Powerful API service tool for Web Scraping

Rotating Residential Proxies, Geolocation, Javascript Rendering and other Proxybot API services making Web Scraping easy and fun process!

Proxybot service simplifies web scraping experience.

The main idea is to focus on writing your web scraper and not to worry about problems like proxy management.

In the following section of this article, you will learn about Proxybot features.

Powerful API service tool for Web Scraping

Contents:

The Key Features

Explore how you can use Proxybot API.

Rotating Residential Proxy servers - Send your request through a proxy server.

Proxy with Geolocation -Send your request through a proxy server located in a specific country.

Javascript Rendering - Get rendered HTML content from a page built with Javascript

Web Scraping - Scrap web page content by simply sending HTTP requests.

 

The following sections will explore these features in more detail.

Rotating Residential Proxy

Proxy management is a crucial element of web scraping.

A rotating proxy server assigns a new random IP address for every incoming connection. It takes IP addresses from a proxy pool

The Proxybot service maintains a pool of rotating residential proxies in different countries. All proxies in our pool are extremely reliable, unique, high quality and provide anonymous connections.

When you send your request through the Proxybot API it will be redirected through a random proxy server. Your request will use the IP address of a randomly assigned proxy server.

Your target website will be not able to see your actual IP address but the IP address of the proxy, allowing you to scrape the web anonymously and avoid IP bans.

In order to take advantage of rotating proxies, you simply need to send HTTP GET requests to Proxybot API. See the below example

https://proxybot.io/api/v1/API_KEY?url=YOUR_URL_HERE

More information on how to use rotating proxy servers can be found here.

Proxy with geographic location

By default, every API request will be routed through a server located in a random country.

However, it is possible to send your request through a proxy server located in a specific country.

It is very easy, you simply need to send an HTTP request and specify county code of your desired proxy location.

Below you can see an example, showing how to send your request through proxy located in Germany.

https://proxybot.io/api/v1/API_KEY?url=YOUR_URL_HERE&geolocation_code=de

Currently, the service offer proxy servers in the following geographic locations:

- us (United States)
- ca (Canada)
- uk (United Kingdom)
- de (Germany)
- fr (France)
- es (Spain)
- br (Brazil)
- mx (Mexico)
- in (India)
- jp (Japan)
- cn (China)
- au (Australia)

- eu (European Union)

- na (North America)

The 'eu' country code will find a random proxy server located in one of the European countries.

The below video demonstrates how you can use a proxy server located in a specific country.

Youtube video showing How to Proxy HTTP Requests

Web Scraping Javascript page

Why it is problematic to scrape content from websites build with Javascript?

It happens because you need to have a browser environment in order to execute Javascript code that will render HTML. We don't have this browser environment when sending requests from a server-side.

Proxybot allows you to simulate the browser environment.

Simply send an HTTP GET request with additional URL param - "render_js=true" and get HTML back.

https://proxybot.io/api/v1/API_KEY?url=YOUR_URL_HERE&render_js=true

The service will virtually run a browser, execute Javascript code, render HTML content and send it back to you.

The below video demonstrates how you can scrape HTML content from a page with Javascript.

Web Scraping Javascript page

Web Scraping with no coding

Maybe you even don't need to create a web scraper?

Proxybot allows extracting content from a website ( text, HTML, links, images, etc..) by simply sending an HTTP POST requests.

The request Body should contain CSS selectors of elements you want to target.

[
  {
    "selector": "#someId .someClass a",
    "get": "text"
  }
]

A response can be an array with extracted values or it could be in JSON like format.

{
    "data": [
        {
            "title": "Our Band Could Be ...",
            "price": "£57.25",
            "image": "media/cache/54/60/54607fe8945897cdcced0044103b10b6.jpg",
            "link": "catalogue/our-band-could-be-your-life-scenes-from-the-american-indie-underground-1981-1991_985/index.html"
        },
        {
            "title": "Libertarianism for Beginners",
            "price": "£51.33",
            "image": "media/cache/0b/bc/0bbcd0a6f4bcd81ccb1049a52736406e.jpg",
            "link": "catalogue/libertarianism-for-beginners_982/index.html"
        },
        {
            "title": "It's Only the Himalayas",
            "price": "£45.17",
            "image": "media/cache/27/a5/27a53d0bb95bdd88288eaf66c9230d7e.jpg",
            "link": "catalogue/its-only-the-himalayas_981/index.html"
        }
    ]
}

More information about web scraping can be found here.

The below video demonstrates how you can scrape HTML content from a webpage by simply sending an HTTP request.

Web Scraping without coding

Web scraping service benefits

How does Proxybot simplify your life?

Avoid IP Blocks and escape CAPTCHAs

For each request, Proxybot assigns a new IP address. It means that a website you trying to open will treat your request as a new user.

Focus on Web Scraping

Don't worry about setting up proxies and configuring server - we have done it for you! Focus on scraping and we will do the rest!

Reliable

In case of failure, Proxybot will retry to execute your request. Moreover, we constantly examine the health of our proxy servers and delete proxies not confirming our standards.

Web Scraping can be an easy and fun process.

I hope that this article was useful and you found information that will help you.


Have a nice day