Web Scraping API Documentation
Read our getting started guide here →
1. Quickstart
The API endpoint is https://scraperbox.com/api/scrape
You can send a GET
or POST
request.
Note that you should replace YOUR_API_TOKEN
with your API token.
2. Authentication
Add the token
field to the request to authenticate yourself.
https://scraperbox.com/api/scrape?token=YOUR_API_TOKEN
Alternatively you can set your token in the Authorization
header.
Authorization: YOUR_API_TOKEN
3. Javascript Rendering
Default value
javascript_enabled=true
A lot of websites require Javascript to display their data correctly.
To enable Javascript rendering set the javascript_enabled
field to true
An API call with Javascript enabled will cost 5 credits.
4. Proxies
Each API requests uses our proxy network by default. Geotargetting is not possible with our normal proxy network.
5. Residential Proxies
Default value
residential_proxy=false
We also offer residential proxies. A residential proxy allows the API to connect to a real internet connection. With this you can counter almost all IP bot detection systems.
To enable residential proxies set the residential_proxy
field to true
An API call with residential proxies enabled will cost 10 credits. If Javascript is enabled each API call with residential proxies will cost 25 credits.
You can also set the proxy_location
parameter when using residential proxies.
Currently we support the following values for proxy_location
proxy_location
will only work with residential_proxy=true
6. Additional Parameters
6.1 Custom Headers
All headers send to our API with a SB-
prefix will be forwarded to the target website.
You will have to add the SB-
prefix so that the API recognizes what headers to forward.
The API will remove the prefix before forwarding the header.
For example, if you only speak klingon, you can set the SB-Accept-Langauge
header to klingon
like this.
7. Response Headers
With each request the following response headers are sent.
Header | Description |
---|---|
X-Request-Cost |
The cost of the request. |
X-Credits-Remaining |
Your remaining credits this period. |
X-Final-Url |
The final URL of the scraped webpage. This is useful for detecting redirects. |
8. Errors
If the API encounters a 4xx
or 5xx
status on the website it will fail along with a JSON error message.
These failed requests will not cost any credits
For example, if the API encounters a 403
response code it will return the following.
{
"errors": {
"response": `The website returned a 403 status code.
You can try again with:
1) javascript_enabled=true
2) residential_proxy=true
3) check out our troubleshooting guide:
https://scraperbox.com/blog/troubleshooting-guide`
}
}