Troubleshooting Guide
By
Dirk Hoekstra
on 29 Dec, 2020
Scraping the web can be hard!
In this article I will outline the most common problems that you may encounter when using ScraperBox.
Javascript Rendering
Some websites require Javascript in order to work.
A lot of issues can be fixed by setting javascript_enabled
to true
When enabling Javascript we spin up a real Chrome browser. This will also help with avoiding bot checks.
403 Forbidden
Our basic proxy network consists of datacenter proxies.
These are relatively easy to detect by websites, and as a result you might encounter some 403 Forbidden
responses.
The servers are constantly refreshed, so on some days you might encounter more 403
responses than on other days.
A way to overcome almost all 403
responses is to use residential proxies by setting residential_proxy
to true
.
Use Our Help
You can always contact us using the live-chat on the website. We'll be happy to help you setting up something custom!
Dirk Hoekstra has a Computer Science and Artificial Intelligence degree and is the co-founder of Scraperbox.
He is a technical author on Medium where his articles have been read over 100,000 times.
Founder of multiple tech companies of which one was acquired in 2020.