The ScraperBox Blog

Hi , we are Dirk and Terry. We're building an easy to use web scraping API. You can read about our journey and what we've learned along the way on this blog.

How I spend $500 per day because of a misconfiguration

In one weekend Google Cloud had burned through €1,200 - which is roughly $1,500. Scraperbox is bootstrapped so this was going to come out of my own pocket.

By Dirk on 05 Feb, 2021
Troubleshooting Guide

Scraping the web can be hard! In this article I will outline the most common problems that you may encounter when using ScraperBox.

By Dirk on 29 Dec, 2020
How to Scrape Google Search Results with Python

In this article, we're going to build a Google search result scraper in Python! We'll start with creating everything ourselves. And then...

By Dirk on 28 Dec, 2020
Web Scraping with Ruby

In this article, we're going to set up a web scraper with Ruby! I think that I'll be fun to try and scrape all developer jobs from indeed.com

By Dirk on 22 Dec, 2020
Solving a Geetest Slider Captcha with Puppeteer

I recently had to solve the Geetest slider captcha. The Captcha is basic enough, you must slide a puzzle piece into the slot...

By Dirk on 01 Dec, 2020
How to scrape webpages using NodeJS

With web scraping, we can automatically extract data from websites!

By Terry on 30 Nov, 2020
Getting started with ScraperBox

Before we dive into how to set up Scraperbox, let's talk about what problem it solves.

By Terry on 06 Sep, 2020