You'll find 2 different types of web scraper in here
- Scrapinghub_bot: A webscraper using scrapinghub as platform for parallelising a headless scraping bot. You will also find sample code for running optical character recognition on google and AWS. This bot contains code autogenerated by scrapinghub and is best understood in conjunction with scrapinghub docs.
- In folder selenium, a scraper acting with a browerser window (head) and lite measures the circumvent automatic detection (javascript injection and captcha solving). This scraper makes use of (a) captcha solving and (b) javascript injection to hide that it is run by a webdriver. It requires to complete the initial login manually