(Use at your own risk, I'm not condoning anything illegal)
Websites such as Amazon uses bot detection software such as Imperva and FingerprintJS to block automated tools from interacting with their website. And they're rightly doing so to keep web scrapers and hackers away from stealing their precious data.
If they only knew...
Those security controls are fairly easy to bypass if you know what you're doing.
There are a few ways we can bypass those security controls:
- Rotate proxies
- Using cookies
- Rotate User Agents and HTTP Headers
- Remote Javascript signatures in your webdriver(we'll explore this one soon! 😎)
- Avoiding patterns when interacting with websites
If you're using Selenium and you're getting blocked by automation software.
There's a little trick we can implement that can save us the hassle of setting up rotating proxies, headers or user agents.
Disable Selenium Automation Flag
But here's another simpler way that might do the trick - disabling Selenium's automation flag.
Selenium by default has automation flags enabled which might block you from accessing websites using anti bot software.
To bypass the block you'll have to disable those automation flags in your Selenium code, let's see how we can do that using a simple one liner.
First, let's call the webdriver.Chromeoptions() function and instantiate it to a variable called options.
This enable's us to add custom options to our webdriver:
options = webdriver.ChromeOptions
Now, let’s add the option disabling the automation flag by adding the following line to our Selenium code:
options.add_argument("--disable-blink-features=AutomationControlled")
This simple one-liner disables the automation flag and tells the website that you're not a bot.
Top comments (0)