Originally posted on cri.dev
Browser automation built for enterprises, loved by developers.
browserless.io is a neat service for hosted puppeteer scraping, but there is also the official Docker image for running it locally.
I was amazed when I found out about it 🤯!
Find the whole source code on Github christian-fei/browserless-example!
Running browserless in docker
A one-liner is enough to have a full puppeteer backend, with configured concurrency etc., to leverage using puppeteer.
You can connect to a browserless backend by passing the option browserWSEndpoint like this:
async function createBrowser () {
return puppeteer.connect({ browserWSEndpoint: 'ws://localhost:3000' })
}
To start the backend you can use the following command, using the docker image browserless/chrome:
docker run \
-e "MAX_CONCURRENT_SESSIONS=15" \
-e "MAX_QUEUE_LENGTH=0" \
-e "PREBOOT_CHROME=true" \
-e "DEFAULT_BLOCK_ADS=true" \
-e "DEFAULT_IGNORE_HTTPS_ERRORS=true" \
-e "CONNECTION_TIMEOUT=600000" \
-p 3000:3000 \
--rm -it browserless/chrome
Source code
Find the whole source code on Github christian-fei/browserless-example!
You'll find a web crawler with puppeteer!
git clone https://github.com/christian-fei/browserless-example.git
cd browserless-example
npm i
npm run start-browserless
node crawl-with-api.js https://christianfei.com
Puppeteer using browserless docker backend
You simply connect to the Browser WebSocket Endpoint ws://localhost:3000 and you're connected to the browserless backend!
Here is a short example of getting all links <a> on christianfei.com:
const puppeteer = require('puppeteer')
main(process.argv[2])
.then(err => console.log('finished, exiting') && process.exit(0))
.catch(err => console.error(err) && process.exit(1))
async function main (url = 'https://christianfei.com') {
const browser = await createBrowser()
const page = await browser.newPage()
await page.goto(url)
console.log('title', await page.title())
const links = await page.evaluate(selector => [...document.querySelectorAll(selector)], 'a')
console.log('links.length', links.length)
}
async function createBrowser () {
return puppeteer.connect({ browserWSEndpoint: 'ws://localhost:3000' })
}
An example video:
Top comments (0)