DEV Community

Cover image for Let's Build a Web Scraper with Cypress!
Brett Cassette
Brett Cassette

Posted on

Let's Build a Web Scraper with Cypress!

Check out the Free Lesson on Egghead.io

What would you build if you had a web scraper?

  • Maybe your fitness tracker has no public API?
  • Or you want to always outprice your competitors?
  • Maybe you want to correlate and visualize data from multiple, disconnected platforms?
  • Want to automate common tasks on an external site?

Or maybe you're like me, and the website that lists all your open mic comedy events doesn't have any ability to search, filter, or show event locations on a map.

So how do we ingest this data for fun and profit?

Cypress to the Rescue

Cypress is a blazing-fast end-to-end testing framework. Since it can interact with any webpage, we can also use it to automate interactions with external sites!

Build it along with me in this free lesson on Egghead.io!

Check out the Free Lesson on Egghead.io

Top comments (2)

Collapse
 
venkatcoderbloom profile image
Venkatesh vellingiri

Hi Brett,
I enjoyed scrapping of data using cypress.

I scrapped a web application which contains a list of data. Is it possible for me to use that in node.js API for processsing.

Collapse
 
cubiclebuddha profile image
Cubicle Buddha

Cypress is the best. Once it came out I was so happy to stop using CasperJS and Slimer. I don’t know if I could do (enterprise grade) UI development without Cypress.