Javascript is a vast language that is used in developing many different things nowadays. The one little thing it can do is scrape Google Search Results.
In this tutorial, we will be scraping Google Search Results with Node JS using Unirest and Cheerio as web scraping libraries.
Requirements:
To start scraping Google Search Results, we need to install some npm libraries to move forward and prepare our scraper.
To extract our HTML data we will use Unirest JS. And for parsing the HTML data, we will use Cheerio JS.
Target:
Process:
The Google Search Results scraping is divided into parts:
- Making HTTP request on the target URL to extract the raw HTML data.
- Parsing the HTML data to extract the required data.
We will target this URL:
https://www.google.com/search?q=apple+linkedin&gl=us&hl=en
We will make this request by passing the headers to the URL, which in this case is User-Agent.
User-Agent is used to identify the application, operating system, vendor, and version of the requesting user agent, which can save help in making a fake visit to Google by acting as a real user.
You can also rotate User Agents, read more about this in this article: How to fake and rotate User Agents using Python 3
If you want to further safeguard your IP from being blocked by Google, you can use Serpdog's Google Search API which solves all the problems of proxies and CAPTCHAs for a smooth scraping journey.
unirest
.get("https://www.google.com/search?q=apple+linkedin&gl=us&hl=en")
.headers({
"User-Agent":
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36",
})
.proxy("PROXY SERVER URL")
Now, we will search for the required tags from HTML for our title, description, link and displayed link.
So, according to this image, the tag for the title is .yuRUbf > a > h3 , for the link is .yuRUbf > a , for the snippet is .g .VwiC3b and for the displayed link is .g .yuRUbf .NJjxre .tjvczx.
Here is our code:
const unirest = require("unirest");
const cheerio = require("cheerio");
const getOrganicData = () => {
return unirest
.get("https://www.google.com/search?q=apple+linkedin&gl=us&hl=en")
.headers({
"User-Agent":
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36",
})
.then((response) => {
let $ = cheerio.load(response.body);
let titles = [];
let links = [];
let snippets = [];
let displayedLinks = [];
$(".yuRUbf > a > h3").each((i, el) => {
titles[i] = $(el).text();
});
$(".yuRUbf > a").each((i, el) => {
links[i] = $(el).attr("href");
});
$(".g .VwiC3b ").each((i, el) => {
snippets[i] = $(el).text();
});
$(".g .yuRUbf .NJjxre .tjvcx").each((i, el) => {
displayedLinks[i] = $(el).text();
});
const organicResults = [];
for (let i = 0; i < titles.length; i++) {
organicResults[i] = {
title: titles[i],
links: links[i],
snippet: snippets[i],
displayedLink: displayedLinks[i],
};
}
console.log(organicResults)
});
};
getOrganicData();
Or you can copy this code from the following link for better understanding: Google Organic Search Results Scraper
Results:
Our result should look like this 👇🏻.
[
{
title: 'Apple - LinkedIn',
links: 'https://www.linkedin.com/company/apple',
snippet: "Apple | 17180290 followers on LinkedIn. We're a diverse collective of thinkers and doers, continually reimagining what's possible to help us all do what we ...",
displayedLink: 'https://www.linkedin.com › company › apple'
},
{
title: 'Apple: Jobs - LinkedIn',
links: 'https://www.linkedin.com/company/apple/jobs',
snippet: "Apple | 17158618 followers on LinkedIn. We're a diverse collective of ...",
displayedLink: 'https://www.linkedin.com › company › apple › jobs'
},
{
title: 'Apple Inc - LinkedIn',
links: 'https://www.linkedin.com/company/ihasmd',
snippet: 'Apple Inc | 1980 followers on LinkedIn. iHASMD develops collaborative software for the healthcare industry based on a SAAS model. The software brings in a ...',
displayedLink: 'https://www.linkedin.com › company › ihasmd'
},
{
title: 'James Huang - Director - Apple | LinkedIn',
links: 'https://www.linkedin.com/in/james-huang-83040936',
snippet: 'Skilled in Consumer Electronics, Management, Software Documentation, Test Planning, and Research and Development (R&D). Strong quality assurance professional ...',
displayedLink: 'https://www.linkedin.com › james-huang-83040936'
},
....
With Google Search API
If you don't want to code and maintain the scraper in the long run then you can try this Google Search API to scrape Google Search Results. Serpdog APIs also supports all advanced featured snippets like knowledge graph, answer box results, etc.
We also offer 100 free requests on the first sign-up.
const axios = require('axios');
axios.get('https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us')
.then(response => {
console.log(response.data);
})
.catch(error => {
console.log(error);
});
Result:
{
"meta": {
"api_key": "APIKEY",
"q": "coffee",
"gl": "us"
},
"knowledge_graph": {
"title": "Coffee",
"type": "Drink",
"header_images": [
{
"source": "https://en.wikipedia.org/wiki/Coffee",
"image": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"source": "https://www.tastingtable.com/718678/coffee-brands-ranked-from-worst-to-best/",
"image": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"source": "https://www.healthline.com/nutrition/top-evidence-based-health-benefits-of-coffee",
"image": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"source": "https://www.bbcgoodfood.com/recipes/collection/coffee-recipes",
"image": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"description": "Coffee is a brewed drink prepared from roasted coffee beans, the seeds of berries from certain flowering plants in the Coffea genus. From the coffee fruit, the seeds are separated to produce a stable, raw product: unroasted green coffee.",
"source": {
"name": "Wikipedia",
"link": "https://en.wikipedia.org/wiki/Coffee"
},
"Acidity level": "4.85 to 5.10",
"list": {
"Total Fat": [
"0 g",
"0%"
],
"Saturated fat": [
"0 g",
"0%"
],
"Trans fat regulation": [
"0 g",
""
],
"Cholesterol": [
"0 mg",
"0%"
],
"Sodium": [
"5 mg",
"0%"
],
"Potassium": [
"116 mg",
"3%"
],
"Total Carbohydrate": [
"0 g",
"0%"
],
"Dietary fiber": [
"0 g",
"0%"
],
"Sugar": [
"0 g",
""
],
"Protein": [
"0.3 g",
"0%"
],
"Caffeine": [
"95 mg",
""
],
"Vitamin C": [
"",
"0%"
],
"Calcium": [
"",
"0%"
],
"Iron": [
"",
"0%"
],
"Vitamin D": [
"",
"0%"
],
"Vitamin B6": [
"",
"0%"
],
"Cobalamin": [
"",
"0%"
],
"Magnesium": [
"",
"1%"
]
},
"Compounds In Coffee": [
{
"title": "Chlorogenic acid",
"link": "https://google.com/search?gl=us&hl=en&q=Chlorogenic+acid&stick=H4sIAAAAAAAAAONgFuLUz9U3MCorTMtVAjPNizNy4rVEspOt9JPzc3Pz86xS8svzyhOLUopXMQo6Z6TmZiYn5jjn5xbkl-alFC9iFXDOyMkvyk9PzctMVkhMzkzZwcoIAJ5I0VJYAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhlEAU",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Trigonelline",
"link": "https://google.com/search?gl=us&hl=en&q=Trigonelline&stick=H4sIAAAAAAAAAONgFuLUz9U3MCorTMtVAjNNqwwq47VEspOt9JPzc3Pz86xS8svzyhOLUopXMQo6Z6TmZiYn5jjn5xbkl-alFC9i5QkpykzPz0vNycnMS93ByggAa_A38FQAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhlEAc",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Quinic acid",
"link": "https://google.com/search?gl=us&hl=en&q=Quinic+acid&stick=H4sIAAAAAAAAAONgFuLUz9U3MCorTMtVAjPNi5NNLbREspOt9JPzc3Pz86xS8svzyhOLUopXMQo6Z6TmZiYn5jjn5xbkl-alFC9i5Q4szczLTFZITM5M2cHKCAAvEyRDUwAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhlEAk",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Kahweol",
"link": "https://google.com/search?gl=us&hl=en&q=Kahweol&stick=H4sIAAAAAAAAAONgFuLUz9U3MCorTMtV4gIxTdLMDLJNtESyk630k_Nzc_PzrFLyy_PKE4tSilcxCjpnpOZmJifmOOfnFuSX5qUUL2Jl907MKE_Nz9nByggA3TsOH1AAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhlEAs",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"Starbucks Coffee": [
{
"title": "Starbucks Roast Dark Ro...",
"link": "https://google.com/search?gl=us&hl=en&q=Starbucks+Roast+Dark+Roast+Ground+Coffee&stick=H4sIAAAAAAAAAC3IMQrCMBQAUEQKOriIByhO4vKtLtJVwV0PUNLkJ8Yk_8ekteBxHD2Bx9PB7fEm4_kUAmy2j7sOyxkYqKqblrun27frhZM1SA6BqVY80CCSyu9R9TvvUXaWCRzx4FEZbKIg9BnylWO0ZBoXG_TW2Nbjq1hdOpHaXrpcnlnkrjyK5P48Je5JlQfWGvFTjL5shC6JlAAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhmEAU",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Starbucks Frappuccino, Caramel...",
"link": "https://google.com/search?gl=us&hl=en&q=Starbucks+Frappuccino,+Caramel,+13.7+Oz&stick=H4sIAAAAAAAAAA3IMQrCMBQAUIoUdHARDxBcBCmmqYLQVXB18AAlTX9jyP9JTFoLHsfRE3g8feObz1YLTrysno-eNkuuuRC6OpZEttqtraq58kTe1Z2f3CRjlz6Z-B8iqMF4x63zE0KnoQnSASae7j4E43RjQwNotGkR3vn2NsjYjsomdokyhFEp43zBzjJKAiyYOOxP7Pr65tkPqGF6KJMAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhmEAc",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Starbucks Coffee Drink Sw...",
"link": "https://google.com/search?gl=us&hl=en&q=Starbucks+Coffee+Drink+Sweet+Cream&stick=H4sIAAAAAAAAAA3IMQ7CIBQA0DSmiQ4uxgOQji7fmtSYrvUGPQCh9BcJ8KGAch9HT-Dx9I1vuznswMH58loX1-xBQdtSl1d55bfT0cgepHfOUz_7QkXEOX2q9n_WoszaExjyxeKskAdBaBOkhw9Bk-ImcLRa6cniu27GLOL0lCaxwS8LIrtHTYaNBTGzIaJw37r6AeTgLQeOAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhmEAk",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Starbucks Roast Coffee...",
"link": "https://google.com/search?gl=us&hl=en&q=Starbucks+Roast+Coffee+Whole+Bean&stick=H4sIAAAAAAAAAA3IMQ6CMBQA0BBDooOL8QDV0eUDgwOj3kAHR1LKpzRt_y98lPs4egKPp29869VuAxGK6jX28bgFC2UpRZLxnOJp700NhmNkqjteaNFTJ5-s_F8IaGbHBJ54CdhZbJImDAIycEqObONTg8FZ1wZ854f7rKf2abyoG2uZ1ZX7HlE9Bg6oLqjpm2c_as3ybY0AAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhmEAs",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"People also search for": [
{
"title": "Tea",
"link": "https://google.com/search?gl=us&hl=en&q=Tea&si=AC1wQDCwN61-ebmuwbQCO5QCrgOvEq5bkWeIzJ5JczItzAKNdTku-1rX0tcMn1LKdXOIpKJwlLmVSfa90_0znjAMcL2sAzX4J_UwMexVra0SvN9zBEkwGwA5u6HSaoI47SyRCD9nb-8Glzx2c5rfIJe-aizf7IAxGOhJh_kG0qL9FIdiWvdohKRx4gRfu0e1hMv_cus-OR01faxKayI3akdtDTPtZF4-BCaT_0qctWJy1y5xtkAOr0_dsLhobE2rbdSGteM1GLxNZDARFa9umM8-6UkPFTmf4Q%3D%3D&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhnEAU",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Espresso",
"link": "https://google.com/search?gl=us&hl=en&q=Espresso&si=AC1wQDCwN61-ebmuwbQCO5QCrgOvEq5bkWeIzJ5JczItzAKNdeHFN2RiKqNUOGcF0pmVdz3rIrXiuMEhNdO-SfahftpqQIYK3-hB1rA_kOlkAeWqSr6HTBB8G84PquMrIFHofy3tS0AENRg2iT4kQhARW-cqiwd5mwe4HWoM7NfCdXPHKL-9vcPCuQWsMQH0QsiUPQef1wfYHgpF76EoCEd9WgK2RCOVSWy2_UvrDO9ycBWVfnNdmkvoeclcDeyjTxdu3vRmri1-sGC3Vxg1Xf8C1P-tSU7Ac6Db7sVDHZWBhhei19_FP0o%3D&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhnEAc",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Drink",
"link": "https://google.com/search?gl=us&hl=en&q=Beverage&si=AC1wQDC0KliLIoeyh35S2UVSh9d0at62ormA4JRB9xLdTgvleVEPuIJW3qN0rEFDMoRO5TaIi-eUtrO7i-yBcPsfUzSls6PoTU-buveIcXrKc8j6j8RTk6levipGBVfMDb2AmQYFIlFivgjEWtrw2TTOjzXWO7h0VnyOV-MXRLUeB3vIgKfdyVVgegWm85plPQ3qoUo00QwqxhwByVeRhWtTwWxm0nV7Mhs4OnmQz88vCcewdr1PGbonYzW4YfEznNahmd_nLsKERiRzf7PW6RoKR9MqwLl7tKt3KMgvOj5nUbDOmdXibLM%3D&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhnEAk",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Iced coffee",
"link": "https://google.com/search?gl=us&hl=en&q=Iced+coffee&si=AC1wQDDagiMg03ncxeOQZbwVe-CJxRCchC-jr2hCPTxjc9wbgDb1tUqijDJX85Pg7-Fd4h4ECiqrTLXgOQJbaZmoXfWb8MZKyZ7uj6FykoR0HPdYz7APuA16Lh3IDaqobb6GC1g6vA53Q0ZorgA-FFa18joGMuV0h3kKB2XsyLL0UeBqUl5Ia_r_Auk9F-wZzVIhpHKkmIq6gXo1DP_vcZa8jyTzXy4VXG7vcXdoDmzCjZTsJtClzIOpna_wgbYFRtYbRWqfGISfDDvhjFbeGQDR5bN1gsLT_ra5gs0FdkoknXCL4O-AXnM%3D&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhnEAs",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"see_results_about": [
{
"title": "Coffee",
"snippet": "Plant",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
]
},
"inline_videos": [
{
"title": "Miguel - Coffee (Official Video)",
"source": "YouTube",
"date": "Jun 16, 2015",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Kelly Rowland - COFFEE (Official Music Video)",
"source": "YouTube",
"date": "Apr 17, 2020",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Beabadoobee - Coffee",
"source": "YouTube",
"date": "Mar 26, 2020",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"local_results": [
{
"position": 1,
"title": "Scooter's Coffee",
"place_id": "5225762576898740739",
"rating": "4.4",
"address": "Coffeyville, KS",
"hours": "Closed â‹… Opens 7AM",
},
{
"position": 2,
"title": "Terebinth Coffee House & Roastery",
"place_id": "18371722425790879519",
"rating": "4.9",
"address": "Coffeyville, KS",
"hours": "Closed â‹… Opens 6:30AM Mon",
},
{
"position": 3,
"title": "Ane Mae's Coffee and Sandwich Shop",
"place_id": "1687994686344788766",
"rating": "4.7",
"address": "Independence, KS",
"hours": "Closed â‹… Opens 7AM Mon",
},
{
"more_locations_link": "https://www.google.com/search?gl=us&hl=en&q=coffee&npsic=0&rflfq=1&rldoc=1&rllag=37129806,-95662854,11423&tbm=lcl&sa=X&ved=2ahUKEwiGzIer7qL5AhXCKEQIHdh0D5YQtgN6BAgmEAE"
}
],
"recipes_results": [
{
"title": "Coffee recipes",
"link": "https://www.bbcgoodfood.com/recipes/collection/coffee-recipes",
"source": "BBC Good Food",
"rating": "",
"reviews": "",
"total_time": "",
"ingredients": [
"Instant coffee"
],
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "20 Great Coffee Drinks From Around the World",
"link": "https://insanelygoodrecipes.com/coffee-recipes/",
"source": "Insanely Good Recipes",
"rating": "4.5",
"reviews": "8",
"total_time": "",
"ingredients": [
"Turkish coffee",
" vietnamese coffee",
" white chocolate mocha",
" cold brew coffee",
" frappuccino"
],
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Bulletproof Coffee Recipe",
"link": "https://www.bulletproof.com/recipes/bulletproof-diet-recipes/bulletproof-coffee-recipe/",
"source": "Bulletproof",
"rating": "",
"reviews": "",
"total_time": "5 min",
"ingredients": [
"Mct oil",
" bulletproof coffee",
" grass fed"
],
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"peopleAlsoAskedFor": [
"What are benefits of coffee?",
"Which is the best coffee to drink?",
"Is it healthy to eat coffee?",
"Is coffee made from poop?"
],
"organic_results": [
{
"title": "9 Health Benefits of Coffee, Based on Science - Healthline",
"link": "https://www.healthline.com/nutrition/top-evidence-based-health-benefits-of-coffee",
"displayed_link": "https://www.healthline.com › Wellness Topics › Nutrition",
"snippet": "Coffee is a popular beverage that researchers have studied extensively for its many health benefits, including its ability to increase energy levels, promote ...",
"rank": 1
},
{
"title": "The Coffee Bean & Tea Leaf | CBTL",
"link": "https://www.coffeebean.com/",
"displayed_link": "https://www.coffeebean.com",
"snippet": "Born and brewed in Southern California since 1963, The Coffee Bean & Tea Leaf® is passionate about connecting loyal customers with carefully handcrafted ...",
"rank": 2
},
{
"title": "Peet's Coffee: The Original Craft Coffee",
"link": "https://www.peets.com/",
"displayed_link": "https://www.peets.com",
"snippet": "Since 1966, Peet's Coffee has offered superior coffees and teas by sourcing the best quality coffee beans and tea leaves in the world and adhering to strict ...",
"rank": 3
},
{
"title": "The History of Coffee - National Coffee Association",
"link": "https://www.ncausa.org/about-coffee/history-of-coffee",
"displayed_link": "https://www.ncausa.org › ... › History of Coffee",
"snippet": "Coffee grown worldwide can trace its heritage back centuries to the ancient coffee forests on the Ethiopian plateau. There, legend says the goat herder ...",
"inline_sitelinks": [
{
"title": "An Ethiopian Legend",
"link": "https://www.ncausa.org/about-coffee/history-of-coffee#:~:text=An%20Ethiopian%20Legend"
},
{
"title": "The Arabian Peninsula",
"link": "https://www.ncausa.org/about-coffee/history-of-coffee#:~:text=The%20Arabian%20Peninsula,-Coffee%20cultivation%20and%20trade%20began"
},
{
"title": "Coffee Comes To Europe",
"link": "https://www.ncausa.org/about-coffee/history-of-coffee#:~:text=Coffee%20Comes%20to%20Europe"
}
],
"rank": 4
},
{
"title": "coffee | Origin, Types, Uses, History, & Facts | Britannica",
"link": "https://www.britannica.com/topic/coffee",
"displayed_link": "https://www.britannica.com › ... › Food",
"snippet": "coffee, beverage brewed from the roasted and ground seeds of the tropical evergreen coffee plants of African origin. Coffee is one of the three most popular ...",
"rank": 5
},
{
"title": "Starbucks Coffee Company",
"link": "https://www.starbucks.com/",
"displayed_link": "https://www.starbucks.com",
"snippet": "More than just great coffee. Explore the menu, sign up for Starbucks® Rewards, manage your gift card and more.",
"rank": 6
},
{
"title": "#coffee hashtag on Instagram • Photos and videos",
"link": "https://www.instagram.com/explore/tags/coffee/",
"displayed_link": "https://www.instagram.com › explore › tags › coffee",
"snippet": "156M Posts - See Instagram photos and videos from 'coffee' hashtag.",
"rank": 7
}
],
"relatedSearches": [
{
"query": "coffee near me",
"link": "www.google.com/search?gl=us&q=coffee near me"
},
{
"query": "coffee shop",
"link": "www.google.com/search?gl=us&q=coffee shop"
},
{
"query": "coffee benefits",
"link": "www.google.com/search?gl=us&q=coffee benefits"
},
{
"query": "best coffee",
"link": "www.google.com/search?gl=us&q=best coffee"
},
{
"query": "coffee - wikipedia",
"link": "www.google.com/search?gl=us&q=coffee - wikipedia"
},
{
"query": "coffee origin",
"link": "www.google.com/search?gl=us&q=coffee origin"
}
],
"pagination": {
"current": "1",
"next": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=10&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8NMDegQIAxBK",
"page_no": {
"2": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=10&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxA4",
"3": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=20&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxA6",
"4": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=30&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxA8",
"5": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=40&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxA-",
"6": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=50&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxBA",
"7": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=60&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxBC",
"8": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=70&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxBE",
"9": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=80&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxBG",
"10": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=90&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxBI"
}
},
"serpdog_pagination": {
"current": "1",
"page_no": {
"2": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=10",
"3": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=20",
"4": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=30",
"5": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=40",
"6": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=50",
"7": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=60",
"8": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=70",
"9": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=80",
"10": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=90"
}
}
}
Conclusion:
In this tutorial, we learned to scrape Google Search Results using Node JS. Feel free to message me anything you need clarification on. Follow me on Twitter. Thanks for reading!
Additional Resources
- Scrape Google Shopping Results
- Scrape Google Images Results
- Scrape Google News Results
- Scrape Google Maps Reviews
Frequently Asked Questions
Q. Is it legal to scrape Google Search Results?
Yes, it is legal to scrape Google Search Results. According to First Amendment of the United States Constitution scraping of any public data is completely legal.
Q. Can you scrape Google Search Results?
If you don't want to code and maintain the scraper in the long run then you can definitely try Serpdog Google Search API to scrape Google Search Results. This API also supports all advanced featured snippets like knowledge graph, answer box results, etc.
Q. How do I scrape Google Search Results for free?
- Login to Serpdog | Google Search API.
- Insert the query you want to scrape from Google.
- Click the Send button to send the request at the API server and collect your response form the dashboard.
Author:
My name is Darshan, and I am the founder of serpdog.io. I am currently working for several MNCs to provide them Google Search Data through a seamless data pipeline.
Top comments (0)