Data mining is a hard job and it gets harder when you are supposed to deal with giants like Google. A couple of years ago, I needed to deal with Google SERPs data and I searched for available options. I found a couple of web services as well as open-source code to get Google SERPs data but each solution had its own limitations.
I found a couple of services to be reliable but they were costly. Some were affordable but I had to wait for several minutes before they returned me a result. And there, I started my research.
I opted for Python as the programming language and thanks to several other tools that I used, I was able to build a solution. My web service is https://serpsbot.com and now it is in full production mode. You can use it to consume Google SERPs data for any purpose.
To cover the server costs, I'm charging $0.001 per API call there. For high-volume customers, the price can be reduced down to $0.0005 per API call.
How to get Google SERPs?
You can get Google SERP results as JSON data. Here is an example of usage for the awesome Python. Please beware that you need to create an account and get your API key at our website SerpsBot first.
import requests
# Your API key
apikey = 'your-api-key-here'
headers = {
'Authorization': 'Bearer {}'.format(apikey)
}
payload = {
'q': 'programming',
'pages': 2,
'hl': 'en-US',
'gl': 'us'
}
res = requests.get('https://serpsbot.com/api/v1/google-serps/', headers=headers, params=payload)
# JSON data
res.json()
How to build a custom site search engine?
Using our API service, you can build a search engine for your website. As we support all Google search filters, you can pass in your website URL using site:
filter and only results for your provided URL will be returned.
import requests
# Your API key
apikey = 'your-api-key-here'
headers = {
'Authorization': 'Bearer {}'.format(apikey)
}
payload = {
'q': 'python site:dev.to',
'pages': 2,
'hl': 'en-US',
'gl': 'us'
}
res = requests.get('https://serpsbot.com/api/v1/google-serps/', headers=headers, params=payload)
# JSON data
res.json()
Possibilities are limitless. You can use any programming language to consume data from our API endpoint and you can build your next tool that consumes data from our API. If you have any questions, you can ask me in the comments.
p.s. I'll publish the website source at GitHub in the upcoming months.
Top comments (0)