Natasha Lekh for Apify

Posted on Dec 1, 2023 • Originally published at blog.apify.com on Aug 31, 2023

Top Google Search APIs

#google #api #seo

Not all websites share their APIs with the public. Luckily for us, the biggest website in the world, Google Search, offers quite a few Google APIs. Or does it? Let's break down the most common misconceptions around Google Search API.

We're Apify . The Apify platform gives you access to 1,500+ data extraction tools and unofficial APIs for popular websites. Check us out .

Instead of going deep into what an API is, heres a short recap: APIs make the web open. The main point of an API is to connect different web programs with a link that they will use to communicate and exchange data. If the API is well-written, it enables two applications created with different technologies and languages to interact smoothly with each other without the need to resort to some sort of techno-compromise.

For various reasons, not all websites share their APIs with the public. Luckily for us, the biggest website in the world seems to be among the most generous, offering quite a few public APIs along with documentation on how to use them. Not without caveats of course, but well talk about that in due time.

Which API is used in Google?

Google offers many public APIs for different Google services; they are usually JSON APIs based on RESTful requests. These APIs are publicly available in Google API GitHub repository. As for a Google Search API, theres no such thing from Google. The Custom Search API provided by Google wont allow you to get a JSON from Google search results. What it does instead is perform a small version of Google Search on your own website.

Is there a Google Search API?

Not at the moment. Although the original Google Web Search API has been deprecated in 2011, you can still stumble upon its replacement, Custom Search API. However, this replacement API is different. While provided by Google as a stand-in for Google Search API, this API wont return you a detailed list of Google search results in JSON - which is what youd expect. Instead, it allows you to program your own little search engine by applying the logic of Google Search to your website.

In addition, this Google Search API imposes two key limitations: the number of queries and the number of websites you can search. It supports a limited number of queries per day (10K). It may also restrict search capabilities which means that you cant use it to search the whole web programmatically. Whichever limitation you choose, it will cost you $5/1K queries to search websites programmatically via the official Google API. More on the pricing in the next part.

While searching for the Google Search API, you may also come across the Google Search Console API, which is also not exactly what you might be looking for since it's geared specifically towards the GSC tool. It will allow you to interact with and control various aspects of Google Search Console programmatically but that's closer to a scraping session customized for your website than an objective look at what Google has to offer for a given query.

Is Google's API free?

Google has many APIs and all of them are free but with many limitations. The Custom Search API, for example, offers the first 100 search queries per day for free. If you wish to lift this limitation and make more queries, it will cost you $5 per 1,000 requests but still limits you to 10,000 queries per day. Theres no way to exceed those 10,000 requests per day even if youre ready to pay more. The only way to go above the 10K limitation is if you are applying your search engine to less than 10 websites.

How do I create a search API?

To create a search API of your own, you can follow Googles step-by-step tutorial on creating a Custom Search Engine which is freely available on the Google developers page. In short, regardless of the programming language, you will need to get an API Key, a Custom Search ID, and install a relevant API Client to get your first results. If you follow through with all the steps, youll be able to make your own search engine, customize your search engine's ranking, and do other surface-level modifications.

🎓More of a Bing scraping fan? See this tutorial

How do I search Google using an API?

Although Google does offer some sort of Search API, its impossible to use that one to send a search query to the whole wide web and get a machine-readable doc from this search. Which makes an official Google Search API, in the way youd usually understand it, non-existent. However, many developers have noticed it and have come up with alternatives. By the nature of their task, all those Google Search API alternatives are essentially scrapers, heres just a few of them:

Top Google Search API alternatives:

1. SERP API

Offers some web scraping infrastructure, regular organic results, structured data in JSON, reliable location search via encrypted parameters, solving all CAPTCHAs, guaranteed 20% throughput, and no request queues. Free plan can get you 100 searches/month. Pricing starts from $50/month per 5,000 searches/month and offers a so-called Legal US Shield.

2. Apify 's SERP API

Provides SERP proxies, reliable scraping infra and monitoring, freedom in request location and language, organic results, ads, prices, reviews, related queries, wide proxy network, datasets of 5 formats (JSON, Excel, CSV, XML, RSS), and offers Google APIs for other Google Services. Free trial for 30 days, then $49/month for a platform subscription plan. Try it out for free right now!

3. Serpstack

The free plan promises no request queue, customized search queries, proxy network, data in JSON format, SERP proxies, and up to 1,000 searches monthly. Able to scrape the web, images, videos, news and shopping data. Pricing starts at $29.99/month for expanded features of API premium.

4. Rapid API

One available API for Google Search suggests speed and accuracy, easy-to-use but advanced enough to support special parameters, with zero proxies but clean IPs from the US instead, as well as some monitoring. Free plan is limited to 50 requests/month and 1 request/second, paid plan starts at $15/monthly and 5,000 requests/month. Maximum capacity plan offers 50,000 requests per month.

Another one called Google Web Search offers Free plan with 100 requests/month and 5 requests/second. Paid plan starts at $30/monthly and 10,000 requests/month.

5. ScrapingBee

Scrapes organic results, ads, local results, and related queries using SERP proxies. Offers extensive documentation. Monthly subscription plan starts at $49 with 6,000 searches/month and 5 concurrent requests. Proxies availability depend on the plan.

So Google Search API is powered by web scraping?

Essentially, yes. That's why you can confidently call Google the world's biggest web scraping company. As Google stopped providing a proper Search API allowing to get machine-readable data, other solutions such as scrapers inevitably turned up. They now remain the only proper solution to extract data from Google via an API.

How does a Google SERP API work?

The Google SERP bot takes your query, [e.g. weather tomorrow], takes it to google.com, performs the search and extracts the raw Google data from the results pages. You can customize the language [e.g. English or Spanish], geolocation of the results [e.g. US or Mexico], number of scraped results pages [e.g. first 10] and many other parameters.

Watch this simple video explaining how a Google Search API aka Google SERP Scraper retrieves data from the search engine. It will only take you 3 minutes to learn how to extract the data you need from Google Search!

DEV Community