If you're interested in building a SERP ranking tracker app, you'll love our API. SerpApi provides a simple API to access live data from various search engines, including Google, Bing, DuckDuckGo, Yahoo, and others. It enables you to build an app like a SERP ranking tracker.
The idea
To get the ranking position of a website, we need to access the organic results and check where the domain first appears. The organic results data is available through our API. Here are the three APIs we're going to use:
- Google Search API: Scrape the results from Google search.
- Bing Search API: Scrape the results from the Bing search.
- DuckDuckGo Search API: Scrape the results from the DuckDuckGo search.
API Design
We'll create a single endpoint where people can receive the ranking results from the above search engine using this parameter:
- Domain name (String): The website domain we want to track.
- Keywords (Array): List of keywords that we want to search for.
- Engines (Object): List of search engines we want to search on. We can also adjust the parameter details based on the API. Refer to the relevant documentation to check the available parameters for each APIs.
POST: /api/rankings
Data:
- domain (string)
- keywords (array[string])
- engines ((array[name, params]))
Example:
{
"domain": "archive.org",
"keywords": ["internet archive"],
"engines": [
{
"name": "google",
"params": {
"domain": "google.com",
"gl": "es"
}
}
]
}
The source code is available on GitHub; feel free to take a look at the detailed implementation here:
https://github.com/hilmanski/rank-tracker-api/
Let's write the code!
I'll use Nodejs for this API; feel free to use other languages/frameworks.
Install Express
Let's use Express to help us clean up the code structure.
npm i express --save
Export your API Key
You can either export your API key in a terminal like the sample below or save it on an .env
file.
export SERPAPI_API_KEY=YOUR_ACTUAL_API_KEY
Basic route
Prepare the POST endpoint with relevant parameters
const express = require('express')
const app = express()
const port = 3000
app.use(express.json());
app.post('/api/rankings', async(req, res) => {
const { keywords, engines, domain } = req.body;
// detail implementation later
res.json({ keywords, engines });
})
app.listen(port, () => {
console.log(`Example app listening on port ${port}`)
})
Validate the input type
Make sure API users use the correct types.
app.post('/api/rankings', async(req, res) => {
const { keywords, engines, domain } = req.body;
// Validate keywords
if (!Array.isArray(keywords) || !keywords.length) {
return res.status(400).json({ error: 'Keywords and engines must be arrays.' });
}
// Validate engines
for (const engine of engines) {
if (typeof engine !== 'object' || !engine.name) {
return res.status(400).json({ error: 'Each engine must be an object with a "name" property.' });
}
if (engine.params && typeof engine.params !== 'object') {
return res.status(400).json({ error: 'Engine "params" must be an object.' });
}
}
// coming soon
})
Run parallel search
Since we're enabling multiple keywords and multiple search engines, we need to run the function in parallel to save us some time.
// Parallel search
const results = await Promise.all(engines.map(async engine => {
const rankings = await Promise.all(keywords.map(async keyword => {
return await getRanking(keyword, engine, cleanDomain);
}));
// map keywords - rankings in one array
const rankingResults = keywords.map((keyword, index) => {
return [keyword, rankings[index]];
});
console.log(rankingResults);
return { domain, engine, rankingResults };
}))
*getRanking method coming soon.
GetRanking method implementation
Here is the function that is responsible for running the search for each search engine.
const suportEngines = ['google', 'bing', 'duckduckgo'];
async function getRanking(keyword, engine, domain) {
const engineName = engine.name.toLowerCase();
if(!suportEngines.includes(engineName)) {
console.error(`Error: Engine ${engineName} is not supported.`);
return;
}
return new Promise(async (resolve, reject) => {
switch(engineName) {
case 'google':
resolve(await searchGoogle(keyword, engine.params, domain))
break;
case 'bing':
resolve(await searchBing(keyword, engine.params, domain))
break;
case 'duckduckgo':
resolve(await searchDuckDuckGo(keyword, engine.params, domain))
break;
default:
break;
}
})
}
We'll use the native fetch method in NodeJS to request the actual SerpApi endpoint for each search engine.
API to access ranking position in Google
function searchGoogle(keyword, params, domain) {
let endpoint = `https://serpapi.com/search?q=${keyword}&engine=google&num=100&api_key=${SERPAPI_API_KEY}`
if(params) {
endpoint += `&${new URLSearchParams(params).toString()}`
}
return fetch(endpoint)
.then(response => response.json())
.then(data => {
const organic_results = data.organic_results;
let ranking = organic_results.findIndex(result => result.link.includes(domain))
return ranking + 1;
})
.catch(error => {
console.error(error);
});
}
API to access ranking position in Bing
function searchBing(keyword, params, domain) {
let endpoint = `https://serpapi.com/search?q=${keyword}&engine=bing&count=50&api_key=${SERPAPI_API_KEY}`
if(params) {
endpoint += `&${new URLSearchParams(params).toString()}`
}
return fetch(endpoint)
.then(response => response.json())
.then(data => {
const organic_results = data.organic_results;
let ranking = organic_results.findIndex(result => result.link.includes(domain))
return ranking + 1;
})
.catch(error => {
console.error(error);
});
}
API to access ranking position in DuckDuckGo
function searchDuckDuckGo(keyword, params, domain) {
let endpoint = `https://serpapi.com/search?q=${keyword}&engine=duckduckgo&api_key=${SERPAPI_API_KEY}`
if(params) {
endpoint += `&${new URLSearchParams(params).toString()}`
}
return fetch(endpoint)
.then(response => response.json())
.then(data => {
const organic_results = data.organic_results;
let ranking = organic_results.findIndex(result => result.link.includes(domain))
return ranking + 1;
})
.catch(error => {
console.error(error);
});
}
Final Endpoint
Here's what our endpoint looks like
app.post('/api/rankings', async(req, res) => {
const { keywords, engines, domain } = req.body;
// Validate keywords
if (!Array.isArray(keywords) || !keywords.length) {
return res.status(400).json({ error: 'Keywords and engines must be arrays.' });
}
// Validate engines
for (const engine of engines) {
if (typeof engine !== 'object' || !engine.name) {
return res.status(400).json({ error: 'Each engine must be an object with a "name" property.' });
}
if (engine.params && typeof engine.params !== 'object') {
return res.status(400).json({ error: 'Engine "params" must be an object.' });
}
}
// CLean up domain
// Since people can include https:// or http:// or a subdomain, strip all of it?
const cleanDomain = domain.replace(/^https?:\/\//, '').replace(/\/$/, '');
// Parallel search
const results = await Promise.all(engines.map(async engine => {
const rankings = await Promise.all(keywords.map(async keyword => {
return await getRanking(keyword, engine, cleanDomain);
}));
// map keywords - rankings in one array
const rankingResults = keywords.map((keyword, index) => {
return [keyword, rankings[index]];
});
console.log(rankingResults);
return { domain, engine, rankingResults }
}))
res.json(results);
})
Test the API
Let's try out this API via cURL.
No parameter example
curl -X POST http://localhost:3000/api/rankings \
-H 'Content-Type: application/json' \
-d '{
"keywords": ["internet archive"],
"domain": "archive.org",
"engines": [
{
"name": "google"
}
]
}'
Using multiple keywords and search engine parameter sample
curl -X POST http://localhost:3000/api/rankings \
-H 'Content-Type: application/json' \
-d '{
"keywords": ["internet archive", "digital library archived internet"],
"domain": "archive.org",
"engines": [
{
"name": "google",
"params": {
"google_domain": "google.co.id",
"gl": "id"
}
}
]
}
Using multiple search engines
curl -X POST http://localhost:3000/api/rankings \
-H 'Content-Type: application/json' \
-d '{
"keywords": ["internet archive", "digital archive", "internet library"],
"domain": "archive.org",
"engines": [
{
"name": "google",
"params": {
"domain": "google.com",
"gl": "es"
}
},
{
"name": "Bing",
"params": {
"cc": "gb"
}
},
{
"name": "duckduckgo",
"params": {
"kl": "uk-en"
}
}
]
}'
That's it! I hope you like this post. Please let us know if you have any questions. Feel free to also contribute to this project on GitHub.
Top comments (0)