The Issue
Ever since launching my site, https://irian.to/, I found that I couldn't google any page on my site. For example, when I google "irian.to redux" (my redux article), only site home page showed on google result. (This was done 2/1/2020.)
The Solution
I went to Google Search Console, then I checked URL inspection and entered "https://irian.to/blog/redux-101". Google notified me that it didn't exist.
To make that page searchable, I needed to tell google to index that page. Here's how you can do it:
- Visit search.google.com
- Click your domain (you need to add it if you haven't. Check here to learn how)
- URL Inspection
- Enter the site URL
You can request for indexing ("Request Indexing"). I usually "Test Live URL" before doing that to make sure that it returns green.
In this case, it does.
When requesting google for reindexing, it may take a few days. Google also tells you that:
URL was added to a priority crawl queue. Submitting a page multiple times will not change its queue position or priority. Learn more.
I checked my google search again on 2/3/2020 and it showed up on search.
The Conclusion
Moral of the story? If you own personal site, make sure people can google your page! 👀
Added 2/4/2020
Thanks to helpful inputs in comments (check below 👇), here are more things you can do to help with your site's discoverability!
Adding sitemap.xml
Per suggestions below, it is best to add sitemap if you have many pages. For more information on sitemap, check this sitemap 101 article or moz article on search engines (go to sitemap section).
Thankfully, there are many sitemap generators. I created mine with Nuxt sitemap-module.
Testing your site index
How to quickly test indexed pages?
If you do site:yoursite.com (in my case, I did site:irian.to), google will show all results google have.
If you see screenshot below, you'll see I have 68 results (at the moment of writing), which is about right.
Top comments (18)
Hey Igor,
You can google this "site:your_site" to see the indexed psots by google.
For example, site:ajeet.dev
On a side note if you tried
site:facebook.com -www
You would get all the subdomains that are indexed by google.
Oh nice! What's the difference between googling with site: and without it? (What is "indexed" search?)
Quoting from this site: shoutmeloud.com/google-crawling-an...
"In layman’s terms, indexing is the process of adding web pages into Google search."
If you use "site:your_site", you will get a list of your indexed posts/pages/categories/etc that are indexed by google.
You should also read this article - how search engine works :)
moz.com/beginners-guide-to-seo/how...
Just did this today at work. We were researching something and my coworker suggested using "site:further_query" method. Much thanks for sharing Ajeet!
Your welcome 😊
This seems like an appropriate place to remind folks that they're able to use canonical URLs when posting on DEV!
From the DEV FAQ:
If you are sharing any work on DEV that you previously posted elsewhere, it's a good idea to add "canonical_url: blogname.com/postname" to the Front Matter of the post if using the "basic markdown" editor. If using the "rich + markdown" version of the editor, just click on the three-dots menu when writing your post and you'll see the option to input your canonical.
By the way, if you're not quite sure which editor version you're using or didn't even realize that there are different editor versions, 🤯 just go here in your settings and poke around!
Very nice!! I wasn't aware of this feature either. I am going to go back to every single blogs that I cross-posted and adding this.
You are awesome. So glad you posted this. 😎
Btw for those who don't know canonical url: yoast.com/what-is-a-canonical-url/
No problem! Happy to help. 😀
And good thinking on providing the explanation of what a canonical url is — definitely important for anyone who plans on cross posting to be aware of this!
You should submit a sitemap.xml file to Google, rather than manually adding each page.
Agree with Patryk. Generate a sitemap and submit to google console periodically. Sitemaps looks like this: ajeet.dev/sitemap.xml
Is it a good practice to update sitemap and submit to google every time I add new page?
If not, what's a good frequency?
Normally, as long as Google knows to index your site and follow links, it shouldn't really be necessary, but it doesn't really hurt. It's just a PITA to do it manually.
You can always submit a
sitemapindexto Google, and generate a new sitemap every time you add a page (I presume you mean blog posts). Then Google will pull your sitemap periodically, and should update its index without intervention.If you can automate the sitemap generation as part of your workflow (e.g. in the
savesignal on aPostmodel in Django), that would require basically zero manual intervention.Google can also discover your site on its own if another site links to it. This + submitting my sitemap initially is usually enough for my site.
If you're cross-posting content to DEV from your blog you could try adding an "Originally posted on " line to the DEV post.
I knew that more cross-referencing (backlinking?) helps with SEO but didn't know that it helps with unindexed files discoverability. Lots of good insight here 👍
Thank you for an easy to follow guide! 🙌Making my site searchable didn't cross my mind when I created my website recently! I managed to set up Google indexing on my own site successfully.
Regarding the sitemap, I submitted it specifically to the Google Search Console here:
When submitting the sitemap a popup told me that Google would index the sitemap automatically and would let me know if there were any problems indexing the sitemap in future.
Glad you found this helpful 👍
The method you described is useful also, especially if the page has not been indexed already.