Google Site Index and Sitemap – Working together

Many a times, there have been questions posted up at the WebMasters forums, asking the solution for the discrepancy in the number of pages indexed from your website shown on the Google site index and the Webmasters console.

site:www.yoursite.com gives the number of pages indexed by Google on its regular index while the webmasters console shows the number of indexed pages in the dashboard based on your sitemap submitted.

According to this post by Charlene – there is nothing to worry about it nor is it a bug with Google.

The Google site index operator only displays you an approximate estimate of the number of pages from your site on the Google’s regular index. It isn’t accurate, nor it may have the recently added / new pages. Sometimes it may also have more pages than the sitemap, and this could be due to the additional pages not indexed on your sitemap but found through links.

So essentially, here are some possible conclusions we can make regarding the sitemap and google indexing patterns.

1 – A Sitemap is not an Order to Google

Submitting a sitemap can only help tell Google that – “These files ought to be indexed by Google. And here are they”.
While Google in all possibility will index them, one cannot make sure that they will be indexed, especially when there are lot of files and large sitemaps.
Google has its own ideas when it comes to finding out if a page from a  website is worth keeping in its normal index or not, and submitting a sitemap is not a resolution to it.

Situations when Google might not index all the files on the sitemap.

- When there are files in the sitemap that have no external links/internal links at all .
- When there are pages that are strikingly similar and could be considered as duplicate content.
- Other reasons that tells Google that the pages aren’t relevant or technically correct.

2 – The Google site index operator is a hint of your general site health

Google says -

Think of the site operator as a quick diagnosis of the general health of your site in Google’s index. Site operator results can show you:

• a rough estimate of how many pages have been indexed
• one indication of if your site has been hacked
• if you have duplicate titles or snippets

So sitemaps might not be the surest way to get all the pages indexed on Google. Technically, getting relevant links to each page would be the surest way, but that’s going to be really hard a possibility, so the best solution is to do a mix- and – match of both.

To ensure that Google indexes all (maximum?) pages from your site -

1. Submit a sitemap with proper weightage assigned to each page/category.
2. Get relevant links to categories/subcategories and spread the google juice.
3. Remove possibilities of dangling pages
4. Keep the internal navigation sound so that Google can crawl them all properly.

Possibly related SEO & Social Media Articles

• New site not indexed on Google yet ? Here’s what to do
  One of the most popular problems among webmasters is not being able to get indexed by Google despite search engine submissions and long waiting. Google takes time to index some new sites and this can be frustrating. But most of the times, the reasons for Google not indexing your site is from your side, although [...]...
• Google Sitemap Generator – Efficient and better !
  Now that was a surprise. With all the third party sitemap gnerators available, this one is a Killer ! Google has announced it’s own homegrown Sitemap generator for webmasters that is more efficient than any other sitemap generators available yet. Well, sitemaps are just a bunch of links for the spiders after all, but there’s quite a [...]...
• Submit a sitemap to Google (in whatever possible ways you can)
  Submitting sitemaps to Google just got easier. Now, you don’t have to worry about the sitemap type, location etc, Google will auto detect it. All you have to do is tell them where it is through the Google webmasters tool. Here are the steps you need to follow to successfully submit a sitemap to Google. Be [...]...
• Is your site indexed on Google after all?
  Back to some basics today. For all novices out there – Do you know if your site is indexed on Google? Meaning – Is your site recognized by Google? Or Do your site appear on Google search results? Not sure? Okay goto www.google.com/yahoo.com/msn and search for site:www.yourdomain.com If it gives a result like – “Site not found”, then it [...]...
• Create a free Google Sitemap
  Google Sitemaps is an excellent way to submit your site to Google. If you are not sure of what a site map is - Sitemaps are an easy way for webmasters to inform search engines about pages on their sites that are available for crawling. In its simplest form, a Sitemap is an XML file that [...]...