Increase Google indexing frequency – get the edge over others.
But why?
Indexing frequency can be explained as the count or the number of times Google or the other search engines, visit your website to index your content.
And, the higher the frequency, the better you chances for a better rank or your importance in the SERPs.
Can’t believe? Neither did I, once upon a time.
Well, do you know that the best and the top blogs on the internet gets the best indexing frequencies? That is the content published on them gets indexed quicker that anyone else and gets more importance on the SERPs. Examples are TechCrunch.com and Problogger.net.
Content published on TechCrunch and Problogger gets indexed in minutes. The closest we have measured is like around 8 minutes or so. Yeah that’s right, when a content is published on Problogger, it is very likely that the next moment it is on the SERP’s.
At DailySEOblog, I’ve seen content indexed in 30 minutes or so, which is good.
Now the point here is that, if you are writing the same content as the above blogs, like for say example a movie review or so, it is highly likely that even if you have better content, TechCrunch like blogs are likely to get higher on the SERPs.
So they clearly have an edge of having the crawlers index them faster.
Now, how can you get this edge? How can you make crawlers index your content quicker as well?
Several factors.
1. Rapidly changing content.
2. A good sitemap structure.
3. Good pinging tools.
4. Good meta tags that tell the crawlers to visit more often.
5. Easy navigation structure.
Well, so many other factors as well. But these are the best of them I’d say.
Essentially, if you have rapidly changing content, that is made available to the crawlers through a well maintained and updated sitemap, that itself should take care of the problem.
For maintaining the sitemap, of course you’d be using the AllinOne SEO plugin right? That’s a good one.
Also, make sure that you have a regularity in posting. If the crawlers find that your content is being updated on a regular basis, like daily or hourly, then they will set their crawl frequency(if not manipulated by META tags) to hourly or daily basis, however it is followed on your site.
This is exactly the same reason why blogs have an edge over static websites. It is easy for crawlers to find a pattern in your posts and hence they find it easy to follow the pattern.
You can find more information about this concept over here and here.
dear Mani
thank you for useful information
best regard for you
Indexing in google is speed not only for the big blogs but my blog content also got indexed within 30 minutes.. Indexing is different from cache recording. I will show you a screenshot of the quick indexing in google next time when i publish my next post.
Chetan’s last blog post..Participating in the seocontest2008 ?
Dude,
I have a question on sitemaps created by Google Sitemap XML plugins. Well, in the options page, we are allowed to change the weight placed on posts, home page, category page, etc. Do you think that the default values are good enough?
Wayne Liew’s last blog post..WordPress Plugin: Clean Up Broken Images
great info buddy. Are there any tools through which we can find out how much time does it take for our content to get indexed or we have to find it manually by visiting google.
Madhur Kapoor’s last blog post..Zeiss Cinemizer video glasses for iPod
Its not just the big names that get indexed fast, the Blogs that use FeedBurner as their RSS provider are getting into to Google with lighting speed…
I can send you a screen shot that says,”wrote 1 minute ago”… that’s right, 1 Minute.
But in real time it took about 7 or * minutes to show up.
This lighting speed indexing has just begon after Google bought FeedBurner, go figure
Chetan, That’s exactly the point I’m trying to make. If “ONLY” big blogs were able to make it, then I wouldn’t discuss it for others here. In fact, if you follow certain preliminary stuff, the “so called” small blogs can also achieve the same indexing frequency.
Wayne,
The default values can be changed for your advantage. Having said that, if SE’s find that the page weightage(which can be easily guessed from your posting frequency) is not the same as what’s given in the sitemap details, then the whole exercise is fatal.
Madhur, doing it manually will be more precise than using any tool, which I’m yet to find. Makes Sense?
Mani Karthik’s last blog post..Which is the best satellite television in Chennai?
Madhur, I found one!
You can find a tool that will help you find the crawl frequency here – http://www.seometer.com/
Mani Karthik’s last blog post..Increase your indexing frequency – get the edge over others.
Nice post…I do agree that posting frequency has a lot to do with indexing frequncy…
Techblissonline Dot Com’s last blog post..MySpace to launch developer platform
Mani,
I get indexed in under 10 minutes and my blog doesnt have any gang bang content or doesnt have super links.
I believe its all about getting your content “travel” and make it travel real fast.
Think about it laterally, there are two ways in which a bot crawls you:
1. It DECIDES to crawl you.
2. It follows your link from elsewhere.
Make use of that thing called the “RSS Feed”
( That was a hint )
Ashwin’s last blog post..Is It Okay To Submit Your Own Articles To Content Voting Sites
That’s a good point there Ashwin.
Mani Karthik’s last blog post..Increase your indexing frequency – get the edge over others.
I have heard this same concept on courtneytuttle as well….
btw, missed you at the barcamp in cochin….was hoping to meet you up.
Hope it was a nice one Anand. There’s always another time or another conference.
Even though I don’t post that frequently, my posts get indexed within 20 minutes. It looks like the Google indexing speed has improved a lot.
For a second there I thought you were going to say to set a higher priority on your XML sitemaps lol
Very good points there. I believe the two main factors are PR (not toolbar PR, but actual PR or “link juice”) and frequently updated content.
Josh Spaulding’s last blog post..How Often Should I Update my Web Pages?
Sorry about the bragging, but just came back to tell that my last post: Taking your users for granted was indexed in less than 2 minutes. Wow!!!
Nice blog.. Nice information.. nice contents..
I got lots of knowledge from here..
Cheers guys..
Nice Blog and Simply Nice Information Dude. Thanks
Visit my Blog for More Latest Movies and Games @ http://directlinksheaven.blogspot.com
Good article!! thanks!!!
I created a blog last friday and it still has not been crawled, does anybody know why?
http://mactipstricksandhacks.blogspot.com/
Hey,
That’s a nice piece of info….
Is there a specific time frame that Google follows for refreshing the Google’s search positioning… If yes, please let me know about it.
FourPx
http://www.fourpx.com
Thanks for the useful info. We suffer because of this. In our very competitive business of Rome vacation rental, we change our website not so often, and Google re-indexes us every 5-10 days, while we have some competitors using smar techniques which are indexed every 1 or 2 days. Every time Google reindexes (you can see it from the cached version) it generally improves the SERPs.