Once upon a time AJAX was taboo, or many feared to use it because Google couldn’t figure out what it was all about (at least in the code). But AJAX got popular. AJAX is here to stay. Now Google gives up, finds new ways to get around knowing AJAX better.

Google has now proposed a new standard for making AJAX based websites crawlable.

In Google’s words -

This will benefit webmasters and users by making content from rich and interactive AJAX-based websites universally accessible through search results on any search engine that chooses to take part. We believe that making this content available for crawling and indexing could significantly improve the web.

Some of the goals that will be achieved with this proposal.

• Minimal changes are required as the AJAX website grows
• Users and search engines see the same content (no cloaking)
• Search engines can send users directly to the AJAX URL (not to a static copy)
• Site owners have a way of verifying that their AJAX website is rendered correctly and thus that the crawler has access to all the content

And, here’s how Google think webmasters should build their content, so as to allow effective web crawling of their pages.

1. Slightly modify the URL fragments for stateful AJAX pages

2. Use a headless browser that outputs an HTML snapshot on your web server

3. Allow search engine crawlers to access these URLs by escaping the state

4. Show the original URL to users in the search results

And, here’s a whole document on making your AJAX websites crawlable.

More details here on making AJAX crawlable