The Search Engine Blog

Google is experimenting with new directives in the robots.txt file. Here is a great article to read regarding robots.txt and what you should be doing before Google fully implements there new methods. (Don’t panic….it’s really a good thing.)

Validate your robots.txt – Googlebot becomes smarter