What’s special about Live search new crawler msnbot 2.0b

To crawl and collect more information from the web pages, live search has taken a step forward by implementing a new crawler msnbot/2.0b.

Currently this new crawler is under testing process and will replace the current one (msnbot/1.1) soon. This new crawler has come up with entirely new algorithm to crawl & understand the web pages.

What must be the special things in this new crawler?

1) Efficiency to crawl the special characters
2) Exactly obeying for tags like no follow, no index etc
3) Exact algorithm to pass back link weightage from a page having back kink.
4) Exactly obeying for robots command like allow and disallow.
5) Better understanding about Images.

These are the new things that I have guessed in msnbot/2.0b. Also the new spider will obey the current robots.txt protocol and we don’t need to change anything in our robots.txt. Lets expect a good and relevant results from Live in future.

Click Here to read the official announcement from MSN on their new crawler release.