|
They are all synonyms for an application which visits each link found on a web page and indexes the contents of the page for search engines to use.
Web crawlers, web bots (robots), web spiders, web slurpers, etc.
Crawlers we've had on our site so far this month: Googlebot, MSNBot, GigaBot, Inktomi Slurp (this is what Yahoo uses), The Web Wombat, MSIECrawler, Alexa (IA Archiver), plus a few others. Google, MSN and Yahoo (Inktomi) generally are the busiest and most effective.
__________________
Sim'
This is a general comment only and does not constitute advice. Before making financial decisions you should seek advice from a professional adviser, who can take into account your specific circumstances and investment goals.
|