Crawlers (or bots) are used to assemble info out there on the internet. By using web site navigation menus, and studying internal and exterior links, the bots begin to understand the context of a page. Of course, the words, photographs, and other information on pages also help search engines like go