Crawl

Crawl

A crawl, or web crawl, involves the process of spiders traveling from site to site, gathering information for search engines to index all while paying attention to broken links, new sites and changes to existing sites. The internet is a constantly changing place, and crawls allow search engines to have a better idea of what is out there at all times. During a web crawl, spiders gather information about site content based on keywords and backlinks, and use that to help keep SERPs accurate and relevant.