Crawl Budget Definition & Meaning

What Is Crawl Budget?

Crawl budget is the number of URLs a search engine will crawl on a website on any given day. Crawl budgets are based on two factors: crawl rate and crawl demand.

Crawl budgets are important for websites because the budget allowed for crawling web pages affects how quickly new content is identified and indexed in the search engines. If a website wastes its crawl budget, then new content will not get indexed or ranked in the search engine results pages (SERPs). Additionally, updated content won’t be reassessed for reindexing in the SERPs.

Website owners cannot change the crawl budget for their sites. Crawl budgets are determined by the search engine spiders based on the crawl rate (i.e., the number of requests per second a search engine crawler can make on a website when crawling it) and crawl demand (i.e., how much demand is specified for the URLs based on the popularity and staleness of the pages in the index).

Learn more about the SEO Chatter mentorship

Visit the SEO Glossary

Go to the SEO Glossary to find more terms and definitions that relate to the field of search engine optimization.