Crawl Budget: SEO Mythbusting

In the second episode of SEO Mythbusting season 2, Martin Splitt (Developer Advocate, Google) and Alexis Sanders (Senior Account Manager, Merkle) discuss the most common SEO questions and myths around crawl budget.

Specific timestamped topics discussed in this episode:
Why is crawl budget an interesting topic to discuss (0:00)
What is crawl budget? (1:15)
What is crawl rate, and what is crawl demand? (1:47)
How does Googlebot make its crawl rate and crawl demand decisions? (2:44)
ETags, HTTP headers, last modified dates, and similar (3:43)
What size of sites should worry about crawl budget? (4:35)
Server setup vs crawl budget (5:00)
Crawl frequency vs quality of content (6:18)
What to expect to see in one’s log files if Google is testing one’s server? (7:45)
Tips on how to get your site crawled accurately during a site migration (8:18)
Crawl budget and the different levels of one’s site’s infrastructure (9:40)
Does crawl budget affect rendering as well? (10:37)
Caching of resources and crawl budget (11:46)
Crawl budget and specific industries such as publishing (13:34)
What can be - generally speaking - recommended to help Googlebot out when crawling one’s site? (15:03)
What are the usual pitfalls people get into with crawl budget? (16:52)
Can one tell Googlebot to crawl one’s site more? (17:40)

Documentation mentioned in this episode:

Watch more SEO Mythbusting episodes →

#SEO #GoogleSearch

[speedrunfantasychess]

This series is excellent. Martin is an excellent speaker and gives publishers a feeling Google isn’t “out to get us” 😃

[chris-palmer]

*Thank you for this video, excellent! Love these myth busting videos* - WE WANT MORE

[Techjackie]

Interesting to see Martin actually mentioned it quite a few times in the video that Google won't choose to index pages that are not high-quality enough. This basically confirms what some of the SEOs like Marie Haynes have specticulated few months back as of why many people have reported that their content wouldn't get indexed

[hhgj]

TRUNG THACH Trungs2T5n1968 chuc cac ban co CHUA di Cung cac ban bahia Dinh binh AN Y CHA duoc nen A MEN

[dmy_tro]

Alexis: *asks any question*
Martin: Well, it depends...

[ProphetBusinessGrp]

Great information! Looking forward to hearing more in upcoming episodes.

[SeanClarke]

Great video but how many people have sites with a million pages?

[RicardMenor]

What a shame I did not watch this earlier (thankfully this is due to being very busy :)
This is one of the most valuable contents I've learned from for years.
Content is absolutely significant plus format is plain enough to share with customers and project portfolio stakeholders (though highly technical background).
Thanks to all those involved in this production.

[ChaitanyaACV]

I just loved this video - thank you so much - Alexis & Martin!

[Zapzockt]

Very interesting information, I love myth busting, especially with SEO, as there are so many maybes floating around

[davetaylor]

Yes, would be good to add more of these as there a way more SEO myths to debunk!

[cianmurphydigital]

Excellent discussion on crawl budget. This is one for the resources.

[cristhianalmora]

YouTube: Do you recommend this video? Me: Definitely

[pincus]

What happens when google bot goes nuts? Our site went from 160k-260k to 22 million...what's going on?

[marcinlewicki]

6m in where they talk about not updating web pages enough - how often is enough?

[nitishsundriyal]

I see many quotes and Shayari websites they all write the same content on our website so why google give them ranking and why google does not penalize these websites?

[mikemaloon]

GraphQL? Or something like prerender or puppeteer?

[khageshgopalpatel]

Why is the video quality so bad and blurry ?

[ItamarBlauer]

There was definitely a lot of nuggets in there - thank you both!