SEO Basics: Crawling, Indexing, and Ranking | Lesson 2/31

Learn how the internet and search engines work.

00:00 Understanding Search Engines' Working Principle
00:40 Crawling, or How Google Discovers Content on the Internet
01:14 Indexing, or How Google Stores Crawled Content
01:47 Ranking, or how Google's Algorithm Prioritizes Indexed Pages

✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹
You might find it useful:
Try out Semrush SEO tools!

Learn how to boost your SEO with Semrush tools with our in-depth course:

✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹ ✹

When you search for a keyword phrase, Google is going to provide you with a list of websites that matches the pattern that's related to your search.

Most people don't realize that when they do a search on Google, they're not actually searching the live web. They're actually searching Google's index of the web… the stored copy of the sites that Google has crawled.

Crawling, or How Google Discovers Content on the Internet

Google uses little programs called spiders to crawl the web. The spiders are how Google actually discovers content. Basically, a spider will start on a page and check out all the content on that page, and then it follows the links on that page and looks at the content on those pages. Then it follows the links on those pages to discover even more content, and then the links on those pages lead to more content… The spiders keep crawling at a mind-blowingly massive scale until most of the internet has been crawled.

Indexing, or How Google Stores Crawled Content

As the spiders crawl all of these pages, a copy is stored on Google's servers. This is Google's index, and it's stored in a way that allows Google to quickly search through the billions of pages in the index. When you search for a certain phrase, Google checks the index to get a list of every page that's related to your search phrase.

Here's where ranking comes into play - this is the part that SEOs work to influence. Google uses an algorithm to look at that list of pages it just pulled from the index and rank the pages based on relevancy.

Ranking, or how Google's Algorithm Prioritizes Indexed Pages

Let's say you did a search for “Denver omelette recipe” - Google's going to search the index and come up with a list of every website with a recipe for a Denver omelette. Then it's going to use the algorithm to sort that list so that the most relevant sites will be at the top.

There are hundreds of ranking factors, each with a different assigned weight or value. The algorithm will look at a few hundred different factors that influence relevancy, like the content on the page, the number of other sites that link to that page, and the overall quality of the website.

When we do SEO, we're hoping to influence those relevancy scores. We know that if we optimize the right signals, Google's algorithm will decide that the page is more relevant - that it's a better answer to the question being asked - and show that page higher than the other options in the index.

#SEO #SearchEngineRanking #SEOtutorial #SemrushAcademy

[adrianabarrios]

I am so happy I came across this video, I speak Spanish so the Spanish subtitles helped me a lot, thank you

[semrush]

0:05 Understanding Search Engines' Working Principle
0:40 Crawling, or How Google Discovers Content on the Internet
1:14 Indexing, or How Google Stores Crawled Content
1:47 Ranking, or how Google's Algorithm Prioritizes Indexed Pages

[daisymonsalve]

You provide the information that is needed, you also have the ability to indicate the necessary objectives of crawling, indexing and classification, excellent

[tsungaimubaiwa]

Wow after a lot of searches I have understood the difference between crawling, index and algarythms. Thank you. All in less than 3minutes.

[rafimalik]

It's really wonderful to understand the crawling and indexing. Now clearly understand it. Thank You SEMrush.

[southernfilipino]

Says the guy that showed up first in MY search! Good Job! Love the socks

[MahdiBanners]

Thank you very much for simplified explanation ..very easy to understand

[itachiseyes]

Great video. A legit 10/10. simple, concise .just perfect

[santhoseaghilsanthose]

On first impression, i had no idea on what is crawler in search engine terms. But after seeing the video, i got good insights. Thanks for the information

[hamdanshakoor]

It was simple to understand and I really like it.

[sreekumargopinathanpillai]

The perfect explanation the way you have done it!

[daniloserra]

Very good video, very simple, thanks!

[xavi]

really informative. thanks for sharing

[jakobhuetter]

Best video i could find to that topic!

[akosikardia]

Wow! That was very informative.. thank you so much!. 🍀👌

[TimeDragonGod]

Nice and clear video.
I think the books on the bookshelf are not yet indexed?

[RizwanYahya]

can google crawler submit forms to follow the pages which show after submission ?

[halashahrouri]

عمي و عم اولادي روح يا شيخ الله يسهل عليك الحياة

[ramzialarja]

What do you mean by " google crawler follows the links on those pages to discover even more content..???" what if a page doesn't contain any links.. If Google crawler doesn't find links on my website then it'll not be visible for users?

[samudrachakravorty]

How will Search Engine deal with poor sites? like will the crawler come back for a site again if it doesn't show up a good site structure earlier... I got this doubt. HELP ME OUT PLEASE!!!!