How to Create robots.txt File

1. if you want to let certain parts of a website not be crawled by Googlebot
2. if Googlebot is requesting to many URLs thus causing server performance issues.

Basically, to be able to control Google to not index certain URLs on a website, you will need to use other methods such as noindex.

RankYa is a passionate digital marketer, website optimizer, content creator, and a fully qualified web developer helping businesses of all sizes (big or small) to achieve better results online.

We love sharing our passion through freely available how to videos and courses related to Google (Search Console, Ads, Analytics, YouTube), SEO, HTML5, Structured Data and WordPress on RankYa YouTube channel.
Rest assured that these proven insights will serve you well as well.

Securely purchase your new private SEO courses online (instant access). Insights that also come with Guaranteed Results for your business website.

[alizaidi]

this is an amazing tutorial. thank you sir

[makingtheweb]

Do I create this in text editor such as atom or visual studio? Isn't the root index.html?