Python Web Crawler Tutorial - 1 - Creating a New Project

[thenewboston]

I uploaded all the source code to GitHub (link in the description) so if anyone wants to help out with development feel free :-)

[slavkodem]

This was my first python tutorial and Im now a junior dev, thank you

[ThePirates]

The timing is impeccable cause i was just thinking about making one.

[briancrane]

Hey thank Bucky! I've wanted to do something like this for a long time. Also I want to learn python, web scraping, web crawling and all the data-sciency kind of stuff. Thanks for all the hard work and the leg up!!

[dayanh]

Short and to the point. Thank you for bringing this tutorial to us. ^^

[pallavhkust]

I'm not gonna pay 100 bucks a year. - ha ha .. loved it man

[peterrusznak]

Cool stuff. Well done. Subscription earned

[nozarsafari]

there is crawl() in create_jobs() that call create_jobs() again it cause RecursionError: maximum recursion depth exceeded

[AndyT]

did he say hello hostage at the start of the vid?

[Thechomania]

It says Cannot run program (in directory CreateProcess error=193, %1 is not a valid Win32 application

[nakshatranahar]

well the crawler even end after crawling the page

[jakeanderson]

Here is the code that he typed out:

import os

def
if not os.path.exists(directory):
print('Creating project ' + directory)
os.makedirs(directory)



Your are welcome P)

[QuirtSatterfield]

I've been watching this tutorial and wanted to know if I am allowed to run the tutorials you have created on this specific website?

[sangrampatra]

it only works in your website only, i have tried many other sites but its not workig, help me out

[aditiaggarwal]

Hi, I followed your awesome tutorial and now wanted to make a custom search engine for my website...Can you please help me with that? Thanks.

[raghavrastogi]

Sir in video lecture 12, can we gather links by using beautifulSoup too? If yes, then how?
Please help me out sir

[raihantanvir]

Can one get access to all websites ?!If not how to overcome that ?!

[pedroaquino]

My Uni project from 2010 :D you too long mate, this would have been useful then hehe

[VivekKumar-qxmy]

how can download all the photos(more than 1) from a website by python
make detailed vedio on that please

[ShiroiWoolf]

hey bucky can you make a tutorial on using selenium for crawling through websites that use javascripts to render?