Web Scraping with Beautiful Soup - Make Databases from Scratch

In this video we'll extract information from web pages and store it in a CSV file.
STEP 1. We'll scrape a webpage with Beautiful Soup.
STEP 2. We'll fine-tune the extracted information with Regex.
STEP 3. We'll store the information in a DataFrame.
STEP 4. We'll save the DataFrame to a CSV file.

Webpage URL:

Jupyter Notebook Code:

Read Blog Post in Medium:

Read more about Beautiful Soup in the Documentation:

[iansjackson]

Could I respectfully mention a slight efficiency in getting onto one line "item = item.text.replace('\n', ' ').

[jesusarmentasegura]

U r so clean at explaining and makes it look so easy. U makes me feel like a senior programmer although I barely know how to turn on my PC. Thank you very much.

[webscrapingwithandy]

⭐️ Timestamps ⭐️
0:59 - import libs
1:33 - load html code from a url
2:37 - using Chrome Inspector (DevTools)
3:06 - find all function names
4:53 - find all function descriptions
7:02 - store the data inside a DataFrame
8:50 - export the data into a CSV file

[jaznarossi]

This was INCREDIBLY helpful! Thank you for explaining all of this so clearly.

[MrJackod]

I saw a 3 hours course about this but it was hard for me, now you did it so easy and I understood everything!!! Thanks!!!

[securitydogma]

I love this channel. I'm learning so much from these fantastic tutorials.

[johntessier]

I appreciate you making this, I am currently using this and your example code to do some scraping for work. I have no real prior knowledge of using python, although I do have a background in computers.

This is very helpful, although I have to scale the script to work for multiple sites.

Thank you. :)

[billreed]

This was good, clear, and concise. Thank You

[dorotamarkowska]

It is so simple if you are explaining it! Thanks a lot.

[sebastianmt]

Beautiful Teacher !!, thank you very much !

[srini]

Excellent, you are my teacher, explanation is very simple.

[mmchaava]

very good presentation, easy to follow. thank you

[angelfoodcake]

As usual, a great tutorial to follow along. I don't know though why my lists are both len 25 :D
I got confused about the jumping to a second soup.findAll. I would have liked to see that second example finished to the end to where you get the DataFrame, but it's straight back to the previous example. Specifically, I would have liked to see more on re usage to filter out all the stuff around the useful information because it comes with more I feel like. I found that straight to_csv also works also in colab, it then shows up in the left panel under files from where it can be downloaded. But connecting your own drive is more elegant, of course. Fab job. I'm really enjoying these tutorials you put out! Keep them coming :)

[tav]

Big respect: Beautifulsoup in 10 mins. Wow

[cccccccccccccccc]

Legal, curti bastante, muito interessante e bem explicado, vou usar bastante seus vídeos como guia

[thesouthsidedev]

You just earned a new sub keep them videos coming

[neilgyverabangan]

The best! Thank you soo much for the tutorial! I hope you'll make a video about scraping LinkedIn profiles from a search result. Thanks!

[vijaykumarlokhande]

Good Work. It is the shortest one and the useful one.

[_mytube_]

I like that relaxing music.. as this can be stressful

[nczioox]

Nice! I've been using BS and selenium to scrape and store in a Heroku Postgres database. Its been an awesome small project to play with