Python Tutorial: Browser Automation & Web Scraping with Selenium - Part 2

In part 2 we auto-login with Selenium then use Python BeautifulSoup to scrape the contents of the pages to create a Pandas dataframe at the end.

[trinb]

I learnt more in 30mins than what I learned in 4+ hours from studying the same subject matter in a WebScraping book

[sarcasmasaservice]

Thanks for these tutorials, you have an excellent teaching style. I look forward to sharing your videos with my students as supplemental materials (and "scraping" them for potential assignment ideas). Keep up the great work!

[Zancb]

Great video! I love your example use of Python code in each of those sections. Very helpful in visualizing the code being executed and the data being returned. Thank you very much for putting these together!

[toobstr]

Just stumbled across your channel and this is some of the best content I have seen related to Python! Your teaching style is fantastic. I hope you keep making these videos. It would be cool if you explored making a video on using Selenium, scraping data, adding that data to a postgres db, making a UI that displays that db data. Oooor another thing I've been wanting to try is taking the data I scraped and adding it to an Airtable sheet for a quick and easy/shareable visualizer. Anyways, keep it up, really enjoying it!

[pursuing.perfection]

Keep doing what you are doing this is GREAT

[thewilltejeda]

Definitely some of the best scraping tutorials I’ve found for sure !

I’m curious if you have anything planned for crawling with scrapy

[muhammadawon]

Thank You for bringing high-quality education with super easy conceptual techniques for all levels of learners.
Just a thing, where someone should ask questions for a specified problem regarding web scraping? Thanks

[SOFRADAKAOS]

insane tutorial thanksss! also your beard looks really good

[alexkotov]

Hi!
Your videos are great, it helps me a lot. But one thing I still can't completely understand is how to work with network page in DevTools, how to pick an element you need etc. Would be really interesting to get a little bit deeper. Watched all videos so far and still couldnt find explanation.

[lemontap]

I like your beard, also your tutorials are awesome I have learned alot going through your videos. Thank you!!

[creapygames]

we need new tutorial the driver thingy was changed, but thanks alot for you content is really cool :D

[originalkundukulangara]

The part where you described calling the function login() and process_products()...Does this work in python IDE because when I use similar code in pycharm by getting all the code from jupyter notebook, it doesn't bring anything when I call login function

[sandeepyadav]

thanx for username and password. It still works ;-)

[elshroomness]

Omg! Such an awesome video!

Quick question. Selenium has a known issue with drag and drop function. Do you know a work around for it? I've been stuck on this isssue for two months now.

[ShahzaibAnwaar]

Just came across your channel and went through all the videos this week. Top notch content mate. I noticed you haven't uploaded anything new for a while. Hope all is well. Will you resume uploading new content anytime soon?

[roberthuff]

Thank you! Late, but did you ever release your code as a Collab resource? Yes, I am lazy.

[snawfel]

Thnks
how would you login into an log.asp site ? I would like to scrape data behind ASP login
plz help

[netbin]

hey 👋 do i need to raise up my hands every time i run login function?

[djordjevojimirovic]

Great video! Which IDE you are using in this video?

[jonnyboii]

Hey 👋thanks for the amazing tutorial. I've got a small question. After web scraping how do you place an order on items that meet a certain criteria?