HELIUM for simple DYNAMIC web scraping with Python

After using Selenium a lot to scrape dynamic sites i thought i would give Helium a go - its built on Selenium but is said to be lighter and easier. it comes with some handy features out of the box and is installed with pip.

-------------------------------------
-------------------------------------
Disclaimer: These are affiliate links and as an Amazon Associate I earn from qualifying purchases

[farmersneed]

I used Selenium to automate testcases in an old school commerce website. Navigating iframes was like navigating the circles of hell. This looks very promising and I will use this for present and future prospects. Very good presentation. Thank you.

[samwisethelanky]

Awesome, great video, love how easy it is! Just tried scraping with Javascript, took me all day yesterday to achieve what I just did with Python + Helium in about 5 minutes lol. You've gained a subscriber here :) Thanks

[EmanueleCannizzaro]

John
Thank you for the introduction to this tool.
I tried it and I spotted that it has a strict dependency to Selenium 3.
I played with the code few minutes and I got it working with selenium 4 and the latest version of Chrome.
I have also submitted a pull request to Michael.

This is the power of open source and knowledge sharing. 😀

[xilllllix]

i've ignored helium for a long time bec i don't see the need for it but this vid convinced me to give it a few tries

[dzeykop]

Thank you John, great video again. Easy to understand.

[JohnMusicbr]

Excellent again. Saved my day. Thank you.

[Klausi]

I Just started with BS4...but Helium looks nicer. Thank you for the good tuts

[AgeRon]

Very good tip, thanks I'll try it!

[tejasgalande]

Thanks buddy. I got the new corner to explore.

[martpagente]

I hope you can also talk about freelancing web scraping services, talk like how to price your service, scope etc.

[dishydez]

Thanks a lot! This was great - got it working!

[jonathanfriz]

Thanks man, way fast and easy that only with selenium. Thanks for share. ¿Part 2?

[CompThatHouse]

John, i have been watching many of your videos as you have a great way of explaining Web Scraping. I upgraded from requests and selenium and and started using Helium about 6 months ago. I also found the X PATH explanations very helpful. Thank you so much.
I also like how easy it is to understand as you speak very clear English which is helpful to me.
Today, i upgraded my Helium and Chrome and it all came crashing down. wont go onto the web. Wondering if you or anybody else has this problem, and what to do about it.

Tarlton John

[Robls]

Subscribed to your channel. This channel offers priceless content. Thank you sir for posting all these awesome videos.

[NeilSpeed]

Really like this approach, this ssems super valuable. Any ideas on how to get "everything" ie... Page Down until it won't let you not page down any further?

[coyoteden]

Wow. So simple. Is this your preferred library for dynamic scraping?

[fatimaelmansouri]

Awesome! Could you please do a video on how to webscrape using APIs ? (when neither BeautifulSoup nor Selenium can detect the html)

[diegomairena]

Hey John, great video, thanks a lot. Can you mix BeautifulSoup and Helium? So use bs4 to output a series of links and for Helium to go into each link, find a button, press it and output data?

[ns-jw]

Excellent tutorial! Do i need to scroll to the bottom a dynamic website to get all results? If yes, then do i loop press(PAGE_DOWN)? Thanks!

[mattmovesmountains]

Wow - didn't know about helium until this video, but I may start using this moving forward. Have you noticed any shortcomings since you've been working with helium? Any tradeoffs to the ease of use? Seems like even if there were any, they could be mitigated by adding standard selenium commands where needed.