Python Code Review: Refactoring a Web Scraper, PEP 8 Style Guide Compliance, requirements.txt

Python Code Review: Unplugged – Episode 2: Code Review for Milton

This is a Python code review I did for Milton's Python project on GitHub. Milton is definitely on the right track with his Python journey. I liked how he used functions to split up his web scraper program into functions that each handle a different phase, like *fetch the html*, *parse it*, and *generate the output file*.

The main thing that this code base could benefit from would be consistent formatting. Making the formatting as regular and consistent as possible really helps with keeping the "mental overhead" low when you're working on the code or handing it off to someone else.

Besides formatting, the video also covers things like writing a great GitHub README, how to name functions and modules, and the use of constants to simplify your Python code.

Again, I left the video completely unedited. That’s why I’m calling this series Code Review: Unplugged. It’s definitely not a polished tutorial or course. But based on the feedback I got so far that seems to be part of the appeal :D

* * *

FREE Python Tutorials & News:

[piotralanp]

Your reviews put a really big smile on my face! I like watching how you clean up that messy code, and fix everything that drives me nuts in it! It's so relaxing, like popping bubble wrap or observing flowing water.

I think it can be a thing like ASMR, and relaxation music for programmers, did you consider to make a film "10 Hours Of Cleaning Messy Python Code with Rain Sounds and Tibetian Singing Bowls in the Background"? You really should! Or I will do it - and I mean it!

[sunnz]

another great review! I didn't know that you can select multiple variables and change their names at the same time, I shown that to my colleagues today and they were all impressed! Thanks Daniel!

[karlhopkinson-turrell]

Great video thank you. It's just as useful to hear more reasons to consistently format code, as it is to hear about the more complex possibilities for race conditions. If you need any more material for future videos like this, I'd be happy to volunteer some of my code. Keep it up.

[MyselfMil]

Thanks Daniel, I really loved all your suggestions and I missed committing my run me file which will execute the main file.

[legacy_kyleconnolly]

Based on requirements.txt maybe this script was apart of a Flask application?
Love these code reviews!

[yujinyuz]

Thanks for this one! I would probably recommend this to someone after learning the basics. I really don't like it when reading someone elses ugly messy code. Well, as Pep20 says, "Beautiful is better than ugly." I am no great programmer, I just want to follow the best practices even if the program only does is print your name :)

[gelbander]

Thanks for a great code-review session! What color-scheme do you use or is it "home-made" :)? I really liked that you have different color depending on single/double quoates.

[kwomyurii]

Hi Dan, awesome video! Learned a lot today. I'm just curious how did you edit the single quotes at the same time just by highlighting one?

[dpicollege]

could you make video for how to read python document AT docs.python.org ? for example explain meaning of [{}{}{}{}{}]

[mberone]

Why not just run the code and show us what it does.

[SandhanSarma]

Hi Dan! Great series. I think its a very fast way to pro up by analyzing at other people's code.
It would be great if you could also do a code review for a very basic python game I wrote few years back