Pythonic code: Tip 4 Processing large data sets with yield and generators

Any time you are writing a method that returns a sequence (especially if that sequence is a list), I recommend you pause and think whether that method should actually be a generator. They can add tremendous flexibility and performance to your app.

What's a generator? Well, with regard to methods, it involves the keyword yield rather than return keyword in Python. The best way to describe how yield works is to write a method which is NOT a generator.

This video will show you how it works.

[zeppelin]

Unbelievable. I just watched ~3 videos to learn this. One video explained yields and why they give better performance, but his examples were extremely meager AND he barely glanced over how yields actually work! I can't believe that video had 15, 000 views and was twice as long as this one.

Great tutorial. You should make an extension of this video to demonstrate the time/memory these functions consume with yields versus returns.

[BharatKumar-mkdc]

Michael Kennedy its a nice video I like the example you provided, Can you tell me which editor are you using? it really looks cool

[fatimak]

i have way too much more learning to do. have NO IDEA what's going on here.
currently working with (what i thought was) a somewhat large dataset (50krowsx100columns). Have an assignment to:
1 - Clean and Tidy Dataset
2 - Build Machine Learning Models
3 - Make Predictions
i came here to see what was best way to save/load large datasets, but now im more confused and eventually want to know more about yield/generators and COULD I use that in my assignment?