Iterators vs. Generators in Python : Data Science Code

Can Generators help speed up your data science code?

[steevensonemile]

If you're watching this video in 2023, please note that xrange has been removed from Python 3, and the range function now behaves like xrange did in Python 2.

[antonboiko]

Hey! Thank you for the video, the main idea behind usage of generators and iterators was clearly explained.
I'm wondering is there any explanation why we have the picks on the graph? Just a little doubtful concerning the time evaluation method.

[garrettwill]

I personally would redo the test at the end, as I think it was misleading. I tested it on my laptop and my PC and experience no speed difference in small numbers for a single iteration. On my pc the difference between a list comprehension [x for x in vs a generator only makes a difference when my memory exceeds a certain threshold. And i would show an example like with open (file.txt) as file for row in file <- a generator to grab / create the next line. to show how each row in a file is a generator which is very helpful since your file can be gigabytes and you don't want to load that whole file into memory.

[sungchulyonseiackr]

I think you have to compare memory usage, not time to see the real advantage of generator against iterator. When I use a generator, my major concern is memory, not time. Suppose you have 100GB data to handle. I saw such big data from astronomy. You cannot load that big one on memory. However, this can be handled by a generator.

[chap_eau]

great video, useful and clear, thank you!

[ramarao_amara]

In python3, seems range is not a list, it will returns range object, like xrange in python2
and xrange in python2 & range in python3 are Iterables not Iterators/Generators

import collections
issubclass(range, collections.Iterator) -> False
issubclass(range, collections.Generator) -> False
issubclass(range, collections.Iterable) -> True

same as with xrange.

and its not loading entire data into memory
import sys
sys.getsizeof(range(100)) -> 48
-> 1008


issubclass(collections.Iterator, collections.Iterable) ->True
issubclass(collections.Generator, collections.Iterable) ->True
issubclass(collections.Generator, collections.Iterator) ->True

** python iterator is much more memory efficient than generator **

[ashishk]

Is range have more space complexity than xrange
And xrange has more time complexity than range ?

[pulkitaneja]

I never drop comments on youtube videos, but @ritvikmath YOU ARE AWESOME. Beautiful explanation :)

[kaushit]

it's wrong generator and range function both are similarly in terms of memory usage that means xrange and range both generate elements on fly without pre loading whole range in memory first, just double checked things by coding itself.

[CardiganBear]

For an integer n, loading each value into memory is ridiculous.

[sdaiwepm]

Iterators and generators are not distinct - generators are a subset of iterators.

[sgamer]

maybe also touch on iter() and next() vs a generator object =(for *num for num in range(10) ) for example

[sandyjust]

Python 2 xrange to Python 3 range. I think in python3, now both are same.

[jay_wright_thats_right]

I'm not feeling the neck beard but the chest hair is super sexy.