Iterators
- Understand the difference between iterables and iterators
- Build custom iterators with
__iter__and__next__ - Use
iter()andnext()explicitly for finer control - Chain iterators with
itertoolsfunctions
itertools, pandas' .iterrows(), and every streaming parser (csv.reader, json.JSONDecoder.raw_decode) rely on the iterator protocol. Staff engineers prefer generators over materialized lists for log processing and ETL pipelines because they compose lazily and hold O(1) memory over billion-row streams.
- Iterating the same iterator twice — it's exhausted after one pass; wrap in
list()or useitertools.tee. - Forgetting to
raise StopIterationin a manual__next__— infinite loops and hungforblocks follow. - Using
.iterrows()on large DataFrames — 100× slower than vectorized ops; reach for.itertuples()or vectorization.
The iterator protocol is dead simple:
__iter__()returns an iterator (oftenself)__next__()returns the next item, or raisesStopIteration
Every for-loop, comprehension, and sum()/min()/max() builds on this.
itertools is your friend
chain,islice,cycle,count,takewhile,dropwhilegroupby,combinations,permutations,product
Generators vs hand-rolled iterators
Generators (with yield) are usually clearer for one-off iteration. Build a class only when you need state or methods beyond __next__.
Try it
- Write a hand-rolled iterator for the Fibonacci sequence.
- Use
itertools.productto enumerate every(rank, suit)of a card deck.
3 graded exercises
Write the code, click Check. We’ll tell you exactly what to fix.
- Exercise 1
Manually iterate over
[10, 20, 30]usingiter()+next(). Store the results in a listpulledand print it. Expected:[10, 20, 30]. - Exercise 2
Write a class
CountUpwith__init__(self, limit)and both__iter__(self)and__next__(self)methods that yield 1, 2, ..., limit and then raiseStopIteration. Test:list(CountUp(4))should return[1, 2, 3, 4]. Print that list. - Exercise 3
Use the two-argument form of
iter():iter(callable, sentinel). Givenimport random; random.seed(0), useiter(lambda: random.randint(1, 100), 42)and pull items until it stops. Print the count of items pulled before the sentinel (should be > 0, deterministic with the seed).