~/problems / Iterators & parsers / Iterators and generators

Basics: an iterator class that yields chunks

easy basics ~10 min

Python's for loop works on anything that follows the iterator protocol: __iter__ returns the iterator itself, and __next__ returns the next item or raises StopIteration when there are none left. Write one by hand.

Implement class Chunks:

  • Chunks(iterable, size) wraps any iterable (a list, a string, a generator, even an infinite one) and yields its items in lists of size, in order.
  • The last chunk may be shorter. Never yield an empty chunk.
  • size must be a positive integer; otherwise the constructor raises ValueError.
  • iter(chunks) returns the object itself, and once it's exhausted every further next() keeps raising StopIteration.
  • It must be lazy: pull only as many items from the source as the chunk you're building needs. Don't turn the whole input into a list up front, because it may never end.
list(Chunks([1, 2, 3, 4, 5], 2))   # [[1, 2], [3, 4], [5]]
list(Chunks("abc", 5))             # [["a", "b", "c"]]
list(Chunks([], 3))                # []

import itertools
c = Chunks(itertools.count(), 3)
next(c)   # [0, 1, 2]
next(c)   # [3, 4, 5]

Write it as a class with __iter__ and __next__ (not a generator function); that's the skill being practised.

Show hint

keep iter(iterable) in an attribute, and in __next__ call next() on it up to size times, stopping early when it runs dry; an empty chunk means you're done, so raise StopIteration.

Topic: Iterators and generators. Resumable/serializable iterators, merging streams, lazy pipelines.

0:00
Ctrl ' run · Ctrl ↵ submit
esc