~/problems / Iterators & parsers / Iterators and generators

Iterator with Filter

easy ~15 min Coinbase

A price feed produces trades one at a time, and a dashboard only wants some of them (say, trades above a size threshold). Wrap the feed in an iterator that hands out only the items a test function accepts, in the order they arrived.

Implement class FilterIterator:

  • FilterIterator(source, keep): source is any iterable (a list, a generator, possibly infinite) and keep(item) -> bool decides whether an item is wanted.
  • has_next() -> bool: True if at least one more wanted item is left. It may read ahead in source to find out, but calling it several times in a row must not skip or lose anything.
  • next(): return the next wanted item and move past it. If there is none, raise StopIteration.
  • It also follows Python's iterator protocol: iter(it) returns it itself and __next__ behaves like next(), so for x in it and list(it) work.

Rules:

  • Items can be anything, including None, 0, False or "". Those are ordinary items: whether they are returned depends only on keep.
  • Lazy: keep is called exactly once for each source item that gets read, and the iterator reads from source only when it must: has_next() and next() stop reading as soon as they have found the next wanted item. Never turn source into a list.
  • Once exhausted, has_next() stays False and next() keeps raising StopIteration.
it = FilterIterator([4, 7, 10, 3, 8], lambda x: x % 2 == 0)
it.has_next()    # True
it.has_next()    # True (still 4 waiting)
it.next()        # 4
it.next()        # 10
list(it)         # [8]
it.has_next()    # False

import itertools
sq = FilterIterator(itertools.count(1), lambda n: int(n ** 0.5) ** 2 == n)
[sq.next() for _ in range(4)]    # [1, 4, 9, 16]

list(FilterIterator([0, None, "", 5], lambda v: not v))   # [0, None, ""]
Show hint

keep a one-item buffer plus a separate flag saying whether the buffer is full. has_next() fills the buffer if it's empty (pulling and testing items until one passes or the source ends); next() calls has_next(), then empties the buffer. A None sentinel breaks as soon as None is a real item.

Topic: Iterators and generators. Resumable/serializable iterators, merging streams, lazy pipelines.

0:00
Ctrl ' run · Ctrl ↵ submit
esc