~/problems / Streams & durability / Fault-tolerant work queue

OA: Fault-tolerant work queue

medium 3 levels ~60 min OpenAI

Level 1 Reserve, complete, fail

Build WorkQueue(timeout=None, max_attempts=None). The constructor arguments come in later levels. At this level there are no timeouts and no retry limit.

Every method takes now (an int clock) first. Across calls now never decreases. You can ignore it until level 2.

Each item is in one of four states: "ready", "reserved", "completed" or "dead" (you'll need "dead" from level 3).

  • enqueue(now, item_id, payload) -> bool adds a ready item at the back of the queue. Returns False and changes nothing if the id has been seen before.
  • reserve(now) -> tuple | None takes the item at the front of the ready queue and marks it reserved. Returns (item_id, payload, token), or None if nothing is ready. The token is an opaque value that identifies this particular reservation. Every reservation must get a token that has never been handed out before.
  • complete(now, item_id, token) -> bool: if token is the item's current reservation, the item becomes "completed" (a final state) and the call returns True. Otherwise it returns False and changes nothing.
  • fail(now, item_id, token) -> bool: same check. The worker gave up, so the item goes back to the back of the ready queue.
  • state(now, item_id) -> str | None returns the state, or None for an unknown id.
q = WorkQueue()
q.enqueue(0, "a", {"n": 1}); q.enqueue(0, "b", {"n": 2})
_, _, ta = q.reserve(1)          # ("a", {"n": 1}, <token>)
q.fail(2, "a", ta)               # True; the ready order is now b, a
q.complete(3, "a", ta)           # False: that reservation is over
q.reserve(4)[0]                  # "b"

Queue operations and id lookups should be O(1).

Level 2 unlocks when level 1 passes.

Level 3 unlocks when level 2 passes.

Topic: Fault-tolerant work queue. Leases with deadlines in a min-heap; explicit task states.

0:00
Ctrl ' run · Ctrl ↵ submit
esc