~/problems / Streams & durability / Serialization and durability

Encode and decode with RLE and bit-packing

medium ~30 min Databricks

Columnar storage engines compress integer columns with a mix of two schemes. Build an encoder that turns a list of non-negative integers into a list of readable run strings, and a decoder that turns them back.

def encode(nums: list[int]) -> list[str]
def decode(runs: list[str]) -> list[int]

There are two kinds of run string (note the single space after each comma):

  • Run-length: "RLE[v, c]" stands for the value v repeated c times (c >= 1).
  • Bit-packed: "BP[w, k, h]" stands for k values (k >= 1), each stored in w bits. Value number j (0-based) occupies bits j*w to j*w + w - 1 of one big integer, so the big integer is v0 + v1 * 2^w + v2 * 2^(2w) + .... h is that integer in lowercase hexadecimal without a 0x prefix ("0" when it is zero).

encode must produce exactly this output, so the result is deterministic:

  1. Split nums into maximal groups of equal adjacent values.
  2. A group of length 3 or more becomes one RLE run.
  3. Values from shorter groups are collected, in order, into a pending list. Just before an RLE run is emitted, and at the very end, the pending list is flushed: cut it into chunks of 8 values from the left (the last chunk may be shorter), and emit one BP run per chunk with w = the bit length of the largest value in that chunk, but at least 1.

decode must reverse encode, and must also accept any other well-formed list of runs: RLE runs with any count >= 1 (even 1 or 2), and BP runs with any k and any w wide enough for their values, with hex digits in either case. encode([]) is [] and decode([]) is [].

encode([0, 0, 0, 0, 5, 1, 2, 9, 9])
# ["RLE[0, 4]", "BP[4, 5, 99215]"]
#   5 + 1*16 + 2*16^2 + 9*16^3 + 9*16^4 = 0x99215   (w = 4, since 9 needs 4 bits)

encode([3, 3, 7, 7, 7, 1])
# ["BP[2, 2, f]", "RLE[7, 3]", "BP[1, 1, 1]"]

decode(["RLE[4, 2]", "BP[8, 3, 0A0B0C]"])
# [4, 4, 12, 11, 10]

Constraints: up to 300,000 values, each below 2^31. Both functions should run in linear time; beware of building a result by repeated slicing or pop(0) from the front of a list.

Show hint

For encode, walk the list with two indices to find each group, append short groups to a pending list, and write a small flush() helper. To pack a chunk, OR each value shifted left by j*w into an int and format it with f"{x:x}"; to unpack, shift right by j*w and mask with (1 << w) - 1.

Topic: Serialization and durability. Length-prefixed encodings, write-ahead logs, checkpoints and recovery.

0:00
Ctrl ' run · Ctrl ↵ submit
esc