~/techniques/bit-manipulation

Bit manipulation

Read a number as a row of on/off bits. AND, OR, XOR and shifts test, set, clear and count them in a step or two.

what

Treat an integer as a row of bits and change or query them with AND, OR, XOR and shifts. `x & (x - 1)` drops the lowest 1.

use when

Flags or small sets packed into one integer, pairs that cancel out, powers of two, IP addresses and masks.

time

O(1) per operation; O(number of 1s) to count them

space

O(1)

You’ll recognise it when

  • A handful of yes/no flags (weekdays, chosen items, visited cities) would fit in the bits of one integer.
  • Every value appears twice except one, and you must find the odd one out with O(1) extra space.
  • The question is about the binary form itself: count the 1s, reverse the bits, test for a power of two.
  • You’re handling IP addresses or masks, where the top k bits of a 32-bit number mean something.
  • The rules ban +, or ask for O(1) space where a set would be the obvious answer.

It sits next to bitmask DP, which uses these same masks as the states of a DP table. This page is the toolkit that DP is built on.

The idea

Think of an integer as a row of light switches. Switch i is worth 2^i, so 13 is 1101: switches 0, 2 and 3 are on. A bitwise operator works on every switch at once, in a single machine instruction.

Operator Bit by bit 12 (1100) with 10 (1010)
a & b (AND) 1 only where both are 1 1000 = 8
a | b (OR) 1 where either is 1 1110 = 14
a ^ b (XOR) 1 where they differ 0110 = 6
~a (NOT) flips every bit ~12 = −13
a << k moves bits up k places, 0s fill in: times 2^k 12 << 1 = 24
a >> k moves bits down, the low k fall off: divide by 2^k, rounding down 12 >> 2 = 3

One bit at a time

1 << i is a number with only bit i on. Combine it with x to test, set, clear or toggle that bit:

(x >> i) & 1 # test: 1 if bit i is on, else 0
x | (1 << i) # set bit i
x & ~(1 << i) # clear bit i
x ^ (1 << i) # toggle bit i

A mask of several bits works the same way. (1 << k) - 1 is k ones, so x & 0xFF keeps the low 8 bits and (x >> 8) & 0xFF reads the next 8. That’s how an IPv4 address packs into one 32-bit number: (a << 24) | (b << 16) | (c << 8) | d.

Three tricks worth knowing by heart

  • x & (x - 1) clears the lowest 1. Subtracting 1 turns the lowest 1 into a 0 and every 0 below it into a 1. The AND keeps only the bits that didn’t change. So x > 0 and x & (x - 1) == 0 means x is a power of two, and repeating it counts the 1s (below).
  • x & -x keeps only the lowest 1. For 12 (1100) it gives 4 (0100): the largest power of two that divides x.
  • XOR cancels pairs. a ^ a == 0, a ^ 0 == a, and the order doesn’t matter. XOR a whole list together and every value that appears twice vanishes, leaving the one that appears once. XOR the list with 0, 1, …, n and the one missing number survives the same way.

Negative numbers: two’s complement

A 32-bit int stores a negative number by wrapping round. -1 is 32 ones, because adding 1 to it carries all the way off the top and leaves 0. In general -x == ~x + 1: flip every bit, then add one. The top bit ends up as the sign, 1 for negative, so an int holds −2³¹ to 2³¹ − 1.

An unsigned type reads the same 32 bits with no sign: 32 ones is 4,294,967,295, not −1. The bits are identical; only the reading differs. That’s also why x & -x works: -x is ~x + 1, and the + 1 carries up through the flipped zeros to land exactly on x’s lowest 1. Above that bit, -x is the opposite of x, so the AND clears everything else.

How it works

We’ll count the 1 bits of x with the lowest-bit trick, often called Kernighan’s method. The graphic follows the steps as you scroll. Try editing x: a power of two takes one round, and a negative number shows its two’s complement bits.

  1. Start with count = 0. The graphic draws x = 180 as 8 bits, bit 7 on the left and bit 0 on the right.
  2. While x isn’t 0, it still has at least one 1 bit.
  3. Work out x - 1. Subtracting 1 borrows from the lowest 1: that bit becomes 0, every 0 below it becomes 1, and every bit above stays the same.
  4. AND the two rows: x &= x - 1. Above the lowest 1 the rows agree, so those bits survive. From there down they disagree, so they all come out 0.
  5. Add one to count and go round again with the smaller x.
  6. When x reaches 0, count is the number of 1 bits: 4 for 180.
loading bit-manipulation…

Why it’s correct: each round removes exactly one 1 and leaves every higher bit alone, and the bits below it were already 0. So after k rounds x has k fewer 1s, and the loop stops exactly when none are left.

def popcount(x):
"""Number of 1 bits in x, read as a 32-bit value (so -1 has 32)."""
# Python ints never run out of bits: a negative number has infinitely
# many 1s on the left. Keep the low 32 bits, as C++ and Java would.
x &= 0xFFFFFFFF
count = 0
while x:
x &= x - 1
count += 1
return count
#include <cstdint>
using namespace std;
// Number of 1 bits in x. Unsigned, so x - 1 and the shifts behave the same
// for every bit pattern, and -1 converts to 32 ones.
int popcount(uint32_t x) {
int count = 0;
while (x != 0) {
x &= x - 1;
count++;
}
return count;
}
class Bits {
// Number of 1 bits in x. Java ints are 32-bit two's complement, so a
// negative x works too: -1 has 32 ones.
static int popcount(int x) {
int count = 0;
while (x != 0) {
x &= x - 1;
count++;
}
return count;
}
}

The Python version masks with 0xFFFFFFFF first. Python integers never overflow, so -1 has infinitely many 1s on the left and the loop would never finish. The mask keeps the low 32 bits, the same pattern C++ and Java store.

Why it’s O(number of 1s)

Each round clears one 1 and never creates one, so the loop runs once per 1 bit: at most 32 times for a 32-bit value, and often far fewer. Checking every bit with (x >> i) & 1 always takes 32 steps.

x bit-by-bit steps rounds of x & (x - 1)
180 (10110100) 32 4
2³¹ 32 1
2³² − 1 (all ones) 32 32

Each bitwise operation on a fixed-width integer is O(1). Space is O(1).

Common mistakes

Forgetting that Python ints never overflow

Python keeps as many bits as a number needs, and a negative number behaves as if it had infinitely many 1s on the left. Loops that wait for x to reach 0 never end, and sums never wrap to negative. Mask to 32 bits while you work, and turn the pattern back into a signed value at the end.

while x: x &= x - 1 # ✗ never ends for x = -1
x &= 0xFFFFFFFF # ✓ keep 32 bits, as C++ and Java do
x - (1 << 32) if x >> 31 else x # ✓ read a 32-bit pattern as signed again

Shifting a 32-bit 1 too far

In C++ and Java the literal 1 is a 32-bit int. 1 << 31 lands on the sign bit and comes out negative (older C++ standards call it undefined), and 1 << 40 is undefined in C++. Java quietly uses only the low 5 bits of the shift amount, so 1 << 40 is 1 << 8, which is 256.

long long bit = 1 << 40; // ✗ shifted as an int, before the widening
long long bit = 1LL << 40; // ✓ shift a 64-bit one (Java: 1L << 40)
uint32_t top = 1u << 31; // ✓ unsigned, so no sign bit to hit

Using >> on a negative number

In C++ and Java, >> on a signed number copies the sign bit into the gap, so -8 >> 1 is -4 and -1 >> 1 stays -1 for ever. A loop that shifts until it reaches 0 never stops. Java’s >>> fills with 0s instead; in C++, use an unsigned type. Python’s >> also keeps the sign, so mask first.

while (x != 0) { x >>= 1; } // ✗ stuck at -1 when x is negative
while (x != 0) { x >>>= 1; } // ✓ zeros come in from the left

Leaving out the brackets

In C++ and Java, == binds tighter than &, so x & 1 == 0 means x & (1 == 0). C++ compiles it and the test is always false; Java refuses to compile it. Python happens to group it the way you meant, which makes the habit easy to miss. Bracket every bitwise expression.

if (x & 1 == 0) // ✗ x & (1 == 0), always false
if ((x & 1) == 0) // ✓ x is even

Variations

  • Adding without +. a ^ b adds each column but drops the carries, and (a & b) << 1 is exactly those carries, moved to the column they go into. Replace a, b with those two and repeat until the carry is 0. In Python, mask both to 32 bits on every round, or a negative input keeps carrying for ever.
  • Enumerating subsets. With n items (up to about 20), for mask in range(1 << n) visits every subset once, and item i is in it when (mask >> i) & 1. To walk the subsets of one mask, repeat sub = (sub - 1) & mask until it reaches 0. This is where bitmask DP starts.
  • Counts for every number. ones[i] = ones[i & (i - 1)] + 1 fills a table of 1-counts from 0 to n in O(n), because i & (i - 1) is smaller than i and already done.
  • Building a number bit by bit. ans = (ans << 1) | (x & 1) then x >>= 1 moves the lowest bit of x onto the end of ans. After 32 rounds the bits are reversed.
  • Library versions. Python x.bit_count() (3.10+) and x.bit_length(), C++20 std::popcount and std::countr_zero from <bit> (unsigned types only), Java Integer.bitCount and Integer.numberOfTrailingZeros. Use them in real code, and know the loop for interviews.

Check yourself

5 quick questions. Pick an answer to see why it's right or wrong.

  1. 1

    A list holds up to 10⁵ integers. Every value appears exactly twice, except one value that appears once. You must find it in O(n) time with O(1) extra space. What’s the approach?

  2. 2

    What does this print?

    x = 12
    print(x & -x, x & (x - 1), x | 1, x ^ 0b1111)
  3. 3

    This counts the 1 bits of a 32-bit value. Called as count_ones(-1) in Python, it never returns. Why?

    def count_ones(x):
    count = 0
    while x:
    x &= x - 1
    count += 1
    return count
  4. 4

    In Java, what is the value of mask after this line?

    long mask = 1 << 40;
  5. 5

    How many times does x &= x - 1 run before x reaches 0, for x = 2**31 and for x = 2**32 - 1?

Practice problems

Solve these right here, in Python, C++ or Java. Tests run as you go.

All 14 problems on this topic

Further reading

esc