~/techniques/bit-manipulation
Bit manipulation
Read a number as a row of on/off bits. AND, OR, XOR and shifts test, set, clear and count them in a step or two.
Treat an integer as a row of bits and change or query them with AND, OR, XOR and shifts. `x & (x - 1)` drops the lowest 1.
Flags or small sets packed into one integer, pairs that cancel out, powers of two, IP addresses and masks.
O(1) per operation; O(number of 1s) to count them
O(1)
You’ll recognise it when
- A handful of yes/no flags (weekdays, chosen items, visited cities) would fit in the bits of one integer.
- Every value appears twice except one, and you must find the odd one out with O(1) extra space.
- The question is about the binary form itself: count the 1s, reverse the bits, test for a power of two.
- You’re handling IP addresses or masks, where the top
kbits of a 32-bit number mean something. - The rules ban
+, or ask for O(1) space where a set would be the obvious answer.
It sits next to bitmask DP, which uses these same masks as the states of a DP table. This page is the toolkit that DP is built on.
The idea
Think of an integer as a row of light switches. Switch i is worth 2^i, so 13 is 1101: switches 0, 2 and 3 are on. A bitwise operator works on every switch at once, in a single machine instruction.
| Operator | Bit by bit | 12 (1100) with 10 (1010) |
|---|---|---|
a & b (AND) |
1 only where both are 1 | 1000 = 8 |
a | b (OR) |
1 where either is 1 | 1110 = 14 |
a ^ b (XOR) |
1 where they differ | 0110 = 6 |
~a (NOT) |
flips every bit | ~12 = −13 |
a << k |
moves bits up k places, 0s fill in: times 2^k |
12 << 1 = 24 |
a >> k |
moves bits down, the low k fall off: divide by 2^k, rounding down |
12 >> 2 = 3 |
One bit at a time
1 << i is a number with only bit i on. Combine it with x to test, set, clear or toggle that bit:
(x >> i) & 1 # test: 1 if bit i is on, else 0
x | (1 << i) # set bit i
x & ~(1 << i) # clear bit i
x ^ (1 << i) # toggle bit i
A mask of several bits works the same way. (1 << k) - 1 is k ones, so x & 0xFF keeps the low 8 bits and (x >> 8) & 0xFF reads the next 8. That’s how an IPv4 address packs into one 32-bit number: (a << 24) | (b << 16) | (c << 8) | d.
Three tricks worth knowing by heart
x & (x - 1)clears the lowest 1. Subtracting 1 turns the lowest 1 into a 0 and every 0 below it into a 1. The AND keeps only the bits that didn’t change. Sox > 0 and x & (x - 1) == 0meansxis a power of two, and repeating it counts the 1s (below).x & -xkeeps only the lowest 1. For 12 (1100) it gives 4 (0100): the largest power of two that dividesx.- XOR cancels pairs.
a ^ a == 0,a ^ 0 == a, and the order doesn’t matter. XOR a whole list together and every value that appears twice vanishes, leaving the one that appears once. XOR the list with0, 1, …, nand the one missing number survives the same way.
Negative numbers: two’s complement
A 32-bit int stores a negative number by wrapping round. -1 is 32 ones, because adding 1 to it carries all the way off the top and leaves 0. In general -x == ~x + 1: flip every bit, then add one. The top bit ends up as the sign, 1 for negative, so an int holds −2³¹ to 2³¹ − 1.
An unsigned type reads the same 32 bits with no sign: 32 ones is 4,294,967,295, not −1. The bits are identical; only the reading differs. That’s also why x & -x works: -x is ~x + 1, and the + 1 carries up through the flipped zeros to land exactly on x’s lowest 1. Above that bit, -x is the opposite of x, so the AND clears everything else.
How it works
We’ll count the 1 bits of x with the lowest-bit trick, often called Kernighan’s method. The graphic follows the steps as you scroll. Try editing x: a power of two takes one round, and a negative number shows its two’s complement bits.
- Start with
count = 0. The graphic drawsx = 180as 8 bits, bit 7 on the left and bit 0 on the right. - While
xisn’t 0, it still has at least one 1 bit. - Work out
x - 1. Subtracting 1 borrows from the lowest 1: that bit becomes 0, every 0 below it becomes 1, and every bit above stays the same. - AND the two rows:
x &= x - 1. Above the lowest 1 the rows agree, so those bits survive. From there down they disagree, so they all come out 0. - Add one to
countand go round again with the smallerx. - When
xreaches 0,countis the number of 1 bits: 4 for 180.
Why it’s correct: each round removes exactly one 1 and leaves every higher bit alone, and the bits below it were already 0. So after k rounds x has k fewer 1s, and the loop stops exactly when none are left.
def popcount(x):
"""Number of 1 bits in x, read as a 32-bit value (so -1 has 32)."""
# Python ints never run out of bits: a negative number has infinitely
# many 1s on the left. Keep the low 32 bits, as C++ and Java would.
x &= 0xFFFFFFFF
count = 0
while x:
x &= x - 1
count += 1
return count#include <cstdint>
using namespace std;
// Number of 1 bits in x. Unsigned, so x - 1 and the shifts behave the same
// for every bit pattern, and -1 converts to 32 ones.
int popcount(uint32_t x) {
int count = 0;
while (x != 0) {
x &= x - 1;
count++;
}
return count;
}class Bits {
// Number of 1 bits in x. Java ints are 32-bit two's complement, so a
// negative x works too: -1 has 32 ones.
static int popcount(int x) {
int count = 0;
while (x != 0) {
x &= x - 1;
count++;
}
return count;
}
}The Python version masks with 0xFFFFFFFF first. Python integers never overflow, so -1 has infinitely many 1s on the left and the loop would never finish. The mask keeps the low 32 bits, the same pattern C++ and Java store.
Why it’s O(number of 1s)
Each round clears one 1 and never creates one, so the loop runs once per 1 bit: at most 32 times for a 32-bit value, and often far fewer. Checking every bit with (x >> i) & 1 always takes 32 steps.
x |
bit-by-bit steps | rounds of x & (x - 1) |
|---|---|---|
180 (10110100) |
32 | 4 |
| 2³¹ | 32 | 1 |
| 2³² − 1 (all ones) | 32 | 32 |
Each bitwise operation on a fixed-width integer is O(1). Space is O(1).
Common mistakes
Forgetting that Python ints never overflow
Python keeps as many bits as a number needs, and a negative number behaves as if it had infinitely many 1s on the left. Loops that wait for x to reach 0 never end, and sums never wrap to negative. Mask to 32 bits while you work, and turn the pattern back into a signed value at the end.
while x: x &= x - 1 # ✗ never ends for x = -1
x &= 0xFFFFFFFF # ✓ keep 32 bits, as C++ and Java do
x - (1 << 32) if x >> 31 else x # ✓ read a 32-bit pattern as signed again
Shifting a 32-bit 1 too far
In C++ and Java the literal 1 is a 32-bit int. 1 << 31 lands on the sign bit and comes out negative (older C++ standards call it undefined), and 1 << 40 is undefined in C++. Java quietly uses only the low 5 bits of the shift amount, so 1 << 40 is 1 << 8, which is 256.
long long bit = 1 << 40; // ✗ shifted as an int, before the widening
long long bit = 1LL << 40; // ✓ shift a 64-bit one (Java: 1L << 40)
uint32_t top = 1u << 31; // ✓ unsigned, so no sign bit to hit
Using >> on a negative number
In C++ and Java, >> on a signed number copies the sign bit into the gap, so -8 >> 1 is -4 and -1 >> 1 stays -1 for ever. A loop that shifts until it reaches 0 never stops. Java’s >>> fills with 0s instead; in C++, use an unsigned type. Python’s >> also keeps the sign, so mask first.
while (x != 0) { x >>= 1; } // ✗ stuck at -1 when x is negative
while (x != 0) { x >>>= 1; } // ✓ zeros come in from the left
Leaving out the brackets
In C++ and Java, == binds tighter than &, so x & 1 == 0 means x & (1 == 0). C++ compiles it and the test is always false; Java refuses to compile it. Python happens to group it the way you meant, which makes the habit easy to miss. Bracket every bitwise expression.
if (x & 1 == 0) // ✗ x & (1 == 0), always false
if ((x & 1) == 0) // ✓ x is even
Variations
- Adding without
+.a ^ badds each column but drops the carries, and(a & b) << 1is exactly those carries, moved to the column they go into. Replacea, bwith those two and repeat until the carry is 0. In Python, mask both to 32 bits on every round, or a negative input keeps carrying for ever. - Enumerating subsets. With
nitems (up to about 20),for mask in range(1 << n)visits every subset once, and itemiis in it when(mask >> i) & 1. To walk the subsets of one mask, repeatsub = (sub - 1) & maskuntil it reaches 0. This is where bitmask DP starts. - Counts for every number.
ones[i] = ones[i & (i - 1)] + 1fills a table of 1-counts from 0 tonin O(n), becausei & (i - 1)is smaller thaniand already done. - Building a number bit by bit.
ans = (ans << 1) | (x & 1)thenx >>= 1moves the lowest bit ofxonto the end ofans. After 32 rounds the bits are reversed. - Library versions. Python
x.bit_count()(3.10+) andx.bit_length(), C++20std::popcountandstd::countr_zerofrom<bit>(unsigned types only), JavaInteger.bitCountandInteger.numberOfTrailingZeros. Use them in real code, and know the loop for interviews.
Check yourself
5 quick questions. Pick an answer to see why it's right or wrong.
-
1
A list holds up to 10⁵ integers. Every value appears exactly twice, except one value that appears once. You must find it in O(n) time with O(1) extra space. What’s the approach?
a ^ a == 0anda ^ 0 == a, and XOR doesn’t care about order, so every pair cancels and only the single value survives. The hash set is correct but uses O(n) space; sorting takes O(n log n) time; the sum trick needs the distinct values, which again means a set. -
2
What does this print?
x = 12print(x & -x, x & (x - 1), x | 1, x ^ 0b1111)12 is
1100.x & -xkeeps only the lowest 1, which is 4.x & (x - 1)clears that same bit, leaving1000= 8, notx - 1= 11.x | 1sets bit 0 to give 13, and XOR with1111flips the low four bits to0011= 3. That’s only~x(-13) if every bit is flipped. -
3
This counts the 1 bits of a 32-bit value. Called as
count_ones(-1)in Python, it never returns. Why?def count_ones(x):count = 0while x:x &= x - 1count += 1return countIn C++ or Java,
-1is exactly 32 ones and the loop runs 32 times. A Python int grows as needed, and a negative one acts as if it had endless 1s on the left, so clearing one at a time never finishes. Masking keeps the same 32-bit pattern the other languages store.while x > 0doesn’t fix it: it stops at once and returns 0, a wrong answer. -
4
In Java, what is the value of
maskafter this line?long mask = 1 << 40;The literal
1is a 32-bitint, so the shift happens in 32 bits before the result is widened tolong. Java uses only the low 5 bits of anintshift amount, so 40 acts as 8 and the result is 2⁸ = 256. In C++ the same line is undefined behaviour. Write1L << 40in Java or1LL << 40in C++. -
5
How many times does
x &= x - 1run beforexreaches 0, forx = 2**31and forx = 2**32 - 1?Each round clears exactly one 1 bit, so the loop runs once per 1. 2³¹ has a single 1 (bit 31), and 2³² − 1 is 32 ones. Only the worst case matches the 32 steps of checking every bit; sparse numbers finish much sooner. “31” mixes up the position of the bit with how many bits are set.
Practice problems
Solve these right here, in Python, C++ or Java. Tests run as you go.