When Integers Misbehave: Overflow, Sign Bits, and Other Surprises
Integers seem simple: they’re just whole numbers. But when you squeeze them into fixed-width slots of 8, 16, 32, or 64 bits, they start doing strange things. Following up on our earlier look at floating-point problems, here are some of the most instructive integer pitfalls, gathered from real-world reports and language quirks.
The Database Primary Key Ceiling
One classic scenario: you create a table with a 32-bit unsigned integer primary key, confident that four billion rows is plenty. Then your service takes off, and the rows creep toward that limit. When you hit the maximum, new inserts likely fail, and you’re stuck doing a painful migration to a 64-bit key while keeping the service alive.
When Subtraction Goes Negative (or Not)
Integer underflow can produce surprising results, as in this Go example:
package main
import "fmt"
func main() {
var x uint32 = 5
var length uint32 = 0
if x < length-1 {
fmt.Printf("%d is less than %d\n", x, length-1)
}
}
This prints:
5 is less than 4294967295
That may look wrong, but it’s not a bug in the usual sense. The operation 0 - 1 yields the 4-byte pattern 0xFFFFFFFF. Whether that represents -1 or 4294967295 depends entirely on signedness. Because x and length are declared as uint32, the comparison treats length - 1 as the unsigned value 4294967295, so 5 < 4294967295 is true.
Languages handle this differently. Python, Java, and Ruby don’t have unsigned integers at all, so this specific trap doesn’t exist there. In C, you can compile with clang -fsanitize=unsigned-integer-overflow to crash on such an error. Rust only checks for overflow in debug builds; release mode will happily compute 0 - 1 = 4294967295. The reason for this laxness is performance: checking every addition for overflow is costly in tight loops.
How Negative Integers Work: Two’s Complement
The reason 0xFFFFFFFF can be -1 has to do with how modern hardware encodes signed numbers. With 8 bits, you can represent 256 values. By convention, any pattern that would be 128 or greater is reinterpreted as a negative number, computed by subtracting 256 from the unsigned interpretation.
So the mappings look like this:
00000000 -> 0
00000001 -> 1
00000010 -> 2
...
11111111 -> 255
00000000 -> 0
00000001 -> 1
00000010 -> 2
01111111 -> 127
10000000 -> -128 (previously 128)
10000001 -> -127 (previously 129)
10000010 -> -126 (previously 130)
...
11111111 -> -1 (previously 255)
Thus, the 8-bit pattern 11111111 (or 0xFF) means -1. For 32-bit numbers, the rule scales to “patterns above 2^31 are negative, computed by subtracting 2^32.” This scheme is called two’s complement, and while it’s dominant, other representations exist in the wild.
Edge Case: The Absolute Value That Isn’t
With signed 8-bit integers, the range is -128 to 127. There’s no +128. Yet this simple function still compiles:
package main
import (
"fmt"
)
func abs(x int8) int8 {
if x < 0 {
return -x
}
return x
}
func main() {
fmt.Println(abs(-127))
fmt.Println(abs(-128))
}
It outputs:
127
-128
Even though abs(-128) should return 128, the hardware can’t represent it, resulting in an overflow (or, in this case, a wrap-around to a negative value). Some languages crash on this; Go doesn’t.
Shifting Bytes in Java: A Sign-Extension Trap
Suppose you’re parsing bytes in Java and want the first 4 bits of 0x90. The correct answer is 9, but this code prints -7:
public class Main {
public static void main(String[] args) {
byte b = (byte) 0x90;
System.out.println(b >> 4);
}
}
The culprit is two Java specifics: Java has no unsigned integers, and it can’t right-shift a byte directly. The byte 0x90 (binary 10010000) starts with a 1, so it’s negative. When Java promotes it to an int for the shift, it does so by sign-extension, turning 0x90 into 0xFFFFFF90. Then, a standard >> is a signed shift, so it keeps padding with 1 bits, yielding 0xFFFFFFF9 — which is -7 as a signed int.
The fix is to mask before shifting:
b >> 4
becomes:
(b & 0xFF) >> 4
Here, b & 0xFF isn’t redundant; it strips the sign-extension, turning 0xFFFFFF90 into 0x00000090 first. The result is then shifted to 0x00000009, which is the expected 9.
When Strings and IPs Become Integers
It’s easy to misinterpret data that isn’t inherently numeric as if it were an integer. For example, the ASCII string “HTTP” corresponds to the hex pattern 0x48545450, and the integer 2130706433 is the representation of 127.0.0.1. Tools like ping will happily take a bare integer and convert it to an IP address:
$ ping 2130706433
PING 2130706433 (127.0.0.1): 56 data bytes
$ ping 132848123841239999988888888888234234234234234234
PING 132848123841239999988888888888234234234234234234 (251.164.101.122): 56 data bytes
Security Consequences of Overflow
Overflow bugs aren’t just academic. A search for CVEs related to integer overflows turns up many real vulnerabilities. One example is a JSON parsing library bug (CVE-2022-24795). The issue was roughly this: a JSON file is 3 GB in size. Due to an overflow, the parser computes a size of near zero and allocates a tiny buffer. The full 3 GB of data then gets copied into that buffer, overwriting memory it shouldn’t touch. That can crash the program, but sometimes such bugs can be escalated to arbitrary code execution.
The Mystery of Byte Order
Reading raw binary data with an unknown endianness is another well-known trap. If a file contains the bytes 00, 00, 12, and 81, you don’t know if it’s meant to be read big-endian as 0x00001281 (4737) or little-endian as 0x81120000 (2165440512). The correct answer might come from file metadata, from knowing the generating machine’s architecture, or just from which value makes more sense in context.
This isn’t just a problem with integers; floats have byte order too. Network data is a special case: the network order is defined to be big-endian, so if you read it on a little-endian x86 machine, you have to swap the bytes of every number.
Modulo with Negative Numbers Is Not Universal
Different programming languages define % differently when negative numbers are involved. Take -13 % 3. In Python, the result is 2. In JavaScript, it’s -1. Even when the dividend is positive and the divisor is negative (like 13 % -3), languages make different choices. These aren’t bugs, but they are design decisions that can cause portability headaches.
The Optimizer That Eliminates Your Overflow Check
Let’s say you try to be careful and add checks for signed integer overflow in C:
#include <stdio.h>
#define INT_MAX 2147483647
int check_overflow(int n) {
n = n + 100;
if (n + 100 < 0)
return -1;
return 0;
}
int main() {
int result = check_overflow(INT_MAX);
printf("%d\n", result);
}
Your intention is for check_overflow to return -1 because INT_MAX + 100 exceeds the maximum integer size. When you compile without optimizations, that’s what you get:
$ gcc check_overflow.c -o check_overflow && ./check_overflow
-1
$ gcc -O3 check_overflow.c -o check_overflow && ./check_overflow
0
But with gcc -O3, the behavior changes. The compiler assumes that signed integer overflow is undefined behavior in C — and it is. That means the compiler is allowed to do anything once that overflow happens, including assuming it never will. Since the if condition n + 100 < 0 would only be true if undefined behavior occurred, the optimizer is free to remove the check entirely.
Undefined behavior is a C and C++ concept mainly; other languages generally don’t have an equivalent, except when they call into C code that misbehaves.
The && Typo That Changes Everything
A single-character typo can bridge logic and bitwise operations. To check that two integers are both non-zero in JavaScript, you write:
if a && b {
/* some code */
}
A typo might give you this, which is still perfectly valid code:
if a & b {
/* some code */
}
Now it’s a bitwise AND, not a logical one. For example, in a console:
> 9 && 4
4
> 9 & 4
0
> 4 && 5
5
> 4 & 5
4
This can lead to an intermittent bug: often x & y will be truthy when x && y is true, but not always. Linters like ESLint have a no-bitwise rule that flags this, requiring you to explicitly justify every use of a bitwise operator.
