Threads and Shared State

Concurrency is about structure: a program is concurrent if it has multiple independent tasks that can overlap in time. Parallelism is about execution: tasks that literally run simultaneously on different hardware threads.

A single-core CPU can run concurrent code (time-sliced), but it cannot run parallel code. A four-core machine can do both. The distinction matters because:

• A concurrent, non-parallel program (Node.js event loop, Python asyncio) has no true simultaneous execution — but it still has coordination complexity.
• A parallel program does run multiple instruction streams at once, which forces you to reason about cache lines, memory ordering, and torn reads.

Rule of thumb: concurrency is a design property; parallelism is a runtime property. Almost all bugs discussed in this cursus are concurrency bugs — they arise from structure, not hardware count.

A process gets its own virtual address space, file descriptor table, and kernel bookkeeping. Spawning one is expensive (~1 ms, ~several MB). Communication requires IPC (pipes, sockets, shared memory mapped explicitly).

A thread is a lightweight execution context inside a process. All threads share the same heap and globals — this is the whole point, and the whole problem. Thread creation is cheaper (~10–50 µs) but threads must coordinate every shared write.

Go goroutines, Erlang processes, and Python greenlets are different again: user-space threads multiplexed onto OS threads, cheap enough to spawn millions.

Every programmer's first race condition looks like this:

import threading

count = 0

def increment():
    global count
    for _ in range(100_000):
        count += 1  # NOT atomic!

threads = [threading.Thread(target=increment) for _ in range(2)]
for t in threads: t.start()
for t in threads: t.join()
print(count)  # prints 112,847 or 153,201 or ... rarely 200,000

count += 1 compiles to three operations: LOAD count into a register, ADD 1, STORE back. If two threads both load the value 42, both add 1, and both store 43 — the count should be 44 but it's 43. That's a lost update. With 100k iterations per thread, you can lose tens of thousands of updates. The output is non-deterministic and depends on OS scheduling, CPU speed, and the phase of the moon.

A data race occurs when two threads access the same memory location, at least one write is involved, and there is no synchronization ordering the accesses. Data races are undefined behavior in C/C++ — the compiler is allowed to assume they don't exist and optimize accordingly, producing surreal results.

The solution is a critical section: a region of code that at most one thread executes at a time. You demarcate it with a mutual exclusion primitive (a mutex). The critical section must be:

• Minimal — only the shared-state touch, not the whole function.
• Always paired — every lock must unlock, even under exceptions. Use RAII (C++), try/finally (Java/Python), or defer (Go).
• Consistent — every thread that touches the shared variable must take the same lock. One unprotected access breaks the invariant for everyone.

Gotchas: holding a lock during I/O, doing expensive computation inside a lock, or locking at the wrong granularity (per-object vs per-field).

Even after you add locks, reasoning about visibility is subtle. Does thread B see the value thread A wrote? The answer depends on the happens-before relation in the language's memory model.

For now, the safe mental model:

• Everything inside a lock-protected critical section happens-before everything inside a subsequent acquisition of the same lock.
• Without synchronization, there is no guarantee a write by thread A is ever visible to thread B — even if A finished before B started on your particular machine today.

This isn't theoretical paranoia. Modern CPUs have per-core store buffers and caches. A write to a variable by core 0 may sit in core 0's L1 cache for microseconds before being flushed to shared L3 and invalidated in core 1's cache. We'll revisit this in detail in the memory models lesson.

Thread A and Thread B both read count=42, both compute 43, both store — one increment is lost.

Adding cores does not scale linearly. If a fraction $s$ of your program is inherently serial (I/O, initialization, lock-contended sections), the maximum speedup with $N$ processors is:

$S(N) = \frac{1}{s + \frac{1 - s}{N}}$

As $N \to \infty$, $S \to 1/s$. If 20% of your code is serial, you can never go faster than 5×, no matter how many cores you add.

Practical takeaways:

• Profile before parallelizing. A 5% serial bottleneck caps you at 20×.
• Lock contention is serial time. A lock held 30% of the time caps speedup at ~3×.
• Gustafson's law is the optimistic counterpoint: as problem size grows, the parallel fraction often dominates — but only for embarrassingly parallel workloads.

Amdahl's law is why multicore doesn't automatically make sequential code faster — you have to redesign for parallelism.

A function or class is thread-safe if it behaves correctly when called concurrently from multiple threads. Thread safety is not a binary property — it depends on what is shared and how:

• Stateless functions are trivially thread-safe (pure functions, stack-local data only).
• Immutable data is thread-safe — reads require no synchronization.
• Thread-local storage (TLS) gives each thread its own copy of a variable — thread-safe by isolation, not coordination.
• Synchronized data uses locks, atomics, or channels to serialize access.

Common misconceptions to stamp out:

• "It worked in testing, so it's fine." Race conditions are timing-dependent; a lightly loaded test machine may never trigger the interleaving a production server hits instantly.
• "Python's GIL makes everything thread-safe." The GIL prevents true CPU parallelism but does not prevent logical races across multiple bytecode operations.
• "Using a thread-safe queue means my whole program is thread-safe." Individual safe components do not compose automatically into a safe system.

Races are hard to reproduce, so reach for static and dynamic tools early:

• ThreadSanitizer (TSan): compiles into your binary (clang/gcc -fsanitize=thread); instruments every memory access at runtime and reports races with the full stack trace of both conflicting accesses. Near-zero false positives. ~5–15× slowdown. Use this in CI.
• Helgrind / DRD: Valgrind-based race detectors for C/C++. Slower but no recompile needed.
• Go race detector: built into go test -race. Same TSan engine, first-class support.
• Java: no built-in TSan equivalent; use stress tests + code review. FindBugs/SpotBugs flag some patterns.
• Code review heuristics: every mutable global accessed from more than one goroutine/thread is suspect. Every if (x != null) { use(x); } without a lock is likely a TOCTOU race.

Discipline: establish an ownership model — every mutable piece of state has exactly one owner at a time, transferred via channels or handed off under a lock. Races become structurally impossible.