Language-Level Concurrency and Multithreading Questions

Direct answer

The Global Interpreter Lock (GIL, the lock that lets only one thread execute Python bytecode at a time) does not make counts[user] = counts.get(user, 0) + 1 safe. That line is several bytecode steps (read the old value, add one, store it; bytecode is the low-level instruction list CPython compiles each function to), and the interpreter can hand the GIL to another thread part-way through them (it checks for a pending switch request only at certain points, such as returns from calls and backward jumps, so not literally between any two instructions), so two threads can read the same old value and one increment is lost. With real numbers, if counts["alice"] is 5:

Two events were recorded and the count rose by one. The fix inside one process is to put the whole read-modify-write under a threading.Lock. The fix does not carry across processes or machines: every process has its own dictionary, so the counts must move to a shared store with an atomic increment (Redis HINCRBY, a SQL upsert) or be sent to a single aggregator (one component that receives every event and does all the counting). Atomic means the store applies the whole increment as one indivisible step. HINCRBY is the Redis command that adds a number to a field of a hash (a key with named fields) in one step; a SQL upsert is an INSERT that updates the existing row instead when the key already exists (ON CONFLICT ... DO UPDATE in PostgreSQL).

What goes wrong in one process

The Python library FAQ lists D[x] = D[x] + 1 among operations that are NOT atomic, while single operations such as D[x] = y are. The GIL protects the interpreter's internal data, not your multi-step logic. A thread switch is requested after a time slice (5 ms by default, adjustable with sys.setswitchinterval) and takes effect at the interpreter's next check point, which can fall inside a multi-step update.

The program below runs eight threads, each recording 200,000 events across three users (expected total 1,600,000), ten rounds each for an unsafe and a locked version. It also starts four processes that each record 1,000 events into their own correctly locked counter. sys.setswitchinterval(1e-6) asks for very frequent switches, which only makes the race more likely to show.

import sys
import threading
from collections import defaultdict
from multiprocessing import Process, Queue

THREADS = 8
EVENTS_PER_THREAD = 200_000
USERS = ["alice", "bob", "carol"]
EXPECTED = THREADS * EVENTS_PER_THREAD


class UnsafeCounts:
    def __init__(self):
        self.counts = defaultdict(int)

    def record(self, user):
        self.counts[user] = self.counts.get(user, 0) + 1  # read, add, write: three steps

    def total(self):
        return sum(self.counts.values())


class LockedCounts:
    def __init__(self):
        self.counts = defaultdict(int)
        self._lock = threading.Lock()

    def record(self, user):
        with self._lock:  # the whole read-add-write is one critical section
            self.counts[user] = self.counts.get(user, 0) + 1  # same three steps as the unsafe class, but now under the lock

    def total(self):
        with self._lock:
            return sum(self.counts.values())


def hammer(counter):
    def work(offset):
        for i in range(EVENTS_PER_THREAD):
            counter.record(USERS[(i + offset) % len(USERS)])

    threads = [threading.Thread(target=work, args=(t,)) for t in range(THREADS)]
    for t in threads:
        t.start()
    for t in threads:
        t.join()
    return counter.total()


def process_worker(q):
    local = LockedCounts()  # a perfectly thread-safe object, but private to this process
    for i in range(1000):
        local.record(USERS[i % len(USERS)])
    q.put(local.total())


if __name__ == "__main__":
    sys.setswitchinterval(1e-6)  # ask for very frequent thread switches to expose the race
    for name, cls in (("unsafe", UnsafeCounts), ("locked", LockedCounts)):
        wrong = 0
        worst = 0
        for _ in range(10):
            got = hammer(cls())
            if got != EXPECTED:
                wrong += 1
                worst = max(worst, EXPECTED - got)
        print(f"{name}: wrong totals in {wrong} of 10 rounds, worst loss {worst} of {EXPECTED}")

    q = Queue()
    procs = [Process(target=process_worker, args=(q,)) for _ in range(4)]
    for p in procs:
        p.start()
    totals = sorted(q.get() for _ in procs)
    for p in procs:
        p.join()
    print("4 processes, each sent 1000 events; per-process totals:", totals, "global view: none")

The target is 8 threads x 200,000 events = 1,600,000 per round. Output from a run in python:3.12-slim (the lost-event figure differs on every run; the unsafe version usually loses events in every round and the locked version never does; the GIL means the loss shows even when the process is limited to one CPU):

unsafe: wrong totals in 10 of 10 rounds, worst loss 538896 of 1600000
locked: wrong totals in 0 of 10 rounds, worst loss 0 of 1600000
4 processes, each sent 1000 events; per-process totals: [1000, 1000, 1000, 1000] global view: none

Reading the numbers: in the first line, worst loss 538896 means the unsafe version's worst round counted only 1,600,000 - 538,896 = 1,061,104 events, so about a third were lost. Two notes about the race. First, whether the loss shows depends on the exact code. Disassembling both forms on CPython 3.12 (dis.dis) shows why they differ: the get form reads through a method CALL (d.get), and the interpreter can act on a pending thread-switch request when that call returns, before the add and the store, while the augmented form d["a"] += 1 goes straight from BINARY_SUBSCR (read) to BINARY_OP (add) to STORE_SUBSCR (write) with no call in between. A short check makes the contrast reproducible: eight threads each run 200,000 increments on a single key of a plain dict, once as d["a"] = d.get("a", 0) + 1 and once as d["a"] += 1, at the default switch interval and at sys.setswitchinterval(1e-6). On CPython 3.12 the augmented form lost nothing at either setting, while the get form lost events at both: in every round at the short interval, and at the default interval in many rounds (some rounds, especially with several CPUs, lose nothing). The check:

import sys
import threading


def lost(form, interval):
    sys.setswitchinterval(interval)
    d = {"a": 0}

    def work():
        for _ in range(200_000):
            if form == "get":
                d["a"] = d.get("a", 0) + 1
            else:
                d["a"] += 1

    threads = [threading.Thread(target=work) for _ in range(8)]
    for t in threads:
        t.start()
    for t in threads:
        t.join()
    return 8 * 200_000 - d["a"]


for form in ("get", "aug"):
    for interval in (0.005, 1e-6):
        print(form, interval, [lost(form, interval) for _ in range(5)])

The difference is consistent with the interpreter handling thread-switch requests at calls and similar points, but that is an implementation detail of one CPython version, not a guarantee. So the unsafe line is the get form above; the augmented form merely happens not to lose on this build, and the FAQ's advice applies to both: when in doubt, use a lock. For the same reason the LockedCounts class uses the get form too: with += inside it, deleting the lock would still print zero losses on this build, and the demo could not tell a working lock from a missing one. Second, in the free-threaded build (the build of CPython that runs without the GIL, so threads really execute Python code in parallel; PEP 779 describes it as officially supported but still optional from Python 3.14, experimental in 3.13) even the quiet forms can race, so the lock is required there too.

Making it safe in one process

1. One lock around the whole read-modify-write (the LockedCounts class). The lock must cover the read as well as the write; locking only the store leaves the same race.
2. If the lock becomes a bottleneck, shard it. Keep, say, 16 dictionaries each with its own lock and pick one with hash(user) % 16. Each user maps to exactly one shard, so totals stay exact.
3. Or batch per thread and merge. Each thread counts into a private dictionary with no lock, and a periodic merge adds those deltas into the shared one under the lock. Sums commute (the order of adding does not change the total), so merging deltas is exact. The cost is that a reader sees counts that lag by up to one merge interval.
4. Or confine the dictionary to one aggregator thread that reads events from a queue.Queue (thread-safe by design). No other thread touches the dictionary, so it needs no lock.

Also bound memory: a dictionary keyed by user grows with the number of distinct users. Use a TTL (expire entries after a set time) or LRU (evict the least recently used entry) eviction, or accept that the in-memory view covers only recent users and keep the source of truth elsewhere.

What changes across processes or machines

With four gunicorn workers (gunicorn is a common Python web server that runs several worker processes) or four pods (copies of the service running as separate containers in Kubernetes), each has its own dictionary (the demo's last line: four counters of 1,000 and no global view). A user's requests land on different workers, so each worker holds a fraction and no one holds the total. A lock cannot help, because threading.Lock only coordinates threads inside one process.

Choose by what the count is for (the lock-protected local buffer with delta flush, in the recommendation below, is the usual starting point):

• Needs to be exact and immediately readable (quota enforcement, billing): increment in a shared store that applies the update atomically, such as a SQL statement INSERT ... ON CONFLICT (user_id) DO UPDATE SET n = counts.n + 1 in PostgreSQL or HINCRBY in Redis. The database or Redis serialises the increments, so no application lock is needed.
• Approximate and read later (analytics): each process keeps its local counts under a lock and flushes deltas every few seconds with one HINCRBY per user, or writes events to a log such as Kafka keyed by user so a single consumer per key partition does the counting (Kafka splits a log into partitions and sends all events with the same key to the same one, so one consumer sees all of a user's events in order). The trade-off is that events held in memory are lost if the process crashes before a flush.
• Retries: an atomic increment is not idempotent (doing it twice has a different effect from doing it once). If a client retries after a timeout, the event may count twice. Attach an event id and record seen ids (a set with a TTL, a time to live after which old entries expire) when exactness matters.

Recommendation: for a typical web service, use a lock-protected local buffer flushed as deltas to Redis or the database, because it keeps the hot path off the network and the shared store receives one write per user per interval. Switch to a direct atomic increment per event if the count gates an action such as a rate limit, since the buffered version lags by up to one flush interval.

Pitfalls

• Believing the GIL makes compound dictionary updates safe.
• Locking the write but not the read that decides it.
• Using multiprocessing and a plain dict, expecting sharing.
• Holding the lock while calling the database or doing I/O, which serialises every request thread behind the slowest call.

Direct answer

Concurrency means several tasks are in progress during overlapping time periods, so a program is structured to deal with many things at once, even if only one of them is executing at any instant (one CPU core switching between them). Parallelism means several tasks are literally executing at the same instant on different CPU cores. Concurrency is about structure and waiting; parallelism is about hardware. A program can be concurrent without being parallel (one thread running an event loop), and parallel work needs more than one core.

The practical consequence: I/O-bound work (mostly waiting on network, disk or a database) benefits from concurrency alone, because while one task waits another can run. CPU-bound work (mostly computing) only speeds up with real parallelism, and then only up to the number of cores.

Python, Go and Java side by side

Terms in the table: an event loop is a loop on one thread that waits for any of many pending operations to be ready and runs the code waiting on it; a coroutine is a function that can pause at an await and be resumed later by the loop; a virtual thread (Java 21) is a very lightweight thread managed by the JVM rather than the OS, so thousands can block cheaply on a few OS threads; a ForkJoinPool is a Java thread pool that splits a big job into pieces for its worker threads. In an interview, one pair per language is enough: a concurrent-but-not-parallel example and a truly-parallel one.

The GIL (global interpreter lock) is a mutex inside CPython that lets only one thread execute Python bytecode (the low-level instructions the interpreter runs) at a time. The Python 3.12 glossary says it exists to make the object model safe against concurrent access, at the expense of much of the parallelism of multi-processor machines, and that it is always released while doing I/O (and by some extension modules during heavy computation such as compression or hashing). So in CPython threads give concurrency for I/O-bound work but not parallel speedup for pure-Python CPU-bound code. A free-threaded build (PEP 703, the Python design document that removes the GIL in a separate build) exists as an optional build variant starting with Python 3.13, and it is not the default distribution, so do not assume it in an interview answer without saying which build you mean.

Runnable examples

Python (3.12). Asyncio overlaps three coroutines on one thread; a thread pool overlaps four blocking calls; a process pool computes the CPU-bound part in other processes:

import asyncio
import os
import threading
import time
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor


def cpu_task(n):
    """CPU-bound: sum of squares below n. Returns (worker pid, result)."""
    return os.getpid(), sum(i * i for i in range(n))


async def io_coroutine(name, log):
    log.append((name, "start", threading.get_ident()))
    await asyncio.sleep(0.1)          # stands in for a network call; yields to the event loop
    log.append((name, "end", threading.get_ident()))


async def run_coroutines():
    log = []
    await asyncio.gather(*(io_coroutine(f"c{i}", log) for i in range(3)))
    starts_before_first_end = [e[1] for e in log].index("end")
    thread_ids = {e[2] for e in log}
    print("asyncio: events before the first 'end':", starts_before_first_end,
          "| distinct threads used:", len(thread_ids))


def blocking_io(state):
    with state["lock"]:
        state["now"] += 1
        state["max"] = max(state["max"], state["now"])
    time.sleep(0.1)                   # blocking call; the GIL is released while sleeping
    with state["lock"]:
        state["now"] -= 1


def run_threads():
    state = {"lock": threading.Lock(), "now": 0, "max": 0}
    with ThreadPoolExecutor(max_workers=4) as pool:
        list(pool.map(blocking_io, [state] * 8))
    print("threads: most blocking calls in flight at once:", state["max"], "(pool size 4)")


def run_processes():
    sizes = [200_000] * 8
    with ProcessPoolExecutor(max_workers=4) as pool:
        results = list(pool.map(cpu_task, sizes))
    expected = sum(i * i for i in range(200_000))
    pids = {pid for pid, _ in results}
    print("processes: all results correct:", all(r == expected for _, r in results),
          "| ran in other processes:", os.getpid() not in pids,
          "| distinct worker processes <= 4:", len(pids) <= 4)


if __name__ == "__main__":
    asyncio.run(run_coroutines())
    run_threads()
    run_processes()

The log holds a start and an end entry per coroutine. If the coroutines ran one after another, the first end would come at index 1 (right after c0 started). Three start entries before any end means all three were started and waiting at the same time, and one distinct thread id means they did it without a second thread: concurrency without parallelism. In blocking_io, a counter (now) is raised on entry and lowered on exit under a lock, and max remembers its highest value, so 4 means four blocking calls were in flight together even though the pool had to cycle through 8 jobs.

asyncio: events before the first 'end': 3 | distinct threads used: 1
threads: most blocking calls in flight at once: 4 (pool size 4)
processes: all results correct: True | ran in other processes: True | distinct worker processes <= 4: True

Go (run with go run -race main.go, Go 1.22 or later: each goroutine below captures the loop variable c, and only from Go 1.22 does every loop iteration get its own c; under an older language version, for example a go.mod saying go 1.21 with the package run as go run . (naming main.go directly does not apply the go.mod language version), the same program shares one c, which is a data race, and typically panics with an index out of range). Four blocking goroutines overlap; the same CPU-bound split is run with GOMAXPROCS set to 1 and then to the CPU count, and the answer is identical, which is the point: the code is concurrent by structure, and whether it is parallel depends on the setting and the hardware:

package main

import (
	"fmt"
	"runtime"
	"sync"
	"sync/atomic"
	"time"
)

func sumSquares(lo, hi int) int {
	s := 0
	for i := lo; i < hi; i++ {
		s += i * i
	}
	return s
}

func main() {
	// Concurrency: 4 goroutines that block (simulated I/O) overlap in time.
	var now, max int32
	var wg sync.WaitGroup
	for i := 0; i < 4; i++ {
		wg.Add(1)
		go func() {
			defer wg.Done()
			cur := atomic.AddInt32(&now, 1)
			for {
				m := atomic.LoadInt32(&max)
				if cur <= m || atomic.CompareAndSwapInt32(&max, m, cur) {
					break
				}
			}
			time.Sleep(50 * time.Millisecond)
			atomic.AddInt32(&now, -1)
		}()
	}
	wg.Wait()
	fmt.Println("blocking goroutines in flight at once:", max)

	// Parallelism: split CPU-bound work into 4 chunks; the same code runs with 1 or many OS threads.
	const n = 4_000_000
	want := sumSquares(0, n)
	for _, procs := range []int{1, runtime.NumCPU()} {
		runtime.GOMAXPROCS(procs)
		parts := make([]int, 4)
		for c := 0; c < 4; c++ {
			wg.Add(1)
			go func() {
				defer wg.Done()
				parts[c] = sumSquares(c*n/4, (c+1)*n/4)
			}()
		}
		wg.Wait()
		got := parts[0] + parts[1] + parts[2] + parts[3]
		fmt.Printf("GOMAXPROCS=%d result correct: %v\n", runtime.GOMAXPROCS(0), got == want)
	}
	fmt.Println("CPUs visible to this container:", runtime.NumCPU())
}

blocking goroutines in flight at once: 4
GOMAXPROCS=1 result correct: true
GOMAXPROCS=14 result correct: true
CPUs visible to this container: 14

(The 14 is the CPU count of the Linux container these programs ran in; it differs on other machines.)

The loop inside each goroutine records the highest number of goroutines in flight. now counts goroutines currently inside the sleep. A plain if cur > max { max = cur } would be a data race (two goroutines read and write max at once, and one could overwrite a larger value with a smaller one). So the loop uses compare-and-swap (CAS): CompareAndSwapInt32(&max, m, cur) sets max to cur only if max still equals the m that was just read, and returns false if another goroutine changed it first, in which case the loop reloads and tries again. When it ends, max holds the largest cur any goroutine saw. The race detector (the -race flag, which instruments the program and reports unsynchronized accesses at runtime) reported nothing.

Java 21. Four tasks that wait for each other can only finish if all four are in flight at once: each task counts a CountDownLatch (a counter that releases all waiters when it reaches zero) down by one and then waits on it, so the latch reaches zero only after all four tasks have started and are waiting together. If the pool ran them one at a time, the first would wait forever and the program would hang. The second part starts 1,000 virtual threads that all block at once, which shows that blocking is cheap; the parallel stream splits the CPU-bound sum across the common fork-join pool:

import java.util.concurrent.*;
import java.util.stream.IntStream;

public class ConcurrencyDemo {
    public static void main(String[] args) throws Exception {
        // Concurrency with few threads: 4 tasks that all wait for each other prove 4 are in flight at once.
        CountDownLatch fourStarted = new CountDownLatch(4);
        try (ExecutorService pool = Executors.newFixedThreadPool(4)) {
            for (int i = 0; i < 4; i++) {
                pool.submit(() -> { fourStarted.countDown(); fourStarted.await(); return null; });
            }
        }
        System.out.println("fixed pool of 4: all 4 tasks were in flight together");

        // Many cheap blocking tasks: 1,000 virtual threads (Java 21) all waiting on one latch.
        int n = 1000;
        CountDownLatch allStarted = new CountDownLatch(n);
        try (ExecutorService vt = Executors.newVirtualThreadPerTaskExecutor()) {
            for (int i = 0; i < n; i++) {
                vt.submit(() -> { allStarted.countDown(); allStarted.await(); return null; });
            }
        }
        System.out.println("virtual threads: all " + n + " tasks were blocked at once");

        // Parallelism: a CPU-bound sum split across the common ForkJoinPool.
        long expected = 0;
        for (int i = 0; i < 4_000_000; i++) expected += (long) i * i;
        long got = IntStream.range(0, 4_000_000).parallel().mapToLong(i -> (long) i * i).sum();
        System.out.println("parallel stream result correct: " + (got == expected)
            + " | common pool parallelism: " + ForkJoinPool.getCommonPoolParallelism()
            + " | availableProcessors: " + Runtime.getRuntime().availableProcessors());
    }
}

fixed pool of 4: all 4 tasks were in flight together
virtual threads: all 1000 tasks were blocked at once
parallel stream result correct: true | common pool parallelism: 13 | availableProcessors: 14

Run repeatedly, each program prints the same lines apart from the machine-specific CPU numbers (the 13 and 14 in the Java line and the 14 in the Go lines). On a container limited to one CPU the Java line reports a common pool parallelism of 1 and one available processor: the stream is still correct, but with a single core nothing can execute at the same instant (the pool's one worker and the calling thread only take turns), which is the point that parallelism depends on the hardware. What these outputs show: tasks overlap in time (concurrency), the work splits correctly, and the process pool really runs in other processes. They do not show a speedup; that depends on core count and load, so measure it with a timer on your own machine rather than quoting a number.

What this means for CPU-bound versus I/O-bound work

Worked example: a service makes 50 calls that each wait 200 ms on the network. Run one after another that is 50 x 200 ms = 10 s of waiting. With 50 concurrent tasks the waits overlap and the total is close to one call's 200 ms plus overhead, on a single core. Now take 8 CPU-heavy jobs of 1 s each on a 4-core machine: with real parallelism the best case is 8 / 4 = 2 s of work time, while CPython threads running pure-Python code would still take roughly 8 s (this is the arithmetic of the example, not a timing from the demos above) because only one thread executes bytecode at a time. Mixed workloads split the two: fetch concurrently, compute in parallel.

Pitfalls

• Calling a blocking function inside an async def freezes the event loop, which turns "concurrent" into "stuck".
• Virtual threads are not faster threads: JEP 444 says they do not run code faster than platform threads and they help throughput of waiting tasks, not CPU-bound work. They also should not be pooled; limit concurrency with a semaphore instead.
• Shared mutable state between parallel workers needs synchronization in Go and Java (Go's race detector flagged none here because each goroutine wrote its own slot), whereas separate Python processes share nothing by default and must send results back.