Copy-on-write (COW) after fork: Linux creates a child that shares the parent’s physical pages read-only. At page granularity, when either process attempts a write to a shared page the CPU traps to the kernel (page fault). The kernel then allocates a new physical page, copies the old page contents into it, updates the faulting process’s page table to point to the new writable page, and clears the COW mapping—only the writing process gets the private copy.
Interactions:
- madvise(MADV_DONTNEED): marking a region MADV_DONTNEED tells the kernel the pages aren’t needed; pages become reclaimable and may be zero-filled on next access. For COWed pages, a MADV_DONTNEED from the parent can drop backing pages so a subsequent child write will get a zeroed page (no copy), reducing memory pressure.
- mlockall / mlock: locked pages cannot be paged out or reclaimed; COW still occurs on write (a copy must be made), but the kernel must allocate a locked physical page. This increases memory pressure because both parent and child private pages may remain resident and pinned.
- OOM killer: COW increases peak memory usage when many children write pages: the kernel accounts private RSS per process; if system memory is exhausted, the OOM killer may select processes (often the largest memory consumers) to kill. Because COW can vastly increase actual RAM used post-fork, services delaying writes may avoid immediate OOM but still trigger it when writes happen.
SRE best practices for frequent-fork large-address-space services:
- Prefer posix_spawn or vfork where semantics allow (avoids full COW of large address space).
- Use fork+exec: minimize time between fork and exec to avoid children inheriting large address spaces that get COWed.
- Touch only necessary memory after fork; avoid background threads in parent that will cause writes.
- Use MADV_DONTNEED or MADV_FREE on large caches before fork to reduce COW burden.
- Avoid mlockall unless necessary; if using, account for doubled pinned memory.
- Monitor anon/rss/COW metrics (smaps, /proc/<pid>/status, /proc/vmstat) and set alerts on sudden RSS growth.
- Set resource limits (rlimit/RSS, cgroups memory) to contain blasts and prefer cgroup OOM policies.
- Test failure modes under memory pressure and tune OOM score/oom_adj for critical services.
- Where possible, use techniques like copy-on-write-aware allocators, or pre-fork small working sets (fork pool) instead of forking full process repeatedly.
