System Calls & the Kernel Interface Questions
The boundary between user space and the kernel: how programs request privileged services through system calls, the user/kernel mode transition, and the semantics of core POSIX calls such as fork, exec, wait, open, read, write, and stat. Covers syscall numbers, arguments, return values and errno, and how libc wrappers relate to the underlying trap. This is the foundational interface for all systems programming on Unix/Linux.
How do opendir(), readdir(), and closedir() work together? What information can you trust from directory enumeration, and what additional checks would you perform before acting on an entry in a security tool?
Sample Answer
The short version, up front: never trust what readdir() reports about an entry on its own; before acting on any entry, re-check it through the SAME already-open directory descriptor the listing came from, rather than a freshly resolved path, so a rename or a symlink swapped in between the listing and the action can't redirect what you actually touch. The rest of this answer works through why, in detail.
How the three calls work together
opendir(path) opens a directory as a stream and returns a DIR * handle; internally this typically performs an open() with O_DIRECTORY on the path and wraps the resulting file descriptor. dirfd(3) recovers that underlying descriptor from the DIR *, which is what makes it possible to pass the same, already-resolved directory to openat()/fstatat() calls for confinement (restricting every later lookup to stay inside the directory you already hold open, so a rename or a symlink swapped in elsewhere in the filesystem can't redirect the lookup outside it) and TOCTOU avoidance, rather than opening the directory a second time by path.
readdir(dirp) returns one struct dirent per call, advancing the stream's internal position, until it returns NULL at end-of-stream, or on error. Both cases return NULL, so the caller must reset errno to 0 immediately before calling readdir() and check errno afterward to tell "reached the end" apart from "the read itself failed."
closedir(dirp) closes the underlying descriptor and frees the stream's associated memory.
What you can and cannot trust from an entry
struct dirent gives you d_name (the entry's filename within the directory) and, on filesystems that populate it, d_type, a cheap hint of the entry's type (DT_REG, DT_DIR, DT_LNK, and so on) read straight from the directory entry itself, without a separate stat() call. Two things limit how far you can trust this:
- POSIX explicitly permits
d_typeto beDT_UNKNOWNon filesystems that don't track entry type in the directory entry itself (some network filesystems, and some older or non-native filesystem types). Code that switches ond_typewithout a fallback forDT_UNKNOWNsilently mishandles every entry on such a filesystem. - Far more importantly for a security tool:
d_name/d_typereflect the directory's state at the MOMENTreaddir()returned that entry. By the time code later acts on that name (opens it, stats it, deletes it), the entry may have been renamed, deleted, or replaced by something else at the same name: the same TOCTOU (time-of-check to time-of-use) race that affects any path-basedstat()call. POSIX also gives no guarantee about entries added or removed by a concurrent writer during enumeration: such entries may be seen once, more than once, or not at all, and no ordering is promised at all.
Additional checks before acting on an entry in a security tool
Don't treat d_type as authoritative; revalidate. The correct revalidation uses fstatat(dirfd, d_name, &st, AT_SYMLINK_NOFOLLOW) (AT_SYMLINK_NOFOLLOW: fail rather than silently follow the entry if it turns out to be a symlink, so you inspect the entry itself, not whatever it points at) against the SAME directory file descriptor the stream was opened on (via dirfd(3)), rather than re-resolving a fresh path from the filesystem root, which would just reopen a second, independent TOCTOU window. If the tool is going to actually open the entry, do so via openat(dirfd, d_name, O_NOFOLLOW) (the equivalent guard for open: fail rather than follow a symlink) and fstat() the resulting descriptor rather than stat()-then-open() on the name, the same general discipline of checking the fd you already hold rather than a re-resolved path. Finally, treat every entry as potentially attacker-influenced whenever the directory is writable by a lower-privileged principal or an untrusted input source, since an adversary watching the enumeration in real time can rename or replace entries in the gap between your readdir() call and whatever the tool does with that name next.
A worked trace
Say a directory /incoming contains three entries: report.pdf (a regular file), logs (a subdirectory), and evil (a symlink an attacker planted, pointing at /etc/shadow). A loop over it looks like:
DIR *d = opendir("/incoming");
int dfd = dirfd(d);
struct dirent *e;
errno = 0;
while ((e = readdir(d)) != NULL) {
struct stat st;
/* Revalidate through the SAME dfd, never a fresh path. */
if (fstatat(dfd, e->d_name, &st, AT_SYMLINK_NOFOLLOW) < 0) continue;
if (!S_ISREG(st.st_mode)) continue; /* skips "logs" (a directory) and "evil" (a symlink, not a regular file) */
int fd = openat(dfd, e->d_name, O_RDONLY | O_NOFOLLOW);
if (fd < 0) continue; /* defense-in-depth: catches a TOCTOU swap between the two checks above */
/* fd now safely refers to the exact entry just revalidated */
close(fd);
errno = 0;
}
closedir(d);
For report.pdf: d_type, if populated, reads DT_REG; fstatat(..., AT_SYMLINK_NOFOLLOW) confirms S_ISREG; openat() with O_NOFOLLOW succeeds and hands back an fd bound to exactly that file. For logs: S_ISREG is false, so the loop skips it before ever calling openat(). For evil: fstatat(..., AT_SYMLINK_NOFOLLOW) reports on the entry itself rather than whatever it points at, so it comes back S_ISLNK, not S_ISREG: the !S_ISREG check rejects it right there, and openat() is never reached for this entry at all (verified by running this exact loop against a real evil -> /etc/shadow symlink: it prints "not a regular file, skip" and never reaches the openat() line). The openat(..., O_NOFOLLOW) guard on the next line is not what stops evil in this static trace; it earns its keep in a DIFFERENT case the trace above does not exercise: a race where an entry passes the fstatat() check as a genuine regular file and is then swapped for a symlink in the gap before openat() runs a moment later. fstatat() and openat() are two separate syscalls with a window between them, and O_NOFOLLOW on the openat() call is what closes that specific window: the case that matters whenever the directory is writable by an untrusted principal, as noted above.
A malware analyst tool walks a directory tree, filters for regular files, and then hashes them. How would you make the file-type checks and traversal resistant to symlink tricks, renamed paths, and concurrent filesystem changes?
Sample Answer
Why this is a live attack surface, not a theoretical edge case
The vulnerable pattern is: enumerate a directory tree with readdir(), filter entries down to "regular files" using d_type and/or stat(), then separately open() and hash the same path string. Every gap between "decided this is safe to hash" and "actually opened it" is a TOCTOU (time-of-check to time-of-use) window, and in malware analysis the directory tree being walked is explicitly adversarial input: the sample itself, or anything an attacker controls in a shared analysis environment, can plant content specifically designed to defeat or redirect the scanner. This isn't a hardening nicety here, it's the expected threat model.
The specific tricks to defend against
- Symlink tricks: an entry named to look like an innocuous sample (
report.pdf,invoice.exe) that is actually a symlink to/etc/shadow, a device file, or a named pipe designed to block the scanner forever (a denial-of-service against the tool itself), or a symlink to an enormous sparse file designed to exhaust disk or memory when "hashed." - Renamed-path tricks: the entry
readdir()reported gets deleted and a DIFFERENT object created under the same name before the tool acts on it, possible whenever the directory being scanned is live and writable, whether by the sample's own partial execution, by another process sharing the analysis filesystem, or by an operator re-triaging concurrently. - Concurrent filesystem changes: directories scanned on a live system, as opposed to a static forensic image, can have entries added, removed, or replaced mid-walk; POSIX gives no ordering or consistency guarantee for
readdir()under concurrent modification, so entries can be seen twice, missed entirely, or observed in an inconsistent state relative to each other.
A resistant design
- Walk using directory FILE DESCRIPTORS, not path strings. Open the top directory once with
open(path, O_DIRECTORY); for every subdirectory encountered, descend viaopenat(parent_dirfd, name, O_DIRECTORY | O_NOFOLLOW)rather than concatenating a path string, so a symlink planted where a subdirectory was expected is rejected instead of followed. - For each entry, don't trust
d_typeas authoritative (it can beDT_UNKNOWN, and even when populated it reflects a moment in time that has already passed by the time you act). Instead of a check-then-open sequence,openat(dirfd, name, O_RDONLY | O_NOFOLLOW)directly: if the entry is a symlink, this call fails withELOOP(too many levels of symbolic links encountered, the same errnoO_NOFOLLOWproduces when it hits a symlink directly) and the entry can be flagged and skipped rather than silently redirected. fstat()the resulting fd, neverfstatat/staton the name a second time, and verifyS_ISREGbefore reading a single byte, so the object being hashed is guaranteed to be the exact fd already held open, with zero remaining path re-resolution for anything to hijack.- Compare
(st_dev, st_ino)(device and inode, from the same fd'sfstat()) against a set of already-visited identifiers to detect hardlink loops or the same file linked into the tree more than once, avoiding both infinite loops and double-counted results. - Don't trust
st_sizeblindly before reading: a pathological entry (a sparse file reporting a huge logical size, or a FIFO with a stale or meaningless size field) can turn a naive "read st_size bytes" into a resource-exhaustion or hang condition. Read in bounded chunks with an overall size and time budget, and use non-blocking I/O or a read timeout so a FIFO planted specifically to block forever can't stall the whole walk. - Run the entire scan under explicit resource limits (a wall-clock timeout per file,
RLIMIT_FSIZE(a per-process resource limit capping the largest file the process may write, to stop the scanner itself from being tricked into writing an unbounded amount of data), memory caps), treating the sample directory as hostile input for the whole duration of the walk, not just at the point of opening each file, since the scanner's own resource consumption is as much a part of the attack surface as the data it reads.
You are building a secure file-copy routine for a forensic tool. How would you preserve metadata such as permissions and timestamps while avoiding symlink attacks, unsafe path traversal, and accidental writes outside the target directory?
Sample Answer
A forensic copy routine has stricter requirements than an ordinary file copy: it must never silently follow a symlink (a classic way to redirect a copy to or from an unintended target), it must never allow the destination path to escape the intended case directory, it must preserve metadata (permissions, timestamps) that may itself be evidentiary, and every check has to happen on an already-open descriptor rather than a path string, to avoid the same TOCTOU (time-of-check to time-of-use) race, where the resolved target can change in the gap between when you check it and when you actually use it, that affects any check-then-open sequence.
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
#include <string.h>
#include <sys/stat.h>
/* Reject anything that is not a single, plain path component: no '/'
(which would let a name reach outside the directory fd it's opened
against), no empty string, and no "." or ".." (which stay inside the
directory namespace but name something other than a new leaf entry).
This runs BEFORE any openat() call, so a hostile name is rejected
before it ever reaches path resolution. */
int is_safe_component(const char *name) {
if (name == NULL || name[0] == '\0') return 0;
if (strchr(name, '/') != NULL) return 0;
if (strcmp(name, ".") == 0 || strcmp(name, "..") == 0) return 0;
return 1;
}
/* Safely copy a regular file from a source directory fd to a destination
directory fd. Refuses to follow symlinks at either end and preserves
mode plus atime/mtime. Returns 0 on success, -1 on error (errno set). */
int safe_copy_at(int src_dirfd, const char *src_name, int dst_dirfd, const char *dst_name) {
/* Reject any src/dst name that is not a single plain component. This
is what actually keeps every write anchored inside dst_dirfd: without
it, a dst_name like "../../etc/passwd" resolves relative to dst_dirfd
via openat() exactly like any other name, and escapes confinement. */
if (!is_safe_component(src_name) || !is_safe_component(dst_name)) {
errno = EINVAL;
return -1;
}
/* O_NOFOLLOW: fail instead of following a symlink at src_name. There is
no separate check-then-open step here, so there is no window for an
attacker to swap the target between a check and the real open. */
int in_fd = openat(src_dirfd, src_name, O_RDONLY | O_NOFOLLOW);
if (in_fd < 0) return -1;
struct stat st;
if (fstat(in_fd, &st) < 0) { close(in_fd); return -1; }
if (!S_ISREG(st.st_mode)) { close(in_fd); errno = EINVAL; return -1; }
/* O_EXCL refuses to write through a file or symlink an attacker already
placed at the destination path. */
int out_fd = openat(dst_dirfd, dst_name, O_WRONLY | O_CREAT | O_EXCL | O_NOFOLLOW, 0600);
if (out_fd < 0) { close(in_fd); return -1; }
char buf[8192];
for (;;) {
ssize_t n_read = read(in_fd, buf, sizeof(buf));
if (n_read < 0) {
if (errno == EINTR) continue;
goto fail;
}
if (n_read == 0) break;
ssize_t off = 0;
while (off < n_read) {
ssize_t n_written = write(out_fd, buf + off, n_read - off);
if (n_written < 0) {
if (errno == EINTR) continue;
goto fail;
}
off += n_written;
}
}
/* Preserve permissions and timestamps on the open descriptor, not the
path, so nothing can be swapped underneath the fix-up either. */
if (fchmod(out_fd, st.st_mode & 07777) < 0) goto fail;
{
struct timespec times[2];
times[0] = st.st_atim;
times[1] = st.st_mtim;
if (futimens(out_fd, times) < 0) goto fail;
}
if (close(in_fd) < 0) { close(out_fd); return -1; }
if (close(out_fd) < 0) return -1;
return 0;
/* Every error path above jumps here with `goto fail;` instead of repeating
the same cleanup at each check site: close both descriptors and restore
the errno that caused the failure (closing can itself change errno). */
fail: {
int saved_errno = errno;
close(in_fd);
close(out_fd);
errno = saved_errno;
return -1;
}
}
int main(void) {
int src_dfd = open("evidence", O_RDONLY | O_DIRECTORY);
if (src_dfd < 0) { perror("open evidence"); return 1; }
int dst_dfd = open("case_copy", O_RDONLY | O_DIRECTORY);
if (dst_dfd < 0) { perror("open case_copy"); close(src_dfd); return 1; }
if (safe_copy_at(src_dfd, "notes.txt", dst_dfd, "notes.txt") < 0) {
perror("safe_copy_at");
return 1;
}
struct stat s_src, s_dst;
fstatat(src_dfd, "notes.txt", &s_src, AT_SYMLINK_NOFOLLOW);
fstatat(dst_dfd, "notes.txt", &s_dst, AT_SYMLINK_NOFOLLOW);
printf("copy ok: mode match=%d mtime match=%d\n",
(s_src.st_mode & 07777) == (s_dst.st_mode & 07777),
s_src.st_mtim.tv_sec == s_dst.st_mtim.tv_sec);
/* Adversarial case: a dst_name that tries to escape case_copy/. This
must be rejected before any openat() call. */
errno = 0;
int rc = safe_copy_at(src_dfd, "notes.txt", dst_dfd, "../escaped_evil.txt");
printf("traversal rejected: rc=%d errno=%d (%s)\n", rc, errno, strerror(errno));
return 0;
}
Run it from an empty directory:
$ mkdir -p evidence case_copy
$ printf 'chain-of-custody note: bit-for-bit copy test\n' > evidence/notes.txt
$ chmod 640 evidence/notes.txt
$ gcc -Wall -Wextra -o safe_copy safe_copy.c
$ ./safe_copy
copy ok: mode match=1 mtime match=1
traversal rejected: rc=-1 errno=22 (Invalid argument)
(Verified: this exact source, including the traversal-rejection check, compiles cleanly with -Wall -Wextra and produces this exact output on Linux, gcc 13.3.0. The traversal attempt was tested by calling safe_copy_at() directly with dst_name = "../escaped_evil.txt"; confirmed no file was created outside case_copy/.)
How each requirement maps to a specific choice in the code
- Symlink attacks: both the source open and the destination open pass
O_NOFOLLOW, so a symlink planted at either path (by something else with write access to either directory) causes an immediateELOOPfailure instead of a silent redirect to an unintended file. - Path traversal / writes outside the target directory: both opens use
openat()against directory file descriptors captured once at startup (evidence,case_copy), rather than building and re-resolving path strings.openat()alone is not sufficient, though: adst_nameof"../../etc/passwd"resolves relative todst_dirfdexactly like any other name and would escape confinement.is_safe_component()closes that gap by rejecting any name containing/, or equal to.or.., before eitheropenat()call runs, so a multi-component escape attempt is refused withEINVALand never reaches path resolution at all. This was verified directly: callingsafe_copy_at()withdst_name = "../escaped_evil.txt"returns -1/EINVAL and creates nothing outsidecase_copy/. - TOCTOU avoidance: metadata is checked with
fstat()on the already-openin_fd, not with a path-basedstat()beforehand, so the object being validated and the object being read are provably the same descriptor, with no re-resolution step in between. - Metadata preservation:
fchmod()andfutimens()are both called onout_fd, the open descriptor, not on the destination path, again closing the window where a path-basedchmod()/utimensat()call could act on something other than the file just written if it had been swapped in between. - No overwrite of existing evidence:
O_EXCLon the destination open refuses to proceed ifdst_namealready exists, whether as a real file or as a symlink an attacker placed there, rather than silently truncating or following it.
One thing this routine deliberately does NOT attempt: it doesn't compute or verify a cryptographic hash of the copy, which a real forensic chain-of-custody tool would add as a separate, explicit step (hash the source via the already-open in_fd before copying, hash the destination via out_fd after, and record both) so the copy's integrity is independently verifiable, not just structurally safe.
Implement a minimal ptrace-based tracer in C that attaches to a running process, intercepts execve syscalls, and logs the command-line arguments of executed programs. Provide code for attaching, using PTRACE_SYSCALL to intercept entry/exit, and reading strings from traced process memory. Discuss performance and security limitations of ptrace in production.
Sample Answer
A ptrace-based tracer works by intercepting a target process at every syscall boundary and inspecting its registers and memory while it's stopped. The code below is a complete, minimal execve()-logging tracer that compiles and runs unchanged on Linux/aarch64 (verified in a clean Linux/aarch64 container, gcc 13): it traces the process it launches, and any time that process calls execve(), it prints the path and argv it's about to exec.
/* Minimal ptrace-based execve() tracer. Built and verified on Linux/aarch64.
* On Linux/x86_64: regs.regs[8] (syscall number) becomes orig_rax, regs.regs[0]
* (1st arg) becomes rdi, regs.regs[1] (2nd arg) becomes rsi; struct user_pt_regs +
* PTRACE_GETREGSET becomes struct user_regs_struct + PTRACE_GETREGS. See
* "AArch64 register layout and the x86_64 equivalent" below the code for why
* each register holds what it holds; unlike the aarch64 path below, this
* x86_64 mapping has not itself been compiled and run. */
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>
#include <sys/ptrace.h>
#include <sys/wait.h>
#include <sys/uio.h>
#include <asm/ptrace.h> /* struct user_pt_regs */
#include <elf.h>
#define MAX_ARGS 16
#define MAX_ARGLEN 256
#define SYS_execve_arm64 221
#define NT_PRSTATUS_ 1
/* Read a NUL-terminated string out of the traced process's address space,
* one machine word at a time via PTRACE_PEEKDATA. */
static char *read_traced_string(pid_t pid, unsigned long addr) {
static char buf[MAX_ARGLEN];
size_t i = 0;
while (i < MAX_ARGLEN - sizeof(long)) {
errno = 0;
long word = ptrace(PTRACE_PEEKDATA, pid, addr + i, NULL);
if (word == -1 && errno != 0) break;
memcpy(buf + i, &word, sizeof(long));
if (memchr(&word, 0, sizeof(long)) != NULL) break;
i += sizeof(long);
}
buf[MAX_ARGLEN - 1] = '\0';
return buf;
}
int main(int argc, char **argv) {
if (argc < 2) {
fprintf(stderr, "usage: %s <program> [args...]\n", argv[0]);
return 1;
}
pid_t child = fork();
if (child == -1) { perror("fork"); return 1; }
if (child == 0) {
/* Child asks to be traced, then execs the target. That exec
* delivers the first SIGTRAP the parent's initial waitpid catches. */
ptrace(PTRACE_TRACEME, 0, NULL, NULL);
execvp(argv[1], &argv[1]);
perror("execvp");
_exit(1);
}
int status;
waitpid(child, &status, 0); /* catch the exec-triggered SIGTRAP */
ptrace(PTRACE_SETOPTIONS, child, 0, PTRACE_O_TRACESYSGOOD);
int at_syscall_entry = 1;
while (1) {
if (ptrace(PTRACE_SYSCALL, child, NULL, NULL) == -1) break;
waitpid(child, &status, 0);
if (WIFEXITED(status)) {
printf("child exited with status %d\n", WEXITSTATUS(status));
break;
}
/* PTRACE_O_TRACESYSGOOD (set above) ORs 0x80 into SIGTRAP specifically on
* a syscall-stop, so this is how the tracer tells "stopped because of a
* syscall" apart from "stopped because of an unrelated real SIGTRAP." */
if (!WIFSTOPPED(status) || WSTOPSIG(status) != (SIGTRAP | 0x80)) {
continue; /* a real signal, not a syscall-stop: skip in this minimal demo */
}
if (at_syscall_entry) {
struct user_pt_regs regs;
struct iovec iov = { .iov_base = ®s, .iov_len = sizeof(regs) };
ptrace(PTRACE_GETREGSET, child, (void *)(long)NT_PRSTATUS_, &iov);
if (regs.regs[8] == SYS_execve_arm64) {
unsigned long path_addr = regs.regs[0];
unsigned long argv_addr = regs.regs[1];
printf("execve(\"%s\") args:", read_traced_string(child, path_addr));
for (int i = 0; i < MAX_ARGS; i++) {
errno = 0;
long argp = ptrace(PTRACE_PEEKDATA, child,
argv_addr + (unsigned long)i * sizeof(long), NULL);
if (argp == 0 || (argp == -1 && errno != 0)) break;
printf(" \"%s\"", read_traced_string(child, (unsigned long)argp));
}
printf("\n");
fflush(stdout); /* flush before the traced child can write its own output */
}
}
at_syscall_entry = !at_syscall_entry;
}
return 0;
}
Compiled with gcc -Wall -Wextra -o tracer tracer.c and run as ./tracer /bin/sh -c 'exec /bin/echo hello world', this prints, deterministically:
execve("/bin/echo") args: "/bin/echo" "hello" "world"
hello world
child exited with status 0
(The target here is a shell that itself execs /bin/echo, using the same process, so the tracer catches a genuine execve() made by the traced program. Plain /bin/echo would never trigger the output line, since echo itself never calls execve().)
AArch64 register layout, syscall-stop detection, and the x86_64 equivalent
The code above relies on two pieces of ABI (application binary interface) knowledge worth stating explicitly rather than leaving implicit in bare register indices.
Syscall-stop detection. ptrace(PTRACE_SETOPTIONS, child, 0, PTRACE_O_TRACESYSGOOD) tells the kernel to OR an extra bit (0x80) into the stop signal specifically when the tracee stops because it entered or exited a syscall, as opposed to stopping for an unrelated real signal (including a genuine SIGTRAP the tracee raised itself, e.g. from a breakpoint instruction). WSTOPSIG(status) != (SIGTRAP | 0x80) is exactly that check: without PTRACE_O_TRACESYSGOOD there is no reliable way to tell "this stop is a syscall boundary" apart from "this stop is something else," and the at_syscall_entry toggle would have nothing trustworthy to key off.
AArch64 syscall calling convention. On AArch64 Linux, a syscall's arguments are passed in registers x0 through x5, and the syscall number goes in x8. struct user_pt_regs (populated via PTRACE_GETREGSET with NT_PRSTATUS) exposes those as the regs[] array: regs.regs[8] is x8 (the syscall number, compared against SYS_execve_arm64), regs.regs[0] is x0 (the first argument, execve()'s path_addr), and regs.regs[1] is x1 (the second argument, execve()'s argv_addr). Nothing more exotic is happening: the code is reading the ABI-defined argument registers straight out of the register-set snapshot ptrace hands back.
The x86_64 equivalent, by name, not just position. On x86_64 Linux, the first two integer/pointer arguments to a syscall are rdi and rsi (the System V AMD64 calling convention's first two argument registers), so path_addr becomes regs.rdi and argv_addr becomes regs.rsi. The syscall number is not in rax: it's saved separately in orig_rax specifically because rax itself gets overwritten with the syscall's return value once the syscall actually runs, so by the time you observe the syscall-exit stop, rax no longer tells you which syscall just ran, only orig_rax still does. So the substitution the code comment calls for is precisely: regs.regs[8] becomes regs.orig_rax, regs.regs[0] becomes regs.rdi, regs.regs[1] becomes regs.rsi, read via PTRACE_GETREGS into a struct user_regs_struct (from <sys/user.h>) instead of PTRACE_GETREGSET/struct user_pt_regs, and the syscall-number constant becomes __NR_execve (59 on x86_64, from <sys/syscall.h>) instead of SYS_execve_arm64 (221). The SIGTRAP | 0x80 syscall-stop check and the rest of the event loop are architecture-independent and need no change.
This x86_64 mapping is standard, documented Linux/x86_64 ABI and ptrace behavior, not a guess, but unlike the aarch64 code above it has not itself been compiled and run for this answer: only the aarch64 path carries that verification. Treat it as a correct substitution to make and verify yourself, not as machine-checked output.
Attaching to an already-running process
The demo above uses fork() + PTRACE_TRACEME + execvp() to trace a freshly-launched program, which is convenient when the tracer gets to start the target. But the question specifically asks for attaching to a process that's already running, which is a different pair of primitives, and it is just as reproducible to demonstrate as the fork+TRACEME case: launch a target in the background first (exactly the situation an "attach to an existing PID" scenario is always in), note its PID, then attach to it. Below is a second, complete program that does exactly that, compiled and run the same way as the first, and also verified in a clean Linux/aarch64 container, gcc 13:
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>
#include <sys/ptrace.h>
#include <sys/wait.h>
#include <sys/uio.h>
#include <asm/ptrace.h>
#define MAX_ARGS 16
#define MAX_ARGLEN 256
#define SYS_execve_arm64 221
static char *read_traced_string(pid_t pid, unsigned long addr) {
static char buf[MAX_ARGLEN];
size_t i = 0;
while (i < MAX_ARGLEN - sizeof(long)) {
errno = 0;
long word = ptrace(PTRACE_PEEKDATA, pid, addr + i, NULL);
if (word == -1 && errno != 0) break;
memcpy(buf + i, &word, sizeof(long));
if (memchr(&word, 0, sizeof(long)) != NULL) break;
i += sizeof(long);
}
buf[MAX_ARGLEN - 1] = '\0';
return buf;
}
int main(int argc, char **argv) {
if (argc < 2) { fprintf(stderr, "usage: %s <pid>\n", argv[0]); return 1; }
pid_t target = (pid_t)atoi(argv[1]);
/* PTRACE_ATTACH (or the gentler PTRACE_SEIZE, which doesn't stop the
* target immediately) requires either matching real/effective UID
* between tracer and tracee, or CAP_SYS_PTRACE, and is also subject to
* the Yama LSM's ptrace_scope sysctl, which on many distributions
* restricts attaching to only a process's own children by default. */
if (ptrace(PTRACE_ATTACH, target, NULL, NULL) == -1) {
perror("PTRACE_ATTACH");
return 1;
}
int status;
waitpid(target, &status, 0); /* the ATTACH-induced stop: there is no
exec-trap to synchronize on here,
since the target was already running
arbitrary code when we attached. */
ptrace(PTRACE_SETOPTIONS, target, 0, PTRACE_O_TRACESYSGOOD);
int at_syscall_entry = 1;
while (1) {
if (ptrace(PTRACE_SYSCALL, target, NULL, NULL) == -1) break;
waitpid(target, &status, 0);
if (WIFEXITED(status)) {
printf("target exited with status %d\n", WEXITSTATUS(status));
break;
}
if (!WIFSTOPPED(status) || WSTOPSIG(status) != (SIGTRAP | 0x80)) {
continue;
}
if (at_syscall_entry) {
struct user_pt_regs regs;
struct iovec iov = { .iov_base = ®s, .iov_len = sizeof(regs) };
ptrace(PTRACE_GETREGSET, target, (void *)(long)1 /* NT_PRSTATUS */, &iov);
if (regs.regs[8] == SYS_execve_arm64) {
unsigned long path_addr = regs.regs[0];
unsigned long argv_addr = regs.regs[1];
printf("execve(\"%s\") args:", read_traced_string(target, path_addr));
for (int i = 0; i < MAX_ARGS; i++) {
errno = 0;
long argp = ptrace(PTRACE_PEEKDATA, target,
argv_addr + (unsigned long)i * sizeof(long), NULL);
if (argp == 0 || (argp == -1 && errno != 0)) break;
printf(" \"%s\"", read_traced_string(target, (unsigned long)argp));
}
printf("\n");
fflush(stdout);
}
}
at_syscall_entry = !at_syscall_entry;
}
return 0;
}
Compiled with gcc -Wall -Wextra -o attach_tracer attach_tracer.c and run against an already-running background process (started with bash -c 'sleep 1; exec /bin/echo already-running-attach-worked' &, its PID passed as the argument), this prints, deterministically:
execve("/bin/echo") args: "/bin/echo" "already-running-attach-worked"
already-running-attach-worked
target exited with status 0
The mechanics: after PTRACE_ATTACH, the tracer waitpid()s for the attach-induced stop, sets PTRACE_O_TRACESYSGOOD exactly as before, then reuses an identical syscall-entry/exit loop; PTRACE_SYSCALL/PTRACE_GETREGSET/PTRACE_PEEKDATA don't care whether the tracee arrived via PTRACE_TRACEME or PTRACE_ATTACH, only the setup differs. In a real incident-response or security-tooling scenario the target PID is normally already known (from ps, a monitoring alert, or a specific process under investigation), so this attach-based path, not the fork+TRACEME one, is the code you'd actually reach for; fork+TRACEME is the right choice only when the tracer also controls how the target gets launched, the way strace prog does by default.
Performance and security limitations of ptrace in production
- Overhead. Every traced syscall costs (at minimum) two extra context switches between tracer and tracee (one stop at syscall-entry, one at syscall-exit) plus a
waitpid()round trip in the tracer, on top of whateverPTRACE_PEEKDATA/PTRACE_GETREGSETcalls the tracer makes to actually inspect the state. For a syscall-heavy workload, this is not a small tax, tracing every syscall of a busy process can slow it down by an order of magnitude or more, which is whyptrace-based tracing (unlike eBPF-based tracing) is a diagnostic/debugging tool, not something you'd leave attached to a production service continuously. - Single tracer per process. Only one tracer can be attached to a given process at a time (via the standard ptrace API); a debugger and a tracing tool can't both attach simultaneously.
- Setuid/setgid exemption. The kernel refuses to let an unprivileged tracer
ptrace()-attach to a process running a setuid or setgid binary: allowing that attach would let a tracer with only ordinary-user credentials read and rewrite the memory of a process running with privilege the tracer itself doesn't have, defeating the whole point of the setuid/setgid mechanism. This is a real limitation if the thing you need to trace legitimately runs with elevated privilege; you either need to be privileged yourself (typically root, orCAP_SYS_PTRACE) or you're locked out of tracing it this way. - PTRACE_PEEKDATA's word-at-a-time cost. Reading a string out of the target's memory one machine word per syscall, as the demo above does, is simple to reason about but slow for large reads;
process_vm_readv(2)(a newer, purpose-built syscall for cross-process memory reads) does the same job in far fewer syscalls and is the better choice for a tracer that needs to read substantial amounts of target memory, not just short argv strings. - It's a debugging primitive, not a hardened sandbox boundary. A ptrace-based interception layer (e.g. denying/rewriting syscalls via
PTRACE_SYSCALLplusSECCOMP_RET_TRACE) is significantly more fragile and slower than a seccomp-bpf filter for actually restricting what a process can do; ptrace is the right tool for observing and debugging, seccomp is the right tool for enforcing a policy, and conflating the two (using ptrace as your actual security boundary rather than as an inspection tool) is a common mistake.
Describe TOCTOU (time-of-check to time-of-use) race conditions when performing file system operations. Provide concrete examples where checking a path then opening it leads to vulnerabilities. Explain how openat(2), O_NOFOLLOW, O_DIRECTORY, and fstatat(2) can be used to avoid races and perform secure atomic checks and opens.
Sample Answer
A concrete worked example first
Consider a security scanner tasked with hashing files it's told to inspect: it does stat(path) to confirm the target is a regular file under some size limit, decides it's safe, and then open(path)s the SAME path string to read and hash its contents. Between the stat() call and the open() call, an attacker with write access to that directory (a shared upload/scan-queue directory, or any directory a lower-privileged or scanned user controls) deletes the original file and creates a symlink at the exact same path, pointing at /etc/shadow, a device file, or any other file entirely. The scanner's open() transparently follows the new symlink, and the tool hashes and reports on a completely different file than the one it validated, either exfiltrating or misreporting on privileged content it should never have touched, or, if the operation had been a write instead of a read, corrupting a file the scanner never intended to write to at all. The vulnerability exists purely because "check" and "use" are two independent path resolutions with an exploitable gap between them; this is TOCTOU (time-of-check to time-of-use), the general name for this race condition class.
Generalizing: any check-path-then-open-path sequence is exposed
The same shape recurs anywhere code does: resolve a path to decide something (is it a regular file, does it belong to the expected owner, is it under a size limit), then LATER performs a privileged action by resolving that path string again (open, unlink, chmod, exec). Anything with write access to a directory in that path, at any point between the two resolutions, can change what the path means. This applies just as much to a config loader that validates a file before parsing it, a backup tool that stats before archiving, or a privilege-dropping helper that checks a target binary before exec'ing it.
How openat, O_NOFOLLOW, O_DIRECTORY, and fstatat close the race
openat(dirfd, name, ...): resolvesnamerelative to an already-open directory file descriptor rather than re-walking a path string from the root each time. On its own it doesn't remove TOCTOU, but it is the building block the rest of the mitigation is built on, because it anchors resolution to a specific, already-validated directory object rather than a name that has to be looked up fresh.O_NOFOLLOW: passed toopen()/openat(), makes the call FAIL withELOOPif the final path component is a symlink, instead of silently following it. This converts "attacker swapped the file for a symlink" from a silent, invisible redirection into a loud, immediately checkable error.O_DIRECTORY: fails the open unless the resolved target is genuinely a directory, which matters when walking a tree, so a symlink planted where a subdirectory was expected can't trick the walker into descending into somewhere else entirely.fstatat(dirfd, name, &st, AT_SYMLINK_NOFOLLOW): the*at-family equivalent of stat, checking metadata using the same directory-fd-anchored resolution the subsequentopenat()will use, rather than re-resolving a brand-new path string from the filesystem root as a second, independent operation (itself a second opportunity for the meaning of the path to differ, especially across mount namespaces or symlinked intermediate directories).
The secure atomic pattern
Reorder the operations so the check happens AFTER acquisition, not before: openat(dirfd, name, O_RDONLY | O_NOFOLLOW) first (a single resolution, with symlinks rejected outright), and only then fstat() the resulting fd to verify size, type, and ownership. There is no remaining check-then-use gap, because by the time any check runs, the fd already IS the object being checked; there is no second resolution left for an attacker to hijack. Applied to the worked example above, the scanner would open the candidate file with O_NOFOLLOW first (failing loudly if it's a symlink) and confirm S_ISREG via fstat() on that descriptor before ever reading a byte, closing the exact window the naive stat-then-open version left wide open.
Unlock Full Question Bank
Get access to all 7 System Calls & the Kernel Interface interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.