Linux System Administration Fundamentals Questions

Core Linux administration knowledge and hands on operational skills required to install, configure, and maintain Linux systems. Covers user and group management, file permissions and ownership, process management and signals, package management across distributions, the boot process and runlevels or targets, basic systemd service control, filesystem navigation and basic disk management, common system configuration files, shell and command line proficiency, and differences between major enterprise and community distributions. Candidates should demonstrate practical troubleshooting of routine issues, patching and updates, and an ability to perform day to day administration tasks reliably.

MediumTechnical

33 practiced

Explain how to load and unload kernel modules safely (using modprobe/rmmod) and make module options persistent. Demonstrate how to enable IPv4 forwarding immediately and persistently (sysctl -w and /etc/sysctl.d/), and discuss potential risks to verify before changing these kernel parameters on production hosts.

Sample Answer

Load/unload kernel modules safely- To load a module use modprobe (handles dependencies):

bash

sudo modprobe <module_name>

- Verify loaded and dependencies:

bash

lsmod | grep <module_name>
modinfo <module_name>
dmesg | tail -n 50

- To remove:

bash

sudo modprobe -r <module_name>   # removes respecting dependencies
# or rmmod <module_name>  (fails if in use)

- Before unloading check usage/count and consumers:

bash

cat /sys/module/<module_name>/refcnt
lsmod | grep <module_name>
lsof /proc/*/fd | grep <module_device>   # find users

- If module is in use, stop services or unbind devices first; avoid forcing removal on production.

Make module options persistent- Create a file under /etc/modprobe.d/, e.g. /etc/modprobe.d/custom.conf:

text

options <module_name> some_option=1 other_flag=0
install <module_name> /sbin/modprobe --ignore-install <module_name> && /bin/echo "..."  # optional

- For blacklisting:

text

blacklist <module_name>

- Changes apply on next load/reboot; to apply immediately, rmmod/modprobe -r then modprobe (if safe).

Enable IPv4 forwarding immediately and persistently- Immediate (non-persistent):

bash

sudo sysctl -w net.ipv4.ip_forward=1

- Persistently via /etc/sysctl.d/:Create /etc/sysctl.d/99-ipforward.conf:

text

net.ipv4.ip_forward = 1

Apply all sysctl config:

bash

sudo sysctl --system

Risks and verifications before changing on production- Security: Enabling ip_forward turns host into a router; check firewall (iptables/nft) rules, ensure forwarding policies are tight to avoid becoming an open router.- Routing & topology: Validate routing tables, prevent asymmetric routing, loops, or accidentally bridging networks.- NAT/conntrack: Increased connection tracking memory usage; check /proc/sys/net/netfilter/nf_conntrack_max and monitor conntrack table to avoid exhaustion.- Performance: Forwarding increases CPU/network path load; benchmark expected throughput and ensure NIC and kernel offloads handle it.- Module changes: Unloading modules in use can crash services or kernel panic. Verify no consumers, check dmesg for errors, test in staging, and have rollback plan (console access, rescue).- Compatibility: Ensure module options are supported by current kernel and preserved across kernel updates.- Observability & rollback: Add monitoring/alerts, document change, and schedule maintenance window if impact risk > low.

This approach ensures safe changes, persistence across reboots, and mitigates operational risks.

HardTechnical

53 practiced

Users report intermittent packet loss between two datacenter hosts. Provide a detailed, prioritized debugging plan using ping, traceroute, mtr, ss, tcpdump and ethtool. Explain how to check NIC counters, offload settings, firmware issues, and how to determine whether the problem originates in the OS, the NIC, or upstream network/switch.

Sample Answer

Start with clarifying scope: which hosts, when (times), frequency, affected flows (TCP/UDP/ICMP), and SLA. Prioritize reproducible, non-disruptive tests first.

1) Quick connectivity checks (scope narrowing)- From source to dest run ping for loss and RTT jitter:

bash

ping -c 100 -i 0.2 dest

Look for intermittent packet loss, spikes in rtt/stddev, and whether loss correlates with time-of-day.

- Path verification:

bash

traceroute -n dest
mtr --report --report-cycles=100 dest

mtr shows per-hop loss; if loss appears at a hop then persists, likely upstream. If loss appears only at endpoints, suspect NIC/OS.

2) Live flow-level checks- Use ss to list sockets and retransmits (TCP):

bash

ss -s
ss -ti dst:PORT

High retransmits or zero-window indicate TCP-layer problems (could be receiver overload).

3) Capture packets around events- tcpdump on both hosts with timestamps and ring buffer:

bash

tcpdump -i eth0 -w /tmp/cap.pcap -s 0 host other_host and '(tcp or icmp)' &

Compare captures at both ends: if sender saw packet transmitted but receiver never received, likely network/switch. If sender didn't transmit, problem is OS/NIC. Use sequence numbers to detect duplication/reordering.

4) Inspect NIC and driver- ethtool status, stats, and offloads:

bash

ethtool -S eth0   # vendor counters
ethtool -k eth0   # offload settings
ethtool -i eth0   # driver/firmware info
ethtool -T eth0   # hardware timestamping

Check rx_errors, tx_errors, rx_missed, fifo errors, and if GRO/TSO/GSO/XDP offloads are on. Disable offloads temporarily to rule out offload bugs:

bash

ethtool -K eth0 tso off gso off gro off rx off tx off

If loss disappears, offload/driver issue likely.

5) Hardware counters and interface state- ip -s link show eth0- ethtool -S for vendor counters (e.g., rx_no_buffer, extspr).Interpret: incrementing queue drops or CRC errors -> physical/link issue (cable, SFP, switch port).

6) Firmware/driver and BIOS- Compare ethtool -i driver and firmware versions to vendor release notes. If known buggy versions, plan upgrade. Check dmesg for NIC driver warnings, firmware resets, or queue warnings:

bash

dmesg | egrep -i 'eth|e1000|ixgbe|bnx2|mlx'

7) Switch/upstream validation- If mtr/tcpdump indicate loss before destination, request switch-side counters: interface errors, drops due to policing, TC, buffer exhaustion. Ask network team for sFLOW/ERSPAN capture or mirror port. Verify no LACP flapping or port-channel misconfig.

8) Correlation and automation- Correlate metrics: NIC counters, CPU load, interrupt/softirq storms (top, sar), irqbalance, and tc/iptables rules. Use perf/top to check process causing packet drops due to overload.- If intermittent, schedule continuous tcpdump + ethtool -S snapshot every minute and store for post-analysis.

Decision heuristics:- Loss shows in mtr at intermediate hop → upstream/switch.- Loss on both sends/receives at endpoints and NIC counters show rx/tx errors → physical NIC/link.- Sender never emits packets (tcpdump) but app expects them → OS/kernel/resource exhaustion (queue drops, overwhelmed NIC rings).- Offload toggling fixes issue → driver/firmware or offload bug.

Final: Document findings, revert temporary changes (re-enable offloads if necessary), and coordinate with network vendor for firmware/firmware patches or switch config changes.

EasyTechnical

37 practiced

You attached a new block device /dev/sdb to a Linux VM. Provide a step-by-step command sequence to: create a single partition, format it as ext4, mount it at /data, set ownership to user 'app' and group 'app', and add a persistent /etc/fstab entry using the partition UUID. Explain each command used and how to verify everything is correct.

Sample Answer

Approach: create one partition on /dev/sdb, format ext4, mount at /data, set ownership, and add a persistent /etc/fstab entry using the partition UUID so it survives reboots.

Commands and explanations:

1) Inspect device:

lsblk -f /dev/sdb

Shows block devices, existing partitions and filesystems.

2) Create a single partition (GPT) that fills the disk:

sudo parted -s /dev/sdb mklabel gpt mkpart primary 0% 100%

- parted -s: script mode (no interactive).- mklabel gpt: create GPT partition table.- mkpart primary 0% 100%: one partition using entire device (creates /dev/sdb1).

3) Verify partition created:

lsblk -o NAME,SIZE,FSTYPE,TYPE,MOUNTPOINT /dev/sdb

4) Format the new partition as ext4:

sudo mkfs.ext4 -F /dev/sdb1

- -F forces; mkfs.ext4 creates an ext4 filesystem on /dev/sdb1.

5) Create mount point and mount:

sudo mkdir -p /data
sudo mount /dev/sdb1 /data

- mount attaches the filesystem to /data.

6) Set ownership to user and group 'app':

sudo chown app:app /data
sudo chmod 750 /data    # optional: limits access

7) Get the partition UUID (use this in /etc/fstab):

sudo blkid -s UUID -o value /dev/sdb1

This prints the UUID (e.g. 1234-abcd-...).

8) Add persistent /etc/fstab entry (using the UUID you obtained). Example append:

UUID=YOUR-UUID-HERE /data ext4 defaults,noatime 0 2

To append safely:

sudo sh -c "echo 'UUID=$(blkid -s UUID -o value /dev/sdb1) /data ext4 defaults,noatime 0 2' >> /etc/fstab"

- defaults,noatime: sensible defaults and avoid updating atime.- dump=0, fsck order=2 (root is 1).

9) Test fstab entry without reboot:

sudo umount /data
sudo mount -a

- mount -a mounts all fstab entries; if there is an error it will report.

Verification:

- Confirm mounted:

findmnt /data
df -h | grep /data
lsblk -f
mount | grep /dev/sdb1

- Confirm ownership:

stat -c '%U %G %a' /data

- Confirm fstab entry:

grep /data /etc/fstab

Edge cases / notes:- If /dev/sdb already had partitions, adapt commands (use correct partition name).- Use careful maintenance windows if device holds critical data.- If using MBR instead of GPT, change mklabel to msdos.- For large disks or special alignment needs, specify start/end in MiB.

MediumTechnical

50 practiced

Explain the differences between SIGTERM, SIGKILL, SIGHUP, SIGQUIT and SIGSTOP on Linux. For a long-running Java application, describe how you would implement graceful shutdown on SIGTERM and configuration reload on SIGHUP, and how to configure the systemd unit to support graceful termination (e.g. ExecStop, TimeoutStopSec).

Sample Answer

SIG semantics (brief):- SIGTERM: polite request to terminate — process can catch, clean up, and exit.- SIGKILL: uncatchable, immediate termination (cannot be handled).- SIGHUP: traditionally “hangup” — often used to reload configuration; catchable.- SIGQUIT: like SIGTERM but produces core dump; catchable but usually means debugging.- SIGSTOP: uncatchable stop (suspend) the process; resumed by SIGCONT.

Graceful shutdown (SIGTERM) in a long-running Java app:- Catch shutdown and stop accepting work, finish in-flight tasks, flush state, close resources, deregister from load balancers, then exit.- Use Runtime.addShutdownHook for JVM-level shutdown. If you need immediate signal semantics (SIGHUP), use a signal library or an external control endpoint.

Example (simplified):

java

import java.util.concurrent.*;

public class App {
  private final ExecutorService pool = Executors.newFixedThreadPool(10);
  public static void main(String[] args) {
    App app = new App();
    Runtime.getRuntime().addShutdownHook(new Thread(() -> {
      System.out.println("SIGTERM received: shutting down");
      app.stop();
    }));
    app.start();
  }
  void start() { /* accept work, schedule tasks on pool */ }
  void stop() {
    try {
      // stop accepting new tasks
      pool.shutdown();
      if (!pool.awaitTermination(30, TimeUnit.SECONDS)) {
        pool.shutdownNow();
      }
      // close DB, flush metrics, notify LB
    } catch (InterruptedException e) {
      Thread.currentThread().interrupt();
    }
  }
}

Configuration reload (SIGHUP):- Prefer: expose an admin HTTP endpoint (e.g., /-/reload) that triggers a config reload (thread-safe swap of config objects), then have system process or admin call it on SIGHUP.- If you handle signals in-JVM: use a safe signal handler library (e.g., jnr-posix or sun.misc.Signal with caution) to invoke reload logic on SIGHUP. Ensure reload is atomic (immutable config object + volatile reference) and validate new config before swap.

systemd unit settings for graceful termination:- Use ExecStart to run your app; set ExecStop to a command to request graceful shutdown if needed (or rely on SIGTERM).- Configure: - KillMode=process (or control-group) depending whether children should be killed - KillSignal=SIGTERM - TimeoutStopSec=60 (give app 60s to clean up) - Send SIGQUIT or SIGKILL afterward if not stopped.

Example snippet:[Service]ExecStart=/usr/bin/java -jar /opt/myapp/app.jar# optional: ExecStop can call curl to admin endpoint for graceful drainExecStop=/usr/bin/curl -sS --max-time 5 http://127.0.0.1:8081/-/drainKillMode=control-groupKillSignal=SIGTERMTimeoutStopSec=60Restart=on-failure

Notes and best practices:- Test with systemd (systemctl stop) to ensure shutdown completes within TimeoutStopSec.- Make shutdown idempotent and fast; prefer cooperative draining (stop accepting new traffic first).- Keep SIGHUP reload safe: validate config, fallback on errors, and expose health checks so orchestrator can detect bad reloads.

MediumTechnical

36 practiced

How would you use SMART to check the health of /dev/sda? Show the commands to run a short and long self-test, how to inspect attributes like Reallocated_Sector_Ct and Current_Pending_Sector, and explain thresholds or trends that indicate a drive should be replaced.

Sample Answer

Use smartctl (from smartmontools) to query SMART, run self-tests, and inspect key attributes. Basics first: install smartmontools and ensure SMART is enabled.

Commands to check basic SMART health and full attributes:

bash

# Basic health summary
sudo smartctl -H /dev/sda

# Full SMART report including attributes and tests
sudo smartctl -a /dev/sda

Run short and long self-tests:

bash

# Start a short self-test (typically a few minutes)
sudo smartctl -t short /dev/sda

# Start a long/extended/extended offline self-test (can take hours)
sudo smartctl -t long /dev/sda

# List self-test log/results
sudo smartctl -l selftest /dev/sda

Inspect key attributes (example names appear in smartctl -a output):- Reallocated_Sector_Ct (ID 5): count of remapped sectors. RAW value > 0 means sectors were remapped.- Current_Pending_Sector (ID 197): sectors awaiting reallocation (unstable reads).- Offline_Uncorrectable (ID 198): count of sectors that could not be corrected.- For SSDs: Wear_Leveling_Count, Media_Wear_Leveling_Indicator, or Percent_Lifetime_Remain; also Total_LBAs_Written.

Example grep for attributes:

bash

sudo smartctl -a /dev/sda | egrep 'Reallocated_Sector_Ct|Current_Pending|Offline_Uncorrectable|Wear_Leveling'

How to interpret thresholds and trends:- Single reallocated sectors: not immediate panic, but it's a warning. Track growth over time.- Current_Pending_Sector > 0: strong signal of imminent data loss risk — run a long test and off-line backup, schedule replacement if it doesn't clear.- Rapidly increasing Reallocated_Sector_Ct or Pending counts over days/weeks → replace drive.- Any Offline_Uncorrectable > 0 is serious; replace promptly.- For SMART overall-health (-H) returning FAILED → replace immediately.- SSDs: decreasing Percent_Lifetime_Remain or high wear-leveling/low remaining endurance → plan replacement before it reaches vendor EOL.

Monitoring and alerting recommendations:- Capture smartctl -a regularly (cron or node_exporter textfile) and send metrics to monitoring (Prometheus/Grafana).- Alert if: - Reallocated_Sector_Ct increases by >0 since last check - Current_Pending_Sector > 0 - Offline_Uncorrectable > 0 - smartctl -H returns FAILED - SSD lifetime remaining below a threshold (e.g., 10-20%)

Operational steps on detection:1. Snapshot/backup important data immediately.2. Run sudo smartctl -t long and inspect results.3. If failures persist or counts grow, schedule planned replacement and rebuild RAID/replicate before removing device.

Notes:- SMART attribute names and thresholds vary by vendor; focus on trends and critical flags.- For production, automate collection and correlating SMART trends with I/O errors in system logs.

Unlock Full Question Bank

Get access to hundreds of Linux System Administration Fundamentals interview questions and detailed answers.

Join thousands of developers preparing for their dream job.