Shell Scripting Basics Questions

Practical ability to write and maintain simple shell scripts using Bash and other Unix shells to automate routine system and DevOps tasks. Core skills include variable assignment and substitution, control flow with conditionals such as if and else, looping constructs such as for and while, defining and invoking functions, basic error handling and exit code checks, and logging for observability. Candidates should also be comfortable with basic file and directory operations, invoking and composing commands, and text processing using tools such as grep, sed, and awk. Emphasis is placed on scripting best practices including input validation, defensive programming, clear code organization and readability, making scripts idempotent where appropriate, and handling common edge cases when automating system administration or deployment workflows.

MediumTechnical

87 practiced

Write a Bash script that finds files under /var/log modified in the last 24 hours and compresses them with gzip. The script must correctly handle filenames with spaces, newlines, and other special characters. Provide the command(s) and explain choices between -print0/xargs -0 and -exec approaches.

Sample Answer

Approach: use find to locate regular files under /var/log modified in the last 24 hours, exclude already-compressed files, and pass those filenames safely to gzip without breaking on spaces/newlines/special chars. Two safe options: null-delimited pipeline (find -print0 | xargs -0) or find -exec ... {} +. Both avoid word-splitting; choose based on availability and simplicity.

Script (recommended):

bash

#!/usr/bin/env bash
set -euo pipefail

# Find regular files modified in last 24 hours and compress with gzip.
# Exclude files already ending with .gz to avoid double-compressing.
# Use -print0 and xargs -0 to handle any character in filenames safely.
find /var/log -type f -mtime -1 ! -name '*.gz' -print0 \
  | xargs -0 -r gzip --

Alternate (no xargs, using find builtin batching):

bash

#!/usr/bin/env bash
set -euo pipefail

# find's -exec ... {} + batches arguments and is also safe for special chars.
find /var/log -type f -mtime -1 ! -name '*.gz' -exec gzip -- {} +

Why -print0 / xargs -0 vs -exec:- Both handle special characters (spaces, newlines, tabs) correctly when using null-termination or {} +.- -print0 | xargs -0: efficient, portable to do batching, and lets you control xargs options (like parallelism with -P). Requires xargs that supports -0.- -exec ... {} +: simpler (no pipeline), POSIX find supports it, also batches to reduce process overhead. Slightly more straightforward and fewer moving parts.- Avoid -exec ... \; (runs gzip per file — slower) and avoid plain find | xargs without -0 (unsafe for filenames with whitespace/newlines).

Edge cases & notes:- Ensure you exclude files already compressed (.gz) or adapt behavior if you want to compress them too.- If logs are being written to, compressing active files can break rotation; prefer integrating with logrotate or coordinating with services (e.g., restart or copytruncate).- For parallel compression: use xargs -0 -P N gzip -- to speed up on multi-core systems, but be cautious with I/O load.

MediumTechnical

81 practiced

Explain the effects and trade-offs of using 'set -euo pipefail' in Bash scripts. Provide a small example script that demonstrates each option (-e, -u, -o pipefail) and show how to safely run commands whose non-zero exit codes you want to ignore. Mention any pitfalls of set -e with compound commands or while loops.

Sample Answer

Using set -euo pipefail makes Bash scripts more robust by failing fast on common classes of bugs, but it changes control-flow semantics so you must be deliberate.

Effects:- -e: exit immediately if a simple command returns non‑zero (fails fast).- -u: treat unset variables as errors (helps catch typos).- -o pipefail: cause a pipeline to fail if any element fails (not just the last).

Trade-offs:- Pros: surface errors early, avoid silently proceeding after a failure, catch typos.- Cons: can break legitimate control flow (commands used for tests), requires explicit handling when non-zero exits are acceptable, and surprises appear in compound constructs (if/while/&&/||, subshells, and pipelines) because set -e is ignored in certain contexts.

Example script demonstrating each and safe ignores:

bash

#!/usr/bin/env bash
set -euo pipefail

# -u demonstration (will error if MISSING not set)
# echo "Missing is $MISSING"  # uncommenting would exit

# -e demonstration: simple failing command causes exit
false || true   # safe: we explicitly ignore failure with || true

# unsafe: this would cause exit because -e triggers on 'false' here if not guarded
# false

# -o pipefail demonstration: pipeline fails if any command fails
# Here grep fails (non-zero) so pipeline exits with failure
echo "a" | grep -q b || echo "grep failed as expected"

# Safely ignore a command that may fail:
if ! command_that_may_fail; then
  echo "non-fatal failure handled"
fi

# Alternatively temporarily disable -e for a region
set +e
command_that_may_fail
rc=$?
set -e
echo "rc=$rc"

# Pitfall: in while/read loops, pipeline behavior matters
# Using process substitution or pipefail matters:
echo -e "line1\nline2" | while read -r line; do
  # if a command here fails, with -e it may not exit the outer script reliably
  echo "got: $line"
done

Pitfalls to watch:- set -e does NOT abort when a failing command is part of if/while/until condition, part of &&/|| lists, or when prefixed with ! — this is by design. That means tests you expect to trigger exits may be suppressed.- In pipelines, without pipefail only the last command's status is considered; use pipefail to catch earlier failures.- For loops reading from pipelines, test behavior can be surprising—use process substitution or check exit codes explicitly.

Best practices:- Use explicit checks (if/then) or || true where appropriate.- Capture and check return codes if you need selective tolerance.- Keep scripts small, test them, and document places where failures are intentionally ignored.

HardTechnical

66 practiced

Write a Bash monitoring script that polls an HTTP health endpoint (URL provided as an argument) every N seconds. On failures, it should perform exponential backoff up to a maximum delay, send an alert via a webhook when consecutive failures exceed a threshold, and reset the failure count on a successful check. The script must be idempotent and avoid duplicate alerts when multiple instances run simultaneously.

Sample Answer

This script polls a health endpoint every INTERVAL seconds, uses exponential backoff on failures up to MAX_DELAY, sends a single alert via a webhook when failures reach THRESHOLD, resets failure count on success, and avoids duplicate alerts when multiple instances run by using an atomic lock directory for alerting and a state file for failure count. It is idempotent (no global side effects beyond state files) and safe to run from multiple hosts that share the same filesystem for locks/state (or adapt to a distributed lock if needed).

bash

#!/usr/bin/env bash
# Usage: monitor.sh URL WEBHOOK_URL [INTERVAL=5] [THRESHOLD=3] [MAX_DELAY=300]
set -euo pipefail

URL="$1"
WEBHOOK="$2"
INTERVAL="${3:-5}"      # base polling interval (seconds)
THRESHOLD="${4:-3}"     # consecutive failures before alert
MAX_DELAY="${5:-300}"   # max backoff delay (seconds)

STATE_DIR="/var/run/monitor_$(echo "$URL" | sha256sum | cut -d' ' -f1)"
mkdir -p "$STATE_DIR"
FAIL_FILE="$STATE_DIR/fail_count"
LOCK_DIR="/var/lock/monitor_alert_lock"  # atomic lock via mkdir

# init
: > "$FAIL_FILE" 2>/dev/null || true

send_alert() {
  # acquire atomic alert lock so only one instance sends alert
  if mkdir "$LOCK_DIR" 2>/dev/null; then
    # we hold the lock
    payload="{\"text\":\"ALERT: $URL failed ${1} times at $(date -u +%Y-%m-%dT%H:%M:%SZ)\"}"
    curl -sS -X POST -H "Content-Type: application/json" -d "$payload" "$WEBHOOK" || {
      echo "Warning: webhook POST failed" >&2
    }
    # optionally write a marker so duplicates across restarts are avoided until reset
    echo "$(date +%s)" > "$STATE_DIR/alert_sent"
    rmdir "$LOCK_DIR"
  else
    # another instance is sending alert or already sent it
    :
  fi
}

get_fail_count() {
  if [[ -f "$FAIL_FILE" && -s "$FAIL_FILE" ]]; then
    cat "$FAIL_FILE"
  else
    echo 0
  fi
}

set_fail_count() {
  printf "%d" "$1" > "$FAIL_FILE"
}

while true; do
  # perform health check
  if curl -sS -m 10 -o /dev/null -w "%{http_code}" "$URL" | grep -qE '^2|^3'; then
    # success: reset counters and remove alert marker
    set_fail_count 0
    rm -f "$STATE_DIR/alert_sent"
    sleep "$INTERVAL"
  else
    # failure: increment failure count
    fail_count=$(get_fail_count)
    fail_count=$((fail_count + 1))
    set_fail_count "$fail_count"

    # send alert when threshold reached and not already alerted
    if (( fail_count >= THRESHOLD )) && [[ ! -f "$STATE_DIR/alert_sent" ]]; then
      send_alert "$fail_count"
    fi

    # exponential backoff: min(MAX_DELAY, INTERVAL * 2^(fail_count-1))
    delay=$(( INTERVAL * (2 ** (fail_count - 1)) ))
    if (( delay > MAX_DELAY )); then delay=$MAX_DELAY; fi

    sleep "$delay"
  fi
done

Key points:- Uses atomic mkdir for alert locking to avoid duplicate alerts from concurrent instances.- Stores fail count and alert marker in STATE_DIR so script is idempotent across restarts.- Resets failure count and alert marker on success to allow future alerts.- Curl timeout prevents hangs; http codes 2xx/3xx considered success.Edge cases & recommendations:- If running across multiple hosts without shared FS, replace mkdir lock with a distributed lock (e.g., Redis SETNX or consul).- Ensure permissions allow writing to STATE_DIR (/var/run) or change to /tmp for user-level.- Consider HMAC/auth for webhook and retry logic for webhook failures.

MediumTechnical

77 practiced

Create a Bash script that moves files older than X days (configurable) into a quarantine directory instead of deleting them. The script should preserve ownership and permissions, support a --dry-run mode, and handle filename collisions by renaming files using a checksum suffix. Include restore instructions for quarantined files.

Sample Answer

Approach: use find to locate files older than X days, compute checksum to avoid name collisions, move files into quarantine preserving ownership/permissions with rsync or cp --preserve then remove original, support --dry-run flag to print actions only, record metadata (original path, checksum, timestamp) in a manifest for restore.

bash

#!/usr/bin/env bash
# quarantine.sh - move files older than X days into quarantine
set -euo pipefail

QUARANTINE_DIR="/var/quarantine"
DAYS=30
DRY_RUN=0
MANIFEST="${QUARANTINE_DIR}/manifest.csv"

usage(){ echo "Usage: $0 [-d days] [--dry-run] <path>"; exit 2; }

# parse args
while [[ $# -gt 0 ]]; do
  case "$1" in
    -d|--days) DAYS="$2"; shift 2;;
    --dry-run) DRY_RUN=1; shift;;
    -h|--help) usage;;
    *) TARGET="$1"; shift;;
  esac
done
[[ -v TARGET ]] || usage
mkdir -p "$QUARANTINE_DIR"

find "$TARGET" -type f -mtime +"$DAYS" -print0 | while IFS= read -r -d '' file; do
  # compute checksum
  checksum=$(sha256sum "$file" | awk '{print $1}')
  base=$(basename "$file")
  dest="${QUARANTINE_DIR}/${base}"
  # handle collisions by suffixing checksum
  if [[ -e "$dest" ]]; then
    dest="${QUARANTINE_DIR}/${base}.${checksum:0:8}"
  fi
  relpath=$(realpath --relative-to=/ "$file")
  ts=$(date --iso-8601=seconds)
  if [[ $DRY_RUN -eq 1 ]]; then
    echo "[DRY-RUN] would move: $file -> $dest"
    echo "[DRY-RUN] manifest entry: $ts,$checksum,$file,$dest"
  else
    # copy preserving perms/ownership, then remove original
    rsync -aHAX --remove-source-files --chmod=ugo=rwX "$file" "$dest"
    # ensure ownership preserved (rsync -a does this if run as root)
    echo "$ts,$checksum,$file,$dest" >> "$MANIFEST"
  fi
done

Key points:- Uses sha256 checksum suffix to avoid name collisions.- rsync -aHAX preserves ownership, permissions, xattrs; --remove-source-files moves safely.- --dry-run prints planned actions.

Restore instructions:- Inspect manifest at /var/quarantine/manifest.csv.- To restore one entry: verify checksum matches: sha256sum -c <(echo "checksum quarantined_path") then move back with rsync preserving metadata: rsync -aHAX /var/quarantine/filename /original/path/- For bulk restore, iterate manifest entries, verify checksums, recreate directories, rsync back, and remove manifest lines after successful restore.

Edge cases:- Hardlinks, special files, not enough permissions (run as root), concurrent runs—use locking for production.

MediumTechnical

63 practiced

Describe reliable ways to parse JSON from a Bash script. Provide example commands using jq to extract fields, including how to handle missing fields and arrays. Also show a fallback using python -c for environments without jq and discuss pros/cons of each method.

Sample Answer

Reliable JSON parsing in bash is best done with a proper JSON tool (jq). Avoid fragile hacks (awk/grep/sed). Below are practical jq examples, handling missing fields, arrays, and a python fallback.

Approach: use jq to parse and validate JSON, return exit codes or defaults when keys are missing, and map arrays safely.

Extract a top-level field:

bash

# JSON: {"service":"api","port":8080}
service=$(jq -r '.service // empty' <<<"$json")   # prints value or empty string
port=$(jq -r '.port // 0' <<<"$json")            # default to 0 if missing

Handle nested fields and detect missing keys:

bash

host=$(jq -r '.network.host // empty' <<<"$json")
if [[ -z $host ]]; then
  echo "network.host missing" >&2
  exit 2
fi

Parse arrays robustly:

bash

# JSON: {"nodes":[{"name":"a"},{"name":"b"}]}
jq -r '.nodes[]?.name // empty' <<<"$json"        # safe: ? avoids error if nodes missing/null
# To join into CSV:
nodes_csv=$(jq -r '[.nodes[]?.name // empty] | join(",")' <<<"$json")

Return structured exit status when JSON invalid:

bash

if ! echo "$json" | jq -e . >/dev/null 2>&1; then
  echo "invalid json" >&2; exit 3
fi

Fallback when jq unavailable (python -c):

bash

# extract service with fallback default using python stdlib
service=$(python -c 'import sys,json; j=json.load(sys.stdin); print(j.get("service",""))' <<<"$json")

Pros/Cons:- jq: fast, purpose-built, expressive, safe for streaming and complex transforms. Requires installation on minimal images.- python -c: available on many systems, fine for simple extracts, but harder to read, maintain, and risky for complex queries (escaping/quoting). subprocess overhead and less concise for streaming.- Avoid regex hacks: brittle with whitespace/ordering/escaping.

Best practice: prefer jq; validate JSON first; provide clear defaults and exit codes for automation; include python fallback for environments where jq cannot be installed.

Unlock Full Question Bank

Get access to hundreds of Shell Scripting Basics interview questions and detailed answers.

Join thousands of developers preparing for their dream job.