Shell Scripting and Automation Questions

Approach (brief): iterate over regular files in <logdir> older than <days>, for each log compress to a temp file next to the original using gzip -c, copy ownership/permissions from the original, atomically move the compressed file into place, remove the original log, then enforce keeping only the newest <keep> compressed files per prefix. Use per-file locking with flock to be safe under concurrent runs and use temp files + atomic mv to avoid partial states.

Code:

#!/usr/bin/env bash
set -euo pipefail

logdir=$1
days=$2
keep=$3

# ensure inputs valid
if [[ ! -d $logdir ]]; then echo "logdir not found"; exit 2; fi
if ! [[ $days =~ ^[0-9]+$ && $keep =~ ^[0-9]+$ ]]; then echo "days and keep must be integers"; exit 2; fi

# For each file older than $days (strictly older than)
find "$logdir" -maxdepth 1 -type f -mtime +"$days" -print0 |
while IFS= read -r -d '' file; do
  (
    # per-file lock to avoid races: lockfile per original
    lockfile="$file.lock"
    exec 9>"$lockfile"
    flock -x 9

    base=$(basename "$file")
    prefix="${base%%.*}"        # adjust if other prefix rule required
    target="$file.gz"
    tmp="$file.$$.gz.tmp"

    # skip if compressed already exists with same inode (rare) or another process compressed
    if [[ -e "$target" ]]; then
      # already compressed: remove original safely
      rm -f -- "$file" || true
      exit 0
    fi

    # create compressed temp
    if ! gzip -c -- "$file" > "$tmp"; then
      rm -f -- "$tmp"
      exit 1
    fi

    # preserve ownership and permissions using reference to original
    # (chown may require root; ignore errors conservatively)
    if command -v chown >/dev/null 2>&1; then
      chown --reference="$file" "$tmp" 2>/dev/null || true
    fi
    chmod --reference="$file" "$tmp" 2>/dev/null || true

    # atomic move into final name
    mv -T -- "$tmp" "$target"

    # remove original log (now compressed)
    rm -f -- "$file"

    # enforce keep: keep newest $keep compressed files matching prefix.*
    # sorting by mtime newest first
    mapfile -t gzlist < <(ls -1t -- "$logdir"/"$prefix"* 2>/dev/null | grep -E '\.gz$' || true)
    if (( ${#gzlist[@]} > keep )); then
      for ((i=keep; i<${#gzlist[@]}; i++)); do
        rm -f -- "${gzlist[i]}"
      done
    fi

  ) &
done

wait

Key points / reasoning:

• Use find -mtime +"$days" to pick files older than days.
• Per-file flock (lockfile + fd 9) serializes operations on a single log; different files can proceed in parallel.
• Write gzip output to a temp file in same directory, then chown/chmod --reference to copy ownership and permissions, then mv -T for atomic replacement: avoids partial .gz exposure.
• Removing original only after successful mv prevents data loss on failure.
• Enforcing retention using ls -1t keeps newest compressed files by mtime; deletion is done under the same lock, preventing races.
• Assumptions: prefix rule uses filename up to first dot; script must run with permissions to chown (optional) — if not root, chmod/chown lines may silently fail. Filesystem supports atomic mv within same FS. If high concurrency across processes on different hosts, use cluster-aware locking (e.g., lock files on shared storage supporting locks, or use a coordination service).

Edge / failure handling:

• gzip failure cleans up temp file.
• flock ensures two concurrent runs don't compress the same file simultaneously.
• Atomic mv prevents partial .gz files visible to readers.
• For stronger guarantees on distributed filesystems, swap per-file lock for an exclusive NFS/cluster lock or use leader election.

A shell variable is a name/value bound in the current shell process only. An environment variable is a name/value that has been marked for export so it is included in the process environment and therefore inherited by child processes.

In your snippet:
VAR=1
bash -c 'echo $VAR'

VAR=1 creates a shell variable in the current shell but is not exported. When you run bash -c, the parent spawns a new child process whose environment does not include VAR, so the child prints nothing.

To make VAR visible to child processes export it:
export VAR=1

or

VAR=1
export VAR

Now bash -c 'echo $VAR' will print 1. Note that exporting only affects children, not the parent.

Sourcing a script (source script.sh or . script.sh) runs the script in the current shell process rather than a subshell. Any variables it sets or exports remain in your current shell. That’s why you source profile scripts to set up environment variables for your session.

Difference between export and inline assignment for a single command:

• export VAR=val (or export VAR after setting it) sets VAR in the current shell and exports it to all future child processes.
• FOO=bar cmd sets FOO only in the environment of that single command and its children; it doesn’t modify the current shell’s variables after cmd finishes. Example:
  FOO=bar env | grep FOO # visible inside this command
  echo $FOO # empty in current shell unless exported

Remember: exported variables propagate down the process tree; nothing exported in a child comes back up to the parent.

Key differences that affect portability (POSIX /bin/sh vs Bash) and concrete rewrites.

1. Arrays

• Bash: indexed arrays (a=(one two)); ${a[1]}.
• /bin/sh: no arrays. Use positional params or delimited strings.
  Example bashism:

arr=(one two); echo "${arr[1]}"

Portable rewrite:

set -- one two
echo "$2"

2. [[ vs [

• Bash: [[ is richer (pattern matching, &&, || without quoting) and safer.
• POSIX: use [ (test) or use case for pattern matching.
  Bash:

if [[ $x == foo* && -n $y ]]; then ...

Portable:

case "$x" in foo*) :;; *) false;; esac
[ -n "$y" ] && ...

3. Process substitution <( )

• Bash-only. Use named pipes, temporary files, or plain pipes.
  Bash:

diff <(cmd1) <(cmd2)

Portable:

cmd1 > /tmp/out1.$$; cmd2 > /tmp/out2.$$
diff /tmp/out1.$$ /tmp/out2.$$
rm /tmp/out1.$$ /tmp/out2.$$

Or use pipes if diff supports reading from stdin.

4. Here-strings (<<<)

• Bash-only. Use printf/echo piped to stdin.
  Bash:

grep pattern <<<"$var"

Portable:

printf '%s\n' "$var" | grep pattern

5. Associative arrays

• Bash-only (declare -A). No POSIX equivalent; use files, kv pairs, or external tools (awk, sed).
  Example: replace with awk or use two parallel lists or a temporary file/db.

6. Regex operator =~

• Bash: [[ string =~ regex ]] returns match groups.
• POSIX: use grep, awk, expr, or case for basic patterns.
  Bash:

if [[ $s =~ ^[0-9]+$ ]]; then ...

Portable:

case "$s" in ''|*[!0-9]* ) false;; *) true;; esac
# or: printf '%s' "$s" | grep -Eq '^[0-9]+$'

7. Function declaration differences

• POSIX: name() { ...; } — portable.
• Bash allows: function name { ...; } (non-POSIX). Use name() style.

Typical bashisms checklist (avoid or rewrite):

• [[ ... ]], =~, (( arithmetic with C-syntax), arrays, associative arrays, process substitution, here-strings, ${var@Q}, ${!var*}, declare/local (local is in some shs but prefer POSIX 'set -f' patterns).

Team-wide guidelines to maximize portability

• Start scripts with explicit shebang: #!/bin/sh when you need POSIX, #!/usr/bin/env bash only when bash-specific features are required; document why.
• Prefer POSIX constructs: use set --, printf, case, [ ].
• Run shellcheck with POSIX target (SC2039 warnings). Configure CI to run scripts under dash (or /bin/sh) for integration tests.
• Document required shell features in README and add automated tests (lint + run under dash).
• Encapsulate non-portable logic in small, well-tested bash-only helpers and guard execution (check $BASH_VERSION).
• Use external tools (awk, sed, grep, jq) when complex data structures needed rather than bash extensions.

These practices keep SRE automation reliable across minimal containers and BSD/Linux variants.

Approach (brief): parse mounts to detect and exclude remote/pseudo filesystems, then use find limited to local /var filesystems to locate regular world-writable files, log their current mode and path, and chmod them to 0644. Run with least privilege by granting only this script (or its underlying commands) limited sudo rights; test with a dry-run first.

Script:

#!/usr/bin/env bash
set -euo pipefail

LOG="/var/log/fix_world_writable.log"
DRY_RUN=${DRY_RUN:-1}   # set to 0 to apply changes

# Filesystem types to treat as pseudo or remote
EXCLUDE_FSTYPES="proc sysfs devtmpfs tmpfs cgroup devpts fuse.lxcfs overlay"
REMOTE_FSTYPES="nfs nfs4 cifs smbfs sshfs afpfs"

# Build list of mountpoints under /var that are remote or pseudo
mapfile -t BAD_MPS < <(awk -v root="/var" -v excl="$EXCLUDE_FSTYPES $REMOTE_FSTYPES" '
  $2 ~ "^/var" {
    split(excl, a);
    ok=1;
    for(i in a) if($3==a[i]) ok=0;
    if(!ok) print $2 "::" $3
  }' /proc/mounts)

if [ "${#BAD_MPS[@]}" -ne 0 ]; then
  echo "Detected excluded mounts under /var:" | tee -a "$LOG"
  for m in "${BAD_MPS[@]}"; do echo "$m" | tee -a "$LOG"; done
  echo "Aborting to avoid touching remote/pseudo mounts." | tee -a "$LOG"
  exit 2
fi

# Find world-writable regular files under /var, avoid crossing into other mounts with -xdev per mount
# Determine local /var mountpoints to scan
mapfile -t VAR_MPS < <(findmnt -rn -o TARGET -S /var || echo "/var")

for mp in "${VAR_MPS[@]}"; do
  # Use -xdev to keep inside this mount
  while IFS= read -r file; do
    # log previous perms
    old=$(stat -c "%a %n" -- "$file")
    echo "$(date -Iseconds) BEFORE $old" >> "$LOG"
    if [ "$DRY_RUN" -eq 0 ]; then
      chmod 0644 -- "$file"
      new=$(stat -c "%a %n" -- "$file")
      echo "$(date -Iseconds) AFTER  $new" >> "$LOG"
    fi
  done < <(find "$mp" -xdev -type f -perm -0002 -print 2>/dev/null)
done

Key points / reasoning:

• Use /proc/mounts or findmnt to detect mountpoints and types; treat nfs/nfs4/cifs/sshfs as remote and proc/sysfs/devtmpfs/tmpfs etc. as pseudo — abort if any such mounts are under /var.
• find -type f guarantees regular files (no symlinks). -perm -0002 selects world-writable. -xdev prevents crossing mount boundaries for each mount scanned.
• Logging previous permissions with stat gives audit trail before change.
• Least privilege: changing perms requires root, so avoid running arbitrary root shells—place this script as a managed tool and grant a narrowly scoped sudoers entry (e.g., allow this script path with NOEXEC where supported). Always run dry-run first and test on a staging host.

Approach: validate argument, normalize unit name (append .service if missing), call systemctl is-active --quiet --unit to avoid output parsing, print human-friendly messages, and return 0 only when systemctl reports active. Note: systemctl is-active is generally readable by non-root; however, querying some units or private system namespaces may require root.

#!/usr/bin/env bash
# check-service.sh - exit 0 if service is active, non-zero otherwise
set -u

if [ "$#" -ne 1 ]; then
  echo "Usage: $0 <service-name>"
  exit 2
fi

unit="$1"
# normalize: allow "nginx" or "nginx.service"
if [[ "$unit" != *.service && "$unit" != *.socket && "$unit" != *.target ]]; then
  unit="${unit}.service"
fi

# Use --quiet so we rely on exit code not text (avoids false positives)
if systemctl --no-pager is-active --quiet -- "${unit}"; then
  echo "Service '${unit}' is active."
  exit 0
else
  # Provide reason by asking is-active without --quiet for user info
  state=$(systemctl --no-pager is-active -- "${unit}" 2>/dev/null || echo "unknown")
  echo "Service '${unit}' is NOT active (state: ${state})."
  exit 1
fi

Permissions: reading unit state usually works as non-root. If you get "Access denied" or checks against system-wide/private units fail, run as root (sudo) or ensure your user has permission to query systemd for that scope.