Malware Analysis and Reverse Engineering Questions

Approach summary
As a digital forensic examiner I perform cross‑view memory analysis with Volatility/Volatility3 to find injected/hidden processes, threads, network artifacts and keyloggers, corroborating with disk artifacts to reduce false positives.

Key commands / plugins & evidence

• Process listing / hidden processes
  • Volatility:

    volatility -f image.raw --profile=Win7x64 pslist
    volatility -f image.raw --profile=Win7x64 psscan
    

    Evidence: pslist shows active PIDs and parent/creation times; psscan finds terminated/hidden processes and orphaned EPROCESS structures.
  • Volatility3:

    vol.py -f image.raw windows.pslist
    vol.py -f image.raw windows.psscan
    

• DLLs / handles / suspicious injections

  • volatility dlllist
    volatility handles
    volatility -f image.raw --profile=... malfind
    

    Evidence: malfind shows suspicious RWX memory, injected code regions, dumped shells; dlllist/handles show loaded modules and unexpected handles.
• Threads & APCs

  • volatility threads
    volatility apihooks
    

    Evidence: unexpected threads, queued APCs used for stealthy injection or keylogging.
• Network connections

  • volatility netscan
    volatility connscan
    

    Evidence: in‑memory TCP/UDP endpoints, foreign IPs, sockets not present in network logs.
• Keylogger indicators
  • malfind (hooks), apihooks, userassist/clipboard, keystrokes (if plugin available)
    • Evidence: inline hooks on GetAsyncKeyState/SetWindowsHookEx and dumped memory containing plaintext keystrokes.
• Cross‑view and corroboration
  • Compare pslist/psscan, netscan vs. disk artifacts (prefetch, event logs, MFT), module hashes vs. known good, YARA on dumped regions:

    volatility dumpfiles / vol.py windows.yarascan
    

    Evidence: absent disk file for running process, inconsistent timestamps, YARA matches.

False positive reduction strategies

• Cross‑view: require presence in both dynamic (pslist/netscan) and static scans (psscan/connscan) before flagging.
• Correlate to disk: check PE on disk, prefetch, Registry Run keys, Scheduled Tasks, and Event Logs.
• Behavioral signs: RWX pages + suspicious API hooks + network connections + persistence entries = high confidence.
• Hash and reputation checks for dumped binaries; timeline analysis to align with incident.

Notes
Document every step, keep recovered artifacts and hashes, and produce an evidence chain for reporting or legal use.

Situation & goal (brief)
Detect an in‑memory Windows rootkit that hides processes and sockets, collect forensic evidence without contaminating the host, and produce analysis-ready artifacts for reporting.

Detection approach (cross‑view techniques)

• Compare multiple views of the same artifact: userland API / tool output vs. raw kernel memory.
  • Userland: Task Manager, WMI (Win32_Process), ToolHelp / CreateToolhelp32Snapshot, GetExtendedTcpTable.
  • Kernel/raw: PsActiveProcessHead traversal (pslist), psscan / signature scan for EPROCESS, PsGetNextProcess, kernel TCP endpoint structures and ARP/TCP control blocks.
• Use cross‑view tools: Volatility/Volatility3, Rekall, and Sysinternals + PsXview to highlight discrepancies (process visible in psscan but absent in pslist/API).

Kernel structures & hooks to inspect

• Processes: EPROCESS structure fields (UniqueProcessId, ImageFileName, ActiveProcessLinks), PsActiveProcessHead list integrity (check forward/backward links), Token/Job/HandleTable.
• Threads: ETHREAD, ThreadListHead vs. user threads.
• Drivers/hooks: SSDT and Shadow SSDT entries, Inline hooks in ntoskrnl/Win32k, IRP MajorFunction table in DRIVER_OBJECT.
• Network: TCP/IP stack structures (TCP_ENDPOINT/TCB equivalents), AFD/TCB related FILE_OBJECTs, TCP connection tables (kernel MIB), and NDIS/filter drivers for packet hiding.
• Objects: Object namespace inconsistencies (ObReferenceObjectByHandle), hidden device objects (DEVICE_OBJECT), hidden registry keys (KCB).

Safe evidence extraction

• Triage first: isolate host (network disconnected), photograph console, record volatile state.
• Acquire a memory image with a trusted, forensically sound tool (DumpIt, FTK Imager – live RAM capture, Magnet ACQUIRE), or use WinDbg kernel memory dump via .dump /f for kernel dumps. Prefer tools signed and validated; record tool versions and hashes.
• If live dumping from compromised host is too risky, take a full disk image and perform offline live analysis from a trusted forensic workstation using a memory acquisition appliance if available.
• Export suspicious structures: use Volatility to extract EPROCESS, memory ranges for kernel modules, and raw socket/TCB memory to standalone files. Save associated metadata (offsets, timestamps, parent PID).
• Preserve chain of custody: compute SHA256/MD5 of images and extracted files, log commands, operators, and time. Work on copies only.

Analysis & reporting

• Show cross‑view evidence: e.g., process present in psscan with EPROCESS at 0x..., missing from ActiveProcessLinks and from WMI; matching memory pages contain injected code or inline SSDT hooks.
• Demonstrate rootkit persistence artifacts: unsigned drivers, altered driver signing checks, scheduled components.
• Provide timelines, artifacts’ hashes, and recommended containment/remediation (unplug, patch, rebuild from known good images).

Notes & precautions

• Avoid running heavy tools on suspect host that could trigger rootkit anti‑forensic behavior; prefer read‑only memory acquisition and offline analysis.
• Correlate findings with network captures and logs to strengthen attribution and reporting.

Approach (one‑sentence):
I would build a multi-signal, scored detector that combines in‑memory PE verification, protection & VAD anomalies, PEB/module cross‑view, and cross‑process/page table checks to minimize false positives while retaining high recall for process‑hollowing and in‑memory injection on x86/x64 Windows.

Input signals

• PE headers in memory: image base vs. in‑memory DOS/NT headers, mismatched SizeOfImage/Sections, absent certificate directory.
• Page protections: RX pages that are RW then RX (protection flip), non‑image sections with RX.
• PEB module list vs. kernel view: modules present in PEB but missing from kernel LDR list or vice versa (cross‑view inconsistency).
• VAD mappings: VAD owner, protection attributes, large anonymous RX VADs, executable stacks.
• PTE/PFN checks: physical backing (pagefile vs. image mapped), copy‑on‑write differences.
• Temporal signals: fast write+protect sequences, CreateRemoteThread/QueueUserAPC sequences.
• Process metadata: parent relationship, signing of mapped image, known benign injection tool whitelist.

Scoring heuristics

• Assign weighted scores: critical anomalies (suspicious RX anonymous VAD, header absent) = 40, protection flips = 25, cross‑view inconsistency = 30, temporal evidence = 20, weak indicators (unusual parent, unsigned module) = 10.
• Aggregate score thresholding with adaptive thresholds per host baselining (e.g., dev vs. server).
• Suppression rules: known legitimate loaders (DLL‑sideloaders, JIT compilers) with verified signatures and behavior profiles reduce score.
• Correlation: require >=2 high‑confidence and one supporting temporal/PEB signal to alert.

Evaluation

• Construct labeled corpus: clean enterprise traces, benign injection tools (debuggers, profilers), and multiple attack samples (process‑hollowing, reflation, reflective DLL, shellcode loaders) on x86/x64 across Windows versions.
• Metrics: ROC, precision/recall, FPR at target TP (e.g., measure FPR at 95% TPR).
• Red team validation: run A/B tests in live environments, measure alerts, perform triage to estimate FP triage cost.
• Continuous tuning: analyze FPs to add heuristics/whitelists; measure FN by synthetic and mutated payloads.

Forensics output

• Produce reproducible artifact report: memory offsets, header snapshots, VAD map dump, PTE/PFN evidence, timeline of protection changes, and mapping to suspect thread handles for court‑grade evidence.

High-level goal
Design a Volatility plugin that scans process memory for in-memory C2 beaconing indicators (timers/sleeps, repeated DNS/socket usage, TLS artifacts) and outputs scored, forensically defensible alerts with evidence snippets and provenance.

Required telemetry

• Sleep/timer patterns: detect repeated WaitForSingleObject/Sleep calls or timer queue objects with consistent intervals; extract interval histogram per thread.
• Network handles: list socket handles, associated IP/port strings in memory, repeated connect() sequences, and FD bookkeeping structures.
• DNS activity in memory: repeated hostname strings, resolver cache entries, or encoded domain patterns (DGAs) in memory pools.
• TLS metadata: in-process TLS session structures, SNI strings, certificate subjects/serials cached in memory, session ticket blobs.
• Temporal correlation: timestamps from process memory/registry and image capture to build sequence of beacon events.

Heuristic scoring

• Weighted factors: interval regularity (0–30), DNS repetition/entropy (0–25), socket/connect churn (0–20), TLS anomalies (self-signed/mismatched SNI, 0–15), code patterns (beacon function signatures, 0–10).
• Score = sum(weights × normalized feature). Thresholds: >=70 high-confidence, 40–69 medium, <40 low.
• Produce per-evidence provenance and explainability (which memory offsets/strings produced score).

Performance constraints

• Prefer scanning live process address space ranges selectively (heap, stacks, .rdata/.data) not full image.
• Use streaming parsing and regex/state machines; limit regex backtracking; parallelize per-process up to CPU count; memory-bounded buffers.
• Timeout per-process (configurable, default 5s) and sampling mode for large images.

Evasion considerations

• Anti-forensics: encrypted or obfuscated strings — include entropy checks, near-entropy-decoding attempts (XOR, ROL), and YARA rules for common obfuscators.
• Variable/randomized intervals: detect statistical periodicity (autocorrelation) rather than strict fixed sleep.
• Native API whitelisting: maintain benign-process behavioral baseline to reduce false positives.
• TLS in userland vs kernel: account for in-memory TLS proxies and split stacks; correlate with network artifacts on disk (pcap) if available.

As an examiner I’d ensure outputs are evidence-grade: stable offsets, extracted bytes, and reproducible command-line options for courtroom use.

High-level approach

• Enumerate processes and loaded modules.
• For each imported/exported function, resolve runtime target address.
• Compare runtime bytes at target against on-disk module bytes at the same RVA.
• Flag mismatches as potential inline hooks; apply heuristics to suppress false positives (hotpatch, JIT, runtime trampolines).

Data structures

• ModuleInfo { pid, base_va, image_bytes_hash, export_table, import_table }
• HookCandidate { pid, module, func_name, iat_rva, runtime_va, on_disk_bytes, runtime_bytes, score }
• Cache: Map[module_path -> on-disk image bytes + parsed EAT]

Pseudo-code (core loop)

for pid in enumerate_processes():
  for module in list_modules(pid):
    disk = load_on_disk_image(module.path)
    exports = parse_exports(disk)
    imports = parse_imports(module.memory)
    for imp in imports:
      runtime_va = read_pointer(pid, module.base_va + imp.iat_rva)
      if not in_module_range(runtime_va, module):
        continue
      on_disk_addr = rva_to_file_offset(disk, runtime_va - module.base_va)
      on_disk_prolog = disk.read(on_disk_addr, PROLOGUE_LEN)
      runtime_prolog = read_memory(pid, runtime_va, PROLOGUE_LEN)
      if runtime_prolog != on_disk_prolog:
        candidate = HookCandidate(...)
        score = score_candidate(candidate)
        if score >= THRESHOLD:
          report(candidate)

Heuristics to reduce false positives

• Hotpatching: detect 5-byte stub pattern (push ebp / jmp rel32 at module base +/- 0x??) and check IMAGE_DIRECTORY_ENTRY_LOAD_CONFIG / hotpatch flag; suppress if pattern matches.
• Legit JIT/webassembly/runtime: whitelist processes (w3wp, java, node, python, dotnet, v8) or detect executable memory regions with RWX and nearby allocation timestamps.
• ASLR/forwarded exports: resolve forwarded names via EAT and compare to correct module.
• Short prologues: require N bytes (e.g., 8) and a minimum entropy check; require consistent trampoline sequences (jmp->detour module) before flagging.
• Multiple-hit correlation: raise priority if same detour target used across many processes/modules.

Complexity & reliability

• Time: O(P * M * I) where P=processes, M=modules per process, I=imports per module; optimized by caching on-disk parsing and early skips.
• Limitations: packed modules, runtime-modified on-disk images, kernel hooks not covered. Validate findings with full disassembly and cross-process correlation before reporting as evidence.