Embedded Systems Architecture Questions

Approach summary
Design around a dedicated hardware timer driven by a stable clock, keep the 100 μs loop triggered in hardware with minimal ISR work, and move non-deterministic work to lower-priority tasks or DMA.

Clock sources

• Use a low-drift crystal/PLL (e.g., HSE + PLL) for MCU core and timer. If absolute long-term accuracy needed, discipline with external TCXO or GPS.
• Avoid internal RC for the control loop; it adds ppm jitter.

Hardware timers

• Use a 32-bit general-purpose timer in compare mode or a hardware PWM/timer interrupt that can generate an interrupt every 100 μs.
• Where possible use timer in “one-shot” reload or master mode to trigger ADC via hardware TRGO and use DMA to move samples to memory—this removes software latency from sensing.

Interrupt priorities & preemption

• Assign the 100 μs timer ISR highest deterministic priority. Keep ISR very short: acknowledge timer, snapshot data pointers, kick a control task (e.g., set flag or use RTOS semaphore).
• If running bare-metal: handle full control step inside ISR only if WCET ≤ budget and nesting is controlled. Otherwise, ISR should be minimal and schedule a non-preemptive control context.
• Use priority grouping so only the timer and critical fault handlers can preempt the control path; mask lower priority IRQs during the deterministic window if necessary.

Kernel vs bare-metal

• Prefer a small preemptive RTOS with priority-based scheduler (e.g., FreeRTOS) if you need threads and drivers. Use static priorities and avoid priority inheritance surprises.
• For the tightest jitter (≤5 μs) a bare-metal or tickless RTOS approach is often safer: use hardware timer as source-of-truth and avoid periodic kernel tick interference. If using RTOS, make it tickless or ensure kernel ticks don’t coincide with loop.

CPU budgeting

• Compute WCET for ISR and control function (measure on target). Budget: keep control path ≤ 30–40% of 100 μs ideally (30–40 μs) leaving headroom for interrupts and jitter. Reserve slack and a low-priority background for logging/comms. Use DMA and hardware peripherals to offload work.

Verification & testing

• Instrument: use GPIO toggle at loop start/end and measure with high-bandwidth oscilloscope or logic analyzer to capture period and jitter distribution.
• Use hardware trace (ETM/SWO/ITM) and cycle counters to measure latency and WCET. Run synthetic worst-case loads (enable all interrupts, DMA bursts, bus contention).
• Metrics: report min/mean/max/stddev and worst-case latency; ensure max jitter < 5 μs with margin.
• Add watchdog and diagnostic counters; run long soak tests and thermal/voltage variation tests.

Trade-offs

• Full in-ISR control minimizes scheduling jitter but reduces maintainability and complicates debugging. RTOS increases structure but must be configured (tickless, priorities) to preserve determinism.

This architecture yields deterministic 100 μs loops with controllable jitter by using stable clocks, hardware-triggered sensing, minimal high-priority ISRs, measured WCET budgeting, and rigorous lab verification.

Design summary (constraints)
Target: constrained MCU, no external storage, minimal RAM, signed images, atomic recoverable updates, controlled rollback.

Partitioning

• Bootloader (fixed, protected flash region)
• Primary slot (A) — active firmware image
• Secondary slot (B) — incoming update image (same size as A)
• Metadata sector (small, wear‑leveled): stores image headers, sequence numbers, flags, and public-key ID

Image format & header (stored at B start)

• Magic, version, length, sequence number, flags, SHA256 (or SHA512) of image, RSA/ECDSA signature over header+hash, CRC32 for quick sanity check

Verification flow (minimal RAM)

1. On reset, bootloader reads metadata header from A and B.
2. If B has valid header with higher sequence number and flag “candidate”, bootloader streams B to compute hash (small buffer, e.g., 4 KB) and verifies signature using public key in bootloader (validate header+hash). Signature verification can be done with incremental hash API to limit RAM.
3. If signature & hash OK, mark B as “verified”. Swap pointers logically: update metadata to mark B as “active‑pending” and boot into B once. Use single-word atomic writes for flags.
4. On successful boot, application calls bootloader to set B “active” (clear A). If boot fails or watchdog resets before confirming, bootloader will roll back.

Flags & metadata

• slot_state: EMPTY / CANDIDATE / VERIFIED / ACTIVE_PENDING / ACTIVE / INVALID
• seq_num: monotonic update counter to prevent replay
• rollback_allowed: boolean per-image controlled via signature policy or config

Failure & recovery handling

• Interrupted write: B remains CANDIDATE; bootloader ignores incomplete images (invalid CRC/hash).
• Failed verification: mark B INVALID, keep running A.
• Failed boot of ACTIVE_PENDING: on next boot, if ACTIVE_PENDING not confirmed and A is previous ACTIVE with higher seq, bootloader restores previous ACTIVE (no flash copy; use slot roles). If both fail, bootloader enters recovery mode (serial / DFU) with minimal resources.
• Controlled rollback: only allow rollback if rollback_allowed set and seq_num policy satisfied; otherwise reject older images.

Wear & atomicity

• Use small metadata sector with circular wear‑leveling. Use atomic single‑word writes for flags; protect against power loss using sequence numbers and redundant copies.

Security notes

• Public key burned into bootloader or OTP. Protect metadata integrity with signature over header and hash. Use anti‑rollback via monotonic seq and optional secure monotonic counter if available.

This approach guarantees safe OTA with minimal RAM by streaming verification, recovers from interrupted updates, supports cryptographic verification and controlled rollbacks while avoiding in‑place destructive updates.

Approach summary
Make firmware testable by separating hardware access behind small, mockable interfaces, exercising logic on the host with unit tests, and validating integration with QEMU or hardware-in-the-loop (HIL) in CI.

Dependency injection for peripherals

• Define a HAL interface (function pointers/struct) for each peripheral.
• Inject the HAL into modules rather than calling registers directly.

Example:

// hal_uart.h
typedef struct {
  int  (*init)(void);
  int  (*write)(const uint8_t *buf, size_t len);
} uart_hal_t;

// module.c receives uart_hal_t* in init

Mocking / test harness strategies

• Provide fake HAL implementations for unit tests that record calls and simulate faults.
• Use CMock or hand-written mocks for C; for C++ use gmock.
• Write a small test harness that sets peripheral state deterministically (timers, ADC values).

Host-based unit tests

• Move most logic into platform-independent files and compile them on host.
• Use Unity + CMock or GoogleTest for assertions and mocks.
• Keep hardware-specific code confined to thin wrappers that are trivial to mock.

Integration: QEMU & HIL

• Use QEMU/peripheral models to run integration tests for device drivers and boot sequences.
• For final validation, run automated HIL tests that exercise real peripherals; run smoke tests in CI gating.

Trade-offs & best practices

• Keep HAL minimal and pure C for testability.
• Balance mocking vs. real hardware coverage; rely on host tests for fast feedback and HIL/QEMU for confidence before release.

Overview — scope & goal
I’d choose between a lightweight filesystem (example: littlefs) and a separate wear‑leveling layer based on device constraints: flash geometry, RAM budget, frequency of writes, and need for strong power‑loss guarantees.

Approaches compared

• littlefs (filesystem + wear‑leveling + power‑loss safety)
  • Strengths: built-in atomic metadata, copy‑on‑write, CRCs, designed for small flash; strong resilience to sudden power loss; minimal integration (single dependency).
  • Costs: CPU and flash write amplification from metadata updates; RAM for inodes/cache (typical tens to a few hundred bytes per open file/block); code size moderate.
• Dedicated wear‑leveling layer (block-level)
  • Strengths: smaller API footprint, can be extremely RAM‑light if only remapping blocks; lower per‑operation overhead if FS above is optimized for sequential writes.
  • Costs: you still need a filesystem above it for metadata; achieving power‑loss atomicity is complex; GC and translation tables can require extra RAM or reserved flash.

Trade-offs

• Performance: wear‑leveling layer + simple contiguous FS can be faster for large sequential writes; littlefs incurs extra ops for metadata and atomic moves (higher write amplification).
• RAM usage: pure block wear‑leveler can be smallest. littlefs has predictable small buffers but costs increase with open file count and cache sizes.
• Power‑loss resilience: littlefs is designed for safe recovery; block-level wear‑levelers usually require additional journaling to match that guarantee.
• Integration complexity: littlefs is easiest to integrate on MCUs. Custom wear‑leveling demands careful GC, bad‑block handling, and testing across power‑fail scenarios.

Recommendation
For constrained MCUs with intermittent power and frequent small writes (config, logs), pick littlefs. For extreme RAM limits and very predictable write patterns, consider a minimal remapping layer plus a tiny append-only application protocol to avoid complex FS guarantees. Always reserve spare blocks for GC and run power‑fail tests on hardware.

Investigation plan (step‑by‑step)

1. Reproduce deterministically
   • Build instrumented firmware (extra logging around flash HAL, timestamps).
   • Add compile-time stack usage reports and link map; enable link-time optimization to preserve symbols.
2. Capture runtime traces
   • Use SWD/JTAG to enable live breakpoints and core registers on fault.
   • Attach a hardware logic analyzer / scope to flash control pins, flash CS/CLK lines and power rail to catch timing and power dips.
   • Enable ITM / UART trace for entry/exit of ISRs and flash calls; log with timestamps.
3. Fault analysis
   • When crash occurs, use core registers + stack dump to see return addresses and caller frames.
   • Reproduce with Watchdog disabled/short timeout to avoid masking fault.
4. Stress tests
   • Fuzz flash write sizes and alignments, concurrent DMA, and interrupt load; run under low battery / brownout conditions.
   • Run with CONFIG_FORTIFY/stack canaries and MPU to catch stack overflow/illegal writes.
5. Isolate subsystem
   • Stub out flash HAL with safe mock to see if corruption disappears.
   • Run flash writes from single thread with interrupts masked to test reentrancy.

Likely root causes

• Stack overflow (large local buffers or deep recursion) clobbering SP before flash call.
• ISR or high‑priority interrupt writing through bad pointer (races during flash unlock/state changes).
• Flash HAL non‑reentrant: concurrent calls from ISR + task or DMA side effects.
• Brownout/current sag during flash write causing MCU transient corruption.
• DMA or cache coherency issues corrupting stack in RAM region.

Short‑term mitigations

• Add stack canaries, increase task stack sizes for writer tasks, enable MPU region protection.
• Serialize flash access: grab a mutex, disallow flash writes from ISRs; defer to worker task.
• Add watchdog and robust bootloader with rollback/recovery and CRC of firmware.
• Disable DMA/interrupts around critical flash sequence if safe.

Long‑term fixes

• Enforce flash API safety: make HAL reentrant-safe or single-producer/consumer model.
• Static analysis for stack usage; automate stack-sizing tests; add runtime stack watermarking.
• Hardware: add brownout detector, decoupling capacitors, or increase power supply margins.
• Use transactional flash/atomic update + dual-bank firmware to prevent bricking.

I would prioritize reproducing with power and trace capture, add stack canaries + MPU to pinpoint corruption, then harden HAL and bootloader to prevent field bricking.