IoT and Edge Device System Architecture Questions

Requirements (clarify): reliable telemetry ingestion despite intermittent connectivity; exactly-once or at-least-once semantics with dedup/out-of-order handling; secure OTA updates; minimize egress by balancing edge/cloud; scalable, observable, and testable in-field.

High-level architecture:

• Edge device (device agent + local runtime) ↔ intermittent network ↔ Cloud Ingress (API Gateway / Message Broker) → Stream Processor → Time-series DB / Object Store → Control Plane (OTA manager, device registry, auth).
• Optional Edge Gateway for clusters of devices with local buffering and stronger compute.

Core components & responsibilities:

• Device agent: local durable queue (SQLite/leveldb) for telemetry + sequence numbers / logical timestamps + signing of payloads; retry/backoff, exponential, network-aware; local processing/aggregation (summaries, compression) to reduce egress.
• Gateway/broker: MQTT broker (TLS + client certs) or HTTP+MQTT hybrid with message retention. Use MQTT QoS 1 and 2 where supported.
• Cloud ingress: API Gateway + message queue (e.g., Kinesis, Pub/Sub, IoT Hub) for durable, ordered ingestion per device key.
• Stream processor: idempotent consumer using dedup store (Redis with TTL or cloud-built dedup) keyed by deviceID+sequence; reordering window with watermarking to reorder within tolerance.
• Storage: append-only time-series DB (InfluxDB/TSDB or Bigtable) and cold object store for raw batches.
• OTA manager: signed firmware artifacts (code-signing), A/B update strategy, staged rollout, rollback capability; mutual TLS + device attestation (TPM or secure element) to verify identity before applying update.
• Security: PKI for device identity, mTLS, end-to-end encryption, signed telemetry if needed.

Edge vs Cloud processing trade-offs:

• Push event-level filtering/aggregation to edge to reduce egress (aggregate per-minute metrics, compress, delta encoding).
• Keep heavy ML models either as lightweight edge models for inference or in cloud for training; use model hot-swapping via OTA.
• Use gateways to centralize devices with constrained radios.

Handling duplicates/out-of-order:

• Device adds monotonic sequence numbers + per-message UUIDs; cloud dedup uses idempotent checks (persist processed sequence per device with sliding window).
• Reordering: buffering in stream processor for configurable window (e.g., 2 minutes), then apply watermark and emit; provide metrics for late-arriving messages.

Reliability & testing in field:

• Simulate intermittent links in lab: network partitioning, high-latency, packet loss; fuzz message order/dup; chaos testing on gateway and cloud.
• Field validation: staged pilot with telemetry shadow mode (device sends both raw and aggregated); compare lost vs received counts; inject controlled OTA to small cohort using canary rollouts; measure end-to-end delivery SLA, duplicate rate, reorder rate.
• Observability: device-side logs, heartbeat metrics, cloud telemetry metrics, alerting on queue growth, percent late arrivals, and OTA failures.
• Runback tests: induce firmware failure on canary and verify automatic rollback.

Trade-offs & scalability:

• Dedup store cost vs accuracy: per-device stateful checks scale with devices—use sharded TTL stores or CRDTs for wide scale.
• Ordering window increases latency — tune per application.
• Use serverless ingestion for burst elasticity.

This design provides durable ingest, idempotency, secure OTA, and configurable edge/cloud balance to minimize egress while preserving reliability and testability.

Requirements & constraints:

• Hard requirement: end-to-end inspection feedback <50 ms from camera exposure to actuation signal.
• Throughput: up to N cameras per line (example 4–8), 60–120 FPS per camera possible.
• Environment: factory floor, private 5G (or campus 5G), intermittent WAN.

High-level architecture:

• Cameras → Local Edge Node (per cell/line) → Local Decision Router → Aggregation Edge / MEC → Cloud (for sync, model training, analytics)
• Control loop: Camera frames → preproc → model inferencing on Edge Node → decision logic → PLC/robot actuator (actuation <=50ms)

Edge Node design:

• Hardware: industrial GPU/TPU-enabled appliance (e.g., NVIDIA Jetson AGX Orin or Ampere eMAG + NPU), dual NIC (5G modem + wired), ECC storage, TPM.
• Software stack: containerized inference runtime (TensorRT/ONNX Runtime), real-time OS or low-latency Linux kernel, lightweight orchestration (k3s), local message bus (NATS/Redis Streams) for microsecond messaging.
• Local storage: ring buffer for last 30–60s of raw frames; metadata store (SQLite or RocksDB).

Latency budget (example):

• Camera exposure + transfer to edge (over wired/5G): 5–10 ms
• Preprocessing: 2–5 ms
• Inference: 5–15 ms (optimized model)
• Decision logic & actuation signaling: 1–5 ms
• Margin for jitter: <=10 ms
  Total target: <=50 ms

Model inferencing at edge:

• Use quantized, pruned models (INT8 / mixed precision) exported to TensorRT/ONNX
• Batch size = 1, pipelined async inference
• Warm model resident in memory; model hot-swap via atomic update mechanism
• Fallback to lightweight rule-based detection when model update fails

Data synchronization with cloud:

• Two-tier sync:
  • Metadata & metadata-derived results: streamed in near-real-time (kafka/MQTT over TLS) to cloud for analytics, dashboards.
  • Raw frames: only sent on-demand (exceptions/anomalies), sampled (1% or triggered by anomaly), or batched during low-load windows via secure uplink.
• Model lifecycle: cloud trains models using aggregated labeled data; model versioning (MLflow/Model Registry); CI/CD to push signed model artifacts to edge nodes with A/B testing and rollback.

Data reduction strategies:

• Edge-side ROI cropping, compression (JPEG2000 or H.264 hardware encode), delta-frame detection (send only changed frames), event-driven upload (only anomalies), feature-level export (send embeddings instead of raw images).
• Use adaptive sampling based on production state (full capture during initial ramp, aggressive reduction in steady-state).

Security boundaries:

• Zero-trust within plant: mutual TLS between camera → edge → cloud, device identity via TPM and X.509 certs.
• Network segmentation: camera VLAN, edge control VLAN, OTA/management VLAN; firewall rules restrict ports to control plane and telemetry only.
• Secure boot, disk encryption, signed model artifacts, runtime attestation, role-based access control for ops.
• Audit logging locally and in cloud; SIEM integration; OT–IT DMZ when integrating with PLCs.

Failure modes & mitigations:

• Edge node failure: cold standby node on same cell; rapid failover via HA router; last-known-good decision policy in PLC for safe state.
• 5G link degradation: fall back to wired Ethernet or reduce upload rates to preserve control loop; local-only operation continues.
• Model corruption: health checks and signatures; automatic rollback to previous stable model.
• Power failure: UPS for edge node to flush state to persistent storage; safe-stop signals to actuators.
• High inference latency: auto-scale additional edge nodes or reduce resolution/batch to meet SLAs.

Pilot plan (12 weeks realistic):

• Week 0–2: Discovery & design — validate camera types, 5G coverage test, latency measurements; define KPIs.
• Week 3–6: Build & deploy pilot cell — deploy 1 edge node, 2 cameras, integrate with PLC in shadow mode (no actuation).
• Week 7–8: Model tuning & performance optimization — iterate model quantization, latency tuning, test <50ms path.
• Week 9–10: Controlled go-live — enable closed-loop actuation on non-critical line; run for 2 weeks.
• Week 11–12: Scale & handoff — deploy runbook, monitoring dashboards, SOPs, security review, roadmap for fleet rollout.

Success metrics:

• Primary: 99th percentile closed-loop latency <=50 ms.
• Accuracy: detection precision/recall within agreed thresholds (e.g., precision >=98%, recall >=95%).
• Reliability: edge node uptime >=99.9% during pilot.
• Data reduction: >90% reduction in raw-frame bandwidth to cloud vs naive streaming.
• MTTD/MTTR for failures: MTTD <5 min, MTTR <30 min.
• Business: defect catch rate increase and reduction in false rejects by X% (as agreed with client).

Trade-offs:

• Pushing all inference to edge reduces latency and bandwidth but increases edge ops complexity and fleet management.
• Model size vs accuracy: pruning/quantization reduces latency but requires careful validation.

Deliverables for client:

• Detailed architecture diagram, latency budget, pilot test plan, security controls checklist, cost estimate (edge hardware, private 5G, ops), and rollout & support plan.