Client-Side and Mobile Data Persistence Questions

Overview
Below are primary client-side storage options on iOS and Android with use cases, durability, performance, size and security trade‑offs, plus decision criteria.

UserDefaults / SharedPreferences

• Typical use: small app settings, flags, lightweight preferences.
• Durability: persisted to disk; survives restarts and backups (may be cleared on uninstall).
• Performance: very fast for small payloads; synchronous API can block UI if abused.
• Size: intended for < few KB; not for large blobs.
• Security: plaintext; use keychain/keystore for secrets.

NSCache / In-memory cache

• Typical use: transient in-memory caches (images, computed data).
• Durability: not durable; can be evicted anytime (memory pressure).
• Performance: fastest (RAM).
• Size: limited by memory and system eviction.
• Security: in-memory only; still accessible if device compromised.

File storage (Documents/Files / internal storage)

• Typical use: larger blobs, downloaded files, logs, exported data.
• Durability: persistent, included/excluded from backups depending on location.
• Performance: depends on file size and I/O patterns; use streaming for large files.
• Size: limited by device storage; consider quotas.
• Security: plaintext unless encrypted; use file protection APIs (iOS) or encrypted file APIs.

SQLite / Core Data / Room

• Typical use: structured relational data, complex queries, indexing.
• Durability: durable on disk, transactional.
• Performance: good for structured queries; indexing improves reads; writes can block without background threading.
• Size: scales to many MBs/GBs as device permits.
• Security: plaintext DB by default; can use SQLCipher or platform file encryption.

Core Data (Apple) specifics

• Higher-level ORM, object graph, change tracking; good for complex models and performance tuning.

Realm

• Typical use: reactive, object-oriented local DB with simpler APIs.
• Durability: durable, ACID.
• Performance: high read/write performance, built for mobile.
• Size: similar to SQLite; depends on data.
• Security: supports encryption; must manage keys securely.

Keychain / Keystore

• Typical use: small secrets, tokens, credentials.
• Durability: persistent and survives app reinstalls if properly configured; protected by device security.
• Performance: slower (secure enclave access), designed for small items.
• Size: small items (credentials, keys).
• Security: highest on device; hardware-backed where available.

Decision criteria

• Secrets → Keychain/Keystore.
• Tiny config/flags → UserDefaults/SharedPreferences.
• Transient cache → NSCache / in-memory.
• Structured relational data/complex queries → SQLite/Room/Core Data.
• Object-oriented, reactive DB → Realm.
• Large files/blobs → File storage (consider streaming, encryption).
• Need encryption/compliance → use encryption at rest (SQLCipher/File encryption) + secure key storage in Keychain/Keystore.
• Consider performance: prefer in-memory for hot reads, asynchronous DB I/O for writes, index DBs for queries.

Choose by data size, access patterns (read-heavy vs write-heavy), durability needs, security/compliance, and complexity of queries/relationships.

Overview / Goals
I’d implement an append-only client oplog with per-record metadata, deterministic conflict detection, and simple automatic rules (LWV for non-ambiguous) plus a user-facing merge UI for complex conflicts.

Data model

• Task { id: UUID, title: string, notes: string, dueDate: ISO8601, completed: bool, tags: [string], lastModifiedAt: timestamp, lastModifiedBy: clientId, tombstone: bool }
• Server stores authoritative task state + per-task version vector or lastWriteTimestamp.

Client oplog (queue) format
Each operation appended:
{ opId: UUID, opType: CREATE|UPDATE|DELETE, taskId: UUID, payload: {fields changed}, prevVersion: versionToken, timestamp, clientId }

• Persisted locally (SQLite/Realm) until ack by server.
• Retry/safe idempotency using opId.

Sync flow

1. On network: push local oplog entries to server in order.
2. Server validates op.prevVersion; if matches, apply and return new version token.
3. If mismatch, server detects conflict and returns conflicting remote state + merge hints.
4. Client pulls server state, reconciles, and either replays ops or prompts user.

Conflict detection & resolution

• Detect when op.prevVersion != currentVersion OR overlapping updates to same field.
• Automatic rules:
  • If non-overlapping fields changed → merge fields.
  • For single-field concurrent edits → Last-Write-Wins using timestamp with clientId tie-breaker.
  • For deletes vs edits → delete wins if delete timestamp > edit timestamp; otherwise revive with prompt.
• Use vector clock or simple monotonically increasing version token from server for strong ordering.

User conflict presentation

• If automatic merge ambiguous (e.g., title edited differently on two devices), present a compact 2-column merge screen:
  • Show both versions, highlight differing fields, allow user to pick or edit combined result.
  • Offer “Keep mine”, “Keep theirs”, or “Merge manually”.
  • Show timestamps and device names to help decision.
• After user resolves, push a new resolved op to oplog and sync.

Mobile considerations

• Keep oplog in lightweight local DB, compact old ops after ack.
• Minimize UX friction: resolve most conflicts automatically, surface only when meaningful.
• Secure ops with TLS and authenticate clientId; encrypt local DB if needed.
• Test with airplane-mode scenarios and simultaneous edits on emulator/device farms.

Approach (brief)

• Apply optimistic update: update local model & UI immediately, mark task as pending, enqueue network request, show spinner/ghost state. On success clear pending; on error/conflict rollback and surface error.

Code (Swift — simplified)

struct Task { var id: String; var completed: Bool; var pending: Bool = false }

class TaskStore {
  var tasks: [String:Task] = [:]
  let queue = OperationQueue() // or custom serial queue for requests
  let network = NetworkClient()

  func toggleCompleted(taskId: String) {
    guard var task = tasks[taskId] else { return }

    // 1. Optimistic update: flip locally and mark pending
    let newCompleted = !task.completed
    task.completed = newCompleted
    task.pending = true
    tasks[taskId] = task
    notifyUI()

    // 2. Enqueue network operation (supports offline via persist)
    let op = BlockOperation {
      let request = ToggleRequest(id: taskId, completed: newCompleted)
      let result = self.network.send(request) // synchronous for clarity
      DispatchQueue.main.async {
        switch result {
        case .success(let serverTask):
          var updated = serverTask
          updated.pending = false
          self.tasks[taskId] = updated
          self.notifyUI()
        case .failure(let error):
          // 3. Rollback on error/conflict
          var original = self.tasks[taskId] ?? task
          original.completed.toggle() // revert
          original.pending = false
          self.tasks[taskId] = original
          self.notifyUI()
          self.handleError(error)
        }
      }
    }

    persistOperation(op, for: taskId) // for offline/replay
    queue.addOperation(op)
  }

  // helpers omitted: notifyUI(), persistOperation(_:for:), handleError(_:)
}

Key concepts & reasoning

• Marking pending (task.pending = true) lets UI show a subtle spinner/ghost state; keeps user-informed.
• Persisting the queued operation (e.g., CoreData, Realm, or a file) allows offline queuing and replay when connectivity returns.
• Serializing operations per-task avoids race conditions; merge or dedupe subsequent toggles before sending.
• Conflict handling: on 409 or validation error fetch authoritative server state, present conflict UI if needed, or auto-merge where safe.

Edge cases & considerations

• Multiple toggles quickly: collapse to last state before sending, or send sequential ops with idempotency tokens.
• Idempotency: include client-generated operation IDs so retries are safe.
• Consistency: eventual consistency acceptable; provide pull-to-refresh to fetch canonical state.
• Security: authenticate requests and protect persisted queue.

Complexity

• UI/state ops are O(1). Network/offline persistence cost depends on storage; ensure background retry/backoff logic.

Test plan summary

• Goal: verify app behavior offline, graceful sync, conflict resolution, data integrity, and recovery from corruption under varied network conditions and multi-device scenarios.
• Phases: unit + local integration, device/emulator automation, multi-device sync tests, chaos/flaky-network tests, CI gating and post-deploy monitoring.

Automation approach & tools

• iOS: XCTest / XCUITest for UI, XCTest for unit; Network Link Conditioner profiles; Charles Proxy + Map Local for scripted responses; Simulator control via simctl for launch/config.
• Android: Espresso for UI, Robolectric for unit; Android Emulator network shaping (adb shell network delay/packet loss) and tc/netem for advanced conditions; Charles or mitmproxy.
• Cross-platform: Appium or Detox for higher-level flows; Toxiproxy to inject failures between app and backend; WireMock for deterministic backend stubs.

Simulating network conditions

• Deterministic: use Network Link Conditioner (iOS) and emulator tc/netem (Android) with profiles: offline, 2G/3G/4G, 100ms–5000ms latency, packet loss 0–10%, bandwidth caps.
• Flaky/chaos: run randomized schedules with Toxiproxy or scripted Charles scenarios; run long-duration intermittent toggle tests.

Multi-device sync scenarios

• Orchestrate multiple emulators/devices in CI (Firebase Test Lab / AWS Device Farm) or local device lab.
• Create test harness that: seeds device A -> perform offline edits on B -> reconnect A then B -> assert conflict resolution rules and eventual consistency.
• Use deterministic timestamps/mocks for conflict reproducibility.

Datastore corruption

• Inject corrupted DB files: for SQLite use malformed WAL/PRAGMA integrity_check; for Realm replace DB file with truncated bytes; verify app detects corruption, backs up/logs, and recovers (migrate/rebuild) without crashes.
• Include unit tests that mock storage layer to simulate IOExceptions.

CI integration

• Run unit + emulator smoke on PRs (GitHub Actions / Bitrise); run full matrix (network profiles, multi-device sync) nightly on device farm.
• Use Fastlane to orchestrate builds/tests, collect logs/artifacts, and upload flaky test reports.
• Add retry policy for flaky tests, quarantine flaky cases, and alert on regression thresholds.

Validation & metrics

• Assert UX messages, retry/backoff behavior, data integrity, sync convergence within SLA.
• Track flaky test rate, average sync time, recovery success rate.

This plan provides repeatable automation, deterministic simulation, and CI gating to ensure robust offline and sync behavior on iOS and Android.

Approach (high level)
I persist secrets using platform-provided secure storage, protect keys with hardware where available, and avoid storing long-lived tokens unprotected.

iOS

• Use Keychain for tokens/keys (kSecAttrAccessibleWhenUnlocked or WhenPasscodeSetThisDeviceOnly).
• Use Secure Enclave for private keys (generate with kSecAttrTokenIDSecureEnclave) so keys never leave hardware.

Android

• Use Android Keystore to generate asymmetric/symmetric keys (hardware-backed when available).
• For simple prefs, wrap data with EncryptedSharedPreferences or EncryptedFile (uses AES-GCM with KEK from Keystore).

Key derivation & storage best practices

• Derive symmetric keys via HKDF or PBKDF2 with salt and adequate iterations if deriving from user secret.
• Prefer generated random keys stored in Keystore/Keychain; avoid raw secrets in app storage.
• Limit token lifetime, use refresh tokens stored securely and rotate keys.

Backups & restore

• Mark sensitive Keychain items as non-backup when required (ThisDeviceOnly) on iOS.
• On Android, use BackupAgent cautiously; prefer requiring re-authentication to re-provision secrets after restore.
• For Secure Enclave / hardware-backed keys, handle key loss on device change by using server-side key escrow or re-issuance.

Other considerations

• Enforce biometric/credential gating for high-value operations.
• Log nothing sensitive.
• Test on devices without hardware crypto and handle fallbacks.