Skip to content

DVR, persistence, and replay

RF Sentry separates raw IQ and decoded observation evidence from presentation caches while keeping routine Live work bounded. DVR evidence remains subject to its explicit rolling retention policy.

IQ DVR tiers

flowchart TB
    IQ[Incoming IQ blocks] --> HOT[Hot RAM ring]
    HOT -->|age / pressure| OPFS[Warm OPFS segments]
    HOT --> LIVE[Realtime consumers]
    OPFS --> CACHE[Bounded replay cache]
    CACHE --> REPLAY[Replay ingress]
    LIVE --> MARKS[Event/time indexes]
    REPLAY --> MARKS

iq.ingress.worker owns the raw timeline before the realtime, spectrum, analysis, candidate, and decoder fan-out. The hot tier is optimized for immediate timeshift. Sealed older batches transfer to dvr.storage.worker in browser-local OPFS; ingress releases their payloads after successful storage. Replay prefetches a bounded neighborhood rather than hydrating the entire retained interval. The Pi neither stores this DVR nor continues the browser's decoding after its owning session closes.

Defaults are a 60-second Hot RAM Minimum, 1 GB Hot RAM Target, six-hour total retention, and 100 GB warm-storage budget, with warm storage enabled. These are policy inputs, not guaranteed browser capacity. Hot duration takes precedence over the nominal RAM target; there is no project-imposed RAM ceiling. When rebalance is possible, crossing or reducing the target moves raw IQ toward 90% of it to avoid per-frame RAM/disk churn. Warm IQ and waterfall sidecars share the configured disk budget and browser quota. Only small per-lane preferences enter localStorage.

Acquisition runs and replay isolation

Every raw frame retains its original sample counter, monotonic time, logical RF and tuner centers, rate/format, gain, PPM, capture options/flags, and applicable UTC anchor. Geometry, settings, sample-counter, and continuity changes form explicit acquisition runs. An OPFS batch may contain several such runs; its index preserves all boundaries. One tiny storage transaction per acquisition gap is unnecessary and would create overhead without improving provenance.

Historical frames re-enter the ordinary DSP and decoder paths. A presentation source epoch identifies live/history transitions and seeks, resets affected local stream state, and rejects stale worker output. Acquisition identity and presentation identity are distinct: seeking does not rewrite source samples. Detection time follows source time, not the speed of browser callbacks.

Pause/seek/playback changes presentation only. Hardware keeps receiving live; routing, physical center, sample rate, tuner gain and PPM stay read-only during historical review. A historical VFO may move only where its complete passband fits in the selected acquisition span. IQ playback is 1x source-time playback, not an implicit catch-up that relabels old samples as live.

The default protect-history policy protects the investigated interval under retention pressure and sacrifices newer live history when necessary. Optional protect-live instead returns presentation to live. Go Live preserves a continuous timeline when possible; it clears/restarts history only after newer history was deliberately sacrificed. Storage failures remain visible and degrade to bounded RAM rather than silently moving DVR work to the appliance.

See operator controls and the clock contract.

Waterfall visual history

The lane-local WaterfallTimelineStore is a derived visual index, not the authoritative IQ record. Rows carry capture time, acquisition RF center/rate, wide/fine kind and source-epoch context. Replaying the same captured row replaces its prior visual entry instead of appending a duplicate.

Hot rows stay in RAM. Exact configured older representations move through waterfall.storage.worker into an OPFS sidecar and load around the replay cursor. A presentation-source reset preserves the index; a full session release clears it. Pruning follows the oldest retained IQ time. Restoring historical rows uses the existing WebGPU, WebGL2 or Canvas2D renderer without remounting or blanking the waterfall. Arrival-ordered detection overlays reset and rebuild from replay analysis rather than projecting future live evidence into history.

Native Float32, every emitted bin is the default. It means the active display path's row, not a promise that all renderers emit the same FFT geometry: the GPU fine path currently emits a 32,760-bin capture row, while WASM fallback may emit the selected scientific FFT row. There is no implicit archive reduction or quantization and no insertion-time copy for the native row representation. Explicit per-lane options can peak-reduce new rows to 1/2, 1/4 or 1/8 resolution, or encode dB values at 16 or 8 bits. Older rows are not synchronously repacked or deleted when preferences change. These settings do not change raw IQ or the scientific FFT selection.

Report raw-IQ payload and visual-row payload separately, and expose combined nominal-memory overruns. JavaScript objects, worker/WASM heaps and renderer resources add further costs; the raw-IQ target is not a hidden visual-history ceiling. Header memory figures are partial accounting, not total browser RAM or physical VRAM; see client memory readouts for their inputs and exclusions.

DVR ledger and qualification

docs/roadmap/receiver-dsp-scanning.md Rolling IQ History and ADR docs/adr/browser-0008-client-iq-timeshift.md carry the intended contracts. Original IQ timeshift was operator-accepted on 2026-09-05. The later 2026-09-22 waterfall enhancement and 2026-09-24 RAM/OPFS amendment add current tiering details; the ledger still records the tiered candidate's live acceptance as open. Preserve that distinction: neither the older acceptance nor focused regressions close the newer gate.

Implementation anchors are src/transport/iq.ingress.worker.ts, src/timeshift/IqTimelineBuffer.ts, TieredIqTimeline.ts, dvrStorage.ts, waterfallStorage.ts, their storage workers, and src/dsp/WaterfallTimelineStore.ts. timeshiftBuffer.test.ts and waterfallTimelineStore.test.ts cover focused raw-IQ/visual-history behavior. DVR retention is not a permanent recorder: durable SigMF recording/export remains a separate future capability in this ledger.

Storage roles

Store Appropriate content
JavaScript/worker RAM compact live state, visible previews, active job buffers
WASM linear memory DSP/decoder working sets only
IndexedDB compact metadata, indexes, observations, preferences
OPFS bulk IQ segments, reconstructed files, native imagery, large history
localStorage small UI preferences only; never bulk binary data

Original evidence and derived accelerators are separated. A thumbnail, natural-color derivative, replay checkpoint, or latest-aircraft snapshot can be regenerated from retained inputs and provenance.

Reference time

Live reference time advances with the current capture. Historical mode uses an explicit cursor. Every result carries enough generation/range context to be discarded if it belongs to a superseded cursor.

sequenceDiagram
    participant UI as Reference-time controller
    participant W as History worker
    participant DB as IndexedDB / OPFS
    UI->>W: query(range, cursor, generation 41)
    UI->>W: query(new cursor, generation 42)
    W--xUI: obsolete generation 41 discarded
    W->>DB: indexed bounded read for generation 42
    DB-->>W: observations/checkpoint
    W-->>UI: generation 42 as-of snapshot

Latest-request-wins/coalescing prevents old scrubs from forming a stale work queue. Returning to Live invalidates historical generations and obtains a compact current snapshot plus subsequent deltas.

This is RF Sentry's temporal prime directive: future evidence does not travel backward merely because a worker finished late. There are no slingshots around the Sun in the query planner.

Aviation history

Raw aircraft observations are durable and capture-time ordered. Startup no longer performs an unbounded getAll() hydration. The ordinary browsing horizon is 24 hours, but even that range is paged at high volume. Compact latest identities, current state, and useful presentation trails stay resident; detailed older observations load for an explicit range or aircraft investigation and are evicted afterward.

Live aircraft updates never call historical snapshotAt(). Trails contain decoded position observations only; interpolated marker positions are presentation state and are not persisted as history.

SATCOM products and imagery

Reconstructed original files, metadata, provenance, and native derived images are bulk OPFS assets indexed by compact records. Gallery previews and nearby playback frames use bounded caches. Native generation is separately scheduled from acquisition and uses stable logical derivative identities so opening an overview cannot repeatedly regenerate an archive.

Server products and browser retention

Optional Server processing reads bounded committed canonical records, releasing archive/database ownership before CPU work. The generic compiled processor registry, scheduler, worker supervision and product store are separate from GOES/HRIT decoding and imagery recipes. One owner handles each ordered context; independent contexts and immutable recipe tasks share aggregate CPU/RAM/I/O admission. Cancelled generations cannot publish and remain charged until reaped. Processing, checkpointing and product publication never hold ingestion admission or determine acquisition ACKs. Appliance payload retention remains RAM-only.

The additive product catalog lives under local state; assets, staging, opaque checkpoints and bounded canonical replay live in the explicitly selected archive. Batch-owned packs group small-asset durability without a whole-pack RAM buffer. Synchronized data/directory entries precede recoverable preparation; catalog revisions, outcomes, progress and compatible recovery commit together. Old recovery state survives replacement. Original identities/hashes and observation/reception meaning stay distinct from Server/build/backend execution provenance.

The foreground handles current reception while historical ranges are selected newest-first and decoded forwards, with bounded assembly context. Explicit gaps and exact coverage prevent a recent live timestamp from implying complete older history. Failed items have durable dispositions; resource/storage waits cannot pretend a successful processing commit. See the detailed product contract.

Browser Server-product mode uses a synchronization worker and typed output adapters to publish into the existing archive/viewers. Discovery cursors commit with pending work. Verified ranged transfers close staged OPFS assets before IndexedDB atomically publishes references/revision/completed-fetch state. OPFS and IndexedDB are not one transaction: abandoned staging is recoverable and old assets remain until replacement metadata commits. Matching local reconstruction and recipes stay disabled; caches accelerate durable assets rather than own them.

Record retention, Server-product retention and browser retention are independent. Obsolete non-conflicting partial revisions retire after safe supersession; current products, original evidence, conflicts and active task/read ownership retain their required lifetimes. Dead pack space remains charged until physical reclamation. Browser replacement coalesces obsolete work and retires replaced assets after commit; Server expiry never orders deletion of a separately retained local copy. A selected asset absent locally still needs Server availability and eligible retention. Missing native assets are not made offline-ready by a thumbnail.

Temporal correctness rules

  • Use capture/observation time for historical truth; reception and processing times remain separate metadata.
  • A checkpoint contains only information known at its boundary.
  • Reprocessing old IQ or products must not masquerade as newly received live weather.
  • An old product processed today remains old in observation order.
  • Replay-derived records retain input and processing provenance.
  • Eviction removes caches, not authoritative evidence, unless the operator's explicit retention policy deletes the evidence.