Skip to content

Validation and acceptance

Repeatable fixtures, implementation checks and live operator acceptance answer different questions. This page preserves the WEBSDR validation contract; it is not a report that every metric below has been measured or every gate closed.

Implemented local harness

Paths below are relative to components/rfsentry-web/:

Boundary Current harness / evidence
CU8/WAV/SigMF replay src/replay/readers.ts, ReplaySource.ts, test/replay.test.ts
Reproducible input generation tools/generate-fixtures.mjs writes ignored test/generated/ and verifies test/fixture-hashes.json; run npm run fixtures through the pinned toolchain
DSP ABI/numerical checks tools/test-dsp.sh and instantiated browser-WASM regression tests; native C alone does not prove shipped exports
Detection / candidate / activation test/signalDetector.test.ts, candidateAnalyzer.test.ts, activation.test.ts, activationPolicy.test.ts
Input clocks / decoder geometry capture-clock, channelizer and continuity tests referenced by the time contract
Protocol correctness independent fixtures and qualification limits on each decoder page, with pinned source provenance
Transport / routes / origin test/rfsentryClient.test.ts and native tests under components/rfsentry-api/test/

Generated tone fixtures establish reproducibility and input geometry, not weak-signal sensitivity or real-world false-alarm rates. Reference WAVs, synthetic IQ, independently decoded recordings and live reception must remain separately labelled. Unimplemented packet/paging/RDS/ISM paths do not acquire qualification just because an upstream fixture source is listed below.

The build guide describes focused and full checks. scripts/validate.sh --full selects appliance, API, client, decoder, detection, DSP, Server and web groups. Browser, hardware and sustained-load acceptance have their own gates below.

Server acceptance evidence and limits

The operator accepted Server Phase 8 and narrow Phase 9 product consumption on 2026-11-09 at 744b840a5c9b3c15967d84d45e777b686779c8da, originally released as v0.6.19-phase8-q2 (since retired). The Server acceptance/future scope keeps Phase 6 performance acceptance and future Server phases separate.

Evidence What it establishes
Main run 42536097284 Selected repository, Core native/memory64/record-browser and Server UI/native amd64/arm64 checks passed. Installed jobs were not run by this main event.
Release run 42536106408 Required exact-source proofs resolved, remaining repository/client checks passed, both installed Debian VM gates passed and unchanged qualified packages were published. Reused native proofs are not repeated executions.
Installed amd64 and arm64 Package lifecycle, API/security, trusted browser, enrollment, ingestion/products and mounted faults, including independent index-volume exhaustion. ARM64 guest emulation is not a physical Pi performance test.

The q1 release failed on an offline configuration-reader lock race in the installed browser fixture. The q2 source uses authenticated persisted API reads and passed both installed gates. Earlier pending installed/acceptance statements for that checkpoint are superseded. Historical reports and private ignored qualification receipts remain evidence for their own sources and workloads; this documentation commit does not transfer release qualification to a new SHA.

Remaining limits are explicit:

  • Phase 6 performance acceptance is still open. Short catch-up observations do not establish sustained throughput, fleet-wide minimum GOES working sets, aggregate multi-stream Pi headroom or real-time service for every native product. The product record retains scoped results and the unresolved reliability limits.
  • The 2026-11-02 local sanitized scheduler hit the five-second archive-I/O watchdog on an 822-byte commit. ACKs stopped safely. A targeted rerun did not explain the stall; later green qualification is not a diagnosis or reproduction of that incident. It remains unresolved, without relaxed timeouts/assertions.
  • Genuine memory64 >4-GiB execution, wider browser/device coverage (including Safari/iPad-specific recovery/rendering), long-run resources/soak and the integrated multi-server matrix remain separately unqualified where no matching evidence exists. MEMORY64=2 compatibility is not >4-GiB proof. Normal worker address-space containment and sanitizer execution are different test conditions.
  • File/directory synchronization and SIGKILL recovery tests do not prove physical power-loss behavior of every disk/controller/NAS. SQLite stays on qualified local storage; a writable archive probe does not qualify arbitrary backends.
  • #5 preserves the demonstrated hidden-main-document radar decode stall; #6 preserves deferred Full Disk cold-load measurement/tuning. #7 retains the unresolved mount-table trigger and stalled automatic recovery, deferred unless it recurs. A successful restart did not diagnose its cause.

Benchmark interpretation

tools/benchmark.mjs records host/toolchain facts and local conversion, header, instantiated-WASM DSP/channelizer/diagnostic and FFT measurements. Its Node microbenchmarks are not production-browser GPU/frame pacing, total process memory, over-the-air continuity or wall-power measurements. Historical reports retain their machine/date and selected workload; never reuse them as current-device proof.

The benchmark writes ignored test/generated/benchmarks/latest.json under the browser component. It does not create a permanent report in docs/. Publish only reviewed, source-bound measurements needed to substantiate a retained claim.

For live acceptance, record acquisition gaps and flags, queue/backlog depth, audio underruns, displayed cadence/frame-time tails, input responsiveness and settled/peak memory across main thread, workers, WASM and browser/GPU resources. JS heap alone is incomplete. Keep presentation refresh distinct from signal evidence and retain source loss accounting.

Required validation matrix

The original requirements below remain test/acceptance scope, not a list of all currently automated checks. The two-receiver wording is the original minimum isolation example; use today's configured inventory, not a fixed two-device assumption. The original 30-second shutdown target is dated configuration context: inspect route grace, receiver idle and deep-idle deadlines separately. The on-demand native decoder broker has future power-test requirements. The appliance already supports the separate continuous HRIT job.

Recorded-Signal Harness

Development should begin with recorded CU8, WAV, and SigMF fixtures. Each decoder and DSP path needs repeatable golden tests.

Useful fixture sources include:

  • Locally recorded, user-confirmed signals.
  • Long local receiver-noise sessions across routes, bands, gains, and times, including empty spectrum, crowded spectrum, DC, I/Q mirrors, overload, impulsive interference, switching combs, and dropped/repeated blocks.
  • rtl_433 regression samples.
  • FT8/FT4 reference WAV files.
  • WSJT-X sample files.
  • Synthetic AM, FM, SSB, CW, FSK, and PSK generators.
  • Packet, paging, and RDS fixtures from their upstream projects.

Detection Metrics

Measure:

  • Probability of detecting signals by SNR and bandwidth.
  • False alarms per MHz-hour, both overall and by receiver/path/band.
  • Parent-emission over-segmentation and under-segmentation rates.
  • Center-frequency error.
  • Bandwidth-estimation error.
  • Track stability.
  • Time to first detection.
  • Performance around DC and known spurs.
  • Artifact-screen precision and recall.
  • Warmup, fade, recurrence, and identity-reuse behavior.
  • Zero detector analysis and publication while the master toggle is off, with uninterrupted waterfall, audio, and manual tuning.

Classification Metrics

Measure:

  • Confusion matrix by modulation family.
  • Accuracy by SNR.
  • Unknown-signal rejection.
  • Confidence calibration.
  • Performance on real over-the-air captures not used for training.
  • Improvement from band-plan and frequency priors, reported separately from waveform-only accuracy.

Decoder Metrics

Measure:

  • Messages or frames decoded against known captures.
  • CRC/FEC success and failure.
  • Comparison with authoritative upstream decoders.
  • Behavior across frequency offset, drift, fading, clipping, and dropped samples.
  • CPU and memory per active decoder.

Appliance Integration Tests

Validate:

  • Both receivers can operate independently.
  • Different receivers can be used simultaneously.
  • Manual display frequencies select the expected configured route, including stable hysteresis and a target-before-release boundary handoff.
  • A busy, absent, or faulty receiver produces an eligible fallback or a clear route error without disturbing the other receiver.
  • Adding a valid serial-referenced route requires no executable or frontend change.
  • An active SDR++ session cannot be interrupted.
  • Browser disconnect returns through the 30-second clean idle path.
  • Stale USB mapping is repaired through the existing controller.
  • Hot-unplug affects only the exact serial.
  • The web gateway never connects directly to backend ports.
  • No new component changes USB power itself.
  • No endless absent-device, decoder, or reconnect logging occurs.
  • Queues and memory remain bounded during a slow client.

Performance and Power Tests

Record for every profile:

  • Pi CPU and resident memory while idle.
  • Pi CPU and temperature while streaming.
  • Pi CPU with each native decoder worker.
  • Network throughput.
  • Browser CPU, GPU, and memory.
  • Waterfall frame cadence.
  • Audio underruns and glitches.
  • Cold-wake time.
  • Disconnect-to-clean-shutdown behavior.
  • Total wall power where an external power meter is available.

Evidence, privacy, and closeout

Retain specimen provenance, sample rate/format and time anchors, expected values, independent decoder/reference identity, applied processing and discontinuities. Keep captures, tokens and private diagnostic reports out of Git; exact hashes and licence/redistribution permission belong with imported fixtures.

Report new results separately from historical checkboxes. A nominal scheduling target is not achieved cadence; a manifest/artifact is not protocol proof; a generated fixture is not an independent recording. Preserve Phase 2's long-capture audits, Phase 1 carryovers, wider acceleration/FT/NRSC-5 matrices and each Aviation/SATCOM gate until the required operator evidence exists. See the ledger map.