Validation and acceptance¶
Repeatable fixtures, implementation checks and live operator acceptance answer different questions. This page preserves the WEBSDR validation contract; it is not a report that every metric below has been measured or every gate closed.
Implemented local harness¶
Paths below are relative to components/rfsentry-web/:
| Boundary | Current harness / evidence |
|---|---|
| CU8/WAV/SigMF replay | src/replay/readers.ts, ReplaySource.ts, test/replay.test.ts |
| Reproducible input generation | tools/generate-fixtures.mjs writes ignored test/generated/ and verifies test/fixture-hashes.json; run npm run fixtures through the pinned toolchain |
| DSP ABI/numerical checks | tools/test-dsp.sh and instantiated browser-WASM regression tests; native C alone does not prove shipped exports |
| Detection / candidate / activation | test/signalDetector.test.ts, candidateAnalyzer.test.ts, activation.test.ts, activationPolicy.test.ts |
| Input clocks / decoder geometry | capture-clock, channelizer and continuity tests referenced by the time contract |
| Protocol correctness | independent fixtures and qualification limits on each decoder page, with pinned source provenance |
| Transport / routes / origin | test/rfsentryClient.test.ts and native tests under components/rfsentry-api/test/ |
Generated tone fixtures establish reproducibility and input geometry, not weak-signal sensitivity or real-world false-alarm rates. Reference WAVs, synthetic IQ, independently decoded recordings and live reception must remain separately labelled. Unimplemented packet/paging/RDS/ISM paths do not acquire qualification just because an upstream fixture source is listed below.
The build guide describes focused and full
checks. scripts/validate.sh --full selects appliance, API, client, decoder,
detection, DSP, Server and web groups. Browser, hardware and sustained-load
acceptance have their own gates below.
Server acceptance evidence and limits¶
The operator accepted Server Phase 8 and narrow Phase 9 product consumption on
2026-11-09 at 744b840a5c9b3c15967d84d45e777b686779c8da, originally released as v0.6.19-phase8-q2 (since retired).
The Server acceptance/future scope
keeps Phase 6 performance acceptance and future Server phases separate.
| Evidence | What it establishes |
|---|---|
| Main run 42536097284 | Selected repository, Core native/memory64/record-browser and Server UI/native amd64/arm64 checks passed. Installed jobs were not run by this main event. |
| Release run 42536106408 | Required exact-source proofs resolved, remaining repository/client checks passed, both installed Debian VM gates passed and unchanged qualified packages were published. Reused native proofs are not repeated executions. |
| Installed amd64 and arm64 | Package lifecycle, API/security, trusted browser, enrollment, ingestion/products and mounted faults, including independent index-volume exhaustion. ARM64 guest emulation is not a physical Pi performance test. |
The q1 release failed on an offline configuration-reader lock race in the installed browser fixture. The q2 source uses authenticated persisted API reads and passed both installed gates. Earlier pending installed/acceptance statements for that checkpoint are superseded. Historical reports and private ignored qualification receipts remain evidence for their own sources and workloads; this documentation commit does not transfer release qualification to a new SHA.
Remaining limits are explicit:
- Phase 6 performance acceptance is still open. Short catch-up observations do not establish sustained throughput, fleet-wide minimum GOES working sets, aggregate multi-stream Pi headroom or real-time service for every native product. The product record retains scoped results and the unresolved reliability limits.
- The 2026-11-02 local sanitized scheduler hit the five-second archive-I/O watchdog on an 822-byte commit. ACKs stopped safely. A targeted rerun did not explain the stall; later green qualification is not a diagnosis or reproduction of that incident. It remains unresolved, without relaxed timeouts/assertions.
- Genuine memory64 >4-GiB execution, wider browser/device coverage (including Safari/iPad-specific recovery/rendering), long-run resources/soak and the integrated multi-server matrix remain separately unqualified where no matching evidence exists. MEMORY64=2 compatibility is not >4-GiB proof. Normal worker address-space containment and sanitizer execution are different test conditions.
- File/directory synchronization and SIGKILL recovery tests do not prove physical power-loss behavior of every disk/controller/NAS. SQLite stays on qualified local storage; a writable archive probe does not qualify arbitrary backends.
- #5 preserves the demonstrated hidden-main-document radar decode stall; #6 preserves deferred Full Disk cold-load measurement/tuning. #7 retains the unresolved mount-table trigger and stalled automatic recovery, deferred unless it recurs. A successful restart did not diagnose its cause.
Benchmark interpretation¶
tools/benchmark.mjs records host/toolchain facts and local conversion,
header, instantiated-WASM DSP/channelizer/diagnostic and FFT measurements.
Its Node microbenchmarks are not production-browser GPU/frame pacing, total
process memory, over-the-air continuity or wall-power measurements. Historical
reports retain their machine/date and selected workload; never reuse them as
current-device proof.
The benchmark writes ignored test/generated/benchmarks/latest.json under the
browser component. It does not create a permanent report in docs/. Publish
only reviewed, source-bound measurements needed to substantiate a retained claim.
For live acceptance, record acquisition gaps and flags, queue/backlog depth, audio underruns, displayed cadence/frame-time tails, input responsiveness and settled/peak memory across main thread, workers, WASM and browser/GPU resources. JS heap alone is incomplete. Keep presentation refresh distinct from signal evidence and retain source loss accounting.
Required validation matrix¶
The original requirements below remain test/acceptance scope, not a list of all currently automated checks. The two-receiver wording is the original minimum isolation example; use today's configured inventory, not a fixed two-device assumption. The original 30-second shutdown target is dated configuration context: inspect route grace, receiver idle and deep-idle deadlines separately. The on-demand native decoder broker has future power-test requirements. The appliance already supports the separate continuous HRIT job.
Recorded-Signal Harness¶
Development should begin with recorded CU8, WAV, and SigMF fixtures. Each decoder and DSP path needs repeatable golden tests.
Useful fixture sources include:
- Locally recorded, user-confirmed signals.
- Long local receiver-noise sessions across routes, bands, gains, and times, including empty spectrum, crowded spectrum, DC, I/Q mirrors, overload, impulsive interference, switching combs, and dropped/repeated blocks.
rtl_433regression samples.- FT8/FT4 reference WAV files.
- WSJT-X sample files.
- Synthetic AM, FM, SSB, CW, FSK, and PSK generators.
- Packet, paging, and RDS fixtures from their upstream projects.
Detection Metrics¶
Measure:
- Probability of detecting signals by SNR and bandwidth.
- False alarms per MHz-hour, both overall and by receiver/path/band.
- Parent-emission over-segmentation and under-segmentation rates.
- Center-frequency error.
- Bandwidth-estimation error.
- Track stability.
- Time to first detection.
- Performance around DC and known spurs.
- Artifact-screen precision and recall.
- Warmup, fade, recurrence, and identity-reuse behavior.
- Zero detector analysis and publication while the master toggle is off, with uninterrupted waterfall, audio, and manual tuning.
Classification Metrics¶
Measure:
- Confusion matrix by modulation family.
- Accuracy by SNR.
- Unknown-signal rejection.
- Confidence calibration.
- Performance on real over-the-air captures not used for training.
- Improvement from band-plan and frequency priors, reported separately from waveform-only accuracy.
Decoder Metrics¶
Measure:
- Messages or frames decoded against known captures.
- CRC/FEC success and failure.
- Comparison with authoritative upstream decoders.
- Behavior across frequency offset, drift, fading, clipping, and dropped samples.
- CPU and memory per active decoder.
Appliance Integration Tests¶
Validate:
- Both receivers can operate independently.
- Different receivers can be used simultaneously.
- Manual display frequencies select the expected configured route, including stable hysteresis and a target-before-release boundary handoff.
- A busy, absent, or faulty receiver produces an eligible fallback or a clear route error without disturbing the other receiver.
- Adding a valid serial-referenced route requires no executable or frontend change.
- An active SDR++ session cannot be interrupted.
- Browser disconnect returns through the 30-second clean idle path.
- Stale USB mapping is repaired through the existing controller.
- Hot-unplug affects only the exact serial.
- The web gateway never connects directly to backend ports.
- No new component changes USB power itself.
- No endless absent-device, decoder, or reconnect logging occurs.
- Queues and memory remain bounded during a slow client.
Performance and Power Tests¶
Record for every profile:
- Pi CPU and resident memory while idle.
- Pi CPU and temperature while streaming.
- Pi CPU with each native decoder worker.
- Network throughput.
- Browser CPU, GPU, and memory.
- Waterfall frame cadence.
- Audio underruns and glitches.
- Cold-wake time.
- Disconnect-to-clean-shutdown behavior.
- Total wall power where an external power meter is available.
Evidence, privacy, and closeout¶
Retain specimen provenance, sample rate/format and time anchors, expected values, independent decoder/reference identity, applied processing and discontinuities. Keep captures, tokens and private diagnostic reports out of Git; exact hashes and licence/redistribution permission belong with imported fixtures.
Report new results separately from historical checkboxes. A nominal scheduling target is not achieved cadence; a manifest/artifact is not protocol proof; a generated fixture is not an independent recording. Preserve Phase 2's long-capture audits, Phase 1 carryovers, wider acceleration/FT/NRSC-5 matrices and each Aviation/SATCOM gate until the required operator evidence exists. See the ledger map.