Skip to content

Archive ingestion and recovery

Curated source-specific evidence, not a new run or qualification of current source. Dates, measurements, failure outcomes and original source identifiers retain their recorded meaning. Old tags/assets have been retired and original commits may no longer resolve after the privacy rewrite. Where identifiers were sanitized earlier, the original hardware/run association cannot be independently recovered from the public derivative. Current acceptance lives in the roadmap.

Original implementation: 3ba45e36c82525848e25cbe964a47fc6ed266d5e; fixture/GUI corrections: 818923d564dab55164913c28786ec0a1f8ffd1d6 and 886ffdaf54defca777fdb26b5045ed4db016be0a. The local tables below are the Phase 6 ARM64 checkpoint recorded before the subsequent readback correction. They are not measurements of the later q5 release or current source.

Architecture and durability contract

The optional C++20 Beast/Asio/OpenSSL service subscribes over verified TLS with paired full grants or scoped readers. Schema 3 adds explicit persistent per-stream ingestion/retention switches; pairing and upgrades preserve inactive Phase 5 intent. The single React/TypeScript administration UI compiles local static assets served by C++, preserving setup/trust/optional authentication/pairing/discovery, revision conflicts, CSRF and roles. Installed tests consume packaged assets; installed Server requires no Node runtime, external assets or JavaScript backend.

One supervised archive child serializes bounded commits: canonical validation and dependency checks; append; segment and required directory synchronization; FULL SQLite index/coverage/resume commit; then ACK completed connection delivery ordinals. Resume uses original identities, owner/run epochs and exact uint64 text keys. Supplemental context precedes dependent data and never advances the forward cursor. Groups are bounded by 16 records, 1 MiB and 25 ms within the protocol's 32-delivery/4-MiB window; pending disk work is bounded to 16 requests and active subscriptions to eight. Processing never gates storage ACKs.

The archive retains exact canonical envelopes and GOES-kind 892-byte payloads with configuration, timing and validation evidence. Identical identity/content retries are harmless; distinct receptions remain distinct, and conflicting content is an integrity fault. Recovery verifies committed content, reconciles valid unindexed tails idempotently, discards only incomplete uncommitted tails and refuses committed corruption. Context segments recover before dependent data even across rotation. Retention journals deletion intent and tombstones before unlink/directory sync; required context survives and expired retries cannot repopulate deleted history.

Bound-root/no-follow/current-boot mount protections cover writes, recovery and reads. Archive UUID/storage generation binds indexes and cursors to their actual root. Missing, full, read-only, replaced or stalled storage stops ACKs; administration and independent consumers remain responsive. Bounded permission-checked coverage/ range APIs distinguish committed progress, retained records, explicit gaps, corruption and recorded deletion. Min/max sequence does not claim continuity. The exact format, first-receipt retention time basis, continuation behavior and filesystem assumptions are in ARCHIVE.md.

Executed local qualification

Gate Result
React/TypeScript Type checks, six meaningful DOM tests and self-contained production build pass
Native CTest Eight cases pass in 25.70 s; archive case contains 6,244 assertions
ASan/UBSan CTest All eight cases pass in 107.41 s with leak detection
Native and sanitizer API/enrollment/ingestion 169 / 182 / 67 each pass
Corrected large synthetic fixture 67 native and 67 with gateway and Server both instrumented pass
Trusted owned-browser enrollment 202 pass, including actual React/C++ ingestion controls and preserved administration flows
Private owned mount namespace 87 ingestion + nine mount assertions pass; native namespace evidence, not installed/systemd evidence
Repository validation Appliance/gateway/sanitizer record groups pass; corrected remaining full-validation groups pass, including web and decoders
Shared core Complete native/sanitizer/interchange/genuine memory64 browser receive/DSP gates pass
Gateway/package Gateway integration and ARM64 package/assets/runtime-dependency inspection pass

Archive cases exercise termination around header/append/sync/index boundaries, partial tails, actual fsync EIO and short-write/ENOSPC, committed corruption, rotated context, first-receipt age/byte retention, four deletion crash boundaries, index loss and exact identities beyond 2^53 through uint64 MAX. Real isolated connections exercise real enrollment/full and narrow grants, future streams, disable/rotate/revoke isolation, dependencies, exact readback, lost ACK/restart, ring expiry including completely expired forward history, empty/new owners, multiple consumers, archive replacement/rebinding and large-envelope credit/range pagination. Native syscall wrappers and the never-installed FUSE fixture exercise real storage faults; fault injection is absent from the installed executable.

The private namespace uses disposable 64-MiB tmpfs/FUSE mounts for full/read-only/ unmounted storage, fsync stall/EIO and short-write ENOSPC. Each stops affected ACK advancement while independent consumers continue; administration responds under one second, recovery is exact and the underlying mountpoint remains untouched. The real owned trusted Chromium fixture verifies uint64-MAX telemetry/readback, persistent disable after remount, exact large retention values and no mutation from polling/remount. It preserves sandbox/TLS policy and uses no remote CDP.

Expanded end-to-end tests exposed and repaired a drained-commit socket read rearm, continuation when a new owner remained empty, cancellation overwriting reconnect state, and discovery reads sharing the UI mutation guard. These paths pass the actual connection/browser regressions. Production decoder output remains unchanged; the shared-core reference retains 92 frames, 17 corrections and SHA-256 70e147554060f5be9aa117737a6061fcc3f54f46accd58b3f9608862e8450677.

Resource measurements

Regular ARM64 owned synthetic workload; samples include HTTPS telemetry polling. Latency percentiles are sampled group observations, not exhaustive per-commit instrumentation or RF/DSP performance measurements.

Measurement Catch-up Paced GOES-sized records
Records / elapsed 2,002 / 4.262 s 450 / 8.234 s
Throughput 469.8/s 54.7/s
Commit median / p95 22.9 / 59.7 ms 20.9 / 64.6 ms
Peak record / byte backlog 16 / 19,968 6 / 7,488
Parent / archive-worker CPU 0.59 / 1.22 s 0.63 / 0.66 s
Combined sampled current RSS 48,562,176 bytes 52,101,120 bytes

Storage-fault administration read: 4.61 ms. Sampling covers owned children spawned by all owned threads; current RSS is separate from inherited/startup getrusage high-water figures. ASan/UBSan catch-up is 42.6/s with roughly 1.55 GiB combined instrumentation RSS including transient readiness helpers. Instrumented builds are never installed; those figures do not describe production package RAM. Installed-architecture measurements remain pending.

Regression and acceptance limits

The retained readback diagnosis separates a repaired local boundary from an unproven original hosted timeout. The fixture repair records ACK/barrier and listener reservation regressions; test-only fixes do not prove runtime performance. The installed attempts include a failed preflight and must not be read as a pass of that attempt.

The 2026-10-18 release q4 ARM64 GUI user-creation assertion failed. A held-save regression showed the old fixture could submit a disabled form. Corrected q5 source b5a54ac13781e05c24c9dccf7d67896b9968d0f0 passed main 39084382034 and release 39085247715, including both installed architectures. That later pass does not establish the original CI interleaving or erase the failed attempt.

Physical power-loss behavior on arbitrary disks/controllers/NAS is not qualified. Local SQLite/WAL and honest synchronization remain assumptions. Entire index loss cannot prove prior intentional deletion from segments alone. Phase 6 operator performance acceptance remains open. Later Server phases do not close it.