Compare commits
21 Commits
4f8eb711f3
...
218a65068a
| Author | SHA1 | Date | |
|---|---|---|---|
| 218a65068a | |||
| 0bd3b9e035 | |||
| c01f10c474 | |||
| 165dc21842 | |||
| e53f6a3f46 | |||
| bda86be02a | |||
| 220f996928 | |||
| 6a34dee598 | |||
| b6db15ca03 | |||
| 75241d0774 | |||
| 7f1b1727d4 | |||
| 25e41c1fba | |||
| c709c3e9df | |||
| 8fd451f6ba | |||
| 44c751154c | |||
| e33a19d38e | |||
| 220bdd6c1f | |||
| d476b2fb18 | |||
| 49a6fb5af8 | |||
| c30680fcf3 | |||
| 361ceb8393 |
|
|
@ -3,6 +3,79 @@
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## 2026-06-24 (cont.) — R4 analyst mode: MITM-everything + media reverse-catcher + clone (#736)
|
||||||
|
|
||||||
|
New "R4" doctrine — visibility over performance. Delivered + live on gk2:
|
||||||
|
- **Splice flip** — `tls-splice-seed.conf` reduced from a media-CDN perf list to
|
||||||
|
breakers-only (`api.anthropic.com`); splice now applied ONLY where MITM provably
|
||||||
|
breaks (cert pinning). Banner reaches every page; catcher sees media URLs. Live:
|
||||||
|
learned splices cleared, autolearn gated (`tls_splice=off`).
|
||||||
|
- **sbxmitm media reverse-catcher** (`cmd/sbxmitm/mediacatch.go`, toolbox-ng 0.1.20)
|
||||||
|
— 2xx MITM'd flows → cloneable media URLs (HLS/DASH manifests, direct A/V,
|
||||||
|
googlevideo videoplayback) appended to `/run/secubox/media-catch.jsonl` (URLs
|
||||||
|
only, deduped, atomic, fail-open). `--media-catch` default on; worker unit
|
||||||
|
`ReadWritePaths=/run/secubox`.
|
||||||
|
- **mediaflow Discovered Media + Clone** (2.1.0) — `/discovered`, `/clone`
|
||||||
|
(yt-dlp→ffmpeg queue, lazy worker for the aggregator), `/library`,
|
||||||
|
`/download/{id}`, DELETE; dashboard cards. Verified: HLS caught → ffmpeg →
|
||||||
|
464 MiB mp4 in library. yt-dlp installed.
|
||||||
|
- Also fixed the empty mediaflow dashboard (2.0.2 contract + 2.0.3 cumulative
|
||||||
|
services): cards/streams live, Top Media Services from DPI cumulative store.
|
||||||
|
KEY: dashboard routes via the **aggregator** (in-process import) — restart
|
||||||
|
`secubox-aggregator` to pick up mediaflow code changes.
|
||||||
|
- Phase 4 done — R4 button added to the banner topbar (R0..R4) + set-level + by-MAC
|
||||||
|
validation + analytics buckets; gated to the wg path like R3 (secubox-toolbox 2.7.20).
|
||||||
|
- yt-dlp upgraded 2023.03.04 → 2026.06.09 (standalone binary; YouTube works).
|
||||||
|
- Recos: catcher now captures YouTube watch **pages** (kind=page, toolbox-ng 0.1.22);
|
||||||
|
Discovered Media persisted off tmpfs into a durable capped store (mediaflow 2.1.1);
|
||||||
|
yt-dlp packaged (Recommends + weekly refresh timer + postinst).
|
||||||
|
- **Catch-log ownership bug** — `/run/secubox/media-catch.jsonl` was created
|
||||||
|
`secubox`-owned while the worker runs as `secubox-toolbox`, so O_APPEND failed
|
||||||
|
silently → nothing captured. Fixed with a tmpfiles.d entry pre-creating it owned
|
||||||
|
by the writer every boot (zz-secubox-toolbox-ng.conf). Live: rm + worker recreate.
|
||||||
|
|
||||||
|
## 2026-06-24 (cont.) — Banner on nonce-CSP sites + Claude API splice + YouTube unblock (#728)
|
||||||
|
|
||||||
|
Three distinct root causes behind "no banner on youtube / news", fixed in order:
|
||||||
|
|
||||||
|
1. **Trusted Types** (0.1.17) — `require-trusted-types-for` blocked DOM injection. Stripped.
|
||||||
|
2. **Nonce-based CSP** (0.1.18) — the banner is *inlined* (service-worker-proof), but a CSP
|
||||||
|
nonce/hash makes `'unsafe-inline'` IGNORED → the bare inline `<script>` was silently
|
||||||
|
blocked. `relaxCSPForLoader` now **borrows the page's own nonce** and stamps it on the
|
||||||
|
injected `<script nonce=…>` (surgical: page CSP/nonces/hashes untouched), falling back to
|
||||||
|
forcing `unsafe-inline` (drop nonce/hash/strict-dynamic) only when there's no nonce.
|
||||||
|
Nonce validated to base64 charset (attribute-breakout guard). Threaded nonce through
|
||||||
|
injectIntoBody → injectHTML → injectInlineBanner. Tests rewritten for inline semantics.
|
||||||
|
3. **YouTube wholly blocked** (runtime) — autolearn false-positive put `youtube.com` in
|
||||||
|
`/var/lib/secubox/toolbox/learned-trackers.txt` → `Decide()` returned `block` (204) →
|
||||||
|
page never loaded. Removed from learned + added to `ad-allowlist.txt` (hot-reloaded).
|
||||||
|
Latent-bug tracker: **#735** (autolearn must not block apex/first-party nav targets).
|
||||||
|
|
||||||
|
**Claude API splice** (user request) — `api.anthropic.com` added to `tls-splice-seed.conf`
|
||||||
|
(+ live seed): cert-pinned Claude API/SDK clients reject the MITM CA, so pass them through;
|
||||||
|
`claude.ai` web stays MITM'd (browser trusts the CA → still gets the banner).
|
||||||
|
|
||||||
|
Verified end-to-end on gk2: YouTube 200 + banner nonce == page nonce; lemonde/lefigaro
|
||||||
|
banner via unsafe-inline fallback. DPI confirmed healthy — collector writes to
|
||||||
|
`/var/lib/secubox/dpi/` (state.json/cumulative.json fresh), `/exfil` returns categorized
|
||||||
|
flows; the earlier "empty" was me checking the wrong paths (`/run/secubox/dpi`).
|
||||||
|
|
||||||
|
## 2026-06-24 — DPI YouTube bannering: strip Trusted Types CSP (#728)
|
||||||
|
|
||||||
|
- **Root cause** — YouTube serves a standalone `Content-Security-Policy:
|
||||||
|
require-trusted-types-for 'script'` header. sbxmitm's `relaxCSPForLoader` already
|
||||||
|
relaxed `script-src` (drop `strict-dynamic`, add `'self'`/`'unsafe-inline'`) so the
|
||||||
|
banner loader runs, but Trusted Types still blocked the banner's DOM injection →
|
||||||
|
banner silently never mounted on YouTube.
|
||||||
|
- **Fix** (`cmd/sbxmitm/csp.go`, toolbox-ng 0.1.17) — drop `require-trusted-types-for`
|
||||||
|
and `trusted-types` directives during the relax; omit the resulting empty CSP header
|
||||||
|
line. Local Go unit tests cover both the relax and the empty-header drop.
|
||||||
|
- **DPI capture half** — collector `state.json` was stale (frozen 09:44); restarted
|
||||||
|
`secubox-dpi-flowcap` → fresh windows, YouTube/media flows now visible in mediaflow.
|
||||||
|
- Deployed to gk2; R3 workers `secubox-toolbox-ng-worker@1..4` restarted on 0.1.17.
|
||||||
|
- Filed for later: #729 wireguard peers/tabs, #730 yacy, #731 lyrion, #732 magicmirror,
|
||||||
|
#733 firewall dashboard misreport, #734 webui.conf hardcoded-route cleanup.
|
||||||
|
|
||||||
## 2026-06-22 — DPI exfil engine + Netrunner report (HTML+PDF) + sbxmitm fixes
|
## 2026-06-22 — DPI exfil engine + Netrunner report (HTML+PDF) + sbxmitm fixes
|
||||||
|
|
||||||
Big session: full per-device DPI exfiltration pipeline, the kbin report reborn as a
|
Big session: full per-device DPI exfiltration pipeline, the kbin report reborn as a
|
||||||
|
|
|
||||||
|
Before Width: | Height: | Size: 32 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 32 KiB After Width: | Height: | Size: 31 KiB |
|
Before Width: | Height: | Size: 35 KiB After Width: | Height: | Size: 34 KiB |
|
Before Width: | Height: | Size: 34 KiB After Width: | Height: | Size: 50 KiB |
BIN
docs/screenshots/thumbnails/authelia-thumb.png
Normal file
|
After Width: | Height: | Size: 48 KiB |
|
Before Width: | Height: | Size: 25 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 36 KiB After Width: | Height: | Size: 53 KiB |
|
Before Width: | Height: | Size: 33 KiB After Width: | Height: | Size: 60 KiB |
|
Before Width: | Height: | Size: 31 KiB After Width: | Height: | Size: 43 KiB |
BIN
docs/screenshots/thumbnails/certs-thumb.png
Normal file
|
After Width: | Height: | Size: 32 KiB |
|
Before Width: | Height: | Size: 34 KiB After Width: | Height: | Size: 48 KiB |
|
Before Width: | Height: | Size: 33 KiB After Width: | Height: | Size: 32 KiB |
|
Before Width: | Height: | Size: 28 KiB After Width: | Height: | Size: 34 KiB |
|
Before Width: | Height: | Size: 40 KiB After Width: | Height: | Size: 54 KiB |
|
Before Width: | Height: | Size: 71 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 51 KiB After Width: | Height: | Size: 32 KiB |
|
Before Width: | Height: | Size: 28 KiB After Width: | Height: | Size: 31 KiB |
|
Before Width: | Height: | Size: 32 KiB After Width: | Height: | Size: 42 KiB |
|
Before Width: | Height: | Size: 31 KiB After Width: | Height: | Size: 15 KiB |
|
Before Width: | Height: | Size: 32 KiB After Width: | Height: | Size: 46 KiB |
|
Before Width: | Height: | Size: 29 KiB After Width: | Height: | Size: 43 KiB |
|
Before Width: | Height: | Size: 35 KiB After Width: | Height: | Size: 50 KiB |
|
Before Width: | Height: | Size: 27 KiB After Width: | Height: | Size: 42 KiB |
|
Before Width: | Height: | Size: 33 KiB After Width: | Height: | Size: 45 KiB |
|
Before Width: | Height: | Size: 47 KiB After Width: | Height: | Size: 36 KiB |
BIN
docs/screenshots/thumbnails/fmrelay-thumb.png
Normal file
|
After Width: | Height: | Size: 50 KiB |
|
Before Width: | Height: | Size: 30 KiB After Width: | Height: | Size: 45 KiB |
|
Before Width: | Height: | Size: 39 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 45 KiB |
BIN
docs/screenshots/thumbnails/grafana-thumb.png
Normal file
|
After Width: | Height: | Size: 43 KiB |
|
Before Width: | Height: | Size: 40 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 34 KiB After Width: | Height: | Size: 44 KiB |
BIN
docs/screenshots/thumbnails/health-thumb.png
Normal file
|
After Width: | Height: | Size: 65 KiB |
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 46 KiB |
|
Before Width: | Height: | Size: 31 KiB After Width: | Height: | Size: 47 KiB |
|
Before Width: | Height: | Size: 33 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 33 KiB After Width: | Height: | Size: 46 KiB |
|
Before Width: | Height: | Size: 31 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 32 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 28 KiB After Width: | Height: | Size: 45 KiB |
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 46 KiB |
|
Before Width: | Height: | Size: 32 KiB After Width: | Height: | Size: 47 KiB |
|
Before Width: | Height: | Size: 29 KiB After Width: | Height: | Size: 44 KiB |
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 38 KiB |
|
Before Width: | Height: | Size: 32 KiB After Width: | Height: | Size: 30 KiB |
|
Before Width: | Height: | Size: 44 KiB After Width: | Height: | Size: 51 KiB |
|
Before Width: | Height: | Size: 34 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 38 KiB After Width: | Height: | Size: 52 KiB |
|
Before Width: | Height: | Size: 34 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 33 KiB After Width: | Height: | Size: 46 KiB |
|
Before Width: | Height: | Size: 28 KiB After Width: | Height: | Size: 44 KiB |
|
Before Width: | Height: | Size: 31 KiB After Width: | Height: | Size: 30 KiB |
|
Before Width: | Height: | Size: 28 KiB After Width: | Height: | Size: 40 KiB |
|
Before Width: | Height: | Size: 33 KiB After Width: | Height: | Size: 34 KiB |
|
Before Width: | Height: | Size: 35 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 41 KiB After Width: | Height: | Size: 44 KiB |
|
Before Width: | Height: | Size: 32 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 34 KiB After Width: | Height: | Size: 16 KiB |
BIN
docs/screenshots/thumbnails/metoblizer-thumb.png
Normal file
|
After Width: | Height: | Size: 46 KiB |
|
Before Width: | Height: | Size: 36 KiB After Width: | Height: | Size: 53 KiB |
|
Before Width: | Height: | Size: 30 KiB After Width: | Height: | Size: 43 KiB |
|
Before Width: | Height: | Size: 37 KiB After Width: | Height: | Size: 47 KiB |
|
Before Width: | Height: | Size: 33 KiB After Width: | Height: | Size: 33 KiB |
|
Before Width: | Height: | Size: 27 KiB After Width: | Height: | Size: 43 KiB |
|
Before Width: | Height: | Size: 36 KiB After Width: | Height: | Size: 40 KiB |
|
Before Width: | Height: | Size: 37 KiB After Width: | Height: | Size: 46 KiB |
|
Before Width: | Height: | Size: 38 KiB After Width: | Height: | Size: 51 KiB |
|
Before Width: | Height: | Size: 38 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 37 KiB After Width: | Height: | Size: 46 KiB |
|
Before Width: | Height: | Size: 42 KiB After Width: | Height: | Size: 50 KiB |
|
Before Width: | Height: | Size: 40 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 34 KiB After Width: | Height: | Size: 31 KiB |
|
Before Width: | Height: | Size: 25 KiB After Width: | Height: | Size: 48 KiB |
|
Before Width: | Height: | Size: 32 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 33 KiB After Width: | Height: | Size: 28 KiB |
|
Before Width: | Height: | Size: 36 KiB After Width: | Height: | Size: 51 KiB |
|
Before Width: | Height: | Size: 37 KiB After Width: | Height: | Size: 45 KiB |
|
Before Width: | Height: | Size: 25 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 27 KiB After Width: | Height: | Size: 48 KiB |
|
Before Width: | Height: | Size: 30 KiB After Width: | Height: | Size: 43 KiB |
BIN
docs/screenshots/thumbnails/podcaster-thumb.png
Normal file
|
After Width: | Height: | Size: 51 KiB |
|
Before Width: | Height: | Size: 37 KiB After Width: | Height: | Size: 29 KiB |
|
Before Width: | Height: | Size: 34 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 39 KiB After Width: | Height: | Size: 46 KiB |
|
Before Width: | Height: | Size: 37 KiB After Width: | Height: | Size: 49 KiB |
|
Before Width: | Height: | Size: 36 KiB After Width: | Height: | Size: 48 KiB |
|
Before Width: | Height: | Size: 29 KiB After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 34 KiB After Width: | Height: | Size: 48 KiB |
|
Before Width: | Height: | Size: 48 KiB After Width: | Height: | Size: 46 KiB |
|
Before Width: | Height: | Size: 36 KiB After Width: | Height: | Size: 45 KiB |
|
Before Width: | Height: | Size: 37 KiB After Width: | Height: | Size: 46 KiB |
BIN
docs/screenshots/thumbnails/rustdesk-thumb.png
Normal file
|
After Width: | Height: | Size: 43 KiB |
|
Before Width: | Height: | Size: 27 KiB After Width: | Height: | Size: 16 KiB |
BIN
docs/screenshots/thumbnails/security-posture-thumb.png
Normal file
|
After Width: | Height: | Size: 48 KiB |
BIN
docs/screenshots/thumbnails/sentinelle-thumb.png
Normal file
|
After Width: | Height: | Size: 36 KiB |
|
Before Width: | Height: | Size: 28 KiB After Width: | Height: | Size: 49 KiB |
|
Before Width: | Height: | Size: 44 KiB After Width: | Height: | Size: 46 KiB |