Compare commits

...

4 Commits

Author SHA1 Message Date
CyberMind
da71515d79
Merge pull request #664 from CyberMind-FR/fix/662-restore-ng-source
Some checks are pending
License Headers / check (push) Waiting to run
fix(#662): restore Go PoC source lost to .gitignore
2026-06-18 17:13:42 +02:00
73e79b85b4 fix(#662): restore Go PoC source — .gitignore 'sbxmitm' wrongly ignored cmd/sbxmitm/ dir (anchored to /sbxmitm) 2026-06-18 17:13:28 +02:00
CyberMind
56d1bee9fb
Merge pull request #663 from CyberMind-FR/feature/662-epic-migrate-toolbox-mitm-engine-off-pyt
epic: migrate toolbox MITM engine off Python mitmproxy (gomitmproxy/hudsucker/Squid analysis + phased switch)
2026-06-18 17:09:26 +02:00
6daacb1987 feat(#662 Phase 1): MITM-engine migration analysis + phased plan + compiled/tested Go forging-MITM PoC
Analysis: gomitmproxy (unmaintained, dropped) vs martian/goproxy (Go) vs hudsucker
(Rust) vs Squid+ICAP, mapped to the 18-addon capability set. Recommendation: Go
hot-path core + retained Python analysis sidecars. Phased plan with shadow-run +
nft-DNAT-flip rollback (no big-bang cutover). Phase-1 PoC (packages/secubox-toolbox-ng,
stdlib-only): forge from ca-wg CA, 204-block, body-inject, SNI-splice, ClientHello/JA4
capture — go vet clean, tests green, arm64 cross-compile OK. NOT wired to live R3.
2026-06-18 17:07:48 +02:00
8 changed files with 739 additions and 0 deletions

View File

@ -0,0 +1,55 @@
# Toolbox MITM engine migration — phased plan (#662)
> Engine: **Go hot-path core + retained Python analysis sidecars** (see analysis doc).
> Discipline: shadow-run before cutover; nft-DNAT flip = instant rollback at every step; NEVER big-bang. This is a multi-PR epic — each phase is its own PR with a gate.
## Invariants (must hold every phase)
- Reuse the existing CA `/etc/secubox/toolbox/ca-wg/{ca.pem,key.pem}` (what R3 clients already trust) — no new CA, no client re-enroll.
- Live R3 keeps running on the Python mitmproxy workers (8081-8084) until the final cutover. The Go core runs on **separate ports (8090-8093)**, no DNAT, until Phase 6.
- Ad-blocking + anti-track must never regress (the whole point of the appliance).
- arm64; one static Go binary; systemd `secubox-toolbox-ng-worker@N`.
## Phase 1 — PoC (THIS PR) — GATE: compiles + smoke test passes
**packages/secubox-toolbox-ng/** (Go module). NOT wired to live R3.
- `go.mod`, `cmd/sbxmitm/main.go`: a forging MITM that loads `ca-wg/{ca.pem,key.pem}`, listens on a port, and demonstrates the discriminating capabilities:
- request short-circuit **204** for a sample ad host (proves ad_ghost block),
- response **body inject** of a marker (proves banner/ad CSS),
- **SNI splice** passthrough for a sample host (proves tls_splice),
- **JA4 ClientHello capture** via a `crypto/tls` shim logging cipher suites/exts (proves the Go JA4 gap is closable).
- Smoke test (`make test` / a shell script): build for host, run, `curl -x`/transparent a request through it, assert the 204 + the injected marker + a JA4 line.
- `README.md`: build (`GOOS=linux GOARCH=arm64 go build`), the capability map, and the phase roadmap.
- **No deb packaging, no board deploy, no DNAT.** Pure de-risking spike.
## Phase 2 — arm64 build + board bench (no traffic) — GATE: forge+throughput ≥ mitmproxy
- CI/build: cross-compile arm64 static binary; debian packaging stub `secubox-toolbox-ng` (binary + systemd unit, unit DISABLED).
- Deploy the binary to gk2, run on :8090 (no DNAT). Bench: cert-forge latency (cold/warm), req/s, multi-core CPU under synthetic load vs a mitmproxy worker. Confirm it reuses ca-wg certs (client trusts forged leaf).
## Phase 3 — hot-path feature parity — GATE: parity tests green
Port the cheap per-request rewrites into the Go core, reading the SAME data files:
- block 204 from `_AD_HOST`-equivalent + learned-trackers.txt + pure-trackers.txt, with `ad-allowlist.txt` + own-infra guard (#658) honored.
- header/cookie strip (utiq/protective/anonymize), XFF.
- serve `/__toolbox/loader.js` + `/__toolbox/bundle`; banner inject (buffer + streaming).
- SNI splice from the media seed + learned-splice (the safe, no-auto-promote version).
- Parity harness: feed recorded request/response fixtures to both engines, diff the block/inject/strip decisions.
## Phase 4 — analysis sidecars + anti-track poison — GATE: sidecar contract tests
- Go core fires unix-socket events (fire-and-forget) to the EXISTING Python services for social-graph / dpi / cookies / avatar / soc / ja4-scoring — reuse their socket contracts; they stay Python, off the hot path.
- Port the deterministic anti-track **HMAC jar + Set-Cookie forge** to Go (small, security-critical → exhaustive tests vs the Python `privacy.py` jar output for identical inputs).
- Contextual ad metrics (ad_block_stats / per-visitor) written by a sidecar or the Go core's bg writer.
## Phase 5 — SHADOW run — GATE: N-day output parity, zero client breakage
- Run the Go core on :8090-8093. Mirror a SMALL fraction of R3 (e.g. one fanout slot, or a passive tee) to it; compare its would-block/would-inject/recorded against the live mitmproxy for the same flows. Do NOT serve clients from it yet.
- Soak; review divergences; fix; repeat until parity.
## Phase 6 — CUTOVER — GATE: soak, instant rollback ready
- Flip the nft `numgen inc mod 4` fanout from 8081-8084 (mitmproxy) → 8090-8093 (Go core). Keep the mitmproxy workers RUNNING (stopped from receiving DNAT, but up) so rollback = flip the map back (seconds).
- Soak under real load; watch ad-blocking, banner, anti-track, JA4, latency, CPU.
## Phase 7 — decommission — GATE: stable post-cutover window
- Stop/disable the mitmproxy workers; keep the package installed (rollback) for one release, then remove.
## Rollback
At every phase the live path is the mitmproxy workers until Phase 6's DNAT flip; Phase 6 rollback is an nft map edit (seconds). No phase removes the fallback until Phase 7.
## Effort/risk (honest)
Weeks across 7 PRs. Highest-risk areas: JA4-in-Go (de-risked in Phase 1), the anti-track poison port (Phase 4, exhaustively tested), and the cutover (Phase 6, shadow-gated + instant rollback). Recommend pausing after each gate for review.

View File

@ -0,0 +1,120 @@
# Toolbox MITM engine migration — analysis (gomitmproxy / martian·goproxy / hudsucker / Squid+ICAP)
- **Date:** 2026-06-18 · **Issue:** #662 · **Status:** analysis + recommendation
## Why
The R3 path runs Python **mitmproxy**: GIL-bound, ~1 core total across 4 workers,
the tunnel's CPU/latency ceiling (#646). Goal: a multi-core engine **without
losing the 18-addon feature set**. TLS termination was never the bottleneck —
the single-thread L7 work is — so a bare TLS proxy is a non-starter (loses every
feature). The only worthwhile target is a faster **L7 engine** that re-implements
the inline logic.
## The real requirement: our 18 addons' capabilities
| # | Addon | Capability it needs |
|---|-------|---------------------|
| 1 | inject_xff | requestheaders: set XFF from real peer IP |
| 2 | utiq_defense | requestheaders: detect/strip operator (Utiq) headers; short-circuit |
| 3 | protective_mode | requestheaders: strip tracker headers/cookies, spoof |
| 4 | privacy_guard (anti-track v2) | **request 204 / forge Set-Cookie (HMAC jar) / strip headers**; classify; file+key reads |
| 5 | ad_ghost | request **204** + candidate/per-visitor capture; response **CSS body inject**; allowlist; bg SQLite |
| 6 | media_cache | response synthesis from disk cache (range) |
| 7 | local_store | **tls_clienthello** read + async SQLite |
| 8 | social_graph | response cookie-id correlation + **body peek** + SQLite |
| 9 | inject_banner | request short-circuit **serve** /__toolbox/*; **streaming** body inject + buffered inject; CSP detect |
| 10 | dpi | async fire-and-forget POST (unix socket) |
| 11 | cookies | response Set-Cookie read → async POST |
| 12 | avatar | UA → async POST |
| 13 | ja4 | **raw TLS ClientHello** (cipher suites, extensions, ALPN) |
| 14 | soc_relay | events → async POST |
| 15 | cert_pin_detect | **TLS handshake-error** hook → learn ignore_hosts |
| 16 | media_stats | response headers → stats |
| 17 | tls_splice | **tls_clienthello SNI → connection passthrough** (ignore_connection) |
| 18 | (dpi dup/util) | — |
Capability buckets that discriminate the engines:
- **(C)** request short-circuit (return 204/synth without upstream) — ad_ghost, privacy_guard, inject_banner, media_cache.
- **(E)** **streaming** response body rewrite (inject into first chunk, no buffering) — inject_banner TTFB path.
- **(G)** **raw ClientHello introspection** for JA4 — ja4, local_store.
- **(H)** **TLS-layer SNI passthrough/splice** — tls_splice, cert_pin_detect, bypass list.
- **(I)** TLS handshake-error hook — cert_pin_detect.
- **(J)** async side-effects (socket POST / bg SQLite) — 7 addons.
## Engine assessment
### gomitmproxy (Go, AdguardTeam) — DROP
Purpose-built for ad-blocking MITM, but **last release v0.2.1 (2021), effectively
unmaintained**. Reusing an abandoned TLS-handling core for a security appliance
is the wrong bet. Cross off.
### martian (Google) / goproxy (elazarl) — Go, maintained
- Strong on **B/C/D/F/J** (modifier/handler APIs return custom responses, modify
headers/cookies/body; goroutines for async). Easy **arm64 cross-compile**
(`GOOS=linux GOARCH=arm64`), single static binary — great fit for the appliance.
- **Gaps:** **(G) JA4** — both abstract TLS at the HTTP layer; raw ClientHello
isn't exposed by the modifier API. *Workaround:* wrap the listener with our own
`crypto/tls` `Config.GetConfigForClient`/`GetCertificate` to capture the
ClientHello before handing to the proxy — feasible, extra code. **(E) streaming
inject** is manual (wrap the response body reader). **(H/I)** host-level
splice/cert-error handling is doable at the CONNECT layer.
- Verdict: pragmatic, lowest-friction toolchain, but JA4 + streaming need custom
glue.
### hudsucker (Rust, omjadas + ideamans fork) — maintained
- **Best technical coverage:** tokio/hyper async (**multi-core**), `HttpHandler`
(C/D/F), **streaming bodies (E)** native, WebSocket. Critically, **rustls
exposes the ClientHello** (Acceptor/`ClientHello` peek pre-handshake) → **JA4
(G) is clean**, and SNI-based **splice (H)** is natural.
- **Costs:** Rust **arm64 cross-compile friction** (no toolchain here; needs
`cross`/musl setup), and porting 18 addons + the anti-track HMAC-jar/classify
brain to Rust is the **highest re-implementation + re-validation effort**.
- Verdict: technically the strongest (only one covering JA4 + streaming cleanly),
but the heaviest port + ops.
### Squid + ssl-bump + ICAP — mature C, multi-process
- **Native wins:** ssl-bump forges from one root key (A), **peek-and-splice (H)
is literally tls_splice + the bypass list**, native cert-error handling (I),
multi-process scaling. ICAP REQMOD/RESPMOD covers **C/D/F** (204, body rewrite,
header/cookie mod) — ad_ghost/banner-buffer/poison can live in an ICAP service.
- **Gaps:** **(E) streaming** inject — ICAP buffers, no first-chunk inject.
**(G) JA4** — ICAP is post-decrypt HTTP; ClientHello isn't exposed to ICAP
(Squid logs its own TLS details, not via ICAP). Heavy **ops/config**; each ICAP
call is a round-trip; the anti-track HMAC-jar/poison + social-graph logic in an
ICAP service is awkward (still Python, still off-core for analysis).
- Verdict: least *custom proxy* code + native splice/cert handling, but loses
JA4 + streaming-banner and trades Python addons for Squid-config + an ICAP
service. Good if we drop JA4/streaming; otherwise a poor fit.
## Recommendation — **Go hot-path core + retained Python analysis sidecars** (hybrid)
Single-engine "rewrite everything in Rust" is the highest risk; Squid loses JA4 +
streaming. The lowest-risk path to multi-core that **preserves the
security-validated Python brain**:
1. **Go core** (goproxy/martian or a thin `net/http`+`crypto/tls` forging proxy)
owns the **hot path**: TLS forge (reusing `ca-wg`), SNI splice (H), the cheap
per-request rewrites — block 204 (ad_ghost/privacy_guard), header/cookie strip
(utiq/protective/anonymize), banner inject (E via body-reader wrap), serve
/__toolbox/*. Multi-core, one static arm64 binary.
2. **JA4 (G)** in Go via a `crypto/tls` ClientHello-capture shim (no Python).
3. **Heavy/off-path analysis stays Python sidecars** the Go core feeds
fire-and-forget over unix sockets (J): social-graph correlation, classify,
DB/report writers, SOC/DPI relays. These are already async + off the hot path,
so they don't need to be fast — and we DON'T re-validate the anti-track
HMAC-jar/poison + cookie-graph security logic in a new language.
4. The anti-track **poison** (forge Set-Cookie from the HMAC jar) is hot-path +
security-critical → port the *deterministic* jar/forge to Go (small, testable),
keep classify (which list a host is on) as data the Go core reads from the
learned/pure files (already file-based).
This gets multi-core on the hot path, keeps the risky brain in validated Python,
and only ports the small, mechanical, hot pieces. If JA4-in-Go proves painful, the
fallback is **hudsucker** (Rust) for the core (clean JA4) at higher port cost.
## Honest effort/risk
- **Weeks, multi-PR.** 18 addons; security-critical; production board.
- Must **shadow-run** the new core alongside mitmproxy (mirror a fraction of R3
traffic) and compare before any cutover. **Never** big-bang.
- Rollback = the nft fanout still points at the mitmproxy workers until the final
cutover flips the DNAT to the Go core's ports.
See the phased plan: `docs/superpowers/plans/2026-06-18-mitm-engine-migration.md`.

View File

@ -0,0 +1,3 @@
/sbxmitm
*.test
cmd/sbxmitm/sbxmitm

View File

@ -0,0 +1,60 @@
# secubox-toolbox-ng — Go MITM engine (migration spike, #662 Phase 1)
De-risking PoC for migrating the R3 toolbox MITM engine off Python **mitmproxy**
(GIL-bound, ~1 core) onto a multi-core **Go** core, **without losing the
18-addon feature set**. See:
- Analysis: `docs/superpowers/specs/2026-06-18-mitm-engine-migration-analysis.md`
- Phased plan: `docs/superpowers/plans/2026-06-18-mitm-engine-migration.md`
> **Status: Phase 1 — PoC only. NOT wired into the live R3 path.** The live
> tunnel still runs on the Python mitmproxy workers (8081-8084). This binary is
> a standalone CONNECT-proxy spike that proves the risky capabilities.
## What the PoC proves (the discriminating risks from the analysis)
- **CA-compat forging** — loads the *existing* `ca-wg/{ca.pem,key.pem}` and forges
per-host leaf certs the R3 clients already trust (no re-enroll). Cached per host.
- **request 204** — short-circuit block (ad_ghost / privacy_guard).
- **response body inject** — marker before `</head>`/`</body>` (banner / ad-CSS).
- **SNI splice** — raw passthrough, no MITM, by SNI suffix (tls_splice).
- **JA4 material capture**`crypto/tls` `GetCertificate` receives the
`ClientHelloInfo` (SNI, cipher suites, ALPN, TLS versions) → proves the `ja4`
addon's handshake fingerprint is reachable in Go (full JA4 extension-hash needs
a raw-ClientHello peek — Phase 4).
All stdlib (no external modules → builds offline). Tests are network-free
(localhost handshake + temp self-signed CA).
## Build & test
```sh
cd packages/secubox-toolbox-ng
go test ./... # network-free PoC tests
GOOS=linux GOARCH=arm64 go build -o sbxmitm ./cmd/sbxmitm # appliance target
```
## Try it (CONNECT proxy, against the board CA)
```sh
./sbxmitm --ca-cert /etc/secubox/toolbox/ca-wg/ca.pem \
--ca-key /etc/secubox/toolbox/ca-wg/key.pem --listen :8090
curl -x localhost:8090 --cacert /etc/secubox/toolbox/ca-wg/ca.pem https://doubleclick.net/ # → 204
curl -x localhost:8090 --cacert /etc/secubox/toolbox/ca-wg/ca.pem https://example.com/ # → body has the sbx-ng marker
# logs print `ja4 t0304_cNN_a... sni=...` per handshake
```
## Capability → engine map (recap)
Go covers request-204 / body-rewrite / header-cookie-mod / splice / async-sidecars
cleanly; JA4 needs the ClientHello shim (proven here); streaming inject + the
anti-track HMAC-jar/poison port land in Phase 3/4. Heavy analysis (social-graph,
classify, DB/report writers) stays in the existing Python sidecars, fed
fire-and-forget over unix sockets.
## Roadmap (do NOT cut over without the gates)
1. ✅ PoC (this) — forge + 204 + inject + splice + ClientHello capture, compiled + tested.
2. arm64 packaging + board bench on :8090 (no DNAT) — forge/throughput vs mitmproxy.
3. Hot-path feature parity (block lists + allowlist + own-infra guard, header/cookie strip, banner, splice) — parity harness vs the Python addons.
4. Analysis sidecars (unix-socket fire-and-forget) + anti-track HMAC-jar/forge port (exhaustively tested vs `privacy.py`).
5. **Shadow run** — mirror a fraction of R3, compare outputs. No client served yet.
6. **Cutover** — flip nft `numgen` fanout 8081-8084 → 8090-8093; mitmproxy stays up for instant rollback.
7. Decommission mitmproxy after a stable soak.
Rollback is an nft DNAT-map edit at every step; the Python engine is the live
path until Phase 6.

View File

@ -0,0 +1,311 @@
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
//
// SecuBox-Deb :: toolbox-ng :: forging MITM PoC (#662 Phase 1)
//
// De-risking spike for migrating the R3 MITM engine off Python mitmproxy onto a
// multi-core Go core. Pure standard library (no external modules) so it builds
// offline and cross-compiles to arm64 with `GOOS=linux GOARCH=arm64 go build`.
//
// It is NOT wired into the live R3 path. It proves the discriminating
// capabilities the engine analysis flagged as risky:
// - forge per-host leaf certs from the EXISTING ca-wg CA (client trust intact),
// - request short-circuit 204 (ad_ghost block),
// - response body inject (banner / ad-CSS),
// - SNI splice passthrough (tls_splice),
// - TLS ClientHello capture for JA4 (ja4 addon) via crypto/tls.GetCertificate.
//
// Runs as an HTTP CONNECT proxy for easy smoke-testing (`curl -x`). The live
// engine will run transparent (SO_ORIGINAL_DST) — same handlers, different
// accept path (Phase 2+).
package main
import (
"bytes"
"crypto"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/pem"
"flag"
"fmt"
"io"
"log"
"math/big"
"net"
"net/http"
"os"
"strings"
"sync"
"time"
)
// ── CA + per-host leaf forging ──────────────────────────────────────────────
// CA holds the loaded forging CA (reused from ca-wg) + a per-host leaf cache.
type CA struct {
cert *x509.Certificate
key crypto.Signer
mu sync.Mutex
cache map[string]*tls.Certificate
}
func loadCA(certPath, keyPath string) (*CA, error) {
cpem, err := os.ReadFile(certPath)
if err != nil {
return nil, fmt.Errorf("read ca cert: %w", err)
}
kpem, err := os.ReadFile(keyPath)
if err != nil {
return nil, fmt.Errorf("read ca key: %w", err)
}
cblk, _ := pem.Decode(cpem)
if cblk == nil {
return nil, fmt.Errorf("ca cert: no PEM block")
}
cert, err := x509.ParseCertificate(cblk.Bytes)
if err != nil {
return nil, fmt.Errorf("parse ca cert: %w", err)
}
kblk, _ := pem.Decode(kpem)
if kblk == nil {
return nil, fmt.Errorf("ca key: no PEM block")
}
key, err := parseKey(kblk.Bytes)
if err != nil {
return nil, fmt.Errorf("parse ca key: %w", err)
}
return &CA{cert: cert, key: key, cache: map[string]*tls.Certificate{}}, nil
}
func parseKey(der []byte) (crypto.Signer, error) {
if k, err := x509.ParsePKCS8PrivateKey(der); err == nil {
if s, ok := k.(crypto.Signer); ok {
return s, nil
}
}
if k, err := x509.ParsePKCS1PrivateKey(der); err == nil {
return k, nil
}
if k, err := x509.ParseECPrivateKey(der); err == nil {
return k, nil
}
return nil, fmt.Errorf("unsupported CA key format")
}
// forge returns a leaf cert for host signed by the CA, cached.
func (c *CA) forge(host string) (*tls.Certificate, error) {
host = strings.ToLower(strings.TrimSpace(host))
c.mu.Lock()
if tc, ok := c.cache[host]; ok {
c.mu.Unlock()
return tc, nil
}
c.mu.Unlock()
serial, _ := rand.Int(rand.Reader, new(big.Int).Lsh(big.NewInt(1), 128))
tmpl := &x509.Certificate{
SerialNumber: serial,
Subject: pkix.Name{CommonName: host},
NotBefore: time.Now().Add(-1 * time.Hour),
NotAfter: time.Now().Add(24 * time.Hour),
KeyUsage: x509.KeyUsageDigitalSignature | x509.KeyUsageKeyEncipherment,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
DNSNames: []string{host},
}
der, err := x509.CreateCertificate(rand.Reader, tmpl, c.cert, c.key.Public(), c.key)
if err != nil {
return nil, err
}
leaf, err := x509.ParseCertificate(der) // parsed cert has Raw populated (Verify needs it)
if err != nil {
return nil, err
}
tc := &tls.Certificate{Certificate: [][]byte{der, c.cert.Raw}, PrivateKey: c.key, Leaf: leaf}
c.mu.Lock()
c.cache[host] = tc
c.mu.Unlock()
return tc, nil
}
// ── Pure handler logic (the ported addon decisions) ─────────────────────────
type Policy struct {
AdHosts []string // ad_ghost: 204 these (suffix match)
SpliceHosts []string // tls_splice: passthrough, no MITM (suffix match)
Inject []byte // banner / ad-CSS marker injected before </head> or </body>
}
func suffixMatch(host string, pats []string) bool {
h := strings.ToLower(strings.TrimSpace(host))
for _, p := range pats {
p = strings.ToLower(p)
if h == p || strings.HasSuffix(h, "."+p) {
return true
}
}
return false
}
// action: "block" (204), "splice" (passthrough), or "mitm".
func (p Policy) action(host string) string {
if suffixMatch(host, p.SpliceHosts) {
return "splice"
}
if suffixMatch(host, p.AdHosts) {
return "block"
}
return "mitm"
}
// injectMarker inserts p.Inject before </head> (else </body>, else prepends).
func (p Policy) injectMarker(body []byte) []byte {
if len(p.Inject) == 0 || bytes.Contains(body, p.Inject) {
return body
}
for _, tag := range [][]byte{[]byte("</head>"), []byte("</body>")} {
if i := bytes.Index(bytes.ToLower(body), bytes.ToLower(tag)); i >= 0 {
out := make([]byte, 0, len(body)+len(p.Inject))
out = append(out, body[:i]...)
out = append(out, p.Inject...)
out = append(out, body[i:]...)
return out
}
}
return append(append([]byte{}, p.Inject...), body...)
}
// ── JA4 ClientHello capture (the Go-feasibility proof for the ja4 addon) ─────
// ja4ish builds a compact handshake fingerprint from the fields crypto/tls
// exposes in ClientHelloInfo (SNI, TLS versions, cipher count, ALPN). A FULL
// JA4 also needs the extension list, which requires a raw-ClientHello-bytes
// peek before stdlib parsing — feasible (Phase 4); this proves the material is
// reachable in Go without Python.
func ja4ish(h *tls.ClientHelloInfo) string {
maxVer := uint16(0)
for _, v := range h.SupportedVersions {
if v > maxVer {
maxVer = v
}
}
alpn := "none"
if len(h.SupportedProtos) > 0 {
alpn = h.SupportedProtos[0]
}
return fmt.Sprintf("t%04x_c%02d_a%s_sni=%s", maxVer, len(h.CipherSuites), alpn, h.ServerName)
}
// ── CONNECT-proxy MITM wiring ────────────────────────────────────────────────
type Proxy struct {
ca *CA
pol Policy
jaSink func(string) // JA4 observations (logged; a sidecar in prod)
}
func (px *Proxy) serverTLSConfig() *tls.Config {
return &tls.Config{
GetCertificate: func(h *tls.ClientHelloInfo) (*tls.Certificate, error) {
if px.jaSink != nil {
px.jaSink(ja4ish(h)) // capture handshake fingerprint
}
name := h.ServerName
if name == "" {
name = "unknown.local"
}
return px.ca.forge(name)
},
}
}
func (px *Proxy) handleConnect(w http.ResponseWriter, r *http.Request) {
host := r.URL.Hostname()
hj, ok := w.(http.Hijacker)
if !ok {
http.Error(w, "no hijack", 500)
return
}
client, _, err := hj.Hijack()
if err != nil {
return
}
defer client.Close()
io.WriteString(client, "HTTP/1.1 200 Connection Established\r\n\r\n")
if px.pol.action(host) == "splice" {
// passthrough: raw TCP to upstream, no TLS interception (tls_splice).
up, err := net.DialTimeout("tcp", r.URL.Host, 10*time.Second)
if err != nil {
return
}
defer up.Close()
go io.Copy(up, client)
io.Copy(client, up)
return
}
// MITM: TLS-terminate the client with a forged cert (+ ClientHello capture).
tconn := tls.Server(client, px.serverTLSConfig())
if err := tconn.Handshake(); err != nil {
return
}
defer tconn.Close()
br := newReader(tconn)
req, err := http.ReadRequest(br)
if err != nil {
return
}
req.URL.Scheme, req.URL.Host = "https", r.URL.Host
if px.pol.action(host) == "block" {
writeRaw(tconn, 204, "No Content", map[string]string{"X-SecuBox-Ng": "blocked"}, nil)
return
}
// proxy upstream, inject into HTML bodies.
up := &http.Client{Timeout: 30 * time.Second}
req.RequestURI = ""
resp, err := up.Do(req)
if err != nil {
writeRaw(tconn, 502, "Bad Gateway", nil, nil)
return
}
defer resp.Body.Close()
body, _ := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
if strings.Contains(resp.Header.Get("Content-Type"), "text/html") {
body = px.pol.injectMarker(body)
}
hdr := map[string]string{"Content-Type": resp.Header.Get("Content-Type")}
writeRaw(tconn, resp.StatusCode, resp.Status, hdr, body)
}
func main() {
caCert := flag.String("ca-cert", "/etc/secubox/toolbox/ca-wg/ca.pem", "CA cert PEM")
caKey := flag.String("ca-key", "/etc/secubox/toolbox/ca-wg/key.pem", "CA key PEM")
addr := flag.String("listen", ":8090", "CONNECT proxy listen addr")
flag.Parse()
ca, err := loadCA(*caCert, *caKey)
if err != nil {
log.Fatalf("CA load: %v", err)
}
px := &Proxy{
ca: ca,
pol: Policy{
AdHosts: []string{"doubleclick.net", "googlesyndication.com"},
SpliceHosts: []string{"googlevideo.com", "fbcdn.net"},
Inject: []byte("<!-- sbx-ng banner -->"),
},
jaSink: func(s string) { log.Printf("ja4 %s", s) },
}
srv := &http.Server{Addr: *addr, Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodConnect {
px.handleConnect(w, r)
return
}
http.Error(w, "CONNECT only (PoC)", 405)
})}
log.Printf("sbxmitm PoC listening on %s (CA %s)", *addr, *caCert)
log.Fatal(srv.ListenAndServe())
}

View File

@ -0,0 +1,156 @@
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
package main
import (
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/pem"
"math/big"
"net"
"os"
"path/filepath"
"sync"
"testing"
"time"
)
// genTestCA writes a self-signed CA (cert+key PEM) to dir, mirroring ca-wg.
func genTestCA(t *testing.T, dir string) (certPath, keyPath string) {
t.Helper()
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatal(err)
}
tmpl := &x509.Certificate{
SerialNumber: big.NewInt(1),
Subject: pkix.Name{CommonName: "SecuBox Test CA"},
NotBefore: time.Now().Add(-time.Hour),
NotAfter: time.Now().Add(24 * time.Hour),
IsCA: true,
KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageDigitalSignature,
BasicConstraintsValid: true,
}
der, err := x509.CreateCertificate(rand.Reader, tmpl, tmpl, key.Public(), key)
if err != nil {
t.Fatal(err)
}
certPath = filepath.Join(dir, "ca.pem")
keyPath = filepath.Join(dir, "key.pem")
cf, _ := os.Create(certPath)
pem.Encode(cf, &pem.Block{Type: "CERTIFICATE", Bytes: der})
cf.Close()
kder, _ := x509.MarshalPKCS8PrivateKey(key)
kf, _ := os.Create(keyPath)
pem.Encode(kf, &pem.Block{Type: "PRIVATE KEY", Bytes: kder})
kf.Close()
return certPath, keyPath
}
func TestForgeChainsToCA(t *testing.T) {
cp, kp := genTestCA(t, t.TempDir())
ca, err := loadCA(cp, kp)
if err != nil {
t.Fatalf("loadCA: %v", err)
}
leaf, err := ca.forge("ads.example.com")
if err != nil {
t.Fatalf("forge: %v", err)
}
pool := x509.NewCertPool()
pool.AddCert(ca.cert)
if _, err := leaf.Leaf.Verify(x509.VerifyOptions{Roots: pool, DNSName: "ads.example.com"}); err != nil {
t.Fatalf("forged leaf does not chain to CA / wrong SAN: %v", err)
}
leaf2, _ := ca.forge("ads.example.com")
if leaf2 != leaf {
t.Fatal("forge not cached")
}
}
func TestActionDecision(t *testing.T) {
p := Policy{AdHosts: []string{"doubleclick.net"}, SpliceHosts: []string{"googlevideo.com"}}
cases := map[string]string{
"ads.doubleclick.net": "block",
"doubleclick.net": "block",
"r1.googlevideo.com": "splice",
"news.example.com": "mitm",
"notdoubleclick.net": "mitm",
}
for host, want := range cases {
if got := p.action(host); got != want {
t.Errorf("action(%q)=%q want %q", host, got, want)
}
}
}
func TestInjectMarker(t *testing.T) {
p := Policy{Inject: []byte("<!--SBX-->")}
out := string(p.injectMarker([]byte("<html><head></head><body>hi</body></html>")))
if !contains(out, "<!--SBX--></head>") {
t.Fatalf("marker not injected before </head>: %s", out)
}
if string(p.injectMarker([]byte(out))) != out {
t.Fatal("inject not idempotent")
}
}
func contains(s, sub string) bool {
for i := 0; i+len(sub) <= len(s); i++ {
if s[i:i+len(sub)] == sub {
return true
}
}
return false
}
// TestClientHelloCaptureAndForge: a real localhost TLS handshake proves the Go
// core forges a per-SNI cert from the CA that the client trusts AND that the
// ClientHello (JA4 material) is captured.
func TestClientHelloCaptureAndForge(t *testing.T) {
cp, kp := genTestCA(t, t.TempDir())
ca, err := loadCA(cp, kp)
if err != nil {
t.Fatal(err)
}
var mu sync.Mutex
var captured string
px := &Proxy{ca: ca, jaSink: func(s string) { mu.Lock(); captured = s; mu.Unlock() }}
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
defer ln.Close()
go func() {
c, err := ln.Accept()
if err != nil {
return
}
s := tls.Server(c, px.serverTLSConfig())
s.Handshake()
s.Close()
}()
pool := x509.NewCertPool()
pool.AddCert(ca.cert)
conn, err := tls.Dial("tcp", ln.Addr().String(), &tls.Config{ServerName: "example.com", RootCAs: pool})
if err != nil {
t.Fatalf("client handshake against forged cert failed (CA not trusted / forge broken): %v", err)
}
conn.Close()
mu.Lock()
defer mu.Unlock()
if captured == "" {
t.Fatal("ClientHello not captured")
}
if !contains(captured, "sni=example.com") {
t.Fatalf("JA4 capture missing SNI: %q", captured)
}
t.Logf("captured JA4-ish: %s", captured)
}

View File

@ -0,0 +1,31 @@
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
package main
import (
"bufio"
"fmt"
"io"
"net"
)
func newReader(c net.Conn) *bufio.Reader { return bufio.NewReader(c) }
// writeRaw writes a minimal HTTP/1.1 response onto a (TLS) conn.
func writeRaw(c io.Writer, code int, status string, headers map[string]string, body []byte) {
if status == "" {
status = "OK"
}
fmt.Fprintf(c, "HTTP/1.1 %d %s\r\n", code, status)
fmt.Fprintf(c, "Content-Length: %d\r\n", len(body))
fmt.Fprintf(c, "Connection: close\r\n")
for k, v := range headers {
if v != "" {
fmt.Fprintf(c, "%s: %s\r\n", k, v)
}
}
io.WriteString(c, "\r\n")
if len(body) > 0 {
c.Write(body)
}
}

View File

@ -0,0 +1,3 @@
module github.com/CyberMind-FR/secubox-deb/secubox-toolbox-ng
go 1.22