mirror of
https://github.com/CyberMind-FR/secubox-deb.git
synced 2026-06-29 15:31:31 +00:00
Compare commits
No commits in common. "4ef6d3aa7680b606af1bf6eb9f33b7ebe6c51536" and "223f81ac636d6227db7f87cb9766f2f15c903f40" have entirely different histories.
4ef6d3aa76
...
223f81ac63
|
|
@ -3,51 +3,6 @@
|
|||
|
||||
---
|
||||
|
||||
## 2026-06-18 — #662 R3 CUTOVER to the Go MITM engine (PR #670) — LIVE + banner ported
|
||||
|
||||
- **Cutover executed and live.** The Go engine now serves **100% of R3 traffic**,
|
||||
replacing the Python mitmproxy workers. Found + fixed 4 blockers that made the dark
|
||||
package unable to serve the live path: (1) it forged with the wrong CA (ca-wg "WG CA"
|
||||
vs the "R3 CA" clients trust) → now uses the mitmproxy confdir bundle; (2) root-only
|
||||
key vs non-root user → R3 CA bundle is group-readable; (3) bound 127.0.0.1 vs the
|
||||
10.99.1.1 DNAT target → now binds 10.99.1.1; (4) ran CONNECT vs transparent → now
|
||||
`--transparent`. `loadCA` scans PEM blocks by type (combined cert+key bundle).
|
||||
- **Validated on real arm64 hardware** then rolled out gated: localhost forge against
|
||||
the real R3 CA → scoped-DNAT transparent capture → **canary slot 3 (~25%, dead-man
|
||||
armed)** → **widen to 100%**. At 100%: 0 restarts, 0 errors, ~64MB total
|
||||
(vs Python ~280-470MB), even round-robin, 142 distinct SNIs/75s.
|
||||
- **Banner ported** (the one regression the user caught — "no more banner but fast").
|
||||
Go now injects the real loader `<script src="/__toolbox/loader.js" data-mh=.. data-wg=..>`
|
||||
(guard-idempotent, R3 wg flag, mac_hash identity) and reverse-proxies
|
||||
`/__toolbox/loader.js`+`/__toolbox/bundle` to the portal (127.0.0.1:8088, fail-open),
|
||||
keeping bundle/level logic in Python. Verified live: loader injected + assets 200.
|
||||
- **Rollback** = one `nft replace` (Python workers kept warm). **Persistence gap**: the
|
||||
nft flip is a live edit, not yet in the drift-managed generator → reboot safely falls
|
||||
back to Python (workers enabled, banner intact). Phase 7 (decommission Python +
|
||||
persist nft) deferred to a soak'd follow-up.
|
||||
|
||||
## 2026-06-18 — #662 MITM engine migration: P5-prep + P6-prep (PRs #668, #669, all DARK)
|
||||
|
||||
- **P5-prep (PR #668).** Wired the ported `Decide`+jar into the Go engine's request/
|
||||
response handlers: `handleConnect` runs allow/splice/block/mitm; `anonymizeRequest`
|
||||
(strip operator/re-id headers + DNT/GPC) on every MITM'd flow; cookie-poison gated
|
||||
to mitm+tracker only (never allow/own-infra; fail-closed-to-clean; benign cookies +
|
||||
Set-Cookie attrs preserved). New `secubox-toolbox-ng` debian pkg builds an arm64
|
||||
`.deb` shipping `/usr/sbin/sbxmitm` + a **DISABLED** `worker@.service` on `:809%i`
|
||||
(no enable/start, no nft). 22 Go tests, reviewed APPROVED.
|
||||
- **P6-prep (PR #669).** No-traffic build-out of the live transparent path, still DARK.
|
||||
`machash.go` ports `mac_hash_of`/`_wg_hash_of` (WG peers → `sha256(pubkey)[:16]`,
|
||||
mtime-cached, fail-open) wired into `clientHashFromConn`, cross-engine parity vs
|
||||
Python (anti-rig verified). Transparent `SO_ORIGINAL_DST` accept (`--transparent`,
|
||||
default off): peeks ClientHello SNI WITHOUT decrypting → Decide → **splice = true raw
|
||||
passthrough** (never `tls.Server`) / else forge via replayable `prefixConn`; upstream
|
||||
TLS verifies by SNI, pins captured ip:port. Two-stage review caught + fixed a
|
||||
splice-decrypt defect. Builds linux/arm64+amd64+darwin, vet clean, race green, Python
|
||||
parity 10 passed. CONNECT path + poison gate byte-unchanged.
|
||||
- **Engine now functionally complete + packaged, entirely DARK.** Remaining work =
|
||||
the production DEPLOYMENT phases (shadow → cutover → decommission), which touch live
|
||||
R3 traffic and are deferred to a deliberate watched session — NOT chained off "go".
|
||||
|
||||
## 2026-06-18 — #656 Ad Intelligence (PR #657, toolbox 2.6.56) + splice reverted
|
||||
|
||||
- **Ad Intelligence — learn/act/measure.** `ad_ghost` now records every
|
||||
|
|
|
|||
|
|
@ -1,167 +0,0 @@
|
|||
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
//
|
||||
// SecuBox-Deb :: toolbox-ng :: transparency-banner loader inject (#662)
|
||||
//
|
||||
// Ports the LIVE transparency-banner injection from the authoritative Python
|
||||
// addon (../secubox-toolbox/mitmproxy_addons/inject_banner.py) into the Go
|
||||
// engine. With stream_inject ON the Python addon injects a tiny LOADER
|
||||
// <script src="/__toolbox/loader.js" data-mh=.. data-wg=.. async></script> and
|
||||
// SERVES /__toolbox/loader.js + /__toolbox/bundle itself for ANY origin (the
|
||||
// injected same-origin URL resolves to whatever MITM'd host the client is on).
|
||||
//
|
||||
// To avoid re-porting the bundle/level business logic to Go, this engine
|
||||
// REVERSE-PROXIES /__toolbox/* to the portal (default http://127.0.0.1:8088),
|
||||
// which already serves both endpoints. The injection (injectLoader) mirrors the
|
||||
// Python _loader_script + _LoaderInjector byte-for-byte on the tag shape and
|
||||
// placement; the guard makes it idempotent (matches Python _GUARD).
|
||||
//
|
||||
// Pure standard library — no external modules.
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// bannerGuard matches the Python _GUARD ("__GONDWANA_MITM_BANNER__"): an HTML
|
||||
// comment marker that makes injection idempotent across stream chunks / repeat
|
||||
// passes. If the body already contains it, we never inject again.
|
||||
const bannerGuard = "__GONDWANA_MITM_BANNER__"
|
||||
|
||||
// asciiOnly drops every non-ASCII byte from s, mirroring the Python
|
||||
// `s.encode("ascii", "ignore")` used on the client hash before it lands in the
|
||||
// data-mh attribute. The clientHash is normally a hex mac_hash (already ASCII),
|
||||
// but a non-WG fallback could carry odd bytes — strip defensively.
|
||||
func asciiOnly(s string) string {
|
||||
var b strings.Builder
|
||||
b.Grow(len(s))
|
||||
for i := 0; i < len(s); i++ {
|
||||
if s[i] < 0x80 {
|
||||
b.WriteByte(s[i])
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// loaderScript builds the loader <script> tag EXACTLY like the Python
|
||||
// _loader_script: a guard comment followed by the same-origin loader.js tag
|
||||
// carrying the client identity (data-mh) + WG flag (data-wg). wg → "1" else "0";
|
||||
// clientHash is ascii-sanitised. The src is same-origin so it resolves to the
|
||||
// MITM'd host and is intercepted by the /__toolbox/* short-circuit.
|
||||
func loaderScript(clientHash string, wg bool) []byte {
|
||||
wgVal := "0"
|
||||
if wg {
|
||||
wgVal = "1"
|
||||
}
|
||||
mh := asciiOnly(clientHash)
|
||||
tag := `<script src="/__toolbox/loader.js" data-mh="` + mh +
|
||||
`" data-wg="` + wgVal + `" async></script>`
|
||||
return []byte("<!-- " + bannerGuard + " -->" + tag)
|
||||
}
|
||||
|
||||
// injectLoader inserts the loader <script> into an HTML body once. Placement
|
||||
// mirrors the Python _LoaderInjector.__call__:
|
||||
// - guard idempotency: if the body already contains bannerGuard → unchanged.
|
||||
// - find the first (case-insensitive) "<head"; if present, find the next ">"
|
||||
// after it and insert the tag right after that ">".
|
||||
// - else find the first "<body" and insert the tag right BEFORE it.
|
||||
// - if neither is present → return the body unchanged (no inject).
|
||||
func injectLoader(body []byte, clientHash string, wg bool) []byte {
|
||||
if bytes.Contains(body, []byte(bannerGuard)) {
|
||||
return body
|
||||
}
|
||||
script := loaderScript(clientHash, wg)
|
||||
low := bytes.ToLower(body)
|
||||
|
||||
if h := bytes.Index(low, []byte("<head")); h >= 0 {
|
||||
if j := bytes.IndexByte(body[h:], '>'); j >= 0 {
|
||||
at := h + j + 1
|
||||
out := make([]byte, 0, len(body)+len(script))
|
||||
out = append(out, body[:at]...)
|
||||
out = append(out, script...)
|
||||
out = append(out, body[at:]...)
|
||||
return out
|
||||
}
|
||||
}
|
||||
if b := bytes.Index(low, []byte("<body")); b >= 0 {
|
||||
out := make([]byte, 0, len(body)+len(script))
|
||||
out = append(out, body[:b]...)
|
||||
out = append(out, script...)
|
||||
out = append(out, body[b:]...)
|
||||
return out
|
||||
}
|
||||
return body
|
||||
}
|
||||
|
||||
// ── /__toolbox/* reverse-proxy to the portal ─────────────────────────────────
|
||||
|
||||
// isToolboxAssetPath reports whether a request path is one of the banner assets
|
||||
// the engine must serve itself (by reverse-proxying to the portal) for ANY
|
||||
// origin. STARTSWITH (not exact) is REQUIRED: the path includes the query
|
||||
// string and the bundle is fetched as /__toolbox/bundle?mh=..&wg=.. — an exact
|
||||
// match would never fire. Mirrors the Python request() p.startswith(...) checks.
|
||||
func isToolboxAssetPath(path string) bool {
|
||||
return strings.HasPrefix(path, "/__toolbox/loader.js") ||
|
||||
strings.HasPrefix(path, "/__toolbox/bundle")
|
||||
}
|
||||
|
||||
// portalTargetURL builds the absolute portal URL for an intercepted asset
|
||||
// request: <portal-base> + the original request path (which already includes
|
||||
// the query string). The portal base's trailing slash is trimmed so the result
|
||||
// never doubles the leading "/" of the path.
|
||||
func portalTargetURL(portal, pathWithQuery string) string {
|
||||
return strings.TrimRight(portal, "/") + pathWithQuery
|
||||
}
|
||||
|
||||
// portalClient is the short-timeout HTTP client used to fetch banner assets from
|
||||
// the portal. Shared (stdlib http.Client is goroutine-safe) so we don't churn
|
||||
// connections per request.
|
||||
var portalClient = &http.Client{
|
||||
Timeout: 5 * time.Second,
|
||||
// Never follow redirects: the portal is a fixed loopback base, so not
|
||||
// following 3xx means a misbehaving/compromised portal can't steer the
|
||||
// worker into fetching an arbitrary outbound host (SSRF hygiene). The 3xx
|
||||
// is relayed to the client as-is.
|
||||
CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse },
|
||||
}
|
||||
|
||||
// servePortalAsset reverse-proxies a /__toolbox/* request to the portal and
|
||||
// writes the portal's response (status + Content-Type + Cache-Control + body)
|
||||
// back to the client over the already-established (TLS) conn. It returns true
|
||||
// once it has written a response — the caller MUST NOT then forward upstream.
|
||||
//
|
||||
// Fail-open: if the portal request errors (portal down, timeout, non-2xx read
|
||||
// failure) we serve a minimal 204 No Content so the navigation is never broken,
|
||||
// and log at most a warning. We never 502 the whole page over a banner asset.
|
||||
func servePortalAsset(w io.Writer, portal, pathWithQuery string) bool {
|
||||
target := portalTargetURL(portal, pathWithQuery)
|
||||
resp, err := portalClient.Get(target)
|
||||
if err != nil {
|
||||
log.Printf("portal asset fetch failed for %s: %v", target, err)
|
||||
writeRaw(w, 204, "No Content", nil, nil)
|
||||
return true
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
body, rerr := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
|
||||
if rerr != nil {
|
||||
log.Printf("portal asset read failed for %s: %v", target, rerr)
|
||||
writeRaw(w, 204, "No Content", nil, nil)
|
||||
return true
|
||||
}
|
||||
headers := map[string]string{}
|
||||
if ct := resp.Header.Get("Content-Type"); ct != "" {
|
||||
headers["Content-Type"] = ct
|
||||
}
|
||||
if cc := resp.Header.Get("Cache-Control"); cc != "" {
|
||||
headers["Cache-Control"] = cc
|
||||
}
|
||||
// writeRaw formats "HTTP/1.1 <code> <status>"; pass only the reason phrase
|
||||
// (not resp.Status, which already embeds the code → would double it).
|
||||
writeRaw(w, resp.StatusCode, http.StatusText(resp.StatusCode), headers, body)
|
||||
return true
|
||||
}
|
||||
|
|
@ -1,132 +0,0 @@
|
|||
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
//
|
||||
// SecuBox-Deb :: toolbox-ng :: transparency-banner loader inject tests (#662)
|
||||
//
|
||||
// Mirrors the authoritative Python tests of inject_banner._loader_script /
|
||||
// _LoaderInjector / the /__toolbox/* request() short-circuit. The portal
|
||||
// reverse-proxy integration (a live portal) is validated on-board, NOT here;
|
||||
// these unit tests cover the pure injection logic + the path/url helpers.
|
||||
package main
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestInjectLoaderGuardIdempotent(t *testing.T) {
|
||||
// Body already carrying the guard → returned byte-for-byte unchanged.
|
||||
body := []byte("<html><head><!-- " + bannerGuard + " --><script></script></head><body>hi</body></html>")
|
||||
out := injectLoader(body, "abc123", false)
|
||||
if string(out) != string(body) {
|
||||
t.Fatalf("guarded body must be unchanged.\n got: %s", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInjectLoaderHeadInsertion(t *testing.T) {
|
||||
body := []byte(`<html><head lang="en"><title>x</title></head><body>hi</body></html>`)
|
||||
out := string(injectLoader(body, "deadbeef", true))
|
||||
// The tag must land right AFTER the first <head ...>'s closing '>'.
|
||||
headOpen := `<head lang="en">`
|
||||
idx := strings.Index(out, headOpen)
|
||||
if idx < 0 {
|
||||
t.Fatalf("head open lost: %s", out)
|
||||
}
|
||||
after := out[idx+len(headOpen):]
|
||||
wantTag := `<!-- ` + bannerGuard + ` --><script src="/__toolbox/loader.js" data-mh="deadbeef" data-wg="1" async></script>`
|
||||
if !strings.HasPrefix(after, wantTag) {
|
||||
t.Fatalf("tag not inserted right after <head>'s '>'.\n got: %s", after)
|
||||
}
|
||||
// <title> must still follow the injected tag (we inserted, not replaced).
|
||||
if !strings.Contains(out, wantTag+`<title>x</title>`) {
|
||||
t.Fatalf("original head content displaced: %s", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInjectLoaderBodyFallback(t *testing.T) {
|
||||
// No <head> → insert right BEFORE the first <body>.
|
||||
body := []byte(`<html><body class="x">hi</body></html>`)
|
||||
out := string(injectLoader(body, "cafe", false))
|
||||
wantTag := `<!-- ` + bannerGuard + ` --><script src="/__toolbox/loader.js" data-mh="cafe" data-wg="0" async></script>`
|
||||
if !strings.Contains(out, wantTag+`<body class="x">`) {
|
||||
t.Fatalf("tag not inserted right before <body>.\n got: %s", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInjectLoaderNeitherHeadNorBody(t *testing.T) {
|
||||
body := []byte(`<p>just a fragment</p>`)
|
||||
out := injectLoader(body, "x", true)
|
||||
if string(out) != string(body) {
|
||||
t.Fatalf("no head/body → must be unchanged.\n got: %s", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInjectLoaderWGAttr(t *testing.T) {
|
||||
cases := []struct {
|
||||
wg bool
|
||||
want string
|
||||
}{
|
||||
{true, `data-wg="1"`},
|
||||
{false, `data-wg="0"`},
|
||||
}
|
||||
for _, c := range cases {
|
||||
out := string(injectLoader([]byte(`<head></head>`), "mh1", c.wg))
|
||||
if !strings.Contains(out, c.want) {
|
||||
t.Fatalf("wg=%v: want %q in %s", c.wg, c.want, out)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestInjectLoaderNonASCIIHashStripped(t *testing.T) {
|
||||
// Non-ascii bytes in the client hash are dropped (Python .encode("ascii","ignore")).
|
||||
out := string(injectLoader([]byte(`<head></head>`), "abécÿ12", false))
|
||||
if !strings.Contains(out, `data-mh="abc12"`) {
|
||||
t.Fatalf("non-ascii bytes not stripped: %s", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInjectLoaderHeadCaseInsensitive(t *testing.T) {
|
||||
body := []byte(`<HTML><HEAD></HEAD><BODY>hi</BODY></HTML>`)
|
||||
out := string(injectLoader(body, "z", false))
|
||||
if !strings.Contains(out, `<HEAD><!-- `+bannerGuard) {
|
||||
t.Fatalf("case-insensitive <HEAD> match failed: %s", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsToolboxAssetPath(t *testing.T) {
|
||||
cases := []struct {
|
||||
path string
|
||||
want bool
|
||||
}{
|
||||
{"/__toolbox/loader.js", true},
|
||||
{"/__toolbox/loader.js?v=2", true},
|
||||
{"/__toolbox/bundle", true},
|
||||
{"/__toolbox/bundle?mh=abc&wg=1", true},
|
||||
{"/__toolbox/other", false},
|
||||
{"/index.html", false},
|
||||
{"/", false},
|
||||
{"", false},
|
||||
{"/__toolboxbundle", false},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := isToolboxAssetPath(c.path); got != c.want {
|
||||
t.Errorf("isToolboxAssetPath(%q) = %v, want %v", c.path, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPortalTargetURL(t *testing.T) {
|
||||
cases := []struct {
|
||||
portal, path, want string
|
||||
}{
|
||||
{"http://127.0.0.1:8088", "/__toolbox/loader.js", "http://127.0.0.1:8088/__toolbox/loader.js"},
|
||||
{"http://127.0.0.1:8088", "/__toolbox/bundle?mh=abc&wg=1", "http://127.0.0.1:8088/__toolbox/bundle?mh=abc&wg=1"},
|
||||
// Trailing slash on the portal base must not double up.
|
||||
{"http://127.0.0.1:8088/", "/__toolbox/loader.js", "http://127.0.0.1:8088/__toolbox/loader.js"},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := portalTargetURL(c.portal, c.path); got != c.want {
|
||||
t.Errorf("portalTargetURL(%q,%q) = %q, want %q", c.portal, c.path, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -61,21 +61,17 @@ func loadCA(certPath, keyPath string) (*CA, error) {
|
|||
if err != nil {
|
||||
return nil, fmt.Errorf("read ca key: %w", err)
|
||||
}
|
||||
// Scan for the right block TYPE rather than assuming position: the live R3
|
||||
// CA the toolbox forges with (mitmproxy confdir `mitmproxy-ca.pem`) is a
|
||||
// COMBINED cert+key bundle, and --ca-key may point at it. Tolerate cert and
|
||||
// key co-residing in either file, in any order.
|
||||
cblk := firstPEMBlock(cpem, func(b *pem.Block) bool { return b.Type == "CERTIFICATE" })
|
||||
cblk, _ := pem.Decode(cpem)
|
||||
if cblk == nil {
|
||||
return nil, fmt.Errorf("ca cert: no CERTIFICATE PEM block")
|
||||
return nil, fmt.Errorf("ca cert: no PEM block")
|
||||
}
|
||||
cert, err := x509.ParseCertificate(cblk.Bytes)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("parse ca cert: %w", err)
|
||||
}
|
||||
kblk := firstPEMBlock(kpem, func(b *pem.Block) bool { return strings.Contains(b.Type, "PRIVATE KEY") })
|
||||
kblk, _ := pem.Decode(kpem)
|
||||
if kblk == nil {
|
||||
return nil, fmt.Errorf("ca key: no PRIVATE KEY PEM block")
|
||||
return nil, fmt.Errorf("ca key: no PEM block")
|
||||
}
|
||||
key, err := parseKey(kblk.Bytes)
|
||||
if err != nil {
|
||||
|
|
@ -84,22 +80,6 @@ func loadCA(certPath, keyPath string) (*CA, error) {
|
|||
return &CA{cert: cert, key: key, cache: map[string]*tls.Certificate{}}, nil
|
||||
}
|
||||
|
||||
// firstPEMBlock returns the first PEM block in data satisfying want, or nil.
|
||||
// Used to pull a specific block (CERTIFICATE / PRIVATE KEY) out of a file that
|
||||
// may hold several (e.g. mitmproxy's combined CA bundle).
|
||||
func firstPEMBlock(data []byte, want func(*pem.Block) bool) *pem.Block {
|
||||
for {
|
||||
blk, rest := pem.Decode(data)
|
||||
if blk == nil {
|
||||
return nil
|
||||
}
|
||||
if want(blk) {
|
||||
return blk
|
||||
}
|
||||
data = rest
|
||||
}
|
||||
}
|
||||
|
||||
func parseKey(der []byte) (crypto.Signer, error) {
|
||||
if k, err := x509.ParsePKCS8PrivateKey(der); err == nil {
|
||||
if s, ok := k.(crypto.Signer); ok {
|
||||
|
|
@ -202,7 +182,6 @@ type Proxy struct {
|
|||
jaSink func(string) // JA4 observations (logged; a sidecar in prod)
|
||||
jarKey []byte // anti-track HMAC fake-identity seed (nil → poison off)
|
||||
poison bool // master gate: poison tracker Set-Cookies (default on when jarKey present)
|
||||
portal string // portal base URL for /__toolbox/* reverse-proxy (banner assets)
|
||||
}
|
||||
|
||||
func (px *Proxy) serverTLSConfig() *tls.Config {
|
||||
|
|
@ -259,8 +238,7 @@ func (px *Proxy) handleConnect(w http.ResponseWriter, r *http.Request) {
|
|||
|
||||
// Shared post-TLS pipeline. CONNECT dials upstream by the request URL host
|
||||
// (req.URL.Host set inside), so dialHost is "" → mitmPipeline derives it.
|
||||
// CONNECT PoC is never an R3 WG client → wg=false.
|
||||
px.mitmPipeline(tconn, client, host, verdict, "", false)
|
||||
px.mitmPipeline(tconn, client, host, verdict, "")
|
||||
}
|
||||
|
||||
// mitmPipeline runs the shared post-TLS-handshake MITM logic used by BOTH the
|
||||
|
|
@ -278,9 +256,7 @@ func (px *Proxy) handleConnect(w http.ResponseWriter, r *http.Request) {
|
|||
// CONNECT semantics: dial by req.URL.Host (the request URL / host). Non-""
|
||||
// → transparent: TCP-connect the captured original-dst while doing TLS with
|
||||
// ServerName=host and verifying the cert against host (not the bare IP).
|
||||
// - wg : the client is an R3 WireGuard peer (10.99.1.0/24); threaded
|
||||
// into the injected loader's data-wg attribute. CONNECT path passes false.
|
||||
func (px *Proxy) mitmPipeline(tconn *tls.Conn, rawClient net.Conn, host, verdict, dialHost string, wg bool) {
|
||||
func (px *Proxy) mitmPipeline(tconn *tls.Conn, rawClient net.Conn, host, verdict, dialHost string) {
|
||||
br := newReader(tconn)
|
||||
req, err := http.ReadRequest(br)
|
||||
if err != nil {
|
||||
|
|
@ -290,16 +266,6 @@ func (px *Proxy) mitmPipeline(tconn *tls.Conn, rawClient net.Conn, host, verdict
|
|||
if req.URL.Host == "" {
|
||||
req.URL.Host = host
|
||||
}
|
||||
|
||||
// #636/#662 — serve the banner loader + bundle for ANY origin so the injected
|
||||
// <script src="/__toolbox/loader.js"> resolves (R3 clients hit arbitrary
|
||||
// hosts whose origin can't serve /__toolbox/*). Short-circuit BEFORE dialing
|
||||
// the real upstream by reverse-proxying to the portal. Mirrors the Python
|
||||
// InjectBanner.request() startswith checks (path includes the query string).
|
||||
if isToolboxAssetPath(req.URL.RequestURI()) {
|
||||
servePortalAsset(tconn, px.portal, req.URL.RequestURI())
|
||||
return
|
||||
}
|
||||
// Transparent: the upstream request must carry the SNI host (for Host header,
|
||||
// SNI, and cert verification); the actual TCP dial is pinned to the captured
|
||||
// original-dst by transparentTransport. We do NOT put the bare ip:port in
|
||||
|
|
@ -353,12 +319,8 @@ func (px *Proxy) mitmPipeline(tconn *tls.Conn, rawClient net.Conn, host, verdict
|
|||
}
|
||||
|
||||
body, _ := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
|
||||
// Inject the transparency-banner loader only on 2xx text/html responses
|
||||
// (mirrors the Python addon, which skips non-200). The loader's same-origin
|
||||
// <script src="/__toolbox/loader.js"> is served by the short-circuit above.
|
||||
if resp.StatusCode >= 200 && resp.StatusCode < 300 &&
|
||||
strings.Contains(resp.Header.Get("Content-Type"), "text/html") {
|
||||
body = injectLoader(body, clientHash, wg)
|
||||
if strings.Contains(resp.Header.Get("Content-Type"), "text/html") {
|
||||
body = px.pol.injectMarker(body)
|
||||
}
|
||||
writeResponse(tconn, resp, body)
|
||||
}
|
||||
|
|
@ -393,8 +355,6 @@ func main() {
|
|||
"poison tracking Set-Cookies on MITM'd tracker flows (needs --jar-key; never touches allow/own-infra)")
|
||||
transparent := flag.Bool("transparent", false,
|
||||
"transparent mode: accept nft-DNAT'd conns + recover SO_ORIGINAL_DST (live R3); default is the CONNECT proxy PoC")
|
||||
portal := flag.String("portal", "http://127.0.0.1:8088",
|
||||
"portal base URL; /__toolbox/loader.js + /__toolbox/bundle are reverse-proxied here (banner assets, served for any MITM'd origin)")
|
||||
flag.Parse()
|
||||
ca, err := loadCA(*caCert, *caKey)
|
||||
if err != nil {
|
||||
|
|
@ -420,7 +380,6 @@ func main() {
|
|||
jaSink: func(s string) { log.Printf("ja4 %s", s) },
|
||||
jarKey: jarKey,
|
||||
poison: *poison,
|
||||
portal: *portal,
|
||||
}
|
||||
if *transparent {
|
||||
// Transparent R3 mode: raw accept loop, each conn carries its pre-DNAT
|
||||
|
|
|
|||
|
|
@ -72,65 +72,6 @@ func TestForgeChainsToCA(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
// TestLoadCACombinedPEM proves loadCA pulls the right blocks out of a COMBINED
|
||||
// cert+key bundle — the real shape of mitmproxy's confdir `mitmproxy-ca.pem`,
|
||||
// which the live R3 CA uses and the worker unit points --ca-key at. mitmproxy
|
||||
// writes the PRIVATE KEY block first, then the CERTIFICATE; loadCA must scan by
|
||||
// type, not position.
|
||||
func TestLoadCACombinedPEM(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
tmpl := &x509.Certificate{
|
||||
SerialNumber: big.NewInt(7),
|
||||
Subject: pkix.Name{CommonName: "Gondwana ToolBoX R3 CA (test)"},
|
||||
NotBefore: time.Now().Add(-time.Hour),
|
||||
NotAfter: time.Now().Add(24 * time.Hour),
|
||||
IsCA: true,
|
||||
KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageDigitalSignature,
|
||||
BasicConstraintsValid: true,
|
||||
}
|
||||
der, err := x509.CreateCertificate(rand.Reader, tmpl, tmpl, key.Public(), key)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
kder, _ := x509.MarshalPKCS8PrivateKey(key)
|
||||
keyPEM := pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: kder})
|
||||
certPEM := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der})
|
||||
|
||||
// mitmproxy-ca.pem layout: key THEN cert in one file.
|
||||
combined := filepath.Join(dir, "mitmproxy-ca.pem")
|
||||
if err := os.WriteFile(combined, append(append([]byte{}, keyPEM...), certPEM...), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// mitmproxy-ca-cert.pem: cert only.
|
||||
certOnly := filepath.Join(dir, "mitmproxy-ca-cert.pem")
|
||||
if err := os.WriteFile(certOnly, certPEM, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// The unit's exact arg shape: --ca-cert <cert-only> --ca-key <combined>.
|
||||
ca, err := loadCA(certOnly, combined)
|
||||
if err != nil {
|
||||
t.Fatalf("loadCA(cert-only, combined): %v", err)
|
||||
}
|
||||
leaf, err := ca.forge("ads.example.com")
|
||||
if err != nil {
|
||||
t.Fatalf("forge: %v", err)
|
||||
}
|
||||
pool := x509.NewCertPool()
|
||||
pool.AddCert(ca.cert)
|
||||
if _, err := leaf.Leaf.Verify(x509.VerifyOptions{Roots: pool, DNSName: "ads.example.com"}); err != nil {
|
||||
t.Fatalf("forged leaf does not chain to combined-PEM CA: %v", err)
|
||||
}
|
||||
// Belt-and-braces: the combined file works as BOTH cert and key source.
|
||||
if _, err := loadCA(combined, combined); err != nil {
|
||||
t.Fatalf("loadCA(combined, combined): %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// NOTE (#662 Phase 3): the old TestActionDecision drove the removed hardcoded
|
||||
// Policy{AdHosts, SpliceHosts} fields. The decision surface now loads from
|
||||
// disk (LoadPolicy) and mirrors the Python addons; coverage moved to
|
||||
|
|
|
|||
|
|
@ -29,7 +29,6 @@ import (
|
|||
"io"
|
||||
"log"
|
||||
"net"
|
||||
"strings"
|
||||
"syscall"
|
||||
"unsafe"
|
||||
)
|
||||
|
|
@ -340,13 +339,6 @@ func (px *Proxy) handleTransparent(client net.Conn) {
|
|||
if !ok {
|
||||
return // transparent mode only accepts raw TCP conns
|
||||
}
|
||||
// R3 WG client? The data-wg attribute of the injected loader mirrors the
|
||||
// Python _loader_script (ip.startswith("10.99.1.")) — derived from the same
|
||||
// client conn peer IP that feeds clientHashFromConn.
|
||||
wg := false
|
||||
if peer, _, perr := net.SplitHostPort(client.RemoteAddr().String()); perr == nil {
|
||||
wg = strings.HasPrefix(peer, "10.99.1.")
|
||||
}
|
||||
dstHost, dstPort, err := origDst(tcp)
|
||||
if err != nil {
|
||||
return // no original-dst (not DNAT'd) → drop; nothing safe to do
|
||||
|
|
@ -394,5 +386,5 @@ func (px *Proxy) handleTransparent(client net.Conn) {
|
|||
return
|
||||
}
|
||||
defer tconn.Close()
|
||||
px.mitmPipeline(tconn, client, decisionHost, verdict, dialAddr, wg)
|
||||
px.mitmPipeline(tconn, client, decisionHost, verdict, dialAddr)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,24 +1,3 @@
|
|||
secubox-toolbox-ng (0.1.2-1~bookworm1) bookworm; urgency=medium
|
||||
|
||||
* banner: port the real transparency-banner inject — inject the loader
|
||||
<script src="/__toolbox/loader.js" data-mh=.. data-wg=..> (guard-idempotent,
|
||||
R3 wg flag, mac_hash identity) and reverse-proxy /__toolbox/loader.js +
|
||||
/__toolbox/bundle to the portal (127.0.0.1:8088), replacing the invisible
|
||||
marker comment. Fail-open to 204. (ref #662)
|
||||
|
||||
-- Gerald KERMA <devel@cybermind.fr> Wed, 18 Jun 2026 19:20:00 +0000
|
||||
|
||||
secubox-toolbox-ng (0.1.1-1~bookworm1) bookworm; urgency=medium
|
||||
|
||||
* worker@ unit: forge with the LIVE R3 CA clients trust (mitmproxy confdir
|
||||
bundle, group-readable) instead of the root-only ca-wg WG-CA key; bind
|
||||
transparent on 10.99.1.1:809%i (the nft R3 DNAT target) instead of CONNECT
|
||||
on 127.0.0.1; add wg-quick@wg-toolbox dependency. (ref #662)
|
||||
* loadCA: scan PEM blocks by type so a combined cert+key bundle
|
||||
(mitmproxy-ca.pem) is accepted for --ca-key. (ref #662)
|
||||
|
||||
-- Gerald KERMA <devel@cybermind.fr> Wed, 18 Jun 2026 19:00:00 +0000
|
||||
|
||||
secubox-toolbox-ng (0.1.0-1~bookworm1) bookworm; urgency=medium
|
||||
|
||||
* Initial packaging of the Go MITM engine migration target (#662 Phase 5-prep).
|
||||
|
|
|
|||
|
|
@ -8,45 +8,31 @@
|
|||
# (secubox-toolbox-mitm-wg-worker@{1..4}, ports 8081-8084) until the cutover is
|
||||
# performed manually.
|
||||
#
|
||||
# Mirrors the Python worker@ fanout: each %i ∈ {1..4} listens TRANSPARENT on
|
||||
# 10.99.1.1:809%i — the SAME wg-toolbox interface IP the nft R3 DNAT targets
|
||||
# (`iif wg-toolbox tcp dport 443/80 → 10.99.1.1:numgen inc mod 4 → 808{1..4}`),
|
||||
# on 809%i ports so the Go and Python fleets coexist during a side-by-side
|
||||
# canary. The engine recovers the original destination via SO_ORIGINAL_DST
|
||||
# (works for this non-root user under NoNewPrivileges, same as mitmdump).
|
||||
# Mirrors the Python worker@ fanout: each %i ∈ {1..4} listens on 127.0.0.1:809%i
|
||||
# (distinct from the Python 808%i ports so both fleets can coexist during a
|
||||
# side-by-side cutover validation). Enable ONLY at Phase 6:
|
||||
#
|
||||
# Forges with the LIVE R3 CA clients already trust — mitmproxy's confdir bundle
|
||||
# (CN "Gondwana ToolBoX R3 CA"), group-readable by secubox-toolbox — NOT the
|
||||
# root-only ca-wg key.pem (CN "WG CA"), which clients do NOT trust.
|
||||
# systemctl enable --now secubox-toolbox-ng-worker@{1,2,3,4}.service
|
||||
# # then re-point the nft DNAT at 809%i and retire the Python workers
|
||||
#
|
||||
# Enable ONLY at Phase 6 canary:
|
||||
#
|
||||
# systemctl enable --now secubox-toolbox-ng-worker@1.service # one slot first
|
||||
# # canary: nft ... map { ... 3 : 8091 } (was 3:8084), watch, then widen
|
||||
#
|
||||
# Rollback: re-point the nft DNAT map slot back at the Python 808%i worker,
|
||||
# then disable this unit.
|
||||
# Rollback: disable these, re-point DNAT at the Python 808%i workers.
|
||||
|
||||
[Unit]
|
||||
Description=SecuBox ToolBoX-NG Go MITM worker %i (migration target, transparent 10.99.1.1:809%i)
|
||||
Description=SecuBox ToolBoX-NG Go MITM worker %i (migration target, port 809%i)
|
||||
Documentation=https://github.com/CyberMind-FR/secubox-deb/issues/662
|
||||
After=network.target wg-quick@wg-toolbox.service
|
||||
Wants=wg-quick@wg-toolbox.service
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=secubox-toolbox
|
||||
Group=secubox-toolbox
|
||||
|
||||
# Forge with the LIVE R3 CA the clients trust: cert = mitmproxy-ca-cert.pem,
|
||||
# key = mitmproxy-ca.pem (a combined cert+key bundle — loadCA scans for the
|
||||
# PRIVATE KEY block). Both are group-readable by secubox-toolbox. The anti-track
|
||||
# jar key is best-effort: absent → poison stays off.
|
||||
# Reuse the EXISTING ca-wg CA (R3 clients already trust it — no re-enroll).
|
||||
# The anti-track jar key is best-effort: absent → poison stays off.
|
||||
ExecStart=/usr/sbin/sbxmitm \
|
||||
--transparent \
|
||||
--listen 10.99.1.1:809%i \
|
||||
--ca-cert /etc/secubox/toolbox/ca-wg/mitmproxy-ca-cert.pem \
|
||||
--ca-key /etc/secubox/toolbox/ca-wg/mitmproxy-ca.pem \
|
||||
--listen 127.0.0.1:809%i \
|
||||
--ca-cert /etc/secubox/toolbox/ca-wg/ca.pem \
|
||||
--ca-key /etc/secubox/toolbox/ca-wg/key.pem \
|
||||
--jar-key /etc/secubox/secrets/privacy-jar.key
|
||||
|
||||
Restart=on-failure
|
||||
|
|
|
|||
Loading…
Reference in New Issue
Block a user