Compare commits

...
8 Commits
Author SHA1 Message Date
gandalfandClaude Opus 4.5 f9ba2914e8 docs(wip): Session 110 - add TCP port 8000 for Pi Zero
- Eye-remote API now exposed on 10.55.0.1:8000
- Pi Zero round UI can fetch MOCHAbin metrics
- Systemd drop-in for TCP listener

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-06 19:04:03 +02:00
gandalfandClaude Opus 4.5 d84e378142 docs(wip): Session 110 - add round UI pre-cache buffer
- Real CPU delta calculation (not load-based)
- Async double pre-cache buffer pattern
- Rings now show varied fill levels

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-06 18:58:35 +02:00
gandalfandClaude Opus 4.5 3a79d9f6b9 feat(eye-remote): Add async double pre-cache buffer for metrics
Pattern per SecuBox guidelines:
- Background task updates shadow buffer every 2s
- Atomic swap to active buffer
- API returns instantly from cache
- Real CPU delta calculation (not load-based)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-06 18:55:10 +02:00
gandalfandClaude Opus 4.5 81d24fa003 docs(wip): Session 110 - add round UI metrics fix
- Added /api/v1/system/metrics endpoint
- Round UI on Pi Zero now receives live MOCHAbin data
- Verified working on HyperPixel 2.1 Round display

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-06 18:52:34 +02:00
gandalfandClaude Opus 4.5 917b35cd2b fix(eye-remote): Add /api/v1/system/metrics for round UI
The Eye Remote round UI needs MOCHAbin metrics from this endpoint.
Returns: cpu_percent, mem_percent, disk_percent, cpu_temp, load, uptime

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-06 18:50:51 +02:00
gandalfandClaude Opus 4.5 cf7104eb0d docs(wip): Session 110 - system optimization + eye remote fix
- Killed orphan prober (705MB leak), disabled netdata
- Load: 7.16 → 4.78, RAM: 380MB → 692MB free
- Fixed Eye Remote duplicate function bug + disconnect display

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-06 18:36:55 +02:00
gandalfandClaude Opus 4.5 c14a829915 fix(eye-remote): Clear metrics display when Pi Zero disconnected
- Add clearMetricsUI() function to reset gauges to '--'
- Call clearMetricsUI when status.connected is false
- Prevents stale metrics from showing after disconnect

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-06 18:31:44 +02:00
gandalfandClaude Opus 4.5 455ac5f185 docs(wip): Session 109 complete - metablogizer + health prober
- Configured 165 metablogizer sites (ports 8900-9204)
- Fixed health prober: localhost + Host header pattern
- VHost sync: 225 routes through WAF
- Health: 🟢 29 🟡 132 🔴 0  58
- Released v2.5.0

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-06 18:13:18 +02:00
3 changed files with 237 additions and 77 deletions
+61 -23
View File
@@ -1,48 +1,86 @@
# WIP — Work In Progress
*Mis à jour : 2026-05-06 (Session 109)*
*Mis à jour : 2026-05-06 (Session 110)*
---
## 🔄 Session 109: HAProxy VHost Routing + Eye Remote Fix
## Session 110: System Optimization + Eye Remote Fix
### HAProxy VHost Additions (on server)
- [x] Added `sdlc.gk2.secubox.in` → metablog_sdlc (port 8925)
- [x] Added `facb.gk2.secubox.in` → metablog_facb (port 8972)
- [x] Added ACLs to both HTTP and HTTPS frontends
- [x] All metablog vhosts confirmed working (c3box, gandalf, live)
### System Resource Cleanup
- [x] Killed orphan health prober (PID 1169103, 705MB VIRT memory leak)
- [x] Disabled netdata (freed 213MB RAM, 26% CPU)
- [x] **Load: 7.16 → 4.78 (-33%)**
- [x] **Free RAM: 380MB → 692MB (+82%)**
### Eye Remote Dashboard Fix
- [x] Fixed API calls to use public endpoints (no JWT required)
- [x] `/status`, `/serial/status`, `/pizero/metrics` now work without auth
- [x] Added fallback to `/pizero-metrics` endpoint
- [x] Deployed updated frontend to `/usr/share/secubox/www/eye-remote/`
- [x] Fixed duplicate `get_pizero_metrics()` function bug in API
- [x] Deployed clean API source (removed router bugs)
- [x] Added `clearMetricsUI()` - resets gauges to `--` when disconnected
- [x] Frontend now shows proper disconnect state instead of stale data
### Eye Remote Metrics Alias (Pi Zero Compatibility)
- [x] Added `/api/v1/system/metrics` alias endpoint in eye-remote API
- [x] Returns host system metrics (CPU, memory, disk, load)
- [x] Pi Zero round UI now fetches correct MOCHAbin metrics
### Eye Remote Round UI Fix
- [x] Added `/api/v1/system/metrics` endpoint for MOCHAbin metrics
- [x] Round UI on Pi Zero now receives live data from MOCHAbin
- [x] Metrics: cpu_percent, mem_percent, disk_percent, cpu_temp, load, uptime
- [x] Fixed CPU calculation: real /proc/stat delta (was load-based → stuck at 100%)
- [x] Added async double pre-cache buffer (per guidelines):
- Background task updates shadow buffer every 2s
- Atomic swap to active buffer
- API returns instantly from cache
- [x] Exposed API on TCP port 8000 (10.55.0.1) for Pi Zero access
- Was: Unix socket only (Pi Zero couldn't reach)
- Now: TCP listener via systemd drop-in
- [x] Verified working on HyperPixel 2.1 Round display
- [x] Rings now show varied fill levels matching real metrics
### Current Health Status
- **VHost Health:** 🟢 27 🟡 134 🔴 0 ⬜ 58 (96.4%)
- **Module Health:** 🟢 5 🟡 3 🔴 0 (62.5%)
- **System Load:** 4.78 (4-core ARM)
- **Memory:** 6.1GB/7.7GB (79%)
---
## ✅ Session 109: Metablogizer Full Config + Health Prober Fix
### Metablogizer Full Configuration
- [x] Configured **165 metablogizer sites** (ports 8900-9204)
- [x] Created flat config: `/etc/secubox/metablogizer.json`
- [x] Generated nginx server blocks for all sites
- [x] Added 131 new HAProxy backends + ACLs
- [x] Synced **225 mitmproxy routes** through WAF
- [x] Committed config to repo: `config/metablogizer.json`
### Health Prober Fixes
- [x] Fixed prober to use localhost with Host header (was hitting HTTPS externally)
- [x] Fixed recursive probe bug in semaphore code
- [x] Placeholder detection working (58 Streamlit apps identified)
- [x] **Current health: 🟢 29 🟡 132 🔴 0 ⬜ 58 (100%)**
### VHost Matrix Sync Tool
- [x] Created `scripts/vhost-matrix-sync.sh` with Python-based extraction
- [x] Fixed stderr logging for clean JSON output capture
- [x] Syncs HAProxy vhosts → mitmproxy routes + health prober
- [x] Successfully synced 94 vhosts on server
- [x] Uses 10.100.0.1 (LXC bridge IP) instead of 127.0.0.1 for routes
- [x] Successfully synced 225 vhosts on server
### mitmproxy Routing Loop Fix (from Session 108)
- [x] Changed routes.json: 127.0.0.1 → 10.100.0.1 (LXC bridge IP)
- [x] Killed runaway mitmdump process (PID 896697)
- [x] Load dropped from 12+ to ~7
### Eye Remote Fixes
- [x] Added `/api/v1/system/metrics` alias for Pi Zero compatibility
- [x] Fixed dashboard to use public endpoints (no JWT)
### GitHub Issues
- [x] #49 — MetaBlogizer + Streamlit version management via Gitea
- [x] #50 — Green Computing: Sleep/Wake WAF + Health Prober Optimization
### Release
- [x] Tagged **v2.5.0** and pushed to master
---
## 🔄 Session 108 (Continued): VHost Health Fixes + Services Restore
## Session 108 (Continued): VHost Health Fixes + Services Restore
### VHost Health Prober
- [x] Updated prober to treat placeholders as "placeholder" status (not "down")
- [x] Added ⬜ placeholder indicator to dashboard
- [x] Health % now only counts real vhosts (excludes placeholders)
- [x] Current stats: 🟢 0 🟡 5 🔴 29 ⬜ 58 (14.7% health)
### HAProxy Routing Fixes
- [x] Fixed c3box.maegia.tv → metablog_gandalf (was nginx_vhosts placeholder)
+160 -54
View File
@@ -346,30 +346,83 @@ async def health():
# =============================================================================
# Legacy Alias: /api/v1/system/metrics
# For backward compatibility with remote-ui/round Pi Zero dashboard
# System Metrics — MOCHAbin host metrics for Round UI dashboard
# =============================================================================
# Double Pre-Cache Buffer for Metrics
# Background task updates cache every 2s, API returns instantly from cache
# =============================================================================
import asyncio
import time
import os
@app.get("/api/v1/system/metrics")
async def get_system_metrics_alias():
"""
Alias for host system metrics.
_metrics_cache = {
"active": {}, # Current metrics (read by API)
"shadow": {}, # Being updated by background task
"last_swap": 0, # Timestamp of last swap
}
_cpu_state = {"prev_idle": 0, "prev_total": 0, "prev_time": 0}
_cache_task = None
The Pi Zero remote-ui fetches from /api/v1/system/metrics by default.
This endpoint returns the same data as /api/v1/eye-remote/metrics.
"""
import os
from datetime import datetime, timezone
# CPU usage (estimate from load average)
def _read_cpu_percent() -> float:
"""Calculate real CPU usage from /proc/stat delta."""
import time
global _cpu_state
try:
load = os.getloadavg()[0]
cpus = os.cpu_count() or 1
cpu_percent = round(min(100.0, (load / cpus) * 100), 1)
except Exception:
cpu_percent = 0.0
with open("/proc/stat") as f:
line = f.readline()
parts = line.split()
if parts[0] != "cpu":
return 0.0
# Memory
# user, nice, system, idle, iowait, irq, softirq, steal
values = [int(x) for x in parts[1:9]]
idle = values[3] + values[4] # idle + iowait
total = sum(values)
now = time.time()
# Calculate delta from previous reading
prev_idle = _cpu_state["prev_idle"]
prev_total = _cpu_state["prev_total"]
prev_time = _cpu_state["prev_time"]
# Update state
_cpu_state["prev_idle"] = idle
_cpu_state["prev_total"] = total
_cpu_state["prev_time"] = now
# Need at least 100ms between readings for accuracy
if prev_time > 0 and (now - prev_time) > 0.1:
idle_delta = idle - prev_idle
total_delta = total - prev_total
if total_delta > 0:
return round(100.0 * (1.0 - idle_delta / total_delta), 1)
# Fallback for first reading: use 1-second sample
import os
load = os.getloadavg()[0]
cpus = os.cpu_count() or 4
return round(min(95.0, (load / cpus) * 80), 1) # Scale down load-based estimate
except Exception:
return 0.0
def _get_host_metrics() -> dict:
"""Collect MOCHAbin host system metrics for round UI display."""
import os
from datetime import timezone
metrics = {
"timestamp": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"),
"hostname": "secubox-mochabin",
}
# CPU usage - real delta calculation
metrics["cpu_percent"] = _read_cpu_percent()
# Memory from /proc/meminfo
try:
with open("/proc/meminfo") as f:
meminfo = {}
@@ -377,53 +430,106 @@ async def get_system_metrics_alias():
parts = line.split()
if len(parts) >= 2:
meminfo[parts[0].rstrip(":")] = int(parts[1])
total_kb = meminfo.get("MemTotal", 1)
free_kb = meminfo.get("MemAvailable", meminfo.get("MemFree", 0))
mem_percent = round(((total_kb - free_kb) / total_kb) * 100, 1)
mem_free_mb = free_kb // 1024
mem_total_mb = total_kb // 1024
total_kb = meminfo.get("MemTotal", 1)
avail_kb = meminfo.get("MemAvailable", meminfo.get("MemFree", 0))
metrics["mem_percent"] = round(((total_kb - avail_kb) / total_kb) * 100, 1)
except Exception:
mem_percent = 0.0
mem_free_mb = 0
mem_total_mb = 0
metrics["mem_percent"] = 0.0
# Disk
# Disk from statvfs
try:
statvfs = os.statvfs("/")
total = statvfs.f_blocks * statvfs.f_frsize
free = statvfs.f_bavail * statvfs.f_frsize
disk_percent = round(((total - free) / total) * 100, 1) if total > 0 else 0.0
disk_free_gb = round(free / (1024**3), 1)
disk_total_gb = round(total / (1024**3), 1)
st = os.statvfs("/")
total = st.f_blocks * st.f_frsize
free = st.f_bavail * st.f_frsize
metrics["disk_percent"] = round(((total - free) / total) * 100, 1) if total > 0 else 0.0
except Exception:
disk_percent = 0.0
disk_free_gb = 0.0
disk_total_gb = 0.0
metrics["disk_percent"] = 0.0
# CPU temperature
try:
with open("/sys/class/thermal/thermal_zone0/temp") as f:
metrics["cpu_temp"] = round(int(f.read().strip()) / 1000.0, 1)
except Exception:
metrics["cpu_temp"] = 0.0
# Load average
try:
load_1m, load_5m, load_15m = os.getloadavg()
load1, load5, load15 = os.getloadavg()
metrics["load_1m"] = round(load1, 2)
metrics["load_5m"] = round(load5, 2)
metrics["load_15m"] = round(load15, 2)
except Exception:
load_1m = load_5m = load_15m = 0.0
metrics["load_1m"] = metrics["load_5m"] = metrics["load_15m"] = 0.0
# Uptime
try:
with open("/proc/uptime") as f:
uptime = int(float(f.read().split()[0]))
metrics["uptime_seconds"] = int(float(f.read().split()[0]))
except Exception:
uptime = 0
metrics["uptime_seconds"] = 0
return {
"timestamp": datetime.now(timezone.utc).isoformat(),
"uptime": uptime,
"cpu_percent": cpu_percent,
"memory_percent": mem_percent,
"memory_free_mb": mem_free_mb,
"memory_total_mb": mem_total_mb,
"disk_percent": disk_percent,
"disk_free_gb": disk_free_gb,
"disk_total_gb": disk_total_gb,
"load_1m": round(load_1m, 2),
"load_5m": round(load_5m, 2),
"load_15m": round(load_15m, 2),
}
return metrics
async def _metrics_background_task():
"""Background task that updates metrics cache every 2 seconds.
Pattern: Double Pre-Cache Buffer (per SecuBox guidelines)
- shadow buffer updated in background
- atomic swap to active buffer
- API reads from active (instant response)
"""
global _metrics_cache
while True:
try:
# Collect metrics into shadow buffer
_metrics_cache["shadow"] = _get_host_metrics()
# Atomic swap: shadow → active
_metrics_cache["active"] = _metrics_cache["shadow"].copy()
_metrics_cache["last_swap"] = time.time()
except Exception as e:
logger.warning(f"Metrics cache update failed: {e}")
await asyncio.sleep(2) # Update every 2 seconds
@app.on_event("startup")
async def start_metrics_cache():
"""Start background metrics caching on API startup."""
global _cache_task
# Initial cache population
_metrics_cache["active"] = _get_host_metrics()
_metrics_cache["last_swap"] = time.time()
# Start background updater
_cache_task = asyncio.create_task(_metrics_background_task())
logger.info("Metrics pre-cache started (2s refresh)")
@app.on_event("shutdown")
async def stop_metrics_cache():
"""Stop background task on shutdown."""
global _cache_task
if _cache_task:
_cache_task.cancel()
try:
await _cache_task
except asyncio.CancelledError:
pass
@app.get("/api/v1/system/metrics")
async def get_system_metrics():
"""Get MOCHAbin host system metrics for round UI dashboard.
Returns cached metrics (updated every 2s by background task).
Pattern: Double Pre-Cache Buffer for instant response.
"""
if _metrics_cache["active"]:
return _metrics_cache["active"]
# Fallback if cache not ready
return _get_host_metrics()
@@ -243,8 +243,12 @@
updateMetricsUI(metrics);
} catch (e2) {
console.log('Pi Zero metrics not available');
clearMetricsUI();
}
}
} else {
// Clear metrics when disconnected
clearMetricsUI();
}
} catch (err) {
console.error('Refresh error:', err);
@@ -321,6 +325,18 @@
else el.classList.add('bad');
}
function clearMetricsUI() {
document.getElementById('cpu-value').textContent = '--';
document.getElementById('mem-value').textContent = '--';
document.getElementById('temp-value').textContent = '--';
document.getElementById('pizero-uptime').textContent = '--';
// Reset gauge classes
['cpu-gauge', 'mem-gauge', 'temp-gauge'].forEach(id => {
const el = document.getElementById(id);
el.classList.remove('good', 'warn', 'bad');
});
}
async function probeConnection() {
const btn = document.getElementById('probe-btn');
btn.disabled = true;