mirror of
https://github.com/CyberMind-FR/secubox-deb.git
synced 2026-08-16 21:21:22 +00:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f9ba2914e8 | ||
|
|
d84e378142 | ||
|
|
3a79d9f6b9 | ||
|
|
81d24fa003 | ||
|
|
917b35cd2b | ||
|
|
cf7104eb0d | ||
|
|
c14a829915 | ||
|
|
455ac5f185 |
+61
-23
@@ -1,48 +1,86 @@
|
||||
# WIP — Work In Progress
|
||||
*Mis à jour : 2026-05-06 (Session 109)*
|
||||
*Mis à jour : 2026-05-06 (Session 110)*
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Session 109: HAProxy VHost Routing + Eye Remote Fix
|
||||
## ✅ Session 110: System Optimization + Eye Remote Fix
|
||||
|
||||
### HAProxy VHost Additions (on server)
|
||||
- [x] Added `sdlc.gk2.secubox.in` → metablog_sdlc (port 8925)
|
||||
- [x] Added `facb.gk2.secubox.in` → metablog_facb (port 8972)
|
||||
- [x] Added ACLs to both HTTP and HTTPS frontends
|
||||
- [x] All metablog vhosts confirmed working (c3box, gandalf, live)
|
||||
### System Resource Cleanup
|
||||
- [x] Killed orphan health prober (PID 1169103, 705MB VIRT memory leak)
|
||||
- [x] Disabled netdata (freed 213MB RAM, 26% CPU)
|
||||
- [x] **Load: 7.16 → 4.78 (-33%)**
|
||||
- [x] **Free RAM: 380MB → 692MB (+82%)**
|
||||
|
||||
### Eye Remote Dashboard Fix
|
||||
- [x] Fixed API calls to use public endpoints (no JWT required)
|
||||
- [x] `/status`, `/serial/status`, `/pizero/metrics` now work without auth
|
||||
- [x] Added fallback to `/pizero-metrics` endpoint
|
||||
- [x] Deployed updated frontend to `/usr/share/secubox/www/eye-remote/`
|
||||
- [x] Fixed duplicate `get_pizero_metrics()` function bug in API
|
||||
- [x] Deployed clean API source (removed router bugs)
|
||||
- [x] Added `clearMetricsUI()` - resets gauges to `--` when disconnected
|
||||
- [x] Frontend now shows proper disconnect state instead of stale data
|
||||
|
||||
### Eye Remote Metrics Alias (Pi Zero Compatibility)
|
||||
- [x] Added `/api/v1/system/metrics` alias endpoint in eye-remote API
|
||||
- [x] Returns host system metrics (CPU, memory, disk, load)
|
||||
- [x] Pi Zero round UI now fetches correct MOCHAbin metrics
|
||||
### Eye Remote Round UI Fix
|
||||
- [x] Added `/api/v1/system/metrics` endpoint for MOCHAbin metrics
|
||||
- [x] Round UI on Pi Zero now receives live data from MOCHAbin
|
||||
- [x] Metrics: cpu_percent, mem_percent, disk_percent, cpu_temp, load, uptime
|
||||
- [x] Fixed CPU calculation: real /proc/stat delta (was load-based → stuck at 100%)
|
||||
- [x] Added async double pre-cache buffer (per guidelines):
|
||||
- Background task updates shadow buffer every 2s
|
||||
- Atomic swap to active buffer
|
||||
- API returns instantly from cache
|
||||
- [x] Exposed API on TCP port 8000 (10.55.0.1) for Pi Zero access
|
||||
- Was: Unix socket only (Pi Zero couldn't reach)
|
||||
- Now: TCP listener via systemd drop-in
|
||||
- [x] Verified working on HyperPixel 2.1 Round display
|
||||
- [x] Rings now show varied fill levels matching real metrics
|
||||
|
||||
### Current Health Status
|
||||
- **VHost Health:** 🟢 27 🟡 134 🔴 0 ⬜ 58 (96.4%)
|
||||
- **Module Health:** 🟢 5 🟡 3 🔴 0 (62.5%)
|
||||
- **System Load:** 4.78 (4-core ARM)
|
||||
- **Memory:** 6.1GB/7.7GB (79%)
|
||||
|
||||
---
|
||||
|
||||
## ✅ Session 109: Metablogizer Full Config + Health Prober Fix
|
||||
|
||||
### Metablogizer Full Configuration
|
||||
- [x] Configured **165 metablogizer sites** (ports 8900-9204)
|
||||
- [x] Created flat config: `/etc/secubox/metablogizer.json`
|
||||
- [x] Generated nginx server blocks for all sites
|
||||
- [x] Added 131 new HAProxy backends + ACLs
|
||||
- [x] Synced **225 mitmproxy routes** through WAF
|
||||
- [x] Committed config to repo: `config/metablogizer.json`
|
||||
|
||||
### Health Prober Fixes
|
||||
- [x] Fixed prober to use localhost with Host header (was hitting HTTPS externally)
|
||||
- [x] Fixed recursive probe bug in semaphore code
|
||||
- [x] Placeholder detection working (58 Streamlit apps identified)
|
||||
- [x] **Current health: 🟢 29 🟡 132 🔴 0 ⬜ 58 (100%)**
|
||||
|
||||
### VHost Matrix Sync Tool
|
||||
- [x] Created `scripts/vhost-matrix-sync.sh` with Python-based extraction
|
||||
- [x] Fixed stderr logging for clean JSON output capture
|
||||
- [x] Syncs HAProxy vhosts → mitmproxy routes + health prober
|
||||
- [x] Successfully synced 94 vhosts on server
|
||||
- [x] Uses 10.100.0.1 (LXC bridge IP) instead of 127.0.0.1 for routes
|
||||
- [x] Successfully synced 225 vhosts on server
|
||||
|
||||
### mitmproxy Routing Loop Fix (from Session 108)
|
||||
- [x] Changed routes.json: 127.0.0.1 → 10.100.0.1 (LXC bridge IP)
|
||||
- [x] Killed runaway mitmdump process (PID 896697)
|
||||
- [x] Load dropped from 12+ to ~7
|
||||
### Eye Remote Fixes
|
||||
- [x] Added `/api/v1/system/metrics` alias for Pi Zero compatibility
|
||||
- [x] Fixed dashboard to use public endpoints (no JWT)
|
||||
|
||||
### GitHub Issues
|
||||
- [x] #49 — MetaBlogizer + Streamlit version management via Gitea
|
||||
- [x] #50 — Green Computing: Sleep/Wake WAF + Health Prober Optimization
|
||||
|
||||
### Release
|
||||
- [x] Tagged **v2.5.0** and pushed to master
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Session 108 (Continued): VHost Health Fixes + Services Restore
|
||||
## ✅ Session 108 (Continued): VHost Health Fixes + Services Restore
|
||||
|
||||
### VHost Health Prober
|
||||
- [x] Updated prober to treat placeholders as "placeholder" status (not "down")
|
||||
- [x] Added ⬜ placeholder indicator to dashboard
|
||||
- [x] Health % now only counts real vhosts (excludes placeholders)
|
||||
- [x] Current stats: 🟢 0 🟡 5 🔴 29 ⬜ 58 (14.7% health)
|
||||
|
||||
### HAProxy Routing Fixes
|
||||
- [x] Fixed c3box.maegia.tv → metablog_gandalf (was nginx_vhosts placeholder)
|
||||
|
||||
@@ -346,30 +346,83 @@ async def health():
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Legacy Alias: /api/v1/system/metrics
|
||||
# For backward compatibility with remote-ui/round Pi Zero dashboard
|
||||
# System Metrics — MOCHAbin host metrics for Round UI dashboard
|
||||
# =============================================================================
|
||||
# Double Pre-Cache Buffer for Metrics
|
||||
# Background task updates cache every 2s, API returns instantly from cache
|
||||
# =============================================================================
|
||||
import asyncio
|
||||
import time
|
||||
import os
|
||||
|
||||
@app.get("/api/v1/system/metrics")
|
||||
async def get_system_metrics_alias():
|
||||
"""
|
||||
Alias for host system metrics.
|
||||
_metrics_cache = {
|
||||
"active": {}, # Current metrics (read by API)
|
||||
"shadow": {}, # Being updated by background task
|
||||
"last_swap": 0, # Timestamp of last swap
|
||||
}
|
||||
_cpu_state = {"prev_idle": 0, "prev_total": 0, "prev_time": 0}
|
||||
_cache_task = None
|
||||
|
||||
The Pi Zero remote-ui fetches from /api/v1/system/metrics by default.
|
||||
This endpoint returns the same data as /api/v1/eye-remote/metrics.
|
||||
"""
|
||||
import os
|
||||
from datetime import datetime, timezone
|
||||
|
||||
# CPU usage (estimate from load average)
|
||||
def _read_cpu_percent() -> float:
|
||||
"""Calculate real CPU usage from /proc/stat delta."""
|
||||
import time
|
||||
global _cpu_state
|
||||
|
||||
try:
|
||||
load = os.getloadavg()[0]
|
||||
cpus = os.cpu_count() or 1
|
||||
cpu_percent = round(min(100.0, (load / cpus) * 100), 1)
|
||||
except Exception:
|
||||
cpu_percent = 0.0
|
||||
with open("/proc/stat") as f:
|
||||
line = f.readline()
|
||||
parts = line.split()
|
||||
if parts[0] != "cpu":
|
||||
return 0.0
|
||||
|
||||
# Memory
|
||||
# user, nice, system, idle, iowait, irq, softirq, steal
|
||||
values = [int(x) for x in parts[1:9]]
|
||||
idle = values[3] + values[4] # idle + iowait
|
||||
total = sum(values)
|
||||
now = time.time()
|
||||
|
||||
# Calculate delta from previous reading
|
||||
prev_idle = _cpu_state["prev_idle"]
|
||||
prev_total = _cpu_state["prev_total"]
|
||||
prev_time = _cpu_state["prev_time"]
|
||||
|
||||
# Update state
|
||||
_cpu_state["prev_idle"] = idle
|
||||
_cpu_state["prev_total"] = total
|
||||
_cpu_state["prev_time"] = now
|
||||
|
||||
# Need at least 100ms between readings for accuracy
|
||||
if prev_time > 0 and (now - prev_time) > 0.1:
|
||||
idle_delta = idle - prev_idle
|
||||
total_delta = total - prev_total
|
||||
if total_delta > 0:
|
||||
return round(100.0 * (1.0 - idle_delta / total_delta), 1)
|
||||
|
||||
# Fallback for first reading: use 1-second sample
|
||||
import os
|
||||
load = os.getloadavg()[0]
|
||||
cpus = os.cpu_count() or 4
|
||||
return round(min(95.0, (load / cpus) * 80), 1) # Scale down load-based estimate
|
||||
|
||||
except Exception:
|
||||
return 0.0
|
||||
|
||||
|
||||
def _get_host_metrics() -> dict:
|
||||
"""Collect MOCHAbin host system metrics for round UI display."""
|
||||
import os
|
||||
from datetime import timezone
|
||||
|
||||
metrics = {
|
||||
"timestamp": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"),
|
||||
"hostname": "secubox-mochabin",
|
||||
}
|
||||
|
||||
# CPU usage - real delta calculation
|
||||
metrics["cpu_percent"] = _read_cpu_percent()
|
||||
|
||||
# Memory from /proc/meminfo
|
||||
try:
|
||||
with open("/proc/meminfo") as f:
|
||||
meminfo = {}
|
||||
@@ -377,53 +430,106 @@ async def get_system_metrics_alias():
|
||||
parts = line.split()
|
||||
if len(parts) >= 2:
|
||||
meminfo[parts[0].rstrip(":")] = int(parts[1])
|
||||
total_kb = meminfo.get("MemTotal", 1)
|
||||
free_kb = meminfo.get("MemAvailable", meminfo.get("MemFree", 0))
|
||||
mem_percent = round(((total_kb - free_kb) / total_kb) * 100, 1)
|
||||
mem_free_mb = free_kb // 1024
|
||||
mem_total_mb = total_kb // 1024
|
||||
total_kb = meminfo.get("MemTotal", 1)
|
||||
avail_kb = meminfo.get("MemAvailable", meminfo.get("MemFree", 0))
|
||||
metrics["mem_percent"] = round(((total_kb - avail_kb) / total_kb) * 100, 1)
|
||||
except Exception:
|
||||
mem_percent = 0.0
|
||||
mem_free_mb = 0
|
||||
mem_total_mb = 0
|
||||
metrics["mem_percent"] = 0.0
|
||||
|
||||
# Disk
|
||||
# Disk from statvfs
|
||||
try:
|
||||
statvfs = os.statvfs("/")
|
||||
total = statvfs.f_blocks * statvfs.f_frsize
|
||||
free = statvfs.f_bavail * statvfs.f_frsize
|
||||
disk_percent = round(((total - free) / total) * 100, 1) if total > 0 else 0.0
|
||||
disk_free_gb = round(free / (1024**3), 1)
|
||||
disk_total_gb = round(total / (1024**3), 1)
|
||||
st = os.statvfs("/")
|
||||
total = st.f_blocks * st.f_frsize
|
||||
free = st.f_bavail * st.f_frsize
|
||||
metrics["disk_percent"] = round(((total - free) / total) * 100, 1) if total > 0 else 0.0
|
||||
except Exception:
|
||||
disk_percent = 0.0
|
||||
disk_free_gb = 0.0
|
||||
disk_total_gb = 0.0
|
||||
metrics["disk_percent"] = 0.0
|
||||
|
||||
# CPU temperature
|
||||
try:
|
||||
with open("/sys/class/thermal/thermal_zone0/temp") as f:
|
||||
metrics["cpu_temp"] = round(int(f.read().strip()) / 1000.0, 1)
|
||||
except Exception:
|
||||
metrics["cpu_temp"] = 0.0
|
||||
|
||||
# Load average
|
||||
try:
|
||||
load_1m, load_5m, load_15m = os.getloadavg()
|
||||
load1, load5, load15 = os.getloadavg()
|
||||
metrics["load_1m"] = round(load1, 2)
|
||||
metrics["load_5m"] = round(load5, 2)
|
||||
metrics["load_15m"] = round(load15, 2)
|
||||
except Exception:
|
||||
load_1m = load_5m = load_15m = 0.0
|
||||
metrics["load_1m"] = metrics["load_5m"] = metrics["load_15m"] = 0.0
|
||||
|
||||
# Uptime
|
||||
try:
|
||||
with open("/proc/uptime") as f:
|
||||
uptime = int(float(f.read().split()[0]))
|
||||
metrics["uptime_seconds"] = int(float(f.read().split()[0]))
|
||||
except Exception:
|
||||
uptime = 0
|
||||
metrics["uptime_seconds"] = 0
|
||||
|
||||
return {
|
||||
"timestamp": datetime.now(timezone.utc).isoformat(),
|
||||
"uptime": uptime,
|
||||
"cpu_percent": cpu_percent,
|
||||
"memory_percent": mem_percent,
|
||||
"memory_free_mb": mem_free_mb,
|
||||
"memory_total_mb": mem_total_mb,
|
||||
"disk_percent": disk_percent,
|
||||
"disk_free_gb": disk_free_gb,
|
||||
"disk_total_gb": disk_total_gb,
|
||||
"load_1m": round(load_1m, 2),
|
||||
"load_5m": round(load_5m, 2),
|
||||
"load_15m": round(load_15m, 2),
|
||||
}
|
||||
return metrics
|
||||
|
||||
|
||||
async def _metrics_background_task():
|
||||
"""Background task that updates metrics cache every 2 seconds.
|
||||
|
||||
Pattern: Double Pre-Cache Buffer (per SecuBox guidelines)
|
||||
- shadow buffer updated in background
|
||||
- atomic swap to active buffer
|
||||
- API reads from active (instant response)
|
||||
"""
|
||||
global _metrics_cache
|
||||
|
||||
while True:
|
||||
try:
|
||||
# Collect metrics into shadow buffer
|
||||
_metrics_cache["shadow"] = _get_host_metrics()
|
||||
|
||||
# Atomic swap: shadow → active
|
||||
_metrics_cache["active"] = _metrics_cache["shadow"].copy()
|
||||
_metrics_cache["last_swap"] = time.time()
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"Metrics cache update failed: {e}")
|
||||
|
||||
await asyncio.sleep(2) # Update every 2 seconds
|
||||
|
||||
|
||||
@app.on_event("startup")
|
||||
async def start_metrics_cache():
|
||||
"""Start background metrics caching on API startup."""
|
||||
global _cache_task
|
||||
|
||||
# Initial cache population
|
||||
_metrics_cache["active"] = _get_host_metrics()
|
||||
_metrics_cache["last_swap"] = time.time()
|
||||
|
||||
# Start background updater
|
||||
_cache_task = asyncio.create_task(_metrics_background_task())
|
||||
logger.info("Metrics pre-cache started (2s refresh)")
|
||||
|
||||
|
||||
@app.on_event("shutdown")
|
||||
async def stop_metrics_cache():
|
||||
"""Stop background task on shutdown."""
|
||||
global _cache_task
|
||||
if _cache_task:
|
||||
_cache_task.cancel()
|
||||
try:
|
||||
await _cache_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
|
||||
|
||||
@app.get("/api/v1/system/metrics")
|
||||
async def get_system_metrics():
|
||||
"""Get MOCHAbin host system metrics for round UI dashboard.
|
||||
|
||||
Returns cached metrics (updated every 2s by background task).
|
||||
Pattern: Double Pre-Cache Buffer for instant response.
|
||||
"""
|
||||
if _metrics_cache["active"]:
|
||||
return _metrics_cache["active"]
|
||||
# Fallback if cache not ready
|
||||
return _get_host_metrics()
|
||||
|
||||
@@ -243,8 +243,12 @@
|
||||
updateMetricsUI(metrics);
|
||||
} catch (e2) {
|
||||
console.log('Pi Zero metrics not available');
|
||||
clearMetricsUI();
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// Clear metrics when disconnected
|
||||
clearMetricsUI();
|
||||
}
|
||||
} catch (err) {
|
||||
console.error('Refresh error:', err);
|
||||
@@ -321,6 +325,18 @@
|
||||
else el.classList.add('bad');
|
||||
}
|
||||
|
||||
function clearMetricsUI() {
|
||||
document.getElementById('cpu-value').textContent = '--';
|
||||
document.getElementById('mem-value').textContent = '--';
|
||||
document.getElementById('temp-value').textContent = '--';
|
||||
document.getElementById('pizero-uptime').textContent = '--';
|
||||
// Reset gauge classes
|
||||
['cpu-gauge', 'mem-gauge', 'temp-gauge'].forEach(id => {
|
||||
const el = document.getElementById(id);
|
||||
el.classList.remove('good', 'warn', 'bad');
|
||||
});
|
||||
}
|
||||
|
||||
async function probeConnection() {
|
||||
const btn = document.getElementById('probe-btn');
|
||||
btn.disabled = true;
|
||||
|
||||
Reference in New Issue
Block a user