From f3e24bb7a86e4e8329d7d95daa33d7d577418a16 Mon Sep 17 00:00:00 2001 From: Roman Buryan Date: Mon, 28 Sep 2026 16:54:40 +0300 Subject: [PATCH] docs: layered self-healing architecture --- docs/self-healing-architecture.md | 38 +++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) create mode 100644 docs/self-healing-architecture.md diff --git a/docs/self-healing-architecture.md b/docs/self-healing-architecture.md new file mode 100644 index 0000000..9a41ff1 --- /dev/null +++ b/docs/self-healing-architecture.md @@ -0,0 +1,38 @@ +# Self-Healing Infrastructure (Layered) + +## Слои +| L0 | Docker restart policy | ~1-2с | все контейнеры | +| L1 | Kuma→n8n→Remediation API | 60-120с | 11 whitelist | +| L2 | Gitea issue + Telegram | мин-часы | человек | + +## Layer 0 +Docker `restart: unless-stopped` поднимает контейнер за 1-2с. +Kuma часто не успевает заметить падение. + +## Layer 1 +Kuma (57 мониторов) → webhook → n8n (TSqu00NJx2JOrtoN, 10 нод) → +Remediation API (http://192.168.1.10:5000, Bearer auth, whitelist) → +pct exec docker restart → Telegram ✅/❌ + +## Компоненты +- Kuma: CT 107:3001, https://status.buryans.ru +- n8n: CT 108:5680 (прямой) / 5678 (oauth2-proxy) +- Remediation API: PVE host, systemd remediation-api.service + - /opt/remediation-api/remediation_api.py (stdlib) + - Токен: /opt/remediation-api/api_token.txt + +## Whitelist (11 сервисов) +matrix-synapse, element-web, mautrix-telegram, matrix-coturn, livekit (CT 155) +vaultwarden, nextcloud (CT 109) +gitea (CT 140) +authentik (CT 114) +uptime-kuma, adguard (CT 107) + +## Тест +curl -X POST http://192.168.1.108:5680/webhook/uptime-kuma -H "Content-Type: application/json" -d '{"monitorName":"Matrix Synapse","status":"0","shouldAlert":true}' + +## Логи +- Remediation: journalctl -u remediation-api -f +- n8n: pct exec 108 -- docker logs n8n -f + +Создано: 2026-09-28. Статус: ✅ активно.