Files
buryan-infra/docs/self-healing-architecture.md
T

1.4 KiB
Raw Blame History

Self-Healing Infrastructure (Layered)

Слои

| L0 | Docker restart policy | ~1-2с | все контейнеры | | L1 | Kuma→n8n→Remediation API | 60-120с | 11 whitelist | | L2 | Gitea issue + Telegram | мин-часы | человек |

Layer 0

Docker restart: unless-stopped поднимает контейнер за 1-2с. Kuma часто не успевает заметить падение.

Layer 1

Kuma (57 мониторов) → webhook → n8n (TSqu00NJx2JOrtoN, 10 нод) → Remediation API (http://192.168.1.10:5000, Bearer auth, whitelist) → pct exec docker restart → Telegram ✅/❌

Компоненты

  • Kuma: CT 107:3001, https://status.buryans.ru
  • n8n: CT 108:5680 (прямой) / 5678 (oauth2-proxy)
  • Remediation API: PVE host, systemd remediation-api.service
    • /opt/remediation-api/remediation_api.py (stdlib)
    • Токен: /opt/remediation-api/api_token.txt

Whitelist (11 сервисов)

matrix-synapse, element-web, mautrix-telegram, matrix-coturn, livekit (CT 155) vaultwarden, nextcloud (CT 109) gitea (CT 140) authentik (CT 114) uptime-kuma, adguard (CT 107)

Тест

curl -X POST http://192.168.1.108:5680/webhook/uptime-kuma -H "Content-Type: application/json" -d '{"monitorName":"Matrix Synapse","status":"0","shouldAlert":true}'

Логи

  • Remediation: journalctl -u remediation-api -f
  • n8n: pct exec 108 -- docker logs n8n -f

Создано: 2026-09-28. Статус: ✅ активно.