docs(adr): finalize ADR-0021 and add ADR-0022

ADR-0021 (Accepted): - ntfy as central notification hub - Alertmanager integration for critical/warning alerts - Service readiness notifications via Flux notification-controller - Standardized topic naming ADR-0022 (Proposed): - ntfy-discord-bridge Python service design - SSE subscription with reconnection logic - Message transformation to Discord embeds - Priority/tag to color/emoji mapping - Kubernetes deployment with ExternalSecret for webhook
2026-02-02 11:58:52 -05:00
parent 7b77d6c29f
commit e85deaa642
2 changed files with 284 additions and 55 deletions
--- a/docs/adr/ADR-0022-ntfy-discord-bridge.md
+++ b/docs/adr/ADR-0022-ntfy-discord-bridge.md
@@ -0,0 +1,241 @@
+# ADR-0022: ntfy-Discord Bridge Service
+
+## Status
+
+Proposed
+
+## Context
+
+Per ADR-0021, ntfy serves as the central notification hub for the homelab. However, Discord is used for team collaboration and visibility, requiring notifications to be forwarded there as well.
+
+ntfy does not natively support Discord webhook format. Discord expects a specific JSON structure with embeds, while ntfy uses its own message format. A bridge service is needed to:
+
+1. Subscribe to ntfy topics
+2. Transform messages to Discord embed format
+3. Forward to Discord webhooks
+
+## Decision
+
+### Architecture
+
+A dedicated Python microservice (`ntfy-discord-bridge`) will bridge ntfy to Discord:
+
+```
+┌─────────────────┐     ┌──────────────────┐     ┌─────────────┐
+│ CI/Alertmanager │────▶│      ntfy        │────▶│  ntfy App   │
+│ Gatus/Flux      │     │  (notification   │     │  (mobile)   │
+└─────────────────┘     │      hub)        │     └─────────────┘
+                        └────────┬─────────┘
+                                 │ SSE/JSON stream
+                                 ▼
+                        ┌──────────────────┐     ┌─────────────┐
+                        │ ntfy-discord-    │────▶│  Discord    │
+                        │ bridge           │     │  Webhook    │
+                        └──────────────────┘     └─────────────┘
+```
+
+### Service Design
+
+**Repository**: `ntfy-discord-bridge`
+
+**Technology Stack**:
+- Python 3.12+
+- `httpx` for async HTTP (SSE subscription + Discord POST)
+- `pydantic` for configuration validation
+- `structlog` for structured logging
+- Poetry/uv for dependency management
+
+**Core Features**:
+
+1. **SSE Subscription**: Connect to ntfy's JSON stream endpoint for real-time messages
+2. **Automatic Reconnection**: Exponential backoff on connection failures
+3. **Message Transformation**: Convert ntfy format to Discord embed format
+4. **Priority Mapping**: Map ntfy priorities to Discord embed colors
+5. **Topic Routing**: Configure which topics go to which Discord channels/webhooks
+6. **Health Endpoint**: `/health` for Kubernetes probes
+
+### Configuration
+
+Environment variables or ConfigMap:
+
+```yaml
+NTFY_URL: "http://ntfy-svc.observability.svc.cluster.local"
+DISCORD_WEBHOOK_URL: "${DISCORD_WEBHOOK_URL}"  # From Vault via ExternalSecret
+
+# Topic routing (optional - defaults to single webhook)
+TOPIC_WEBHOOKS: |
+  gitea-ci: ${DISCORD_CI_WEBHOOK}
+  alertmanager-alerts: ${DISCORD_ALERTS_WEBHOOK}
+  deployments: ${DISCORD_DEPLOYMENTS_WEBHOOK}
+
+# Topics to subscribe to (comma-separated)
+NTFY_TOPICS: "gitea-ci,alertmanager-alerts,deployments,gatus"
+```
+
+### Message Transformation
+
+ntfy message:
+```json
+{
+  "id": "abc123",
+  "topic": "gitea-ci",
+  "title": "Build succeeded",
+  "message": "ray-serve-apps published to PyPI",
+  "priority": 3,
+  "tags": ["package", "white_check_mark"],
+  "time": 1770050091
+}
+```
+
+Discord embed:
+```json
+{
+  "embeds": [{
+    "title": "✅ Build succeeded",
+    "description": "ray-serve-apps published to PyPI",
+    "color": 3066993,
+    "fields": [
+      {"name": "Topic", "value": "gitea-ci", "inline": true},
+      {"name": "Tags", "value": "package", "inline": true}
+    ],
+    "timestamp": "2026-02-02T11:34:51Z",
+    "footer": {"text": "ntfy"}
+  }]
+}
+```
+
+**Priority → Color Mapping**:
+| Priority | Name | Discord Color |
+|----------|------|---------------|
+| 5 | Max/Urgent | 🔴 Red (15158332) |
+| 4 | High | 🟠 Orange (15105570) |
+| 3 | Default | 🔵 Blue (3066993) |
+| 2 | Low | ⚪ Gray (9807270) |
+| 1 | Min | ⚪ Light Gray (12370112) |
+
+**Tag → Emoji Mapping**:
+Common ntfy tags are converted to Discord-friendly emojis in the title:
+- `white_check_mark` / `heavy_check_mark` → ✅
+- `x` / `skull` → ❌
+- `warning` → ⚠️
+- `rotating_light` → 🚨
+- `rocket` → 🚀
+- `package` → 📦
+
+### Kubernetes Deployment
+
+```yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: ntfy-discord-bridge
+  namespace: observability
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: ntfy-discord-bridge
+  template:
+    spec:
+      containers:
+        - name: bridge
+          image: registry.daviestechlabs.io/ntfy-discord-bridge:latest
+          env:
+            - name: NTFY_URL
+              value: "http://ntfy-svc.observability.svc.cluster.local"
+            - name: NTFY_TOPICS
+              value: "gitea-ci,alertmanager-alerts,deployments"
+            - name: DISCORD_WEBHOOK_URL
+              valueFrom:
+                secretKeyRef:
+                  name: discord-webhook-secret
+                  key: webhook-url
+          ports:
+            - containerPort: 8080
+              name: health
+          livenessProbe:
+            httpGet:
+              path: /health
+              port: health
+            initialDelaySeconds: 5
+            periodSeconds: 30
+          resources:
+            limits:
+              cpu: 100m
+              memory: 128Mi
+            requests:
+              cpu: 10m
+              memory: 64Mi
+```
+
+### Secret Management
+
+Discord webhook URL stored in Vault at `kv/data/discord`:
+
+```yaml
+apiVersion: external-secrets.io/v1beta1
+kind: ExternalSecret
+metadata:
+  name: discord-webhook-secret
+  namespace: observability
+spec:
+  refreshInterval: 1h
+  secretStoreRef:
+    name: vault
+    kind: ClusterSecretStore
+  target:
+    name: discord-webhook-secret
+  data:
+    - secretKey: webhook-url
+      remoteRef:
+        key: kv/data/discord
+        property: webhook_url
+```
+
+### Error Handling
+
+1. **Connection Loss**: Exponential backoff (1s, 2s, 4s, ... max 60s)
+2. **Discord Rate Limits**: Respect `Retry-After` header, queue messages
+3. **Invalid Messages**: Log and skip, don't crash
+4. **Webhook Errors**: Log error, continue processing other messages
+
+## Consequences
+
+### Positive
+
+- **Robust**: Proper reconnection and error handling vs shell script
+- **Testable**: Python service with unit tests
+- **Observable**: Structured logging, health endpoints
+- **Flexible**: Easy to add topic routing, filtering, rate limiting
+- **Consistent**: Follows same patterns as other Python services (handler-base, etc.)
+
+### Negative
+
+- **Operational Overhead**: Another service to maintain
+- **Build Pipeline**: Requires CI/CD for container image
+- **Latency**: Adds ~100ms to notification delivery
+
+### Neutral
+
+- Webhook URL must be maintained in Vault
+- Service logs should be monitored for errors
+
+## Implementation Checklist
+
+- [ ] Create `ntfy-discord-bridge` repository
+- [ ] Implement core bridge logic with httpx
+- [ ] Add reconnection with exponential backoff
+- [ ] Implement message transformation
+- [ ] Add health endpoint
+- [ ] Write unit tests
+- [ ] Create Dockerfile
+- [ ] Set up CI/CD pipeline (Gitea Actions)
+- [ ] Add ExternalSecret for Discord webhook
+- [ ] Create Kubernetes manifests
+- [ ] Deploy to observability namespace
+- [ ] Verify notifications flowing to Discord
+
+## Related
+
+- ADR-0021: Notification Architecture
+- ADR-0015: CI Notifications and Semantic Versioning