From 51e6cee8ab2a69d696fdb66e26e3470c234c8db9 Mon Sep 17 00:00:00 2001 From: "Billy D." Date: Fri, 13 Feb 2026 15:31:29 -0500 Subject: [PATCH] feat: ADR-0056 custom voice support, ADR-0057 Renovate per-repo configs - ADR-0056: custom voice support in tts-module (VoiceRegistry) - ADR-0057: shared Renovate preset rollout to all app repos - Update ADR-0013: add tts-module and stt-module to CI table - Update ADR-0036: cross-reference ADR-0057 --- decisions/0013-gitea-actions-for-ci.md | 2 + decisions/0036-renovate-dependency-updates.md | 1 + .../0056-custom-voice-support-tts-module.md | 109 ++++++++ decisions/0057-renovate-per-repo-configs.md | 251 ++++++++++++++++++ 4 files changed, 363 insertions(+) create mode 100644 decisions/0056-custom-voice-support-tts-module.md create mode 100644 decisions/0057-renovate-per-repo-configs.md diff --git a/decisions/0013-gitea-actions-for-ci.md b/decisions/0013-gitea-actions-for-ci.md index d2bf5f4..a6faaeb 100644 --- a/decisions/0013-gitea-actions-for-ci.md +++ b/decisions/0013-gitea-actions-for-ci.md @@ -58,6 +58,8 @@ Chosen option: "Gitea Actions", because it provides native integration with our | pipeline-bridge | Python | Lint, Test | | companions-frontend | Go | Lint (golangci-lint), Test, Build | | mlflow | Python | Lint, Test | +| tts-module | Python | Lint, Test, Docker build+push | +| stt-module | Python | Lint, Test, Docker build+push | ## Workflow Patterns diff --git a/decisions/0036-renovate-dependency-updates.md b/decisions/0036-renovate-dependency-updates.md index 730983b..829065c 100644 --- a/decisions/0036-renovate-dependency-updates.md +++ b/decisions/0036-renovate-dependency-updates.md @@ -254,3 +254,4 @@ sum(kube_job_status_succeeded{job_name=~"renovate-.*"} + kube_job_status_failed{ * [Gitea Platform Support](https://docs.renovatebot.com/modules/platform/gitea/) * Related: [ADR-0013](0013-gitea-actions-for-ci.md) - Gitea Actions for CI * Related: [ADR-0031](0031-gitea-cicd-strategy.md) - Gitea CI/CD Strategy +* Extended by: [ADR-0057](0057-renovate-per-repo-configs.md) - Per-Repository Renovate Configurations diff --git a/decisions/0056-custom-voice-support-tts-module.md b/decisions/0056-custom-voice-support-tts-module.md new file mode 100644 index 0000000..c5daccf --- /dev/null +++ b/decisions/0056-custom-voice-support-tts-module.md @@ -0,0 +1,109 @@ +# Custom Trained Voice Support in TTS Module + +* Status: accepted +* Date: 2026-02-13 +* Deciders: Billy Davies +* Technical Story: Enable the TTS module to use custom voices generated by the `coqui-voice-training` Argo workflow + +## Context and Problem Statement + +The `coqui-voice-training` Argo workflow trains custom VITS voice models from audio samples and exports them to NFS at `/models/tts/custom/{voice-name}/`. The TTS streaming module currently only supports the default XTTS speaker or ad-hoc voice cloning via base64-encoded reference audio (`speaker_wav_b64`). There is no mechanism to discover and use the fine-tuned models produced by the training pipeline. + +How should the TTS module discover and serve custom trained voices so that callers can request a trained voice by name without providing reference audio? + +## Decision Drivers + +* Voices trained by the Argo pipeline should be usable immediately without service restarts +* Callers should be able to request a trained voice by name (e.g. `"speaker": "my-voice"`) +* Existing ad-hoc voice cloning via `speaker_wav_b64` must continue to work +* Other services need a way to enumerate available voices +* No external database or registry should be required — the file system is the source of truth + +## Considered Options + +1. **File-system VoiceRegistry with periodic refresh** — scan the NFS model store on startup and periodically +2. **Database-backed voice catalogue** — store voice metadata in PostgreSQL +3. **NATS KV bucket for voice metadata** — store voice info in NATS Key-Value store + +## Decision Outcome + +Chosen option: **Option 1 — File-system VoiceRegistry with periodic refresh**, because it introduces zero new infrastructure, uses the model store as the single source of truth, and aligns with the export layout already produced by the `coqui-voice-training` workflow. + +### Positive Consequences + +* Zero additional infrastructure — reads directly from the NFS volume +* Single source of truth — the trained model directory is the registry +* Newly trained voices appear automatically within the refresh interval +* On-demand refresh available via NATS for immediate availability +* Fully backward compatible — existing `speaker_wav_b64` cloning unchanged + +### Negative Consequences + +* Polling-based discovery adds slight latency (mitigated by configurable interval and on-demand refresh) +* No metadata beyond what `model_info.json` contains (sufficient for current needs) +* Requires NFS volume mounted at `VOICE_MODEL_STORE` path in the TTS pod + +## Implementation + +### VoiceRegistry + +A `VoiceRegistry` class scans `VOICE_MODEL_STORE` (default `/models/tts/custom`) for voice directories. Each directory must contain: + +| File | Required | Description | +|------|----------|-------------| +| `model_info.json` | Yes | Metadata: name, language, type, created_at | +| `model.pth` | Yes | Trained model weights | +| `config.json` | No | Model configuration | + +The registry is refreshed: +- On service startup +- Periodically every `VOICE_REGISTRY_REFRESH_SECONDS` (default 300s) +- On demand via `ai.voice.tts.voices.refresh` NATS subject + +### Synthesis Routing + +When a TTS request specifies a `speaker`, the service checks the registry first: + +``` +Request with speaker="my-voice" + ├─ Found in VoiceRegistry → send model_path + config_path to XTTS + ├─ Not found + speaker_wav_b64 present → ad-hoc voice cloning (existing) + └─ Not found + no speaker_wav_b64 → use default speaker +``` + +### New NATS Subjects + +| Subject | Pattern | Description | +|---------|---------|-------------| +| `ai.voice.tts.voices.list` | Request-reply | List default speaker + all custom voices | +| `ai.voice.tts.voices.refresh` | Request-reply | Trigger immediate registry rescan | + +### New Environment Variables + +| Variable | Default | Description | +|----------|---------|-------------| +| `VOICE_MODEL_STORE` | `/models/tts/custom` | NFS path to trained voice models | +| `VOICE_REGISTRY_REFRESH_SECONDS` | `300` | Periodic rescan interval (seconds) | + +### Integration with Training Pipeline + +``` +coqui-voice-training Argo Workflow + └─ export-trained-model step + └─ Writes to /models/tts/custom/{voice-name}/ + ├── model.pth + ├── config.json + └── model_info.json + +TTS Streaming Service + └─ VoiceRegistry + └─ Scans /models/tts/custom/ + └─ Registers {voice-name} → CustomVoice(model_path, config_path, ...) +``` + +## Links + +* Related: [ADR-0009](0009-dual-workflow-engines.md) — Argo/Kubeflow workflow engines +* Related: [ADR-0011](0011-kuberay-unified-gpu-backend.md) — XTTS runs on the KubeRay GPU backend +* Workflow: `argo/coqui-voice-training.yaml` +* Module: `tts-module/tts_streaming.py` diff --git a/decisions/0057-renovate-per-repo-configs.md b/decisions/0057-renovate-per-repo-configs.md new file mode 100644 index 0000000..b8b755a --- /dev/null +++ b/decisions/0057-renovate-per-repo-configs.md @@ -0,0 +1,251 @@ +# Per-Repository Renovate Configurations + +* Status: accepted +* Date: 2026-02-13 +* Deciders: Billy +* Technical Story: Roll out Renovate configs to all application repos so the self-hosted CronJob (ADR-0036) can scan them for dependency and security updates + +## Context and Problem Statement + +ADR-0036 deployed a **self-hosted Renovate CronJob** that auto-discovers all `daviestechlabs/*` repos, and the `homelab-k8s2` GitOps repo already has a detailed `.renovaterc.json5`. However, none of the application repositories contain a `renovate.json` yet, which means: + +- Renovate falls back to its bare defaults (no grouping, no auto-merge, no schedule control). +- Python repos with both `pyproject.toml` and `requirements.txt` get duplicate PRs. +- No security-update fast-path is configured. +- Major updates are auto-merged without review because no rule prevents it. + +We need a consistent per-repo configuration that applies the correct managers, grouping, auto-merge policy, and security rules to every repo. + +## Decision Drivers + +* Consistent behaviour across all repos in the org +* Correct manager selection per ecosystem (Python/Go/Node/Docker) +* Security updates treated with highest priority +* Non-major updates grouped and auto-merged to reduce PR noise +* Major updates require manual review +* Schedule aligned with CI runner availability + +## Considered Options + +1. **Shared org-preset in a dedicated `renovate-config` repo** +2. **Identical standalone `renovate.json` copied into every repo** +3. **No per-repo config (rely on autodiscover defaults)** + +## Decision Outcome + +Chosen option: **Option 1 — Shared org-preset with thin per-repo `renovate.json`** + +A central `renovate-config` repo holds a `default.json` preset that every application repo extends. Repo-specific overrides (extra managers, ignored paths) live in each repo's `renovate.json`. This keeps configuration DRY while allowing per-repo tailoring. + +### Positive Consequences + +* Single place to update grouping strategy, schedule, and auto-merge policy +* Each repo's `renovate.json` is 5-10 lines — easy to audit +* Security updates auto-merge immediately across all repos +* Major updates always require manual review + +### Negative Consequences + +* Extra repository (`renovate-config`) to maintain +* Preset changes propagate to all repos on next run — regressions possible + +## Shared Preset (`renovate-config` repo) + +### `default.json` + +```json +{ + "$schema": "https://docs.renovatebot.com/renovate-schema.json", + "description": "DaviesTechLabs shared Renovate preset for application repos", + "extends": [ + "config:recommended", + "group:allNonMajor", + ":automergeMinor", + ":automergePatch", + ":semanticCommits" + ], + "dependencyDashboard": true, + "platformAutomerge": true, + "schedule": ["before 6am on monday"], + "timezone": "America/New_York", + "prCreation": "immediate", + "vulnerabilityAlerts": { + "enabled": true, + "labels": ["security"] + }, + "packageRules": [ + { + "description": "Auto-merge security updates immediately", + "matchCategories": ["security"], + "automerge": true, + "schedule": ["at any time"], + "prPriority": 10 + }, + { + "description": "Major updates require manual review", + "matchUpdateTypes": ["major"], + "automerge": false, + "labels": ["major-update"] + }, + { + "description": "Group Gitea Actions updates", + "matchManagers": ["gitea-actions"], + "groupName": "gitea-actions" + }, + { + "description": "Group Docker base-image updates", + "matchManagers": ["dockerfile"], + "groupName": "docker-base-images" + }, + { + "description": "Pin uv in CI to digest for reproducibility", + "matchManagers": ["gitea-actions"], + "matchPackageNames": ["astral-sh/setup-uv"], + "pinDigests": true + } + ] +} +``` + +### `python.json` (supplemental preset for Python repos) + +```json +{ + "$schema": "https://docs.renovatebot.com/renovate-schema.json", + "description": "Python-specific rules for uv/pip repos", + "packageRules": [ + { + "description": "Group Python dev dependencies", + "matchManagers": ["pep621", "pip_requirements"], + "matchDepTypes": ["devDependencies", "optional-dependencies"], + "groupName": "python-dev-deps" + }, + { + "description": "Prefer pyproject.toml over requirements.txt when both exist", + "matchManagers": ["pip_requirements"], + "matchFileNames": ["requirements.txt"], + "enabled": false + } + ] +} +``` + +### `golang.json` (supplemental preset for Go repos) + +```json +{ + "$schema": "https://docs.renovatebot.com/renovate-schema.json", + "description": "Go-specific rules", + "packageRules": [ + { + "description": "Group all Go module updates", + "matchManagers": ["gomod"], + "groupName": "go-modules" + } + ], + "postUpdateOptions": ["gomodTidy"] +} +``` + +## Per-Repo Configuration + +Each application repo gets a minimal `renovate.json` that extends the org preset. + +### Python repos (chat-handler, voice-assistant, handler-base, pipeline-bridge, stt-module, tts-module, ray-serve, mlflow, gradio-ui) + +```json +{ + "$schema": "https://docs.renovatebot.com/renovate-schema.json", + "extends": [ + "local>daviestechlabs/renovate-config", + "local>daviestechlabs/renovate-config:python" + ] +} +``` + +### Go repos (companions-frontend, ntfy-discord) + +```json +{ + "$schema": "https://docs.renovatebot.com/renovate-schema.json", + "extends": [ + "local>daviestechlabs/renovate-config", + "local>daviestechlabs/renovate-config:golang" + ] +} +``` + +### companions-frontend (Go + Node hybrid) + +```json +{ + "$schema": "https://docs.renovatebot.com/renovate-schema.json", + "extends": [ + "local>daviestechlabs/renovate-config", + "local>daviestechlabs/renovate-config:golang" + ], + "packageRules": [ + { + "description": "Group npm dev dependencies", + "matchManagers": ["npm"], + "matchDepTypes": ["devDependencies"], + "groupName": "npm-dev-deps" + } + ] +} +``` + +## Repo Coverage Matrix + +| Repository | Ecosystem | Preset(s) | Notes | +|---|---|---|---| +| homelab-k8s2 | Helm/Flux/K8s/Docker | Own `.renovaterc.json5` | Already configured | +| chat-handler | Python, Docker, Gitea Actions | default + python | | +| voice-assistant | Python, Docker, Gitea Actions | default + python | | +| handler-base | Python, Docker, Gitea Actions | default + python | | +| pipeline-bridge | Python, Docker, Gitea Actions | default + python | | +| stt-module | Python, Docker, Gitea Actions | default + python | requirements.txt disabled | +| tts-module | Python, Docker, Gitea Actions | default + python | requirements.txt disabled | +| ray-serve | Python, Gitea Actions | default + python | requirements.txt disabled | +| mlflow | Python, Gitea Actions | default + python | requirements.txt disabled | +| gradio-ui | Python (requirements.txt only), Docker | default + python | No pyproject.toml — requirements.txt stays enabled | +| kuberay-images | Python (amdsmi-shim), Docker | default + python | Multiple Dockerfiles | +| companions-frontend | Go, Node, Docker | default + golang + npm | Hybrid repo | +| ntfy-discord | Go, Docker, Gitea Actions | default + golang | | +| kubeflow | Gitea Actions only | default | Pipeline definitions only | +| argo | None | default | Workflow templates only | + +## Update Flow + +``` + Renovate CronJob (every 8h) + │ + ▼ + Autodiscover daviestechlabs/* + │ + ▼ + Read repo's renovate.json + │ + ├── extends local>daviestechlabs/renovate-config + │ │ + │ └── Fetches default.json + python.json/golang.json + │ + ▼ + Scan dependencies (pyproject.toml, Dockerfile, go.mod, etc.) + │ + ▼ + Create/update PRs + │ + ├── Security → auto-merge immediately + ├── Patch/Minor → auto-merge (minor after 3-day stabilisation) + └── Major → label "major-update", await manual review +``` + +## Links + +* Supersedes: nothing (extends [ADR-0036](0036-renovate-dependency-updates.md)) +* Related: [ADR-0036](0036-renovate-dependency-updates.md) — Renovate CronJob deployment +* Related: [ADR-0013](0013-gitea-actions-for-ci.md) — Gitea Actions for CI +* Related: [ADR-0031](0031-gitea-cicd-strategy.md) — Gitea CI/CD Strategy +* Related: [ADR-0012](0012-use-uv-for-python-development.md) — uv for Python development +* [Renovate Shareable Config Presets](https://docs.renovatebot.com/config-presets/)