Billy D.
f6623d5473
- Replace gitea-http.gitea.svc.cluster.local:3000 with registry.lab.daviestechlabs.io - Use docker/login-action@v3 for Gitea registry auth (proper buildx integration) - Remove manual base64 auth to ~/.docker/config.json (not picked up by buildkit) - Remove insecure registry daemon.json config and Docker restart - Remove buildkitd insecure registry config - Remove cache-from/cache-to type=gha (not supported on Gitea Actions) Fixes 401 Unauthorized: reqPackageAccess on Docker push
Chat Handler
Text-based chat pipeline for the DaviesTechLabs AI/ML platform.
Overview
A NATS-based service that handles chat completion requests with RAG (Retrieval Augmented Generation). It uses the handler-base library for standardized NATS handling, telemetry, and health checks.
Pipeline: Query → Embeddings → Milvus → Rerank → LLM → (optional TTS)
Architecture
NATS (ai.chat.request)
│
▼
┌───────────────────┐
│ Chat Handler │
└───────────────────┘
│
├──▶ BGE Embeddings (drizzt)
│ │
│ ▼
├──▶ Milvus Vector Search
│ │
│ ▼
├──▶ BGE Reranker (danilo)
│ │
│ ▼
├──▶ vLLM (khelben)
│ │
│ ▼ (optional)
└──▶ XTTS TTS (elminster)
│
▼
NATS (ai.chat.response.{id})
NATS Message Format
Request (ai.chat.request)
{
"request_id": "uuid",
"query": "What is the capital of France?",
"collection": "knowledge_base",
"enable_tts": false,
"system_prompt": "Optional custom system prompt"
}
Response (ai.chat.response.{request_id})
{
"request_id": "uuid",
"response": "The capital of France is Paris.",
"sources": [
{"text": "Paris is the capital...", "score": 0.95}
],
"audio": "base64-encoded-audio (if TTS enabled)"
}
Configuration
| Environment Variable | Default | Description |
|---|---|---|
NATS_URL |
nats://nats.ai-ml.svc.cluster.local:4222 |
NATS server |
EMBEDDINGS_URL |
http://embeddings-predictor.ai-ml.svc.cluster.local |
Embeddings |
RERANKER_URL |
http://reranker-predictor.ai-ml.svc.cluster.local |
Reranker |
VLLM_URL |
http://llm-draft.ai-ml.svc.cluster.local:8000 |
LLM service |
TTS_URL |
http://tts-predictor.ai-ml.svc.cluster.local |
TTS (optional) |
MILVUS_HOST |
milvus.ai-ml.svc.cluster.local |
Vector DB |
COLLECTION_NAME |
knowledge_base |
Default Milvus collection |
ENABLE_TTS |
false |
Enable audio responses |
Building
docker build -t chat-handler:latest .
# With specific handler-base tag
docker build --build-arg BASE_TAG=latest -t chat-handler:latest .
Related
- handler-base - Base handler library
- voice-assistant - Voice pipeline
- homelab-design - Architecture docs