feat: Add Kubeflow Pipeline definitions

- voice_pipeline: STT → RAG → LLM → TTS
- document_ingestion_pipeline: Extract → Chunk → Embed → Milvus
- document_ingestion_mlflow_pipeline: With MLflow tracking
- evaluation_pipeline: Model benchmarking
- kfp-sync-job: K8s job to sync pipelines

README.md

@@ -1,2 +1,97 @@
-# kubeflow
+# Kubeflow Pipelines
 
+Kubeflow Pipeline definitions for the DaviesTechLabs AI/ML platform.
+
+## Pipelines
+
+| Pipeline | Description | Output |
+|----------|-------------|--------|
+| `voice_pipeline.py` | STT → RAG → LLM → TTS | voice_pipeline.yaml |
+| `document_ingestion_pipeline.py` | Ingest docs → chunk → embed → Milvus | document_ingestion.yaml |
+| `document_ingestion_mlflow_pipeline.py` | Same with MLflow tracking | document_ingestion_mlflow.yaml |
+| `evaluation_pipeline.py` | Evaluate models against benchmarks | evaluation.yaml |
+
+## Usage
+
+### Compile Pipelines
+
+```bash
+pip install kfp==2.12.1
+
+# Compile all
+python voice_pipeline.py
+python document_ingestion_pipeline.py
+python evaluation_pipeline.py
+
+# Or compile individually
+python -c "from kfp import compiler; from voice_pipeline import voice_assistant_pipeline; compiler.Compiler().compile(voice_assistant_pipeline, 'voice.yaml')"
+```
+
+### Upload to Kubeflow
+
+Upload the generated `.yaml` files to Kubeflow Pipelines UI or use the SDK:
+
+```python
+import kfp
+client = kfp.Client(host='http://kubeflow.ai-ml.svc.cluster.local/pipeline')
+client.upload_pipeline('voice_pipeline.yaml', pipeline_name='Voice Assistant')
+```
+
+## Pipeline Details
+
+### voice_pipeline
+
+Full voice assistant with RAG:
+1. Transcribe audio (Whisper)
+2. Generate embeddings (BGE)
+3. Search Milvus
+4. Rerank documents (BGE Reranker)
+5. Generate response (vLLM)
+6. Synthesize speech (XTTS)
+
+### document_ingestion_pipeline
+
+Ingest documents into vector DB:
+1. Extract text (PDF, DOCX, HTML, TXT)
+2. Chunk with overlap (tiktoken)
+3. Generate embeddings
+4. Store in Milvus collection
+
+### evaluation_pipeline
+
+Benchmark model quality:
+1. Load eval dataset (MMLU, etc.)
+2. Run inference
+3. Calculate metrics (accuracy, F1)
+4. Log to MLflow
+
+## Integration
+
+### kfp-sync-job.yaml
+
+Kubernetes Job to sync compiled pipelines to Kubeflow:
+
+```bash
+kubectl apply -f kfp-sync-job.yaml
+```
+
+### From Argo Workflows
+
+Pipelines can be triggered from Argo via the kfp-integration workflow in the [argo](https://git.daviestechlabs.io/daviestechlabs/argo) repo.
+
+## Service Endpoints
+
+| Service | Endpoint |
+|---------|----------|
+| Whisper STT | `http://whisper-predictor.ai-ml.svc.cluster.local` |
+| Embeddings | `http://embeddings-predictor.ai-ml.svc.cluster.local` |
+| Reranker | `http://reranker-predictor.ai-ml.svc.cluster.local` |
+| vLLM | `http://llm-draft.ai-ml.svc.cluster.local:8000` |
+| TTS | `http://tts-predictor.ai-ml.svc.cluster.local` |
+| Milvus | `milvus.ai-ml.svc.cluster.local:19530` |
+
+## Related
+
+- [argo](https://git.daviestechlabs.io/daviestechlabs/argo) - Argo Workflows for training
+- [voice-assistant](https://git.daviestechlabs.io/daviestechlabs/voice-assistant) - Real-time voice handler
+- [homelab-design](https://git.daviestechlabs.io/daviestechlabs/homelab-design) - Architecture docs