Billy D.
1c5dc7f751
Each UI now logs per-request metrics to MLflow: - llm.py: latency, tokens/sec, prompt/completion tokens (gradio-llm-tuning) - embeddings.py: latency, text length, batch size (gradio-embeddings-tuning) - stt.py: latency, audio duration, real-time factor (gradio-stt-tuning) - tts.py: latency, text length, audio duration (gradio-tts-tuning) Uses try/except guarded imports so UIs still work if MLflow is unreachable. Persistent run per Gradio instance, batched metric logging via MlflowClient.log_batch().
Gradio UI
Interactive Gradio web interfaces for the DaviesTechLabs AI/ML platform.
Apps
| App | Description | Port |
|---|---|---|
embeddings.py |
BGE Embeddings demo with similarity comparison | 7860 |
stt.py |
Whisper Speech-to-Text demo | 7861 |
tts.py |
XTTS Text-to-Speech demo | 7862 |
Features
- Consistent theme - Shared DaviesTechLabs theme via
theme.py - MLflow integration - Metrics logged for demo usage
- Service endpoints - Connect to KServe inference services
Running Locally
pip install -r requirements.txt
# Run individual apps
python embeddings.py # http://localhost:7860
python stt.py # http://localhost:7861
python tts.py # http://localhost:7862
Docker
# Build
docker build -t gradio-ui:latest .
# Run specific app
docker run -p 7860:7860 -e APP=embeddings gradio-ui:latest
docker run -p 7861:7861 -e APP=stt gradio-ui:latest
docker run -p 7862:7862 -e APP=tts gradio-ui:latest
Kubernetes Deployment
# Deploy all apps
kubectl apply -k .
# Or individual apps
kubectl apply -f embeddings.yaml
kubectl apply -f stt.yaml
kubectl apply -f tts.yaml
Configuration
| Environment Variable | Default | Description |
|---|---|---|
EMBEDDINGS_URL |
http://embeddings-predictor.ai-ml.svc.cluster.local |
Embeddings service |
WHISPER_URL |
http://whisper-predictor.ai-ml.svc.cluster.local |
STT service |
TTS_URL |
http://tts-predictor.ai-ml.svc.cluster.local |
TTS service |
MLFLOW_TRACKING_URI |
http://mlflow.mlflow.svc.cluster.local:80 |
MLflow server |
App Details
embeddings.py
- Generate embeddings for text input
- Batch embedding support
- Cosine similarity comparison
- Visual embedding dimension display
stt.py
- Upload audio or record from microphone
- Transcribe using Whisper
- Language detection
- Timestamp display
tts.py
- Text input for synthesis
- Voice selection
- Audio playback and download
- Speed/pitch controls
File Structure
gradio-ui/
├── embeddings.py # Embeddings demo
├── stt.py # Speech-to-Text demo
├── tts.py # Text-to-Speech demo
├── theme.py # Shared Gradio theme
├── requirements.txt # Python dependencies
├── Dockerfile # Container image
├── kustomization.yaml # Kustomize config
├── embeddings.yaml # K8s deployment
├── stt.yaml # K8s deployment
└── tts.yaml # K8s deployment
Related
- kuberay-images - Ray workers
- handler-base - Handler library
- homelab-design - Architecture docs