Billy D.
8f9b2203ca
- e2e_test.go: Argo/Kubeflow full payload validation, concurrent requests - Pipeline dispatch validation, unknown pipeline error handling - Benchmarks: Argo 218µs/op, Kubeflow 179µs/op
pipeline-bridge
Bridges NATS events to Argo Workflows and Kubeflow Pipelines. Subscribes to
ai.pipeline.trigger and submits workflow runs to the appropriate engine,
publishing status updates to ai.pipeline.status.{request_id}.
Supported Pipelines
| Name | Engine | Template / Pipeline ID |
|---|---|---|
| document-ingestion | Argo | document-ingestion |
| batch-inference | Argo | batch-inference |
| model-evaluation | Argo | model-evaluation |
| rag-query | Kubeflow | rag-pipeline |
| voice-pipeline | Kubeflow | voice-pipeline |
Configuration
| Variable | Default | Description |
|---|---|---|
KUBEFLOW_HOST |
http://ml-pipeline.kubeflow.svc.cluster.local:8888 |
Kubeflow Pipelines API |
ARGO_HOST |
http://argo-server.argo.svc.cluster.local:2746 |
Argo Server API |
ARGO_NAMESPACE |
ai-ml |
Namespace for Argo workflows |
Plus all standard handler-base settings (NATS_URL, OTEL_*, HEALTH_PORT, etc.).
Build
go build -o pipeline-bridge .
Test
go test -v -race ./...
Docker
docker build -t pipeline-bridge .