Billy D.
aeb1b749be
Replace Python implementation with Go for smaller container images. Uses handler-base Go module for NATS, health, and telemetry. - main.go: pipeline bridge with Argo/Kubeflow HTTP submission - main_test.go: 8 tests covering helpers and HTTP submit functions - Dockerfile: multi-stage golang:1.25-alpine → scratch - CI: Gitea Actions with lint/test/release/docker/notify
1.1 KiB
1.1 KiB
pipeline-bridge
Bridges NATS events to Argo Workflows and Kubeflow Pipelines. Subscribes to
ai.pipeline.trigger and submits workflow runs to the appropriate engine,
publishing status updates to ai.pipeline.status.{request_id}.
Supported Pipelines
| Name | Engine | Template / Pipeline ID |
|---|---|---|
| document-ingestion | Argo | document-ingestion |
| batch-inference | Argo | batch-inference |
| model-evaluation | Argo | model-evaluation |
| rag-query | Kubeflow | rag-pipeline |
| voice-pipeline | Kubeflow | voice-pipeline |
Configuration
| Variable | Default | Description |
|---|---|---|
KUBEFLOW_HOST |
http://ml-pipeline.kubeflow.svc.cluster.local:8888 |
Kubeflow Pipelines API |
ARGO_HOST |
http://argo-server.argo.svc.cluster.local:2746 |
Argo Server API |
ARGO_NAMESPACE |
ai-ml |
Namespace for Argo workflows |
Plus all standard handler-base settings (NATS_URL, OTEL_*, HEALTH_PORT, etc.).
Build
go build -o pipeline-bridge .
Test
go test -v -race ./...
Docker
docker build -t pipeline-bridge .