Files

Billy D. aeb1b749be feat: rewrite pipeline-bridge in Go

Replace Python implementation with Go for smaller container images.
Uses handler-base Go module for NATS, health, and telemetry.

- main.go: pipeline bridge with Argo/Kubeflow HTTP submission
- main_test.go: 8 tests covering helpers and HTTP submit functions
- Dockerfile: multi-stage golang:1.25-alpine → scratch
- CI: Gitea Actions with lint/test/release/docker/notify

2026-02-19 17:52:31 -05:00

1.1 KiB

Raw Permalink Blame History

pipeline-bridge

Bridges NATS events to Argo Workflows and Kubeflow Pipelines. Subscribes to ai.pipeline.trigger and submits workflow runs to the appropriate engine, publishing status updates to ai.pipeline.status.{request_id}.

Supported Pipelines

Name	Engine	Template / Pipeline ID
document-ingestion	Argo	document-ingestion
batch-inference	Argo	batch-inference
model-evaluation	Argo	model-evaluation
rag-query	Kubeflow	rag-pipeline
voice-pipeline	Kubeflow	voice-pipeline

Configuration

Variable	Default	Description
`KUBEFLOW_HOST`	`http://ml-pipeline.kubeflow.svc.cluster.local:8888`	Kubeflow Pipelines API
`ARGO_HOST`	`http://argo-server.argo.svc.cluster.local:2746`	Argo Server API
`ARGO_NAMESPACE`	`ai-ml`	Namespace for Argo workflows

Plus all standard handler-base settings (NATS_URL, OTEL_*, HEALTH_PORT, etc.).

Build

go build -o pipeline-bridge .

Test

go test -v -race ./...

Docker

docker build -t pipeline-bridge .

1.1 KiB Raw Permalink Blame History