Billy D. 87d0545d2c feat: replace fake streaming with real SSE StreamGenerate ...

Use handler-base StreamGenerate() to publish real token-by-token
ChatStreamChunk messages to NATS as they arrive from Ray Serve,
instead of calling Generate() and splitting into 4-word chunks.

Add 8 streaming tests: happy path, system prompt, RAG context,
nil callback, timeout, HTTP error, context canceled, fallback.

2026-02-21 09:23:57 -05:00

6.4 KiB

Raw Blame History

View Raw