Skip to main content
Back to Projects

Mamba Server — Streaming AI Backend

AI/ML
PythonFastAPIPydantic AIOpenAIServer-Sent EventsVercel AI SDKDockerKubernetes
View Source

A production-ready FastAPI backend that wraps OpenAI models with Pydantic AI's agent architecture and delivers real-time streaming responses over Server-Sent Events. Built streaming-first with a layered architecture, it is a robust foundation for AI-powered chat applications and integrates natively with the Vercel AI SDK message format.

It can route requests to specialized, pre-configured agents (such as research or code review) and exposes display tools that let the model render structured UI.

Key capabilities:

  • Streaming chat completions over SSE in Vercel AI SDK format, with exponential-backoff retry for OpenAI calls
  • Four display tools — generateForm, generateChart, generateCode, generateCard — plus Mamba Agents integration for specialized routing
  • Kubernetes-ready with liveness/readiness probes, flexible auth (none, API key, or JWT), and multi-source configuration