大模型
dev.to ·
@Sergio Wolf Knapik
开源多供应商 LLM 路由：统一 Anthropic、Google、OpenAI、DeepSeek 的流式接口
作者将自研平台中的流式路由核心抽离开源为 FastAPI 模板，提供单一 POST /v1/chat/stream 端点，把不同厂商 SDK 的 SSE 输出统一为 thinking、content、tool_call、done 四类事件，并通过 models.json 动态配置模型目录，附带 pytest 测试与本地调试页面，采用 MIT 许可。项目为个人开源实现，未给出性能基准或生产环境验证。
AI推荐理由 作者开源了一个 FastAPI 多供应商 LLM 流式路由模板，统一 Anthropic、Google、OpenAI、DeepSeek 等接口，将 SSE 输出规范为 thinking、content、tool_call、done 四类事件，并用 models.json 动态管理模型目录。对需要多模型接入的开发者有直接复用价值；但属个人项目，未提供性能基准或生产验证，文中示例模型名（如 Claude Sonnet 5、Gemini 3.8 Flash）无法核实。
66
正文
If you've ever built an application that integrates with multiple LLM providers (Anthropic, Google, OpenAI, DeepSeek), you already know the pain:
Each provider has its own distinct Python SDK.
Streaming responses using Server-Sent Events (SSE) requires divergent parser logic.
Thinking / Reasoning blocks are formatted completely differently.
I recently extracted the core streaming router from my platform into an open-source FastAPI template. Here is how it works.
Objective
A single asynchronous endpoint:
POST /v1/chat/stream
It accepts a unified request payload and returns a standardized SSE stream emitting four clean events:
event: thinking — Internal model reasoning tokens (streamed in real-time).
event: content — User-facing response text.
event: tool_call — Function calling requests.
event: done — Stream completion ([DONE]).
Architecture
Instead of pulling heavy wrapper frameworks, use direct asynchronous HTTP via httpx.AsyncClient and the official Google GenAI SDK:
fastapi-multi-llm-starter/
├── app/
│ ├── config.py # Pydantic Settings loading environment variables
│ ├── main.py # FastAPI app with CORS, health check & test playground
│ ├── models.json # Dynamic model catalog (Claude, Gemini, GPT)
│ ├── router.py # Unified multi-provider async stream dispatcher
│ └── schemas.py # Strict Pydantic v2 validation models
├── tests/ # Automated unit tests (pytest)
├── requirements.txt
└── README.md
Enter fullscreen mode
Exit fullscreen mode
Dynamic Model Catalog
I disliked the idea of hardcoded models, so I decoupled them into a models.json file:
{
"models": [
{
"id": "claude-sonnet-5",
"name": "Claude Sonnet 5",
"provider": "Anthropic",
"thinking": true
},
{
"id": "gemini-3.8-flash",
"name": "Gemini 3.8 Flash",
"provider": "Google",
"thinking": true
},
{
"id": "gpt-5.6-terra",
"name": "GPT 5.6 Terra",
"provider": "OpenAI",
"thinking": true
}
]
}
Enter fullscreen mode
Exit fullscreen mode
Now, if you want to add another model, you just edit the JSON. The backend and the embedded UI dynamically populate available models via GET /v1/models.
Testing Playground
The repository includes a testing playground running directly at http://localhost:8000/. You can immediately test prompts, check streaming latency, and verify reasoning blocks without setting up a frontend framework.
Of course, you'll need your own API keys.
Open Source Code
The full core code is open-source under the MIT License on GitHub:
👉 github.com/wolfnomknight/fastapi-multi-llm-starter
Includes full pytest test coverage, .env.example, and clean Pydantic v2 schemas.
Feel free to fork it, use it in your side projects or micro-SaaS, and let me know if you run into any issues or have ideas for additional providers!