大模型
x.com ·
@andrewchen
开发者分享本地 AI 实验 homelab 配置：5090 eGPU、双 DGX Spark 与本地/云端路由
开发者 andrew chen 描述其本地 AI 实验环境：Framework Desktop AI Max+ 395 主机、5090 eGPU 运行 Qwen 3.8 27B（有时 150+ tok/s）、两台 DGX Spark 运行 Deepseek v4 Flash 用于批处理后台任务、树莓派 5 监控、Mac mini 作开发机；Hermes 默认走本地 AI，并通过自研路由插件 Arch-Router 判断走本地还是云端，目前约 60-70% 本地化，目标是逐步全本地。作者强调该配置并非必需，并称后悔购买 eGPU。
AI推荐理由 作者以第一人称列出本地 AI 实验的完整硬件与模型配置，包括 Framework Desktop AI Max+ 395、5090 eGPU 跑 Qwen 3.8 27B 达 150+ tok/s、2x DGX Spark 跑 Deepseek v4 Flash、本地/云端路由插件 Arch-Router 及约 60-70% 本地化比例，对关注本地推理与算力配置的开发者有参考价值；但属个人经验分享，无基准测试或可复现数据，且明确表示配置并非必需、后悔购买 eGPU，需保留其主观性。
66
浏览 5,394
点赞 45
评论 18
转发 6
正文
current homelab setup for local AI experimentation:
- hermes box hosted on a Framework Desktop Mainboard AI Max+ 395
- 5090 eGPU running Qwen 3.8 27B for fast tok/s LLM use - sometimes 150+ tok/s
- 2x DGX Spark: running Deepseek v4 Flash 0731 - better but slower model
- pi 5 for monitoring
- Mac mini as a dev box - use Herdr and ohmypi/codex/claude depending on the use case
- housed in a 10" DeskPi mini rack (mostly)
Hermes is defaulted to local AI but with a homegrown routing plugin hitting a small low TTFT model (Arch-Router) to decide whether to go local or upgrade to cloud/frontier. Trying to get to 100% local over time, but right now probably more like 60-70%
The Sparks are for batch processing background runs (all the cron jobs, longer dev builds, etc)
do you need all of this? Absolutely not lol. I started with the mac mini and couldn't help myself but to add over time!
Also I regret getting the eGPU so I wouldn't recommend that to anyone