算力商业
prismml.com ·
@Hacker News
Bonsai 2 27B：以约 9 倍更小体积实现近无损压缩
PrismML 发布 Bonsai 2 27B，宣称在约 9 倍更小体积下实现近无损压缩。HN 评论提到 Ternary Bonsai 2 27B 在 NVIDIA GeForce RTX 5090 上最高 143 tokens/s、M5 Max 上 46.8 tokens/s，在 RTX 4090 上每 token 耗电 0.714 mWh，比全精度 8B 模型能效高 40%。有用户表示将在 16GB Intel B50 上尝试，但 XPU 核心能否加速该量化格式尚不确定。
AI推荐理由 Bonsai 2 27B 主打近无损压缩与更小占用，评论中给出 Ternary 版本在 RTX 5090 上最高 143 tokens/s、M5 Max 上 46.8 tokens/s，以及 RTX 4090 上 0.714 mWh/token、比全精度 8B 模型能效高 40% 的具体数字，对本地部署和消费级显卡用户有参考价值；但正文仅来自厂商页面与 HN 评论，缺少独立复现，且 16GB 显卡能否加速仍属用户猜测，需保留不确定性。
62
正文
Domain: prismml.com
Points: 7 | Comments: 3
[Top Comments]
- Love this for the folks with 16gb graphics cards - 3.8 27b has been incredible but not quite runnable on anything less than 32gb - will try loading this up on my 16gb intel b50 and see how it goes - not sure these quants can be accelerated by the XPU cores yet but maybe in time!
- I'm not following the local mdoel scene too closely but this seems quite amazing. Is this able to be run on Apple silicon too?
- "Ternary Bonsai 2 27B reaches up to 143 tokens/second on NVIDIA GeForce RTX 5090 and 46.8 tokens/second on M5 Max. On an RTX 4090, Ternary Bonsai 2 27B consumes just 0.714 mWh/token, making it 40% more energy-efficient than an 8B model running in full-precision."