大模型
gradientflow.com ·
@Ben Lorica
并非每个 AI 任务都需要 LLM
Gradient Flow 文章指出，LLM 擅长生成文本，但大量 AI 自动化并不需要文本，只需要一个小决策，例如判断是否为垃圾邮件、工单该分给哪个团队、紧急程度如何。作者认为强行让 LLM 返回结构化输出，本质上仍是逐 token 生成，未必是最优选择。
AI推荐理由 Gradient Flow 文章指出并非所有 AI 任务都需要 LLM：垃圾邮件判定、工单分派、紧急度评估等只需小型决策，用 LLM 生成结构化输出仍是逐 token 生成，成本与延迟不划算。属方法论观点，对 AI 工程选型有实际参考价值，但摘要未给出具体替代方案或基准数据。
66
正文
LLMs are built to generate text, but a surprising amount of AI automation does not need text at all. It needs a small decision: Is this spam? Which team should get this ticket? How urgent is it? We can force an LLM to return structured output, but it is still generating that answer token by token, which adds latency and cost when you do it millions of times. The model can usually handle these tasks, but I increasingly think we are using more model than the job requires.
Jev is an attempt to build a model specifically for these small decisions. You give it some application data and a narrow question with fixed possible answers, and it returns an answer plus a probability rather than generating prose. Your software can then use that probability to act automatically or hand the case to a person or a larger model. The early numbers are interesting, with latency measured in tens or hundreds of milliseconds and very low input costs. I would still treat the benchmarks cautiously, since most come from the company itself, and a well-formed answer can still be the wrong answer.
What makes this category particularly interesting to me is how quickly open alternatives started appearing. Laya was an open source precursor, and several OpenJev-style projects show that ordinary open models can be adapted to return probabilities over a fixed set of choices. Jev may still have an important lead in model quality and calibration, meaning how closely its confidence scores track its actual accuracy. But the interface itself is simple enough that I suspect decide eventually joins generate, embed, and rerank as a standard model call. If that happens, the lasting differentiation will come from quality, reliability, and trust rather than ownership of the interface.
Subscribe to our weekly newsletter
Share this:
Share on X (Opens in new window)
X
Share on Facebook (Opens in new window)
Facebook
Share on LinkedIn (Opens in new window)
LinkedIn
Email a link to a friend (Opens in new window)
Email
Like this:
Like Loading…