大模型
x.com ·
@omarsar0
作者用 Jev 重标注 2.3K 篇 AI 论文：成本 0.14 美元、耗时 83 秒
X 用户 omarsar0 分享了一个 Jev 的生产用例：对约 2.3K 篇 AI 研究论文重新打标签，总成本 0.14 美元、耗时约 83 秒。此前论文标签由旧开放模型 DeepSeek V4 Flash 生成，作者对其分类置信度不足。Jev 重标注后与旧标签一致率约 75%，识别出约 579 处高置信度主题变更；作者人工标注 30 个分歧样本并全部接受，称已在生产环境应用并验证这些变更。作者认为通过组合 System One 与 System Two 模型可显著改进流水线，并称仍认为有改进空间。
AI推荐理由 给出具体生产用例：用 Jev 重新标注约 2.3K 篇 AI 论文，成本 0.14 美元、耗时约 83 秒，与旧标签一致率 75%，发现约 579 处高置信度主题变更，并人工标注 30 个分歧样本全部接受。数据具体、可指向公开论文库，但为个人单方陈述，无第三方验证，且 Jev 与 DeepSeek V4 Flash 的对比条件未完全披露。
62
浏览 1,091
点赞 6
评论 4
转发 1
正文
Found a great production use case for Jev.
I used Jev to organize ~2.3K AI research papers.
The total cost was $0.14, and it took about 83 seconds.
The process:
The papers already had old tags, which I ran through a previous open model (DeepSeek V4 Flash).
However, I wasn't confident in the classifications, and I didn't want to spend more on tokens unless I spent time tuning it into a good LLM classifier via few-shot (more expensive). Too tedious, too costly, and unsustainable.
Luckily for us, we now have Jev to help us with organizing papers better.
So Jev first went through all the papers and retagged them. It agreed with 75% of the previous tags.
Jev found about 579 high-confidence topic changes.
I evaluated reliability by manually labeling 30 disagreements and accepted all of them. I was astonished by Jev's classification capabilities. We applied and verified all changes in production.
I'm much more satisfied with the classifications, but I think there is still room for improvement.
Check out the papers here: [academy.dair.ai/papers](https://academy.dair.ai/papers)
I will experiment with Jev more. The takeaway is that pipelines can be significantly improved by carefully combining System One and System Two models.
Jev clearly unlocks more interesting ways to organize papers and offer a more useful discovery layer for research papers. More updates on that soon.
[Your browser does not support the video tag.](https://video.twimg.com/amplify_video/2102061930106015745/vid/avc1/3776x2160/l9YOtP5mYKmtiVzj.mp4?tag=29)
[🔗 View on Twitter](https://x.com/omarsar0/status/2102066232383979749/video/1)