智能体
x.com ·
@petergyang
观点：AI agent 的参与度不等于价值，应看 trace 与 eval
Peter Yang 认为，对 AI agent 而言用户停留时长不代表价值，应通过对话、响应和工具调用等 trace 结合 eval 来评估 agent 是否完成任务。他借此介绍 Amplitude 的 Agent Analytics，并引用其数据：The Economist 达到 96.9% 任务成功率、周任务失败减少 84%；在 2 万+ Amplitude 用户中，首次正向 agent 体验与 3 倍留存相关。上述数据为厂商自述，未经独立验证。
AI推荐理由 提出 agent 场景下“参与度不等于价值”、应看 trace 与 eval 的观点，并给出 Amplitude Agent Analytics 的具体指标（The Economist 任务成功率 96.9%、周任务失败下降 84%；2 万+用户中首次正向 agent 体验与 3 倍留存相关）。但内容带有明显产品推广性质，数据来自厂商自述，缺乏独立验证，需谨慎看待。
62
浏览 3,134
点赞 16
评论 10
转发 2
正文
With AI agents, more user engagement does not necessarily mean more value.
A user can spend five minutes talking to an agent and leave frustrated because it never answered their question.
Instead, you have to look at the traces - the actual conversations, responses, and tool calls, to understand whether the agent did its job.
That’s why I’m excited to try [@Amplitude\_HQ](https://x.com/Amplitude_HQ)'s new Agent Analytics. It combines traces with evals that measure the quality of agent interactions, then connects both to what users do next.
This way, teams can see:
- What the user was trying to accomplish
- Where the agent failed
- Whether good and bad answers affected KPIs
Early results show why this matters:
- The Economist (see below) reached 96.9% task success and cut weekly task failures by 84%.
- Across 20K+ Amplitude users, a positive first agent experience was associated with 3× higher retention.
Check out Agent Analytics here: [amplitude.com/agent-analytics](https://amplitude.com/agent-analytics)
Amplitude@Amplitude\_HQ
Since 1843 [@TheEconomist](https://x.com/TheEconomist) has been fact-checking the world. Now they're fact-checking their AI.
Here’s how The Economist improved their AI research assistant with Amplitude Agent Analytics.
[bit.ly/4zkd2x0](https://bit.ly/4zkd2x0)
[Your browser does not support the video tag.](https://video.twimg.com/amplify_video/2088323272479797249/vid/avc1/1294x720/BzzJqG6dfMrlb8gH.mp4?tag=14)
[🔗 View on Twitter](https://x.com/Amplitude_HQ/status/2088323335817941328/video/1)
[🔗 View Quoted Tweet](https://x.com/Amplitude_HQ/status/2088323335817941328)