EVA-Bench Data 2.0基准发布：覆盖3个领域、121个工具和213个测试场景

This article is a deep-dive from JudyAI Lab — an AI engineering playbook series with 100+ published guides, 5,000+ weekly readers across 60+ countries, focused on the practical side of running AI agents, trading systems, and content pipelines in production. 📰 Key Summary ServiceNow AI's research team has released EVA-Bench Data 2.0, an enterprise-grade benchmark designed specifically for voice agents. This release dramatically expands the scope, moving from a single domain to three major enterprise scenarios: airline customer service management (CSM), enterprise IT service management (ITSM), and healthcare human resources service delivery (HRSD). Together, the three domains cover 213 evaluation scenarios and 121 tools, roughly four times the coverage of the original version. Broken down by domain: airline has 50 scenarios, ITSM has 80, and HRSD has 83. This benchmark places particular emphasis on real-world voice scenarios — every data point was filtered starting from actual phone-based customer service workflows, with tool schemas modeled after production API specs. The healthcare HRSD domain goes even deeper, tying into real US healthcare policy details like NPI provider identifiers, FMLA family leave regulations, and insurance coverage rules, ensuring the evaluation scenarios match what practitioners actually deal with day to day. All 213 scenarios were cross-validated for solvability by three frontier models — OpenAI's GPT-5.4, Google's Gemini 3.1 Pro, and Anthropic's Claude Opus 4.6 — to keep the benchmark challenging while ensuring results stay fair and trustworthy. All three datasets are fully open-sourced and can be loaded directly through Hugging Face Datasets. The team has also announced an upcoming multilingual expansion, which will push the benchmark's scope beyond its current English-only enterprise deployment limitations. The full design principles and generation process are documented in the original post — a solid implementation reference for anyone looking to build their own eval dataset. 💬 JudyAI Lab Take ServiceNow expanding its voice agent benchmark from a single domain to three major enterprise scenarios — airline, ITSM, and healthcare — and open-sourcing all of it signals, in our view, that standardizing enterprise AI voice evaluation has moved from a conceptual discussion to something you can actually deploy as a tool. The most notable design principle here is filtering scenarios "starting from real phone-based customer service workflows" rather than dreaming up test questions from scratch. This approach reflects a consensus that's taking shape: if enterprise voice AI evaluation is disconnected from actual business workflows, the resulting scores often fail to predict production performance. Having all 213 scenarios cross-validated for solvability across GPT-5.4, Gemini 3.1 Pro, and Claude Opus 4.6 — this multi-model consensus design ensures the benchmark stays both challenging and fair, rather than being friendly to just one model family. The healthcare HRSD scenarios incorporating details like NPI identifiers, FMLA leave rules, and insurance coverage also show that eval data for highly regulated domains needs to hit a certain density of business-level detail before it can actually measure meaningful differences. If you're designing an enterprise AI eval dataset, the scenario generation files in this open-source release are a solid starting point to reference directly — reverse-engineering test scenarios from business workflows exposes production gaps more effectively than forward-designing from model capability dimensions. 📅 Original Article Info Published: 2026-06-04T12:24 Source: https://huggingface.co/blog/ServiceNow-AI/eva-bench-data 🔗 Further Reading The Rise of Personalized AI Models: Tailoring Intelligence for Your Business From Trading Idea to Live Execution: The Real Workflow of AI-Assisted Strategy Development References EVA-Bench Data 2.0: 3 Domains, 121 Tools, 213 Scenarios AI evals are broken, but builders still need them - DEV Community OpenAI Officially Deprecates SWE-bench Verified: The "Gold Standard" for Coding Ability ... Originally published at Judy AI Lab. Visit for more articles on AI engineering and development.