大模型
arxiv.org ·
EnterpriseVal：量化生成式 AI 在企业中的有效性、可靠性与价值
该 arXiv 预印本指出，前沿模型已能在相当比例的经济价值任务上产出与人类相当的专业交付物，但多数企业 GenAI 项目难以证明可测量的业务效果，作者认为这主要是测量问题：公开基准回答“模型能做什么”，而部署决策需要回答“该工作流是否适配、可靠、安全且值得扩展”。为此提出 EnterpriseVal 用例级评估系统，包含用例与冻结配置（模型、提示、检索、工具、护栏、人工监督）的形式化规范及自主性等级与后果分级、六类指标目录、结合盲评专家判断与校准 LLM-as-judge 的评分协议、将带置信区间的指标向量映射为 REJECT/CONDITIONAL/SCALE 的两级阈值门算法，以及以审核者捕获率为可测参数的价值-风险模型。作者报告在一家全球银行三个工作流的试点结果：信贷备忘录起草中最佳模型人工评分引用精确率 88%、幻觉率 1.6%（门槛 70% 与 5%）；流程改造中分析师精修工作量从估计 27.4 小时降至 2.9 小时。论文区分已确立结果、试点证据、所提系统与开放假设，并列出完整验证所需实验。
AI推荐理由 arXiv 预印本，提出面向企业用例的生成式 AI 评估系统 EnterpriseVal，包含用例与冻结社会技术配置的形式化规范、覆盖保真度/效用/效率/可靠性/保障/监督的指标目录、结合盲评专家判断与校准 LLM-as-judge 的评分协议、两级阈值门（REJECT/CONDITIONAL/SCALE）以及把审核者捕获率作为可测参数的价值-风险模型。作者报告在一家全球银行的三个工作流试点：信贷备忘录起草中最佳模型人工评分的引用精确率 88%、幻觉率 1.6%（门槛为 70% 和 5%）；流程改造中分析师精修工作量从估计的 27.4 小时降至 2.9 小时。论文明确区分已确立结果、试点证据、所提系统与开放假设，并列出完整验证所需实验，未声称已全面验证。局限是仅单机构试点、样本有限，且为预印本未经同行评审。
78
正文
Computer Science > Artificial Intelligence
arXiv:2609.21841 (cs)
[Submitted on 18 Sep 2026]
Title:EnterpriseVal: Quantifying the Efficacy, Reliability and Value of Generative AI in the Enterprise
Authors:Abbas Raza Ali, Muhammad Ajmal Siddiqui, Moona Zahid
View a PDF of the paper titled EnterpriseVal: Quantifying the Efficacy, Reliability and Value of Generative AI in the Enterprise, by Abbas Raza Ali and Muhammad Ajmal Siddiqui and Moona Zahid
View PDF
HTML (experimental)
Abstract:Frontier language models now produce professional deliverables that expert graders judge to match human work on a substantial share of economically valuable tasks, yet most enterprise GenAI initiatives fail to show a measurable business effect and a large fraction of agentic projects are expected to be cancelled. We argue that this is substantially a measurement problem: public benchmarks answer "what can the model do?", whereas a deployment decision requires "is this workflow fit, reliable, safe and worth scaling - here, on our data, under our controls?". We present EnterpriseVal, a use-case-level evaluation system that closes this gap. It comprises (i) a formal specification of the use case and of the frozen socio-technical configuration under test, model, prompts, retrieval, tools, guardrails and human oversight, with an autonomy level and consequence tier that jointly set the required evaluation intensity; (ii) a metric catalogue spanning fidelity, utility, efficiency, reliability, assurance and oversight; (iii) a grading protocol that scales blinded expert judgement with calibrated LLM-as-judge scoring through prediction-powered inference; (iv) a two-tier threshold gate, stated as an executable algorithm, that maps metric vectors with confidence bounds to REJECT/CONDITIONAL/SCALE decisions; and (v) a value-and-risk model in which the reviewer catch rate is a measured parameter. We report a pilot across three workflows in a global bank. In credit-memo drafting, human-graded citation precision reached 88% and hallucination rate 1.6% for the best model against gates of 70% and 5%; in procedure transformation, analyst refinement effort fell from an estimated 27.4 to 2.9 hours per document. We separate established results, documented pilot evidence, the proposed system and open hypotheses, and specify the experiments required for full validation
Subjects:
Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
Cite as:
arXiv:2609.21841 [cs.AI]
(or
arXiv:2609.21841v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.21841
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Abbas Raza Ali [view email]
[v1]
Fri, 18 Sep 2026 14:39:26 UTC (1,319 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled EnterpriseVal: Quantifying the Efficacy, Reliability and Value of Generative AI in the Enterprise, by Abbas Raza Ali and Muhammad Ajmal Siddiqui and Moona Zahid
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.AI
new
|
recent
| 2026-09
Change to browse by:
cs
stat
stat.ML
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)