大模型
全球精选 ·
@Sungjune Lee, Myungjoo Kang
LatentMD：评测 LLM 生成文本中的 Markdown 边界失败
该论文提出 LatentMD 基准与评测协议，用于诊断 LLM 生成 Markdown 在 CommonMark 层面的围栏边界失败，将内容正确性与边界正确性分开评估。基准包含 4,179 条 prompt 及一个可对任意模型输出打分的 CLI。在 9 个 LLM、约 37,600 次生成中，38.0% 的有效主网格输出为内容正确但边界破损；未指定提示及小型人工验证集上也出现明显边界破损。消融表明失败主要源于同族对称分隔符冲突而非嵌套，提示提示只能部分缓解，并泛化到 Python 三引号 docstring，而 JSON 作为非对称分隔符对照保持稳健。
AI推荐理由 arXiv 论文，提出 LatentMD 基准与评测协议，专门诊断 LLM 生成 Markdown 的 CommonMark 围栏边界失败，将内容正确性与边界正确性分离。含 4,179 条 prompt、CLI 评分工具，并在 9 个 LLM、约 37,600 次生成上报告 38.0% 的有效主网格输出属于内容正确但边界破损；消融显示主因是同族对称分隔符冲突而非嵌套，提示提示只能部分缓解，且该现象可泛化到 Python 三引号 docstring，而 JSON 作为非对称分隔符对照保持稳健。对解析器敏感的 LLM 评测与 Agent/代码抽取管线有实际参考价值，但属特定格式边界问题，影响面有限。
72
正文
Computer Science > Software Engineering
arXiv:2609.06993 (cs)
[Submitted on 7 Sep 2026 (v1), last revised 11 Sep 2026 (this version, v2)]
Title:LatentMD: Benchmarking Markdown Boundary Failures in LLM-Generated Text
Authors:Sungjune Lee, Myungjoo Kang
View a PDF of the paper titled LatentMD: Benchmarking Markdown Boundary Failures in LLM-Generated Text, by Sungjune Lee and 1 other authors
View PDF
HTML (experimental)
Abstract:Large language models (LLMs) increasingly generate Markdown that is consumed by renderers, agents, code extractors, and structured downstream pipelines. Yet existing evaluations often conflate content quality with format adherence, leaving Markdown boundary failures under-measured. We introduce LatentMD, a benchmark and evaluation protocol for diagnosing CommonMark-level fence-boundary failures in LLM-generated Markdown. LatentMD separates content correctness from boundary correctness, enabling detection of outputs that are content-correct but boundary-broken. The benchmark contains 4,179 prompts and a CLI for scoring arbitrary model outputs. Across 9 LLMs and roughly 37,600 generations, we find that Markdown boundary failures are widespread: 38.0% of valid main-grid outputs are content-correct but boundary-broken, with substantial boundary breakage under unspecified prompts and in a small human-authored validation set. Ablations show that failures are driven primarily by same-family symmetric-delimiter collisions rather than nesting alone, are only partially mitigated by prompt hints, and generalize to Python triple-quote docstrings while JSON remains robust as an asymmetric-delimiter control. LatentMD provides a reproducible diagnostic target for parser-sensitive LLM evaluation.
Comments:
Subjects:
Software Engineering (cs.SE); Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as:
arXiv:2609.06993 [cs.SE]
(or
arXiv:2609.06993v2 [cs.SE] for this version)
https://doi.org/10.48550/arXiv.2609.06993
Focus to learn more
arXiv-issued DOI via DataCite
Submission history
From: Sungjune Lee [view email]
[v1]
Mon, 7 Sep 2026 03:35:26 UTC (109 KB)
[v2]
Fri, 11 Sep 2026 15:23:03 UTC (122 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled LatentMD: Benchmarking Markdown Boundary Failures in LLM-Generated Text, by Sungjune Lee and 1 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.SE
new
|
recent
| 2026-09
Change to browse by:
cs
cs.CL
cs.LG
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)