大模型
arxiv.org ·
MiST：面向网络安全的中期训练大语言模型
论文提出 MiST（Mid-trained Security Transformer），包含 8B 和 32B 两个模型，在通用预训练与网络安全训练之间引入 mid-training 阶段。方法不采用大规模原始领域文本的持续预训练，而是构建经专家审核的紧凑种子语料，并转化为高质量领域合成训练数据。相比对应 Qwen 基线，8B 和 32B 模型的网络安全平均准确率分别提升 13.1 和 8.6 个绝对百分点，相对提升 27.0% 和 15.8%；消融显示增益主要来自 mid-training 与监督微调阶段的合成数据生成流程，且 MiST 可为下游任务微调和强化学习提供更强的初始化。
AI推荐理由 arXiv 预印本，来源可信、摘要信息密度高，给出了具体模型规模（8B/32B）、基线（Qwen）和量化提升（8B +13.1 绝对百分点/+27.0% 相对，32B +8.6/+15.8%），并说明增益来自 mid-training 与 SFT 阶段的合成数据流程，还指出可为下游微调和 RL 提供更好初始化。局限在于仅为论文自述结果，未提供代码或第三方复现，benchmark 细节与合成数据质量未在摘要中展开。
78
正文
Computer Science > Cryptography and Security
arXiv:2609.18496 (cs)
[Submitted on 16 Sep 2026]
Title:MiST: Mid-Training LLMs for Cybersecurity
Authors:Oded Ovadia, Elad Ben Zaken, Elad Guttman, Orly Moreno Kadosh
View a PDF of the paper titled MiST: Mid-Training LLMs for Cybersecurity, by Oded Ovadia and 3 other authors
View PDF
HTML (experimental)
Abstract:Cybersecurity combines high-stakes analysis with complex technical language, making it an impactful and challenging domain for LLMs. We present MiST (Mid-trained Security Transformer), a suite of 8B and 32B models that achieve strong performance on public cybersecurity benchmarks. We use mid-training as an intermediate adaptation stage between general pre-training and cybersecurity training. Rather than performing continual pre-training over large volumes of raw domain text, we curate a compact, expert-vetted seed corpus, and transform it into high-quality domain-specific synthetic training data. The final MiST checkpoints improve mean cybersecurity accuracy by +13.1 and +8.6 absolute percentage points over the corresponding Qwen baselines for 8B and 32B, respectively, corresponding to relative gains of +27.0% and +15.8%. Ablation results further show that these cybersecurity gains arise in the mid-training and supervised fine-tuning stages through a combination of the synthetic data generation flows. Furthermore, we show that MiST provides a stronger initialization for downstream task-specific fine-tuning adaptation and reinforcement learning.
Subjects:
Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
Cite as:
arXiv:2609.18496 [cs.CR]
(or
arXiv:2609.18496v1 [cs.CR] for this version)
https://doi.org/10.48550/arXiv.2609.18496
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Oded Ovadia [view email]
[v1]
Wed, 16 Sep 2026 11:33:12 UTC (629 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled MiST: Mid-Training LLMs for Cybersecurity, by Oded Ovadia and 3 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.CR
new
|
recent
| 2026-09
Change to browse by:
cs
cs.AI
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)