大模型
huggingface.co ·
用 100 步 GRPO 微调 350M 模型以提升结构化输出能力
Hugging Face 博客介绍了一种公开、低成本的微调方法，使用 TRL 库对 LFM2.5-350M 模型进行 GRPO 微调，在 IFStruct 基准上从 22.6% 提升至 29.7%，仅需约 500 个样本和 100 步训练，可在免费 Colab 或 Kaggle GPU 上运行。文章提供了完整代码和评估流程，展示了小模型通过轻量微调可接近更大模型的性能。
AI推荐理由 Hugging Face 官方博客，提供可复现的微调配方，包含具体步骤、基准分数提升（22.6% 到 29.7%）和开源资源，信息密度高，时效性好。但方法并非全新，且未提供实际应用效果，兴趣度中等。
78
正文
返回文章列表
在100步GRPO中微调350M模型以获得更好的结构化输出
发布时间
2026年9月3日
在GitHub上更新
点赞
80
+74
Leonie Monigatti
iamleonie
关注
LiquidAI
ben burtenshaw
burtenshaw
关注
Sergio Paniego
sergiopaniego
关注
本指南是一个完全公开、成本低廉的配方，用于让小型模型在结构化输出合规性方面显著提升。我们使用TRL库，通过组相对策略优化（GRPO）对LFM2.5-350M进行微调，并在IFStruct基准上对其进行评估。整个运行大约需要500个样本和100个训练步骤，小到可以在免费版Colab或Kaggle GPU上运行，并且可在GitHub上获取。结果表明，即使是轻量级的微调过程也能将IFStruct基准上的性能从22.6%提升到29.7%。
结构化输出是LLM最常见的现实世界任务之一，但大多数基准将其纳入更广泛的推理或提取分数中，而不是单独衡量。模型是否能够可靠地返回符合请求格式和形状的有效、可解析输出——即模式合规性——通常决定了它是否能够接入下游系统。
请注意，此处描述的训练流程并非用于训练IFStruct博客中描述的RL模型的流程。本笔记本的目的不是重现IFStruct基准分数，而是展示针对特定任务的小型模型微调如何提升性能并匹配更大模型的性能。
先决条件
本指南分为两部分，在不同的环境中运行：
微调在GPU上运行。随附的笔记本适用于免费版Colab或Kaggle GPU。
评估可以在MacBook上本地运行（此处为配备Apple M5 Max和36 GB统一内存的MacBook Pro），通过llama.cpp实现，它暴露了一个与OpenAI兼容的服务器，IFStruct评估器可以与之通信。
我们将需要uv用于Python工具，以及llama.cpp用于服务。按照Liquid AI llama.cpp部署文档，使用Homebrew安装llama.cpp，并验证llama-server可用：
brew install llama.cpp
llama-server --version
在LFM2.5-350M（基础模型）上的IFStruct评估
在开始之前，让我们在IFStruct基准上评估LFM2.5-350M，看看我们能否复现报告的21.1%的分数。
IFStruct是一个用于测试LLM输出有效性和模式遵循度的基准。该基准在Liquid4All/ifstruct中开源，公共基准数据集可在Hugging Face上的LiquidAI/ifstruct-v1.0获取。
git clone https://github.com/Liquid4All/ifstruct.git
为了进行评估比较，我们在MacBook上使用llama.cpp本地提供模型服务。我们将使用BF16 GGUF（LiquidAI/LFM2.5-350M-GGUF）。
然后我们使用以下命令启动基础模型服务器：
llama-server \
-hf LiquidAI/LFM2.5-350M-GGUF:BF16 \
-c 32768 \
-np 4 \
-ngl 99 \
--alias LiquidAI/LFM2.5-350M \
--host 127.0.0.1 \
--port 8080
--alias：IFStruct发送到OpenAI兼容端点的模型名称
-ngl 99：要求llama.cpp在可用时将所有层卸载到GPU
-np 4：并行处理四个请求
-c 32768：提示上下文的大小
一旦服务器运行，我们可以使用2000个样本运行完整基准：
uv run ifstruct-eval \
--model LiquidAI/LFM2.5-350M \
--base-url http://localhost:8080/v1 \
--api-key dummy \
--dataset data/test.jsonl \
--results-file results/lfm2.5-350m-llamacpp-base.json \
--n-threads 4 \
--max-tokens 2048 \
-v
============================================================
模型：LiquidAI/LFM2.5-350M
============================================================
总体：452/2000通过（22.6%）
平均延迟：1453毫秒
按格式：
JSON：180/1000通过（18.0%）
YAML：272/1000通过（27.2%）
按顶层结构：
包装键 288/1011通过（28.5%）
裸列表 164/989通过（16.6%）
按实体类型：
test__camera_review 6/83通过（7.2%）
test__clinical_trial 20/104通过（19.2%）
test__conference_schedule 7/87通过（8.0%）
test__escaping__bug_report_batch 24/89通过（27.0%）
test__escaping__config_snippet_audit 15/85通过（17.6%）
test__escaping__customer_email_thread 5/73通过（6.8%）
test__escaping__dialogue_sample 14/95通过（14.7%）
test__escaping__interview_transcript_segment 21/80通过（26.2%）
test__escaping__log_parser_examples 21/72通过（29.2%）
test__escaping__pr_discussion 22/87通过（25.3%）
test__escaping__repro_steps_batch 16/73通过（21.9%）
test__escaping__screenplay_scene 16/92通过（17.4%）
test__escaping__short_story_chapter 15/84通过（17.9%）
test__escaping__support_ticket_batch 27/73通过（37.0%）
test__escaping__terminal_session_notes 20/70通过（28.6%）
test__event_ticket_booking 49/107通过（45.8%）
test__gpu_review 6/94通过（6.4%）
test__invoice 28/86通过（32.6%）
test__job_posting 25/85通过（29.4%）
test__real_estate_listing 31/82通过（37.8%）
test__recipe 3/70通过（4.3%）
test__rental_car_booking 27/79通过（34.2%）
test__scientific_experiment 13/69通过（18.8%）
test__travel_itinerary 21/81通过（25.9%）
常见错误：
7228次缺少必填字段
738次项目数量错误
540次类型不匹配
317次未闭合的代码块
190次多余字段'notes'
181次多余字段'path'
175次多余字段'constraints'
170次多余字段'type'
170次缺少代码块
100次期望裸列表，但得到包装键
IFStruct发布博客报告LFM2.5-350M的分数为21.1%。我们本地的llama.cpp/BF16设置测得22.6%，接近IFStruct博客中报告的21.1%。我们使用此本地结果作为相同服务栈比较的基线。
使用TRL对结构化输出进行GRPO微调
完整、可运行的流程位于随附的笔记本中。本节仅涵盖相关部分。
训练数
据
我们使用nvidia/Nemotron-RL-instruction_following-structured_outputs，它将每个提示与目标JSON Schema和期望的字段数量配对。我们使用大约500个样本进行训练。
由于Nemotron数据分布与IFStruct评估不同，我们增强提示以缩小两者之间的两个差距：
40%的提示附加“在围栏代码块内返回输出”的指令，以便模型学习遵循格式指令，而不是总是发出原始JSON。
另外不重叠的20%被转换为顶层数组任务（模式被包装在数组中，并带有必需的项目数量），这训练了裸列表输出和项目数量合规性。
模型和LoRA
我们加载LiquidAI/LFM2.5-350M并附加LoRA适配器。由于LFM2.5使用混合注意力/卷积架构，我们针对LFM特定的模块名称：
lora_config = LoraConfig(
r=16,
lora_alpha=32,
bias="none",
task_type="CAUSAL_LM",
target_modules=[
"q_proj", "k_proj", "v_proj", "out_proj", "in_proj",
"w1", "w2", "w3",
],
)
这训练了约600万个参数，约占模型的1.66%。
奖励函数
然后我们定义三个奖励函数，每个都在[0, 1]范围内，对每个完成的结构提取是否正确进行评分：
json_format_reward：输出是否可解析，并且格式是否符合要求？完全符合要求的形式（围栏代码块与原始）得满分（1.0），格式错误但可解析得0.2，不可解析得0.0。
field_count_reward：对象是否具有期望的顶层字段数量？完全匹配得1.0，分数随缺失量线性衰减。
schema_validation_reward：输出是否根据行的JSON Schema进行验证？它计算每个约束违规，并根据必填键覆盖率给予部分分数。
我们将这三个奖励组合为加权和，权重为reward_weights=[1.0, 0.5, 2.0]。
训练
我们训练100步，每组提示生成8个样本，适用于免费版16 GB GPU：
from trl import GRPOConfig
training_args = GRPOConfig(
output_dir="./outputs/lfm25-350m-nemotron-schema-grpo",
learning_rate=5e-5,
max_steps=100,
warmup_steps=10,
num_generations=8,
per_device_train_batch_size=4,
gradient_accumulation_steps=8,
steps_per_generation=2,
max_completion_length=1024,
mask_truncated_completions=False,
temperature=1.1,
beta=0.01,
reward_weights=[1.0, 0.5, 2.0],
logging_steps=1,
save_steps=100,
)
正如您在笔记本中看到的，在运行过程中，所有三个奖励组件都在攀升，来自参考模型的KL在预热后从零开始上升，截断完成的比例保持在接近零。
合并并保存模型
最后，我们将LoRA适配器合并回基础权重，并将其保存为单个自包含检查点，准备转换为GGUF以进行服务：
MERGED_DIR = f"{training_args.output_dir}-merged"
merged_model = trainer.model.merge_and_unload()
merged_model.save_pretrained(MERGED_DIR)
tokenizer.save_pretrained(MERGED_DIR)
在GRPO调优的LFM2.5-350M上的IFStruct评估
GRPO微调后，我们重新运行IFStruct评估。为此，我们需要将合并的模型检查点转换为BF16 GGUF。转换器脚本随llama.cpp源代码一起提供，因此我们克隆一次仓库并安装转换器的gguf包。
git clone --depth 1 https://github.com/ggml-org/llama.cpp
pip install ./llama.cpp/gguf-py
mkdir -p models
python llama.cpp/convert_hf_to_gguf.py \
PATH_TO_YOUR_MERGED_MODEL \
--outfile ./models/lfm25-350m-grpo-bf16.gguf \
--outtype bf16
然后我们使用以下命令提供合并模型的服务：
llama-server \
-m ./models/lfm25-350m-grpo-bf16.gguf \
--alias lfm25-350m-grpo-structured-output \
-c 32768 \
-np 4 \
-ngl 99 \
--host 127.0.0.1 \
--port 8081
然后，我们将使用微调后的模型再次运行完整的IFStruct评估：
uv run ifstruct-eval \
--model lfm25-350m-grpo-structured-output \
--base-url http://localhost:8081/v1 \
--api-key dummy \
--dataset data/test.jsonl \
--results-file results/lfm25-350m-grpo.json \
--n-threads 4 \
--max-tokens 2048 \
-v
============================================================
模型：lfm25-350m-grpo-structured-output
============================================================
总体：594/2000通过（29.7%）
平均延迟：1518毫秒
按格式：
JSON：319/1000通过（31.9%）
YAML：275/1000通过（27.5%）
按顶层结构：
包装键 300/1011通过（29.7%）
裸列表 294/989通过（29.7%）
按实体类型：
test__camera_review 5/83通过（6.0%）
test__clinical_trial 31/104通过（29.8%）
test__conference_schedule 11/87通过（12.6%）
test__escaping__bug_report_batch 32/89通过（36.0%）
test__escaping__config_snippet_audit 24/85通过（28.2%）
test__escaping__customer_email_thread 9/73通过（12.3%）
test__escaping__dialogue_sample 17/95通过（17.9%）
test__escaping__interview_transcript_segment 13/80通过（16.2%）
test__escaping__log_parser_examples 33/72通过（45.8%）
test__escaping__pr_discussion 26/87通过（29.9%）
test__escaping__repro_steps_batch 23/73通过（31.5%）
test__escaping__screenplay_scene 34/92通过（37.0%）
test__escaping__short_story_chapter 24/84通过（28.6%）
test__escaping__support_ticket_batch 36/73通过（49.3%）
test__escaping__terminal_session_notes 23/70通过（32.9%）
test__event_ticket_booking 62/107通过（57.9%）
test__gpu_review 7/94通过（7.4%）
test__invoice 36/86通过（41.9%）
test__job_posting 33/85通过（38.8%）
test__real_estate_listing 32/82通过（39.0%）
test__recipe 7/70通过（10.0%）
test__rental_car_booking 37/79通过（46.8%）
test__scientific_experiment 14/69通过（20.3%）
test__travel_itinerary 25/81通过（30.9%）
常见错误：
7331次缺少必填字段
890次项目数量错误
555次类型不匹配
102次期望裸列表，但得到包装
键
62次多余字段'metadata.tone'
55次6大于最大值5
49次多余字段'speaker_labels'
47次多余字段'tone'
44次'cups'不在允许值['mg', 'g', 'kg', 'oz', 'lb', 'ml', 'l', 'cl', 'dl'中
44次多余字段'notes'
在相同的服务栈上比较两次运行：
IFStruct组
基础
GRPO调优
Δ
总体
22.6%
29.7%
+7.1
JSON
18.0%
31.9%
+13.9
YAML
27.2%
27.5%
+0.3
包装键
28.5%
29.7%
+1.2
裸列表
16.6%
29.7%
+13.1
收益恰好落在训练目标上：JSON通过率上升了近14个百分点（18.0% → 31.9%），而YAML基本保持不变。虽然这仍低于Qwen3.5-2B的33.15%分数，但它表明即使是轻量级的任务特定微调也能使小型模型接近更大的模型。
结论
一次简短的GRPO运行，使用约500个样本和100步，可以将一个350M参数的小模型在IFStruct上的分数从22.6%提升到29.7%。要点是，一个廉价、任务特定的奖励信号可以使小型模型在格式上显著更可靠，缩小与数倍于其大小的模型之间的差距。
要复现或扩展这项工作，请参阅原始IFStruct v1.0博客文章、Liquid4All/ifstruct基准仓库和LiquidAI/ifstruct-v1.0数据集。
本文提到的模型 2
本文提到的数据集 2
来自我们博客的更多文章
nlpguidecommunity
使用Sentence Transformers训练和微调多向量嵌入模型
120
2026年8月26日
nlpguidecommunity
使用Sentence Transformers的多向量（后期交互）嵌入模型
106
2026年8月18日
社区
helpmehere
3天前
这是我们安全扫描的测试评论
查看翻译
回复
helpmehere
3天前
无描述。
查看翻译
回复
helpmehere
3天前
无描述。
查看翻译
回复
x0mhb
2天前
回复
x0mhb
2天前
点击我
查看翻译
回复
编辑预览
通过拖拽文本输入、粘贴或点击此处上传图像、音频和视频。
点击或粘贴此处上传图像
评论
· 注册或登录以评论
点赞
80
+68
本文提到的模型 2
本文提到的数据集 2
查看原文（中文正文为 AI 翻译）
Back to Articles
Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps
Published
September 3, 2026
Update on GitHub
Upvote
80
+74
Leonie Monigatti
iamleonie
Follow
LiquidAI
ben burtenshaw
burtenshaw
Follow
Sergio Paniego
sergiopaniego
Follow
This guide is a fully public, inexpensive recipe for making a small model substantially better at structured-output compliance. We fine-tune LFM2.5-350M with Group Relative Policy Optimization (GRPO) using the TRL library and evaluate it on the IFStruct benchmark. The full run takes around 500 samples and 100 training steps, small enough for a free-tier Colab or Kaggle GPU, and is available on GitHub. The results show that even a light fine-tuning procedure improves performance from 22.6% to 29.7% on the IFStruct benchmark.
Structured output is one of the most common real-world tasks for LLMs, yet most benchmarks fold it into broader reasoning or extraction scores rather than measuring it on its own. Whether a model reliably returns valid, parseable output in the requested format and shape — schema compliance — is often what decides whether it can be wired into a downstream system at all.
Note that the training pipeline described here is not the one used to train the RL model described in the IFStruct blog. This notebook doesn't aim to recreate the IFStruct benchmark score, but to show how task-specific fine-tuning of smaller models can improve performance and match that of far larger models.
Prerequisites
This guide has two halves that run in different places:
Fine-tuning runs on a GPU. The accompanying notebook is sized for a free-tier Colab or Kaggle GPU.
Evaluation can run locally on a MacBook (here, a MacBook Pro with an Apple M5 Max and 36 GB of unified memory) through llama.cpp, which exposes an OpenAI-compatible server that the IFStruct evaluator talks to.
We will need uv for the Python tooling and llama.cpp for serving. Following the Liquid AI llama.cpp deployment docs, install llama.cpp with Homebrew and verify that llama-server is available:
brew install llama.cpp
llama-server --version
IFStruct Evaluation on LFM2.5-350M (Base model)
Before we begin, let's evaluate LFM2.5-350M on the IFStruct benchmark and see whether we can reproduce the reported score of 21.1%.
IFStruct is a benchmark for testing the validity of LLM outputs and schema adherence. The benchmark is open-source in Liquid4All/ifstruct, with the public benchmark dataset available on Hugging Face at LiquidAI/ifstruct-v1.0.
git clone https://github.com/Liquid4All/ifstruct.git
For the eval comparison, we serve the model locally on the MacBook with llama.cpp. We will use the BF16 GGUF (LiquidAI/LFM2.5-350M-GGUF).
Then we start the base-model server with the following command:
llama-server \
-hf LiquidAI/LFM2.5-350M-GGUF:BF16 \
-c 32768 \
-np 4 \
-ngl 99 \
--alias LiquidAI/LFM2.5-350M \
--host 127.0.0.1 \
--port 8080
--alias: model name IFStruct sends to the OpenAI-compatible endpoint
-ngl 99: asks llama.cpp to offload all layers to the GPU when available
-np 4: serves four requests in parallel
-c 32768: size of the prompt context
Once the server is running, we can run the full benchmark with 2000 samples:
uv run ifstruct-eval \
--model LiquidAI/LFM2.5-350M \
--base-url http://localhost:8080/v1 \
--api-key dummy \
--dataset data/test.jsonl \
--results-file results/lfm2.5-350m-llamacpp-base.json \
--n-threads 4 \
--max-tokens 2048 \
-v
============================================================
Model: LiquidAI/LFM2.5-350M
============================================================
Overall: 452/2000 passed (22.6%)
Average latency: 1453ms
By format:
JSON: 180/1000 passed (18.0%)
YAML: 272/1000 passed (27.2%)
By top-level structure:
Wrapper key 288/1011 passed (28.5%)
Bare list 164/989 passed (16.6%)
By entity type:
test__camera_review 6/83 passed (7.2%)
test__clinical_trial 20/104 passed (19.2%)
test__conference_schedule 7/87 passed (8.0%)
test__escaping__bug_report_batch 24/89 passed (27.0%)
test__escaping__config_snippet_audit 15/85 passed (17.6%)
test__escaping__customer_email_thread 5/73 passed (6.8%)
test__escaping__dialogue_sample 14/95 passed (14.7%)
test__escaping__interview_transcript_segment 21/80 passed (26.2%)
test__escaping__log_parser_examples 21/72 passed (29.2%)
test__escaping__pr_discussion 22/87 passed (25.3%)
test__escaping__repro_steps_batch 16/73 passed (21.9%)
test__escaping__screenplay_scene 16/92 passed (17.4%)
test__escaping__short_story_chapter 15/84 passed (17.9%)
test__escaping__support_ticket_batch 27/73 passed (37.0%)
test__escaping__terminal_session_notes 20/70 passed (28.6%)
test__event_ticket_booking 49/107 passed (45.8%)
test__gpu_review 6/94 passed (6.4%)
test__invoice 28/86 passed (32.6%)
test__job_posting 25/85 passed (29.4%)
test__real_estate_listing 31/82 passed (37.8%)
test__recipe 3/70 passed (4.3%)
test__rental_car_booking 27/79 passed (34.2%)
test__scientific_experiment 13/69 passed (18.8%)
test__travel_itinerary 21/81 passed (25.9%)
Common errors:
7228x required field missing
738x wrong item count
540x type mismatch
317x Unclosed code block
190x extraneous field 'notes'
181x extraneous field 'path'
175x extraneous field 'constraints'
170x extraneous field 'type'
170x missing code block
100x expected bare list, got wrapper
The IFStruct release blog reports 21.1% for LFM2.5-350M. Our local llama.cpp/BF16 setup measures 22.6%, close to the 21.1% reported in the IFStruct blog. We use this local result as the baseline for the same serving stack comparison.
GRPO Fine-tuning with TRL on Structured Outputs
The full, runnable pipeline lives in the accompanying notebook. We will cover only the relevant pieces in this section.
Training data
We use nvidia/Nemotron-RL-instruction_following-structured_outputs, which pairs each prompt with a target JSON Schema and an expected field count. We use about 500 samples for training.
Because the Nemotron data distribution differs from the IFStruct evaluation, we augment the prompts to close two gaps between them:
40% get a "return the output inside a fenced code block" instruction appended, so the model learns to follow the format instruction rather than always emitting raw JSON.
A disjoint 20% are converted into top-level-array tasks (the schema is wrapped in an array with a required item count), which trains bare-list output and item-count compliance.
Model and LoRA
We load LiquidAI/LFM2.5-350M and attach a LoRA adapter. Because LFM2.5 uses a hybrid attention/convolution architecture, we target the LFM-specific module names:
lora_config = LoraConfig(
r=16,
lora_alpha=32,
bias="none",
task_type="CAUSAL_LM",
target_modules=[
"q_proj", "k_proj", "v_proj", "out_proj", "in_proj",
"w1", "w2", "w3",
],
)
This trains ~6M parameters, about 1.66% of the model.
Reward functions
Then we define three reward functions, each on a [0, 1] scale, which score every completion on whether the extracted structure is correct:
json_format_reward: Is the output parseable, and in the requested form? Full credit (1.0) for the requested form (fenced vs. raw), 0.2 for the wrong-but-parseable form, 0.0 for unparseable output.
field_count_reward: Does the object have the expected number of top-level fields? An exact match earns 1.0, and the score decays linearly with the miss.
schema_validation_reward: Does the output validate against the row's JSON Schema? It counts every constraint violation and gates partial credit on required-key coverage.
We combine the three as a weighted sum with reward_weights=[1.0, 0.5, 2.0].
Training
We train for 100 steps with 8 generations per prompt group, sized for a free-tier 16 GB GPU:
from trl import GRPOConfig
training_args = GRPOConfig(
output_dir="./outputs/lfm25-350m-nemotron-schema-grpo",
learning_rate=5e-5,
max_steps=100,
warmup_steps=10,
num_generations=8,
per_device_train_batch_size=4,
gradient_accumulation_steps=8,
steps_per_generation=2,
max_completion_length=1024,
mask_truncated_completions=False,
temperature=1.1,
beta=0.01,
reward_weights=[1.0, 0.5, 2.0],
logging_steps=1,
save_steps=100,
)
As you can see in the notebook, over the run, all three reward components climb, the KL from the reference model lifts off zero after warmup, and the truncated-completion fraction stays near zero.
Merging and saving the model
Finally, we merge the LoRA adapter back into the base weights and save it as a single self-contained checkpoint, ready to convert to GGUF for serving:
MERGED_DIR = f"{training_args.output_dir}-merged"
merged_model = trainer.model.merge_and_unload()
merged_model.save_pretrained(MERGED_DIR)
tokenizer.save_pretrained(MERGED_DIR)
IFStruct Evaluation on GRPO Tuned LFM2.5-350M
After GRPO fine-tuning, we rerun the IFStruct evaluation. For this, we need to convert the merged model checkpoint into a BF16 GGUF. The converter script ships with the llama.cpp source, so we clone the repo once and install the converter's gguf package.
git clone --depth 1 https://github.com/ggml-org/llama.cpp
pip install ./llama.cpp/gguf-py
mkdir -p models
python llama.cpp/convert_hf_to_gguf.py \
PATH_TO_YOUR_MERGED_MODEL \
--outfile ./models/lfm25-350m-grpo-bf16.gguf \
--outtype bf16
Then we serve the merged model with the following command:
llama-server \
-m ./models/lfm25-350m-grpo-bf16.gguf \
--alias lfm25-350m-grpo-structured-output \
-c 32768 \
-np 4 \
-ngl 99 \
--host 127.0.0.1 \
--port 8081
Then, we will run the full IFStruct evaluation again with the fine-tuned model:
uv run ifstruct-eval \
--model lfm25-350m-grpo-structured-output \
--base-url http://localhost:8081/v1 \
--api-key dummy \
--dataset data/test.jsonl \
--results-file results/lfm25-350m-grpo.json \
--n-threads 4 \
--max-tokens 2048 \
-v
============================================================
Model: lfm25-350m-grpo-structured-output
============================================================
Overall: 594/2000 passed (29.7%)
Average latency: 1518ms
By format:
JSON: 319/1000 passed (31.9%)
YAML: 275/1000 passed (27.5%)
By top-level structure:
Wrapper key 300/1011 passed (29.7%)
Bare list 294/989 passed (29.7%)
By entity type:
test__camera_review 5/83 passed (6.0%)
test__clinical_trial 31/104 passed (29.8%)
test__conference_schedule 11/87 passed (12.6%)
test__escaping__bug_report_batch 32/89 passed (36.0%)
test__escaping__config_snippet_audit 24/85 passed (28.2%)
test__escaping__customer_email_thread 9/73 passed (12.3%)
test__escaping__dialogue_sample 17/95 passed (17.9%)
test__escaping__interview_transcript_segment 13/80 passed (16.2%)
test__escaping__log_parser_examples 33/72 passed (45.8%)
test__escaping__pr_discussion 26/87 passed (29.9%)
test__escaping__repro_steps_batch 23/73 passed (31.5%)
test__escaping__screenplay_scene 34/92 passed (37.0%)
test__escaping__short_story_chapter 24/84 passed (28.6%)
test__escaping__support_ticket_batch 36/73 passed (49.3%)
test__escaping__terminal_session_notes 23/70 passed (32.9%)
test__event_ticket_booking 62/107 passed (57.9%)
test__gpu_review 7/94 passed (7.4%)
test__invoice 36/86 passed (41.9%)
test__job_posting 33/85 passed (38.8%)
test__real_estate_listing 32/82 passed (39.0%)
test__recipe 7/70 passed (10.0%)
test__rental_car_booking 37/79 passed (46.8%)
test__scientific_experiment 14/69 passed (20.3%)
test__travel_itinerary 25/81 passed (30.9%)
Common errors:
7331x required field missing
890x wrong item count
555x type mismatch
102x expected bare list, got wrapper
62x extraneous field 'metadata.tone'
55x 6 is greater than maximum 5
49x extraneous field 'speaker_labels'
47x extraneous field 'tone'
44x 'cups' not in allowed values ['mg', 'g', 'kg', 'oz', 'lb', 'ml', 'l', 'cl', 'dl'
44x extraneous field 'notes'
Comparing the two runs on the identical serving stack:
IFStruct group
base
GRPO-tuned
Δ
Overall
22.6%
29.7%
+7.1
JSON
18.0%
31.9%
+13.9
YAML
27.2%
27.5%
+0.3
Wrapper key
28.5%
29.7%
+1.2
Bare list
16.6%
29.7%
+13.1
The gains land exactly where the training aimed: the JSON pass rate rises by nearly 14 points (18.0% → 31.9%), while YAML stays mostly the same. While this is still below the Qwen3.5-2B score of 33.15%, it shows that even light task-specific fine-tuning can bring a small model close to a larger one.
Conclusion
A short GRPO run with about 500 samples and 100 steps can lift a small 350M parameter model from 22.6% to 29.7% on IFStruct. The takeaway is that a cheap, task-specific reward signal can make a small model substantially more reliable about form, closing much of the gap to models several times its size.
To reproduce or extend this work, see the original IFStruct v1.0 blog post, the Liquid4All/ifstruct benchmark repo, and the LiquidAI/ifstruct-v1.0 dataset.
Models mentioned in this article 2
Datasets mentioned in this article 2
More Articles from our Blog
nlpguidecommunity
Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers
120
August 26, 2026
nlpguidecommunity
Multi-Vector (Late Interaction) Embedding Models with Sentence Transformers
106
August 18, 2026
Community
helpmehere
3 days ago
This is a test comment from our security scan
See translation
Reply
helpmehere
3 days ago
No description provided.
See translation
Reply
helpmehere
3 days ago
No description provided.
See translation
Reply
x0mhb
2 days ago
Reply
x0mhb
2 days ago
clickme
See translation
Reply
EditPreview
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Comment
· Sign up or log in to comment
Upvote
80
+68
Models mentioned in this article 2
Datasets mentioned in this article 2