随模型增长的数据集：用于研究与 CI 的确定性合成对象

Illustration for building a synthetic dataset for training AI models. Computer Vision Dataset Data Science Data Visualization Announcements A Dataset That Grows With Your Model: Deterministic Synthetic Objects for Research and CI Same seed, same pixels, with nothing stored. Exact labels for four tasks, and difficulty you dial up when the detector stops failing. イルカ Borovec 9 min read · 1 hour ago -- Listen Share Every project starts with the same annoying question. Which dataset do we use for this? The first answer is always “something simple”. It is never simple. You sign up somewhere. You click through a web UI that was built for clicking, not scripting. You hit export, wait, download a zip, and get a folder layout that is almost, but not quite, what your loader wants. Then the Wi-Fi drops, or the mirror moved, or the archive is 900 MB when you wanted 20 images. Or it is 20 images when you wanted 2,000. Then the format is wrong, so you write a converter, and the converter has a bug, and now you are debugging the fixture instead of the model. And then the question changes. New task, new label type, a harder case you want to probe. So you go find another dataset and run the whole loop again. None of that is the work. It is the toll you pay before the work. I got tired of paying it, so I wrote a generator that makes the dataset a function of a seed instead of a file: from synth_datasets import generate_dataset counts = generate_dataset(tmp_path, num_images=8, fmt="coco", seed=0, img_size=64) # {'train': 5, 'val': 2, 'test': 1} - 21 ms on one CPU core No network. No storage unless you want it. No PyTorch needed just to write files. Any size you ask for. And the labels are exact, because they come from the geometry that drew the pixels, not from anyone’s mouse. Stream it into a DataLoader and nothing touches disk. Write it out once and the files on your laptop are byte-for-byte the same as the ones on the CI runner. This post is about what you can build on that. It goes from a fixture that takes 21 milliseconds to a held-out mAP gate that gets harder as your model gets better. Scope. Everything here is measured on vision-synth from main, ahead of the 0.15.0.rc release, CPU only, on 25 September 2026. The generator imports as synth_datasets and needs only NumPy and Pillow. The fused augmentations import as fused_transforms and pull in PyTorch through the [torch] extra. Timings are single-core, on my machine, and each one says what it counts. Press enter or click to view image in full size Illustration for real samples in the package. What comes out of the generator Flat-coloured objects on a canvas, drawn with Pillow. Every label comes from the geometry that drew the object, not from an annotation pass afterward. There are 49 object templates in four families: Primitives: 4 shapes (square, rectangle, triangle, circle). Boxes, masks and oriented boxes, no keypoints. Symbols: 7 shapes with concave outlines like arrow, anchor and house. 5 to 6 keypoints each. Letters: 26 glyphs built skeleton-first, so every keypoint sits inside the stroke. 3 to 10 keypoints each. Animals: 12 traced silhouettes with named landmarks. 11 to 16 keypoints each. You get four tasks from the same scene and the same seed: detection, segmentation, oriented boxes, and keypoints. Two writers cover COCO (the Roboflow flavor) and YOLO (the Ultralytics flavor, with its data.yaml). There is also a SyntheticIterableDataset that streams samples straight into training. The class vocabulary scales too. class_mode gives you 3 colour classes, 49 shape classes, or 147 colour-times-shape classes from the same generator. The category ids in each written file narrow down to what that run actually used, so a fixture is always self-consistent. Determinism is the thing everything else rests on. One integer seed fixes every pixel and every label. The package pins its own generated bytes to a checked-in baseline, so a change to the renderer trips a test before it ever changes a fixture downstream. The iterable dataset takes rank, world_size and epoch and folds them into the seed. Eight workers get eight disjoint shards, and no scene shows up twice in an epoch. A fixture you cannot reproduce is a fixture you cannot bisect. Two axes of difficulty Difficulty moves along two axes, and you pick where you sit on each. The first axis is what the model has to learn: the object family. Primitives are convex and nearly symmetric, so they are easy. Symbols add concavity and a real orientation. Letters add thin strokes, near-duplicate classes, and symmetry broken on purpose for nine glyphs, so an angle head cannot cheat. Animals add irregular silhouettes and the densest keypoints. Turn class_mode up and the vocabulary grows on top of all that. The second axis is what the model has to ignore: everything around the object. Backgrounds go from plain grey to gradients, noise, procedural textures, or random crops of your own images. Degradations bake into the render: noise, blur, JPEG, contrast, colour cast, vignette, quantisation. Distractor clutter and occluders are two more integers on the config, and a covered keypoint drops from visible to hidden the way a real annotation would. from synth_datasets import JPEG, GaussianBlur, Task, TextureBackground, generate_dataset generate_dataset( out, num_images=200, fmt="yolo", seed=0, img_size=320, task=Task.OBB, background=TextureBackground(), degrade=(GaussianBlur(0.5), JPEG(75)), distractors=3, occluders=1, ) # {'train': 140, 'val': 40, 'test': 20} - 1.9 s on one CPU core Each setting is a knob, not a preset. A preset is just someone’s opinion about what “hard” means. The honest version of that opinion is a measurement, so the docs rank the knobs by how much held-out mAP each one costs on a fixed gate, and leave the pick to you. Get イルカ Borovec’s stories in your inbox Join Medium for free to get updates from this writer. Subscribe Subscribe Remember me for faster sign in The dataset does not get harder on its own. It gets exactly as hard as you tell it to, and it remembers. Press enter or click to view image in full size Illustration for raising difficulty. Five ways you can benefit from it A test fixture that costs nothing The eight-image call up top is a whole dataset setup for a test suite. It runs in conftest.py, on every platform your suite runs on, with nothing to fetch. RF-DETR's tests use it since #1435. The only wrinkle was a format detail: the writer emits dense category IDs, and those fixtures wanted gaps, so it took a six-line helper. If your repo has a data/folder with an archive in it, or a script that pulls eight images off a mirror, this replaces both. And it still works the day the mirror goes down. A DDP smoke test that does not need a cluster Sharding is baked into the seed, not bolted on around it. So you can build the dataset as worker 3 of 8 on a single machine, and check it yields exactly what worker 3 of 8 should, at any epoch, with no torchrun: from synth_datasets import SyntheticIterableDataset shard = SyntheticIterableDataset( num_images=64, seed=0, img_size=128, rank=3, world_size=8, epoch=2 ) This is the one piece that needs pip install vision-synth[torch], since it is a real PyTorch IterableDataset. The contract is one line: no persistent_workers, so the epoch counter advances. That is your whole distributed data path, testable on a laptop. A converter and metric test bed One seed, one scene, four label types, two formats. If your COCO-to-YOLO conversion disagrees with itself, or your OBB loss, or your keypoint visibility handling, or your mAP code, the data is not the suspect. It is the same geometry written four ways. The letters are the sharp tool here. Symmetry is broken on purpose, so an angle regression that folds orientation gets caught in the metric instead of sliding by. An oracle for augmentation This is the one thing real images cannot give you at any price. Apply a geometric augmentation to a generated scene, and you can compute the label it should produce. So you can test an augmentation library against ground truth, not against another implementation. A small harness pushes the image, the mask, the polygon, the keypoints, the boxes, and the oriented boxes through one fixed transform, then compares each label to the polygon pushed through the same matrix. Four families, six warps, 25 scenes each, about two thousand objects. It runs in under ten seconds on a CPU. The identity warp is the self-test, and it reads 1.000 IoU and 0.000 pixels everywhere. It earned its place on the first run. In most augmentation code, boxes and points follow slightly different pixel conventions: boxes measured from pixel edges, points from pixel centers. On a photo, you would never notice. On a generated square you notice immediately, because you know where the label is supposed to land. The harness caught a one-pixel mismatch between the two conventions on a plain horizontal flip, before release. On a real dataset, that would just look like slightly noisy labels, forever. It also put numbers on some folklore. The oriented-box path is exact to 0.000 pixels under rotation and scale. An axis-aligned box transformed by its corners scores IoU 0.48 to 0.53 against the true tight box at 37 degrees, so a detector trained that way sees boxes that are about half background. Letters and symbols keep every keypoint on the ink through every warp, 5,141 checks between them. A regression gate that grows with the model lucid-YOLO uses the generator for a held-out gate. An n-scale detector trains on 1,800 generated scenes and evaluates on 200 held-out ones, 320 pixels, six epochs, mosaic on. The gate pins three numbers: NMS mAP50–95 at 0.871, end-to-end mAP50–95 at 0.857, NMS mAP50 at 0.986. Three runs on one machine reproduced them to four decimals. Overfit gates on a fixed 100-scene slice pin detection recall at IoU 0.5 to 0.997, rotated mAP50 to 0.965 for OBB, mask IoU to 0.865 for segmentation, and OKS-AP to 0.575 for keypoints. Those catch coordinate-frame slips and starved heads without spending a COCO epoch. A gate should stay useful after the model clears it. This is where the two axes pay off. When the detector stops failing on plain primitives, take one step along either axis, a harder family or a busier scene, and the gate moves with it. The seed stays put, so the run you compare against is still the run you ran last month. Press enter or click to view image in full size Illustration for addoption patterns. Adopting it, in the order that pays off first Start by replacing whatever produces your test dataset with one generate_dataset call and a pinned seed. Delete the download step and the folder it filled. If you train distributed, add the one-machine shard test next: build rank r of n, and assert the samples are disjoint across ranks and stable across restarts at the same epoch. Then generate the same seed in both formats and all four tasks, and point your converters and metrics at it before you point them at anything real. If you maintain or vendor an augmentation stack, run the parity harness against it, starting with a horizontal flip. And once the smoke tests are green, pin a held-out gate on plain primitives, then take one step along either difficulty axis per model milestone, keeping the seed. A test dataset should be a function of a seed, not a file someone uploaded once. What it is not The scenes are geometry with color. Converging on them proves your pipeline, labels, losses, and metrics are wired up right. It does not rank backbones or augmentation policies for real photos. A knob that costs 0.05 mAP here might cost nothing on a camera feed, or everything. The difficulty ranking is a regression instrument, not a benchmark, and the docs say so. The animal landmarks are traced onto silhouettes, not built skeleton-first like the letters. So a landmark on the tip of a thin limb can sit within a pixel of the outline at small sizes. I drew each template 120 times at random sizes and rotations and checked every landmark against the rendered pixels. All 7 symbols and all 26 letters kept every landmark on the ink, and so did all 12 animals. Any near-miss is shape geometry, not a broken label. The generator ships in vision-synth. The harness, the raw results, and the difficulty ranking live in its docs. So here is my question: if your augmentation stack uses one convention for boxes and another for points, does it agree with itself when you flip?