When Generating Data Beats Collecting It

Start with the number everyone starts with. Suppose you need 50,000 instruction-response pairs.

Human annotation at two dollars per example, which is modest for anything requiring expertise, costs 100,000 dollars and takes months of coordination.

Generating the same volume at 800 output tokens each, at an assumed three dollars per million output tokens, costs about 120 dollars and runs overnight. That is a ratio of roughly 830 to one.

The ratio is real and it is the least interesting thing about synthetic data, because cost was rarely the binding constraint. Teams that could not afford annotation usually also could not afford to train.

What changes decisions is that generation reaches places collection cannot. Data for a product that has no users yet. Coverage of failure modes that occur once in ten thousand real interactions. Examples in a format no one has ever written down.

The honest framing is that synthetic data is a coverage tool that happens to be cheap, and treating it as a cheap substitute for real data is where the failures start.

Generation is worth reaching for when collection fails, and it fails in four distinguishable ways. Naming which one you have determines what technique applies.

The data does not exist. A new product, a new interface, a new task. There is no corpus because nobody has done this yet.

The data exists and cannot be used. Medical records, financial transactions, anything under a licence that forbids training. The information is there and legally out of reach.

The data exists and is unlabelled. Millions of support tickets, no categories. Here the raw material is real and only the annotation is missing, which is the cheapest case and the one most likely to succeed.

The data exists and is unbalanced. Ninety-nine percent routine, one percent the cases you actually need to handle well. Collection gives you more of what you already have.

The fourth is where generation shines and the first is where it disappoints, because a task nobody has performed is also a task your generator has never seen.

The constraint that governs everything else is easiest to state as an accounting identity.

A generator's outputs cannot contain more information about the world than the generator holds. Sampling a model a million times produces a great deal of text and no new facts. If the teacher does not know your API's error codes, no volume of generated examples will teach a student about them.

This sounds obvious and is routinely violated in practice. Teams generate training data for a domain the generator has never encountered, fine-tune on it, and are surprised when the student learns to produce confident, well-formatted, wrong answers. The student learned exactly what was in the data.

What generation genuinely does is redistribute and reformat capability that already exists. It can move knowledge from a large model into a small one. It can convert latent ability into a consistent output format. It can amplify a rare pattern until a model trained on it responds reliably.

All three are valuable. None of them create knowledge, and a pipeline designed as though they do produces fluent fabrication at scale.

There is one genuine exception to the accounting identity above, and it is the most important idea in this path.

When you can check an answer independently of the model that produced it, generation stops being redistribution and starts being search.

Consider a model that solves a class of maths problems correctly 35 percent of the time. Sample it once per problem and you get correct answers on 35 percent of prompts. Sample it eight times and keep only the verified-correct attempts, and 96.8 percent of prompts yield at least one correct solution. Sixteen samples takes that to 99.9 percent.

Train on the verified survivors and the student learns from a distribution far stronger than the teacher's own sampling behaviour. The information was not in any single generation; it came from the checker.

This is why verifiable domains behave differently. Code with tests, mathematics with checkable answers, formal proofs, SQL you can execute, structured extraction you can validate against the source.

Where a verifier exists, generate many and keep few. Where none exists, you are redistributing what the teacher already knew, and the ceiling is the teacher.

Synthetic data pipelines come in four shapes, and they differ in where the information comes from, which determines what each can achieve.

Distillation takes a stronger teacher model and trains a weaker student on its outputs. Information flows downhill. The student can approach the teacher and cannot exceed it.

Self-improvement uses one model to generate its own training data. With no filter this is circular and is the configuration that collapses. With a filter it is the next shape.

Verified generation adds a checker between generation and training. The checker is an external information source, so the student can exceed the generator's single-sample reliability. This is the only shape with a ceiling above its generator.

Programmatic generation builds examples from templates and rules, with ground truth known by construction because you generated the answer before the question. Perfect labels, limited realism.

Most production pipelines combine them: programmatic scaffolding for structure, a strong teacher for language, a verifier where one is available.

Distillation is the most common shape and carries a constraint that is contractual rather than technical.

Most commercial model providers' terms prohibit using their outputs to train competing models. The wording varies and the enforcement posture varies, but the clause is standard enough that assuming it is present is the safe default.

The practical consequences are worth separating. Generating data to fine-tune a model for your own internal use is treated differently from generating data to train a model you will sell. Open-weight models with permissive licences remove the question, which is one reason they are heavily used as teachers even when a stronger closed model is available.

And provenance survives. A dataset generated from a given model is often identifiable from stylistic signatures, and published models have been challenged on exactly this basis.

The engineering habit that follows is to record, for every example, which model produced it, under which terms, and on what date. That record costs nothing at generation time and is impossible to reconstruct afterwards, which is the same argument the distributed training path made about run configuration.

A subtlety that decides whether a distillation project succeeds: the student does not learn the teacher's knowledge. It learns the teacher's outputs.

Those are different in a way that shows up under pressure. The teacher answers correctly because it has broad capability it can bring to bear. The student trained on ten thousand of those answers learns the shape of correct answers on the covered distribution.

On inputs resembling the training data, the two are indistinguishable. Off that distribution, the teacher degrades gracefully and the student degrades into confident imitation, producing responses with the right structure and tone and the wrong content.

This is why coverage of the synthetic set matters more than its size, and it is the subject of the next lesson. Ten thousand examples spanning the input space beat a hundred thousand clustered in one region, because the second set teaches the shape of one region very well and says nothing about the rest.

It is also why evaluating a distilled student on data drawn from the same generation process is nearly meaningless. That evaluation measures how well the student learned the covered region, which is the one thing you can already assume.

Synthetic data is frequently proposed as a privacy solution: generate a synthetic version of sensitive records and the privacy problem disappears. The claim needs qualification.

When a generator is trained on sensitive data, its outputs are a function of that data. Generative models can memorise and reproduce training examples, particularly rare and distinctive ones, which are exactly the records that identify individuals. A synthetic dataset produced this way is not automatically anonymous; it is derived data whose disclosure risk has to be argued rather than assumed.

The stronger position is a formal guarantee. Training the generator under differential privacy bounds how much any single record can influence its outputs, and that bound is inherited by anything generated from it. Utility falls, sometimes sharply, and the guarantee is precise rather than rhetorical.

The weaker but common position is a generator that never saw the sensitive data at all, prompted with schemas and descriptions rather than records. The output has no derivation from real individuals and, correspondingly, no realism from them either.

Under European data protection law, whether output counts as personal data turns on identifiability, so the assessment is a legal one and not settled by the word synthetic.

Matching the failure mode to the pipeline shape resolves most of the design question before any code is written.

The last row is the one worth defending. When no generator understands your domain, synthetic data produces a confident, well-formatted model that is wrong in ways your evaluation will not catch, because the evaluation came from the same misinformed process.

The third row is the most underused. When real inputs exist and only annotation is missing, you keep the true input distribution and use the model only for what it is genuinely good at.

Before generating anything, three questions determine whether the project can work, and answering them takes an hour.

Where is the information coming from? Name the source. A teacher model, a verifier, a template with known ground truth, or real inputs with model labels. If the honest answer is that the information comes from the model you are about to train, the pipeline is circular and the next lessons explain what happens.

What is the ceiling? Distillation ceilings at the teacher. Verified generation ceilings above the generator. Programmatic generation ceilings at the fidelity of your templates. If the ceiling is below your requirement, no volume of data reaches it.

How will you know it worked? The answer must involve real data the generation process never touched. This is the discipline the whole path depends on, because a synthetic evaluation of a synthetically trained model measures self-consistency rather than capability.

Those three answers determine the technique. The next lesson covers generating with coverage, the one after covers filtering, and the last covers what happens when the loop closes on itself.