Where Synthetic Respondents Break Down

Before cataloguing failures, it helps to fix what "working" means, because studies that appear to disagree are often measuring different things.

A silicon sample can be judged on at least three criteria, and they are genuinely independent.

Direction. Does it get the sign right? If human respondents prefer A to B, does the synthetic sample?

Distribution. Does it reproduce the spread of human responses, including disagreement and minority positions, or only the central tendency?

Magnitude. Does it get the size right? A willingness to pay of 12 dollars against a true 40 dollars has the right direction and a useless number.

A method can pass one and fail the others, and the published record contains exactly that pattern. This is why "do silicon samples work?" has no single answer, and why the most common reporting error is demonstrating one criterion and implying all three.

The most consistent failure is that synthetic samples are too agreeable with themselves.

Real survey data is messy. Ask a hundred people to rate a product and you get a spread: enthusiasts, detractors, people who misread the scale, people with idiosyncratic reasons. That dispersion is not noise to be cleaned away. It is the signal in questions about segmentation, polarisation, and risk.

A silicon sample tends to compress it. Asked to answer as a described person, the model outputs a high-probability response for that description, and repeated draws cluster tightly around it. Minority positions thin out or vanish, because they were low-probability continuations.

The consequence is specific and dangerous. Means can look approximately right while variances are badly understated, which makes findings look more statistically decisive than the underlying reality. A study reporting a clear preference may be reporting the model's confidence about a stereotype rather than a real consensus.

Anything about disagreement, niche appeal, or the shape of a distribution is the wrong use of this instrument.

There is a subtler problem underneath the variance issue, and it explains why more prompting rarely fixes things.

When you condition a model on "a 34-year-old teacher in Ohio", you are not instantiating that person. You are asking for text that would plausibly be written about or by such a person. The model draws on how that demographic is portrayed in its corpus, which includes journalism, fiction, marketing copy, and online commentary, alongside anything those people actually wrote.

So the output is closer to a cultural composite than a sample from the population. Where portrayal and reality diverge, the synthetic respondent follows the portrayal.

This predicts the observed pattern well. Fidelity is best where a group's self-expression is abundant online and worst where a group is mostly written about. It also explains why adding demographic detail can sharpen a stereotype rather than a person.

The practical test: ask whether your target population writes a lot online in the register your question probes. If not, be sceptical regardless of how coherent the answers read.

The failure that should most worry anyone using this for a decision is not imprecision. It is being confidently backwards.

Recall from the previous lesson that Brand, Israeli, and Ngwe found willingness-to-pay estimates sometimes comparable to human studies, often inaccurate, and in some cases wrong-signed. Wrong-signed means the synthetic data indicated buyers valued a feature that real buyers discount, or the reverse.

This matters more than an equivalent error in a human study, for a reason worth internalising. Human survey error tends to be noisy: respondents are inattentive, samples are unrepresentative, and the errors partly cancel. Model error is systematic. If the corpus encodes a mistaken association between a demographic and a preference, every synthetic respondent reproduces it, and adding rows makes the wrong answer more precise rather than less wrong.

So the usual defence against noise, collect more data, actively hurts here. It converts a wrong estimate into a wrong estimate with a tight confidence interval, which is exactly the artefact most likely to survive review and reach a decision.

The two error sources behave in opposite ways. Sampling noise shrinks as you collect more observations, which is the entire logic of larger samples. Systematic corpus bias does not move at all, so beyond a small number of draws you are buying precision around a fixed offset.

Every survey has coverage problems. The difference here is that you cannot inspect or correct them by the usual means.

In conventional research the population is defined, the sampling frame is documented, non-response is measured, and results are weighted to known population margins. When a group is underrepresented, you can see it in the data and adjust.

A silicon sample has no sampling frame at all. The effective population is whoever produced the training text, filtered through alignment training that shapes what the model will say. Neither is documented in a form you can audit, and both change between model versions.

Two consequences follow. Groups that write less online, or in languages and registers less represented in the corpus, are approximated worse, and you cannot quantify how much worse without human data, which is the thing you were trying to avoid collecting. And alignment training pushes toward inoffensive, moderate responses, which is a systematic distortion precisely on sensitive topics where accurate measurement matters most.

Weighting cannot fix this, because there is nothing to weight toward.

A one-sided account would be its own distortion, and there are uses with real support.

Pretesting instruments. Running a draft questionnaire through a model surfaces ambiguous wording, broken skip logic, and questions that cannot be answered as asked. Failure here is informative and success costs nothing, because no substantive claim depends on the answers.

Augmenting rather than replacing. Marketing research has explored calibrating synthetic responses against a smaller human sample, and using models to reduce the volume of human data needed for conjoint estimation. This keeps a human anchor and uses the model to extend it, which is a fundamentally different claim from replacement.

Hypothesis generation. Producing candidate segments or objections to investigate is a legitimate use, provided the investigation actually happens.

Qualitative-style exploration. Surfacing the range of arguments people make about a topic is closer to what the model genuinely encodes than any point estimate.

The unifying rule: silicon samples are defensible when they are an input to human research, and indefensible when they are a substitute for it.

Mapping tasks against the failure modes gives a usable decision aid.

Two gotchas that recur in practice. First, plausibility is not validity: synthetic responses read well because fluency is what the model optimises, and reviewers routinely mistake that for realism. Second, a validation in one domain does not transfer; fidelity for political attitudes tells you nothing about fidelity for snack preference.

All of this assumes a single fixed setup. The next lesson removes that assumption, and the picture gets substantially worse.