Why synthetic participants and digital twins can't replace real people
By Hey Juno · Published · Updated
Synthetic participants and digital twins cannot currently replace real people when a decision depends on the answer. A language model works from what has already been written, so in the studies below it is weakest on new questions and unfamiliar topics, which is where customer research is most needed. Averages can look right while the differences that drive decisions are wrong, and nothing in the output tells you which is which.
This is the argued case. For a balanced overview of what synthetic respondents are and how they are used, read real participants versus synthetic respondents. This guide covers synthetic respondents, AI personas, silicon samples and digital twins alike, and says where each is weakest. Juno, the AI researcher from Hey Juno, interviews real people for this reason.
They lean on what is already written
A model produces what is plausible given the text it was trained on. That can be a fair stand-in when a question has been asked many times, and a poor one when it has not.
Verasight, a company that runs human survey panels, tested this in 2025. Its second synthetic sampling report (Morris, Leff and Enns) built a persona for each of 1,500 real panel respondents. Results were reasonably close on a familiar, polarised question, Trump approval, with error of around 4 points in the original study. Immigration attitudes were still off by 11.3 points with the most detailed setup. The authors say models "badly predicted" less polarised and novel questions.
New products and new situations are novel by definition. There is little written record for a model to draw on.
The averages look right; the differences do not
Bisbee and colleagues (2024, Political Analysis) asked ChatGPT to answer as personas drawn from real American National Election Study respondents. Average scores were close to the real averages. But the synthetic data varied less than real data, and regression results often led to different conclusions from the real data. Small changes to the prompt moved results, and the same prompt gave different results three months later. The authors name this as a replication problem.
Pew Research Center reached a similar view in 2026 (Chapekis and colleagues). Pew notes that aggregate estimates from models are generally more accurate, but puts this down to the averaging effect of the model. So it paired each real panellist with an AI counterpart, to see where synthetic answers fail to reflect real opinion.
Decisions rarely turn on the average. They turn on which segment differs, which relationship holds, and what drives what. Those are the parts that come out wrong.
They flatten people
Wang, Morgenstern and Dickerson (accepted at Nature Machine Intelligence; arXiv 2024, revised 2025) ran human studies with 3,200 participants across 16 identities and four models. They argue that models are likely to misportray demographic groups and flatten the variety within them.
A 2026 conference abstract from Bocconi University (Peng and colleagues, presented at the European Population Conference) found similar compression when five models answered factorial survey questions in China and the United States. Model variance was well below human variance. The authors warn that uncalibrated models may overstate consensus and understate contested trade-offs. It is an early abstract, so treat it as supporting evidence only.
What gets squeezed out is minority and unusual views. Those often show where a finding stops being true.
Even a digital twin does not predict the person
A digital twin is built from a real person's own earlier answers, so it should be the strongest case. A 2025 mega-study led by Olivier Toubia and 23 co-authors tested it (arXiv 2509.19088, "Digital Twins as Funhouse Mirrors: Five Key Distortions"). It ran 19 pre-registered studies comparing twins with the people they were built from, across 164 outcomes. Each twin was built from that person's answers to more than 500 questions.
The average correlation between twin and human responses was 0.197, roughly r = 0.20. The authors found that detailed personal information improved correlation but had only a modest effect on individual-level accuracy. Twins tended to be more accurate for people with higher education and income, and for people with moderate political views. In this study, a twin was a weak predictor of the one person it was built from, and a worse one for some groups than others.
The errors are patterns, not noise
The same paper names five distortions: insufficient individuation, stereotyping, representation bias, ideological bias and hyper-rationality. Twins' answers were less spread out than humans' in 154 of 164 outcomes, and full-persona twins answered more like demographics-only twins than like the real person. These are patterns, not random noise, so generating more answers is unlikely to remove them. A thousand synthetic respondents can carry the same lean as ten. Verasight's authors make the same point: errors on broader questions are model error, not sampling error.
Verasight's October 2025 coffee report (Morris and Leff) shows the size of the problem in a market research setting. Compared with a survey of 1,519 US adults, errors averaged 19.8 percentage points. In the real data, 56 per cent of adults drank coffee daily; 91 per cent of the model's answers did. None of the model's answers came from the 17 per cent who drink none. The authors sell human panel research, so read their conclusion with that in mind. The pattern, though, matches the academic work above.
A handful of real people beats them
Ivetta, Moradbakhti and Calvo (2025, preprint, "Synthetic social data: trials and tribulations") compared six models with World Values Survey data from four countries. They asked how often the average of a random human sample was closer to the true population value than the synthetic average. One real respondent was closer in 47.5 per cent of cases, two in 60.8 per cent, sixteen in over 86 per cent and 128 in 97.6 per cent.
A small sample of real people is not a good sample. On this measure, in these data, it still beat the model's average once there were a few respondents, and it is real.
The validation trap
You can only check a synthetic answer against a real one. If you already have the real data, the synthetic version adds little. If you do not, you cannot easily tell when the synthetic answer is wrong, because wrong answers are as fluent as right ones. The method asks you to trust it in the cases where you cannot test it.
They cannot surprise you
Qualitative research earns its keep when someone says what you did not expect. A model returns what is plausible, and fluency is not evidence. A persona also has no real last time something happened, so questions about specific events get invented answers. Good interviewing follows the specific event, the workaround or the hesitation. See how Juno interviews for how that works with real people.
No one stands behind the quote
A synthetic finding cannot be traced to a real person. Nobody can check it against what they said. Putting invented words into the mouths of real groups, such as "older rural customers said", is also an ethical problem. With real participants, every finding can be traced to a verbatim quote and a transcript, as set out in how to check AI research findings.
They are frozen in time
A model knows the world as of its training data, unless it is given fresh information. Prices, products, competitors and opinions keep moving. A real participant can tell you about this month.
The edge test
This section is reasoning, not evidence, and it makes no claim about what any firm does or does not use.
Take a simple argument. A model trained on public text knows only what was written down. If a signal is in public text, many others can read it too, so it gives no particular edge. An edge comes from something others lack.
Customer research works the same way. Its value is in what your customers have not written anywhere yet. Asking a model to stand in for them is like making a decision from yesterday's newspaper.
Where synthetic data does help
Used openly, it has honest uses: rehearsing an interview guide, checking questions for confusion or leading wording, and generating hypotheses to test with real people. Each is about the study, not about the market.
Juno uses simulated interviews to rehearse a study before launch. They are never research data or findings.
Limits of this argument
Much of this evidence is recent, and some of it is preprint, working paper or conference abstract. Models improve, and a well-validated tool used within a narrow, known range may do better than the general results here. Several tests also ask models to reproduce answers people already gave. The claim is limited: on current evidence, synthetic data cannot substitute for real people when decisions depend on it. If that changes, it will show up as validated results on new questions, not on questions already answered.
Frequently asked questions
Are digital twins better than synthetic personas?
Somewhat, but not enough to change the conclusion. In the 2025 mega-study, twins built from more than 500 answers per person were only modestly more accurate than a generic model, and the average correlation with the real person was about 0.20. See real participants versus synthetic respondents for the difference between the two.
Can synthetic data replace a pilot?
No, but it can prepare one. A simulated interview helps you fix unclear questions before you meet anyone. A pilot with real people tells you whether the questions work with the people you are asking, and whether you are asking the right ones.
Will better models fix this?
They may help in places. Verasight's second report found that a newer model and added detail did not necessarily improve accuracy, and for Trump approval the most detailed setup was the least accurate. The distortions in the mega-study are patterns in how models answer, so scale alone may not remove them. The test is evidence on new questions that were not in anything the model saw.
Does Juno use synthetic participants for findings?
No. Findings come from real participants and trace back to their words. Simulated interviews only rehearse a study before launch.