Synthetic respondents vs surveys in brand research

Published tests find that synthetic respondents come close to the average answer of a population but miss much of the spread around it. In brand measurement the spread between groups, markets and competitors is usually the result, so we use synthetic data to generate hypotheses and never to report a KPI.

The debate about synthetic data in market research is often framed as a question of replacement, whether generated answers will make surveys unnecessary. A more useful question is which decisions each kind of data can support, and what has to be checked before it is trusted.

What synthetic respondents are

A synthetic respondent is an answer generated by a language model that has been asked to respond as a person with given characteristics, for example a 35-year-old woman in Manchester who buys skincare online.

Some services add real survey data to the prompt or train the model further on it. Either way, the answer is a prediction of what such a person might say, built from the text the model was trained on and the data it is given.

What published research on synthetic respondents shows

The research is young, and the main academic studies agree on where synthetic samples do well and where they fall short:

Relationships between variables can be reproduced. Argyle and colleagues (2023) gave GPT-3 the background characteristics of real respondents in the American National Election Studies. The generated answers reproduced many of the relationships between demographics and political attitudes in the real data.
Averages come close, but the spread is smaller. Bisbee and colleagues (2024) asked ChatGPT to answer as different personas and compared the results with the same survey. Average scores corresponded closely, but the synthetic answers varied less than the real ones, and regression estimates often differed significantly from those in the real data.
Answers move with the prompt and over time. In the same study, small changes in prompt wording changed the distribution of answers, and the same prompt gave significantly different results three months apart.
Groups are flattened. Wang, Morgenstern and Dickerson (2025) argue that the way current models are trained leads them to misportray demographic groups and to flatten the differences within them.

In political science these are problems of statistical inference. Brand measurement is more exposed, because they affect exactly the properties a tracker exists to capture.

Why the spread matters in brand tracking

Most results in brand tracking are differences:

between a brand and its competitors
between one market and another
between segments
between this quarter and the last

A brand manager rarely acts on the average for a category. Decisions follow the gap to a named competitor, or the market where consideration does not turn into purchase.

Those are the properties the published tests find hardest to reproduce. A method whose documented weakness is a compressed spread is therefore weakest where brand decisions are made.

There is a timing problem as well. A language model answers from text written before its training cut-off, so a change in how a market sees a brand after that point can only reach it through what is put into the prompt. A tracker exists to record such changes.

Where synthetic data helps

Synthetic data is fast and cheap, and some tasks only need a plausible answer:

Generating hypotheses to test, such as which needs a new segment might have or which objections a concept might meet.
Testing a questionnaire before fieldwork, to see whether a question can be misread, whether the answer options cover the likely answers and whether the routing works.
Exploring scenarios internally, as a structured way to think through how different groups might react.

Our own rule follows from that. Synthetic audiences are never a substitute for representative respondent data or for KPI reporting.

We use them for hypothesis generation, scenario exploration and internal analysis support, kept clearly separate from measured results.

Synthetic data, surveys and AI visibility: what each source answers

The same discipline applies to every source that sits next to a survey, including social conversation and the answers AI assistants give about a category. Each one observes something different, so choosing a source starts from the question at hand and from what the answer should be checked against.

Before using synthetic respondents

Decide what the output may be used for before it is generated. Hypotheses and question wording are within scope. Levels, segment sizes and trends are not.
Test it against data you already have. Generate answers for a population you have measured, and compare the spread and the differences between subgroups as well as the average.
Fix the prompt and the model version, then rerun. If the result moves with the wording or over time, it is not a measurement.
Label synthetic output wherever it is used, and keep it out of KPI dashboards. The ICC/ESOMAR International Code, revised in 2025, was updated with AI and synthetic data in mind and puts weight on transparency and human oversight.
Take the hypotheses that hold up into fieldwork, and let measured data decide.

As more of the data in circulation is generated by models, answers recorded from real people become the scarcer input and the reference that every other source has to be checked against.

That is a reason to keep measured data at the centre of brand tracking, and to use synthetic data for the work it does well.

Synthetic respondents can generate hypotheses but cannot measure a brand