Synthetic Sampling Report
Your Polls On ChatGPT
The challenges of “synthetic sampling” for public opinion polling.
By G. Elliott Morris; Verasight data team
Verasight data is trusted by the world's leading researchers
Insights from This Report
Topline Accuracy
We found substantial variation across LLM models, with an average absolute error rate that ranged from 4 percentage points to 23 percentage points.
Crosstab Errors
The average absolute difference between the population proportion and LLM proportion for these demographic subgroups is 8 percentage points.
Out of Sample Questions
When we move beyond the information in the LLM’s training data, we start to see even larger differences between LLM and actual respondent percentages.
Recommended Citation
Morris, Elliott G. 2025. “Your Polls on ChatGPT.” Verasight White Paper Series. https://www.verasight.io/
Download the Report Now
Serious questions deserve serious data. That’s why the world’s top researchers trust Verasight.
