The Impostor is Among Us: Can Large Language Models Capture the Complexity of Human Personas?

Citation

Lazik, C., Katins, C., Kauter, C., Jakob, J., Jay, C., Grunske, L., & Kosch, T. (2025). The Impostor is Among Us: Can Large Language Models Capture the Complexity of Human Personas? In Mensch und Computer 2025 (MuC ‘25). ACM.

Summary

Study comparing human-crafted personas (from 10 HCI experts) with AI-generated personas (GPT-4o) through a survey of 54 participants. Examines whether users can distinguish between the two and what features drive that distinction.

Key Findings

Users can distinguish AI from human personas (p = .003 for human recognition, p = .002 for AI recognition), contradicting some prior work suggesting indistinguishability.

AI personas scored higher on:

• Informativeness (p < .001)
• Consistency (p < .001)
• Clarity (p < .001)
• Positivity (p < .001)
• Stereotypicality (p = .03)

No significant difference on:

• Believability
• Relatability
• Likability

Qualitative signals of AI-generated content:

• “Robotic” writing style, unusual vocabulary choices
• Overly positive depictions (no flaws)
• Hobbies that suspiciously align with occupation
• Lack of technology-critical attitudes
• Narrow demographic representation

Demographics of generated vs. human-crafted:

• AI age range: 27-50 (mean 35.9)
• Human age range: 10-72 (mean 51.4)
• AI occupations: all white-collar tech/business
• AI gender: 5M/5F/0NB; Human: 7M/2F/1NB

Methodology Notes

• Human personas created by HCI experts familiar with concept but not routine persona designers
• AI personas generated via zero-shot prompting using structured approach from prior work
• Used Persona Perception Scale constructs from Salminen et al.
• Participants evaluated 20 randomized personas (10 each type)

Extracted Content

→ 05-atom—polish-authenticity-paradox → 05-atom—flaws-as-authenticity-signals → 05-atom—occupation-hobby-alignment-tells → 05-atom—llm-stereotype-defaults → 07-atom—distinguishing-ai-from-human-output → 07-molecule—authenticity-vs-quality-in-ai-output → 01-molecule—ai-persona-generation-risks