Skip to content
Synthetic Users

Why synthetic users?

Read this before you decide whether Synthetic Users fits your problem.

What it’s good for

  • The first few rounds of feedback you’d otherwise get from user interviews on a staging URL, before you pay for a real research study.
  • The dogfooding pass that keeps getting scheduled and never happens.
  • The “is this actually clear to someone who isn’t me?” check at 11pm the night before you ship.

What it doesn’t replace

  • Real user research on a launched product. Synthetic personas have a shape; real users have history, habits, and stakes.
  • Quantitative usability metrics. Time-on-task and success rates across a large sample need a real test pool.
  • Buyer interviews. Purchasing decisions live in a context the persona doesn’t have — budget cycles, org politics, procurement.

What makes the output trustworthy

Three things keep the report grounded in what actually happened, not in what the model guessed. They’re described in detail in persona voice:

  1. The react() tool records how the persona feels, in its own words, as first-class data — separate from page snapshots.
  2. The report is written from those reactions first, and the transcript second.
  3. A deterministic check rejects any quote that doesn’t trace back to a recorded reaction, so the quotes section can’t be invented.

We test this against apps with a hand-written list of known UX problems and measure how many the personas surface. Results so far are directional rather than a published benchmark: the ground-truth lists are still being calibrated against real observed runs, so we don’t quote a headline accuracy number yet. What we can say is that persona voice and the reactions-first report make the difference between a report that reads like a test log and one that reads like research.

When it’s wrong

Known failure modes:

  • Page-string regurgitation — the model quotes on-screen copy as if a user said it. Most likely when tool calls keep failing; the session stops rather than degrade.
  • Persona over-reach — a non-technical persona objecting to terminology its real-world counterpart would actually understand. Cross-check with a second persona.
  • Synthesis dropout on very long sessions. The medium duration budget caps a session at roughly 20 minutes to keep this rare.

If a report doesn’t feel right, the transcript is the source of truth — everything in the report is reproducible from it.

What we’re explicitly not building yet

Some of these land in V1.5 or later:

  • Conversational replay against a finished transcript (V1.5).
  • A persistent synthetic identity that carries across runs (V1.5).
  • Logging your personas into your product through your own OAuth (V1.5).
  • Multi-agent coordination (V2+).

What to pay attention to in a report

  • The aggregate report on a multi-persona run is where patterns show up — the same confusion point, in three different voices.
  • The quotes section is the most reusable artifact; it drops straight into slides and tickets.
  • Every persona deciding it wouldn’t sign up is the loudest signal you can get: the value proposition isn’t landing clearly enough to move anyone.