Technical White Paper
Methodology & Honest Limits — for teams deciding whether to trust a simulated market test.
Every product launch, price change, and market-entry decision is a bet placed under uncertainty. The traditional way to reduce that uncertainty — focus groups, survey panels, consulting studies — costs $50K–$200K and takes weeks to months. Most teams skip it and launch on instinct. The expensive failures that follow (Quibi, Google Stadia, Amazon's Fire Phone) weren't unknowable; the demand-side warning signs were visible, but nobody could afford to look in time.
EchoTest is a simulation layer for that decision. Before you spend real money, you run your product, price, or campaign against a population of synthetic consumers and read how the market is likely to react — in minutes, not weeks.
This paper exists because a fair question deserves a real answer: is this a genuine simulation, or an AI telling plausible stories? We'll show you the actual machinery, and we'll be equally clear about what it can't do. The short version:
Market research answers the question “what do people think of this?” That's useful, but it's a snapshot — a single moment, a fixed sample, no sense of what happens next. The decisions that actually keep founders and operators awake are dynamic:
These aren't “what do people think” questions. They're “if I actually run this plan, what happens?” questions. Answering them requires a model of behavior over time, not a transcript of opinions. That's what EchoTest is built to be.
The phrase “2.5 million personas” invites a fair skepticism: are these distinct models, or just 2.5 million AI prompts? Here is the honest, precise answer. Each persona is a structured profile, not a chatbot character. It exists in three places at once:
Personas are generated deterministically through four layers. No language model is involved in generating a persona — it's a reproducible, auditable construction.
| Layer | What it adds | Real-world source |
|---|---|---|
| 1. Statistical foundation | Demographic distributions per country (age, gender, income, education, sector, urban/rural) | UN population data, World Bank indicators, national census |
| 2. Cultural DNA | How a person from this culture tends to think, trust, and decide | Hofstede 6-dimension model + World Values Survey (Wave 7) |
| 3. Emotional & religious calibration | Expression style, sensitivities, communication norms | Cultural and religious behavioral research |
| 4. Profile assembly | A coherent, natural-language description of the individual | Template-based composition of layers 1–3 |
Beyond demographics, every persona carries a quantified behavioral profile:
That's 35+ behavioral fields, plus a Big Five profile, on top of the demographic and cultural base.
We did not survey 2.5 million individuals — that isn't feasible, and any vendor claiming otherwise should be pressed on it. Instead, each behavioral field is estimated from established correlations in the psychometric and consumer-behavior literature (for example, price sensitivity rises as income falls; luxury orientation rises with income; agreeableness trends higher in collectivist cultures). These estimates are seeded deterministically, so the same persona always resolves to the same profile — results are reproducible, not random. What we validate is the direction and strength of these relationships, not per-person ground truth. A persona is a statistically faithful archetype of a population slice, not a real human and not a claim to be one.
The ~2.5 million figure comes from scoping personas not just by country but by sub-country zone: roughly 50 priority countries broken into hundreds of local zones. This is what lets the simulation distinguish a luxury buyer in Palm Jumeirah from a value-driven shopper in Deira — two profiles a “UAE average” would blur together. Critically, each persona carries a population weight: how many real people it represents. This is what allows a panel of a few hundred simulated agents to extrapolate honestly to a market of millions.
This is the heart of the matter, and the clearest answer to “is it real or is it theater.”
Principle: the math decides what happens; the language model only explains why. A simulated customer's decision to buy, stay, or cancel is the output of an explicit formula. The AI is then asked to write a believable, culturally-grounded reason for the decision the math already made — and even if the AI's narrative leans the other way, the numbers govern the outcome.
We never ask the model, “does this person cancel?” We calculate a cancellation risk from concrete factors:
These combine into a churn probability, bounded to a sane range, and the outcome is then drawn stochastically — so the simulation produces a distribution of behavior across the panel, not a single deterministic verdict. The AI writes the human reason (“the price hike finally pushed me to compare alternatives”) after the math has spoken.
The customer lifecycle — never heard of you → aware → signed up → paying → churned — is governed by flow-conservation equations. Inflow at each stage must equal outflow from the previous one; customers can't appear or vanish without being accounted for. This is the same accounting discipline a real funnel obeys, and none of it touches a language model.
EchoTest runs two synchronized layers:
The scaling factor is applied to flows (new sign-ups, new churns this quarter), not to stocks (the existing base) — which preserves baseline customer counts instead of naively multiplying everything. This is why a small panel can project to a large market without the numbers becoming nonsense.
Real markets aren't sums of independent opinions; people hear each other out and reconsider. EchoTest models this with a multi-round debate:
That silent-majority veto is genuine emergent filtering: a persuasive minority can move the market, but only if the market actually finds the argument persuasive.
| Decided by math | Voiced by the language model |
|---|---|
| Who churns, who buys, who converts | The human-readable reason each persona gives |
| Funnel flows and conservation | The wording of objections and endorsements |
| Population scaling | The narrative summary and example customer journeys |
| Which arguments cascade vs. get vetoed | The phrasing of those arguments |
The model supplies authenticity and voice. It does not get to overrule the simulation.
A prediction without an error bar is marketing, not analysis. Every EchoTest result carries two distinct uncertainty signals — and we're careful not to conflate them.
Each headline metric ships with a p10 / p50 / p90 band, computed by resampling the agent panel a thousand times. This answers: “if we'd drawn a different sample of personas, how much would the answer move?” A tight band means the result is robust to sampling; a wide one is a signal to run a larger panel.
Each report gets a letter grade (A through D) tied to panel size, using Wilson confidence intervals for proportions. A 500+ agent panel earns an A with a margin of error around ±3.5%; a sub-50 panel earns a D and a wide margin. This keeps small, cheap runs honest about their own limits.
Confidence bands measure sampling uncertainty, not predictiveaccuracy. A tight band tells you the simulation is internally consistent — it does not by itself tell you the simulation matches the real world. Establishing that second thing requires validation against real outcomes (Section 5). We draw this line explicitly because blurring it is exactly how “impressive simulation” becomes “unfounded prediction.”
When you run a real campaign, you can feed the actual results back in. The system compares its prediction to what happened, identifies systematic bias, and adjusts future predictions for your specific market. Two deliberate guardrails keep this honest:
The fairest criticism any simulation platform can face is: prove it predicts reality. We take that seriously enough not to fake it. Our validation protocol is deliberately strict:
This is a held-out test, not a self-graded one: the model is scored on launches it had no hand in. We are collecting design-partner outcomes for this study now, and we will publish the results — including where we were wrong — as the sample grows, in a dedicated follow-up paper. Until then, you will not find a headline accuracy percentage in our materials, because we haven't earned the right to print one.
A methodology paper that only lists strengths isn't a methodology paper. Here is where the model is weak or silent:
We'd rather you know these limits going in than discover them later. A tool you can calibrate your trust in is worth more than one that asks for blind faith.
EchoTest is a decision-support tool. Its outputs are probabilistic estimates intended to inform human judgment, not to replace it. This paper describes the system as of June 2026; methodology evolves as validation data accumulates.
A designed PDF to share with your team — plus a heads-up when the validation results land.