Design an A/B test for {{hypothesis}}. Feature/change: {{feature_description}} Primary metric: {{primary_metric}} Current baseline: {{baseline}} Provide: **Hypothesis Statement** - If we [change], then [outcome] because [reasoning] **Test Design** - Control: What users see today - Variant(s): What changes - Targeting: Who is included/excluded - Allocation: % per variant **Metrics** - Primary metric (what we optimize for) - Secondary metrics (what we also watch) - Guardrail metrics (what should not degrade) **Sample Size & Duration** - Minimum detectable effect we care about - Estimated sample needed - Expected test duration **Success Criteria** - What result would lead to ship? - What result would lead to iterate? - What result would lead to abandon? **Risks & Mitigations** - Potential confounds - How we address them
A/B Test Design Framework
Create rigorous A/B test designs with clear success criteria
78 copies0 forks
Details
Category
AnalysisUse Cases
Experiment designFeature validationData-driven decisions
Works Best With
gpt-4claude-3
Created Updated Shared