A good experiment changes your mind. A great one changes your roadmap. Most startup A/B tests do neither because they're under-powered, poorly scoped, or test cosmetic changes.
Score every experiment on three axes: impact (how much could this move the metric?), effort (engineering days), and confidence (do we have data suggesting it'll work?). Only run experiments where impact × confidence > 2 × effort.
Define your success metric and minimum detectable effect before you start. If you need 50,000 users to detect a 1% lift and you have 2,000 weekly actives, you're not running an experiment — you're flipping a coin.
ProductBlaze's experiments board automatically scores ideas on impact, effort, and confidence so you never waste a sprint on theater.