How to smoke test a startup idea

What a smoke test is

A smoke test measures what people do, not what they say. A survey asks “would you use this?” and almost everyone says yes. A smoke test shows a page for the finished product and waits: an email typed, a plan button clicked, a login tried. Each of those cost the visitor something, so each is evidence.

A fake door test is one kind of smoke test: a button for something that isn’t built yet. A smoke test is usually a whole small site of them.

Steps

1. Write down the bet. Who it’s for, the problem, what they use today, and the number that would make you build it. For example: “If 1 in 20 freelancers who click my Reddit ad joins the waitlist, I’ll build the invoice chaser.” Pick the number from your own economics, before you see any results.
2. Build a page that sells the finished product. A headline that names the outcome, three reasons, a pricing page with real prices on the plan buttons, and a sign-up. Write it as if the product exists, because that’s the decision you’re testing.
3. Add the actions you’ll count. A waitlist form, a tracked click on each plan button, and, if your buyers expect an app, a log-in form. A login attempt is strong intent: they expected the product to exist.
4. Tell the truth right after the click. The sign-up says the product isn’t built yet and that they’re on the waitlist. Never take a password or a payment you can’t honour.
5. Send traffic you can tell apart. Give each channel and ad its own tagged link (?utm_source=reddit&utm_medium=paid&utm_campaign=launch). A small paid budget reaches strangers; friends and followers are kind, not evidence.
6. Read it in order. Visitors, then whether they read the page (time on page, bounces), then plan clicks, then sign-ups. The step with the biggest drop is where the idea or the page loses people.
7. Decide. Build it, change one thing and run again, or drop it. Write the decision down next to the bet from step 1.

What to measure

• Conversion: the share of visitors who signed up. Compare ideas and ads on conversion, not raw sign-ups.
• Paid and free apart: judge a paid test on paid traffic alone. Free traffic from people who know you inflates the rate.
• Cost per email: ad spend divided by sign-ups. If a visitor costs you $1 and you can afford $10 per interested person, you need 1 in 10 to sign up.
• Sample size: don’t call one ad or channel better than another on fewer than about 100 visitors each. Small differences need more.

When a smoke test is the wrong tool

• You can name your buyers. If there are 30 companies who could buy it, call them. A page can’t beat a conversation with your whole market.
• A page can’t explain it. Deep tech, regulated products and anything people must try to understand get low sign-ups for the wrong reason.
• A two-sided marketplace. Test each side separately, and expect the harder side to decide it.
• People already ask to pay. Then sell it. Pre-sales beat any smoke test.

Running one on Hunchwire

Hunchwire hosts the page at its own address (yourname.hunchwire.site, or your own domain), adds an honest waitlist form, tracked buttons and a fake login form that never sends the password or email, and shows visitors, conversion, the funnel, channels and cost per email. You can build it yourself or let an AI agent run it. Hunchwire is in private beta: free, by invitation from the waitlist.

Questions