Customer interviews: how to test demand before you write code

The most expensive early mistake is building a product for a problem that barely hurts. People will politely say “interesting”, take the free version and never pay. Customer interviews exist to find that out in a few weeks of conversations, not a year of development.

What you're actually testing

Not “will people like the idea”. Almost everyone likes ideas; it costs nothing. You need to test three things:

Does it hurt. Does the person have this problem right now, not “it happens in principle”.
Are they trying to solve it. Do they spend time or money on it, keep a spreadsheet, an intern, a second service for it. If they do nothing, it's bearable.
What it costs them. In hours, in money, in nerves. That's where your price will come from.

How to ask

The main rule: ask about the past, not the future. About the future, people answer the way they'd like to see themselves.

Bad: “Would you use a service that does X?” Almost everyone will say yes.
Good: “Tell me about the last time you did X. What was awkward? What did you try?”
Bad: “How much would you pay?”
Good: “How much do you spend on this now — in money or in staff time?”

Don't tell them about your idea in the first half of the conversation. As soon as a person knows what you're building, they start answering about your product instead of their life.

How many conversations you need

Fewer than you think to see repeats, and more than you'd like to trust them.

The first five conversations prove almost nothing: one enthusiastic person outweighs everything else. Precision grows slowly — to halve the error of your estimate, you need four times as many conversations. Fifteen to twenty interviews in one segment usually show whether it really hurts there.

And an important point: talk to one segment at a time. “Small business” isn't a segment. “Dental clinics with two or three chairs” is. You can't average the answers of an accountant and a warehouse manager.

A conversation is also your first customers

A good interview leaves a trace. A person who's in pain often asks: “When will it be ready? Will you show me?” Write those people down. They're your first users, and later they're the easiest people to check with whether you've solved the problem.

When conversations aren't enough — a demand test

Interviews answer “does it hurt”. “Will they click” is better answered by a simple test: a page describing the product with a “sign up” button, and a little advertising pointed at it. It costs money, but it measures an action, not words.

A combination works well: conversations first, to understand whom to write to and about what, then a test, to check whether strangers respond to it.

How it works in Runway 200

In the game, interviews are one of the first actions. One click is three conversations in the chosen segment, and what you get back isn't “they liked it” but an estimate of the pain with an error margin. Every person answers with noise, so after the first three conversations the margin is huge, and it narrows as slowly as in real life: to halve it, you need four times as many conversations. And you take the time for them from everything else.

If you talk to the segment you're targeting, some of the people ask to see the product when it's ready — and become your future customers. Conversations with another segment give you only knowledge.

To test demand there's the landing test: $37.50 for 70 views, and you see how many people signed up. Conversion accumulates from test to test, and the more views, the more you can trust it.

You can pick the wrong segment in the game just as in life: after that, neither advertising nor quality will help. Why — in Customer churn.

Try it yourself: the first ten weeks are free, no sign-up. See how many conversations you need before you stop doubting.