Key Assumptions Testing

The convergence of your market research with empirical results from your experiments prepares you to move forward with confidence.

After having completed this step and the previous step, you have de-risked your product at the level of individual assumptions as much as you reasonably can. This accomplishment does not mean that, when all the assumptions are put into one product, the fully assembled solution is assured of market success, but you have done the best you can. Some assumptions will never be able to be fully tested until there is a product and it is put into production. That testing comes in the next two steps.

Content will be published as it becomes available.

Content will be published as it becomes available.

Designing good experiments requires that you be systematic and think creatively. There is an entire field of research, behavioral economics, that explains why people make the decisions they make and how to design proper and efficient experiments that will illuminate this behavior, so I won’t go into detail within the confines of this workbook. Instead, I suggest that you read some of the literature in this field. One book I find immensely useful in describing the value of well-designed experiments as well as the emerging field of behavioral economics and some of its basic principles is Think Like a Freak by Steven Levitt and Stephen Dubner, who are well-known for their bestselling book Freakonomics.

It should be fun to design experiments that test your assumptions, and you don’t need a team of Ph.D. economists, especially if you’re only testing one assumption in each experiment. If an experiment is more complex, such as one that tests multiple assumptions at the same time, it gets much more difficult and less fun. Hence, try to decouple assumptions and design your experiments carefully.

On the first worksheet provided, describe some detail about the experiments you plan to run, including which assumption(s) will be tested, what resources you need for the experiment, and most importantly, what outcomes will validate your assumption(s). If you don’t define your outcomes before the experiment, you will inevitably skew the experiment in favor of your assumptions, making the outcomes worthless as indicators of the strength of your new product’s plans.

After you conduct each experiment, use the second worksheet to make notes about each experiment, including the outcomes, whether your assumption(s) were validated, and what your next steps are based on those outcomes—whether that means revising work you did earlier in the 24 steps, conducting additional experiments with revised variables, or taking other appropriate actions. Sometimes the result of a successful experiment is more experiments. You have to be patient.

Notice how simple the tests can be. More complicated is not better, it is actually worse. Some may be as simple as calling your vendors to validate your list of how much it costs for each part that goes into your product (the cost of goods sold), as that cost is often a key assumption. Or, if getting a certain highly influential customer in your chosen market segment is a crucial validation point, you’ll want to more energetically measure their commitment level. The expectations and commitment thresholds are still low because you should not be communicating to customers that you have a product being sold—you’re still in inquiry mode for a bit longer, which is a much better environment for experiments. Enjoy it while it lasts because it will be ending soon!

Finally, remember that a valid assumption is still only an assumption, because only with paying customers do you truly know that your startup has a sustainable market. I will guide you through how to start determining whether customers will actually buy your product in Step 23, Show That “The Dogs Will Eat the Dog Food.” An experiment that invalidates an assumption will often tell you more than an experiment that seems to give you confirming evidence regarding an assumption. Nevertheless, the more you test in advance, the more problems you can identify while fixing problems is still relatively inexpensive.

Content will be published as it becomes available.

Content will be published as it becomes available.