CommonLounge Archive

How do you run an MVP experiment - Part 2

December 26, 2017

To recap the 6 steps to a successful MVP:

  1. Determine the problem you’re trying to solve and the solution your product provides.
  2. Identify your assumptions about the problem and solution. (Covered in the previous tutorial.)
  3. Build testable hypotheses based on your assumptions. (Covered in the previous tutorial.)
  4. Figure out how you’ll determine whether the test was successful or not. (Keep reading to learn how.)
  5. Pick the best MVP type for your unique situation. (Keep reading to learn how.)
  6. Execute the MVP experiment and collect the data. Decide if your hypothesis is true, false, or somewhere in between. (Keep reading to learn how.)

Defining the Minimum Criteria for Success

In order to know whether your hypothesis (and related assumption) is true or false, you must define what “success” means for your MVP experiment.

In order to know if your product is worth pursuing, you’ll need to create a Minimum Criteria for Success.

To do this, first write out the entire cost of your product (developer’s time, materials, your time, wages, advertising expenses, brand effect, opportunity cost, etc.).

Ask each person involved how long it will take them to work on the project, and figure out the entire cost for each person. Ask them what else they could do in that time to figure out the opportunity cost.

Next, write out your product’s expected return (increased revenue, more time spent on your website, more likes or shares, higher CTR, higher LTV, etc.). Combine these to estimate the total expected return (in dollars) of your product.

Now compare the cost to the expected rewards. What kind of rewards do you need to see to have a profitable product?

Make a statement that says: “If it costs X, we need to see this metric Y improve this much to know it’s worth it.”

If you can prove this return with your MVPs, you’ll know that your idea is worth pursuing.

Which metrics do startups focus on?

Startups need to build initial traction for their idea. They need to see signs of real interest in their product.

This can be shown through something called validation metrics. These include number of sign ups, number of social shares, the average purchase price, or email open rate—metrics that show customers’ interest in a product.

Startups also need to focus on the economic sustainability of their idea as a whole, with metrics like gross margins and customers’ LTV.

For startups, opportunity cost is not a big issue.

Choosing an MVP type

There are multiple ways to conduct an MVP experiment, from setting up “fake” buttons on your app or website to piecing together handmade versions of products to simulate the actual experience.

You will learn more about each type in the following tutorials.

Evaluating the results

MVP experiments are like any other scientific experiment.

They need to be precise and reproducible. They need to be measured and recorded faithfully.

You’ll need to get quantitative data (numbers) from your tests.

At the same time, you need to get qualitative data (observations and insights) to get an accurate picture of the results.

Numbers tell you what is happening (whether customers buy your product or not), but observations tell you why it’s happening (what motivates your customers to buy).

When you take both into account, you’ll be able to determine whether your assumption is true or false.

Make the call

Once all the data is in, analyze it and make your decision:

Do you make the feature or new product? Do you give up on the idea? Or do you run more tests on another hypothesis?


© 2016-2022. All rights reserved.