The art of writing a great hypothesis.

Prologue

In my previous article, I spoke about hypothesis generation and how important it is to design a product and user experiences around hypotheses and then validate/invalidate them. That got me thinking. I wanted to write about how I think we should be coming up with these hypotheses and what a good hypothesis would look like and a few checkboxes that it should tick etc.

For today’s topic let’s dive deeper into how to formulate a hypothesis and what a good hypothesis would look like.

Topic Introduction

Coming up with any hypothesis isn’t the point. Coming up with the right one is. Hypothesis generation is an art and a science, and a good hypothesis leads to a great product.

There are two kinds of hypotheses I have seen:

1. Hypothesis based on the company’s/product’s internal limitations.
2. Hypothesis based on the user’s needs and pain points.

The problem with the first kind is that you are actively thinking about the problems you have internally and assuming that the customer’s problems are because of your limitation.

Think about this example: I think us not having the feature “Blah” is stopping the customers from making the purchase.

If you were the product manager reading this hypothesis, what would you do? you would think “Ah, let me start introducing “Blah”.

This hypothesis is not helping us with:

1. Understanding the customer more and what they are trying to do.
2. Understanding the product user experience more and where the limitations are.
3. Understanding what it would take from an engineering standpoint to get this to production.

In reality, this is biasing us towards one solution and only one and that is not what we want to do.

Now think about this example: I think customers do not complete the purchase because we aren’t able to tell them more about the product and what we have to offer.

This hypothesis is doing two things:

1. Telling the users what we have and letting them decide if they want to buy it.
2. Validating/invalidating the fact that because we do not have “blah” people aren’t buying it.

As you can see, hypotheses are powerful and with great power comes great responsibility :)

The conceptual Framework

I wanted to talk to you guys about two things here:

1. What are the various characteristics of a good hypothesis?
2. How can we word a hypothesis?

Various characteristics of a great hypothesis:

A great hypothesis can be tested: This in my opinion is a very powerful statement. You are “testing” your hypothesis, you are not trying to “prove” your hypothesis. A hypothesis that you could not prove is as powerful as a hypothesis that you could prove because now you know what did not work. In fact, a hypothesis that you could not prove teaches you so much more than a proven one. You have a unique opportunity to rethink the whole domain starting from users and their pain points and as a product manager that is golden.

This also hits home a point that’s very dear to my heart, if you have not failed you are not trying new things.

A great hypothesis reduces risk: Along the same lines are point #1 a hypothesis can be either proven or disproven which means you are moving one step closer to where you want to be or your north star. With a disproved hypothesis you know which path not to take and with a proven one you know which one to take. Either way, you are moving closer to your goal. Know that if you do not test enough and often you are in the same place you started and that is not the place you want to be.

A great hypothesis is specific: Let me explain this with an example, let’s say you have the following services on your website — childcare, cooks, cleaners, and all this for your home. You started with the following hypothesis: “I believe that as a homeowner they will use our service over others because of our web pages ease of use”, this is so unspecific in my opinion — everyone wants ease of use so how are you defining ease of use in your webpage? I would instead think about people who are in a household — mom, dad, kids. Most moms and dads are probably the ones looking for the services on your webpage. I would then think about as a mother who is looking for childcare, cooking, and cleaning what their pain points could be and formulate the following hypothesis — “I believe then when a mother with young kids is looking at our website for services that are looking for someone whom they can trust and open their homes to.”

The difference between the first and the second hypothesis is that you can craft better solutions to test the second hypothesis and is more specific to a particular persona.

A great hypothesis can be measured: I am going to take the same two hypotheses as the above section for this one too:

Hypothesis #1: “I believe that as a homeowner they will use our service over others because of our web pages’ ease of use.”

Hypothesis #2: “I believe then when a mother with young kids is looking at our website for services that they are looking for someone whom they can trust and open their homes to.”

In the first hypothesis you can see that you are trying to make it easy for someone to use your product, what does that mean exactly? are you looking at increasing the # of people who land on your product? Are you trying to make more people stay with your product? Are you trying to say they spend less time finding what they are looking for? The problem with such a hypothesis is that it is so hard to measure that you do not know where to start, where you are, and where to go. In the case of the second hypothesis, you know that by increasing the aspects of trust (maybe with peer reviews, community feedback, etc.) you think that there will be more mothers who use your product for their needs and that is easy to measure, measure the # of mothers who use your product without the trust feature and measure after you introduce the trust feature. With this, you know exactly where to start, where you are, and where to go.

Wording a hypothesis:

Are your customers looking for a drill? a hole? Or a way to hang a picture? Or a way to get two wood pieces together?

Think about this for a minute — a user is buying your product not because they are doing you a favor but because they have a job to do, and your product is helping with that job. From this stems the Jobs-to-be-done or JTBD framework. When you work your hypothesis try to use this as your structure — I believe that a [user] who has a [JTBD], will use [product] because we will solve for this [pain point] using this [feature]. Here you are validating everything that’s under [].

The Nexus Beyond:

I want to go back to the example from the previous post where I spoke about the companies that have been successful with successful hypotheses generation and the validation phase. In one of the conversations, Jeff Bezos made a very good point about how companies think about their decisions in a do or die fashion and how every decision can also be backtracked, and the process restarted all over again but in this case, I want to add one other factor with a great hypothesis you do not have to go that far out to change things.