New research on Lean Startup: hypotheses good; MBAs bad

How we learn to learn has a big impact on success

New research on the Lean Startup methodology finds that a business school background (i.e. an MBA) can hinder a founder’s ability to adapt to new evidence in an early-stage lean startup context, if they don’t embrace the hypothesis-testing approach through customer discovery interviews. On the other hand, if they do embrace the approach — getting out of the building and gathering evidence from customers — they move faster than non-MBAs.

As an MBA graduate who runs startup accelerators and teaches a Lean Startup course much like the one in the research, I am very interested in what this means. So, let’s dig in!

The research by Michael Leatherbee and Riitta Katila is published in the Strategic Entrepreneurship Journal, which is paywalled — but they discuss the findings in this Stanford podcast, joined by Steve Blank. The research covers 388 teams who went through an 8-week experiential lean startup course.

I’ve read the published journal article and highlight some of the key findings, with my comments in italics, below:

1. Probing hypotheses via customer interviews leads to faster convergence, both through invalidating hypotheses and discarding them, and also inspiring new hypotheses and new business ideas. But also that as more aspects of the business model are validated, the generation of new hypotheses slows: we “find that convergence leads to fewer new hypotheses; that is, there is a natural stopping mechanism.”

In other words, talking to customers can open up pathways you hadn’t considered, and inform your pivots.

2. More hypotheses isn’t better. Teams that generated more hypotheses tested fewer of them. The researchers propose that this means there should be an emphasis on quality rather than quantity.

I disagree. I think ideation and divergence is an important part of the process, and especially in a team setting it allows everyone’s ideas to be shared and have a chance to be considered. I would instead focus on the process of ideating and the process of selecting which hypotheses to test and which to abandon.

3. Some teams with MBAs resisted the process more than average, because traditional business school education teaches “learning-by-thinking” (e.g. Porter’s Five Forces). The researchers propose that once MBAs have thought through the hypothesis they feel no need to test it through “learning-by-doing”. “[T]eams with an MBA are significantly less likely to conduct a major change in their business idea”.

I’m reminded of the TED talk about making a tower out of spaghetti with a marshmallow on top, in which kindergarteners outperform business students.

And the reason is that business students are trained to find the single right plan, right? And then they execute on it. And then what happens is, when they put the marshmallow on the top, they run out of time and what happens? It’s a crisis. …

What kindergarteners do differently is that they start with the marshmallow, and they build prototypes, successive prototypes, always keeping the marshmallow on top, so they have multiple times to fix when they build prototypes along the way. Designers recognize this type of collaboration as the essence of the iterative process. And with each version, kids get instant feedback about what works and what doesn’t work. [Tom Wujec: TED website]

4. On the other hand, when MBAs embrace the process they excel. Probes (testing hypotheses through customer interviews) “by MBA-teams are more likely to lead to more new ideas (with the corresponding hypotheses) and more business idea convergence, suggesting that MBA training helps interpret data from probes.”

Obviously as an MBA graduate myself I will ignore finding three and tell everyone that this finding confirms how awesome I am.

5. Teams that engaged more with the method, i.e. hypotheses and interviews, had better outcomes in the 18 months after the course.

This finding surprised me a bit. Because while I know that the evidence-gathering process is important in the early stages of a startup I wasn’t sure that would translate to success down the line. There are so many other things a founder needs to do beyond customer discovery and product creation.

Learning and failure

The research makes a distinction between a learning-by-thinking and learning-by-doing mindset, and says this is part of the way the MBAs were taught to learn. In business school they teach by using frameworks and case studies, so students learn to learn-by-thinking.

I had a similar conversation this week with a fellow Entrepreneurship lecturer about how we have to re-teach our startup students that failure is OK.

Other university subjects teach students to prove how smart they are. In assignments and presentations students are supposed to tell us their conclusion and then prove that they are right.

In a startups class we want them to tell us about their process and how they learned they were wrong.

In my class I go out of my way to praise and reward students who do that — when they invalidate a hypothesis I make a big deal about how great it is that they’ve listened to the evidence and made a change in their approach. (And I give them good marks for that week’s presentation, to show them I mean it.) Once the first team admits to having been wrong, others start to follow. But it can take some students a month or two to embrace it.

I think this avoidance of failure is learned. Or rather, taught.

I don’t think it’s quite the same as a “fixed mindset” vs “growth mindset” in the Carol Dweck sense. I don’t think the students think that being wrong means they’re not smart. Or not good at the thing. I think our curriculum designs and assessment techniques have taught them—or even trained them, through rewards in the form of marks and grades — that it’s important to be right.

As my friend Keira semi-joked about what it means to be a nerd: “Sometimes, I’d rather be right than happy.”

Keira Daley: LadyNerd promo

Final thoughts

“The way we do things” is a powerful force, whether it’s explicit or not. Spend a long time doing something a particular way — thinking in a particular way — and being rewarded for it … it becomes a default.

It’s worth reflecting on your defaults, whether you’re launching a new business or working in an established one.

Do you think things through and assume they’re a done deal (learn-by-thinking), or do you have a bias for action (learn-by-doing)? Are you willing to have your assumptions challenged or do you only look for data that confirms that you were right?

Recognising what your defaults are, and knowing that there are other ways of doing things, that’s half the battle.

Actually taking the leap is the other half.

What will you do differently this week?

Maybe not this | Photo by Quino Al on Unsplash

I work with startups, teach entrepreneurship, and freelance in improv and leadership coaching.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store