Experimentation is not about right or wrong. It’s about learning things that you genuinely didn’t know. The secret is to become comfortable with the uncomfortable and to make room for your own sense of vulnerability. When you’re able to embrace not knowing something, or have experiments come back that disprove your hypotheses, you’re going to discover amazing insights that benefit you, your team, and your organization.
In this episode of the Product Momentum Podcast, Sean and Paul welcome Holly Hester-Reilly, Founder and CEO of H2R Product Science. In this dynamic and fast-paced conversation, Holly discusses her approach to the product science method, one that focuses on using science and empathy to manage risk while building high-growth products and teams.
“Our job as product people is to manage the risk of product failure,” Holly says. “Part of that risk is to avoid looking bad in front of our teams, peers, and managers. We have to shift the mindset and the conversation away from right or wrong so that we can begin to pride ourselves on learning new things.”
Product leaders have an enormous role to play here, Holly adds. “The only way for us to make that mindset shift is for us to be the example by calling out when the people around us learn something new and saying, ‘that’s what we want to see more of!’”
Listen in to catch more from Holly:
[02:16] The product science method. It’s really about the difference between what people say they will do and what they actually do.
[02:58] Design experiments around past behaviors, not abstracts and hypotheticals.
[04:51] The role of data and metrics. The cool thing about software is we can actually measure how users behave. The right metrics …that’s the best possible predictor of future behavior.
[07:42] Why smart companies with reams of data still make flawed product launches. They’re too comfortable.
[08:25] The Emperor’s New Clothes. Do we have the willingness to be uncomfortable, to be the person who will stand up and say to the boss, “here are the reasons why your pet project is going to fail.”?
[10:12] Confirmation bias. Channeling Richard Feynman, “you must not fool yourself, and you are the easiest person to fool.”
[10:45] Rapid research. You have to be super-focused on the most important thing to learn and let go of the idea that you might not learn the other things.
[12:38] Exposure therapy. The more times that you’re exposed to something, the more comfortable you become with it.
[15:05] Optimism bias. Gets in the way of making good business decisions like so, so much.
[16:14] How long does it take to change somebody’s mind about their pet project?
[17:49] The role of experimentation. It’s not about being right. It’s about learning things we don’t already know.
[21:19] Premortem risk assessment. Put yourself in a place where risk is already assumed to be real.
[22:34] Our job as product people is to manage the risk of product failure.
[24:02] The difference between good and fantastic product research.
[25:46] Take a snapshot. Make sure that your team is situating who your customer is within the strategy of the product.
[27:22] Practicing discovery. As a product leader, you should have a strategy that is a series of product-market fits.
[28:27] Measuring the value of research. Two parts: quantify the value of research and know when you’ve done enough of it.
[32:10] “Faster horses.” At least you know what outcome your users want.
[33:48] Innovation. Innovation drives a significant change. It doesn’t just increase the amount of something you’re selling: the revenue, the number of users. It changes the rate of that.
Holly’s Recommended Reading
Indistractable, by Nir Eyal.
Holly Hester-Reilly is the Founder and CEO of H2R Product Science, a product management coaching and consulting firm that teaches the science of high-growth product development. Holly is a former Columbia University research scientist and has led over a dozen successful digital product initiatives at startups, high-growth companies, and enterprises like MediaMath, Shutterstock, Lean Startup Co., and WeightWatchers. With those experiences, she has developed the Product Science Method, a framework to discover the strongest product opportunities and lay the foundations for high-growth products, teams, and businesses.
Her team at H2R Product Science partners with startup founders and product leaders to share this framework, helping them to figure out which product growth opportunities they should pursue and build the product management skill to deliver on their goals.
Holly also teaches public and private workshops and has spoken about building high-growth products for events such as Lean Startup Summit Europe, growth equity firm General Atlantic’s CIO summit, top boutique design and development agency Thoughtbot’s employee summit, ProductTankNYC, Parsons School of Design, and the Product School.
Be sure to tune in as Holly hosts The Product Science Podcast.