Product discovery pitfalls and anti-patterns

Product discovery: The good, the bad and the ugly

If you want to create great products and not realize after they have been delivered that they have fallen short of your expectations, you need to do product discovery.

Or, in the words of Marty Cagan:

“First, you need to discover whether there are real users out there that want this product … second, you need to discover a product solution to this problem that is usable, useful, and feasible”.

And you need to do it in a good way!

What this means is not only understanding the “best” practices and ways of thinking but understanding product discovery as a whole. So, not only the good but also the bad and the ugly. To be truly successful and not get surprised when it’s already too late, you need to embrace it from all sides, and only then will you be in control of the process.

To make you more successful on that path, here is an overview of the most impactful discovery anti-patterns and pitfalls. They can get in your way and even lead to failure if you are not aware of them, but they can also help you to be in control and thrive if you get to know them better. What is also very important to know that if you just take one anti-pattern and do the opposite, you will just stumble upon another anti-pattern, and we’ve noted this.

In most cases, the key is not just to balance between two pitfalls but to understand how to tackle both of them and combine these approaches, taking context into account.

But enough talk – let’s see this in practice!

#1 Not doing discovery vs. spending too much time on discovery

The first and foremost discovery anti-pattern is not doing discovery. Although it may sound like an exaggeration, this anti-pattern can be found in companies of all sizes and is quite deadly.

Why?

This way, you start discovering facts about your product (as well as your market and customers) only when you deliver the product for the first time, and this is almost never the version that is successful. Also, if this version took you too long to deliver (at this point, you’ve already spent too much money or time), which happens more often than not, this may also be the last version you’ll ship.

This anti-pattern mostly boils down to being too junior in product management or overly confident, thinking you already know what your customers want even if you don’t talk directly to them. This way, you won’t realize you’re perceiving assumptions as facts until it’s too late. So, whenever you feel confident about solving customers’ biggest problem, creating a likable solution at a good price, and without harming the world,- take a step back and reflect. You may have missed an important part of the discovery process if you’re taking the riskiest, most difficult, and expensive road to developing products, which could increase your chances of business failure. Check again.

The flip side of this anti-pattern is spending too much time on discovery.

This pitfall can be found in people with an overemphasized analytical side and who are not confident enough. This is mostly shown by not feeling comfortable taking risks or making decisions on your own without some of the facts missing. The risk of delaying a decision for too long can also be costly, as you may never get all the data you are searching for.

Besides personal characteristics, company culture may be adverse to any risk-taking, which can cause the whole system to get stuck. This may not be as deadly as the first anti-pattern (although it can be if we get stuck too long and start missing important opportunities), but it often slows things down and demotivates people, which is also not optimal and has an effect on the product’s success.

The goal of discovery is to de-risk product development, but the goal of the product is to be delivered so it can bring user and business value. If we make de-risking our overall goal and put too much time into it without spending enough time delivering value, we’re missing the true goal of product development.

The key is to balance between these two by surfacing as many assumptions as we have and then focusing our discovery efforts on the riskiest assumptions.

#2 Big Bang product discovery vs. Not doing big research

The second most common pitfall is the Waterfall / Project / Big Bang approach to product discovery.

What this mostly means in practice is that one person or a dedicated group of people specialized for research and design plan the discovery project, go in, gather data, structure it, revise it, create solutions, validate them, and usually after a couple of months (2-12) come back with the results and solutions that are presented to the people who need to implement this… Yeah. Specialized people provide great benefits to this approach, but waiting for project completion isn’t feasible when there are daily questions requiring discovery.

Don’t wait 2-12 months for solutions or research summary; consider it done. New questions arise, people become misaligned, and may even pull in opposite directions. Because of this, in most situations it’s not really feasible to do a big bang project-like discovery and then turn to delivery. Discovery needs to be continuous as we’re learning daily. We have questions and decisions that we need to make each day, and this is why we need constant opportunities to talk to customers and be in touch with them to get quick insights and make the best decisions possible.

On the other hand, we should be careful not to fall into the trap of not doing any big research. We’ve understood by now that we should do most of the discovery activities in the continuous mode, but we should not forget that there will always be certain types of research which are best to leave to the mentioned group of research experts and have that done in a bigger initiative or a project.

Most wide market researches will fall into this category (especially the studies where you want to measure a really big sample to see some bigger patterns), but again, this will heavily depend on your context. What we’re trying to emphasize here is always to try to keep an open and critical mind about various approaches.

The key for most of these situations is to use the continuous discovery approach:

“At a minimum, weekly touchpoints with customers, by the team building the product, where they conduct small research activities, in pursuit of a desired outcome” –Teresa Torres

And combine this approach with an occasional big(ger) research project (like competition or market research) where and when needed.

With this in mind, always feel free to experiment because who knows what the next approach will be!

#3 Discovery in isolation vs. Including too many people

With this third anti-pattern, we are covering the three most common mistakes in the traditional discovery approach. Doing discovery in isolation can come in various forms, but we will mostly encounter these two:

1) People doing the discovery are in a silo, disconnected from the rest of the organization or

2) Discovery activities are outsourced to an agency.

As for the first one, the problem is that you are getting only one discovery perspective (mostly from research and design), but you are omitting all others, which leads to misalignment and poor discovery results.

The second one is even worse since agencies will never know your context and strategy as well as you, and they will leave you with research summaries, but there will be no one to answer constant questions that pop up.

This is why modern discovery is advocating for a different approach where you do discovery in Triads (or Trios), which consists mostly of a product manager, a product designer and a lead engineer.

These roles map down to three common discovery areas which are viability, usability and feasibility (see the next pitfall for more information on this topic), so omitting any of these roles radically brings down the chances of success for any discovery effort and alignment inside the team. So, if you don’t include a product manager, you may be focusing on a problem that will not bring enough business value and have multiple unpleasant discussions while trying to use the discovery results; if you don’t bring in a product designer, you are probably going to have a product that is not really usable, no matter how sure you are of your skills.

But the most impactful changes from your plan will come (or won’t come) if you don’t bring an engineer along. In this case, you are likely to end up with a solution that is either too expensive or not feasible at all, while you are losing the greatest potential in your team for innovation. In our world, most of the innovation comes from new technical solutions that are applied to common problems.

Of course, the product trio doesn’t necessarily mean three people but a mix of these different perspectives and skills. This can mean less than three, but also more than three with additional roles included when and where needed (product analyst, subject matter experts, marketing specialist, etc.).

But, be careful here or you’ll run into including too many people in discovery.

One of the goals of discovery is to move fast (you may already hear Marty Cagan saying that good discovery teams do over 10-20 experiments each week), and if you bring too many people with you, this will slow you down, especially when you need to make a number of fast decisions. Aligning with more people takes a lot more time and you just want to uncover the riskiest and false assumptions, rapidly move from one source to another and create super short and fast feedback loops, which just isn’t possible with too many people on your team.

The key here is to include only a small number of people inside your team (or company) who bring the needed perspectives and move with them through discovery cycles as fast as possible.

#4 Not covering different dimensions of discovery AND Validation as discovery

When doing discovery, most of us tend to emphasize one dimension of discovery while forgetting about the others (we can see some of the examples in the previous anti-pattern). This approach can again lead us to miss some of the key points in discovery, and thus fail in developing a good product. To constantly have this on our mind, it’s very useful to be explicit about these dimensions at the beginning and to try to find the underlying assumptions through these aspects:

Desirability – Does anyone want to buy our product? (do they have this problem, are they getting value, etc.)

Usability – Can people use our product? (can they find what they need, understand it, etc.)

Feasibility – Can we build this product? (is it feasible for our business, etc.)

Viability – Can we create a sustainable business out of it? (should we build it?)

Ethics – Is there any potential harm in building this idea?

This will make sure that we have a problem worth solving, that people want our product, that they can use it, that we can make it, that we can create a viable business out of it, and that we’re not doing anything that will bring harm to the world and the people around us.

Also, when we’re assessing opportunities that we want to focus on, it’s important to use different lenses that fit our context and have a balanced approach for prioritizing them (e.g., Teresa Torres suggests using opportunity sizing, market factors, company factors and customer factors; Marty Cagan has his own set that you can look into, etc.).

What is an additional pitfall in this process is to do validation as discovery.

This means we are going into discovery with a solution that we already like and are trying to validate, which heavily falls into the confirmation bias anti-pattern. Even if you are trying to figure out if a solution is working or not working or if this is a good opportunity, you are not using the potential of discovery to its fullest.

It’s not about binary yes or no questions or answers but about comparing and contrasting different opportunities and solutions. It’s about understanding various opportunities and solutions and combining them to get the most optimal path you can take. There are very few simple and straight answers in discovery.

One more thing that will help you fight our ever-present confirmation bias is to define the experiment success criteria upfront so that you can see what your expectations are (and if they are not realistic enough and just too biased), and also it will help you in assessing opportunities and solutions much better.

The key to these pitfalls is to go as wide as possible in surfacing all those different kinds of assumptions (and assumption testing will enable you to do those 10-20 experiments a week best discovery teams do), and making sure that all your success criteria are clearly defined and agreed upon upfront.

#5 Concluding too fast vs. Analysis paralysis

We are back at the topic of discovery speed and while you want to move as quickly as possible, concluding too fast can get you in trouble. This mostly means that if you do a customer interview and pinpoint an opportunity, it’s not really good to conclude that you have a good case for solving it and instantly jump into creating a solution. The same goes for analyzing analytical data without double-checking it through qualitative interviews or assessing how your prototype works on a small set of users (or your friends and family members).

The flip side of this is the mentioned analysis paralysis, the anti-pattern where you always need one more test, one more metric, one more fact, and more and more pieces of data until you finally make up your mind and go with a decision. We just have to work with the fact that we don’t have all metrics in place (nor do we need all metrics, but only those that are crucial to our goals), nor will we ever have all data with thorough analysis done. And even if we did, we’re just reducing our chances of failure, but we can still fail so we need to act with what we know today while being prepared to be wrong.

In both cases, the key is to move fast using good practices – doing assumption testing, starting with super small, fast and cheap research (for instance – talking to a few of your customers), and gradually scaling it until you are ready to start delivering. Have in mind that most of the decisions you make throughout the process are going to be “two-way door” decisions, meaning – if you are wrong, you can always just go back and choose another path. And this is exactly what good discovery practices enable you to do.

#6 Customer-centric vs. Business-centric

If you’re in digital product development, you’re probably hearing the “be customer-centric” mantra each day. And this is definitely one of the keys to success, but even when you obsess with your customers you have to keep the business goal in your mind, otherwise you are not creating a viable product but you are introducing a very expensive and potentially deadly discovery anti-pattern.

Just imagine, when doing discovery, you will come across multiple opportunities – which one will you choose? If you are not connecting your opportunities to the business goals, you are maybe creating short-term customer value, but in the long run, you are not creating business value and ultimately this pattern can lead to bad business performance and prevent delivering value to customers. This may sound naïve, but many of us still make a lot of efforts in discovery and delivery that do not connect to the desired business outcomes. You can instantly check this by quickly going through your opportunities and backlog to see if you can connect all your efforts with the desired outcome.

The questions you want to ask here are: What metric should this opportunity tackle? How are we going to know that we are successful once we launch this? What changes on our dashboard are we targeting with these opportunities? And so on. Always keep the business value on your mind, throughout the whole process.

But you can also be business-centric and fail. How’s that?

This mostly boils down to choosing opportunities that don’t have proven customer value. These are the opportunities that mostly business wants us to tackle and while they are often very likely to move the needle on the business impact, this method also fails since these are short-term gains that will go in the opposite direction once customers stop seeing the true value of our efforts or even feel the shortcomings themselves.

Again, if you want to scan for this pitfall, make sure always to ask probing questions like – What problem are we solving? How many customers have this problem? Where is customer data that backs this up? How is this going to bring value to customers? And so on.

The key here is to start with business outcomes, discover and choose opportunities based on them, check throughout the process if we are on track or did we drift away from the goal or from the customer, and, at the end, check again and learn – Did we really move the needle and achieved the outcome that we wanted or fell short in choosing the right opportunity and tackling it in an impactful way?

#7 This can’t work at my company vs. This is the only way to do discovery

These are the final pitfalls that block the discovery improvement of most teams and organizations. Let’s reflect on the first one – believing that this can’t work at my place.

It’s always scary and unmotivating when you are in a system that has been doing things in the same way for a while or when you have certain people who are just very strong stakeholders of the current system and they are not too eager to hear about new things nor even think about changing them (and mostly there are some control and power-plays in these cases or even insecurity about how this will affect their role).

But there’s always something that you can do.

If you are not doing customer interviews, start by doing one each month. If you do them each month, start bi-weekly or weekly.

If you don’t have access to customers, see who has and jump on a call with them and the customers. Start by just listening and then gradually, meeting by meeting, find ways to ask questions, and start doing interviews yourself. If this is not working, see if you know someone who is like your customers and talk to them, and so on.

Maybe you’ve delivered something that was asked by stakeholders and it’s not performing as they imagined – that’s a great opportunity for you to shine and start implementing some new discovery activities.

Be creative, grab opportunities as they come and don’t get bogged down by asking for permissions.

The key is not to force changes but to meet the people where they are and move in small steps. See what is the smallest new discovery activity that you can do or improve today or this week and just do it. And then, there’s the other blocker for improvement – this is the only way to do discovery.

Although there are better and worse ways to do discovery, they exist only in a certain context. So, what’s written in a book, an article, shown on training, or have worked in your previous company may not be the best way to do discovery in your current context, and it may even hurt your discovery improvements if you start forcing it.

The key here is to have in mind that there is no one, true way to do discovery but that you have to discover what is working in your context and to co-create your discovery process by including the people around you and using opportunities that will appear along the way to continually improve your discovery process.

Product discovery anti-patterns: Next steps

With all this said, we wouldn’t go deeper into the rabbit hole more at the moment but be on the lookout for part two of this article. It’s up to you now to continuously reflect on your discovery process and be on a constant watch for any of these anti-patterns. If you didn’t have a chance to go through some of the best materials on this topic, feel free to check out the materials in the references session.

To give credit where credit is due, I can say that, in this article, I relied heavily on these authors and their work, and if you still haven’t read some of those, start now! (I’ve pointed out the best ones and sorted them by how much of their work I’ve used in this article)

References:

Teresa Torres: Continuous Discovery Habits: Discover Products That Create Customer Value and Business Value, Product Talk Blog
Marty Cagan and SVPG: Books, videos, blog
Jeff Patton: Books, articles