How to Avoid a Product Manager’s Worst Nightmare

How to Avoid a Product Manager’s Worst Nightmare
Daniel Elizalde

I originally wrote this article as a guest post on MindTheProduct.com

A failed launch is a terrible situation in which your product starts breaking or misbehaving as soon as it is launched into production. You and your team have spent the past few months planning and executing the requirements, but the day you launch to production, something goes wrong.

Users can’t access the software. Errors appear all over the place. The site is non-responsive. Etc. And this is all happening while your marketing team is working at full steam letting the world know about your brand new product release. Indeed, this is a Product Manager’s worst nightmare.

So as a Product Manager, what can you do to mitigate these risks? In this post, I share some of the precautions you can take to minimize the risk of a failed launch. With strategy, planning, hard work, and a bit of good luck, you can avoid these failed situations.

Define an MVP That Fits Realistically into Your Timeframe

In any product release, no matter how low or high the stakes, it is very important to have a clear understanding of the requirements and to propose an MVP (minimum viable product) that is doable within that timeframe.

Overall, it’s better to release something that has limited functionality but is very stable than to release extra bells and whistles at the expense of reliability.

Remember that software is always a work in progress. The only certainty after a successful “version 1.0” release party is that the following Monday, you’re back to work on version 1.1.

Related article: The Simple Rule for Feature Prioritization

Be Realistic About Your Team’s Capabilities

As you create the product roadmap and pitch it to executives, it is important to be honest about the capabilities of your team. We often fall prey to the “halo effect” and believe that since we have a great team of back-end developers, they can tackle any other area, such as mobile, UI, etc. However, software disciplines are very specialized these days. For example, in SaaS software teams, it’s common to find specialists for services, databases, networking, front-end, security, mobile, etc.

Make sure you work with Engineering leaders to understand what it will take to build the product. If your product is very complex, needs high availability, or has very tight deadlines, be realistic about whether your existing team is sufficient to meet those needs. You might need to change the scope or convince the executive team to hire the resources you need. Otherwise, it’ll be almost impossible to deliver on the product promise you are making.

Don’t Forget to Include a Project Manager

Agile zealots might not agree with the idea of having a Project Manager. After all, the Product Owner should be able to handle it all, right? I don’t think so. This approach might work for small products with a handful of developers, but when the scope grows, this model doesn’t hold.

Complex products will have multiple teams working on related items, and there need to be people dedicated to making sure everything fits together. Scrum of scrums can help, but in my experience, it is better to have people dedicated to tracking progress and timelines and making sure everything is advancing in the right way. This need is accentuated if your product includes both hardware and software.

A strong Project Manager can own the day-to-day operations, keep track of the goals vs. the schedule, and keep everybody on the same page—vendors, Engineering, and other departments. This person can be the central point of communication for reporting status and risks.

A note of caution: The fact that PRODUCT Managers often have a PROJECT background doesn’t mean we should take on both roles at the same time. The more we can offload those responsibilities to a dedicated Project Manager, the more risk we mitigate and the more time we’ll have to focus on key Product needs. If you find yourself doing more PROJECT than PRODUCT management, then you are probably not adding all the value you could be.

Agile Development Can Reduce Risk

Agile provides a framework for minimizing risk by implementing iterations that produce working software. It is not a silver bullet, but it does help a lot. It provides us with a framework for continuous improvement, one sprint at a time.

Each iteration is self-contained, meaning that it includes design, development, and testing. Therefore, the code on every single iteration is tested to ensure it meets the desired levels of quality.

As part of the testing strategy, the QA team should develop unit tests, perform regression testing, and build a comprehensive automated framework to validate all the use cases defined by the Product Manager. There will always be some manual testing on every iteration, but the better the automated coverage, the better the chances of success.

Now, making sure the Development team implements the right delivery processes is not the responsibility of the Product Manager. It is the responsibility of the delivery team, and their engineering leaders, to make sure that their execution process yields good results. But in many companies, especially at small startups, these processes are often not in place.

Since the product launch is your responsibility, I recommend doing some investigation to ensure all the right processes are in place. If they are not, then it is okay to start conversations with Development leaders and begin shepherding the creation of those processes. Otherwise, your chances of success will be greatly diminished.

Related articles:

Plan for Performance and Stress Testing

Product Managers must understand the impact to customers when their product is down. For example, if my system fails, will businesses cease to operate? Will electricity go down? Will people’s lives be in danger? The bigger the impact, the more you have to plan for performance and stress testing.

As a Product Manager, you also need to understand your audience and the overall usage of your product. This understanding should be part of the product requirements. You should be able to answer questions like:

  • Will we have 100 or 100,000 concurrent users?
  • Which times of day are more likely to have peak usage?
  • What redundancy and elasticity specifications does the product need in order to meet demand at peak times?

Understanding the constraints is the first step. The next one is to come up with a plan to test them and ensure you’ll be ready to handle them when they occur. Make sure to have some projections on how your adoption will grow and how your infrastructure needs to scale in order to meet that demand. Chances are you’ll be way off, but you need to start somewhere.

To ensure this planning for performance and stress testing gets done, it should be part of your standard process, and not a one-off exercise. Performance testing must be done in every sprint, and it should be done in an environment similar to the real world. Anything less, and you risk maxing out your website very fast. The performance team should have their own stories in the backlog and should have deliverables for each sprint. Additionally, I recommend doing end-to-end performance testing several times before the release. Which brings me to my next point…

Allocate Time for Integration Testing

Even though Agile promotes creating production-ready code on every sprint, the reality is that it is very hard to get there. If you have a big Development team, each “Agile Pod” might be able to test their part of the code, but there is no guarantee that everything will work flawlessly when you merge the code of hundreds of developers.

To mitigate this risk, I recommend putting aside at least one sprint for integration testing and stabilization before every major release. The goal of this sprint is to stabilize the system and get it ready for production. No new features will be developed during this sprint.

On top of the technical challenges associated with integration testing, be prepared to convince your Executive Team that no new features are going into the product during that final sprint. They might not be happy, but it’s our job to educate them on the process and help them understand that in the end, this effort will benefit the product, the users, and the company.

Communicate Delays

Let’s face it. Delays happen. Building complex products is hard, and it is not an exact science. Many things might come up, and delays are likely to occur.

When faced with a delay or any other roadblock, it’s imperative to have open communication with the Executive Team. Sponsors need to have timely access to information so they can make decisions that might affect the whole company (not only your product).

In this scenario, take advantage of Agile and use the end of each sprint to inform your Executive Team about the progress, and more importantly, about whether the product is on track to meet the deadlines you agreed upon.

It’s Better to Delay Than to Implode

If after all these precautions, you realize that your product still won’t be “ready” by the deadline, then you need to make a decision: to push the release out further or to cut scope to ensure that at least something is released. Notice that I say “ready” in quotes, meaning that it’s up to you, along with your team, to determine what ready means. You can play with the scope and with the timelines, but I do not recommend playing with quality as a variable to meet a deadline. That’s how Product Management’s worst nightmares become a reality.

When faced with release delays, cut scope or extend release dates. Never sacrifice quality.

I’m not saying that delaying is easy, but it might be less expensive than pushing forward with a faulty product. To make these decisions/recommendations, it’s important to understand both the technical and business side of your company, so you can understand the impact even before you make the recommendation to your Executive team.

A good example of managing a delay comes from Gran Turismo, the best-selling car racing game in history. The release of Version 5 was delayed for a couple of years due to “executive decisions” because they believed the game was not ready to meet expectations. In the meantime, the company released a “reduced” version called Prologue to keep the fans engaged while they worked on the final product.

The approach worked. Although Microsoft and others launched competing products during the waiting period, the launch of Gran Turismo 5 was a huge success, in part due to the credibility they maintained with their audience and the fact that when the product came out, it truly delivered on its promise. Not all companies have this luxury, but it’s worth considering whether there might be some middle ground.

The Bottom Line

At the end of the day, it’s the Product Manager’s responsibility to bring the product to light. Failure to launch can be a disaster not only for your company but also for your career as head of the product. Luckily, many of the risks can be mitigated with good planning, foresight, and great communication.

If you enjoyed this article, please share it with others who might find it useful. Thank you!

8 Comments

  1. Nasir 5 years ago

    Points well covered. Going completely Agile with iterative development & prototyping with SCRUM based planning and execution is essential to avoiding unsuccessful launches.

    Rockstar Product Managers

    • Daniel Elizalde 5 years ago

      Thanks Nasir!

  2. Evan Bradley 7 years ago

    Solid article, but its missing something…and if it was 2010 I would have left it alone, but with today’s tools, I can’t. This article seems very enterprise, for example I totally agree with bring a Project Mgr, and I have had that discussion with execs about how tracking, administration and coordination can deplete time from ideation, design, and validation BUT in a smaller group that is focused on launching a new product a project manager is really unnecessary.

    Validation is actually what is missing from this article. There are several good books on lean product management that focus on the need to validate early and often, that is how you build products that are desirable. When you have “milestones” on the roadmap that are date oriented, when you have a launch date that is more than 6 months out…you may actually just be doing “scrumerfall”…While lean does not solve everything, validation can significantly lower risk when coupled with design thinking and using tools to get feedback from customers, or ultimately get a beta program going.

    More and more, I believe that great product managers should practice a melange of things to be close to the user: First off, practicing design and creating UX flows gives the ability to “speak the language of design to a designer,” at first the designer may tell you what sucks in your design, but over time they may tell you what is great with your design. Some great tools for design are Balsamiq and Sketch. Second, I would say you should use Intercom for gathering user insight and validating. Intercom gives you the ability to reach out to one or many of your customers to get feedback on experiences, walk them through onboarding, fix support issues or user issues (when users can’t figure out how to make the product work, which in turn gives you good feedback to fix broken UX). You can supplement Intercom using Survey Monkey to ask your users (or potential users) questions about the job they are hiring your product for (do a search on “Intercom job story” for more info on job story). You can also supplement Intercom with Invision which allows you to have clickable mock up flows that you can walk your users (or potential users) through.

    With these tools, you can talk to prospective customers about your product design and take steps to validate the “value” of it prior to developing…as well, you can launch MLP (minimum loveable product) quickly and get feedback on new features or existing features to help prioritize work. Most importantly you can validate assumptions about design, features, and technology efficiently and keep your developers working on the most important features/bugs as necessary. With every step in product ownership, validation should be involved because it takes out the risk of the product managers worst nightmare.

    • Daniel Elizalde 7 years ago

      Evan, thanks for such a thoughtful comment.
      I agree that validation is a key part of every development cycle. I really like your points there, even on using Intercom or similar tools to track user engagement and fit. I wrote the article thinking of the stage after validation. My assumption is that you already did validation and are now going into the heavy lifting of development.

      Not sure I agree with your comment about “this is valid in 2010” but not now. I think there’s an over-simplification of what Lean can do. I know that Lean and Agile advocate for smaller cycles of validation, and that in theory, smaller teams don’t need project managers. That is true in some cases, but it really depends on your industry and your product. In my experience, Lean is not very applicable to complex products that support critical infrastructure or where the results of failures might be life-threatening. For example, software for the electric grid infrastructure or the software that automates many of the functions of a jet fighter plane. It’s a lot harder to create a “minimum loveable product” in these scenarios, so Product Managers keep an eye on what is at stake and apply the best methodologies and tools to ensure success.

      Let me know what you think. This is a very interesting discussion that might deserve it’s own post =)

      Thanks for reading,
      Daniel

  3. Marc Chanliau 7 years ago

    In my experience as a Scrum PO, QA doesn’t develop unit tests, developers do. Developers also need to have integration tests to make the microservice they developed work well with other related microservices. QA needs to enter their tests in the form of scrum tasks that map to dev. scrum tasks. These tasks must tie into the scrum user stories written by the scrum product owner.

    • Daniel Elizalde 7 years ago

      March, thanks for your comment.
      I’ve seen this done both ways. I’ve worked with teams where developers write their own tests and also with teams where the QA Engineers are responsible for writing the tests. They both have their pros and cons so I can’t really say which one I prefer.

      -Daniel

  4. John Laws 7 years ago

    Outstanding article. Every line I read I felt like I’d been there before. Scary!

    Really appreciate your insight, spot on.

    Thanks!

    • Daniel Elizalde 7 years ago

      Thank you so much John. I’m glad the article resonated with you.

Leave a reply

Your email address will not be published.