Uncategorized

10 Reasons Why You (Probably) Need a Data Science Product Manager

Data Science Product Manager

The Product Manager Role

Evolving from the consumer package goods brand manager role, the software product role has become common for software teams (mindtheproduct.com). Wearing many different hats, this product person researches market needs, defines how to map the solution to the problem space, and sets the product vision.

Product people who tend to be more inwardly focused on the product development team or work on a Scrum team often carry the title of “product owner” while those more market-facing tend to be known as the “product manager.” Many organizations have both who work together on a product. For simplicity, I’ll just refer to the general role as “product manager”.

Does Data Science Need Product Managers?

As Jeff mentioned in his previous post on team roles, the product role is often appropriate for many data science projects. Yet, in some circumstances, one may not be needed. For example:

  • Rigid contract arrangements where the customer is paying you to deliver a specific solution
  • Smaller, less complex, or low impact projects
  • Research projects where a team is trying to prove out that a certain problem could be modeled
  • When the end deliverable is one-time analysis or otherwise not operationalized

However, as the data science function matures, teams are more frequently delivering operationalized solutions where a product manager is needed to own the product and its development. I can speak to this from experience…

A Tale of Two Projects

A project without a product manager

I worked on a project with clearly defined requirements to deliver a daily-generated forecast file and a user-facing dashboard. The team consisted of two data scientists, one data engineer, one Tableau developer, two subject matter experts, and me (the project manager).  None of us truly researched the underlying business problem but proceeded to follow (somewhat blindly) the pre-defined requirements with some guidance from the subject matter experts. Sound familiar to a project you might have worked on? If so, you won’t be surprised to learn that we missed a key unstated requirement that rendered the final solution useless.

A replacement project with a product manager

So, we opened another project. This time – learning from the past mistake – I first developed a deep understanding of the fundamental business problem with the main stakeholder and his analyst team, identified the root issues, and then proposed some initial product features to fill both the stated and implied needs. In tandem, I worked with my data science development team to design a dynamic querying web app that generates forecasts on demand based on the user’s filter selections. I had weekly conversations with the business team to iteratively define features and demo work completed. After months, my team delivered the initial product, and today we continue to release features about every month. By stepping in to research and understand the business needs and by defining and owning the product roadmap, I avoided the pitfalls of the first project to ensure this second one was a success.

10 Reasons Why You Probably Need a Product Manager

There are many reasons why a product manager is becoming more important for data science teams. Here’s ten:

The customer doesn’t understand what it needs

In the first project of the “Tale of Two Projects”, the business stakeholders defined requirements for a dashboard sitting on top of a forecast file when they really needed a dynamic querying and forecasting tool. Such problem-solution mismatch scenarios happen often because stakeholders tend to have only a partial view of data science (often involving a Tableau dashboard or some other tool they’re familiar with). Without a product manager to dig deeper and investigate what is truly needed, the ill-defined requirements might pass right on to the data scientist team who waste their and the stakeholders’ time building something that no one needs.

The customer expects a data science solution when it’s not feasible

Similarly, customers might think that a problem is solve-able with machine learning when for some reason (e.g. lack of data, legal/ethics issues) it might not be. Or if ML can solve the problem, often a simpler analytics solution is all that is needed. This is where the product manager can step in and set expectations that perhaps the data science team should first explore but not commit to the requested solution.

The customer can’t identify good use cases

On the flip side, the business probably cannot identify the best use cases for machine learning and artificial intelligence. Someone who understands data science and can work with the stakeholders and data scientists to design a solution is much better equipped to identify use cases and deliver a data science product.

The customer lacks the time and the skillset

Managing a size-able product can be a full-time job. If it is a side job for a business stakeholder (after all they have a business to run), the data science team may not get enough feedback or direction. Likewise, product management is a profession that requires a broad range of skills that the customer might lack.

The customer doesn’t know how to use the results

The nuances of model interpretation are often not understood by the customer, and data scientists might not be able to effectively set expectations or explain why the most “accurate” model might not be the best. A product manager who understands data science can better facilitate communication among the data scientists and stakeholders and help the business understand and use the results.

The data scientists don’t understand the business need

A common flip-side of the above scenario is that the data scientists don’t understand the business need for the product. By translating the business needs into a language the data scientists can understand and by explaining the “why” behind a product, the product manager focuses the team on value-delivery.

No one is balancing competing demands

A common fallacy is to treat the project sponsor as the only stakeholder. However, any sufficiently sized project has multiple stakeholders, each with competing demands. A good product manager understands these various competing demands, prioritizes development on the most important needs, and aligns the product with the broader organizational strategy.

Mis-timed launches can cause problems

Kind of a sub-point of the above…If you wait for the best possible results, you’re probably delaying value delivery and stifling the product feedback loop. But if you deliver too early, you might adversely impact the business. Someone needs to make this call and it’s probably not the data scientists (who might want to perfect their work) nor the stakeholders (who might insist on a hard deadline for delivery that’s often too soon). Yet, an effective product manager can often guide an optimal timeline to deliver a series of small or low-accurate solutions first (perhaps for A / B testing) before the wider market launch.

Models need to be managed post-launch

Unlike traditional software that doesn’t need to be re-trained, data science products may often deviate from desired behavior over time. Someone needs to step in and manage the full product life cycle. This is generally recognized in software and is even more critical in data science.

Maybe you should be developing a product suite

A data science team is often called on to solve a specific problem for a specific department. Sometimes this is what is needed. However, often the same work that is going into one data science product might also have another use case for a different (or even the same) department. Take for example, a customer churn model. Yes, the retention department might request this, but this same model can likewise serve product development, marketing, sales, and other teams. A good product manager looks across organizational and market needs and incorporates these broader needs into the product roadmap.

Conclusion

In short, a product manager is not always needed, particularly for smaller, non-stakeholder focused projects that don’t deliver operationalized products. But in cases like my “Tale of Two Projects”, I realized that a product manager can make the difference between failure and success. Microsoft likewise realized the critical need for product management when it hired Kurl DelBene to help Microsoft build its”AI factory” (hbr.org). As data science continues to mature and become ever more integrated with operationalized systems, the role of data science product manager is becoming more critical.

Leave a Reply

Your email address will not be published. Required fields are marked *