Are you a new data science manager? Or a data scientist who wants to become a data science manager? Or, perhaps, a manager who wants to start managing data science teams?
If so, welcome to the world of data science management!
In short, data science projects often have a high degree of uncertainty with respect to time, budget and people, as well as risks that are difficult to mitigate. This makes being a data science manager especially challenging.
Related: 5 Tips to Manage Data Science Projects
So, below are specific actions that can help a data science manager and improve the team’s satisfaction as well as their work output.
Before we get into the the details below, if you are curious about how to hire a data science manager, or potential questions to ask (or be asked) during a data science manager interview, check out this related post on Tips for Hiring a Data Science Manager.
Who is a Typical Data Science Manager?
Before exploring how to become a better data science manager, let’s look at three different job postings. All three of these are from well-known organizations (consulting and tech firms). By exploring these qualifications, we can get a sense of the skills that are often brought to the job (as well as the skills that might be lacking).
Company A required qualifications:
o 3+ years of experience managing a large team of Data Scientists o A passion for selflessly leading a team o A desire to help each member reach their career objectives o An eagerness to share your knowledge with your colleagues o 5+ years of experience with R or Python o 5+ years of experience with predictive modeling o Understanding of relational data structures and SQL
This is the ideal set of qualifications. However, most people do not have experience managing data science teams as well as being an experienced data scientist. This means that this job will be hard to fill, and typically, the company will need to reduce the qualifications.
Company B required qualifications:
o 5+ years of experience doing statistical modeling and machine learning o 5+ years of developing machine learning methods o 5+ years of experience with data science tools, including Python, R, or Scala o Strong quantitative and analytical skills
Company B has given up trying to find the “unicorn data science manager”, and rather, focused on the technical expertise. The job description didn’t even mention managing people, managing projects or even experience working within a data science team. Unfortunately, being a great data scientist does not ensure that one will be a great data science manager! The responsibilities (and hence skills) are very different. For example, to be an effective data science manager, one needs to focus on people, budgets and stakeholder management, but these are not typically the focus of a data scientist. On the other hand, being a former data scientist can provide insights (and empathy) that other managers, without direct hands-on experience (i.e., general managers or software development managers) will struggle to get.
Company C required qualifications:
o 5+ years managing or mentoring data scientists or software developers o 5+ years of experience communicating project results to leadership teams o Experience initiating and completing projects with minimal guidance o Ability to work in a highly collaborative environment o Deep understanding of and empathy for end-users o Excellent written and oral communication skills
Clearly, Company C took the opposite approach, as compared to Company B, in that Company C valued technical management experience, but was not looking for experience / knowledge in actually doing data science. Hence, the person hired by Company C will be a good “people manager” but might struggle to understand “why data science is different”.
What does this suggest about data science managers?
The key takeaway from this is that many new (and current) data science managers could improve their effectiveness with training that is specific to data science management.
This has been reinforced in Harvard Business Review’s Managing a Data Science Team and Sequoia Capital’s discussion of the Role of a Data Science Manager, in that they note that being a data science manager can be a challenge.
6 Actions for a Data Science Manager
1) Focus on Impact
A data science manager needs to help keep the team focused on delivering impact (which is different from interesting insights or even actionable insights). One key aspect to achieving this impact is by having the team use a process framework that facilitates / encourages / requires active communication with stakeholders throughout the project. Another aspect is to define appropriate metrics, both for the team members and the specific project.
2) Manage Client Expectations
Requirements gathering can be a challenge for a data science project and is very different than requirements gathering for software development projects. For example, a client might think they have specified the requirements by stating “find value in my data”. Furthermore, stakeholders and clients often want well-defined deadlines and budgets. However, budgets and delivery dates are often a challenge when conducting exploratory data science efforts.
Hence, a manager needs to provide the correct context for any target deadlines and budgets. Furthermore, the team should collectively focus on understanding what data is available and how potential insights could be used within the organization.
As one can see, this is another area where having a well-defined team collaboration process can help the data science manager work with stakeholders. For example, they can enable stakeholders to understand the project’s status and help prioritize future work within the project as well as understanding the uncertainties within the project and what the team is doing to reduce uncertainty as the project progresses).
3) Manage Team Member Expectations
Many data scientists want to focus on building the best machine learning model possible. But, oftentimes, much of the work during a project will focus on data munging or some other task not directly related to model building. Furthermore, sometimes a “good” model is good enough (the impact of a better model would not be material to the organization). Setting appropriate expectations, along with defining multiple roles, where a person might have different roles on different projects, can help with these challenges.
4) Protect Your Team
Oftentimes, teams have to deal with projects that have vague requirements and/or unrealistic timelines (see above for setting appropriate client expectations). This naturally causes team burnout, dissatisfaction, and in general, a desire for team members to look for new teams where these challenges do not exist. To get the most from a data science team’s time, each team member needs to have a clear understanding of what is the business goal behind the project (see the previous focus on impact).
Hence, a data science manager needs to define a process framework that minimizes the chance of this occurring. If these challenges (requirements and/or unrealistic timelines) do start to exist, the manager needs to get the team to collectively identify how the team’s process broke and how to improve the process going forward (for this project as well as future projects).
5) Develop Your Team
Data Science managers need to ensure that the organization is investing in developing the team members, so that they can operate at increasingly higher levels of problem-solving as well as increasing their overall job satisfaction. However, sometimes, especially with new managers, things like individualized development plans are given a low priority (or are just not done). Don’t short-change your team – invest in your team and the organization will see the returns.
6) Help the Team Prioritize
Most data science teams have more than enough work, and hence, prioritization is essential – both to ensure the data science team is working on the tasks with the highest potential value, but also to ensure that clients / stakeholders are efficiently using their “data science dollars”.
Hence, the team should be using a framework that provides structure to how the team prioritizes their potential work. This prioritization is with respect to different projects as well as prioritizing what might be the most appropriate next step in the current project. In other words, a data science manager should not be helping each team member prioritize their tasks in a vacuum, but rather, leverage a framework that engages the clients / stakeholders / product owners. While prioritization needs to be done in conjunction with stakeholders / product owners, the lack of an effective prioritization will most negatively impact the data science team members.
Learn More
This post is part of the Team Management series which includes posts where you can:
- Explore How to Lead Data Science Teams
- Learn about the 8 Key Roles for Data Science Team
- Understand the difference between Data Science and Software Engineering
- Assess 10 Ethical Questions for data science
- Know Why you (probably) Need a Product Manager
- Explore how to apply CRISP-DM for Teams
- Get 5 Tips for Remote Data Science Teams
- Know the pros and cons of Centralized vs Decentralized Teams
