Managing AI projects: 6 Concepts to Help Lead an AI Team

If you are curious on how an AI project relates to a machine learning or data science project, take a look at this post.

Otherwise, read below for the 6 key concepts to help lead an AI project.

1: To effectively manage AI projects, know how AI insights will be leveraged

While it might seem obvious to know the problem to be solved, the data that might be useful to build a predictive model, and how that model will be used within the organization, this is actually an area where teams often fail! In fact, I have seen many teams jump to talking about using machine learning to build a specific model, with specific attributes. Or other teams build models that were not useful since, for example, the business already knew the insight generated via the model.

Taking the time to step back and understand the real business / organizational challenge that could be improved via an AI / Machine learning solution is an important aspect not to be overlooked. Ensure the team has this context will enable the team to appropriately brainstorm and prioritize the full range of tasks (e.g., what data might be useful, what to predict, how to analyze if an AI predictive model is useful). It is equally important to explore what data might be used, and how the team might have access to that data.

2: Know the conceptual architecture for the AI project

In building an AI solution, it is helpful to think about the system as three key interrelated components. Just as with software systems, there is a front-end component (e.g., a user interface), and a back-end component (e.g., store and access data). However, AI systems also have an ML component (e.g., generate and use predictive models).  

For example, a recommendation system, such as those used at Amazon or Netflix, has a back-end component that keeps track of different users (e.g., previous purchases) and a front-end component that shows the user interface (e.g., movies that you might want to watch). The ML component is what generates the movie suggestions.

For a “normal” software system (i.e., not an AI system), we might just show previous movies that the user has watched or the most popular shows. This type of information would be passed from the back-end to the front-end user interface. Yet, the predictions (e.g., what the person might want to watch) are where machine learning algorithms are important!

Of course, some AI systems are embedded into real-time systems (e.g., self-driving cars), and in this situation, the architecture is a bit different. In the self-driving car example, there are components that collect data (e.g., ‘sensors’ to collect information), that are then integrated with front-end systems (e.g., where does the person want the car to go), and back-end system (e.g., most recent trips). But just as with the amazon example, the key component of this AI system is the Machine Learning (ML) component. In fact, sometimes there are multiple ML components (e.g., one for avoiding hitting objects, one for following the road, etc).

3: Know the life cycle you will use for AI project management and execution

While there are many resources that discuss the SDLC (software development life cycle), fewer resources are available to help you understand the life cycle required to build a machine learning predictive model. At a high level, the team will need to iterate through the following steps:

• Understand the business problem and the that might be data available
• Clean and “munge” the data
• Use Machine Learning to build a predictive model
• Deploy the model
• Observe and analyze how the model performs

Note that, at any point in this life cycle, teams might need to loop back to the previous step. For example, while building a model, it might become clear that additional data is required for the model to be useful.

To read more on life cycles, explore our post on data science life cycles or machine learning life cycles.

4: Know how to coordinate within and across your IT and AI project Teams

While knowing the steps to build a predictive model are useful, there needs to be a process to coordinate efforts across the team (e.g., how to prioritize different potential tasks), and within an AI/Data science effort (e.g., when to loop back to a previous phase of the project).

IT teams typically use Scrum, a team coordination framework defined in the 1990s, that is now used by more than 16M developers.

However, as we have noted, Scrum is often not appropriate for Machine Learning projects. So, something such as Data Driven Scrum, might be more appropriate.

Both the Scrum and Data Driven Scrum frameworks define how the team can work in an agile fashion – with short iterations of work, and then after each iteration, the team meets to learn from that iteration, brainstorm next steps, and prioritize future potential work. Achieving short iterations, and then analyzing the results of each iteration to inform future potential iterations is a way to ‘fail early’ on some ideas, and in general, increase focus on promising ideas.

5: Know when / how to scale the solution

One also needs to understand when to scale (and the steps to scale) a solution.

In other words, most of the time, it makes sense to start small (e.g., a proof of concept) and then incrementally scale the solution in future iterations (rather than directly deploying the system at scale). To achieve this incremental scaling, the data science / ML team should not be ‘throwing the code over the wall’ to an IT DevOps team. The data science team needs to work with the devOps team (sometimes known as MLOps). Thinking through how the group will collectively do “machine learning operational support” is something that should be explored at the start of the project and refined as the project scales in its usage.

In addition, this incremental approach is helpful since it is often not clear how much value the system will deliver (i.e., it needs to be evaluated), and deploying a system at scale can be expensive (as well as risky, since it might not predict as well as hoped).

6. Explore possible bias in the model via active AI project management

Using a training dataset that is not fully representative of the population of where the model will be used can lead to bias in the model. For example, this bias might be due to not getting the full range of applications. While eliminating all bias is difficult, the collective team should explore where bias might be introduced, and how to mitigate any potential bias.

Note that we often think of bias as being something where a segment of the population is disadvantaged (e.g., google ‘apple credit card gender bias’ to find many articles, such as this one from Wired.com). Clearly, this type of bias is problematic. But other forms of bias might impact the accuracy of the model without adversely impacting a segment of the population (e.g., bias in how the team collects data).

To explore other possible ethical conundrums, check out this post on 10 data science ethics questions.

Summary

These 6 key concepts will help improve the chance that you can deliver a useful AI project. In short, you should make sure that the team:

• Understands the problem
• Understands how Machine Learning is integrated with the rest of the AI solution
• Understands the ML life cycle
• Works together to appropriately prioritize short iterations
• Collectively works to identify and mitigate any potential bias
• Incrementally scales the usage of the model (and also thinking through the increasing demands of operational support of the system)

To learn more, explore training via our AI project management course, which more deeply explores how to effectively lead data science teams and build AI systems.

Or jump in and learn more through these articles:

•  Data Science Workflows
•  Coordination Frameworks
•  Agile Data Science