Agile AI

As AI practitioners
We would like a blog post exploring some underlying concepts and practical tips of Agile AI
So that we can more effectively drive results with our AI initiatives

Agile is a flexible approach for managing initiatives in ever-changing environments. Acknowledging the complexities of AI projects, many organizations take an Agile approach to their AI initiatives. But results are mixed. Often this is because practitioners often misunderstand the underlying concepts of Agile and AI. To help unravel this, we’ll walk through:

• What is Agile?
• What is AI?
• 4 ways not to deliver AI projects
• 7 Agile AI best practices

What is Agile?

“It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is the most adaptable to change.”

– Charles Darwin

Contrary to common misconceptions, Agile is not about doing things faster. Rather Agile strives to deliver sooner, solicit feedback, learn, and adapt. To understand this better, let’s look at the animal kingdom.

Cheetahs are the fastest land animals. Their evolutionary strength lies in their ability to run faster than any other animal. However, they can only sustain this advantage by sprinting in more or less one direction. They are not the appropriate comparison animal.

Rather, gazelles are agile – They evolved to observe their surroundings and to quickly prance in different directions that make it difficult for predators to pounce on them.

Coming back to the way of work, Agile practitioners strive to be gazelles. Agilists continually observe their surroundings and adjust each next step based on what they learn. Their evolutionary strength in the business world rests on their ability to quickly observe their surroundings, learn from them, and quickly adjust plans accordingly.

What is Artificial Intelligence (AI)?

Artificial intelligence simulates human-like cognitive processes using computer systems. It is a multi-disciplinary field that encompasses data science, data engineering, software engineering, and human computer interaction. Each AI project also intersects a domain-specific field such as marketing, finance, or medicine.

AI operates through diverse methodologies with machine learning being a cornerstone. These machine learning algorithms allow computers to learn from examples and improve their performance over time without being explicitly programmed.

AI projects flow through an AI life cycle that includes 1) problem definition, 2) data acquisition and preparation, 3) model development and training, 4) model evaluation and refinement, 5) deployment, and 6) machine learning operations.

With advances in algorithms, the proliferation of data, and the ever-decreasing cost of compute, AI is emerging as one of the most promising fields to revolutionize nearly every industry.

4 ways not to deliver AI projects

Despite this progress, most AI initiatives fail. One of the biggest challenges to AI is that many organizations struggle to understand and implement effective project management practices that build and improve AI systems. Typically these failed practices fall into one of at least four patterns.

1.      Being ad hoc with AI projects

Some organizations don’t have a repeatable approach to their AI projects. Rather, they just “wing it”. This might be appropriate for low-risk and straight-forward one-off projects. However, AI projects are rarely straight-forward. Moreover, to realize significant value of AI investments, organizations should build tools and processes that drive sustainable growth in AI. Thus, ad hoc AI usually doesn’t work. And if it does for a one-off project, it won’t scale across multiple projects.

2.      Treating AI projects as pure research projects

Many data science professionals have advanced academic degrees. Thus, it’s no surprise that many data scientists attempt to apply an academic research process in their jobs after school. This might be partially appropriate in some environments such as research arms of large tech firms. But this approach doesn’t fully fit the reality of real-world projects. Specifically:

• Most AI projects are delivered by a diverse team – not an individual or small set of researchers.
• Real-world projects face many environmental changes. Ignoring these likely means that your project is set to deliver something that perhaps made sense yesterday but not for tomorrow.
• Many academic projects deliver something at the end of the project (like a publication). Delivering meaningful interim results may not be important. However, in real-world projects, If you don’t show meaningful value at least within a few months, your project likely won’t survive budget allocations.
• Most real-world projects need to build infrastructure so that the system can operate even after your project closes. This is often beyond the scope of research projects.
• Academic training enforces detailed documentation. This is too cumbersome for most real-world projects.

Thus, academic research training teaches many great qualities that professionals benefit from. However, adhering to a rigorous academic research methodology likely won’t yield results in real-world projects.

3.      Overplanning AI projects

Given the complexity of AI projects, you might think that you need to set a plan upfront and execute the plan. This approach (known as waterfall) works well if your project has known inputs, known outputs, and the overall project environment remains stable.

Such predictability makes sense for building a bridge. It’s wishful thinking for AI.

As soon as your project takes off, you might feel like everything changes. Take just a few common examples:

• You think you’ll be able to build an effective model but every model you try doesn’t fit.
• You are able to build a model but it takes several months instead of the expected handful of days.
• The model you build works great on training data but drifts significantly once it is live.
• The model works using top-end hardware. But this is costly and you have to figure out how to deliver a functional model based on more cost-effective hardware.
• Your compliance department or a regulatory agency restricts you from using a key data set due to privacy concerns.
• A key project member (like the lead data scientist or project champion) leaves the organization.
• Your stakeholders requested to solve problem X. In reality, they need problem Y solved first.

No one will reward you if you perfectly follow a plan that winds up leading you off a cliff. Rather, you must be flexible in your planning so that you can adjust for the inevitable changes along the way.

4.      Confusing AI projects as software projects

A fourth common failure pattern for AI projects is to treat them like software projects. It’s all just code at the end of the day, right?

Indeed, AI projects and software projects are like cousins. They share a lot of similarities. But their differences are substantial. Don’t treat them like identical twins. Specifically, AI projects tend to:

• Have fuzzier objectives
• Focus more heavily on data collection, data cleaning, and data governance
• Require more experimentation
• Be more difficult to know whether the output is functioning
• Face trickier ethics issues

Taking an approach built for software and applying it to AI will likely frustrate your team. Search online and you’ll find many data scientists complaining about this. The most common area where Agile and AI butt heads is when teams blindly apply Scrum (the most common Agile approach) to AI.

While you might be able to effectively apply Scrum to AI, you shouldn’t rigidly apply a software implementation of Scrum. Specifically, avoid many of the common software practices like a software-specific definition of done or writing user stories that overly specify functionality.

In short, take the relevant practices from software when they make sense. But don’t confuse AI and software as the same.

---

Don’t Miss Out on the Latest

---

7 Agile AI Best Practices

So what does make sense?

For best results, manage a project in a way that respects the nuances of AI projects and allows you to quickly learn and adjust course. In other words, take an Agile AI approach. To effectively deploy and manage Agile AI, use these seven best practices.

1. Strive toward foundational principles first

Contrary to another misconception, Agile is not about adhering to a set process. Rather, it is a philosophy that empowers teams to self-manage, deliver often, to learn, and to adapt. Teams should first identify the principles they aspire toward. The Agile Manifesto has 4 initial statements and 12 principles that serve as a great foundation that you can use or build off of.

2. Then layer a framework on top

Unfortunately, many teams fail in their Agile implementations by just trying to implement a framework without a strong Why or understanding behind it. These are typically the teams that apply a software approach to AI as mentioned above.

Don’t be one of these teams that just does an Agile framework for the sake of “doing Agile”. Rather, build a framework on top of the Agile principles. Common options include Scrum, kanban, Data Driven Scrum, and building a custom framework that works for your specific team. Simply doing one of these won’t make you Agile. But rather view these as approaches that can help on your journey toward being Agile.

3. Collaborate across functions

Each AI project spans multiple disciplines. To effectively implement Agile, build diverse AI teams with members who collectively have a holistic understanding of both the technical and business context. See the guide to data science roles to learn more.

4. Build a Minimum Viable AI (MVAI)

To reduce complexity, accelerate development, and to learn early in the project, a team should strive to deliver a scaled-down but valuable AI solution as early as possible. Breaking this down, an MVAI (also known as a Data Science MVP) is:

• Minimal – so that the team can deliver a solution quickly
• Valuable – so that stakeholders have a core need addressed
• Artificial Intelligence – mimics human intelligence

As explained in the next section, your MVAI does not have to be based on machine learning.

5. Incrementally improve your model

Your first working model probably isn’t great. In most circumstances, that’s OK. If you try to engineer the “best model possible”, then you’ll take too long to deliver something. Rather, embrace the Agile principle of continuous improvement by starting with a basic solution.

Think creatively. Some teams deliver a basic ML model. Others start with a heuristic (business rules) solution. And some might even start with a human behind the scenes (the Wizard of Oz approach) who delivers answers. Deploy this initial solution, collect real-world data, and refine your approach based on user interactions. This approach leads to more accurate and effective AI implementations over time.

6. Measure your progress

Agile AI is a journey. And it’s challenging to understand where you are on this journey unless you effectively measure your performance. This includes many of the model performance metrics that data scientists might jump to such as recall, precision, or RSME. But be comprehensive and look at business KPIs, stakeholder and team happiness, and process metrics like cycle times. Having a rich set of data science project metrics will provide a deeper understanding of where your project currently is and how you are progressing. See 10 data science metrics to dive deeper.

7. Learn quickly and effectively

Learning is a foundational thread that weaves through Agility, Artificial Intelligence, and professional development. Specifically, Agile focuses on setting up practices to help the team learn and react quickly. Likewise, Artificial Intelligence is built on top of computer systems that can learn from experience (machine learning). And to help establish yourself as an effective practitioner in these realms, you should likewise focus on learning. Some great resources include:

• Scrum in AI – Paolo Sammicheli’s book explores many of the concepts in this post and specifically emphasizes the use of Scrum in AI initiatives.
• Agile AI – This O’reilly book covers the processes and tools that AI teams use to help them become more Agile.
• The Data Science Project Management Blog is the most comprehensive set of knowledge on how to manage data science projects. See the posts tagged as AI.
• The Data Science Team Lead course empowers professionals to lead data science projects.