Kanban, which literally means billboard in Japanese, started as a supply chain and inventory control system for Toyota manufacturing in the 1940s to minimize work in progress and to match the supply of automotive parts with demand.
Kanban is simple and at its core focuses on two principles:
- visualize workflow and
- reduce work in progress.
Visualize Workflow
Kanban starts with a list of potential features or tasks, similar to the backlog concept of Scrum, that are placed in the initial To Do column of a Kanban board, which is a visual representation of workflow. In a simple three-column (or three-bin) Kanban board (see image at the top of the page), when the team decides to start working on the task, the Kanban card (sticky note) is moved from the To Do to the Doing column. When the team completes its task, it is moved to the Done column.
Kanban boards often have additional columns. For example, software teams might split Doing into In Development and Testing. Some teams might further split Testing into Verification and Validation.
Minimize Work In Progress
Uncompleted work, or work in progress or simply WIP, is an investment whose value has yet to be realized. To reduce WIP, Kanban teams set WIP Limits that define the maximum number of tasks that can simultaneously exist in given columns.
For example, if a team places a WIP limit of three in Doing in the above Kanban, then they are only allowed to simultaneously work on three items. If they want to bring in a fourth item from To Do into Doing, they would first have to complete one of the tasks in Doing to Done to free up space for the new item.
Kanban Popularity
Although it started in manufacturing, other industries including software have since adopted Kanban.
More recently, data science teams have also adopted it. In our 2020 survey conducted on this site, Kanban was the third most popular response (12%) to the question: “Which process do you most commonly use for data science projects?”
Part of Kanban’s popularity stems from its simplicity and strong alignment with Agile practices.
Kanban vs Scrum
Kanban is less prescriptive than Scrum. Roles, meetings, and time boxes are not defined in Kanban. Scrum teams often use Kanban as a secondary approach to manage workflow during the sprint, although often without WIP limits.
Is Kanban for Data Science?
Kanban can be effective for data science. Its fluid and less rigorous processes provide data scientists with greater flexibility to execute their work without having to hit constant deadlines.
Like other agile approaches, work is divided into small increments, which allows for rapid iterations and continuous delivery. Kanban provides some structure that is more than what a lot of data science teams currently have (Saltz, Shamshurin & Crowston, 2017).
Benefits
- Highly Visual: The highly visual and simple nature of Kanban boards make them very effective at quickly communicating work in progress for team members and stakeholders (Brechner, 2015).
- Very Flexible: By pulling in work items one at a time as opposed to Scrum’s batch cycle approach, Kanban provides teams with greater flexibility to shift gears (Rigby, Sutherland, & Takeuchi, 2016). Conflicts about not being able to complete items by the sprint deadline are avoided.
- Lightweight and Adaptable: By not prescribing time boxes, roles, and meetings (Rigby, Sutherland, & Takeuchi, 2016), the overhead requirements to manage Kanban is significantly less than both waterfall and scrum. Kanban provides teams with the freedom to adopt their own additional processes.
- Avoids Culture Clash: As a very simple system that does not redefine team roles, Kanban is met with less cultural and organizational resistance than Scrum (Rigby, Sutherland, & Takeuchi, 2016). Managers don’t necessarily need to cede authority and data scientists generally don’t need to forecast what they can accomplish within a rigid time boundary. Kanban is not an invasive process; rather, teams can adopt it seamlessly without an awkward shift to a radically different system.
- Better Coordination: The simplicity, visual nature, lightweight, and flexible structure without stressful deadlines might make Kanban more conducive to teamwork than other approaches. Student teams that used Kanban in a controlled experiment were more likely to report that they worked well together as a team than under any under approach (Saltz, Shamshurin, & Crowston, 2017).
- Minimize Work in Progress: The WIP limits can increase total throughput and reduce investments in uncompleted work because they prevent too much WIP from piling up in a given process (Brechner, 2015).
Weaknesses and Challenges
However, as a collaboration framework for coordinating tasks, Kanban lacks the necessary rigor for a comprehensive project management approach and generally needs augmentation from other approaches.
- Customer Interaction not Defined: As an inward-facing process, Kanban does not directly prescribe outward-facing processes for rich and frequent customer feedback loops. Customers may not feel as committed to the process without the structured cadence of sprint reviews (Akred, 2015).
- Lack of Deadlines: Relative to Scrum, without the motivation to hit the constantly looming deadlines, teams may work on certain tasks for an excessive length of time. A higher level of team discipline is needed to ensure that tasks do not slide longer than needed.
- Kanban Column Definition: How do you define the columns for a data science Kanban board? There doesn’t seem to be a good answer. To set up a data science-specific board, you would either need to use a generic board or attempt to create a board that encompasses all steps of a data science process. The former is of limited value because it is missing information, but the latter option is cumbersome because steps in data science vary drastically based on the problem at hand (Saltz, Shamshurin & Crowston, 2017).
- No added value: One data scientist we interviewed found Kanban to be counter-productive. The engineering department at his former company used an electronic Kanban tracker but he “refused to use it [because] it adds an unnecessary managerial burden that doesn’t really result in any improved outcomes…I tried it but it didn’t change how I work.” (Foxworthy, 2017)
Recommendations
Kanban is only part of the solution for a data science project management approach. Teams that use Kanban need other processes in place that encourage effective customer interaction. However, Kanban can be effective for several types of teams.
Kanban is ideal for teams…
- With low process maturity who want to transition into agile or want at least some structure
- That use Scrum but are consistently not completing their sprints or find the meeting and cadence requirements overly burdensome
- That don’t need the rigor of Scrum in order to be accountable
- That are looking to augment their work processes and focus further on collaboration, continuous improvement, and minimizing work in progress
- In “process-oriented cultures that prefer evolutionary improvements with overarching values but no prescribed practices” (Rigby, Sutherland, & Takeuchi, 2016)
Learn More
Related Posts:
- What is the Data Science Process? – How to combine coordination frameworks (like Kanban) with workflows (like CRISP-DM)
- Agile Data Science – What it means to be agile in data science
- Scrum for Data Science – The ins and outs of leveraging Scrum in a data science context
- Data Driven Scrum – A new agile framework specifically designed for data science.
LinkedIn Post: Ken Johnston has a good three-post series on LinkedIn describing his journey with data science and kanban.
