Agilists juxtapose waterfall as the antithesis of agility. However, Eric Stolterman, Senior Executive Associate Dean at Indiana University, believes that crystalline processes such as waterfall and liquid processes such as Scrum should simultaneously co-exist. The same problem can be viewed from the lenses of both crystalline or liquid processes, “similar to the way that light can be understood scientifically as having attributes of either practices or waves depending on which of the two different points of view is used” (Nelson & Stolterman, 2014).
Mark Schiffman, Senior IT Project Manager at KSM Consulting, agrees. He describes an ideal data science project management approach as something “like Scrum with a waterfall wrapper around it”. He suggests that the waterfall view is better for customer-focused aspects such as requirements gathering to “keep them one step ahead of development” and that the Scrum view is better for development (Schiffman, 2017). Carol Choksy, Associate Chair of Information and Library Science at Indiana University, similarly believes that data projects require more intense upfront planning that is best handled using the traditional planning principles from the PMI Project Management Book of Knowledge. Once, the initial requirements are scoped, then agile is appropriate (Choksy, 2017).
Hybrid approaches could also facilitate rapid development without having to hit extensive validation requirements. Donald Jennings, Manager of Data Integration at Eli Lilly, reports that they use a hybrid approach with two-week sprints for development. However, these time periods are not sufficient to prove their results within a high degree of certainty required by Eli Lilly’s quality assurance protocols and by Food and Drug Administration requirements. Therefore, a series of a few development sprints might be followed by months of a waterfall-style testing process that features a series of sign-offs (Jennings, 2017).
Evaluation
Although it may be tempting to pick and choose the best aspects from various models, such hybrid approaches have been criticized for failing to provide neither the structural benefits of waterfall nor the flexibility benefits of agile (Sutherland, 2014). Agilists label such approaches with pejorative terms such as “wagile”, “fragile” or “scrumfall” to discredit their use.
Regardless, bimodal approaches could be useful for project management functions to look at a project from different lenses. They might also be appropriate when specific constraints like regulation or organizational policies override development process freedom.
Learn More
Learning how to manage data science projects is broad topic. For an initial start, look into:
- What is the Data Science Process?
- What is a Data Science Life Cycle?
- The Data Science Methodologies Guide
