This is the third and final part of a three-part blog series about:
- The U.S. Federal Government’s practice of implementing hybrid-Agile/”Agilish” software development projects
- The perils of Agilish projects
- How we can make Agilish projects more Agile
Table of Contents
“Agilish” software development projects, also known as “hybrid-Agile”, “Scrumfall”, or “Agilefall” projects, attempt to gain the speed and flexibility of Agile while enforcing traditional project management or “Waterfall” project controls. In Agilish projects, development teams engage in Agile “process mechanics” while other project stakeholders and contributors continue working under a Waterfall program construct. This mismatch in project approaches leads to what I call, “the worst of both worlds“:
- Application of predictive planning and project controls (e.g., Earned Value Management (EVM)) that sap the agility out of Agile while;
- Lacking detailed program baseline data necessary to measure project progress and apply project controls a la Waterfall
In part one of this blog series, I explained what Agilish projects are and why we should avoid them. In part two, I discussed the reasons why government Agile initiatives tend to move away from Agile principles and towards becoming primarily Waterfall projects with Agile practices added (Agilish). In this article, I discuss how to make Agilish projects more Agile when contractual and acquisition rules make an Agilish project management approach the only choice.
As I stated in the two previous articles, my perspective comes from years of working Agile projects for U.S. Federal Government clients. While many of the references are specific to that environment, equivalents exist at all levels of government, as well as in many private organizations, worldwide.
When Agilish is the Only Choice
In Agilish projects, software developers and other technical contributors are the only people “doing” Agile while everyone else continues working as before. U.S. Federal Government acquisitions and contracting culture make it practically impossible to “do Agile by the book.” The question is not whether federal software system projects will be Agilish. The question is to what degree can we make those projects approximate Agile?
We have two choices:
- “Tailor” Agile to fit within existing Waterfall-centric programmatic processes
- Change programmatic processes to be more Agile
The first choice is the default for most Agilish projects. It typically results in failed Agile implementations corrupted by traditional project management biases. The second choice is harder but more likely to yield success. It requires acquisitions, contracting, and project management professionals to understand how modern software development practices affect their work and to leverage recent Agile-friendly changes in federal acquisitions regulations and policies.
Break the “Iron Triangle”
Firm-fixed-price contracts can work with Agile as long as we allow scope to “float”. In other words, keep budget and event milestones fixed while closely managing project scope. Rather than attempting to define a large number of detailed requirements, define a much smaller set of coarse-grained requirements to flesh out, decompose, and prioritize during development. Some degree of requirements ambiguity is necessary to achieve the Agile-style flexibility. This is the idea behind managing a Product Backlog in Scrum.
Keeping the specter of “scope creep” at bay requires two things:
- Empowered Product Managers and Product Owners
- Adopting a Minimum Viable Product (MVP) approach to defining, developing, and fielding software systems
Formalize Product Management/Ownership
The concept of managing software systems as products is a foreign concept for government program and project managers. The U.S. Federal Government treats software systems development as a management activity for project managers to lead while Agile takes a product management approach to software development.
I detail the difference between project management and product management in another blog article. The main takeaway is that project management focuses on managing people, schedules, and activities while product management focuses on managing the product and its creation.
In the U.S. Federal Government’s acquisition model, program management offices (PMOs) manage contracts rather than software products/solutions. Requirements definition and solution approaches are left largely for vendors/contractors to figure out from initial sets of requirements developed by a different set of contractors. Developing solutions through active and ongoing engagement between the PMO, other government stakeholders, and vendors goes against how most federal acquisitions and contracting organizations are structured.
In Scrum, Product Owners are ultimately responsible for defining the solutions Scrum Teams develop. The Product Owner serves as the “Voice of the Customer”, synthesizing the various needs, wants, and perspectives of product stakeholders (both mission and technical) and customers into a coherent and cohesive product vision. Development teams depend on the Product Owner for direction and guidance concerning mission needs so they can define and build valuable software functionality.
This work of defining solutions cannot be outsourced wholesale to vendors in an Agile effort. Ascertaining mission needs and determining how solutions will address them are inherently governmental activities. PMOs must formalize the Product Owner role as part of how they do business. This is a move away from outsourcing the lead systems integrator (LSI) role to vendors and towards assuming a government-as-integrator role. Under this model, government collaborates with vendors as part of the team as opposed to assuming a distant customer stance.
Product Owners need to be government employees empowered to make decisions regarding solution scope and mission-facing functionality and be accountable for solution alignment with mission needs and requirements. Product ownership is a full-time job, not an extra duty for project managers. Government personnel who assume product management/owner positions must receive training, coaching, and support.
Adopt an MVP Approach
The only way to know if a solution is valuable is to put it in the hands of customers and end users. The key to doing so agilely is by releasing increments of valuable functionality (something people can use to get real work done) iteratively while ensuring the entire system delivers real mission value as it evolves. One way to accomplish this is to start with an MVP.
An MVP is a hypothesis of the minimum functionality required to provide mission value. The first MVP is an initial version of the solution. It is “viable” meaning that it is not a crude prototype but is instead a stable, releasable capability. MVPs go through a cyclical process of definition, development, release, usage monitoring, and adjudication.
MVP adjudication is a process step where we decide whether to add, modify, or remove functionality based on lessons learned from monitoring usage and performance data and from customer/end-user feedback. We may find that the current solution approach is not viable or will not satisfy requirements and decide to change the approach or abandon the MVP altogether; also referred to as pivoting away from the current solution path. Each iteration through the MVP cycle results in a new version of the MVP that delivers additional value. The MVP eventually matures into a solution that maximizes value.
The MVP cycle is an empirical process model that supports adaptive planning. The combination of empowered product ownership with an MVP-based development approach allows projects to control scope, while keeping within budget and schedule constraints and without engaging in heavyweight, agility-stifling, predictive planning.
Avoid EVM, Use MVPs Instead
Earned Value Management (EVM) is used to determine whether a project is on target to finish (i.e., deliver all planned functionality/scope), within a predetermined schedule and budget. The logic being that the sooner negative cost and schedule variances are discovered, the greater the likelihood the project can be managed back to plan. However, Agile is an empirical practice in which we monitor progress, make decisions, and take action based on what teams have released throughout the project. EVM is a definitive method in which project management predicts progress and works to prevent deviations from established plans. This fundamental lack of philosophical alignment between Agile and EVM prompts the question whether employing them together is reasonable and worthwhile.
An MVP-based solution development approach can help. MVPs break up the scope of large system implementations into smaller iterations. This allows programs to stay below mandatory EVM cost thresholds while lowering risk and providing the choice of stopping further development when mission needs are satisfied or when it is otherwise deemed necessary (e.g., bad contractor performance).
Focus on Shortening Program Lead and Cycle Times
Many organizational leaders think of Agile as a way to speed up software development, and nothing more. They obsess over adopting Scrum practices, DevOps pipelines, and Agile tracking tools because they think in terms of increasing development team efficiency and productivity rather than on continually improving program lead-time and cycle time.
Lead time is the time from when a customer orders a product to when they receive it. Cycle time is the time between gaining authorization to produce an order and completion of a deliverable product. We replace the word “product” with functionality, feature, capability, component, etc. when speaking about software development.
Focusing on Agile and DevOps practices and tools affects cycle time. Developers are able to crank out more code faster, but what happens before developers start coding a feature and after the feature is ready for deployment? How big a percentage of cycle time is tied up in user acceptance testing, security checks, approvals, and integration activities (things out of the control of software teams)? How much time do these activities add to cycle time and lead time?
In a true DevOps environment, the amount of time actually spent writing, building, and testing code is a small percentage of lead time. In government programs, lead times dwarf cycle times and time-consuming project or program-specific processes lengthen cycle times. Often, organizations narrowly focus on improving the shortest time span within program lead and cycle times while ignoring huge time sinks outside of the work performed by developers.
The actions organizations need to take to lower lead and cycle times are organization-specific. However, one recommendation applies to any organization sponsoring Agile software development projects: They should identify and manage dependencies between software teams and organizational, vendor, and customer stakeholders from day one.
In Scrum, the Product Owner works to integrate stakeholders in ongoing requirements and dependencies management. It may be advantageous to include stakeholder representatives in Scrum Teams if the development teams need their input on an ongoing basis to develop valuable and compliant functionality.
Conclusion
Getting the most agility out of Agilish projects begins understanding the rules of the chosen Agile approach and the statutory, policy, programmatic, procedural, and technological constraints within which the project must operate. Based on that understanding, some mix of tailoring the Agile approach and changing existing organizational processes is in order.
There is no “cookbook” on how to do this. Different organizations and projects will require different approaches and mistakes will be made along the way.
The important thing is to treat the shift towards Agile in an agile way. In other words, whenever possible, work in ways that align with the chosen Agile approach first. Only modify the Agile approach when laws, regulations, and policies absolutely require it. Work to become proficient in the Agile approach before attempting to change it to better suit existing ways of working.
This is guaranteed to bring stress and discomfort in the short term as people grapple with how they will accomplish their work under the new paradigm. However, as long as we treat changes as experiments from which to learn and improve and we implement them in an iterative and incremental manner, we will mitigate negative effects while progressively moving in the direction of greater agility, both from a technological and organizational perspective.
