NFRs are part of Agile projects too. Here’s how to handle them.
In this video, Frank Velazquez explains Non-Functional Requirements (NFRs) in the context of requirements management and how to address them in Agile projects.
It is quite common for people who sponsor, manage, and develop software systems to misunderstand NFRs and struggle in applying them to Agile requirements management. User Stories make mapping of system-level NFRs to functional requirements easier. Acceptance criteria link functional requirements to tests that demonstrate satisfaction of NFRs. Coupled with frequent integrations and regression testing, this approach to NFRs ensures the delivery of expected system behavior to customers and end users.
If you are managing software development efforts as projects, you are doing it wrong. Instead of managing projects, organizations that sponsor and develop software solutions need to shift towards managing solutions as products. A product management approach to software development aligns with modern software development and with Agile’s inherent product management bias.
Project Management vs. Product Management
In a nutshell, project management differs from product management in that the former is primarily concerned with managing people, schedules, and tasks while the latter focuses on managing the product and how it is created.
Table 1: Product Management vs. Project Management
Traditional project management is misaligned with modern software development for two reasons:
An overemphasis on managing the triple constraints of the “Iron triangle” to the detriment of technical execution, product quality, and value delivery
An overemphasis on control to the detriment of speed and flexibility
The “Iron Triangle”
Traditional program management focuses on enforcing the triple constraints of the Iron Triangle: cost, schedule, and scope (the amount of work done). The objective is to deliver expected value on time and on budget. Unfortunately, fixing the three constraints leaves quality as the only variable, opening the door to sacrificing quality when things don’t go as planned. Because no one can predict the future, most traditional projects fail to deliver on at least one of these constraints. In other words, “I can get it to you cheap, fast, or good. Pick any two.”
Figure 1: The Iron Triangle
Traditional project managers manage against detailed plans that attempt to fix all three constraints for the life of the project. They base plans on estimates of how long it will take a certain number of people, with specific skills, to accomplish a set amount of work at a certain cost. As problems arise, managers work to keep the project as close to the original plan as possible. If project reality strays too far from the plan, they develop a new project plan or scrap the project altogether. Replanning is a risky and expensive proposition, so project managers work mightily to avoid it.
Fixing cost and schedule begins by fixing scope. The more detailed project requirements are upfront, the more precise the cost and schedule estimates become. However, these calculations are not necessarily more accurate because they are based on estimates and on requirements that will likely change the bigger and more complex project scope becomes.
How Enforcing the Iron Triangle Affects Software Development
The problem with applying traditional project management to software development is that development of non-trivial and/or large-scale software solutions is too complex for linear predictive planning. As software solutions grow and evolve, their complexity increases, sometimes at a seemingly exponential rate. Since we can develop and modify software solutions in infinite ways, they are often significantly more complex than physical systems of comparable cost.
Enforcement of the Iron Triangle in software development projects often leads to the following problems:
Narrowing requirements and technical approaches too early – Predictive project planning typically forces teams to define what they will build and how before they really understand the business/mission and technology domains they will work in. This often leads to developing things that customers and end users do not need or that do not adequately address their needs.
Tendency to overengineer solutions – Specifying technical scope upfront often leads to overengineering. Defining and developing features that few, if any, customers or end users will actually use is quite common. Also, overengineering can lead to creating complex features or functionality where simpler versions could have satisfied requirements. All those rarely-used or overly complex features cost money to develop and maintain. They rob time and budget from developing and releasing features customers and end users truly value.
Difficulty managing risks and taking advantage of opportunities – In the zeal to keep projects “on plan”, planners often identify emerging risks too late to keep them from becoming issues or forgo opportunities to improve the product and how it is developed.
Forgoing opportunities to validate solutions – Projects based on predictive planning, such as Waterfall projects, often afford only one chance to validate what teams develop. Typically, validation of the solution occurs towards the end of the project, when the time to address problems is most limited.
Delaying delivery of business or mission value – With only one chance to deliver / release, sponsors, stakeholders, customers, and end users must wait until the end of the project to gain access to solution features.
Agile “Breaks” the Iron Triangle
Instead of attempting to fix all three constraints of the Iron Triangle, Agile fixes cost and schedule while allowing scope to “float”. In other words, Agile efforts are managed primarily by managing scope.
Under an Agile approach, there are no surprises. We have a certain amount of budget and time to spend developing software capabilities. We continuously verify (test) the solution as we build it to ensure we are building things right and demonstrate and release functionality often to ensure we built the right things. Since we build and deliver solutions iteratively and incrementally, customers and end users get valuable business or mission capabilities sooner.
Figure 2: Waterfall, Agilefall, and Agile Release Frequencies
For as long as we have enough budget and schedule, we have the option to continue developing functionality or stop after realizing enough value. We also have the option to pivot to a different solution approach or cancel the solution altogether. When budget and schedule run out, we can choose to invest more on the solution based on demonstrated functionality and customer and end-user feedback.
Figure 3: Pivot or Persevere
Overemphasizing Control
Enforcing the constraints of the Iron Triangle causes project management to implement heavyweight management and reporting processes. These processes are attempts to lower risk by increasing control. Overemphasizing control slows software development and decreases the flexibility necessary to tackle risks and challenges. On the other hand, underemphasizing control can turn flexibility into chaos, robbing software development teams of both planning and process stability, thereby slowing progress. An optimal balance between control, flexibility, and speed requires empowering technical contributors to plan their own work based on sponsor, stakeholder, customer, and end user needs and enterprise technical constraints. The main reason for this is the inherent complexity of software systems.
High-performing teams are empowered teams. They take on or participate in many duties traditionally left to project management: Devising work plans, decomposing work and deliverables, monitoring task and reporting progress, and meeting commitments.
Software is never finished. We’ve moved away from delivering discrete, standalone software package products (think Windows 95) to continual delivery of new software functionality and bug fixes via updates distributed over the internet. Even in organizational IT and enterprise system integration efforts, development never truly finishes. If any solution or product is to survive, it must adapt to meet changing customer needs while maintaining quality and performance.
Also, software is infinitely changeable. Since software is abstract, we can add and modify it at will with much less effort and expense than physical solutions or products. Thus, we don’t see the same level of resistance to updating software solutions as we see with physical products. The very nature of software solutions makes them likely to continue changing and evolving long after initial development.
Thus, managing the development and evolution of software solutions as projects that start and stop does not make sense. All software solutions go through the stages of the Product Lifecycle.
The Product Lifecycle
Figure 4: The Product Lifecycle
The Product Lifecycle includes the phases all products go through during their useful lives: from when they are first conceived, to when they are retired from the market or their mission domains. This cycle focuses on the performance of the product in its intended market or mission domain. All products, including software solutions, commercial or otherwise:
Start out as an idea developed into an initial version of a releasable product
Are introduced / released to their intended consumers or user bases
See their market or user base expand, if successful. This phase tends to drive the most product changes that keep the product relevant and/or profitable
See an eventual stabilization or flattening in customer use, sales, or revenue growth as the product’s market or mission domain matures
See their market or mission domain decline as it becomes saturated or customers move on to new solutions
Are retired when they cease to be profitable, effective, or relevant
Projects Incur Wasteful Overhead
Traditional project management’s focus on projects is wasteful. Starting and stopping projects incurs significant overhead. Examples include:
Contracting activities (and acquisition activities, for government projects) – Finding contractors / vendors, putting them on contract, monitoring performance, “Color of Money” concerns, etc.
Ramping up new employees or vendors to be productive – Accesses, equipment, software tools, training (formal and on-the-job), etc.
Ramping down employees or vendors as projects end – Removing accesses, returning equipment, managing software licenses, etc.
Not having the flexibility to easily terminate failing projects or projects considered no longer necessary or desirable – It is much cheaper and easier to cancel a project after spending $10 million than after spending $100 million
Agile’s Inherent Product Management Bias
So, if we should manage software development as product development efforts, rather than as projects, what does that look like? It looks like Agile software development. Agile software development in general, and Scrum specifically, have an inherent product management bias. Table 2 below shows how Scrum fulfills the product management duties listed in Table 1:
Table 2: How Scrum Fulfills Product Management Duties
Conclusion
In my experience, the single biggest reason why so many Agile software development efforts fail to achieve the benefits of Agile is the conflict between traditional project management thinking and Agile’s inherent product management approach to software development. This is not just my opinion, it is a growing realization Lean-Agile circles[i].
Today, business and mission requirements change too quickly for slow-moving predictive planning and management approaches. To meet the challenges brought about by the quickening pace of technology, we must leverage flexibility and control to achieve speed while maintaining quality. This requires a shift from a project management to a product management mindset.
Improve Product Backlog Ordering and Prioritization
In this video, Frank Velazquez explains what a User Story Map is and how it supports the ordering and refinement of product backlogs.
User Story Maps can help your Scrum Team order and refine its product backlog, plan releases, and communicate status and plans to sponsors and stakeholders. They help ensure teams take into account technical dependencies between product backlog items when ordering and refining product backlogs. They also help ensure that all planned and implemented product backlog items add up to a business or mission-valuable solution.
Organizations rooted in traditional project management often fail to reap the benefits of Agile. This is primarily because they try to implement Agile approaches in ways that align with traditional project management biases. One of the most common examples of this corruption of Agile practices is developing project schedules that attempt to fix the constraints of the Iron Triangle: Schedule, Scope, and Budget / Resources.
In this article, I explain why project roadmaps are a much better fit for Agile projects than traditional project schedules.
Traditional Project Schedules
Obviously, all projects have a beginning and an end. How much time transpires in between depends on how long it takes to accomplish a certain amount of work based on some amount of resources (e.g., equipment, people). The total cost of the project depends on the per hour cost of all those resources and the number of hours they contribute to the project.
Traditional projects attempt to fix or set the constraints of the Iron Triangle from the start. This entails identifying all of the work that needs to be done, estimating how long each item of work will take, and identifying who will do the work and at what rate. Planners determine the project’s budget from these data. To keep schedule, cost, and scope as close to the original plan as possible, project management works mightily to minimize changes to planned project scope (i.e., minimize “scope creep”). The traditional project management artifact that ties all three constraints together is the project schedule, often referred to in larger traditional projects as the Integrated Master Schedule (IMS).
Thus, in traditional software projects, planners develop schedules not just to set expectations of when teams will deliver and release software functionality, but also to track:
Work task progress
Dependencies between tasks
Estimated schedule impacts of early / late task starts and completions
Costs incurred as well as estimated future costs
The project schedule is the main instrument of traditional project management control. Project managers depend on it to manage tradeoffs between Iron Triangle constraints as the project progresses.
Project Critical Path
As I discussed in the previous section, traditional project managers use project schedules to track dependencies between work tasks. This allows them to track how the actual start and completion dates of any given work task will impact subsequent tasks. It is important to stress, however, that these schedule impacts are based on estimates that may (likely) be inaccurate.
The longest sequence or path of consecutive schedule activities is known as the project’s critical path. Usually, the critical path is the path that determines duration of the project. Tracking a project’s critical path is important in traditional project management because task delays along the critical path impact project schedule more than those not on the critical path.
Figure 1: Critical Path Example (Original from onepager.com)
Agile Projects Break the Iron Triangle
Unlike traditional projects, we manage Agile projects primarily by managing scope. Agile projects hold budget and schedule milestones fixed while managing scope to fit available time and resources throughout the project. This approach to planning aligns with Agile’s principle of welcoming “changing requirements, even late in development.” Rather than defining the total scope of the project upfront, we define it as we go. To pull this off without going over budget or schedule, Agile teams employ timeboxing, sizing, ordering, and frequent delivery of business / mission capabilities.
Timeboxing
Agile planning breaks up the project planning horizon into short time periods or “timeboxes” throughout the project timeline. Instead of creating a plan and schedule for the entire duration of a project, teams break up planning across each and every timebox on the project timeline. Agilists usually call these timeboxes “sprints” or “iterations.”
Figure 2: Waterfall vs. Agile Project Planning Horizons
Sizing
Agile teams plan enough work to fill the timeboxes. They don’t change timebox duration to fit the work. They do so by estimating the “size” of the business or mission features / functionality they plan to build. The size of a given feature (or user story) represents the combination of perceived or estimated:
Feature complexity, risk, and uncertainty
Level of effort required to complete the feature
Teams estimate features by “sizing” them relative to each other. More complex features typically require more time and effort to complete, thus making them relatively “bigger” than less complex features.
Based on these “sizes”, and the team’s record of finishing different sized features during previous timeboxes (known in Scrum as “velocity”), Agile teams determine which features to break down into smaller ones and which will fit into the next timebox or two.
Ordering
Rather than following a detailed and static plan of component deliveries for later integration, Agile teams plan their work based on adaptive planning. Ordering the development, delivery, and release of features / deliverables takes into account both what is most important or valuable from a business / mission perspective and the technical dependencies between the items. I cover ordering, and how it differs from prioritization, here.
Frequent Deliveries
Agile teams deliver small increments of valuable business or mission functionality. Each increment builds on previous ones. The system grows and evolves an increment at a time. Every delivery contains working functionality that is potentially releasable. This means that the functionality is ready for deployment and release when authorized.
Figure 3: Waterfall, Agilefall, and Agile Release Frequencies
Agile teams avoid delivering individual components. They strive to deliver fully functional features. Doing so allows project sponsors, stakeholders, customers, and end users to validate that teams delivered the right things.
The faster and more often teams deliver, the more opportunities exist to validate the solution as it grows and evolves. This Agile “empiricism” provides teams the information they need to plan, not just the next timebox or two, but the direction of the solution / product and its development.
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
https://agilemanifesto.org/principles.html
No Critical Paths in Agile Projects
Since teams define and refine the project plan continually throughout the project, there is no predefined set of work tasks to form a critical path. Also, in the case of Scrum, for example, teams organize work around implementation of user stories, not accomplishment of developer tasks. Rather than focusing on tracking task completion (as traditional project management does), Agile teams track delivered features.
Working software is the primary measure of progress.
https://agilemanifesto.org/principles.html
So, if traditional schedules don’t align to Agile projects, what should we use instead?
The Project Roadmap
Project roadmaps afford teams and project stakeholders the flexibility to shape the evolution of solutions throughout development. Instead of following and tracking a detailed predictive plan via a project schedule, Agile projects can leverage project roadmaps to chart the course of development. The course charted is informed by ongoing verification and validation of potentially releasable and released functionality throughout development.
Figure 4: Charting a course to a solution while navigating changing requirements, impediments, and risks
A project roadmap should depict the delivery of business or mission capabilities at a coarse-grained level. In the case of Scrum, for example, I recommend keeping the project roadmap at the epic level. The idea here is not to track every feature / user story and all of their associated tasks. Instead, the project roadmap should present the major slices of system functionality that deliver complete business or mission utility. However, some development team collaboration tools offer features that make drilling down below the epic level on the roadmap easy.
As depicted in Figure 5, we arrange the epics along a timeline that depicts their estimated start and end dates. Teams arrange the epics in the order they expect to work them, taking into account business / mission priorities and technical dependencies between them.
Figure 5: Agile Project Roadmap Example
Finally, we include major milestones and deadlines that teams must meet as part of project activities. Since Agile teams separate deployments to production from releases, and every timebox offers an opportunity to release, there is no need to clutter up the roadmap with lower-level details.
Conclusion
Project roadmaps are a better way of tracking planned completion and release of business / mission capabilities by Agile teams than traditional project schedules. They align with Agile scope management principles and provide the planning flexibility necessary for Agile projects. They are also much more lightweight and easier to maintain than complex project schedules. Most important, leveraging project roadmaps helps us avoid the tendency to over-plan and over-analyze that often turns Agile projects into traditional project management efforts.
