A definition of Agile applicable to all technical and non-technical projects that avoids discussing particular flavors of Agile or comparing Agile to Waterfall
Definitions matter. They matter because they frame how we think about the concepts they define. In turn, how we think about concepts affects how we engage with and apply them. Definitions also help us communicate by allowing us to agree on the meaning of things. A clear, concise definition of a concept helps crystalize in our minds what we may otherwise perceive as a disorganized collection of ideas and facts.
A Definition of Agile
No official definition for Agile exists. Instead, Agilists have a set of tenets and principles guiding how they work. As long as work practices foster agility and do not run counter to Agile tenets and principles, they are considered Agile.
As Agile thought leaders and practitioners develop and share new ways of working, the most successful concepts and practices eventually become part of an informal canon of Agile best practices. No formal body exists to regulate or promote this innovation and knowledge sharing. Rather than a methodology, Agile is a community that shares norms, values, conventions, and, to some degree, identity.
This lack of a prescriptive definition for Agile is powerful. It provides practitioners a great deal of latitude in creating Agile approaches best aligned to their unique business or mission context. It gives people permission to try widely-adopted practices, modify them, or reject them based on their own experience applying them.
However, such latitude also brings differences of opinion over what is or is not Agile. Without a formal definition, answers to the question, “What is Agile?” often devolve into Scrum jargon-filled word salad or attempts to define Agile by what it is not, namely Waterfall-based systems development.
Another problem with the standard approach to defining Agile is that it relies too much on the two foundational principles of Agile: The Agile Manifesto and the 12 Principles of Agile. While a deep understanding of the two is essential for anyone attempting to work in an Agile way, they were written from the perspective of software developers who build IT systems. While adoption of Agile practices is growing in many industries outside of IT, the IT-centric focus of most Agile literature and instruction often leaves practitioners outside of IT unsure as to how to apply them within their own context.
Taking into account the limitations in how Agile is typically defined, while adhering to the Agile Manifesto and the 12 Principles of Agile, Agile Cheetah proposes the following definition of Agile:
An approach to developing complex systems and solutions in which work teams self-organize to deliver increments of value, within and across short iterations, guided by collaboration with and among other work teams, project sponsors, stakeholders, customers, and end-users.
Developing Complex Systems and Solutions
Agile development approaches are particularly well-suited for developing complex technical and non-technical systems and solutions of all kinds. Software solutions, in particular, are inherently complex because they can be developed and modified in an infinite number of ways. Software tends to grow in complexity as we add new functionality to address new requirements.
Traditional project management attempts to cope with this complexity through predictive planning while Agile takes an adaptive planning approach. Rather than attempting to identify and plan for all project tasks and milestones prior to project start, Agile planning breaks up the planning horizon into increments. This allows us to adapt plans when circumstances change, which they invariably do.
Adaptive planning is most appropriate when we lack enough certainty about business or mission requirements, solution approaches, or technology options to formulate a predictive plan. This is typically the case for most non-trivial software development projects. Agile’s iterative and incremental development and delivery model supports, and is supported by, adaptive planning.
Agile Teams Self-Organize
High-performing teams are crucial to tackling complex projects and the best Agile teams are high-performing teams. They are self-organized and capable and empowered to plan their own work and to collaborate effectively among themselves and with other teams, project sponsors, stakeholders, customers, and end users. This team-level independence, coupled with effective organizational and project governance, affords the flexibility necessary to thrive in less-structured, more uncertain environments. We give up administrative command-and-control bureaucracy, in favor of project flexibility and speed while keeping effective control over projects.
Delivering Increments of Value, Iteratively
Agile teams deliver value iteratively and incrementally. Every increment of value Agile teams deliver provides real business or mission value. Therefore, technical Agile teams are primarily feature-based as opposed to component-based. The traditional project management focus on delivering system components prevents system validation until all components are integrated into the completed system.
This is important because the longer we wait to deliver useful capability to customers and end users, the later it will be before they can validate assumptions behind what we develop. Finding out that the solution is not what is needed or wanted at the end of a project is an unacceptable, yet common, occurrence in systems development. The faster and more frequently teams release functionality, the more opportunities we gain to incorporate what we learn into solutions as they are developed.
Agile Teams are Guided by Collaboration
What Agile teams create is determined through collaboration. Rather than deciding on a solution approach at the beginning and developing it with little to no input from sponsors, stakeholders, customers, and end users, those parties are actively engaged in defining requirements and validating system increments throughout development. This allows us to control scope by limiting what is built to what we know is currently needed and only to the necessary level of complexity. Achieving this level of collaboration requires Agile teams to work transparently by making information about their progress and the nature of what they develop openly and easily accessible. This way of working affords plenty of opportunities to inspect and adapt emerging solutions and team processes and practices.
Conclusion
Our wish is for this definition to help Agilists at all levels of experience explain what Agile is without resorting to talking about a particular flavor of Agile (Scrum, XP, Kanban, etc.) or engaging in comparisons between Agile and Waterfall. We expect our definition to simplify discussions about Agile across diverse audiences.
What do you think of our definition? How would you improve it? Do you have one you like better? Let us know by leaving a comment.
It has been almost 20 years since the publication of the Agile Manifesto and the 12 Principles of Agile that undergird it. During that time, the Manifesto has been a catalyst for profound changes in how we work, not in just software development, but across industries, governments, and nonprofits worldwide. Despite having attained such a level of influence, the Agile Manifesto is regularly misinterpreted in ways that undermine its intent.
This article sets the record straight. To do so, I walk through the Manifesto, line by line, provide context, and dispel a few myths. First, let’s read the Manifesto:
The Agile Manifesto
I’ll now go over each sentence starting with the first sentence:
We are uncovering better ways of developing software by doing it and helping others do it.
The Agile Manifesto and the 12 Principles of Agile together serve as the foundation of Agile practices. Rather than prescribing how to “do” Agile, Agile describes ways of working that steer away from a bureaucratic, command-and-control style of management to a decentralized, self-organized, and adaptive approach to building and delivering software systems.
Agile is a bottom up approach to software development in which developers and other technical contributors manage and continually improve the systems or solutions they develop as well as how they develop them.
Next, I skip to the last sentence:
That is, while there is value in the items on the right, we value the items on the left more.
The Agile Manifesto states a preference for lightweight processes supported by face-to-face communication and sustained and meaningful engagement between technical teams, project sponsors and stakeholders, and customers and end users. It does not prohibit traditional project management practices such as communicating via documentation, instituting processes, leveraging tools, planning activities, and engaging in contract negotiations. Agile does not prohibit them because they are all necessary to some extent for any technical project to succeed. What Agile argues against is measuring progress based on accomplishing these internal project activities. Agile argues for determining project progress based on delivery of useful and valuable solution functionality and structuring both the work and how teams are organized to best incorporate changing requirements and lessons learned into emerging software solutions.
Now I continue with the first tenet:
Individuals and interactions over processes and tools
Software development is a creative endeavor. As such, people are at the heart of it. Tools and processes should support open and direct collaboration between people, not serve as substitutes for it. Using what are often described as “Agile tools” such as JIRA or VersionOne does not equate to working in an Agile way. Processes do not make up for a lack of training and support for those who do the work. A focus on improving people and the teams they work in yields much greater dividends than instituting processes and using tools.
The second tenet:
Working software over comprehensive documentation
The only real way to verify and validate solutions and gauge project progress is by developing and delivering working functionality. However, documentation is necessary, even in Agile projects.
The question is how much documentation is necessary and for what purposes? Too much documentation generates waste, slows down development, and fosters a false sense of progress. Too little documentation can lead to a loss of project knowledge, hampers the ability to manage projects, and may leave customers and end users without the information they need to use the resulting solutions.
There is no magic formula for striking the right balance. Collaboration between technical teams, sponsors, stakeholders, customers and end users will bring to light what documentation is necessary and to what level of detail. Just keep in mind that, when it comes to documentation, “less is more”.
It is also important to remember that documentation has inherent drawbacks. It is often out of date, its usefulness is often time limited, and it can be a source of confusion. Think hard over these drawbacks before engaging in documentation activities to ensure they are worth the effort.
The third tenet:
Customer collaboration over contract negotiation
Traditional project management focuses on contract negotiation and enforcement, often to the detriment of collaboration. While Agile projects are also bound by contracts, the focus is on fostering a win-win atmosphere of collaboration between those who develop and deliver software and those who sponsor and use it. Collaboration in Agile occurs across the entire software development lifecycle.
The fourth tenet:
Responding to change over following a plan
For most non-trivial software development projects, predictive planning is the wrong approach. Software is intangible and infinitely changeable. The opportunities to change a software system are as infinite as the number of reasons to change it. Predictive software development plans do not survive first contact with project reality.
Agile planning is adaptive planning. Project plans evolve as the project proceeds. This affords the flexibility to stop development when a solution satisfies the need or when the project is no longer viable. We also avoid over-engineering solutions and developing features that few, if any people, actually use. This allows us to control project cost and schedule, by managing scope, without sacrificing quality.
Conclusion
Saying that understanding the Agile Manifesto and its implications is necessary for conducting successful Agile projects and Agile organizational transformations is an understatement to say the least. Too many Agile implementations fail because people rush to learn and apply the mechanics of a particular Agile approach (e.g., Scrum, XP, Kanban) without understanding the principles behind it. Agile is not a methodology, a product you can buy, or a process. You cannot templatize and checklist your way into agility. Agile is a way of working born from a small set of principles. It is up to all of us to employ Agile best practices, and develop new ones, that align with those principles.
In part one of this blog series, I explained what “Agilish” projects are and why we should avoid them. In this article, I discuss the reasons why government Agile initiatives tend to move away from Agile principles and towards becoming primarily Waterfall projects with Agile practices added (Agilish).
As I stated in part one, my perspective comes from years of working Agile projects for U.S. Federal Government clients. While many of the references are specific to that environment, equivalents exist at all levels of government, as well as in many private organizations, worldwide.
Process over Outcomes
Many government organizations possess process-centered cultures where progress is judged more on process execution than on outcomes achieved. The fundamental assumption being that if people correctly follow processes and procedures, desired outcomes will follow.
It is easy to see why. Governments are much more constrained than private organizations by laws, regulations, and policies (LRPs). Government institutions are ultimately accountable to the public, thus they must abide by extensive oversight rules. Documenting and enforcing processes and procedures help to ensure adherence to the law and to promote fairness and equality in government dealings.
With respect to managing software projects, the issue is not so much about having processes and procedures. They are necessary. The issue is about replacing judgement and understanding with highly-detailed and prescriptive processes and procedures. Much of current U.S. Federal acquisitions and contracting LRPs were written with traditional project management in mind. When carried too far, this urge to codify and template everything results in risk adverse acquisitions and contracting cultures hamstrung by their own rules.
This mentality leads to mechanistic Agile implementations that are at odds with Agile. Agencies “tailor” their Agile approaches to fit within their existing rules, processes, and procedures. While there are limits to how much agencies can do to align acquisitions and contracting processes to Agile, government agencies tend to overlook (and often reject) opportunities to do so.
Blaming Waterfall
For many organizations, years (if not decades) of managing
software development projects through traditional project management coincide
with years of project failures.
Frustration over the inability to deliver planned project scope on time,
on budget, and at acceptable levels of quality serves as a powerful motivator
for instituting Agile practices.
While traditional project management is increasingly out of step with modern software development, for many organizations, Waterfall is not the root cause of software project failures. Many organizations, public and private, lack the programmatic, engineering, and governance maturity necessary to effectively manage complex technology implementations, regardless of approach.
Organizations struggling to deliver under traditional project management processes will likely struggle even more using Agile. You see, Agile does not on its own fix problems; it makes them plainly visible for people to take action. Agile is a catalyst for change, not a solution.
When lack of programmatic, engineering, and governance maturity render organizations unable to address problems uncovered by Agile, they often retreat into familiar ways of working more in line with their actual capabilities. This, rather than traditional project management processes and culture, may be the single greatest obstacle to government Agile adoption.
Waterfall-Based Funding Mechanisms
Government builds software systems the way it funds them. For the most part, government agencies have not done enough to align how they fund software systems development with Agile and DevOps. This lack of alignment is a principal factor in turning potential Agile projects Agilish and is most prominently manifested by:
Establishing projects rather than managing services
Adhering to outmoded “color of money” funding allocations between new development, test and evaluation, and O&M support
The Trouble with Projects
The trouble with structuring software system development and funding around projects is that projects begin and end. A lot of coordination, planning, contracts management, and funding allocations happen before project start. That overhead is costly and time consuming, especially in government projects. These activities frustrate the very agility organizations try to achieve by imposing a predictive planning approach on software development.
Traditional projects afford one opportunity to get the solution right. As a result, there is a great deal of pressure to include as many requirements as possible for a wide range of users. This increases project scope and, consequently, increases project complexity, planning overhead, risk, and cost. Organizations also lose flexibility to address new requirements, manage risks, and seize opportunities during development.
All of this predictive planning relies exclusively on estimates against a plan or program baseline. If the plan turns out to be sufficiently wrong, a new plan is formulated (the program is re-baselined). Changes to traditional project plans are highly disruptive, so project managers work mightily to avoid them. In many cases, this effectively prevents projects from acting on empirical reality, until it is too late.
As I described in part one, software systems evolve for as long as they are in operation. We cannot accurately predict what customer needs and priorities will be months or years from now. Requirements are highly perishable, sometimes to the point that software systems are obsolete or irrelevant by the time they are fielded. A funding model that aligns with Agile and DevOps treats software systems as long-standing managed services rather than as serial “one-time” investments.
“Color of Money” Concerns
As stated in part one, Agile and DevOps compress planning, analysis, design, testing, and release activities into short, fast, and frequent iterations or cycles. Under DevOps, software development and operations teams share development and production support duties. Development and maintenance work are included in the ongoing flow of work done by teams. Traditional discrete project phases for these activities, performed by separate teams, no longer happen under Agile and DevOps. Thus, it does not make sense to segregate funding and related planning along the lines of Waterfall project phases.
The US Federal Government and the Department of Defense (DoD) in particular, are coming to this realization. Unfortunately, it will take changes in law and policy to move away from “color of money” funding mechanisms and that moves slowly.
Earned Value Management (EVM) Requirements
Applying Earned Value Management (EVM) to Agile projects imposes a vastly different definition of value on Agile. Agile does not attempt to quantify value in terms of dollars and cents. Instead, the focus is on delivering capabilities that project stakeholders, customers, and end users consider most important to mission accomplishment. Imposing EVM requirements on Agile projects tends to subvert Agile practices to accommodate the needs of EVM.
Vendors Switching to Agile while Government Lags Behind
Agile software development is the new normal in IT. Each succeeding batch of newly minted software development professionals entering software development is increasingly less likely to work on Waterfall projects. This reality is now a forcing function on government agencies to change how they work with vendors to support agility. Some agencies are doing better than others, but organizational cultures do not change overnight.
Conclusion
Recognizing the driving factors behind Agilish projects helps us understand the organizational environments within which we attempt to institute Agile approaches. Such understanding is invaluable to successful Agile adoptions/transformations, specially in the government space. Success depends on leveraging existing LRPs to align processes with Agile. While “textbook” Agile may never be realistic in many government settings, achieving the benefits of Agile requires us to understand the trade offs involved and strive to move towards greater organizational agility whenever possible.
In the third and final article of this series, I’ll cover ways to make Agilish projects more Agile.
Leveraging Impact Maps, User Story Maps, and Product Roadmaps to Align Organizational and IT Product Strategy using Scrum
This is the third and final part of a three-part blog series about aligning IT solutions/capabilities to organizational needs via an Agile-Scrum product management approach. You’ll find part one here and part two here.
In this article, I walk through how a Scrum Product Owner (PO), in collaboration with the rest of the Scrum Team and others, might bridge the “Product Management Vacuum” discussed in part II by leveraging impact maps, User Story maps, and product roadmaps for Product Backlog refinement and release planning.
Part II Recap
In part II, I explained how the PO, as part of the Scrum Team, serves as the linchpin connecting the Scrum Team to the larger organizational product management structure. The PO is responsible for fostering and maintaining ongoing collaboration between and among the Scrum Team(s), project/solution sponsors and stakeholders, customers, and end users to:
Define and maximize IT solution value in terms of expected and realized business/mission value
Define a product vision and strategy that aligns with organizational vision and strategy
Align the work performed by development teams to product vision and strategy
This product management function bridges what Ralph Jocham and Don McGreal call the “Product Management Vacuum.” In the absence of a true product strategy, many organizations fill the vacuum with traditional project management activities that focus on managing tasks and people, rather than on defining the product/solution and improving development processes. This typically leads to wasted effort borne out of disconnects between the Scrum Team, the sponsor organization, customers, and end users.
Finally, I introduced Jocham’s and McGreal’s “Three Vs” construct. The “Three Vs” – Vision, Value, and Validation – sum up the product definition/management activities POs lead as part of their role. The “Three Vs” are mutually dependent and reinforcing:
Vision drives the creation of value (features/functionality)
Value created enables validation of business assumptions behind product development choices
Validation informs changes to the product vision
The Official Scrum Artifacts
I am taking the time to clarify that most of the artifacts I discuss in this article are not official Scrum artifacts. The Scrum framework includes three official artifacts: the Product Backlog, the Sprint Backlog, and the Increment.
…is an ordered list of everything that is known to be needed in the product. It is the single source of requirements for any changes to be made to the product. The Product Owner is responsible for the Product Backlog, including its content, availability, and ordering.
The Product Backlog is the single source of requirements for a product/solution
Only the features and functionality compiled and ordered in the Product Backlog make it into the product
Helps the PO manage development scope
The PO owns the Product Backlog
Tracks product requirements evolution as the product and its environment change towards providing greater value
…the set of Product Backlog items selected for the Sprint, plus a plan for delivering the product Increment and realizing the Sprint Goal. The Sprint Backlog is a forecast by the Development Team about what functionality will be in the next Increment and the work needed to deliver that functionality into a “Done” Increment.
The Sprint Backlog makes visible all the work the Development Team plans to do during the Sprint
The highest ordered Product Backlog items become Sprint Backlog items
Sprint Backlog items have enough detail for the Development Team to understand what needs to be done
The Development Teams owns the Sprint Backlog
Only the Development Team can change its Sprint Backlog during a Sprint
The Development Team tracks progress and changes in work scope through the Sprint Backlog
…the sum of all the Product Backlog items completed during a Sprint and the value of the increments of all previous Sprints.
The increment is not just what the Development Team develops during the current Sprint. It includes whatever system/solution functionality the team developed in previous sprints
Every new Increment must be “Done” at the end of the Sprint
A “Done” increment is ready for release and capable of providing usable and valuable functionality regardless of whether the PO decides to release it
Product Backlog Refinement
Product Backlog refinement is the management of product requirements by the Scrum Team. The PO leads Product Backlog refinement in collaboration with the rest of the Scrum Team (i.e., the Development Team and Scrum Master) and with input from product/project sponsors, stakeholders, customers, and end users. During Product Backlog refinement:
As the product evolves, higher ordered Product Backlog items become more detailed and the estimates become more precise. With help from the PO, the Development Team defines and decomposes the highest ordered items they plan to work during the next iteration/Sprint into User Stories. The Development Team manages those stories in the Sprint Backlog.
Product Backlog refinement is an ongoing process. The Scrum Team decides how and when to refine
the Product Backlog. However, the Scrum
Guide recommends the Scrum Team expend no more than 10% of its “capacity”
(i.e., the sum of productive hours across the team) on Product Backlog
refinement. Product Backlog refinement is
not restricted to specific timeframes or meetings. The PO can make changes to the backlog at any
time or have others do so at his/her discretion.
A Product Management Scenario Example
In part I of this series, I discussed organizational and product visions and strategies generically. To help visualize how all the parts fit together, I walk through a product definition and management scenario.
Over the last year, an online retailer has experienced a significant loss of market share to competitors. The company investigated reasons for the drop in returning customers and new registrations. The findings boiled down to customer dissatisfaction with business service functions across the value chain that negatively impacted the company’s reputation. While the company enjoyed an early market lead for years, competitors were quickly closing the gap with more innovative customer-centric service options and better delivery performance. The most significant problems found were:
Delivery problems
Customers receiving their orders late at a quarterly rate higher than industry average
Customer orders lost at a quarterly rate higher than industry average
Customers receiving the wrong orders at a quarterly rate higher than industry average
Customer service issues
Customer complaints about being bounced from one customer representative to another with a lack of continuity between representatives
Customer complaints about needing faster and more accessible help with “non-standard” issues requiring additional attention
Customer wish for an online customer service feedback option integrated with their ordering experience
Falling behind in offering new payment service options (Paypal, Google Pay, etc.)
Management shifted company strategy from a narrow focus on delivering
an “outstanding” website experience to improving the entire buying experience,
from online shopping to product delivery.
They revamped the company’s vision to match this new value stream-based
approach to customer satisfaction and updated their business strategy to
support it.
Company leadership recognized that a lack of customer focus caused the company to fall behind competitors and a subsequent decline in customer satisfaction. To address this, the company changed how it defines its services by instituting a product management approach. The company started looking at its website as a collection of online service offerings. Company strategy needed to be tightly coupled to customer needs and demands, changing as the market changes.
The table below breaks out the flow down of strategic vision and business strategy to tactical product vision and strategy I explained in part I of this series. The area highlighted in yellow is the “Product Management Vacuum” filled by organizational product management and POs in collaboration with the Scrum Team and with input from project/solution sponsors, stakeholders, customers, and end users. I focus on the strategic goal to increase customer retention and acquisition through better customer service down to the tactical goal of improving the company’s Customer Service System. Text of that tactical goal and its decomposed elements is colored red.
Table 1: Strategic Strategy to Product Strategy Decomposition
Defining Product Features
Armed with product goals, the PO works with the Scrum Team and others to define features they believe will help achieve them based on their current understanding of the business and market context. The Scrum Team continually refines that understanding as the Scrum Team releases features to users. The ultimate validation of the business assumptions behind new features is end-user validation. New insights shape future features/capabilities and business/mission strategic and tactical strategies.
Figure 1: Ongoing Organizational and Product Strategy Validation
Impact Mapping
Impact mapping is a strategic planning technique Scrum Teams
can leverage to develop Product Backlog items such as Epics and User Stories as
extensions to a defined product strategy. Impact maps are a great tool for
visualizing alignment from product vision to coarse-grained product backlog
items that Scrum Teams can order and further decompose and refine over time.
The diagram below depicts the use of an impact map to decompose the product vision and product objectives for the Customer Service System. One or more coarse-grained Product Backlog items support each product objective. These backlog items will collectively form the initial version of system requirements. Over the course of Product Backlog refinement and Sprint PlanningA time-boxed meeting where the Scrum Team plans the work to be done during the Sprint., the Scrum Team refines these items into User Stories. Each backlog item includes of three elements:
Actor – The role for whom the backlog item is for. The beneficiary of new feature functionality
Impact – The desired outcome from using the feature
functionality
Deliverable – The new feature functionality that will
help the actor achieve the desired impact
We avoid creating compound requirements. In other words, each backlog item is specific to one role, one deliverable, and one impact. These are high-level requirements, so the Scrum Team will identify, through their decomposition, different flavors of the same type of actor (e.g., administrator vs. regular user), related impacts for different flavors of actor, and related deliverables/features. It is important to remember that these are system/solution-level items. Too much specification now is counter-productive.
Figure 2: Impact Map
Product Backlog
Product Backlog Recap
After defining a set of course-grained requirements, we create the initial version of the Product Backlog. The Scrum Team writes requirements in the form of Epics and User Stories. User Stories are backlog items the Development Team estimates will take no more than one Sprint to complete and make ready for release, while Epics are estimated to take more than one Sprint. The Scrum Team decomposes Epics into User Stories as part of Product Backlog refinement.
At this point, we do not know how long it will take to complete each backlog item, so we do not know which items are Epics vs. User Stories. It is likely that these coarse-grained items will be Epics but I call them User Stories and group them under “themes.” The concept of themes is not part of the Scrum framework but they can come in handy when structuring the Product Backlog. Scrum does recommend adding backlog attributes to group items when multiple Scrum Teams are working from the same backlog.
The Product Backlog table below maps each backlog item to the desired impacts and product objectives listed in in the impact map. The product objectives become the value measures by which we measure whether each item’s feature delivers the expected business impact/outcome. I highlighted the Actor and Deliverable elements in each User Story.
Table 2: Initial Product Backlog
Based on the coarse-grained User Stories listed in the table above, the Scrum Team knows enough about what it plans to build to provide the Customer Service System’s tactical plan.
Table 3: Tactical Goal Decomposition with Tactical Plans
User Story Sizing and Product Backlog Ordering
I will not go into detail about how to size and order Product Backlog items in this article. However, I will highlight some key points that support the topics covered in this article:
The Scrum Guide refers to estimates rather than sizes. However, the most popular estimation techniques in Scrum leverage the concept of “relative sizing.” Instead of attempting to estimate how long it will take to complete a User Story, Scrum Teams use techniques like Planning Poker to estimate the story’s “size” relative to the other Product Backlog items. Relative sizing is faster to do, more intuitive, and becomes increasingly more accurate as the Scrum Team “matures” than estimating in hours.
As part of the Scrum Team, the Development Team is solely responsible for all estimates. The PO provides the business/mission context the Development Team needs to estimate the relative size of each Product Backlog item.
Typically, Scrum Teams estimate User Story sizes in “story points”, with “larger” stories earning higher point estimates. Often, the point scale used is the Fibonacci Series or a modified version of it. Story points are heuristically determined, numerical representations of User Story complexity, risk, uncertainty, and level of effort combined.
By tracking the number of User Story points the Scrum Team completes across multiple Sprints (at least three Sprints), the team determines its “velocity.” Scrum Team velocity is a heuristic measure of a Scrum Team’s throughput (not productivity). It is the rate at which a team delivers fully completed, tested, and accepted User Stories during a Sprint.
Scrum replaced Product Backlog “prioritization” with “ordering.” Ordering better connotes the concept of ongoing backlog refinement. Rather than periodically prioritizing backlog items according to vague business value categorizations, like High, Medium, and Low, and awarding multiple items the same priority, the Scrum Team continually orders backlog items based on their business value, risk, cost/size, and dependencies.
As User Stories rise in order, the Scrum Team fleshes out story details, provides more precise story point estimates, and decomposes larger stories into smaller stories the team can complete during a Sprint.
User Story Mapping
What are User Story Maps?
A Product Backlog is essentially an ordered queue. As the number of User Stories grows, it
becomes increasingly difficult see how all of the stories come together into a
system/solution. This is especially true
for solutions that involve workflows. A
list of ordered features, written as User Stories, does not communicate how
those features work together and what dependencies exist between them.
Scrum Teams can use User Story maps to organize an ordered Product Backlog across solution workflow steps, Sprints, and releases. Scrum Teams may use User Story maps in tandem with Product Backlogs to help identify dependencies as part of backlog refinement. It is a valuable tool to help plan releases as well.
Below is a User Story map example. The map is a simplified workflow for coarse-grained User Stories #4 and #5 in the initial Product Backlog I discussed earlier in this article. The User Stories read:
4. As a customer, I want the choice to provide feedback about the customer service I received so that I can communicate whether my issue was resolved to my satisfaction and why.
5. As a Customer Service Representative, I want access to customer feedback about issues I handle so that I learn how best to improve my performance.
The deliverable for User Story #4, listed in the example impact map, is a user feedback form. However, as described in User Story #5, there is much more to the form then just a webpage to enter data. The Scrum Team needs to develop a workflow that links multiple roles across the organization. User Story #5 is likely an Epic in need of decomposition while User Story #4 is likely a User Story the Development Team can finish in one Sprint. As Product Backlog refinement occurs, the scope of individual stories and of the entire workflow likely changes.
In this workflow, the Customer Service Coordinator reviews
incoming feedback submittals, categorizes them, and assigns them to the
appropriate part of the organization for resolution/acknowledgement. Should the initial assignment be inappropriate
or fail to yield a satisfactory outcome, the Customer Service Coordinator works
across the organization to identify the right people to address the
feedback. Otherwise, the customer
receives a response.
Below is a sample User Story map for this workflow.
Table 4: User Story Map Example
Let us walk through the User Story map’s elements:
Table 5: User Story Map Element Descriptions
The yellow cells in the User Story map example represent individual User Stories decomposed from the Epic directly above them in the “Walking Skeleton.” This decomposition is not one person’s idea of how the Epic should be broken down, not even the PO’s. It is instead the result of ongoing collaboration within the Scrum Team, informed by project sponsors and stakeholders, customers, and end users, and synthesized into a coherent product vision and strategy by the PO.
Development of a User Story map is an iterative process. As the solution takes shape iteratively and incrementally, more users, User Stories, business activities, and business tasks emerge, requiring changes to the User Story map and the Product Backlog. Fundamental changes in approach, predicated by insights learned during product validation, with actual customers and end users, may require changes to the product vision and strategy as well.
Product Roadmap
A product roadmap is a planning tool used to visualize when Scrum Teams estimate they will complete coarse-grained capabilities/Epics and in what order, over a given time horizon. A product roadmap facilitates planning, not just for the Scrum Team, but also for project stakeholders outside the Scrum Team. Common uses for product roadmaps include:
Communicating status and longer-term plans to
organizational decision makers
Identifying business/mission dependencies across
planned releases
Informs longer term business/mission planning
(e.g., funding, contracting, agreements with external partners)
A product roadmap is
not a schedule. It is a snapshot of
current planning, not a set of fixed milestone delivery dates. Short-term release timelines (no more than
three Sprints ahead) are based on User Story sizing and ordering (done as part
of Product Backlog refinement) as well as demonstrated Scrum Team
velocity. The PO bases longer-term
release timelines on the rank order of lower-ordered Product Backlog items and
current knowledge of dependencies between those items.
Like everything else in Agile and Scrum, product roadmaps
are always subject to change. As the
solution emerges and new learning happens, plans change. Agile planning is adaptive planning. Rather than creating detailed plans that span
months or years, Scrum Teams wait until “the last responsible moment” to plan only
the highest-ordered Product Backlog items in detail.
The recommended time horizon for Scrum Team planning is no
more than the next Sprint and two Sprints beyond that. Based on the recommended two to four-week
Sprint length, that is no more than 12 weeks.
The amount of implementation detail behind a User Story increases the
closer that story comes to being implemented by the Scrum Team. User Stories planned for the next Sprint are
much more detailed, and their estimates are better informed, than lower ordered
User Stories tentatively scheduled for later Sprints. Therefore, plans to implement solution
functionality beyond the next three Sprints are highly imprecise and likely to
change.
The sample Product Roadmap below maps the product strategy components I discuss throughout this article into a quarterly based plan for the year.
Table 6: Product Roadmap
Release Strategy
Depending on the size of the sponsor organization/company,
the PO either contributes to an organizationally agreed-upon release strategy
or collaborates with the Scrum Team, project sponsors and stakeholders,
customers, and end users in developing one.
A product release strategy dictates:
Release Frequency – How often Scrum Team(s) release functionality: the more frequently, the better. In Waterfall projects, there may only be only one a major release at the end of development. Release frequency for Agile/Scrum Teams ranges from releasing once at the end of each Sprint, to releasing multiple times throughout each Sprint, to achieving a steady flow of “done” functionality released on “demand” (Releasing functionality when customers/end users are ready to accept it)
Validation Frequency – How often customers and end users validate newly released functionality. Obviously, the more often Scrum Teams release functionality, the more opportunities for validation exist
In our company example, the company’s release strategy is to release at the end of every four-week Sprint. Scrum Teams monitor usage and performance metrics for the capabilities they release and the organization monitors customer feedback.
Table 7: Tactical Goal Decomposition with Release Strategy
Release Planning
The product roadmap provides a high-level, “broad strokes”
view of how planned releases fit together over an extended period of time. The Scrum Team plans releases at a greater
level of detail and across much shorter timeframes (no more than three
Sprints). A PO develops a release plan,
which aligns with the operative release strategy, in collaboration with the
rest of the Scrum Team, project sponsors and stakeholders, customers, and end
users. A release plan takes into account
the following considerations:
The Scrum Team’s capacity to develop “done”
capability at a sustainable pace
Technical and business/mission dependencies on
or arising from planned functionality
The ability and willingness of customers and end
users to “absorb” or accept planned changes
Alignment with business/mission strategy and
product strategy
Just like a product roadmap, the release plan is subject to change. Certainty over what a Development Team will develop and release over the next three Sprints is greatest for the next Sprint and becomes progressively less certain for the next two. However, since the release plan’s time horizon is so short and immediate, the frequency and magnitude of the changes should be significantly less than for a product roadmap.
Below is an example of a release plan for the Customer Feedback Workflow capability based on the User Story map discussed earlier:
Table 8: Release Plan
Conclusion
Scrum is a framework and as such avoids over-prescribing practices and artifacts. The expectation is that practitioners will adhere to its rules and tailor existing processes to align with the framework, not the other way around. The concepts and tools presented in this series are meant to be used as enablers for aligning organizational vision and business strategy with product vision and strategy using Scrum.
Scrum has an inherent product management bias that is a definite departure from the traditional project management mentality that has reigned over organizational IT for decades. A product management approach to organizational IT shifts project-centric, internally-focused sponsor organizations towards putting the customer first and focusing on achieving outcomes, rather than executing plans.
