Author: Efrain (Frank) Velazquez

Why So Many Developers are Fed Up with Agile (Pt. 1)

The Never-Ending Agile Grind

Agile has become the way we build software.  Across industries, government institutions, and non-profits, it is the default software development approach.  Debates over whether Agile should supplant traditional project management approaches to software development (i.e., Waterfall) have long been passé.  The focus now is on accelerating the product development lifecycle through DevOps.

However, all is not well in “Agileland”.  Many organizations are not enjoying the benefits promised by the Agile gurus.  Many software developers are fed up with Agile.  If Agile is so great, why are so many people, especially developers, souring on it?  The bottom line upfront (BLUF) is that what passes for Agile in many places is anything but Agile.

This article is the first of a four-part series that describes the most common pain points experienced by software developers and other technical contributors engaged in Agile software development, the root causes, and potential solutions.  The pain points covered in the series are:

  • The Never-Ending Agile Grind – Developers often feel overwhelmed by the constant pressure to produce under tight deadlines.
  • Greater Visibility & Oversight with Less Control – The “radical transparency” of Agile approaches, such as Scrum and Kanban, is often exploited by management as an opportunity to micromanage development teams.
  • Top-Down Driven Agile – Technical contributors often have little say in how their organizations or sponsors implement Agile, leading to the imposition of unnecessary, ineffective, and, often, counterproductive practices on development teams.
  • Changing Expectations for Developers – Developers are moving away from functioning as “order takers” focused almost exclusively on coding to collaborating with product owners/managers, sponsors and stakeholders, and customers and end users to define, verify, and validate what they develop.  Many technical contributors find this level of direct collaboration difficult to manage alongside their growing list of technical responsibilities.

The Never-Ending Agile Grind

Software development is a demanding intellectual activity.  It requires study, perseverance, and focus.  The work requires time to think and experiment.  Some degree of failure is necessary as long as it yields valuable insights and teams use those insights to continually improve the product and how they work.

The experience of too many developers on Agile teams, however, is working in “feature factory” environments and “death march” projects.  They feel pressured and rushed to deliver code to the point of sacrificing quality.

Sprints become Sisyphean ordeals where overburdened teams are under constant pressure to deliver working code as quickly as possible while also maintaining the existing code base and contributing DevOps pipeline automation components.  The end of one Sprint leads to the beginning of another in a seemingly never-ending loop of stress that eventually leads to burnout. 

Teams are not afforded time to think through and experiment with alternative approaches, leading them to commit to suboptimal approaches that will likely take longer to implement and maintain than more elegant approaches they could have conceived given sufficient time.

“Feature factory” teams are denied time “on the clock” to learn and experiment with new technologies, design patterns, and architectural approaches.  As a result, implementations based on those approaches are often half-baked, filled with technical debt, under-performant, and difficult to maintain.  The benefits expected from adopting new technological approaches are not realized.  Meanwhile system complexity explodes exponentially.

Management consistently chooses delivery of new functionality over addressing technical debt.  This short-sided preference leads to increasingly brittle systems that force developers to expend increasing amounts of energy and focus on fixing bugs and babysitting systems.

Overtime, developers start to feel like robotic coding units who get paid to crank out code.  A mentality of “getting it done on time no matter what” becomes culture.  Any vestige of work/life balance is thrown out the window as the backlog of features grows and the system or product becomes ever more unmanageable.

The Causes

The Agile Manifesto for Software Development and the 12 Principles of Agile describe the characteristics of Agile software development.  Software development approaches that adhere to the values of the Manifesto and the 12 principles are considered Agile.  Approaches that deviate from any of the values or principles are not.  One Agile principle “feature factory” teams violate is principle number #8:

Agile processes promote sustainable development.  The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

http://agilemanifesto.org/principles.html

Why do so many software development organizations and their sponsors ignore this principle?  Many blame the increasing pace of business competition or mission threats for this single-minded focus on production and speed.  While pressure from increased competition is a contributing factor, I posit that the root cause is cultural.  Many (if not most) of these organizations take a project management approach to software development.  Agile, in general, and Scrum in particular, have a product management orientation.

Traditional project management, rooted partly in Taylorism, focuses on managing resources, schedule, and budget.  People are a category of resources managed by managers.  Traditional project management places a very high value on maximizing resource utilization to achieve greater efficiencies.  People are the most important (and expensive) resources to maximize.  In the drive to do more with less and meet tight (often arbitrary) deadlines, management often loses sight of the link between team welfare and morale and superior performance.

Product management focuses on managing the product and how the product is made and delivered.  Through Agile, management shifts its focus from micromanaging developers to collaborating with them in continually improving product offerings as well as the development and delivery of those offerings.  Ensuring development teams are not overwhelmed becomes a major concern since it has such a large bearing on product quality, timeliness, cost, etc.

Another cultural factor that leads to “feature factory” cultures is confusion over the role of planning in Agile.  A popular misconception is that there is no planning in Agile software development.  This misunderstanding leads to underemphasizing requirements definition, analysis, and design which, in turn, leads to badly conceived and executed software implementations.

Agile teams plan constantly.  The difference between Agile planning and planning in Waterfall projects is that Agile planning is adaptive while traditional project planning is predictive.  In other words, rather than attempting to plan the entire project upfront, Agile plans change based on current circumstances and new information and insights.

The Software Development Lifecycle (SDLC)

Just like development teams in Waterfall projects, Agile teams perform all the activities encompassed by each of the phases of the Software Development Lifecycle (SDLC).  The difference is that the “Agile SDLC” (DevOps) does not segregate those activities into discrete scheduled phases arranged one after the other for the entire system or solution.  Instead, every feature/deliverable an Agile development team delivers is the product of its own “mini” SDLC.  The resulting system or solution is the sum of all integrated features.

Potential Solutions

So, what are some concrete steps software development organizations and their sponsors can take to avoid overwhelming and burning out technical contributors?

The first step is understanding and internalizing that Agile development teams are self-organized.  They collectively manage their own work, determine technical approaches, develop their own estimates, report their progress, and help define the functionality they build.  Team self-organization is a core tenant of Agile supported by three of the 12 Principles of Agile:

Principle #11:  The best architectures, requirements, and designs emerge from self-organizing teams.

Principle #5:  Build projects around motivated individuals.  Give them the environment and support they need and trust them to get the job done.

Principle #12:  At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

http://agilemanifesto.org/principles.html

This contrasts with traditional project management practice, in which management is responsible for defining what to build, breaking down and assigning work, developing plans and schedules, and reporting progress.

Align the Work to Team Capacity

Agile teams break down their work with an eye towards delivering valuable functionality within short timeframes.  They can only achieve this if they align the amount of work they commit to with the team’s ability to get it done.  In other words, break down solution scope into slices of capability (features) that can be accomplished within agreed timeframes.  By ordering/prioritizing features along the lines of business or mission value, Agile releases deliver the most valuable features/capabilities first.

Agile teams commit to additional work as they complete existing work.  Rather than having work pushed on Agile teams, Agile teams pull work when they are ready to do it.  Software development teams that do not have control over the amount of work they commit to are not Agile teams. 

Allocate Time for Specific Tasks during Sprints

One way to ensure that Agile teams address technical debt, and other necessary tasks outside the scope of developing new features, is to allocate some percentage of their time to those activities.  For example, during Sprint planning, a Scrum Team may allocate some percentage of the next Sprint to addressing technical debt.  They could do the same for backlog refinement and DevOps automation work.

Establish Innovation & Planning Sprints

The Scaled Agile Framework (SAFe) includes Innovation and Planning (I&P) iterations (Sprints).  I&P iterations occur at the end of each Program Increment (PI), which lasts eight to twelve-weeks.  Each PI includes two to six iterations depending on the length of each iteration (two to four weeks).  Agile teams do not have to adopt SAFe to experiment with I&P iterations or Sprints.

I&P iterations provide teams time to work important tasks, such as the ones below, without significantly disrupting the flow/cadence of development and release of new functionality:

  • Explore new and innovative approaches (e.g., experiment with new technologies; evaluate products, tools, and services)
  • Work on technical infrastructure, tooling, and other impediments to delivery
  • Engage in continuous learning and improvement
  • Cross train in new skills
  • Engage in longer-term planning (e.g., product roadmap discussions, architecture design)

Conclusion

It is common knowledge that productivity, quality, and innovation suffer when workers are stressed and burned out.  However, too often responsibility for coping with stressful work environments is placed solely on the shoulders of individual employees/contributors while management sticks to outdated ideas about maximizing efficiency.

Agile, implemented in the spirit of Agile, opens the door to sustainable progress and achievement.  Again, development teams that do not control the amount of work they commit to delivering are not Agile teams.  Such teams risk being pressured into over-promising while under-delivering.

In the next article of this series, I’ll discuss how the greater visibility into team activities and progress afforded by Agile is often used to against the development teams themselves.  Just like this article I will also discuss root causes for the dysfunction and suggest solutions.



Will an Agile Certification help me get a Job?

It could, but choose wisely

Getting a certification can be a time consuming and expensive proposition.  Most people do not commit to paying for and following a certification program unless they feel reasonably certain they will benefit from doing so.  For many people, getting a job, advancing their career, or making more money are the primary motivations for becoming certified.

However, rarely does getting a certification, on its own, lead directly to getting a job or promotion.  Most people understand this, at least intuitively.  However, Agile-related internet forums are filled with posts asking:  Will an Agile certification help me get a job?

This article describes how to decide which Agile certification program is most likely to bring you professional success.  While my focus here is on Agile certifications, this same decision-making approach applies to any kind of certification you may be considering.

Deciding on a Certification

When deciding which Agile certification to pursue, evaluate your options based on how each certification aligns to your professional goals, industry and job considerations, and life circumstances.

Your Professional Goals

We can group professional goals into three categories:  Professional development, marketability, and changing careers.  Professional development is about keeping up relevant skills, or learning new ones, to improve your current job performance or get promoted.  Your marketability depends on how attractive a job candidate you are, across a profession or industry, based on your skills, knowledge, and experience.  Changing careers may require gaining education, skills, and certifications.

Industry and Position Considerations

The Agile approach(es) you choose to become certified in must align with the needs of your target profession or industry and with individual position requirements.  For example, a software product development company may need certified Scrum Masters to help development teams become more effective by applying Scrum.  In that case, a Scrum Master certification would be valid differentiator.  Another company might be a systems integration company that develops large-scale systems that include both software and hardware components.  In that case, a certification in an Agile scaling approach such as the Scaled Agile Framework (SAFe) would be appropriate.

Let’s delve further into industry and position considerations in choosing Agile certifications.

Table 1: Industry and Position Considerations Relevant to Choosing an Agile Certification

Industry Considerations

Taking into account the following industry-related considerations will help you choose which certifications are most aligned with your professional goals.

1. Does the industry recognize the certification and certifying body?

A quick look through job postings and descriptions for your target industry should quickly bring to light that industry’s required or most desired certifications.  Some certifications require more time and money to get than others.  Many Agile certifying organizations or bodies require attending expensive classes and taking expensive tests to become certified.  Other bodies, like Scrum.org, offer classes but allow people to take their certification exams without class attendance.  Ensure that the certification is recognized and accepted within the industry before spending time and money acquiring it.

Be on the lookout for references to specific certifying bodies.  For example, ScrumAlliance and Scrum.org are two well-known and widely-accepted Scrum certifying bodies for Scrum. 

Also look at industry forums and ask people in the industry about which Agile certifying bodies they recognize and why.  Getting an Agile certification through a recognized body increases its value.

2. Is the certification in demand?

Just because a certification is recognized within an industry does not mean it is necessary to gain entry into that industry.  Your prior experience is often more valuable than a certification.

However, a certification may be a minimum requirement to gain employment in a particular industry.  You need to determine your relative value compared to other job candidates.  If most candidates possess certifications and you don’t, you’ll be at a disadvantage.

3. Will the certification set you apart from other candidates?

On the flip side, having a certification may actually set you apart from other job candidates in your industry.  This is truer for more specialized Agile certifications like the different SAFe role certifications. 

Understand, however, that a certification is not a substitute for real experience.  It is a credential that attests to your knowledge about, and understanding of, a particular Agile approach.

Position Considerations

After narrowing down the choices based on your professional goals and target industry, consider which certification options most closely align with the positions you seek.

1. Is the position an Agile role?

Are you interested in positions where you’ll play a formal role within an Agile approach or framework such as Scrum or SAFe?  For example, are you going for a Scrum Master or Product Owner position for a Scrum Team?  Do you want to be a Release Train Engineer (RTE) for a SAFe Agile Release Train (ART)?  If so, then becoming certified for such roles will likely benefit you both in terms of meeting position requirements and preparing you for the roles.

2. Does the certification lower the number of years of experience required?

Some organizations lower the number of years of experience required for a position if the candidate possesses a specific certification.

3. Is your position changing because of Agile?

Perhaps your current position or industry is moving towards Agile and you need to understand how to best deal with the change.  For example, you are a traditional project manager who is now working Agile projects.

Learn about the organization’s chosen Agile approach and understand its roles.  Then ask yourself, “Which of these roles most aligns with the work I do today?”  Determine whether it makes sense for you to transition to that role or learn how best to support that role from your current position.  Either way, a certification in that Agile role will arm you with the knowledge necessary to participate and contribute in an Agile environment.

Gaining Multiple Certifications

Obviously, if you intend to become an Agile coach, mastery of multiple Agile approaches (e.g., Scrum, Kanban, XP, SAFe, LESS) is expected.  However, even if becoming an Agile coach is not your goal, you may find it beneficial to possess more than one Agile certification. Many Agilists do.  You may want to start with becoming certified in the different roles of your organization’s chosen Agile approach and later expand into other Agile approaches and Lean process management practices. How you expand your Agile knowledge is entirely up to you.

Your Professional Experience and Life Circumstances

The next set of factors to consider when deciding whether to pursue an Agile certification are your professional experience and life circumstances.

Table 2: Professional Experience and Life Circumstances Relevant to Choosing an Agile Certification

Professional Experience

If you do not have a background in developing, testing, and supporting software systems, or managing those who do, you will have a steeper learning curve.  Agile philosophy is applicable to team-centered product or solution development in practically any industry or domain.  However, Agile approaches, such as Scrum, were originally conceived as better ways of developing software.  As such, translating Agile practices to work outside the realm of software development requires knowledge, skills, and experience.

Everyone’s familiarity and background in Agile is different.  If you are new to Agile, introductory courses and beginner-level certifications make sense.  If you are an experienced Agilist, then progressing through intermediate and advanced certification levels may be right for you.  Make sure that you meet course and certification prerequisites.

One course or certification does not make anyone proficient in Agile.  Attaining a certification is really the beginning of an ongoing journey of learning, application, and improvement. The journey involves hands-on application, Agile community participation, and individual study augmented by structured training opportunities.

Life Circumstances

Do you have the time and money necessary to get certified and stay certified?  As stated earlier, many Agile certification programs require attending certification courses that they sponsor.  Many certifying bodies also require periodic (often annual) re-certification payments as wells as documentation of ongoing professional development activities related to the certification.  Understand all of the time and money requirements associated with any Agile certification you pursue.

Conclusion

Committing to attaining an Agile certification is a big step.  You want to be certain that your chosen certification program aligns with your professional goals, your target industry/positions, and your life circumstances.  Rather than thinking about Agile certification in terms of landing a particular job, think of it as an opportunity for professional and personal growth that will open up opportunities.



The Five Most Important Factors in Prioritizing Product Backlogs

Achieve effective Product Backlog ordering by optimally balancing five competing factors

Note: This article includes a video detailing how to calculate Weighted Shortest Job First (WSJF) values.

Product Backlog “prioritization” is a popular topic in Agile and Scrum forums.  It seems like a simple concept.  Teams prioritize the most valuable or important product backlog items or features and begin work on the highest priority items.  However, teams quickly discover that there is much more to prioritizing backlog items than that.

The difficulty boils down to determining what is more or less valuable or important when every item addresses valid needs.  How do we deal with the ambiguity of determining what is “most valuable” or “more important”?  How do we account for dependencies and other factors that influence what we build and when? How do we do so in a consistent, non-biased way that ensures delivery of valuable solutions, cost effectively, and in time to be most valuable?

Understanding Product Backlogs and their Refinement

For the uninitiated, before reading more about Product Backlog prioritization, I recommend reading our article on Product Backlog refinement.  At a minimum, understanding what a Product Backlog is and what Product Backlog refinement entails is crucial to understanding backlog prioritization.

Product Backlog refinement includes three main activities:

  1. Management of Product Backlog Items:  The creation, retrieval/tracking, update, and deletion of items

  2. Sizing of Product Backlog Items:  Relative size estimation of items

  3. Ordering:  Deciding in what order to address each item

In this article, I focus on Product Backlog ordering. This article does not apply to Sprint or iteration backlogs. Sprint backlogs are solely composed of user stories that take two to four weeks to complete. These items are likely too small in scope to warrant multi-factor prioritization/ordering schemes.

Prioritization vs. Ordering

As of 2011, the official Scrum Guide began defining the Product Backlog as “an ordered list” instead of a “prioritized” list.  Why was this changed and what is the difference between prioritizing a list and ordering it?

As Barry Overeem states in his Scrum.org article:

The focus on ordering (over ‘prioritization’) underscores the active role that the Product Owner continuously has to play in the ordering and reordering, of the work in a manner that maximizes value.

https://www.scrum.org/resources/blog/myth-5-scrum-product-backlog-prioritized

Simply stated, many factors influence Product Backlog Ordering, not just what is considered most important from a business or mission perspective.

Non-trivial software solutions service many sponsors, stakeholders, customers, and end users.  Their interests and priorities differ from, and sometimes contradict, each other.  It is the job of the Product Owner (PO) to fuse the various needs, wants, and perspectives of that solution community into one complete, coherent, and cohesive product vision.

There are also project and technical realities that must be taken into account when deciding what portions of a solution to build and deliver and when.  Examples:

  • Environments (e.g., development, integration, test, and production environments) may not be ready for use at a particular point in the project schedule

  • Key system dependencies (e.g., features, components, libraries, frameworks) may be missing or incomplete, thus impeding progress

  • The customer may add or take away project scope during the project

Business or mission circumstances, solution community needs, solution requirements, and project priorities change as the solution takes shape through iterative and incremental delivery of system capabilities.  A PO must lead continuous refinement and ordering of Product Backlog items to align with such changes.

Five Factors that Influence Product Backlog Ordering

The most optimal ordering of Product Backlog items is a balance between these competing factors:

  1. Business/Mission Value

  2. Cost/Size

  3. Time Criticality

  4. Technical Dependencies

  5. Risk Reduction/Opportunity Enablement

Figure 1: Product Backlog Ordering Factors

Factor 1: Business/Mission Value

How Agile defines value is different from how traditional project management defines it.  A simple definition of value is whatever solution sponsors, stakeholders, customers, and end users care about or makes them happy.  We discuss Agile’s concept of value further here.

A feature’s business or mission value is the value created by implementing the feature (e.g., revenue, cost savings, customer retention, prospective customers, future opportunities, mission accomplishment).  Features that map closest to the product vision likely rank highest for this factor.[1]

Factor 2: Cost/Size

It is common Agile practice to use relative sizing to estimate how much of a development team’s capacity will be consumed by a particular Product Backlog item.  One way Agile teams perform relative sizing is by assigning story points to Product Backlog items. Story points are heuristically-determined, numerical proxies representing the combination of perceived or estimated:

  • User story complexity, risk, and uncertainty

  • Level of effort required to complete the user story

This combination is often referred to as the “bigness” or “size” of a user story. The development team estimates user story points by “sizing” stories relative to each other. More complex stories typically require more time and effort to complete, thus earning more story points than less complex stories.

Just as there is a correlation between story complexity and size, there is also one between story size and the cost of implementing the story.  While not a perfect correlation, it is good enough to support efficient ordering of Product Backlog items.

Factor 3: Time Criticality

How soon solution capabilities or system dependencies are needed also influences Product Backlog ordering.  Again, the goal of Agile is to deliver valuable capabilities, cost effectively, and in time to be valuable.  Delivering a capability before it is needed risks spending resources on something that may never be needed (YAGNY:  You Aren’t Gonna Need It!).  Delivering after it is needed risks delivering it too late to be valuable.

Agile achieves a “just-in-time” balance by constraining planning, development, and release scope to fit within relatively short time frames (iterations/Sprints/releases) and delivering the highest ordered items first.  By waiting until “the last responsible moment” to define, develop, and release solution functionality, we avoid over-engineering solutions or losing opportunities to provide value.

Factor 4: Technical Dependencies

Software solutions are inherently complex.  As such, their technical components often rely on other components to function.  For example, multiple features may require a database backend to store data and state.  Product Backlog ordering must take into account these dependencies as solutions grow and evolve.

Factor 5: Risk Reduction or Opportunity Enablement

Software system complexity introduces technical and project risks and opportunities.  Diligent backlog refinement spreads risk management actions across the entire software development effort.  For example, a team may choose to tackle riskier Product Backlog items earlier.  Another may identify an opportunity to shorten the schedule by accelerating delivery of a dependency common across multiple parts of the solution.  This differs from traditional project management efforts that, even when risks and opportunities are identified early, tend to accumulate risks, address them after they become issues, and miss opportunities.

Prioritization Techniques

Backlog ordering is the art of balancing these five factors to best facilitate Agile development and delivery of software solutions.  The three prioritization techniques briefly discussed in this section are not specific to Agile or Scrum.  They are well-known product management techniques. 

The first two, the Kano Model and the MoSCoW Method, have narrow perspectives and reduce business or mission value considerations to simplistic categories.  The third technique, Weighted Shortest Job First (WSJF), takes into account all five factors discussed in the previous section.

Kano Model

Professor Noriaki Kano developed the Kano Model in the 1980s.  The model categorizes product features according to customer needs and expectations.  In other words, what will satisfy or please customers?  Many versions of the model exist.  The original classifies items using five categories: Must-be, Attractive, One-Dimensional, Indifferent, and Reverse.

The Kano Model is most useful in organizations that create mostly “Must-Be” features: Features expected or taken for granted by customers.  However, products should include “Attractive” and “One-Dimensional” features to succeed in the marketplace.

Table 1: Kano Model Categories

Using this technique, the Product Backlog would include items from every category except “Reverse”.  Releases and Minimum Viable Products (MVPs) would include some set of items from those categories.  As MVPs grow and evolve, more items from across those categories would be added.

The problem with this approach is that it narrowly focuses on customer needs and expectations, which is one of many factors in gauging business or mission value.  However, Kano Model category definitions are useful in helping to shape thinking about the business or mission value of solution features.

MoSCoW Method

The MoSCoW Method, developed by Dai Clegg, has roots in Agile software development.  It is an acronym loosely based on the first letter of each of its categories: Must-Have, Should-Have, Could-Have and Won’t-Have.  The categories are defined from the point of view of what a product includes or offers vis-a-vis what customers or users want.  It answers the question, “What features will be included in the release or MVP?”

Table 2: MoSCoW Method Categories

Similar to the Kano Model, using this technique produces a Product Backlog that includes items from every category.  “Won’t-Have” items may either be postponed to future iterations or releases or dropped from the backlog altogether.

However, like the Kano Model, the MoSCoW Method focuses on one dimension of value.  In the case of the MoSCoW Method, it is determining which set of features best aligns with customer and end user expectations about the product/solution.

Weighted Shortest Job First (WSJF)

In his book, Principles of Product Development Flow: Second Generation Lean Product Development, Don Reinertsen explained the Weighted Shortest Job First (WSJF) prioritization model.  The model is used to sequence work in the most economically efficient way.

The folks at Scaled Agile Inc. explain a simplified version of Reinersten’s model here.  The simplified approach consists of two formulas:

Figure 2: A Formula for WSJF


Figure 3: Calculating Relative Cost of Delay

This simplified model is used in the Scaled Agile Framework (SAFe) to sequence (order) individual pieces of work or “jobs” (e.g., SAFe Features, Capabilities, and Epics) within their respective backlogs.

Unlike Reinertsen’s model, which uses monetary amounts to calculate Cost of Delay (CoD) and job duration, SAFe’s simplified model uses a relative sizing construct.  The construct uses a modified Fibonacci sequence as the basis for parameter values.  In calculating CoD, values for User-Business Value, Time Criticality, and Risk Reduction/Opportunity Enablement are assigned greater or smaller values relative to other jobs being ordered.  For job duration, “size” values are assigned to individual jobs.  Every job is sized relative to the other jobs being ordered.

We can create a simple table to track the relative values and calculate CoD and WSJF values:

Figure 4: WSJF Table

The following video provides more details on calculating WSJF values:

Calculating WSJF Values

The initial formulaic answer is a start to the ordering, not a final answer.  The computed sequence of Product Backlog items is adjusted to account for technical dependencies between them.

Teams may find it helpful to keep in mind the Kano Model and MoSCoW Method categories as they assign relative business or mission values to Product Backlog items/features and determine which items are ranked too low to include in the next Sprint or release.  Just remember that those categories do not represent the entirety of business or mission value.

We can use WSJF sequencing whenever Product Backlog we perform refinement.  Since Product Backlog refinement is a continual process, there will be future opportunities to apply WSJF.  Aim for “good enough” and trust that whatever issues surface later will be taken into account during the next round of Product Backlog ordering.

It is good practice to not let Product Backlogs grow too large.  Clearing items that may never be implemented is a good idea.  They can always be reintroduced later for ordering.  In the same vein, avoid ordering the entire Product Backlog.  Instead, focus on the next three Sprints or so.  Ordering items that may never rise high enough in the backlog to be implemented is wasteful.

Remember that Product Backlog ordering is an art, not a science.  As leads of this process, POs must leverage input from their Development Teams and solution community stakeholders and their own intuition and experience.

When Not to Employ WSJF

If the product backlog is small or mainly populated with user story-sized items, computation of WSJF values may be overkill. However, it is still advisable to take the five ordering factors discussed here into account when ordering product backlogs.

Leveraging User Story Maps

Scrum Teams can leverage user story maps to organize an ordered Product Backlog across solution workflow steps, Sprints, and releases.  Scrum Teams may use them in tandem with Product Backlogs to help identify dependencies as part of backlog refinement.  We explain user story mapping and provide an example here.

Capacity Allocation

The Product Backlog contains all of the solution requirements (e.g., epics and user stories) for which the Development Team may implement features.  If it is not in the Product Backlog, the Development Team does not implement it.  However, just because something is in the Product Backlog does not mean it will be done.

In light of this, it is often difficult for teams to balance delivery of business or mission value with work necessary to:

  • Implement back-end functionality or infrastructure that supports business/mission-facing functionality

  • Perform system maintenance to ensure overall system performance and quality

  • Address technical debt

  • Explore new requirements and technical approaches to address them

One way Scrum Teams achieve the right balance between different types of work is through capacity allocation.  Teams allocate their capacity to do work within an iteration/Sprint.  They allocate percentages of that timebox to specific types of work.  For example, a team may agree to devote most of the next Sprint to developing new business or mission capabilities, but allocate a percentage of that time to address technical debt and maintenance.  The team agrees to the types of work they will perform, and the percentages of the Sprint those work types will consume, based on current needs and constraints.

Conclusion

The key to effective Product Backlog prioritization is achieving an optimal balance between competing factors.  “Optimal” does not mean perfect. 

The WSJF technique provides a simple way of accounting for those factors.  WSJF is only a tool to help make decisions.  It is not a deterministic process that renders an exact solution. 

Arriving at a “good enough” Product Backlog ordering depends much more on effective collaboration between POs, development teams, and solution community stakeholders than on applying the techniques discussed in this article.  Ultimately, the quality of Product Backlog ordering depends on Agile’s empirical inspect and adapt approach to defining and executing work.

[1] Don McGreal and Ralph Jocham (2018).  The professional Product Owner: Leveraging Scrum as a Competitive Advantage 1st Edition. Scrum.org



Business Agility Depends on Culture

While most organizations would benefit greatly from attaining business agility, many do not possess cultures amenable to Agile transformation.

Organizations, of all types, are jumping on the “business agility” or “Agile transformation” bandwagon. Business agility is the ability to compete and thrive in volatile business or mission environments by responding quickly to changing market or mission circumstances.  Agile transformations involve the application of Agile and Lean principles to business or mission processes and decision making.  Organizations seek to emulate the efficiency gains and flexibility achieved through Agile software development, but in a business or mission context. Business (or organizational) agility is the goal of Agile transformations.

The value proposition for Agile transformation is compelling.  According to proponents of Agile transformation initiatives, organizations stand to become more competitive and thrive in increasingly volatile markets and mission domains by achieving:

  • Cost savings through greater efficiency
  • Faster responses to changing market or mission conditions
  • Better business performance or mission execution

Also driving growth in the number of Agile transformation initiatives is a growing appreciation for the need to align enterprise information technology (IT) investment, governance, and operations with Agile’s iterative and incremental delivery of software system capabilities.

Figure 1: Agile Transformation Taxonomy

These incentives have fueled a multi-billion-dollar Agile transformation services industry.  According to a report published by Allied Market Research in August 2019, “… the U.S. enterprise Agile transformation services market was pegged at $4.91 billion in 2018 and is expected to hit $18.19 billion in 2026 …”.

Despite all of the money spent (or, perhaps, partly because of it), many Agile transformation initiatives fail.  We can only estimate how many, but according to Jeff Sutherland, co-creator of the Scrum framework, 47% of Agile transformations do.

The Organizational Culture Challenge

Most Agile transformation initiatives failures are tied to organizational culture.  The 14th Annual State of Agile Report, published in 2020, supports this assertion.  Four out of the top five reasons, cited by survey participants, for failed Agile scaling initiatives are directly tied to organizational culture:

Figure 2: Challenges to Scaling Agile

For organizations that develop software systems, or sponsor their development, scaling of Agile software development and system operations is a crucial part of Agile transformation.  Cultural issues that frustrate their success also negatively affect business or organizational agility.  The first and third challenges, marked in red in Figure 2 above, explicitly call out organizational culture.  The second and fourth marked challenges point to lack of commitment and participation from organizational leadership and management.  Organizational leadership is responsible for setting the tone for the rest of the organization, so a lack of support from them translates into a lack of support for Agile transformation across the organization.

Addressing the Organizational Culture Challenge

As common as the “culture” challenge is, it is hard to tackle because it is so vague.  What are the reasons for “resistance to change” and how do they manifest themselves?  What is “enough” leadership and management participation and support and what do they entail?  Beyond performing the mechanics of a chosen Agile approach (e.g., Scrum, Kanban, SAFe), when do we know the organization has achieved an “Agile culture”?

Organizational culture influences behavior.  The inverse is also true: changing behavior changes culture.  Merely exhorting people to “think Agile” and “adopt Lean thinking” is a waste of time because they likely do not know how to do so within the context of the organization and their jobs.  On the other hand, engaging in a mechanical implementation of an Agile approach, without understanding its values and principles, guarantees that existing organizational cultural biases will corrupt the approach.

Organizational culture change starts with laying out a clear and compelling vision for the organization.  The organization’s chosen Agile approach becomes the means by which the vision is implemented.  An Agile transformation effort entails some degree of uncertainty and risk.  It will likely be the first time the organization embarks on such an effort.  Mistakes will be made and lessons will be learned.  This is why closely aligning to the chosen Agile approach and understanding its values and principles is so important.

The Agile approach serves as the foundation upon which organization-specific practices eventually emerge.  While the Agile approach likely requires some degree of tailoring to ensure it aligns to the constraints and needs of the organization, such tailoring must be informed by a deep understanding of the implications of each change.  It is best to explore ways of changing existing organizational processes to align to the Agile approach before tailoring the approach to fit the organization’s processes.

As people learn how to do their jobs, in alignment with the Agile approach, through training, coaching, and actual experience, their understanding of the approach’s values and principles take root and increasingly shift organizational culture towards a shared, agile-enabled vision.

Four Organizational Characteristics Most Likely to Derail Agile Transformations

Before embarking on an Agile transformation initiative, organizational leadership would be wise take time to reflect on how well positioned they and their organizations are to achieve a successful Agile transformation.  While it is true that the only way to know for sure is to try, an honest self-assessment enables leaders to approach Agile transformation with a clear-eyed understanding of what cultural obstacles they are likely to encounter. Leaders may come to the conclusion that their organization is not ready for the transformation.  In our experience, the organizations least likely to achieve business agility through Agile transformation share the following four cultural characteristics:

  1. They emphasize process execution over actual progress
  2. They emphasize control to the detriment of speed and flexibility
  3. They discourage experimentation and punish failure
  4. They possess a consistent record of failure

1. Emphasizing Process Execution over Actual Progress

Many organizations operate under the assumption that following plans and procedures ensures good outcomes.  While no organization can operate without them, many organizations lose sight of the purpose of plans and procedures: to accomplish business or mission goals and objectives.  When this happens, the following cultural dysfunctions arise:

  • People are rewarded for following plans and procedures, rather than for achieving desired outcomes
  • Failure is accepted so long as procedures were followed
  • Project status reporting tracks process activity completion rather than progress towards desired outcomes

One of the 12 Principles of Agile Software Development is, “Working software is the primary measure of progress.”  The only way to validate the value of any software capability is by getting it into the hands of customers and end users.  Only they can determine whether expected business or mission outcomes were achieved through use of the software. 

Similarly, the effectiveness of business plans and processes should be validated against business or mission outcomes.  Doing so involves quantifying expected outcomes and continually measuring their attainment against relevant metrics.  Failure to achieve desired outcomes should prompt process changes or the formulation of new processes.  This empiricism is a critical characteristic of Agile and Lean.

2. Emphasizing Control to the Detriment of Speed and Flexibility

The point of carrying out an Agile transformation is to achieve business or organizational agility.  Agile Cheetah defines agility as the combination of speed, control, and flexibility:

Agility = Speed + Control + Flexibility

The elements of this “Agility Triad” are mutually dependent constraints.  In other words, more or less of any one constraint influences the other two.  For example:

  • Accelerating the pace of work requires some degree of control and flexibility
  • Overemphasizing control slows progress and decreases flexibility
  • Underemphasizing control can turn flexibility into chaos, robbing teams of both planning and process stability, thereby slowing progress

Many organizations are guilty of the second bullet.  Overemphasizing control to the detriment of speed and flexibility is emblematic of fear-based organizational cultures.  In such cultures, Taylorism prevails.  Organizational structure tends to be hierarchical with a strong command-and-control management approach.  There is often an unwillingness to delegate decision making down to the most responsible levels.  In the most extreme cases, position, status, and tenure trump expertise, knowledge, and skills.

Source: geek & poke under CC-BY-3.0 license. No changes made.

Rather than accepting that some amount of risk is an inevitable part of solving tough problems, fearful organizations avoid taking risks.  Fear of failure produces cultures where failure to act is deemed preferable to acting and failing.  When failure happens, finding blame takes precedence over finding solutions.  Additional control mechanisms are put in place to ensure it does not happen again. 

Unfortunately, such mechanisms often feed a vicious cycle where people lose some autonomy and latitude to act, resulting in a loss of effectiveness, which breeds further failure, which then prompts further controls.  Left unchecked, this cycle can whittle down organizational effectiveness to the point where the organization loses the ability to reach previous levels of performance.

Figure 3: Vicious Cycle of Degraded Organizational Performance

Organizations with strong hierarchical and command-and-control cultures struggle through Agile transformation more than those with more egalitarian and permissive cultures.  Leaders should ask themselves; how much authority are they willing to delegate to achieve greater flexibility and speed?  What changes need to happen to enable their people to assume greater ownership and responsibility for their work and resulting outcomes?

3. Discouraging Experimentation and Punishing Failure

Organizations that overemphasize control also tend to discourage experimentation and punish failure.  Rather than exploring different solution options, they rush to settle on one approach.  Time that could have been spent investigating options is spent improving a solution that was likely suboptimal from the start.  Fear of ambiguity and risk leads to the formulation of unrealistic plans (In search for certainty, they conjure fantasies).  Premature commitment to plans takes away the flexibility needed to adjust to changing circumstances and new information.  Closed off to alternatives and fearing the consequences of admitting failure, the organization continues to sink resources into failing approaches.

As mentioned earlier, Agile and Lean are based on empiricism.  In an empirical process, information gained through experimentation and observation informs and validates plans, decisions, and actions.  Planning is adaptive as opposed to predictive.  In other words, Agile plans change based on current circumstances and new information and insights. 

Agile plans skew towards shorter planning horizons measured in weeks to months rather than months to years.  When problems or mistakes happen, Agile teams re-plan and correct course.  Short planning horizons afford the flexibility to make new plans and pivot to new solution approaches.

Like experiments, plans and decisions are treated as a hypothesis.  When a hypothesis proves to be wrong, another one is formulated and tried.  The faster teams cycle through these learning cycles, the faster they arrive at solutions that provide real business or mission value, cost effectively, and in time to be valuable.

4. Possessing a Consistent Record of Failure

Agile is not a silver bullet.  Agile, on its own, does not solve problems, it makes them plain for everyone to see so they may take action.  Transforming organizations from traditional project management structures to Agile requires courage, vision, and competence.  Organizations with long and consistent track records of underperformance and failure are most likely to fail at implementing Agile transformations.

Underperforming organizations may actually be missing core competencies that must be developed or acquired prior to embarking on any type of organizational transformation.  Such organizations are more likely to corrupt their chosen Agile approach to accommodate their deficiencies.

Conclusion

While most organizations would benefit greatly from attaining business agility, many do not possess cultures amenable to Agile transformation.  However, cultural change results from changing behavior.  Organizational leaders maximize the odds of successful Agile transformations by acknowledging organizational culture problems and addressing them before and during Agile transformation initiatives.