Month: October 2017

At Large Scale, DevOps is not Enough

For the large-scale, software-intensive, systems-of-systems integration projects typically funded by the Federal Government, DevOps alone is not enough. The Scaled Agile Framework (SAFe) fully incorporates DevOps while providing a more holistic approach to organizing multiple Agile teams into programs that live the DevOps culture.

I am astonished at the speed with which DevOps has gained traction across private industry and government organizations. This is especially surprising to me with respect to the Federal Government, which is often a (very) late adopter of new IT technologies and methodologies. Although, to my knowledge, no definitive studies exist that confirm this, DevOps seems to be gaining acceptance and adoption within the Federal Government at a faster rate than Agile did during the last 16 years.

The general consensus of when DevOps started is 2008, the year Andrew Shafer and Patrick Debois met over a talk on “Agile Infrastructure.” I worked my first Agile project as a developer that year. The Agile Manifesto was published seven years prior and adoption of Agile software development within Federal software development programs was in its infancy. Within private industry, Agile development was not as prominent as it is today, but it was further ahead in acceptance and practice and the momentum was definitely building. Many “Gray Beards” across the Federal Government and contracting industry were convinced that Agile was a fad and that it could not work within a Federal acquisitions environment (Many still believe this today!). As depicted in the chart below (source: “Agile by the Numbers,” Deloitte Insights, 5/5/2017), in 2008, roughly 17% of all major Federal IT projects were described as Agile or iterative. By 2017, that percentage rose to approximately 80%: a 370% increase.

Since the Federal Government does not yet collect information on the DevOps implementations it sponsors, I am making an anecdotal comparison. I started hearing government contractors pitching DevOps to Federal agencies in 2014. As of 2017, most, if not every RFI or RFP for Agile software development services I encounter include requirements for DevOps services. It took Agile roughly 16 years to become a standard practice for Federal software development efforts. Ten years after it was first proposed, a large number of Federal agencies see DevOps as desirable and necessary and are willing to spend significant sums of money to implement it.

I suppose this relatively swift embrace of DevOps by the Federal Government should not surprise me. DevOps is an easy sell because of three main factors. First, Agile laid the groundwork for acceptance of DevOps. The second factor is a realization across the Federal Government that iterative and incremental software development approaches are superior to traditional monolithic (“Big Bang”) development and delivery. Third, the shift to cloud architectures enables and encourages DevOps implementations. Agile and cloud adoption serve as catalysts for DevOps adoption. In addition, DevOps has a compelling value proposition: Increased opportunity for efficiencies, faster and more frequent deployments, and tighter collaboration and alignment between developers, operations, and organizational sponsors.

While DevOps is, indeed, a game changer, for the large-scale, software-intensive, systems-of-systems integration projects typically funded by the Federal Government, it is not enough. Despite DevOps’ core emphasis on developing a collaborative culture between development, operations, and across the organization, there is little guidance on how to do that as part of a DevOps approach. The Scaled Agile Framework (SAFe) fully incorporates DevOps while providing a more holistic approach to organizing multiple Agile teams into programs that live the DevOps culture.

SAFe Approach to DevOps

The Continuous Delivery Pipeline is SAFe’s DevOps approach. This approach tightly integrates with SAFe’s concept of the Agile Release Train (ART). The ART is a cross-functional team/organization of 50 to 125 people that delivers technical capabilities (i.e., software, hardware, firmware, systems). The ART is a “virtual” or matrix organization in the sense that it brings together people who, under traditional software program management practices, are typically siloed apart from each other: Engineers, testers, security, enterprise architects, configuration managers, release managers, program managers, product managers, operations, maintenance, business stakeholders, etc.

The ART aligns very well with the DevOps concept of collaboration. However, collaboration in an ART goes well beyond the work performed by developers and operations. It includes representation from all stakeholders who affect or who are affected by the ART. SAFe also aligns program and portfolio-level concerns with the work performed by Agile teams. DevOps emphasizes the need for collaboration at all levels, but there is next to no guidance on how to foster it.

An ART fulfills the IT capabilities needs of a value stream. A value stream is the chain of processes, systems, and people that that delivers business value (goods or services) to a customer. As an example, consider the steps required for a delivery company to deliver a package. Since value streams operate for as long as customers procure value from them, value streams, and the systems that support them, are long-lived. Just like the flow of value, the flow of IT capabilities is continuous: unencumbered by wasteful handoffs, reviews, queues, delays, and project starts and stops. For a given value stream, developers and operations personnel apply DevOps concepts and practices as part of an ART. This emphasis on eliminating waste and promoting flow aligns with DevOps philosophy and practice.

With respect to SAFe’s Continuous Delivery Pipeline, the ART performs three interlocking and reinforcing activities: Continuous Exploration (CE), Continuous Integration (CI), and Continuous Deployment (CD). The goal of CE is to develop a program-level vision, roadmap, and backlog that the ART fulfills by delivering features. As the name suggests, CI encompasses the activities that develop, integrate, and test prioritized features resulting from CE activities. The CD process deploys validated features into the production environment where they are further tested and prepared for release. SAFe’s CI and CD concepts closely align with the same-named concepts in DevOps. Automation is a critical enabler of CI and CD activities in both DevOps and SAFe. However, SAFe goes further by enabling the creation of portfolio and program-level ecosystems that align with DevOps.

Bottom line: Like Agile, attempts to solve problems and improve performance by applying new technology and practices, while ignoring the cultural and organizational prerequisites that support those approaches, has colored many DevOps implementations. Also like Agile, DevOps is not solely or even primarily about tools, technologies, and technical practices. It is a mindset. Rather than merely exhorting practitioners to change their organizational culture to one that is more collaborative and integrated, SAFe provides a framework that enables the creation of such a culture. SAFe enables and enforces a balance between DevOps technical practices and the cultural and organizational behaviors and structures that enable its success.



Last Updated on February 2, 2022

Five Things You Must Understand about Agile Team Velocity

Agile team velocity is an often misunderstood concept that is prone to misapplication. This article sets the record straight.

Agile software development is the way we build software now. The general consensus on management of software development projects is that predictive planning approaches (e.g., Waterfall) are ill suited for the job. However, almost two decades after the release of the Agile Manifesto and the rise of Scrum as the most popular Agile framework, team velocity is still a point of confusion.

Some organizations are under the mistaken idea that they can “buy” Agile-Scrum software development services à la carte based on team velocities. Under this contracting model, service vendors offer prepackaged teams rated at some velocity. Based on that velocity, the organization could then pay for a certain amount of that team’s time to complete predefined tasks. This arrangement treats Scrum teams as plug-and-play units of development capability. This mentality is evidence of a fundamental misunderstanding of Agile and Scrum, of what teams require to be effective, and of the concept of team velocity itself.

What is Team Velocity?

The concept of team velocity is not officially part of the Scrum framework. It is a best practice for estimating Development Team throughput. More specifically, it is the rate at which the Development Team, in collaboration with the other members of the Scrum Team (i.e., the Product Owner and ScrumMaster), delivers discrete items of functionality that are finished, tested, and accepted during an iteration/Sprint. The Scrum Team documents requirements for those items as User Stories and tracks and orders User Stories in the Product Backlog.

Rather than simply counting the number of completed stories at the end of a Sprint, the Scrum Team determines its velocity by adding together story points from every story completed during the Sprint. Story points form the basis for determining team velocity. Calculating team velocity allows Development Teams to estimate the amount of capability (value) they can deliver at a sustainable pace during a Sprint.

Σ Story Points of Completed User Stories During a Sprint = Velocity

It is important to view team velocity results as data points along a trend, rather than as definitive stand-alone measures.  In other words, to understand how much value a Development Team is capable of delivering, we need to look at velocity results from the previous two or three Sprints to get a sense of whether the latest result is part of a trend or an anomaly.  Such anomalies often indicate issues requiring attention.

Like the concept of team velocity, story points are not part of the Scrum framework. The Development Team assigns story points to Product Backlog items (i.e., Epics and User Stories). Story points are heuristically-determined, numerical proxies representing the combination of perceived or estimated:

  • User Story complexity, risk, and uncertainty
  • Level of effort required to complete the User Story

This combination is often referred to as the “bigness” or “size” of a User Story. The Development Team estimates User Story points by “sizing” User Stories relative to each other. More complex User Stories typically require more time and effort to complete, thus earning more story points than less complex stories.

Story point estimation is not restricted to specific timeframes or meetings. It is part of a continual refinement of Product Backlog items by the Scrum Team that includes:

  • Adding, updating, or deleting items
  • Decomposing items into smaller items
  • Adding details and estimating story points
  • Ordering/prioritizing items

Five Key Team Velocity Concepts:

1.  Only the Development Team estimates story points with input from the Product Owner and other relevant stakeholders. They are not determined by a chief engineer or some other senior authority as an estimation exercise prior to the establishment of Development Teams and the appointment of Product Owners.

2.  Team velocity is specific to the team.  Velocity cannot be used as a point of comparison between teams.  Team X is not necessarily faster or able to attain a higher throughput than Team Y because they earned a higher number of story points during a Sprint.  The story points upon which team velocity is based are specific to the relative sizing performed by the team on their particular backlog of User Stories.

3.  A team’s velocity is the end result of many factors, including:
  • Individual team member productivity across the team
  • Team cohesion, collaboration, and effectiveness
  • Team skill set and experience mix
  • Level of programmatic support (i.e., the factors that fall under the purview of program management)
  • Empowerment and level of engagement of Product Owners
  • The team’s familiarity with the business domain, technology mix, and enterprise architecture
  • The amount of automation leveraged in building, testing, and deploying code

Thus, the makeup of the Scrum Teams and the environment or ecosystem within which they operate determine velocity.  Team velocity is ultimately a proxy for the overall health of the software development program and organization.  When team velocity stagnates or drops, the search for potential causes must be systemic as opposed to a hunt for individual culpability.

4.  It is normal for team velocity to fluctuate.  What should concern us are large fluctuations or downward trends in velocity.  Changes in personnel, technology, organizational structure, etc. will affect development teams in different ways and to varying degrees.

5. Improving team velocity is not the goal!  Improving team capabilities, practices, tools, and program support improves team velocity.  Focusing on improving velocity results in attempts to game the system rather than systemic improvement.

Conclusion

Scrum Development Teams are composed of people. People are not machines calibrated to deliver a predetermined rate of output. Teams need time and stability to gel into highly effective teams. Helicoptering Scrum Teams, even experienced ones, in and out of tasks significantly degrades velocity except perhaps for the most mundane and menial tasks (In which case, why employ Agile/Scrum at all?). Decades of reducing people to resources allocated across Gantt charts has led many to also think of Scrum Teams as abstract productivity units. Scrum requires much more of us.



Last Updated on February 2, 2022