Well-run organizations invest in information systems to achieve specific and measurable goals and objectives.  They determine the return on information technology (IT) investments by how much they lower cost, increase productivity, lower risk, and increase quality.  This expectation of business or mission value achieved for money spent carries over to how organizations conceive, develop, field, and maintain information systems.  Valuable information systems allow users to leverage technology in ways that improve how they work to accomplish organizational goals and objectives.

Improving this alignment between business or mission needs and information systems development has been the holy grail of analysts, engineers, and managers since the dawn of the Information Age.  Unfortunately, past engineering and management approaches actually widened the communication gap between business / mission stakeholders and software developers.  This state of affairs made information systems:

• More expensive to develop and operate;
• Take longer to develop than expected;
• Less likely to generate expected value after fielding; and
• More likely to be cancelled before initial fielding.

Over the last 20 years, innovations in how we develop and field software systems, of all kinds, are breaking down barriers to collaboration between business / mission stakeholders and development teams.  Arguably, the three most important innovations are Agile software development, DevOps, and Microservices Architecture.

Here, I discuss how the three approaches can work together to address the aforementioned shortfalls. We aim to foster tighter collaboration between business / mission stakeholders and development teams as we improve development and organizational agility.

The Agility Triad

Today, mature software development organizations can maximize development and organizational agility by integrating Agile software development, DevOps and microservice architectural practices.  Agile Cheetah defines agility as:

Agility = Speed + Control + Flexibility

We call this definition “The Agility Triad.”  The Agility Triad’s elements apply to organizational, process, and technological agility.  The three elements are mutually dependent and reinforcing constraints defined as follows:

DevOps and Microservices Contribute to Agility

DevOps and microservice architectures provide the means of implementing the Agility Triad’s elements.  DevOps serves as the Agile implementation of the Software Development Lifecycle (SDLC).  Microservices help align software deliverables directly to the business / mission domain and requirements.  They also help structure software development teams, their work, and what they deploy to directly address business or mission needs.

DevOps as the Agile SDLC

Agile projects perform the same SDLC activities as Waterfall projects.  The difference is that software teams execute the Agile SDLC as a continuous flow of activity rather than as a linear, one-time progression through time-bound phases.

Agile projects are feature-based.  In the context of Agile, features are cohesive and complete slices of system functionality capable of performing useful work from the perspective of customers and end users (human or automated). 

Throughout development, Agile teams define, develop, test, deploy, and release features.  Rather than waiting until the end of the project for a finished system, project stakeholders, customers, and end users gain access to released features throughout the project.  These features, individually and collectively, are valuable business / mission capabilities ready for use upon release.

Every feature goes through its own mini–Agile SDLC iteration.  Mature Agile teams leverage automation to continually test and integrate features into complete increments of releasable functionality.  Newly deployed and released increments work with all previous increments.  The system grows and evolves iteratively and incrementally.

At the project level, the Agile SDLC is essentially a “swirl” or “flow” of activity occurring concurrently across multiple features and increments.  The most mature Agile teams release so quickly and often that their releases approach a continuous flow.

DevOps is the implementation of this flow-based approach to software development.  DevOps is not an Agile concept, nor did it start with Agile.  However, it aligns perfectly with Agile because it reinterprets the SDLC in ways that support Agile’s iterative and incremental value delivery and team-based culture. It also aligns with Lean’s emphasis on achieving flow and promoting continuous improvement.

Ways DevOps Enables Agility

I now explain how DevOps enables the three elements of the Agility Triad.

DevOps Contribution to Project Speed

DevOps accelerates software delivery by leveraging modern software development techniques, tools, and technologies and by eliminating wasteful handoffs between development teams and operations staff.  This culture of automation and continuous improvement facilitates faster iterations through the SDLC for every feature developed.

DevOps automation goes beyond automating the testing, packaging, and deployment of source code.  Automated building and configuration of host environments is also a critical facet of DevOps.  This capability is termed, “Infrastructure as Code (IaC).”

IaC allows for the repeatable provisioning and configuration of host environments via execution of code.  Rather than incurring the time and cost of manually setting up environments, infrastructure automation platforms provision and configure virtual environments by executing scripts and implementing configuration files. 

DevOps Contribution to Controlling Projects

Faster and more frequent iterations through the SDLC support greater control of project scope, budget, and schedule through increased empiricism.  In other words, Agile teams and project management monitor progress, make decisions, and act based on lessons learned from previous releases.  The faster and more frequently teams iterate through the SDLC, the faster and more frequently they receive feedback that informs upcoming work plans (scope).

Project management can make decisions about the state of the project based, not on estimated levels of completion but, on information gained through actual deployment, release, and use of functionality.  This arms managers with the information they need to decide whether to continue the current course, update project plans, or terminate the project.  All along, teams deliver complete increments of functionality capable of providing value immediately.

DevOps Contribution to Project Flexibility

DevOps contributions to software project flexibility include:

• Faster and more frequent iterations through the SDLC that allow experimentation with different approaches and quickly pivoting away from failing or undesirable approaches.
• The ability to separate software deployments in production from releases, which enables:
  • Testing in Production – Teams can deploy and test new software features in production before making them available to users.  This allows teams to experiment with different versions of the same functionality and see which performs best.
  • Releasing on Demand – Rather than deploying and releasing on a fixed schedule, teams can deploy functionality at any time and make it available to users when they are ready to use it.

Monolithic vs. Microservices Architectures

Traditional software projects delivered solutions all at once.  In other words, software teams deployed and released all application components as one unit.  Changes to an application often required complete redeployment.  In microservices parlance, we refer to applications architected this way as monoliths.[1]

While architects and developers typically split monoliths into packages, modules, or components, interdependencies between those components make it difficult to develop, test, and deliver them separately.  In systems engineering terms, these system components are “tightly coupled.”

A microservices architecture, on the other hand, builds applications as sets of independently deployable services that represent key entities of a particular business or mission domain.  For example, an inventory system may include services that handle transactions associated with customers, invoices, warehouses, payment, etc.  All of the information / data associated with each of these entities belongs to its respective service.  A service needing information from outside the confines of its own entity (e.g., the warehouse service needs a customer’s address) must get it from the corresponding service.

Three-Tiered (Web) Applications

Since the 1990’s, the predominant architectural model for monolith applications has been the three-tiered (web) application.  This distributed architecture grouped together components based on where they fit within each of the following tiers:

1. Front End:  Implements the application’s user interface(s) or externally-facing interface(s)/API(s)
2. Middle Tier:  Performs the bulk of the application’s business logic
3. Datastore:  Manages application data and state

---

Microservices Cut Across Three-Tiered Architectures

In contrast, since microservices are end-to-end slices of business or mission functionality, each microservice includes just enough of the three tiers to perform its job. 

Individual microservices are not coupled to the internal workings of other services.  Thus, microservices are independent or “loosely coupled.” As long as the interfaces they expose do not change, internal changes to individual microservices will not ripple across the application or system.  Teams can deploy and release microservices independently, allowing applications to grow and evolve iteratively and incrementally. 

Features Map to Microservices

As stated earlier, Agile value delivery is feature-based.  This differs from the component-based delivery of functionality performed in traditional software projects.

In traditional software projects, management organizes teams to develop system components which they later integrate into complete solutions.  Until that integration happens, nothing of business or mission value exists.  Complete and cohesive functionality / features only exist after system integration.

System integration occurs infrequently in traditional software projects. This increases project risk.  Validation of an application’s actual business / mission value typically occurs towards the end of the project, when budget and schedule are scarce.

As end-to-end slices of business or mission functionality, microservices help development teams develop, test, and deliver applications, iteratively and incrementally, as features, from the start.  A feature may be satisfied by one microservice or through orchestration of multiple microservices.

Ways Microservices Enable Agility

Agile approaches, such as Scrum, help teams organize and manage their work as increments of functionality (features / user stories), developed within short timeframes (timeboxes), as part of frequent releases.  DevOps provides the automation and coordination required to continually and sustainably deliver features agilely.  Microservices are as close as we have come to directly implementing, deploying, and releasing requested features as features.

Microservices Contribution to Project Speed

Microservices accelerate software applications development and fielding by:

• Breaking down application scope into smaller and more manageable units of work – A concept of Lean is that smaller batches of work increase flow.  Microservices make it easier for development teams to manage their work through Agile and DevOps practices.
• Allowing customers and end users access to valuable features earlier – Rather than waiting until full system deployment to validate and use a solution, customers and end users gain access to useful functionality throughout development.  Lessons learned from released functionality help inform ongoing development.
• Allowing development teams to work in parallel – Development teams can work on different microservices in parallel with little worry about inter-service dependencies beyond the interfaces the services expose.
• Facilitating mixing and matching of technologies and technical approaches – Microservice independence also applies to the technology stacks and platforms used to implement them.  Teams can develop microservices in whatever technologies they are most comfortable with and still achieve service integration and interoperability.  Since microservices communicate via networks, they do not need to run on the same platform or under the same operating system.  This technology independence lowers the amount of learning and coordination required across development teams.
• Lowering the risk associated with fielding new functionality – Adding or replacing individual microservices is much easier and less risky than redeploying monolith applications.  This flexibility lowers outage times.

Microservices Contribution to Controlling Projects

Microservices help teams and project management control projects by:

• Facilitating Development Around Features – Agile teams control projects by managing scope.  Breaking up system scope into features helps teams manage their work.  Microservices support the iterative and incremental development and release of features.  This strategy affords teams multiple opportunities throughout development to evaluate progress and update project plans based on actual use.
• Lowering Inter-Team Dependencies – Agile software development teams are cross-functional.  Members possess the skills necessary to develop complete, coherent, and deployable features across short timeboxes.  This organization around features, rather than components, aligns with the tenet of building microservices that include all of the components necessary to perform as independently deployable services.  Organizing teams along the lines of the microservices they develop lowers dependencies across teams and associated integration risks. 

Microservices Contribution to Project Flexibility

Spreading application scope across individual microservices simplifies changes and promotes experimentation.  Teams can change their piece of the application without fear of affecting the rest. Thus, there is less resistance to trying out different approaches and settling on the most optimal.  For example, we can deploy different versions of the same microservice with different user interfaces in production, selectively release them to users, and evaluate them.  This technique is known as A/B testing.

Summary of DevOps and Microservices Contributions to Agility

I summarize the contributions DevOps and Microservice Architecture can make to agility below:

Conclusion

Proper integration of Agile, DevOps, and microservice architectural approaches can narrow the gulf between business or mission stakeholders and those who deliver software solutions.  We do this by improving development and organizational agility.  Under this construct:

• Agile planning and execution are supported by DevOps processes
• DevOps processes accelerate development, testing, packaging, and deployment of microservice-based applications
• Resulting applications are better aligned with Agile delivery and business / mission needs

The decision to adopt any of the three approaches is not be taken lightly.  There is no one-size-fits-all approach to making them work and few organizations make all three work well.  It may also be the case that any of three approaches are not right for your organization or specific project.

However, these approaches are firmly in the canon of modern software development practice.  It is worthwhile investigating how your organization can improve how it practices these approaches or begin adopting them.

[1] Sam Newman provides a comprehensive explanation of the ideas covered in this section in his book:  Monolith to Microservices, 2020, O’Reilly, Chp 1