No one likes being told how to do their job, especially by people less knowledgeable about the work than themselves.  One of the defining characteristics of knowledge workers is that they know more about the work they do than their supervisors.

Knowledge workers amass a wealth of education, training, and experience in their specialties.  Their success depends on how well they apply those intellectual resources to their work.  Knowledge workers do not merely produce tangible or quantifiable products and services. They help employers or clients achieve business or mission outcomes.  Thus, they usually enjoy much greater latitude in determining how they work than workers in traditional service or manufacturing sectors.  Software development teams are made up of knowledge workers.

Unfortunately, traditional project management culture is still highly influenced by Taylorist notions regarding work organization and worker management.  Too many organizations reflexively assume that management should determine how employees work.  This predisposition runs directly counter to the Agile principle of:

Build[ing] projects around motivated individuals, [giving] them the environment and support they need, and trust[ing] them to get the job done.

http://agilemanifesto.org/principles.html

Organizations possessing traditional project management cultures often adopt Agile in ways that largely ignore the perspectives and input of those most affected by the change: software development teams.

This article is the third of a four-part series about why so many developers are fed up with Agile and why so many organizations fail to achieve Agile’s much-touted benefits.  The common thread across the series is that what often passes for Agile is anything but Agile.

The other topics covered in the series are:

• The Never-Ending Agile Grind – Developers often feel overwhelmed by the constant pressure to produce under tight deadlines.
• Greater Visibility & Oversight with Less Control – The “radical transparency” of Agile approaches, such as Scrum and Kanban, is often exploited by management as an opportunity to micromanage development teams.
• Changing Expectations for Developers – Developers are moving away from functioning as “order takers”, focused almost exclusively on coding, to collaborating with product owners/managers, sponsors and stakeholders, and customers and end users to define, verify, and validate what they develop.  Many technical contributors find this level of direct collaboration difficult to manage alongside their growing list of technical responsibilities.

A Reaction Against Traditional Project Management

Agile software development started as a reaction against the decades-old, heavyweight, project management approach to software development commonly referred to as Waterfall.  It began as a collection of “lightweight” software development approaches implemented by a few like-minded, software industry thought leaders.

Realizing their approaches shared a common philosophy, they gathered at a ski resort in Utah in February 2001.  There they drafted “The Agile Manifesto for Software Development”.  Adoption of the Manifesto and its supporting 12 Principles spread quickly across the software industry.  That formulation of what constitutes software development agility helped popularize Agile and shape the ongoing evolution and refinement of Agile approaches. 

Agile was a bottom-up movement started by software developers.  As stated in the first sentence of the Agile Manifesto:

We are uncovering better ways of developing software by doing it and helping others do it.

http://agilemanifesto.org/

Viewed through the lens of management practice at the time, what these early Agilists advocated was disruptive; almost subversive.  They carved out space within which they could work without the distractions or frustrations of managerial micromanagement and organizational politics. They broke down organizational silos and included customers and end users in defining, verifying, and validating software solutions.  While Agile helped shift software and IT management culture towards becoming more cooperative and participatory, progress has been spotty.

Few organizations fundamentally changed their traditional management cultures to align with Agile.  While many digital native companies established during the first two decades of the 21^st century internalized Agile and Lean principles from the start, many (if not most) organizations did not.  Traditional project management biases still color how most organizations view Agile.  They perceive Agile as a new way of developing software rather than a new way of delivering value through software.  For those organizations, Agile is something developers do while the rest of the organization functions as before.

This fundamental mismatch between Agile’s bottom-up culture and top-down traditional project management culture is the primary reason so many Agile implementations go wrong.  It is also why those with the most at stake in adopting Agile often have the least say in its implementation.

Aligning Organizations to Agile Delivery

Many organizational leaders are unaware of the need to align their organizations with Agile’s iterative and incremental mode of value delivery.  In Agile, value is whatever solution sponsors, stakeholders, customers, and end users (i.e., the Stakeholder Community) care about or makes them happy.  This definition equates value with real business or mission value rather than the cost incurred in developing functionality.  Agile teams help organizations achieve predefined and measurable outcomes by delivering valuable software capabilities.

Agile teams determine what to build through negotiation and collaboration.  The Stakeholder Community is actively engaged with development teams in defining requirements and verifying and validating solution increments throughout development.  This way of working affords plenty of opportunities to inspect and adapt emerging solutions as they grow and evolve.  Frequent releases provide the most important validation of all: customer and end user acceptance of solutions and achievement of desired outcomes.  What teams learn along the way informs future development.

Organizations with strong traditional project management cultures are not organized to participate in this form of value delivery and empiricism.  They are organized to outsource the definition and development of software solutions to development teams after a period of requirements definition.  They then wait until the solution is finished to validate its value. 

Outsourcing Agile Adoption

When organizational leadership sees Agile as a process for developers to follow, instead of a culture change for the entire organization, they are more likely to “outsource” Agile adoption to Agile trainers, coaches, and consultants.

Many organizations spend large sums training developers on the mechanics of an Agile approach but fail to internalize the values and principles behind the approach.  The result are development teams that perform the practices of the chosen Agile approach without understanding how those practices foster agility.  Meanwhile, management continues to operate under Waterfall assumptions and expectations.  Organizations end up with crops of newly Agile-certified employees but no plans to apply what they learned towards achieving greater agility.

Many developers feel they do not get enough value from Agile trainers and coaches.  Some trainers and coaches lack enough understanding of the day-to-day work of software developers or the organizational environment to be effective.  Others are Ill-equipped to help organizations move beyond executing team-level Agile practices and establish Agile-based organizational cultures.

Another manifestation of organizational leadership outsourcing Agile adoption is hiring consultants to develop “Agile strategies.”  Rather than leveraging consultants to help them develop Agile strategy, organizational leadership pays consultants to develop their Agile strategy with little organizational input or understanding of the organizational culture.  When this happens, the resulting strategies tend to be long on theory and short on actionable objectives for the organization to pursue. 

“Hybrid Agile” (Scrumfall, Agilefall)

Many organizations struggling to transform from traditional project management to Agile decide to go for a “Hybrid Agile” approach.  “Hybrid Agile” is not a recognized Agile software development approach. It is an attempt to gain the benefits of Agile while avoiding internalizing and applying Agile values and principles. Rather than reevaluating existing processes to achieve greater agility, these organizations contort Agile approaches to fit within existing organizational culture. This leads to a tailoring of Agile misinformed by traditional project management biases.

Development teams working in “Hybrid Agile” projects experience the worst of both Waterfall and Agile.  Teams are expected to:

• “Plan” their work while being denied control over the amount of work they commit to
• “Self-organize” while having no say over team structure, tools, and processes
• Provide “estimates” that are treated as deadlines
• Document what they work in “Agile” productivity tools to keep management informed
• Develop what they are told to according to an “approved” design while expected to accommodate customer and end-user change requests

Proposed Solution

Developing valuable software solutions and products takes more than writing code.  Regardless of the software development approach followed, all projects are accountable to their target Stakeholder Communities and must provide business or mission value, cost effectively, and in time to be valuable.  Management and development teams have to support each other to be successful.

Bottom-up Agile adoption typically fails to foster organizational agility beyond development teams.  Top-down Agile adoption tends to lack in-depth understanding of the realities of software development and buy-in from development teams.  A “meet-in-the-middle” approach accounts for the strengths and weaknesses of the other two approaches.  The meet-in-the-middle approach entails:

• Preparing everyone to work agilely
• Hiring knowledgeable and experienced Agile trainers and coaches
• Hiring, educating, and mentoring developers for leadership roles

Prepare Everyone to Work Agilely

While Agile is inherently bottom-up, organizational transformations required to achieve agility are not.  Organizational leadership and management must play an active role in leading the transformation.  However, to do so in ways that do not crush agility, they must adopt a “Servant Leadership” model.  This starts with a recognition that the primary role of management is to empower those who deliver value to the customer and to create work environments and customer relationships in which value providers can succeed and flourish.

A big piece of leading the change involves training, coaching, and mentoring employees, at all levels and roles, to understand Agile values and principles and how to apply them to their work.  This may require different types of training/mentoring or individual study for different roles throughout the organization.  For example, coaching and mentoring for organizational leadership and management could be geared towards:

• Understanding the differences between traditional project management and Agile
• Learning and internalizing the tenets of Agile Servant Leadership
• Aligning organizations to Agile value delivery in ways that align with Agile values and principles

I stress the need for coaching and mentoring.  Setting people loose after a two-day Agile certification class and telling them to “be Agile” guarantees failure.  Most people outside of software development have not worked in Agile environments. Thus, they do not understand how an organization rooted in Agile values and principles would impact how they work.  Organizational agility is not about everyone organizing themselves as a Scrum team. Like any new skill, Agile thinking and practice must be honed to gain the most benefit.

Hire Knowledgeable and Experienced Trainers & Coaches

This point is obvious but warrants explanation: hire knowledgeable and experienced trainers and coaches.  They need experience in more than just teaching classes and holding Agile certifications.  The more experience trainers and coaches possess in developing software solutions or leading their development, the better. 

I temper this with the caveat that the best trainers and coaches possess subject matter expertise and instruction, facilitation, and communication skills specific to their role.  The most technical people are often not the best fit for teaching and coaching Agile.  What are needed are generalists who possess a mix of theoretical knowledge, relevant industry or mission experience, exposure to the work of software development, and strong interpersonal and communication skills.

Hire, Educate, Train, & Mentor Developers for Leadership Roles

Employers in technical fields tend to select out “soft skills” while overemphasizing technical skills.  They do this at their peril.  Agile depends on the development of self-organizing “high-performance” teams.

Agile teams develop leaders.  Hiring well-rounded people, who work well in teams, and developing their technical and leadership skills ensures development teams have the skills they need to self-organize and to build productive relationships with their Stakeholder Communities.

Conclusion

Reconciling Agile’s inherent bottom-up perspective to organizational change with the need for transformative leadership from above is difficult.  There are no one-size-fits-all approaches for doing so.  Successful implementation of any Agile approach requires deep understanding of Agile, the chosen Agile approach, the organizational environment, the Stakeholder Community, and the day-to-day realities of development teams.  It requires discernment and a willingness to make tough decisions.

Software developers are on the front lines of Agile transformations because they create solutions that enable business or mission success.  They have a huge stake in the direction Agile adoptions take because they are directly affected by the outcome.  Thus, they need and deserve a significant voice in their implementation.

In the next and final article of this series, I’ll discuss difficulties software developers face managing Agile and DevOps-driven changes to their role.

Last Updated on February 2, 2022