Shaping and Implementing DevSecOps Culture in IT Environments

A Little Background: DevSecOps is About People and Culture

From a definition standpoint, DevSecOps is the union of software development (Dev), security (Sec), and operations (Ops). It is a set of principles and practices that aim to integrate software development and software operations by facilitating a culture of collaboration and shared responsibility.

In the 2021 State of DevOps Report, Patrick Debois, advisor to the open-source security platform, Snyk, stated, "Models such as The Three Ways of DevOps, CAMS, and CALMS all emphasize that while DevOps was made possible by automation, programmable infrastructure, and more accessible programming languages and APIs, it is fundamentally a human-centered movement, focused on improving the interactions between people."

One reason why MetroStar has successfully brought DevSecOps transformation to multiple enterprise IT missions across the Federal government is because of our understanding that DevSecOps, at its core, is more about people and culture than it is about the tools. DevSecOps is a treasure chest of opportunity for an organization, and the principles you find within the chest make it possible.

DevSecOps as a Culture Sounds Nice, but What Does that Really Mean?

You may have heard DevSecOps has cultural implications, but what does that mean? For starters, DevSecOps culture is a separate concept from the technologies and practices that arise from it. DevSecOps culture exhibits traits of shared responsibility, bridging friction caused by silos, and supporting fully empowered product teams.

It is more important than ever to invest in DevSecOps concepts and culture—your agency's mission may fail without it. In a 2019 study, Gartner predicted that through 2022, 75 percent of DevSecOps initiatives would fail to meet expectations due to issues around organizational learning and change.

How Can You Shape and Implement DevSecOps Culture?

DevSecOps culture is explained through the lens of the relationship between leadership and product teams. When it's understood that leadership ultimately cares about outcomes, and product teams ultimately care about delivering value to the end-user, then we can begin to construct a cohesive organization that supports these desires synergistically. To do so, organizations must break down team silos, trust teams with the responsibility of outcome over output, track the right metrics, and instill DevSecOps principles across different teams.

Culture Change Starts at the Top: Fostering a Culture of Trust and Shared Ownership

As Todd Alexander, Director at the Leadership Research Institute, writes in this publication, an organization's culture is not merely written down and socialized; it is forged through actions.

Cultural transformations are a journey that both leadership and their teams must walk, but leadership must take the first step.

Trust is essential between leadership and their product teams and within the product teams themselves. A lack of trust in an organization leads to excessive processes and bureaucracy that calcifies the value chain and prevents the easy flow of communication.

What can leadership do to build a culture of trust and shared

ownership with their product teams?

Break Down Team Silos by Creating Cross-Functional Teams

• By building cross-functional and self-sufficient teams, leadership is sending the message that we're all dedicated towards the same end goal despite our differences in roles.

Trust Teams with the Responsibility of the Outcome Over Output

• When teams are introduced to the reasons and thought processes behind a set of activities or decisions instead of just hearing what the outputs are, leadership can help provide direction, motivation, and clarity to their teams by showing them the why of outcomes.

Find True-North Metrics to Track Outcomes

• To measure the organization's success, choosing metrics that value outcomes over output may help team members articulate and attain the most important goals for the organization.

Take the Journey Together by Working with Teams

• Transitioning into a trusting DevSecOps culture may feel as daunting to leaders as it does for engineers. Some teams will embrace change more efficiently, while others will struggle. Continuing to lean back on DevSecOps principles, even when the going gets tough, may turn missteps into opportunities for growth.

Break Down Team Silos by Creating Cross-Functional Teams

Developers are not necessarily trying to pull the wool over a QA team's eyes. They are often trying their best to deliver value, and they want to know what's wrong with their code and why it might not be working in production. They typically want to know if an end-user is facing an issue, so they can fix it quickly.

One of the best ways to break down team silos is to embed testers within development teams. Quality Assurance (QA) teams can still properly audit the work of the developers while working as a cohesive team. Separate testing and development teams send the message that developers have certain self-interests that would conflict with organizational goals if their outside check was removed. Thus, they cannot be trusted to collaborate. DevSecOps culture is about helping teams do what they naturally want to do—build awesome things for end-users.

Trust Teams with the Responsibility of the Outcome Over Output

Sometimes it is worth asking, are you just twisting balloons or delivering a room full of happy children? Understanding the difference between the output of balloon animals (what you are doing) vs. the outcome of children having fun (why you are doing it) can be an "ah-ha" moment for some organizations.

Leadership can create a sense of stronger ownership from the teams when those teams are more aligned with outcome-focused goals over output-driven ones. One way to empower product teams is to trust them with the responsibility of the outcomes. After extending this trust, product teams may meet leadership halfway and earn deeper trust by displaying ownership of the end user's experience through automated testing and deployments.

An organization's culture may dramatically shift when leadership extends trust and product teams deepen that trust by delivering on outcomes:

• Suppose leadership can cede some control of the minutiae of the automation pipeline while aligning teams with outcomes rather than output. In that case, a positive shift may occur in the team's incentives and expectations, increasing their freedom to solve problems creatively and effectively.

• When product teams show they can manage the potential risks of deploying to production (e.g., by implementing feature flags or the ability to roll back releases), leadership will feel more comfortable knowing their trust was well placed and mission outcomes were achieved.

This shift is sometimes difficult for leaders who are used to pulling output-driven levers, which is why trust is a necessary ingredient for successful DevSecOps.

That said, trust alone does not bolster a DevSecOps team. To be clear, when product teams are given the responsibility of owning outcomes it is not necessarily a green light to say automation is a "Wild West" free-for-all. Certain requirements for releasing to production, automated testing, and the ability to roll back releases should still be a part of the picture.

Find True-North Metrics to Track Outcomes

While it is certainly helpful to track the output of how many dog balloon animals were made vs. sword balloons, this might cause you to lose track of what's more important: following the overall outcome of how many children had fun!

A significant part of DevSecOps is using metrics to gauge if teams are moving in the right direction. While many companies are implementing KPIs and OKRs, picking metrics that value outcome over output may help leadership create the desired guardrails that bring clarity to the teams trying to deliver value.

By 2013, the State of DevOps Report had established the relationship between a true DevSecOps practice—the combination of people, methods, and culture—and high-performance results. Organizations practicing DevSecOps consistently report:

• More frequent deployments

• Shorter lead time to change (LTTC)

• Lower change failure rates

• Faster mean time to recovery (MTTR)

These four metrics don't encompass all DevSecOps, but if you're looking for high-level metrics to begin baselining, these are a good start. They illustrate the measurable, concrete benefits of pairing engineering expertise with a focus on minimizing friction across the entire software development lifecycle.

Take the Journey Together by Working with Teams

Transitioning into a trusting DevSecOps culture can be radical and feel as unfamiliar to product teams as it does to leaders. Inevitably some teams will embrace change more readily than others and knowing how to handle failures gracefully is just as important as knowing when to celebrate successes.

While struggling teams should still be held accountable for the outcomes they might fail to achieve, it can be helpful to approach situations with a mentality of assuming positive intent.

Most teams are trying their best to succeed and are not trying to fail. When an organization's culture builds itself around encouraging honesty and accepting failure rather than head-hunting for blame, teams are given the opportunity to learn and grow from their mistakes rather than becoming defensive and compounding the problem.

For teams that successfully embrace change—those champions become the exemplars. When teams reach the desired goals, it's a good idea to pause and shine a light on what went right. Successful implementation of certain tools or processes may be replicated and made available to all; struggling teams can see who the go-to people are for assistance. The success of one team becomes the success of all.

Applying DevSecOps Across MetroStar

MetroStar has built a growing DevSecOps presence in the Federal space and continues to expand our capabilities rapidly. Through our understanding of Lean and Agile principles, systems thinking, ITIL, and the social and psychological beliefs that motivate people, we have successfully used DevSecOps to bring meaningful change for the nation's leading data provider and the nation's leader of domestic fiscal and financial policy.

Our engineers are DevSecOps change agents who combine their technical knowledge of various automation tools and CI/CD pipeline best practices with an intrinsic understanding and motivation to shift an organization's culture through their daily work and interactions with others.

Ultimately, anyone can be a DevSecOps change agent when they inspire the organization to break down silos and foster a culture of collaboration, shared ownership, and trust.

Interested in working with our DevSecOps Change Agents? Reach out!