DevSecOps: More Work Ahead

DevOps has come a long way since it got underway in full force nearly ten years ago.  As was recently made clear at this year’s DevOps Enterprise Summit (DOES) in Las Vegas, DevOps organizations have been successful when it comes to knocking down organizational silos, optimizing the delivery of software services and functionality, and shortening the time it takes to deliver digital value to customers. DevOps organizations are delivering better business outcomes.

However, one area where there’s still much more work to do remains the stubborn area of software security.

Based on the presentations I attended and my discussions with DevOps practitioners at DOES, it’s clear that most mid-sized and large organizations are running multi-cloud and on-premises environments, and they’re all in when it comes to virtualization, containerization, and microservices.

While all of this is helping to speed the delivery of software services, it is also increasing management and security complexity in some ways.

Organizations are taking steps to help better secure their systems. In one talk I attended, DevOps for the Enterprise: 2018 Trends and Insights, Wesley Pullen, chief strategy officer at Electric Cloud said that they are seeing enterprises embrace artificial intelligence and machine learning as ways to identify patterns using in-depth statistical analysis, predict future risk using previous patterns, and to provide recommendations to reduce risk, Pullen said.

The conversations at the summit reminded me of the findings from this year’s Sonotype report, DevSecOps Realities and Opportunities, released this spring. "While some DevOps teams are starting to incorporate application security into their CI/CD workflows, driven by factors such as improved software quality, compliance, and risk avoidance, there is ample room for improvement," Jay Lyman, principal analyst at 451 Research said when the report was published. "In many cases, security testing is not being integrated often or early enough in the process for organizations to fully benefit from reduced risk and rework headaches," he said.

The report is based on the survey results of 350 decision makers from large enterprises across numerous industries.

The study found half of CI/CD pipelines are equipped with application security testing. It’s not as if respondents don’t know better, in the survey they themselves stated the importance of application security.

What are some of the holdbacks to successful DevSecOps? The DevSecOps Realities and Opportunities report found lack of automation and consistency to be big drivers. That’s no surprise, and security automation is a subject we’ve been pounding on here for some time. See our previous posts on this here and here. The TL;DR takeaway from those posts is the only way to keep up with the required software tests is to automate as many tests and processes as can be reasonable automated.

When it comes to moving testing into continuous delivery, another common challenge that has been known to be a bugaboo in security forever: false positives. When teams move automated testing tools earlier into the software development cycle, they often get plagued with false positives. The key here is to get developers trained in application security and to have application security experts embedded early in the design process. Having testing directly in the integrated development environment can also help developers to spot flaws early. It may make sense to also play triage to an extent, so that low level and easily fixable defects move forward so they can be remediated later.

The Sonotype’s DevSecOps survey confirmed that false positives are a serious challenge with 46 percent of respondents. The report stated that “the noise of false positives can drown out the benefits of security scanning and other elements in CI/CD processes. We believe organizations can help address this issue by choosing security software and services that specialize in effectively reducing false positives and the noise that comes with them.” The report authors cited static application security testing be tuned to every organization’s unique environment to reduce false positives.

Another interesting area that the DevSecOps Realities and Opportunities report found that known at-risk open source software components being installed is a critical application security challenge in continuous pipelines. Surprisingly, 40 percent of organizations either don’t look for vulnerable open source software or they think they aren’t using any open source components. The latter is highly unlikely as the vast majority of software utilize open source and this is especially so in CI/CD environments.

What was also interesting from the conference in talking with various organizations is that DevOps may have a foothold in organizations, it’s not evenly distributed — even if the benefits of DevOps have been realized. This, too, was reflected in the DevSecOps Realities and Opportunities report. Of its survey respondents, “36 percent reported developer/administrator teams were focused on continuous integration. Another 35 percent of respondents stated teams focused on continuous delivery, and 35 percent also reported a DevOps focus, although DevOps was ranked lowest in most industries, including traditional retail, SaaS and healthcare.”

Fortunately, the survey also found that one of the big drivers of DevSecOps and the integration of good security practices and testing into CI/CD processes. The survey found that that software quality was ranked highest at 75 percent of firms indicating its importance. Software quality even beat out compliance and regulatory requirements, which ranked 68 percent. Interesting, risk avoidance over ranked at 64 percent while 48 percent believed that application security testing can slow down the release process.

The challenges reflected in the survey have been reflected in related surveys. Enterprises know the benefits of DevOps, they understand that security needs to be integrated into their processes, and they have yet to properly do so. This was also what was largely reflected in my discussions this year at the DevOps Enterprise Summit, as well. Some may find this state of affairs disheartening, especially after a focus on the importance of web application security for nearly 20 years. But I don’t.

Still, I see hope. I think organizations are on the right path to achieving healthy states of DevSecOps. They’re learning how to deploy the right tools and processes to maintain the right balance between the speed of software delivery and the level of security that software needs to be compliant and resilient form attack.