Tool proliferation has created a data volume challenge that is making it harder than ever to understand the security risks that really matter, which in turn makes it harder to secure the software development and deployment lifecycle. As threats continue to mount, this challenge will only get worse unless there is a better way of separating relevant risks from the background noise. Fortunately, AI now offers a solution

Data, data, and more data

Cybersecurity threats are ever evolving, and to prevent them or mitigate their impact, cybersecurity pros must have real-time insight into what is happening in their environments. This requires the analysis of the security scan reports, log files and data sets produced by a very fragmented set of tools used by security professionals, DevOps, network administrators, compliance and governance teams and others. In larger organizations, users in each of these groups often adopt a variety of overlapping tools, generating dozens of reports and log files with information in different formats, creating tremendous complexity for the security professionals who must manually review them when hunting for the cause of a security incident.

In the past, this manual review was feasible if tedious. Today, it’s impossible. In addition to the array of DevOps tools, including the security tools themselves, and the logs produced by different systems and applications, reliance on the cloud means a new set of logs to analyze. In AWS, for example, there are load balancers, firewalls and other monitoring systems, all generating huge log files.

Typically, as soon as software code enters the merge process, tools looking at the code immediately start generating extensive reports. Then there’s the build process, the basic vulnerability scanning process, the software composition analysis process (including open source dependencies) and more before the code goes into production. Next, the cloud-based infrastructure hosting the application generates another dozen or so additional sets of logs.

So just one application can create gigabytes of data in a relatively short time. And most organizations deploy multiple applications and rely on continuous deployments, leading to massive amounts of log data that can reach cloud scale.

In this environment, it’s impossible for manual reviews to distinguish valuable signals from all that noise and make meaningful correlations that point toward root causes — let alone reveal how to respond. Thus, incident response takes far too long, giving attackers more time to do damage while creating a greater public relations nightmare when the incident is finally made public. 

AI to the rescue

AI is the perfect technology to solve this problem. A properly trained AI model can cut through the noise to identify patterns and small bits of data that hold the key to an issue. AI applications can also pull huge amounts of data from disparate sources to identify patterns and make correlations that would never be apparent during a manual review.

Specifically, AI could help security professionals:

1. Quickly identify where to look for a problem. For example, cloud environments have rules and best practices that should be followed (e.g., don’t leave ports open or run containers as a root), but various tools report errors related to these rules differently. AI can correlate the error messages from different tools and systems. 
2. Prioritize what’s important. For example, AI can combine and analyze data related to deployment configuration in conjunction with the Exploit Prediction Scoring System (EPSS), the Common Vulnerability Scoring System (CVSS) and Knowledge of Exploit Vulnerability (KEV), enabling security teams to quickly prioritize the threats they should address first.

Generative AI can also be used to make security assessments interactive, so the system can be queried in natural language, allowing staff with different backgrounds and skill sets to use the system effectively.

Is prevention possible?

A problem identified and fixed in a test environment never becomes a problem in a production environment. AI can be used early in the software development lifecycle to proactively identify obvious gaps in security or policy non-compliance, which if not addressed could become issues in production.

So yes, AI can enable some level of prevention. However, no matter how well trained a model may be and how many patterns it can recognize, unidentifiable threats will continually emerge because the possibilities are endless, and threat actors are hard at work on something new. This means the need for a fast, AI-powered reactive strategy will always exist.

Effective AI-powered software security risk detection is possible 

There is no technical challenge to using AI for software vulnerability detection. It’s a standard AI problem. However, AI works most effectively when models are trained on abundant data, and in this case, much of the data required for training is siloed in the hands of the various vendors. Each tool and each cloud platform has its own structures and types of messages they understand. Thus, vendors must make their data accessible via APIs, if they aren’t already doing so, enabling AI solutions to access, unify, and analyze all the available information.

The volume of security data will continue exploding for the foreseeable future, so using AI to improve software security isn’t just an option. It’s a necessity. And the sooner AI-powered solutions are delivered and adopted, the sooner security teams will achieve better overall risk management decisions, faster mean time to remediation and the ability to catch more problems before they cause harm.