CardinalOps has released its State of SIEM Detection Risk report. The report, which analyzed 3,000 detection rules and 1.2 million log sources, found that SIEMs only cover 19% of MITRE ATT&CK tactics. This accounts for 38 out of the 201 techniques in the MITRE ATT&CK v14 framework. Yet, the report also found that organizations have the ability to cover 87% of the techniques. 

Key findings from the report include: 

• Multiple SIEM environments are becoming more common, as 43% of organizations have two or more SIEMs.
• 18% of SIEM rules are broken, often due to missing fields and misconfigured data sources.

Security leaders weigh in

Adam Neel, Senior Threat Detection Engineer at Critical Start:

“Looking at the state of SIEM from the angle of detection engineering and threat hunting it is interesting to see that 43% of organizations report having more than one SIEM. While there are certain use cases where having multiple SIEM tools could be beneficial (i.e. cost savings by sending a bulk of data to a less expensive SIEM and then forwarding the most important data to a more robust and expensive SIEM), it can also lead to a concern of complexity. 

“Managing integrations, rules, detections and automation for one SIEM can already be a daunting task. Adding on a second SIEM with its own idiosyncrasies that every team member must learn can lead to falling behind by trying to make sure that both SIEM tools are at parity with their capabilities and detections. If there is a critical threat, response times to create capable detections could be slowed since engineers will need to test across two different platforms rather than one. There is also a greater chance for misconfigured rules since engineers will need to learn syntax for both SIEMs, rather than focusing on mastering one.

“Another potential concern is related to threat hunting and investigation, tracking lateral movement could become much more difficult. If logs are split between two different SIEMs then it is possible to lose track of lateral movement if data from two impacted devices are going to separate SIEM tools. At the very least this could slow down investigation by having hunters jumping between both SIEM solutions.

“Making sure that detections are properly tested and working as intended takes a large effort. The reporting that 18% of SIEM rules are broken does not come as a surprise, it is easy for teams to enable rules that come with the product out-of-the-box with little testing. This can quickly result in SIEM rules that are enabled, but will never fire due to basic issues like misconfigured data sources. It is important for engineers to keep in mind that whenever possible, rules should be tested and confirmed to be working before relying on them to cover a MITRE ATT&CK technique. Improper detection testing can result in a false sense of security and false negatives that go unnoticed.”

Tamir Passi, Senior Product Director at DoControl:

“The findings in the CardinalOps report highlight a critical issue in the cybersecurity landscape: the significant gap between SIEM systems’ capabilities and the actual detection coverage they provide. This gap underscores a fundamental challenge for security operations centers (SOCs) worldwide.

“The rise in the use of multiple SIEM environments, reported by 43% of organizations, highlights that the data is not being utilized correctly. Fact is, SIEMs are too much of a Swiss army knife. This is why companies should be using purpose-built systems for detection such as SaaS Security Posture Management and Cloud Security Posture Management. These highly focused systems understand specific event data as well as the context the events were generated under. Context includes configuration and policy information, which can make a large impact on the detection level. While you can ultimately place event data into a security data lake and have multiple focused systems analyze the data from a single datastore, you will still need to have systems to understand and analyze the contextual information.”

John Bambenek, President at Bambenek Consulting:

“Detection engineering has often lacked thoroughness and efficiency for decades. Many rules, even those written by vendors, are ineffective against anything except very specific examples of attacks. Enterprises are often left to fill the gap from various security tools. What is worse is we often lack sample logs to test alerts and detections in a test environment so we’re left always trying to stop the last breach and not prevent the next one.

“It isn’t as intellectually attractive, but organizations need to focus their detection rules on foundational behaviors covered in MITRE ATT&CK instead of specific IOCs or TTPs that change between attacks. Almost every attack uses malicious scripting, phishing and privilege escalation so those should get a priority for generic but comprehensive coverage.”