Attackers Work With the Tools Defenders Already Trust

CISA and its partner agencies reported in February 2024 that Volt Typhoon had held access inside some U.S. critical infrastructure environments for at least five years, working through valid accounts and built-in system utilities rather than malware. Every action was permitted, so recognizing the intrusion meant examining how the actions were related. 

Security investigations are fundamentally contextual. Attackers frequently use legitimate tools, valid credentials, and native administrative functionality, so a Kerberos ticket request or a remote management session can be entirely normal on its own. The risk becomes visible when those events are evaluated together and placed in context: user, device, privilege level, timing, location, and the normal behavior of the environment. That is especially true in identity security, where administration and abuse are separated by intent and scope rather than by a malicious executable.

Speed Matters Because Analyst Attention Runs Out

AI is well suited to working across that context. It can process information from multiple sources, identify relationships and anomalies, and present related activity in a form that is easier to investigate, which helps a team prioritize what deserves human attention. An experienced analyst may eventually reach the same conclusion manually, though that review relies on the analyst's skills and access and competes with every other signal in the queue including speed/timing.

However, that help AI can provide must meet two conditions: conclusions must be reproducible, and they must remain traceable to the underlying evidence. A confident answer with no supporting data behind it amounts to a well-written guess.

Start With One Investigation and Define the Boundaries Up Front

Often, these conditions are easier to design into a security framework than to retrofit, which argues for starting small. Organizations should begin with a narrow, well-defined use case rather than attempting to build an autonomous AI security platform. A good initial use case might be summarizing identity-related incidents, correlating authentication activity across several data sources, or helping analysts investigate changes involving privileged accounts. The organization should define exactly what data the system can access, what questions it is expected to answer, and what actions it is prohibited from taking. 

Containment and remediation belong behind a human approval step, as there is a major difference between suggesting that an account should be disabled and actually disabling it in production. These systems need the monitoring, testing, and change management any other security-sensitive application gets.

Two Kinds of Expertise Must Be in the Room

This type of build takes expertise from both the AI team and the security team, and neither of them should treat the other as an implementation detail. AI specialists understand model behavior and its limitations; the security and IT staff who own the environment understand the operational consequences of getting a decision wrong. Structure the effort around real security scenarios instead of broad goals such as "use AI to improve detection": give the team a specific investigation and ask what information an analyst would need to reach the correct conclusion.

Review failures jointly. "The model failed" covers three faults with different owners: incomplete data, a prompt that encouraged speculation, or flawed security logic. The output may be generated by AI, but the risk belongs to the organization.

Measure Outcomes, and Test With Dirty Evidence

Whether the investigation actually improved is measurable and should be examined. Baseline the current process first: the time an investigation takes, the number of data sources an analyst needs to review, and how often incidents are escalated incorrectly or important signals missed. Then hold the assisted process to those same criteria, adding analyst acceptance of recommendations, false-positive rates, and the  conclusions fully supported by cited evidence.

Hallucination and omission rates belong in the test plan, and the test set should include known incidents, ordinary activity, incomplete telemetry, and deliberately misleading signals. A system that performs well only when the evidence is clean and complete will not perform well in a real enterprise.

A Confident Explanation Widens the Attack Surface

To make the most of your AI use in investigation, you should remember that it still makes mistakes. Generative AI can produce explanations that sound remarkably similar to those written by an experienced security professional, which creates a unique risk. A traditional security tool may produce a false positive; an AI system can produce a false positive wrapped in a persuasive narrative explaining why it must be correct, which invites a team to trust the explanation before validating the evidence.

AI therefore deserves treatment as both a security capability and a new part of the attack surface: its data sources can be manipulated, and attackers may generate activity specifically to shape what it concludes. Research published in 2023 showed that instructions planted in content a model later retrieves can redirect its behavior, and OWASP now ranks prompt injection as the top risk to LLM applications. In a security workflow, the retrieved content includes log fields, ticket text, filenames, and user-agent strings, several of which accept attacker-supplied input.

Authority Should Follow Verification

Organizations should resist granting AI more authority simply because its output is fast and confident. Define which conclusions a person signs off on and how any conclusion can be checked against the records behind it. Handled that way, AI makes a security team significantly more effective; trusted solely on the strength of its prose, the same system helps a team reach an incorrect decision more quickly. Evidence, testing, and governance separate the two.