Threat hunting involves actively searching for threats instead of waiting for alerts from defense systems. It is about taking the initiative to find irregularities and deviations in computer systems. Often, there are no alerts to signal an intrusion. Threat hunting targets threats that have already bypassed automated detection systems. It facilitates the identification of attackers who have penetrated the infrastructure and are practically indistinguishable from legitimate users by standard security measures.

Threat hunting complements automated threat detection tools by closing potential gaps in the security system and expanding the monitoring area. It also helps identify complex or hard-to-detect threats.

15 to 20 years ago, comprehensive tools for information security did not exist, so the focus was on monitoring events through individual security tools. If something deviated from the norm or a new Indicator of Compromise (IoC) emerged, it would trigger an investigation. In fact, this early practice was a form of threat hunting.

Threat hunting can be initiated from specific data obtained by a specialist, or it may start from a hypothesis. If testing the hypothesis yields a positive result, it can contribute to improving threat detection mechanisms. Threat hunting comprises several components, including preliminary research and automation. The direct application of this knowledge involves a manual search for anomalies.

Threat hunting approaches

• In an unstructured approach, a threat hunting expert examines the normal operations of the infrastructure to spot any unusual activities or deviations.
• A structured approach involves studying the attack patterns, tactics, techniques and procedures specific to certain Advanced Persistent Threat (APT) groups. The expert aims to identify which methods may slip past automated defenses and determine how to address these security vulnerabilities.

Who needs threat hunting?

Organizations at risk of targeted APT campaigns find proactive threat hunting particularly relevant. However, with the increasing trend of supply chain attacks, even small companies could attract skilled attackers’ attention.

The effectiveness of proactive threat hunting can be constrained by various factors, primarily the organization’s maturity level. Without essential components (like unified log storage, a SIEM system or the ELK framework for threat hunting) or the establishment of necessary audits, there would be little to base the search for cyber threats on.

Since the goal of threat hunting is to enhance threat detection quality, companies should have these procedures in place. It is crucial for a company to recognize the importance of threat hunting. This involves establishing solid information security processes and assessing risks to gauge the vulnerability of the organization’s critical resources to attacks.

The Hunting Maturity Model (HMM) outlines the stages of an organization's readiness for proactive threat hunting. Here are these stages:

• Level zero: The organization relies solely on automated detection tools like IDS, SIEM or antivirus, using their detection results.
• Level one: Involves gathering some data from endpoints and maintaining a storage location for these logs.
• Level two: The company utilizes a suite of SOC tools and conducts extensive audits, yet often lacks dedicated specialists for threat hunting.
• Levels three and four: These levels indicate a mature organization with a deep understanding of its business processes and enhanced threat monitoring capabilities, including additional logs and network telemetry. Companies at the fourth level also have a dedicated team focused exclusively on threat hunting.

Threat hunting tools

What tools and services are crucial for threat hunting? At the core, you need telemetry data, which tracks where and what processes are initiated, and what exactly is entered into the command line. This data allows for the detection of a significant number of attacks and serves as the foundation for developing detection strategies.

Threat hunting experts also value access to event data and tools for gathering it, as well as network insights and means to examine disks and memory. Moreover, threat intelligence tools are highly sought after in the world of threat hunting.

In theory, a company does not necessarily need its own SOC to implement threat hunting processes. It all comes down to having dedicated staff to analyze and sift through the data. Yet, in practice, organizations capable of conducting effective cyber hunts usually have a SOC in place, which offers a more holistic approach to information security.

It is important to note that no number of automated tools can substitute the intuition and experience of a seasoned threat hunter. The value of a curious mind and innate threat intelligence far surpasses that of any technology or service. Actually, threat hunting is not possible without human involvement.

Forensics and threat hunting

Threat hunting is closely linked with another important area of information security: investigation or forensics. Investigation involves collecting, processing, storing and analyzing evidence of potential cyber incidents, whereas threat hunting primarily aims at uncovering potential threats to preempt attacks. A critical role of investigators is determining when and how to react to a threat, but the information gathered during investigations frequently aids in threat hunting efforts. Forensic analysis plays an important role, for example, when backups come into play. Bare metal backups can provide clean data states from before any breach occurred, offering insights into the attacker's movements and methods, thus enhancing both forensic investigations and threat hunting efforts.

The starting point for threat hunting

Threat hunting relies heavily on threat intelligence, meaning it often kicks off in response to news of other attacks or attackers’ moves. An important part of boosting information security awareness involves specialists keeping up with the latest insights in specialized publications and researchers’ blogs. This information can prompt them to look for similar threats within their own systems.

You can gather information from various sources and prioritize among them. The following are sources from which threat hunting often begins:

• Information about vulnerabilities likely present in the network under observation.
• Analysis of critical digital assets that could be targeted by attackers.
• External indicators of compromise, including data on attacks against other similar organizations.

It is vital to investigate hypotheses that fall outside your security system's detection capabilities and are likely to be used in an attack. Here, a specialist’s personal experience and their own assessment of the risks associated with specific methods play a crucial role. Regardless of how sophisticated an attack might be, at least one of its Indicators of Attack (IoA) will trigger, initiating the threat hunting process.

While threat hunting might uncover only a few incidents, measuring its effectiveness solely on this count is inaccurate. Those few security events it does detect are overlooked by automated security tools and could pose a critical threat.

Finding the best threat hunting provider

Choosing your provider largely depends on how well your company’s information security service aligns with the methodologies used by a specific vendor. The metrics for evaluating a threat hunting service provider’s effectiveness are less about the quantity of threats identified and more about their ability to detect attacks that are particularly relevant to your industry and company type. It is also important to consider the types of data the specialists work with. Reliance solely on IDS and IPS logs and antivirus might indicate a lower quality of threat hunting. Additionally, the speed at which specialists respond to threats indicates their experience and the quality of their processes.

Becoming a specialist in threat hunting

There are numerous online threat hunting certifications, training programs and boot camps, yet this specialty is rarely found in university programs. People working in security operations centers who have gained experience in spotting and reacting to incidents often venture into threat hunting. Penetration testers and red team members, with their deep understanding of attack strategies and attacker mindsets, can also excel as threat hunting specialists.

Embarking on a threat hunting career begins with a solid foundation in cybersecurity basics, including networks, systems and security principles. Enhancing one’s skills further involves practical experiences like Capture the Flag (CTF) competitions and cybersecurity labs, which refine problem-solving abilities and deepen essential knowledge.

Tomorrow’s threat hunting landscape

Security leaders should not expect threat hunting to be fully automated. Automated security tools cannot cover the entire threat landscape. Human involvement in threat hunting will always be needed.

AI and ML are also unlikely to replace humans. While machine learning tools can assist by processing large volumes of information and generating recommendations, the final step of distinguishing “good” from “bad” events truly requires a human touch. It is a creative endeavor that demands the ability to interpret and assess new threats, something that goes beyond the scope of machine learning, which relies on existing data.

Threat hunting operates on two fronts: technical and marketing. Typically, marketing takes the lead. It is mainly large corporations that can afford traditional full-fledged threat hunting. For most others, it is likely to be bundled with products offering EDR, XDR and similar systems that include basic threat hunting features.

The surge in data volume, traffic and threats might nudge threat hunting closer to the detection layer. Here, events might be treated as a form of telemetry. Initially, the focus may shift from the events’ content to the behavior change in event generation by each source. This could lead to creating basic profiles for organizations, considering factors like size and industry, to enhance the effectiveness of the threat detection process.

A persistent challenge in this field is the shortage of skilled professionals. Not every company looking to hire a threat hunting specialist will manage to find one, leading to a growing trend towards outsourcing these services.

As companies mature, there is a growing need for proactive threat hunting. This trend is driven by an increase in cyber-attacks and the evolution of security operations centers that are now incorporating threat hunting into their operations.

Many attack types previously considered irrelevant by companies are now significant threats. This includes attacks by hacktivists or pro-government groups, targeting not just government entities but also banks and various private sector businesses. Additionally, the rise in remote work has expanded the list of potential compromise indicators. These developments have made threat hunting more critical than ever. 

Finally, threat hunting is not a set-it-and-forget-it operation; it is an ongoing journey that adapts as the threat landscape shifts.