Modern security teams are not unlike the tenacious forensic investigators featured on many popular network television shows. In order to determine ‘who done it’ they must piece together small and seemingly unrelated strains of evidence.

However, unlike a TV investigator, the vast majority of security teams today are often unaware that a malicious incident has even occurred - which is why according to IBM, it takes the average enterprise almost 200 days to even recognize the fact that they’ve been breached in the first place.

Which is why security analysts are taking a closer look at network metadata which represents one of the most underutilized and overlooked sources of threat intelligence. For an attacker to successfully exfiltrate data from a network, the attacker must use the network itself to move the data. No matter how stealthy an attacker might be, traces of network metadata will invariably be left behind.

The sheer volume of metadata generated by the network is staggering and comes in a variety of forms, from DNS queries and firewall or proxy access logs to even lowly spam messages – the metadata attached to each one of these sources represents a pixel of threat intelligence. Just as an isolated pixel in a digital image by itself won’t tell you very much, once you collect, correlate and organize all these pixels, a picture will begin to come into focus – one that can help answer the most important question for today’s cybersecurity practitioner: have we been compromised?

What follows are four distinct insights that network metadata that can help you definitively answer this question.

Four insights generated by network metadata

Insight 1: DNS metadata can verify an adversary is communicating with your infrastructure

The Domain Naming System (DNS) is the phonebook of the Internet, providing a distributed and hierarchical naming system for the devices, services and resources - without it, there is no Internet. More than 90% of our daily usage of the internet involves DNS queries, hence it's a gold mine to assess whether or not a device is interacting with adversarial infrastructure.

Once a device is successfully compromised, the first thing it will typically do is attempt to resolve a domain that belongs to the adversary and if successful, the conclusion is self-evident: your organization has been compromised.

Analysis of DNS metadata can also be used to detect other telltale compromise indicators such as domain spoofing and unresolved IP addresses. One aspect where analyzing DNS metadata can be particularly useful is in identifying Domain Generating Algorithms (DGAs) which threat actors regularly employ to dynamically generate and activate random domains from which they can orchestrate and obscure their attacks.

Blocking malicious DNS requests is a logical first step, however the ultimate goal should be to eliminate the residual compromise from the device that triggered the initial blocked DNS request. Otherwise, a compromised device will continuously trigger malicious DNS requests generated by the DGA until it succeeds in connecting to its Command & Control master console.

Insight 2: Proxy and firewall log metadata can be used to visualize compromise connection points

If an attacker does not use domain infrastructure to communicate with compromised infrastructure, their only other option is to use an IP address. In this case instance, the traces of adversarial contact will lie in the access logs of firewalls or proxies, which when analyzed and correlated with other network metadata components, can help connect the dots between a constellation of compromised devices.

For instance, many of the most persistent strains of malware will automatically scan a network looking for accessible hosts from which to escalate their privileges and in response, the firewall will either block or drop the connection. When the firewall drops a lot of different connections from a single source host that’s usually a pretty good indication that an intruder is already inside and scanning the network to identify lateral openings.

In most instances, the adversary will opt to use DNS because it provides the attacker with greater flexibility and can persist undetected for a longer period of time. However, if an attacker is deprived of this option they will instead default to an IP-based communication method, making it all the more important to incorporate this source of metadata.

Insight 3: Analyzing netflow metadata can illuminate how an attacker is moving across the network

Once an attacker has gained a foothold in the network, they must then use the network itself to move laterally in order to achieve the goal: the exfiltration or encryption of host data. During this crucial phase of an attack, specific and distinct types of network traffic are being generated, leaving the attacker vulnerable to discovery.

However, because the network is a noisy place, identifying and isolating these markers can be especially tricky. Analyzing netflow metadata can provide important clues as to how they are traversing the network, providing network defenders with valuable insight into how and at which point an attacker is attempting to escalate privileges or propagate their payload.

Insight 4: There is more than spam in your spambox

Your email spambox is more than just a receptacle of junk emails and phishing lures. By identifying the machines that are contacting the spam infrastructure based on the metadata from the spambox itself can tell us a great deal about other malicious activities that might be related to the same organization - especially if a phishing link was the initial point of incursion.

Correlating metadata from the spambox with other network metadata elements can also help identify compromised endpoints which can be mapped back to individual users, providing incident responders and forensic teams with a vehicle to trace the route of a compromise, update its heuristics to improve detection of future threats, and ultimately limit its impact.

Stopping spam is a good first step, however, analyzing it is better as it can tell you who and how your organization is being attacked.

The age old axiom, ‘the whole is greater than the sum of its parts’ has become a popular refrain for technology leaders looking to cultivate a holistic ethos. However, when it comes to seeing the big picture of your network, you must be able to see all of those individual parts to understand the whole picture. The real power of network metadata becomes fully realized when you are able to fully assess and integrate all of these discrete parts – only then will you be able to transform them into something that’s both actionable and meaningful.