Whether it’s done to meet compliance requirements or just as a general best practice, most organizations are now testing their own networks for security weaknesses, and if they’re not, they should be. The many different types of tests can be confusing for the uninitiated; we will take a look at the common types with their strengths and weaknesses.

Network Vulnerability Scans

Network vulnerability scans are the most basic type of test, relying on a scanning tool that looks for known common vulnerabilities and reports them with severity ratings. These scans are quick, cheap, and have little risk of causing damage. They are great for making sure that system patches are up to date and security configurations settings are locked down. When run regularly, network vulnerability scans can provide an early warning if a system is missing patches or misconfigured.

Many organizations only scan their network from the Internet. While Internet-facing vulnerabilities may seem like the easiest for an attacker to exploit, scanning only from this perspective leaves an organization blind to the vulnerabilities that an attacker would leverage to move between systems once they gained a foothold inside the network. Scans should also be performed from behind the firewall to identify these internal vulnerabilities.

Yet even internal network scans leave blind spots: by default the scanners can only check services that listen for network communications. Many of today’s attacks are the result of phishing campaigns that target web browsers, PDF viewers, and other “client” software not visible to a network scan. These attackers usually go on to exploit other local operating system vulnerabilities to get administrator privileges. To solve this problem scanning tools can be configured with authentication credentials so they can log in to their targets during internal scans and check local software as well. This approach is strongly recommended as it will give the most complete view of the patch and configuration status of an organization.

The final flaw with network vulnerability scanners is that they are only as good as their vulnerability signatures which are based on databases of known vulnerabilities. This means that scanners can’t pick up anything that hasn’t been publicly reported yet, including vulnerabilities in obscure or custom applications. This is an issue as attackers regularly leverage vulnerabilities in custom applications in order to gain access to the data they contain or the underlying network.

Application Vulnerability Scans

Unlike network scanners, application scanners are designed specifically to look for previously undocumented vulnerabilities in custom applications. Rather than checking for a list of known vulnerability signatures, they will exercise all of an application’s functionality in order to find common types of flaws. Because of the amount of data they send to an application these scanners must be used carefully; everyone has a horror story about a scanner dumping garbage data into a database or triggering thousands of emails.

As advanced as application scanners are, there are still vulnerabilities that they miss. Some vulnerabilities are too subtle for an automated scanner to detect but will be obvious to a human with a little intuition. As with network scans, a clean report is a good start but doesn’t necessarily mean there are no problems.

Penetration Testing

The down side of any kind of scanning is that most organizations concentrate on fixing the “high” or “critical” severity vulnerabilities reported by these scans while real-world breaches are rarely the result of a single critical network vulnerability; real attackers may chain together a few low to medium severity network vulnerabilities or combine them with “local” vulnerabilities that aren’t visible from the network.

Penetration tests build on the concept of network and application vulnerability scanning by adding skilled hackers that can simulate real-world attacks against network services, applications, or both simultaneously. The testers will try to combine and exploit vulnerabilities found by the scanners as well as look for the types of vulnerabilities that scanners miss. While this is more time consuming and expensive than scanning tools alone it provides a much more realistic assessment of how much effort would be required to breach an organization.

Testing is not without its risks, however. Although testers will take great care to avoid causing damage, they are still attempting to exploit flaws in software, and there will always be a chance of unintended consequences.

Many organizations try to cut costs by limiting penetration testing to “critical” systems, but the value of a penetration test is only as good as its scope. Real attackers have no qualms about exploiting other unrelated systems to get to the ones with sensitive data, so testers should be free to do so as well.

Organizations should also remember that the results of a penetration test are highly dependent on the skill of the team performing it. I’ve personally seen many cases where an organization contracted a new penetration testing firm and found severe vulnerabilities that a previous firm missed. It’s worth paying a little extra for a team with a solid reputation.