Security Enterprise Services

How Security Can Use Big Data Best

Third of a Four-Part Series


money_enewsIn our last article, we looked at sources of security data and emphasized the importance of a data strategy before you get started. Now, assuming that your organization has formulated a strategy and set goals for what you want to achieve with Big Data, there are several paths toward implementation.

One way to evaluate options is to think about where they fall on the build vs. buy continuum and what sort of appetite your company has for each of those options. If you have deep IT resources, perhaps with other Big Data projects already in place, it may be more attractive to graft your security data strategy onto the tools and talents already at work in other departments or business domains. If, on the other hand, you have few IT resources and always look to outside vendors for solutions, you will approach this more as a “shopping” exercise than a build-out.

In any case, there are three broad approaches worth considering:

Export, Aggregate and Build

To my knowledge, there are no on-premise physical security systems with a Big Data solution already built into their core deployments. In part, that’s because Big Data is a relatively new technology, and no one has seen fit to include it in their feature set. It’s also because the technology platforms used for Big Data are very different from the technology platforms typically used for security. Finally, few if any security systems have been set up to marshal all the necessary data into one place, where it can be usefully analyzed with Big Data techniques. Large video storage systems may be an exception but, even then, it’s usually not seen as economical to transport massive volumes of enterprise-wide data from hundreds of locations to one data warehouse.

Given these many challenges, the “roll your own” option begins with exporting data from your current systems and aggregating it onto a Big Data platform where you can perform subsequent analysis. In the data warehousing sector, this is known as ETL, or Extract, Transform and Load. You’ll need to do this because the typical security database platform will not support Big Data operations.

After ETL, the difficult task of programming one of the many Big Data technologies to perform your particular analysis will begin. This is not for the faint of heart, and you’ll want to have access to someone called a “data scientist” in addition to your software developers.

Dump it in the Cloud

If building IT solutions from the ground up is not your cup of tea, a second option is to transfer your data to an online Big Data solutions provider and work with their experts to extract the value identified in your data strategy. Along with many industry stalwarts, there are now many dozens of start-ups operating in this arena. Big Data is a hot investment trend, and we’re sure to see many more entrants before the winners emerge.

The advantage of this approach is that you avoid both ramp-up and capital expenses. The learning curve for Big Data technologies can be significant, depending on what you want to accomplish, and you may not wish to burden your initiative with either the expense or time for that process to play out. By the same token, you may not wish to invest in the technology up front, and cloud solutions offer the same flattened expense profile as traditional SaaS offerings for other business applications.

The one disadvantage, given that this is a new field, is vendor longevity. If your vendor is not one of the market winners, you may find yourself having to migrate to a new provider, and that’s tough given there aren’t really any standards for data portability in this domain.

Go for a Built-in Solution

The last option is to wait and see which security vendors emerge with built-in solutions. This is likely to occur first among enterprise systems providers, with an advantage toward cloud offerings. Cloud vendors can distribute the cost of a Big Data solution across everyone in their customer base who chooses to use it, rather than asking you to buy a whole Big Data stack to put in the data center next to your other servers.

The trick here is to recognize that every industry vertical will have different Big data strategies, with the goal to extract different types of value. That means there will not be a one-size-fits-all product offering, and you’ll do best choosing a vendor that can extensively customize the solution.

In the last installment of this article series, we’ll take up the importance of normative references, or why your own security data becomes more meaningful in comparison to industry standards.

Did you enjoy this article? Click here to subscribe to Security Magazine. 

Recent Articles by Steve Van Till

You must login or register in order to post a comment.



Image Galleries

ASIS 2013 Product Preview

ASIS International 59th Annual Seminar and Exhibits, September 24-27 in Chicago, Illinois, will include an exhibit hall packed with innovative security solutions. Here are some of the products that will be shown at ASIS this year.


Virtualization and Data Center Security: What You Need to Know for 2014

Data centers are increasingly becoming the center of the enterprise, and data center and cyber security is following the same path for security departments. According to Justin Flynn, a consultant at the Burwood Group, the virtualization of data centers allows enterprises to scale more easily and faster, with a smaller footprint.

However, hosting enterprise data in the cloud can make intrusion detection more difficult – how can enterprise security leaders team up with other departments to keep aware of cyber risks and traffic, and physical and data compliance during the virtual transition? How can CISOs and CSOs discuss cyber threats with the C-Suite to get the resources they need? And how can the proper infrastructure test and verify possible malicious attacks? 

More Podcasts

Security Magazine

August 2014

2014 August

In the August issue of Security Magazine, read about the public-private partnerships and the future of DHS with Frank Taylor, sneak a peek at the ASIS 2014 security products, and read a special report on cyber risk and security. Also in this issue find out why America is in desperate need of a CSO and the most common mistakes in Cyber incident response. The security game has dramatically changed since September 11th, read about what enterprises are doing to keep Americans safe and sound.

Table Of Contents Subscribe

Adopting New Technology

How long do you wait before adopting a new technology?
View Results Poll Archive


CompTIA Security+ Certification Study Guide
CompTIA's Security+ certification is a globally-recognized, vendor neutral exam that has helped over 60,000 IT professionals reach further and higher in their careers. The current Security+ exam (SY0-201) focuses more on being able to deal with security issues rather than just identifying them.
More Products

Clear Seas Research

Clear Seas ResearchWith access to over one million professionals and more than 60 industry-specific publications,Clear Seas Research offers relevant insights from those who know your industry best. Let us customize a market research solution that exceeds your marketing goals.

Vertical Sector Focus: Critical Infrastructures

criticalhomepagethumbFrom terrorism to vandalism, it’s preparedness, response, training and partnerships. Learn about some of the critical security issues facing this sector.

Visit the Critical Infrastructure page to


Facebook 40px 2-12-13 Twitter logo 40px 2-12-13  YouTube  LinkedIn logo 40px 2-12-13Google+