In our last article, we looked at sources of security data and emphasized the importance of a data strategy before you get started. Now, assuming that your organization has formulated a strategy and set goals for what you want to achieve with Big Data, there are several paths toward implementation.

One way to evaluate options is to think about where they fall on the build vs. buy continuum and what sort of appetite your company has for each of those options. If you have deep IT resources, perhaps with other Big Data projects already in place, it may be more attractive to graft your security data strategy onto the tools and talents already at work in other departments or business domains. If, on the other hand, you have few IT resources and always look to outside vendors for solutions, you will approach this more as a “shopping” exercise than a build-out.

In any case, there are three broad approaches worth considering:

Export, Aggregate and Build

To my knowledge, there are no on-premise physical security systems with a Big Data solution already built into their core deployments. In part, that’s because Big Data is a relatively new technology, and no one has seen fit to include it in their feature set. It’s also because the technology platforms used for Big Data are very different from the technology platforms typically used for security. Finally, few if any security systems have been set up to marshal all the necessary data into one place, where it can be usefully analyzed with Big Data techniques. Large video storage systems may be an exception but, even then, it’s usually not seen as economical to transport massive volumes of enterprise-wide data from hundreds of locations to one data warehouse.

Given these many challenges, the “roll your own” option begins with exporting data from your current systems and aggregating it onto a Big Data platform where you can perform subsequent analysis. In the data warehousing sector, this is known as ETL, or Extract, Transform and Load. You’ll need to do this because the typical security database platform will not support Big Data operations.

After ETL, the difficult task of programming one of the many Big Data technologies to perform your particular analysis will begin. This is not for the faint of heart, and you’ll want to have access to someone called a “data scientist” in addition to your software developers.

Dump it in the Cloud

If building IT solutions from the ground up is not your cup of tea, a second option is to transfer your data to an online Big Data solutions provider and work with their experts to extract the value identified in your data strategy. Along with many industry stalwarts, there are now many dozens of start-ups operating in this arena. Big Data is a hot investment trend, and we’re sure to see many more entrants before the winners emerge.

The advantage of this approach is that you avoid both ramp-up and capital expenses. The learning curve for Big Data technologies can be significant, depending on what you want to accomplish, and you may not wish to burden your initiative with either the expense or time for that process to play out. By the same token, you may not wish to invest in the technology up front, and cloud solutions offer the same flattened expense profile as traditional SaaS offerings for other business applications.

The one disadvantage, given that this is a new field, is vendor longevity. If your vendor is not one of the market winners, you may find yourself having to migrate to a new provider, and that’s tough given there aren’t really any standards for data portability in this domain.

Go for a Built-in Solution

The last option is to wait and see which security vendors emerge with built-in solutions. This is likely to occur first among enterprise systems providers, with an advantage toward cloud offerings. Cloud vendors can distribute the cost of a Big Data solution across everyone in their customer base who chooses to use it, rather than asking you to buy a whole Big Data stack to put in the data center next to your other servers.

The trick here is to recognize that every industry vertical will have different Big data strategies, with the goal to extract different types of value. That means there will not be a one-size-fits-all product offering, and you’ll do best choosing a vendor that can extensively customize the solution.

In the last installment of this article series, we’ll take up the importance of normative references, or why your own security data becomes more meaningful in comparison to industry standards.