At its most essential, said John McClurg, vice president - global security for Honeywell International, "Metrics meets a need to advance the business as a whole."
Measuring performance has always been a business and security exercise. “What has happened subtly is that our (security) organization has focused on metrics, not thrust on us. It continues to evolve and help advance each of the four Honeywell businesses,” said McClurg.
John Martinicky, director, corporate security at Navistar, Inc. - International Truck and Engine Corp., agrees with McClurg about evolving metrics. “We have developed a number of programs to measure the effectiveness of the security group.”
Business Management Strategies
Six Sigma seeks to improve the quality of process outputs by identifying and removing the causes of defects (errors) and variation in manufacturing and business processes. It uses a set of quality management methods, including statistical methods such as metrics, has quantified targets and creates a special infrastructure of people within the organization who are experts in these methods.
Incident analysis, security metrics and Six Sigma, these and other business management strategies are in force or evolving at many enterprises and within their security operations. The techniques also call for “refinement and sophistication,” observed McClurg. And the result of a firm embrace of metrics by security is the reality that the operation – big or small – is indispensable to the business as a whole. “It solidifies our position with all stakeholders,” added Honeywell’s McClurg.
Measuring UpOperating on “guts” and “instinct,” making assumptions on how security should operate or change based on how the enterprise overall is performing without knowing the facts could lead to problems.
Welcome to metrics. And there’s more to come.
“As a security professional, I believe we are seeing more convergence in this arena and ultimately we will see the fusion of security metrics with more advanced concepts in risk identification and analysis,” said Messemer, who has worldwide responsibility for all security operations across all Nielsen businesses.
Messemer added, “I believe an effective security metrics scorecard will help drive a fact-based decision on the proposed allocation of scarce capital investment resources. In today’s economy it is more important than ever.”
No doubt, metrics plays well in today’s economy. Said Maria Chadwick, director of surveillance, Wynn/Encore, “During these economic times, everything in our work environment is subject to scrutiny, whether it’s budget, staffing, procedures, and/or projects. For our organization, we are often asked to ‘quantify’ requests for spending whether it is for the purchasing of new equipment, increasing manpower for the department or adding staffing for a particular venue or project. If money is spent, will it create a substantial return on investment? Returns do not necessarily have to be monetary; they can be in the form of reducing wasted time, improving methods of operation, and enhancing training. Can the same tasks can be accomplished faster with less downtime and can certain training improve the operational efficiency of the department?”
Completing the Overall Mission
Comparables are important to John Williams, director of security at Prince William Hospital. “We use several methods to measure security operational performance. Some of them are what I consider standard measures such as response time to emergencies, number of escorts. Then we add additional measurements that focus on a broader comparison such as meeting or exceeding 75 percent of the measure used by other hospital security departments in our region. And we conduct an annual customer quality survey and look at such things as the visibility of security staff and staff feeling about their overall security while at work.” (For comparison, Williams conducts a benchmark survey.)
As McClurg pointed out, metrics is nothing new. One way or another, organizations have been using metrics forever – measuring themselves internally as well as against others.
Basically, a business metric is any type of measurement used to gauge some quantifiable component of performance, often as return on investment (ROI), employee and customer churn rates, revenues, income before interest and taxes and depreciation, and so forth.
Business metrics are part of the larger arena of business intelligence, those applications and technologies used to gather, store, analyze and provide access to information to help business people – from the CEO and CFO to chief security officers – make better decisions and adjustments. Such systematic approaches, balanced scorecard methodology, for one, can transform an organization’s or security’s vision, mission statement and strategies into specific and quantifiable goals, and to monitor the performance in terms of achieving those goals.
Beyond Financial Measures
The bottom line, no matter the approach, is that you can’t manage it if you can’t measure it.
There are a number of benefits to security leaders who use metrics.
There is greater focus. Defining metrics most important to the security operation and its business fit lets security executives tune out things not related to those more specific key measurements.
Vision is clearer. CSOs and security directors who monitor metrics regularly spot threats and opportunities faster than those who don’t. An outcome: more accurate insights into what’s happening within the operation as well as overall trends industry-wide.
Decisions turn out better. The numbers, stripped of emotions and “what ifs,” lead to reasonable, actionable decisions that flow more naturally with a higher degree of confidence.
Mike Tarter, executive director, safety and security with the Rio Rancho Public School District, uses incidents as one marker. “I have documented incidents over the last six years to show increases and decreases. That includes the number of reported student and employee incidents based on the total number of incidents. It’s sort of like the FBI annual Uniform Crime Report.”
“We look at use of equipment, who, where, how, what type of assignment; overtime versus comp time; private security versus our own; and analysis of information to determine future operation of security. We also look at training costs and take surveys of customers about how they feel.”
George Campbell, a member of the emeritus faculty of the Security Executive Council and managing partner in the Business Security Advisory Group, sees a clear connection from incident trends to metrics. Incident analysis involves some form of assessment, measurement and consideration of related metrics. More importantly, looking at risk this way helps form a more reliable assessment of root causes and the success of the revised security measures that are proposed.
Martinicky, for instance, measures incident rates, dollar loss, cost avoidance, security audit scores, security officer reports, the budget, system uptime (security video, DVRs, access readers, alarms, etc.) and supplier and vendor responsiveness. “Our value in reducing risk, meeting compliance requirements and cost avoidance is quantified, "he said."
But security executives must go beyond counting, of course.
For Mike Cummings, director, loss prevention services at Aurora Health Care and 2009 ASIS International President, numbers come to life when fed into a productivity model. “Relative to ‘activity’ or productivity performance, we use activity tracking software whereby we simply count the numbers of each service task that we do so we can empirically measure how busy we are. We work with internal staff to feed this data into a productivity model that also takes into consideration patrol hours, patient standby hours and nonproductive training time. Through this we can and do determine our productivity against a standard on a percentage basis. This helps justify and substantiate staffing levels.”
Added Chadwick, “In my opinion, staffing is the primary component to measuring any operation. Departments and organizations may operate within a budget; but how many people does it take to efficiently and effectively reach your departmental/organizational goals? Adequate staffing is the key to handling any type of venue, event, or day to day business.”
Drilling Up and Down
Added Cummings, “We identify specific performance measurement goals (Joint Commission) by hospital based on priority issues at that site. For example, we may look at (track) such things as false alarms, thefts, unlocked doors and then put in place tactics to reduce the risk. We determine the performance measures each year, measure monthly and document tactics and results that impacted the changes. We share this between hospitals to reduce duplication.”
Chadwick said, “Measuring security performance varies depending on the type of business and industry. But standards and procedures may differ within the same industry. It is dependent on business trends, location of the business, and the goals and objectives of the company.”
In addition, metrics can play a very specific role.
In a recent presentation for the Security Executive Council, Campbell used the example of a disturbing trend of more frequent workplace violence incidents at a particular location. Helpful metrics are in the incident reports. For example:
In his presentation, Campbell showed what gaps in the security program may be contributing to this increase in frequency and severity of workplace violence incidents.
- Is there a pattern in the findings that suggests a broader set of risks?
- What business processes failed? Which ones should have mitigated risks like these? Who owns them?
- What has been learned about the victims and perpetrators?
Outside Metrics Help
For example, earlier this summer ASIS International issued its Facilities Physical Security Measures Guideline outlining the main types of physical security measures that can be applied to minimize the security risks at a facility. It can make a solid complement to or drill-down for a metrics model.
Concentrating on physical security to safeguard or protect an enterprise’s assets, it outlines eight categories of measures used to protect facilities: physical barriers; physical entry and access control; security lighting; intrusion detection systems; video surveillance; security personnel; security policies and procedures; and crime prevention through environmental design.
Said Guideline Committee Chairman Geoff Craighead, with Securitas Security Services USA, “It’s a useful, succinct body of knowledge that’s not occupancy specific. A risk assessment, accompanied by a basic understanding of physical security measures provided by the guideline, makes it possible, either alone or with the help of security consultants or vendors, to select and implement appropriate physical security measures to reduce the assessed risks and accomplish the protective task.” And when it comes to system-centric metrics, especially to novice security executives, the guideline can overlay key performance indicators.
A number of years ago, the National Fire Protection Association issued premises security guidelines. Observed Craighead, “ASIS International, of its own volition, decided some years ago to create the ASIS Facilities Physical Security Measures Guideline for the projected audience, including its own membership. It was not a matter of what other organizations had created.”
In late June, the New York Police Department issued a report – Engineering Security: Protective Design for High Risk Buildings – that’s both a call to action as well as a metrics helper for security leaders and building owners who face a higher level of terrorism. It provides recommendations on how to prevent and mitigate the effects of a terrorist attack on a building, and shows a tiering system designed to categorize buildings based on risk.
On the Standards Side
E. John Sutton, with ManTech International and security consultant to the Port Authority of New York and New Jersey, looks at both the big and small picture. “For me, the ‘security operation’ encompasses everything from risk assessment to system delivery, operation and maintenance. For these and all steps in between, we utilize various means to quantify our performance (execution). High on the list of ways to quantify and qualify is taking into consideration those standards and specifications used throughout industry and governmental systems that require a high level of security, especially those involving intermodal transportation systems.”
On top of metrics, there also are value engineering reviews that pull experts from applicable fields. Commented Sutton, “Getting past this into actual construction and installation, our project managers, in conjunction with the construction department, use internal resources to ensure appropriate quality controls are applied including detailed schedule and cost tracking. After commissioning, the actual operation and monitoring of the systems typically use contract forces that have direct and instantaneous communication with responders. The typical systems employ numerous ways to query the databases to establish the actual response times for the operational personnel such as elapsed time between event announcement, acknowledgment of the event and time to final disposition. This type of data is readily available from most systems today and the challenge is to ensure the system administrators establish proper procedures and then execute them routinely.”
While facilities and premises guidelines, standards and specifications are an educating effort for metrics makers, more use of outside sources also push for more and more accurate performance evaluations.
The Nielson Company’s Messemer added, “Another trend we see in the marketplace owing to the new economy is a corporation’s reliance on fewer vendors in order to achieve its strategic security goals. This form of convergence also brings its own potential risks if a security professional ignores effective supply chain risk management.” In this month’s Zalud Report, Messemer talks about an externally-imposed metric – the state of Massachusetts’ privacy regulations and its impact on citizens and businesses.
KPI Plays a Role
Those professionals evangelizing metrics encourage a simple, stepped approach.
Define security’s goals and then define the metrics. For each goal, write down a metric that will help track progress to success.
Then it’s time to benchmark security’s current status – determine exactly how security is doing, even if the truth is hard to face. By establishing the current value of each metric, security executives are better able to track future improvements.
Value is a key concept for Williams, too, as well as measuring success beyond traditional security. he said “I have seen a major change in security measures. Days have long past where you can show that you found X number of doors unlocked and locked them so you potentially stopped a certain number of thefts. Security departments now must be able to show value to those in the C-suite.”
Emphasis on Measuring Service
Another performance measurement step is to create a system to monitor and report metrics. Some leaders interviewed for this article found it necessary to add new processes to help them calculate and report their metrics. For example, if the number of employees or associates who view security’s customer service as being “excellent” is low, then the security leader may want to survey those people more regularly to ask them how you are doing.
With his ever-changing customer base, Shawn Reilly, director of security at Greenville Hospital System, observed, “We use a customer satisfaction survey that asks people to rate us in a number of areas including responsiveness, courtesy and knowledge level of confidence in our force. We also gather and analyze crime information.”
Another measurement step is to communicate metrics with security staff. With key metrics, get the team informed and on board. Everyone then can make decisions to help improve the metrics. Obviously, it’s also essential to educate the boss on security’s metrics.
Just as loss prevention and security test and review disaster plans, review the metrics and make adjustments to improve results in an ongoing fashion.
The end-of-the-process charm of metrics is the list of successes and failures identified.
What to Avoid
Industry veteran Ray Humphrey, president of Humphrey & Company and past president of both the International Security Management Association and ASIS International, has a strong, working list of elements and factors to measure. These included:
- Periodic employee sensing and/or satisfaction surveys
- Variance profiles from year to year (e.g., loss and/or recovery statistics compared to previous years)
- Annual “feedback” solicited from all business units within the enterprise regarding the effectiveness, efficiency, and/or value-add of the security function and security personnel
- Recurring assessment and/or re-evaluation of security costs (e.g., headcount reductions vis-à-vis prudent use of technology – software and/or hardware)
- Contribution of the security function to such business initiatives as shipping, delivery, marketing, vendor relationships, purchasing, etc., in eliminating or reducing the “drainage of corporate profitability” because of product diversion, counterfeiting, fraud, collusion, thereby contributing to bottom line corporate profitability
- Enhancement of the “security profile” by having security management firmly and continuously involved in corporate governance and business continuity task forces, think tanks, work groups, and similar engagements that demonstrate the maturity, business understanding, intelligence, and contributions of individuals that in earlier years were viewed as “tunnel-vision guards”
- Involving senior security management in frequent presentations and exposures to business unit and corporate governance
Said Humphrey, “All aspects of the security function need to be evaluated or measured on a periodic basis. Measurement also involves identifying what isn’t working and/or what aspects of the security function should be restructured, refocused, or eliminated.”
Pick Actionable Metrics
- Developing metrics for which you cannot collect accurate or complete data.
- Developing metrics that measure the right thing, but cause people to act in a way contrary to the best interest of security simply to “make their numbers.”
- Developing so many metrics that you create excessive overhead and red tape.
- Developing metrics that are complex and difficult to explain to others.
Most metrics practitioners see two types of measurements. Performance metrics are high-level measures of what is happening. They are external in nature and are most closely tied to outputs, incidents, losses, recoveries, customer ratings and business needs. Diagnostic metrics are measures that delve into why something is not performing up to expectations. They tend to be internally focused.
Advised Trimble, a common mistake is to start first with diagnostic measures – measuring internally within the security department, for example, rather than beginning with an external focus.
Charts Make Conclusions Stand Out
Graphics, even on a daily basis, is a great metrics tool for Guy Grace, director of security and emergency planning for the Littleton (Colo.) Public Schools. “The performance of our security operations is measured by the overall response rate for the entire operation. For example, every type of response is tracked for each of security’s areas of responsibility. This is defined as four major areas; patrol operations, dispatching/monitoring, maintenance, and emergency planning. Each day, data is compiled and included in a daily report which is forwarded to key administrators within the district. At the end of each month data is included in a month-end summary along with an executive summary. Each response by security is cataloged for each site. Using this data the organization is able to measure the performance of security operations.
"This is a tremendous amount of data. However, when it’s illustrated in simple graphics it gives the organization the data it needs in a format that is easily understood. It is very helpful to have a section that includes the actual reports written by the security officers on key incidents. Reading how a crime was thwarted can be very interesting to the reader and personalizes it beyond the numbers game.”
There still are factors that defy measurement.
Commented Williams, “Metrics is a great way to show progress and see areas where improvement would benefit the organization as long as the metrics are validated, reasonable and emphasis value. However, some of the work that security departments do is performed as a deterrent. It is difficult to show that you prevented something from happening that others might say never would have happened in the first place.”
Grace agrees. “Security can be tough to measure because a good security operation is always going to be evolving. The bar always rises and there is always a new challenge to be met around the corner. Often our customers do not see the issues that are dealt with or how we respond to them. When customers go to work or school, the issues security may respond to are dealt with little or no disruption to the operations.”
Sample Security/Safety Performance Measures
- Number of incidents
- Security violations per audit
- Percent of audits conducted on schedule
- Percent of safety equipment checked per schedule
- Number of safety problems identified by management versus total safety problems identified
- Safety accidents per X hours worked
- Safety violations by department
- Number of safety suggestions
- Employee satisfaction surveys
- Customer satisfaction surveys
- Vendor surveys
- Benchmarking to industry guidelines
- Meeting government rules, regulations
Now for that “Gut Check”
- Do the metrics make sense?
- How do they compare with existing metrics elsewhere in the enterprise and in similar operations within the same industry?
- Do they form a complete set?
- Do they reinforce the desired behavior for the long haul as well as for today?
Rid Yourself of Analysis Paralysis
Rose has five steps to help security leaders avoid analysis paralysis and back into action.
One: Set the timer. Give yourself a defined period of time to finish the process of data gathering. It is important to accept the fact that there will always be unknown factors; security executives will never have all of the data about a particular subject. When coming to terms with the inevitable unknown, make a decision based on what’s known, including metrics and measurements in hand.
Two: For security executives who must balance their business acumen with their protection expertise, instinct counts. It’s a talented business analyst.
Three: Ask for a second or even third opinion. In picking metrics as well as analyzing results, run arguments and potential actions to be taken past trusted others. Ask for feedback both on content and delivery. Where in the presentation did the colleague see the most energy? Confidence and enthusiasm are always good indicators of the most appropriate solution.
Four: Do a mental dry run. Once a path has been chosen, Rose suggests beginning the journey in your mind first. “Mentally play out decisions exactly as you wish it to be. Fill in all of the details of the outcome.”
Five: Take the leap. Rest assured, there may be a level of second-guessing. But metrics, well crafted and smartly analyzed, reduces the tone of such second-guessing.
Metrics and Managing Emergencies
According to Williams, emergency management planning is much more than the 5-inch thick binder. The basis of every emergency management response plan is first and foremost people who are educated and train regularly on how to properly respond and overcome obstacles to achieve their goals. Having a thorough emergency management plan that addresses all-hazard for your area and industry is a great step forward. But if plans are not communicated to stakeholders, tested for effectiveness and reviewed for improvement they are worth less than the paper they are written on.
To make the metrics relative to emergency drills valid, Williams urges security leaders not keep the disruption of the drill to a minimum. “Build your drills with enough injects to stress your plans, resources, and team to the breaking point and beyond to see if it can stand the surge.”
Planning drills that educate staff, the community and others takes time, thought, and, in some cases, a lot of imagination and knowledge about what is happening around the world and in a particular region. “Effective emergency management drills more often take real-life events that have occurred in an industry or region and modify them to fit the particular facility drill objectives. After reviewing and revising the all-hazards vulnerability assessment from the previous year, the next step is to set up the annual emergency management drill calendar.” Annual or multi-year emergency management drill planning allows a facility a better opportunity to draw in their local community agencies who may also need to hold an annual drill and would jump at the change to also get involved.
Prince William Hospital holds at least four major emergency management drills annually. These drills can include: mass casualty; chemical, biological, explosive, radiological and nuclear (CBERN) decontamination (Teen volunteers are used as mock patients in the summer for this drill.); infant abduction; weather-related (which affects this hospital most often); police action (active shooter, hostage, barricade, or civil disturbance); lock down; hazardous chemical spill, and others.
Advised Williams, “When developing an emergency management drill, staff buy-in is greater the more realistic the events leading up to the drill and the actual drill seems to them.” For the incident command center team, pulling in representatives from outside agencies and organizations to participate in the simulation cell (SIMCELL), and using this SIMCELL to answer questions from participants, is a great way to add value to the learning process, improve relationships with outside agencies and educate agencies about what the facility is capable of doing. Using a SIMCELL is a little more work for those planning the emergency management drill, as injects need to be written and briefings with agency representatives conducted, but the exercise simulates what it takes to interact with outside agencies and the limits of their help.
Drills can have a more realistic feel when a software presentation program, such as Microsoft PowerPoint, is used to develop simulated news report injects. Photos from past drills or un-copyrighted disaster photos found on Web sites add a higher level of realism to the drill. “For example,” added Williams, “in a recent police action drill, we used video previously taken from one of the hospital’s exterior security cameras during an active shooter drill and played it back as if it were actually happening.”
Useful measures of the success of emergency drills also call for involvement and feedback from numerous groups.
“I am not advocating communicating everything in you management plans to the general public,” Williams said. “However, in some cases, the general public should be aware a facility has thorough and well-thought out plans based on the hazards and emergencies the community is likely to experience and that, in this case, the hospital is equipped, staff is trained and response has been tested on a regular basis.” Specific to healthcare facilities and their security operations, Williams advised to “take every opportunity (hurricane season, tornado season, ice storm season, etc.) to educate the public how to prepare themselves for an emergency management event.”
This year, one of the multi-day drill events for Prince William was a 7-day homeland security event culminating in a facility bomb threat, which required a full response from staff. “We communicated the scenario to local police, fire, and emergency management departments, as well as the county emergency management department and the regional health coordinating center.”
There is a concern about bench strength when it comes to measuring the performance of people involved in a lengthy emergency drill. “Our goal this year is to bring in more staff to be trained in the incident command system application and set them up with a mentor who has sat in a similar position for at least three drills or actual events, and have him or her help new staff member learn about their position,” Williams said. “We decided on a mentorship program rather than a ‘sink or swim’ process.”
Emergency management drills are far more successful when they are designed as learning and growing events as opposed to being held as “just a drill.” They should be as realistic and current as possible. And metrics comes into play when it comes to the measurement of the drill events.