The IP Camera Debacle
It just drives me nuts when someone tells me that their cameras are Internet Protocol-enabled.
I typically respond, “Oh really, but what about the Transmission Control Protocol (TCP)?” The only people that really laugh at this comment are the client’s or manufacturer’s information technology (IT) networking gurus.
The reason this is scary is because TCP and IP are intertwined and essential for inter- and intra-computer networking. Without them, networking computer, cameras, phone and any other “appliance” would not be possible.
Yes, in guru speak, anything that connects to a TCP/IP-based network such as a camera or a server is called an appliance. As we move “further down the rabbit hole,” you will see other types of terminology exchanged. The purpose here is to share this author’s opinion of TCP/IP-based camera technologies: the move towards the IP revolution, and the problems associated with this new technology and how to determine if it is right for your organization.
BACKWARDTo understand where we are going, we must first understand where we have come from.
To that end, enter the world of analog cameras. Analog cameras represent the majority of today’s installed base. From pan, tilt zoom cameras, fixed cameras, interior and exterior cameras, you don’t have to look very far to find an analog camera staring back. Analog video is founded on standards — including Phase Alternating Line (PAL) and Sequential Color with Memory (SECAM). As it relates to North America, the National Television System Committee (NTSC) and Electronics Industry Association (EIA) created standards so that manufacturers could develop products that are interoperable. Under this approach, video recorded in PAL (European standard) would be unreadable to U.S-based NTSC standards and vice-versa. Without the proper hardware, designed to the correct standard, in an analog world, you would be unable to access video source signals that were produced under a different standard.
This is extremely important, because contrary to popular belief, the digital world does not follow the feeling of openness and consensus, which is what standardization is all about.
In fact, digitally-ready TCP/IP connectable cameras may not operate with a different manufacturer’s digital head-end equipment or vice versa. In some ways ironically, the fast passed progress has actually moved us backwards in interoperability/integration.
INTEGRATION / ACRONYMSIntegration has been around for a very long time and it takes on different contexts to different people.
Recently, integration has been re-defined as convergence. However, integration to this author is when multiple pieces of equipment from different and unaffiliated manufacturers come together to make it part of a whole system customized to a specific end-user’s needs. For instance, the ability to integrate a security video switcher and an access control system though a single cable, rather than multiple cables, is integration to me, but may differ from someone else’s perspective.
More than likely, many chief security officers today have been pitched a TCP/IP solution because some manufacturers seem to be giving incentives to sales persons that are selling TCP/IP in lieu of analog. Salespersons are throwing words out such as integrate-able (is that a word?) and cost effective. In some instances, security practitioners are being removed from the decision process by other stakeholders within the organization. These terms should be taken into serious context when evaluating a digital solution. There is a need, however, to question why a manufacturer sales person would attempt to sell a digital solution to an organization that is fully vested in analog. However, this is what happens, more often than not. These sales, in this author’s opinion, are “beach-heads” for the removal of analog cameras system from respective organizations, which may or may not be required. Most sales persons, when it comes to digital TCP/IP technologies do not try and determine the needs of their customers, but pitch the benefits of a digital solution.
First and foremost, there are numerous types of digital technologies, each with drawback and advantages. There are digital video recorders (DVRs), network video recorders (NVRs), hybrid NVRs, hybrid DVRs. These in turn support different types of codecs (formats of video) MPEG, MPEG-4 MJPEG, H.263 and others.
To complicate matters, these products come in varying resolutions and rates of capture. This is when things really start to get complicated. Many think that because their recorder uses a standardized format or codec that the quality is equal to another product using this same codec. In actuality, this is untrue. The quality of the recorded video is a result of the type recording and the information in that recording. Types of recording include and further subdivided as frames per second, fields per second (FPS) images per second (IPS) and pictures per second (PPS). It is important to note that the only difference between IPS/PPS is well, nothing. They are exactly the same, but manufacturers will infer that their technology is superior to that of the others by using different terms. Frames and PPS are a bit different, but to really evaluate it, we need to look at analog frames/fields, which would take us off course. It is somewhat like ready an army field manual when comparing digital equipment.
This acronym soup just confuses everyone and makes a selection process very difficult. However, when comparing these technologies, the manufacturer will utilize this terminology to their benefit to “distinguish” their product from the rest.
Over the course of a few months, there’s a trend toward DVRs and NVRs being capable of recording video at MPEG–4, which is group of pictures (GOP) type of recording. GOP captures a reference or key/I/reference frame, which is constantly compared to the video source and is updated when there is a change in the picture. However, only the area in the picture that has changed is updated. This typically results in fuzziness or blurring of the picture for the area that changed. This recording is preferred, because of the increased compression, reduced file size and limited storage requirements. Conversely, IPS/PPS captures a still frame that has less compression, and video is recorded regardless of changes in the scene much like the multiplexer/recorder combination we previously used. However, the drawback as indicated is size. With less compression, the size of the picture increases, which means more storage requirements, which means higher cost.
SIZE MATTERSAs previously noted, security video image size matters. Video manufacturers, realizing the benefits of digital video, failed to realize one of its biggest stumbling blocks – storage. Yes, storage, even a medium- to large-sized frame of 15k, recorded at 15 fps over a 30-day period, per camera can add up extremely quickly. The storage of this amount of data requires large amounts of storage in the form of network appliance devices such as the redundant array of independent discs (RAID), network area storage (NAS) and storage area network (SAN).
Now, again, without standards, it is a little bit like the Wild West. One digital recorder may record its images at a size of 320 x 240, while another recorder may record 640 x 480. Obviously, a 640 x 480 picture would look better, because there is more information. As you increased the image size, you increased the information captured and the associated frames, thereby increasing the overall storage requirements. As it relates to storage, you need to keep in mind these two independent factors: frame resolution and frames of capture. These two items greatly affect storage and your pocket book.
As it relates to comparable resolutions, many people use the common intermediate format (CIF) to equate quality of video when comparing other video standards such as MPEG. This really is not accurate. CIF was developed to standardize a set of video formats used in videoconferencing; but it is now being applied to digital security recordings. Sadly, CIF is currently the closest thing to a semi “apples-to-apples” comparison of a manufacturer’s digital recording method. There still is bending. You will ask for a 4 CIF resolution in a product shoot-out only to find out later that the resolution was higher, such as “special CIF.” This is important to watch for, because it makes it more difficult to compare products from an “apples-to-apples” approach.
As it relates to storage, the analog world was very forgiving. Typically security directors knew when a VCR was failing and or when a tape needed to be replaced. The digital world is very different; the loss of storage will typically be immediate and will result in a hard-drive crash. Therefore, if your organization is regulated or mission critical, you will need to consider the use of a RAID 5 or a mirrored drive arrangement. Not to be too technical, a RAID allocates storage across a number of drives. If a single drive fails, the data from the other drives can help re-create the data on the drive that became corrupt/failed. However, the function of RAIDs is at a price, which is additional storage. RAIDs are expensive and should be allocated based on an assessment of specific security video needs. However, as Murphy’s Law dictates, the time you will need the video will be when a drive fails. RAIDs will also limit downtime for the replacement of the corrupt or crashed hard-drive.
No matter the digital recording solution selected by a CSO, the storage requirements will make most information technology (IT) groups salivate. 2.5 Terabytes of storage are very common, and this author has seen systems as large as 8 Terabytes, or 8192 gigabytes per recorder.
As you can see, the storage demands for digital video are massive. Depending on the manufacturer being reviewed by the CSO or his/her integrator, the capability of an “open” storage platform may exist. The opportunity for open storage must not be discounted. Significant monies could be saved by leveraging the purchasing power of an organization to purchase a NAS, SAN or other type direct attached storage device via the network, SCSI, ISCSI, USB 2.0 or firewire connection. It must be stressed, not all manufacturers will support an open system, because they want to make the money on the mark-up associated to the “certified” equipment they sell.
DIGITAL RECORDING TECHNOLOGIESFor the purposes of this article, there are basically three types of digital recording technologies, which have embedded or fully functional operating systems. The DVR takes the place of an analog multiplexer and video recorder solution. The DVR incorporates a looping encoder and storage device into a single location. The DVR is a recorder that is typically accessible through a shared network, but does not offer the features that the second type of recorder provides.
An NVR is essentially digital transmission from the camera/encoder to the recorder. For this discussion and simplicity, there are two types of basic network video recorder systems. The first is a true digital solution. The camera is connected through a shared network (TCP/IP). The camera acts as the encoder and converts video into a proprietary format for recording at a central location. Cameras are one-to-one, which, as implied, use one Ethernet (TCP/IP) cable back to a TCP/IP network switcher/router/hub for transmission back to the centralized recorder. The second type of NVR system is what may be called a hybrid. This hybrid NVR is no different than a DVR except that the video capture has been segregated from the storage device. The video capture device (encoder) is located in an area where signals from multiple cameras centralize, such as a riser, prior to being transmitted back to its respective head-end system location. Under this design approach, analog cameras terminate at the encoder, which converts the analog video into the appropriate format and transmits it back to the central recording area via a network switcher. Both types of NVRs are total video solutions and are not designed to inter-operate with existing analog technologies. If these technologies are being used, then it is likely you will need to covert a whole analog infrastructure to a digital-based one over a period of time. There are exceptions to the application and architecture of hybrid NVRs, but limits the value of the digital solution.
NETWORK CONNECTIVITYNetwork connectivity is based on a distributed infrastructure, which is built upon telecom or intermediate distribution frame (IDF) rooms and main distribution frame (MDF) rooms and the horizontal Ethernet cable (Category 5,6) and vertical fiber optic cable in between them. TCP/IP network connectivity is very important to all types of digital recording technologies. The loss of a shared network could mean the loss of viewing or accessing a DVR or the loss of all cameras connected to the network.
There are some important things to realize when implementing a digital solution.
One, there is no direct connection to the camera or digital recording technology. TCP/IP cameras/encoders shall convert video in the appropriate format and transmit the video in a series of data packets back to the digital recording device though a Web address. In actuality, camera data can pass through multiple areas, prior to be transmitted back to the recording device. Each of these “hops” as the gurus like to call them are points of partial or total failure of the video system that is being run over a network. Second, passing camera data over a shared network has inherent problems. One, as indicated, the network is shared, and therefore susceptible to “packet sniffing” or the interception of data more readily than the physical interception of analog cameras.
It is possible that someone could remotely access and monitor cameras within the organization. Third, the shared network is controlled by others, and is subject to manipulation. A network administrator has the capability to throttle back individual computer appliances or slow the throughput of data on to the network without knowledge or consent. If you intend to use Web cameras, then you really need to do some in-depth research and implement a number of controls both on the IP camera device, your network and preferably though encryption or tunneling. Tunneling is establishing a secure connection through an existing shared network.
Security systems will not be the only service that runs on a shared network, and as a result priorities will need to be established for its use. For instance, in a healthcare scenario, patient imaging will take prevalence over security. In this example, a growing healthcare network shall prioritize network connections and limit the throughput of other connections that are perceived less important.
VIDEO TRANSMISSIONWithout a doubt, digital technologies excel in the transmission of video over shared networks for remote sites. Previously video connections between remote sites were done by microwave or dedicated fiber optic cable. Now, multiple cameras can be simultaneously routed to a separate or concurrent location very cost effectively. This is one of the hidden advantages of TCP/IP connectivity and makes a lot of sense as compared to the static networks that would be required otherwise.
Shared networks are based on standards -- each camera would most likely connect to a telecom room/IDF. However, security personnel and their associated contractors will have limited accessibility to these rooms. Cabling, to support any network or Ethernet connections shall need to be run in accordance with IT cabling standards. Typically, Ethernet cabling can only be run 300 feet as compared to coax, and unshielded twisted pair (UTP) can run much further. The lack of accountability when installing a security system can lead to a great deal of finger pointing. When a Web camera does not work, security integrators often point to the network. Network personnel point to the manufacturer; and, ultimately, the end-user will be sitting in the middle with a system that has limited or no functionality.
One of the biggest issues that appears to be a point of contention with my clients when evaluating TCP/IP cameras is latency. Latency is a fancy way of saying delay. The data conversion and network hopping cause delay. As a result, unlike analog, this is not real-time, but it is more semi-real-time. This can differ from network and manufacturer. Latency can be affected by network architecture, and also throttling as described earlier. Latency really becomes apparent when an operator is trying to control an integral pan/tilt/zoom camera. The delay is on both ends and can be further elongated when using encryption to minimize interception of camera data. The operator sends the command and the camera reacts. Both have delay as a result of the conversion to packets and routing though the same network.
The simplest way to see the impact of the latency is to use an TCP/IP encoder routed/looped through an analog security video switcher. When you operate the camera from the analog switcher, you will see the type of delay that could be expected with a digital system. Currently, manufacturers address this problem by camera setting pre-sets, which minimize the perception of delay compared to a variable control that were all used to from an analog system.
Last but not least, CSOs need to describe the final and most important part of the networked digital solution. This is typically overlooked, which results in the throttling by network administrators. We talked about the storage requirements of video. As you recall there are systems requiring storage requirements as large as 8 terabytes. As more is added on to a shared network, more available bandwidth is consumed. Bandwidth is related to storage in that it is finite and not limitless. In general terms, bandwidth is the data and its size in transit. For this reason, many camera and digital recording manufacturers have recommended gigabyte stand-alone TCP/IP networks. As primary networks become congested with the prevalence of other shared resources, IT will set its eyes on these reserved networks. Make no mistake, this author has seen firsthand IT professionals take over backbones that were reserved for security in an effort to further their own goals.
Outside network infrastructure (cabling) requirements must be met. Networks establish communication through open ports and through either unicast or multi-cast environments. Some TCP/IP cameras and systems may require specific ports to be opened, which will allow communication. In addition to ports, cameras/encoders may require specific network environments that may not be possible with existing network infrastructure. It is important to involve IT stakeholders when contemplating a Web-based system, if their not already involved.
OPERATING SYSTEMSPreviously with analog systems, the transmission of video was based on standards. However the migration to digital has opened up a whole new realm of flexibility and options that were previously not possible.
These options are the result of software packages, which were loaded on computers. When digital recording technologies first emerged, other manufacturers quickly reacted to minimize the loss of market share. However, the quickest way to get a product to market was to use standard off the shelf parts. PCI video capture cards and computer operating systems were used to run software, which regulated the output of the digital recordings. The use off the shelf commercial operating systems, in my opinion was in error. The reason being is that the digital recording technologies were connected to shared networks, which were susceptible to viruses, Trojans, worms, hacking and so forth. I am aware of multiple organizations, which have tracked down network attacks to malware loaded on security networks or digital video recorders. Manufacturers responded with the only solution they had -- to load memory hungry anti-virus software that required updates to stay current. In some instances, the memory required to operate the virus software took away from the resources required to record video. At that time, this was the only solution, other than the disconnection or creation of a secondary network or subnet
Later, embedded operating systems emerged. These systems are different in that the operating system that regulates the software options and recording was located on a non-volatile memory chip, which virus or malware cannot access and therefore cannot modify. As a result, embedded systems did not require third party virus software and were immune to malware previously used to exploit computer networks that these digital recorders were attached to. Although there is still a number of digital recording technologies that use off the shelf operating systems, there is a trend towards embedded operating systems because of their malware and anti-hacking properties.
COMPRESSION/IMAGE MODIFICATIONWith analog camera systems, the biggest concern that end-users had was image quality, because storage and recording were somewhat defined. Initially, the push for digital technologies, as discussed, came about because of the ability to access video randomly.
However, in many cases, digital technologies perform worse than digital analog technologies. That’s right. Many believe that there is better picture quality on an analog camera system than a digital solution. So why all the buzz about digital? It seems that we have forgotten about one of the biggest issues in lieu of all “digital buzz” and software features that are being thrust upon us. When it comes down to it, CSOs should not forget picture quality. Unfortunately, there really is no comparison between manufacturers to manufacturers, which requires s “shoot-out.” A shoot out is exactly what it implies, which is a comparison of products based on a similar set-up and field of view. It is from this, we can evaluate the quality of each of the products.
Why is this required? Well, picture quality can be modified based on the size of the video capture, colors and other factors. There are two types of compression, lossy and lossless. Compression is somewhat of a misnomer. Although we are reducing the size of the image, we are not “compressing” but rather extracting “un-necessary” information to reduce the file size. As an experiment, take a data compression tool such as Win-zip, Win-RAR or some other type. Take a standard image or video file (group of pictures), and using the software compression tool, compress it. You will notice that the file size does not differ much from the original image file and the new compressed file.
Beyond compression, there are other image modifications that take place to further reduce the size of the video file, or “group-of-pictures,” which MPEG–4 uses.
Another factor in the digital arena is the quality that can be achieved by using a camera that acts as an encoder, and digitally transmits the information to a digital recorder. When it comes to picture quality, cameras directly attached to the network excel and provide crisp pictures because there is no loss in the conversion from analog to digital. Digital cameras directly connected to a TCP/IP network will always look better than traditional analog cameras. However, the final compression may not be equal to what you are viewing on the monitor. There is always a loss when converting digital to analog and vice versa. Each time the video is converted, there will be a loss of information. This includes the transmission of non-connected TCP/IP cameras that terminate to an encoder. These analog cameras will terminate on an encoder and will have to be converted to digital, prior to transmission on the network
Once in a digital format, they will have to be converted back to analog. This is why, if you have a digital recording solution, they almost always use Web cameras in lieu of analog cameras, because of the lack of loss and better image quality. Don’t be misled, analog cameras will not look like the video and pictures generated by directly connected TCP/IP cameras. CSOs need to look at video quality, which will use infrastructure/equipment that is comparable to the enterprise. Also, obtain samples of video and Bitmap (.BMP) screen captures for further review.
INTEROPERABILITYThe move to directly connected TCP/IP cameras means that many chief security officers are moving out of the analog world. Many manufacturers are making this change very subtly. But make no mistake, TCP/IP means the move away from analog. In addition, although some manufactures support a MJPEG video stream from their cameras, for the most part Web cameras are not open like their analog ancestors. There are no standards like those of analog, which provided design criteria for interoperability. In lieu of this interoperability, manufactures will offer software development kits (SDKs). SDK is a shield that a manufacturer offers, which is somewhat like telling the owner, here are the keys to the system. But often there are no directions to where the “lock” is and how the keys are inserted. Coming are common standards, which will offer reduced functionality/quality, but a common platform for all TCP/IP manufacturers to design their equipment too.
THE POLITICS OF IT ALLMore security video manufacturers are trying to enlist the assistance of IT professionals in the furtherance of their goals. However, even IT professionals are getting concerned about the massive data requirements, but still push for this technology, because ultimately the more they can put on the network, the bigger their profit center begins.
Make no mistake. IT is an internal profit center, which obtains its budget from the services they provide. During a catastrophe, the focus for IT will be to restore network and crucial network services, not necessarily security. In addition, end-users should realize that manufacturers are in constant competition, and TCP/IP and convergence buzzwords are the result of real or perceived market share. In an effort for manufactures to minimize the loss of market share, companies have invested millions of dollars in research and development. Losing market share means losing business. However, these manufacturers want to recoup the cost of R&D, and regardless if the market is ready will push their technology. Is it no wonder why that every technology we look at is TCP/IP? In addition, end-users will be shocked by a new word when evaluating NVR technologies. That word? License! Manufacturers are requiring annuity or one-time license fees for all analog and digital cameras connected to their recorders through encoders. The fine print can be a hard pill to swallow when CSOs have invested so much money in the conversion from analog to digital. In addition, the move towards IP technology leaves salesmen without the technical acuity to understand or provide the equipment that the end user desires. This results in change orders to the end-user when it does not work based on the user’s preconceived notion of their previous analog system.
SIDEBAR: Redundant, Redundancy – The Rule Of 2The “Rule of 2” is what Sean Ahrens came up with to describe the infrastructure required for digital technologies. Everything, with exception of cabling, must be redundant to provide the same level of concurrent service current to analog systems. This means everything from TCP/IP data switches in each closet that cameras are connected to and data recorders storage devices must have a redundant counterpart. Data switchers should be redundant because they are typically serviced during the evening when the least amount of network traffic and commensurate disruption would occur. Taking a data switcher down for service would be segregating an TCP/IP camera from the network and recorder. Most digital TCP/IP network technologies will transmit video back to a series of recorders, which will buffer video and transmit this video to a archive NAS/SAN or other type of connected storage device.
The benefit of digital TCP/IP recording technologies provides a level of continuity unsurpassed by analog. Specifically, with the proper redundancy, the loss of a recorder connected to TCP/IP cameras could automatically re-adjust to a spare recorder or shared across multiple existing recorders. To do this, a separate computer, properly backed up would administer the video on the recorder as well as the archival storage unit. In addition to this infrastructure, if chief security officers wish to replace existing analog viewing monitors with a digital Web-based solution, the CSO or his/her integrator would again need a computer with a digital video interface (DVI) connection. As with the other digital infrastructure, there is that “Rule of 2” with these computers. too. The perception by the end-user is that the system will operate just like an analog system, which is far from the truth. When considering all of these computers, you should consider the costs for maintenance (patches, virus updates, etc) by your IT group.