The Beginning

Video recording is one of the most important tasks in the field of Closed Circuit Television (CCTV).  Originally standard VHS type Video Cassette Recorders (VCRs) were used.  Using endless tapes in combination with slow recording speeds, from 4 to 8 hours could be recorded on a single VHS-type cassette.  Without changing the tape, the recording would be endlessly overwritten, thereby keeping a history of events for the duration of the tape.  Slower recording speeds and the time-lapse mode brought longer coverage per tape.

For longer storage times, tapes have to be changed either manually or by robotic systems.  Because of the endless operation and the mechanical nature of the physical recording, tapes and video heads are subject to significant wear, necessitating the exchange of tapes and video heads during regular maintenance cycles.  For economic reasons, exchange intervals were stretched and sometimes forgotten, resulting in the infamous washed out video recordings, where hardly anything could be recognized.

Retrieval is also a challenge, with endless tape rewindings and with no clues other than the hopefully accurate labeling of the tapes.  On the positive side, VHS tapes can be played back on all of VCR decks, and everybody is able to handle them.

DVRs to the Rescue

The combination of affordable image compression technology and large capacity hard disks made the development of digital video recorders feasible.  Here video is stored in digitized and compressed form on regular computer hard disks.

Although hard disk recording is also a magnetic-recording technique, in contrast to video tapes, the magnetic heads do not touch the surface of the disks.  Instead, the heads hover at a small distance above the surface, causing no wear to the surfaces.  Moreover, the digital nature of the data ensures that all retrieved footage is an identical copy of the recorded signals.

Another advantage of DVRs is instantaneous retrieval, even while recording is still ongoing.  Any part of the recording is available for instantaneous playback without rewinding or the need for any other access time.  And additional information like motion activity or meta data may be stored in parallel with the video data itself in order to aid in the retrieval process.

All these advantages have led to a significant growth in DVR technology and an almost complete replacement of analog solutions.  In previous years, DVRs came at a premium.  But recently, they come at the same or even lower cost than older VCRs.  DVRs are now the technology of choice, especially for small installations.

For larger installations, solutions based on standard DVRs have several disadvantages:

• Local Video Inputs: All video inputs are local to the DVR.  Any video source, i.e. camera, has to be wired to the location of the DVR(s).  Significant investment in cabling is the result.
• Analog Video-based Design: With the future being digital, current DVRs only support conventional analog CCTV cameras.  Current DVRs are unable to work with the growing base of all-digital (network) cameras.
• Limited Capacity Design: DVRs are designed for a certain capacity, based on the number of internal hard disks times available and supported hard disk sizes.  For a larger number of cameras and/or larger recording times, separate DVR units have to be used.  Retrieval across distinct units then becomes impractical.
• Multiplexed operation: Due to the legacy of VCR, most DVRs are one-channel designs.  In order to accommodate 4, 9, 16 or more video inputs, internal or external multiplexers are used.  The performance per camera then typically drops to a few frames per second.
• Non-standard CCTV-specific equipment: DVRs are mostly specific designs that do not take advantage of common IT principles like RAID storage or standard, cost-efficient IT equipment.

The development of network video recording overcomes these drawbacks.

NVR - NETWORK VIDEO RECORDING

We are currently witnessing a significant growth in the use of network cameras in today’s CCTV applications.  Instead of a conventional analog video output, these cameras interface directly with IT-style Ethernet networks.  The reason for the popularity of so-called IP cameras is the availability of a network in most buildings and integrated features like motion detection, scene analysis and alarm features.  Using the network as the transmission medium saves the cost of dedicated cabling and compensates for the higher price-tag of today’s IP-cameras.

But how do we interface these IP cameras with DVRs?  In most cases, IP cameras cannot be connected to standard DVRs and cannot be mixed with conventional analog cameras.  The only way to connect an IP camera to a standard DVR would be through a decoder box, which converts the digital signal back into an analog video signal.  This analog signal can be recorded with conventional DVRs, which in turn digitize and compress the signal.  It is easy to grasp that this cascade of coding, decoding and recording is likely to degrade quality and is not a very elegant solution, to say the least.

The solution to the above problem is the so-called ‘Networked Video Recording (NVR)’ principle.  With the NVR, all digital video sources or analog cameras connected to video servers feed their digital data streams into the network.  A computer with sufficient storage capacity serves as the recorder and the NVR accesses the data streams of the remote network cameras and video servers and stores them on the computer hard disk.

An NVR is basically a standard networked PC with a software application that controls the flow of digital video data.  Usually a scheduling agent is used that controls access to video streams driven by time or activity in the scene.  NVRs usually also have mechanisms for retrieval of archived footage and backup capabilities.

NVR technology is a cost-efficient and modern way of recording digital video.  With IT-concepts such as ‘Storage Area Networks (SAN)’ or ‘Network Attached Storage (NAS)’ the capacity can be scaled into the terabyte and even the pentabyte range.  Despite these advantages, the weak point is the network.  Without the network there is no recording!

COPING WITH NETWORK DOWNTIME

The core of the Automatic Network Replenishment (ANR) concept is the ability to account for network downtimes.  The cause of the network downtime can be many.  Examples are:

• Equipment Failure: Any of the network components may fail, from simple cabling over switches, to routers or the storage server itself.
• Operator Fault: The wrong cables may be pulled during an attempt to reroute wiring or a configuration change may introduce an error.
• Network Congestion: Heavy traffic might cause the network service to become unavailable for video recording

Step1: Detection of Network Status

The basis of the ‘ANR’ concept is the detection of network availability.  To this end, the video server or the IP-camera on one side and the NVR application on the other side exchange so-called LIVE-CHECK messages.

Whenever the network status changes, i.e. the network becomes unavailable and later becomes available again, both the video source and the NVR create an event.  This event is logged in the database and may be used to send a message over alternative networks or at a later time when the network becomes available again.

The Live-Check mechanism effectively ensures that both sender and receiver monitor and keep a log of network availability.

Step2: Dual Recording Principle

To cope with times when the network is not available, video recording has to take place independently at both sides, namely at the sender and the receiver.

To achieve this, all Bosch Security Systems video servers and IP-cameras are equipped with sender-side recording capabilities.  Without actual user interaction, digital video data is streamed through a ring-buffer inside the camera or video server.

Physical implementation of the ring-buffer may vary.  Examples are standard hard drives, flash-memories or flash drives, which offer the added benefit of having no moving parts, making them well suited to harsh environments.  Even standard volatile memory may be used to buffer video data.  However, capacity is usually limited and due to the volatile nature, data is not preserved when power is disconnected.

The capacity of the sender-side storage is directly proportional to expected network downtimes plus a safety margin.  If, for example, a 1 MBit/s video data stream recorded on a central NVR is to be secure against network downtimes of 8 hours, a local storage capacity of app. 3.5 GBytes is required.

A safety margin must be added to account for delayed detection of network status changes and for clean edit marks inside the video stream.  So in the above example, 4 GBytes would be the recommended.  For reasons explained below, this figure should be doubled, which brings capacity to around 8 GBytes, a capacity easily achievable with today’s hard-disk technology.

It can also be seen that flash memories and even more so standard memory can only serve to bridge short network failure or spurious unavailability since affordable flash memory capacity today is in the order of 32 MBytes up to 4 GBytes.

Step3: Automatic Replenishment

The last step in the ‘ANR’ concept aims at repairing the corrupted recording at the NVR.

Thanks to the detection of network status changes, both the sender and the NVR are aware of the gap in the recording at the NVR.  When the network becomes available again, an automatic replenishment process is initiated.  In this, the missing data at the NVR side is copied from the intact recording at the sender side, as illustrated by the picture below.  In effect, the corrupted recording is seamlessly repaired, resulting in a flawless NVR recording despite the network fault.

The reason for the doubling of the required recording capacity at the sender side is to allow for the time required to copy the missing data from the sender to the NVR.  Assuming that the network is capable of the higher rate, it is further assumed that the copy process runs at the same speed as the recording itself, e.g. 1 MBit/s in the above example.  During the copy process of the missing data, recording of the current digital video data has to continue both at the sender and at the NVR side, thereby doubling the data rate on the network for the duration of the replenishment process.

In practice and with today’s networks, the copy process will take place at a significantly higher speed, e.g. 5 MBit/s, effectively lowering the requirement for buffer capacity at the sender side.

VIRTUAL HARD DISK

We have already mentioned above, that DVR technology is plagued by the limited local capacity of the hard disks used.  Physical limitations in the chassis of the DVR prevent the number of disks from being extended.  And while external extensions are usually possible, they only push the limit to the next step, simply postponing but not solving the problem.

Moreover, recording on more than a single drive usually makes retrieval and backup very cumbersome.  In most cases, recording stops when a hard drive fills to capacity and resumes on a second hard disk, effectively creating a gap in recording.

The concept of ‘Virtual Hard Disk’ is borrowed from today’s computer systems, where a virtual memory concept is commonplace.  The idea is that the physically present memory is seamlessly extended by a much larger virtual memory that is held on hard disk.  The CPU (central processing unit) is fooled into believing that it is operating with the larger memory by a logic unit called ‘Memory Management Unit (MMU)’  which is responsible for loading and unloading just the section of memory that the CPU currently needs.

The same concept may be readily employed for digital video recording.  The local hard disk inside the video server or the IP-camera is just a segment of a much larger virtual disk.  During the process of recording video to the local hard disk, the same amount of data is being copied over the network to a network storage device.  Data that has been successfully copied over the network may then be erased from the local hard disk, effectively freeing capacity on the local drive.  The net effect is that the local hard disk will never fill up, as long as the network storage device can take the data.

The copy process itself may run continuously at the same speed as the actual video recording or at a much faster speed in bursts.  In the latter case, the copying commences as soon as a certain threshold has been reached, e.g. half of the local hard disk capacity has been used.  According to the FIFO (first in - first out) principle, the oldest data on the local hard disk is backed-up over the network to a network video recorded and deleted locally.  The copy process is totally transparent for the video server, which just operates with a hard disk that never fills up.

The virtual hard disk makes the retrieval of recorded video footage especially convenient. Instead of searching over several physical disk volumes, the user always sees a single disk of sufficient capacity.

The Best of All Worlds

The two concepts introduced above are not mutually exclusive.  In fact, it becomes quite natural to combine both concepts for added benefit.  There are two aspects in the design of the overall system that will ensure the feasibility of the combination of the ‘ANR’ and the ‘Virtual Hard Disk’vconcept:

• Threshold: The trigger point for the start of the backup process should be engineered carefully.  An earlier start, e.g. less than 50% capacity of the local hard disk has been exhausted, ensures ample capacity for buffering network downtimes.  In the worst case scenario, the networks could become unavailable just when the copy process is about to commence.
• Data Integrity: It is essential to check that the data backed-up from the local drive to the network storage server is intact before deleting the local recording portion.  Even if the network fails during the copy process, network protocols are designed to cope with that.  The available capacity on the local drive should, moreover, ensure sufficient headroom for buffering.

MORE TO COME

Compared with early tape-based time lapse video recording, today’s Digital Video Recording (DVR) is a vast improvement.  With the advent of digital cameras, new recording principles are required that interface with network cameras.

Most of the network video recorders on the market today are not yet ready for professional use, because they rely totally on the availability of the network.  Only when the complete system is capable of handling network downtimes without sacrificing recording coverage and integrity will network video recording be applicable to professional installations.

This article introduced the concept of ‘ANR’  which goes beyond network video recording (NVR).  A combination of local recording for caching and large background archives serves even the largest installation.  The large centralized archive facilitates backup and searching across the complete available footage.  The system, moreover, is error-proof and scales nicely to any size, thanks to its combination of decentralized security and centralized convenience.

Even smaller installations can benefit from the concepts introduced above.  Hard disk capacity, often seen as the only limitation to the capabilities of a DVR, now becomes less meaningful since the concept of ‘Virtual Hard Drive’ extends the capacity of a physical disk over the network.  The least recently recorded data is copied over the network to a centralized background storage device.

Even with small economical physical drives, ring and time-lapse recording times of any size are achievable.  To the user, the system is absolutely transparent.  One cannot tell whether the recording resides on a single large drive or is spread out over a network.

It has been shown that today’s DVRs do not represent the end of the technological advancement but rather are the beginning of intelligent recording devices that are more user-friendly and economical.

Konrad Simon is Product Manager of IP Network Video, Bosch Security Systems.