COMPUTER-ASSISTED VISION

Sight is probably the most valuable sense humans possess, and modern computer-assisted vision dramatically increases safety and security.  Nowhere is this more evident than in driving and in CCTV surveillance, two areas where it is vitally important to minimize the effects of human fallibility.  In both areas, humans are often bombarded with a massive amount of visual information that needs to be digested and interpreted very quickly, often in conditions of severely impaired visibility.  Both activities also demand undivided attention.  Even small distractions behind the wheel of a car can result in potentially disastrous situations.  And in surveillance, experience has shown that the fallibility of the human brain resulting from imperfect memory recall, biases, momentary distractions and boredom places severe limits on the reliability of conventional human-based video surveillance.  Interestingly, whereas the challenges of driving and CCTV surveillance appear to have a lot in common, so too do the solutions.  In particular, many of the advanced algorithms developed for intelligent video content analysis are generic in nature, which means they can be applied to equal advantage in either application area. 

SHARED DEVELOPMENT

It was recognition of this fact that prompted Bosch to establish its research center for video-based surveillance systems within its Competence Center for Surround Sensing Systems (CCS) in Hildesheim, Germany.  CCS has an unrivalled reputation for fundamental research in computer vision systems.  The fruits of this research have long been used in automotive systems -- primarily in-car driver- assistance and navigation systems, in which the company is acknowledged as a world market leader.  More recently, the fundamental research being performed at CCS has also greatly benefited Bosch Security Systems in the development of the company’s advanced Intelligent Video Analysis (IVA) software that has now become an important feature of its network IP video solutions.  A recent example of developments in the automotive area is the Night Vision system for the Mercedes-Benz S Class based on an infrared-sensitive video camera.  Infrared high-beam headlights illuminate an area 150 meters in front of the vehicle.  The infrared image is picked up by the video camera whose high-performance electronics converts the signals into an image visible to the human eye.  A central display enables drivers to identify dangerous situations more quickly, giving them more time to react.  The next stage in this development is the creation of driver-assistance systems that recognize defined characteristics and give the driver information about them.  New technology now ready for series production includes highly advanced processing technology that is able to use the image data delivered by the video camera to recognize whether pedestrians are standing or moving.  This means it can highlight pedestrians in color on the display, for example, and draw the driver’s attention to them.  Bosch’s road-sign recognition is a further example of video-based driver-assistance functions developed by CCS.  Here, developers have made the electronics systems so intelligent that they can pick out and recognize pre-defined road signs such as speed restrictions and ‘no overtaking’ signs from the video image, and then display the signs and remind the driver to observe them.  The advanced intelligent video-analysis algorithms used in developments like these are also the perfect basis for intelligent CCTV surveillance.  By being located within CCS, Bosch’s video-based surveillance research group is able to benefit from the center’s many years’ experience in this area and the high degree of synergy that exists between the various research groups.

MEETING SPECIAL CHALLENGES

Bosch Security Systems has been benefiting from the extensive know-how in video content analysis of the company’s Competence Center for Surround Sensing Systems since the late 1990s.  Initially the products concentrated on simple video motion detection.  The VMD01 video motion detector, introduced by Bosch in 2003, was already based on simple tracking of objects.  Developments accelerated from 2006 with the introduction of the company’s highly acclaimed IVA (Intelligent Video Analytics) software for its range of network IP (Internet Protocol) video products.  The introduction of ‘intelligence’ to video motion detection significantly increased the possibilities for ‘guard assisted’ CCTV surveillance and reduced demands on bandwidth and storage capacity.  The new software introduced a more event-based surveillance regime, ensuring that only scenes in which important changes occur would be captured, transmitted and stored.  Changes in environmental conditions are one of the major causes of false alarms in video motion detection systems.  To minimize these, Bosch’s IVA software includes an advanced background learning algorithm developed by CCS that allows for changes in background and saves computational power by suppressing unwanted notification from, for example, moving trees, branches, leaves, clouds, shadows and falling rain and snow.  Moreover, traditional video motion detection systems are implemented as stand-alone software systems running on an industrial PC platform.  These PCs are installed in a central location (e.g., in the control room), and continuous streams of monitored video are transmitted over the network for analysis and archiving.  This places a huge strain on network bandwidth, and the use of PC-based platforms for video analysis adds unnecessary costs for hardware, operating system, etc. and increases the risks of virus attacks.  Bosch overcame this problem by embedding the VCA functionality in the encoders and cameras themselves.  Video content is analyzed, compared against ‘known rules’, and events are generated at the edge of the network -- i.e., in the cameras.  Only video footage of interest (e.g., abnormal events) is transmitted to the control center.  This ‘smart camera’ approach of Bosch’s was a major departure from traditional video over IP systems as it eliminated the Single-Point-Of-Failure (SPOF), reduced network traffic and eliminated the overhead of a separate PC for running the VCA software.  In later developments, Bosch extended the range of alarm criteria to include object identification on the basis of aspect ratio, ‘idle-object’ detection for detecting items left at a scene or cars parked in sensitive locations, objectremoval detection for monitoring displays in, for example, museums and retail stores, and trajectory mapping for detecting suspicious behavior such as loitering.  An image stabilization feature for pole-mounted cameras was also

incorporated.  In IVA version 3.5, the latest release of its IVA software, Bosch has built on the features of the earlier versions with further enhanced detection possibilities.  These include new color filtering capabilities that allow object color or even a combination of colors to be detected.  This is embodied in a ‘color histogram’ function that allows object color or colors, saturation and precision to be set as monitoring criteria.  The filter set has also been extended with new powerful features such as object trajectories, line crossing alerts and aspect ratio filtering.  Triggers can be set to transmit alerts if, for example, objects cross a predefined line or multiple lines, or change speed (running), shape (crouching) or aspect ratio (falling).  Bosch’s IVA 3.5 also includes a new powerful forensic search function. Content analysis information, in the form of metadata, is automatically generated by the IVA software and stored with the video images.  The recorded metadata, comprising simple text strings describing specific image details, is much smaller and easier to search through than the recorded video.  Searches which may take days or even weeks when done manually can now be completed within seconds just by searching the metadata with smart search facilities like those provided by an Internet search engine.

DEVELOPMENTS IN THE PIPELINE

While the latest version of Bosch’s IVA software contains filters for detecting suspicious behavior such as loitering, even more powerful algorithms for analyzing crowd scenes are in development at CCS.  New exciting developments are also in the pipeline in video management, specifically in Bosch’s Video Management System (BVMS).  Today the system is 2D map based but future developments are expected to lead to a 3D interactive map-based system utilizing the video input from multiple cameras.  A greater level of interaction between cameras is also expected allowing powerful extensions to the IVA functionality.  This will include tools such as “camera handover of information”, where object identification and descriptors are automatically handed over from one camera to another, enabling objects to be recognized and reliably tracked over multiple scenes.  This will have a major impact on the effectiveness of guard-assisted surveillance, enabling security personnel to draw more confident conclusions about suspicious behavior.  In the case of an object left at a scene, for example, they would be able to conclude whether the owner is still in the neighborhood or has really left the building.  This function also opens up tremendous potential for more powerful forensic searching over several cameras. 

ENHANCING ROBUSTNESS AND RELIABILITY

Today’s video-based motion detectors are characterized by a low false alarm rate for perimeter protection.  But for large video installations with hundreds of cameras, further reduction of false alarm rates is still essential.  To contend with this, CCS has developed a stringent performance-evaluation process aimed at assessing the effects of every algorithmic change in order to optimize detection rate during development.  Based on the established process of ground truthing, the evaluation process compares manually annotated images (the ground truth) with images annotated automatically by the algorithm.  Discrepancies are then collected and, through a feedback process, the algorithm is incrementally adapted until its results agree with the ground truth data set (at least to within specified limits of precision).  Although time-consuming and involving many man-hours of labor, the process clearly leads to significant improvements in performance, and is one of the principal reasons why Bosch’s IVA software is now widely regarded as one of the most reliable and robust guard-assistance systems on the market.