Good test data is crucial for driving new developments in computer vision (CV), but two questions remain unanswered: which situations should be covered by the test data, and how much testing is enough to reach a conclusion? In this paper we propose a new answer to these questions using a standard procedure devised by the safety community to validate complex systems: the hazard and operability analysis (HAZOP). It is designed to systematically identify possible causes of system failure or performance loss. We introduce a generic CV model that creates the basis for the hazard analysis and—for the first time—apply an extensive HAZOP to the CV domain. The result is a publicly available checklist with more than 900 identified individual hazards. This checklist can be utilized to evaluate existing test datasets by quantifying the covered hazards. We evaluate our approach by first analyzing and annotating the popular stereo vision test datasets Middlebury and KITTI. Second, we demonstrate a clearly negative influence of the hazards in the checklist on the performance of six popular stereo matching algorithms. The presented approach is a useful tool to evaluate and improve test datasets and creates a common basis for future dataset designs.

Test data plays an important role in computer vision (CV) but is plagued by two questions: Which situations should be covered by the test data and have we tested enough to reach a conclusion? In this paper we propose a new solution answering these questions using a standard procedure devised by the safety community to validate complex systems: The Hazard and Operability Analysis (HAZOP). It is designed to systematically search and identify difficult, performance-decreasing situations and aspects. We introduce a generic CV model that creates the basis for the hazard analysis and, for the first time, apply an extensive HAZOP to the CV domain. The result is a publicly available checklist with more than 900 identified individual hazards. This checklist can be used to evaluate existing test datasets by quantifying the amount of covered hazards. We evaluate our approach by first analyzing and annotating the popular stereo vision test datasets Middlebury and KITTI. Second, we compare the performance of six popular stereo matching algorithms at the identified hazards from our checklist with their average performance and show, as expected, a clear negative influence of the hazards. The presented approach is a useful tool to evaluate and improve test datasets and creates a common basis for future dataset designs.

Geometry generators are commonly used in video games and evaluation systems for computer vision to create geometric shapes such as terrains, vegetation or airplanes. The parameters of the generator are often sampled automatically which can lead to many similar or unwanted geometric shapes. In this paper, we propose a novel visual exploration approach that combines the abstract parameter space of the geometry generator with the resulting 3D shapes in a composite visualization. Similar geometric shapes are first grouped using hierarchical clustering and then nested within an illustrative parallel coordinates visualization. This helps the user to study the sensitivity of the generator with respect to its parameter space and to identify invalid parameter settings. Starting from a compact overview representation, the user can iteratively drill-down into local shape differences by clicking on the respective clusters. Additionally, a linked radial tree gives an overview of the cluster hierarchy and enables the user to manually split or merge clusters. We evaluate our approach by exploring the parameter space of a cup generator and provide feedback from domain experts.

Data acquisition by multidomain data acquisition provides means for environment perception usable for detecting unusual and possibly dangerous situations. When being automated, this approach can simplify surveillance tasks required in, for example, airports or other security sensitive infrastructures. This paper describes a novel architecture for surveillance networks based on combining multimodal sensor information. Compared to previous methodologies using only video information, the proposed approach also uses audio data thus increasing its ability to obtain valuable information about the sensed environment. A hierarchical processing architecture for observation and surveillance systems is proposed, which recognizes a set of predefined behaviors and learns about normal behaviors. Deviations from “normality” are reported in a way understandable even for staff without special training. The processing architecture, including the physical sensor nodes, is called smart embedded network of sensing entities (SENSE).

Distributed sensor networks, as opposed to centralized networks, offer several advantages in terms of versatility and increased safety, which make their use particularly relevant for applications of security surveillance. A challenge of such systems is how to build autonomously a global description of the sensed environment without supervision of a central processing unit and with minimal configuration effort. We present an approach to ubiquitous computing, based on a semantic representation of the world view in terms of correlation of local information learned at the local level. There, a statistical description of the sensed activity is provided. Correlations of events among nodes are learned using a Boltzmann machine approach and used in order to establish neighborhood correspondences. Moreover, the communication between nodes is used to enrich the local description of the sensed environment by approximating the a-posterior distributions by marginal distributions computed with the loopy belief propagation algorithm. We present results of simulations emulating a security surveillance environment in which the sensors are cameras and activity is learned by processing video data.