In this work a CNN-based rail track detection algorithm and two novel evaluation metrics are proposed. Rails define the region of interest for object detection and localization algorithms of railbound vehicles, like lane markings do for automotive driver assistance functions. Looking at the analogies in significance and appearance of both, it becomes apparent that rail and lane marking detection could be solved similarly. Hence, this paper firstly introduces rail detection using an adopted version of PINet, a regression net for lane marking detection. The network is completely re-trained using a novel loss function and our own railway dataset. Secondly, a post-processing approach for clustering the detected rails into tracks using geometric constraints is proposed. Finally, two track detection metrics are introduced: The rail position offset metric (RPOM) and the track centerline offset metric (TCOM), which allow precise assessment of rail and track centerline detection results and can be cornerstones to foster future developments in this area.

Neural Networks are the state-of-the-art technology for environmental perception in applications such as autonomous driving. However, they require a large amount of training data in order to perform well, making the selection and annotation of sensor data a time-consuming and expensive task. Active learning is a promising approach to reduce the required amount of training data by selecting samples for annotation that are expected to improve the neural network the most. In this work, we propose a sequence-based active learning approach that selects sequences of consecutive frames instead of individual images. This allows to evaluate tracking algorithms and to reduce the annotation effort by interpolating labels between frames. Our approach is compared to a random sampling strategy as baseline. Over 15 iterations, both approaches select 1000 additional images for training in each iteration. The performance of the neural network trained on the data selected by our sequence-based active learning approach is compared to the performance of the network trained on the data select by the baseline approach. The results show that sequence-based active learning can reduce the required amount of training data by up to 25% while reaching a similar performance. Furthermore, sequence-based active learning can improve the neural network's overall performance by 2 % compared to a random sampling strategy. In this work, the proposed method was evaluated with a new dataset consisting of 15 scenes in railway environments. The dataset has 45,888 frames in total, 14,513 frames contain persons on railway stations or close to tracks.

We propose a scale-adaptive crowd detection and counting approach for drone images. Based on local feature points and density estimation considering the image scale, we detect dense crowds over multiple distances and introduce an extremely fast counting strategy with high accuracy for our detected crowd regions. We compare our results with a recent CNN-based state-of-the-art approach and validate both methods for different scaling factors on a novel crowd dataset. The results show that our proposed method outperforms the pre-trained CNN-based approach and receives very precise counting results for different zoom factors, resolutions and crowd sizes. Its low computational complexity makes it highly suitable for real-time analysis or embedded systems.

Tracking-by-detection is a common approach to multi-object tracking. With ever increasing performances of object detectors, the basis for a tracker becomes much more reliable. In combination with commonly higher frame rates, this poses a shift in the challenges for a successful tracker. That shift enables the deployment of much simpler tracking algorithms which can compete with more sophisticated approaches at a fraction of the computational cost. We present such an algorithm and show with thorough experiments its potential using a wide range of object detectors. The proposed method can easily run at 100K fps while outperforming the state-of-the-art on the DETRAC vehicle tracking dataset.

Tracking-by-detection becomes more and more popular for visual pedestrian tracking applications. However, it requires accurate and reliable detections in order to obtain good results. In this work, we propose two different post-detection filters designed to enhance the performance of custom person detectors. Using a popular deformable-parts-based pedestrian detector as a baseline, a detailed comparison over multiple test videos is performed and the gain of both algorithms is proven. Further analysis shows that the improved detection outcomes also lead to improved tracking results. We thus found that the usage of the proposed post-detection filters is recommendable as they do not impose a high computational load and are not limited to a specific detector method.

This work applies the Gaussian Mixture Probability Hypothesis Density (GMPHD) Filter to multi-object tracking in video data. In order to take advantage of additional visual information, Kernelized Correlation Filters (KCF) are evaluated as a possible extension of the GMPHD tracking-by-detection scheme to enhance its performance. The baseline GMPHD filter and its extension are evaluated on the UA-DETRAC benchmark, showing that combining both methods leads to a higher recall and a better quality of object tracks to the cost of increased computational complexity and increased sensitivity to false-positives.

The rapid advances of transportation infrastructure have led to a dramatic increase in the demand for smart systems capable of monitoring traffic and street safety. Fundamental to these applications are a community-based evaluation platform and benchmark for object detection and multi-object tracking. To this end, we organize the AVSS2017 Challenge on Advanced Traffic Monitoring, in conjunction with the International Workshop on Traffic and Street Surveillance for Safety and Security (IWT4S), to evaluate the state-of-the-art object detection and multi-object tracking algorithms in the relevance of traffic surveillance. Submitted algorithms are evaluated using the large-scale UA-DETRAC benchmark and evaluation protocol. The benchmark, the evaluation toolkit and the algorithm performance are publicly available from the website http://detrac-db.rit.albany.edu.

Scientific interest in automated abandoned object detection algorithms using visual information is high and many related systems have been published in recent years. However, most evaluation techniques rely only on statistical evaluation on the object level. Therefore and due to benchmarks with commonly only few abandoned objects and a non-standardized evaluation procedure, an objective performance comparison between different methods is generally hard. We propose a new evaluation metric which is focused on an end-user application case and an evaluation protocol which eliminates uncertainties in previous performance assessments. Using two variants of an abandoned object detection method, we show the features of the novel metric on multiple datasets proving its advantages over previously used measures.

Lagrangian theory provides a rich set of tools for analyzing non-local, long-term motion information in computer vision applications. Based on this theory, we present a specialized Lagrangian technique for the automated detection of violent scenes in video footage. We present a novel feature using Lagrangian direction fields that is based on a spatio-temporal model and uses appearance, background motion compensation, and long-term motion information. To ensure appropriate spatial and temporal feature scales, we apply an extended bag-of-words procedure in a late-fusion manner as a classification scheme on a per-video basis. We demonstrate that the temporal scale, captured by the Lagrangian integration time parameter, is crucial for violence detection and show how it correlates to the spatial scale of characteristic events in the scene. The proposed system is validated on multiple public benchmarks and non-public, real-world data from the London Metropolitan Police. Our experiments confirm that the inclusion of Lagrangian measures is a valuable cue for automated violence detection and increases the classification performance considerably compared with the state-of-the-art methods.

Convolutional neural networks are a popular choice for current object detection and classification systems. Their performance improves constantly but for effective training, large, hand-labeled datasets are required. We address the problem of obtaining customized, yet large enough datasets for CNN training by synthesizing them in a virtual world, thus eliminating the need for tedious human interaction for ground truth creation. We developed a CNN-based multi-class detection system that was trained solely on virtual world data and achieves competitive results compared to state-of-the-art detection systems.

During the standardization of HEVC, new motion information coding and prediction schemes such as temporal motion vector prediction have been investigated to reduce the spatial redundancy of motion vector fields used for motion compensated inter prediction. In this paper a general motion model based vector coding scheme is introduced. This scheme includes estimation, coding and dynamic recombination of parametric motion models to generate vector predictors and merge candidates for all common HEVC inter coding settings. Bit rate reductions of up to 4.9% indicate that higher order motion models can increase the efficiency of motion information coding in modern hybrid video coding standards.

We propose a new methodology for crowd analysis by introducing the concept of Multi-Person Density. Using a state-of-the-art feature tracking algorithm, representative low-level features and their long-term motion information are extracted and combined into a human detection model. In contrast to previously proposed techniques, the proposed method takes small camera motion into account and is not affected by camera shaking. This increases the robustness of separating crowd features from background and thus opens a whole new field for application of these techniques in non-static CCTV cameras. We show the effectiveness of our approach on various test videos and compare it to state-of-the-art people counting methods.

In this paper we present a method of improving a human detector by means of crowd density information. Human detection is especially challenging in crowded scenes which makes it important to introduce additional knowledge into the detection process. We compute crowd density maps in order to estimate the spatial distribution of people in the scene and show how it is possible to enhance the detection results of a state-of-the-art human detector by this information. The proposed method applies a self-adaptive, dynamic parametrization and as an additional contribution uses scene-adaptive learning of the human aspect ratio in order to reduce false positive detections in crowded areas. We evaluate our method on videos from different datasets and demonstrate how our system achieves better results than the baseline algorithm.

The Probability Hypothesis Density (PHD) filter is a multi-object Bayes filter which has recently attracted a lot of interest in the tracking community mainly for its linear complexity and its ability to deal with high clutter especially in radar/sonar scenarios. In the computer vision community however, underlying constraints are different from radar scenarios and have to be taken into account when using the PHD filter. In this article, we propose a new tree-based path extraction algorithm for a Gaussian Mixture PHD filter in Computer Vision applications. We also investigate how an additional benefit can be achieved by using a second human detector and justify an approximation for multiple sensors in low-clutter scenarios.

This paper is motivated by the problem of local motion estimation via robust regression with linear models. In order to increase the robustness of the motion estimates, we propose a novel robust local optical flow approach based on a modified Hampel estimator. We show the deficiencies of the least squares estimator used by the standard Kanade-Lucas-Tomasi (KLT) tracker when the assumptions made by Lucas-Kanade are violated. We propose a strategy to adapt the window sizes to cope with the generalized aperture problem. Finally, we evaluate our method on the Middlebury and MIT dataset and show that the algorithm provides excellent feature tracking performance with only slightly increased computational complexity compared to KLT. To facilitate further development, the presented algorithm can be downloaded from http://www.nue.tu-berlin.de/menue/forschung/projekte/rlof.

In this paper we present a decentralized surveillance network composed of IP video cameras, analysis devices and a central node which collects information and displays it in a 3D model of the complete area. The exchange of information between all components in the surveillance network takes place according to the ONVIF specification, therefore ensuring interoperability between products complying with the specification and flexibility regarding the integration of new devices and services. The collected information is displayed in a 3D model of the surveilled area, therefore providing a comfortable overview of the activity in large environments and offering the user an intuitive way to eventually interact with network devices.

In this paper we present an approach for the efficient computation of optical flow fields in real-time and provide implementation details. Proposing a modification of the popular Lucas-Kanade energy functional based on integral projections allows us to speed up the method notably. We show the potential of this method which can compute dense flow fields of 640×480 pixels at a speed of 4 fps in a GPU implementation based on the OpenCL framework. Working on sparse optical flow fields of up to 17,000 points, we reach execution times of 70 fps. Optical flow methods are used in many different areas, the proposed method speeds up current surveillance algorithms used for scene description and crowd analysis or Augmented Reality and robot navigation applications.

This paper describes a robust method for the local optical flow estimation and the KLT feature tracking performed on the GPU. Therefore we present an estimator based on the L2norm with robust characteristics. In order to increase the robustness at discontinuities we propose a strategy to adapt the used region size. The GPU implementation of our approach achieves real-time (>;25 fps) performance for High Definition (HD) video sequences while tracking several thousands of points. The benefit of the suggested enhancement is illustrated on the Middlebury optical flow benchmark.