Abstract This paper presents an in depth comparison of state‐of‐the‐art precise shadowing techniques for an omnidirectional point light. We chose several types of modern shadowing algorithms, starting from stencil shadow volumes, methods using traversal of acceleration structures to hardware‐accelerated ray‐traced shadows. Some methods were further improved – robustness, increased performance; we also provide the first multi‐platform implementations of some of the tested algorithms. All the methods are evaluated on several test scenes in different resolutions and on two hardware platforms – with and without dedicated hardware units for ray tracing. We conclude our findings based on speed and memory consumption. Ray‐tracing is the fastest and one of the easiest methods to implement with small memory footprint. The Omnidirectional Frustum‐Traced Shadows method has a predictable memory footprint and is the second fastest algorithm tested. Our stencil shadow volumes are faster than some newer algorithms. Per‐Triangle Shadow Volumes and Clustered Per‐Triangle Shadow Volumes are difficult to implement and require the most memory; the latter method scales well with the scene complexity and resolution. Deep Partitioned Shadow Volumes does not excel in any of the measured parameters and is suitable for smaller scenes. The source codes of the testing framework have been made publicly available.

The aim of this paper is to provide a comprehensive overview of the MICCAI 2020 AutoImplant Challenge. The approaches and publications submitted and accepted within the challenge will be summarized and reported, highlighting common algorithmic trends and algorithmic diversity. Furthermore, the evaluation results will be presented, compared and discussed in regard to the challenge aim: seeking for low cost, fast and fully automated solutions for cranial implant design. Based on feedback from collaborating neurosurgeons, this paper concludes by stating open issues and post-challenge requirements for intra-operative use. The codes can be found at https://github.com/Jianningli/tmi.

In this paper, we propose a new approach to automatic calibration of surveillance cameras. The proposed method is based on observing rigid objects in the scene and automatically estimating landmarks on these objects. The proposed approach can use arbitrary rigid objects, as was verified by experiments with a synthetic dataset, but vehicles were used during our experiments with real-life data. Landmarks on objects automatically detected by a convolutional neural network together with corresponding 3D positions in the object coordinate system are exploited during the camera calibration process. To determine 3D positions of the landmarks, fine-grained classification of the detected vehicles in the image plane is necessary. The proposed calibration method consists of dual optimization-optimization of objects' positions in the world coordinate system and also optimization of the calibration parameters to minimize the re-projection error of the localized landmarks. The experiments show improvement in calibration accuracy over the existing method solving a similar problem furthermore with fewer restrictions on the input data. The calibration error on a real world dataset decreased from 6.88% to 2.85%.

In this paper, we focus on traffic camera calibration and a visual speed measurement from a single monocular camera, which is an important task of visual traffic surveillance. Existing methods addressing this problem are difficult to compare due to a lack of a common data set with reliable ground truth. Therefore, it is not clear how the methods compare in various aspects and what factors are affecting their performance. We captured a new data set of 18 full-HD videos, each around 1 hr long, captured at six different locations. Vehicles in the videos (20 865 instances in total) are annotated with the precise speed measurements from optical gates using LiDAR and verified with several reference GPS tracks. We made the data set available for download and it contains the videos and metadata (calibration, lengths of features in image, annotations, and so on) for future comparison and evaluation. Camera calibration is the most crucial part of the speed measurement; therefore, we provide a brief overview of the methods and analyze a recently published method for fully automatic camera calibration and vehicle speed measurement and report the results on this data set in detail.

In this paper, we focus on fine-grained recognition of vehicles mainly in traffic surveillance applications. We propose an approach that is orthogonal to recent advancements in fine-grained recognition (automatic part discovery and bilinear pooling). In addition, in contrast to other methods focused on fine-grained recognition of vehicles, we do not limit ourselves to a frontal/rear viewpoint, but allow the vehicles to be seen from any viewpoint. Our approach is based on 3-D bounding boxes built around the vehicles. The bounding box can be automatically constructed from traffic surveillance data. For scenarios where it is not possible to use precise construction, we propose a method for an estimation of the 3-D bounding box. The 3-D bounding box is used to normalize the image viewpoint by “unpacking” the image into a plane. We also propose to randomly alter the color of the image and add a rectangle with random noise to a random position in the image during the training of convolutional neural networks (CNNs). We have collected a large fine-grained vehicle data set BoxCars116k, with 116k images of vehicles from various viewpoints taken by numerous surveillance cameras. We performed a number of experiments, which show that our proposed method significantly improves CNN classification accuracy (the accuracy is increased by up to 12% points and the error is reduced by up to 50% compared with CNNs without the proposed modifications). We also show that our method outperforms the state-of-the-art methods for fine-grained recognition.

This work is focused on recognition of license plates in low resolution and low quality images. We present a methodology for collection of real world (non-synthetic) dataset of low quality license plate images with ground truth transcriptions. Our approach to the license plate recognition is based on a Convolutional Neural Network which holistically processes the whole image, avoiding segmentation of the license plate characters. Evaluation results on multiple datasets show that our method significantly outperforms other free and commercial solutions to license plate recognition on the low quality data. To enable further research of low quality license plate recognition, we make the datasets publicly available.

We are dealing with the problem of fine-grained vehicle make&model recognition and verification. Our contribution is showing that extracting additional data from the video stream - besides the vehicle image itself - and feeding it into the deep convolutional neural network boosts the recognition performance considerably. This additional information includes: 3D vehicle bounding box used for "unpacking" the vehicle image, its rasterized low-resolution shape, and information about the 3D vehicle orientation. Experiments show that adding such information decreases classification error by 26% (the accuracy is improved from 0.772 to 0.832) and boosts verification average precision by 208% (0.378 to 0.785) compared to baseline pure CNN without any input modifications. Also, the pure baseline CNN outperforms the recent state of the art solution by 0.081. We provide an annotated set "BoxCars" of surveillance vehicle images augmented by various automatically extracted auxiliary information. Our approach and the dataset can considerably improve the performance of traffic surveillance systems.

We present a novel way of odometry estimation from Velodyne LiDAR point cloud scans. The aim of our work is to overcome the most painful issues of Velodyne data - the sparsity and the quantity of data points - in an efficient way, enabling more precise registration. Alignment of the point clouds which yields the final odometry is based on random sampling of the clouds using Collar Line Segments (CLS). The closest line segment pairs are identified in two sets of line segments obtained from two consequent Velodyne scans. From each pair of correspondences, a transformation aligning the matched line segments into a 3D plane is estimated. By this, significant planes (ground, walls, ...) are preserved among aligned point clouds. Evaluation using the KITTI dataset shows that our method outperforms publicly available and commonly used state-of-the-art method GICP for point cloud registration in both accuracy and speed, especially in cases where the scene lacks significant landmarks or in typical urban elements. For such environments, the registration error of our method is reduced by 75% compared to the original GICP error.

We present an object detector coupled with pose estimation directly in a single compact and simple model, where the detector shares extracted image features with the pose estimator. The output of the classification of each candidate window consists of both object score and likelihood map of poses. This extension introduces negligible overhead during detection so that the detector is still capable of real time operation. We evaluated the proposed approach on the problem of vehicle detection. We used existing datasets with viewpoint/pose annotation (WCVP, 3D objects, KITTI). Besides that, we collected a new traffic surveillance dataset COD20k which fills certain gaps of the existing datasets and we make it public. The experimental results show that the proposed approach is comparable with state-of-the-art approaches in terms of accuracy, but it is considerably faster - easily operating in real time (Matlab with C++ code). The source codes and the collected COD20k dataset are made public along with the paper.

Augmented reality does not make any sense for fixed cameras. Or does it? In this work, we are dealing with static cameras and their usability for interactive augmented reality applications. Knowing that the camera does not move makes camera pose estimation both less and more difficult — one does not have to deal with pose change in time, but on the other hand, obtaining some level of understanding of the scene from a single viewpoint is challenging. We propose several ways how to gain advantage from the camera being static and a pipeline of a system for broadcasting a video stream enriched by information needed for its interactive visual augmenting — Interactive Camera Streams, INCAST. We present a proof-of-concept system showing the usability of INCASt on several use-cases — non-interactive demos and simple AR games.

This paper deals with automatic calibration of roadside surveillance cameras. We focus on parameters necessary for measurements in traffic-surveillance applications. Contrary to the existing solutions, our approach requires no a priori knowledge, and it works with a very wide variety of road settings (number of lanes, occlusion, quality of ground marking), as well as with practically unlimited viewing angles. The main contribution is that our solution works fully automatically-without any percamera or per-video manual settings or input whatsoever-and it is computationally inexpensive. Our approach uses tracking of local feature points and analyzes the trajectories in a manner based on cascaded Hough transform and parallel coordinates. An important assumption for the vehicle movement is that at least a part of the vehicle motion is approximately straight-we discuss the impact of this assumption on the applicability of our approach and show experimentally that this assumption does not limit the usability of our approach severely. We efficiently and robustly detect vanishing points, which define the ground plane and vehicle movement, except for the scene scale. Our algorithm also computes parameters for radial distortion compensation. Experiments show that the obtained camera parameters allow for measurements of relative lengths (and potentially speed) with ~2% mean accuracy. The processing is performed easily in real time, and typically, a 2-min-long video is sufficient for stable calibration.

This paper deals with the detection of orthogonal vanishing points. The aim is to efficiently cope with the clutter edges in real-life images and to determine the camera orientation in the Manhattan world reliably. We are using a modified scheme of the Cascaded Hough Transform where only one Hough space is accumulated – the space of the vanishing points. The parameterization of the vanishing points – the “diamond space” – is based on the PClines line parameterization and it is defined as a mapping of the whole real projective plane to a finite space. Our algorithm for detection of vanishing points operates directly on edgelets detected by an edge detector, skipping the common step of grouping edges into straight lines or line segments. This decreases the number of configuration parameters and reduces the complexity of the algorithm. Evaluated on the York Urban DB, our algorithm yields 98.04 % success rate at 10◦ angular error tolerance, which outperforms comparable existing solutions. Our parameterization of vanishing points is in all aspects linear; it involves no goniometric or other non-linear operations and thus it is suitable for implementation in embedded chips and circuitry. The iterative search scheme allows for finding orthogonal triplets of vanishing points with high accuracy and low computational costs. At the same time, our approach can be used without the orthogonality constraint.

We introduce here an improved design of the Uniform Marker Fields and an algorithm for their fast and reliable detection. Our concept of the marker field is designed so that it can be detected and recognized for camera pose estimation: in various lighting conditions, under a severe perspective, while heavily occluded, and under a strong motion blur. Our marker field detection harnesses the fact that the edges within the marker field meet at two vanishing points and that the projected planar grid of squares can be defined by a detectable mathematical formalism. The modules of the grid are grey scale and the locations within the marker field are defined by the edges between the modules. The assumption that the marker field is planar allows for a very cheap and reliable camera pose estimation in the captured scene. The detection rates and accuracy are slightly better compared to state-of-the-art marker-based solutions. At the same time, and more importantly, our detector of the marker field is several times faster and the reliable real-time detection can be thus achieved on mobile and low-power devices. We show three targeted applications where the planarity is assured and where the presented marker field design and detection algorithm provide a reliable and extremely fast solution.

In this article, we propose an instant matchmoving solution for green screen. It uses a recent technique of planar uniform marker fields. Marker fields are an extension of planar markers used in augmented reality, offering better reliability and performance suitable for our task: tolerance to occlusion, speed of detection, and use of arbitrary low-contrast colors. We show that marker fields of shades of green (or blue or other color) can be used to obtain an instant and effortless camera pose estimation. We provide exemplar applications of the presented technique: virtual camera/simulcam and live storyboarding or shot prototyping. The matchmoving technique based on marker fields of shades of green is very computationally efficient - our measurements show that the matchmoving preview and living storyboard editing and recording can be easily done on today's ultramobile devices. Our technique is thus available to anyone at low cost and with easy setup, opening space for new levels of filmmakers' creative expression.

One limitation of existing fiduciary markers is that the camera motion is tightly limited: the marker (one of the markers) must be visible and it must be observed at a proper scale. This paper introduces a fractal structure of markers similar to matrix codes (such as QR-code or Data Matrix): the Fractal Marker Field. The FMF allows for embedding markers of a virtually unlimited number of scales. At the same time, for each of the scales it guarantees a constant density of markers at that scale over the whole marker field's surface. The Fractal Marker Field can provide unprecedented freedom of motion to camera-based augmented reality applications.

In various applications, a wider area needs to be covered by fiduciary markers but a large marker cannot be used because only a fraction of the area is to be viewed by the camera. Such an area can be covered by a number of small markers with unique identifiers. However, with the camera freely moving in the scene and with occluders present, it is difficult to ensure that at least one of the individual markers is completely visible, unless the markers are small and numerous. In that case, the markers are not recognizable from larger distances.

Detection of lines in raster images is often performed using Hough transform. This paper presents a new parameterization of lines and a modification of the Hough transform–PClines. PClines are based on parallel coordinates, a coordinate system used mostly or solely for high-dimensional data visualization. The PClines algorithm is described in the paper; its accuracy is evaluated numerically and compared to the commonly used line detectors based on the Hough transform. The results show that PClines outperform the existing approaches in terms of accuracy. Besides, PClines are computationally extremely efficient, require no floating-point operations, and can be easily accelerated by different hardware architectures.

This article presents an optimized algorithm for Nonnegative Tensor Factorization (NTF), implemented in the CUDA (Compute Uniform Device Architecture) framework, that runs on contemporary graphics processors and exploits their massive parallelism. The NTF implementation is primarily targeted for analysis of high-dimensional spectral images, including dimensionality reduction, feature extraction, and other tasks related to spectral imaging; however, the algorithm and its implementation are not limited to spectral imaging. The speedups measured on real spectral images are around 60-100{\times} compared to a traditional C implementation compiled with an optimizing compiler. Since common problems in the field of spectral imaging may take hours on a state-of-the-art CPU, the speedup achieved using a graphics card is attractive. The implementation is publicly available in the form of a dynamically linked library, including an interface to MATLAB, and thus may be of help to researchers and engineers using NTF on large problems.

Ray-triangle intersection is an important algorithm, not only in the field of realistic rendering (based on ray tracing) but also in physics simulation, collision detection, modeling, etc. Obviously, the speed of this well-defined algorithm's implementations is important because calls to such a routine are numerous in rendering and simulation applications. Contemporary fast intersection algorithms, which use SIMD instructions, focus on the intersection of ray packets against triangles. For intersection between single rays and triangles, operations such as horizontal addition or dot product are required. The SSE4 instruction set adds the dot product instruction which can be used for this purpose. This paper presents a new modification of the fast ray-triangle intersection algorithms commonly used, which-when implemented on SSE4-outperforms the current state-of-the-art algorithms. It also allows both a single ray and ray packet intersection calculation with the same precomputed data. The speed gain measurements are described and discussed in the paper.