The field of computer graphics was revolutionized by models such as NeRF and 3D Gaussian Splatting, displacing triangles as the dominant representation for photogrammetry. In this paper, we argue for a triangle comeback. We develop a differentiable renderer that directly optimizes triangles via end-to-end gradients. We achieve this by rendering each triangle as differentiable splats, combining the efficiency of triangles with the adaptive density of representations based on independent primitives. Compared to popular 2D and 3D Gaussian Splatting methods, our approach achieves competitive rendering and convergence speed, and demonstrates high visual quality. On the Mip-NeRF360 dataset, our method outperforms concurrent nonvolumetric primitives in visual fidelity and achieves higher perceptual quality than the state-of-the-art Zip-NeRF on indoor scenes. Triangles are simple, compatible with standard graphics stacks and GPU hardware, and highly efficient. Our results highlight the efficiency and effectiveness of triangle-based representations for high-quality novel view synthesis. Triangles bring us closer to mesh-based optimization by combining classical computer graphics with modern differentiable rendering frameworks. The project page is https://trianglesplatting.github.io/

In this simulator study, we adopt a human-centered approach to explore whether and how drivers' cognitive state and driving environment complexity influence reliance on driving automation features.Besides, we examine whether such reliance affects driving performance.Participants operated a vehicle equipped with adaptive cruise control (ACC) in a simulator across six predefined driving scenarios varying in traffic conditions while either performing a cognitively demanding task (i.e., responding to mental calculations) or not.Throughout the experiment, participants had to respect speed limits and were free to activate or deactivate ACC.In complex driving environments, we found that the overall ACC engagement time was lower compared to less complex driving environments.We observed no significant effect of cognitive load on ACC use.Furthermore, while ACC use had no effect on the number of lane changes, it impacted the speed limits compliance and improved lateral control.

Recent advances in radiance field reconstruction, such as 3D Gaussian Splatting (3DGS), have achieved high-quality novel view synthesis and fast rendering by representing scenes with compositions of Gaussian primitives. However, 3D Gaussians present several limitations for scene reconstruction. Accurately capturing hard edges is challenging without significantly increasing the number of Gaussians, creating a large memory footprint. Moreover, they struggle to represent flat surfaces, as they are diffused in space. Without hand-crafted regularizers, they tend to disperse irregularly around the actual surface. To circumvent these issues, we introduce a novel method, named 3D Convex Splatting (3DCS), which leverages 3D smooth convexes as primitives for modeling geometrically-meaningful radiance fields from multi-view images. Smooth convex shapes offer greater flexibility than Gaussians, allowing for a better representation of 3D scenes with hard edges and dense volumes using fewer primitives. Powered by our efficient CUDA-based rasterizer, 3DCS achieves superior performance over 3DGS on benchmarks such as MipNeRF360, Tanks and Temples, and Deep Blending. Specifically, our method attains an improvement of up to 0.81 in PSNR and 0.026 in LPIPS compared to 3DGS while maintaining high rendering speeds and reducing the number of required primitives. Our results highlight the potential of 3D Convex Splatting to become the new standard for high-quality scene reconstruction and novel view synthesis. The project page is https://convexsplatting.github.io

Ranking entities such as algorithms, devices, methods, or models based on their performances, while accounting for application-specific preferences, is a challenge. To address this challenge, we establish the foundations of a universal theory for performance-based ranking. First, we introduce a rigorous framework built on top of both the probability and order theories. Our new framework encompasses the elements necessary to (1) manipulate performances as mathematical objects, (2) express which performances are worse than or equivalent to others, (3) model tasks through a variable called satisfaction, (4) consider properties of the evaluation, (5) define scores, and (6) specify application-specific preferences through a variable called importance. On top of this framework, we propose the first axiomatic definition of performance orderings and performance-based rankings. Then, we introduce a universal parametric family of scores, called ranking scores, that can be used to establish rankings satisfying our axioms, while considering application-specific preferences. Finally, we show, in the case of two-class classification, that the family of ranking scores encompasses well-known performance scores, including the accuracy, the true positive rate (recall, sensitivity), the true negative rate (specificity), the positive predictive value (precision), and F1. However, we also show that some other scores commonly used to compare classifiers are unsuitable to derive performance orderings satisfying the axioms.

Camera calibration and localization, sometimes simply named camera calibration, enables many applications in the context of soccer broadcasting, for instance regarding the interpretation and analysis of the game, or the insertion of augmented reality graphics for storytelling or ref-ereeing purposes. To contribute to such applications, the research community has typically focused on single-view calibration methods, leveraging the near-omnipresence of soccer field markings in wide-angle broadcast views, but leaving all temporal aspects, if considered at all, to general-purpose tracking or filtering techniques. Only a few contributions have been made to leverage any domain-specific knowledge for this tracking task, and, as a result, there lacks a truly performant and off-the-shelf camera tracking system tailored for soccer broadcasting, specifically for elevated tripod-mounted cameras around the stadium. In this work, we present such a system capable of addressing the task of soccer broadcast camera tracking efficiently, robustly, and accurately, outperforming by far the most precise methods of the state-of-the-art. By combining the available open-source soccer field detectors with carefully designed camera and tripod models, our tracking system, BroadTrack, halves the mean reprojection error rate and gains more than 15% in terms of Jaccard index for camera calibration on the SoccerNet dataset. Furthermore, as the SoccerNet dataset videos are relatively short (30 seconds), we also present qualitative results on a 20-minute broadcast clip to show-case the robustness and the soundness of our system.

We introduce CURDIS, a template for algorithms to discretize arcs of regular curves by incrementally producing a list of support pixels covering the arc. In this template, algorithms proceed by finding the tangent quadrant at each point of the arc and determining which side the curve exits the pixel according to a tailored criterion. These two elements can be adapted for any type of curve, leading to algorithms dedicated to the shape of specific curves. While the calculation of the tangent quadrant for various curves, such as lines, conics, or cubics, is simple, it is more complex to analyze how pixels are traversed by the curve. In the case of conic arcs, we found a criterion for determining the pixel exit side. This leads us to present a new algorithm, called CURDIS-C, specific to the discretization of conics, for which we provide all the details. Surprisingly, the criterion for conics requires between one and three sign tests and four additions per pixel, making the algorithm efficient for resource-constrained systems and feasible for fixed-point or integer arithmetic implementations. Our algorithm also perfectly handles the pathological cases in which the conic intersects a pixel twice or changes quadrants multiple times within this pixel, achieving this generality at the cost of potentially computing up to two square roots per arc. We illustrate the use of CURDIS for the discretization of different curves, such as ellipses, hyperbolas, and parabolas, even when they degenerate into lines or corners.

In video understanding, action spotting consists in temporally localizing human-induced events annotated with single timestamps. In this paper, we propose a novel loss function that specifically considers the temporal context naturally present around each action, rather than focusing on the single annotated frame to spot. We benchmark our loss on a large dataset of soccer videos, SoccerNet, and achieve an improvement of 12.8% over the baseline. We show the generalization capability of our loss for generic activity proposals and detection on ActivityNet, by spotting the beginning and the end of each activity. Furthermore, we provide an extended ablation study and display challenging cases for action spotting in soccer videos. Finally, we qualitatively illustrate how our loss induces a precise temporal understanding of actions and show how such semantic knowledge can be used for automatic highlights generation.

Semantic background subtraction (SBS) has been shown to improve the performance of most background subtraction algorithms by combining them with semantic information, derived from a semantic segmentation network. However, SBS requires high-quality semantic segmentation masks for all frames, which are slow to compute. In addition, most state-of-the-art background subtraction algorithms are not real-time, which makes them unsuitable for real-world applications. In this paper, we present a novel background subtraction algorithm called Real-Time Semantic Background Subtraction (denoted RT-SBS) which extends SBS for real-time constrained applications while keeping similar performances. RT-SBS effectively combines a real-time background subtraction algorithm with high-quality semantic information which can be provided at a slower pace, independently for each pixel. We show that RT-SBS coupled with ViBe sets a new state of the art for real-time background subtraction algorithms and even competes with the non real-time state-of-the-art ones. Note that we provide python CPU and GPU implementations of RT-SBS at https://github.com/cioppaanthony/rt-sbs.

There exist many background subtraction algorithms to detect motion in videos. To help comparing them, datasets with ground-truth data such as CDNET or LASIESTA have been proposed. These datasets organize videos in categories that represent typical challenges for background subtraction. The evaluation procedure promoted by their authors consists in measuring performance indicators for each video separately and to average them hierarchically, within a category first, then between categories, a procedure which we name “summarization”. While the summarization by averaging performance indicators is a valuable effort to standardize the evaluation procedure, it has no theoretical justification and it breaks the intrinsic relationships between summarized indicators. This leads to interpretation inconsistencies. In this paper, we present a theoretical approach to summarize the performances for multiple videos that preserves the relationships between performance indicators. In addition, we give formulas and an algorithm to calculate summarized performances. Finally, we showcase our observations on CDNET 2014.

We develop a powerful probabilistic framework for the local characterization of surfaces and edges in range images. We use the geometrical nature of the data to derive an analytic expression for the joint probability density function (pdf) for the random variables used to model the ranges of a set of pixels in a local neighborhood of an image. We decompose this joint pdf by considering independently the cases where two real world points corresponding to two neighboring pixels are locally on the same real world surface or not. In particular, we show that this joint pdf is linked to the Voigt pdf and not to the Gaussian pdf as it is assumed in some applications. We apply our framework to edge detection and develop a locally adaptive algorithm that is based on a probabilistic decision rule. We show in an objective evaluation that this new edge detector performs better than prior art edge detectors. This proves the benefits of the probabilistic characterization of the local neighborhood as a tool to improve applications that involve range images.

We introduce the notion of semantic background subtraction, a novel framework for motion detection in video sequences. The key innovation consists to leverage object-level semantics to address the variety of challenging scenarios for background subtraction. Our framework combines the information of a semantic segmentation algorithm, expressed by a probability for each pixel, with the output of any background subtraction algorithm to reduce false positive detections produced by illumination changes, dynamic backgrounds, strong shadows, and ghosts. In addition, it maintains a fully semantic background model to improve the detection of camouflaged foreground objects. Experiments led on the CDNet dataset show that we managed to improve, significantly, almost all background subtraction algorithms of the CDNet leaderboard, and reduce the mean overall error rate of all the 34 algorithms (resp. of the best 5 algorithms) by roughly 50% (resp. 20%).

Given a video sequence acquired with a fixed camera, the generation of the stationary background of the scene is a challenging problem which aims at computing a reference image for a motionless background. For that purpose, we developed our method named LaBGen, which emerged as the best one during the Scene Background Modeling and Initialization (SBMI) workshop organized in 2015, and the IEEE Scene Background Modeling Contest (SBMC) organized in 2016. LaBGen combines a pixel-wise temporal median filter and a patch selection mechanism based on motion detection. To detect motion, a background subtraction algorithm decides, for each frame, which pixels belong to the background. In this paper, we describe the LaBGen method extensively, evaluate it on the SBI 2016 dataset and compare its performance with other background generation methods. We also study its computational complexity, the performance sensitivity with respect to its parameters, and the stability of the predicted background image over time with respect to the chosen background subtraction algorithm. We provide an open source C++ implementation at http://www.telecom.ulg.ac.be/labgen.

Estimating the stationary background of a video sequence is useful in many applications like surveillance, segmentation, compression, inpainting, privacy protection, and computational photography. To perform this task, we introduce the LaBGen-P method based on the principles of LaBGen and the conclusions drawn in the corresponding paper. It combines a pixel-wise median filter and a pixel selection mechanism based on a motion detection performed by the frame difference algorithm. By working with pixels instead of patches, as originally done in LaBGen, it avoids some discontinuities between different spatial areas and generates better visual results. In this paper, we describe the LaBGen-P method, study its performance on the sequences of the SBMnet dataset, and compare it to that of LaBGen and other methods on the same dataset. Both algorithms emerged as the best ones during the IEEE Scene Background Modeling Contest (SBMC) organized in 2016. However, as there is not yet a good understanding of the recommended metrics, and due to the small amount of video sequences provided with the corresponding ground truth, we have performed a subjective evaluation. More precisely, 35 human experts were asked to compare background images estimated by LaBGen-P and LaBGen, and select the best one. From these experiments, it turns out that the results of LaBGen-P are preferred for about two thirds of the video sequences. Note that we provide an open-source C++ implementation at http://www.telecom.ulg.ac.be/labgen.

Reliable measurements of feet trajectories are needed in some applications, such as biomedical applications. This paper describes the data processing pipeline used in GAIMS, which is a non-intrusive system that measures feet trajectories based on multiple range laser scanners. Our processing pipeline relies on a new tracking paradigm, and it is based on two innovative algorithms: the first algorithm localizes the feet directly from the observed point cloud without any clustering, and the other algorithm identifies the feet. After reviewing the various types of noise affecting the point cloud, this paper explains the limitations of the classical processing approach and gives an overview of our new pipeline. The effectiveness of the proposed approach is established by discussing the results that have been obtained in several studies based on GAIMS.

Positioning is a fundamental issue in mobile robot applications, and it can be achieved in multiple ways. Among these methods, triangulation based on angle measurements is widely used, robust, accurate, and flexible. This paper presents BeAMS, which is a new active beacon-based angle measurement system used for mobile robot positioning. BeAMS introduces several major innovations. One innovation is the use of a unique unsynchronized channel with on-off keying modulated infrared signals to measure angles and to identify the beacons. We also introduce a new mechanism to measure angles: Our system detects a beacon when it enters and leaves an angular window. We show that the estimator resulting from the center of this angular window provides an unbiased estimate of the beacon angle. A theoretical framework for a thorough performance analysis of BeAMS is provided. We establish the upper bound of the variance and validate this bound through experiments and simulations; the overall error measure of BeAMS is lower than 0.24° for an acquisition rate of 10 Hz. In conclusion, BeAMS is a low-power, flexible, and robust solution for angle measurement and a reliable component for robot positioning.

Positioning is a fundamental issue in mobile robot applications. It can be achieved in many ways. Among them, triangulation based on angles measured with the help of beacons is a proven technique. Most of the many triangulation algorithms proposed so far have major limitations. For example, some of them need a particular beacon ordering, have blind spots, or only work within the triangle defined by the three beacons. More reliable methods exist; however, they have an increasing complexity, or they require to handle certain spatial arrangements separately. In this paper, we present a simple and new three object triangulation algorithm, known as ToTal, that natively works in the whole plane and for any beacon ordering. We also provide a comprehensive comparison between many algorithms and show that our algorithm is faster and simpler than comparable algorithms. In addition to its inherent efficiency, our algorithm provides a very useful and unique reliability measure that is assessable anywhere in the plane, which can be used to identify pathological cases, or as a validation gate in Kalman filters.

This paper focuses on cover song recognition over a large dataset, potentially containing millions of songs. At this time, the problem of cover song recognition is still challenging and only few methods have been proposed on large scale databases. We present an efficient method for quickly extracting a small subset from a large database in which a correspondence to an audio query should be found. We make use of fast rejectors based on independent audio features. Our method mixes independent rejectors together to build composite ones. We evaluate our system with the Million Song Dataset and we present composite rejectors offering a good trade-off between the percentage of pruning and the percentage of loss.

Human detection in video streams is an important task in many applications including video surveillance. Surprisingly, only few papers have been devoted to this topic. This paper presents a new approach to detect humans in video streams. Our approach is based on the temporal information present in videos. A background subtraction algorithm is first used to segment the silhouettes of the users and the moving objects. Then a classification process in two steps determines for each connected component if it corresponds to the silhouette of a human or not. During the first step, a probabilistic information is computed for each pixel independently. The information from a subset of pixels is then gathered to predict the class of the observed silhouette. This paper presents the principles and some results obtained on real silhouettes. It is shown that our approach is efficient for the detection of humans in video streams.

This paper presents a technique for motion detection that incorporates several innovative mechanisms. For example, our proposed technique stores, for each pixel, a set of values taken in the past at the same location or in the neighborhood. It then compares this set to the current pixel value in order to determine whether that pixel belongs to the background, and adapts the model by choosing randomly which values to substitute from the background model. This approach differs from those based upon the classical belief that the oldest values should be replaced first. Finally, when the pixel is found to be part of the background, its value is propagated into the background model of a neighboring pixel. We describe our method in full details (including pseudo-code and the parameter values used) and compare it to other background subtraction techniques. Efficiency figures show that our method outperforms recent and proven state-of-the-art methods in terms of both computation speed and detection rate. We also analyze the performance of a downscaled version of our algorithm to the absolute minimum of one comparison and one byte of memory per pixel. It appears that even such a simplified version of our algorithm performs better than mainstream techniques.

Many computer vision systems try to infer semantic information about a video scene content by looking at the time series of the silhouettes of the moving objects. This paper proposes a new inter-frame feature set (signature) based on piecewise surfacic descriptions of binary silhouettes. It captures the dynamics of moving objects and compacts it into a robust set of features suitable for classification. To assess its ability to represent motion information, we use it to build a complete gait recognition algorithm that we test on a database of 21 different subjects. To highlight the efficiency of our signature, we use frontal views instead of side views of persons, which is less discussed in literature and is considered to be harder as the movement of legs is not visible. In that context, the high recognition rates obtained (over 95% of correct identifications) proves that our signature is appropriate to describe moving objects.

Background subtraction is a crucial step in many automatic video content analysis applications. While numerous acceptable techniques have been proposed so far for background extraction, there is still a need to produce more efficient algorithms in terms of adaptability to multiple environments, noise resilience, and computation efficiency. In this paper, we present a powerful method for background extraction that improves in accuracy and reduces the computational load. The main innovation concerns the use of a random policy to select values to build a samples-based estimation of the background. To our knowledge, it is the first time that a random aggregation is used in the field of background extraction. In addition we propose a novel policy that propagates information between neighboring pixels of an image. Experiment detailed in this paper show how our method improves on other widely used techniques, and how it outperforms these techniques for noisy images.

We address the problem of detecting slow-moving targets using a non-sidelooking monostatic space-time adaptive processing (STAP) radar. The construction of optimum weights at each range implies the estimation of the clutter covariance matrix. This is typically done by straight averaging of neighboring data snapshots. The range-dependence of these snapshots generally results in poor performance. We present two new methods that handle the range-dependence by exploiting the geometry of the direction-Doppler curves.