Online Multi-Object Tracking (MOT) plays a pivotal role in autonomous systems. The state-of-the-art approaches usually employ a tracking-by-detection method, and data association plays a critical role. This paper proposes a learning and graph-optimized (LEGO) modular tracker to improve data association performance in the existing literature. The proposed LEGO tracker integrates graph optimization, which efficiently formulates the association score map, facilitating the accurate and efficient matching of objects across time frames. To further enhance the state update process, the Kalman filter is added to ensure consistent tracking by incorporating temporal coherence in the object states to further enhance the state update process. Our proposed method, utilising LiDAR alone, has shown exceptional performance compared to other online tracking approaches, including LiDAR-based and LiDAR-camera fusion-based methods. LEGO ranked 3rdamong all trackers (both online and offline) and 2ndamong all online trackers in the KITTI MOT benchmark for cars1, at the time of submitting results to KITTI object tracking evaluation ranking board. Moreover, our method also achieves competitive performance on the Waymo open dataset benchmark.

This paper uses multi-object tracking methods known from the radar tracking community to address the problem of pedestrian tracking using 2D bounding box detections. The standard point-object (SPO) model is adopted, and the posterior density is computed using the Poisson multi-Bernoulli mixture (PMBM) filter. The selection of the model parameters rooted in continuous time is discussed, including the birth and survival probabilities. Some parameters are selected from the first principles, while others are identified from the data, which is, in this case, the publicly available MOT-17 dataset. Although the resulting PMBM algorithm yields promising results, a mismatch between the SPO model and the data is revealed. The model-based approach assumes that modifying the problematic components causing the SPO model-data mismatch will lead to better modelbased algorithms in future developments.

The concept of 6G distributed integrated sensing and communications (DISAC) builds upon the functionality of integrated sensing and communications (ISAC) by integrating distributed architectures, significantly enhancing both sensing and communication coverage and performance. In 6G DISAC systems, tracking target trajectories requires base stations (BSs) to hand over their tracked targets to neighboring BSs. Determining what information to share, where, how, and when is critical to effective handover. This paper addresses the target handover challenge in DISAC systems and introduces a method enabling BSs to share essential target trajectory information at appropriate time steps, facilitating seamless handovers to other BSs. The target tracking problem is tackled using the standard trajectory Poisson multi-Bernoulli mixture (TPMBM) filter, enhanced with the proposed handover algorithm. Simulation results confirm the effectiveness of the implemented tracking solution.

Diffusion models excel in high-fidelity image generation but face scalability limits due to transformers’ quadratic attention complexity. Plug-and-play token reduction methods like ToMeSD and ToFu reduce FLOPs by merging redundant tokens in generated images but rely on GPU-inefficient operations (e.g., sorting, scattered writes), introducing overheads that negate theoretical speedups when paired with optimized attention implementations (e.g., FlashAttention). To bridge this gap, we propose Token Merge with Attention (ToMA), an off-the-shelf method that redesigns token reduction for GPU-aligned efficiency, with three key contributions: 1) a reformulation of token merging as a submodular optimization problem to select diverse tokens; 2) merge/unmerge as an attention-like linear transformation via GPU-friendly matrix operations; and 3) exploiting latent locality and sequential redundancy (pattern reuse) to minimize overhead. ToMA reduces SDXL/Flux generation latency by 24%/23% (DINO $\Delta <$ 0.07), outperforming prior methods. This work bridges the gap between theoretical and practical efficiency for transformers in diffusion.

Lexically Constrained Translation (LCT) refers to controlling Neural Machine Translation (NMT) models to generate accurate translations containing specific words or phrases by predefining constraint segments. Existing LCT methods lack full utilization of constraint segments and struggle to balance translation quality and constraint accuracy. To address these issues, we propose a multi-task learning-based two-stage lexically constrained translation method. First, by jointly modeling the generation of the constraint prefix composed of the target constraint segments and the decoding of the target translation and, the model utilizes the constraint prefix as a prompt to guide the generation of coarse translations containing constraint tags. Second, to detect and correct errors in the coarse translations, we introduce a cascaded multi-task framework to refine the coarse translations. Finally, the NMT task and the Target-Side Monolingual Text Generation (TMTG) task were incorporated as auxiliary tasks in the joint training with the LCT task, which further improved the overall translation quality. Experimental results demonstrate that the proposed LCT method outperforms state-of-the-art baseline methods, achieving an average improvement of 2.8 in BLEU, 1.08 in BLEURT, and 2.05 in Window Overlap on ZH-EN translation, and 0.49 in BLEU, 0.38 in BLEURT, and 0.65 in Window Overlap on DE-EN translation.

As the adoption of Generative AI in real-world services grow explosively, energy has emerged as a critical bottleneck resource. However, energy remains a metric that is often overlooked, under-explored, or poorly understood in the context of building ML systems. We present the ML.ENERGY Benchmark, a benchmark suite and tool for measuring inference energy consumption under realistic service environments, and the corresponding ML.ENERGY Leaderboard, which have served as a valuable resource for those hoping to understand and optimize the energy consumption of their generative AI services. In this paper, we explain four key design principles for benchmarking ML energy we have acquired over time, and then describe how they are implemented in the ML.ENERGY Benchmark. We then highlight results from the early 2025 iteration of the benchmark, including energy measurements of 40 widely used model architectures across 6 different tasks, case studies of how ML design choices impact energy consumption, and how automated optimization recommendations can lead to significant (sometimes more than 40%) energy savings without changing what is being computed by the model. The ML.ENERGY Benchmark is open-source and can be easily extended to various customized models and application scenarios.

Achieving fully autonomous driving with enhanced safety and efficiency relies on vehicle-to-everything (V2X) cooperative perception (CP), which enables vehicles to share perception data, thereby enhancing situational awareness and overcoming the limitations of the sensing ability of individual vehicles. V2X CP plays a crucial role in extending the perception range, increasing detection accuracy, and supporting more robust decision-making and control in complex environments. This article provides a comprehensive survey of recent developments in V2X CP, introducing mathematical models that characterize the perception process under different collaboration strategies. Key techniques for enabling reliable perception sharing, such as agent selection, data alignment, and feature fusion, are examined in detail. In addition, major challenges are discussed, including differences in agents and models, uncertainty in perception outputs, and the impact of communication constraints such as transmission delay and data loss. This article concludes by outlining promising research directions, including privacy-preserving artificial intelligence methods, collaborative intelligence, and integrated sensing frameworks to support future advancements in V2X CP.

Accurate 3D multi-object tracking (MOT) is crucial for autonomous driving, as it enables robust perception, navigation, and planning in complex environments. While deep learning-based solutions have demonstrated impressive 3D MOT performance, model-based approaches remain appealing for their simplicity, interpretability, and data efficiency. Conventional model-based trackers typically rely on random vector-based Bayesian filters within the tracking-by-detection (TBD) framework but face limitations due to heuristic data association and track management schemes. In contrast, random finite set (RFS)-based Bayesian filtering handles object birth, survival, and death in a theoretically sound manner, facilitating interpretability and parameter tuning. In this paper, we present OptiPMB, a novel RFS-based 3D MOT method that employs an optimized Poisson multi-Bernoulli (PMB) filter while incorporating several key innovative designs within the TBD framework. Specifically, we propose a measurement-driven hybrid adaptive birth model for improved track initialization, employ adaptive detection probability parameters to effectively maintain tracks for occluded objects, and optimize density pruning and track extraction modules to further enhance overall tracking performance. Extensive evaluations on nuScenes and KITTI datasets show that OptiPMB achieves superior tracking accuracy compared with state-of-the-art methods, thereby establishing a new benchmark for model-based 3D MOT and offering valuable insights for future research on RFS-based trackers in autonomous driving.

Multiple extended target tracking (ETT) has gained increasing attention due to the development of high-precision LiDAR and radar sensors in automotive applications. For LiDAR point cloud-based vehicle tracking, this paper presents a probabilistic measurement-region association (PMRA) ETT model, which can describe the complex measurement distribution by partitioning the target extent into different regions. The PMRA model overcomes the drawbacks of previous data-region association (DRA) models by eliminating the approximation error of constrained estimation and using continuous integrals to more reliably calculate the association probabilities. Furthermore, the PMRA model is integrated with the Poisson multi-Bernoulli mixture (PMBM) filter for tracking multiple vehicles. Simulation results illustrate the superior estimation accuracy of the proposed PMRA-PMBM filter in terms of both the positions and extents of vehicles compared with PMBM filters using the gamma Gaussian inverse Wishart and DRA implementations.

Accurate and timely determination of a vehicle’s current lane within a map is a critical task in autonomous driving systems. This paper utilizes an Early Time Series Classification (ETSC) method to achieve precise and rapid ego-lane identification in real-world driving data. The method begins by assessing the similarities between map and lane markings perceived by the vehicle’s camera using measurement model quality metrics. These metrics are then fed into a selected ETSC method, comprising a probabilistic classifier and a tailored trigger function, optimized via multi-objective optimization to strike a balance between early prediction and accuracy. Our solution has been evaluated on a comprehensive dataset consisting of 114 hours of real-world traffic data, collected across 5 different countries by our test vehicles. Results show that by leveraging road lane-marking geometry and lane-marking type derived solely from a camera, our solution achieves an impressive accuracy of 99.6%, with an average prediction time of only 0.84 seconds.

High-definition map with accurate lane-level information is crucial for autonomous driving, but the creation of these maps is a resource-intensive process. To this end, we present a cost-effective solution to create lane-level roadmaps using only the global navigation satellite system (GNSS) and a camera on customer vehicles. Our proposed solution utilizes a prior standard-definition (SD) map, GNSS measurements, visual odometry, and lane marking edge detection points, to simultaneously estimate the vehicle’s 6 D pose, its position within a SD map, and also the 3D geometry of traffic lines. This is achieved using a Bayesian simultaneous localization and multi-object tracking filter, where the estimation of traffic lines is formulated as a multiple extended object tracking problem, solved using a trajectory Poisson multi-Bernoulli mixture (TPMBM) filter. In TPMBM filtering, traffic lines are modeled using B-spline trajectories, and each trajectory is parameterized by a sequence of control points. The proposed solution has been evaluated using experimental data collected by a test vehicle driving on highway. Preliminary results show that the traffic line estimates, overlaid on the satellite image, generally align with the lane markings up to some lateral offsets.

Online 3D multi-object tracking (MOT) has recently received significant research interests due to the expanding demand of 3D perception in advanced driver assistance systems (ADAS) and autonomous driving (AD). Among the existing 3D MOT frameworks for ADAS and AD, conventional point object tracking (POT) framework using the tracking-by-detection (TBD) strategy has been well studied and accepted for LiDAR and 4D imaging radar point clouds. In contrast, extended object tracking (EOT), another important framework which accepts the joint-detection-and-tracking (JDT) strategy, has rarely been explored for online 3D MOT applications. This paper provides the first systematical investigation of the EOT framework for online 3D MOT in real-world ADAS and AD scenarios. Specifically, the widely accepted TBD-POT framework, the recently investigated JDT-EOT framework, and our proposed TBD-EOT framework are compared via extensive evaluations on two open source 4D imaging radar datasets: View-of-Delft and TJ4DRadSet. Experiment results demonstrate that the conventional TBD-POT framework remains preferable for online 3D MOT with high tracking performance and low computational complexity, while the proposed TBD-EOT framework has the potential to outperform it in certain situations. However, the results also show that the JDT-EOT framework encounters multiple problems and performs inadequately in evaluation scenarios. After analyzing the causes of these phenomena based on various evaluation metrics and visualizations, we provide possible guidelines to improve the performance of these MOT frameworks on real-world data. These provide the first benchmark and important insights for the future development of 4D imaging radar-based online 3D MOT algorithms.

As an emerging technology and a relatively affordable device, the 4D imaging radar has already been confirmed effective in performing 3D object detection in autonomous driving [1] . Nevertheless, the sparsity and noisiness of 4D radar point clouds hinder further performance improvement, and in-depth studies about its fusion with other modalities are lacking. On the other hand, as a new image view transformation strategy, sampling has been applied in a few image-based detectors and shown to outperform the widely applied depth-based splatting proposed in Lift-Splat-Shoot (LSS) [2] , even without image depth prediction [3] . However, the potential of sampling is not fully unleashed. As a result, this paper investigates the sampling strategy on the camera and 4D imaging radar fusion-based 3D object detection. In the proposed LiDAR Excluded Lean (LXL) model, predicted image depth distribution maps and radar 3D occupancy grids are generated from image perspective view (PV) features and radar bird’s eye view (BEV) features, respectively. They are sent to the core of LXL, called radar occupancy-assisted depth-based sampling , to aid image view transformation.

This paper presents an efficient implementation of the trajectory Poisson multi-Bernoulli (PMB) filter for multiple extended object tracking (EOT), which directly estimates a set of object trajectories. The trajectory PMB filter propagates a PMB density on the posterior of sets of trajectories through the filtering recursions over time, where the multi-Bernoulli (MB) mixture in the PMB mixture (PMBM) posterior after each update step is approximated as a single MB. The efficient MB approximation is achieved by first running a blocked Gibbs sampler on the joint posterior of the set of trajectories and the measurement association variables. The single-object measurement model is assumed to be a Poisson point process which enables us to parallelize the sampling across all objects and association variables, respectively. Then, samples of object states are utilized to form the approximate MB density via Kullback-Leibler divergence minimization. Simulation results on EOT with known and constant elliptical shapes show that the TPMB implementation using blocked Gibbs sampling outperforms the state-of-the-art TPMB implementation using loopy belief propagation with significantly reduced runtime.

The fusion of multiple probability densities has important applications in many fields, including, for example, multi-sensor signal processing, robotics, and smart environments. In this paper, we demonstrate that deep learning based methods can be used to fuse multi-object densities. Given a scenario with several sensors with possibly different field-of-views, tracking is performed locally in each sensor by a tracker, which produces random finite set multi-object densities. To fuse outputs from different trackers, we adapt a recently proposed transformer-based multi-object tracker, where the fusion result is a global multi-object density, describing the set of all alive objects at the current time. We compare the performance of the transformer-based fusion method with a well-performing model-based Bayesian fusion method in several simulated scenarios with different parameter settings using synthetic data. The simulation results show that the transformer-based fusion method outperforms the model-based Bayesian method in our experimental scenarios. The code is available at https://github.com/Lechili/DeepFusion.

Evaluating the performance of multi-object tracking (MOT) methods is not straightforward, and existing performance measures fail to consider all the available uncertainty information in the MOT context. This can lead practitioners to select models which produce uncertainty estimates of lower quality, negatively impacting any downstream systems that rely on them. Additionally, most MOT performance measures have hyperparameters, which makes comparisons of different trackers less straightforward. We propose the use of the negative log-likelihood (NLL) of the multi-object posterior given the set of ground-truth objects as a performance measure. This measure takes into account all available uncertainty information in a sound mathematical manner without hyperparameters. We provide efficient algorithms for approximating the computation of the NLL for several common MOT algorithms, show that in some cases it decomposes and approximates the widely-used GOSPA metric, and provide several illustrative examples highlighting the advantages of the NLL in comparison to other MOT performance measures.

This paper presents two multi-Bernoulli filters on sets of trajectories for multiple target tracking. The first filter provides a multi-Bernoulli approximation of the posterior density over the set of alive trajectories at the current time step. The second filter provides a multi-Bernoulli approximation of the posterior density over the set of all trajectories (alive and dead) up to the current time. We also explain the Gaussian implementation of the filters and compare them with other multiple target tracking algorithms in a simulated scenario.

In this paper we introduce spatiotemporal constraints for trajectories, i.e., restrictions that the trajectory must be in some part of the state space (spatial constraint) at some point in time (temporal constraint). Spatiotemporal contraints on trajectories can be used to answer a range of important questions, including, e.g., “where did the person that were in area A at time t, go afterwards?”. We discuss how multiple constraints can be combined into sets of constraints, and we then apply sets of constraints to set of trajectories densities, specifically Poisson Multi-Bernoulli Mixture (PMBM) densities. For Poisson target birth, the exact posterior density is PMBM for both point targets and extended targets. In the paper we show that if the unconstrained set of trajectories density is PMBM, then the constrained density is also PMBM. Examples of constrained trajectory densities motivate and illustrate the key results.

This paper presents a solution for recovering full trajectory information, via the calculation of the posterior of the set of trajectories, from a sequence of multitarget (unlabelled) filtering densities and the multitarget dynamic model. Importantly, the proposed solution opens an avenue of trajectory estimation possibilities for multitarget filters that do not explicitly estimate trajectories. In this paper, we first derive a general multitrajectory forward-backward smoothing equation based on sets of trajectories and the random finite set framework. Then we show how to sample sets of trajectories using backward simulation when the multitarget filtering densities are multi-Bernoulli processes. The proposed approach is demonstrated in a simulation study.

Motivated by real-world automotive radar measurements that are distributed around object (e.g., vehicles) edges with a certain volume, a novel hierarchical truncated Gaussian measurement model is proposed to resemble the underlying spatial distribution of radar measurements. With the proposed measurement model, a modified random matrix-based extended object tracking algorithm is developed to estimate both kinematic and extent states. In particular, a new state update step and an online bound estimation step are proposed with the introduction of pseudo measurements. The effectiveness of the proposed algorithm is verified in simulations.

The Poisson multi-Bernoulli mixture (PMBM) is an unlabelled multi-target distribution for which the prediction and update are closed. It has a Poisson birth process, and new Bernoulli components are generated on each new measurement as a part of the Bayesian measurement update. The PMBM filter is similar to the multiple hypothesis tracker (MHT), but seemingly does not provide explicit continuity between time steps. This paper considers a recently developed formulation of the multi-target tracking problem as a random finite set (RFS) of trajectories, and derives two trajectory RFS filters, called PMBM trackers. The PMBM trackers efficiently estimate the set of trajectories, and share hypothesis structure with the PMBM filter. By showing that the prediction and update in the PMBM filter can be viewed as an efficient method for calculating the time marginals of the RFS of trajectories, continuity in the same sense as MHT is established for the PMBM filter.

This paper proposes an efficient implementation of the Poisson multi-Bernoulli mixture (PMBM) trajectory filter. The proposed implementation performs track-oriented N-scan pruning to limit complexity, and uses dual decomposition to solve the involved multi-frame assignment problem. In contrast to the existing PMBM filter for sets of targets, the PMBM trajectory filter is based on sets of trajectories which ensures that track continuity is formally maintained. The resulting filter is an efficient and scalable approximation to a Bayes optimal multi-target tracking algorithm, and its performance is compared, in a simulation study, to the PMBM target filter, and the delta generalized labelled multi-Bernoulli filter, in terms of state/trajectory estimation error and computational time.

In this paper, we evaluate the performance of labelled and unlabelled multi-Bernoulli conjugate priors for multi-target filtering. Filters are compared in two different scenarios with performance assessed using the generalised optimal sub-pattern assignment (GOSPA) metric. The first scenario under consideration is tracking of well-spaced targets. The second scenario is more challenging and considers targets in close proximity, for which filters may suffer from coalescence. We analyse various aspects of the filters in these two scenarios. Though all filters have pros and cons, the Poisson multi-Bernoulli filters arguably provide the best overall performance concerning GOSPA and computational time.