Implementing fine-grained emotion control is crucial for emotion generation tasks because it enhances the expressive capability of the generative model, allowing it to accurately and comprehensively capture and express various nuanced emotional states, thereby improving the emotional quality and personalization of generated content. Generating fine-grained facial animations that accurately portray emotional expressions using only a portrait and an audio recording presents a challenge. In order to address this challenge, we propose a visual attribute-guided audio decoupler. This enables the obtention of content vectors solely related to the audio content, enhancing the stability of subsequent lip movement coefficient predictions. To achieve more precise emotional expression, we introduce a fine-grained emotion coefficient prediction module. Additionally, we propose an emotion intensity control method using a fine-grained emotion matrix. Through these, effective control over emotional expression in the generated videos and finer classification of emotion intensity are accomplished. Subsequently, a series of 3DMM coefficient generation networks are designed to predict 3D coefficients, followed by the utilization of a rendering network to generate the final video. Our experimental results demonstrate that our proposed method, EmoSpeaker, outperforms existing emotional talking face generation methods in terms of expression variation and lip synchronization. Project page:https://peterfanfan.github.io/EmoSpeaker/

Trajectory anomaly detection is becoming increasingly important in various fields such as smart cities, autonomous vehicles, and video surveillance. In recent years, pattern learning-based trajectory anomaly detection methods have emerged as a research hotspot, demonstrating strong modeling capabilities and flexibility. This review focuses on pattern learning-based trajectory anomaly detection technologies, providing a comprehensive and in-depth review of the research progress and emerging challenges in this field over the past decade. First, the unique characteristics of trajectory anomaly detection problems and the current research challenges are thoroughly analyzed. Then, existing pattern learning-based trajectory anomaly detection algorithms are classified. For each category of algorithm, its principles, representative literature, and an objective evaluation of its advantages and disadvantages are discussed. Furthermore, common anomaly detection evaluation techniques, open-source trajectory datasets, and the comparison of representative methods on related datasets are explored. Finally, the critical open issues in the field are summarized, and a prospective outlook on future research trends and potential solutions is provided, particularly regarding the potential and challenges of deep learning-based models in handling multifarious trajectory data.

Virtual traffic simulation is a crucial aspect within the autonomous driving field. To guarantee the safety of autonomous driving tests, it is imperative to generate traffic flows that closely approximate real-world scenarios. In this paper, we present an unsupervised approach grounded in traffic trajectory anomaly detection to quantitatively assess the authenticity of arbitrary trajectories. This method can also serve to compare the modeling capabilities of existing simulation algorithms. Initially, we utilize the original trajectory data as input and leverage the variational autoencoder (VAE) architecture to accomplish the spatiotemporal feature learning and reconstruction of the trajectory. Subsequently, trajectory anomaly detection is achieved by contrasting the disparity between the model’s output and the original data. Normal trajectories typically exhibit favorable continuity and smoothness, resulting in reconstruction outcomes that are closer to the original trajectories, whereas anomalous trajectories display the opposite characteristics. Finally, we introduce perceptual evaluation and derive the trajectory authenticity evaluation function via polynomial fitting. Experiments conducted on multiple trajectories demonstrate that our method can effectively measure the authenticity of both real-world and synthetic data. Notably, our method directly employs the original position information of the trajectory for model training, thereby enhancing the practicality and robustness of the approach.

Different from large-scale classification tasks, fine-grained visual classification is a challenging task due to two critical problems: 1) evident intra-class variances and subtle inter-class differences, and 2) overfitting owing to fewer training samples in datasets. Most existing methods extract key features to reduce intra-class variances, but pay no attention to subtle inter-class differences in fine-grained visual classification. To address this issue, we propose a loss function named exploration of class center, which consists of a multiple class-center constraint and a class-center label generation. This loss function fully utilizes the information of the class center from the perspective of features and labels. From the feature perspective, the multiple class-center constraint pulls samples closer to the target class center, and pushes samples away from the most similar nontarget class center. Thus, the constraint reduces intra-class variances and enlarges inter-class differences. From the label perspective, the class-center label generation utilizes class-center distributions to generate soft labels to alleviate overfitting. Our method can be easily integrated with existing fine-grained visual classification approaches as a loss function, to further boost excellent performance with only slight training costs. Extensive experiments are conducted to demonstrate consistent improvements achieved by our method on four widely-used fine-grained visual classification datasets. In particular, our method achieves state-of-the-art performance on the FGVC-Aircraft and CUB-200-2011 datasets.

Virtual traffic benefits a variety of applications, including video games, traffic engineering, autonomous driving, and virtual reality. To date, traffic visualization via different simulation models can reconstruct detailed traffic flows. However, each specific behavior of vehicles is always described by establishing an independent control model. Moreover, mutual interactions between vehicles and other road users are rarely modeled in existing simulators. An all-in-one simulator that considers the complex behaviors of all potential road users in a realistic urban environment is urgently needed. In this work, we propose a novel, extensible, and microscopic method to build heterogeneous traffic simulation using the force-based concept. This force-based approach can accurately replicate the sophisticated behaviors of various road users and their interactions in a simple and unified manner. We calibrate the model parameters using real-world traffic trajectory data. The effectiveness of this approach is demonstrated through many simulation experiments, as well as comparisons to real-world traffic data and popular microscopic simulators for traffic animation.

Measuring the fidelity of synthesized virtual traffic has become an important and fundamental concern for evaluating the performance of different traffic simulation techniques and applications of autonomous vehicle testing. In this work, we propose a novel method to evaluate the fidelity of any trajectory data from the perspective of anomalous trajectory detection. First, given the trajectory data to be evaluated as input, the method learns spatio-temporal traffic features and reconstructs the input trajectory through a Long Short-Term Memory (LSTM)-based autoencoder architecture. Then, the anomalous trajectories are detected by comparing the reconstructed trajectories and the input ones using the reconstruction error as the benchmark. Our method can detect eight different kinds of anomalous trajectory in terms of changes in velocity and moving direction. In order to evaluate the fidelity of the input trajectory, we design a perceptual evaluation on virtual traffic fidelity and derive a mapping from the reconstruction error to the evaluation score. We demonstrated the effectiveness and robustness of our metric through many experiments on real-world and synthetic trajectory data containing different types of motion anomalies.