Multi-view object tracking (MVOT) offers promising solutions to challenges such as occlusion and target loss, which are common in traditional single-view tracking. However, progress has been limited by the lack of comprehensive multi-view datasets and effective cross-view integration methods. To overcome these limitations, we compiled a Multi-View object Tracking (MVTrack) dataset of 234K high-quality annotated frames featuring 27 distinct objects across various scenes. In conjunction with this dataset, we introduce a novel MVOT method, Multi-View Integration Tracker (MITracker), to efficiently integrate multi-view object features and provide stable tracking outcomes. MI-Tracker can track any object in video frames of arbitrary length from arbitrary viewpoints. The key advancements of our method over traditional single-view approaches come from two aspects: (1) MITracker transforms 2D image features into a 3D feature volume and compresses it into a bird’s eye view (BEV) plane, facilitating inter-view information fusion; (2) we propose an attention mechanism that leverages geometric information from fused 3D feature volume to refine the tracking results at each view. MI-Tracker outperforms existing methods on the MVTrack and GMTD datasets, achieving state-of-the-art performance. The code and the new dataset will be available at mii-laboratory.github.io/MITracker. 1

Alignment between human knowledge and machine learning models is crucial for achieving efficient and interpretable AI systems. However, conventional self-supervised pre-training methods often suffer from low efficiency, as they do not incorporate human knowledge during the pre-training process and instead rely mainly on post-hoc alignment techniques. We propose Gaze Pre-Training (GzPT), a novel approach that introduces early alignment with human eye gaze information during the pre-training process to enhance both the learning efficiency and performance of self-supervised models. By leveraging contrastive learning to pull together images with similar gaze patterns, GzPT can effectively align the model with human attention during the pre-training. We demonstrate the effectiveness of our approach on three diverse medical image datasets, showing that GzPT can consistently outperform baseline methods and learn more meaningful and interpretable representations. Our findings also highlight the potential of incorporating human eye gaze as a form of passive knowledge to bridge the gap between human and machine learning in the self-supervised pre-training. Our code is available at Github.

Brain serves as a critical cornerstone of human intelligence, which involves a series of complex neuropsychological activities that lead to the coordination of various functions in the brain network. In recent years, brain network analysis methods based on graph neural networks (GNNs) have attracted increasing attention for the identification of brain disorders. However, these methods generally assume that the brain network is a homogeneous graph while ignoring its heterogeneity among human brain activities, which is reflected in both the complex connectivity of the brain network and distinctive brain functions. To overcome this problem, we propose a heterogeneous subdivision GNN (HSGNN), which captures the heterogeneous connections and functions of the brain network simultaneously. Specifically, we first employ two fundamental brain connectivity patterns to capture both statistical dependency and directional information flow among different brain regions and construct a heterogeneous brain connectivity network for each subject. Then, we develop a functional subdivision method that encodes brain networks into multiple latent feature subspaces corresponding to heterogeneous brain functions and extracts features of brain networks accordingly. Considering the intricate interactions of brain functions to facilitate cognitive activities within the brain network, we further employ the self-attention mechanism to obtain comprehensive representations of brain networks in a joint latent space. Finally, we propose a composite loss function to train the model for obtaining the heterogeneous brain network representation, which can be utilized for disease classification. The experimental results in the Alzheimer's Disease Neuroimaging Initiative (ADNI) and Autism Brain Imaging Data Exchange (ABIDE) datasets demonstrate that our method outperforms several state-of-the-art (SOTA) methods to identify different types of brain cognitive-related disorders.