Vision-language pretrained models have achieved significant success across various tasks. However, the lack of interpretability limits the application of multimodal models, especially for those that require the security of systems, data, and users. A key challenge in enhancing the interpretability of the models is the tradeoff between transparency and model performance in terms of accuracy and computing efficiency. We propose a vision-language multimodal pretrained model called multiway-fuzzy-experts bidirectional retention network (VL-MFER), which is designed to effectively interpret model decision processes while enhancing the consistent mapping between text and image features. We first propose a bidirectional retention network to handle and integrate the cross-modal high-dimensional data, which can effectively enhance both the performance and inference efficiency. Then, to improve the interpretability of vision-language pretrained models such as the unified vision-language pretraining with mixture-of-modality-experts, we propose a multiway fuzzy experts pool based on multiple different layered deep neuro-fuzzy systems for diverse downstream tasks. After training on diverse datasets consisting of images, text, and paired image-text data, we fine-tuned models for multiple vision and vision-language tasks. Experimental results show that VL-MFER performs well in all performance metrics of downstream tasks, and VL-MFER leads to enhancements in transparency compared to other methods, while improving computational efficiency and reducing inference time by up to 13%.

Few-shot object detection (FSOD) within remote sensing imagery has achieved great advancements in recent years. However, most existing methods are facing one key challenge while handling remote sensing images: many unlabeled instances in few-shot images are treated as background, which tends to degrade the generalization of the trained model severely. This paper presents HPMF, a Hypergraph-guided Prototype Mining Framework that addresses the challenge through joint optimization from three perspectives. The first is Hierarchical Reference Mining (HRM) which constructs a class-instance dual-driven prototype space that enables mining the unlabeled instances via cross-hierarchical similarity fusion. The second is a Robust Pseudo-box Estimator (RPE) that generates high-quality pseudo bounding boxes for the HRM-mined instances via adaptive density clustering and multi-statistic aggregation. The third is a Hypergraph-Guided Decoder (HGD) that introduces hypergraphs into the transformer decoder for group semantic modeling, enhancing high-order semantic association and similarity of instance features, thereby further improving the mining performance of the HRM module. Extensive experiments under various settings show that the proposed HPMF outperforms state-of-the-art methods consistently across multiple widely adopted remote-sensing FSOD benchmarks such as DIOR, NWPU-VHR10 v2, and HRRSD.

To provide effective services for intelligent transportation systems (ITS), such as optimizing ride services and recommending trips, it is important to predict the distributions of passenger flows from various origins to destinations. However, existing crowd flow prediction models have not sufficiently addressed this problem, and most methods have only focused on in and out flows of individual regions. The main challenges of origin-destination (OD) crowd flow prediction are diverse flow patterns across city networks and data sparsity. To solve these problems, we propose a Multi Attention 3D Residual Network (MAThR) to predict city-wide OD crowd flows. In particular, we develop a multi-component 3D residual structure with a novel global self-attention mechanism to dynamically aggregate the OD spatial-temporal dependencies, by modeling three components: contextual information of the region, and long and short term periodic crowd flows. For each component, we design a tensor criss-cross self-attention block, which can simultaneously discover the global and local correlation of spatial (where), temporal (when) and contextual (which) information between all OD pairs. Evaluation on real-world crowd flow data demonstrates the advantages of our MAThR method on prediction accuracy, compared to other existing state-of-the-art methods.

On-ground aircraft trajectory information plays a key role in airport situations awareness prediction and management. Airport administration needs to arrange and schedule the time and order of aircraft landing and take-off events based on a precise and real-time information of on-ground aircraft. Recently, a large dataset of GPS-derived aircraft at airports, available from the Federal Aviation Administration (FAA), provides researchers with an opportunity to monitoring on-ground aircraft trajectory. In this paper, we present a framework to incrementally cluster airport aircraft trajectories based on the GPS data. The framework consists of two steps: 1) Classifying airport aircraft data according to spatial and temporal information. 2) Merging the similar aircraft trajectories incrementally. We evaluate our framework experimentally using a state-of-the-art test-bed technique, and find that it can effectively and efficiently construct and update on-ground aircraft trajectory map.