Pedestrian Trajectory Prediction (PTP) aims to predict the future trajectory of pedestrians based on a historical trajectory. Transformer-based approaches have demonstrated unparalleled performance for PTP tasks, encoding long-term temporal dependencies and heterogeneous spatial interactions of pedestrians. However, Transformer often involves redundant information and noisy interactions from irrelevant regions by considering all available trajectory features. Recently, the structured state space model, Mamba has been proposed, which captures long-range dependency in sequences with a selective mechanism to filter out redundant information. To further tap into the potential of the novel Mamba architecture for the PTP task, in this paper, we presentMambaPTP, which predicts future trajectories based purely on Mamba mechanisms, to mitigate the noisy interactions of irrelevant trajectory features and avoid repetitive trajectory modeling, while maintaining high-performance trajectory prediction. Specifically, we propose a new Bidirectional Gating Mamba (BGM) module with bidirectional state space models, which leverages the sparse gate mechanism to select informative temporal patterns and spatial interactions. Moreover, we design a Bidirectional Trajectory Alignment (BTA) module towards aligning the predicted trajectory to the ground truth, ensuring that the model to learn the effective sparse feature representation of trajectories. We conduct extensive experiments on several mainstream pedestrian trajectory prediction datasets. The results demonstrate that the proposed MambaPTP achieves competitive performance compared to advanced Transformer-based models. We hope this paper can further inspire research in Mamba for the PTP task, leading to a tighter integration of the Mamba and PTP communities.

Comprehensive evaluation of Multimodal Large Language Models (MLLMs) has recently garnered widespread attention in the research community. However, we observe that existing benchmarks present several common barriers that make it difficult to measure the significant challenges that models face in the real world, including: 1) small data scale leads to a large performance variance; 2) reliance on model-based annotations results in restricted data quality; 3) insufficient task difficulty, especially caused by the limited image resolution. To tackle these issues, we introduce MME-RealWorld. Specifically, we collect more than $300$ K images from public datasets and the Internet, filtering $13,366$ high-quality images for annotation. This involves the efforts of professional $25$ annotators and $7$ experts in MLLMs, contributing to $29,429$ question-answer pairs that cover $43$ subtasks across $5$ real-world scenarios, extremely challenging even for humans. As far as we know, **MME-RealWorld is the largest manually annotated benchmark to date, featuring the highest resolution and a targeted focus on real-world applications**. We further conduct a thorough evaluation involving $29$ prominent MLLMs, such as GPT-4o, Gemini 1.5 Pro, and Claude 3.5 Sonnet. Our results show that even the most advanced models struggle with our benchmarks, where none of them reach 60\% accuracy. The challenges of perceiving high-resolution images and understanding complex real-world scenarios remain urgent issues to be addressed. The data and evaluation code are released in our Project Page.

Fine-grained ship detection in remote sensing images (RSIs) depends heavily on numerous training data with expensive manual annotations. Learning novel ship categories from very few labeled samples and without forgetting the learned knowledge of seen categories is important to real-world applications. In this letter, we formulate fine-grained ship detection in RSIs as a problem of generalized few-shot object detection (G-FSOD). Existing methods often neglect the structured information in ship taxonomy, and thus result in mutually exclusive representations between base and novel classes and hinder the transfer of the learned knowledge to the novel concepts under the few-shot settings. To handle this problem, we propose to incorporate the inherent hierarchical taxonomy in ship classes into the generalized few-shot ship detection to leverage the shared knowledge among base and novel classes. In particular, a ship detector is trained based on the coarsest class labels and a multitask classification network is built to distinguish various ships at both coarse and fine-grained levels on base classes, which leads to a generalized ship representation between base classes to novel classes. To build the classifier of novel classes, a prototype bank is constructed with the few-shot samples of novel classes, without the wreck of the feature extractor so as to maintain the performance on base classes. Extensive experiments on two large-scale ship detection datasets demonstrate the effectiveness of our method against state-of-the-art methods.

Existing IoU-based loss functions have achieved promising performance for bounding box regression in object detection. However, they cannot fully reflect the relation between the predicted and target boxes in the case of box inclusions, and might thus deteriorate detection accuracy and efficiency. In this paper, we design a novel similarity measurement based on the box diagonal called Diag-IoU to well represent the divergence between the predicted and target boxes even in the case of box inclusions, and thus achieve superior localization accuracy and fast convergence. In particular, we equivalently represent a rectangular box with its box diagonal, which contains exclusive and informative geometrical factors, and define the Diag-IoU based on the similarities of a set of sampled point pairs from the predicted and target box diagonals. Based on the Diag-IoU, we design a general Diag-IoU loss, which can provide holistic information in measuring two boxes and thus differentiate the two boxes in the case of box inclusions. To validate the effectiveness of the proposed method, we apply the Diag-IoU loss to several representative object detectors, including YOLO v5s, Faster R-CNN, and FCOS. Extensive experiments on the synthetic data and two challenging object detection benchmark datasets, i.e., MS COCO and PASCAL VOC, demonstrate the superior performance of the proposed Diag-IoU loss compared to previous IoU-based losses as well as other metrics.