This paper introduces MISTA, a novel dataset for visual instruction tuning on aerial imagery, designed to enhance large multi-modal model applications in remote sensing. Originating from the renowned DOTA-v2.0 aerial object detection benchmark, MISTA uniformly processes high-resolution images into 2048×2048 pixels, creating a detailed and complex dataset tailored for remote sensing analysis. To craft this dataset, we design an automated annotation pipeline, employing advanced language models such as GPT-4 and LLaVA-1.5, to generate diverse and specialized instruction-following data. The annotations include various instruction types like multi-turn conversation, detailed description, and complex reasoning, each reflecting the intricacies inherent in remote sensing tasks. The innovative approach of subdividing aerial images into individually annotated sub-patches significantly enhances the richness of the dataset and allows for a more granular analysis of visual content. As a robust foundation for multi-modal model development in remote sensing, MISTA represents a significant advancement, setting the stage for future research and further applications in the field.

The development of detection methods for oriented object detection remains a challenging task. A considerable obstacle is the wide variation in the shape (e.g., aspect ratio) of objects. Sample selection in general object detection has been widely studied as it plays a crucial role in the performance of the detection method and has achieved great progress. However, existing sample selection strategies still overlook some issues: (1) most of them ignore the object shape information; (2) they do not make a potential distinction between selected positive samples; and (3) some of them can only be applied to either anchor-free or anchor-based methods and cannot be used for both of them simultaneously. In this paper, we propose novel flexible shape-adaptive selection (SA-S) and shape-adaptive measurement (SA-M) strategies for oriented object detection, which comprise an SA-S strategy for sample selection and SA-M strategy for the quality estimation of positive samples. Specifically, the SA-S strategy dynamically selects samples according to the shape information and characteristics distribution of objects. The SA-M strategy measures the localization potential and adds quality information on the selected positive samples. The experimental results on both anchor-free and anchor-based baselines and four publicly available oriented datasets (DOTA, HRSC2016, UCAS-AOD, and ICDAR2015) demonstrate the effectiveness of the proposed method.