Recent developments in the construction and operation of high-altitude platforms (HAPs) close the gap between airborne and spaceborne remote sensing. The possibility of permanent monitoring of regions from the stratosphere offers new fields of application requiring bespoke instruments for operation on HAP carriers.The MACS-HAP aerial camera system was developed specifically for platforms operating in the stratosphere. Based on a refractive optical design, it captures aerial images with 127 megapixels per image and achieves a ground sampling distance of 15 cm per pixel at 20 km altitude. One of the special system features is the embedded onboard processing hardware with specific algorithms for real-time information extraction.This paper presents an overview of the instrument and the current state of development. Technical solutions to meet the specific payload requirements are described, such as stress-optimized design and the lightweight optical system with its capabilities for automatic onboard image processing.

Recent advances in machine learning and the rise of new large-scale remote sensing datasets have opened new possibilities for automation of remote sensing data analysis that make it possible to cope with the growing data volume and complexity and the inherent spatio-temporal dynamics of disaster situations. In this work, we provide insights into machine learning methods developed by the German Aerospace Center (DLR) for rapid mapping activities and used to support disaster response efforts during the 2021 flood in Western Germany. These include specifically methods related to systematic flood monitoring from Sentinel-1 as well as road-network extraction, object detection and damage assessment from very high-resolution optical satellite and aerial images. We discuss aspects of data acquisition and present results that were used by first responders during the flood disaster.

The integration of deep learning and active learning has achieved great success in polarimetric synthetic aperture radar (PolSAR) image classification. However, the training samples provided by the active learning approach are inade-quate to promote the performance of deep learning methods. Also, in the initial learning stages, querying a small amount of informative and complex samples, which are plagued by significant speckle noise, not only increases the risk of overfitting, but also makes the further annotations of less importance. To alleviate these problems, by utilization of curriculum learning (CL), we propose a novel classification method for PolSAR images, considering the complexity of informative samples before applying them to the deep learning model. Furthermore, we develop a new lightweight 3D convolutional neural network with high-level feature extraction ability while having a very low computational cost. Experimental results with the two PolSAR benchmark data sets of AIRSAR Flevoland and ESAR Oberpfaffenhofen indicate our proposed method achieved the state-of-the-art classification results with a significantly smaller amount of training data.

Multi-class vehicle detection from airborne imagery with orientation estimation is an important task in the near and remote vision domains with applications in traffic monitoring and disaster management. In the last decade, we have witnessed significant progress in object detection in ground imagery, but it is still in its infancy in airborne imagery, mostly due to the scarcity of diverse and large-scale datasets. Despite being a useful tool for different applications, current airborne datasets only partially reflect the challenges of real-world scenarios. To address this issue, we introduce EAGLE (oriEnted vehicle detection using Aerial imaGery in real-worLd scEnarios), a large-scale dataset for multi-class vehicle detection with object orientation information in aerial imagery. It features high-resolution aerial images composed of different real-world situations with a wide variety of camera sensor, resolution, flight altitude, weather, illumination, haze, shadow, time, city, country, occlusion, and camera angle. The annotation was done by airborne imagery experts with small-and large-vehicle classes. EAGLE contains 215,986 instances annotated with oriented bounding boxes defined by four points and orientation, making it by far the largest dataset to date in this task. It also supports researches on the haze and shadow removal as well as super-resolution and in-painting applications. We define three tasks: detection by (1) horizontal bounding boxes, (2) rotated bounding boxes, and (3) oriented bounding boxes. We carried out several experiments to evaluate several state-of-the-art methods in object detection on our dataset to form a baseline. Experiments show that the EAGLE dataset accurately reflects real-world situations and correspondingly challenging applications. The dataset will be made publicly available.

Multi-pedestrian tracking in aerial imagery has several applications such as large-scale event monitoring, disaster management, search-and-rescue missions, and as input into predictive crowd dynamic models. Due to the challenges such as the large number and the tiny size of the pedestrians (e.g., 4 × 4 pixels) with their similar appearances as well as different scales and atmospheric conditions of the images with their extremely low frame rates (e.g., 2 fps), current state-of-the-art algorithms including the deep learning-based ones are unable to perform well. In this paper, we propose AerialMPTNet, a novel approach for multi-pedestrian tracking in geo-referenced aerial imagery by fusing appearance features from a Siamese Neural Network, movement predictions from a Long Short-Term Memory, and pedestrian interconnections from a GraphCNN. In addition, to address the lack of diverse aerial pedestrian tracking datasets, we introduce the Aerial Multi-Pedestrian Tracking (AerialMPT) dataset consisting of 307 frames and 44,740 pedestrians annotated. We believe that AerialMPT is the largest and most diverse dataset to this date and will be released publicly. We evaluate AerialMPTNet on AerialMPT and KIT AIS, and benchmark with several state-of-the-art tracking methods. Results indicate that AerialMPTNet significantly outperforms other methods on accuracy and time-efficiency.

Understanding the complex urban infrastructure with centimeter-level accuracy is essential for many applications from autonomous driving to mapping, infrastructure monitoring, and urban management. Aerial images provide valuable information over a large area instantaneously; nevertheless, no current dataset captures the complexity of aerial scenes at the level of granularity required by real-world applications. To address this, we introduce SkyScapes, an aerial image dataset with highly-accurate, fine-grained annotations for pixel-level semantic labeling. SkyScapes provides annotations for 31 semantic categories ranging from large structures, such as buildings, roads and vegetation, to fine details, such as 12 (sub-)categories of lane markings. We have defined two main tasks on this dataset: dense semantic segmentation and multi-class lane-marking prediction. We carry out extensive experiments to evaluate state-of-the-art segmentation methods on SkyScapes. Existing methods struggle to deal with the wide range of classes, object sizes, scales, and fine details present. We therefore propose a novel multi-task model, which incorporates semantic edge detection and is better tuned for feature extraction from a wide range of scales. This model achieves notable improvements over the baselines in region outlines and level of detail on both tasks.

The fusion of complementary information from co-registered multi-modal image data enables a more detailed and more robust understanding of an image scene or specific objects, and is important for several applications in the field of remote sensing. In this paper, the benefits of combining RGB, near infrared (NIR) and thermal infrared (TIR) aerial images for the task of semantic vehicle segmentation through deep neural networks are investigated. Therefore, RGB, NIR and TIR image triplets acquired by the Modular Aerial Camera System (MACS) are precisely co-registered through the application of a virtual camera system and subsequently used for the training of different neural network architectures. Various experiments were conducted to investigate the influence of the different sensor characteristics and an early or late fusion within the network on the quality of the segmentation results.

This paper describes the winning contribution to the 2019 IEEE GRSS Data Fusion Contest Multi-view Semantic Stereo Challenge. In this challenge, a digital surface model (DSM) and a semantic segmentation should be derived from a large number of multi-spectral WorldView-3 images. Results from 50 stereo pairs matched using Semi-Global Matching (SGM) are fused into a DSM. Semantic segmentation is performed with an ensemble of FCN networks taking as input RGB, multi-spectral and height data. Their results are then merged with pixel-wise detectors for the classes water and high vegetation. Compared to the second and third placed teams (mIOU-3 scores of 0.73 and 0.7295), our contribution reached a significantly higher score of 0.745.

The knowledge about the placement and appearance of lane markings is a prerequisite for the creation of maps with high precision, necessary for autonomous driving, infrastructure monitoring, lanewise traffic management, and urban planning. Lane markings are one of the important components of such maps. Lane markings convey the rules of roads to drivers. While these rules are learned by humans, an autonomous driving vehicle should be taught to learn them to localize itself. Therefore, accurate and reliable lane-marking semantic segmentation in the imagery of roads and highways is needed to achieve such goals. We use airborne imagery that can capture a large area in a short period of time by introducing an aerial lane marking data set. In this paper, we propose a symmetric fully convolutional neural network enhanced by wavelet transform in order to automatically carry out lane-marking segmentation in aerial imagery. Due to a heavily unbalanced problem in terms of a number of lane-marking pixels compared with background pixels, we use a customized loss function as well as a new type of data augmentation step. We achieve a high accuracy in pixelwise localization of lane markings compared with the state-of-the-art methods without using the third-party information. In this paper, we introduce the first high-quality data set used within our experiments, which contains a broad range of situations and classes of lane markings representative of today's transportation systems. This data set will be publicly available, and hence, it can be used as the benchmark data set for future algorithms within this domain.

This article describes the workflow of the classification algorithm which ranked at 2ndplace in the 2018 GRSS Data Fusion Contest. The objective of the contest was to provide a classification map with 20 classes on a complex urban scenario. The available multi-modal data were acquired from hyperspectral, LiDAR and very high-resolution RGB sensors flown on the same platform over the city of Houston, TX, USA. The classification was obtained by merging deep convolutional and shallow fully-connected neural networks on a simplified set of classes, complemented by a series of specific detectors and ad hoc classifiers.

Remote sensing is extensively used in cartography. As transportation networks grow and change, extracting roads automatically from satellite images is crucial to keep maps up-to-date. Synthetic aperture radar (SAR) satellites can provide high-resolution topographical maps. However, roads are difficult to identify in these data as they look visually similar to targets, such as rivers and railways. Most road extraction methods on SAR images still rely on a prior segmentation performed by the classical computer vision algorithms. Few works study the potential of deep learning techniques, despite their successful applications to optical imagery. This letter presents an evaluation of fully convolutional neural networks (FCNNs) for road segmentation in SAR images. We study the relative performance of early and state-of-the-art networks after carefully enhancing their sensitivity toward thin objects by adding the spatial tolerance rules. Our models show promising results, successfully extracting most of the roads in our test data set. This shows that although FCNNs natively lack efficiency for road segmentation, they are capable of good results if properly tuned. As the segmentation quality does not scale well with the increasing depth of the networks, the design of specialized architectures for roads extraction should yield better performances.