Scene understanding of aerial imagery is essential for proper emergency response during catastrophic events such as hurricanes, earthquakes, and floods. Unmanned Aerial Vehicles (UAVs) capture aerial images and analyze the context by passing images into a semantic segmentation model for monitoring damaged areas. However, the state-of-the-art semantic segmentation models are mainly trained and evaluated on ground-based datasets such as Cityscapes, MS-COCO, and CamVid, unsuitable for aerial image segmentations. For example, extracted features from objects in aerial perspective are distinct from objects on the ground view. Hence, neural networks cannot properly segment an aerial scene, especially on deformed or damaged objects during disasters. This research analyzes current semantic segmentation models to explore the feasibility of applying these models for emergency response during catastrophic events. We compare the performance of real-time semantic segmentation models with non-real-time counterparts constrained by aerial images under adversarial settings. Furthermore, we train several models on the FloodNet dataset, containing UAV images captured after Hurricane Harvey, and benchmark their execution on special classes such as flooded-buildings vs. non-flooded buildings or flooded-roads vs. non-flooded roads. In this research, real-time UNet-MobileNetV3 yields 59.3% test mIoU while non-real-time PSPNet [1] attains 79.7% test mIoU on the FloodNet, demonstrating the trade-off between accuracy and efficiency in the segmentation models.

Visual Question Generation (VQG) is one of the most challenging problems since it aims to produce relevant and meaningful questions from images. As the VQG process is able to generate a diverse set of questions that do not exist in the training set, various models (e.g., visual question answering) can be beneficial from this question generation task by evaluating the model’s performance in unknown settings. In this paper, we explored the visual question generation task on images collected by an unmanned aerial vehicle (UAV). We highlight the significant role of the question generation task and present a variational attention-based model that focuses on creating diversified and meaningful questions from images. In comparison to baseline approaches, our presented method has demonstrated the ability to create a broad and meaningful set of questions.

In this paper, we present a large-scale hurricane Michael dataset for visual perception in disaster scenarios, and analyze state-of-the-art deep neural network models for semantic segmentation. The dataset consists of around 2000 high-resolution aerial images, with annotated ground-truth data for semantic segmentation. We discuss the challenges of the dataset and train the state-of-the-art methods on this dataset to evaluate how well these methods can recognize the disaster situations. Finally, we discuss challenges for future research.

Global warming is rapidly reducing glaciers and ice sheets across the world. Real time assessment of this reduction is required so as to monitor its global climatic impact. In this paper, we introduce a novel way of estimating the thickness of each internal ice layer using Snow Radar images and Fully Convolutional Networks. The estimated thickness can be used to understand snow accumulation each year. To understand the depth and structure of each internal ice layer, we perform multiclass semantic segmentation on radar images, which hasn't been performed before. As the radar images lack good training labels, we carry out a pre-processing technique to get a clean set of labels. After detecting each ice layer uniquely, we calculate its thickness and compare it with the processed ground truth. This is the first time that each ice layer is detected separately and its thickness calculated through automated techniques. Through this procedure we were able to estimate the ice-layer thicknesses within a Mean Absolute Error of approximately 3.6 pixels. Such a Deep Learning based method can be used with ever-increasing datasets to make accurate assessments for cryospheric studies.

Artificial intelligence (AI) techniques have displayed impressive success in many practical fields. Deep neural networks (DNNs) owe their success to the availability of massive labeled data. However, in many real-world problems, even when a large dataset is available, deep learning methods have shown less success, due to causes such as lack of large labeled dataset, presence of noise in data, or missing data. In the present work, we intend to examine the application of deep learning methods on radar data gathered from polar regions. Our goal is to track internal ice layers in radar imagery. In such data, the presence of noise is one of the main obstacles in utilizing popular deep learning methods such as transfer learning. Our experiments show that if the neural network is trained to detect contours of objects in electro-optical imagery, it can only track a low percentage of contours in radar data. Fine-tuning and further training do not provide any better results. However, we will show that selecting the right model and training the model on the radar imagery from the base, is going to yield far better results. We also discuss another possible learning approach that can save us time for data annotation.

Emergency departments treat around 2.5 million older people for fall injuries each year. Preserving the elderlys' right of aging in a home of their own choice is mandatory in today's world, as more elderly people are willing to live independently. Current implementations of fall detection systems lack accuracy. Despite efforts to detect elderly falls, it is possible that daily life activities, such as lying down, trigger false alarms. Moreover, privacy is the main concern for visual cameras. In this research we used deep convolutional neural networks to describe the overall space-time appearance pattern of a fall-event in depth video cameras. We developed a 3D convolutional neural network to capture both the spatial information available in video frames, and the temporal information presented through successive video frames. Our method outperformed the state-of-the art accuracy with a large margin.

Historical documents frequently suffer from arbitrary geometric distortions (warping and folds) due to storage conditions, use and to, some extent, the printing process of the time. In addition, page curl can be prominent due to the scanning technique used. Such distortions adversely affect OCR and print-on-demand quality. Previous approaches to geometric restoration either focus only on the correction of page curl or require supplementary information obtained by additional scanning hardware - not practical for existing scans. This paper presents a new approach to detect and restore arbitrary warping and folds, in addition to page curl. Warped text lines and the smooth deformation between them are precisely modelled as primary and secondary flow lines that are then restored to their original linear shape. Preliminary, but representative, experimental results, in comparison to a leading page curl removal method and an industry-standard commercial system, demonstrate the effectiveness of the proposed method.