Maryam Rahnemoonfar

dblp:03/10400 · DBLP profile ↗
← Back
39ranked-venue papers
11as first author
24since 2021 · last 2026
0000-0001-9358-2836ORCID · corroborated

Domains — the database's venue-derived domains; a paper can count in several

Applied, interdisciplinary, general and emerging computing · 29 · 9 first-author · 15 since 2021Artificial intelligence and machine learning · 10 · 2 first-author · 5 since 2021Databases, data management, data science and information retrieval · 7 · 2 first-author · 2 since 2021Graphics, computer vision, multimedia, augmented reality and games · 7 · 1 first-author · 7 since 2021Systems, architecture and hardware · 1 · 1 since 2021
YearPublicationVenuePosition
2026 OPTNet: Ordering Point Transformer Network for Post-disaster 3D Semantic Segmentation
Nhut Le, Ehsan Karimi, Maryam Rahnemoonfar
ICPR (15)3
2026 From Short Histories to Long Futures: Horizon-Aware Graph Neural Networks for Long Horizon Forecasting
Zesheng Liu, Maryam Rahnemoonfar
ICPR (15)2
2026 K-STEMIT: Knowledge-informed spatio-temporal efficient multi-branch graph neural network for subsurface stratigraphy thickness estimation from radar data
Zesheng Liu, Maryam Rahnemoonfar
Neurocomputing2
2025 ST-GRIT: Spatio-Temporal Graph Transformer For Internal Ice Layer Thickness Prediction
abstract
Understanding the thickness and variability of internal ice layers in radar imagery is crucial for monitoring snow accumulation, assessing ice dynamics, and reducing uncertainties in climate models. Radar sensors, capable of penetrating ice, provide detailed radargram images of these internal layers. In this work, we present ST-GRIT, a spatiotemporal graph transformer for ice layer thickness, designed to process these radargrams and capture the spatiotemporal relationships between shallow and deep ice layers. ST-GRIT leverages an inductive geometric graph learning framework to extract local spatial features as feature embeddings and employs a series of temporal and spatial attention blocks separately to model long-range dependencies effectively in both dimensions. Experimental evaluation on radargram data from the Greenland ice sheet demonstrates that ST-GRIT consistently outperforms current state-of-the-art methods and other baseline graph neural networks by achieving lower root mean-squared error. These results highlight the advantages of self-attention mechanisms on graphs over pure graph neural networks, including the ability to handle noise, avoid oversmoothing, and capture long-range dependencies. Moreover, the use of separate spatial and temporal attention blocks allows for distinct and robust learning of spatial relationships and temporal patterns, providing a more comprehensive and effective approach.
Zesheng Liu, Maryam Rahnemoonfar
ICIP2
2024 Deep Spectral Siamese Network For Heterogeneous Object Verification In Amazon Robotic Warehouse
abstract
Automatic object verification is an important task in Amazon fulfillment centers where millions of shelves are filled with a wide assortment of packages of various sizes. In the current Amazon unstructured warehouse environment it is still cheaper to hire humans (even in the U.S.) than to develop customized robotic solutions. New advances in computer vision and deep learning can revolutionize Amazon robotic warehouse by developing new algorithms for learning to find objects by robots. The success of many deep learning algorithms depends on a large number of correct annotated data and high quality images. The images captured by Amazon robots are partially or completely occluded with plastic tapes and the content of some bins may not match the recorded inventory of that bin. Moreover, since most of the objects appear only once or twice in the entire dataset of incredibly large number of categories, it is difficult to achieve sufficient training for each class at a satisfactory level. To address these issues, we developed a novel deep spectral Siamese architecture for efficient verification with improved accuracy. Our proposed spectral Siamese network can accurately learn correlation between objects that have only a few examples for training in the noisy and blurry images. Experimental results on Amazon Bin Image demonstrate the effectiveness of the proposed framework both qualitatively and quantitatively.
Maryam Rahnemoonfar
ICIP1
2024 Flood Simulation: Integrating UAS Imagery and Ai-Generated Data With Diffusion Model
abstract
The primary goal of early disaster impact assessments is to gather georeferenced data about affected areas. Floods, a major natural calamity, pose challenges in data collection and response coordination. The use of Unmanned Aerial Systems (UAS) has significantly improved data acquisition in flood-impacted regions, offering a cost-effective method for obtaining high-quality images. However, issues like accurate image description and data scarcity remain. Addressing these, we propose two novel methods: GPT models for precise image-to-text conversion and diffusion models for flood data simulation. GPT models enhance data quality with accurate descriptions, while diffusion models extend our dataset by simulating diverse flood scenarios, enriching our training resources. Further analysis of this augmented dataset for similarity demonstrates a wide-ranging variety in the data, highlighting a diverse array of elements within images.
Xiyang Hu, Maryam Rahnemoonfar
IGARSS2
2024 Graph Neural Network as Computationally Efficient Emulator Of Ice-Sheet And Sea-Level System Model (ISSM)
abstract
The Ice-sheet and Sea-level System Model (ISSM) provides solutions for Stokes equations relevant to ice sheet dynamics by employing finite element and fine mesh adaption. However, since its finite element method is compatible only with Central Processing Units (CPU), the ISSM has limits on further economizing computational time. Thus, by taking advantage of Graphics Processing Units (GPUs), we design a graph convolutional network (GCN) as a fast emulator for ISSM. The GCN is trained and tested using the 20-year transient ISSM simulations in the Pine Island Glacier (PIG). The GCN reproduces ice thickness and velocity with a correlation coefficient greater than 0.998, outperforming the traditional convolutional neural network (CNN). Additionally, GCN shows 34 times faster computational speed than the CPU-based ISSM modeling. The GPU-based GCN emulator allows us to predict how the PIG will change in the future under different melting rate scenarios with high fidelity and much faster computational time.
Younghyun Koo, Maryam Rahnemoonfar
IGARSS2
2024 Physics-informed Machine Learning for Deep Ice Layer Tracing in SAR images
abstract
The precise prediction and tracking of deep internal layers of ice sheets is becoming increasingly important as we deal with the impacts of climate change and the rise of global atmospheric temperatures. Synthetic Aperture Radar (SAR) is the only sensor capable of penetrating through ice and providing us with information about what lies beneath the ice surface, allowing us to monitor changes on a large scale. Forecasting and tracking these internal ice sheet layers is crucial for calculating snow mass balance, inferring otherwise difficult-to-observe ice dynamic processes, and extrapolating ice age from direct measurements of the subsurface. To achieve this, we developed a geometric deep learning model that uses a supervised, multi-target, adaptive long short-term memory graph convolutional network to predict the thicknesses of multiple deep ice layers at specific coordinates in an ice sheet given the thicknesses of a few shallow ice layers. Furthermore, we expanded the model to consider additional physical features of the ice, alongside layer thickness. We found that the inclusion of snow mass balance, meltwater refreezing, and height change due to melting as node features give our model better and more consistent performance.
Maryam Rahnemoonfar, Benjamin Zalatan
IGARSS1
2024 Reg-Tune: A Regression-Focused Fine-Tuning Approach for Profiling Low Energy Consumption and Latency
abstract
Fine-tuning deep neural networks is pivotal for creating inference modules that can be suitably imported to edge or field-programmable gate array (FPGA) platforms. Traditionally, exploration of different parameters throughout the layers of deep neural networks has been done using grid search and other brute force techniques. Although these methods lead to the optimal choice of network parameters, the search process can be very time consuming and may not consider deployment constraints across different target platforms. This work addresses this problem by proposing Reg-Tune, a regression-based profiling approach to quickly determine the trend of different metrics in relation to hardware deployment of neural networks on tinyML platforms like FPGAs and edge devices. We start by training a handful of configurations belonging to different combinations of \(\mathcal {NN}\scriptstyle \langle q (quantization),\,s (scaling)\rangle \displaystyle\) or \(\mathcal {NN}\scriptstyle \langle r (resolution),\,s\rangle \displaystyle\) workloads to generate the accuracy values respectively for their corresponding application. Next, we deploy these configurations on the target device to generate energy/latency values. According to our hypothesis, the most energy-efficient configuration suitable for deployment on the target device is a function of the variables q , r , and s . Finally, these trained and deployed configurations and their related results are used as data points for polynomial regression with the variables q , r , and s to realize the trend for accuracy/energy/latency on the target device. Our setup allows us to choose the near-optimal energy-consuming or latency-driven configuration for the desired accuracy from the contour profiles of energy/latency across different tinyML device platforms. To this extent, we demonstrate the profiling process for three different case studies and across two platforms for energy and latency fine-tuning. Our approach results in at least 5.7 \(\times\) better energy efficiency when compared to recent implementations for human activity recognition on FPGA and 74.6% reduction in latency for semantic segmentation of aerial imagery on edge devices compared to baseline deployments.
Arnab Neelim Mazumder, Farshad Safavi, Maryam Rahnemoonfar, Tinoosh Mohsenin
ACM Trans. Embed. Comput. Syst.3
2024 Hierarchical Information-Sharing Convolutional Neural Network for the Prediction of Arctic Sea Ice Concentration and Velocity
abstract
Forecasting sea ice concentration (SIC) and sea ice velocity (SIV) in the Arctic Ocean is of great significance as the Arctic environment has been changed by the recent warming climate. Given that physical sea ice models require high computational costs with complex parameterization, deep learning techniques can effectively replace the physical model and improve the performance of sea ice prediction. This study proposes a novel multitask fully conventional network architecture named hierarchical information-sharing U-net (HIS-Unet) to predict daily SIC and SIV. Instead of learning SIC and SIV separately at each branch, we allow the SIC and SIV layers to share their information and assist each other’s prediction through the weighting attention modules (WAMs). Consequently, our HIS-Unet outperforms other statistical approaches, sea ice physical models, and neural networks without such information-sharing units. The improvement of HIS-Unet is more significant to when and where SIC changes seasonally, which implies that the information sharing between SIC and SIV through WAMs helps learn the dynamic changes of SIC and SIV. The weight values of the WAMs imply that SIC information plays a more critical role in SIV prediction, compared to that of SIV information in SIC prediction, and information sharing is more active in marginal ice zones [e.g., East Greenland (EG) and Hudson/Baffin Bays (HBB)] than in the central Arctic (CA).
Younghyun Koo, Maryam Rahnemoonfar
IEEE Trans. Geosci. Remote. Sens.2
2023 Prediction of Deep Ice Layer Thickness Using Adaptive Recurrent Graph Neural Networks
abstract
As we deal with the effects of climate change and the increase of global atmospheric temperatures, the accurate tracking and prediction of ice layers within polar ice sheets grows in importance. Studying these ice layers reveals climate trends, how snowfall has changed over time, and the trajectory of future climate and precipitation. In this paper, we propose a machine learning model that uses adaptive, recurrent graph convolutional networks to, when given the amount of snow accumulation in recent years gathered through airborne radar data, predict historic snow accumulation by way of the thickness of deep ice layers. We found that our model performs better and with greater consistency than our previous model as well as equivalent non-temporal, non-geometric, and non-adaptive models.
Benjamin Zalatan, Maryam Rahnemoonfar
ICIP2
2023 ECHOVIT: Vision Transformers Using Fast-And-Slow Time Embeddings
abstract
This paper details the preliminary efforts of applying the deep learning transformer architecture to automatically track annual layer stratigraphy in echogram images obtained from mapping near-surface ice layers using airborne radars. Following the success of the transformer architecture in the natural language processing and computer vision communities, we explore a variant termed Echogram Vision Transformer (EchoViT) on the radar echogram layer tracking (RELT) problem. The proposed approach divides the echogram images into patches using different schemes inspired by tokenization methods in natural language processing. We then apply a soft-attention mechanism to model interdependencies between the patches, capturing spatiotemporal stratigraphic information. Experiments conducted on the CREED dataset demonstrate the superiority of transformer-based architectures over existing convolutional-based architectures. Furthermore, the EchoViT fast-time and EchoViT slow-time patchifying schemes achieved precise tracking of the layers with submeter MAE of 3.39 and 3.55, respectively, while the use of cropped patches led to suboptimal results.
Ibikunle Oluwanisola, Debvrat Varshney, Jilu Li, Maryam Rahnemoonfar, John Paden
IGARSS4
2023 Efficient Large-Scale Damage Assessment After Natural Disasters With UAVS and Deep Learning
abstract
Frequent and increasingly severe natural disasters due to climate change threaten human health and infrastructure. The provision of accurate, timely, and understandable information has the potential to revolutionize disaster management. While traditional analyses provide some insights into the data, the complexity, scale, and multi-disciplinary nature of the data necessitate advanced, intelligent solutions. The main barrier is the lack of near real-time data analysis. Recently there has been a surge of practical applications of real-time semantic segmentation. Real-time semantic segmentation requires fast and high-quality predictions. As a result, lightweight architectures with low latency and computational costs are necessary for efficient semantic segmentation. In this article, we developed several encoder-decoder and two-pathway architectures and compared their performance on a novel RescuNet dataset.
Maryam Rahnemoonfar, Farshad Safavi
IGARSS1
2023 RESCUENet-VQA: A Large-Scale Visual Question Answering Benchmark for Damage Assessment
abstract
In order to advance the research on AI-assisted efficient damage assessment during a natural disaster, we present in this study a large-scale visual question answering (VQA) dataset on remote sensing images, namely RescueNet-VQA. Visual question answering is the task of getting query-based scene information from images. The main advantage of this approach is that it can provide high-level scene information while interacting with users. For this merit, VQA has the potential to be considered in the decision-making processes for rapid response and recovery during any disaster. To conduct substantial research in this context, we present a novel VQA dataset for damage assessment on remote sensing imagery. Images in our dataset were collected after hurricane Michael. We have generated 1,03,192 image-question-answer triplets from 4,375 images. This dataset is the only large-scale remote-sensed imagery-based visual question-answering dataset for damage assessment purposes. We have presented image collection and question generation procedures along with dataset statistics in this work.
Argho Sarkar, Maryam Rahnemoonfar
IGARSS2
2023 Prediction of Annual Snow Accumulation Using a Recurrent Graph Convolutional Approach
abstract
The precise tracking and prediction of polar ice layers can unveil historic trends in snow accumulation. In recent years, airborne radar sensors, such as the Snow Radar, have been shown to be able to measure these internal ice layers over large areas with a fine vertical resolution. In our previous work, we found that temporal graph convolutional networks perform reasonably well in predicting future snow accumulation when given temporal graphs containing deep ice layer thickness. In this work, we experiment with a graph attention network-based model and used it to predict more annual snow accumulation data points with fewer input data points on a larger dataset. We found that these large changes only very slightly negatively impacted performance.
Benjamin Zalatan, Maryam Rahnemoonfar
IGARSS2
2023 SAM-VQA: Supervised Attention-Based Visual Question Answering Model for Post-Disaster Damage Assessment on Remote Sensing Imagery
abstract
Each natural disaster leaves a trail of destruction and damage that must be effectively managed to reduce its negative impact on human life. Any delay in making proper decisions at the post-disaster managerial level can increase human suffering and waste resources. Proper managerial decisions after any natural disaster rely on an appropriate assessment of damages using data-driven approaches, which are needed to be efficient, fast, and interactive. The goal of this study is to incorporate a deep interactive data-driven framework for proper damage assessment to speed up the response and recovery phases after a natural disaster. Hence, this paper focuses on introducing and implementing the Visual Question Answering (VQA) framework for post-disaster damage assessment based on drone imagery, namely Supervised Attention-Based VQA (SAM-VQA). In visual question answering, query-based answers from images regarding the situation in disaster-affected areas can provide valuable information for decision-making. Unlike other computer vision tasks, visual question answering is more interactive and allows one to get instant and effective scene information by asking questions in natural language from images. In this work, we present a VQA dataset and propose a novel supervised attention-based VQA framework (SAM-VQA) for post-disaster damage assessment on remote sensing images. Our model outperforms state-of-the-art attention-based VQA techniques, including Stacked Attention Networks (SAN) [1] and Multi-modal Factorized Bilinear (MFB) with Co-Attention [2]. Furthermore, our proposed model can derive appropriate visual attention based on questions to predict answers, making our approach trustworthy.
Argho Sarkar, Tashnim Chowdhury, Robin R. Murphy, Aryya Gangopadhyay, Maryam Rahnemoonfar
IEEE Trans. Geosci. Remote. Sens.5
2022 Grad-Cam Aware Supervised Attention for Visual Question Answering for Post-Disaster Damage Assessment
abstract
In this paper, we present a Grad-Cam aware supervised attention framework for visual question answering (VQA) tasks for post-disaster damage assessment purposes. Visual-attention in visual question-answering tasks aims to focus on relevant image regions according to questions to predict answers. However, the conventional attention mechanisms in VQA work in an unsupervised manner, learning to give importance to visual contents by minimizing only task-specific loss. This approach fails to provide appropriate visual attention where the visual contents are very complex. The content and nature of UAV images in FlooNet-VQA dataset are very complex as they depict the hazardous scenario after Hurricane Harvey from a high altitude. To tackle this, we propose a supervised attention mechanism that uses explainable features from Grad-Cam to supervise visual attention in the VQA pipeline. The mechanism we propose operates in two stages. In the first stage of learning, we derived the visual explanations through Grad-Cam by training a baseline attention-based VQA model. In the second stage, we supervise our visual content for each question by incorporating the Grad-Cam explanations from the previous phase of the training process. We have improved the model performance over the state-of-the-art VQA models by a considerable margin on FloodNet dataset.
Argho Sarkar, Maryam Rahnemoonfar
ICIP2
2022 Learning Snow Layer Thickness Through Physics Defined Labels
abstract
Increasing global temperatures are adversely affecting the polar ice sheets and contributing to sea level rise. The situation requires constant monitoring and analysis of the change in thickness of snow layers accumulated on top of ice sheets. The monitoring can be performed through radar sensors, but current methods aren't efficient enough to process the radar images since they are noisy, and lack quality annotations, which are required by state-of-the-art deep learning algorithms. In this work, we show that first learning the thickness of snow layers simulated through a physical model helps in building robust deep learning networks. Specifically we show that transfer learning from a network trained with physics-defined labels improves snow layer thickness estimates by 6-29% on the test set.
Debvrat Varshney, Ibikunle Oluwanisola, John Paden, Maryam Rahnemoonfar
IGARSS4
2021 Comparative Study Between Real-Time and Non-Real-Time Segmentation Models on Flooding Events
abstract
Scene understanding of aerial imagery is essential for proper emergency response during catastrophic events such as hurricanes, earthquakes, and floods. Unmanned Aerial Vehicles (UAVs) capture aerial images and analyze the context by passing images into a semantic segmentation model for monitoring damaged areas. However, the state-of-the-art semantic segmentation models are mainly trained and evaluated on ground-based datasets such as Cityscapes, MS-COCO, and CamVid, unsuitable for aerial image segmentations. For example, extracted features from objects in aerial perspective are distinct from objects on the ground view. Hence, neural networks cannot properly segment an aerial scene, especially on deformed or damaged objects during disasters. This research analyzes current semantic segmentation models to explore the feasibility of applying these models for emergency response during catastrophic events. We compare the performance of real-time semantic segmentation models with non-real-time counterparts constrained by aerial images under adversarial settings. Furthermore, we train several models on the FloodNet dataset, containing UAV images captured after Hurricane Harvey, and benchmark their execution on special classes such as flooded-buildings vs. non-flooded buildings or flooded-roads vs. non-flooded roads. In this research, real-time UNet-MobileNetV3 yields 59.3% test mIoU while non-real-time PSPNet [1] attains 79.7% test mIoU on the FloodNet, demonstrating the trade-off between accuracy and efficiency in the segmentation models.
Farshad Safavi, Tashnim Chowdhury, Maryam Rahnemoonfar
IEEE BigData3
2021 UAV-VQG: Visual Question Generation Framework on UAV Images
abstract
Visual Question Generation (VQG) is one of the most challenging problems since it aims to produce relevant and meaningful questions from images. As the VQG process is able to generate a diverse set of questions that do not exist in the training set, various models (e.g., visual question answering) can be beneficial from this question generation task by evaluating the model’s performance in unknown settings. In this paper, we explored the visual question generation task on images collected by an unmanned aerial vehicle (UAV). We highlight the significant role of the question generation task and present a variational attention-based model that focuses on creating diversified and meaningful questions from images. In comparison to baseline approaches, our presented method has demonstrated the ability to create a broad and meaningful set of questions.
Argho Sarkar, Maryam Rahnemoonfar
IEEE BigData2
2021 Self Attention Based Semantic Segmentation on a Natural Disaster Dataset
abstract
Global image dependencies help in full image understanding. Self-attention based methods can map the mutual relationship and dependencies among pixels of an image and thus improve semantic segmentation accuracy. In this paper, we propose two segmentation networks based on a novel baseline self-attention network. Compared to existing self-attention methods we utilize lower level feature maps to generate position attention modules which constitute a baseline network. This baseline network is incorporated with global average pooling and U-Net to create two segmentation schemes. These two segmentation networks are evaluated on a natural disaster dataset and perform excellent in damage assessment with a Mean IoU score of 95.61%.
Tashnim Chowdhury, Maryam Rahnemoonfar
ICIP2
2021 Attention Based Semantic Segmentation on UAV Dataset for Natural Disaster Damage Assessment
abstract
The detrimental impacts of climate change include stronger and more destructive hurricanes happening all over the world. Identifying different damaged structures of an area including buildings and roads are vital since it helps the rescue team to plan their efforts to minimize the damage caused by a natural disaster. Semantic segmentation helps to identify different parts of an image. We implement a novel self-attention based semantic segmentation model on a high resolution UAV dataset and attain Mean IoU score of around 88% on the test set. The result inspires to use self-attention schemes in natural disaster damage assessment which will save human lives and reduce economic losses.
Tashnim Chowdhury, Maryam Rahnemoonfar
IGARSS2
2021 VQA-Aid: Visual Question Answering for Post-Disaster Damage Assessment and Analysis
abstract
Visual Question Answering system integrated with Unmanned Aerial Vehicle (UAV) has a lot of potentials to advance the post-disaster damage assessment purpose. Providing assistance to affected areas is highly dependent on real-time data assessment and analysis. Scope of the Visual Question Answering is to understand the scene and provide query related answer which certainly faster the recovery process after any disaster. In this work, we address the importance of visual question answering (VQA) task for post-disaster damage assessment by presenting our recently developed VQA dataset called HurMic-VQA collected during hurricane Michael, and comparing the performances of baseline VQA models.
Argho Sarkar, Maryam Rahnemoonfar
IGARSS2
2021 Regression Networks for Calculating Englacial Layer Thickness
abstract
Ice thickness estimation is an important aspect of ice sheet modelling. In this work, we use convolutional neural networks (CNN) with multiple output nodes to regress and learn the thickness of internal ice layers in Snow Radar images captured over northwest Greenland. We experiment with some state-of-the-art CNNs to obtain a mean absolute error of 1.251 pixels of thickness estimation over the test set. Such regression-based networks can further be improved by embedding domain knowledge and radar information in the neural network in order to reduce the requirement of manual annotations.
Debvrat Varshney, Maryam Rahnemoonfar, Masoud Yari, John Paden
IGARSS2
2020 Comprehensive Semantic Segmentation on High Resolution UAV Imagery for Natural Disaster Damage Assessment
abstract
In this paper, we present a large-scale hurricane Michael dataset for visual perception in disaster scenarios, and analyze state-of-the-art deep neural network models for semantic segmentation. The dataset consists of around 2000 high-resolution aerial images, with annotated ground-truth data for semantic segmentation. We discuss the challenges of the dataset and train the state-of-the-art methods on this dataset to evaluate how well these methods can recognize the disaster situations. Finally, we discuss challenges for future research.
Tashnim Chowdhury, Maryam Rahnemoonfar, Robin R. Murphy, Odair Fernandes
IEEE BigData2
2020 Deep Ice Layer Tracking and Thickness Estimation using Fully Convolutional Networks
abstract
Global warming is rapidly reducing glaciers and ice sheets across the world. Real time assessment of this reduction is required so as to monitor its global climatic impact. In this paper, we introduce a novel way of estimating the thickness of each internal ice layer using Snow Radar images and Fully Convolutional Networks. The estimated thickness can be used to understand snow accumulation each year. To understand the depth and structure of each internal ice layer, we perform multiclass semantic segmentation on radar images, which hasn't been performed before. As the radar images lack good training labels, we carry out a pre-processing technique to get a clean set of labels. After detecting each ice layer uniquely, we calculate its thickness and compare it with the processed ground truth. This is the first time that each ice layer is detected separately and its thickness calculated through automated techniques. Through this procedure we were able to estimate the ice-layer thicknesses within a Mean Absolute Error of approximately 3.6 pixels. Such a Deep Learning based method can be used with ever-increasing datasets to make accurate assessments for cryospheric studies.
Debvrat Varshney, Maryam Rahnemoonfar, Masoud Yari, John Paden
IEEE BigData2
2020 Snow Radar Layer Tracking Using Iterative Neural Network Approach
abstract
This paper presents preliminary results using a fully connected neural network (NN) to automatically track the internal layers of snow radar echograms using an iterative “row-block-column” approach. Snow radar images, when accurately tracked, provide relevant information for estimating snow accumulation rates in polar regions which is a key measurement needed to understand and predict the impact of climate warming in Greenland and Antarctica. A multiclass NN was designed and trained with a training set of 121,408 columns of simulated snow radar data and learns to automatically track the internal layers with an accuracy of 92.8%, a RMSE of 0.24 pixels, and with 98% of pixel errors less than or equal to 1 pixel.
Ibikunle Oluwanisola, John Paden, Maryam Rahnemoonfar, David Crandall, Masoud Yari
IGARSS3
2020 Radar Sensor Simulation with Generative Adversarial Network
abstract
Significant resources have been spent in collecting and storing large and heterogeneous radar datasets during expensive Arctic and Antarctic fieldwork. The vast majority of data available is unlabeled, and the labeling process is both time-consuming and expensive. One possible alternative to the labeling process is the use of synthetically generated data with artificial intelligence. In this research, we evaluated the performance of synthetically generated snow radar images based on modified cycle-consistent adversarial networks. We conducted several experiments to test the quality of the generated radar imagery. Our experiments show a very good similarity between real and synthetic snow radar images.
Maryam Rahnemoonfar, Masoud Yari, John Paden
IGARSS1
2020 Multi-Scale and Temporal Transfer Learning for Automatic Tracking of Internal Ice Layers
abstract
Pragmatic Deep Learning techniques in recent years have greatly influenced our approaches to data analysis. However, in many real-world problems, even when a large dataset is available, Deep Learning methods have shown less success, for the lack of large labeled dataset, presence of noise, or missing data. In this work, our goal is to track internal ice layers in radar images gathered with various sensors in different years. We will show that transfer learning will not generally work well. However, if the Deep Learning model gets trained on noisy images, there would be a significant improvement. Unlike spatial Transfer Learning, our experiments show that temporal Transfer Learning can provide considerably better results.
Masoud Yari, Maryam Rahnemoonfar, John Paden
IGARSS2
2019 Smart Tracking of Internal Layers of Ice in Radar Data via Multi-Scale Learning
abstract
Artificial intelligence (AI) techniques have displayed impressive success in many practical fields. Deep neural networks (DNNs) owe their success to the availability of massive labeled data. However, in many real-world problems, even when a large dataset is available, deep learning methods have shown less success, due to causes such as lack of large labeled dataset, presence of noise in data, or missing data. In the present work, we intend to examine the application of deep learning methods on radar data gathered from polar regions. Our goal is to track internal ice layers in radar imagery. In such data, the presence of noise is one of the main obstacles in utilizing popular deep learning methods such as transfer learning. Our experiments show that if the neural network is trained to detect contours of objects in electro-optical imagery, it can only track a low percentage of contours in radar data. Fine-tuning and further training do not provide any better results. However, we will show that selecting the right model and training the model on the radar imagery from the base, is going to yield far better results. We also discuss another possible learning approach that can save us time for data annotation.
Masoud Yari, Maryam Rahnemoonfar, John Paden, Ibikunle Oluwanisola, Lora Koenig, Lynn Montgomery
IEEE BigData2
2019 Semantic Segmentation of Underwater Sonar Imagery with Deep Learning
abstract
Majority of deep learning methods are developed for RGB imagery. However, for many applications such as detecting objects underwater other types of sensors such as sonar or radar are required. One of the most precise sensors to map the seagrass disturbance is side scan sonar. Here we developed a new deep learning framework based on dilated convolution, dense module, and inception to perform semantic segmentation for automatic extraction of potholes in underwater sonar imagery. We tested our proposed approach on a collection of underwater sonar images taken from Laguna Madre in Texas. Experimental results in comparison with the ground-truth and state-of-the-art semantic segmentation methods show the efficiency and improved accuracy of our proposed method.
Maryam Rahnemoonfar, Dugan Dobbs
IGARSS1
2018 Spatio-Temporal Convolutional Neural Network for Elderly Fall Detection in Depth Video Cameras
abstract
Emergency departments treat around 2.5 million older people for fall injuries each year. Preserving the elderlys' right of aging in a home of their own choice is mandatory in today's world, as more elderly people are willing to live independently. Current implementations of fall detection systems lack accuracy. Despite efforts to detect elderly falls, it is possible that daily life activities, such as lying down, trigger false alarms. Moreover, privacy is the main concern for visual cameras. In this research we used deep convolutional neural networks to describe the overall space-time appearance pattern of a fall-event in depth video cameras. We developed a 3D convolutional neural network to capture both the spatial information available in video frames, and the temporal information presented through successive video frames. Our method outperformed the state-of-the art accuracy with a large margin.
Maryam Rahnemoonfar, Hend Alkittawi
IEEE BigData1
2018 Deep Hybrid Wavelet Network for Ice Boundary Detection in Radra Imagery
abstract
This paper proposes a deep convolutional neural network approach to detect Ice surface and bottom layers from radar imagery. Radar images are capable to penetrate the Ice surface and provide us with valuable information from the underlying layers of ice surface. In recent years, deep hierarchical learning techniques for object detection and segmentation greatly improved the performance of traditional techniques based on hand-crafted feature engineering. We designed a deep convolutional network to produce the images of surface and bottom ice boundary. Our network take advantage of undecimated wavelet transform to provide the higest level of information from radar images, as well as multilayer and multi-scale optimized architecture. In this work, radar images from 2009-2016 NASA Operation IceBridge Mission are used to train and test the network. Our network outperformed the state-of-the art accuracy.
Hamid Kamangir, Maryam Rahnemoonfar, Dugan Dobbs, John Paden, Geoffrey C. Fox
IGARSS2
2018 Flooded Area Detection from Uav Images Based on Densely Connected Recurrent Neural Networks
abstract
The emergence of small unmanned aerial vehicles (UAV) along with inexpensive sensors presents the opportunity to collect thousands of images after each natural disaster with high flexibility and easy maneuverability for rapid response and recovery. Despite the ease of data collection, data analysis of the big datasets remains a significant barrier for scientists and analysts. Here we propose an integration of densely connected CNN and RNN networks, which is able to accurately segment out semantically meaningful object boundaries with end-to-end learning. The proposed network is applied on UAV aerial images of flooded areas in Houston, TX. We achieved 96% accuracy in detecting flooded areas on a large UAV dataset.
Maryam Rahnemoonfar, Robin R. Murphy, Marina Vicens Miquel, Dugan Dobbs, Ashton Adams
IGARSS1
2017 Automatic Ice thickness estimation in radar imagery based on charged particles concept
abstract
Accelerated loss of ice from Greenland and Antarctica has been observed in recent decades. Ice thickness is a key factor in making predictions about the future of massive ice reservoirs and can be estimated by calculating the exact location of the ice surface and bottom in radar imagery. Identifying the locations of ice boundaries is typically performed manually which is a very time consuming procedure. Here we propose a novel approach which automatically detects the complex topology of ice surface and bottom boundaries based on charged particle concept. Here we first applied anisotropic diffusion to remove the noise and enhance the image. At the second step, we detected the contours in the image based on Coulomb's electrostatic law and the assumption that each pixel is an electrically charged particle. The final ice surface and bottom are detected based on the projection profile of the contours. The results are evaluated on a large dataset of airborne radar imagery collected during IceBridge mission over Antarctica and show promising results with respect to hand-labeled ground truth.
Maryam Rahnemoonfar, Amin Abbasi Habashi, John Paden, Geoffrey C. Fox
IGARSS1
2017 Real-time scene understanding for UAV imagery based on deep convolutional neural networks
abstract
Real-time scene understanding is important for many applications of Unmanned Aerial Vehicles (UAVs) such as reconnaissance, surveillance, mapping, and infrastructure inspection. With the recent growth of computation power, it is feasible to use Deep Learning for real-time applications. Deep Convolutional Neural Networks (CNNs) have emerged as a powerful model for classifying image content, and are widely considered in the computer vision community to be the de facto standard approach for most problems. Current Deep learning approaches for image classification and object detection are designed and evaluated on lab setting human-centric photographs taken horizontally from a height of 1-2 meters. UAV images are taken vertically in high altitude; therefore the objects of interest are relatively small with a skewed vantage point which creates a real challenge in detection and classification of such images. Here we present a deep convolutional approach for classification of Aerial imagery taken by UAV. We applied our network on optical imagery taken with UAV RS-16 from Port Mansfield, TX. Experimental results in comparison with ground-truth show 93.6 % accuracy for UAV image classification.
Clay Sheppard, Maryam Rahnemoonfar
IGARSS2
2017 Automatic Ice Surface and Bottom Boundaries Estimation in Radar Imagery Based on Level-Set Approach
abstract
Accelerated loss of ice from Greenland and Antarctica has been observed in recent decades. The melting of polar ice sheets and mountain glaciers has considerable influence on sea level rise in a changing climate. Ice thickness is a key factor in making predictions about the future of massive ice reservoirs. The ice thickness can be estimated by calculating the exact location of the ice surface and subglacial topography beneath the ice in radar imagery. Identifying the locations of ice surface and bottom is typically performed manually, which is a very time-consuming procedure. Here, we propose an approach, which automatically detects ice surface and bottom boundaries using distance-regularized level-set evolution. In this approach, the complex topology of ice surface and bottom boundary layers can be detected simultaneously by evolving an initial curve in the radar imagery. Using a distance-regularized term, the regularity of the level-set function is intrinsically maintained, which solves the reinitialization issues arising from conventional level-set approaches. The results are evaluated on a large data set of airborne radar imagery collected during a NASA IceBridge mission over Antarctica and show promising results with respect to manually picked data.
Maryam Rahnemoonfar, Geoffrey C. Fox, Masoud Yari, John Paden
IEEE Trans. Geosci. Remote. Sens.1
2011 Restoration of Arbitrarily Warped Historical Document Images Using Flow Lines
abstract
Historical documents frequently suffer from arbitrary geometric distortions (warping and folds) due to storage conditions, use and to, some extent, the printing process of the time. In addition, page curl can be prominent due to the scanning technique used. Such distortions adversely affect OCR and print-on-demand quality. Previous approaches to geometric restoration either focus only on the correction of page curl or require supplementary information obtained by additional scanning hardware - not practical for existing scans. This paper presents a new approach to detect and restore arbitrary warping and folds, in addition to page curl. Warped text lines and the smooth deformation between them are precisely modelled as primary and secondary flow lines that are then restored to their original linear shape. Preliminary, but representative, experimental results, in comparison to a leading page curl removal method and an industry-standard commercial system, demonstrate the effectiveness of the proposed method.
Maryam Rahnemoonfar, Apostolos Antonacopoulos
ICDAR1
2005 Two dimensional phase unwrapping of interferometric SAR data by means of wavelet technique
abstract
One of the reliable methods in least-squares method is multigrid technique which overcomes the problem of slow convergence and less-accurate of Gauss-Seidel by transforming problem to coarser grid. It makes a pyramid of grids. Each grid has half the resolution of its predecessor. It uses two restriction and prolongation operators called fine-to-coarse and coarse-to-fine operators respectively. In this research, discrete wavelet decomposition and its reconstruction have been applied on the two operators. One of the assumptions made on this operator is that as long as the wavelet transformation decomposes the 2-D signal to one low frequency and three high frequency components, it should converge faster and more accurate than the multigrid method. This is due to the fact that the transformation of only low frequency component would suffice rather than transforming the whole grid to coarser grid. The idea has been implemented and tested on simulation data and the results confirm the assumption. In this paper the results of implementation of various wavelet filters and also multigrid techniques on various simulation data (with and without noise) are presented. In all cases, wavelet techniques have shown improved results than multigrid techniques.
Maryam Rahnemoonfar, Mohammad Rahmati, Ahad Tavakoli, M. R. Saradjian
IGARSS1