Deep learning is a powerful technique for semantic change detection (SCD) of bitemporal remote sensing images. In this work, we propose to improve SCD accuracy using deep learning with frequency feature enhancement. Specifically, we develop a frequency feature enhancement module that aims to enhance the performance of both binary change detection and semantic segmentation, two main key components for obtaining high SCD accuracy, by integrating the Fourier transform and attention mechanisms. Experimental results on the SECOND and LandSat-SCD datasets demonstrate the effectiveness of the proposed method, and it achieves high resolution for change boundaries.

Deep learning techniques have been used to build velocity models (VMs) for seismic traveltime tomography and have shown encouraging performance in recent years. However, they need to generate labeled samples (i.e., pairs of input and label) to train the deep neural network (NN) with end-to-end learning, and the real labels for field data inversion are usually missing or very expensive. Some traditional tomographic methods can be implemented quickly, but their effectiveness is often limited by prior assumptions. To avoid generating and/or collecting labeled samples, we propose a novel method by integrating deep learning and dictionary learning to enhance the VMs with low resolution by using the traditional tomography-least square method (LSQR). We first design a type of shallow and simple NN to reduce computational cost followed by proposing a two-step strategy to enhance the VMs with low resolution: (1) Warming up. An initial dictionary is trained from the estimation by LSQR through dictionary learning method; (2) Dictionary optimization. The initial dictionary obtained in the warming-up step will be optimized by the NN, and then it will be used to reconstruct high-resolution VMs with the reference slowness and the estimation by LSQR. Furthermore, we design a loss function to minimize traveltime misfit to ensure that NN training is label-free, and the optimized dictionary can be obtained after each epoch of NN training. We demonstrate the effectiveness of the proposed method through the numerical tests on both synthetic and field data.

Deep learning techniques have been widely used for semantic change detection (SCD) of remote sensing images (RSIs) and have shown encouraging performance. In this paper, we propose a novel neural network by embedding the difference enhancement (DE) module into the adjacent layers of ResNet for SCD of RSIs (DESNet), which can pay more attention to the changes of bi-temporal RSIs. Furthermore, we deploy the module of multi-scale parallel sampling spatial-spectral non-local (SSN) after feature extraction, which can effectively improve the robustness to large-scale changes and the integrity of the changed objects by fusing global features that sampled from the multi-scale feature space. The experimental tests demonstrate that our DESNet can achieve state-of-the-art accuracy on the SECOND dataset and the LandSat-SCD dataset.

The nonlinear conjugate gradient (NLCG) algorithm is one of the popular linearized methods used to solve the frequency-domain electromagnetic (EM) geophysical inverse problem. During NLCG iterations, the model gradient guides the searching direction while the line-search algorithm determines the step length of each iteration. Normally, the line search requires solving the corresponding forward problem a few times. Since line search is usually computationally inefficient, we introduce the adaptive gradient descent (AGD) algorithm to accelerate solving the frequency-domain EM inverse problem within the linearized framework. The AGD algorithm is a variant of the classical gradient descent method and has been well-developed and widely used in deep learning. Rather than the time-consuming line search, its core idea is to algebraically manipulate the cumulative gradients and updates of the model from previous iterations to estimate the model parameter variables at the current iteration. For the inversion of magnetotelluric (MT) data, we here designed and implemented a framework using the AGD algorithm combined with the cool-down scheme to tune the regularization parameter. To improve the convergence performance of the AGD algorithm [specifying to Adam and root-mean-square propagation (RMSProp)], we proposed a tolerance strategy which has been tested numerically. To optimize the global learning rate, we carried out some comparative trials in the proposed inversion framework. The inverted results of synthetic and real-world data showed that both the AGD algorithms (Adam and RMSProp) can recover comparable results and save more than a third of CPU time compared with the NLCG algorithm.

The efficiency of solving geophysical inverse problem largely relies on the efficiency of solving the corresponding forward problem. As for electromagnetic (EM) data forward modeling in frequency domain, the conventional numerical methods, e.g. finite difference method (FDM), discretize the governing equations resulting a large linear system which is usually expensive to solve. Meanwhile, for inversion iteration we normally do not need to solve the forward problem in high precision. Thus a rapid surrogate modeling approach which uses the neural network is promising for replacing the forward modeling module in the inversion scheme. Here we proposed an algorithm which uses the neural operator to solve the EM data modeling problem in the frequency domain. To develop a surrogate model for EM data forward problem, we introduce an extended Fourier neural operator (EFNO) that enables the calculation at least 100 times faster than the conventional FDM solver while maintaining good precision. Moreover, by adding a sub-network the proposed neural operator has good generalization which has the capacity of predicting solution at any site locations and frequencies. Due to the discretization-invariance of Fourier neural operator, the neural operator trained on coarse grids can easily transfer to fine grids with only retraining part of parameters, resulting in a super-resolution prediction capability. We test our proposed method with 2-D and 3-D magnetotelluric (MT) data modeling problems, demonstrating that the EFNO has great potentials for severing as a general rapid surrogate forward solver in EM data inversion scheme.

For seismic random noise attenuation, deep learning has attracted much attention and achieved promising performance. However, compared with conventional methods, the denoising performance of supervised learning-based methods heavily depends on massive training samples with high-quality labeled data, which makes their generalization capabilities limited. Even though deep neural networks (DNNs) usually outperform the conventional denoising methods, their performance is not guaranteed since neural networks still lack good mathematical interpretability at present. To alleviate the dependency on labeled data and explore insights into the denoising system, we proposed an unsupervised denoising method based on model-based deep learning, which combined domain knowledge and a data-driven method. We designed a network based on the modified iterative soft threshold algorithm (ISTA), which omitted the soft threshold to alleviate uncertainties introduced by empirically selected thresholds. In this network, we set the dictionary and code as trainable parameters. A loss function with a smooth penalty was designed to ensure that the network training can be implemented in an unsupervised manner. In the proposed method, we set the denoised result by$f-x$deconvolution as the input for our network, and the further denoised data can be obtained after each epoch of the training, which means that our method does not need the testing procedure. Experiments on synthetic and field seismic data demonstrate that our method exhibits competitive performance compared to the conventional, supervised, and unsupervised methods, including$f-x$deconvolution, curvelet, the Denoising Convolutional Neural Network (DnCNN), and the integration of neural network and Block-matching and 3-D filtering method (NN + BM3D).

Over the last decades, seismic random noise attenuation has been dominated by transform-based denoising methods over the last decades. However, these methods usually need to estimate the noise level and select an optimal transformation in advance, and they may generate some artifacts in the denoising result (e.g., nonsmooth edges and pseudo-Gibbs phenomena). To overcome these disadvantages, we trained a deep convolutional neural network (CNN) with residual learning for seismic data denoising. We used synthetic seismic data for network training rather than seismic images, and we adopted a method to preprocess the seismic data before it was inputted in the network to help network training. We demonstrate the performance of the deep CNN in seismic random noise attenuation based on the synthetic seismic data. Results of numerical experiments show that our network adaptively and effectively suppresses noise of different levels and exhibits a competitive performance in comparison with the traditional transform-based methods.