Weakly supervised visible-to-infrared image translation in remote sensing scenes is a challenging task that attempts to generate corresponding infrared images from visible images with cross-model manner, thus tackling the acute scarcity of infrared data in certain crucial task scenes. Existing methods frequently concentrate solely on style transfer, resulting in stylistic similarity but physical inconsistency, which limits the authenticity of cross-modal translation. Infrared thermal features stemming from brightness and darkness contrasts among distinct regions due to temperature disparities, are associated with visual representations in foreground objects in visible images. Therefore, this paper proposes ThermalMask, a framework for infrared image generation guided by thermal features, aiming to make the generated images more closely with infrared features in remote sensing. Specifically, a saliency mask generation network (SMGN) and a semantic attention generation network (SAGN) are designed in the generator. The SMGN is used to generate background and foreground masks to preserve the background and most prominent features of the input remote sensing image. Meanwhile, the SAGN generates attention maps through pyramid non-local attention mechanism, guiding the network to adaptively focus on salient regions within both visible and infrared feature maps. In addition, to refine the generated infrared features, a pixel-spectrum multi-spatial constraint (PSC) targeting high-frequency components is designed, enabling the generative network to focus more on expressing infrared features. Extensive experiment results have shown that our method has good performance in visible-to-infrared image translation within remote sensing scenes, surpassing existing state-of-the-art image translation methods. Moreover, it holds promise for application in downstream tasks where infrared data is insufficient.

Due to the limitations of infrared image acquisition conditions, many essential tasks currently rely on visible images as the main source of training data. However, single-modal data makes it difficult for downstream networks to show optimal performance. Therefore, converting the more easily obtainable visible images into infrared images emerges as an effective remedy to alleviate the critical shortage of infrared data. Yet current methods typically focus solely on transferring visible images to infrared style, while overlooking the crucial infrared thermal feature during cross-modal translation. To elevate the authenticity of cross-model translation at the feature level, this paper introduces a translation network based on frequency feature mapping and dual patches contrast, MappingFormer, which can achieve cross-modal image generation from visible to infrared. Specifically, the generator incorporates two branches: low-frequency feature mapping (LFM) and high-frequency feature refinement (HFR), both have embedded the Swin Transformer blocks. The LFM branch captures the fuzzy structural from visible images, while the HFR focuses on mapping edge and texture features. The extracted dual-branch frequency features undergo refinement and fusion through cross-attention mechanisms. Additionally, a dual contrast learning mechanism based on feature patch (DFPC) is designed to infer effective mappings between unaligned cross-modal data. Numerous experimental results prove the effectiveness of this method in cross-modal feature mapping and image generation from visible to infrared. This method holds significant potential for downstream tasks where infrared data is limited.

In recent years, the Geostationary Interferometric Infrared Sounder (GIIRS) has provided important data support for space-based and ground-based synergistic observation. However, the cloudy environment can easily affect the data of GIIRS, whose spatial resolution is nearly 16 km. As far as we know, no previous research has specifically investigated a cloud-clearing method for GIIRS in the synergistic observation of space-based and ground-based infrared sounders. Based on specially designed alternate attention blocks (AABs), we propose the Synergistic Cloud-Clearing Transformer (SCCT), which utilizes ground-based Atmospheric Emitted Radiance Interferometer (AERI) clear-sky observations and a single cloudy field of view (FOV) from the GIIRS to perform cloud-clearing for the GIIRS in synergistic observation. This approach eliminates the requirement of imager data and identical cloud properties assumption for adjacent GIIRS FOVs. The model training and verification were conducted based on the FY-4 A/GIIRS and AERI-measured data collected from 2020 to 2021. Results show that in the experiment using simulated clear-sky observation as a baseline, the average RMSE and MRAE of SCCT are reduced by 2.75 and 0.037, respectively, compared to the traditional${N}^{\ast }$cloud-clearing method. In the experiment using adjacent clear-sky FOV observation as a baseline, the reduction of SCCT on average RMSE and MRAE is 5.876 and 0.058 compared to the traditional method.

Abstract We present parallel single-pixel imaging (PSI), a photography technique that captures light transport coefficients and enables the separation of direct and global illumination, to achieve 3D shape reconstruction under strong global illumination. PSI is achieved by extending single-pixel imaging (SI) to modern digital cameras. Each pixel on an imaging sensor is considered an independent unit that can obtain an image using the SI technique. The obtained images characterize the light transport behavior between pixels on the projector and the camera. However, the required number of SI illumination patterns generally becomes unacceptably large in practical situations. We introduce local region extension (LRE) method to accelerate the data acquisition of PSI. LRE perceives that the visible region of each camera pixel accounts for a local region. Thus, the number of detected unknowns is determined by local region area, which is extremely beneficial in terms of data acquisition efficiency. PSI possesses several properties and advantages. For instance, PSI captures the complete light transport coefficients between the projector–camera pair, without making specific assumptions on measured objects and without requiring special hardware and restrictions on the arrangement of the projector–camera pair. The perfect reconstruction property of LRE can be proven mathematically. The acquisition and reconstruction stages are straightforward and easy to implement in the existing projector–camera systems. These properties and advantages make PSI a general and sound theoretical model to decompose direct and global illuminations and perform 3D shape reconstruction under global illumination.

The ill-posed separation problem of land surface temperature and emissivity (LST and LSE) from ground-based Fourier transform infrared (FTIR) spectrometer data is vulnerable to random errors generated in the preceding measurement and calibration processes. Under the assumption that the influence of random radiative errors is mainly manifested in the overall stretching (multiplicative noise) and high frequency fluctuation (additive noise) upon at-ground radiance spectrum, the traditional iterative retrieval method is improved to an antinoise estimation method. A prepolishing strategy in radiance domain is employed to suppress the additive noise. A spectrally invariant coefficient is introduced during iteration for the estimation and correction of multiplicative noise. The Savitzky-Golay filtering is introduced to the proposed method first. For the simulation data of typical ground-based FTIR spectrometers (spectral sampling interval of 1-4 cm-1, noise equivalent spectral radiance of 5 ×10-9W/(cm2·sr ·cm-1), and multiplicative noise error of 1.015), the root-mean-square error (RMSE) of LST is ≤ 0.4 K, the RMSE of LSE is <; 0.01 in the long-wave infrared range. For field data of a spectrometer under similar noise conditions, the RMSE of LST is about 0.7 K and consistent with the LST measured by a thermocouple thermometer, considering the measurement uncertainty. The proposed method is also generalized to use moving average filtering, derivation filtering, the Fourier transform, and wavelet transform. For the field data with both multiplicative and additive noises, the antinoise estimation methods with these tested filters have similar accuracy; among them, the moving average filtering and the Savitzky-Golay filtering are recommended.

An improved position error model for airborne hyperspectral imaging systems is proposed to quantify the position errors caused by the deficiencies of instruments and the manual installation of integrated systems. The model is based on a thorough analysis of position error sources, which include uncertainties caused by camera position and altitude, interior camera parameters, gyro stabilized mount, position setting error of the global positioning system antenna, and boresight angle error. An improved position error model that describes these error sources is established based on collinear equations. This model is then expanded into the first-order Taylor series. Finally, the error propagation law is applied to estimate the position errors. The performance of the proposed method is evaluated based on real hyperspectral images of Hymap. Results show that the mean square error of the position error is better than 2.1 pixels, which is almost the same as that of ground truth.

Shifts of center wavelengths (CWLs) and related changes of full-widths at half-maximums (FWHMs) occur during in-flight data acquisitions of push-broom imaging spectrometers such as the airborne prism experiment (APEX). Combined with the spectrally changing properties of the dichroic coating that acts as a beam splitter between the visible and near infrared (VNIR) as well as the short-wave infrared (SWIR) channels, these shifts affect both the spectral and radiometric accuracies of the spectrometer data, and hence the accuracy of higher level products. In this paper, two independent standards, i.e., atmospheric absorption features as well as features of the standard reference material filter built in the APEX in-flight characterization facility, are used in a complementary way to improve in-flight spectral calibration. The CWL shift and FWHM change for each detector element are simultaneously detected by using spectrum-matching and surface fitting techniques under constraints from pregenerated shift realizations. Subsequently, the APEX spectroradiometric response model is improved in the aspect of spectral resolution by using performance parameters of optics and detector modules. The radiometric gain and offset for each detector element are corrected according to the detected CWLs and FWHMs, as well as the improved APEX response model. Compared with the spectral and radiometric parameters acquired during laboratory calibration, the detected CWLs and FWHMs promote the accuracy of the atmospheric feature positions in the SWIR channel by 10 nm, whereas the corrected gains and offsets reduce the radiance deviation in the spectral regions 375-550 nm and 950-1080 nm both by 70% on average.

In this paper we propose a new approach to fiber-optic sensor head based on voltage-controlled ferroelectric liquid crystals (FLC). Using nano-grid electrodes, nanometer-scale photo-alignment layers and deformed helix ferroelectric liquid crystals with a sub-wavelength helix pitch of 150 nm, we have realized a fully integrated polarization-based photonic transducer which exhibits fast response to an applied voltage/electric field, excellent linearity and allows multiplexing using standard, low cost telecom optical fibers. In addition, the electro-optical behavior of deformed helix ferroelectric (DHF) liquid crystal in reflective mode is described and tested. The electrically controlled reflectance has been measured at sub-kilohertz driving voltage frequency for different polarizations of the incident light and compared quite successfully with the simulation results.

Infrared spectrums play an important role in the information extraction of rock and minerals. Spectrum simulation is a fundamental issue of land surface scene simulation and image simulation of remote sensing systems. Signal to Noise Ratio is regarded as an essential parameter of instrument and remote sensing image. In this study, we used MODTRAN to simulate apparent radiance and different levels of additive white Gaussian noise was added to the simulated spectrum. In the section of noise impact on object recognition, Spectral Feature Fitting was chosen to compare the fit of simulated spectra with different noise levels to reference apparent radiance spectra without noise. Relative error is also calculated for the accuracy assessment which is helpful for validation and improvement of instrument parameters.

Various types of features can be extracted from very high resolution remote sensing images for object classification. It has been widely acknowledged that the classification performance can benefit from proper feature fusion. In this letter, we propose a softmax regression-based feature fusion method by learning distinct weights for different features. Our fusion method enables the estimation of object-to-class similarity measures and the conditional probabilities that each object belongs to different classes. Moreover, we introduce an approximate method for calculating the class-to-class similarities between different classes. Finally, the obtained fusion and similarity information are integrated into a marginalized kernel to build a support vector machine classifier. The advantages of our method are validated on QuickBird imagery.

Segmentation and classification are important tasks in remote sensing image analysis. Recent research shows that images can be described in hierarchical structure or regions. Such hierarchies can produce the state-of-the-art segmentations and can be used in the classification. However, they often contain more levels and regions than required for an efficient image description, which may cause increased computational complexity. In this letter, we propose a new hierarchical segmentation method that applies graph Laplacian energy as a generic measure for segmentation. It reduces the redundancy in the hierarchy by an order of magnitude with little or no loss of performance. In the classification stage, we apply local self-similarity feature to capture the internal geometric layouts of regions in an image. By incorporating advantages from both semantic hierarchical segmentation and local geometric region description, we have achieved better performance than those from the methods being compared. In the experimental section, we validate the effectiveness of our method by showing results on QuickBird and GeoEye-1 image data sets.

In this paper, we propose an approach for object detection via structural feature selection and part-based shape model. It automatically learns a shape model from cluttered training images without need to explicitly use bounding boxes on objects. Our approach first builds a class-specific codebook of local contour features, and then generates structural feature descriptors by combining context shape information. These descriptors are robust to both within-class variations and scale changes. Through exploring pairwise image matching using fast earth mover's distance, feature weights can be iteratively updated. Those discriminative foreground features are assigned high weights and then selected to build a part-based shape model. Finally, object detection is performed by matching each testing image with this model. Experiments show that the proposed method is very effective. It has achieved comparable performance to the state-of-the-art shape-based detection methods, but requires much less training information.

The experiment plays an important role in the remote sensing applications and theoretical studies. In this study, the image rendering software POV-Ray was used to investigate the errors in the process of near-surface remote sensing experiment, by analyzing the features of four components' fractions (sunlit leaves, shaded leaves, sunlit soil and shaded soil) of typical row plants. The study shows: the model of pointing the sensor toward the targets while moving along the track induces the lowest errors among the three commonly-adopted experimental modes to sample the directional information of the targets; the mode of rotating the sensor around the fixing point brings about fluctuations around the true values, due to the variations of the targets during the experiment; increasing the distance from the sensor to the targets can improve the precision; however, it is not the larger, the better.

Extracting man-made objects in satellite images which are generated from the meter to sub-meter resolution plays an important role in remote satellite image analysis. However, spectral characteristics of urban land objects are so similar. So the classification accuracies are far from satisfactory by using only spectral information. As a result, researchers turn to incorporate geometrical information into satellite image classification. In this paper, we introduce a new local feature, namely local self-similarity(LSS) which captures internal geometric layouts of local self-similarities, into high spatial resolution images classification application. Our method captures self-similarity of color, edges, repetitive patterns and complex textures in a single unified way. With the help of Bag-of-Visual Words and SVMs, the proposed method performs well. Experimental results on Quickbird-image data set show that the proposed local self-similarity representation yields better classification performance than the low-level features, such as the spectral and texture features.

In order to detect the objects of interest, many different approaches have been proposed. One kind of popular approaches are based on template matching, which use a template of the object class to match the image at different positions. The matching can be computed using similarity measures such as the correlation coefficient. These approaches, although easy and robust, has the limitation of not containing to much variability of the object class, especially for shape information. Despite the statistical variation in each kind of object, collecting enough training samples is another problem which is time consuming. Inspire by template matching and incremental learning, a new object-oriented object detection methodology for very high resolution remote sensing images is proposed in this paper. We obtain the first initial query results via the bag-of-visual-words method. Then we introduce two query expansion baseline expansion and PAS expansion to obtain a new incremental model for re-query. In the experiment part, we compare and evaluate the performance of our proposed methods.

Hyperspectral radiance spectra are the sensor's response, through its spectral response functions (SRFs), to the at-sensor radiance field. As the SRFs vary with sensors, hyperspectral radiance data need to be transformed for cross-calibration with another sensor or data simulation of a future sensor. In fact, the hyperspectral radiance data are composed of average radiance in the sensor's passbands and bear a spectral smoothing effect. Current data transformation methods do not fully consider this spectral response effect in hyperspectral data, and it restricts the accuracy of the transformation and, hence, the accuracy of the cross-calibration and data simulation. The spectral response effect is modeled as a correlation-sampling process instead of convolution sampling to fit all the hyperspectral, multispectral, and panchromatic sensors. An iterative method is proposed to get a superresolution spectrum from the hyperspectral radiance spectrum according to the response model. Data of another sensor are simulated by responding to the superresolution spectrum. Validation experiments are performed using MODTRAN4-simulated spectrum, coincident Earth Observing-1 Hyperion and Advanced Land Imager (ALI) radiance data, and nearly coincident Hyperion and Landsat-7 Enhanced Thematic Mapper Plus data. For transformations from the MODTRAN4-simulated spectrum to the hyperspectral spectrum of different spectral resolutions, the proposed transformation method achieves the least root mean square relative error (RMSRE) compared with the linear interpolation, the cubic spline interpolation, and the deconvolution-sampling method. For transformation from the Hyperion data to the ALI multispectral and panchromatic data, the RMSRE is less than 9% and 4%, respectively.

Adjacency effect contaminates pixel spectrum in remotely sensed hyperspectral data and should be fully considered in hyperspectral scene simulation. But existing simulation method cannot model all the mechanisms that induce adjacency effect or complete the simulation with full adjacency effect in a few hours. A method is proposed to simulate the hyperspectral radiance data cube with full adjacency effect in a reasonable time. The method depends on a physically-based radiative transfer equation, in which parameters relating to atmospheric and geometric conditions are calculated through four times of MODTRAN4 running and surrounding average reflectance are calculated based on the atmospheric single scattering Point Spread Function (PSF). The sensor's spatial resolution, spectral response function and Signal-to-Noise Ratio (SNR) are simulated to generate a calibrated data cube. Performance of the method is validated through simulating a set of AVIRIS radiance data and comparing it with the actual one.

Radiation correction is often required in popular spectral unmixing of hyperspectral data, followed by interactive endmember determination methods. It involves much heavy work for the huge amount of hyperspectral data. In this paper, an uncorrected image of Push-broom Hyperspectral Imager (PHI) was automatically unmixed based on the linear mixing model, using minimum noise fraction (MNF) transformation to find the inherent dimensionality of the data, convex geometry concepts to extract endmembers and least squares method to estimate the fractional abundances. The result abundance images indicated that hyperspectral data, in subsection of appropriate size, can be unmixed before radiation correction and no apriori ground information is required.

To avoid the restriction of neuron activation functions of neural learning algorithm and the disadvantage of getting into local optimum solution with general numerical computation method, a novel independent component analysis (ICA) based on improved quantum genetic algorithm (IQGA) is proposed in our paper. Moreover, Han's quantum genetic algorithm (QGA) is improved by adopting the quantum crossover and quantum mutation to overcome the premature convergence and increase the search capability in our work. The proposed algorithm is applied to hyperspectral anomaly detection. The effectiveness of the algorithm is evaluated by HYDICE hyperspectral images. It is demonstrated that the proposed algorithm has better detection effect and time efficiency than QGA