The Earth observation era has bestowed dividends upon supervised land cover classification based on deep learning and optical data. However, limitations, such as insufficient spectral information and reduced quality during inclement weather for optical data, coupled with the need for extensive labeled samples, impede accurate classification. This letter harnesses multimodal images with deep contrastive learning to reduce reliance on labeled data and classify land covers. By employing a well-designed contrastive learning method with triangular similarity loss, our model can learn effective multimodal features without labeled samples. Moreover, the learned features are fused at the early feature level and used for the downstream classification task with fewer labeled samples. Experimental results demonstrate the benefits of incorporating multiple modalities, highlighting the potential of combining multimodal image analysis and contrastive learning for land cover classification with limited labeled samples.

Hyperspectral images (HSIs) provide exceptional spatial and spectral resolution of a scene, crucial for various remote sensing applications. However, the high dimensionality, presence of noise and outliers, and the need for precise labels of HSIs present significant challenges to the analysis of HSIs, motivating the development of performant HSI clustering algorithms. This paper introduces a novel unsupervised HSI clustering algorithm—Superpixel-based and Spatially-regularized Diffusion Learning (S2DL)—which addresses these challenges by incorporating rich spatial information encoded in HSIs into diffusion geometry-based clustering. S2DL employs the Entropy Rate Superpixel (ERS) segmentation technique to partition an image into superpixels, then constructs a spatially-regularized diffusion graph using the most representative high-density pixels. This approach reduces computational burden while preserving accuracy. Cluster modes, serving as exemplars for underlying cluster structure, are identified as the highest-density pixels farthest in diffusion distance from other highest-density pixels. These modes guide the labeling of the remaining representative pixels from ERS superpixels. Finally, majority voting is applied to the labels assigned within each superpixel to propagate labels to the rest of the image. This spatial-spectral approach simultaneously simplifies graph construction, reduces computational cost, and improves clustering performance. S2DL’s performance is illustrated with extensive experiments on four publicly available, real-world HSIs: Indian Pines, Salinas, Salinas A, and WHU-Hi. Additionally, we apply S2DL to landscape-scale, unsupervised mangrove species mapping in the Mai Po Nature Reserve, Hong Kong, using a Gaofen-5 HSI. The success of S2DL in these diverse numerical experiments indicates its efficacy on a wide range of important unsupervised remote sensing analysis tasks.

The brightness temperature (TB) features extracted from Chang’e Lunar Microwave Sounder (CELMS) data have been proved their superiority to study mare basalt. In this paper, dimension reduction and feature space analysis are conducted on TBfeatures to fully understand the data distribution and reduce the feature redundancy in the classification process based on two methods - Principal Component Analysis (PCA) and Nonnegative Matrix Factorization (NNMF). The results showed that PCA and NNMF can effectively enhance the classification capability of early(?)- and late(?)-age mare basalt respectively, and proved the necessity of dimension reduction for CELMS TBfeatures due to the largely-existing redundancy.

The rapid urbanization process over metropolitan cities often came with the dramatic urban land transformation of different natural land cover types into urban impervious surfaces (UIS) or UIS back to natural land covers. However, most UIS mapping studies usually focus on inter-year variation, suggesting that the urbanization process is irreversible. The rapid intra-year urban reconstruction and urban renewal are under-explored. This study develops a new algorithm for intra-year urban renewal using all available Sentinel-1 and Sentinel-2. Time series SAR and optical data are utilized for intra-year UIS change detection, i.e., to detect the changes from UIS to other land cover types. Results showed that the urban renewal process occurred within the metropolitan area, i.e., UIS changed into the bare land or vegetation. The overall accuracy and kappa of urban renewal change detection are 72.79% and 0.4599, respectively.

Urban impervious surfaces (UIS) indicate the environmental and socioeconomic influences of rapid urbanization. Synthetic aperture radar (SAR) reflects the scattering behaviors of different land covers while multispectral data demonstrate their physicochemical properties. Numerous studies reported that the incorporation of SAR and optical data supplement each other for better extracting UIS, nevertheless, the shadow and layover effects remain unclear, especially in very high-resolution observations. This study analyzed the shadow and layover influences from both optical and SAR data for fine resolution UIS estimation. Given the SAR shadow and layover distribution, we proposed a multisource shadow-based fuzzy set (MSFS) approach for fusing optical and SAR in optical shadow areas using decision fusion. SAR layovers showed effectiveness in UIS extraction. MSFS delivered 3% and 7% improvement in overall accuracy compared with SVM and RF using feature fusion respectively.

Sonneratia have posed a threat to native mangrove species in Hong Kong. Early detection of individual Sonneratia when they are introduced and naturalized before invasion is essential for native mangrove species protection, especially for Sonneratia with a strong ability of propagation. This letter aims to provide an effective way to the accurate detection of individual Sonneratia. Specifically, using very high spatial resolution remotely sensed data, we adapt the RetinaNet, incorporating multiscale features for sapling detection and convolutional neural networks for detecting the Sonneratia distributed scatteredly among native species. The Sonneratia were detected with a higher mean average precision (mAP) 0.50 of 0.3891 with a precision of 0.5465 than that from the deformable part model. In addition, 3678 Sonneratia were detected at early stage. This letter can support the government for mangrove forest management and offer a scientific guidance for adequate response to the species invasion, like annual removal of Sonneratia, and then reduce the consumption of labor and time over a large scale. In addition, it can provide a quantitative survey for Sonneratia management.

High-resolution urban impervious surface (UIS) is essential for social and environmental analysis. However, shadows have become a major challenge to the accurate UIS mapping in high-resolution optical images, as the low reflectance usually leads to misclassification of shadows as roads or waters. To solve this problem, we proposed a shadow free multisource stack sparse autoencoder (ShdFree-MS-SSAE) for urban shadow detection and compensation. Multisource data, including optical, SAR and LiDAR were used for the occlusion information recover. First, MS-SSAE was proposed for urban land cover classification, including shadow and non-shadow area. Then, shadow area in optical data was enhanced with a linear compensation method. Finally, MS-SSAE was applied to classify the enhanced shadow area and the non-shadow area. The results demonstrated that ShdFree-MS-SSAE framework was effective for UIS mapping, with an average improvement of 10%.

Zonation of mangrove species is the predictable and discrete ordering of mangrove species caused by a unique, intertidal environment. Mangrove zonation pattern is formed by complex abiotic and biotic environmental factors and, at the same time significantly influences the associate flora and fauna communities even the whole coastal ecosystem and local carbon cycle. In this study, high resolution time-series satellite images were employed to investigate the characteristics and inner structures of mangrove zonation using landscape metrics during the past decade in the Deep Bay area, Hong Kong SAR and Shenzhen, China. Both native and exotic mangrove species were discriminated and analyzed. The results of this study shown that native mangrove stands in the Deep Bay showed relatively clearer zonation during the past decade, presenting a sequence of the species in this tide-dominated shore with higher aggregation and reunion degree. In comparison, the distribution pattern of exotic species is in higher degree of fragmentation and less connectivity. The patch-based landscape metrics were proven to be effective methods in measuring and describing the spatial distribution pattern of mangrove zonation based on high resolution satellite images.

Not only the urbanization development but also its ecological process lays emphasis on the Impervious Surface Area (ISA) extraction, whereas, the ISA extraction from high-resolution images is challenging for both the phenomenon of the mixed pixels and shadow effects. To solve the problem, a Multi-Source Dictionary Sparse Representation Classification (MSD-SRC) method using WorldView-2 and TerraSAR-X dataset is proposed. First, it uses multi-source data and fuzzy samples by Low Pass Filtering (LPF) to solve the problem of road and building misclassification; second, learning Multi-Source Dictionary for non-shadow and shadow classes, then using discriminative sparse coding method for classification, therefore to reduce shadow effects and improve the ISA extraction accuracy. Experimental results demonstrated the effectiveness of the proposed method.

Incorporating optical and SAR data to estimate impervious surface is useful but challenging due to their different geometric imaging mechanism. The recent development of deep convolutional networks (DCN) opens a promising opportunity. In this study, the typical DCN, AlexNet, was modified to estimate the impervious surface from optical and SAR data. GoogLeNet and the Support Vector Machine (SVM) were employed for comparison. Experimental results indicated the effectiveness of AlexNet with an accuracy of over 99%, outperforming both GoogLeNet and SVM. Furthermore, 60~80% of training samples outperformed the results from the whole training set under certain number of epochs, indicating that large number of training samples may not necessarily produce better results, depending on other factors (e.g. number of epochs). Generally, AlexNet was able to fuse the optical and SAR data and improved the accuracy of estimating impervious surface by about 2% compared with that using optical data alone.

In the previous work [2], we proposed an error diffused intra prediction for HEVC to improve its coding performance. The proposed algorithm can achieve average 0.5% BDBR reduction with 21% increase of encoding time, compared to the HEVC intra prediction. To further improve the computation efficiency, in this paper, we suggest incorporating ordered dither technique into HEVC intra prediction to reduce computational complexity, instead of error diffusion. The experimental results reveal that average 0.5% BDBR reduction can be achieved in the proposed algorithm and it brings out only an increase of average 4% of total encoding time compared to the HEVC intra prediction, and that is much lower than the error diffused intra prediction.

HEVC uses up to 35 prediction modes for intra prediction and it can well predict blocks with uni-directional structures or sharp edges, but the intra prediction still suffers from its discontinuous characteristics. To improve coding performance of intra prediction, the inpainting technique has been studied but it is impractical because of its high computational complexity. In this paper, we employ error diffusion technique for HEVC intra prediction to improve its coding efficiency with reasonable increase in computational complexity. The experimental results show that the error diffusion technique outperforms the inpainting technique subjectively and objectively, especially with much lower computational complexity. The results demonstrate that average 0.5% BDBR reduction can be achieved in the proposed algorithm, compared to HEVC intra prediction.

This paper proposes an efficient motion re-estimation using enhanced predictive zonal search (EPZS) algorithm for H.264/AVC video downscaling transcoding to reduce computation cost while maintaining high coding performance. Three prediction motion vectors (PMV) using AAW, ABW and AWW approaches are defined based on CBP and MV information in original H.264/AVC coded videos. The proposed EPZS takes into account four MV predictors (including AAW, ABW and AWW predictors as well as median predictor defined in H.264/AVC). The experimental result reveals that the proposed EPZS algorithm achieves a significant reduction of 66% computation time while maintaining coding efficiency as good as fully decoding and re-encoding. The result also shows that the proposed algorithm outperforms the AWVM algorithm in all respects, including PSNR, bit-rate increment as well as computation saving.

In the literature, most motion compensated frame interpolation (MCFI) methods for frame rate up conversion are performed at the decoder side. To reduce computational complexity of the decoder and improve visual quality of interpolated frames, this paper proposes an advanced MCFI (AMCFI) algorithm which uses skipped frame information to select weight and motion vector for MCFI. The weight and motion vector selections are performed at the encoder side instead of decoder and at the decoder only a simple MCFI with low computation cost is performed. Experimental results show that the proposed algorithm outperforms other MCFI algorithms subjectively and objectively.

This paper proposes an efficient zero-block inter mode decision algorithm for high bit-rate coding in H.264/AVC, to improve computation efficiency of the previous algorithm [1]. In the proposed algorithm we suggest using zero-blocks of both 8×8 and 4×4 DCT coefficients to describe video sequences in detail, including global and local stationary characteristics. The experimental results reveal that significant computation improvement (up to 20% reduction) for high bit-rate coding can be achieved over the previous algorithm.

Our previous work presents an enhanced SATD-based intra mode decision to improve computation cost while maintaining good coding performance. Based on the enhanced SATD-based intra mode decision, this paper proposes an efficient H.264/AVC intra frame transcoding which utilizes original coded mode type information to eliminate unlikely modes before the enhanced intra mode decision. Both coding and computation performances are compared with complex cascaded pixel domain transcoding (CCPDT) and simple cascaded pixel domain transcoding (SCPDT) algorithms.

The popular video coding standard H.264/AVC features many advanced techniques in motion estimation, such as multiple reference frames, variable block sizes, and quarter pixel resolution. These new features significantly improve coding performance, but with extremely high computational complexity. To improve computation efficiency, this paper suggests a selective multiple reference frame motion estimation (SMRFME), which characterizes MRFME as a stationary Gaussian random process, and uses the stationary property to check whether a mode or a block is necessary to perform motion estimation on next reference frames. Experimental results reveal that the proposed SMRFME algorithm momentously improves computational complexity with an average reduction of 68% in total encoding time, and with negligible degradation in PSNR and bit-rate increment. The current research also compares SMRFME algorithm performance with the AFMFSA algorithm. Results demonstrate that the proposed algorithm outperforms the distinct algorithm in all aspects.

In this letter, a rate estimation technique using transform coefficient variance is proposed in the rate-distortion cost function for intra mode decision. The new cost function achieves better coding performance than the cost function using the ¿-domain model. The new cost function brings out an average speed-up factor of 13 when compared with the rate-distortion optimization (RDO) mode decision, but with considerable degradation in rate-distortion performance. To improve the coding performance, we propose a sum of the absolute transformed differences (SATD)-based mode decision algorithm, which incorporates both SATD and the variance information into the RDO mode decision method. The results show that the SATD-based mode decision scheme can achieve a significant improvement in computation (with 21% of total encoding time, or a speed-up factor of 5) with negligible peak signal-to-noise ratio loss. The proposed algorithm is also compared with two other distinct algorithms , and the results indicate that the proposed algorithm outperforms these two edge detection algorithms in coding performance, as well as time saving.

This letter proposes an enhanced sum of absolute transform differences (SATD)-based intra mode decision for H.264/advanced video coding intra frame coding, which is accomplished in two stages. In the first stage, the low-frequency AC components of discrete cosine transform block of a MB is used to select I4 MB or I16 MB mode prediction; while in the second stage SATD coefficients, including SATD value and its variance, is used to skip improper modes for rate-distortion optimization mode decision. This letter compares the proposed algorithm performance with other distinct algorithms such as image structure tensor and dominant edge direction, and so on, and the results indicate that the proposed algorithm significantly outperforms these algorithms, in both rate-distortion performance and computation performance.

In the previous work we presented a SATD-based intra mode decision algorithm for H.264/AVC to reduce the computational complexity, based upon SATD coefficients. In this paper, we extend the previous work and propose an improved SATD-based intra mode decision, which uses low-frequency components to provide local characteristics for intra mode selection. The experimental results reveal that our proposed algorithm can achieve significant reduction of computation time compared to the reference program, while still maintaining good coding performance. The proposed algorithm is also compared with two distinct algorithms and the results indicate that the proposed algorithm brings out higher reduction in computation that these two algorithms.

In this paper, we propose an efficient inter/intra mode decision for H.264/AVC inter frame transcoding. A zero-block decision scheme is first employed to select some candidate inter and intra modes for prediction. Soon afterward, we incorporate a fast multi-reference frame motion estimation with adaptive search window into inter mode decision. For intra mode decision we suggest a mode refinement scheme to eliminate improbable modes for RDO calculations to a greater extent. The experimental results reveal that average 93% of computation time can be saved for the proposed H.264 inter frame transcoding algorithm, when compared with fully decoding/encoding procedure. The degradation in the rate-distortion performance is fairly small.

In the previous paper , we proposed a zero-block intermode decision algorithm for H.264 video coding based upon the number of zero-blocks of 4 x 4 DCT coefficients between the current macroblock and the co-located macroblock. The proposed algorithm can achieve significant improvement in computation, but the computation performance is limited for high bit-rate coding. To improve computation efficiency, in this paper, we suggest an enhanced zero-block decision algorithm, which uses an early zero-block detection method to compute the number of zero-blocks instead of direct DCT and quantization (DCT/Q) calculation and incorporates two adequate decision methods into semi-stationary and nonstationary regions of a video sequence. In addition, the zero-block decision algorithm is also applied to the intramode prediction in the P frame. The enhanced zero-block decision algorithm brings out a reduction of average 27% of total encoding time compared to the zero-block decision algorithm.

In this paper, a rate estimation technique using the transform coefficient variance is proposed in the rate-distortion cost function for intra mode decision. The experiment shows that the rate estimation using the absolute transform coefficient variance can predict rates more accurate than the one using ρ-domain model. The new cost function brings out an average speed-up factor of 13.8 when compared with the RDO mode decision, but with considerable degradation in the rate-distortion performance. To improve the coding performance, we propose a SATD-based mode decision algorithm, which incorporates both SATD and the variance information into the RDO mode decision method. The results show that the SATD-based mode decision scheme can achieve a significant improvement in computation with negligible PSNR loss.

In the previous work, we presented a zero-block mode decision algorithm for H.264/AVC to reduce the computation time, based upon the number of zero blocks of 4times4 DCT coefficients between the current macroblock (MB) and the collocated macroblock. In this paper we extend the previous work and provide a more sophisticated algorithm, which uses an early zero-block detection method instead of DCT/Q computation, and incorporates adequate decision methods into semi-stationary and non-stationary regions of a video sequence. The simulation results demonstrate that the proposed algorithm can achieve up to 20% of time saving on average compared to the original zero-block mode decision algorithm, while maintaining a high coding performance.

Ordered dither is considered to be a simple and effective method among all halftoning techniques. In this paper, compaction of ordered dithered images using arithmetic coding is studied. A preprocessor referred to as pixel interleaving (i.e., grouping pixels with similar dithering thresholds) is employed in such a way that dithered images can be efficiently coded with the JBIG1 code and high compressibility can be achieved. Experimental results reveal that the four-pixel interleaving achieves the best compression performance.

A hybrid block truncation coding (BTC) is presented. In the hybrid BTC, a universal codebook using Hamming codes and a differential pulse code modulation (DPCM) are employed, respectively, to the bit plane and the side information of BTC to reduce coding rate. Simulation results reveal that the performance of the proposed algorithm is only slightly worse than that of the hybrid BTC using vector quantization (VQ) techniques, but with much lower computational or hardware complexity.

This paper proposes a modified dot-overlap printer model used in error diffusion. It can reduce the distortion due to the dot-overlap effect introduced in laser printers. In the modified model-based error diffusion, multipass is not required and the bias in the gray scale of the printed images can be entirely eliminated.

A (0,G/I) code is a modulation code used in the partial response maximum likelihood (PRML) recording system in which G and I represent the maximum number of zeros between two consecutive ones in the global sequence and the interleaved subsequences, respectively. For some magnetic recording systems such as the one with a helical scanning rotary head, a DC-balanced write waveform is required to meet channel requirements. In this study, charge-constrained (0,G/I:C) sequences which can generate a DC-balanced write waveform are investigated, where C represents the upper bound of charge. Based upon the runlength subgraphs and the transition matrices, capacities and power spectra of (0, G/I; C) sequences are derived and computed.