In subsea environments, sound navigation and ranging (SONAR) images are widely used for exploring and monitoring infrastructures due to their robustness and insensitivity to low-light conditions. However, their quality can degrade during acquisition and transmission, where standard SONAR image processing techniques can hardly produce high-quality outcomes. An effective image quality assessment (IQA) method can assess their usefulness and aid to develop refinement techniques by identifying the degradation issues, ensuring the reliability of SONAR data. Existing methods often fail to account for degradations from noise, distortion, and resolution changes simultaneously. To address this challenge, we propose a new blind quality assessment method that measures the overall quality of SONAR images by quantifying both the perceptual and utility qualities using the micro- and macro-scale texture and contour features derived from the wavelet domain. By combining the local binary pattern (LBP) micro-scale texture features with the proposed histograms of Schmid Gabor-like edge maps as macro-scale features, a support vector regression model is learned to map from these features to subjective quality scores. Extensive experiments have demonstrated the superiority of our method over existing SONAR IQA techniques on distorted and reconstructed super-resolution side-scan, acoustic lens, and forward-looking SONAR images. Specifically, our method achieves Pearson’s and Spearman’s correlation metrics of 0.8616 and 0.8541, respectively, for distorted SONAR images, demonstrating improvements of 4.69% and 4.8%. For reconstructed super-resolution SONAR images, our method attains correlation metrics of 0.9415 and 0.9408, reflecting improvements of 0.8% and 1.6% over the second-best method, respectively. To facilitate ease of access, a comprehensive list of key abbreviations and their full names is provided in Table A.9 in the Appendix section. The source code of the proposed method will be shared at https://github.com/hfarhaditolie/BSIQA .

Fuzzy Broad Learning System (Fuzzy BLS) constitutes an effective neural network architecture that has demonstrated remarkable efficacy across various real-world application domains. Nonetheless, Fuzzy BLS may result in suboptimal performance and face challenges in effectively addressing the issue of imbalanced classification. To tackle the challenge mentioned above, a novel Mixture-of-Expert-based Fuzzy Broad Learning System (MFBLS) is proposed. In MFBLS, a fuzzy system is integrated to deal with the fuzziness of input data. Concurrently, based on the advantages of the Mixture-of-expert framework, the feature weight of each expert system is dynamically adjusted via a gating network to enhance the importance of key features, thereby enhancing the overall capability of the model. Besides, several classical over-sampling techniques are employed to address sample imbalance in datasets to achieve a more balanced class distribution. Subsequently, the Extreme Gradient Boosting (XGBoost) algorithm is utilized on the oversampled dataset for feature selection, aiming to enhance the efficiency and the predictive precision of subsequent model training in MFBLS. Finally, through a comparative analysis of state-of-the-art machine learning methods, the superiority of MFBLS in handling imbalanced datasets is validated through a series of experimental evaluations on various imbalanced datasets.

Oriented object detection in remote sensing is a critical task for accurately location and measurement of the interested targets. Despite of its success in object detection, deep learning-based detectors rely heavily on extensive data annotation. However, variations in object appearance significantly increase the difficulty and the cost of creating large-scale annotated datasets. Semi-supervised learning (SSL) aims to utilize unlabeled data to enhance object detectors. Among these, pseudo-label-based methods have shown promising results recently. Nonetheless, as training progresses, the accumulation of errors in pseudo labels leads to prediction bias without corrections. To tackle this particular challenge, we present a SSL pipeline, named “dual teacher,” for improving the reliability of pseudo labels in the semi-supervised oriented object detection. First, to mitigate the bias caused by limited annotated data, a global burn-in (GBI) strategy is introduced at the beginning of training, which guides the student detector to learn the feature extraction on a global scale. In addition, an online bounding box (bbox) correction module is proposed to decrease the occurrence of mislabeled instances and enhance the reliability of detection. These improvements are facilitated by an additional detector, instead of a single teacher model in the teacher-student architecture. Dual teacher reduces the dependency on the quality of pseudo labels related to the model complexity and combines the strengths of both the two-stage and one-stage detectors. With only 20% labeled data, dual teacher outperforms fully supervised rotated fully convolutional one-stage object detection (R-FCOS), you only look once X-small (YOLOX-s), and rotated region-based convolutional neural network (R-RCNN) by up to 2% on both a large-scale dataset for object detection in aerial images (DOTA) and SODA-A datasets. This reveals its potential in reducing labor-intensive tasks and enhancing robustness against environmental interference and noisy labels. The code is available at:https://github.com/ZYFFF-CV/DualTeacher-semisup.git.

World models significantly enhance hierarchical understanding, improving data integration and learning efficiency. To explore the potential of the world model in the remote sensing (RS) field, this article proposes a label-efficient RS world model for multimodal data fusion (FusDreamer). The FusDreamer uses the world model as a unified representation container to abstract common and high-level knowledge, promoting interactions across different types of data, that is, hyperspectral (HSI), light detection and ranging (LiDAR), and text data. Initially, a new latent-spatial multimodal generation (LaMG) paradigm is utilized for its exceptional information integration and detail retention capabilities. Subsequently, an open-world knowledge-guided consistency projection (OK-CP) module incorporates prompt representations for visually described objects and aligns language-visual features through contrastive learning. In this way, the domain gap can be bridged by fine-tuning the pre-trained world models with limited samples. Finally, an end-to-end multitask combinatorial optimization (MuCO) strategy can capture slight feature bias and constrain the diffusion process in a collaboratively learnable direction. Experiments conducted on four typical datasets indicate the effectiveness and advantages of the proposed FusDreamer. The corresponding code will be released athttps://github.com/Cimy-wang/FusDreamer.

In the remote sensing field, detecting small objects is a pivotal task, yet achieving high performance in deep learning-based detectors heavily relies on extensive data annotation. The challenge intensifies as small objects in remote sensing imagery are typically densely distributed and numerous, leading to a substantial increase in the cost of creating large-scale annotated datasets. This elevated cost poses significant limitations on the application and advancement of small object detection. To address this issue, a point-based annotation (PBA) method is proposed, which generates bounding boxes (BBOXs) through graph-based segmentation. In this framework, user annotations categorize nodes into three distinct classes—positive, negative, and to-cut—facilitating a more intuitive and efficient annotation process. Utilizing the max-flow algorithm, our method seamlessly generates oriented BBOXs (OBBOXs) from these classified nodes. The efficacy of PBA is underscored by our empirical findings. Notably, annotation efficiency is enhanced by at least 40%, a significant leap forward. Moreover, the intersection over union (IoU) metric of our OBBOX outperforms existing methods like “segment anything model (SAM)” by 10%. Finally, when applied in training, models annotated with PBA exhibit a 3% increase in the mean average precision (mAP) compared with those using traditional annotation methods. These results not only affirm the technical superiority of PBA but also its practical impact on advancing small object detection in remote sensing.

The widespread dissemination of Deepfake in social networks has posed serious security risks, thus necessitating the development of an effective Deepfake detection technique. Currently, video-based detectors have not been explored as extensively as image-based detectors. Most existing video-based methods only consider temporal features without combining spatial features, and do not mine deeper-level subtle forgeries, resulting in limited detection performance. In this paper, a novel spatiotemporal trident network (STN) is proposed to detect both spatial and temporal inconsistencies of Deepfake videos. Since there is a large amount of redundant information in Deepfake video frames, we introduce convolutional block attention module (CBAM) on the basis of the I3D network and optimize the structure to make the network better focus on the meaningful information of the input video. Aiming at the defects in the deeper-level subtle forgeries, we designed three feature extraction modules (FEMs) of RGB, optical flow, and noise to further extract deeper video frame information. Extensive experiments on several well-known datasets demonstrate that our method has promising performance, surpassing several state-of-the-art Deepfake video detection methods.

Poor real-time performance in multi-QR codes detection has been a bottleneck in QR code decoding-based Internet of Things (IoT) systems. To tackle this issue, we propose in this article a rapid detection approach, which consists of multistage stepwise discrimination (MSD) and a Compressed MobileNet. Inspired by the object category determination analysis, the preprocessed QR codes are extracted accurately on a small scale using the MSD. Guided by the small scale of the image and the end-to-end detection model, we obtain a lightweight Compressed MobileNet in a deep weight compression manner to realize rapid inference of multi-QR codes. The average detection precision (ADP), multiple box rate (MBR) and running time are used for quantitative evaluation of the efficacy and efficiency. Compared with a few state-of-the-art methods, our approach has higher detection performance in rapid and accurate extraction of all the QR codes. The approach is conducive to embedded implementation in edge devices along with a bit of overhead computation to further benefit a wide range of real-time IoT applications.

The principal component analysis (PCA) and 2-D singular spectral analysis (2DSSA) are widely used for spectral- and spatial-domain feature extraction in hyperspectral images (HSIs). However, PCA itself suffers from low efficacy if no spatial information is combined, while 2DSSA can extract the spatial information yet has a high computing complexity. As a result, we propose in this letter a PCA domain 2DSSA approach for spectral–spatial feature mining in HSI. Specifically, PCA and its variation, folded PCA (FPCA) are fused with the 2DSSA, as FPCA can extract both global and local spectral features. By applying 2DSSA only on a small number of PCA components, the overall computational cost can be significantly reduced while preserving the discrimination ability of the features. In addition, with the effective fusion of spectral and spatial features, our approach can work well on the uncorrected dataset without removing the noisy and water absorption bands, even under a small number of training samples. Experiments on two publicly available datasets have fully validated the superiority of the proposed approach, in comparison to several state-of-the-art methods and deep learning models.

Despite of various approaches proposed to smooth the hyperspectral images (HSIs) before feature extraction, the efficacy is still affected by the noise, even using the corrected dataset with the noisy and water absorption bands discarded. In this study, a novel spectral-spatial feature mining framework, Multiscale Superpixelwise Prophet Model (MSPM), is proposed for noise-robust feature extraction and effective classification of the HSI. The prophet model is highly noise-robust for deeply digging into the complex structured features thus enlarging interclass diversity and improving intraclass similarity. First, the superpixelwise segmentation is produced from the first three principal components of an HSI to group pixels into regions with adaptively determined sizes and shapes. A multiscale prophet model is utilized to extract the multiscale informative trend components from the average spectrum of each superpixel. Taking the multiscale trend signal as the input feature, the HSI data are classified superpixelwisely, which is further refined by a majority vote based decision fusion. Comprehensive experiments on three publicly available datasets have fully validated the efficacy and robustness of our MSPM model when benchmarked with eleven state-of-the-art algorithms, including six spectral-spatial methods and five deep learning ones. Besides, MSPM also shows superiority under limited training samples, due to the combined strategies of superpixelwise fusion and multiscale fusion. Our model has provided a useful solution for noise-robust feature extraction as it achieves superior HSI classification even from the uncorrected dataset without prefiltering the water absorption and noisy bands.

Summary The information propagation mechanism in graph‐structured networks such as social networks is the foundation of network security. The graph convolutional network (GCN) is a powerful approach for semisupervised node classification on graph‐structure data. The vertex features which pass through the graph network are affected by the k‐hop neighborhood vertices. However, current high‐order GCN approaches merged the k‐hop neighborhood using coarse pooling and complicated weight parameters. To reduce the computational complexity and preserve topological of the graph data, with weight sharing mechanism we propose a novel GCN based on a novel higher‐order pooling layer for semisupervised classification. The proposed model and its variants are experimental studied on several large‐scale citation network datasets using semisupervised learning. The experimental results show that the proposed model and its variants have lower computational complexity and achieve the state‐of‐the‐art in the node classification accuracy.

Singular spectral analysis (SSA) has recently been successfully applied to feature extraction in hyperspectral image (HSI), including conventional (1-D) SSA in spectral domain and 2-D SSA in spatial domain. However, there are some drawbacks, such as sensitivity to the window size, high computational complexity under a large window, and failing to extract joint spectral-spatial features. To tackle these issues, in this article, we propose superpixelwise adaptive SSA (SpaSSA), that is superpixelwise adaptive SSA for exploiting local spatial information of HSI. The extraction of local (instead of global) features, particularly in HSI, can be more effective for characterizing the objects within an image. In SpaSSA, conventional SSA and 2-D SSA are combined and adaptively applied to each superpixel derived from an oversegmented HSI. According to the size of the derived superpixels, either SSA or 2-D singular spectrum analysis (2D-SSA) is adaptively applied for feature extraction, where the embedding window in 2D-SSA is also adaptive to the size of the superpixel. Experimental results on the three datasets have shown that the proposed SpaSSA outperforms both SSA and 2D-SSA in terms of classification accuracy and computational complexity. By combining SpaSSA with the principal component analysis (SpaSSA-PCA), the accuracy of land-cover analysis can be further improved, outperforming several state-of-the-art approaches.

Band selection has become a significant issue for the efficiency of the hyperspectral image (HSI) processing. Although many unsupervised band selection (UBS) approaches have been developed in the last decades, a flexible and robust method is still lacking. The lack of proper understanding of the HSI data structure has resulted in the inconsistency in the outcome of UBS. Besides, most of the UBS methods are either relying on complicated measurements or rather noise sensitive, which hinder the efficiency of the determined band subset. In this article, an adaptive distance-based band hierarchy (ADBH) clustering framework is proposed for UBS in HSI, which can help to avoid the noisy bands while reflecting the hierarchical data structure of HSI. With a tree hierarchy-based framework, we can acquire any number of band subset. By introducing a novel adaptive distance into the hierarchy, the similarity between bands and band groups can be computed straightforward while reducing the effect of noisy bands. Experiments on four datasets acquired from two HSI systems have fully validated the superiority of the proposed framework.

As an important topic in hyperspectral image (HSI) analysis, band selection has attracted increasing attention in the last two decades for dimensionality reduction in HSI. With the great success of deep learning (DL)-based models recently, a robust unsupervised band selection (UBS) neural network is highly desired, particularly due to the lack of sufficient ground truth information to train the DL networks. Existing DL models for band selection either depend on the class label information or have unstable results via ranking the learned weights. To tackle these challenging issues, in this article, we propose a Gumbel-Softmax (GS) trick enabled concrete autoencoder-based UBS framework (CAE-UBS) for HSI, in which the learning process is featured by the introduced concrete random variables and the reconstruction loss. By searching from the generated potential band selection candidates from the concrete encoder, the optimal band subset can be selected based on an information entropy (IE) criterion. The idea of the CAE-UBS is quite straightforward, which does not rely on any complicated strategies or metrics. The robust performance on four publicly available datasets has validated the superiority of our CAE-UBS framework in the classification of the HSIs.

The pandemic of COVID-19 has become a global crisis in public health, which has led to a massive number of deaths and severe economic degradation. To suppress the spread of COVID-19, accurate diagnosis at an early stage is crucial. As the popularly used real-time reverse transcriptase polymerase chain reaction (RT-PCR) swab test can be lengthy and inaccurate, chest screening with radiography imaging is still preferred. However, due to limited image data and the difficulty of the early-stage diagnosis, existing models suffer from ineffective feature extraction and poor network convergence and optimisation. To tackle these issues, a segmentation-based COVID-19 classification network, namely SC2Net, is proposed for effective detection of the COVID-19 from chest x-ray (CXR) images. The SC2Net consists of two subnets: a COVID-19 lung segmentation network (CLSeg), and a spatial attention network (SANet). In order to supress the interference from the background, the CLSeg is first applied to segment the lung region from the CXR. The segmented lung region is then fed to the SANet for classification and diagnosis of the COVID-19. As a shallow yet effective classifier, SANet takes the ResNet-18 as the feature extractor and enhances high-level feature via the proposed spatial attention module. For performance evaluation, the COVIDGR 1.0 dataset is used, which is a high-quality dataset with various severity levels of the COVID-19. Experimental results have shown that, our SC2Net has an average accuracy of 84.23% and an average F1 score of 81.31% in detection of COVID-19, outperforming several state-of-the-art approaches.

Abstract SiamPRN algorithm performs well in visual tracking, but it is easy to drift under occlusion and fast motion scenes because it uses $$\ell _1$$ ℓ 1 -smooth loss function to measure the regression location of bounding box. In this paper, we propose a multivariate intersection over union (MIOU) loss in SiamRPN tracking framework. Firstly, MIOU loss includes three geometric factors in regression: the overlap area ratio, the center distance ratio, and the aspect ratio, which can better reflect the coincidence degree of target box and prediction box. Secondly, we improve the definition of aspect ratio loss to avoid gradient explosion, improve the optimization performance of prediction box. Finally, based on SiamPRN tracker, we compared the tracking performance of $$\ell _1$$ ℓ 1 -smooth loss, IOU loss, GIOU loss, DIOU loss, and MIOU loss. Experimental results show that the MIOU loss has better target location regression than other loss functions on the OTB2015 and VOT2016 benchmark, especially for the challenges of occlusion, illumination change and fast motion.

Abstract With the development of 5G technology, the short delay requirements of commercialization and large amounts of data change our lifestyle day-to-day. In this background, this paper proposes a fast blind deblurring algorithm for QR code images, which mainly achieves the effect of adaptive scale control by introducing an evaluation mechanism. Its main purpose is to solve the out-of-focus caused by lens shake, inaccurate focus, and optical noise by speeding up the latent image estimation in the process of multi-scale division iterative deblurring. The algorithm optimizes productivity under the guidance of collaborative computing, based on the characteristics of the QR codes, such as the features of gradient and strength. In the evaluation step, the Tenengrad method is used to evaluate the image quality, and the evaluation value is compared with the empirical value obtained from the experimental data. Combining with the error correction capability, the recognizable QR codes will be output. In addition, we introduced a scale control parameter to study the relationship between the recognition rate and restoration time. Theoretical analysis and experimental results show that the proposed algorithm has high recovery efficiency and well recovery effect, can be effectively applied in industrial applications.

Effective compression of hyperspectral (HS) images is essential due to their large data volume. Since these images are high dimensional, processing them is also another challenging issue. In this work, an efficient lossy HS image compression method based on enhanced multivariance products representation (EMPR) is proposed. As an efficient data decomposition method, EMPR enables us to represent the given multidimensional data with lower-dimensional entities. EMPR, as a finite expansion with relevant approximations, can be acquired by truncating this expansion at certain levels. Thus, EMPR can be utilized as a highly effective lossy compression algorithm for hyper spectral images. In addition to these, an efficient variety of EMPR is also introduced in this article, in order to increase the compression efficiency. The results are benchmarked with several state-of-the-art lossy compression methods. It is observed that both higher peak signal-to-noise ratio values and improved classification accuracy are achieved from EMPR-based methods.

Effective acquisition, analysis and reconstruction of multi‐modal data such as colour and multi‐/hyper‐spectral imagery is crucial in smart camera applications, where wavelet‐based coding and compression of images are highly demanded. Many existing discrete wavelet filtering banks have fixed coefficients hence their performance is highly dependent on the signal/image being processed. To tackle this problem, a unified framework is proposed in this study, which can produce a series of discrete wavelet filtering banks, where many existing discrete wavelet filtering banks become special cases of the framework. For each generated filtering bank, it consists of two decomposition filters and two reconstruction filters through an optimisation process. The efficacy of the filtering banks produced by the framework has been validated in two case studies, including colour image decomposition and reconstruction, and hyperspectral image classification. Comprehensive experiments have demonstrated the superior performance of the proposed framework, which will benefit the efficacy of smart camera and camera network applications.

The novel coronavirus disease 2019 (COVID-19) pandemic has led to a worldwide crisis in public health. It is crucial we understand the epidemiological trends and impact of non-pharmacological interventions (NPIs), such as lockdowns for effective management of the disease and control of its spread. We develop and validate a novel intelligent computational model to predict epidemiological trends of COVID-19, with the model parameters enabling an evaluation of the impact of NPIs. By representing the number of daily confirmed cases (NDCC) as a time-series, we assume that, with or without NPIs, the pattern of the pandemic satisfies a series of Gaussian distributions according to the central limit theorem. The underlying pandemic trend is first extracted using a singular spectral analysis (SSA) technique, which decomposes the NDCC time series into the sum of a small number of independent and interpretable components such as a slow varying trend, oscillatory components and structureless noise. We then use a mixture of Gaussian fitting (GF) to derive a novel predictive model for the SSA extracted NDCC incidence trend, with the overall model termed SSA-GF. Our proposed model is shown to accurately predict the NDCC trend, peak daily cases, the length of the pandemic period, the total confirmed cases and the associated dates of the turning points on the cumulated NDCC curve. Further, the three key model parameters, specifically, the amplitude (alpha), mean (mu), and standard deviation (sigma) are linked to the underlying pandemic patterns, and enable a directly interpretable evaluation of the impact of NPIs, such as strict lockdowns and travel restrictions. The predictive model is validated using available data from China and South Korea, and new predictions are made, partially requiring future validation, for the cases of Italy, Spain, the UK and the USA. Comparative results demonstrate that the introduction of consistent control measures across countries can lead to development of similar parametric models, reflected in particular by relative variations in their underlying sigma, alpha and mu values. The paper concludes with a number of open questions and outlines future research directions.

Although extreme learning machine (ELM) has successfully been applied to a number of pattern recognition problems, only with the original ELM it can hardly yield high accuracy for the classification of hyperspectral images (HSIs) due to two main drawbacks. The first is due to the randomly generated initial weights and bias, which cannot guarantee optimal output of ELM. The second is the lack of spatial information in the classifier as the conventional ELM only utilizes spectral information for classification of HSI. To tackle these two problems, a new framework for ELM-based spectral-spatial classification of HSI is proposed, where probabilistic modeling with sparse representation and weighted composite features (WCFs) is employed to derive the optimized output weights and extract spatial features. First, ELM is represented as a concave logarithmic-likelihood function under statistical modeling using the maximum a posteriori estimator. Second, sparse representation is applied to the Laplacian prior to efficiently determine a logarithmic posterior with a unique maximum in order to solve the ill-posed problem of ELM. The variable splitting and the augmented Lagrangian are subsequently used to further reduce the computation complexity of the proposed algorithm. Third, the spatial information is extracted using the WCFs to construct the spectral-spatial classification framework. In addition, the lower bound of the proposed method is derived by a rigorous mathematical proof. Experimental results on three publicly available HSI data sets demonstrate that the proposed methodology outperforms ELM and also a number of state-of-the-art approaches.

Hyperspectral imaging (HSI) classification has become a popular research topic in recent years, and effective feature extraction is an important step before the classification task. Traditionally, spectral feature extraction techniques are applied to the HSI data cube directly. This paper presents a novel algorithm for HSI feature extraction by exploiting the curvelet-transformed domain via a relatively new spectral feature processing technique—singular spectrum analysis (SSA). Although the wavelet transform has been widely applied for HSI data analysis, the curvelet transform is employed in this paper since it is able to separate image geometric details and background noise effectively. Using the support vector machine classifier, experimental results have shown that features extracted by SSA on curvelet coefficients have better performance in terms of classification accuracy over features extracted on wavelet coefficients. Since the proposed approach mainly relies on SSA for feature extraction on the spectral dimension, it actually belongs to the spectral feature extraction category. Therefore, the proposed method has also been compared with some state-of-the-art spectral feature extraction techniques to show its efficacy. In addition, it has been proven that the proposed method is able to remove the undesirable artifacts introduced during the data acquisition process. By adding an extra spatial postprocessing step to the classified map achieved using the proposed approach, we have shown that the classification performance is comparable with several recent spectral–spatial classification methods.

Feature extraction is of high importance for effective data classification in hyperspectral imaging (HSI). Considering the high correlation among band images, spectral-domain feature extraction is widely employed. For effective spatial information extraction, a 2-D extension to singular spectrum analysis (2D-SSA), which is a recent technique for generic data mining and temporal signal analysis, is proposed. With 2D-SSA applied to HSI, each band image is decomposed into varying trends, oscillations, and noise. Using the trend and the selected oscillations as features, the reconstructed signal, with noise highly suppressed, becomes more robust and effective for data classification. Three publicly available data sets for HSI remote sensing data classification are used in our experiments. Comprehensive results using a support vector machine classifier have quantitatively evaluated the efficacy of the proposed approach. Benchmarked with several state-of-the-art methods including 2-D empirical mode decomposition (2D-EMD), it is found that our proposed 2D-SSA approach generates the best results in most cases. Unlike 2D-EMD that requires sequential transforms to obtain detailed decomposition, 2D-SSA extracts all components simultaneously. As a result, the execution time in feature extraction can be also dramatically reduced. The superiority in terms of enhanced discrimination ability from 2D-SSA is further validated when a relatively weak classifier, i.e., the k-nearest neighbor, is used for data classification. In addition, the combination of 2D-SSA with 1-D principal component analysis (2D-SSA-PCA) has generated the best results among several other approaches, demonstrating the great potential in combining 2D-SSA with other approaches for effective spatial-spectral feature extraction and dimension reduction in HSI.