Underwater image quality is seriously degraded due to the insufficient light in water. Although artificial illumination can assist imaging, it often brings non-uniform illumination phenomenon. To this end, we develop an illumination channel sparsity prior (ICSP) guided variational framework for non-uniform illumination underwater image restoration. Technically, the illumination channel sparsity prior is built on the observation that the illumination channel of a uniform-light underwater image in HSI color space contains few pixels whose intensity is very low. Then according to the Retinex theory, we design a variational model with L0 norm term, constraint term, and gradient term, by integrating the proposed ICSP into an extended underwater image formation model. Such three regularizations are effective in enhancing the brightness, correcting color distortion, and revealing structures and fine-scale details. Meanwhile, we exploit a fast numerical algorithm on the base of the alternating direction method of multipliers (ADMM) to accelerate solving this optimization problem. We also collect a benchmark dataset, namely NUID that contains 925 real underwater images of different non-uniform illumination. Extensive experiments demonstrate that our proposed method is effective in terms of qualitative and quantitative comparisons, ablation studies, convergence analysis, and applications. The code and dataset are available athttps://github.com/Hou-Guojia/ICSP.

Estimating depth from a single underwater image is one of the main tasks of underwater visual perception. However, data-driven underwater depth estimation methods have long been challenging to make breakthroughs due to the difficulty of obtaining a large number of true-value references. This is partly due to the high cost of acquisition equipment, which is difficult to be applied to diverse ocean scenes by a wide range of users, and therefore sample diversity is difficult to guarantee; on the other hand, manual annotation of dense depth relationships is almost impossible to achieve. In this paper, we establish a new underwater relative depth estimation benchmark, namely SUIM-SDA, by extending the SUIM dataset with more than 6,000 manually annotated depth trendlines, 25 million pixels with paired depth-ranking labels and 14 million depth-ranked pixel pairs. Using the sparse depth relation annotation provided by SUIM-SDA and the semantic information provided by SUIM, we design a new multi-stage multi-task learning framework to predict a dense relative depth map for a single underwater image. Comprehensive comparison and ablation study on the publicly available dataset and our new benchmark demonstrate the effectiveness of the proposed weakly-supervised strategy for dense relative depth estimation. The new benchmark, source code, and trained models are available on the project home page: https://wangxy97.github.io/WsUIDNet.

Efficient eddy trajectory prediction driven by multi-information fusion can facilitate the scientific research of oceanography, while the complicated dynamics mechanism makes this issue challenging. Benefiting from ocean observing technology, the eddy trajectory dataset can be qualified for data-intensive research paradigms. In this paper, the dynamics mechanism is used to inspire the design idea of the eddy trajectory prediction neural network (termed EddyTPNet) and is also transformed into prior knowledge to guide the learning process. This study is among the first to implement eddy trajectory prediction with physics informed neural network. First, an in-depth analysis of the kinematic characteristics indicates that the longitude and latitude of the trajectory should be decoupled; Second, the directional dispersion prior knowledge of global eddy propagation is embedded into the decoder of the EddyTPNet to improve the performance; Finally, EddyTPNet predicts global eddy trajectories through pre-training and adapts to complex local regions via model transfer. Extensive experimental results demonstrate that EddyTPNet can reliably forecast the motion of eddies for the next 7 days, ensuring a low daily mean geodetic error. This exploratory study provides valuable insights into solving the prediction problem of ocean phenomena by using knowledge-based time series neural networks.

Wheat variety identification from hyperspectral images holds significant importance in both fine breeding and intelligent agriculture. However, the discriminatory accuracy of some techniques is limited due to insufficient datasets, data redundancy, and noise interference. To address these issues, we propose a wheat variety identification framework called generate adversarial-driven cross-aware network (GACNet), comprising a semi-supervised generative adversarial network (GAN) for data augmentation and a cross-aware attention network (CAANet) for variety identification. First, the semi-supervised GAN (SSGAN) alleviates data scarcity by generating fake hyperspectral images as realistically as possible through learning the distribution hypothesis of real hyperspectral images, while the discriminator distinguishes between real and fake hyperspectral images. Subsequently, the CAANet is employed for wheat variety identification, which leverages a cascading cross-learning of 3-D and 2-D convolutions to fully exploit spectral, spatial, and texture features and refines the features through an embedded attention mechanism in the cross-convolutional module. Additionally, we constructed a hyperspectral wheat variety dataset (HWVD) comprising 4560 samples of 19 categories. Extensive experiments on our dataset demonstrate that our GACNet outperforms state-of-the-art methods for wheat variety identification. The HWVD will be made available.

Achieving subjective and objective quality assessment of underwater images is of high significance in underwater visual perception and image/video processing. However, the development of underwater image quality assessment (UIQA) is limited for the lack of publicly available underwater image datasets with human subjective scores and reliable objective UIQA metrics. To address this issue, we establish a large-scale underwater image dataset, dubbed UID2021, for evaluating no-reference (NR) UIQA metrics. The constructed dataset contains 60 multiply degraded underwater images collected from various sources, covering six common underwater scenes (i.e., bluish scene, blue-green scene, greenish scene, hazy scene, low-light scene, and turbid scene), and their corresponding 900 quality improved versions are generated by employing 15 state-of-the-art underwater image enhancement and restoration algorithms. Mean opinion scores with 52 observers for each image of UID2021 are also obtained by using the pairwise comparison sorting method. Both in-air and underwater-specific NR IQA algorithms are tested on our constructed dataset to fairly compare their performance and analyze their strengths and weaknesses. Our proposed UID2021 dataset enables ones to evaluate NR UIQA algorithms comprehensively and paves the way for further research on UIQA. The dataset is available at https://github.com/Hou-Guojia/UID2021 .

Underwater captured images are usually degraded by low contrast, hazy, and blurry due to absorbing and scattering, which limits their analyses and applications. To address these problems, a red channel prior guided variational framework is proposed based on the complete underwater image formation model (UIFM). Unlike most of the existing methods that only consider the direct transmission and backscattering components, we additionally include forward scattering component into the UIFM. In the proposed variational framework, we successfully incorporate the normalized total variation item and sparse prior knowledge of blur kernel together. In addition, we perform the estimation of blur kernel by varying image resolution in a coarse-to-fine manner to avoid local minima. Moreover, for solving the generated non-smooth optimization problem, we employ the alternating direction method of multipliers (ADMM) to accelerate the whole progress. Experimental results demonstrate that the proposed method has a good performance on dehazing and deblurring. Extensive qualitative and quantitative comparisons further validate its superiority against the other state-of-the-art algorithms. The code is available online at:https://github.com/Hou-Guojia/UNTV

Due to the wavelength-dependent light attenuation, refraction and scattering, underwater images usually suffer from color distortion and blurred details. However, due to the limited number of paired underwater images with undistorted images as reference, training deep enhancement models for diverse degradation types is quite difficult. To boost the performance of data-driven approaches, it is essential to establish more effective learning mechanisms that mine richer supervised information from limited training sample resources. In this paper, we propose a novel underwater image enhancement network, called SGUIE-Net, in which we introduce semantic information as high-level guidance via region-wise enhancement feature learning. Accordingly, we propose semantic region-wise enhancement module to better learn local enhancement features for semantic regions with multi-scale perception. After using them as complementary features and feeding them to the main branch, which extracts the global enhancement features on the original image scale, the fused features bring semantically consistent and visually superior enhancements. Extensive experiments on the publicly available datasets and our proposed dataset demonstrate the impressive performance of SGUIE-Net. The code and proposed dataset are available at https://trentqq.github.io/SGUIE-Net.html.

Image regularised features play a critical role in image processing domain, by integrating regularised feature and structural similarity, a new full‐reference image assessment method (IRF_SSIM) is proposed in this study. As well known, the gradient operator always be used to capture the edge information of the image, while the total variational regularised features can be adopted to calculate the detailed change information of image contrast and texture, as well as noise removal and edge retention. Therefore, the IRF_SSIM method extends the gradient features into the image regularised features to measure the structural changes in the image. In addition, image quality is also affected by variations of luminance and contrast. For a more comprehensive image quality assessment, the IRF_SSIM method considers the changes in structure, luminance and contrast simultaneously. In other words, the total image quality is estimated by structural similarity calculated by integrating the effects of image structure, luminance and contrast changes. Comparing with the representative methods, the experimental results illustrate that the IRF_SSIM method is highly consistent with the subjective assessment results.

Single image dehazing and denoising models can simultaneously remove haze and noise with high efficiency. Here, the authors propose three variational models combining the celebrated dark channel prior (DCP) and total variations (TV) models for image dehazing and denoising. The authors firstly estimate the transmission map associated with depth using DCP, then design three variational models for colour image dehazing and denoising based on this estimation and the layered total variation (LTV) regulariser, multichannel total variation (MTV) regulariser, and colour total variation (CTV) regulariser, respectively. In order to improve the computation efficiency of the three models, the authors design their fast split Bregman algorithms via introducing some auxiliary variables and the Bregman iterative parameters. Numerous experiments are presented to compare their denoising effects, edge‐preserving properties, and computation efficiencies. To demonstrate the merits of the proposed models, the authors also conduct some comparisons with several existing state‐of‐the‐art methods. Numerical results further prove that the LTV‐based model is fastest, and the CTV model is the best for denoising with edge‐preserving, and it also leads to the best visually haze‐free and noise‐free images.

The multiplicative noise removal problem has received considerable attention recently. To solve this problem, various variational models have been proposed, which minimise an energy functional composed of the data term and the regularisation term. Regarding the regularisation term, a first‐order model is frequently used to remove multiplicative noise, which may cause staircase effect and loss of contrast in the output image. In this study, the authors use a second‐order model, the total curvature (TC), to solve the above problem. The TC model has the benefit of removing the staircase effect and maintaining image edges, contrasts and corners. The augmented Lagrange method is utilised to solve the proposed TC model by introducing auxiliary variables, Lagrange multipliers and using alternating optimisation strategy. In each loop of optimisation, the fast Fourier transform, generalised soft threshold formulas, projection method and gradient descent method are integrated effectively. The experimental results show that the TC model can effectively remove staircase effect and preserve smoothness, via comparison with the first‐order model (total variation regularisation and Perona–Malik regularisation). Furthermore, the TC model is better than another second‐order model based on bounded Hessian regularisation in preserving contrast and corner.

In this study, a novel underwater colour image enhancement approach based on hue preserving is presented by combining hue–saturation–intensity (HSI) and HS–value (HSV) colour models. In this study, the proposed wavelet‐domain filtering (WDF) and constrained histogram stretching (CHS) algorithms are operated on HSI and HSV colour models, respectively. The degraded image is first converted from red–green–blue colour model into the HSI colour model, wherein the hue component H is preserved and WDF algorithm is executed on the S and I components. Similarly, the image is further converted into the HSV colour model, wherein H component is kept invariant as well and CHS algorithm is applied on the S and V components. The authors' key contribution is that the H preserving method can improve image quality in terms of contrast, colour rendition, non‐uniform illumination, and denoising. In addition, experimental results show that the proposed approach outperforms several other state‐of‐the‐art algorithms.