Adding imperceptible watermarks to artwork images, such as paintings and photographs, can effectively safeguard the copyright of these images without compromising their usability. However, existing blind watermarking techniques encounter two major challenges in addressing this task: imperceptibility and robustness, particularly when subjected to various noise attacks. In this paper, we propose a blind watermarking method for artwork image copyright protection, IWRN, which can ensure both the Imperceptibility of the Watermark and Robustness against Noise attacks. For imperceptibility, we design a Learnable Wavelet Network (LWN) to adaptively embed the watermark into the high-frequency region where the watermark has better invisibility. For robustness, we establish a Deform-Attention based Invertible Neural Network (DA-INN) with a decoding optimization, which offers the advantage of computational reversion, and combines the deform-attention mechanism and decoding optimization to enhance the model's resistance against noises. Additionally, we design a Joint Contrast Learning (JCL) mechanism to improve imperceptibility and robustness simultaneously. Experiments show that our IWRN outperforms other state-of-the-art blind watermarking methods, achieves an average performance of 41.55 PSNR and 99.57% accuracy on the Coco2017, Wikiart, and Div2k datasets when facing 12 kinds of noise attacks.

The white balance methods for sRGB images (sRGB-WB) aim to directly remove their color temperature shifts. Despite achieving promising white balance (WB) performance, the existing methods suffer from WB instability, i.e., their results are inconsistent for images with different color temperatures. We propose a stable white balance network (SWBNet) to alleviate this problem. It learns the color temperature-insensitive features to generate white-balanced images, resulting in consistent WB results. Specifically, the color temperatureinsensitive features are learned by implicitly suppressing lowfrequency information sensitive to color temperatures. Then, a color temperature contrastive loss is introduced to facilitate the most information shared among features of the same scene and different color temperatures. This way, features from the same scene are more insensitive to color temperatures regardless of the inputs. We also present a color temperature sensitivity-oriented transformer that globally perceives multiple color temperature shifts within an image and corrects them by different weights. It helps to improve the accuracy of stabilized SWBNet, especially for multiillumination sRGB images. Experiments indicate that our SWBNet achieves stable and remarkable WB performance.

The performance of nighttime semantic segmentation has been significantly improved thanks to recent unsupervised methods. However, these methods still suffer from complex domain gaps, i.e., the challenging illumination gap and the inherent dataset gap. In this paper, we propose the illumination-coupled domain adaptation framework(ICDA) to effectively avoid the illumination gap and mitigate the dataset gap by coupling daytime and nighttime images as a whole with semantic relevance. Specifically, we first design a new composite enhancement method(CEM) that considers not only illumination but also spatial consistency to construct the source and target domain pairs, which provides the basic adaptation unit for our ICDA. Next, to avoid the illumination gap, we devise the Deformable Attention Relevance(DAR) module to capture the semantic relevance inside each domain pair, which can couple the daytime and nighttime images at the feature level and adaptively guide the predictions of nighttime images. Besides, to mitigate the dataset gap and acquire domain-invariant semantic relevance, we propose the Prototype-based Class Alignment(PCA) module, which improves the usage of category information and performs fine-grained alignment. Extensive experiments show that our method reduces the complex domain gaps and achieves state-of-the-art performance for nighttime semantic segmentation. Our code is available at https://github.com/chenghaoDong666/ICDA.

The sRGB white balance methods aim to correct the nonlinear color cast of sRGB images without accessing raw values. Although existing methods have achieved increasingly better results, their generalization to sRGB images from multiple cameras is still under explored. In this paper, we propose the network named WBFlow that not only performs superior white balance for sRGB images but also generalizes well to multiple cameras. Specifically, we take advantage of neural flow to ensure the reversibility of WBFlow, which enables lossless rendering of color cast sRGB images back to pseudo raw features for linear white balancing and thus achieves superior performance. Furthermore, inspired by camera transformation approaches, we have designed a camera transformation (CT) in pseudo raw feature space to generalize WBFlow for different cameras via few shot learning. By utilizing a few sRGB images from an untrained camera, our WBFlow can perform well on this camera by learning the camera specific parameters of CT. Extensive experiments show that WBFlow achieves superior camera generalization and accuracy on three public datasets as well as our rendered multiple camera sRGB dataset. Our code is available at https://github.com/ChunxiaoLe/WBFlow.

Superpixel is the over-segmentation region of an image, whose basic units "pixels" have similar properties. Although many popular seeds-based algorithms have been proposed to improve the segmentation quality of superpixels, they still suffer from the seeds initialization problem and the pixel assignment problem. In this paper, we propose Vine Spread for Superpixel Segmentation (VSSS) to form superpixel with high quality. First, we extract image color and gradient features to define the soil model that establishes a "soil" environment for vine, and then we define the vine state model by simulating the vine "physiological" state. Thereafter, to catch more image details and twigs of the object, we propose a new seeds initialization strategy that perceives image gradients at the pixel-level and without randomness. Next, to balance the boundary adherence and the regularity of the superpixel, we define a three-stage "parallel spreading" vine spread process as a novel pixel assignment scheme, in which the proposed nonlinear velocity for vines helps to form the superpixel with regular shape and homogeneity, the crazy spreading mode for vines and the soil averaging strategy help to enhance the boundary adherence of superpixel. Finally, a series of experimental results demonstrate that our VSSS offers competitive performance in the seed-based methods, especially in catching object details and twigs, balancing boundary adherence and obtaining regular shape superpixels.

Color Constancy aims to correct image color casts caused by scene illumination. Recently, although the deep learning approaches have remarkably improved on single-camera data, these models still suffer from the seriously insufficient data problem, resulting in shallow model capacity and degradation in multi-camera settings. In this paper, to alleviate this problem, we present a Transfer Learning Color Constancy (TLCC) method that leverages cross-camera RAW data and massive unlabeled sRGB data to support training. Specifically, TLCC consists of the Statistic Estimation Scheme (SE-Scheme) and Color-Guided Adaption Branch (CGA-Branch). SE-Scheme builds a statistic perspective to map the camera-related illumination labels into camera-agnostic form and produce pseudo labels for sRGB data, which greatly expands data for joint training. Then, CGA-Branch further promotes efficient transfer learning from sRGB to RAW data by extracting color information to regularize the backbone's features adaptively. Experimental results show the TLCC has overcome the data limitation and model degradation, outperforming the state-of-the-art performance on popular benchmarks. Moreover, the experiments also prove the TLCC is capable of learning new scenes information from sRGB data to improve accuracy on the RAW images with similar scenes.

Color Constancy aims to eliminate the color cast of RAW images caused by non-neutral illuminants. Though contemporary approaches based on convolutional neural networks significantly improve illuminant estimation, they suffer from the seriously insufficient data problem. To solve this problem by effectively utilizing multi-domain data, we propose the Domain Adversarial Learning Color Constancy (DALCC) which consists of the Domain Adversarial Learning Branch (DALB) and the Feature Reweighting Module (FRM). In DALB, the Camera Domain Classifier and the feature extractor compete against each other in an adversarial way to encourage the emergence of domain-invariant features. At the same time, the Illuminant Transformation Module performs color space conversion to solve the inconsistent color space problem caused by those domain-invariant features. They collaboratively avoid model degradation of multi-device training caused by the domain discrepancy of feature distribution, which enables our DALCC to benefit from multi-domain data. Besides, to better utilize multi-domain data, we propose the FRM that reweights the feature map to suppress Non-Primary Illuminant regions, which reduces the influence of misleading illuminant information. Experiments show that the proposed DALCC can more effectively take advantage of multi-domain data and thus achieve state-of-the-art performance on commonly used benchmark datasets.

The Normalized Cut (NCut) model is a popular graph-based model for image segmentation. But it suffers from the excessive normalization problem and weakens the small object and twig segmentation. In this paper, we propose an Explored Normalized Cut (ENCut) model that establishes a balance graph model by adopting a meaningful-loop and a k-step random walk, which reduces the energy of small salient region, so as to enhance the small object segmentation. To improve the twig segmentation, our ENCut model is further enhanced by a new Random Walk Refining Term (RWRT) that adds local attention to our model with the help of an un-supervising random walk. Finally, a move-making based strategy is developed to efficiently solve the ENCut model with RWRT. Experiments on three standard datasets indicate that our model can achieve state-of-the-art results among the NCut-based segmentation models.

Retrieving occlusion relation among objects in a single image is challenging due to sparsity of boundaries in image. We observe two key issues in existing works: firstly, lack of an architecture which can exploit the limited amount of coupling in the decoder stage between the two subtasks, namely occlusion boundary extraction and occlusion orientation prediction, and secondly, improper representation of occlusion orientation. In this paper, we propose a novel architecture called Occlusion-shared and Path-separated Network (OPNet), which solves the first issue by exploiting rich occlusion cues in shared high-level features and structured spatial information in task-specific low-level features. We then design a simple but effective orthogonal occlusion representation (OOR) to tackle the second issue. Our method surpasses the state-of-the-art methods by 6.1%/8.3% Boundary-AP and 6.5%/10% Orientation-AP on standard PIOD/BSDS ownership datasets. Code is available at https://github.com/fengpanhe/MT-ORL.

The nonlocal-based blocks are designed for capturing long-range spatial-temporal dependencies in computer vision tasks. Although having shown excellent performance, they still lack the mechanism to encode the rich, structured information among elements in an image or video. In this paper, to theoretically analyze the property of these nonlocal-based blocks, we provide a new perspective to interpret them, where we view them as a set of graph filters generated on a fully-connected graph. Specifically, when choosing the Chebyshev graph filter, a unified formulation can be derived for explaining and analyzing the existing nonlocal-based blocks (e.g., nonlocal block, nonlocal stage, double attention block). Furthermore, by concerning the property of spectral, we propose an efficient and robust spectral nonlocal block, which can be more robust and flexible to catch long-range dependencies when inserted into deep neural networks than the existing nonlocal blocks. Experimental results demonstrate the clear-cut improvements and practical applicabilities of our method on image classification, action recognition, semantic segmentation, and person re-identification tasks. Code are available at https://github.com/zh460045050/SNL_ICCV2021.

In this paper, we propose a deep learning-based su-perpixel segmentation algorithm for RGB-D image. The proposed deep neural network called BP-net is composed of boundary detection network (B-net) that exploits multiscale information from depth image to extract the geometry edge of objects, and pixel labeling network (P-net) that extracts pixel features and generates superpixels. A boundary pass filter is proposed to combine the edge information and pixel features and ensures superpixels adhere better to geometry edges. To generate regular superpixels, we design a loss function which takes the shape regularity error and superpixel accuracy into account. In addition, for providing reasonable initial seeds, a new seeds initialization strategy is proposed, in which the density of seeds is investigated from a 2-manifolds space to reduce the number of superpixels that cover multiple objects in the region of rich texture. Experimental results demonstrate that our algorithm outperforms the existing state-of-the-art algorithms in terms of accuracy and shape regularity on the RGB-D dataset.

This letter proposes new estimations of fractional power spectral density (FrPSD) and fractional correlation function (FrCF) for nonuniform sampling of random signals with non-stationarity and limited bandwidths in the fractional Fourier domain. Unlike previous works, the developed FrPSD and FrCF estimations are capable of dealing with unknown sampling instants. In order to obtain them, we first formulate approximations of FrCF and FrPSD making use of uniform sampling instants. Then we convert the approximate FrPSD to a fractional filtered version of the FrPSD for the original random signal, which does not rely on the sampling instants. With such operations, we propose the FrPSD estimation to cancel the bias of FrPSD approximation by means of a fractional inverse filtering and thereby obtain a high accuracy of it. The FrCF estimation is proposed to be the inverse fractional Fourier transform of the FrPSD, and it serves as the fractional interpolation of the previously obtained approximation of the FrCF. Simulation results show the effectiveness of the proposed estimation methods.

In this paper, we propose a novel random walk model, called Dynamic Random Walk (DRW), which adds a new type of dynamic node to the original RW model and reduces redundant calculation by limiting the walk range. To solve the seed-lacking problem of the proposed DRW, we redefine the energy function of the original RW and use the first arrival probability among each node pair to avoid the interference for each partition. Relaxation of our DRW is performed with the help of a greedy strategy and the Weighted Random Walk Entropy(WRWE) that uses the gradient feature to approximate the stationary distribution. The proposed DRW not only can enhance the boundary adherence but also can run with linear time complexity. To extend our DRW for superpixel segmentation, a seed initialization strategy is proposed. It can evenly distribute seeds in both 2D and 3D space and generate superpixels in only one iteration. The experimental results demonstrate that our DRW is faster than existing RW models and better than the state-of-the-art superpixel segmentation algorithms with respect to both efficiency and segmentation effects.

In this paper, we propose a patch-wise super-resolution (SR) method that combines an external-sample classification tree and a nonlinear-mapping learning stage to simultaneously guarantees reconstruction quality and speed at the stage of patch representation and mapping. We use the low-resolution (LR) to high-resolution (HR) mapping kernel of each patch-pair sample (called SIMK) to complete classification by binary tree branching and provide reasonable training sets for mapping-learning. Then a high accuracy but low cost lightweight network is learned for each tree node to choose the reasonable branch path for the testing LR patches. In the mapping-learning stage, the nonlinear mapping for each class is represented as a full-connected network, which provides satisfying generalization ability for LR patch reconstruction. Comparing with state-of-the-art methods, our approach achieves real-time (>24fps) SR of realistic vision and high quality for different upscaling factors.

Recent advances in compressive spectral imagers have demonstrated the potential of spatio-spectral modulation (SSM) for improved reconstruction performance. Existing SSM techniques, however, use either a color filter array or a complex optical arrangement, both of which can only provide limited modulation bandwidth in the spectral dimension. This paper proposes a practical SSM method to help address the "missing cone" problem of chromo-tomography. A high-resolution binary coded aperture is used to modulate the dispersed images, which in the Fourier domain fulfills a 3D convolution of the probed spectrum with the aperture's wide spectrum. This spectrum spreading process facilitates the compressed sensing strategy determined by the Fourier Slice Theorem and we demonstrate the advantages of the proposed approach with numerical experiments.

In this paper, we propose an adaptive occlusion boundary extraction method for depth inference based on an adaptive segmentation and classification. First, an Adaptive DRW is proposed to generate more precise seeds and adaptive segmentation results, which can improve the feature quality and lower the boundary imbalance degree. Then, to deal with the imbalanced classification, we design a cost-sensitive boosting method-Adaptive AdaCost to better classify the imbalanced boundary, which can further improve overall performance and lower the cumulative misclassification cost and cost upper bound. Benefited from our Adaptive DRW and AdaCost, we extract more reliable and precise occlusion boundaries and use them for depth inference. The experiment results demonstrate that the combination of our Adaptive DRW and Adaptive AdaCost can produce more precise occlusion boundaries, and the depth inference result with our occlusion boundaries can be greatly improved.

In this paper, we first define a reality-preserving multiple parameter fractional angular transform (RPMPDFrAT), which is a useful tool for image encryption. Then, we propose a new color image encryption algorithm based on the defined RPMPDFrAT. The encryption process consists of two phases: encryption in the spatial domain and RPMPDFrAT domain. In the spatial domain, three color components of the plain image are mapped by dual cylindrical transform, which can nonlinearly hide the original color information. Then, the intermediate output is scrambled by a coupled logistic map to reduce the correlation of adjacent pixels and uniformly distribute the image energy of different color components. Thereafter, the scrambled image is transformed by the proposed RPMPDFrAT, which can ensure that we obtain the real-value output. Finally, a process similar to the spatial domain is performed in the RPMPDFrAT domain to further improve the security of the cryptosystem. Numerical simulations are performed and demonstrate that the proposed image encryption algorithm is effective and sensitive to keys. Moreover, some potential attacks are tested to verify the robustness of the proposed method, and the performance of our method outperforms previously published ones.

To ensure the confidentiality of color images during their storage or transmission on insecure networks, a number of encryption methods based on fractional transforms have been proposed and widely investigated. However, most of their outputs are complex values that are inconvenient for record and transmission. Also, those methods always deal with a whole color component with the same fractional-order and ignore the textural features that are contained in different image parts. In this paper, we first define a reality-preserving multiple-parameter fractional Hartley transform (RPMPFRHT), whose output is real value, and then a novel color image encryption method is proposed that divides the RGB components into different blocks and uses a constructed pixel scrambling operator to mix and hide the original color information. The outputs are transformed to different RPMPFRHT domains and further scrambled by a non-adjacent coupled map lattices system. Numerical simulations are performed to demonstrate that the proposed encryption algorithm is feasible, secure, sensitive to keys, and robust to potential attacks.

The explosive growth in high-dimensional visual data requires effective regularization techniques to utilize the underlying low-dimensional structure. We consider low-rank matrix recovery, and many existing approaches are based on the nuclear norm regularization. Recently, truncated nuclear norm (TNNR) has been proposed to achieve a better approximation to the rank function than that of the traditional nuclear norm. TNNR was defined by the nuclear norm by subtracting the sum of the largest r singular values. However, the estimation of r is not trivial. In addition, the original algorithm based on TNNR only considers the matrix completion cases and requires double loops, which is not quite computationally efficient. Correspondingly, in this paper, we propose the adaptive bisection method to adaptively estimate r, which can efficiently reduce the cost of computation. Moreover, to further accelerate computing, we apply iteratively reweighted nuclear norm to solve the nonconvex TNNR directly, and the convergence can also be guaranteed. Finally, we extend the applications of TNNR from the matrix completion problems to the general low-rank matrix recovery. Extensive experiments validate the superiority of the proposed algorithm over the state-of the-art methods.

Single image super-resolution (SR) technique is widely used to estimate high-resolution (HR) images from low-resolution (LR) ones. As a research hotspot, many example-based SR methods achieve superior results by learning class-mapping-kernels from classified external LR-HR patch-pair samples. However, in these methods, the classification of samples is generally based on the features of LR patch, and the interference of ill-samples to learn class-mapping-kernels is ignored as well. In this paper, we propose a new SR method with Sample Individual Mapping-Kernel (SIMK) based classification and Ill-Sample Removal Mechanism (ISRM) in learning LR-HR mapping. In the proposed sample classification, we use the SIMK feature which is the LR-to-HR mapping kernel of each sample, to classify samples and obtain more reasonable sample sets for mapping-learning. To prevent overfitting and reduce the complexity of SIMK-based-classification, samples are pre-categorized by relative pixel values of LR patch. In the mapping-learning process, the ill-samples which are far away from the classification center are removed to improve the validity of class-mapping-kernels. In addition, for each testing LR patch, the optimal class is assigned reasonably based on a probabilistic decision model learned from Naive Bayes Classifier. Comparing with state-of-the-art methods, our SR method achieves both visual and performance improvement.

In this paper, a novel method of digital image encryption based on the reality-preserving multiple parameter fractional Hartley transform (RPMPFrHT) is proposed. Firstly, we define an RPMPFrHT that make sure the output of cryptosystem is real-value. Then, based on random address sequences generated by coupled logistic function, we propose the double random scrambling encoding scheme which scrambled an image in the spatial domain and the RPMPFrHT domain respectively. Our method can encrypt an original image into noise-like picture with real-value which is convenient for storage and transmission. Numerical simulations have been performed and demonstrated that the proposed image encryption method is effective and sensitive to keys. Moreover, some potential attacks have also been performed to verify the robustness of the proposed method.

We propose a multichannel random discrete fractional Fourier transform (MRFrFT) with random weighting coefficients and partial transform kernel functions. First, the weighting coefficients of each channel are randomized. Then, the kernel functions, selected based on a choice scheme, are randomized using a group of random phase-only masks (RPOMs). The proposed MRFrFT can be carried out both electronically and optically, and its main features and properties have been given. Numerical simulation about one-dimensional signal demonstrates that the MRFrFT has an important feature that the magnitude and phase of its output are both random. Moreover, the MRFrFT of two-dimensional image can be viewed as a security enhanced image encryption scheme due to the large key space and the sensitivity to the private keys.