Due to the complexity and diversity of practical environments, real-world image dehazing remains an unresolved problem, with one of the key challenges being how to bridge the distribution gap between synthetic and real domains. This paper proposes a Prompt-driven Domain Adaptation (PDDA) framework within the bi-level optimization perspective. Specifically, we introduce hyperparameter optimization-based bi-level modeling: the lower-level optimization emphasizes prior learning within the synthetic domain to stabilize dehazing performance, while the upper-level optimization focuses on enhancing cross-domain adaptability to ensure that the model can generalize across different domains. Given the scarcity of paired real haze images, we train learnable haze prompts by jointly optimizing the text-image similarity between positive/negative prompts and corresponding clear/haze images in the CLIP latent space to more effectively capture real-world haze characteristics. Based on the learned haze prompts, we construct an unsupervised cross-domain loss function that enhances the adaptability to complex real-world scenarios by integrating prompt learning with bi-level optimization strategy. Furthermore, we conduct a comprehensive exploration to uncover the inherent properties of PDDA, including architecture-irrelevant flexibility and domain-agnostic robustness. Extensive experiments across a wide range of benchmark datasets demonstrate that our method achieves both quantitative and qualitative improvements across diverse scenarios, showing robust performance not only in real-world daytime conditions but also exhibiting superior cross-domain adaptation capabilities in nighttime scenarios. Codes are available at https://github.com/YanZhang-zy/PDDA.git.

Learning-based image dehazing algorithms have shown remarkable success in synthetic domains. However, real image dehazing is still in suspense due to computational resource constraints and the diversity of real-world scenes. Therefore, there is an urgent need for an algorithm that excels in both efficiency and adaptability to address real image dehazing effectively. This work proposes a Compression-and-Adaptation (CoA) computational flow to tackle these challenges from a divide-and-conquer perspective. First, model compression is performed in the synthetic domain to develop a compact dehazing parameter space, satisfying efficiency demands. Then, a bilevel adaptation in the real domain is introduced to be fearless in unknown real environments by aggregating the synthetic dehazing capabilities during the learning process. Leveraging a succinct design free from additional constraints, our CoA exhibits domain-irrelevant stability and model-agnostic flexibility, effectively bridging the model chasm between synthetic and real domains to further improve its practical utility. Extensive evaluations and analyses underscore the approach's superiority and effectiveness. The code is publicly available at https://github.com/fyxnl/COA.

Stylized human motion synthesis remains a fundamental challenge in computer animation and graphics, with a wide spectrum of applications spanning gaming, film production, virtual reality, and beyond. While recent advances in text-driven motion generation have shown promise, existing approaches face critical limitations including the inability to maintain consistent trajectory control, the lack of fine-grained stylization intensity adjustment, and inadequate generalization across diverse motion styles. To address these challenges, We introduce MCSMoG, a novel framework for controllable stylized motion synthesis through multi-conditional guidance. First, a new Multi-Conditional Motion Latent Diffusion (MC-MLD) model is proposed to introduce additional trajectory guidance and achieve trajectory decoupling. Second, we develop a Style and Non-Style Feature Fusion Module that dynamically blends motion features through an adjustable parameter, providing control over stylization intensity. Third, we integrate MotionCLIP as our style encoder, enhancing the model’s generalization capability across diverse and unseen motion styles. Extensive experiments conducted on the combined HumanML3D and 100STYLE datasets demonstrate that our approach outperforms state-of-the-art methods, achieving a 4.6% reduction in FID scores and a 4.1% increase in motion diversity. User studies further confirm the superiority of our method in style fidelity, semantic consistency, and motion naturalness.

Existing learning-based dehazing methods perform well on synthetic data but struggle in real scenarios due to the domain gap, causing residual haze and detail loss. To address this, we propose a Multilevel Subspace Distribution Adapter (MSDA) to progressively reduce the feature distribution gap through hierarchical subspace modeling. We also introduce a Dual-Domain Synchronous Optimization (DDSO) strategy that jointly leverages synthetic supervision and adaptation to the real domain in a unified training scheme. Extensive experiments underscore the superiority of our approach and its excellence on no-reference image quality metrics.

For single-view clothed human reconstruction, the fashionable PIFu-like framework depends on the pixel-aligned feature essentially, while this leads to depth ambiguity and inaccuracy of representation. Additionally, this task faces the inherent problem of the lack of invisible information. To solve these two problems, we propose depth-guided pixel-aligned feature and multi-view consistency prior to constrain representation learning of the single-view reconstruction task. The difference, between the depth values of points and estimated depth map, is used to filter pixel-aligned features. Thus, the image encoder can focus on capturing the feature of the visible part which is more effective in feature representation. The method introduces contrastive learning and masked autoencoder to achieve consistency of SMPL vertex features in each view which helps model to imagine invisible information. The experimental results show that the proposed method enables the feature to represent surface details more efficiently, thus achieves more reasonable and accurate representation learning. The qualitative and quantitative evaluations on the public and commercial datasets show that the proposed method can achieve better performance than previous implicit representation based methods.

Accurate segmentation of 3D point clouds in indoor scenes remains a challenging task, often hindered by the labor-intensive nature of data annotation. While weakly supervised learning approaches have shown promise in leveraging partial annotations, they frequently struggle with imbalanced performance between foreground and background elements due to the complex structures and proximity of objects in indoor environments. To address this issue, we propose a novel foreground-aware label enhancement method utilizing visual boundary priors. Our approach projects 3D point clouds onto 2D planes and applies 2D image segmentation to generate pseudo-labels for foreground objects. These labels are subsequently back-projected into 3D space and used to train an initial segmentation model. We further refine this process by incorporating prior knowledge from projected images to filter the predicted labels, followed by model retraining. We introduce this technique as the Foreground Boundary Prior (FBP), a versatile, plug-and-play module designed to enhance various weakly supervised point cloud segmentation methods. We demonstrate the efficacy of our approach on the widely-used 2D-3D-Semantic dataset, employing both random-sample and bounding-box based weak labeling strategies. Our experimental results show significant improvements in segmentation performance across different architectural backbones, highlighting the method's effectiveness and portability.

Abstract In recent years, most video segmentation methods use deep CNN to process the input image, but they did not fully mine the rich intermediate predictions in spatio‐temporal space. And, the segmentation challenges such as occlusion, severe deformation and illumination have not been well solved so far. To alleviate these problems, this paper focuses on constructing multi module network structures that represent multi semantics and proposes a video object segmentation network via coupled‐stream architecture with feature memory mechanism. This network first extracts high‐level semantic features, edge features, long‐term and short‐term stable depth features of the target, and then decode them into the segmentation mask of target. In addition, negative skeleton inhibition and frame interpolation are used to prevent the interference of similar objects and motion blur, respectively. The method has a low GPU memory usage, regardless of the number of object in video. And performs 86.5%and 62.4% in J&F measure on DAVIS 2016 and DAVIS 2017 validation set, without fine‐tuning and online training.

Restoring high-quality images from degraded hazy observations is a fundamental and essential task in the field of computer vision. While deep models have achieved significant success with synthetic data, their effectiveness in real-world scenarios remains uncertain. To improve adaptability in real-world environments, we construct an entirely new computational framework by making efforts from three key aspects: imaging perspective, structural modules, and training strategies. To simulate the often-overlooked multiple degradation attributes found in real-world hazy images, we develop a new hazy imaging model that encapsulates multiple degraded factors, assisting in bridging the domain gap between synthetic and real-world image spaces. In contrast to existing approaches that primarily address the inverse imaging process, we design a new dehazing network following the "localization-and-removal" pipeline. The degradation localization module aims to assist in network capture discriminative haze-related feature information, and the degradation removal module focuses on eliminating dependencies between features by learning a weighting matrix of training samples, thereby avoiding spurious correlations of extracted features in existing deep methods. We also define a new Gaussian perceptual contrastive loss to further constrain the network to update in the direction of the natural dehazing. Regarding multiple full/no-reference image quality indicators and subjective visual effects on challenging RTTS, URHI, and Fattal real hazy datasets, the proposed method has superior performance and is better than the current state-of-the-art methods.

In real-world scenarios, the haze presents diversity and complexity. However, current dehazing researches usually focus solely on specific categories or the removal of common white haze, frequently lacking the ability to adapt across various unknown haze types. In this study, our emphasis is on constructing a model that shows excellent adaptability across diverse haze conditions. Unlike approaches that solely rely on network structure design to enhance model adaptability, we comprehensively improve dehazing model adaptability from three key aspects: constructing the multitype haze dataset from designed haze degradation models, designing the network architecture, and formulating training strategies suitable for cross-scene generalization. Firstly, to meet the diverse haze training data requirements, we design a multitype haze degradation model to generate more realistic pairs of hazy images. Secondly, to ensure thorough haze removal and natural restoration of texture details in the recovered images, we construct a dual-branch ensemble network framework by leveraging pre-trained clear image prior features and the characteristics of 2D discrete wavelet priors. Finally, to further enhance the adaptability for removing various types of haze, we employ a sample reweighting decorrelation strategy during the network training phase to eliminate dependencies between haze and haze-free background features. Through extensive experiments, our approach shows remarkable performance across diverse haze scenarios. Our method not only outperforms state-of-the-art scene-specific dehazing methods in typical scenarios like daytime and nighttime, but it also excels in handling challenging scenarios such as dusty conditions, and color haze. See more resultshttps://github.com/fyxnl/Image-dehazing-CGID.

Joint video moment retrieval and highlight detection is an emerging and challenging research task. It requires the generation of robust joint task features to satisfy the demands of video moment retrieval and video highlight detection. Moreover, it involves the interaction of multiple modalities. Presently, methods typically focus on the design of distinct enhancement modules and the addition of supplementary input data to improve the solution for joint video moment retrieval and highlight detection. However, they overlook subtask interference during joint training. Joint task learning leverages the correlations and complementarities between tasks, yet it also introduces task interference arising from the differences between tasks. In order to address task interference, we proposes a subtask prior-driven optimized mechanism. The mechanism consists of two stages. In the free stage, we train subtask model to get subtask prior features. In the constrained stage, the joint task model is constrained by the subtask. Besides, we propose a cross adaptive-gated mechanism. It addresses the issue of information loss in cross-modal fusion and filters out redundant information by conducting cross-modal interaction during feature compression and an adaptive gating process. Extensive experimental results exhibit the effectiveness of the subtask prior-driven optimized mechanism and the cross adaptive-gated transformer in joint video moment retrieval and highlight detection.

Outfit collocation requires considering the interrelationship and adaptability among the attributes of component items. However, with the numerous and diverse attributes of fashion items, accurately capturing attribute features and modeling the complex relationships between attributes become the key challenges. To address these challenges, we propose a novel scheme Decoupling-driven Multi-level Attribute Parsing for interpretable outfit collocation. First, we decouple a series of attribute features from the item's visual feature by fully supervised, which can improve the robustness of the model in processing both relevant and irrelevant attributes of items. Furthermore, employing a deep deconvolution neural network with attention mechanisms to reconstruct the decoupled attribute features into a visual image that is close to the original item image. It ensures all attribute features can be combined to contain complete item information. Next, graph attention networks are constructed to parse multi-level attribute compatibility relationships from three perspectives: intra-attribute, inter-attribute, and item integration relationships. Finally, we use multi-layer perceptrons to fuse the score distributions of the three and output the outfit compatibility score. Experiments conducted on the IQON3000 dataset demonstrate that our model outperforms existing state-of-the-art methods and exhibits good interpretability.

Abstract Detailed and accurate feature representation is essential for high‐resolution reconstruction of clothed human. Herein we introduce a unified feature representation for clothed human reconstruction, which can adapt to changeable posture and various clothing details. The whole method can be divided into two parts: the human shape feature representation and the details feature representation. Specifically, we firstly combine the voxel feature learned from semantic voxel with the pixel feature from input image as an implicit representation for human shape. Then, the details feature mixed with the clothed layer feature and the normal feature is used to guide the multi‐layer perceptron to capture geometric surface details. The key difference from existing methods is that we use the clothing semantics to infer clothed layer information, and further restore the layer details with geometric height. We qualitative and quantitative experience results demonstrate that proposed method outperforms existing methods in terms of handling limb swing and clothing details. Our method provides a new solution for clothed human reconstruction with high‐resolution details (style, wrinkles and clothed layers), and has good potential in three‐dimensional virtual try‐on and digital characters.

Abstract Clothed human re‐construction from a monocular image is challenging due to occlusion, depth‐ambiguity and variations of body poses. Recently, shape representation based on an implicit function, compared to explicit representation such as mesh and voxel, is more capable with complex topology of clothed human. This is mainly achieved by using pixel‐aligned features, facilitating implicit function to capture local details. But such methods utilize an identical feature map for all sampled points to get local features, making their models occlusion‐agnostic in the encoding stage. The decoder, as implicit function, only maps features and does not take occlusion into account explicitly. Thus, these methods fail to generalize well in poses with severe self‐occlusion. To address this, we present OaIF to encode local features conditioned in visibility of SMPL vertices. OaIF projects SMPL vertices onto image plane to obtain image features masked by visibility. Vertices features integrated with geometry information of mesh are then feed into a GAT network to encode jointly. We query hybrid features and occlusion factors for points through cross attention and learn occupancy fields for clothed human. The experiments demonstrate that OaIF achieves more robust and accurate re‐construction than the state of the art on both public datasets and wild images.

In complex natural haze scenes, image haze removal still faces significant challenges in removing non-homogeneous and dense haze. The double complexity of haze distribution, on the one hand, is reflected in the interference of haze to the global image information, and on the other hand, it is reflected in the imbalance of image brightness and color caused by random haze distribution. In natural scenes with prominent edge and texture features, the above problems may cause severe degradation of image quality and performance of various tasks. Numerous studies on network learning show that the effect of haze removal is closely related to haze feature expression. Therefore, to improve the performance of dehazing, this paper proposes a sliding self-attention wavelet network. Specifically, we first design a sliding self-attention module to identify haze regions in images and capture rich haze-related feature information. Then, considering the uneven distribution of haze in images, discrete wavelet transform (DWT) and inverse transform (IDWT) are used for constructing a hierarchical encoder-decoder structure, which can fully use the multi-resolution characteristics of DWT, locally decompose feature maps of different scales, extract low and high-frequency information, and then gradually recover sharp edges and precise texture details from hazy images. Finally, to enable the proposed network to generate more realistic haze-free images on different complex haze scenes, we develop a DWT-based adversarial loss function to constrain the low and high-frequency components of generated images closer to the corresponding clear images. Experimental results on the relevant public benchmark datasets show that the proposed algorithm achieves favorable dehazing performance.

With the popularization of object detection in fields such as autonomous driving and surveillance, object detection networks are required to be used in more changeable scenarios. However, a multi-target domain adaptive object detection network is a challenging problem. Due to the lack of relevant datasets and the limitation of traditional domain adaptive methods, there is a lack of pertinent research in this field. Traditional single-target domain adaptive methods are often ineffective when used in multi-target domain adaptive problems. To allow the object detection network to be applied to multiple target domains, we propose the “domain transfer module” and the “multi-scale hybrid attention domain alignment module”. At the same time, we synthesized the “BlendedCityspce” dataset for training and testing of a multi-target domain adaptive target object network. The network provided in this paper has a good performance in the multi-target domain adaptation and reaches the state-of-the-art in the single-target domain adaptation. You can find our source code in https://github.com/Chonghuan-Liu/Multi-Target-Domain-Adaptative-Detector.

Haze can cause significant changes in the data domain of the image. Due to domain-shift problem, the model trained on the synthetic haze images has a weak or even invalid dehazing ef-fect on the real data. This paper proposes an unsupervised domain adaptation dehazing method based on the nearest-farthest subspace distance in response to this problem. For the middle convolved features, the matrix singular value de-composition is used to obtain the basis vectors of the support subspace. Narrowing the angle between the basis vectors of natural and synthetic haze domains could reduce the differ-ence between the data domains. In subspace, a newly defined distance with a new penalty term named subspace measure distance is employed to constrain the model. Furthermore, to avoid search blindness of the proposed method in the real data training phase, we propose the nearest-farthest subspace distance inspired by contrastive learning. In addition, we use a new training strategy. Finally, the experimental tests on the synthetic and real images prove the effectiveness of the pro-posed method.

The variety and deformable characteristics of fashions are the challenges in improving the accuracy of fashion landmark detection. Considering the delicate characteristics, this paper proposes a new method for fashion landmark detection based on the cross-rolling attention module and the feature pyramid network. It imports the cross-rolling attention mechanism into the multi-scale structure to extract and fuse different features with rich context, which is beneficial to capture fine-grained fashion keypoints more accurately and improve detection performance. While constructing the network, we also focus on controlling the network scale to avoid a large increase in parameters and longtime training. Finally, we perform some experiments suggesting that the proposed method achieves superior performance on both DeepFashion and FLD datasets. It demonstrates the effectiveness of the cross-rolling attention module in the proposed method.

Existing learning-based dehazing methods do not fully use non-local information, which makes the restoration of seriously degraded region very tough. We propose a novel dehazing network by defining the Feature Dense Relevance module (FDR) and the Shallow Feature Mapping module (SFM). The FDR is defined based on multi-head attention to construct the dense relationship between different local features in the whole image. It enables the network to restore the degraded local regions by non-local information in complex scenes. In addition, the raw distant skip-connection easily leads to artifacts while it cannot deal with the shallow features effectively. Therefore, we define the SFM by combining the atmospheric scattering model and the distant skip-connection to effectively deal with the shallow features in different scales. It not only maps the degraded textures into clear textures by distant dependence, but also reduces artifacts and color distortions effectively. We introduce contrastive loss and focal frequency loss in the network to obtain a realitic and clear image. The extensive experiments on several synthetic and real-world datasets demonstrate that our network surpasses most of the state-of-the-art methods.

Due to wide application prospects and various challenges such as large scale variation, inter-occlusion between crowd people and background noise, crowd counting is receiving increasing attention. In this paper, we propose a scale-aware rolling fusion network (SRF-Net) for crowd counting, which focuses on dealing with scale variation in highly congested noisy scenes. SRF-Net is a two-stage architecture that consists of a band-pass stage and a rolling guidance stage. Compared with the existing methods, SRF-Net achieves better results in retaining appropriate multi-level features and capturing multi-scale features, thus improving the quality of density estimation maps in crowded scenarios with large scale variation. We evaluate our method on three popular crowd counting datasets (ShanghaiTech, UCF_CC_50 and UCF-QNRF), and extensive experiments show its outperformance over the state-of-the-art approaches.

Human parsing, which segments a human image into semantic regions, is a fundamental task in human-centric analysis. Recently, numerous human parsing approaches based on convolutional neural networks (CNNs) have made significant progress. However, these models still suffer the high time-complexity and poor efficiency issues, since they always introduce too many and complex hypothetical priors. Therefore, in this paper, we attempt to solve above issues by mining the deep information hidden in data, such as the scale, spatial information and statistics of the image descriptors, etc. By this way, we propose an effective structure, named the trilateral awareness operation (TAO), which can boost the scale, spatial and fine-grained awareness of a CNN. In practice, it is a simple yet effective architecture without using bells and whistles (such as human pose and edge information), to address human parsing tasks. Comprehensive experiments and corresponding results on three public datasets have demonstrated that the proposed TAO is superior to the state-of-the-art methods.

Virtual try-on system could demonstrate the visual effect of wearing certain clothes, which is in a great demand for the online clothing customers. Due to the neglect of the original person structure, the previous try-on schemes usually encounter with the problems like the loss of body parts, the missing of body details and the deviation of clothing style. In this paper, we propose a novel image-based virtual try-on network, which could maintain the structural consistency between the generated image and the original image by human parsing. Our network consists of three components. Given the original person and the clothing images, the target human parsing maps are generated. Then, the parsing maps are matched with the target clothes to generate the warped clothes. Finally, according to the parsing results, the parts to be replaced the original images are intercepted, and more original information is retained as the input of the network to generate the final results. Experiments on an existing benchmark demonstrate our method maintains the consistency of structure and achieves the state-of-the-art performance.

PM2.5 is an important indicator of the severity of air pollution and its level can be predicted through hazy photographs caused by its degradation. Image-based PM2.5 estimation is thus extensively employed in various multimedia applications but is challenging because of its ill-posed property. In this paper, we convert it to the problem of estimating the PM2.5-relevant haze transmission and propose a learning model called the transmission filtering network. Different from most methods that generate a transmission map directly from a hazy image, our model takes the coarse transmission map derived from the dark channel prior as the input. To obtain a transmission map that satisfies the local smoothness constraint without regional boundary degradation, our model performs the edge-preserving smoothing filtering as the refinement on the map. Moreover, we introduce the attention mechanism to the network architecture for more efficient feature extraction and smoothing effects in the transmission estimation. Experimental results prove that our model performs favorably against the state-of-the-art dehazing methods in a variety of hazy scenes.

Recently, deep learning methods have been successfully applied to single image super-resolution tasks. However, some networks with extreme depth failed to achieve better performance because of the insufficient utilization of the local residual information extracted at each stage. To solve the above question, we propose a Dense Information Flow Network (DIF-Net), which can fully extract and utilize the local residual information at each stage to accomplish a better reconstruction. Specifically, we present a Two-stage Residual Extraction Block (TREB) to extract the shallow and deep local residual information at each stage. The dense connection mechanism is introduced throughout the model and within TREBs to dramatically increase the information flow. Meanwhile this mechanism prevents the shallow features extracted earlier from being diluted. Finally, we propose a lightweight subnet (residual enhancer) to efficiently recycle the overflow residual information from the backbone net for detail enhancement of the residual image. Experimental results demonstrate that the proposed method performs favorably against the state-of-the-art methods with relatively-less parameters.

Abstract Single image rain removal is a challenging ill‐posed problem due to various shapes and densities of rain streaks. We present a novel incremental randomly wired network (IRWN) for single image deraining. Different from previous methods, most structures of modules in IRWN are generated by a stochastic network generator based on the random graph theory, which ease the burden of manual design and further help to characterize more complex rain streaks. To decrease network parameters and extract more details efficiently, the image pyramid is fused via the multi‐scale network structure. An incremental rectified loss is proposed to better remove rain streaks in different rain conditions and recover the texture information of target objects. Extensive experiments on synthetic and real‐world datasets demonstrate that the proposed method outperforms the state‐of‐the‐art methods significantly. In addition, an ablation study is conducted to illustrate the improvements obtained by different modules and loss items in IRWN.

In this study, the authors propose a novel progressive dehazing network to address the single image haze removal problem based on a new mean progressive scattering model. Different from methods that learn atmosphere light and transmission maps with different networks, these two variables are optimised in a unified network. Following the methodology of traditional prior‐based methods that estimate a coarse transmission map first, a progressive refinement branch in the decoder has been designed to restore the fine‐scale transmission map. To improve the prediction accuracy of the transmission map, a novel binomial truncated loss that assigns weights to error values according to the probabilities of error occurrences has been proposed. An ablation study is conducted to verify the effectiveness of the components in the proposed method. Experiments in the synthetic datasets and real images demonstrate that the proposed method outperforms other state‐of‐the‐art methods.

Clothing parsing is significant to many clothing applications. Recently, a lot of clothing parsing methods have been presented, which explore the innovation of the parsing pipeline or try to find more specific prior information. Although these methods perform well in some benchmarks, a few challenging problems have not been solved yet, such as the complicated mutual interference among labels. In this study, the authors propose a Conditional Progressive Network to parse clothing in different scales and prevent the mutual interference among labels. The authors’ solution consists of three sub‐networks, including Conditional Parsing Network (CPN), Pose Estimation Network (PEN) and Label Transform Network (LTN). Specifically, the CPN module generates the intermediate parsing result in the form of the multiple progressive stages, which combines with the previous outputs in each stage and the specific prior conditions. The PEN module provides a series of heat maps about the human pose information. The LTN module suppresses the redundant labels to avoid the mutual interference among labels. They demonstrate their solution in parsing the fashion clothing cases on the ATR and the Fashion dataset. In their experiments, their method obtains a better performance than the state‐of‐the‐art methods.

Human parsing, which segments a human-centric image into pixel-wise categorization, has a wide range of applications. However, none of the existing methods can productively solve the issue of label parsing fragmentation due to confused and complicated annotations. In this paper, we propose a novel Trusted Guidance Pyramid Network (TGPNet) to address this limitation. Based on a pyramid architecture, we design a Pyramid Residual Pooling (PRP) module setting at the end of a bottom-up approach to capture both global and local level context. In the top-down approach, we propose a Trusted Guidance Multi-scale Supervision (TGMS) that efficiently integrates and supervises multi-scale contextual information. Furthermore, we present a simple yet powerful Trusted Guidance Framework (TGF) which imposes global-level semantics into parsing results directly without extra ground truth labels in model training. Extensive experiments on two public human parsing benchmarks well demonstrate that our TGPNet has a strong ability in solving label parsing fragmentation problem and has an obtained improvement than other methods.

Abstract Non‐photorealistic rendering has been an active area of research for decades whereas few of them concentrate on rendering chromatic penciling style. In this paper, we present a framework named as PencilArt for the chromatic penciling style generation from wild photographs. The structural outline and textured map for composing the chromatic pencil drawing are generated, respectively. First, we take advantage of deep neural network to produce the structural outline with proper intensity variation and conciseness. Next, for the textured map, we follow the painting process of artists to adjust the tone of input images to match the luminance histogram and pencil textures of real drawings. Eventually, we evaluate PencilArt via a series of comparisons to previous work, showing that our results better capture the main features of real chromatic pencil drawings and have an improved visual appearance.

Image Ranking is one of the key problems in information science research area. However, most current methods focus on increasing the performance, leaving the semantic gap problem, which refers to the learned ranking models are hard to be understood, remaining intact. Therefore, in this article, we aim at learning an interpretable ranking model to tackle the semantic gap in fine‐grained image ranking. We propose to combine attribute‐based representation and online passive‐aggressive (PA) learning based ranking models to achieve this goal. Besides, considering the highly localized instances in fine‐grained image ranking, we introduce a supervised constrained clustering method to gather class‐balanced training instances for local PA‐based models, and incorporate the learned local models into a unified probabilistic framework. Extensive experiments on the benchmark demonstrate that the proposed framework outperforms state‐of‐the‐art methods in terms of accuracy and speed.

Abstract This paper presents a new two‐step color transfer method which includes color mapping and detail preservation. To map source colors to target colors, which are from an image or palette, the proposed similarity‐preserving color mapping algorithm uses the similarities between pixel color and dominant colors as existing algorithms and emphasizes the similarities between source image pixel colors. Detail preservation is performed by an ℒ0 gradient‐preserving algorithm. It relaxes the large gradients of the sparse pixels along color region boundaries and preserves the small gradients of pixels within color regions. The proposed method preserves source image color similarity and image details well. Extensive experiments demonstrate that the proposed approach has achieved a state‐of‐art visual performance.

Image self‐similarity property is important to super‐resolution reconstruction. However, how to effectively exploit the self‐similarity information to reconstruct an underlying high‐resolution image is still a challenging problem. The authors propose a novel model for solving the single image upsampling problem with the self‐similarity property. First, the authors construct a statistical prior that requires maximising the similarity between the low‐ and high‐resolution image pairs. Then, the authors develop an alternative Gaussian approximation solver based on the Gaussian mixture model to find the optimal high‐resolution output. To obtain a better performance, the authors summarise some refined implementation skills to raise the reconstruction quality. For demonstration, a series of objective and subjective measurements are used to evaluate the performance of the model.

The bilateral filter (BF) is a non‐linear filter that spatially smooths images with awareness of large structures such as edges. The level of smoothness applied to a pixel is constrained by a photometric weight, which can be obtained from the same image to be filtered (in case of the original BF) or from a guided image (in case of the joint/cross BF). In this study, the authors propose a new filter called the semi‐guided BF which is derived from solving a non‐linear constraint least square problem. The proposed filter's photometric weight incorporates information from the image to be filtered and the guided image. They propose a fast implementation of the filter based on layer approximation. They also study the iterative application of the proposed filter and show that the filter can preserve large structures while smoothing out small structures. This makes the proposed filter an efficient and effective tool for structure‐aware image smoothing. Experimental results have demonstrated that performance of the proposed filter is comparable to those of the state‐of‐the‐art algorithms.

Attribute‐based approaches have recently attracted much attention in visual recognition tasks. These approaches describe images by using semantic attributes as the mid‐level feature. However, low recognition accuracy becomes the biggest barrier that limits their practical applications. In this paper, we propose a novel framework termed Boosting Attribute Recognition (BAR) for the image recognition task. Our framework stems from matrix factorization, and can explore latent relationships from the aspect of attribute and image simultaneously. Furthermore, to apply our framework in large‐scale visual recognition tasks, we present both offline and online learning implementation of the proposed framework. Extensive experiments on 3 data sets demonstrate that our framework achieves a sound accuracy of attribute recognition.

Image colorization aims to produce a natural looking color image from a given gray-scale image, which remains a challenging problem. In this paper, we propose a novel example-based image colorization method exploiting a new locality consistent sparse representation. Given a single reference color image, our method automatically colorizes the target gray-scale image by sparse pursuit. For efficiency and robustness, our method operates at the superpixel level. We extract low-level intensity features, mid-level texture features, and high-level semantic features for each superpixel, which are then concatenated to form its descriptor. The collection of feature vectors for all the superpixels from the reference image composes the dictionary. We formulate colorization of target superpixels as a dictionary-based sparse reconstruction problem. Inspired by the observation that superpixels with similar spatial location and/or feature representation are likely to match spatially close regions from the reference image, we further introduce a locality promoting regularization term into the energy formulation, which substantially improves the matching consistency and subsequent colorization results. Target superpixels are colorized based on the chrominance information from the dominant reference superpixels. Finally, to further improve coherence while preserving sharpness, we develop a new edge-preserving filter for chrominance channels with the guidance from the target gray-scale image. To the best of our knowledge, this is the first work on sparse pursuit image colorization from single reference images. Experimental results demonstrate that our colorization method outperforms the state-of-the-art methods, both visually and quantitatively using a user study.

For a given image, it is a challenging task to generate its corresponding counterpart with visual inconspicuous modification. The complexity of this problem reasons from the high correlativity between the editing operations and vision perception. Essentially, a significant requirement that should be emphasized is how to make the object modifications hard to be found visually in the generative counterparts. In this article, we propose a novel dual-domain perceptual framework to generate visual inconspicuous counterparts, which applies the perceptual bidirectional similarity metric (PBSM) and appearance similarity metric (ASM) to create the dual-domain perception error minimization model. The candidate targets are yielded by the well-known PatchMatch model with the strokes-based interactions and selective object library. By the dual-perceptual evaluation index, all candidate targets are sorted to select out the best result. For demonstration, a series of objective and subjective measurements are used to evaluate the performance of our framework.

Example-based color transfer is a critical operation in image editing but easily suffers from some corruptive artifacts in the mapping process. In this paper, we propose a novel unified color transfer framework with corruptive artifacts suppression, which performs iterative probabilistic color mapping with self-learning filtering scheme and multiscale detail manipulation scheme in minimizing the normalized Kullback-Leibler distance. First, an iterative probabilistic color mapping is applied to construct the mapping relationship between the reference and target images. Then, a self-learning filtering scheme is applied into the transfer process to prevent from artifacts and extract details. The transferred output and the extracted multi-levels details are integrated by the measurement minimization to yield the final result. Our framework achieves a sound grain suppression, color fidelity and detail appearance seamlessly. For demonstration, a series of objective and subjective measurements are used to evaluate the quality in color transfer. Finally, a few extended applications are implemented to show the applicability of this framework.

Image retargeting is a critical technique in displaying images on devices with different resolutions. This study presents a new image retargeting algorithm based on aesthetic‐based cropping and scaling. A composite measurement is first constructed under the guidelines of composition aesthetics in photographing. An aesthetic‐based cropping is proposed to yield an optimal candidate retargeted image with maximum aesthetic value computed via a constructed composite measurement. The optimal candidate is uniformly scaled to obtain the retargeted image of target size. Some subjective and objective assessments demonstrate that the proposed scheme significantly improves the aesthetics of retargeted images while preserving the important objects. It also achieves better performance in terms of aesthetics than a number of conventional image retargeting approaches.

Obtaining a texture-smoothing and edge-preserving filtered output is significant to image decomposition. Although the edge and the texture have salient difference in human vision, automatically distinguishing them is a difficult task, for they have similar intensity difference or gradient response. The state-of-the-art edge-preserving smoothing (EPS) based decomposition approaches are hard to obtain a satisfactory result. We propose a novel edge-preserving texture suppression filter, exploiting the joint bilateral filter as a bridge to achieve the purpose of both properties of texture-smoothing and edge-preserving. We develop the iterative asymmetric sampling and the local linear model to produce the degenerative image to suppress the texture, and apply the edge correction operator to achieve edge-preserving. An efficient accelerating implementation is introduced to improve the performance of filtering response. The experiments demonstrate that our filter produces satisfactory outputs with both properties of texture-smoothing and edge-preserving, while compared with the results of other popular EPS approaches in signal, visual and time analysis. Finally, we extend our filter to a variety of image processing applications.

We track the object by separating it from the surrounding with an ensemble of boosted classifiers, which are trained in a discriminative feature space that is determined on the fly. Contour refinement and weight thresholding techniques are used to select good examples for training. While tracking, location calibration and scale adaptation are used to improve the tracker's performance. We update the ensemble of weak classifiers online to adapt to appearance changes, and use the positive occupancy ratio to detect occlusion. A center-surround discrepancy measure is presented to evaluate the discriminative power of the current feature space and to invoke re-initialization of feature selection and classifier training if necessary. Experiments on challenging video sequences demonstrate the effectiveness of the proposed approach.

Automatic global color transfer is a challenging problem in image editing. In this paper, we propose a novel color transfer method, which is based on the gradient-aware decomposition and the color distribution mapping. Firstly, a gradient-aware decomposition model is established to separate the target image into the base and detail layers. Then, the colors of each separated base layer are enforced to match those of a given reference image by Pitie's multi-dimensional probability density function transfer method. After that, all mapped base layers are combined with corresponding boosted detail layers to produce the final output. The experiments demonstrate that our method can achieve a visual satisfied result without post-processing gradient correction.