In long-term series forecasting (LTSF), it is imperative for models to adeptly discern and distill from historical time series data to forecast future states. Although Transformer-based models excel at capturing long-term dependencies in LTSF, their practical use is limited by issues like computational inefficiency, noise sensitivity, and overfitting on smaller datasets. Therefore, we introduce a novel time series lightweight interactive Mamba with an adaptive Fourier filter model (Affirm). Specifically, (i) we propose an adaptive Fourier filter block. This neural operator employs Fourier analysis to refine feature representation, reduces noise with learnable adaptive thresholds, and captures inter-frequency interactions using global and local semantic adaptive Fourier filters via element-wise multiplication. (ii) A dual interactive Mamba block is introduced to facilitate efficient intra-modal interactions at different granularities, capturing more detailed local features and broad global contextual information, providing a more comprehensive representation for LTSF. Extensive experiments on multiple benchmarks demonstrate that Affirm consistently outperforms existing SOTA methods, offering a superior balance of accuracy and efficiency, making it ideal for various challenging scenarios with noise levels and data sizes.

In recent years, large text-to-video (T2V) synthesis models have garnered considerable attention for their abilities to generate videos from textual descriptions. However, achieving both high imaging quality and effective motion representation remains a significant challenge for these T2V models. Existing approaches often adapt pre-trained text-to-image (T2I) models to refine video frames, leading to issues such as flickering and artifacts due to inconsistencies across frames. In this paper, we introduce EVS, a training-free Encapsulated Video Synthesizer that composes T2I and T2V models to enhance both visual fidelity and motion smoothness of generated videos. Our approach utilizes a well-trained diffusion-based T2I model to refine low-quality video frames by treating them as out-of-distribution samples, effectively optimizing them with noising and denoising steps. Meanwhile, we employ T2V backbones to ensure consistent motion dynamics. By encapsulating the T2V temporal-only prior into the T2I generation process, EVS successfully leverages the strengths of both types of models, resulting in videos of improved imaging and motion quality. Experimental results validate the effectiveness of our approach compared to previous approaches. Our composition process also leads to a significant improvement of 1.6x-4.5x speedup in inference time.1

Cross-modal retrieval is essential for interpreting cultural heritage data, but its effectiveness is often limited by incomplete or inconsistent textual descriptions, caused by historical data loss and the high cost of expert annotation.While large language models (LLMs) offer a promising solution by enriching textual descriptions, their outputs frequently suffer from hallucinations or miss visually grounded details.To address these challenges, we propose C 3 , a data augmentation framework that enhances cross-modal retrieval performance by improving the completeness and consistency of LLM-generated descriptions.C 3 introduces a completeness evaluation module to assess semantic coverage using both visual cues and language-model outputs.Furthermore, to mitigate factual inconsistencies, we formulate a Markov Decision Process to supervise Chain-of-Thought reasoning, guiding consistency evaluation through adaptive query control.Experiments on the cultural heritage datasets CulTi and TimeTravel, as well as on general benchmarks MSCOCO and Flickr30K, demonstrate that C 3 achieves state-of-the-art performance in both fine-tuned and zero-shot settings.The code of this paper is available at https://github.com/JianZhang24/C-3.

Diffusion-based garment synthesis tasks primarily focus on the design phase in the fashion domain, while the garment production process remains largely underexplored. To bridge this gap, we introduce a new task: Flat Sketch to Realistic Garment Image (FS2RG), which generates realistic garment images by integrating flat sketches and textual guidance. FS2RG presents two key challenges: 1) fabric characteristics are solely guided by textual prompts, providing insufficient visual supervision for diffusion-based models, which limits their ability to capture fine-grained fabric details; 2) flat sketches and textual guidance may provide conflicting information, requiring the model to selectively preserve or modify garment attributes while maintaining structural coherence. To tackle this task, we propose HiGarment, a novel framework that comprises two core components: i) a multi-modal semantic enhancement mechanism that enhances fabric representation across textual and visual modalities, and ii) a harmonized cross-attention mechanism that dynamically balances information from flat sketches and text prompts, allowing controllable synthesis by generating either sketch-aligned (image-biased) or text-guided (text-biased) outputs. Furthermore, we collect Multi-modal Detailed Garment, the largest open-source dataset for garment generation. Experimental results and user studies demonstrate the effectiveness of HiGarment in garment synthesis. The code and dataset are available at https://github.com/Maple498/HiGarment.

Video editing is a pivotal process for customizing video content according to user needs. However, existing text-guided methods often lead to ambiguities regarding user intentions and restrict fine-grained control for editing specific aspects in videos. To overcome these limitations, this paper introduces a novel approach named \emph{AdaptEdit}, which focuses on reference-based video editing that disentangles the editing process. It achieves this by first editing a reference image and then adaptively propagating its appearance across other frames to complete the video editing. While previous propagation methods, such as optical flow and the temporal modules of recent video generative models, struggle with object deformations and large motions, we propose an adaptive correspondence strategy that accurately transfers the appearance from the reference frame to the target frames by leveraging inter-frame semantic correspondences in the original video. By implementing a proxy-editing task to optimize hyperparameters for image token-level correspondence, our method effectively balances the need to maintain the target frame's structure while preventing leakage of irrelevant appearance. To more accurately evaluate editing beyond the semantic-level consistency provided by CLIP-style models, we introduce a new dataset, PVA, which supports pixel-level evaluation. Our method outperforms the best-performing baseline with a clear PSNR improvement of 3.6 dB.

Instruction-based garment editing enables precise image modifications via natural language, with broad applications in fashion design and customization. Unlike general editing tasks, it requires understanding garment-specific semantics and attribute dependencies. However, progress is limited by the scarcity of high-quality instruction-image pairs, as manual annotation is costly and hard to scale. While MLLMs have shown promise in automated data synthesis, their application to garment editing is constrained by imprecise instruction modeling and a lack of fashion-specific supervisory signals. To address these challenges, we present an automated pipeline for constructing a garment editing dataset. We first define six editing instruction categories aligned with real-world fashion workflows to guide the generation of balanced and diverse instruction-image triplets. Second, we introduce Fashion Edit Score, a semantic-aware evaluation metric that captures semantic dependencies between garment attributes and provides reliable supervision during construction. Using this pipeline, we construct a total of 52,257 candidate triplets and retain 20,596 high-quality triplets to build EditGarment, the first instruction-based dataset tailored to standalone garment editing. The project page is https://yindq99.github.io/EditGarment-project/.

Learning time series representations with sparse labels presents notable challenges. The surge in unsupervised contrastive learning has garnered increasing interest due to its immense advancements in deriving meaningful representations in semi-supervised settings, typically involving a two-stage process: pretraining on large unlabeled data followed by fine-tuning with few labeled samples. However, this approach has inherent drawbacks: poor knowledge transfer, reduced generalizability, and failure to directly utilize unsupervised contrastive loss from pretraining and valuable supervised loss guided by ground truth to impact the downstream tasks. In response, we introduce a novel end-to-end semi-supervised framework, Semi4TSF, for time series forecasting (TSF). It optimizes unsupervised loss on massive unlabeled data and integrates supervised contrastive and forecasting losses on limited labeled data, enabling the model to see other unlabeled embeddings meanwhile learning useful labeled embeddings, improving generalization. The three losses are jointly to refine the encoder and forecaster. Specifically, the unsupervised learning module applies two instance-wise augmentation banks over the entire series to capture long-term dependencies, suggests a learnable Fourier layer, and fuses temporal and frequency information to uncover intricate temporal-frequency correlations through cross-domain interactions to capture nuanced representations. Extensive experiments on five benchmarks demonstrate that Semi4TSF is an effective and superior end-to-end framework that fills the gap in semi-supervised TSF.

Map matching aims to align GPS trajectories to their actual travel routes on a road network, which is an essential pre-processing task for most of trajectory-based applications. Many map matching approaches utilize Hidden Markov Model (HMM) as their backbones. Typically, HMM treats GPS samples of a trajectory as observations and nearby road segments as hidden states. During map matching, HMM determines candidate states for each observation with a fixed searching range, and computes the most likely travel route using theViterbialgorithm. Although HMM-based approaches can derive high matching accuracy, they still suffer from high computation overheads. By inspecting the HMM process, we find that the computation bottleneck mainly comes from improper candidate sets, which contain many irrelevant candidates and incur unnecessary computations. In this paper, we present$\mathtt {LiMM}$– a learned road network index structure for efficient map matching.$\mathtt {LiMM}$improves existing HMM-based approaches from two aspects. First, we propose a novel learned index for road networks, which considers the characteristics of road data. Second, we devise an adaptive searching range mechanism to dynamically adjust the searching range for GPS samples based on their locations. As a result,$\mathtt {LiMM}$can provide refined candidate sets for GPS samples and thus accelerate the map matching process. Extensive experiments are conducted with three large real-world GPS trajectory datasets. The results demonstrate that$\mathtt {LiMM}$significantly reduces computation overheads by achieving an average speedup of$11.7\times$than baseline methods, merely with a subtle accuracy loss of 1.8%.

Learning semantic-rich representations from unlabeled time series data with intricate dynamics is a notable challenge. Traditional contrastive learning techniques predominantly focus on segment-level augmentations through time slicing, a practice that, while valuable, often results in sampling bias and suboptimal performance due to the loss of global context. Furthermore, they typically disregard the vital frequency information to enrich data representations. To this end, we propose a novel self-supervised general-purpose framework called Temporal-Frequency and Contextual Consistency (TFCC). Specifically, this framework first performs two instance-level augmentation families over the entire series to capture nuanced representations alongside critical long-term dependencies. Then, TFCC advances by initiating dual cross-view forecasting tasks between the original series and its augmented counterpart in both time and frequency domains to learn robust representations. Finally, three specially designed consistency modules 'temporal, frequency, and temporal-frequency' aid in further developing discriminative representations on top of the learned robust representations. Extensive experiments on multiple benchmarks demonstrate TFCC's superiority over the state-of-the-art classification and forecasting methods and exhibit exceptional efficiency in semi-supervised and transfer learning scenarios.

Arbitrary style transfer (AST) transfers arbitrary artistic styles onto content images. Despite the recent rapid progress, existing AST methods are either incapable or too slow to run at ultra-resolutions (e.g., 4K) with limited resources, which heavily hinders their further applications. In this paper, we tackle this dilemma by learning a straightforward and lightweight model, dubbed MicroAST. The key insight is to completely abandon the use of cumbersome pre-trained Deep Convolutional Neural Networks (e.g., VGG) at inference. Instead, we design two micro encoders (content and style encoders) and one micro decoder for style transfer. The content encoder aims at extracting the main structure of the content image. The style encoder, coupled with a modulator, encodes the style image into learnable dual-modulation signals that modulate both intermediate features and convolutional filters of the decoder, thus injecting more sophisticated and flexible style signals to guide the stylizations. In addition, to boost the ability of the style encoder to extract more distinct and representative style signals, we also introduce a new style signal contrastive loss in our model. Compared to the state of the art, our MicroAST not only produces visually superior results but also is 5-73 times smaller and 6-18 times faster, for the first time enabling super-fast (about 0.5 seconds) AST at 4K ultra-resolutions.

This paper presents a new adversarial training framework for image inpainting with segmentation confusion adversarial training (SCAT) and contrastive learning. SCAT plays an adversarial game between an inpainting generator and a segmentation network, which provides pixel-level local training signals and can adapt to images with free-form holes. By combining SCAT with standard global adversarial training, the new adversarial training framework exhibits the following three advantages simultaneously: (1) the global consistency of the repaired image, (2) the local fine texture details of the repaired image, and (3) the flexibility of handling images with free-form holes. Moreover, we propose the textural and semantic contrastive learning losses to stabilize and improve our inpainting model's training by exploiting the feature representation space of the discriminator, in which the inpainting images are pulled closer to the ground truth images but pushed farther from the corrupted images. The proposed contrastive losses better guide the repaired images to move from the corrupted image data points to the real image data points in the feature representation space, resulting in more realistic completed images. We conduct extensive experiments on two benchmark datasets, demonstrating our model's effectiveness and superiority both qualitatively and quantitatively.

This paper presents a new task for CLIP-based reference-guided facial image semantic transfer: the source facial image is translated to the output image with the high-level semantic attributes from the reference image while maintaining identity preservation. To this end, we employ the powerful generative capability of StyleGAN generator and the rich semantic knowledge of CLIP encoder to accomplish such a task. Additionally, a novel contrastive loss is designed to comprehensively explore the rich semantic information of CLIP for facial semantic concepts. This loss guides the semantic transfer toward desired directions from different perspectives in the pre-defined CLIP space. Besides, a simple yet effective semantic-preserved modulation module is proposed to explicitly map CLIP embeddings of reference image to the latent space. Experiments demonstrate that our approach achieves realistic facial image semantic transfer driven by reference images with various facial semantics.

Recently proposed LaMa [25] introduce Fast Fourier Convolution (FFC) [4] into image inpainting. FFC empowers the fully convolutional network to have a global receptive field in its early layers, and have the ability to produce robust repeating texture. However, LaMa has difficulty in generating clear and sharp complex content. In this paper, we analyze the fundamental flaws of using FFC in image inpainting, which are 1) spectrum shifting, 2) unexpected spatial activation, and 3) limited frequency receptive field. Such flaws make FFC-based inpainting framework difficult in generating complicated texture and performing faithful reconstruction. Based on the above analysis, we propose a novel Unbiased Fast Fourier Convolution (UFFC) module. UFFC is constructed by modifying the vanilla FFC module with 1) range transform and inverse transform, 2) absolute position embedding, 3) dynamic skip connection, and 4) adaptive clip, to overcome the above flaws. UFFC captures frequency information efficiently and realize reconstruction without introducing additional artifacts, achieving better inpainting results and more efficient training. In addition, we propose two novel perceptual losses for better generation quality and more robust training. Extensive experiments on several benchmark datasets demonstrate the effectiveness of our method, outperforming the state-of-the-art methods in both texture-capturing ability and expressiveness.

Text-driven 3D style transfer aims at stylizing a scene according to the text and generating arbitrary novel views with consistency. Simply combining image/video style transfer methods and novel view synthesis methods results in flickering when changing viewpoints, while existing 3D style transfer methods learn styles from images instead of texts. To address this problem, we for the first time design an efficient text-driven model for 3D style transfer, named TeSTNeRF, to stylize the scene using texts via cross-modal learning: we leverage an advanced text encoder to embed the texts in order to control 3D style transfer and align the input text and output stylized images in latent space. Furthermore, to obtain better visual results, we introduce style supervision, learning feature statistics from style images and utilizing 2D stylization results to rectify abrupt color spill. Extensive experiments demonstrate that TeSTNeRF significantly outperforms existing methods and provides a new way to guide 3D style transfer.

In this paper, we present the texture reformer, a fast and universal neural-based framework for interactive texture transfer with user-specified guidance. The challenges lie in three aspects: 1) the diversity of tasks, 2) the simplicity of guidance maps, and 3) the execution efficiency. To address these challenges, our key idea is to use a novel feed-forward multi-view and multi-stage synthesis procedure consisting of I) a global view structure alignment stage, II) a local view texture refinement stage, and III) a holistic effect enhancement stage to synthesize high-quality results with coherent structures and fine texture details in a coarse-to-fine fashion. In addition, we also introduce a novel learning-free view-specific texture reformation (VSTR) operation with a new semantic map guidance strategy to achieve more accurate semantic-guided and structure-preserved texture transfer. The experimental results on a variety of application scenarios demonstrate the effectiveness and superiority of our framework. And compared with the state-of-the-art interactive texture transfer algorithms, it not only achieves higher quality results but, more remarkably, also is 2-5 orders of magnitude faster.

Gram-based and patch-based approaches are two important research lines of style transfer. Recent diversified Gram-based methods have been able to produce multiple and diverse stylized outputs for the same content and style images. However, as another widespread research interest, the diversity of patch-based methods remains challenging due to the stereotyped style swapping process based on nearest patch matching. To resolve this dilemma, in this paper, we dive into the crux of existing patch-based methods and propose a universal and efficient module, termed DivSwapper, for diversified patch-based arbitrary style transfer. The key insight is to use an essential intuition that neural patches with higher activation values could contribute more to diversity. Our DivSwapper is plug-and-play and can be easily integrated into existing patch-based and Gram-based methods to generate diverse results for arbitrary styles. We conduct theoretical analyses and extensive experiments to demonstrate the effectiveness of our method, and compared with state-of-the-art algorithms, it shows superiority in diversity, quality, and efficiency.

Artistic style transfer is the task of synthesizing content images with learned artistic styles. Recent studies have shown the potential of Generative Adversarial Networks (GANs) for producing artistically rich stylizations. Despite the promising results, they usually fail to control the generated images' style degree, which is inflexible and limits their applicability for practical use. To address the issue, in this paper, we propose a novel method that for the first time allows adjusting the style degree for existing GAN-based artistic style transfer frameworks in real time after training. Our method introduces two novel modules into existing GAN-based artistic style transfer frameworks: a Style Scaling Injection (SSI) module and a Style Degree Interpretation (SDI) module. The SSI module accepts the value of Style Degree Factor (SDF) as the input and outputs parameters that scale the feature activations in existing models, offering control signals to alter the style degrees of the stylizations. And the SDI module interprets the output probabilities of a multi-scale content-style binary classifier as the style degrees, providing a mechanism to parameterize the style degree of the stylizations. Moreover, we show that after training our method can enable existing GAN-based frameworks to produce over-stylizations. The proposed method can facilitate many existing GAN-based artistic style transfer frameworks with marginal extra training overheads and modifications. Extensive qualitative evaluations on two typical GAN-based style transfer models demonstrate the effectiveness of the proposed method for gaining style degree control for them.

Recent studies have shown remarkable success in universal style transfer which transfers arbitrary visual styles to content images. However, existing approaches suffer from the aesthetic-unrealistic problem that introduces disharmonious patterns and evident artifacts, making the results easy to spot from real paintings. To address this limitation, we propose AesUST, a novel Aesthetic-enhanced Universal Style Transfer approach that can generate aesthetically more realistic and pleasing results for arbitrary styles. Specifically, our approach introduces an aesthetic discriminator to learn the universal human-delightful aesthetic features from a large corpus of artist-created paintings. Then, the aesthetic features are incorporated to enhance the style transfer process via a novel Aesthetic-aware Style-Attention (AesSA) module. Such an AesSA module enables our AesUST to efficiently and flexibly integrate the style patterns according to the global aesthetic channel distribution of the style image and the local semantic spatial distribution of the content image. Moreover, we also develop a new two-stage transfer training strategy with two aesthetic regularizations to train our model more effectively, further improving stylization performance. Extensive experiments and user studies demonstrate that our approach synthesizes aesthetically more harmonious and realistic results than state of the art, greatly narrowing the disparity with real artist-created paintings. Our code is available at https://github.com/EndyWon/AesUST.

Deep learning has proven to be an effective mechanism for computer vision tasks, especially for image denoising and burst image denoising. In this paper, we focus on solving the burst image denoising problem and aim to generate a single clean image from a burst of noisy images. We propose to combine the power of block matching and 3D filtering (BM3D) and a convolutional neural network (CNN) for burst image denoising. In particular, we design a CNN with a divide-and-conquer strategy. First, we employ BM3D to preprocess the noisy burst images. Then, the preprocessed images and noisy images are fed separately into two parallel CNN branches. The two branches produce somewhat different results. Finally, we use a light CNN block to combine the two outputs. In addition, we improve the performance by optimizing the two branches using two different constraints: a signal constraint and a noise constraint. One maps a clean signal, and the other maps the noise distribution. In addition, we adopt block matching in the network to avoid frame misalignment. Experimental results on synthetic and real noisy images show that our algorithm is competitive with other algorithms.

Artistic style transfer is an image editing task that aims at repainting everyday photographs with learned artistic styles. Existing methods learn styles from either a single style example or a collection of artworks. Accordingly, the stylization results are either inferior in visual quality or limited in style controllability. To tackle this problem, we propose a novel Dual Style-Learning Artistic Style Transfer (DualAST) framework to learn simultaneously both the holistic artist-style (from a collection of artworks) and the specific artwork-style (from a single style image): the artist-style sets the tone (i.e., the overall feeling) for the stylized image, while the artwork-style determines the details of the stylized image, such as color and texture. Moreover, we introduce a Style-Control Block (SCB) to adjust the styles of generated images with a set of learnable style-control factors. We conduct extensive experiments to evaluate the performance of the proposed framework, the results of which confirm the superiority of our method.

Image style transfer aims to transfer the styles of artworks onto arbitrary photographs to create novel artistic images. Although style transfer is inherently an underdetermined problem, existing approaches usually assume a deterministic solution, thus failing to capture the full distribution of possible outputs. To address this limitation, we propose a Diverse Image Style Transfer (DIST) framework which achieves significant diversity by enforcing an invertible cross-space mapping. Specifically, the framework consists of three branches: disentanglement branch, inverse branch, and stylization branch. Among them, the disentanglement branch factorizes artworks into content space and style space; the inverse branch encourages the invertible mapping between the latent space of input noise vectors and the style space of generated artistic images; the stylization branch renders the input content image with the style of an artist. Armed with these three branches, our approach is able to synthesize significantly diverse stylized images without loss of quality. We conduct extensive experiments and comparisons to evaluate our approach qualitatively and quantitatively. The experimental results demonstrate the effectiveness of our method.

Although existing artistic style transfer methods have achieved significant improvement with deep neural networks, they still suffer from artifacts such as disharmonious colors and repetitive patterns. Motivated by this, we propose an internal-external style transfer method with two contrastive losses. Specifically, we utilize internal statistics of a single style image to determine the colors and texture patterns of the stylized image, and in the meantime, we leverage the external information of the large-scale style dataset to learn the human-aware style information, which makes the color distributions and texture patterns in the stylized image more reasonable and harmonious. In addition, we argue that existing style transfer methods only consider the content-to-stylization and style-to-stylization relations, neglecting the stylization-to-stylization relations. To address this issue, we introduce two contrastive losses, which pull the multiple stylization embeddings closer to each other when they share the same content or style, but push far away otherwise. We conduct extensive experiments, showing that our proposed method can not only produce visually more harmonious and satisfying artistic images, but also promote the stability and consistency of rendered video clips.

Image style transfer is an underdetermined problem, where a large number of solutions can satisfy the same constraint (the content and style). Although there have been some efforts to improve the diversity of style transfer by introducing an alternative diversity loss, they have restricted generalization, limited diversity and poor scalability. In this paper, we tackle these limitations and propose a simple yet effective method for diversified arbitrary style transfer. The key idea of our method is an operation called deep feature perturbation (DFP), which uses an orthogonal random noise matrix to perturb the deep image feature maps while keeping the original style information unchanged. Our DFP operation can be easily integrated into many existing WCT (whitening and coloring transform)-based methods, and empower them to generate diverse results for arbitrary styles. Experimental results demonstrate that this learning-free and universal method can greatly increase the diversity while maintaining the quality of stylization.

Although existing image inpainting approaches have been able to produce visually realistic and semantically correct results, they produce only one result for each masked input. In order to produce multiple and diverse reasonable solutions, we present Unsupervised Cross-space Translation Generative Adversarial Network (called UCTGAN) which mainly consists of three network modules: conditional encoder module, manifold projection module and generation module. The manifold projection module and the generation module are combined to learn one-to-one image mapping between two spaces in an unsupervised way by projecting instance image space and conditional completion image space into common low-dimensional manifold space, which can greatly improve the diversity of the repaired samples. For understanding of global information, we also introduce a new cross semantic attention layer that exploits the long-range dependencies between the known parts and the completed parts, which can improve realism and appearance consistency of repaired samples. Extensive experiments on various datasets such as CelebA-HQ, Places2, Paris Street View and ImageNet clearly demonstrate that our method not only generates diverse inpainting solutions from the same image to be repaired, but also has high image quality.

The image generation model based on generative adversarial networks has recently received significant attention and can produce diverse, sharp, and realistic images. However, generating high-resolution images has long been a challenge. In this paper, we propose a progressive spatial recursive adversarial expansion model(called SpatialGAN) capable of producing high-quality samples of the natural image. Our approach uses a cascade of convolutional networks to progressively generate images in a part-to-whole fashion. At each level of spatial expansion, a separate image-to-image spatial adversarial expansion network (conditional GAN) is recursively trained based on context image generated by previous GAN or CGAN. Unlike other coarse-to-fine generative methods that constraint on generative process either by multi-scale resolution or by hierarchical feature, the SpatialGAN decomposes image space into multiple subspaces and gradually resolves uncertainties in the local-to-whole generative process. The SpatialGAN greatly stabilizes and speeds up the training, which allows us to produce images of high quality. Based on visual Inception Score and Fréchet Inception Distance, we demonstrate that the quality of images generated by SpatialGAN on several typical datasets is better than that of images generated by GANs without cascading and comparative with the state of art methods with cascading.

Existing style transfer methods have achieved great success in artwork generation by transferring artistic styles onto everyday photographs while keeping their contents unchanged. Despite this success, these methods have one inherent limitation: they cannot produce newly created image contents, lacking creativity and flexibility. On the other hand, generative adversarial networks (GANs) can synthesise images with new content, whereas cannot specify the artistic style of these images. The authors consider combining style transfer with convolutional GANs to generate more creative and diverse artworks. Instead of simply concatenating these two networks: the first for synthesising new content and the second for transferring artistic styles, which is inefficient and inconvenient, they design an end‐to‐end network called ArtistGAN to perform these two operations at the same time and achieve visually better results. Moreover, to generate images of higher quality, they propose the bi‐discriminator GAN containing a pixel discriminator and a feature discriminator that constrain the generated image from pixel level and feature level, respectively. They conduct extensive experiments and comparisons to evaluate their methods quantitatively and qualitatively. The experimental results verify the effectiveness of their methods.

Recent studies using deep neural networks have shown remarkable success in style transfer, especially for artistic and photo‐realistic images. However, these methods cannot solve more sophisticated problems. The approaches using global statistics fail to capture small, intricate textures and maintain correct texture scales of the artworks, and the others based on local patches are defective on global effect. To address these issues, this study presents a unified model [global and local style network (GLStyleNet)] to achieve exquisite style transfer with higher quality. Specifically, a simple yet effective perceptual loss is proposed to consider the information of global semantic‐level structure, local patch‐level style, and global channel‐level effect at the same time. This could help transfer not just large‐scale, obvious style cues but also subtle, exquisite ones, and dramatically improve the quality of style transfer. Besides, the authors introduce a novel deep pyramid feature fusion module to provide a more flexible style expression and a more efficient transfer process. This could help retain both high‐frequency pixel information and low‐frequency construct information. They demonstrate the effectiveness and superiority of their approach on numerous style transfer tasks, especially the Chinese ancient painting style transfer. Experimental results indicate that their unified approach improves image style transfer quality over previous state‐of‐the‐art methods.

Under the perspective projection assumption, non-Lambertian photometric stereo is a highly non-linear problem. In this study, we present an optimized framework for reconstructing the surface normal and depth with non-Lambertian reflection models under perspective projection. By decomposing the images into diffuse and specular components, we compute the surface normal and reflectance simultaneously. We also propose a variational formulation that is robust and useful for surface reconstruction. The experiments show that our method accurately reconstructs both the surface shape and reflectance of colorful objects with non-Lambertian surfaces.

Generally, the distributed bundle adjustment (DBA) method uses multiple worker nodes to solve the bundle adjustment problems and overcomes the computation and memory storage limitations of a single computer. However, the performance considerably degrades owing to the overhead introduced by the additional block partitioning step and synchronous waiting. Therefore, we propose a low-overhead consensus framework. A partial barrier based asynchronous method is proposed to early achieve consensus with respect to the faster worker nodes to avoid waiting for the slower ones. A scene summarization procedure is designed and integrated into the block partitioning step to ensure that clustering can be performed on the small summarized scene. Experiments conducted on public datasets show that our method can improve the worker node utilization rate and reduce the block partitioning time. Also, sample applications are demonstrated using our large-scale culture heritage datasets.

Neighbor discovery is important for docking applications, where mobile nodes communicate with static nodes situated at various rendezvous points. Among the existing neighbor discovery protocols, the probabilistic methods perform well in average cases but they have aperiodic, unpredictable, and unbounded discovery latency. Yet, deterministic protocols can provide bounded worst-case discovery latency by sacrificing the average-case performance. In this study, we propose a mobility-assisted slot index synchronization (MASS), which is a new synchronization technique that can improve the average-case performance of deterministic neighbor discovery protocols via slot index synchronization without incurring additional energy consumption. Furthermore, we propose an optimized beacon strategy in MASS to mitigate beaconing collisions, which can lead to discovery failures in situations where multiple neighbors are in the vicinity. We evaluate MASS with theoretical analysis and simulations using real traces from a tourist tracking system deployed at the Mogao Grottoes, which is a famous cultural heritage site in China. We show that MASS can reduce the average discovery latency of state-of-the-art deterministic neighbor discovery protocols by up to two orders of magnitude.

Duty-cycling is generally adopted in existing sensor networks to reduce power consumption and these networks depend on neighbor discovery protocols to ensure that nodes wake up and discover each other. For different neighbor discovery protocols, the discovery latency is determined by two factors: the wake-sleep pattern and slot size. To the best of our knowledge, previous works on neighbor discovery have thus far been focused on improving the wake-sleep pattern. In this paper, we investigate the extent to which we can improve discovery latency by reducing the slot size. We found that by reducing the slot size, i.e., reducing the listening time in active slots, the collisions between beacons and synchronization between nodes become more severe, which can lead to discovery failures that are not predicted by existing theoretical models. We show that we can mitigate these effects by reducing the number of beacons and introducing randomization. We propose a new continuous-listening-based neighbor discovery algorithm called Spotlight. Our evaluations with a practical sensor testbed suggest that Spotlight can achieve a 50% reduction in discovery latency over existing state-of-the-art neighbor discovery protocols without increasing power consumption in existing sensor networks.

To speed up the reconstruction of 3D dynamic scenes in an ordinary hardware platform, we propose an efficient framework to reconstruct 3D dynamic objects using a multiscale-contour-based interpolation from multi-view videos. Our framework takes full advantage of spatio-temporal-contour consistency. It exploits the property to interpolate single contours, two neighboring contours which belong to the same model, and two contours which belong to the same view at different times, corresponding to point-, contour-, and model-level interpolations, respectively. The framework formulates the interpolation of two models as point cloud transport rather than non-rigid surface deformation. Our framework speeds up the reconstruction of a dynamic scene while improving the accuracy of point-pairing which is used to perform the interpolation. We obtain a higher frame rate, spatio-temporal-coherence, and a quasi-dense point cloud sequence with color information. Experiments with real data were conducted to test the efficiency of the framework.

This paper presents a transport traffic estimation method which leverages road network correlation and sparse traffic sampling via the compressive sensing technique. Through the investigation on a traffic data set of more than 4400 taxis from Shanghai city, China, we observe nontrivial traffic correlations among the traffic conditions of different road segments and derive a mathematical model to capture such relations. After mathematical manipulation, the models can be used to construct representation bases to sparsely represent the traffic conditions of all road segments in a road network. With the trait of sparse representation, we propose a traffic estimation approach that applies the compressive sensing technique to achieve a city-scale traffic estimation with only a small number of probe vehicles, largely reducing the system operating cost. To validate the traffic correlation model and estimation method, we do extensive trace-driven experiments with real-world traffic data. The results show that the model effectively reveals the hidden structure of traffic correlations. The proposed estimation method derives accurate traffic conditions with the average accuracy as 0.80, calculated as the ratio between the number of correct traffic state category estimations and the number of all estimation times, based on only 50 probe vehicles' intervention, which significantly outperforms the state-of-the-art methods in both cost and traffic estimation accuracy.

This paper presents CSPR, a compressive-sensing-based approach for path reconstruction in wireless sensor networks. By viewing the whole network as a path representation space, an arbitrary routing path can be represented by a path vector in the space. As path length is usually much smaller than the network size, such path vectors are sparse, i.e., the majority of elements are zeros. By encoding sparse path representation into packets, the path vector (and thus the represented routing path) can be recovered from a small amount of packets using compressive sensing technique. CSPR formalizes the sparse path representation and enables accurate and efficient per-packet path reconstruction. CSPR is invulnerable to network dynamics and lossy links due to its distinct design. A set of optimization techniques is further proposed to improve the design. We evaluate CSPR in both testbed-based experiments and large-scale trace-driven simulations. Evaluation results show that CSPR achieves high path recovery accuracy (i.e., 100% and 96% in experiments and simulations, respectively) and outperforms the state-of-the-art approaches in various network settings.

Neighbor discovery is essential for docking applications, where mobile nodes communicate with static nodes situated at various rendezvous points. In existing neighbor discovery protocols, the probabilistic protocols perform well in the average-case but have a periodic, unpredictable and unbounded discovery latency. While the deterministic protocols can provide a bounded worst-case discovery latency, they achieve this by sacrificing the average-case performance. In this paper, we propose a new synchronization technique, called Mobility-Assisted Slot index Synchronization (MASS). MASS improves the average-case performance of deterministic neighbor discovery protocols via slot index synchronization, without incurring additional energy consumption. We evaluate MASS through both theoretical analysis and simulations of the real traces from a tourist tracking system deployed at Mogao Grottoes, a famous cultural heritage site in China. We show that MASS can reduce the average discovery latency of state-of-the-art deterministic neighbor discovery protocols by up to 2 orders of magnitude.

Abstract Many works focus on multi‐spectral capture and analysis, but multi‐spectral display still remains a challenge. Most prior works on multi‐primary displays use ad‐hoc narrow band primaries that assure a larger color gamut, but cannot assure a good spectral reproduction. Content‐dependent spectral analysis is the only way to produce good spectral reproduction, but cannot be applied to general data sets. Wide primaries are better suited for assuring good spectral reproduction due to greater coverage of the spectral range, but have not been explored much. In this paper we explore the use of wide band primaries for accurate spectral reproduction for the first time and present the first content‐independent multi‐spectral display achieved using superimposed projections with modified wide band primaries. We present a content‐independent primary selection method that selects a small set of n primaries from a large set of m candidate primaries where m > n. Our primary selection method chooses primaries with complete coverage of the range of visible wavelength (for good spectral reproduction accuracy), low interdependency (to limit the primaries to a small number) and higher light throughput (for higher light efficiency). Once the primaries are selected, the input values of the different primary channels to generate a desired spectrum are computed using an optimization method that minimizes spectral mismatch while maximizing visual quality. We implement a real prototype of multi‐spectral display consisting of 9‐primaries using three modified conventional 3‐primary projectors, and compare it with a conventional display to demonstrate its superior performance. Experiments show our display is capable of providing large gamut assuring a good visual appearance while displaying any multi‐spectral images at a high spectral accuracy.

Abstract In this paper, we propose a highly accurate inpainting algorithm which reconstructs an image from a fraction of its pixels. Our algorithm is inspired by the recent progress of non‐local image processing techniques following the idea of ‘grouping and collaborative filtering’. In our framework, we first match and group similar patches in the input image, and then convert the problem of estimating missing values for the stack of matched patches to the problem of low‐rank matrix completion, and finally obtain the result by synthesizing all the restored patches. In our algorithm, how to accurately perform patch matching process and solve the low‐rank matrix completion problem are key points. For the first problem, we propose a robust patch matching approach, and for the second task, the alternating direction method of multipliers is employed. Experiments show that our algorithm has superior advantages over existing inpainting techniques. Besides, our algorithm can be easily extended to handle practical applications including rendering acceleration, photo restoration and object removal .

This paper presents CSPR, a compressive sensing based approach for path reconstruction in wireless sensor networks. By viewing the whole network as a path representation space, an arbitrary routing path can be represented by a path vector in the space. As path length is usually much smaller than the network size, such path vectors are sparse, i.e., the majority of elements are zeros. By encoding sparse path representation into packets, the path vector (and thus the represented path) can be recovered from a small amount of packets using compressive sensing technique. CSPR formalizes the sparse path representation and enables accurate and efficient per-packet path reconstruction. CSPR is invulnerable to network dynamics and lossy links due to its distinct design. A set of optimization techniques are further proposed to improve the design. We evaluate CSPR in both testbed-based experiments and large-scale trace-driven simulations. Evaluation results show that CSPR achieves high path recovery accuracy (i.e., 100% and 96% in experiments and simulations, respectively), and outperforms the state-of-the-art approaches in various network settings.

Real-world, long-running wireless sensor networks (WSNs) require intense user intervention in the development, hardware testing, deployment, and maintenance stages. A majority of network design is network centric and focuses primarily on network performance, for example, efficient sensing and reliable data delivery. Although several tools have been developed to assist debugging and fault diagnosis, it is yet to systematically examine the underlying heavy burden that users face throughout the lifetime of WSNs. In this article, we propose a general Multimode user-CentriC (MC 2 ) framework that can, with simple user inputs, adjust itself to assist user operation and thus reduce the users' burden at various stages. In particular, we have identified utilities that are essential at each stage and grouped them into modes . In each mode, only the corresponding utilities will be loaded, and modes can be easily switched using the customized MC 2 sensor platform. As such, we reduce the runtime interference between various utilities and simplify their development as well as their debugging. We validated our MC 2 software and the sensor platform in a long-lived microclimate monitoring system deployed at a wildland heritage site, Mogao Grottoes. In our current system, 241 sensor nodes have been deployed in 57 caves, and the network has been running for over five years. Our experimental validation shows that the MC 2 framework shortens the time for network deployment and maintenance, and makes network maintenance doable by field experts (in our case, historians).

Grouping sensor nodes with similar readings into the same cluster and scheduling them to alternately report their sensing readings is an effective and efficient method to perform approximate data collection, which exploits the tradeoff between data quality and energy consumption. However, to partition all sensor nodes into exclusive clusters while incurring as less communication overhead as possible in a distributed manner is a challenging task. In this paper, by exploiting inherent spatial and data correlation in wireless sensor network, we have proposed the distributed spatial correlation-based clustering algorithm to complete this tough mission. With nodes information exchange in certain region and the novel ranking strategy, our clustering algorithm can terminate in a small number of iterations. Extensive simulations show that the proposed algorithm outperforms three other noteworthy clustering algorithms, namely EEDC, ASAP and DClocal, on some key metrics, such as number of clusters, energy consumption for clustering, average dissimilarity of clusters and residual energy level of cluster heads.

The surface-based approaches modeling from silhouettes are a popular and useful topic. Intersecting operator of line segments is an essential and crucial operation in these approaches. It consists of two phases: intersecting operator in 2D and overlap testing of the set of line segments in real domain. The former has made considerable progresses while the latter remains insignificant status when there are complex objects in the scene. In order to accelerate the speed of reconstructing visual hulls, this paper addresses the problem and proposes a novel approach based on contribution weights which reflect viewing ray's contributions to the visual hulls. Experimental results demonstrate that the novel approach can attain the promise.

This paper introduces a novel, efficient and accurate algorithm for solving the image completion problem, which is a widely discussed topic in recent years. This algorithm is inspired by recent progress in the matrix completion field. Firstly, corresponding to the RGB channels, an input incomplete image is treated as three incomplete matrices. Then each matrix is recovered by solving a convex relaxation problem using the nuclear norm. In this process, an alternating direction method (ADM) is employed, which is quite easily implementable and able to achieve an optimal solution for the problem efficiently. The experiments show that our methodology is fast and effective and the proposed algorithm is able to handle various applications, such as image restoration, object removal and image denoising.

We present a fast perceptual radiometric compensation method for inter-reflection in immersive projection environment. Radiometric compensation is the inverse process of light transport. As light transport process can be described by a matrix-vector multiplication equation, radiometric compensation for inter-reflection can be achieved by solving the equation to get the vector, during which matrix inversion should be computed. As the dimensions of the matrix are equivalent to the resolution of images, such matrix inversion is both time and storage consuming. Unlike previous methods, our method adopts projector-camera system to simulate the inversion, and treats the compensation as a non-linear optimization problem which is formulated from full light transport matrix and non-linear color space conversion. To make physical multiplication simulation more practical, the method adjusts the range of projector-camera system adaptively and reduces the high-frequency errors caused by clipping error and measured error to make the compensated results smoother. We implement an immersive projection display prototype. The experiments show that our method achieves better results compared with the previous method.

With the prevalence of web services, services orchestration evolves from intra-enterprise integration to cross-domain integration and involves multiple networks. In traditional centralized SOA, the execution engine suffers from single point of failure and performance of composite services will be affected by the network conditions because of cross domain message transmissions. In this paper, we present a decentralized orchestration model for cross-domain multiple networks environment. Based on this model, we propose a Genetic Algorithm to find an optimization deployment solution for component services with minimum cross-domain network cost and thus increase the performance of composite services.

In this work, we will study the problem of applying natural neighbor interpolation in real-time application environments. First, we will see how this method can be used in time critical environments by adopting some fast algorithms for locating points and computing function values. Then, we will build a framework for testing both the time efficiency and the value accuracy of this interpolation method applied for the electromagnetic tracker calibration problem. Experiment results show that this method will be a good candidate for real-time interpolation based applications.

Microclimates in many wildland cultural heritage sites are not under surveillance up to now, due to the lack of power supply and network access. However, accurate microclimate data in cultural heritage sites are very important for research and conservation. In this paper, we present a wireless sensor system for long-term microclimate monitoring in wildland cultural heritage sites, and its deployment at the Mogao Grottoes. The system integrates wireless sensor network (WSN), long-distance wireless polling network (LWPN) and Internet to fit the complex geography of the Mogao Grottoes. Both robust hardware and fault tolerance strategies have been developed to ensure long-term stable monitoring. The currently deployed system consists of 241 data sensors covering 57 typical caves. The reliability and long life-time of the system are verified through network and battery performance evaluations at the end of the paper.

In this paper, we addressed the problem of redundancy allocation for protecting packet loss for better quality of service (QoS) in real-time H.264 video streaming. A novel error-resilient approach is proposed for the transmission of pre-encoded H.264 video stream under bandwidth constrained networks. A novel frame importance model is derived for estimating relative importance index for different H.264 video frames. Combining with the characteristics of the network, the optimal resource allocation strategy for different video frames can be determined for achieving improved error resilience. The model uses frame error propagation index (FEPI) to characterize video quality degradation caused by error propagation in different frames in a GOP when suffer from packet loss. This model can be calculated in DCT domain with the parameters extracted directly from the bitstream. Therefore, the complexity of the proposed scheme is very low and much better for real-time video transmission. Simulation results show that the proposed scheme can improve the receiver side reconstructed video quality remarkably under different channel loss patterns.

In this paper, we propose a fast and reliable impulse noise filter for highly corrupted images. The median filter was once the most popular nonlinear filter for removing impulse noise because of its good denoising power and computational efficiency, but the performance are unsatisfactory when noise ratio is high. So many algorithms have been proposed to improve the filter result. Recently, A switching median filter with boundary discriminative noise detection(BDND) was proposed, it is very effective and outperforms all previously proposed median-based filters, but it's very time-consuming in calculation. The proposed filter use similar scheme as in BDND, it is also very effective and much more efficient than BDND. The proposed method use a new powerful and efficient noise detection method to determine if current pixel has been corrupted by noise, if it's corrupted, a variable window median filter is used to attenuate impulse noise, while those uncorrupted pixels are remain unchanged. Results from computer simulations are used to demonstrate pleasing performance of our proposed method.

In this paper a computer-aided style stroke extraction system is developed. In this system all the objects in a "Dun Huang" fresco is classified into the stroke objects and the non-stroke objects. The styled strokes are extracted as a whole using an effective algorithm, which utilize the relationship between points and curves combined both the spatial information and the color information. While the edges of the non-stroke objects can be extracted by the state-of-the-art edge detection technique. All the extracted style strokes and the edges are formed the digital sketch of the fresco. This research has the potential to provide a computer aided tool for art historians to do imitation work, and improve the efficiency of imitation

Image segmentation has been the subject of considerable research activity over the last three decades. This paper proposes an algorithm for color image segmentation by region growing combined with the image enhancement based on the Bezier model. Using the Bezier curve model has a direct impact on the quality of the image, and enhances the boundary differences between the objects and their background making the image segmentation task easier. The segmentation starts from a seed in the form of 3*3 image blocks to avoid the noise point. It grows the region by adding adjacent pixels that are satisfied with the homogeneous criteria with the seed point, expand point and the growing region respectively. Using the perceptual color clustering, color images is quantized and the similar colors are classified based on NBS color distance. The experimental results illuminate that the algorithm the paper proposed is effective.

Cooperative work is becoming a comprehensive and effective product design method, and the application of multimedia and multiple design modes has made cooperative design system more natural and easier to use. By now, most of the cooperative design is specific-task oriented and carried out through a specific multimedia cooperative design tool. But with the improved requirements of agility and design efficiency of product design, more and more attention is paid to the design platform that supports multiple design tasks using multiple cooperative design tools. Based on the analysis of a comprehensive cooperative design model, this paper presents a framework supporting multimedia and multiple design modes. This framework is based on an open hierarchy, which is able to embed multiple cooperative design tools. And it also provides a series of cooperative design services. Then several key technologies of the framework are discussed: the intelligent agent based cooperative tool configuration technique, synchronous/asynchronous design mode control, consistency control of the cooperative design object, and at last the extensible multimedia organization model. Finally, this paper will present an application example of the framework, and then will evaluate the framework and draw a conclusion.

This paper introduces a virtual Dunhuang mural restoration system in collaborative network environment. It describes the style of Dunhuang mural, analyzes the reasons of mural spoilage, and presents the necessity to develop a collaborative mural restoration GroupWare. It describes the components and the workflow of mural restoration in detail, solves some key technologies in the system. In the end, it introduces the system architecture, and presents the system interface and some restored results.