Single-domain generalization (SDG) in time series classification (TSC) poses significant challenges for current time-series domain generalization methods due to the extremely limited data available from only one source domain. In this study, we propose Segment-dErived Expansion of Domains (SEED), a diffusion-based method that effectively expands domain diversity for SDG. We reveal that individual instances exhibit intrinsic temporal shifts over time, which provides a principled foundation for creating multiple pseudo domains by segmenting each instance into distinct parts. To do so, SEED extracts two complementary representations from each time-series segment: 1) a segment-specific representation that captures diverse distributional variations, and 2) a segment-invariant representation that preserves class semantics. SEED formulates these representations as pseudo-domain prompts to guide a diffusion model in generating diverse yet semantically consistent time-series data. Additionally, SEED introduces a novel prompt-fused sampling for diffusion, enabling flexible recombination of segment-specific features to continuously expand the pseudo-domain space. We provide both theoretical analysis and extensive empirical evaluations on four widely used TSC benchmarks to validate its ability in reducing generalization error and improving model's performances in SDG. In our experiments, SEED significantly improves classification accuracy by 7.68% on average compared to the strong baselines.

Existing domain generalization (DG) methods for cross-person sensor-based activity recognition tasks often struggle to capture both intra- and inter-domain style diversity, leading to significant domain gaps with the target domain. In this study, we explore a novel perspective to tackle this problem, a process conceptualized as domain padding. This proposal aims to enrich the domain diversity by synthesizing intra- and inter-domain style data while maintaining robustness to class labels. We instantiate this concept using a conditional diffusion model and introduce a style-fused sampling strategy to enhance data generation diversity, termed Diversified Intra- and Inter-domain distributions via activity Style-fused Diffusion modeling (DI2SDiff). In contrast to traditional condition-guided sampling, our style-fused sampling strategy allows for the flexible use of one or more random style representations from the same class to guide data synthesis. This feature presents a notable advancement: it allows for the maximum utilization of possible combinations among existing styles to generate a broad spectrum of new style instances. We further extend DI2SDiff into DI2SDiff++ by enhancing the diversity of style guidance. Specifically, DI2SDiff++ integrates a multi-head style conditioner to provide multiple distinct, decomposed substyles and introduces a substyle-fused sampling strategy that allows cross-class substyle fusion for broader guidance. Empirical evaluations on a wide range of datasets demonstrate that our generated data achieves remarkable diversity within the domain space. Both intra- and inter-domain generated data have been proven significant and valuable, enabling DI2SDiff and DI2SDiff++ to surpass state-of-the-art DG methods in various cross-person activity recognition tasks.

Existing domain generalization (DG) methods for cross-person generalization tasks often face challenges in capturing intra- and inter-domain style diversity, resulting in domain gaps with the target domain. In this study, we explore a novel perspective to tackle this problem, a process conceptualized as domain padding. This proposal aims to enrich the domain diversity by synthesizing intra- and inter-domain style data while maintaining robustness to class labels. We instantiate this concept using a conditional diffusion model and introduce a style-fused sampling strategy to enhance data generation diversity. In contrast to traditional condition-guided sampling, our style-fused sampling strategy allows for the flexible use of one or more random styles to guide data synthesis. This feature presents a notable advancement: it allows for the maximum utilization of possible permutations and combinations among existing styles to generate a broad spectrum of new style instances. Empirical evaluations on a broad range of datasets demonstrate that our generated data achieves remarkable diversity within the domain space. Both intra- and inter-domain generated data have proven to be significant and valuable, contributing to varying degrees of performance enhancements. Notably, our approach outperforms state-of-the-art DG methods in all human activity recognition tasks.

The nearest neighbor search is an essential operation for many computer vision, data mining, and machine learning problems. Since it is so widely used, the nearest neighbor search should be as fast as possible. This paper explores lower bound-based approaches to speed up the exact nearest neighbor search in high dimensional Euclidean space. We compute the lower bound of Euclidean Distance by using the block vectors and Cauchy–Schwartz inequality. The proposed lower bound is calculated efficiently and is close to the real Euclidean Distance. Besides, the preprocessing step of the proposal has linear time complexity. Given a query, during the procedure of identifying the nearest neighbor, our method can eliminate many expensive actual distance computations using the lower bound to approximate Euclidean Distance. In addition, we develop a multilevel lower bound strategy, which calculates the lower bound step by step and utilizes the multistep filtering mechanism to improve the searching process further. Theoretical analysis is provided to show that the proposals can guarantee to obtain the same result as the brute-force search. Comprehensive experiments on 16 public data sets collected from various domains demonstrate that our approach performs well in finding the exact nearest neighbor compared to related competitors. The experimental results also illustrate that the multilevel lower bound strategy is effective.

The similarity search on sensor data generated by a myriad of sensing devices is a frequently encountered problem in the era of the Internet of Things (IoT). This sensor data generally appear in the form of time series, a temporally ordered sequence of real numbers obtained regularly in time. It has been widely accepted that the dynamic time warping (DTW) currently is the most prevalent similarity measure in the time-series mining community, mainly due to its flexibility and broad applicability. However, calculating DTW between two time series has quadratic time complexity, leading to unsatisfactory efficiency when performing the similarity search over the large time-series data set. The main contribution of this article is to propose a method called product quantization (PQ)-based DTW (PQDTW) for fast time-series approximate similarity search under DTW. The PQ, a well-known approximate nearest neighbor search approach, is used in PQDTW. Nevertheless, the conventional PQ is developed with the Euclidean distance and is not designed for DTW. To solve this problem, the DTW barycenter averaging (DBA) technique is utilized to adapt the PQ for DTW before using it. We employ PQDTW along with thefilter-and-refineframework to efficiently and accurately perform the time-series similarity search. Our method can reasonably reduce many DTW computations in the filtering phase; thus, the query process is accelerated. We compare PQDTW with related popular algorithms using public time-series data sets. Experimental results verify that the proposal achieves the best tradeoff between query efficiency and retrieval accuracy compared to the competitors.

Deep learning has proven to be an effective mechanism for computer vision tasks, especially for image denoising and burst image denoising. In this paper, we focus on solving the burst image denoising problem and aim to generate a single clean image from a burst of noisy images. We propose to combine the power of block matching and 3D filtering (BM3D) and a convolutional neural network (CNN) for burst image denoising. In particular, we design a CNN with a divide-and-conquer strategy. First, we employ BM3D to preprocess the noisy burst images. Then, the preprocessed images and noisy images are fed separately into two parallel CNN branches. The two branches produce somewhat different results. Finally, we use a light CNN block to combine the two outputs. In addition, we improve the performance by optimizing the two branches using two different constraints: a signal constraint and a noise constraint. One maps a clean signal, and the other maps the noise distribution. In addition, we adopt block matching in the network to avoid frame misalignment. Experimental results on synthetic and real noisy images show that our algorithm is competitive with other algorithms.

Time series similarity search is an essential operation in time series data mining and has received much higher interest along with the growing popularity of time series data. Although many algorithms to solve this problem have been investigated, there is a challenging demand for supporting similarity search in a fast and accurate way. In this paper, we present a novel approach, TS2BC, to perform time series similarity search efficiently and effectively. TS2BC uses binary code to represent time series and measures the similarity under the Hamming Distance. Our method is able to represent original data compactly and can handle shifted time series and work with time series of different lengths. Moreover, it can be performed with reasonably low complexity due to the efficiency of calculating the Hamming Distance. We extensively compare TS2BC with state-of-the-art algorithms in classification framework using 61 online datasets. Experimental results show that TS2BC achieves better or comparative performance than other the state-of-the-art in accuracy and is much faster than most existing algorithms. Furthermore, we propose an approximate version of TS2BC to speed up the query procedure and test its efficiency by experiment.

Under the perspective projection assumption, non-Lambertian photometric stereo is a highly non-linear problem. In this study, we present an optimized framework for reconstructing the surface normal and depth with non-Lambertian reflection models under perspective projection. By decomposing the images into diffuse and specular components, we compute the surface normal and reflectance simultaneously. We also propose a variational formulation that is robust and useful for surface reconstruction. The experiments show that our method accurately reconstructs both the surface shape and reflectance of colorful objects with non-Lambertian surfaces.

Generating an image from a given text description has two goals: visual realism and semantic consistency. Although significant progress has been made in generating high-quality and visually realistic images using generative adversarial networks, guaranteeing semantic consistency between the text description and visual content remains very challenging. In this paper, we address this problem by proposing a novel global-local attentive and semantic-preserving text-to-image-to-text framework called MirrorGAN. MirrorGAN exploits the idea of learning text-to-image generation by redescription and consists of three modules: a semantic text embedding module (STEM), a global-local collaborative attentive module for cascaded image generation (GLAM), and a semantic text regeneration and alignment module (STREAM). STEM generates word- and sentence-level embeddings. GLAM has a cascaded architecture for generating target images from coarse to fine scales, leveraging both local word attention and global sentence attention to progressively enhance the diversity and semantic consistency of the generated images. STREAM seeks to regenerate the text description from the generated image, which semantically aligns with the given text description. Thorough experiments on two public benchmark datasets demonstrate the superiority of MirrorGAN over other representative state-of-the-art methods.

Text-to-image generation, i.e. generating an image given a text description, is a very challenging task due to the significant semantic gap between the two domains. Humans, however, tackle this problem intelligently. We learn from diverse objects to form a solid prior about semantics, textures, colors, shapes, and layouts. Given a text description, we immediately imagine an overall visual impression using this prior and, based on this, we draw a picture by progressively adding more and more details. In this paper, and inspired by this process, we propose a novel text-to-image method called LeicaGAN to combine the above three phases in a unified framework. First, we formulate the multiple priors learning phase as a textual-visual co-embedding (TVE) comprising a text-image encoder for learning semantic, texture, and color priors and a text-mask encoder for learning shape and layout priors. Then, we formulate the imagination phase as multiple priors aggregation (MPA) by combining these complementary priors and adding noise for diversity. Lastly, we formulate the creation phase by using a cascaded attentive generator (CAG) to progressively draw a picture from coarse to fine. We leverage adversarial learning for LeicaGAN to enforce semantic consistency and visual realism. Thorough experiments on two public benchmark datasets demonstrate LeicaGAN's superiority over the baseline method. Code has been made available at https://github.com/qiaott/LeicaGAN.

Collecting a large-scale and well-annotated dataset for image processing has become a common practice in computer vision. However, in the ancient painting area, this task is not practical as the number of paintings is limited and their style is greatly diverse. We, therefore, propose a novel solution for the problems that come with ancient painting processing. This is to use domain transfer to convert ancient paintings to photo-realistic natural images. By doing so, the "ancient painting processing problems" become "natural image processing problems" and models trained on natural images can be directly applied to the transferred paintings. Specifically, we focus on Chinese ancient flower, bird and landscape paintings in this work. A novel Domain Style Transfer Network (DSTN) is proposed to transfer ancient paintings to natural images which employ a compound loss to ensure that the transferred paintings still maintain the color composition and content of the input paintings. The experiment results show that the transferred paintings generated by the DSTN have a better performance in both the human perceptual test and other image processing tasks than other state-of-arts methods, indicating the authenticity of the transferred paintings and the superiority of the proposed method.

Attention mechanisms have been widely applied in the Visual Question Answering (VQA) task, as they help to focus on the area-of-interest of both visual and textual information. To answer the questions correctly, the model needs to selectively target different areas of an image, which suggests that an attention-based model may benefit from an explicit attention supervision. In this work, we aim to address the problem of adding attention supervision to VQA models. Since there is a lack of human attention data, we first propose a Human Attention Network (HAN) to generate human-like attention maps, training on a recently released dataset called Human ATtention Dataset (VQA-HAT). Then, we apply the pre-trained HAN on the VQA v2.0 dataset to automatically produce the human-like attention maps for all image-question pairs. The generated human-like attention map dataset for the VQA v2.0 dataset is named as Human-Like ATtention (HLAT) dataset. Finally, we apply human-like attention supervision to an attention-based VQA model. The experiments show that adding human-like supervision yields a more accurate attention together with a better performance, showing a promising future for human-like attention supervision in VQA.

In order to retrieve unlabeled images by textual queries, cross-media similarity computation is a key ingredient. Although novel methods are continuously introduced, little has been done to evaluate these methods together with large-scale query log analysis. Consequently, how far have these methods brought us in answering real-user queries is unclear. Given baseline methods that use relatively simple text/image matching, how much progress have advanced models made is also unclear. This paper takes a pragmatic approach to answering the two questions. Queries are automatically categorized according to the proposed query visualness measure and later connected to the evaluation of multiple cross-media similarity models on three test sets. Such a connection reveals that the success of the state of the art is mainly attributed to their good performance on visual-oriented queries, which account for only a small part of real-user queries. To quantify the current progress, we propose a simple text2image method, representing a novel query by a set of images selected from large-scale query log. Consequently, computing cross-media similarity between the query and a given image boils down to comparing the visual similarity between the given image and the selected images. Image retrieval experiments on the challenging Clickture dataset show that the proposed text2image is a strong baseline, comparing favorably to recent deep learning alternatives.

How to estimate cross-media relevance between a given query and an unlabeled image is a key question in the MSR-Bing Image Retrieval Challenge. We answer the question by proposing cross-media relevance fusion, a conceptually simple framework that exploits the power of individual methods for cross-media relevance estimation. Four base cross-media relevance functions are investigated, and later combined by weights optimized on the development set. With DCG25 of 0.5200 on the test dataset, the proposed image retrieval system secures the first place in the evaluation.

Abstract In this paper, we propose a highly accurate inpainting algorithm which reconstructs an image from a fraction of its pixels. Our algorithm is inspired by the recent progress of non‐local image processing techniques following the idea of ‘grouping and collaborative filtering’. In our framework, we first match and group similar patches in the input image, and then convert the problem of estimating missing values for the stack of matched patches to the problem of low‐rank matrix completion, and finally obtain the result by synthesizing all the restored patches. In our algorithm, how to accurately perform patch matching process and solve the low‐rank matrix completion problem are key points. For the first problem, we propose a robust patch matching approach, and for the second task, the alternating direction method of multipliers is employed. Experiments show that our algorithm has superior advantages over existing inpainting techniques. Besides, our algorithm can be easily extended to handle practical applications including rendering acceleration, photo restoration and object removal .

This paper introduces a novel, efficient and accurate algorithm for solving the image completion problem, which is a widely discussed topic in recent years. This algorithm is inspired by recent progress in the matrix completion field. Firstly, corresponding to the RGB channels, an input incomplete image is treated as three incomplete matrices. Then each matrix is recovered by solving a convex relaxation problem using the nuclear norm. In this process, an alternating direction method (ADM) is employed, which is quite easily implementable and able to achieve an optimal solution for the problem efficiently. The experiments show that our methodology is fast and effective and the proposed algorithm is able to handle various applications, such as image restoration, object removal and image denoising.

Abstract In this paper, we present a practical framework to generate cartoon style animations of smoke, which consists of two components: a smoke simulator and a rendering system. In the simulation stage, the smoke is modelled as a set of smoothed particles and the physical parameters such as velocity and force are defined on particles directly. The smoke is rendered in flicker‐free cartoon style with two‐tone shading and silhouettes. Both the simulation and rendering are intuitive and easy to implement. In the most moderate scale scene, an impressive cartoon animation is generated with about a thousand particles at real‐time frame rate. Copyright © 2005 John Wiley & Sons, Ltd.

This study is to meet the demands of high realistic and efficient pattern application in textile CAD systems. A method to achieve photo effect by changing patterns in real scenes was given out. According to the scene surface texture trend, the area of the scene surface, which needs to obtain the pattern effect, was divided into control meshes. A mesh division and adjusting idea was developed. A vector operation based texture mapping counterpoint pixels attainting algorithm was presented, which could embody texture distortion demands. In order to achieve the immersion authenticity, a mapping pixels' brightness value synthesis method was put forward, which inherited the surface material characteristics. The validity of the method was confirmed by citing some typical examples. The method can create high realistic visual pattern effect. In the same time, its computational cost is relatively low and will have high efficiency. It is also convenient to implement.