To bridge the gap between artists and non-specialists, we present a unified framework, Neural-Polyptych, to facilitate the creation of expansive, high-resolution paintings by seamlessly incorporating interactive hand-drawn sketches with fragments from original paintings. We have designed a multi-scale GAN-based architecture to decompose the generation process into two parts, each responsible for identifying global and local features. To enhance the fidelity of semantic details generated from users' sketched outlines, we introduce a Correspondence Attention module utilizing our Reference Bank strategy. This ensures the creation of high-quality, intricately detailed elements within the artwork. The final result is achieved by carefully blending these local elements while preserving coherent global consistency. Consequently, this methodology enables the production of digital paintings at megapixel scale, accommodating diverse artistic expressions and enabling users to recreate content in a controlled manner. We validate our approach to diverse genres of both Eastern and Western paintings. Applications such as large painting extension, texture shuffling, genre switching, mural art restoration, and recomposition can be successfully based on our framework.

Halftoning is a time-honored printing technique that simulates continuous tones using ink dots (halftone dots). The resurgence of deep learning has catalyzed the emergence of innovative technologies in the printing industry, fostering the advancement of data-driven halftoning methods. Nevertheless, current deep learning-based approaches produce halftones through image-to-image black box transformations, lacking direct control over the movement of individual halftone dots. In this paper, we propose an innovative halftoning method termed “neural dot-controllable halftoning”. This method allows dot-level image dithering by providing direct control over the motion of each ink dot. We conceptualize halftoning as the process of sprinkling dots on a canvas. Initially, a specific quantity of dots are randomly dispersed on the canvas and subsequently adjusted based on the surrounding grayscale and gradient. To establish differentiable transformations between discrete ink dot positions and halftone matrices, we devise a lightweight dot encoding network to spread dense gradients to sparse dots. Dot control offers several advantages to our approach, including the capability to regulate the quantity of halftone dots and enhance specific areas with artifacts in the generated halftones by adjusting the placement of the dots. Our proposed method exhibits superior performance than previous approaches in extensive quantitative and qualitative experiments.

Crowd counting is widely used in real-time congestion monitoring and public security. Due to the limited data, many methods have little ability to be generalized because the differences between feature domains are not taken into consideration. We propose VGG-embedded adaptive layer normalization (VadaLN) to filter the features that irrelevant to the counting tasks in order that the counting results should not be affected by the image quality, color or illumination. VadaLN is implemented on the pretrained VGG-16 backbone. There is no additional learning parameters required through our method. VadaLN incoporates the proposed scale-shuffling modules (SSM) to relax the distortions in upsampling operations. Besides, nonaligned training methdology for the estimation of density maps is leveraged by an adversarial contextual loss (ACL) to improve the counting performance. Based on the proposed method, we construct an end-to-end trainable baseline model without bells and whistles, namely VadaLNet, which outperforms several recent state-of-the-art methods on commonly used challenging standard benchmarks. The intermediate scale-shuffled results are combined to formulate a scale-complementary strategy as a more powerful network, namely as VadaLNeSt. We implement VadaLNeSt on standard benchmarks, e.g. ShanghaiTech (Part A & Part B), UCF_CC_50, and UCF_QNRF, to show the superiority of our method.

Visual hull is an efficient technique for image-based rendering. It takes a group of images as input, produces an approximate geometry model of the object, and renders it from arbitrary new viewpoints. Explicit visual hull models (as polygon meshes or point sets) are usually insufficient in accuracy. Image-based visual hull (IBVH) method [Matusik 2000] reconstructs implicit models, and produce view-dependent rendering results with higher quality.

Semantic object cutout serves as a basic unit in various image editing systems. In a typical scenario, users are required to provide several strokes which indicate part of the pixels as image background or objects. However, most existing approaches are passive in the sense of accepting input strokes without checking the consistence with user's intention. Here we argue that an active strategy may potentially reduce the interaction burden. Before any real calculation for segmentation, the program can roughly estimate the uncertainty for each image element and actively provide useful suggestions to users. Such a pre-processing is particularly useful for beginners unaware of feeding the underlying cutout algorithms with optimal strokes. We develop such an active object cutout algorithm, named ActiveCut, which makes it possible to automatically detect ambiguity given current user-supplied strokes, and synthesize "suggestive strokes" as feedbacks. Generally, suggestive strokes come from the ambiguous image parts and have the maximal potentials to reduce label uncertainty. Users can continuously refine their inputs following these suggestive strokes. In this way, the number of user-program interaction iterations can thus be greatly reduced. Specifically, the uncertainty is modeled by mutual information between user strokes and unlabeled image regions. To ensure that ActiveCut works at a user-interactive rate, we adopt superpixel lattice based image representation, whose computation depends on scene complexity rather than original image resolution. Moreover, it retains the 2-D-lattice topology and is thus more suitable for parallel computing. While for the most time-consuming calculation of probabilistic entropy, variational approximation is utilized for acceleration. Finally, based on submodular function theory, we provide a theoretic analysis for the performance lower bound of the proposed greedy algorithm. Various user studies are conducted on the MSRC image dataset to validate the effectiveness of our proposed algorithm.

We propose a novel graph-based transductive learning approach for interactive image segmentation. Here the term ldquotransductiverdquo indicates a process that iteratively propagates information from user-labeled regions to unlabeled image pixels. For the application of interactive image segmentation, transductive approach has several advantages compared with traditional color probabilistic model based approach. However, previous transductive approaches for image segmentation usually utilize an 8-connected neighborhood system, which has low efficacy when transferring local information to remote pixels. The main contribution of this paper is to estimate pairwise pixel similarity based on a novel path-based metric (i.e. connectivity similarity), rather than local comparison with 8-connected neighbors. We further theoretically prove the computing complexity is on a polynomial order and provide convergence guarantee for the extra local smoothing operation that is introduced to further refine the initial results. Especially, the proposed method shows promising performance in the multi-label case. Various experiments are presented to illustrate its effectiveness.

Automatic object cutout in personal album is still an open problem for computer graphics and computer vision communities. In this paper we propose a two-stage prior-guided approach for automatic object cutout. Unlike traditional computational illumination or side information-based approaches, we utilize machining learning techniques to obtain prior knowledge for target object category (front-view straight-up person here). Based on the learnt priors, desired objects can be roughly located in an image. Another contribution in this paper is the novel regularization terms defined in line segment Markov network (LSMN). Various experiments are performed to demonstrate the effectiveness of our proposed method.

In recent years, local pattern based object detection and recognition have attracted increasing interest in computer vision research community. However, to our best knowledge no previous work has focused on utilizing local patterns for the task of human detection. In this paper we develop a novel human detection system in personal albums based on LBP (local binary pattern) descriptor. Firstly we review the existing gradient based local features widely used in human detection, analyze their limitations and argue that LBP is more discriminative. Secondly, original LBP descriptor does not suit the human detecting problem well due to its high complexity and lack of semantic consistency, thus we propose two variants of LBP: Semantic-LBP and Fourier-LBP. Carefully designed experiments demonstrate the superiority of LBP over other traditional features for human detection. Especially we adopt a random ensemble algorithm for better comparison between different descriptors. All experiments are conducted on INRIA human database.

Isotropic point distribution is crucial in remeshing process to generate a high-quality mesh. In this paper, we present a novel algorithm of isotropic sampling on two-manifold mesh surface. Our main contribution lies in the successful generalization of a 2D fast Poisson disk sampling algorithm, which makes it able to directly sample 3D mesh surfaces, including feature edges. We adopt geodesic distance as the distance metric for sampling algorithm in 3D to better capture the geometry information. Given a density function over the surface, we derive a close analytic form of the available boundary, which makes our algorithm support efficient adaptive sampling. To further improve the isotropy of point distribution, Lloyd relaxation is performed locally to optimize the location of sampling points. The whole process guarantees that new vertices lie on the original surface. Mutual tessellation is utilized to reconstruct the connectivity of new vertices, which guarantees the fidelity and validity of topology. Experiments show that our algorithm is able to remesh an arbitrary closed manifold into a high-quality mesh with large minimal angles and small number of irregular vertices.

Summary form only given. Our liquid simulator is mainly based on octree-staggered grid. We use the compact difference scheme which is often used in non-staggered grid to achieve fourth order accuracy. We modified the compact scheme suitable to adaptive octree-staggered grid. In stylized rendering procedure, we use 2D texture mapping method to generate still images. Laplacian scheme smoothing in time axes is introduced to maintain temporal coherence in animation. The algorithm we used can reduce the stroke "twinkle" phenomenon among frames.

Visual saliency analysis provides a power tool for many applications. In this paper, we propose a practical and high performance GPU-based visual saliency computational model. Several novel ideas are introduced for saliency computations, such as the feature extraction as signed difference values in the lab color space and the feature fusion based on the information theory. For our implementation on the GPU, besides the programmable shaders we fully exploit other computational resources on the GPU to accelerate the computations. Our experimental results demonstrate the effectiveness of our saliency maps and indicate an order of magnitude speedup over the CPU-based implementation.

In this paper, we describe the use of threshold modulation to remove the visual artifacts contained in the variable-coefficient error-diffusion algorithm. To obtain a suitable parameter set for the threshold modulation, a cost function used for the search of optimal parameters is designed. An optimal diffusion parameter set, as well as the corresponding threshold modulation strength values, is thus obtained. Experiments over this new set of parameters show that, compared with the original variable-coefficient error-diffusion algorithm, threshold modulation can remove visual anomalies more effectively. The result of the new algorithm is an artifact-free halftoning in the full range of intensities. Fourier analysis of the experimental results further support this conclusion.