Diffusion models have achieved remarkable success in image generation, with applications broadening across various domains. Inpainting is one such application that can benefit significantly from diffusion models. Existing methods either hijack the reverse process of a pretrained diffusion model or cast the problem into a larger framework, i.e., conditioned generation. However, these approaches often require nested loops in the generation process or additional components for conditioning. In this paper, we present region-aware diffusion models (RAD) for inpainting with a simple yet effective reformulation of the vanilla diffusion models. RAD utilizes a different noise schedule for each pixel, which allows local regions to be generated asynchronously while considering the global image context. A plain reverse process requires no additional components, enabling RAD to achieve inference time up to 100 times faster than the state-of-the-art approaches. Moreover, we employ low-rank adaptation (LoRA) to fine-tune RAD based on other pretrained diffusion models, reducing computational burdens in training as well. Experiments demonstrated that RAD provides state-of-the-art results both qualitatively and quantitatively, on the FFHQ, LSUN Bedroom, and ImageNet datasets.

Much progress has been made for non-rigid structure from motion (NRSfM) during the last two decades, which made it possible to provide reasonable solutions for synthetically-created benchmark data. In order to utilize these NRSfM techniques in more realistic situations, however, we are now facing two important problems that must be solved: First, general scenes contain complex deformations as well as multiple objects, which violates the usual assumptions of previous NRSfM proposals. Second, there are many unreconstructable regions in the video, either because of the discontinued tracks of 2D trajectories or those regions static towards the camera, which require careful manipulations. In this paper, we show that a consensus-based reconstruction framework can handle these issues effectively. Even though the entire scene is complex, its parts usually have simpler deformations, and even though there are some unreconstructable parts, they can be weeded out to reduce their harmful effect on the entire reconstruction. The main difficulty of this approach lies in identifying appropriate parts, however, it can be effectively avoided by sampling parts stochastically and then aggregate their reconstructions afterwards. Experimental results show that the proposed method renews the state-of-the-art for popular benchmark data under much harsher environments, i.e., narrow camera view ranges, and it can reconstruct video-based real-world data effectively for as many areas as it can without an elaborated user input.

Recent advances of subspace clustering have provided a new way of constructing affinity matrices for clustering. Unlike the kernel-based subspace clustering, which needs tedious tuning among infinitely many kernel candidates, the self-expressive models derived from linear subspace assumptions in modern subspace clustering methods are rigorously combined with sparse or low-rank optimization theory to yield an affinity matrix as a solution of an optimization problem. Despite this nice theoretical aspect, the affinity matrices of modern subspace clustering have quite different meanings from the traditional ones, and even though the affinity matrices are expected to have a rough block-diagonal structure, it is unclear whether these are good enough to apply spectral clustering. In fact, most of the subspace clustering methods perform some sort of affinity value rearrangement to apply spectral clustering, but its adequacy for the spectral clustering is not clear; even though the spectral clustering step can also have a critical impact on the overall performance. To resolve this issue, in this paper, we provide a nonparametric method to estimate the probabilistic cluster membership from these affinity matrices, which we can directly find the clusters from. The likelihood for an affinity matrix is defined nonparametrically based on histograms given the probabilistic membership, which is defined as a combination of probability simplices, and an additional prior probability is defined to regularize the membership as a Bernoulli distribution. Solving this maximum a posteriori problem replaces the spectral clustering procedure, and the final discrete cluster membership can be calculated by selecting the clusters with maximum probabilities. The proposed method provides state-of-the-art performance for well-known subspace clustering methods on popular benchmark databases.

Many recent works on knowledge distillation have provided ways to transfer the knowledge of a trained network for improving the learning process of a new one, but finding a good technique for knowledge distillation is still an open problem. In this paper, we provide a new perspective based on a decision boundary, which is one of the most important component of a classifier. The generalization performance of a classifier is closely related to the adequacy of its decision boundary, so a good classifier bears a good decision boundary. Therefore, transferring information closely related to the decision boundary can be a good attempt for knowledge distillation. To realize this goal, we utilize an adversarial attack to discover samples supporting a decision boundary. Based on this idea, to transfer more accurate information about the decision boundary, the proposed algorithm trains a student classifier based on the adversarial samples supporting the decision boundary. Experiments show that the proposed method indeed improves knowledge distillation and achieves the state-of-the-arts performance.

An activation boundary for a neuron refers to a separating hyperplane that determines whether the neuron is activated or deactivated. It has been long considered in neural networks that the activations of neurons, rather than their exact output values, play the most important role in forming classificationfriendly partitions of the hidden feature space. However, as far as we know, this aspect of neural networks has not been considered in the literature of knowledge transfer. In this paper, we propose a knowledge transfer method via distillation of activation boundaries formed by hidden neurons. For the distillation, we propose an activation transfer loss that has the minimum value when the boundaries generated by the student coincide with those by the teacher. Since the activation transfer loss is not differentiable, we design a piecewise differentiable loss approximating the activation transfer loss. By the proposed method, the student learns a separating boundary between activation region and deactivation region formed by each neuron in the teacher. Through the experiments in various aspects of knowledge transfer, it is verified that the proposed method outperforms the current state-of-the-art.

In this paper, we propose 3D unsupervised reconstruction networks (3D-URN), which reconstruct the 3D structures of instances in a given object category from their 2D feature points under an orthographic camera model. 3D-URN consists of a 3D shape reconstructor and a rotation estimator, which are trained in a fully-unsupervised manner incorporating the proposed unsupervised loss functions. The role of the 3D shape reconstructor is to reconstruct the 3D shape of an instance from its 2D feature points, and the rotation estimator infers the camera pose. After training, 3D-URN can infer the 3D structure of an unseen instance in the same category, which is not possible in the conventional schemes of non-rigid structure from motion and structure from category. The experimental result shows the state-of-the-art performance, which demonstrates the effectiveness of the proposed method.

We propose a symmetric graph convolutional autoencoder which produces a low-dimensional latent representation from a graph. In contrast to the existing graph autoencoders with asymmetric decoder parts, the proposed autoencoder has a newly designed decoder which builds a completely symmetric autoencoder form. For the reconstruction of node features, the decoder is designed based on Laplacian sharpening as the counterpart of Laplacian smoothing of the encoder, which allows utilizing the graph structure in the whole processes of the proposed autoencoder architecture. In order to prevent the numerical instability of the network caused by the Laplacian sharpening introduction, we further propose a new numerically stable form of the Laplacian sharpening by incorporating the signed graphs. In addition, a new cost function which finds a latent representation and a latent affinity matrix simultaneously is devised to boost the performance of image clustering tasks. The experimental results on clustering, link prediction and visualization tasks strongly support that the proposed model is stable and outperforms various state-of-the-art algorithms.

This paper proposes a novel unified framework to solve the 3-D localization and tracking problem that occurs multiple camera settings with overlapping views. The main challenge is to overcome the uncertainty of the back projection arising from the challenges of ground point detection in an environment that includes severe occlusions and the unknown heights of people. To tackle this challenge, we establish a Bayesian learning framework that maximizes a posterior over the trajectory assignments and 3-D positions for given detections from multiple cameras. To solve the Bayesian learning problem in a tractable form, we develop an expectation-maximization scheme based on the variation inference approximation, where the probability distributions are designed to follow Boltzmann distributions of seven terms that are induced from multicamera tracking settings. The experimental results show that the proposed method outperforms the state-of-the-art methods on the challenging multicamera data sets.

Non-rigid structure from motion (NRSfM) is a fundamental problem of computer vision. Recently, it has been shown that incorporating shape alignment in NRSfM can improve the performance significantly compared with the other algorithms, which do not consider shape alignment. However, realizing this idea was at a cost of a heavy, complicated process, which limits its usefulness and possible extensions. In this paper, we propose a novel regression framework for NRSfM, of which the variables (3D shapes) are regularized based on their aligned shapes. We show that this can be casted into an unconstrained problem or a problem with simple bound constraints, which can be efficiently solved by existing solvers. This framework can be easily integrated with numerous existing models and assumptions, such as orthographic or perspective camera models, occlusion, low-rank assumption, smooth deformations, and so on, which makes it more practical for various real situations. The experimental results show that the proposed method gives competitive result to the state-of-the-art methods for orthographic projection with much less time complexity and memory requirement, and outperforms the existing methods for perspective projection.

A well-defined deformation model can be vital for non-rigid structure from motion (NRSfM). Most existing methods restrict the deformation space by assuming a fixed rank or smooth deformation, which are not exactly true in the real world, and they require the degree of deformation to be predetermined, which is impractical. Meanwhile, the errors in rotation estimation can have severe effects on the performance, i.e., these errors can make a rigid motion be misinterpreted as a deformation. In this paper, we propose an alternative to resolve these issues, motivated by an observation that non-rigid deformations, excluding rigid changes, can be concisely represented in a linear subspace without imposing any strong constraints, such as smoothness or low-rank. This observation is embedded in our new prior distribution, the Procrustean normal distribution (PND), which is a shape distribution exclusively for non-rigid deformations. Because of this unique characteristic of the PND, rigid and non-rigid changes can be strictly separated, which leads to better performance. The proposed algorithm, EM-PND, fits a PND to given 2D observations to solve NRSfM without any user-determined parameters. The experimental results show that EM-PND gives the state-of-the-art performance for the benchmark data sets, confirming the adequacy of the new deformation model.

Recently, there have been many progresses for the problem of non-rigid structure reconstruction based on 2D trajectories, but it is still challenging to deal with complex deformations or restricted view ranges. Promising alternatives are the piecewise reconstruction approaches, which divide trajectories into several local parts and stitch their individual reconstructions to produce an entire 3D structure. These methods show the state-of-the-art performance, however, most of them are specialized for relatively smooth surfaces and some are quite complicated. Meanwhile, it has been reported numerously in the field of pattern recognition that obtaining consensus from many weak hypotheses can give a strong, powerful result. Inspired by these reports, in this paper, we push the concept of part-based reconstruction to the limit: Instead of considering the parts as explicitly-divided local patches, we draw a large number of small random trajectory sets. From their individual reconstructions, we pull out a statistic of each 3D point to retrieve a strong reconstruction, of which the procedure can be expressed as a sparse l1-norm minimization problem. In order to resolve the reflection ambiguity between weak (and possibly bad) reconstructions, we propose a novel optimization framework which only involves a single eigenvalue decomposition. The proposed method can be applied to any type of data and outperforms the existing methods for the benchmark sequences, even though it is composed of a few, simple steps. Furthermore, it is easily parallelizable, which is another advantage.

Recently, finding the low-dimensional structure of high-dimensional data has gained much attention. Given a set of data points sampled from a single subspace or a union of subspaces, the goal is to learn or capture the underlying subspace structure of the data set. In this paper, we propose elastic-net subspace representation, a new subspace representation framework using elastic-net regularization of singular values. Due to the strong convexity enforced by elastic-net, the proposed method is more stable and robust in the presence of heavy corruptions compared with existing lasso-type rank minimization approaches. For discovering a single low-dimensional subspace, we propose a computationally efficient low-rank factorization algorithm, called FactEN, using a property of the nuclear norm and the augmented Lagrangian method. Then, ClustEN is proposed to handle the general case, in which the data samples are drawn from a union of multiple subspaces, for joint subspace clustering and estimation. The proposed algorithms are applied to a number of subspace representation problems to evaluate the robustness and efficiency under various noisy conditions, and experimental results show the benefits of the proposed method compared with existing methods.

Learning a low-dimensional structure plays an important role in computer vision. Recently, a new family of methods, such as l1 minimization and robust principal component analysis, has been proposed for low-rank matrix approximation problems and shown to be robust against outliers and missing data. But these methods often require heavy computational load and can fail to find a solution when highly corrupted data are presented. In this paper, an elastic-net regularization based low-rank matrix factorization method for subspace learning is proposed. The proposed method finds a robust solution efficiently by enforcing a strong convex constraint to improve the algorithm's stability while maintaining the low-rank property of the solution. It is shown that any stationary point of the proposed algorithm satisfies the Karush-Kuhn-Tucker optimality conditions. The proposed method is applied to a number of low-rank matrix approximation problems to demonstrate its efficiency in the presence of heavy corruptions and to show its effectiveness and robustness compared to the existing methods.

Recently, there have been many proposals with state-of-the-art results in subspace clustering that take advantages of the low-rank or sparse optimization techniques. These methods are based on self-expressive models, which have well-defined theoretical aspects. They produce matrices with (approximately) block-diagonal structure, which is then applied to spectral clustering. However, there is no definitive way to construct affinity matrices from these block-diagonal matrices and it is ambiguous how the performance will be affected by the construction method. In this paper, we propose an alternative approach to detect block-diagonal structures from these matrices. The proposed method shares the philosophy of the above subspace clustering methods, in that it is a self-expressive system based on a Hadamard product of a membership matrix. To resolve the difficulty in handling the membership matrix, we solve the convex relaxation of the problem and then transform the representation to a doubly stochastic matrix, which is closely related to spectral clustering. The result of our method has eigenvalues normalized in between zero and one, which is more reliable to estimate the number of clusters and to perform spectral clustering. The proposed method shows competitive results in our experiments, even though we simply count the number of eigenvalues larger than a certain threshold to find the number of clusters.

Low-rank matrix approximation plays an important role in the area of computer vision and image processing. Most of the conventional low-rank matrix approximation methods are based on the l2 -norm (Frobenius norm) with principal component analysis (PCA) being the most popular among them. However, this can give a poor approximation for data contaminated by outliers (including missing data), because the l2 -norm exaggerates the negative effect of outliers. Recently, to overcome this problem, various methods based on the l1 -norm, such as robust PCA methods, have been proposed for low-rank matrix approximation. Despite the robustness of the methods, they require heavy computational effort and substantial memory for high-dimensional data, which is impractical for real-world problems. In this paper, we propose two efficient low-rank factorization methods based on the l1 -norm that find proper projection and coefficient matrices using the alternating rectified gradient method. The proposed methods are applied to a number of low-rank matrix approximation problems to demonstrate their efficiency and robustness. The experimental results show that our proposals are efficient in both execution time and reconstruction performance unlike other state-of-the-art methods.

Recovering a non-rigid 3D structure from a series of 2D observations is still a difficult problem to solve accurately. Many constraints have been proposed to facilitate the recovery, and one of the most successful constraints is smoothness due to the fact that most real-world objects change continuously. However, many existing methods require to determine the degree of smoothness beforehand, which is not viable in practical situations. In this paper, we propose a new probabilistic model that incorporates the smoothness constraint without requiring any prior knowledge. Our approach regards the sequence of 3D shapes as a simple stationary Markov process with Procrustes alignment, whose parameters are learned during the fitting process. The Markov process is assumed to be stationary because deformation is finite and recurrent in general, and the 3D shapes are assumed to be Procrustes aligned in order to discriminate deformation from motion. The proposed method outperforms the state-of-the-art methods, even though the computation time is rather moderate compared to the other existing methods.

Tensor decomposition is frequently used in image processing and machine learning for its ability to express higher order characteristics of data. Among tensor decomposition methods, N-mode singular value decomposition (SVD) is widely used owing to its simplicity. However, the data dimension often becomes too large to perform N-mode SVD directly due to memory limitation. An incremental method to N-mode SVD can be used to resolve this issue, but existing approaches only provide a result, which is just enough to solve discriminative problems, not the full factorization result. In this paper, we present a complete derivation of the incremental N-mode SVD, which can be applied to generative models, accompanied by a technique that can reduce the computational cost by reordering calculations. The proposed incremental N-mode SVD can also be used effectively to update the current result of N-mode SVD when new training data is received. The proposed method provides a very good approximation of N -mode SVD for the experimental data, and requires much less computation in updating a multilinear model.

Non-rigid structure from motion is a fundamental problem in computer vision, which is yet to be solved satisfactorily. The main difficulty of the problem lies in choosing the right constraints for the solution. In this paper, we propose new constraints that are more effective for non-rigid shape recovery. Unlike the other proposals which have mainly focused on restricting the deformation space using rank constraints, our proposal constrains the motion parameters so that the 3D shapes are most closely aligned to each other, which makes the rank constraints unnecessary. Based on these constraints, we define a new class of probability distribution called the Procrustean normal distribution and propose a new NRSfM algorithm, EM-PND. The experimental results show that the proposed method outperforms the existing methods, and it works well even if there is no temporal dependence between the observed samples.

Albedo estimation from a facial image is crucial for various computer vision tasks, such as 3-D morphable-model fitting, shape recovery, and illumination-invariant face recognition, but the currently available methods do not give good estimation results. Most methods ignore the influence of cast shadows and require a statistical model to obtain facial albedo. This paper describes a method for albedo estimation that makes combined use of image intensity and facial depth information for an image with cast shadows and general unknown light. In order to estimate the albedo map of a face, we formulate the albedo estimation problem as a linear programming problem that minimizes intensity error under the assumption that the surface of the face has constant albedo. Since the solution thus obtained has significant errors in certain parts of the facial image, the albedo estimate needs to be compensated. We minimize the mean square error of albedo under the assumption that the surface normals, which are calculated from the facial depth information, are corrupted with noise. The proposed method is simple and the experimental results show that this method gives better estimates than other methods.

Albedo estimation from a facial image is crucial for various computer vision tasks such as 3D morphable model fitting, shape recovery and illumination-invariant face recognition, but it has not been addressed well. This paper describes how albedo can be estimated from the combined use of image in tensity and facial depth information even if the image is under general,unknown light with cast shadow. In order to estimate the albedo map, we formulate the albedo estimation problem as a linear programming using 1-norm under the assumption that the surface of a face has a constant albedo. The proposed method is simple and the experiment results show that the proposed approach gives better albedo estimation than other methods.

In this paper, we propose Period-Controlled Medium Access Control (PC-MAC), which can operate in pseudo-TDMA manner and achieves high throughput and fairness in simple networks. PC-MAC works like CSMA/CA initially and becomes a pseudo-TDMA scheme in a few seconds due to the periodic backoff mechanism along with the contention control that tries to maintain the number of idle slots to an optimal level. Simulation results show 10%-50% higher throughput than distributed coordination function (DCF), depending on the number of nodes, while maintaining nearly perfect fairness. Furthermore, we also show how PC-MAC can successfully be applied to complex networks.