Data pruning is the method selecting a subset from an entire training dataset while maintaining the performance after training. Most pruning metrics measure the difficulty of each training sample for the target task and select the hardest samples first. The “hardest-first sampling” works well when the pruning ratio is low (the pruned dataset is large). However, while the pruning ratio increases, the performance decreases significantly and get worse than the random pruning. This is because the hardest samples are the special samples of the class, and a model trained on few hardest samples specializes in them and drops its performance. For solving this problem, we propose a new metric: AgreementLoss. This is computed from two classifier models trained on two datasets obtained by the disjoint decomposition of the original dataset. By the dataset decomposition, the predictions of the models are more different for a sample since either of them trains each sample in the original dataset. The metric is defined as the positive or negative loss value of the trained either model, depending on two predictions agree or disagree. This metric represents the difficulty of each sample, while extremely difficult “noisy” samples get lower scores by the negative sign. By selecting samples in descending order of the AgreementLoss scores, we can reduce the priority of the “noisy” samples. Experiments on various datasets show that our method gets stable performance over all pruning ratio and outperforms other metrics especially when the pruning ratio is high.

Deep Neural Networks (DNNs) are dominant in the field of machine learning. However, because DNN models have large computational complexity, implementation of DNN models on resource-limited equipment is challenging. Therefore, techniques of compressing DNN models without degrading their accuracy is desired. Pruning is one such technique that removes unimportant neurons (or channels). In this paper, we present Pruning with Output Error Minimization (POEM), a method that performs not only pruning but also reconstruction to compensate the error caused by pruning. The strength of POEM lies in its consistent criteria for neuron selection and reconstruction based on output error of the activation function, whereas the previous methods use ad hoc criteria or relax the problem to minimize error before the activation function. The experiments with the well-known DNN models (VGG-16, ResNet-18, MobileNet) and image recognition datasets (ImageNet, CUB-200-2011) were conducted. The results show that POEM significantly outperformed the previous methods in maintaining the accuracy of the compressed models.

This paper presents an image upsampling method. Joint-bilateral filtering has been successfully applied to this problem that upsample “target” images using high-resolutional “control” images. In this filtering, the kernel is a product of weights representing spatial proximity and color (or intensity) proximity of the “control” image. However, when the “target” image involves textures that are invisible in the “control” image, these textures are damaged by this upsampling. In the case of thermal target image and visible-light control image, we can often find such textures. For solving this problem, we first propose trilateral upsampling, which is a simple extension of joint-bilateral upsampling incorporating color (or intensity) proximity weight in the “target” image. For improving this method so as to increase the spatial resolution, we introduce weight control based on local mutual information between target and control images. We confirmed that resulted images have higher spatial resolution and lower noise.

Object re-identification techniques are essential to improve the identification performance in video surveillance tasks. The re-identification problem is equal to a multi-view problem that an unknown individual is identified across spatially disjoint data. For the re-identification techniques, several multi-view feature transformation methods have been proposed. These methods are formulated by the supervised learning framework and show the better performances in multi-view classification tasks in which the training data are observed by the different sensors. However, in the reidentification tasks, these methods may not be required because the simple feature transformation method such as linear discriminant analysis (LDA) shows the reasonable identification rates. In this paper, we propose a novel semi supervised feature transformation method, which is formulated as a natural coupling with PCA and LDA modeled by the graph embedding framework. Our method showed best re-identification performances compared with other feature transformation methods.

Discriminating feature extraction is important to achieve high recognition rate in a classification problem. Fisher's linear discriminant analysis (LDA) is one of the well-known discriminating feature extraction methods and is closely related to the Mahalanobis distance metric learning. Neighborhood component analysis (NCA) is one of the Mahalanobis distance metric learning methods based on stochastic nearest neighbor assignment. The objective function of NCA can be expressed as a within-class coherency by a simple formula, and NCA extracts discriminating features by minimizing the objective function. Unfortunately, the computational cost of NCA significantly increases as the number of input data increases. For reducing the computational cost, we propose a fast distance metric learning method by taking the between-class distinguish ability into account of nearest mean classification. According to the experimental results using standard repository datasets, the computational time of our method is evaluated as 27 times shorter than that of NCA while keeping or improving the accuracy.

Content-based image retrieval (CBIR) is an image retrieval problem with image-content query. This problem is investigated in many applications, such as, human identification, information embedding to real-world objects, life-log, and so on. Through many researches on CBIR, local image features, such as SIFT, SURF, and LBP, defined on image key points are proved to be effective for fast and occlusion-robust image retrieval. In CBIR using local features, it is clear that not all features are necessary for image retrieval. That is, distinctive features have stronger discrimination power than commonly observed features. Also, some local features are fragile against observation distortions. This paper presents an importance measure representing both the robustness and the distinctiveness of a local feature based on diverse density. According to this measure, we can reduce the number of local features related to each database entry. Through some experiments, database having reduced local feature indices performs better than database using all local features as indices.

This paper presents a novel camera calibration method for a system consists of a sensor-less mobile robot and an external camera. This method can estimate the focal length and the viewing direction of the camera relative to a plane where the sensor-less mobile robot moves from a single image of an arbitrary circle on that plane. Since it is easy to let a four-wheel mobile car to move along a circular locus, our method does not requires specially designed calibration objects for camera calibration. The method does not use the position of the center and the radius of the circle and the circle is allowed to be partially occluded. Through computer simulations and experiments with a real camera and a robot, we have confirmed that our method is more robust than the Homography based calibration using four point correspondences.

Viola et al. have introduced a fast object detection scheme based on a boosted cascade of haar-like features. In this paper, we introduce a novel ternary feature that enriches the diversity and the flexibility significantly over haar-like features. We also introduce a new genetic algorithm (GA) based method for training effective ternary features through iterations of feature generation and selection. Experimental results showed that the rejection rate can reach at 98.5% with only 16 features at the first layer of the constructed cascade detector. This indicates the high performance of our method for generating effective features. We confirmed that the training time can be significantly shortened compared with Violas's method while the performance of the resulted cascade detector is comparable to the previous methods.

Viola et al. have introduced a fast object detection scheme based on a boosted cascade of haar-like features. In this paper, we introduce a novel ternary feature that enriches the diversity and the flexibility significantly over haar-like features. We also introduce a new genetic algorithm based method for training effective ternary features. Experimental results showed that the rejection rate can reach at 98.5% with only 16 features at the first layer of the cascade detector. We confirmed that the training time can be significantly shortened while the performance of the resulted cascade detector is comparable to the previous methods.

Hua et al. have proposed a stable and efficient tracking algorithm called ldquoK-means trackerrdquo[2, 3, 5]. This paper describes an adaptive non-target cluster center selection method that replaces the one used in k-means tracker where non-target cluster center are selected at fixed interval. Non-target cluster centers are selected from the ellipse that defines the area for searching the target object in K-means tracker by checking whether they have significant effects for the pixel classification and are dissimilar to any of the already-selected non-target cluster centers. This ensures that all important non-target cluster centers will be picked up while avoiding selecting redundant non-target clusters. Through comparative experiments of object tracking, we confirmed that both the robustness and the processing speed could be improved with our method.

The particle filter estimates a probability distribution of target objectpsilas state by sampled hypotheses and their weights. This method is more expressive than existing method such as Kalman filtering, because the object state is represented as a multi-modal distribution. However, the method canpsilat be directly applied to temporally variable appearance object tracking, for example, a firefly or a flickering neon-sign. For solving this problem, we propose a particle filter for a variable appearance object, which estimates a unique state parameter independent of targetpsilas position. Our method decomposes the state space into disjoint parameter spaces, i.e., object position and posture space and Appearance parameter space. In the appearance parameter space, the likelihood of each hypothesis is evaluated at the position parameters generated in the other space, and the best explain parameter is determined. Based on this parameter, likelihood in the position and posture space is evaluated. By interacting the parameter estimations in different spaces, we can successfully track blinking firefly in the darkness.

This paper presents a new method for simultaneous object tracking and recognition using object image database. This application requires two searches: search for object appearance stored in the database and that for pose parameters (position, scale, orientation, and so on) of the tracking object in each image frame. For simplifying this problem, we propose a new method, pose parameter embedding (PPE) that transforms the original problem to an appearance search problem. The nearest neighbor (NN) appearance search in this problem has a special property that gradually changing queries are given. For this problem, graph based NN search is suitable, because the preceding search result can be used as the starting point of the next search. Delaunay graph can be used for this search, however, both the graph construction cost and the degree (number of mean edges connected to a vertex) drastically increase in high-dimensional space. Instead, we propose nearest first traversing graph (NFTG) for avoiding these problems. Based on these two techniques, we successfully realized video-rate tracking and recognition.

This paper presents a perspective imaging model that is able to eliminate image deformations caused by non-linear lens distortion and to detect the location of the optical center respectively. By using a calibration pattern it can detect the optical center by tracking the optical rays generated by 3D points that have the same representation on the image plane. Calibration pattern is also used in order to generate normalized virtual screens on which captured images are back projected, thus eliminating the deformations. Experimental results for single viewpoint omni-directional cameras demonstrate the effectiveness of our method.

Three-dimensional (3-D) video is a real 3-D movie recording the object's full 3-D shape, motion, and precise surface texture. This paper first proposes a parallel pipeline processing method for reconstructing a dynamic 3-D object shape from multiview video images, by which a temporal series of full 3-D voxel representations of the object behavior can be obtained in real time. To realize the real-time processing, we first introduce a plane-based volume intersection algorithm: first represent an observable 3-D space by a group of parallel plane slices, then back-project observed multiview object silhouettes onto each slice, and finally apply two-dimensional silhouette intersection on each slice. Then, we propose a method to parallelize this algorithm using a PC cluster, where we employ five-stage pipeline processing in each PC as well as slice-by-slice parallel silhouette intersection. Several results of the quantitative performance evaluation are given to demonstrate the effectiveness of the proposed methods. In the latter half of the paper, we present an algorithm of generating video texture on the reconstructed dynamic 3-D object surface. We first describe a naive view-independent rendering method and show its problems. Then, we improve the method by introducing image-based rendering techniques. Experimental results demonstrate the effectiveness of the improved method in generating high fidelity object images from arbitrary viewpoints.

This paper presents a novel background subtraction method for detecting foreground objects in dynamic scenes involving swaying trees and fluttering flags. Most methods proposed so far adjust the permissible range of the background image variations according to the training samples of background images. Thus, the detection sensitivity decreases at those pixels having wide permissible ranges. If we can narrow the ranges by analyzing input images, the detection sensitivity can be improved. For this narrowing, we employ the property that image variations at neighboring image blocks have strong correlation, also known as "cooccurrence". This approach is essentially different from chronological background image updating or morphological postprocessing. Experimental results for real images demonstrate the effectiveness of our method.

Most nearest neighbor (NN) classifiers employ NN search algorithms for the acceleration. However, NN classification does not always require the NN search. Based on this idea, we propose a novel algorithm named k-d decision tree (KDDT). Since KDDT uses Voronoi condensed prototypes, it is less memory consuming than naive NN classifiers. We have confirmed that KDDT is much faster than NN search based classifiers through the comparative experiment (from 9 to 369 times faster).

In a Cooperative Distributed Vision system a group of communicating Active Vision Agents (AVA, in short, i.e. real time image processor with an active video camera and high speed network interface) cooperate to fulfil a meaningful task such as moving object tracking and dynamic scene visualization. A key issue to design and implement an AVA rests in the dynamic integration of Visual Perception, Camera Action, and Network Communication. This paper proposes a novel dynamic system architecture named Dynamic Memory Architecture, where perception, action, and communication modules share what we call the Dynamic Memory. It maintains not only temporal histories of state variables such as pan-tilt angles of the camera and the target object location but also their predicted values in the future. Perception, action, and communication modules are implemented as parallel processes which dynamically read from and write into the memory according to their own individual dynamics. The dynamic memory supports such asynchronous dynamic interactions (i.e. data exchanges between the modules) without wasting time for synchronization. This no-wait asynchronous module interaction capability greatly facilitates the implementation of real time reactive systems such as moving object tracking. Moreover, the dynamic memory supports the virtual synchronization between multiple AVAs, which facilitates the cooperative object tracking by communicating AVAs. A prototype system for real time moving object tracking demonstrated the effectiveness of the proposed idea.

We propose a method for detecting and tracking a human head in real time from an image sequence. The proposed method has three advantages: (1) we employ a fixed-viewpoint pan-tilt-zoom camera to acquire image sequences; with the camera, we eliminate the variations in the head appearance due to camera rotations with respect to the viewpoint; (2) we prepare a variety of contour models of the head appearances and relate them to the camera parameters; this allows us to adaptively select the model to deal with the variations in the head appearance due to human activities; (3) we use the model parameters obtained by detecting the head in the previous image to estimate those to be fitted in the current image; this estimation facilitates computational time for the head detection. Accordingly, the accuracy of the detection and required computational time are both improved and, at the same time, the robust head detection and tracking are realized in almost real time. Experimental results in the real situation show the effectiveness of our method.

This paper presents a multiobject behaviour recognition approach based on assumption generation and verification, i.e., feasible assumptions about the present behaviors consistent with the input image and behavior models are dynamically generated and verified by finding their supporting evidence in input images. This can be realized by an architecture called the selective attention model, which consists of a state-dependent event detector and an event sequence analyzer. The former detects image variation (event) in a limited image region (focusing region), which is not affected by occlusions and outliers. The latter analyzes sequences of detected events and activates all feasible states representing assumptions about multiobject behaviors. We further extend the system by introducing colored-token propagation to discriminate different objects in state space, and integration of multiviewpoint image sequences to disambiguate the single-view recognition results. Extensive experiments of human behavior recognition in real world environments demonstrate the soundness and robustness of our architecture.

Most of behavior recognition methods proposed so far share the limitations of bottom-up analysis, and single-object assumption; the bottom-up analysis can be confused by erroneous and missing image features and the single-object assumption prevents us from analyzing image sequences including multiple moving objects. This paper presents a robust behavior recognition method free from these limitations. Our method is best characterized by 1) top-down image feature extraction by selective attention mechanism, 2) object discrimination by colored-token propagation, and 3) integration of multi-viewpoint images. Extensive experiments of human behavior recognition in real world environments demonstrate the soundness and robustness of our method.

Spatial Reasoning, reasoning about spatial information (i.e. shape and spatial relations), is a crucial function of image understanding and computer vision systems. This paper proposes a novel spatial reasoning scheme for image understanding and demonstrates its utility and effectiveness in two different systems: region segmentation and aerial image understanding systems. The scheme is designed based on a so-called Multi-Agent/Cooperative Distributed Problem Solving Paradigm, where a group of intelligent agents cooperate with each other to fulfill a complicated task. The first part of the paper describes a cooperative distributed region segmentation system, where each region in an image is regarded as an agent. Starting from seed regions given at the initial stage, region agents deform their shapes dynamically so that the image is partitioned into mutually disjoint regions. The deformation of each individual region agent is realized by the snake algorithm14 and neighboring region agents cooperate with each other to find common region boundaries between them. In the latter part of the paper, we first give a brief description of the cooperative spatial reasoning method used in our aerial image understanding system SIGMA. In SIGMA, each recognized object such as a house and a road is regarded as an agent. Each agent generates hypotheses about its neighboring objects to establish spatial relations and to detect missing objects. Then, we compare its reasoning method with that used in the region segmentation system. We conclude the paper by showing further utilities of the Multi-gent/Cooperative Distributed Problem Solving Paradigm for image understanding.

Background subtraction is a simple and effective method to detect anomalous regions in images. In spite of the effectiveness, it cannot be used with an active (moving) camera head because the background image varies with camera-parameter control. This paper presents a background subtraction method with pan-tilt-zoom control. The proposed method consists of an omnidirectional background model called appearance sphere and parallax free sensing. Based on this model, precise background images can be generated and background subtraction can be performed for any combination of pan-tilt-zoom parameters without restoring 3D scene information.

We address the problem to recover the 3D shape of an unfolded book surface from the shading information in a scanner image. From a technical point of view, this shape from shading problem in real world environments is characterized by (1) proximal light source, (2) interreflections, (3) moving light source, (4) specular reflection, and (5) nonuniform albedo distribution. Taking all these factors into account, we first formulate the problem based on an iterative nonlinear optimization scheme. Then we introduce piecewise polynomial models of the 3D shape. Image restoration experiments for a real book surface demonstrated that geometric and photometric distortions are almost completely removed by the proposed method.>

It is known that the rho - theta parameter space has inherent bias, and it has been treated as the appearance of the white noise in the image space. In this paper, the authors first show that the bias is caused by the uniform quantization of the parameter space. To eliminate the bias, a new parameter gamma representing a nonuniform quantization along the rho -axis is introduced and the gamma - theta parameter space is constructed. In this space, the uniform quantization does not introduce any bias. Then, by a nonlinear transformation of theta , the gamma - omega parameter space is derived, in which a voting curve becomes a pair of straight lines preserving the unbiasedness.>

Scale-space filtering is a multiresolutional filtering method which expands a waveform to the set of waveforms in multiscale measurement. The intervals bounded by zero-crossings of expanded waveforms have a hierarchical structure, which can be represented by an interval tree. Another hierarchical description of the waveform, scale-space tree, is introduced as a natural description of the hierarchy. The difference hierarchy between the interval tree and the scale-space tree is discussed.>

Some examples of the structure line are shown and the morphological transition of the structure line is investigated. The waveform that causes the catastrophe of tree structure plays an important role in the transition. This waveform is called a catastrophe waveform. The catastrophe waveform means the transformation of the tree structure. Since the transformations of the branch such as separation and connection do not occur at the same time, an isolated tree must appear between these transformations.>

Scale-space filtering is a multiresolution filtering, which is defined as the convolution of a random waveform f(x) with the Gaussian kernel w(x,σ). The set of filtered waveform f(x,σ), called generalized waveform of f(x), is very useful for the structural analysis of waveforms. In this paper, we introduce an analytic line, named structure line, on the surface of generalized waveform f(x,σ). Structure line describes the relation of the convex and concave region of generalized waveform. Structure line is defined by some derivatives of generalized waveform, and has the same topological structure as a trinary tree. The properties of structure line are discussed and some examples are shown. It is confirmed that the structure line is effective to represent waveforms hierarchically.