We previously proposed a novel lossless coding method that utilizes example search and adaptive prediction within a framework of probability model optimization for monochrome video. In this paper, we improve the adaptive prediction in terms of coding performance and processing time. More precisely, we made modifications to the following three items: (a) reference pel arrangements, (b) motion vector derivation, and (c) optimal selection of predictors. Experimental results show that the proposed method certainly improves the coding performance and the processing time compared to our previous method, and achieves better coding performance than the VVC-based lossless video coding scheme.

We propose a method of virtually flattening the surface of a clothing item to a two-dimensional plane using the geodesic distance over the surface of the item. If a clothing item is flatly opened on a table, the recognition of the item is much easier than that of the same item having an arbitrary shape. However, it is difficult and troublesome to physically flatten a clothing item from its arbitrary shape automatically. We therefore propose to develop the surface of the clothing item held in the air into a two-dimensional shape using three-dimensional observation data of the surface. To this end, boundary points of the observed clothing region are sampled as start/end nodes for calculating geodesic lines, the lengths of which become two-dimensional distances between the points when the surface is flattened to a plane. To robustly calculate geodesic lines using a three-dimensional point cloud of the surface, we adopt a method that interpolates the depth and normal direction at any point on the surface using the element-free Galerkin method. The shortest path along the surface between two points on the surface is then calculated using these depths and normal directions in the framework of the “zero-length spring analogy”. The two-dimensional coordinates of the points on the plane are obtained by solving simultaneous equations determined by the geodesic distances. We also propose a method of using the flattened view for the classification of clothing type and detection of important parts of clothing. Preliminary experiments using long-sleeve shirts and trousers as clothing items demonstrate the promise of the proposed methods.

A method for detecting the position and orientation of freely curved pipes is proposed for the purpose of bin picking. We assume that the geometrical model of the object of interest is given and the bin of objects can be observed using a widely available low-cost 3D sensor. As freely curved pipes do not have distinguishable feature points, typical feature-based matching methods cannot be applied directly in this case. Although the iterative closest point (ICP) algorithm [1] is a suitable strategy for registering a 3D model with free-form surfaces to its observed 3D data, two difficulties arise when applying the ICP algorithm in the case where the background is filled with objects with the same shape. One of these is the initial estimation of the position and orientation of an object from among similar shaped point clouds. The other difficulty is robust point matching between the 3D model and the observed data belonging to a single object. To solve these issues, we investigated two key ideas: 1) estimation of pipe orientation by checking the shape of the cross-section of the observed 3D data while changing cutting directions of the sections; and 2) point matching based on ridge lines with respect to the camera direction. Experiments using actual observed data show the effectiveness of the proposed method.

In this paper, a method for detecting changes from time-sequential images of outside scenes which are taken with several minutes interval is proposed. Recently, statistical background intensity model per pixel using Gaussian mixture model (GMM) has shown its effectiveness for detecting changes from video streams. However, when the time interval between consecutive images is long, enough number of frames can not be sampled for building useful GMM. To robustly build a pixel wise background model at time t0from small number of fore and aft frames, we propose to use the joint intensity histogram of the images at time t0and t0+ 1, H(It0, Ito+ 1). Under “background dominance” condition, background probability distribution for each intensity level at t0can be estimated from H(It0, Ito+ 1). By taking this background probability distribution per intensity as a prior probability, GMM which models the variation in each pixel is robustly calculated even from several frames. Experimental results using actual field monitoring images have shown the advantage of the proposed method.

In this paper, we propose a method of visual recognition in cooperation with actions for automatic handling of clothing by a robot. Difficulty in visual recognition of clothing largely depends on the observed shape of the clothing. Therefore, strategy of actively making clothing into the shape easier to recognize should be effective. First, after clothing is observed by a trinocular stereo vision system, it is checked whether the observation gives enough information to recognize the clothing shape robustly or not. If not, proper “recognition-aid” actions, such as rotating and/or spreading the clothing, are automatically planned based on the visual analysis of the current shape. After executing the planned action, the clothing is observed again to recognize. The effect of the action of spreading clothes was demonstrated through experimental results using an actual humanoid.

In this paper, we propose a deformable-model-driven method to recognize the state of hanging clothes using three-dimensional (3D) observed data. For the task to pick up a specific part of the clothes, it is indispensable to obtain the 3D position and posture of the part. In order to robustly obtain such information from 3D observed data of the clothes, we take a deformable-model-driven approach[4], that recognizes the clothes state by comparing the observed data with candidate shapes which are predicted in advance. To carry out this approach despite large shape variation of the clothes, we propose a two-staged method. First, small number of representative 3D shapes are calculated through physical simulations of hanging the clothes. Then, after observing clothes, each representative shape is deformed so as to fit the observed 3D data better. The consistency between the adjusted shapes and the observed data is checked to select the correct state. Experimental results using actual observations have shown the good prospect of the proposed method.

In this paper, we propose a strategy for a dual-arm robot to pick up a specific part of clothing with one hand while holding the item of clothing with its other hand. Due to the large deformability of clothing, the handling requirements differ from those required for the handling of rigid objects. In the case of holding a specific part of clothing, large deformation leads to a large variety of positions and orientations of the target part, requiring flexibility of both visual recognition and motion control. On the other hand, since the clothing can flexibly curve over the hand, a relatively large range of suitable actions is allowed for grasping the clothing. By considering these characteristics, the following three-stage strategy is proposed. First, the state of the clothing is recognized from visual observation of the clothing using a deformable-model [1]. Then, the theoretically optimal position and orientation of the hand for handling a specific part of the clothing is calculated based on the recognition results. Finally, the position and orientation of the hand is modified by considering the executable motion range of the dual arms. Preliminary experimental results using actual observations of a humanoid robot were used to validate the effectiveness of the proposed strategy.

Detection of appearance/disappearance of objects from satellite images is generally very difficult since background pixels also change their intensity values owing to various factors. In this paper, we propose a method to estimate such background intensity changes by analyzing the joint intensity histogram of compared images. Considering the fact that background intensity changes tend to make clusters with ridges on the joint histogram, dominant ridges are detected using Hough transformation. Then, the joint histogram is classified into clusters according to the dominant ridges. Clusters of background intensity changes are distinguished based on covariance of each cluster. Pixels showing intensity change other than the background intensity changes are extracted as candidates of ¿significant¿ change. The prospect of this method is examined through experiments using real satellite images.

In this paper, we propose a deformable model-driven method to obtain the 3D information necessary for handling clothes by manipulators from observation with stereo cameras. The task considered in this paper is to hold up a target part of clothes (e.g. one shoulder of a pullover) by the second manipulator, when the clothes are held in the air at any point by the first manipulator. First, the method calculates possible 3D shapes of the hanging clothes by simulating the clothes deformation. The 3D shape whose appearance gives the best fit with the observed images is selected as estimation of the current state. Then, based on the estimated shape, the 3D position and normal direction of the part where the second manipulator should hold are calculated. The results of preliminary experiments using actual two manipulators have shown the good potential of the proposed method.

Aiming at manipulating clothes, a model-driven method of estimating the state of hanging clothes is proposed. We suppose a system consisting of two manipulators and a camera. The task considered in this paper is to hold a pullover at its two shoulders by two manipulators respectively, as a first step for folding it. The proposed method estimates the state of the clothes held by one manipulator in a model-driven way and indicates the position to be held next by the other manipulator. First, the possible appearances of the pullover when it is held at one point are roughly predicted. Using discriminative features of the predicted appearances, the possible states for the observed appearance are selected. Each appearance of the possible state is partially deformed so as to get close to the observed appearance. The state whose appearance successfully approaches closest to the observed appearance is selected as the final decision. The point to be held next is determined according to the state. The results of preliminary experiments using actual images have shown the good potential of the proposed method.

Classification of benign/malignant microcalcification clusters is a major diagnostic challenge for radiologists. Clinical studies have revealed that the shape of the cluster, and the spatial distribution of individual microcalcifications within it, are important indicators of its malignancy. However, mammographic images of clustered microcalcifications confound their three-dimensional (3-D) distribution with image projection and breast compression. This paper presents a novel model-based method for reconstructing microcalcification clusters in 3-D from two mammographic views (cranio-caudal and medio-lateral oblique--"shoulder to the opposite hip" or lateral-medio). We develop a 3-D breast representation and a parameterised breast compression model which constraints geometrically the possible 3-D positions of a calcification in a two-dimensional image. Corresponding calcifications in the two views are matched using an estimate of the calcification volume. Both the geometric constraint and the matching criterion are utilized in the final reconstruction step to build the 3-D reconstructed clusters. Validation experiments are described using 30 clusters to verify the individual steps of the model, and results consistent with known ground truth are obtained. Some of the approximations in the model and future work are discussed in the concluding section.

This is an elementary research into assigning color values to voxels of multi-channel magnetic resonance imaging (MRI) volume data. The MRI volume data sets obtained under different scanning conditions are transformed into components by independent component analysis (ICA), which enhances the physical characteristics of the tissue. The transfer functions for generating color values from the independent components are obtained by using a radial basis function network, a kind of neural net, by training the network with sample data chosen from the Visible Human female data set (VHF). The resultant color volume data sets correspond well with the full-color cross-sections of the Visible Human data sets.

A method for quick determination of the position and pose of a 3D free-form object with respect to its 2D projective image(s) is proposed. It is a precondition of the method that a 3D model of the object and an initial estimation of the state are given. We (1998) have previously proposed a 3D-2D registration method for the real-time registration of a 3D model of a cerebral vessel tree to an X-ray image of the vessel. Here we extend this method to meet a more general purpose. First, the formulae for obtaining the 3D model transformation from 3D-2D point pairs are generalized by describing camera's coordinates independently from world coordinates. The second generalization is to use the occluding contour instead of the skeleton of the blood vessel as the feature for matching. We use a 3D graphics system like OpenGL into our 3D-2D registration method. The applicability improvements are shown using two applications: a position and pose estimation of a 3D vessel model using multiple views, and visual feedback on the position and pose of an active camera head.

A method to determine the position and pose of an active camera-head by aligning a 3D model of its surrounding environment with an observed 2D image is proposed. The camera-head is mounted on a mobile robot and freely moves in a 3D space. We aim at visual feedback to correct the estimation error of its position and pose obtained from dead reckoning. Since the nuclear power plant where the robot moves about consists of many pipes without particular marks, most of features in the observed images are occluding edges of the pipes. For robustly finding 3D-2D point correspondences on the occluding edges, two-type predicted images which are calculated from the 3D environmental model by a view simulator are used as follows: 1) 3D model points which correspond to the observed occluding edges are quickly obtained from the predicted depth image; 2) The predicted intensity image is used to select only the 3D model points which are expected to appear clearly in the observed image. As a result, point correspondences between the observed image and the 3D model can be robustly found even in complicated scenes. Preliminary experiments using an actual plant mock-up have shown that the method is promising.

This is basic research for assigning color values to voxels of multichannel MRI volume data. The MRI volume data sets obtained under different scanning conditions are transformed into their components by independent component analysis (ICA), which enhances the physical characteristics of the tissue. The transfer functions for generating color values from independent components are obtained using a radial basis function network, a kind of neural net, by training the network with sample data chosen from the Visible Female data set. The resultant color volume data sets correspond well with the full-color cross-sections of the Visible Human data sets.

We develop a method to find correspondences between a Cranio-Caudal (CC) and a Medio-Lateral Oblique (MLO) X-ray image of the same breast. Matching between such pairs of images is considered essential by radiologists for more reliable diagnosis of early breast cancer. The two images are taken while the breast is compressed between the cassette and plate of the X-ray machine, but, almost always, to a different extent in each direction. The deformations of the breast caused by the different compressions in the different directions causes corresponding points to appear far from the straight "epipolar lines" familiar from binocular stereo vision. The method developed in this paper calculates the line in a MLO image corresponding to a point in the CC image through simulation of the deformation and the projection of a 3D line (curve) corresponding to the point. Experiments using actual images show that the method gives good predictions which can be used to find exact correspondences between points in the two images.

In this paper, a force-based method for registration of a pattern with consideration of attributes (color, grey level, curvature) is described. In the method, a pattern is represented as a group of points which are sampled from the pattern. When we consider registration of a two dimensional pattern with color (red, green, blue) information, model pattern (the pattern at initial state) and aimed pattern (the pattern after some movement) are input in the same form: respectively, P/sub i/(x/sub i/,y/sub i/,r/sub i/,g/sub i/,b/sub i/;i=1/spl sim/n) and P'/sub j/(x'/sub j/,y'/sub j/,r'/sub j/,g'/sub j/,b'/sub j/;j=1/spl sim/m). The model pattern is registered to the aimed pattern by simulating the movement of the model pattern caused by attractive forces applied to each point of P/sub i/ from all points of P'/sub j/. The magnitude of the force is defined using five dimensional space (x, y, r, g, b) so that stronger forces are exerted between "closer" points while considering similarity of their attribute values. In this paper the algorithm of the method is represented and its performance characteristics are examined. Experiments on registration of pieces of a jigsaw puzzle show that this simple method is robust and useful.

This paper proposes a method to extract regions of a deformable object from several views of it while finding the correspondence of the object among the views. The method has been developed to analyze X-ray images of a stomach. Owing to the physical (not physiological) deformation of the stomach and changes of the camera angle, the shape of the stomach regions are fairly different among the images. In order to collectively analyze these images, we use an elastic stomach model. Firstly, our method builds an elastic stomach model based on the stomach shape in one image. Considering each photographing condition, the deformation of the stomach in each image is simulated with the elastic model. Referring to the predicted contour which is obtained by projecting the deformed model from the camera angle of each image, the contour is robustly extracted from noisy images in a model-driven way. Since the predicted contour registered in each image corresponds with the elastic model, the position of each stomach part in the image is simultaneously obtained; corresponding parts can be found among the images through the model. Experimental results of analyzing several types of stomach X-ray images are shown and discussed.

Proposes a model-driven method for extracting the contour of a physically deformed object using a flexible model which simulates the deformation. This method has been developed to extract a stomach contour from a double contrast X-ray image (abbreviated as the DC image). First, the method automatically constructs a stomach model based on the stomach shape extracted from a different type of X-ray image of the same stomach. The model consists of springs to represent the elasticity of a stomach. By using the model to simulate the stomach deformation between the two images, the stomach shape in the DC image is predicted. The predicted shape is registered in the DC image and is made closer to the exact contour by a two-staged method. Stomach contours were accurately obtained using 13 actual DC images.>