This paper proposes a new object handling device for a robot, in which a conventional vacuum suction cup is provided with force sensing ability by coating a piezoelectric thin film polymer, i.e., PolyVinylidene DiFluoride (PVDF), on its surface. It can detect the applied force by piezoelectric effect of coated thin film. The feature of our study is that the suction cup can work not only as a grasping tool but also as a force sensor. PVDF thin film was prepared by dip-coating method which can coat thin film directly on a three dimensional (3D) surface. Although the film with sufficient piezoelectric d 33 constant, e.g., 34 pC/N, was achieved after polarization process in case of flat surface, it was not achieved, however, on 3D bellows surface. To address this problem, as an alternative method, the feasibility of lamination of already polarized film was investigated.

We propose a temporal spotting method of a hand gesture from a low-resolution far-infrared image sequence captured by a far-infrared sensor array. The sensor array captures the spatial distribution of far-infrared intensity as a thermal image by detecting far-infrared waves emitted from heat sources. It is difficult to spot a hand gesture from a sequence of thermal images captured by the sensor due to its low-resolution, heavy noise, and varying duration of the gesture. Therefore, we introduce a voting-based approach to spot the gesture with template matching-based gesture recognition. We confirm the effectiveness of the proposed temporal spotting method in several settings.

In the way of biomimetic research, we have developed a painless needle mimicking mosquito's proboscis. A real three-dimensional (3D) sharp and jagged needle shape elaborately like mosquito, however, was difficult to achieve as far as using conventional fabrication methods. For this purpose, a 3D laser lithography with ultrafine resolution of 200 nm (0.2 μm) is employed in this article. Unlike other reported sole needles, presented microneedle is divided into two half parts, each is shaped like eaves-trough and dependently moved. The combined halves act as one hollow microneedle, which can draw up human blood by capillary action. The resistive force during insertion can be reduced when the two halves are alternately advanced, which mimics the motion of mosquito's proboscis.

This paper describes a vacuum gripper that imitates the sucker of the octopus which is useful for the handling of industrial parts. It is difficult to use other sucker type vacuum grippers, but this gripper can adhere to rigid materials since the sucker is flexible. In the present work, we evaluated the effect of a liquid membrane on vacuum suction, since the industrial part often has a liquid membrane of, for example, oil and water. The success rate of grip was increased by wetting the surface, regardless of the object's shape and weight, since the liquid membrane filled the gap between the gripper and object. For the same reason, the maximum adsorption force was improved up to 1.8 times of 46 N.

Crowd analysis using cameras has attracted much attention for public safety and marketing. Among techniques of the crowd analysis, we focus on spatial people density estimation which estimates the number of people for each small area in a floor region. However, spatial people density cannot be estimated accurately for an area far from the camera because of the occlusion by people in a closer area. Therefore, we propose a method using a memory based regression method with images captured from cameras from multiple viewpoints. This method is realized by looking up a table that consists of correspondences between people density maps and crowd appearances. Since the crowd appearances include situations where various occlusions occur, an estimation robust to occlusion should be realized. In an experiment, we examined the effectiveness of the proposed method.

This paper proposes a camera-based visibility estimation method for a traffic sign. The visibility here indicates how a visual target is easy to be detected and recognized by a human driver (not a machine). This research aims at realizing a nuisance-free driver assistance system which sorts out information depending on the visibility of a visual target, in order to prevent driver distraction. Our previous study on estimating the visibility of a traffic sign considered only the effect of the local region around a target, assuming the situation that a driver's gaze is around it. The proposed method integrates both the local features and global features in a driving environment without such an assumption. The global features evaluate the positional relationships between traffic signs and the appearance around the fixation point of a driver's gaze, which considers the effect of the driver's entire field of view. Experimental results showed the effectiveness of incorporating the global features for estimating the visibility of a traffic sign.

A flexible vacuum gripper having miniature arrayed valves is proposed; the valves are normally closed, and opened when contacted with object. Since the gripper is made of flexible polymer material and only the valves contacted with the object are opened to suck the surface, this gripper can hold a free-form surface such as having steps, curvatures, etc. Namely, the valve is autonomously switchable between open and close states. As the valve becomes small, the gripper can cope with higher step and higher curvature. We fabricate this gripper using Micro Electro Mechanical Systems (MEMS) technology; this technology makes it possible to fabricating many elements with micrometer order high precision by batch processing. Two types of gripper were fabricated; one is a large-scale gripper for verifying the working principle, another is a miniature gripper based on MEMS. The absorbed weights of the large-scale and miniature grippers were 19.6 N (2 kgf) and 120 mN (12 gf), respectively.

Recently, the demand for face recognition to identify persons from surveillance video cameras has rapidly increased. Since surveillance cameras are usually placed at positions far from a person's face, the quality of face images captured by the cameras tends to be low. This degrades the recognition accuracy. Therefore, aiming to improve the accuracy of the low-resolution-face recognition, we propose a video-based super-resolution method. The proposed method can generate a high-resolution face image from low-resolution video frames including non-rigid deformations caused by changes of face poses and expressions without using any positional information of facial feature points. Most existing techniques use the facial feature points for image alignment between the video frames. However, it is difficult to obtain the accurate positions of the feature points from low-resolution face images. To achieve the alignment, the proposed method uses a free-form deformation method that flexibly aligns each local region between the images. This enables super-resolution of face images from low-resolution videos. Experimental results demonstrated that the proposed method improved the performance of super-resolution for actual videos in terms of both image quality and face recognition accuracy.

The mosquito proboscis is composed of seven micro parts. Among them, labrum and maxillae play important roles in insertion. The labrum is a central needle which sucks blood; two maxillae are left/right needles aside the labrum, respectively. They cooperatively move, i.e., one maxilla goes forward, labrum goes forward, another maxilla goes forward, successively. We noticed that the maxilla has a jagged harpoon-like shape on its tip area, which has an anchoring effect. We proposed the easy insertion mechanism as follows: the equivalent stiffness, which is the ratio of the resistance force relative to the moving distance, is considered. The total stiffness in pushing out the central needle can be made smaller than zero due to the anchoring effect of the side needles. We named it as “negative stiffness mechanism.” Equivalent stiffness during pulling/pushing a fabricated needle was experimentally estimated, which indicated the validity of proposed mechanism.

This demo presents "Smart VideoCooKing" which is a multimedia cooking recipe browsing application on portable Android devices. A multimedia cooking recipe is a cooking recipe where each cooking operation is associated with a corresponding video clip describing it, aimed to facilitate the understanding of cooking operations. In combination with third-party applications, "Smart VideoCooKing" provides useful functions such as playing cooking video clips describing cooking operations quickly, searching information of ingredients easily, and reading aloud cooking directions.

This paper proposes a method for low resolution QR-code recognition. A QR-code is a two-dimensional binary symbol that can embed various information such as characters and numbers. To recognize a QR-code correctly and stably, the resolution of an input image should be high. In practice, however, recognition of a QR-code is usually difficult due to low resolution when it is captured from a distance. In this paper, we propose a method to improve the performance of low resolution QR-code recognition by using the super-resolution technique that generates a high resolution image from multiple low-resolution images. Although a QR-code is a binary pattern, it is observed as a grayscale image due to the degradation through the capturing process. Especially the pixels around the borders between white and black regions become ambiguous. To overcome this problem, the proposed method introduces a binary pattern constraint to generate super-resolved images appropriate for recognition. Experimental results showed that a recognition rate of 98% can be achieved by the proposed method, which is a 15.7% improvement in comparison with a method using a conventional super-resolution method.

We propose a method to detect the inconsistency between a subject and the speaker for extracting speech scenes from news videos. Speech scenes in news videos contain a wealth of multimedia information, and are valuable as archived material. In order to extract speech scenes from news videos, there is an approach that uses the position and size of a face region. However, it is difficult to extract them with only such approach, since news videos contain non-speech scenes where the speaker is not the subject, such as narrated scenes. To solve this problem, we propose a method to discriminate between speech scenes and narrated scenes based on the co-occurrence between a subject's lip motion and the speaker's voice. The proposed method uses lip shape and degree of lip opening as visual features representing a subject's lip motion, and uses voice volume and phoneme as audio feature representing a speaker's voice. Then, the proposed method discriminates between speech scenes and narrated scenes based on the correlations of these features. We report the results of experiments on videos captured in a laboratory condition and also on actual broadcast news videos. Their results showed the effectiveness of our method and the feasibility of our research goal.

We propose a visibility estimation method for traffic signs considering temporal environmental changes, as a part of work for the realization of nuisance-free driver assistance systems. Recently, the number of driver assistance systems in a vehicle is increasing. Accordingly, it is becoming important to sort out appropriate information provided from them, because providing too much information may cause driver distraction. To solve such a problem, we focus on a visibility estimation method for controlling the information according to the visibility of a traffic sign. The proposed method sequentially captures a traffic sign by an in-vehicle camera, and estimates its accumulative visibility by integrating a series of instantaneous visibility. By this way, even if the environmental conditions may change temporally and complicatedly, we can still accurately estimate the visibility that the driver perceives in an actual traffic scene. We also investigate the performance of the proposed method and show its effectiveness.

Road image is becoming important for several applications such as car navigation systems, traffic environment research, city modeling. Usually, a road image can be obtained from an aerial image but the resolution of the aerial image is often low, or it contains occlusions by obstacles. Therefore, the update of road image is required. In this paper, we propose a road image mosaicing method using in-vehicle camera images and an aerial image. We first perform image registration of road regions between these images, and then, we generate a large road image by performing image mosaicing of road regions in invehicle camera images. In an experiment, we achieved resolution improvement and occlusions removal, and also succeeded in update of a large road image.

Free space detection is very important for vehicle navigation and safe driving. 3-D line segment reconstruction of a street is important for the free space detection because a street-view includes many line segments. For the free space detection, we propose a method for reconstructing 3-D line segments in a streetscape using a monocular in-vehicle camera. The 3-D reconstruction of the line segments is achieved by using each three images from an image sequence. Once accurate camera poses of these images are obtained, one of the remaining crucial problems is to match the line segments between the images correctly. A strategy for finding correspondence of the line segments is as follows: First, the correspondences of line segment candidates are searched by using a two-view constraint. However, the two-view constraint has difficulty on determining an unique correspondence geometrically. Therefore, the candidates of the line segment correspondences are reduced using a three-view constraint. In order to improve the accuracy, the proposed method exploits a color feature of the line segment and a preliminary knowledge of the vehicle motion. Finally, the line segments are reconstructed using the correspondences. From an experimental result, we confirmed the effectiveness of the proposed method. Application to the free space detection demonstrated the usefulness of the reconstructed line segments.

We propose a method for scene segmentation of a wedding party video. Recently, it has become popular to take videos of a wedding ceremony and its party. Especially, because of its length, each scene of a wedding party video needs to be indexed with each event for efficient browsing. The proposed method segments a wedding party video into scenes of events by scenario- based matching with example videos that are synthesized by combining scenes from other wedding party videos according to a scenario.

We propose a method that analyzes the structure of a large volume of general broadcast video data by the appearance patterns of near-duplicate video segments. We define six classification rules based on the appearance patterns of near-duplicate video segments according to their roles, and evaluated them over more than 1,000 hours of actual broadcast video data.

We propose a method of region-based image transform to achieve accurate transition between object appearances. A view-transition model (VTM) is one of the statistical methods that learn appearance transition from a sample image dataset of a large number of objects with various appearances. However, the VTM method has a practical problem that the appearance transition cannot be performed accurately if a sufficient number of learning samples is not available in the dataset. To cope with the problem, the proposed method first determines the regions of input and output images whose pixel values mutually affect each other during appearance transition, then transforms iteratively between partial images in the regions. We conducted experiments using actual image datasets. The results show that the proposed method could accurately transform appearances compared with the VTM method.

We propose a method to remove moving objects from an in-vehicle camera image sequence by fusing multiple image sequences. Driver assistance systems and services such as Google Street View require images containing no moving object. The proposed scheme consists of three parts: (i) collection of many image sequences along the same route by using vehicles equipped with an omni-directional camera, (ii) temporal and spatial registration of image sequences, and (iii) mosaicing partial images containing no moving object. Experimental results show that 97.3% of the moving object area could be removed by the proposed method.

Assisting culinary activities for inexperienced users has been considered as an important task in most existing works in the field. On the other hand, recipe texts are becoming available on the Internet in increasing numbers. However, they tend to be written simply by mostly non-professional people, and thus are sometimes difficult for an inexperienced person to follow the steps and manage to cook as they are supposed to. In this paper, we propose a method that detects difficult descriptions for an inexperienced user in an existing text recipe, and supplements them with multimedia contents including text information extracted from a large number of recipes, and also images and video clips on certain kinds of cooking operations, to facilitate the understanding of the recipe. Experimental results showed promising ability of the proposed method to assist inexperienced users understand the descriptions in a recipe.

We propose a visibility estimation method for traffic signs as part of work for realization of nuisance-free driving safety support systems. Recently, the number of driving safety support systems in a car has been increasing. As a result, it is becoming important to select appropriate information from them for safe and comfortable driving because too much information may cause driver distraction and may increase the risk of a traffic accident. One of the approaches to avoid such a problem is to alert the driver only with information which could easily be missed. Therefore, to realize such a system, we focus on estimating the visibility of traffic signs. The proposed method is a model-based method that estimates the visibility of traffic signs focusing on the difference of image features between a traffic sign and its surrounding region. In this paper, we investigate the performance of the proposed method and show its effectiveness.

In this paper, we propose a method for recognizing low-resolution characters using a super-resolution technique. Although portable digital cameras can be used for camera based character recognition, the captured images contain several types of noises which make the recognition task difficult. We introduce a phase of super-resolution before the recognition to enhance the resolution of images obtained from a video. The proposed method uses the subspace method for the recognition of characters which are integrated from multiple low-resolution characters by the super-resolution technique. Experimental results show that the proposed method improves the recognition accuracy; we confirmed that the recognition rate for the input size of 7 times 7 pixels was 90.35%, and for the input size of 9 times 9 pixels was 99.97%.

Near-duplicate video detection is becoming a core-technology for analyzing the structure of a large-scale video archive. It, however, is naturally an O(n2) problem, where n is a value proportional to the total length of an input video stream. We have previously challenged this time-consuming task by reducing the cost required for each of the O(n2) comparisons. This paper, on the other hand, proposes a method that reduces the number of comparisons by adaptively dividing the feature space according to the distribution of feature points.

Wikipedia is a famous online encyclopedia. However most Wikipedia entries are mainly explained by text, so it will be very informative to enhance the contents with multimedia information such as videos. Thus we are working on a method to extend information of Wikipedia entries by means of broadcast videos which explain the entries. In this work, we focus especially on news videos and Wikipedia entries about news events. In order to extend information of Wikipedia entries, it is necessary to link news videos and Wikipedia entries. So the main issue will be on a method that labels news videos with Wikipedia entries automatically. In this way, explanations could be more detailed with news videos can be exhibited, and the context of the news events should become easier to understand. Through experiments, news videos were accurately labeled with Wikipedia entries with a precision of 86% and a recall of 79%.

We present a method for recognizing characters from image sequences captured by moving camera. In the proposed method, the sequence of the captured images is compared with those of reference character patterns using the concept of analytic signal. Since the captured image sequence can be nonlinearly warped along the time axis due to the movement of a hand-held camera, phase synchronization of two analytic signals is used for the alignment of two image sequences. Hilbert transform is used to convert all the image sequences into analytic signals whose phases are supposed to be increasing. Experimental results showed the usefulness of the proposed phase-based alignment algorithm.

We propose a time-warping algorithm for recognizing finger actions by a camera. In the proposed method, an input image sequence is aligned to the reference sequences by phase-synchronization of the analytic signals, and then classified by comparing the cumulative distances. A major benefit of this method is that over-fitting to sequences of incorrect categories is restricted. The proposed method exhibited high recognition accuracy in finger-writing character recognition.

An eigenspace interpolation method smoothly interpolates between two different eigenspaces using high dimensional rotation. However, up to now its effectiveness in object recognition and the validity of the interpolation algorithm have not been discussed sufficiently. We therefore propose an appearance-based object recognition method combining the eigenspace interpolation method and a subspace method. We conducted face recognition experiments using images captured from multiple camera positions with various illumination conditions. Experimental results demonstrate the effectiveness of the proposed method and the validity of the interpolation algorithm.

This paper proposes a fast and accurate method to detect all near-duplicate segments in a video stream. To reduce the computation time while ensuring the detection accuracy equivalent to that by brute-force frame-by-frame comparison, a two-step detection method is proposed; a fast but rough detection applied in a compressed feature vector space spanned by the result of a PC A, followed by confirmation of candidates in the original high dimension space. The results show that the proposed method accelerates the detection by more than 1,000 times while maintaining the detection accuracy. We also propose an entropy-based pixel selection scheme to generate feature vectors optimized for comparison of video segments within programs with mostly common pictures. The results show that the proposed scheme eliminates the false positives drastically, which should lead to even faster detection.

We introduce a video browsing interface 'mediaWalker' that lets users explore a news video archive based on a time-series semantic structure; the 'topic thread' structure. The interface lets users efficiently track up and down the development of news in an archive with more than 1,000 hours of video.