This article discusses the feasibility of classifying various situations for collaboration among medical staff engaged in surgery from videos taken in operating rooms. In video processing for medical use, especially for supporting surgery, many previous studies have attempted to recognize, from intraoperative videos, surgical phases constituting the workflows of specific surgical operations. However, it should also be significant to analyze, from videos obtained from observation cameras in operating rooms, how medical staff engaged in surgeries collaborate, with the aim of improving the efficiency and safety of collaboration among medical staff. The type of collaborative situations that occur are not predetermined but rather depend on actual behavior of medical staff during surgeries. Therefore, this study attempts to segment videos of surgical operating rooms into different collaborative situations by clustering the video frames based on similarities in the arrangements of medical staff. The arrangements are represented by the distributions of medical staff in the operating rooms as well as their observed postures observed in the video frames. In the results of our experiment with the video of a real surgery, some meaningful clusters of collaborative situations for medical staff could be obtained.

This article discusses the segmentation of indoor daily living environments, such as rooms in a house, into regions that are habitually used for different purposes by the residents living in the environment. Because these regions usually include purpose-related furnishings, previous studies have focused on the functionalities of these furnishings or their predefined parts to characterize the purposes of the regions, including those furnishings used by the residents. However, the regions used for specific purposes do not necessarily correspond to the furnishings or their predefined parts. Even for a single dining table, its central area typically serves as the region used for placing food, drink, salt shakers etc., whereas the peripheral area typically constitutes a region for dining together. This article attempts to extract the regions used for different purposes as a spatial range from the given environment by segmenting it based on the similarity in the variation of postures observed in each position and the occupancy of its surrounding space. In the experiments using two datasets for indoor daily living environments observed by RGB-D cameras, the obtained regions primarily correspond to our understanding of the regions used for different purposes.

In this paper, we present a regression for predicting 3D rigid transformations from real-valued vectors. We use a unit dual quaternion to represent the transformation. The regression is formulated as blending unit dual quaternions. To formulate it in a closed form, we introduce an approximation based on error metrics according to geometric algebra. Finally, we take an articulated motion and an elastic deformation as examples to present the descriptive power of our method in modeling the motion and the deformation.

Object tracking from camera images is a key technique in the field of computer vision. Because the most representative application of this technique is security or surveillance, objects such as pedestrians or vehicles in public spaces are considered as tracking targets. However, it is also useful to track objects such as daily items found in indoor living spaces, including offices, private rooms, and other locations, when they go missing or need to be located. However, it is not known in advance which part of the living space can move as a single object to be tracked. Moreover, because objects in a living space do not move by themselves but are moved by human hands, the bodies of objects in motion are usually occluded by the hands for a relatively long period of time, and drastic changes in the position and appearance of the objects may occur after a long-term occlusion. In this article, we propose an approach to finding the correspondence between objects observed before and after their occlusion for object tracking purposes by detecting the grasp and release of such objects by the same hand. Segmentation of the body of each object is also realized through the detection of its grasp and release.

Conventional studies on human behavior recognition have mainly focused on individual actions, including facial expressions and postures. However, most human behavior involves face-to-face interactions with other humans or objects, such as PCs. The previous work on recognizing face-to-face interaction focused on recognition of each group of humans engaged in conversation in an open space, which is considered a free space without obstacles. However, face-to-face conversation in our daily activities also occurs in a space such as an office with various obstacles, which could interfere with the interaction. To cope with this interference, the arrangement of obstacles and humans in the space should be considered. Moreover, in an office space, humans interact not only with other humans but also with certain interactive objects, such as PCs. This study aims to detect the target of interaction of each person in a space with obstacles by considering the 3D positions and orientations of humans, interactive objects, and other obstacles. The 3D arrangement of humans and obstacles is obtained by observing the space with an RGB-D camera.

This article discusses the problem of recognizing groups of people in conversation with each other in an open space. Previous work on this problem takes an approach based on the knowledge that people in the same conversation group often makes a circular formation called an F-formation, by referring to the work in social psychology. Since the F-formation describes spatial and orientational characteristics of the lower bodies of people, their positions and orientations need to be obtained for employing the approach. However, it is difficult to observe lower bodies by cameras, especially for people in a circular formation, due to occlusions between their bodies. We propose to recognize conversation groups while estimating the positions and orientations of the lower bodies of the people only from the information of their head positions and facial orientations observable from a single camera, by considering that the lower bodies of the people are arranged in a circular pattern if they are in the same conversation group.

This paper discusses virtual object manipulation using a human hand in an AR environment. For this purpose, we need to measure the configuration, which includes the position, pose and posture, of the human hand. However, if we employ a data glove for the measurement, the glove appears in the video images synthesized by AR. If we otherwise employ a vision-based approach, the obtained configuration often includes larger errors due to occlusions among fingers. Moreover, the configuration of the human hand in anyway often includes some cognitive errors in manipulating the virtual object, which is only visible in the synthesized videos. To cope with these problems, we propose to replace the human hand with a data glove with a virtual hand in the video and adjust its configuration so that it properly grasps the virtual objects without errors.

This paper presents an artifact-free super resolution texture mapping from multiple-view images. The multiple-view images are upscaled with a learning-based super resolution technique and are mapped onto a 3D mesh model. However, mapping multiple-view images onto a 3D model is not an easy task, because artifacts may appear when different upscaled images are mapped onto neighboring meshes. We define a cost function that becomes large when artifacts appear on neighboring meshes, and our method seeks the image-and mesh assignment that minimizes the cost function. Experimental results with real images demonstrate the effectiveness of our method.

This paper describes our work in computer vision to track food materials in the food preparation process. Tracking such food materials is difficult, because they are often hidden when moved by hand. Furthermore, their appearance may change in hand when they are cut or peeled. For tracking these objects in such situations, we propose a novel method that matches an object on a cooking table to one grasped in the past. We use the following three criteria to match the objects even when they are cut or peeled: the similarity in their appearance, the validity of their change in appearance, and the grasped order. We experimentally evaluated our method by applying it to the scenes of cutting and peeling food materials. As a result, we achieved an accuracy of 83.6% in matching the objects.

In this paper, we address a novel task "cooking recognition task". Cooking recognition task is to keep recognizing a cooking action of a cook and the target food product of the action at any time of the cooking. The target object of the recognition could change its visual feature completely by peeling, cutting, and mixed with others during the task. Nonetheless, such food product is called by the name that is given at the beginning of the cooking. Because the existing image recognition methods recognizes an object based on such idea that objects of the same name have similar visual feature, such methods cannot solve this cooking recognition task. Therefore, we propose new methods that recognize an object based on such idea that object keeps its identity even after manufactured and/or mixed with others.

We propose a method for synthesizing an undistorted highresolution image of a lecture room. A certain degree of high-resolution videos can be captured using commercially available high-resolution cameras. However, the resolution of objects captured by these cameras is not enough for viewers of a lecture video. To get sufficient high-resolution lecture videos, we synthesize multiple images captured by low cost cameras using a planar perspective projection. However, lecturers are distorted by the projection because they are not planar objects and do not exist on planar objects in a lecture room. To get an undistorted figure of a lecturer, we introduce a fixed-viewpoint projection using a lecturer tracking camera. As a result, we could get more high-resolution images including an undistorted figure of a lecturer than ones captured by a standard high-resolution camera.

Photometric stereo is a method of recovering surface normals (needle map) from images. The surface integral of surface normals is used to reconstruct a depth map; however, the depth edges, which are discontinuous boundaries of the depth map, pose a problem for photometric stereo. When the surface of objects includes depth edges, the reconstructed depth map may contain errors. To solve this problem, we detect depth edges using shadows and compute a relative depth between two distant points using the widths of the corresponding shadows. We define an error function and reconstruct the depth map by minimizing the error function. Experimental results with synthetic and with real image data demonstrate the effectiveness of our approach.

3D shapes are reconstructed from silhouettes obtained by multiple cameras with the volume intersection method. In recent work, methods of integrating silhouettes in time sequences have been proposed. The number of silhouettes can be increased by integrating silhouettes in multiple frames. The silhouettes of a rigid object in multiple frames are integrated with its rigid motion. This motion is often estimated with 3D feature points extracted from silhouettes. When the estimated motion has large error, shapes are reconstructed with missing parts. This error is given by the incomplete extraction of 3D feature points, which is caused by additional and missing regions of extracted silhouettes. We cannot prevent silhouettes from being extracted with the additional and missing regions in real environments. Here, we propose an intelligent method of integrating incomplete silhouettes where outcrop points, which are 3D feature points for estimating motion, play an important role. The reconstructed shape can be evaluated referring to how many outcrop points have been included in the reconstructed shape of another frame. Although the evaluation does not represent the accuracy of estimated motion directly, it does guarantee that outstanding parts will be preserved in the reconstructed shape. Silhouettes in multiple frames can be integrated with fewer missing and additional parts based on this evaluation.

This paper describes the implementation of blog system in an international distance course between Japan and Taiwan. In the study, blog was used as a tool to encourage students' reflective learning and communication. Findings suggested that blog was effective for students to document their learning, share experience and knowledge, have direct interaction with peers especially internationally. Students preferred blog over the LMS where this course builds an course Web site and they proved to use blog in developing their e-learning experience. We proposed the use of blog could lead to a technologically enhanced support for instructional strategy that serves as an informative model for other Web-assisted international courses

In this article, we discuss 3D shape reconstruction of an object in a rigid motion with the volume intersection method. When the object moves rigidly, the cameras change their relative positions to the object at every moment. To estimate the motion correctly, we propose new feature points called outcrop points on the reconstructed 3D shape. These points are guaranteed to be located on the real surface of the object. If the rigid motion of the object can be correctly estimated, cameras at different moments serve as the cameras in different positions virtually. With these cameras in time sequences, we can increase accuracy of the reconstructed 3D shape without increasing the number of cameras. Based on this idea, we reconstruct an accurate shape of the object in motion from images obtained by limited number of cameras. As the result, we can acquire an accurate shape from images in time sequences

This article discusses face translation (FT), which replaces the user's face with an expression by a different synthetic face, such as the face of an avatar or a cartoon character, with the same expression. In order to realize FT, we have to establish a criterion to define which expression on the synthetic face is the “same ” expression as the user wears on his/her face. Different from previous work, it is proposed in this article to adopt the user’s subjective viewpoint for this criterion. The user’s viewpoint is acquired online through the interaction with the user. When the user is not satisfied with the synthetic facial expression given as the result of FT by the system, the user corrects it by a graphical user interface (GUI). The system obtains examples of the FT from the user's subjective viewpoint every time the user corrects the results of translation. With increase of examples, the result of FT by the system gradually comes to reflect the user’s subjective viewpoint, so that the user does not need to correct the result. 1.

This paper describes a task-oriented approach to cope with the difference between the three dimensional (3D) space in a virtual reality (VR) system and that perceived by the user from its stereoscopic images with binocular disparity. This difference occurs for the reason that human visual perception is affected not only by binocular disparity but also by various kinds of visual cues. Although it is possible to set some of those cues consistent with each other when they are presented by a stereoscopic display, it is difficult to do the same for other cues especially vergence and focus. As the result, the users of VR systems often fail to manipulate virtual objects due to this difference. In this article, we propose to adjust the virtual space in a VR system through object manipulation by the user so that the user does not fail in each manipulation. We verified by experiments that this adjustment is effective to improve the rate of successful manipulation.

We propose a route planning algorithm for agent navigation under linguistic instructions of a route, aiming at man-agent flexible communication by a natural language. Instructions given to the agent describe a route in terms of landmarks like intersections, for example, "turn right at the second intersection". The contextual information in instructions of a route provides some constraints on the interpretation of landmarks and their geometric relationships. This information helps to reduce ambiguities in landmark recognition.

The authors view a document as a multi-modal media in the sense that it presents information by using figures and/or images as well as text. By showing examples of media conversion and media integration, the authors discuss the essential processing techniques of media information, especially document media information. First, media conversion is briefly addressed by an example of converting a graph diagram in a document to either an alternative graph diagram or a table. The main topic is the generation of a manual of a mechanical parts assembly as an example of media integration in a document. Thereby, necessary input information is presented for generating the manual.

We discuss the recognition of social dancing from a real image sequence and an acoustic signal of music. Since social dancing involves somewhat complicated movements by multiple humans, it is difficult to recover a detailed description of each dancer's posture by the conventional approach based on matching of an articulated human body model to each frame of an image sequence. Using the results of the recognition process for annotating a video of social dancing, we focus on the bare minimum of motion description. We show that the motion description can be acquired from a real image sequence and an acoustic signal by simple familiar techniques of computer vision and music information processing.

In this paper, we present COCE (COordinative Character Extractor), a new method for extracting printed Japanese characters from an unformatted document image. This research aims to exploit knowledge independent of the layouts. COCE is based on a multiagent scheme where each agent is assigned to a single character line and tries to extract characters by making use of the knowledge about features of a character line as well as shapes and arrangement of characters. Moreover, the agents communicate with each other to keep consistency between their tasks. We have favourable results for the effectiveness of this method.

Three-dimensional (3D) shape recovery from a monocular image is the inverse process of optical projection and necessitates the use of additional constraints on 3D features of the objects. We have proposed the method of shape recovery based on some hypothetical constraints and its realization by energy minimization using a parallel network. In this paper, we will discuss an extension of our proposal with the aim of correcting noise and errors in an input image through the process of shape recovery. It is hardly possible to extract edges and vertexes free from noise and errors from a real image. For the robustness of the system, we must provide our system with the ability to correct such noise and errors. In this extension, we can also modulate the inclination of the process toward model-driven or data-driven.