This article discusses the feasibility of classifying various situations for collaboration among medical staff engaged in surgery from videos taken in operating rooms. In video processing for medical use, especially for supporting surgery, many previous studies have attempted to recognize, from intraoperative videos, surgical phases constituting the workflows of specific surgical operations. However, it should also be significant to analyze, from videos obtained from observation cameras in operating rooms, how medical staff engaged in surgeries collaborate, with the aim of improving the efficiency and safety of collaboration among medical staff. The type of collaborative situations that occur are not predetermined but rather depend on actual behavior of medical staff during surgeries. Therefore, this study attempts to segment videos of surgical operating rooms into different collaborative situations by clustering the video frames based on similarities in the arrangements of medical staff. The arrangements are represented by the distributions of medical staff in the operating rooms as well as their observed postures observed in the video frames. In the results of our experiment with the video of a real surgery, some meaningful clusters of collaborative situations for medical staff could be obtained.

We propose a new type of object models that includes functional attributes like edible, portable and supportable, instead of or in addition to the conventional appearance based ones like color, texture, shape and so forth. It leads to not conventional recognition of objects one by one individually and separately but understanding a scene where humans are interacting with objects. The detection of the functional attributes defined newly in a proposed model requires recognition of humans' actions that actually exhibit interactions with mainly an object, and in some cases interaction between objects each other. This has ever been ignored in the traditional object recognition, since a dominant situation for recognition of objects is set up as a stationary circumstance without human activities

An extension of the ability of the image based rendering (IBR) is described by presenting a method of generating images corresponding to the virtual change of the lighting color. It is an additional ability to the previous proposal that allows generating images by virtually changing the lighting orientation. It is also suggested that the basic idea developed in this paper might be applicable to the virtual varying object material.

Video abstraction is defined as creating shorter video clips or video posters from an original video stream. In this paper, we propose a method of generating a personalized abstract of broadcasted American football video. We first detect significant events in the video stream by matching textual overlays appearing in an image frame with the descriptions of gamestats in which highlights of the game are described. Then, we select highlight shots which should be included in the video abstract from those detected events reflecting on their significance degree and personal preferences, and generate a video clip by connecting the shots augmented with related audio and text. An hour-length video can be compressed into a minute-length personalized abstract. We experimentally verified the effectiveness of this method by comparing man-made video abstracts.

This paper presents a model to represent a broadcasted sports video in a semantical way and proposes a method to segment the sports video into the semantical units for the representation. Representation of a video should clarify its semantical content as accurately as possible. Our model structurizes the video and gives suitable semantical descriptions to its particular time locations based on the structures of the sports video. We consider the speech transcript as the useful information stream for the generation of the semantical descriptions. The proposed method tries to segment the speech transcript into the units of the structurization in a probabilistic framework based on Bayesian networks. Moreover, the association with the image stream enables us to find the corresponding video segments. We discuss some experimental results.

The necessity for a video portal which supports an access to a specific video or its part from a huge video media space consisting of a large number of structured video streams is increasing. The video portal for a media space can be defined as a system which helps us to access videos through various kinds of media segments such as image, text, audio and video segments. In order to construct the video portal, we deal with structured video with metadata. Our system defines the structure of video in terms of document type definition (DTD), and describes the metadata of video with XML according to the DTD. We propose the video portal for structured video streams for the sports domain, and verify its usefulness based on a prototype system's performance.

In order to handle video media such as TV programs efficiently and effectively, we need to segment a video stream into video segments and structuralize them based on their contents. We focus on similarity, which is one of the important relations between video segments, and describe a method to cluster similar segments in a video stream. The conventional clustering methods are based on shots, but no complete method to detect shot boundaries has yet been established. Our method is based on fixed length video stream segments, called video packets. Generating spatio-temporal images, we employ cooccurrence matrices to express features in the time dimension explicitly. From clustering experiments for actual TV programs, we obtained clustering accuracy of 81%.

In this paper, we propose event-based video indexing, which is a kind of indexing by its semantical contents. Because video data is composed of multimodal information streams such as visual, auditory, and textual [closed caption (CC)] streams, we introduce a strategy of intermodal collaboration, i.e., collaborative processing taking account of the semantical dependency between these streams. Its aim is to improve the reliability and efficiency in contents analysis of video. Focusing here on temporal correspondence between visual and CC streams, the proposed method attempts to seek for time spans in which events are likely to take place through extraction of keywords from the CC stream and then to index shots in the visual stream. The experimental results for broadcasted sports video of American football games indicate that intermodal collaboration is effective for video indexing by the events such as touchdown (TD) and field goal (FG).

Video abstraction is defined as creating a shorter video clip from an original video stream. In this paper, we propose a method of generating a personalized abstract of broadcasted sports video. We first detect significant events from the video stream by matching with gamestats in which highlights of the game are described. Textual information in an overlay appearing on an image frame is recognized for this matching. Then, we select highlight shots from these detected events, reflecting on personal preferences. Finally, we connect each shot augmented with related audio and text in temporal order. From experimental results, we verified that an hourlength video can be compressed into a minute-length personalized abstract.

To effectively deal with the vast amount of videos, we need to construct a content-based representation for each video. As a step towards this goal, this paper proposes a method to automatically generate the semantical annotations for a sports video by integrating the text(c1osed-caption) and image stream. we first segment the text data and extract segments which are meaningful to grasp the story of the video, then extract the actors, the actions and the events of each scene which are useful for information retrieval by using the linguistic cues and the domain knowledge. We also segment the image stream so that each segment can associate with each text segment extracted above by using the image cues. Finally we can annotate the video by associating the text segments with the image segments. Some experimental results are presented and discussed in this paper.

Describes a new document analysis method for unformatted documents such as advertisements or catalogs. Conventional model-based approaches to the extraction of logical structures are hard to apply to advertisements or catalogs, because a model of a page can't be defined. However, these kinds of documents have similar configurations of the regions that represent each product, where a local model of a local layout and logical structures can be defined. This model, which we call a sub-structure model, can be used as a template to extract the logical structures from other regions that represent the same kinds of products. In proposed system, a sub-structure model is captured through an interactive process with a user. The system was tested on advertisements in Japanese computer magazines and the experiments show promising results.

We propose a non-interactive method for solving the contradictions caused by exceptions to ordinary rules. It is realized by: detecting the ordinary rules which have the instances in their bodies; and converting them into the default rules. To reduce the default rules which need much reasoning time, we convert the minimal sets of ordinary rules to solve contradictions. Since the proposed method is executed without human interaction, it contributes to automatic rule-base maintenance.

We propose a nonmonotonic formalism for knowledge handling allowing exceptions, the Exc-Representation (ER), and its goal-directed proof procedure, the SLD-EXC resolution, for the nonmonotonic inference system, NISE. ER always provides a unique extension, which is a set of conclusions, and makes tractable the membership problem in nonmonotonic reasoning by introducing the conditional facts, and classifying exceptions into two types based on the relationships among rules.

We propose a route planning algorithm for agent navigation under linguistic instructions of a route, aiming at man-agent flexible communication by a natural language. Instructions given to the agent describe a route in terms of landmarks like intersections, for example, "turn right at the second intersection". The contextual information in instructions of a route provides some constraints on the interpretation of landmarks and their geometric relationships. This information helps to reduce ambiguities in landmark recognition.

The authors view a document as a multi-modal media in the sense that it presents information by using figures and/or images as well as text. By showing examples of media conversion and media integration, the authors discuss the essential processing techniques of media information, especially document media information. First, media conversion is briefly addressed by an example of converting a graph diagram in a document to either an alternative graph diagram or a table. The main topic is the generation of a manual of a mechanical parts assembly as an example of media integration in a document. Thereby, necessary input information is presented for generating the manual.

We discuss the recognition of social dancing from a real image sequence and an acoustic signal of music. Since social dancing involves somewhat complicated movements by multiple humans, it is difficult to recover a detailed description of each dancer's posture by the conventional approach based on matching of an articulated human body model to each frame of an image sequence. Using the results of the recognition process for annotating a video of social dancing, we focus on the bare minimum of motion description. We show that the motion description can be acquired from a real image sequence and an acoustic signal by simple familiar techniques of computer vision and music information processing.

In this paper, we propose a method of generating a sketch map drawing and its instructions in order to support the route understanding of a human. Both are given from a structured data, called the road network, which is a graph augmented by the attributes of roads and crossings. In generating a sketch map drawing, the pictorial information about the shape and the direction of roads is modified. The sketch map drawing can be generated at any simplification level by controlling a parameter called simpleness. The instructions corresponding to the sketch map are produced by assigning appropriate terms into sentence templates. We verified that favorable results are obtained by this method.

In this paper, we present COCE (COordinative Character Extractor), a new method for extracting printed Japanese characters from an unformatted document image. This research aims to exploit knowledge independent of the layouts. COCE is based on a multiagent scheme where each agent is assigned to a single character line and tries to extract characters by making use of the knowledge about features of a character line as well as shapes and arrangement of characters. Moreover, the agents communicate with each other to keep consistency between their tasks. We have favourable results for the effectiveness of this method.

This paper presents a method of generating a sketch drawing from geographical map images with vector representation. This method consists of two stages: generation of a road network from a vectorized image and generation of a sketch map drawing based on the road network. The road network is fundamental data for any applications, represented as a graph structure that is augmented with related attributes about roads and crossings. The characteristic of this method is to introduce a parameter, called roughness, in order to control sketch map generation. Because of the roughness, we can obtain a variety of sketch map drawings according to user's requirements. We experimentally verified that the proposed method is effective and that the roughness is valid as a psychological measure.>

Three-dimensional (3D) shape recovery from a monocular image is the inverse process of optical projection and necessitates the use of additional constraints on 3D features of the objects. We have proposed the method of shape recovery based on some hypothetical constraints and its realization by energy minimization using a parallel network. In this paper, we will discuss an extension of our proposal with the aim of correcting noise and errors in an input image through the process of shape recovery. It is hardly possible to extract edges and vertexes free from noise and errors from a real image. For the robustness of the system, we must provide our system with the ability to correct such noise and errors. In this extension, we can also modulate the inclination of the process toward model-driven or data-driven.

The authors previously presented a computer generation method of technical illustrations of a machine from its CAD model. However, parts of complicated structures in a machine are more comprehensible when they are expressed in the form of TIs which consist of several illustrations representing functional subgroups of a machine than in a picture containing the total massive structures. There arises a problem of how to divide a machine into functional parts. The authors attempt to provide an answer to it by means of processing a specific graph derived from a CAD model of a given machine. The authors propose a method based on a directed graph representing a few structural aspects of a machine. The graph represents two kinds of relations, namely inserting a part to another part and connecting parts by a gear, by directional arcs, and also the shapes of parts by distinguishably shaped nodes. The basic idea is to describe a kind of semantics by structural features of a machine in a symbolic way based on the above-mentioned mechanical relations and the shapes of parts reflecting functions of parts.>

Assembly illustrations are usually included in an assembly manual to explain assembling operations visually. It is interesting and challenging to generate an assembling plan from an illustration in an assembly manual by means of document image understanding. Most of the systems developed so far are devoted merely to convert paper-based documents into computerized forms but scarcely to deal with dynamic information acquisition from a document image. The system described in the paper aims at actually understanding a document so as to generate such a dynamic object as an assembling plan from an assembly manual.>

A method for creating a discrimination tree and its application in a machine-learning-based power fault diagnosis system are discussed. An algorithm for diagnosing a single distribution line that is developed automatically by the machine learning system is presented. The performance of machine learning for fault diagnosis using a discrimination tree, an artificial neural network, and an expert system are compared.>

We present an improved method of as- sembling plan generation from an assembly illus- tration. We have previously proposed an approach to assembly planning by recognizing auxiliary lines representing assembly relations. However some ambiguity remains in the details of assem- bling operations and also the determination of the execution order of the operations although a plan has been acquired from the pictorial features in an observed illustration. We have solved the problem by referring to the completed configuration in the succeeding illustrations. Explanatory words or phrases are also included in the illustration, which were not considered in our previous work. Through examining the parts of speech of them it is found that they can be classi- fied into two categories: 1) names of mechanical parts and 2) instructions on operations. The latter obviously conveys dynamic information necessary to the assembly planning. The former also implies dynamic information through the functions of the corresponding mechanical parts. The approach dis- cussed in this paper obtains dynamic information from explanatory words and generates an assem- bling plan by only analyzing the observing illus- tration. Index Terms -- Assembly illustration, auxiliary line, explanatory line, explanatory word, assembly relation, assembling operation, dynamic informa- tion, assembling plan ( assembly planning ).

A discussion is presented of how to understand assembly illustrations in an assembly manual without basing it on any model of mechanical parts. The discussion is based on the following two basic prerequisites. The first is that at the present stage mechanical parts are assumed to be in the shape of or composed of cylinders. The second is to recognize the 3-D-structure of mechanical parts by using the assembly information obtained from auxiliary lines. A hierarchical approach is developed to understand an assembly illustration. Also presented is a 'generated' structure understanding scheme. The scheme finds the generated structure in the succeeding illustration, by using preliminary information on the generated structure from the current assembly illustration. The preliminary information is obtained by understanding the current assembly illustration.>

This paper deals with a wire-routing algorithm for a building-block LSI. This algorithm has several features which sharply define it from previous techniques. The most noteworthy one among them is that it can achieve perfect wirability. This feature enables a design automation of an LSI layout pattern without any human supports.

The orthogonal expansion of many-valued logical functions is described and some approximate forms derived from it are discussed. A method of logical design of many-valued functions, especially threshold functions, using a single-threshold element is proposed based on the approximate forms.