The number of traffic accidents has been continuously increasing in recent years worldwide. Many accidents are caused by distracted drivers, who take their attention away from driving. Motivated by the success of Convolutional Neural Networks (CNNs) in computer vision, many researchers developed CNN-based algorithms to recognize distracted driving from a dashcam and warn the driver against unsafe behaviors. However, current models have too many parameters, which is unfeasible for vehicle-mounted computing. This work proposes a novel knowledge-distillation-based framework to solve this problem. The proposed framework first constructs a high-performance teacher network by progressively strengthening the robustness to illumination changes from shallow to deep layers of a CNN. Then, the teacher network is used to guide the architecture searching process of a student network through knowledge distillation. After that, we use the teacher network again to transfer knowledge to the student network by knowledge distillation. Experimental results on the Statefarm Distracted Driver Detection Dataset and AUC Distracted Driver Dataset show that the proposed approach is highly effective for recognizing distracted driving behaviors from photos: (i) the teacher network’s accuracy surpasses the previous best accuracy; (ii) the student network achieves very high accuracy with only 0.42M parameters (around 55% of the previous most lightweight model). Furthermore, the student network architecture can be extended to a spatial-temporal 3D CNN for recognizing distracted driving from video clips. The 3D student network largely surpasses the previous best accuracy with only 2.03M parameters on the Drive&Act Dataset. The source code is available athttps://github.com/Dichao-Liu/Lightweight_Distracted_Driver_Recognition_with_Distillation-Based_NAS_and_Knowledge_Transfer

Fine-Grained Image Classification is an inherently challenging task because of its inter-class similarity and intra-class variance. Most existing studies solve this problem by localization-and-classification strategies, which, however, always causes the problem of information loss or heavy computational expenses. Instead of localization-and-classification strategy, we propose a novel end-to-end optimization procedure named Multi-Task Attention Learning (MTAL), which reinforces the neural network’ correspondence to attention regions. Experimental results on CUB-Birds and Stanford Cars show that our procedure distinctly outperforms the baselines and is comparable with state-of-the-art studies despite its simplicity*.

Occlusion handling is one of the most challenging issues for pedestrian detection, and no satisfactory achievement has been found in this issue yet. Using human body parts has been considered as a reasonable way to overcome such an issue. In this paper, we propose a brand new approach based on the fusion of Mid-level body part mining and Convolutional Neural Network (CNN) to solve this problem, named DP-CNN(Discriminative Parts CNN). Two main discussions are included in this paper. First, we take an exhaustive analysis on how to mine useful body parts that contribute to pedestrian detection. Multiple ingredients (e.g. feature representation, pedestrian attributes) are analyzed through a wide range of experiments. Second, we convert the part detectors to the middle layer of CNN and re-train the model to get a better adaption of the dataset. Compare to existing approaches based on fine-tuning CNN models, our method is not only robust to occlusion handling but also has a smaller computational cost.

The eye gaze behavior of individuals changes depending on their knowledge and experience of the event occurring in their field of view. In past studies, researchers formulated a hypothesis concerning this dependency on a specific scene and then analyzed the gaze behavior of viewers observing the scene. We depart from this hypothesis-testing paradigm. In this paper, we propose a data-mining framework for extracting skilled gaze behaviors of experts while watching a video based on a comprehensive comparison of viewers in terms of the dependency of their gaze patterns on video scenes. To quantitatively analyze the changes in the gaze behavior of experts according to the events in the scene, video and eye movement sequences are classified into video scenes and gaze patterns, respectively, by using an unsupervised clustering method focusing on short-time dynamics. Then, we analyze the dependency based on the distinctiveness and occurrence frequency of gaze patterns for each video scene.

This paper proposes an approach to elderly mental care called model-based reminiscence, which utilizes cognitive modeling to guide a user's mental time travel. In this approach, a personalized cognitive model is constructed by implementing a user's lifelog (a photo library) in the ACT-R cognitive architecture. The constructed model retrieves photos based on human memory characteristics such as learning, forgetting, inhibition, and noise. These memory characteristics are regulated with parameter values corresponding to cognitive and emotional health. The authors assumed that a user's mental health could be assessed from their reactions to photo sequences retrieved by models with various parameter settings. The authors also assumed that it would be possible to motivate a user by guiding their memory recall with photo sequences generated from a healthy optimal state model. A simulation study indicates the potential of this approach presenting a variety of model behaviors corresponding cognitive / emotional states.

We built a multimodal biofeedback system integrated with low-cost sensors and applied the system to support meditation training utilizing multimodal biofeedback to the user. Our meditation support system employs Electroencephalogram, heart rate variability and eye tracking. The first two biosignals are employed to assess the mental stress during meditation, and eye tracking is equipped to detect an interval where the user engages in meditation by monitoring the open/closed states of the eyes.

Multi-camera videos with abundant information and high flexibility are expected to be useful in a wide range of applications, such as surveillance systems, web lecture broadcasting, concerts and sports viewing, etc. Viewers can enjoy a high-presence viewing experience of their own choosing by means of virtual camera switching and controlling viewing interfaces. However, some viewers may feel annoyed by continual manual viewpoint selection, especially when the number of selectable viewpoints is relatively large. In order to solve this issue, we propose an automatic viewpoint-recommending method designed especially for soccer games. This method focuses on a viewer's personal preference for viewpoint-selection, instead of common and professional editing rules. We assume that the different trajectory distributions cause a difference in the viewpoint selection according to personal preference. We therefore analyze the relationship between the viewer's personal viewpoint selecting tendency and the spatio-temporal game context. We compare methods based on a Gaussian mixture model, a general histogram+SVM and bag-of-words+SVM to seek the best representation for this relationship. The performance of the proposed methods are verified by assessing the degree of similarity between the recommended viewpoints and the viewers' edited records.

Humans see things from various viewpoints but nobody attempts to see anything from every viewpoint owing to physical limitations and the great effort required. Intelligent interfaces for viewing multi-viewpoint videos may effectively remove these limitations and open up a new visual world to mankind. We have developed a multi-viewpoint video viewer that incorporates target-centered viewpoint switching. The viewer stabilizes an object at the center of the display field, which helps to focus the user's gaze on the target. We conducted a user study to analyze user behavior, especially eye movement, while watching a multi-viewpoint video on the viewer. Statistical analyses of the results indicated that the target-centered viewpoint switching encouraged the users to gaze at the center of the display where the target was located during the viewing. We believe that these are useful findings that pave the way for the design of even more intelligent viewers.

This paper provides a novel method for building non-task-oriented dialogue agents such as chatbots. The dialogue agent constructed using our method automatically selects a suitable utterance depending on a context from a set of candidate utterances prepared in advance. To realize automatic utterance selection, we rank the candidate utterances in order of suitability by application of a machine learning algorithm. We employed both right and wrong dialogue data to learn relative suitability to rank the utterances. Additionally, we provide a low-cost and quality-assured learning data acquisition environment using crowdsourcing and gamification. The results of an experiment using learning data obtained via the environment demonstrate that the appropriate utterance is ranked on the top in 82.6% of cases and within the top 3 at 95.0% of cases. Results show that using context information that is not used in most existing agents is necessary for appropriate responses.

A Hough transform-based method for action detection can achieve robustness to occlusions because the method casts votes for action classes and spatio-temporal action positions based on the visible local features of partially occluded actions. However, each local feature is prone to a false vote. This paper focuses on the trend of past votes to curb the influence of false votes by extending conventional Hough forests to sensing that trend. Our proposed method, called trendsensitive Hough forests, learns a voting trend model that can be used to discriminate between correct and false votes and calculate the confidence of them. We experimentally confirmed that it outperformed action detection accuracy of conventional Hough forests.

Multi-view videos shot using multiple cameras are highly interested due to their considerable flexibility in enhancing the quality of our daily viewing experience, especially for large-scale events. However, the increase in the number of cameras burdens even experts on suitable viewpoint selection. Therefore, we propose in this paper an automatic viewpoint sequence recommendation system to support multi-view viewpoint selecting with a soccer game example. Unlike existing methods, our proposed system focuses on context-dependency using viewpoint evaluation and transition processes by two types of agents: a camera agent and a producer agent. The camera agent evaluates the view quality based on scene context such as positions of ball and players in given production context such as camera position and user's preference. The producer agent selects the optimal set of viewpoints by taking account of the view quality and the production objectives. The context-dependent optimization has been performed to generate variable viewing patterns which are adequate to various scene and production contexts. Sequences generated by the system and the human selection were experimentally compared to confirm the effectiveness of our proposed system. Our recommendation system has the potential to satisfy both common and personal viewing preferences for sports games.

We developed a novel networked video viewing system for multi-view video contents with video streaming technology. This work's contribution is that our developed system confirmed the validity of simultaneous multiple video streaming architecture that incorporates real-time channel switching and a target-oriented viewing interface. In addition, we newly introduced an inter-channel bandwidth management scheme to achieve cost-effective multi-view streaming.

We have developed a novel viewer for multi-point video with a socially accumulated viewing log for viewing assistance. The viewer uses annotations of objects in the scene and stabilizes the viewpoint to the user-selected object(s) along with the viewing-point selections.Starting from discussion on two viewing interfaces, i.e. camera-centered and target-centered, we propose a novel socially assisted viewing interface as a director-agent assisted target-centered system. A histogram of the viewing log in terms of time, camera and target of many people's viewing experiences, which we call a viewgram, is used as the source of the director agent, which exploits the viewgram as the visualized popular viewing behavior for particular content. The system can compose the most preferred viewing sequence by referring to the viewgram, for example. The viewgram can also be used as a map of preferences useful in choosing the viewing point.

This paper proposes an agent system that semi-automatically creates stories about daily events detected by ubiquitous sensors. These stories are knowledge of inhabitants' daily lives and it may be useful for human-friendly agent. Story flows in daily lives are extracted from interaction between sensor room inhabitants and a symbiotic agent. The agent asks causal relationships among daily events for inhabitants to create the story flow. Experimental results show that created stories let created stories perceive agent's intelligence.

In this paper, we propose and evaluate a video communication system that compensates for user's uncongenial attitudes by coordinating the robot's behaviors and media control of the video. The system facilitates comfortable video communications between elderly or disabled people by an assistant robot for each user that expresses (a) active listening behaviors to compensate for the listener's attitude when he/she is not really listening to another user's talking and (b) a cover-up behavior (gaze turned to the user) to divert attention from the other user's uncongenial attitude when that person is not looking at the talking user but toward the robot at her/his side; this behavior is performed by coordinating the automatic switching of cameras to give the impression that the congenial person is still looking at the user. The results obtained in the system evaluation show the significant effectiveness of this design approach using the robot's behavior and media control of the video to compensate for the problems in video communication that we aimed to overcome.

In this paper, we propose two novel methods for arbitrary listening-point generation for 3D audio-video (3DAV) integration in a large-scale multipoint cameras and microphones system with abilities to process, and display information of any recorded 3D scene in realtime. With this system, users are able to control their own viewpoint/listening-point position, freely. Arbitrary listening-point can be generated by either (i) ray-space representation of sound wave field (i.e. source sound independent) for multi frequency layers, or (ii) acoustic transfer function estimation (i.e. source sound dependent) and blind separation of sources of sounds. Arbitrary viewpoint generation is based on ray-space method, which is enhanced by using multipass dynamic programming for geometry compensation. Integration is done by either (i) ray-space representation of sound wave and image together, or (ii) integrating each camera video signal and acoustic transfer function of the same location as integrated 3DAV data. The prototype system of integrated audio-visual viewer achieves both good image and sound qualities with 15 frames/second.

Pervasive learning promises an exciting learning environment such that users can access content and study them at anytime, anywhere, through any devices. Besides delivering the right content to the learner, it is necessary to provide acceptable Quality-of-Service (QoS) guarantees in terms of presenting the content. In this paper, we propose a recommendation approach based on fuzzy logic theory towards QoS-aware pervasive learning. It determines appropriate presentation form of the learning content according to user's QoS requirements and device/network capability. We also propose an adaptive QoS mapping strategy, which dynamically sets quality parameters at running time according to the capabilities of client devices. The experimental results show the proposed approach is feasible and acceptable to enable QoS-aware pervasive learning.

We describe the design of a novel multi-modal, mobile computer system to support foreign students in learning Japanese kanji characters through creation of mnemonic stories. Our system treats complicated kanji shapes as hierarchical compositions of smaller shapes (following Heisig, 1986) and allows hyperlink navigation to quickly follow whole-part relationships. Visual display of kanji shape and meaning are augmented with user-supplied mnemonic stories in audio form, thereby dividing the learning information multi-modally into visual and audio modalities. A device-naming scheme and color-coding allow for asynchronous sharing of audio mnemonic stories among different users' devices. We describe the design decisions for our mobile multi-modal interface and present initial usability results based on feedback from beginning kanji learners. Our combination of mnemonic stories, audio and video modalities, and mobile device provide a new and effective system for computer-assisted kanji learning.

In this paper, we investigate the reciprocal attention modality in remotecommunication. A remote meeting system with a humanoid robot avatar is proposedto overcome the invisible wall for a video conferencing system. Ourexperimental result shows that a tangible robot avatar provides more effectivereciprocal attention against video communication. The subjects in the experimentare asked to determine whether a remote participant with the avatar is activelylistening or not to the local presenter's talk. In this system, the head motionof a remote participant is transferred and expressed by the head motion of ahumanoid robot. While the presenter has difficulty in determining the extentof a remote participant's attention with a video conferencing system, she/he hasbetter sensing of remote attentive states with the robot. Based on theevaluation result, we propose a vision system for the remote user thatintegrates omni-directional camera and robot-eye camera images to provide a wideview with a delay compensation feature.

We propose a method of extracting important interaction patterns in medical interviews. Because the interview is a major step where doctor-patient communication takes place, improving the skill and the quality of the medical interview will lead to a better medical care. A pattern mining method for multimodal interaction logs, such as gestures and speech, is applied to medical interviews in order to extract certain doctor-patient interactions. As a result, we demonstrated that several interesting patterns are extracted and we examined their interpretations. The extracted patterns are considered to be ones that doctors should acquire in training and practice for the medical interview.

This paper proposes arbitrary listening-point generation of sound without source localization, and a theory based on the ray-space representation of light rays using sub-band signal processing. An array of beam-formed dynamic microphone-arrays (MAs), are set and each MA generates a multi frequency layer sound-image (SImage) by scanning the viewing range of a camera. Each layer is captured by active microphones in MA for a given frequency range which makes the correct interval to capture that frequency layer, and a sound wave ray-space. Arbitrary listening-point generation of each layer is done by geometry compensation of corresponding images in the location of each MA or SImages or their combination. Each layer sound of an SIamge is generated by averaging the sound wave in each pixel or group of pixel. The listening-point is generated after combining each layer sound in frequency domain and inverse transformation from frequency to time domain

We developed a novel multipoint measurement system capable of acquiring video and sound at more than 100 points in a "synchronized" manner. In this paper, we first describe the specification of the system and how the system works in detail. Then we report some experimental results that confirm the performance of the system. We also describe test data set we provided for MPEG (moving picture experts group) multi-viewpoint video coding activities. Using this system, we are planning to conduct projects to measure humans and their activities, collect a large volume of real-world data of video and sound, and release them to the public

The aim of this paper is to clarify the relationship between the expressive strengths of gestures and voice for embodied and personified interfaces. We conduct perceptual tests using a puppet interface, while controlling singing-voice expressions, to empirically determine the naturalness and strength of various combinations of gesture and voice. The results show that (1) the strength of cross-modal perception is affected more by gestural expression than by the expressions of a singing voice, and (2) the appropriateness of cross-modal perception is affected by expressive combinations between singing voice and gestures in personified expressions. As a promising solution, we propose balancing a singing voice and gestural expressions by expanding and correcting the width and shape of the curve of expressive strength in the singing voice.

This paper addresses the problem of segmenting audio data recorded with embedded devices for the purpose of intelligent sensing in the context of multi-modal interactions. We propose a real-time method for robust speech detection in natural, noisy environments. It is based on a fusion of high order statistics of the LPC residual and autocorrelation, and adopts an on-line version of Expectation Maximization algorithm for the classification. Experimental evaluations show that the proposed method provides better detection performance under different types of natural noises, working robustly against other voices in the context of multi-speaker interactive situations. As the proposed method is based on features which have a low computational cost, and has a small latency, it is suitable for real-time tracking applications.

This paper addresses the problem of 3D sound representation without sound source localization and proposes a theory based on the ray-space representation of light rays, which is independent of object's specifications. An array of beam-formed microphone-arrays (MAs), are set and each MA generates a sound-image (SImage) by scanning the viewing range of a camera in the same location. SImage has the same size of an image and contains of blocks of sound wave with duration of one image-frame. Captured SImages with the array of MAs generate the sound wave ray-space. To make a dense SImage ray-space, we propose to use the geometry compensation of corresponding images in the location of each MA. By a dense sound ray-space, any virtual SImage, which corresponds to an arbitrary listening-point, can be generated. The listening-point sound is generated by averaging the sound wave in each pixel or group of pixel of the virtual SImage.

This paper proposes a novel mining method for multimodal interactions to extract important patterns of group activities. These extracted patterns can be used as machine-readable event indices in developing an interaction corpus based on a huge collection of human interaction data captured by various sensors. The event indices can be used, for example, to summarize a set of events and to search for particular events because they contain various pieces of context information. The proposed method extracts simultaneously occurring patterns of primitive events in interaction, such as gaze and speech, that in combination occur more consistently than randomly. The proposed method provides a statistically plausible definition of interaction events that is not possible through intuitive top-down definitions. We demonstrate the effectiveness of our method for the data captured in an experimental setup of a poster-exhibition scene. Several interesting patterns are extracted by the method, and we examined their interpretations.

We have developed a method for synthesizing a singing voice by gradually changing the musical expression based on speech morphing. This paper shows the advantages of this method in comparison with the approach of binary discrete transformation between two expressions, confirmed by statistical analyses of perception tests. In order to synthesize different expressional strengths of a singing voice, a "normal" (without expression) voice of a particular singer is used as the base of morphing, and three different expressions, "dark," "whispery" and "wet," are used as the target. Through our experiments, we confirmed i) the proposed morphing algorithm effectively interpolates the expressional strength of a singing voice, ii) an approximate equation of the perceptual sense can be used to calculate the morph ratio at a perceptually linear interval, and iii) our gradual transformation method can generate a natural singing voice from the interpolation of two different expressions.

We propose an intelligent environment for "co-experience communication" consisting of stationary and wearable sensors, and communication robots. The environment captures many events collaboratively from multiple viewpoints using a combination of stationary sensors that track people's behaviors and interactions, and wearable sensors that observe the viewpoints and areas of interest of individual persons. Humanoid communication robots also play a role as assistants to help visitors pay attention to significant events. First of all, the basic framework of the environment is described. Next, our developing exhibition room is introduced as an example of an intelligent environment.

Commercially available motion capture products give us fairly precise movements of human body segments but do not measure enough information to define skeletal posture in its entirety. This sketch describes how to obtain the complete posture of skeletal structure with the help of marker locations relative to bones that are derived from MRI data sets.

We propose an application that uses music as a multimodal expression to activate and support communication that runs parallel with traditional conversation. We examine a personified doll-shaped interface designed for musical expression. To direct such gestures toward communication, we have adopted an augmented stuffed toy with tactile interaction as a musically expressive device. We constructed the doll with various sensors for user context recognition. This configuration enables translation of the interaction into melodic statements. We demonstrate the effect of the doll on face-to-face conversation by comparing the experimental results of different input interfaces and output sounds. Consequently, we have found that conversation with the doll was positively affected by the musical output, the doll interface, and their combination.

There is a strong correlation between the number of appropriate behaviors an interactive robot can produce and its perceived intelligence. We propose a robot architecture for implementing a large number of behaviors and a visualizing tool for understanding the developed complex system. Behaviors are designed by using knowledge obtained through cognitive experiments and implemented by using situated recognition. By representing relationships between behaviors, episode rules help to guide the robot in communicating with people in a consistent manner. We have implemented over 100 behaviors and 800 episode rules in a humanoid robot. As a result, the robot could entice people to relate to it interpersonally. An Episode Editor is a tool to support the development of episode rules and to visualize the complex relationships among the behaviors. We consider the visualization is to be necessary for the constructive approach.

This paper describes our ongoing attempts to build a communityware system by presenting a project of providing digital assistants to support participants in an academic conference. We provided participants at the conference with a personal assistant system with mobile and ubiquitous computing technologies and facilitated communication among the participants. We also made online services available via the Web to encourage the participants to continue their relationships even after the conference. In this paper, we show the system we provided for the project and report the results.

Toy Interface is a real-world oriented interface that uses modeled objects with “toy”-like shapes and attributes as the interface between the real world and cyberspace. Toy-interface can be categorized into one of three types: the doll type, miniascape type and brick type. We investigate various toy interfaces and present the design detail of a doll-type interface prototype for the purpose of multi-modal interaction and communication.

We conduct an experiment to investigate how interactive amusement media attract the users. For the experiment, we have developed the exploration oriented interactive amusement media, and show that users' behavior tends to segment every similar manipulation in the time scale. We also suggest the hypothesis that such a behavioral pattern is due to a series of taking further interest within the duration of an interest, and argue what forms of interactive amusement media cause such behavioral patterns.

This paper presents an interactive video visualization technique called video cubism. With this technique, video data is considered to be a block of three dimensional data where frames of video data comprise the third dimension. The user can observe and manipulate a cut plane or cut sphere through the video data. An external real-time video source may also be attached to the video cube. The visualization leads to images that are aesthetically interesting as well as being useful for image analysis.

We present a method for visualizing the structure of a discussion in an electronic conference environment, and propose a prototype system called MRV (Multi-Relationship Viewer). MRV assists users in understanding the transitions of an argument and the whole argument structure by visualizing the relationship between the utterances from the view points of cause-and-effect and semantics. MRV provides the cause-and-effect view by using a tree structure, and the semantic view by using a spring model. The transition from one of the views to the other is seamless, enabling users to keep track of the flow of thinking. Results from a series of experiments proved that MRV can shorten the necessary time for users to grasp the transitions of an argument and the point at issue.

We present a method for visualizing a structure of discussion in an electronic conference environment, and propose a prototype system. The system assists users to understand the transition of the argument and the whole structure of the argument by visualizing the relationship between utterances from the view points of the cause-and-effect and the semantics. The system provides the cause-and-effect view by using a tree structure, and semantic view by using a spring model. The transition from one view to other is seamless so as to enable users to keep track of thinking.

We propose a method for presenting information based on individual user interests and knowledge. In previous work on recommending information, such as through WWW pages, the user could not always get his/her needed information, since the user's interests were represented only in keywords or key categories, and the contents of the presented information were previously formed by the information presenter; consequently, this information did not necessarily match the user's interests and knowledge. In the proposed method, the system acquires the user's implicit and/or explicit interests and knowledge from a question-and-answer interaction and presents the user with focus-of-interest information based on them. For personalizing the presentation and recommendation of information, the system provides each user with an information retriever using the user's operation records, a personal conceptual space, and control knowledge.

No abstract available.

A system to manage human interaction in immersive environments was designed and implemented.The interaction is defined by an interval scripi which describes the relationships between the time intervals which command actuators or gather information from sensors.With this formalism, reactive, linear, and tree-like interaction can be equally described, as well as less regular story and interaction patterns.Control of actuators and sensors is accomplished using PNF-restriction, a calculus which propagates the sensed information through the interval script determining which intervals are or should be happening at each moment.The prototype was used in an immersive, story-based interactive environment called SingSong where a user or a performer tries to conduct four computer character singers in spite of the hostility of one of them.

The representation of a visual scene is a key issue in navigating through image-based virtual environments. Using affine and projective invariants, sequential/multiple images from a pivoting camera can be easily mosaiced into a planar sheet. We discuss the importance of field-of-view to share the sensation of the scene observer (i.e. camera person) in a virtual environment and propose a method of obtaining the total field-of-view angle of image mosaics. The extension to an environment with moving objects as well as applications are presented.

This paper proposes a robust method for tracking an object contour in a sequence of images. In this method, both object extraction and tracking problems can be solved simultaneously. Furthermore, it is applicable to the tracking of arbitrary shapes since it does not need a priori knowledge about the object shapes. In the contour tracking, energy-minimizing elastic contour models are utilized, which is newly presented in this paper. The contour tracking is formulated as an optimization problem to find the position that minimizes both the elastic energy of its model and the potential energy derived from the edge potential image that includes a target object contour. We also present an algorithm which efficiently solves energy minimization problems within a dynamic programming framework. The algorithm enables us to obtain optimal solution even when the variables to be optimized are not ordered. We show the validity and usefulness of the proposed method with some experimental results.

This paper presents a realtime pedestrian counting system based on x-t spacetime image analysis. The system counts the number of pedestrians through the combination of three processes: moving object extraction, object counting by region labeling, and direction detection by flow estimation. In order to extract objects reliably under various illumination changes, we propose new moving objects extraction methods that extract moving regions whose shadows have been eliminated by using adaptive background image reconstruction and the color information processing of the images. For labeling and flow estimation, we use the Ortho-sectioning method that analyzes extracted regions on a spacetime slice image of the original three dimensional volume. Experimental results confirm the robustness of the system against illumination changes.

A unified theoretical framework for motion transparency and motion boundaries by devising fundamental constraint equations of multiple optical flow is proposed. This framework can handle flow discontinuities at motion boundaries as well as flow multiplicities due to transparency of objects in a unified manner. The constraint equations are formulated by a composition of homogeneously parametrized differential operators on the space-time image. Fitting algorithms for the constraints which result in eigensystem analyses are described. To determine the number of flows, the authors use the margin energy, a measure of goodness of fit which is the difference between the first and the second lower eigenenergy of the eigensystem. They also hypothesize a criterion for multiplicity. The measure and the criterion are derived from the analogy of quantum mechanics. It is demonstrated that the margin energy can determine the transparency and discontinuities of the flow field as regions of more than one flow.>

The authors propose a simultaneous closed-form estimation method for multiple optical flow from image sequences in which each image point has multiple motions. This method only requires convolution for space-time filtering and low-dimensional eigensystem analysis as an optimization process. The authors propose a mixture flow model of a multiple flow and energy integral minimization as a model fitting method. It is shown that symmetry between component flows of the mixture flow can reduce the dimension of the eigensystem and make the optimization unimodal and stable. Successful experiments on double-flow estimation of random texture patterns and natural scene images are reported.>

This paper presents a new ray-tracing acceleration technique called Pixel Selected Ray-Tracing (PSRT). PSRT uses undersampling based on Iterative Central Point Selection(ICPS) along with checking for similarities among trees in neighboring pixels. By using ICPS and trees, the largest danger of missing object borders can be drastically reduced. Although the speed increase attributable to PSRT varies with the image generation environment, according to experiments comparing PSRT with standard ray-tracing, PSRT is 2.6 to 8.2 times faster than standard ray-tracing for 512 by 512 pixel images, maintaining the same visual image quality. It is true that images generated by this method may contain very small errors. However, such errors can be reduced and may be made visually negligible by using ICPS and the trees of ray-object intersection to check for similarities.