This article introduces the ISO/IEC MPEG Immersive Video (MIV) standard, MPEG-I Part 12, which is undergoing standardization. The draft MIV standard provides support for viewing immersive volumetric content captured by multiple cameras with six degrees of freedom (6DoF) within a viewing space that is determined by the camera arrangement in the capture rig. The bitstream format and decoding processes of the draft specification along with aspects of the Test Model for Immersive Video (TMIV) reference software encoder, decoder, and renderer are described. The use cases, test conditions, quality assessment methods, and experimental results are provided. In the TMIV, multiple texture and geometry views are coded as atlases of patches using a legacy 2-D video codec, while optimizing for bitrate, pixel rate, and quality. The design of the bitstream format and decoder is based on the visual volumetric video-based coding (V3C) and video-based point cloud compression (V-PCC) standard, MPEG-I Part 5.

Electroencephalography (EEG)-based emotion recognition is currently a hot issue in the affective computing community. Numerous studies have been published on this topic, following generally the same schema: 1) presentation of emotional stimuli to a number of subjects during the recording of their EEG, 2) application of machine learning techniques to classify the subjects' emotions. The proposed approaches vary mainly in the type of features extracted from the EEG and in the employed classifiers, but it is difficult to compare the reported results due to the use of different datasets. In this paper, we present a new database for the analysis of valence (positive or negative emotions), which is made publicly available. The database comprises physiological recordings and 257-channel EEG data, contrary to all previously published datasets, which include at most 62 EEG channels. Furthermore, we reconstruct the brain activity on the cortical surface by applying source localization techniques. We then compare the performances of valence classification that can be achieved with various features extracted from all source regions (source space features) and from all EEG channels (sensor space features), showing that the source reconstruction improves the classification results. Finally, we discuss the influence of several parameters on the classification scores.

Quadrotor drones equipped with high-quality cameras have rapidly raised as novel, cheap, and stable devices for filmmakers. While professional drone pilots can create aesthetically pleasing videos in short time, the smooth—and cinematographic—control of a camera drone remains challenging for most users, despite recent tools that either automate part of the process or enable the manual design of waypoints to create drone trajectories. This article moves a step further by offering high-level control of cinematographic drones for the specific task of framing dynamic targets. We propose techniques to automatically and interactively plan quadrotor drone motions in dynamic three-dimensional (3D) environments while satisfying both cinematographic and physical quadrotor constraints. We first propose the Drone Toric Space , a dedicated camera parameter space with embedded constraints, and derive some intuitive on-screen viewpoint manipulators. Second, we propose a dedicated path planning technique that ensures both that cinematographic properties can be enforced along the path and that the path is physically feasible by a quadrotor drone. At last, we build on the Drone Toric Space and the specific path planning technique to coordinate the motion of multiple drones around dynamic targets. A number of results demonstrate the interactive and automated capacities of our approaches on different use-cases.

We propose here an objective study of the emotional impact of a movie on an audience. An affective benchmarking solution is introduced making use of a low-intrusive measurement of the well-known Electro Dermal Answer. A dedicated processing of this biosignal produces a time-variant and normalized affective signal related to the significant excitation variations of the audience. Besides the new methodology, the originality of this paper stems from the fact that this framework has been tested on a real audience during regular cinema shows and a film festival, for five different movies and a total of 128 audience members.

The ultimate goal of the movie industry is to create an emotion from a story. However it is difficult to measure this emotional impact on an user. Questionnaires or other audience measurement techniques are often used, but with well known biases difficult to remove. We propose here an objective solution based on the viewer's physiological response (aka the EDA or GSR signals). The demonstration consists of an on the fly and live capture of an user's physiological signals, corresponding to his unconscious body reaction, and the real-time computing and display of the corresponding user's "affective profile".

This paper presents a novel approach for gaze estimation using only a low-cost camera and requiring no calibration. The main idea is based on the center-bias property of human gaze distribution to get a coarse estimate of the current gaze position as well as benefit from temporal information to enhance this rough gaze estimate. Firstly, we propose a method for detecting the eye center location and a mapping model based on the center-bias effect to convert it to gaze position. This initial gaze estimate then serves to construct the likelihood model of the eye-appearance. The final gaze position is estimated by fusing the likelihood model with the prior information obtained from previous observations on the basis of the particle filtering framework. Extensive experiments demonstrate the good performance of the proposed system with an average estimation error of 3.43° which outperforms state-of-the-art methods. Furthermore, the low complexity of the proposed system makes it suitable for real-time applications.

Identifying violent scenes in a movie may be of high interest as soon as the associated content has to be shown to a specific audience, as children for instance. However, defining what a violent scene is as well as extracting the violent excerpts from the only audiovisual cue are two hard tasks. In this article, we propose a pilot study to evaluate the interest of an approach based on the use of the electrodermal activity (EDA) to address these problems in an objective and human-centered manner. Assuming a consumer context, we especially focus on the use of a commercial sensor to capture the EDA. A comparison with a more professional device is initially performed to validate the accuracy of the commercial sensor. Two main aspects of the violent scene impact measurement with the EDA are then discussed. To tackle the violent scene detection problem, a methodology to correlate the EDA information with manual annotations of violent excerpts is first proposed. The user sensitivity to violence is then addressed and an approach to identify different profiles in the audience is qualitatively proposed. The advantages and limitations of such an approach are discussed and potential improvements are finally proposed.

We introduce a novel way of simulating sensations of motion which does not require an expensive and cumbersome motion platform. Multiple force-feedbacks are applied to the seated user's body to generate a sensation of motion experiencing passive navigation. A set of force-feedback devices such as mobile armrests or headrests are arranged around a seat so that they can apply forces to the user. We have dubbed this new approach HapSeat. A proof of concept has been designed which uses three low-cost force-feedback devices, and two control models have been implemented. Results from the first user study suggest that subjective sensations of motion are reliably generated using either model. Our results pave the way to a novel device to generate consumer motion effects based on our prototype.

In this paper, we propose a methodology to build a real-time affect detector dedicated to video viewing and entertainment applications. This detector combines the acquisition of traditional physiological signals, namely, galvanic skin response, heart rate, and electromyogram, and the use of supervised classification techniques by means of Gaussian processes. It aims at detecting the emotional impact of a video clip in a new way by first identifying emotional events in the affective stream (fast increase of the subject excitation) and then by giving the associated binary valence (positive or negative) of each detected event. The study was conducted to be as close as possible to realistic conditions by especially minimizing the use of active calibrations and considering on-the-fly detection. Furthermore, the influence of each physiological modality is evaluated through three different key-scenarios (mono-user, multi-user and extended multi-user) that may be relevant for consumer applications. A complete description of the experimental protocol and processing steps is given. The performances of the detector are evaluated on manually labeled sequences, and its robustness is discussed considering the different single and multi-user contexts.