Taking a better look at subjects of interest helps humans to improve confidence in their age estimation. Unlike still images, sequences offer spatio-temporal dynamic information that contains many cues related to age progression. A review of previous work on video-based age estimation indicates that this is an underexplored field of research. This may be caused by a lack of well-defined and publicly accessible video benchmark protocol, as well as the absence of video-oriented training data. To address the former issue, we propose a carefully designed video age estimation benchmark protocol and make it publicly available. To address the latter issue, we design a video-specific age estimation method that leverages pseudo-labeling and semi-supervised learning. Our results show that the proposed method outperforms image-based baselines on both offline and online benchmark protocols, while the online estimation stability is improved by more than 50%.

Current best local descriptors are learned on a large data set of matching and non-matching keypoint pairs. However, data of this kind are not always available, since the detailed keypoint correspondences can be hard to establish. On the other hand, we can often obtain labels for pairs of keypoint bags. For example, keypoint bags extracted from two images of the same object under different views form a matching pair, and keypoint bags extracted from images of different objects form a non-matching pair. On average, matching pairs should contain more corresponding keypoints than non-matching pairs. We describe an end-to-end differentiable architecture that enables the learning of local keypoint descriptors from such weakly labeled data. In addition, we discuss how to improve the method by incorporating the procedure of mining hard negatives. We also show how our approach can be used to learn convolutional features from unlabeled video signals and 3D models.

Linking and brushing is an essential technique for interactive data exploration and analysis that leverages coordinated multiple views to identify, select, and combine data points of interest. We propose to augment this technique by directly exploring data space using textual queries. Textual and visual queries are freely combined and modified during the data exploration process. Visual queries are used to refine the results of textual queries and vice versa. This mixed brushing integrates procedural, textual, and visual based data exploration to provide a unified approach to brushing. We also propose an interface --- the Text Query Browser View, that allows users to specify and edit data queries as well as to browse the data query history. Further, we argue why an interactive, on-demand, data aggregation and derivation is necessary, and we provide a flexible mechanism that supports it. We have implemented the proposed approach within an existing visualization tool using a client-server architecture. The approach was illustrated and evaluated using two example data sets.

Current best local descriptors are learned on a large dataset of matching and non-matching keypoint pairs. However, data of this kind is not always available since detailed keypoint correspondences can be hard to establish. On the other hand, we can often obtain labels for pairs of keypoint bags. For example, keypoint bags extracted from two images of the same object under different views form a matching pair, and keypoint bags extracted from images of different objects form a non-matching pair. On average, matching pairs should contain more corresponding keypoints than non-matching pairs. We describe an end-to-end differentiable architecture that enables the learning of local keypoint descriptors from such weakly-labeled data.

Abstract An image mosaic is an assembly of a large number of small images, usually called tiles, taken from a specific dictionary/codebook. When viewed as a whole, the appearance of a single large image emerges, i.e. each tile approximates a small block of pixels. ASCII art is a related (and older) graphic design technique for producing images from printable characters. Although automatic procedures for both of these visualization schemes have been studied in the past, some are computationally heavy and cannot offer real‐time and interactive performance. We propose an algorithm able to reproduce the quality of existing non‐photorealistic rendering techniques, in particular ASCII art and image mosaics, obtaining large performance speed‐ups. The basic idea is to partition the input image into a rectangular grid and use a decision tree to assign a tile from a pre‐determined codebook to each cell. Our implementation can process video streams from webcams in real time and it is suitable for modestly equipped devices. We evaluate our technique by generating the renderings of a variety of images and videos, with good results. The source code of our engine is publicly available.

Abstract Animated virtual human characters are a common feature in interactive graphical applications, such as computer and video games, online virtual worlds and simulations. Due to dynamic nature of such applications, character animation must be responsive and controllable in addition to looking as realistic and natural as possible. Though procedural and physics‐based animation provide a great amount of control over motion, they still look too unnatural to be of use in all but a few specific scenarios, which is why interactive applications nowadays still rely mainly on recorded and hand‐crafted motion clips. The challenge faced by animation system designers is to dynamically synthesize new, controllable motion by concatenating short motion segments into sequences of different actions or by parametrically blending clips that correspond to different variants of the same logical action. In this article, we provide an overview of research in the field of example‐based motion synthesis for interactive applications. We present methods for automated creation of supporting data structures for motion synthesis and describe how they can be employed at run‐time to generate motion that accurately accomplishes tasks specified by the AI or human user.

In this paper we present a new method for mapping a natural speech to the lip shape animation in the real time. The speech signal, represented by MFCC vectors, is classified into viseme classes using neural networks. The topology of neural networks is automatically configured using genetic algorithms. This eliminates the need for tedious manual neural network design by trial and error and considerably improves the viseme classification results. This method is suitable for real-time and offline applications

LiveMail is a prototype system that allows mobile subscribers to communicate using personalized 3D face models created from images taken by their phone cameras. The user takes a snapshot of someone's face - a friend, famous person, themselves, even a pet - using the mobile phone's camera. After a quick manipulation on the phone, a 3D model of that face is created and can be animated simply by typing in some text. Speech and appropriate animation of the face are created automatically by speech synthesis. Animations can be sent to others as real 3D animated messages or as short videos in MMS. They can be used as fun messages, greeting cards etc. The system is based on a client/server communication model. The clients are mobile devices or web clients so messages can be created, sent and received on the mobile phone or on a web page. The client has a user interface that allows the user to input a facial image and place a simple mask on it to mark the main features. The client then sends this data to a server that builds a personalized face model. The client also provides an interface that lets the user request the creation of animated messages using speech synthesis. It is planned to have several versions of the client: Symbian, midlet-based, web-based, wap-based, etc. The server is responsible for sending messages adjusted to the capabilities of the receiving platform. The server, Symbian client, midlet-based client and the web client have been implemented as prototypes. We present the system architecture and the experience gained building LiveMail.

Talking virtual characters are graphical simulations of real or imaginary persons capable of human-like behaviour, most importantly talking and gesturing. Coupled with artificial intelligence (AI) techniques, the virtual characters are expected to represent the ultimate abstraction of a human-computer interface, the one where the computer looks, talks and acts like a human. Such an interface would include audio/video analysis and synthesis techniques combined with AI, dialogue management and a vast knowledge base in order to be able to respond quasi-intelligently to the users by: speech, gesture and even mood. While this goal lies further on in the future, we present an architecture that reaches towards it, at the same time aiming for a possibility of practical applications in the nearer future. Our architecture is aimed specifically at the Web. It involves a talking virtual character capable of involving in a fairly meaningful conversation with the user who types in the input.

Computer simulation of human faces has been an active research area for a long time, resulting in a multitude of facial models and several animation systems. Current interest for this technology is clearly shown by its inclusion in the MPEG-4 standard. However, it is less clear what the actual applications of facial animation (FA) will be. We have therefore undertaken experiments on 190 subjects in order to explore the benefits of FA. Part of the experiment was aimed at exploring the objective benefits, i.e., to see if FA can help users to perform certain tasks more accurately or efficiently. The other part of the experiment was aimed at more subjective benefits, like raising the level of appeal to the user, gaining more users interest, filling in the waiting times for server access so the users do not get bored. We present the experiment design and the results, The results show that FA aids users in understanding spoken numbers in noisy conditions (error rates drop from 16% to 8%); that it can make waiting times more acceptable to the user; and that it makes services more attractive to the users.

The new MPEG-4 standard, scheduled to become an International Standard in February 1999, will include support not only for natural video and audio, but also for synthetic graphics and sounds. In particular, representation of human faces and bodies will be supported. In the current draft specification of the standard, facial animation parameters (FAPs) and facial definition parameters (FDPs) are defined. FAPs are used to control facial animation at extremely low bit rates (approx. 2 kbit/s). FDPs are used to define the shape of the face by deforming a generic facial model or by supplying a substitute model. We present algorithms to interpret the part of FDPs dealing with the deformation of a generic facial model, leading to a personalisation of the model. The implementation starts from a generic model, which is deformed in order to fit the input parameters. The input parameters must include the facial feature points, and may optionally include texture coordinates and a calibration face model. We apply a cylindrical projection to the generic face in order to interpolate any missing feature points and to fit the texture, if supplied. Then we use the Dirichlet free-form deformation interpolation method to deform the generic head according to the set of feature points. If the calibration face model is present, the fitting method is based on cylindrical projection matching and barycentric coordinates to interpolate the non-feature points.

Introducing seemingly autonomous virtual beings into virtual environments to co-habit and collaborate with us is a continuous challenge and source of interest. Latest proof of human excitement for virtual life is the current worldwide craze for electronic pets that must be fed and cared for lest they develop a bad character or die. Even more interesting is the inclusion of autonomous actors in networked collaborative virtual environments (NCVEs). They provide a meeting place for people from different geographical locations and virtual beings. In NCVEs one doesn't see correspondents, only their graphical representations in the virtual world, which is the same as for the virtual ones-therefore the communication with virtual beings can come naturally. There is no single solution to the simulation of autonomous behavior. This is an ongoing research topic. Therefore it is interesting to provide an open NCVE system for easy interfacing with various implementations of autonomous behavior. In this way, the system can serve as an application platform with existing algorithms, as well as a research testbed for new autonomous behavior algorithms. The paper studies the requirements for such an open interface and, based on this study, presents an implementation within the Virtual Life Network (VLNET) system. Results are presented in terms of two case studies. A simple one implementing a dumb servant character and a more complex one connecting VLNET with the autonomous agent program Eliza (Weizenbaum, 1966).

Navigation within the scope of a Networked Collaborative Virtual Environment (NCVE) using an articulated body representation is mores complex then just moving the viewpoint based on user input.In such context, navigation englobes problems such as mapping of user's actions on the embodiment and body constraints in addition to the usual ones such as universal support for different devices and global motion constraints.We take a broader look at the problems, clasify them and present a solution for navigation in NCVEs.Basic navigation involves using some input device to control walk-through or fly-through motion.In the context

Complex virtual human representation provides more natural interaction and communication among participants in networked virtual environments, hence it is expected to increase the sense of being together within the same virtual world. We present a flexible framework for the integration of virtual humans in networked collaborative virtual environments. A modular architecture allows flexible representation and control of the virtual humans, whether they are controlled by a physical user using all sorts of tracking and other devices, or by an intelligent control program turning them into autonomous actors. The modularity of the system allows for fairly easy extensions and integration with new techniques making it interesting also as a testbed for various domains from “classic” VR to psychological experiments. We present results in terms of functionalities, example applications and measurements of performance and network traffic with an increasing number of participants in the simulation.

The research activity in natural/synthetic image processing and representation reported in this paper, initiated under the Esprit project HUMANOID and currently continued under the ACTS project VIDAS, concerns the application of virtual reality methodologies to interpersonal audio/video communication. The 3D videophone scene is modeled in video (the talker's face) and in audio (the talker's speech) so that natural data can be efficiently mixed with synthetic data and adapted onto deformable parameterized structures. Robust image analysis/synthesis tools are necessary to extract the visual primitives associated to the talker's face and to adapt them onto suitable modeling structures (wire-frames). Image/speech analysis performed at the transmitter provides suitable audio/video parameters which are encoded and used at the receiver to synthesize the corresponding facial expressions together with synchronized lip movements.

Abstract We describe the HUMANOID environment dedicated to human modeling and animation for general multimedia, VR, and CAD applications integrating virtual humans. We present the design of the system and the integration of the various features: generic modeling of a large class of entities with the BODY data structure, realistic skin deformation for body and hands, facial animation, collision detection, integrated motion control and parallelization of computation intensive tasks.