Understanding images relies on the understanding of how visible objects are linked to each other. Current approaches of Visual Relation Detection (VRD) are hindered by the high frequency of some relations: when an important focus is put on them, more meaningful ones are overlooked. We address this challenge by learning the relative relevance of relations, and integrating this term into a novel scene graph extraction scheme. We show that this allows our model to predict relations on fewer and more relevant object pairs. It outperforms MotifNet, a state of the art model, on the Visual Genome dataset. It increases the Class Macro recall, the metric we propose to use, from 38.1% to 44.4%. In addition, we propose a new split of Visual Genome, with a more balanced relation distribution, emphasizing on the detection of uncommon relations and validates the use of the previous metric. On this set, our model outperforms MotifNet on all metrics, e.g. from 39.6% to 44.0% at 10 predictions per image on the relation classification task.

Relationships between objects drive our understanding of images, but modelling them poses several challenges due to the combinatorial nature of the problem and the complex structure of natural language. This paper tackles the task of predicting relationships in the form of (subject, predicate, object) triplets from still images. To address these issues, we propose a framework for learning predicate prototypes that aims to capture the multimodal nature of predicate distributions. Concurrently, a network is trained to define a space in which relationships with similar spatial layouts, interacting objects and predicates are clustered together. Finally, at test time, relationships are classified based on the similarity to their train nearest neighbor. We evaluate this approach on two well known datasets and observe comparable performance with state of the art models. Furthermore, we find that coupling prototype learning with a nearest neighbors approach increases the performance from 85.4 % to 87.6 % over a standard classification approach.

A thorough comprehension of image content demands a complex grasp of the interactions that may occur in the natural world. One of the key issues is to describe the visual relationships between objects. When dealing with real world data, capturing these very diverse interactions is a difficult problem. It can be alleviated by incorporating common sense in a network. For this, we propose a framework that makes use of semantic knowledge and estimates the relevance of object pairs during both training and test phases. Extracted from precomputed models and training annotations, this information is distilled into the neural network dedicated to this task. Using this approach, we observe a significant improvement on all classes of Visual Genome, a challenging visual relationship dataset. A 68.5 % relative gain on the recall at 100 is directly related to the relevance estimate and a 32.7% gain to the knowledge distillation.

Emerging idea in asthma phenotyping, incorporating local morphometric information on the airway wall thickness would be able to better account for the process of airway remodeling as indicator of pathology or therapeutic impact. It is thus important that such information be provided uniformly along the airway tree, not on a sparse (cross-section) sampling basis. The volumetric segmentation of the airway wall from CT data is the issue addressed in this paper by exploiting a patient-specific surface active model. An original aspect taken into account in the proposed deformable model is the management of auto-collisions for this complex morphology. The analysis of several solutions ended up with the design of a motion vector field specific to the patient geometry to guide the deformation. The segmentation result, presented as two embedded inner/outer surfaces of the wall, allows the quantification of the tissue thickness based on a locally-defined measure sensitive to even small surface irregularities. The method is validated with respect to several ground truth simulations of pulmonary CT data with different airway geometries and acquisition protocols showing accuracy within the CT resolution range. Results from an ongoing clinical study on moderate and severe asthma are presented and discussed.

We address the problem of action recognition in unconstrained videos. We propose a novel content driven pooling that leverages space-time context while being robust toward global space-time transformations. Being robust to such transformations is of primary importance in unconstrained videos where the action localizations can drastically shift between frames. Our pooling identifies regions of interest using video structural cues estimated by differ ent saliency functions. To combine the different structural information, we introduce an iterative structure learning algorithm, WSVM (weighted SVM), that determines the optimal saliency layout of an action model through a sparse regularizer. A new optimization method is proposed to solve the WSVM' highly non-smooth objective function. We evaluate our approach on standard action datasets (KTH, UCF50 and HMDB). Most noticeably, the accuracy of our algorithm reaches 51.8% on the challenging HMDB dataset which outperforms the state-of-the-art of 7.3% relatively.

Bag-of-Words representation based on trajectory local features and taking into account the spatio-temporal context through static segmentation grids is currently the leading paradigm to perform action annotation.While providing a coarse localization of low-level features, those approaches tend to be limited by the grid rigidity. In this work we propose two contributions on trajectory based signatures. First, we extend a local trajectory feature to characterize the acceleration in videos, leading to invariance to camera constant motion. We also introduce two new adaptive segmentation grids, namely Adaptive Grid (AG) and Deformable Adaptive Grid (DAG). AG is learnt from videos data, to fit a given dataset and overcome static grid rigidity. DAG is also learnt from video data. Moreover, it can be adapted to a specific video through a deformation operation. Our adaptive grids are then exploited by a Bag-of-Words model at the aggregation step for action recognition. Our proposal is evaluated on 4 publicly available datasets.

In this paper, we introduce the Just-In-Time Adaptive Decoder Engine (Jade) project, which is shipped as part of the Open RVC-CAL Compiler (Orcc) project. Orcc provides a set of open-source software tools for managing decoders standardized within MPEG by the Reconfigurable Video Coding (RVC) experts. In this framework, Jade acts as a Virtual Machine for any decoder description that uses the MPEG RVC paradigm. Jade dynamically generates a native decoder representation suitable for X86, ARM and CELL platforms with a possibility of exploiting multi-core CPUs. Thus, according to the MPEG RVC decoder description coupled with a video coded stream, Jade can create, configure and re-configure video decompression algorithms adapting to the video content.

This paper describes a mobile video browsing and retrievalapproach, based on the so-called OVIDIUS (On-line VIDeo Indexing Universal System) platform. In contrast with traditional and commercial video retrieval platforms, where video content is treated in a more or less monolithic manner (i.e. with global descriptions associated with the whole document), the proposed approach makes it possible to browse and access video content in a finer, per-segment basis. The hierarchical metadata structure exploits the MPEG-7 approach for structural description of video content. The MPEG-7 description schemes have been here enriched with both semantic and content-based metadata. The developed approach shows all its pertinence within a multiterminal context and in particular for video access from mobile devices. The platform has been recently (February, 2010) validated within the framework of the Mé[email protected] French national project.

This paper proposes a novel approach for mono-resolution 3D mesh compression, called TFAN (Triangle Fan-based compression). TFAN treats in a unified manner meshes of arbitrary topologies, i.e. manifold or not, oriented or not, while supporting real-time decoding. The proposed approach has been evaluated on both manifold and non-manifold databases, including more than 7000 3D mesh models. Experiments show that the TFAN approach outperforms existing techniques such as MPEG-4 3DMC or Tourna & Gotsman, with decoding times lower by an order of magnitude at equivalent or even better levels of compression efficiency (+/-10% in bitrate). When applied to non-manifold 3D data, the compression performances are significantly enhanced (6% to 30% gain in bitrate). Due to its high compression performances TFAN has been recently retained for ISO MPEG-4 standardization.

Abstract This paper proposes a novel approach for mono‐resolution 3D mesh compression, called TFAN (Triangle Fan‐based compression). TFAN treats in a unified manner meshes of arbitrary topologies, i.e., manifold or not, oriented or not, while offering a linear computational complexity (with respect to the number of mesh vertices) for both encoding and decoding algorithms. In addition, the TFAN compressed representation is optimized for real‐time decoding applications. In order to validate the proposed approach, two databases have been considered for experimentations. The first is the MPEG‐4 test set, which includes over 3500 general purpose manifold meshes. The second, related to the French national project SEMANTIC‐3D, includes over 4000 computer assisted design (CAD) meshes of highly irregular, non‐manifold topologies. In both cases, the TFAN approach outperforms existing techniques such as MPEG‐4/3DMC (3D Mesh Coding) or Touma and Gotsman, with decoding times lower by an order of magnitude at equivalent or even better levels of compression efficiency (±10% in bitrate). In addition, when applied to non‐manifold 3D data, the compression performances are significantly enhanced (6–30% gain in bitrate). Due to its high compression performances the TFAN approach has been recently retained for ISO standardization, within the framework of the MPEG‐4/AFX standard. Copyright © 2009 John Wiley & Sons, Ltd.

In this paper we present an on-line platform, called MyMultiMediaWorld.com, designed to help standardization bodies and communities to benchmark compression algorithms for still 3D objects. Based on MPEG standards, this platform provides an easy-to-use API to integrate the algorithms to be evaluated. The platform automatically benchmarks them from the platform extensible 3D database. When uploading new 3D contents, benchmark results for the various available compression schemes are automatically updated. Filtering tools are provided to help performance assessment at scales ranging from individual 3D content to database subsets.

This paper presents a new compression technique for 3D dynamic meshes, referred to as FAMC - frame-based animated mesh compression, recently promoted within the MPEG-4 standard as Amendment 2 of part 16 AFX (Animation Framework extension). The heart of the method is a skinning model optimally computed from a frame-based representation and exploited for compression purposes within the framework of a motion compensation strategy. The proposed encoder offers high compression performances (gains in bitrate of 60% with respect to the previous MPEG-4 technique and of 20 to 40% with respect to state-of-the-art approaches) and is well suited for compressing both geometric and photometric attributes.

The MPEG-4 standard adopted a novel technology for compression of dynamic 3D meshes with constant connectivity and time-varying geometry, referred to as FAMC - frame-based animated mesh compression. In this paper, we propose two optimizations of the FAMC approach, aiming at improving the compression efficiency. The first one is based on a PCA (principal component analysis) decomposition of the motion compensation error residuals. The second improves the bi-orthogonal (4-2) wavelet coding approach supported by the standard, by introducing an optimal bit allocation procedure, combined with an adapted quantization of wavelet coefficients. Experimental results show that both optimizations lead to significant gains in compression rate (about 20-30%) at low bitrates.

This paper presents a new compression technique for 3D dynamic meshes, referred to as FAMC - frame-based animated mesh compression, promoted within the MPEG-4 standard as amendment 2 of part 16 AFX (animation framework extension). The FAMC approach combines a model-based motion compensation strategy, with transform/predictive coding of residual errors. First, a skinning motion compensation model is automatically computed from a frame-based representation and then encoded. Subsequently, either 1) DCT/lifting wavelets or 2) layer-based predictive coding is employed to exploit remaining spatio-temporal correlations in the residual signal. The proposed encoder offers high compression performances (gains in bit rate of 60% with respect to the previous MPEG-4 technique and of 20% to 40% with respect to state-of-the-art approaches) and is well suited for compressing both geometric and photometric (normal vectors, colors...) attributes. In addition, the FAMC method supports a rich set of functionalities including streaming, scalability (spatial, temporal and quality) and progressive transmission.

This paper addresses the problem of layout analysis of handwritten letters using textural and spatial information with a bidimensional Markovian approach. In this framework, the layout extraction is viewed as a labeling problem which aims to find the optimal configuration of the Markov random field performed by the 2D dynamic programming method by E. Geoffrois (2003). Preliminary experiments have been led on a small part of the RIMES database by E.Augustine el al. (2006) for which an error rate of 15% has been achieved.

Abstract This paper proposes a novel approach for 3D mesh compression, based on a skinning animation technique. The core of the proposed method is a piecewise affine predictor coupled with a skinning model and a DCT representation of the residuals errors. The experimental evaluation shows that the proposed skinning‐based encoder outperforms (with bitrates gains from 47% to 67%) GV, RT, MPEG‐4/AFX‐IC, D3DMC, PCA and Dynapack techniques. Copyright © 2006 John Wiley & Sons, Ltd.

In this paper, we present a two-dimensional approach of the processing of handwriting. It combines a Markovian model, an efficient decoding algorithm, a windowed spectral features extraction scheme and a rigorous evaluation methodology. We applied this principle to a digit recognition task and to a word recognition task.

The paper gives a short overview on the new feature of the recent MPEG-4 standard, namely interactive facilitation of multimedia contents. Some experiments on dealing with BIFS and MPEG-J - the two main motors enabling the interactivity in MPEG-4 - are presented. These experiments serve for two targets. First, they demonstrate a very perspective capability of MPEG-4 based multimedia contents, which are not only flexible in composition but also make interaction with viewer possible. Next, they expose a gap occurring in merging the computing and broadcasting technology in MPEG-4, namely temporal constraint versus random access. The work also outlines some solution to overcome this problem. The goal is to improve the performance of the interactivity enabling feature so that MPEG-4 standard actually integrates the computing capability into compression technique, enabling the generation of sophisticated multimedia scene.

Abstract This paper addresses the issue of non‐invasive investigation and functional assessment of pulmonary airways reconstructed from multi detector computed tomography clinical acquisitions. Such an analysis combines accurate 3D meshing of the inner bronchial wall surface and navigation and interactivity tools based on a robust central axis representation. A reliable endoluminal investigation of airways via virtual bronchoscopy is possible regardless of their anatomical/pathological specificity (small caliber bronchi, severe stenoses, …). Computational fluid dynamics simulations on real airway geometries allow to assess functional modifications induced by physiopathological changes. Copyright © 2004 John Wiley & Sons, Ltd.

Enriched multimedia applications and services aim at combining images, sounds, videos, and synthetic objects into hybrid and interactive scenes. The core technologies discussed here deal with the representation and integration within such complex scenes of a specific kind of synthetic data, namely virtual character animation. This paper analyzes how an integrated and standardized framework is currently emerging in order to ensure application interoperability, universal content access, and user interactivity. We first compare how virtual character animation has been addressed within virtual reality modeling language (VRML) and MPEG-4 synthetic and natural hybrid coding standardization processes. A comparative synthesis between the objectives and the capabilities of each framework is exposed, specifically MPEG-4 Face and Body Animation (FBA) versus H-Anim (Humanoid Animation Working Group, WEB3D Consortium, h-anim.org.) 1.1, and MPEG-4 Bone-Based Animation (BBA) versus H-Anim 2001. The Animation Framework eXtension (AFX) specifications that are a part of MPEG-4 Systems Part 16 include the BBA framework. The BBA animation concepts, based on generic skeleton representation and curve-based deformations, are introduced. The definition of the related nodes and how they successfully address the BBA concepts are as well discussed. Some comments with respect to the rotation representation, the interpolation methods, the animation mask, and value parameters of the animation stream are made. This set of specifications provides an efficient framework, which is appropriate to real-time and animation realistic applications within networked environments.

In the framework of computer-aided diagnosis, this paper proposes a novel functionality for the computerized tomography (CT)-based investigation of the pulmonary airways. It relies on an energy-based three-dimensional (3-D) reconstruction of the bronchial tree from multislice CT acquisitions, up to the sixth- to seventh-order subdivisions. Global and local analysis of the reconstructed airways is possible by means of specific visualization modalities, respectively, the CT bronchography and the virtual bronchoscopy. The originality of the 3-D reconstruction approach consists in combining axial and radial propagation potentials to control the growth of a subset of low-order airways extracted from the CT volume by means of a robust mathematical morphology operator-the selective marking and depth constrained (SMDC) connection cost. The proposed approach proved to be robust with respect to a large spectrum of airway pathologies, including even severe stenosis (bronchial lumen obstruction/collapse). Validated by expert radiologists, examples of airway 3-D reconstructions are presented and discussed for both normal and pathological cases. They highlight the interest in considering CT bronchography and virtual bronchoscopy as complementary tools for clinical diagnosis and follow-up of airway diseases.

controls how far from the winning neuron the input signal can reach. In order to find the global optimum of the objective function, the decrease of the size of the radius of the circular region around the position of the winning neuron should be done gradually. We have found out experimentally that a choice of in the range 150 to 250 and in the range 0.99 to 1.0 gives satisfactory results. Regarding the value of the parameter , experiments have shown that it should be near unity and in particular in the range 0.995 to 1.0. It should be emphasized that the algorithm is not critically influenced by the specific chosen values of the aforementioned parameters. The condition given by (11) is necessary to protect the SOM network from generating a signal that is much more different than the weighting vectors of the neurons in the current iteration. The condition is not too strict in the sense that it is not prohibitive of the production of a winner with a measure of match lower than that in the previous iteration. Consequently, it is achieved the escape from local optima of the objective function, since solutions that do not lead to better values of the objective function are accepted too. This approach is similar to the Metropolis process that takes place in simulated annealing [12]. Another aspect of the proposed registration scheme that should be addressed is the utilization of the gradient difference as a measure of match. The selected measure of match provided accurate results for almost all image pairs that were used in the current study. However, it should be stressed that the gradient difference may fail for images characterized by apparent hyperfluorescence. Hyperfluorescence is caused usually by a breakdown or lack of tight vascular junctions in abnormal blood vessels. Any leakage of fluorescein from a retinal vessel or within the retinal tissues indicates an abnormality. For example, capillary microaneurysms, retinal telangiectasias, arterial macroanuerysm, papilledema, and some vascularized tumors exhibit leakage. Finally, it should be noted that the focus of the paper was to show the feasibility of the SOM theory for registering multimodal retinal images. The proposed implementation of the SOM model could be considered as a method for finding the optimum value of an objective function. Under this framework, the proposed registration scheme provides several “degrees of freedom” regarding its parameters. For example, another measure of match (such as the mutual information) could be used, the bifurcation points could be extracted by means of other methods, another transformation (such as bilinear or elastic) could be adopted after the training of the SOM network. VI. CONCLUSION In this paper, a new registration algorithm for registering multimodal retinal images was presented. Two basic novel implementations were introduced. First, the application of the vessel centerline detection and bifurcations extraction process only on the reference image. This step simplifies the registration methodology since candidate points are identified only on the reference image. Second, the novel implementation of the SOM network to define automatic correspondence of the bifurcation points between the reference and the image to be transformed. The proposed algorithm was tested for 24 pairs of multimodal images providing an accuracy of approximately 40 for all retinal pairs.

We address the problem of robust non rigid image registration and focus our interest on similarity criteria based on generalized information measures within the Ali-Silvey class. We introduce exclusive f-information as a novel class of similarity functionals, and develop a generic variational framework for their optimization over nonparametric and parametric transform spaces. This approach is applied to blind nonrigid alignment of face images with arbitrary lighting conditions and complex deformations.

This paper explores the new features of the recent MPEG- 4 standard, namely the description and the composition of an interactive multimedia scene. Providing with such advanced facilities, the MPEG-4 standard is no longer considered only as an efficient compression tool for conventional video / audio application. It is now granted the power of computing mechanism as any other programming languages to facilitate the creation of high interactive multimedia contents. Game-applications therefore can be also considered as a new output-feature of the MPEG-4 standard. The work describes in detail how game can be created within the MPEG-4 framework. The process of making game with MPEG-4 is studied comprehensively to demonstrate the compatibility as well as the advantages of the MPEG-4 standard regarding to Flash technology, a well-known tool for making games over the Internet.

The MPEG-4 face and body animation (FBA) specifications aims at standardizing an interchange format for specifying virtual face and body modeling and related animation parameters. The scene graph concept, the FBA nodes, semantics and representation of the animation parameters as well as compression and streaming capabilities are successively addressed. The compatibility between virtual character animation techniques and MPEG-4 FBA is analyzed in terms of modeling, deformation behavior and animation capabilities.

The paper presents an advanced animation framework for virtual characters within the MPEG-4 standard. The bone-based animation (BBA) specifications are described and it is shown how they extend the MPEG-4 face and body animation (FBA) specifications. Specifically, the scene graph concept, the BBA nodes related to bone and muscle modeling, the animation parameters and the compression capabilities are successively addressed and discussed within a specific application related to a sign language communication system.

This paper addresses the issue of 3D mesh indexation by using shape descriptors (SDs) under constraints of geometric and topological invariance. A new shape descriptor, the canonical 3D Hough transform descriptor (C3DHTD) is here proposed. Intrinsically topologically stable, the C3DHTD is not invariant to geometric transformations. Nevertheless, we show mathematically how the C3DHTD can be optimally associated (in terms of compactness of representation and computational complexity) with a spatial alignment procedure which leads to a geometric invariant behavior. Experiments carried out upon the categorized MPEG-7 3D model database objectively show that the C3DHTD outperforms both the MPEG-7 3D SD and classic EGIs descriptor.

We present a generic and robust method for model-based global 3D head pose estimation in monocular and non-calibrated video sequences. The proposed method relies on a 3D/2D matching between 2D image features estimated throughout the sequence and 3D object features of a generic head model. Specifically, it combines motion and texture features in an iterative optimization procedure based on the downhill simplex algorithm. A proper initialization of the pose parameters, based on a block matching procedure, is performed at each frame in order to take into account large amplitude motions. For the same reason, we have developed a nonlinear optical flow-based interpolation algorithm for increasing the frame rate. Experiments demonstrate that this method is stable over extended sequences including large head motions, occlusions, various head postures and lighting variations. The estimation accuracy is related to the head model, as established by using an ellipsoidal model and an ad hoc synthesized model. The proposed method is general enough to be applied to other tracking applications.

We propose in this paper a new 3D fully parallel thinning algorithm that we believe to be the most concise due to its simple characterization. The algorithm is indeed completely defined by a set of five patterns, three removing conditions and two non-removing conditions. These patterns are designed from the two fundamental and compatible constraints usually expected in skeleta: (1) Topology preservation and (2) Medial surface. From these two constraints, the removing patterns terns (/spl alpha//sub 1/, /spl alpha//sub 2/ and /spl alpha//sub 3/) detect the non-local maxima, whereas the non-removing patterns (/spl beta//sub 1/ and /spl beta//sub 2/) prevent any topology change that the removing conditions could imply. We show that the three mentioned constraints are respected. The logical conciseness of our procedure, called MB-3D, makes it to our knowledge the easiest 3D thinning algorithm to implement. Some results are displayed, that illustrate the relevance of our approach.

In this paper, we tackle the problem of estimating textural parameters. We do not consider the problem of texture synthesis, but the problem of extracting textural features for tasks such as image segmentation. We take into account nonstationarities occurring in the local mean. We focus on Gaussian Markov random fields for which two estimation methods are proposed, and applied in a nonstationary framework. The first one consists of extracting conditional probabilities and performing a least square approximation. This method is applied to a nonstationary framework, dealing with the piecewise constant local mean. This framework is adapted to practical tasks when discriminating several textures on a single image. The blurring effect affecting edges between two different textures is thus reduced. The second proposed method is based on renormalization theory. Statistics involved only concern variances of Gaussian laws, leading to Cramer-Rao estimators. This method is thus especially robust with respect to the size of sampling. Moreover, nonstationarities of the local mean do not affect results. We then demonstrate that the estimated parameters allow texture discrimination for remote sensing data. The first proposed estimation method is applied to extract urban areas from SPOT images. Since discontinuities of the local mean are taken into account, we obtain an accurate urban areas delineation. Finally, we apply the renormalization based on method to segment ice in polar regions from AVHRR data.

The Voronoi diagram (VD) is a popular tool for partitioning the support of an image. An algorithm is presented for constructing VD when the seed set, which determines the Voronoi regions, can be modified by adding and removing seeds. The number of pixels and seeds revisited for updating the diagram and the neighbor relationships among seeds is minimized. A result on cocircular seeds is presented. The adjacency, or dual, graph of the VD is readily obtained. The use of the algorithm for constructing skeletons by influence zones (SKIZ) is demonstrated.

This paper investigates structural properties of diffusive scale-spaces and develops a Riemannian description based on electromagnetic (EM) field theory. The generalized diffusion equation defining photometric transitions is interpreted as a Lorentz gauge condition expressing the trace Lorentz-invariance of an EM quadripotential with covariant scalar and contravariant vector components, respectively related to photometric and geometric image properties. This gauge condition determines EM quadrifield and quadricharge, which satisfy Maxwell equations. Deriving their general expressions as functions of scale-space geometric or energetic features yields Lorentz-invariants which synthesize intrinsic multiscale image properties.

A fundamental problem in the field of off-line signature verification is the lack of a pertinent shape representation or shape factor. The main difficulty in the definition of pertinent features lies in the local variability of the signature line which is closely related to the intrinsic characteristic of human beings. We propose a new formalism for signature representation based on visual perception. A signature image of 512/spl times/128 pixels is centered onto a grid of rectangular retinas which are excited by a local portion of the signature image. So each retina has only a local perception of the entire scene. Granulometric size distributions have been used for the definition of local shape descriptors in attempt to characterized the amount of signal activity in front of each retina located on the focus of attention grid. Experimental evaluation of this scheme has been made using a signature database of 800 genuine signatures from 20 individuals. Two types of classifiers, a INN and a threshold classifiers show a total error rate below 0.02% and 1.0% respectively in the context of random forgeries.