International audience

10.1145/2348816.2348820

3D triangular meshes are becoming an increasingly prevalent data type in networked applications such as digital museums, online games, and virtual worlds. In these applications, a 3D mesh is typically coded progressively, yielding a multiresolution representation suitable for streaming. While such progressive coding allows incremental rendering for users while data is being transmitted, it introduces dependencies between data, causing delay in rendering when packets are lost. This article quantitatively analyzes the effects of such dependency by modeling the distribution of decoding time as a function of mesh properties and network parameters. We apply our model to study two extreme cases of dependency in progressive meshes and show that the effect of dependencies on decoded mesh quality diminishes with time. Our model provides the expected decoded mesh quality at the receiver at a given time. Based on this expected value, we propose a packetization strategy that improves the decoded mesh quality during the initial stage of streaming. We validate the accuracy of our model under a variety of network conditions, including bursty losses, fluctuating RTT, and varying sending rate. The values predicted from our model match the measured value reasonably well in all cases except when losses are too bursty.

Screen size and display resolution limit the experience of watching videos on mobile devices. The viewing experience can be improved by determining important or interesting regions within the video (called regions of interest, or ROIs) and displaying only the ROIs to the viewer. Previous work focuses on analyzing the video content using visual attention model to infer the ROIs. Such content-based technique, however, has limitations. In this paper, we propose an alternative paradigm to infer ROIs from a video. We crowdsource from a large number of users through their implicit viewing behavior using a zoom and pan interface, and infer the ROIs from their collective wisdom. A retargeted video, consisting of relevant shots determined from historical users behavior, can be automatically generated and replayed to subsequent users who would prefer a less interactive viewing experience. This paper presents how we collect the user traces, infer the ROIs and their dynamics, group the ROIs into shots, and automatically reframe those shots to improve the aesthetics of the video. A user study with 48 participants shows that our automatically retargeted video is of comparable quality to one handcrafted by an expert user

Nowadays, commercial websites use an increasing amount of rich media ads, since these ads grab users' attention more easily. Ad overloading has a counterproductive effect and we advocate a more parsimonious and finely-tuned use of ads. In this paper we stress the importance of using ads in the right sequence and at the right time. We also present how to use our web adaptation system for a classic web marketing multivariable test. The experimental results show the usefulness of optimizing banners' sequences and timing.

We present MARCH, a complete solution for enhanced cultural heritage discovery using the mobile phone. Simply point the camera of a mobile device at prehistoric cave engravings. Then MARCH augments the captured images with the expert's drawings, highlighting in real time the animal engravings, which are almost impossible to observe with the naked eye. We have created a mobile augmented reality application which runs at 14 FPS for 320x240 frames on a Nokia N95 smartphone. We describe the optimizations and the requirements needed to obtain these results on mobile devices.

We demonstrate MobiTree, a system we built that allows progressive streaming and rendering of 3D tree models on a mobile phone. MobiTree enables user to retrieve not only typical media describing a species (such as text and image), but also a 3D model that yields detail information about the structure of the branches and the foliage. MobiTree adopts our previous proposed progressive representation to speed up display of trees at the mobile client, trading off latency and quality. Progressivity also allows MobiTree to flexibly adopt the level of details to the capability of the mobile devices.

Just as in the real world, plants are important objects in virtual worlds for creating pleasant and realistic environments, especially those involving natural scenes. As such, much effort has been made in realistic modeling of plants. As the trend moves towards networked and distributed virtual environments, however, the current models are inadequate as they are not designed for progressive transmissions. In this article, we fill in this gap by proposing a progressive representation for plants based on generalized cylinders. We model the shape and thickness of branches in a plant as Bézier curves, group the curves according to the similarity, and differentially code the curves to represent the plant in a compact and progressive manner. To facilitate the transmission of the plants, we quantify the visual contribution of each branch and use this weight in packet scheduling. We show the efficiency of our representations and the effectiveness of our packet scheduler through experiments over a wide area network.

Recent developments of hardware capabilities on mobile devices have made Location Based Services(LBS) very popular. This triggered an increasing need of rich visual content in mobile guide applications.This paper presents two original approaches for using representative visualizations: artistic views which represent an arbitrary deformation made by an artist and perspective views(3D-like) obtained from 3D models. Both approaches are based on learning GPS-to-image relations. We show an efficient use of the thin-plate spline for registering GPS coordinates with images. We also show the implementation of our guiding system on two mobile platforms.

International audience

3D triangular mesh is becoming an increasingly important data type for networked applications such as digital museums, online games, and virtual worlds. In these applications, a multi-resolution representation is typically desired for streaming large 3D meshes, allowing for incremental rendering at the viewers while data is still being transmitted. Such progressive coding, however, introduces dependencies between data. This paper quantitatively analyzes the effects of such dependency on the intermediate decoded mesh quality when the progressive mesh is transmitted over a lossy network, by modeling the distribution of decoding time as a function of mesh properties and network parameters. To illustrate the usefulness of our analytical model, we describe three of its applications. First, we show how it can be used to analytically compute the expected decoded mesh quality. Second, we study two extreme cases of dependency in progressive mesh and show that the effect of dependencies on decoded mesh quality diminishes with time. Finally, based on the model, we propose a packetization strategy that improves the decoded mesh quality during the initial stage of streaming.

Highly interactive collaborative applications need to offer each user a consistent view of the interactions represented by the streams exchanged between dispersed groups of users. At the coordination level, strong ordering protocols for capturing and delivering streams' interactions (e.g. CAUSAL, TOTAL order) may be too expensive due to the variability of the network conditions. This paper builds upon previous work on expressing streams causality and proposes a flexible coordination middleware in order to integrate different delivery modes (e.g. FIFO, CAUSAL, TOTAL) into a single channel (with respect to each of these protocols). Moreover, the proposed abstract channel can handle the mix of any partial or total order protocols. We present a cooperative streaming scenario and an experimental platform for measuring the cost of combining several protocols on various network conditions and show how our coordination service can adjust the degree of synchronization according to network variations.

Highly interactive collaborative streaming applications express the need for causality. Solutions exist but we argue that more work needs to be done especially from a perceptual point of view. The key question is: given the current state of the Internet and the perceptual tolerance of causal desynchronization, does causality make any difference? This paper proposes a practical answer to this question by comparing different solutions. We support this comparison by producing video results for a live streaming scenario on an experimental platform. Further, this paper proposes a novel approach for handling causality in multimedia and shows that it can perform better than Δ-causality, usually considered the best solution.

Interaction with hypermedia documents is a required feature for new sophisticated yet flexible multimedia applications. This paper presents an innovative adaptive technique to stream hypervideo that takes into account user behaviour. The objective is to optimize hypervideo prefetching in order to reduce the latency caused by the network. This technique is based on a model provided by a Markov Decision Process approach. The problem is solved using two methods: classical stochastic dynamic programming algorithms and reinforcement learning. Experimental results under stochastic network conditions are very promising.