Accurate perception of lunar surfaces is critical for modern lunar exploration missions. However, developing robust learning-based perception systems is hindered by the lack of datasets that provide both geometric and photometric supervision. Existing lunar datasets typically lack either geometric ground truth, photometric realism, illumination diversity, or large-scale coverage. In this paper, we introduce MoonAnything, a unified benchmark built on real lunar topography with physically-based rendering, providing the first comprehensive geometric and photometric supervision under diverse illumination with large scale. The benchmark comprises two complementary sub-datasets : i) LunarGeo provides stereo images with corresponding dense depth maps and camera calibration enabling 3D reconstruction and pose estimation; ii) LunarPhoto provides photorealistic images using a spatially-varying BRDF model, along with multi-illumination renderings under real solar configurations, enabling reflectance estimation and illumination-robust perception. Together, these datasets offer over 130K samples with comprehensive supervision. Beyond lunar applications, MoonAnything offers a unique setting and challenging testbedfor algorithms under low-textured, high-contrast conditions and applies to other airless celestial bodies and could generalize beyond. We establish baselines using state-of-the-art methods and release the complete dataset along with generation tools to support community extension: https://github.com/clementinegrethen/MoonAnything.

The rise of Extended Reality$(X R)$requires efficient streaming of 3D online worlds, challenging current 3DGS representations to adapt to bandwidth-constrained environments. This paper proposes LapisGS, a layered 3DGS that supports adaptive streaming and progressive rendering. Our method constructs a layered structure for cumulative representation, incorporates dynamic opacity optimization to maintain visual fidelity, and utilizes occupancy maps to efficiently manage Gaussian splats. This proposed model offers a progressive representation supporting a continuous rendering quality adapted for bandwidth-aware streaming. Extensive experiments validate the effectiveness of our approach in balancing visual fidelity with the compactness of the model, with up to 50.71 % improvement in SSIM, 286.53% improvement in LPIPS with 23% of the original model size, and shows its potential for bandwidth-adapted 3D streaming and rendering applications. Project page: https://yuang-ian.github.io/lapisgs/

We tackle the challenge of efficiently reconstructing 3D scenes with high detail on objects of interest. Existing 3D Gaussian Splatting (3DGS) methods allocate resources uniformly across the scene, limiting fine detail to Regions Of Interest (ROIs) and leading to inflated model size. We propose ROI-GS, an object-aware framework that enhances local details through object-guided camera selection, targeted Object training, and seamless integration of high-fidelity object of interest reconstructions into the global scene. Our method prioritizes higher resolution details on chosen objects while maintaining real-time performance. Experiments show that ROI-GS significantly improves local quality (up to 2.96dB PSNR), while reducing overall model size by ≈ 17% of baseline and achieving faster training for a scene with a single object of interest, outperforming existing methods.

This paper proposes blind mesh quality measures for a reconstructed 3D model. The assessment of 3D model quality is a fundamental step in the process of 3D reconstruction, to efficiently and iteratively improve the model quality. We first prove that metrics based on a reference model (extrinsic metrics) that have been shown to be correlated to subjective interpretation, are able to capture flaws that may occur while reconstructing. However, no reference is available during an iterative reconstruction process, so we study intrinsic measures on a 3D model to evaluate the quality of the 3D model being reconstructed. As expected, our results show that these intrinsic metrics give high responses in regions locally corrupted by noises.

Navigating in a 3D networked virtual environment with a six-degree of freedom on a mobile device can be disorientating and challenging to users. In this technical demonstration, we show how 3D bookmarks can be used to simplify such interactions. Our system integrates the 3D bookmarks into a DASH-based network virtual environment in a DASH-compliant manner and is available on both the desktop and mobile.

In the context of 2D/3D registration, this paper introduces an approach that allows for matching features detected in two different modalities, photographs, and 3D models, by using a common 2D representation. More precisely, 2D images are matched with a set of depth images representing the 3D model. After introducing the concept of Curvilinear Saliency, which is related to curvature estimation, we propose a new ridge and valley detector for depth images rendered from 3D models. A variant of this detector is adapted to photographs, first by considering multi-scale features and second by integrating the focus curve principle. Finally, a registration algorithm determines the correct view of the 3D model and, thus, the pose of the photograph. This approach relies on the Histogram of Curvilinear Saliency (HCS), an adaptation of the Histogram of Oriented Gradients (HOG) to the proposed features in 2D and 3D. The presented results highlight both the quality of the features detected in terms of repeatability and the interest of the approach for registration and pose estimation.

Diminished Reality (DR) consists in virtually removing objects from a captured scene, thus requiring a coherent filling of the areas originally hidden behind these objects. Indoor DR applications often exploit the planar geometry of the scene to apply an inpainting process on a perspectively undistorted view of the plane. In this paper we propose to integrate a novel, physic-based criterion into classical state-of-the-art inpainting algorithms in order to take into account the variations in image resolution of the undistorted view. The proposed inpainting process selects the patches and avoids the propagation of low-resolution data, i.e. patches corresponding to parts of the plane that are far from the camera or seen under an very skew angle. We illustrate the improvements of DR results on synthetic and real images.

We demonstrate the use of DASH, a widely-deployed standard for streaming video content, for streaming 3D content in an NVE (Networked Virtual Environment) consisting of 3D geometry and associated textures. We have developed a DASH client for NVE to show how NVE benefits from the advantages of DASH: it offers a scalable, easy-to-deploy 3D streaming framework. In our system, the 3D content is first statically partitioned into compliant DASH data, and metadata is provided in order for the client to manage which data to download. Based on a proposed utility metric for geometry and texture at the different resolution, the client can choose the content to request depending on its viewpoint. We effectively provide a Web-based client to navigate through our sample 3D scene, while deriving the streaming requests from its computation of the necessary online parameters, in a receiver-driven manner.

DASH is now a widely deployed standard for streaming video content due to its simplicity, scalability, and ease of deployment. In this paper, we explore the use of DASH for a different type of media content -- networked virtual environment (NVE), with different properties and requirements. We organize a polygon soup with textures into a structure that is compatible with DASH MPD (Media Presentation Description), with a minimal set of view-independent metadata for the client to make intelligent decisions about what data to download at which resolution. We also present a DASH-based NVE client that uses a view-dependent and network dependent utility metric to decide what to download, based only on the information in the MPD file. We show that DASH can be used on NVE for 3D content streaming. Our work opens up the possibility of using DASH for highly interactive applications, beyond its current use in video streaming.

The advantage of skeleton-based 3D reconstruction is to completely generate a single 3D object from well chosen views. Having numerous views is necessary for a reliable reconstruction but projections of skeletons lead to different topologies. We reconstruct 3D objects with curved medial axis (whose topology is a tree) from the perspective skeletons on an arbitrary number of calibrated acquisitions. The main contribution is to estimate the 3D skeleton, from multiple images: its topology is chosen as the closest to those of the perspective skeletons on the set of images, which means that the number of topology changes to map the 3D skeleton topology to topologies on images is minimum. This way, the number of matched branches is maximum.

Object recognition based on local features computed at multiple locations is robust to occlusions, strong viewpoint changes and object deformations. These features should be repeatable, precise and distinctive. We present an operator for repeatable feature detection on depth images (relative to 3D models) as well as 2D intensity images. The proposed detector is based on estimating the curviness saliency at multiple scales in each kind of image. We also propose quality measures that evaluate the repeatability of the features between depth and intensity images. The experiments show that the proposed detector outperforms both the most powerful, classical point detectors (e.g., SIFT) and edge detection techniques.

This paper presents a multilevel analysis of 2D shapes and uses it to find similarities between the different parts of a shape. Such an analysis is important for many applications such as shape comparison, editing, and compression. Our robust and stable method decomposes a shape into parts, determines a parts hierarchy, and measures similarity between parts based on a salience measure on the medial axis, the Weighted Extended Distance Function, providing a multi-resolution partition of the shape that is stable across scale and articulation. Comparison with an extensive user study on the MPEG-7 database demonstrates that our geometric results are consistent with user perception.

A 3D bookmark in a networked virtual environment (NVE) provides a navigation aid, allowing the user to move quickly from its current viewpoint to a bookmarked viewpoint by simply clicking on the bookmark. In this paper, we first validate the positive impact that 3D bookmarks have in easing navigation in a 3D scene. Then, we show that, in the context of a NVE that streams content on demand from server to client, navigating with bookmarks leads to lower rendering quality at the bookmarked viewpoint, due to lower locality of data. We then investigate into how prefetching the 3D data at the bookmarks and precomputation of visible faces at the bookmarks help to improve the rendering quality.

In this paper we present the 2D Shape Structure database, a public, user-generated dataset of 2D shape decompositions into a hierarchy of shape parts with geometric relationships retained. It is the outcome of a large-scale user study obtained by crowdsourcing, involving over 1200 shapes in 70 shape classes, and 2861 participants. A total of 41,953 annotations has been collected with at least 24 annotations per shape. For each shape, user decompositions into main shape, one or more levels of parts, and a level of details are available. This database reinforces a philosophy that understanding shape structure as a whole, rather than in the separated categories of parts decomposition, parts hierarchy, and analysis of relationships between parts, is crucial for full shape understanding. We provide initial statistical explorations of the data to determine representative (“mean”) shape annotations and to determine the number of modes in the annotations. The primary goal of the paper is to make this rich and complex database openly available (through the website http://2dshapesstructure.github.io/index.html), providing the shape community with a ground truth of human perception of holistic shape structure.

We present a novel approach to reconstruct a 3D object from images corresponding to two different viewpoints: we estimate the skeleton of the object instead of its surface. The originality of the method is to be able to reconstruct a complete tubular 3D object from only two input images. Unlike classical reconstruction methods like multiview stereo, this approach does not rely on interest points but estimates the topology of the object and derives its surface. Our contributions are twofold. First, given two perspective images of the 3D shape, the projection of the skeleton is computed in 2D. Second the 3D skeleton is reconstructed from the two projections using triangulation and matching. A mesh is finally derived for each skeleton branch.

Online galleries of 3D models typically provide two ways to preview a model before the model is downloaded and viewed by the user: (i) by showing a set of thumbnail images of the 3D model taken from representative views (or keyviews); (ii) by showing a video of the 3D model as viewed from a moving virtual camera along a path determined by the content provider. We propose a third approach called preview streaming for mesh-based 3D objects: by streaming and showing parts of the mesh surfaces visible along the virtual camera path. This article focuses on the preview streaming architecture and framework and presents our investigation into how such a system would best handle network congestion effectively. We present three basic methods: (a) stop-and-wait , where the camera pauses until sufficient data is buffered; (b) reduce-speed , where the camera slows down in accordance to reduce network bandwidth; and (c) reduce-quality , where the camera continues to move at the same speed but fewer vertices are sent and displayed, leading to lower mesh quality. We further propose two advanced methods: (d) keyview-aware , which trades off mesh quality and camera speed appropriately depending on how close the current view is to the keyviews, and (e) adaptive-zoom , which improves visual quality by moving the virtual camera away from the original path. A user study reveals that our keyview-aware method is preferred over the basic methods. Moreover, the adaptive-zoom scheme compares favorably to the keyview-aware method, showing that path adaptation is a viable approach to handling bandwidth variation.

Publishers of 3D models online typically provide two ways to preview a model before the model is downloaded and viewed by the user: (i) by showing a set of thumbnail images of the 3D model taken from representative views (or keyviews); (ii) by showing a video of the 3D model as viewed from a moving virtual camera along a path determined by the content provider. We propose a third approach called preview streaming for mesh-based 3D object: by streaming and showing parts of the mesh surfaces visible along the virtual camera path. This paper focuses on the preview streaming architecture and framework, and presents our investigation into how such a system would best handle network congestion effectively. We study three basic methods: (a) stop-and-wait, where the camera pauses until sufficient data is buffered; (b) reduce-speed, where the camera slows down in accordance to reduce network bandwidth; and (c) reduce-quality, where the camera continues to move at the same speed but fewer vertices are sent and displayed, leading to lower mesh quality. We further propose a keyview-aware method that trades off mesh quality and camera speed appropriately depending on how close the current view is to the keyviews. A user study reveals that our keyview-aware method is preferred over the basic methods.

This paper introduces a new technique, called peer-assisted rendering, that aims to enable interactive navigation in a 3D networked virtual environment using a resource-constrained device, by speeding up the rendering. A resource-constrained client requests part of the rendered scenes from other peers with similar viewpoints within the virtual environment, and merges the rendered parts into its own view. This approach is more scalable than previous solutions based on server-based pre-rendering. The goal of this paper is to make a strong case for the feasibility of peer-assisted rendering through the following two messages. First, by analyzing a large number of user traces from a popular virtual world called Second Life, we show that there are surprisingly many users with similar viewpoints and encompass large number of common objects in their viewing areas, indicating that a client can potentially find multiple other peers that can assist in rendering. Second, by combining three different rendering methods, each contributing to rendering of different classes of objects in the scene, we show that it is possible for a client to render the scene efficiently with little visual artifacts.

3D triangular meshes are becoming an increasingly prevalent data type in networked applications such as digital museums, online games, and virtual worlds. In these applications, a 3D mesh is typically coded progressively, yielding a multiresolution representation suitable for streaming. While such progressive coding allows incremental rendering for users while data is being transmitted, it introduces dependencies between data, causing delay in rendering when packets are lost. This article quantitatively analyzes the effects of such dependency by modeling the distribution of decoding time as a function of mesh properties and network parameters. We apply our model to study two extreme cases of dependency in progressive meshes and show that the effect of dependencies on decoded mesh quality diminishes with time. Our model provides the expected decoded mesh quality at the receiver at a given time. Based on this expected value, we propose a packetization strategy that improves the decoded mesh quality during the initial stage of streaming. We validate the accuracy of our model under a variety of network conditions, including bursty losses, fluctuating RTT, and varying sending rate. The values predicted from our model match the measured value reasonably well in all cases except when losses are too bursty.

Screen size and display resolution limit the experience of watching videos on mobile devices. The viewing experience can be improved by determining important or interesting regions within the video (called regions of interest, or ROIs) and displaying only the ROIs to the viewer. Previous work focuses on analyzing the video content using visual attention model to infer the ROIs. Such content-based technique, however, has limitations. In this paper, we propose an alternative paradigm to infer ROIs from a video. We crowdsource from a large number of users through their implicit viewing behavior using a zoom and pan interface, and infer the ROIs from their collective wisdom. A retargeted video, consisting of relevant shots determined from historical users behavior, can be automatically generated and replayed to subsequent users who would prefer a less interactive viewing experience. This paper presents how we collect the user traces, infer the ROIs and their dynamics, group the ROIs into shots, and automatically reframe those shots to improve the aesthetics of the video. A user study with 48 participants shows that our automatically retargeted video is of comparable quality to one handcrafted by an expert user

We demonstrate MobiTree, a system we built that allows progressive streaming and rendering of 3D tree models on a mobile phone. MobiTree enables user to retrieve not only typical media describing a species (such as text and image), but also a 3D model that yields detail information about the structure of the branches and the foliage. MobiTree adopts our previous proposed progressive representation to speed up display of trees at the mobile client, trading off latency and quality. Progressivity also allows MobiTree to flexibly adopt the level of details to the capability of the mobile devices.

Just as in the real world, plants are important objects in virtual worlds for creating pleasant and realistic environments, especially those involving natural scenes. As such, much effort has been made in realistic modeling of plants. As the trend moves towards networked and distributed virtual environments, however, the current models are inadequate as they are not designed for progressive transmissions. In this article, we fill in this gap by proposing a progressive representation for plants based on generalized cylinders. We model the shape and thickness of branches in a plant as Bézier curves, group the curves according to the similarity, and differentially code the curves to represent the plant in a compact and progressive manner. To facilitate the transmission of the plants, we quantify the visual contribution of each branch and use this weight in packet scheduling. We show the efficiency of our representations and the effectiveness of our packet scheduler through experiments over a wide area network.

This paper presents a satellite image compression scheme based on a post-processing of the wavelet transform of images. The bandelet transform is a directional post-processing of wavelet coefficients. Thanks to a low computational complexity, this transform is a good candidate for future on-board satellite image compression systems. First, we analyze the ability of the bandelets to exploit directional correlations between wavelet coefficients. This study leads to an improved post-processing with a better decorrelation of adjacent wavelet coefficients in the vertical or in the horizontal direction taking into account the wavelet subband orientations. To perform even better decorrelation, bases are also build by principal component analysis (PCA). This results in an improved compression performance without increasing the computational complexity.

International audience

This paper studies the performance of the ZC contexts in JPEG2000. A quality criterion based on the mutual information is introduced. According to this criterion, there exist ZC contexts which are not effective in any bit-plane. They can thus be merged with others without significant loss of the compression efficiency. Since the contextual conditional probabilities strongly depend on the bit-planes, an adaptation of the contexts to the bit-planes is also proposed. First, an optimal context quantizer is used on each bit-planes, then a simple greedy entropy minimization procedure allows to combine the ZC contexts in JPEG2000. The results are compared with a global combination of JPEG2000 ZC contexts. Tests have been performed on a real satellite image data base. With only four ZC contexts, the PSNR is less than 0.01 dB lower than the PSNR with the nine ZC contexts.

3D triangular mesh is becoming an increasingly important data type for networked applications such as digital museums, online games, and virtual worlds. In these applications, a multi-resolution representation is typically desired for streaming large 3D meshes, allowing for incremental rendering at the viewers while data is still being transmitted. Such progressive coding, however, introduces dependencies between data. This paper quantitatively analyzes the effects of such dependency on the intermediate decoded mesh quality when the progressive mesh is transmitted over a lossy network, by modeling the distribution of decoding time as a function of mesh properties and network parameters. To illustrate the usefulness of our analytical model, we describe three of its applications. First, we show how it can be used to analytically compute the expected decoded mesh quality. Second, we study two extreme cases of dependency in progressive mesh and show that the effect of dependencies on decoded mesh quality diminishes with time. Finally, based on the model, we propose a packetization strategy that improves the decoded mesh quality during the initial stage of streaming.

Poisson approximation extends the Bernstein-Bezier scheme from polynomials to arbitrary analytic functions. A survey of some old and new results in Poisson approximation is presented with special emphasis on algorithmic techniques for analytic functions.