Accurate perception of lunar surfaces is critical for modern lunar exploration missions. However, developing robust learning-based perception systems is hindered by the lack of datasets that provide both geometric and photometric supervision. Existing lunar datasets typically lack either geometric ground truth, photometric realism, illumination diversity, or large-scale coverage. In this paper, we introduce MoonAnything, a unified benchmark built on real lunar topography with physically-based rendering, providing the first comprehensive geometric and photometric supervision under diverse illumination with large scale. The benchmark comprises two complementary sub-datasets : i) LunarGeo provides stereo images with corresponding dense depth maps and camera calibration enabling 3D reconstruction and pose estimation; ii) LunarPhoto provides photorealistic images using a spatially-varying BRDF model, along with multi-illumination renderings under real solar configurations, enabling reflectance estimation and illumination-robust perception. Together, these datasets offer over 130K samples with comprehensive supervision. Beyond lunar applications, MoonAnything offers a unique setting and challenging testbedfor algorithms under low-textured, high-contrast conditions and applies to other airless celestial bodies and could generalize beyond. We establish baselines using state-of-the-art methods and release the complete dataset along with generation tools to support community extension: https://github.com/clementinegrethen/MoonAnything.

The rise of Extended Reality$(X R)$requires efficient streaming of 3D online worlds, challenging current 3DGS representations to adapt to bandwidth-constrained environments. This paper proposes LapisGS, a layered 3DGS that supports adaptive streaming and progressive rendering. Our method constructs a layered structure for cumulative representation, incorporates dynamic opacity optimization to maintain visual fidelity, and utilizes occupancy maps to efficiently manage Gaussian splats. This proposed model offers a progressive representation supporting a continuous rendering quality adapted for bandwidth-aware streaming. Extensive experiments validate the effectiveness of our approach in balancing visual fidelity with the compactness of the model, with up to 50.71 % improvement in SSIM, 286.53% improvement in LPIPS with 23% of the original model size, and shows its potential for bandwidth-adapted 3D streaming and rendering applications. Project page: https://yuang-ian.github.io/lapisgs/

We tackle the challenge of efficiently reconstructing 3D scenes with high detail on objects of interest. Existing 3D Gaussian Splatting (3DGS) methods allocate resources uniformly across the scene, limiting fine detail to Regions Of Interest (ROIs) and leading to inflated model size. We propose ROI-GS, an object-aware framework that enhances local details through object-guided camera selection, targeted Object training, and seamless integration of high-fidelity object of interest reconstructions into the global scene. Our method prioritizes higher resolution details on chosen objects while maintaining real-time performance. Experiments show that ROI-GS significantly improves local quality (up to 2.96dB PSNR), while reducing overall model size by ≈ 17% of baseline and achieving faster training for a scene with a single object of interest, outperforming existing methods.

This paper proposes blind mesh quality measures for a reconstructed 3D model. The assessment of 3D model quality is a fundamental step in the process of 3D reconstruction, to efficiently and iteratively improve the model quality. We first prove that metrics based on a reference model (extrinsic metrics) that have been shown to be correlated to subjective interpretation, are able to capture flaws that may occur while reconstructing. However, no reference is available during an iterative reconstruction process, so we study intrinsic measures on a 3D model to evaluate the quality of the 3D model being reconstructed. As expected, our results show that these intrinsic metrics give high responses in regions locally corrupted by noises.

3D reconstruction of insects from photographs is a challenging task as it requires to tackle several problems such as strong out-of-focus areas in macro-photography, thin structures (insect legs and hairs), flat-colored surfaces (insects shells), non-Lambertian (shells specularities) and even translucent surfaces (wings). In this work, we first present a new lens-based image registration technique for accurate multi-focus stacking suitable for 3D reconstruction purposes while other methods create in-focus images for viewing purpose only. We then evaluate and compare the classical Multi-View-Stereo (MVS) reconstruction pipeline for small and complex objects with recent deep learning-based reconstruction methods such as the Neural Radiance Fields (NeRF) and the Neural Sparse Voxel Fields (NSVF). We present an assessment of different sources of errors for the considered methods. The results are compared both quantitatively and qualitatively across the different methods. From this analysis we present a series of practical guidelines for addressing the common issues of the reconstruction of small objects under challenging conditions.

This paper introduces the Meshroom software and its underlying 3D computer vision framework AliceVision. This solution provides a photogrammetry pipeline to reconstruct 3D scenes from a set of unordered images. It also features other pipelines for fusing multi-bracketing low dynamic range images into high dynamic range, stitching multiple images into a panorama and estimating the motion of a moving camera. Meshroom's node-graph architecture allows the user to customize the different pipelines to adjust them to their domain specific needs. The user can interactively add other processing nodes to modify a pipeline, export intermediate data to analyze the result of the algorithms and easily compare the outputs given by different sets of parameters. The software package is released in open source and relies on open file formats. These features enable researchers to conveniently run the pipelines, access and visualize the data at each step, thus promoting the sharing and the reproducibility of the results.

A major research area in Computer Assisted Intervention (CAI) is to aid laparoscopic surgery teams with Augmented Reality (AR) guidance. This involves registering data from other modalities such as MR and fusing it with the laparoscopic video in real-time, to reveal the location of hidden critical structures. We present the first system for AR guided laparoscopic surgery of the uterus. This works with pre-operative MR or CT data and monocular laparoscopes, without requiring any additional interventional hardware such as optical trackers. We present novel and robust solutions to two main sub-problems: the initial registration, which is solved using a short exploratory video, and update registration, which is solved with real-time tracking-by-detection. These problems are challenging for the uterus because it is a weakly-textured, highly mobile organ that moves independently of surrounding structures. In the broader context, our system is the first that has successfully performed markerless real-time registration and AR of a mobile human organ with monocular laparoscopes in the OR.

Diminished Reality (DR) consists in virtually removing objects from a captured scene, thus requiring a coherent filling of the areas originally hidden behind these objects. Indoor DR applications often exploit the planar geometry of the scene to apply an inpainting process on a perspectively undistorted view of the plane. In this paper we propose to integrate a novel, physic-based criterion into classical state-of-the-art inpainting algorithms in order to take into account the variations in image resolution of the undistorted view. The proposed inpainting process selects the patches and avoids the propagation of low-resolution data, i.e. patches corresponding to parts of the plane that are far from the camera or seen under an very skew angle. We illustrate the improvements of DR results on synthetic and real images.

Using fiducial markers ensures reliable detection and identification of planar features in images. Fiducials are used in a wide range of applications, especially when a reliable visual reference is needed, e.g., to track the camera in cluttered or textureless environments. A marker designed for such applications must be robust to partial occlusions, varying distances and angles of view, and fast camera motions. In this paper, we present a robust, highly accurate fiducial system, whose markers consist of concentric rings, along with its theoretical foundations. Relying on projective properties, it allows to robustly localize the imaged marker and to accurately detect the position of the image of the (common) circle center. We demonstrate that our system can detect and accurately localize these circular fiducials under very challenging conditions and the experimental results reveal that it outperforms other recent fiducial systems.

In this paper we present the Toulouse Vanishing Points Dataset, a public photographs database of Manhattan scenes taken with an iPad Air 1. The purpose of this dataset is the evaluation of vanishing points estimation algorithms. Its originality is the addition of Inertial Measurement Unit (IMU) data synchronized with the camera under the form of rotation matrices. Moreover, contrary to existing works which provide vanishing points of reference in the form of single points, we computed uncertainty regions. The Toulouse Vanishing Points Dataset is publicly available at http://ubee.enseeiht.fr/tvpd

The pervasiveness of mobile devices increases the risk of exposing sensitive information on the go. In this paper, we arise this concern by presenting an automatic attack against modern touchscreen keyboards. We demonstrate the attack against the Apple iPhone - 2010's most popular touchscreen device - although it can be adapted to other devices (e.g., Android) that employ similar key-magnifying keyboards. Our attack processes the stream of frames from a video camera (e.g., surveillance or portable camera) and recognizes keystrokes online, in a fraction of the time needed to perform the same task by direct observation or offline analysis of a recorded video, which can be unfeasible for large amount of data. Our attack detects, tracks, and rectifies the target touchscreen, thus following the device or camera's movements and eliminating possible perspective distortions and rotations In real-world settings, our attack can automatically recognize up to 97.07 percent of the keystrokes (91.03 on average), with 1.15 percent of errors (3.16 on average) at a speed ranging from 37 to 51 keystrokes per minute.

We present a novel calibration technique for all central catadioptric cameras using images of planar grids. We adopted the well-known sphere camera model to describe the catadioptric projection. We show that, using the so-called lifted coordinates, a linear relation mapping the grid points to the corresponding points on the image plane can be written as a 6 × 6 matrix Hcata, which acts like the classical 3 × 3 homography for perspective cameras. We show how to compute the image of the absolute conic (IAC) from at least 3 homographies and how to recover from it the intrinsic parameters of the catadioptric camera. In the case of paracatadioptric cameras one such homography is enough to estimate the IAC, thus allowing the calibration from a single image.

Segment-based maps are emerging as an efficient way to represent the environments in which mobile robots operate. When compared to grid-based maps, maps composed of line segments usually need less space to be stored. However, very little effort has been devoted to methods that allow to reduce the size of segment-based maps by removing redundant line segments that represent the same object in the environment. This problem is usually addressed with ratheradhocmethods that are embedded in mapping systems. In this paper, we put forward the problem of reducing the size of segment-based maps by presenting a survey of the existing methods and by experimentally evaluating some of them. Our results can be used to set out some guidelines for the development of a general approach to reducing redundant line segments in maps.

In this paper we address the problem of sphere localization from a single image taken with a noncentral catadioptric camera. We propose a method for determining both the radius and the position of an unknown sphere from a single, catadioptric image. The method can find its application in the field of robotic vision, especially in mobile robots playing soccer in RoboCup contests, in order to improve robot capabilities related to playing with a flying ball. Recently, a method for sphere reconstruction from single image taken with a noncentral, axial-symmetric, catadioptric camera has been proposed. In an axial symmetric catadioptric cameras, the pinhole of the camera is placed on the mirror axis. Though axial symmetric cameras help to simplify the geometrical treatment of the problem, they are difficult to set up since they require a precise alignment, usually hard to check. In this paper we deal with the general case of off-axis catadioptric cameras, with the camera pinhole placed in a general position w.r.t. the mirror. We devise a simple geometrical method by which we determine both the position of a sphere and its radius from its apparent image contour. Since our approach is based on coplanar viewing rays, it has a wider applicability w.r.t. the previous method, as it relaxes the constraint on camera position, i.e. it does not require a precise alignment, and the constraint on mirror axial symmetry, i.e. it can be applied to a wider class of mirrors. Some preliminary experiments both on simulated and real image are also presented.

Line localization from a single image of a central camera is an ill-posed problem unless other constraints or apriori knowledge are exploited. Recently, it has been proved that noncentral catadioptric cameras allow space lines to be localized from a single image. In this paper we propose two novel localization algorithms. The first method exploits a pair of coplanar viewing rays to localize the space line. The second method follows a constrained non-linear minimization procedure using a suitable parametrization to represent space lines. We compare the accuracy of the proposed method w.r.t. the classical line localization algorithm and two robust variants of it. We carried out both synthetic and real experiments and evaluated the performance in localizing a set of space lines. We also propose a quality index for the viewing surfaces associated to space lines in order to better evaluate the quality of the localization. The experimental results showed the effectiveness and the accuracy of both proposed methods.

We present a novel calibration method for off-axis catadioptric cameras, i.e. standard perspective cameras placed in a generic position w.r.t. an axial-symmetric mirror of unknown shape. The proposed method estimates the intrinsic parameters of the natural perspective camera, the 3D shape of the mirror and its pose w.r.t. the camera. The peculiarity of our approach is that, unlike several other calibration methods, we do not require any cross section of the mirror to be visible in the image. Instead, we require that the catadioptric image contains at least the image of one generic space line. We then derive some constraints that, combined with the harmonic homology relating the apparent contours of the mirror, allow us to calibrate the off-axis camera. We provide experimental results both on synthetic and camera images that prove the validity of the technique.

A way to significantly advance robotic science is to perform experiments that can be replicated by other researchers and be used to compare different methods. This happens rarely in current robotics research. In this paper we present a methodology for performing experimental activities in the area of robotic mapping. The proposed methodology prescribes a number of issues that should be addressed when experimentally validating a mapping method. We present the application of the proposed methodology to a mapping system we have developed.

In-door environments often contain several straight line segments. The 3D reconstruction of such environments can thus reduce to the localization of lines in the 3D space. Multi-view reconstruction requires the solution of the correspondence problem. The use of a single image to localize space lines is attractive, since the correspondence problem can be avoided. However, using a perspective camera (or a central one), a line can not be localized, since its viewing surface is planar, and hence it can contain infinite lines other than the correct one. In this paper we study the number of planar viewing surfaces for a general class of catadioptric cameras, constituted by an axial symmetric mirror and a perspective camera placed at generic relative position. We show that, under broad conditions, there is only a discrete set of planar viewing surfaces for the considered class of cameras. This result establishes a qualitative difference with respect to axial-symmetric cameras (e.g., catadioptric cameras constituted by an axial-symmetric mirror plus a perspective camera, whose viewpoint is constrained to be on the mirror axis), where an infinite set of planar viewing surfaces exists. Then, some conditions are derived for the localization of lines in the 3D space from single images. Preliminary experiments are also reported.

Most map building methods employed by mobile robots are based on the assumption that an estimate of robot poses can be obtained from odometry readings or from observing landmarks or other robots. In this paper we propose methods to build a global geometric map by integrating scans collected by laser range scanners without using any knowledge about the robots' poses. We consider scans that are collections of line segments. Our approach increases the flexibility in data collection, since robots do not need to see each other during mapping, and data can be collected by multiple robots or a single robot in one or multiple sessions. Experimental results show the effectiveness of our approach in different types of indoor environments.

The reconstruction of 3D scenes constituted by straight lines can find many applications both in computer vision and in mobile robotics. Most of the approaches to this problem involve either stereo-vision or the analysis of a sequence of images taken from different viewpoints: in both cases, the solution of the correspondence problem is required. This paper studies the localization of straight lines in 3D space from single ID images, acquired by a catadioptric camera. In general, using a noncentral camera, the viewing rays starting from the points of a straight line constitute a non-planar surface: if this non-planar surface only contains one straight line, other than the viewing rays, then the straight line can univocally be localized. Some conditions for the univocal localization of straight lines are derived. A simple technique for the straight line localization is presented, and some preliminary experimental results are discussed.

The map building methods usually employed by mobile robots are based on the assumption that an estimate of the position of the robot can be obtained from odometry readings. In this paper we propose three methods that build a geometrical global map by integrating partial maps without using any odometry information. We focus on the problem of integrating a sequence of partial maps that specifies the order in which the partial maps must be integrated. Experimental results show the effectiveness of our approach in different types of environments.