This paper addresses the challenge of utilizing learningbased algorithms for 3D scene reconstruction on resourceconstrained end-user devices. Although integrating deep learning methods into the reconstruction pipeline has demonstrated superior performance to classical techniques, the resulting large models could be impractical for resource-limited devices. We propose an efficient solution by introducing a method to compress deep learning models used in 3D reconstruction workflows. Our approach, named DRAFT, employs knowledge distillation (KD), adapted and extended for complex feature and context extraction tasks related to optical flow. New distillation components based on algebraic signpattern matrices (SPM) and inertia enhance the KD process. Empirical validation on KITTI and Sintel benchmark datasets reveals that DRAFT consistently achieves comparable or superior performance to state-of-the-art models such as RAFT, FlowID, GMFlow, and Anyflow while significantly reducing model size. This contribution enhances the feasibility of deploying learning-based 3D scene reconstruction frameworks on edge systems. It contributes to the discourse on resource-efficient deep learning methodologies, particularly for optical flow and stereo matching. Our code is available at https://github.com/christian-tchenko/DRAFT.git.

In this paper, the authors propose a visual-inertial algorithm to estimate the lateral velocity of a motorcycle traveling at high speed along a single-carriageway road. The approach comprises the following steps. First, a monocular camera captures real-time images of the road ahead. Lane markers present in the image are detected and segmented using image processing techniques. Next, a bird's eye view transform is applied, and the dashed center lane markers are isolated. The motion of these markers is computed using an image registration algorithm and is expressed in the motorcycle body frame using orientation estimates from an inertial measurement unit. Finally, this measurement is combined with readings from an accelerometer using a Kalman filter to produce a filtered estimate. The approach was validated using data from simulations of two scenarios created in the BikeSim simulation software suite. In the first scenario, the motorcycle performs a double lane-change across both lanes of a straight road. In the second, the motorcycle navigates an s-shaped bend.

Using spherical moments to estimate the rotation of camera is an effective method. However, after the camera rotates, the scene in the captured image will change, which will cause a certain error. This paper proposes a method to reduce this error by removing the non-common part between the two images during rotation. Through multiple iterations, the error caused by scene changing will become very small. The experimental results using manual generated image are provided to show the efficiency of this proposed method.

This paper introduces a Wearable SLAM system that performs indoor and outdoor SLAM in real time. The related project is part of the MALIN challenge which aims at creating a system to track emergency response agents in complex scenarios (such as dark environments, smoked rooms, repetitive patterns, building floor transitions and doorway crossing problems), where GPS technology is insufficient or inoperative. The proposed system fuses different SLAM technologies to compensate the lack of robustness of each, while estimating the pose individually. LiDAR and visual SLAM are fused with an inertial sensor in such a way that the system is able to maintain GPS coordinates that are sent via radio to a ground station, for real-time tracking. More specifically, LiDAR and monocular vision technologies are tested in dynamic scenarios where the main advantages of each have been evaluated and compared. Finally, 3D reconstruction up to three levels of details is performed.

This paper deals with the problem of 3D human tracking in catadioptric images using particle-filtering framework. While traditional perspective images are well exploited, only a few methods have been developed for catadioptric vision, for the human detection or tracking problems. We propose to extend the 3D pose estimation in the case of perspective cameras to catadioptric sensors. In this paper, we develop an original likelihood functions based, on the one hand, on the geodetic distance in the spherical space SO3 and, on the other hand, on the mapping between the human silhouette in the images and the projected 3D model. These likelihood functions combined with a particle filter, whose propagation model is adapted to the spherical space, allow accurate 3D human tracking in omnidirectional images. Both visual and quantitative analysis of the experimental results demonstrate the effectiveness of our approach.

This paper presents an image-based approach to estimate the motorcycle roll angle. The algorithm estimates directly the absolute roll to the road plane by means of a basic monocular camera. This means that the estimated roll angle is not affected by the road bank which is often a problem for vehicle observation and control purposes. For each captured image, the algorithm uses a numeric roll loop based on some simple knowledge of the road geometry. For each iteration, a bird-eye-view of the road is generated with the inverse perspective mapping technique. Then, a road marker filter associated with the well-known clothoid model are used respectively to track the road separation lanes and approximate them with mathematical functions. Finally, the algorithm computes two distinct areas between the two-road separation lanes. Its performances are tested by means of the motorcycle simulator BikeSim. This approach is very promising since it does not require any vehicle or tire model and is free of restrictive assumptions on the dynamics.

This paper presents a vision-based approach to prevent dangerous steering situations when riding a motorcycle in turns. The proposed algorithm is capable of detecting under, neutral or over-steering behavior using only a conventional camera and an inertial measurement unit. The inverse perspective mapping technique is used to reconstruct a bird-eye-view of the road image. Then, filters are applied to keep only the road markers which are, afterwards, approximated with the well-known clothoid model. This allows the prediction of the road geometry such as the curvature ahead of the motorcycle. Finally, from the predicted road curvature, the measurements of the Euler angles and the vehicle speed, the proposed algorithm is able to characterize the steering behavior. To that end, we propose to estimate the steering ratio and we introduce new pertinent indicators such as the vehicle relative position dynamics to the road. The method is validated using the advanced simulator BikeSim during a steady turn.

This paper presents new schemes to estimate 3D rotation from spherical images. Unlike existing approaches, spherical moment properties are exploited to obtain a closed form solution without iteratively mimimizing a cost function. Actually, three methods using spherical moments are proposed: two of them can be classified as dense approaches, while the third one is hybrid combining geometrical features with dense ones. Experimental results using both synthetic images and acquired images using catadioptric cameras with different scenarios show the effectiveness of our approach.

In this paper, we propose a dense approach for 3D rotation estimation between spherical images, which is simultaneously able to recover the large rotations, robust under clutter and small translations. The key idea is to represent the spherical images by 3D shapes of the triangular mesh surfaces based on image intensity signal. This allows to apply the spherical harmonics representation as 3D shape descriptor. The optimum rotation computation is recovered through the SVD decomposition of the cross covariance matrix, which is obtained from the two 3D shapes spherical harmonics coefficients. The performances of the proposed approach are examined using both synthetic and real image datasets. Experimental results show the effectiveness of our approach for rotation estimation, as well as its robustness against real conditions, image occlusions and small translations. The efficiency of the proposed approach is compared with that of competitive methods.

Omnidirectional cameras are commonly used in computer vision and robotics. Their main advantage is their wide field of view which allows them to acquire a 360 degree view of the scene with only one sensor and a single shot. However, few studies have investigated the human detection problem using this kind of cameras. In this paper, we propose to extend the conventional approach for human detection in perspective images and based on Histogram of Oriented Gradients (HOG) apdapted to spherical images is used for this issue. Our approach uses the Riemannian manifolds in order to adapt the gradient in the omnidirectional images. Several experiments have been done using INRIA image database; the results show that adapting detection and image database to the geometry of omnidirectional camera allows a robust detection, and significantly increases the performances.

This paper presents a novel rotation estimation approach based on 3D mesh representation of spherical images. Indeed, unit sphere is becoming a natural space for projecting images captured from the central cameras (conventional and non-conventional cameras, as omnidirectional camera) and obtaining the spherical images. The proposed method relies on representing the spherical images into a 3D space based on image intensities. Spherical harmonic coefficients are then calculated for the 3D mesh surfaces and used to estimate an initial rotation between the underlying spherical images in spectral domain. The optimal rotation is then refined through the ICP algorithm. Experimental results, using synthetic and real image dataset, demonstrate the effectiveness of the proposed approach for rotation estimation, as well as its robustness against real conditions and images occlusion. A comparison between the proposed method and competitive ones, is performed.

RGB-D sensor, like low cost Kinect, are widely used in robotics applications. Obstacle Avoidance (OA), Simultaneous Localization And Mapping (SLAM), Mobile Object Tracking (MOT) all are needing accurate information about the position of objects in the environment. 3D cameras are really convenient to realize those tasks but as low cost sensors, they have to be completed by other sensors : cameras, laser range finders, US or IR telemeters. In order to exploit all data sensors in a same algorithm, we have to express these data in a common reference frame. In other words we have to know the rigid transformation between sensor frames. In this paper, we propose a new method to retrieve rigid transformation (known as extrinsic parameters in calibration process) between a depth camera and a conventional camera. We show that such a method is accurate enough without the need of an user interaction nor a special calibration pattern unlike other common calibration processes.

This paper deals with the design of a generic visual tracking algorithm suitable for a large class of camera (single viewpoint sensors). It is based on the estimation of the relationship between observations and motion on the sphere. This is efficiently achieved using a kernel-based regression function on a generic linearly-weighted sum of non-linear basis functions. We also present two set of experiments. The first one shows the efficiency of our algorithm through the tracking in video sequences acquired with three types of cameras (conventional, dioptric-fisheye and catadioptric). The real-time performances will be shown by tracking one or several planes. The second set of experiments presents an application of our tracking algorithm to visual servoing with a fisheye camera.

We address the problem of vehicle (mobile robot) navigation by combining visual-based reconstruction and localization with metrical information given by the proprioceptive sensors such as the odometry sensor. The proposed approach extends the navigation system based on a monocular vision which is able to build a map and localize the vehicle in the real time way using only one camera. An extended Kalman filter is used to integrate odometric information to estimate the vehicle position. This position is updated by the localization obtained from the vision system. Experimental result carried out with an urban electric vehicle will show the improvement of the navigation system and its robustness to the temporary loss of images.

This paper extends the recent work proposed in [21]. In this work, it has been noted that three visual features (to control three degrees of freedom) obtained from the spherical projection of 3D spheres allows nice decoupling properties and global stability. However, even if such an approach is theoretically attractive, it is limited by a major practical issue since spherical objects have to be observed while only three degrees of freedom can be controlled. In this paper, we show that similar properties can be obtained by observing a set of points. The basic idea is to build a virtual 3D sphere from two 3D points and to analyse its related spherical projection. Furthermore, to control the six degrees of freedom a 2D 1/2 control scheme is proposed which allows us to fully decouple rotational motions from translational motions.

In this paper, we consider the problem of controlling a 6 DOF holonomic robot and a nonholonomic mobile robot from the projection of 3-D straight lines in the image plane of central catadioptric systems. A generic central catadioptric interaction matrix for the projection of 3-D straight lines is derived using an unifying imaging model valid for an entire class of cameras. This result is exploited to design an image-based control law that allows us to control the 6 DOF of a robotic arm. Then, the projected lines are exploited to control a nonholonomic robot. We show that as when considering a robotic arm, the control objectives are mainly based on catadioptric image feature and that local asymptotic convergence is guaranteed. Simulation results and real experiments with a 6 DOF eye-to-hand system and a mobile robot illustrate the control strategy.

This paper presents a hybrid decoupled vision-based control scheme valid for the entire class of central catadioptric sensors (including conventional perspective cameras). First, we consider the structure from motion problem using imaged 3D points. Geometrical relationships are exploited to enable a partial Euclidean reconstruction by decoupling the interaction between translation and rotation components of a homography matrix. The information extracted from the homography are then used to design a control law which allow us to fully decouple rotational motions from translational motions. Real time experimental results using an eye-to-hand robotic system with a paracatadioptric camera are presented and confirm the validity of our approach.

Motivated by the growing interest for omnidirectional sensors on robotic applications and particularly on vision-based control, we present a new framework to handle in a visual servoing scheme the projection of line features into the image plane of a central catadioptric camera. As it is well known, the projection of a 3D line in the image plane of a central catadioptric camera is a conic curve. We propose to use the polar line of the image center with respect to this conic curve to define the input of the vision-based control scheme. The visual observations obtained from the polar lines lead to a minimal representation of projected lines. An efficient control scheme based only on two image features can then be designed. Simulation and experimental results confirm the validity of our approach

This paper is related to the vision-based control of parallel robots. Indeed, a method is proposed to estimate the reduced set of kinematic parameters appearing in such a control. To do so, it extends a linear method, obtained for a perspective camera, to the case of an omnidirectional camera, using an existing unifying projection model. The proposed method remains linear, once adequate information is extracted from the omnidirectional images, and does not require any calibration pattern. It works both with perspective and omnidirectional cameras, which is underlined by the reported experiments

This paper presents a new hybrid decoupled vision-based control scheme valid for the entire class of central catadioptric sensors (including conventional perspective cameras). First, we consider the structure from motion problem using imaged 3D lines (conics). Polar lines of the principal point with respect to the conic curves are exploited to estimate a generic homography matrix from which a partial Euclidean reconstruction is obtained. The polar lines and the information extracted from the homography are then used to design a control law which allow us to fully decouple rotational motions from translational motions

To close the loop between motion and vision, tracked visual features must remain in the camera field of view (visibility constraint). To overcome the visibility constraint, visual servoing methods can benefit from panoramic sensors such as catadioptric cameras (combining both mirrors and lenses). In this paper, we present a vision-based framework to control a nonholonomic mobile robot using a catadioptric imaging system. We particularly focus on a suitable catadioptric image-based control strategy of a nonholonomic robot in order to follow a 3D straight line. Such strategy can be applied to navigate in indoor or urban environment since the extraction and the tracking of straight lines are natural. First the control objectives are formulated in the catadioptric image space. The control law is then designed according to a well suited chained system for a mobile robot state vector directly expressed in the image space using a generic camera model. Simulation results illustrate the control strategy in the case of hypercatadioptric and paracatadioptric cameras.

In this paper, we present how the 2 1/2 D visual servoing scheme can be used with omnidirectional cameras. Motivated by the growing interest for omnidirectional sensors on robotic applications and particularly on vision-based control, we extend this framework to the entire class of central catadioptric systems. Indeed, conventional cameras suffer from restricted field of view. Central catadioptric systems have larger fields of view thus overcoming the visibility problem encountered when using conventional cameras. The 2 1/2 D visual servoing is based on the estimation of the partial camera displacement between two views, given by the current and desired images. Geometrical relationships are exploited to enable a partial Euclidean reconstruction by decoupling the interaction between translation and rotation components of a homography matrix. First we describe how to obtain a generic homography matrix for central catadioptric cameras from the projection model of an entire class of camera. Then the information obtained from the homography is used to develop a 2 1/2 D visual servoing scheme.

In this paper we consider the problem of controlling a robotic system using the projection of 3D lines in the image plane of central catadioptric systems. Most of the efforts in visual servoing are devoted to points, only few works have investigated the use of lines in visual servoing with traditional cameras and none has explored the case of omnidirectional cameras. First a generic central catadioptric interaction matrix for the projection of 3D straight lines is derived from the projection model of an entire class of camera. Then an image-based control law is designed and validated through simulation results.