In the field of building energy management, the heterogeneity of data arising from the coexistence of multiple standards and ontologies, such as Building Energy Management (BEM) and Building Management Systems (BMS), presents a critical obstacle to achieving efficient, interoperable, and continuous digital twins (DTs). These systems produce disparate and often semantically incompatible data, impeding seamless integration and hindering the realization of holistic energy management strategies. To overcome these challenges, advanced data structuring and AI-based strategies are imperative to ensure semantic interoperability and uninterrupted data continuity across heterogeneous systems. This paper introduces a novel approach for constructing a unified ontology that harmonizes BEM and BMS standards, enabling the seamless fusion of their datasets within a digital twin architecture. By leveraging the capabilities of Large Language Models (LLMs), we enrich the semantic structuring process, facilitating automated and precise reconciliation of heterogeneous data sources. Experimental results underscore the viability and efficacy of the proposed methodology in maintaining robust interoperability and ensuring digital continuity, thereby enhancing the operational efficiency of digital twins for smart building energy management.

We developed time and frequency domains neural network surrogate models of synthetic seismograms stemming from the resolution of the three-dimensional equation of motion. The surrogates predict the time or frequency-series knowing the input variables of the physical model. The surrogate models are used to quantify epistemic uncertainty in ground motion prediction through global sensitivity analysis. We first generate a dataset of time domain synthetic seismograms in a seven-dimensions uncertain space using the spectral-element method. To validate the surrogate model, we evaluate its ability to reproduce at least 80% of the bootstrap resamples. Additionally, the R 2 regression coefficient between the simulations generated by the spectral-element method code and those predicted by the neural network is 0.94 for the validation set, confirming the accuracy of the surrogate model. These surrogates allow fast predictions of velocity time-series or Fourier amplitude spectra where spectral-element simulations are not done (neural networks compute about 100,000 surrogates per second, while a single spectral-element simulation longs approximately 7 h on 48 cores). To quantify the uncertainty of the physical system under study, a global sensitivity analysis is undertaken to better understand how uncertain parameters affect the predicted state of the system. We present two sampling-based estimation methods, the so-called “pick-freeze” Sobol method and the Li and Mahadevan method, to quantify this uncertainty. The Sobol method requires approximately 500,000 simulations to achieve stability, whereas the Li and Mahadevan method requires only 30,000 simulations. Using the metamodel, both methods require only a few seconds to produce results, although the Li and Mahadevan method analyzes 10,000 simulations per second, compared to 3,000 for the Sobol method. The results indicate that the shear wave velocity in the physical system’s layer is the most influential parameter affecting the ground speed on the physical system. In contrast, at a reference station (without considering the geological properties of the physical system), the results show that the shear wave velocity in the first and third deep layers are the most influential parameters.

This paper addresses the complex problem of modeling sinkholes in railway infrastructure using LiDAR (Light Detection and Ranging) point cloud data. Building upon previous research on sinkhole detection with Digital Elevation Models (DEMs), this study evaluates multiple geometric models—an inverted Gaussian, conical, and cylindrical—to determine their effectiveness in accurately representing sinkhole structures. The inverted Gaussian model emerged as the most accurate. The study focused on optimizing the model by identifying and fine-tuning the parameters that most significantly influence detection performance. This led to substantial improvements in detection accuracy, with precision increasing from 55% to 61%, knowing that the recall reached 90%. The proposed methodology enhances railway safety by providing precise, scalable, and robust sinkhole detection, with future improvements aimed at generating synthetic datasets for deep learning model training, further advancing detection capabilities.

International audience

This paper deals with the problem of 3D human tracking in catadioptric images using particle-filtering framework. While traditional perspective images are well exploited, only a few methods have been developed for catadioptric vision, for the human detection or tracking problems. We propose to extend the 3D pose estimation in the case of perspective cameras to catadioptric sensors. In this paper, we develop an original likelihood functions based, on the one hand, on the geodetic distance in the spherical space SO3 and, on the other hand, on the mapping between the human silhouette in the images and the projected 3D model. These likelihood functions combined with a particle filter, whose propagation model is adapted to the spherical space, allow accurate 3D human tracking in omnidirectional images. Both visual and quantitative analysis of the experimental results demonstrate the effectiveness of our approach.

Melanoma is one the most increasing cancers since past decades. For accurate detection and classification, discriminative features are required to distinguish between benign and malignant cases. In this study, the authors introduce a fusion of structural and textural features from two descriptors. The structural features are extracted from wavelet and curvelet transforms, whereas the textural features are extracted from different variants of local binary pattern operator. The proposed method is implemented on 200 images from dermoscopy database including 160 non‐melanoma and 40 melanoma images, where a rigorous statistical analysis for the database is performed. Using support vector machine (SVM) classifier with random sampling cross‐validation method between the three cases of skin lesions given in the database, the validated results showed a very encouraging performance with a sensitivity of 78.93%, a specificity of 93.25% and an accuracy of 86.07%. The proposed approach outperforms the existing methods on the database.

3D face tracking using one monocular camera is an important topic, since it is useful in many domains such as: video surveillance system, human machine interaction, biometrics, etc. In this paper, we propose a new 3D face tracking which is robust to large head rotations. Underlying cascaded regression approach for 2D landmark detection, we build an extension in context of 3D pose tracking. To better work with out-of-plane issues, we extend the training dataset by including a new set of synthetic images. For evaluation, we propose to use a new recording system to capture automatically face pose ground-truth, and create a new test dataset, named U3PT (Unconstrained 3D Pose Tracking). The performance of our method along with the state-of-the-art methods are carried out to analyze advantage as well as limitations need to be improved in the future.

Omnidirectional cameras are commonly used in computer vision and robotics. Their main advantage is their wide field of view which allows them to acquire a 360 degree view of the scene with only one sensor and a single shot. However, few studies have investigated the human detection problem using this kind of cameras. In this paper, we propose to extend the conventional approach for human detection in perspective images and based on Histogram of Oriented Gradients (HOG) apdapted to spherical images is used for this issue. Our approach uses the Riemannian manifolds in order to adapt the gradient in the omnidirectional images. Several experiments have been done using INRIA image database; the results show that adapting detection and image database to the geometry of omnidirectional camera allows a robust detection, and significantly increases the performances.

This paper describes an extension of the popular simultaneous localisation and mapping system (RGB-D SLAM) introduced by Endres et al. In [2]. RGB-D SLAM uses a moving RGB-D sensor (i.e. A Kinect) to incrementally produce a graph-based camera pose trajectory along with a global 3D map of the environment composed of potentially millions of 3D points. The goal here is to produce a global 3D map which is not only composed of millions of 3D points but also 3D planes as indoor scenes are mostly composed of planar features such as floor, walls, desks and other furnitures. These planes are directly detected within the depth images provided by the sensor and therefore tend to "regularize" the noise and missing data in depth images during the whole RGB-D SLAM process. This process represents a first step towards a more semantic approach of RGB-D SLAM.

In this paper we propose a recognition technique of 3D dynamic gesture for human robot interaction (HRI) based on depth information provided by Kinect sensor. The body is tracked using the skeleton algorithm provided by the Kinect SDK. The main idea of this work is to compute the angles of the upper body joints which are active when executing gesture. The variation of these angles are used as inputs of Hidden Markov Models (HMM) in order to recognize the dynamic gestures. Results demonstrate the robustness of our method against environmental conditions such as illumination changes and scene complexity due to using depth information only.

3D camera tracking is an important issue for many kinds of applications such as augmented reality and robotics navigation. When image sequences are used, tracking needs to be initialized by matching 2D data extracted from images with 3D data representing a priori knowledge of the world. However, this process is difficult and the accuracy of the pose estimation strongly depends on the accuracy of the previous matching step. In this paper, we present two original approaches that achieve a 2D/3D points-based matching in order to initialize the 3D camera tracking. The first one is semi-automatic and requires the intervention of the user. The second one is completely automatic. Both approaches are based on SURF descriptors, which have the advantages of being fast and robust against outliers. A description of these two approaches is given and results obtained from experiments performed on real data are exposed and discussed.

In this paper, we present a new solution for the problem of tracking the 3D pose of a hand held camera using particle filtering framework. The proposed approach utilizes a 3D straight-line based model of a structured environment. The principal novelty is in the use of the 2D edges from the image and the 3D lines from the input model within the particle filter, especially to design the likelihood function of the filter observation model. Results from real data are presented, demonstrating the efficiency and robustness of the proposed method.

In this paper we present a novel approach for 3D camera self localization, that uses template matching in a particle filter framework. We propose a tracking scheme that exploits the matching scores between image patches to design the likelihood function of the filter observation model. Indeed, by representing a template image with an ensemble of patches, the method is robust with respect to variations such as local appearance variation, partial occlusion, and scale changes. Experiment results on tracking a hand-held camera have shown that the proposed approach provides more accurate tracking, especially for fast motion or long-term partial occlusions. Comparisons have been made with existing methods; results have shown that the proposed scheme has provided an improved tracking accuracy at the cost of more computations.

This paper describes a multimodal tracking system to resolve occlusions in augmented reality applications. The first module of the proposed architecture is composed of a vision based system and allows identification and tracking of visible targets. When targets are partially occluded by scene elements, a second module relieves the vision based module and tracks feature points using a robust algorithm. Finally, a multi-sensors tracking approach is implemented to handle total occlusion of targets and maintains registration even if all markers are not visible. Experimental results and many evaluations have been performed to show the efficiency and robustness of the proposed multimodal approach of tracking and occlusion handling in augmented reality.

In this paper we present an efficient algorithm for estimating the 3D localization in an urban environments by fusing measurements from GPS receiver, inertial sensor and vision. Such hybrid sensor is important for numerous applications including outdoor mobile augmented reality and 3D robot localization. Our approach is based on non-linear filtering of these complementary sensors using a multi-rate Extended Kalman Filter. Our main contributions concern the modeling of the sensor fusion and the development of an efficient approach for camera pose tracking using only natural features. This method improves the accuracy of the estimated 3D localization. We evaluated the performances of our approach and demonstrated its effectiveness through experiments on real data.

In this paper we present a new robust camera pose estimation approach based on 3D lines features. The proposed method is well adapted for mobile augmented reality applications. We used an extended Kalman filter (EKF) to incrementally update the camera pose in real-time. The principal contributions of our method include first, the expansion of the RANSAC scheme in order to achieve a robust matching algorithm that associates 2D edges from the image with the 3D line segments from the input model. And second, a new powerful framework for camera pose estimation using only 2D-3D straight-lines within an EKF. Experimental results on real image sequences are presented to evaluate the performances and the feasibility of the proposed approach in indoor and outdoor environments.

In mobile outdoor augmented reality applications, accurate localization is critical to register virtual augmentations over a real scene. Vision-based approaches provide accurate localization estimates but are still too sensitive to outdoor conditions (brightness changes, occlusions, etc.). This drawback can be overcome by adding other types of sensors. In this work, we combine a GPS and an inertial sensor with a camera to provide accurate localization. We will present the calibration process and we will discuss how to quantify the 3D localization accuracy. Experimental results on real data are presented.

This paper presents a real-time circular targets tracking approach for camera pose estimation which is based on particle filtering framework. Particle filters are sequential Monte Carlo methods based on point mass (or "particle") representations of probability densities, which can be applied to any state-space model. Their ability to deal with non-linearities and non-Gaussian statistics allows to improve robustness comparing to existing approaches, such as those based on the Kalman filter. We propose to combine a circular fiducials detection algorithm with a particle filter to compute the camera 3D pose parameters. One of the main advantages of our approach comparing to the related camera pose estimation works is its capacity to naturally discard outliers which occur because of either image noise or occlusion. Results from real data in an augmented reality setup are then presented, demonstrating the efficiency and robustness of the proposed method.

This paper presents a robust line tracking approach for camera pose estimation which is based on particle filtering framework. Particle filters are sequential Monte Carlo methods based on point mass (or "particle") representations of probability densities, which can be applied to any state-space model. Their ability to deal with non-linearities and non-Gaussian statistics allows to improve robustness comparing to existing approaches, such as those based on the Kalman filter. We propose to use the particle filter to compute the posterior density for the camera 3D motion parameters. The experimental results indicate the effectiveness of our approach and demonstrate its robustness even when dealing with severe occlusion.

The major problem with augmented reality (AR) systems using see-through head mounted displays (HMD's) is the end-to-end system delay (or latency). This delay exists because the head tracker, scene generator, and communication links require time to perform their tasks, causing a lag between the measurement of head location and the display of the corresponding virtual objects inside the HMD. One way to eliminate or reduce the latency is to predict future head locations. We propose to use optimal Bayesian algorithms for non-linear/non-Gaussian tracking problems, with a focus on particle filters to predict head motion. Particle filters are sequential Monte Carlo methods based upon point mass (or 'particle') representation of probability densities, which can applied to any state space model, and which generalize the traditional Kalman filtering methods. A SIR particle filter is discussed and compared with the standard EKF through an illustrative example.

3D free form object recognition is one of the most difficult problems in computer vision. We present a new approach which exploits only one luminance image of a complex object to recognize it in the scene by identifying its appearance in the input image. We construct a photometric (non geometric) projective invariant to perform matching between local regions of the object in the image and those of the model. We propose an original method based on what we called "photometric aspects" to construct a discriminative data base of the 3D object model. We demonstrate the effectiveness of our approach while implementing it with complex free form objects and we present some obtained results.