Recent work has demonstrated that deep learning approaches can successfully be used to recover accurate estimates of the spatially-varying BRDF (SVBRDF) of a surface from as little as a single image. Closer inspection reveals, however, that most approaches in the literature are trained purely on synthetic data, which, while diverse and realistic, is often not representative of the richness of the real world. In this paper, we show that training such networks exclusively on synthetic data is insufficient to achieve adequate results when tested on real data. Our analysis leverages a new dataset of real materials obtained with a novel portable multi-light capture apparatus. Through an extensive series of experiments and with the use of a novel deep learning architecture, we explore two strategies for improving results on real data: finetuning, and a per-material optimization procedure. We show that adapting network weights to real data is of critical importance, resulting in an approach which significantly outperforms previous methods for SVBRDF estimation on real materials. Dataset and code are available at https://lvsn.github.io/ real-svbrdf.

This paper proposes a complete system for robotic sensor placement in initially unknown arbitrary three-dimensional environments. The system uses a novel approach for computing the quality of acquisition of a mobile sensor group in such environments. The quality of acquisition is based on a geometric model of a camera which allows accurate sensor models and simple occlusion computation. The proposed system combines this new metric with a global derivative-free optimization algorithm to find simultaneously the number of sensors and their configuration to sense accordingly the environment. The presented framework compares favourably with current techniques working in two-dimensional environments. Furthermore, simulation and experimental results demonstrate the ability of the system to cope with full three-dimensional environments, a domain still unexplored by previous methods.

We present a robust and accurate technique for the cross-calibration of 3D remote gaze trackers with stereoscopic scene vision systems between which no common imaging area exists. We empirically demonstrate that a multi-depth calibration approach yields remarkably superior results for obtaining 3D Point-of-Gaze (PoG) when compared with traditional methods using monocular scene cameras and co-planar eye gaze calibration points.

This paper proposes a self-adaptation mechanism to manage the resources allocated to the different species comprising a cooperative coevolutionary algorithm. The proposed approach relies on a dynamic extension to the well-known multi-armed bandit framework. At each iteration, the dynamic multi-armed bandit makes a decision on which species to evolve for a generation, using the history of progress made by the different species to guide the decisions. We show experimentally, on a benchmark and a real-world problem, that evolving the different populations at different paces allows not only to identify solutions more rapidly, but also improves the capacity of cooperative coevolution to solve more complex problems.

In this paper, a 3D bimanual gestural interface using data gloves is presented. We build upon past contributions on gestural interfaces and bimanual interactions to create an efficient and intuitive gestural interface that can be used in immersive environments. The proposed interface uses the hands in an asymmetric style, with the left hand providing the mode of interaction and the right hand acting at a finer level of detail.

This paper presents an immersive Human-centric/built virtual work cell for analyzing complex situations dynamically. This environment is supported by a custom open architecture, is composed of objects of complementary nature reflecting the level of Human understanding. Furthermore, it is controlled by an intuitive 3D bimanual gestural interface using data gloves.

Many fields rely on some stochastic sampling of a given complex space. Low-discrepancy sequences are methods aiming at producing samples with better space-filling properties than uniformly distributed random numbers, hence allowing a more efficient sampling of that space. State-of-the-art methods like nearly orthogonal Latin hypercubes and scrambled Halton sequences are configured by permutations of internal parameters, where permutations are commonly done randomly. This paper proposes the use of evolutionary algorithms to evolve these permutations, in order to optimize a discrepancy measure. Results show that an evolutionary method is able to generate low-discrepancy sequences of significantly better space-filling properties compared to sequences configured with purely random permutations.

A Cable-Driven Locomotion Interface employs two independent cable-driven haptic foot platforms constrained in six degrees of freedom (6-DOF). Its control system and its geometry are designed for performing a wide range of trajectories that could generate cable interferences. This paper presents and analyzes computational methods for determining which cable can be released from an active actuation state while allowing control in a minimal tension state, thereby ensuring that both platforms stay in a controllable workspace. One challaging task is to develop light and fast computational algorithms for hard real time processes included in haptic display applications. Seeing that releasing a cable from an active actuation state might generate discontinuities in tension values in the other cables, this paper proposes collision prediction schemes named Interference Estimated Time of Arrival in order to reduce or completely eliminate such discontinuities.

This paper presents a generic software architecture developed to allow users located at different physical locations to share the same virtual environment and to interact with each other and the environment in a coherent and transparent manner.

The intrinsic interaction of a robotic system that includes two 6-degree-of-freedom cable-driven platforms sharing a common workspace might result in cable interferences for random trajectories. This paper presents and analyzes computational methods for geometrically determining and managing these interferences for any trajectory constrained with variable loads. The algorithms considered determine which cable can be released from an active actuation state while allowing control in a minimal tension state, thereby ensuring that both platforms stay in a controllable workspace. The process of managing cable interferences constitutes a challenge as one must take into account the inherent limitations of the workspace, which not only include the possibility of interference itself, but also the geometry of the cable-driven locomotion interface (CDLI), its dynamics, the nonideal behavior of real cables, and the requirement that both platforms must be completely constrained at any time. As releasing a cable from an active actuation state might generate tension discontinuities in the other cables, this paper also proposes collision prediction schemes that are only applied to redundant actuators in order to reduce or completely eliminate such discontinuities. Finally, a simulation of a CDLI embedded as a peripheral in a virtual environment, in which the load applied on each platform comes from the wrench measured under the foot for a natural gait walking, is thoroughly analyzed.

This paper presents a newly developed method to estimate, in magnetic resonance (MR) images, the temperatures reached within the volume of an iceball produced by a cryogenic probe. Building on the direct measurements of the MR signal intensity and its correlation with independent temperature variations at the phase transition from liquid to solid, the thermal information embedded in the images was accessed. The volume and diameter of the growing iceball were estimated from a time series of MR images. Using regressions over the volume in the time and thermal domains, this method predicted the cryogenic temperatures beyond the range of sensitivity of the MR signal itself. We present a validation of this method in samples of gelatin and ex vivo pig liver. Temperature predictions are shown to agree with independent thermosensor readings over a range extending from 20 degrees C down to -65 degrees C, with an average error of less than 6 degrees C.

Traditional approaches for surface reconstruction from range data require that the input data be either range images or unorganized sets of points. Since a large number of range sensors provide data along curvilinear patterns such as profiles, this paper presents an approach for reconstructing a surface from a set of unorganized curves. A strategy for updating the reconstructed surface during data acquisition is described as well. Curves are accumulated in a volumetric structure in which a vector field is built and updated. The information that is needed for efficient curve registration is also directly available in this vector field. This leads to a unified modeling approach combining surface reconstruction and curve registration. The algorithm implementing the approach is of linear complexity with respect to the number of input curves and makes it suitable for interactive modeling. Simulated data based on a set of six curvilinear patterns as well as data acquired with a range sensor are used to illustrate the various steps of the algorithm.

External radiotherapy is an option used for the treatment of the prostate cancer. In order to achieve the best results, radiation must be precisely delivered to the target volume while preserving healthy tissues and protecting organs at risk. Electronic portal imaging devices (EPID) are now available to verify the dose delivery by using the treatment radiation to obtain a portal image. The localization of the prostate, usually not visible on portal images, is performed by inserting up to three radio-opaque markers into the prostate. The identification and position of the markers relative to bone structures and beam field are used to insure the proper delivery of radiation to the prostate. We present a sequence of algorithms optimized for the automatic detection of the three radio-opaque markers on images obtained with EPID during prostate radiotherapy. A neural network is used to validate the detection of true markers in images.

VR systems were traditionally used for tasks relying on high-quality graphics rendering where virtual environments were entirely made of user-defined objects. We are foreseeing that serious breakthroughs will emerge, where 'augmented reality' environments will be created in a more dynamic fashion, using 2D and 3D data from computer vision sensors. Thus, virtual environments will not be made exclusively of user-defined objects, but also from real data. This data, after proper modeling to provide some behaviour and abstraction levels, will then be used to feed high-performance imaging systems. This paper deals with: 1) local surface modeling of the 3D data visible from each 2D viewpoint using a modified marching cubes algorithm; and 2) region matching of the local models (from neighboring views in the sequence) using color region clustering information from the 2D snapshots. Local models and the region matching information are used to build complete global models from a sequence of images.

In a previous paper by Payeur et al. (1997), probabilistic occupancy modeling has been successfully extended to 3D environments by means of a closed-form approximation of the probability distribution. In this paper, the closed-form approximation is revisited in order to provide more reliable and meaningful models. A merging strategy of local probabilistic occupancy grids originating from each sensor viewpoint is introduced. The merging process takes advantage of the multiresolution characteristics of octrees to minimize the computational complexity and enhance performances. An experimental testbed is used to validate the approach and models computed from real range images are presented.

Probabilistic occupancy grids have proved to be very useful for workspace modeling in 2D environments. Due to the expansion of computational load, this approach was not tractable for mapping a 3D environment in real applications. In this paper, the original occupancy grid scheme is revisited and a generic closed-form function is introduced to avoid numerical computation of probabilities for a range sensor with Gaussian error distribution. Occupancy probabilities are computed and stored in a multiresolution octree for improved performance and compactness. Occupancy models are built in local reference frames and linked to a global reference frame through uncertain spatial relationships that can be updated dynamically. This scheme is used for building a 3D map in a telerobotic maintenance application of electric power lines where perturbations may cause motion of object assembly.

Visibility analysis algorithms use digital elevation models (DEMs), which represent terrain topography, to determine visibility at each point on the terrain from a given location in space. This analysis can be computationally very demanding, particularly when manipulating high resolution DEMs accurately at interactive response rates. Massively data-parallel computers offer high computing capabilities and are very well-suited to handling and processing large regular spatial data structures. In the paper, the authors present a new scanline-based data-parallel algorithm for visibility analysis. Results from an implementation onto a MasPar massively data-parallel SIMD computer are also presented.

We present an algorithm that reduces significantly the level of the registration errors between all pairs in a set of range views. This algorithm refines initial estimates of the transformation matrices obtained from either the calibrated acquisition setup or a crude manual alignment. It is an instance of a category of registration algorithms known as iterated closest-point (ICP) algorithms. The algorithm considers the network of views as a whole and minimizes the registration errors of all views simultaneously. This leads to a well-balanced network of views in which the registration errors are equally distributed, an objective not met by previously published ICP algorithms which all process the views sequentially. Experimental results show that this refinement technique improves the calibrated registrations and the quality of the integrated model for complex multi-part objects. In the case of scenes comprising man-made objects of very simple shapes, the basic algorithm faces problems common to all ICP algorithms and so must be extended.

This paper presents a new and general solution to the problem of range view integration. The integration problem consists in computing a connected surface model from a set of registered range images acquired from different viewpoints. The proposed method does not impose constraints on the topology of the observed surfaces, the position of the viewpoints, or the number of views that can be merged. The integrated surface model is piecewise estimated by a set of triangulations modeling each canonical subset of the Venn diagram of the set of range views. The connection of these local models by constrained Delaunay triangulations yields g non-redundant surface triangulation describing all surface elements sampled by the set of range views. Experimental results show that the integration technique can be used to build connected surface models of free-form objects. No integrated models built from objects of such complexity have yet been reported in the literature, It is assumed that accurate range views are available and that frame transformations between all pairs of views can be reliably computed.>

This paper presents a new approach for the computation of generalized positioning inverse kinematic functions for Puma manipulators. This new approach leads to real solutions of the joint variables for any set of Cartesian coordinates located inside or outside the robot's workspace. Therefore, the inverse kinematic functions derived are termed "generalized inverse kinematic functions". These solutions, which produce the minimum distance between the end effector and a Cartesian point to be reached, are particularly useful in tracking operations. Because of the special architecture of the Puma, closed-form solutions are obtained. This leads to simple algorithms and allows for the computation of all solutions.>

Building integrated models of existing 3-D objects is a key requirement for both reverse engineering and object recognition systems. An automatic 3-D model builder goes through three main steps: i) surface sampling from many views, ii) registration of the sampled views, and iii) integration of the registered views. The accuracy obtained depends on the acquisition and registration errors. The latter is critical since a misalignment of the range views causes their noise distributions to be centered around different means, which makes it difficult to reduce the effect of the acquisition error by simple averaging. In this paper, we propose a general algorithm that reduces significantly the level of the registration errors between all pairs in a set of range views. This algorithm refines initial estimates of the transformation matrices obtained from the calibrated acquisition setup. It considers the network of views as a whole and minimizes the registration errors of all views simultaneously. This leads to a well-balanced network of views in which the registration errors are equally distributed. Experimental results show an improvement of both the calibrated registrations and integrated models.>

This paper is concerned with the reliability of a shape description recovered from a set of scattered measurements. The recovering process of the description should not introduce a bias that is caused by specific acquisition conditions such as the presence of spurious measurements and the relative position of the sensor with respect to the object. Moreover, the recovered description should be stable for a sampling variation. While the fitting stage is based on a measurement error model which takes into account the sensor's viewpoint, the stability with sampling is tested by perturbing an hypothesized section. The validity of the approach is demonstrated by extracting reliable estimates of polynomial sections (lines, conics) from surface profile range data obtained from one or several viewpoints.

Many tasks, such as pose determination, object recognition and model building rely on a geometrical description of the visible surfaces derived from scene measurements. Even though much effort has been invested in surface description, robust extraction of surface parameters from scattered and noisy 3-D measurements is still a delicate task. For a given visible surface section, the extracted parameters should not depend on the sensor position in the scene nor on a particular measurement set on this surface. In this work, we show that a viewpoint invariant and stable local surface description can be extracted on sections where measurement constraints are redundant with respect to a polynomial model. A segmentation approach is developed to identify such stable sections. The approach is based on a measurement error model which takes into account the sensor’s viewpoint. An application of the approach to the extraction of straight line sections from single scan 3-D surface profiles is presented. The extracted stable linear sections are stored in a list that includes the estimated descriptive parameters for each section and indices of reliability for each description. The descriptive parameters obtained from images of the objects is compared with pre-measured parameters.

Robust extraction of surface parameters from multiple view scattered and noisy 3-D measurements is a delicate task. It is shown that a stable local surface description can be extracted on sections where measurement constraints are redundant with respect to a polynomial model. A segmentation approach is developed to identify these sections. The approach is based on a measurement error model which takes into account the sensor's viewpoint. An application of the approach to the extraction of straight line sections from single scan 3-D surface profiles is presented.>

The paper proposes an approach to obtain inverse kinematic functions whose domain is not limited to the manipulator's workspace. As a result, these functions can be used to map points of the Cartesian space that do not belong to the manipulator's workspace. The positioning problem is formulated as a minimization of the distance between the prescribed Cartesian point and the end-effector. Hence, points outside of the workspace of the manipulator are mapped by the inverse kinematic function into joint coordinates that bring the manipulator as close as possible to the prescribed Cartesian location. The formulation is derived for both planar and spatial motion and an extension to problems where the orientation has to be considered is also given. Examples pertaining to 2-DOF and 3-DOF manipulators are solved and both analytical and numerical results are given. The proposed inverse kinematic functions are of great interest for tracking approach and catching operations where the object to be reached or tracked by the manipulator can be outside of the workspace,.>

A multiresolution surface modeling technique is presented. Several registered range views obtained from different viewpoints are first integrated into a nonredundant surface triangulation. The integration technique is based on the reparameterization of the canonic subsets of the Venn diagram of the set of views. The resulting triangulation is then input to a sequential optimization process that computes different levels of resolution of the surfaces of interest.>

Presents a method to compute the inter-frame transformation between two range image views of complex multi-part objects. No exact feature matching is attempted and no initial approximate transformation is provided. The method is naturally decomposed into two stages of initial estimation and final refinement of the transformation. A hierarchical triangulation-based surface representation provides an efficient way to select the control points at which the alignment of the two surfaces is to be evaluated. This representation also permits the selection of a manageable number of initial transformations among which at least one is to be in the parametric neighborhood of the actual transformation. Experimental results show that the computed transformation between two views of a complex multi-part object may provide angles of rotation within a fraction of a degree of the actual ones.>

Presents a dynamic approach for the integration of multiple registered range views. A nonredundant surface model for N views is computed from the model built from N-1 views by reprocessing only the surface segments seen in the Nth view. The integration technique is based on the reparametrization of the canonic subsets of the Venn diagram of the range views. The resulting surface model consists in a triangulation defined in an object-centered reference frame.>

The authors present a surface triangulation technique for surface description at different levels of resolution in a range image. A step discontinuity constrained 2-1/2-D triangulation of all points is first computed. A sequential optimization process then removes, at each iteration. the point that minimizes the local retriangulation approximation error caused by its removal. The iterative process can stop at any time to obtain a desired level of approximation error.>

The complexity of information extraction requires the development of efficient and dedicated hardware systems designed to operate in harmony with robot vision and automation. This paper describes a system which combines optical sensing with integrated focal-plane processing capabilities and integrated image post-processing. It is capable of automatic edge tracking and returns lists of connected pixels. I. INTRODUCTION Autonomous robotic applications require the use of reliable sensors since the capability to extract meaningful information from the surrounding environment is paramount to the accomplishment of any task. Among all sensing modalities, computer vision is the one that is most naturally associated with autonomous robotics. This originates from the fact that visual information can provide a very rich semantic description of the world. Vision operates through a process of successive refinements. The illuminance recorded at each component of the retina is processed, and low-level features such as edges and regions are extracted through relatively simple operations. Higher-level percepts such as depth and motion parameters can then be obtained from these low-level features. A very effective approach to the extraction of edges is based on the convolution of circularly symmetric masks with the illuminance image El). Among the available circularly symmetric masks is the well-known Laplacian of Gaussian (LOG) expressed as:

A surface approach for the integration of registered 3-D range images, is presented. The method is a two-step algorithm based on the theory of sets. The proposed algorithm can deal with any number of views and is tailored for parallel implementation. Experimental results obtained by applying this algorithm to complicated objects showed that it is both effective and reliable. An advantage of the method is that it allows view interlacing. A surface segment not viewed because of the presence of a step discontinuity can be viewed by a view taken from another viewpoint.>

This paper presents a computer vision technique designed to estimate the position and orientation of a robot manipulator in its working environment. The manipulator is expected to execute live-line maintenance tasks. The position and orientation are obtained from the segmentation of two grey-level images of the scene taken from a single point of view, and from a priori knowledge of the scene. Experimental results are presented and the validity of the method is discussed on the basis of accuracy and speed.

An algorithm for extracting edges and plane regions of a polyhedral object in a three-dimensional (3D) range image is described. The object may be Convex or nonconvex. A model of the object is built with the regions extracted. Possible extension to cylindrical objects is also considered. The range images are obtained with a novel range-finder camera that can produce 128 × 256 or 256 × 256 surface element (surfcels) images. The edge detection is accomplished in five steps and yields edges one surfcel wide. The region-finding algorithm relies on the concept of the "hemispheric histogram." The histogram is built with the normals of groups of surfcels (patches) forming the image. Analysis of the hemispheric histogram gives global information on the surface orientation of the visible regions of an object. Once these regions are extracted, they are expanded with a region growing process. Geometric properties of the regions are computed by a simple contour following algorithm. Then, a relational model of the regions is built. The model gathers information that is independent of the position and orientation of the object ill the reference plane and could be Used for object recognition in an unsupervised 3D vision system.