This paper presents the control and perception framework used in the ROBUST EU Horizon 2020 project, aimed at integrate different technologies for developing an underwater autonomous robotic system for exploring deep-sea mining sites. The vehicle firstly collects data of the initial zone of interest and it selects a sub area which is the most probable to contain a manganese nodule field; then it carries out a low altitude survey. When a possible nodule is detected by the cameras, the vehicle lands on the seafloor, allowing the following fixed based manipulation which is designed to perform the nodule analysis. The work reports the implemented control and perception architecture and the preliminary pool experiments results.

Total body photography is used for early detection of malignant melanoma, primarily as a means of temporal skin surface monitoring. In a prior work, we presented a scanner with a set of algorithms to map and detect changes in pigmented skin lesions, thus demonstrating that it is possible to fully automate the process of total body image acquisition and processing. The key procedure in these algorithms is skin lesion matching that determines whether two images depict the same real lesion. In this paper, we aim to improve it with respect to false positive and negative outcomes. To this end, we developed two novel methods: one based on successive rigid transformations of three-dimensional point clouds and one based on nonrigid coordinate plane deformations in regions of interest around the lesions. In both approaches, we applied a robust outlier rejection procedure based on progressive graph matching. Using the images obtained from the scanner, we created a ground truth dataset tailored to diversify false positive match scenarios. The algorithms were evaluated according to their precision and recall values, and the results demonstrated the superiority of the second approach in all the tests. In the complete interpositional matching experiment, it reached a precision and recall as high as 99.92% and 81.65%, respectively, showing a significant improvement over our original method.

Imaging in the underwater environment suffers from color degradation and poor visibility since the light spectrum is selectively absorbed and scattered by water and floating particles. In this paper we introduce a simple but effective underwater dehazing approach that builds on an original color transfer strategy to align the color statistics of a hazy input to the ones of a reference image, also captured underwater, but with neglectable water attenuation. As an original specificity, our proposed color transfer approach is designed to promote the preservation of salient regions, as well as of the details obtained by subtracting from the input an edge-preserving smoothed version of itself. The color-transferred input is then restored by inverting a simplified version of the McGlamery underwater image formation model, using the conventional Dark Channel Prior to estimate the transmission map and the back-scattered light parameter involved in the model. We demonstrate that our color transfer step is crucial for a good transmission estimation but mostly for underwater dehazing where other specialized techniques fail. Extensive qualitative and quantitative results demonstrate the effectiveness of the proposed approach to estimate the transmission, including for cases where traditional specialized techniques fail, and to improve the image quality.

Underwater images suffer from severe perceptual/visual degradation, due to the dense and non-uniform medium, causing scattering and attenuation of the propagated light that is sensed. Typical restoration methods rely on the popular Dark Channel Prior to estimate the light attenuation factor, and subtract the back-scattered light influence to invert the underwater imaging model. However, as a consequence of using approximate and global estimates of the back-scattered light, most existing single-image underwater descattering techniques perform poorly when restoring non-uniformly illuminated scenes. To mitigate this problem, we introduce a novel approach that estimates the back-scattered light locally, based on the observation of a neighborhood around the pixel of interest. To circumvent issue related to selection of the neighborhood size, we propose to fuse the images obtained over both small and large neighborhoods, each capturing distinct features from the input image. In addition, the Laplacian of the original image is provided as a third input to the fusion process, to enhance texture details in the reconstructed image. These three derived inputs are seamlessly blended via a multi-scale fusion approach, using saliency, contrast, and saturation metrics to weight each input. We perform an extensive qualitative and quantitative evaluation against several specialized techniques. In addition to its simplicity, our method outperforms the previous art on extreme underwater cases of artificial ambient illumination and high water turbidity.

Mobile broadband (MBB) networks underpin numerous vital operations of the society and are arguably becoming the most important piece of the communications infrastructure. In this demo paper, our goal is to showcase the potential of a novel multi-homed MBB platform for measuring, monitoring and assessing the performance of MBB services in an objective manner. Our platform, MONROE, is composed of hundreds of nodes scattered over four European countries and a backend system that collects the measurement results. Through a user-friendly web client, the experimenters can schedule and deploy their experiments. The platform further embeds traffic analysis tools for real-time traffic flow analysis and a powerful visualization tool.

The detection of newly appearing and changing pigmented skin lesions (PSLs) is essential for timely diagnosis of cutaneous melanoma. Total body skin examination (TBSE) procedures, currently practiced for this purpose, can be extremely time-consuming for patients with numerous lesions. In addition, these procedures are prone to subjectivity when selecting PSLs for baseline image comparison, increasing the risk of missing a developing cancer. To address this issue, we propose a new photogrammetry-based total body scanning system allowing for skin surface image acquisition using cross-polarized light. Equipped with 21 high-resolution cameras and a turntable, this scanner automatically acquires a set of overlapping images, covering 85%-90% of the patient's skin surface. These images are used for the automated mapping of PSLs and their change estimation between explorations. The maps produced relate images of individual lesions with their locations on the patient's body, solving the body-to-image and image-to-image correspondence problem in TBSEs. Currently, the scanner is limited to patients with sparse body hair and, for a complete skin examination, the scalp, palms, soles and inner arms should be photographed manually. The initial tests of the scanner showed that it can be successfully applied for automated mapping and temporal monitoring of multiple lesions: PSLs relevant for follow-up were repeatedly mapped in several explorations. Moreover, during the baseline image comparison, all lesions with artificially induced changes were correctly identified as "evolved."

Holon and holarchy are widely spread concepts in many domains and are used, among others, for building holonic virtual enterprises. However, the methods applied to create such holonic virtual enterprises are nowadays scarcely specified or they have limited applicability. In order to find detailed methods, the proposal is to give up the holonic paradigm and study the traditional partner selection proposals in virtual enterprises, trying their extrapolation to the holonic paradigm. These proposals are based on groups of key factors, but generally it is neither agreed on such factors nor on the way of performing the selection of member enterprises depending on these key factors. The suggestion on this matter is a well-defined game theory based method with a clear advantage: it is a procedural method which accepts any set of key factors; that is to say, it has a formal basis and it is not limited to a given set of key factors. Moreover, the proposed method is specifically oriented to the partners' selection in holonic virtual enterprises, providing a ranked list of the best potential holonic virtual enterprises instead of a unique holonic virtual enterprise. Finally, a case study is presented in order to illustrate the applicability of the proposal.

The Soil Moisture Active Passive (SMAP) radiometer operates in the L-band protected spectrum (1400-1427 MHz) that is known to be vulnerable to radio-frequency interference (RFI). Although transmissions are forbidden at these frequencies by international regulations, ground-based, airborne, and spaceborne radiometric observations show substantial evidence of out-of-band emissions from neighboring transmitters and possibly illegally operating emitters. The spectral environment that SMAP faces includes not only occasional large levels of RFI but also significant amounts of low-level RFI equivalent to a brightness temperature of 0.1-10 K at the radiometer output. This low-level interference would be enough to jeopardize the success of a mission without an aggressive mitigation solution, including special flight hardware and ground software with capabilities of RFI detection and removal. SMAP takes a multidomain approach to RFI mitigation by utilizing an innovative onboard digital detector back end with digital signal processing algorithms to characterize the time, frequency, polarization, and statistical properties of the received signals. Almost 1000 times more measurements than what is conventionally necessary are collected to enable the ground processing algorithm to detect and remove harmful interference. Multiple RFI detectors are run on the ground, and their outputs are combined for maximum likelihood of detection to remove the RFI within a footprint. The capabilities of the hardware and software systems are successfully demonstrated using test data collected with a SMAP radiometer engineering test unit.

The concepts of holon and holarchy were first applied in the manufacturing world to develop Holonic Manufacturing Systems. Since then, they have been used in many fields and have proved to be applicable concepts for developing applications in any business area. Resulting applications are based on conceptual holonic constructions. Like any model, a holarchy needs to be validated under real circumstances. Such validation assures the quality of the holarchy before it is implemented. In general, validation research tends to target: 1) the specific types of holons handled in each proposal and/or the selected development paradigms; and 2) algorithm performance rather than architecture quality. This paper proposes and evaluates a methodology that focuses on the quality of the architecture. This methodology is able to validate any holonic architecture built to meet trade requirements. Moreover, this is a general-purpose methodology. Therefore, the methodology would be valid for any domain and would not be invalidated by holon types and/or implementation paradigms emerging, changing or falling into disuse. For this purpose, we consider holonic architectures as conceptual models, using the pure holon and holarchy concepts and passing up not only any specific implementation paradigm but also any set of specific holon types.

This paper addresses the challenge of developing techniques for the effective teaching of computer vision. It reports on the experience of using the technique of project-based learning as a motivating method for teaching computer vision strategies within the context of a Master's program module in Visual Perception at the University of Girona (Spain). Students were asked to generate projects which involved the use of invariant features, i.e. algorithms to detect and describe local features in images. Within this project-based context, students were encouraged to search for information resources that would accompany their project ideas. Every student team was supervised by a tutoring instructor and periodic meetings were scheduled during this six-week project to follow their progress.

SMAP is one of four Tier-1 missions recommended by the National Research Council's Committee on Earth Science and Applications from Space [1]. The mission consists of a spacecraft with two instruments: an active L-band 1.26 GHz synthetic aperture radar and a passive L-band radiometer that operates in the 1.400 to 1.427 GHz microwave band. The goal of the mission is to use data derived from both instruments to construct high-resolution, high-accuracy global maps of soil moisture and freeze/thaw states over a 3 year mission duration. The SMAP radiometer has an entirely digital back-end processor that builds upon the findings by Misra et. al. [2], [3] for its digital signal processing (DSP) and radio frequency interference (RFI) mitigation. The implementation of this radiometer is currently under way and the mission is scheduled to launch in 2015. This paper summarizes the design and performance results of the RDE engineering test unit (ETU).

Project planning and control is currently of crucial importance for software engineering. Failed projects waste billions of dollars and hours so the early identification of a troubled project will avoid substantial economic losses. Thus, the ideal situation would be to have a team of experts to analyse the project indicators, detect troubled projects and develop recovery plans. It’s hard to deal with this task in the real-life working because of the economic and temporary cost to businesses. This paper proposes a knowledge-based system to reduce these costs. The knowledge-based system evaluates the level of problems of the projects every x time, so full-time experts in project control are not needed. Just when a project could be in trouble the stakeholders will decide if a team of experts is required to evaluate the project and to take the necessary actions.

Detecting already-visited regions based on their visual appearance helps reduce drift and position uncertainties in robot navigation and mapping. Inspired from content-based image retrieval, an efficient approach is the use of visual vocabularies to measure similarities between images. This way, images corresponding to the same scene region can be associated. State-of-the-art proposals that address this topic use prebuilt vocabularies that generally require a priori knowledge of the environment. We propose a novel method for appearance-based navigation and mapping where the visual vocabularies are built online, thus eliminating the need for prebuilt data. We also show that the proposed technique allows efficient loop-closure detection, even at small vocabulary sizes, resulting in a higher computational efficiency.

The presence of anthropogenic RFI is expected to adversely impact soil moisture measurement by NASA's Soil Moisture Active Passive mission. The digital signal processing approach and preliminary design for detecting and mitigating this RFI is presented in this paper. This approach is largely based upon the work of Johnson and Ruf.

High-Level Languages (HLLs) for Field-Programmable Gate Arrays (FPGAs) facilitate the use of reconfigurable computing resources for application developers by using familiar, higher-level syntax, semantics, and abstractions, typically enabling faster development times than with traditional Hardware Description Languages (HDLs). However, programming at a higher level of abstraction is typically accompanied by some loss of performance as well as reduced transparency of application behavior, making it difficult to understand and improve application performance. While runtime tools for performance analysis are often featured in development with traditional HLLs for sequential and parallel programming, HLL-based development for FPGAs has an equal or greater need yet lacks these tools. This article presents a novel and portable framework for runtime performance analysis of HLL applications for FPGAs, including an automated tool for performance analysis of designs created with Impulse C, a commercial HLL for FPGAs. As a case study, this tool is used to successfully locate performance bottlenecks in a molecular dynamics kernel in order to gain speedup.

Wireless sensor networks (WSNs) are typically composed of very small, battery-operated devices (sensor nodes) containing simple microprocessors with few computational resources. However, the rapidly increasing popularity of WSNs has placed increased computational demands upon these systems, due to increasingly complex operating environments and enhanced data-sensing technology. Whereas introducing more powerful microprocessors into sensor nodes addresses these demands, sensor nodes do not contain sufficient energy reserves to support these microprocessors. In this paper, we present a partially reconfigurable FPGA-based architecture and methodology to provide increased WSN flexibility and computational resources, resulting in superior power consumption and performance compared to a microprocessor capable of satisfying similar demands.

Detecting already-visited regions in vision-based navigation and mapping helps reduce drift and position uncertainties. Inspired from content-based image retrieval, an efficient approach is the use of visual vocabularies for measuring similarities between images. In this way, images corresponding to the same scene region can be associated. The state of the art proposals that address this topic suffer from two main drawbacks: (i) they require heavy user intervention, generally involving trial and error tasks for training and parameter tuning and (ii) they are suitable for batch processing only, where all the data is readily available before data processing. We propose a novel method for visual vocabulary navigation and mapping that overcomes these shortcomings. First, the vocabularies are built and updated online, during robot navigation, in order to efficiently represent the visual information present in the scene. Also, the vocabulary building process does not require any user intervention.

Many applications in mobile and underwater robotics employ 3D vision techniques for navigation and mapping. These techniques usually involve the extraction and 3D reconstruction of scene interest points. Nevertheless, in large environments the huge volume of acquired information could pose serious problems to real-time data processing. Moreover, In order to minimize the drift, these techniques use data association to close trajectory loops, decreasing the uncertainties in estimating the position of the robot and increasing the precision of the resulting 3D models. When faced to large amounts of features, the efficiency of data association decreases drastically, affecting the global performance. This paper proposes a framework that highly reduces the number of extracted features with minimum impact on the precision of the 3D scene model. This is achieved by minimizing the representation redundancy by analyzing the geometry of the environment and extracting only those features that are both photometrically and geometrically significant

Projective homography sits at the heart of many problems in image registration. In addition to many methods for estimating the homography parameters (R.I. Hartley and A. Zisserman, 2000), analytical expressions to assess the accuracy of the transformation parameters have been proposed (A. Criminisi et al., 1999). We show that these expressions provide less accurate bounds than those based on the earlier results of Weng et al. (1989). The discrepancy becomes more critical in applications involving the integration of frame-to-frame homographies and their uncertainties, as in the reconstruction of terrain mosaics and the camera trajectory from flyover imagery. We demonstrate these issues through selected examples.

This paper presents the development of a speech interface for controlling a high fidelity system from natural language sentences. A Bayesian Belief Network approach is proposed for dialog modeling. This solution is applied to infer the user’s goals corresponding to the processed utterances. Subsequently, from the inferred goals, missing or spurious concepts are automatically detected. This is used to drive the dialog prompting for missing concepts and clarifying for spurious concepts allowing more flexible and natural dialogs. A dialog strategy which makes use of the dialog history and the system’s state is also presented.

The registration of full 3-D models is an important task in computer vision. Range finders only reconstruct a partial view of the object. Many authors have proposed several techniques to register 3D surfaces from multiple views in which there are basically two aspects to consider. First, poor registration in which some sort of correspondences are established. Second, accurate registration in order to obtain a better solution. A survey of the most common techniques is presented and includes experimental results of some of them.

Since Chavez proposed the highpass filtering procedure to fuse multispectral and panchromatic images, several fusion methods have been developed based on the same principle: to extract from the panchromatic image spatial detail information to later inject it into the multispectral one. In this paper, we present new fusion alternatives based on the same concept, using the multiresolution wavelet decomposition to execute the detail extraction phase and the intensity-hue-saturation (IHS) and principal component analysis (PCA) procedures to inject the spatial detail of the panchromatic image into the multispectral one. The multiresolution wavelet decomposition has been performed using both decimated and undecimated algorithms and the resulting merged images compared both spectral and spatially. These fusion methods, as well as standard IHS-, PCA-, and wavelet-based methods have been used to merge Systeme Pour l'Observation de la Terre (SPOT) 4 XI and SPOT 4 M images with a ratio 4:1. We have estimated the validity of each fusion method by analyzing, visually and quantitatively, the quality of the resulting fused images. The methodological approaches proposed in this paper result in merged images with improved quality with respect to those obtained by standard IHS, PCA, and standard wavelet-based fusion methods. For both proposed fusion methods, better results are obtained when an undecimated algorithm is used to perform the multiresolution wavelet decomposition.

This paper presents a vision-based localization approach for an underwater robot in a structured environment. The system is based on a coded pattern placed on the bottom of a water tank and an onboard down looking camera. Main features are, absolute and map-based localization, landmark detection and tracking, and real-time computation (12.5 Hz). The proposed system provides three-dimensional position and orientation of the vehicle along with its velocity. Accuracy of the drift-free estimates is very high, allowing them to be used as feedback measures of a velocity-based low-level controller. The paper details the localization algorithm, by showing some graphical results, and the accuracy of the system.

This paper describes the improvements achieved in our mosaicking system to assist unmanned underwater vehicle navigation. A major advance has been attained in the processing of images of the ocean floor when light absorption effects are evident. Due to the absorption of natural light, underwater vehicles often require artificial light sources attached to them to provide the adequate illumination for processing underwater images. Unfortunately, these flashlights tend to illuminate the scene in a nonuniform fashion. In this paper a technique to correct non-uniform lighting is proposed. The acquired frames are compensated through a point-by-point division of the image by an estimation of the illumination field. Then, the gray-levels of the obtained image remapped to enhance image contrast. Experiments with real images are presented.

Obtaining automatic 3D profile of objects is one of the most important issues in computer vision. With this information, a large number of applications become feasible: from visual inspection of industrial parts to 3D reconstruction of the environment for mobile robots. In order to achieve 3D data, range finders can be used. Coded structured light approach is one of the most widely used techniques to retrieve 3D information of an unknown surface. An overview of the existing techniques as well as a new classification of patterns for structured light sensors is presented. This kind of systems belong to the group of active triangulation method, which are based on projecting a light pattern and imaging the illuminated scene from one or more points of view. Since the patterns are coded, correspondences between points of the image(s) and points of the projected pattern can be easily found. Once correspondences are found, a classical triangulation strategy between camera(s) and projector device leads to the reconstruction of the surface. Advantages and constraints of the different patterns are discussed.

This paper deals with the problem of navigation for an unmanned underwater vehicle (UUV) through image mosaicking. It represents a first step towards a real-time vision-based navigation system for a small-class low-cost UUV. We propose a navigation system composed by: (i) an image mosaicking module which provides velocity estimates; and (ii) an extended Kalman filter based on the hydrodynamic equation of motion, previously identified for this particular UUV. The obtained system is able to estimate the position and velocity of the robot. Moreover, it is able to deal with visual occlusions that usually appear when the sea bottom does not have enough visual features to solve the correspondence problem in a certain area of the trajectory.

Addresses the problem of estimating the motion of an autonomous underwater vehicle (AUV), while it constructs a visual map ("mosaic" image) of the ocean floor. The vehicle is equipped with a down-looking camera which is used to compute its motion with respect to the seafloor. As the mosaic increases in size, a systematic bias is introduced in the alignment of the images which form the mosaic. Therefore, this accumulative error produces a drift in the estimation of the position of the vehicle. When the arbitrary trajectory of the AUV crosses over itself, it is possible to reduce this propagation of image alignment errors within the mosaic. A Kalman filter with augmented state is proposed to optimally estimate both the visual map and the vehicle position.

This paper presents an automatic vision-based system for UUV station keeping. The vehicle is equipped with a down-looking camera, which provides images of the sea-floor. The station keeping system is based on a feature-based motion detection algorithm, which exploits standard correlation and explicit textural analysis to solve the correspondence problem. A visual map of the area surveyed by the vehicle is constructed to increase the flexibility of the system, allowing the vehicle to position itself when it has lost the reference image. The testing platform is the URIS underwater vehicle. Experimental results demonstrating the behavior of the system on a real environment are presented.

This paper presents an approach to ameliorate the reliability of the correspondence points relating two consecutive images of a sequence. The images are especially difficult to handle, since they have been acquired by a camera looking at the sea floor while carried by an underwater robot. Underwater images are usually difficult to process due to light absorption, changing image radiance and lack of well-defined features. A new approach based on gray-level region matching and selective texture analysis significantly improves the matching reliability.

Mosaics have been commonly used as visual maps for undersea exploration and navigation. The position and orientation of an underwater vehicle can be calculated by integrating the apparent motion of the images which form the mosaic. A feature-based mosaicking method is proposed in this paper. The creation of the mosaic is accomplished in four stages: feature selection and matching, detection of points describing the dominant motion, homography computation and mosaic construction. In this work we demonstrate that the use of color and textures as discriminative properties of the image can improve, to a large extent, the accuracy of the constructed mosaic. The system is able to provide 3D metric information concerning the vehicle motion using the knowledge of the intrinsic parameters of the camera while integrating the measurements of an ultrasonic sensor. The experimental results of real images have been tested on the GARBI underwater vehicle.

When underwater vehicles perform navigation close to the ocean floor, computer vision techniques can be applied to obtain quite accurate motion estimates. The most crucial step in the vision-based estimation of the vehicle motion consists on detecting matchings between image pairs. Here we propose the extensive use of texture analysis as a tool to ameliorate the correspondence problem in underwater images. Once a robust set of correspondences has been found, the three-dimensional motion of the vehicle can be computed with respect to the bed of the sea. Finally, motion estimates allow the construction of a map that could aid to the navigation of the robot.