Brain-Computer Interfaces (BCIs) are one of the few remaining communication options for individuals in a Completely Locked-In State (CLIS), where all voluntary motor functions are lost. However, decoding electroencephalographic (EEG) signals in CLIS is particularly challenging due to low signal-to-noise ratios, high intra- and inter-session variability, and cognitive fluctuations. In this study, we systematically evaluate classical and deep learning-based (DL) classification methods on a longitudinal P300-based BCI dataset acquired from a CLIS patient over ten months, comprising seven different stimulation paradigms.A systematic approach is followed to assess model generalization across BCI sessions and paradigms. Overall, more than 40 approaches are compared, including spatial filters for feature extraction with standard classifiers, as well as DL methods based on CNNs and Attention-based architectures. All methods are evaluated with raw input data and three different normalization strategies. Additionally, SMOTE data augmentation is applied to upsample the minority class. The results show high generalization performance across sessions and paradigms, with some approaches achieving nearly 100% performance. Normalization strategies significantly influence performance, while SMOTE often leads to performance degradation. These findings offer valuable insights for designing more robust BCI systems tailored to CLIS users, showing that collecting data across sessions and multiple BCI paradigms can improve BCI performance, while reducing or eliminating the need for per session calibration. Despite the very promising results, they are based on offline analysis. Thus, the best-performing approaches now require online validation for deployment in real-world CLIS scenarios.

3D LiDAR-based place recognition has been extensively researched in urban environments, yet it remains underexplored in agricultural settings. Unlike urban contexts, horticultural environments, characterized by their permeability to laser beams, result in sparse and overlapping LiDAR scans with suboptimal geometries. This phenomenon leads to intra-and inter-row descriptor ambiguity. In this work, we address this challenge by introducing SPVSoAP3D, a novel modeling approach that combines a voxel-based feature extraction network with an aggregation technique based on a second-order average pooling operator, complemented by a descriptor enhancement stage. Furthermore, we augment the existing HORTO-3DLM dataset by introducing two new sequences derived from horticultural environments. We evaluate the performance of SPVSoAP3D against state-of-the-art (SOTA) models, including OverlapTransformer, PointNetVLAD, and LOGG3D-Net, utilizing a cross-validation protocol on both the newly introduced sequences and the existing HORTO-3DLM dataset. The findings indicate that the average operator is more suitable for horticultural environments compared to the max operator and other first-order pooling techniques. Additionally, the results highlight the improvements brought by the descriptor enhancement stage. The code is publicly available at https://github.com/Cybonic/SPVSoAP3D.git

Error-related potentials (ErrPs) extracted from electroencephalographic signals hold potential for application in Brain-Machine Interfaces, in contexts such as robot teleoperation or shared control in assistive platforms. Due to difficulties in signal classification, in part caused by its non-stationary and noisy nature, their use has not been fully realized yet.This work proposes a new approach to ErrP classification based on a two-stream deep learning architecture with three training stages. Its first stage is a self-supervised autoencoder architecture with a multi-head attention layer providing relevant latent features. The second stage comprises a supervised contrastive learning approach considering two backbone networks, where one inherits weights from the first stage and the other is updated by considering the feature embeddings distribution. The final stage comprises supervised classification, where the two backbones are fused and used to classify the input EEG signal. At the end of the three stages, a data-driven two-stream ErrP model is obtained.Twenty-five variants of the proposed approach using the Deep Convolutional Network, Shallow Convolutional Network and EEGNet backbones were tested in an ablation study and benchmarked against a large number of classical classification methods, using data from the BNCI dataset intended to assess cross subject generalization capabilities. The proposed approach obtained the best results overall, highlighting the approach’s capabilities in capturing relevant representations of the EEG signal.

This work proposes a novel approach that uses a semantic segmentation mask to obtain a 2D spatial layout of the segmentation-categories across the scene, designated by segmentation-based semantic features (SSFs). These features represent, per segmentation-category, the pixel count, as well as the 2D average position and respective standard deviation values. Moreover, a two-branch network, GS2F2App, that exploits CNN-based global features extracted from RGB images and the segmentation-based features extracted from the proposed SSFs, is also proposed. GS2F2App was evaluated in two indoor scene benchmark datasets: the SUN RGB-D and the NYU Depth V2, achieving state-of-the-art results on both datasets.

Local motion planning is an essential component of autonomous robot navigation systems as it involves generating collision-free trajectories for the robot in real-time, given its current position, the map of the environment and a goal. Considering an a priori goal path, computed by a global planner or as the output of a mission planning approach, this paper proposes a Two-Stream Deep Reinforcement Learning strategy for local motion planning that takes as inputs a local costmap representing the robot’s surrounding obstacles and a local costmap representing the nearest goal path. The proposed approach uses a Double Dueling Deep Q-Network and a new reward model to avoid obstacles while trying to maintain the lateral error between the robot and the goal path close to zero. Our approach enables the robot to navigate through complex environments, including cluttered spaces and narrow passages, while avoiding collisions with obstacles. Evaluation of the proposed approach was carried out in an in-house simulation environment, in five scenarios. Double and Double Dueling architectures were evaluated; the presented results show that the proposed strategy can correctly follow the desired goal path and, when needed, avoid obstacles ahead and recover back to following the goal path.

Visual user interface paradigms are one of the key modules for brain-actuated wheelchairs, which are a class of promising assistive devices that can increase the autonomy and mobility of people suffering from severe motor impairments. This work proposes a Dynamic Environment-based Visual Interface System (DEVIS) for intuitive navigation target selection for brain-actuated wheelchairs. It is composed of a novel Dynamic Visual Interface (DVI), an RGB image-based perception module (indoor scene classification, object detection and classification, and object tracking), and a P300-based BCI. The DVI displays potential navigation goals in three forms of visual cues: an RGB camera image streaming with object bounding boxes overlaid on objects detected and tracked, three global points of interest (indoor places), and two static commands. Hence, the DVI allows a user to select, through the P300-based BCI, navigation targets/commands that are flashing independently and randomly to create an oddball paradigm. Experimental evaluations were carried out in a dynamic setting with 5 participants, who were asked to select targets displayed in the DVI. The dynamic setting was obtained using RGB image sequences of the ISR-RGB-D Dataset, which represents a mission performed by a mobile robotic platform. The target selection was performed through a P300-based BCI, using non-self-paced and self-paced modalities. The obtained results show that the proposed DEVIS can run in real-time and that users are able to correctly select targets with high BCI accuracy rates.

The accurate detection of motor imagery (MI) from electroencephalography (EEG) is a fundamental, as well as challenging, task to provide reliable control of robotic devices to support people suffering from neuro-motor impairments, e.g., in brain-computer interface (BCI) applications. Recently, deep learning approaches have been able to extract subject-independent features from EEG, to cope with its poor SNR and high intra-subject and cross-subject variability. In this paper, we first present a review of the most recent studies using deep learning for MI classification, with particular attention to their cross-subject performance. Second, we propose DynamicNet, a Python-based tool for quick and flexible implementations of deep learning models based on convolutional neural networks. We showcase the potentiality of DynamicNet by implementing EEGNet, a well-established architecture for effective EEG classification. Finally, we compare its performance with the filter bank common spatial pattern (FBCSP) in a 4-class MI task (data from a public dataset). To infer cross-subject classification performance, we applied three different cross-validation schemes. From our results, we show that EEGNet implemented with DynamicNet outperforms FBCSP by about 25 %, with a statistically significant difference when cross-subject validation schemes are applied. We conclude that deep learning approaches might be particularly helpful to provide higher cross-subject classification performance in multiclass MI classification scenarios. In the future, it is expected to improve DynamicNet to implement new architectures to further investigate cross-subject classification of MI tasks in real-world scenarios.

Convolutional Neural Networks (CNNs) have been increasingly applied in visual classification tasks by replacing hand-crafted features with deep features. However, problems such as inter-class similarity and intra-class variation led to the need of obtaining more descriptive features. To accomplish this, a new semantic inter-object relationship approach is proposed, which is based on the distance relationships between recognized objects. This new source of information represents how close or apart objects belonging to two object classes are, which, together with the number of object occurrences, allows to develop a more descriptive semantic feature representation of the scene. To exploit such semantic features, a two-branch CNN architecture based on 1D and 2D convolutional layers, is proposed. Also, an enhancement version, GSF2AppV2, of the Global and Semantic Feature Fusion described in [1] is proposed by integrating the new semantic inter-object relationship approach, as well as the aforementioned two-branch CNN architecture. The GSF2AppV2 is also composed of a CNN-based global feature branch, where five different CNN-based feature extraction approaches were assessed as global feature extraction modules. Moreover, to combine global and semantic features, two feature fusion approaches are proposed and evaluated: correlation and triple concatenation. GSF2AppV2 was evaluated in two benchmark datasets: the SUN RGB-D and NYU Depth V2. State-of-the-art results were achieved on both datasets, showing the effectiveness of the proposed semantic feature approach on the pipeline.

This paper presents a dynamic gesture recognition approach using a novel spatiotemporal 2D skeleton image representation that can be fed to computationally efficient deep convolutional neural networks, for applications on human-robot interaction. Gestures are a seamless modality of human interaction and represent a potentially natural way to interact with the smart devices around us, like robots. The contribution of this paper is the proposal of a visually interpretable representation of dynamic gestures, which has a two-fold advantage: (i) conveys both spatial and temporal characteristics relying on a technique inspired in computer graphics, (ii) and can be used with simple and efficient architectures of convolutional neural networks. In our representation, a 3D skeleton model is projected to a 2D camera’s point-of-view, preserving spatial relations, and through a sliding window the temporal domain is encoded in a fused image of consecutive frames, through a shading motion effect achieved by manipulating a transparency coefficient. The result is a 2D image that when fed to simple custom-designed convolutional neural networks, it is achieved accurate classification of dynamic gestures. Experimmental reuslts obtained with a purposely captured 6 gesture dataset of 11 subjects, and also 2 public datasets, give evidence of a strong performance of our approach, when compared to other methods.

Brain-controlled wheelchairs (BCWs) are a promising solution for people with severe motor disabilities, who cannot use conventional interfaces. However, the low reliability of electroencephalographic signal decoding and the high user's workload imposed by continuous control of a wheelchair requires effective approaches. In this article, we propose a self-paced P300-based brain-computer interface (BCI) combined with dynamic time-window commands and a collaborative controller. The self-paced approach allows users to switch between control and noncontrol states without requiring any additional task or mental strategy, while the dynamic time-window commands allow balancing the reliability and speed of the BCI. The collaborative controller, combining user's intentions and navigation information, offers the possibility to navigate in complex environments and to improve the overall system reliability. The feasibility of the proposed approach and the impact of each system component (self-paced, dynamic time window, and collaborative controller) are systematically validated in a set of experiments conducted with seven able-bodied participants and six physically disabled participants steering a robotic wheelchair in real-office-like environments. These two groups controlled the BCW with a final driving accuracy greater than 99%. Quantitative and subjective results, assessed through questionnaires, attest to the effectiveness of the proposed approach. Altogether, these findings contribute to improving the usability of BCWs and, hence, the potential for their use by target users in home settings.

This paper presents a study in terms of accuracy and inference speed using RGB-D object detection and classification for mobile platform applications. The study is divided in three stages. In the first, eight state-of-the-art CNN-based object classifiers (AlexNet, VGG16-19, ResNet1850-101, DenseNet, and MobileNetV2) are used to compare the attained performances with the corresponding inference speeds in object classification tasks. The second stage consists in exploiting YOLOv3/YOLOv3-tiny networks to be used as Region of Interest generator method. In order to obtain a real-time object recognition pipeline, the final stage unifies the YOLOv3/YOLOv3-tiny with a CNN-based object classifier. The pipeline evaluates each object classifier with each Region of Interest generator method in terms of their accuracy and frame rate. For the evaluation of the proposed study under the conditions in which real robotic platforms navigate, a nonobject centric RGB-D dataset was recorded in Institute of Systems and Robotics facilities using a camera on-board the ISR-InterBot mobile platform. Experimental evaluations were also carried out in Washington and COCO datasets. Promising performances were achieved by the combination of YOLOv3tiny and ResNet18 networks on the embedded hardware Nvidia Jetson TX2.

Robot localization in indoor environments is crucial for achieving flexible automated navigation. In this paper, we propose a novel multi-stage localization approach for mobile robots which combines a commercial beacon-based absolute indoor positioning system with laser scan data. This configuration of sensors can be deployed in a non-intrusive way in most robotic platforms without the use of proprioceptive sensors (e.g. wheel encoders) which often introduce non-negligible maintenance and downtime costs. The data fusion is performed by a particle filter aided by a refinement stage. The proposed approach was evaluated in two different indoor scenarios with a mobile platform equipped with a mobile Marvelmind beacon and a Hokuyo UTM-30LX scanning laser rangefinder. The proposed localization framework, purposely without proprioceptive data, is compared with an AMCL approach having as inputs odometry, calculated from wheel encoders' data, and laser scan data. Preliminary results show that the proposed approach can provide an accurate localization estimate and that using the refinement stage improves localization.

This paper introduces a new mobile robot localization solution consisting of two main modules: a Particle-Filter based Localization (PFL) and a Reinforcement-Learning based map updating, integrating relative measurements and absolute indoor positioning sensor (A-IPS) data. Concerning localization using 2D-LiDARs, featureless areas are known to be problematic. To solve this problem a classic PFL approach was modified to incorporate A-IPS position measurements in the prediction and update stages. The localization approach has the particularity of including the possibility of updating the map whenever major modifications are detected in the environment in relation to the current localization map. Due to the random sampling-based nature of the PFL, an associated map update solution is not trivial since small inconsistencies in the estimated pose can lead to erroneous map associations. The proposed method learns to decide by assigning higher rewards the greater is the overlap between the map and the 2DLIDAR scans, via RL, and then a proper update of the map is achieved. Validation of the proposed pipeline was carried out in a differential drive platform with algorithms developed in ROS. Tests were performed in two scenarios in order to assess the performance of both the localization module and the map update stage. The results show that the proposed localization method offers improvements in relation to known approaches, and consequently suggest promising perspectives for the proposed map update decision framework.

Connected and autonomous/automated vehicle (CAV) technologies are shaping the design and the new developments in the automotive industry and, in a wider perspective, in the mobility sector as well. Despite the recent advances and on-going developments, and the enthusiasm around autonomous mobility systems, real-world testing of CAVs is a crucial element to allow the next generation of intelligent vehicles to come to our daily-life. The importance of realistic testing is recognized by academia, industry, public sector and stakeholders, and is reflected in all projects involving pilots and advanced prototyping. AUTOCITS* is one of the projects where CAVs and interoperability tests have been conducted. This paper concentrates on the assessment and performance evaluation of tests carried out during the AUTOCITS's Lisbon Pilot, in realworld conditions, involving CAVs and C-ITS technologies. New specific quantitative indicators (key performance indicators - KPls) are proposed to back the assessment and evaluation criteria presented in this work. The KPIs' expressions are provided, which demonstrated to be very difficult to find in the literature. Results are reported and discussed according to the scenarios and field-data recorded during the Pilot.

This paper presents a new collaborative approach for robot motion planning of an assistive robotic platform that takes into account the intentions of the user provided through Electrooculographic (EOG) signals, as well as obstacles surrounding the robotic platform. In order to increase human confidence in the operation of robotic platforms with some degree of navigational autonomy, the intent of the user must be included in the decision process. In our system, the humanrobot interface works through ocular movements (saccades and blinks), which are acquired as EOG signals and classified using a Convolutional Neural Network. In our proposed approach, a model-free Reinforcement Learning (RL) layer is used to provide commands to a virtual robotic platform. The RL layer is constantly being updated with the inputs from the user's intent, environment perception and previous machine-based decisions. In order to prevent collisions, machine-based perception using the proposed RL motion planning approach will assist the user by selecting suitable actions while learning from prior driving behaviors. The approach was validated by a set of tests that consisted of driving a robotic platform in an in-house 3D virtual model of our Research Center (ISR-UC). The experimental results show a better performance of the proposed approach with RL when compared to the version without the RL-based motion planning component. Results show that the approach is a promising step in the concept put forward for collaborative Human-Robotic Interaction (HRI), and opens a path for future research.

In this paper, we propose a markerless multi-view-based multi-user head tracking system for VR/AR/MR applications. In the Virtual Reality domain, the user typically interacts with the virtual world through his movements. The point-of-view of the user's head mounted display (HMD) is based on the user's head pose. Most systems track the head of the user based on the tracking of the HMD itself and are typically constrained to a single user. Aiming to improve the user's experience in complex dynamic environments, we propose a head tracking method based on a multi-camera scenario, able to deal with the interaction of multiple users while coping with the existence of dynamic objects. The method relies on the voxelization of dense point cloud data, which is segmented to detect elements in the scene. Each element is holistically tracked along time and each detected user has his head's position tracked using a dedicated Particle Filter. The system was evaluated using an annotated ground-truth, according to several metrics, and demonstrated a mean 3D positioning error below 18 mm. Furthermore, tests with multiple users were performed that successfully validate the proposed head tracking approach. The aforementioned tests also showed that the proposed approach is computationally efficient (running time below 20 ms for 4 users).

This paper proposes a P300-based BCI speller called LSC-4Q that takes advantage of steady state visual evoked potentials (SSVEP) that appear naturally in the brain as a side effect of the inter-stimulus interval of P300 visual paradigms. The LSC-4Q speller has a circular layout divided into quadrants, and symbols flash individually with a given pseudo-random strategy. Controlling the sequence of the events such that consecutive flashes alternate between sides or quadrants of the speller, we research the possibility of detecting the SSVEP phase associated with the side or quadrant for which the user is focusing on the target symbol, without explicitly incorporating a SSVEP flickering stimulator. The SSVEP phase is extracted using a statistical spatio-spectral Fisher criterion beamformer (SSFCB) implemented in the frequency domain. Results show that SSFCB efficiently extracts phase tags from the SSVEPs embedded on the visual evoked potentials of the oddball paradigm. Preliminary results with 4 participants suggest that it is possible to detect with high accuracy the side of the screen to which the user is looking, and with less precision the detection of the quadrant. Main issues are related to phase variability across sessions. Online results show that some participants can benefit from the combined P300-Lateral approach, improving the overall classification when P300 misclassifications occur.

Model-based positioning methods involve nonlinear equations as is the case when using satellite pseudoranges on global navigation satellite systems (GNSSs) and local measurements on road features. As these are nonlinear models, classical estimation methods cannot provide guaranteed position estimation and can converge to local optima, sometimes far away from the global optimum or the true value. Based on interval analysis, set inversion, and constraints propagation on real values provide a framework that guarantees to find the true position with a characterized confidence domain. This paper describes an error bounded set membership algorithm that computes the absolute position of a road vehicle by using raw GNNS pseudoranges, lane boundary measurements, and a 2D road network map as geometric constraints. The algorithm is based on set inversion using interval analysis, and bounds are set on the measurements by taking into account a chosen risk. The GNSS pseudoranges errors are modeled carefully, and road constraints are formalized to provide additional information in the data fusion process. The proposed algorithm, named lane boundary augmented set-membership GNSS positioning (LB-ASGP), provides a novel and inexpensive approach to improve position estimation performance for road vehicles guaranteeing the enclosure of the computed solution with high confidence. Results from simulations and field experiments show that the LB-ASGP significantly reduces GNSS errors in the direction perpendicular to the lane thanks to the lane boundary measurements.

This paper presents a hybrid 3D-like grid-based mapping approach, that we called HMAP, used as a reliable and efficient 3D representation of the environment surrounding a mobile robot. Considering 3D point-clouds as input data, the proposed mapping approach addresses the representation of height-voxel (HVoxel) elements inside the HMAP, where free and occupied space is modeled through HVoxels, resulting in a reliable method for 3D representation. The proposed method corrects some of the problems inherent to the representation of complex environments based on 2D and 2.5D representations, while keeping an updated grid representation. Additionally, we also propose a complete pipeline for SLAM based on HMAPs. Indoor and outdoor experiments were carried out to validate the proposed representation using data from a Microsoft Kinect One (indoor) and a Velodyne VLP-16 LiDAR (outdoor). The obtained results show that HMAPs can provide a more detailed view of complex elements in a scene when compared to a classic 2.5D representation. Moreover, validation of the proposed SLAM approach was carried out in an outdoor dataset with promising results, which lay a foundation for further research in the topic.

This paper presents an approach to robot-assisted navigation on a mobility assistance context, by learning from the user while helping him navigate efficiently and safely in complex environments. Assistive robots such as robotic walkers provide the ability to support a user's body weight on the upper limbs while walking. However, walkers can add an extra layer of distress due to their specific manipulation constraints. For the users of such devices, lack of dexterous upper limb control can be a considerable problem as it means they may be unable to operate these devices efficiently; users may also have visual impairments that reduce their navigational efficiency. The proposed approach uses a Reinforcement Learning (RL) model and a dynamic window-based local motion planning algorithm. It aims to learn needed corrections in the motion command based on the surrounding environment and the user's intent. The proposed solution handles corrections and guides the user through the environment without collisions while learning and aiding the user when he is unable to operate the device efficiently. The RL based approach was tested in indoor scenarios with a robotic walker platform showing preliminary promising results.

Automatic recognition of error-related potentials (ErrPs) requires a long calibration time in order to have enough error-samples to train the classifier. In this paper we analyze whether it is possible to reduce the ErrP-calibration time in a P300-based brain-computer interface (BCI), by calibrating the BCI with a high rate of errors (wrong detections of user intent). We analyze if a high error-rate condition still produces a discriminable ErrP and if its classification model generalizes well in sessions of different error-rates. Results show that the classification model built from a high error-rate calibration can be used successfully on sessions with lower error-rates.

In real-world applications, the assumption of independent and identical distribution is no longer consistent. To alleviate the significant mismatch between source and target domains, importance weighting import vector machine, which is an adaptive classifier, is proposed. This adaptive probabilistic classification method, which is sparse and computationally efficient, can be used for unsupervised domain adaptation (DA). The effectiveness of the proposed approach is demonstrated via a toy problem, and a real-world cross-domain object recognition task. Even though the sparseness, the proposed method outperforms the state-of-the-art in both unsupervised and semisupervised DA scenarios. We also introduce a reliable importance weighted cross validation (RIWCV), which is an improvement of importance weighted cross validation, for parameter and model selection. The RIWCV avoid falling down in local minimum, by selecting a more reliable combination of the parameters instead of the best parameters.

Robotic walkers are assistive robotic devices that provide mobility assistance, in a domestic or clinical scenario, to individuals suffering from a gait disorder, being age related or due to injuries, surgery, or diseases. Walkers also provide a significant potential for lower limb rehabilitation. In this paper, we present a novel multimodal robotic walker platform, the ISR-AIWALKER, where innovative contributions were made both in the human- machine interface (HMI) and in a gait analysis system placed on board the platform. Taking into account the application potential of these devices, an effort was made to use low-cost sensors without sacrificing the overall performance of the system. A change was made in the HMI paradigm, moving from a force-sensing to a vision-based approach, while maintaining a natural user interaction and adding complementary safety features like correct gripping enforcement. To cope with the close proximity of the user's body, a multimodal sensor setup was considered. Using both RGB and depth map data, a kinematic model of the user's lower limbs is obtained, allowing the identification of a set of features that are used in a machine learning approach to discriminate gait asymmetries. Experiments made with several subjects revealed that the proposed HMI is able to correctly estimate the user intention in a natural and intuitive way. The gait analysis system was also evaluated and evidenced a good discrimination capability to distinguish between different gait patterns.

Social activity based on body motion is a key feature for non-verbal and physical behavior defined as function for communicative signal and social interaction between individuals. Social activity recognition is important to study human-human communication and also human-robot interaction. Based on that, this research has threefold goals: (1) recognition of social behavior (e.g. human-human interaction) using a probabilistic approach that merges spatio-temporal features from individual bodies and social features from the relationship between two individuals; (2) learn priors based on physical proximity between individuals during an interaction using proxemics theory to feed a probabilistic ensemble of activity classifiers; and (3) provide a public dataset with RGB-D data of social daily activities including risk situations useful to test approaches for assisted living, since this type of dataset is still missing. Results show that using the proposed approach designed to merge features with different semantics and proximity priors improves the classification performance in terms of precision, recall and accuracy when compared with other approaches that employ alternative strategies.

Understanding human behavior through nonverbal-based features, is interesting in several applications such as surveillance, ambient assisted living and human-robot interaction. In this article in order to analyze human behaviors in social context, we propose a new approach which explores interrelations between body part motions in scenarios with people doing a conversation. The novelty of this method is that we analyze body motion-based features in frequency domain to estimate different human social patterns: Interpersonal Behaviors (IBs) and a Social Role (SR). To analyze the dynamics and interrelations of people's body motions, a human movement descriptor is used to extract discriminative features, and a multi-layer Dynamic Bayesian Network (DBN) technique is proposed to model the existent dependencies. Laban Movement Analysis (LMA) is a well-known human movement descriptor, which provides efficient mid-level information of human body motions. The mid-level information is useful to extract the complex interdependencies. The DBN technique is tested in different scenarios to model the mentioned complex dependencies. The study is applied for obtaining four IBs (Interest, Indicator, Empathy and Emphasis) to estimate one SR (Leading).The obtained results give a good indication of the capabilities of the proposed approach for people interaction analysis with potential applications in human-robot interaction.

For intelligent vehicles to become a reality, further research and development must be performed, addressing the needs of multidisciplinary approaches like integrated control systems, communication and network, security algorithms, artificial intelligence, verification and validation, neural networks, safety assets and other technological concerns. The goal of this workshop is to explore the challenges and innovative solutions regarding intelligent vehicles, considering the implications of security and real-time issues on safety and certification, which emerge when introducing networked, autonomous and cooperative functionalities. It aims at joining together in an active debate, researchers and practitioners from several communities, namely dependability and security, realtime and embedded systems, intelligent transportation and mobile robot systems. This workshop is aimed at exploring the challenges and innovative solutions related to the security and safety of intelligent vehicles.

Instead of measuring photo-similarity, SymStereo is a stereo vision algorithm that uses new cost functions to measure symmetry differences between pairs of images. In this paper we propose the acceleration of a complete signal processing pipeline for generating 3D volumes based on dense SymStereo. The outputs here generated achieve superior reconstruction quality namely for slant based scenarios, so typical in autonomous systems, that have to capture pairs of images and perform moving decisions in real-time. In particular, we analyse several parallelization strategies for the compute-intensive aggregation procedure using different parameters and evaluate a trade-off between processing time, and higher precision of the calculated depths and quality of the final reconstructed 3D volume. The developed parallel pipeline allows to process more than 4.5 volumes per second for high resolution images using commodity GPUs, which conveniently suits its application in a variety of robotics systems.

In this paper a study is made of the problem of classifying scenarios, in terms of semantic categories, based on data gathered from sensors mounted on-board mobile robots operating indoors. Once the data are transformed to feature space, supervised classification is performed by a probabilistic approach called Dynamic Bayesian Mixture Models (DBMM). This approach combines class-conditional probabilities from supervised learning models and incorporates past inferences. In this work, several experiments on multi-class semantic place classification are reported based on publicly available datasets. Such experiments were conducted in a such way that generalization aspects are emphasized, which is particularly important in real-world applications. Benchmark results show the effectiveness and competitive performance of the DBMM method, in terms of classification rates, using features extracted from 2D range data and from a RGB-D (Kinect) sensor.

In this work, we present a human-centered robot application in the scope of daily activity recognition towards robot-assisted living. Our approach consists of a probabilistic ensemble of classifiers as a dynamic mixture model considering the Bayesian probability, where each base classifier contributes to the inference in proportion to its posterior belief. The classification model relies on the confidence obtained from an uncertainty measure that assigns a weight for each base classifier to counterbalance the joint posterior probability. Spatio-temporal 3D skeleton-based features extracted from RGB-D sensor data are modeled in order to characterize daily activities, including risk situations (e.g.: falling down, running or jumping in a room). To assess our proposed approach, challenging public datasets such as MSR-Action3D and MSR-Activity3D [1] [2] were used to compare the results with other recent methods. Reported results show that our proposed approach outperforms state-of-the-art methods in terms of overall accuracy. Moreover, we implemented our approach using Robot Operating System (ROS) environment to validate the DBMM running on-the-fly in a mobile robot with an RGB-D sensor onboard to identify daily activities for a robot-assisted living application.

This paper presents an innovative Human-Machine Interface (HMI) for a robotic walker. Robotic walkers offer their users an aid for sustaining mobility and the potential to rehabilitate their lower limbs. Mobility is a crucial function for a human being and these aids are paramount for improving the independence and quality of life of those who suffer from some form of mobility impairment. However, important factors like cost and safety make them either frequently inaccessible to the public or discarded due to a lack of confidence in their operation. The approach adopted in this work offers an intuitive human-machine interface combined with innovative safety measures. This result was possible due to the resourceful use of the low-cost Leap Motion sensor. Experimental evaluation was divided into two stages. First the system was tested using a simulated environment which served to validate the principle of operation of the HMI. In the second stage the proposed HMI was tested on board a robotic platform to evaluate its performance in a real-world scenario. Experiments performed with healthy volunteers revealed an intuitive user interaction and accurate user intention determination.

Multiplatooning leaders positioning and cooperative behavior strategies are proposed in this paper, to improve the efficiency of a traffic system of communicant automated vehicles evolving on dedicated lanes. Novel algorithms to ensure high traffic capacity are presented and MATLAB/Simulink-based simulation results are reported. In previous research work, we proposed new algorithms to mitigate communication delays effects on platoon string stability using anticipatory information. In this paper we consider constant spacing between platoons' leaders as a fundamental condition to attain high traffic capacity. New algorithms to maintain interplatoon leaders' constant spacing are proposed, as well as novel algorithms allowing vehicles to enter the main track cooperatively. Furthermore, a new set of algorithms to improve safety is also presented. A novel agent-based architecture was developed, in which each vehicle consists of two distinct modules, a leader and a follower. Based on MATLAB/Simulink simulations of several scenarios, the new algorithms are assessed and the simulation results presented, confirming that the proposed algorithms ensure high traffic capacity and vehicle density and avoid traffic congestion. These features were validated through simulations performed on the Simulation for Urban Mobility simulation platform, using a new car-following model implementation [10]. The results proved that the proposed algorithms enable a clear benefit of a platooning system, when compared with bus- and light-rail-based transit systems.

SymStereo is a new algorithm used for stereo estimation. Instead of measuring photo-similarity, it proposes novel cost functions that measure symmetry for evaluating the likelihood of two pixels being a match. In this work we propose a parallel approach of the LogN matching cost variant of SymStereo capable of processing pairs of images in real-time for depth estimation. The power of the graphics processing units utilized allows exploring more efficiently the bank of log-Gabor wavelets developed to analyze symmetry, in the spectral domain. We analyze tradeoffs and propose different parameter-izations of the signal processing algorithm to accommodate image size, dimension of the filter bank, number of wavelets and also the number of disparities that controls the space density of the estimation, and still process up to 53 frames per second (fps) for images with size 288 × 384 and up to 3 fps for 768 × 1024 images.

Advances in autonomous navigation, safety, and natural-landmark based localization, are among the key objectives in the development of the next generation of autonomous vehicles, to be deployed in manufacturing and semi-structured environments. In this paper, autonomous navigation and collision detection will be focused, where it is proposed a novel navigation approach that incorporates a RRT-based dynamic path planning and a path-following controller. Safety issues are taken into account in the form of a laser-based object detection and tracking. Experimental results obtained in a virtual environment provide evidence that our proposed navigation method is promising for real-world applications.

Why is pedestrian detection still very challenging in realistic scenes? How much would a successful solution to monocular depth inference aid pedestrian detection? In order to answer these questions we trained a state-of-the-art deformable parts detector using different configurations of optical images and their associated 3D point clouds, in conjunction and independently, leveraging upon the recently released KITTI dataset. We propose novel strategies for depth upsampling and contextual fusion that together lead to detection performance which exceeds that of the RGB-only systems. Our results suggest depth cues as a very promising mid-level target for future pedestrian detection approaches.

In this work, we propose an approach that relies on cues from depth perception from RGB-D images, where features related to human body motion (3D skeleton features) are used on multiple learning classifiers in order to recognize human activities on a benchmark dataset. A Dynamic Bayesian Mixture Model (DBMM) is designed to combine multiple classifier likelihoods into a single form, assigning weights (by an uncertainty measure) to counterbalance the likelihoods as a posterior probability. Temporal information is incorporated in the DBMM by means of prior probabilities, taking into consideration previous probabilistic inference to reinforce current-frame classification. The publicly available Cornell Activity Dataset [1] with 12 different human activities was used to evaluate the proposed approach. Reported results on testing dataset show that our approach overcomes state of the art methods in terms of precision, recall and overall accuracy. The developed work allows the use of activities classification for applications where the human behaviour recognition is important, such as human-robot interaction, assisted living for elderly care, among others.

The article describes a new algorithm for calibrating a Kinect sensor that achieves high accuracy using only 6 to 10 image-disparity pairs of a planar checkerboard pattern. The method estimates the projection parameters for both color and depth cameras, the relative pose between them, and the function that converts kinect disparity units (kdu) into metric depth. We build on the recent work of Herrera et. al [8] that uses a large number of input frames and multiple iterative minimization steps for obtaining very accurate calibration results. We propose several modifications to this estimation pipeline that dramatically improve stability, usability, and runtime. The modifications consist in: (i) initializing the relative pose using a new minimal, optimal solution for registering 3D planes across different reference frames, (ii) including a metric constraint during the iterative refinement to avoid a drift in the disparity to depth conversion, and (iii) estimating the parameters of the depth distortion model in an open-loop post-processing step. Comparative experiments show that our pipeline can achieve a calibration accuracy similar to [8] while using less than 1/6 of the input frames and running in 1/30 of the time.

The most standard image object detectors are usually comprised of one or multiple feature extractors or classifiers within a sliding window framework. Nevertheless, this type of approach has demonstrated a very limited performance under datasets of cluttered scenes and real life situations. To tackle these issues, LIDAR space is exploited here in order to detect 2D objects in 3D space, avoiding all the inherent problems of regular sliding window techniques. Additionally, we propose a relational parts-based pedestrian detection in a probabilistic non-iid framework. With the proposed framework, we have achieved state-of-the-art performance in a pedestrian dataset gathered in a challenging urban scenario. The proposed system demonstrated superior performance in comparison with pure sliding-window-based image detectors.

Road segmentation plays an important role in many computer vision applications, either for in-vehicle perception or traffic surveillance. In camera-equipped vehicles, road detection methods are being developed for advanced driver assistance, lane departure, and aerial incident detection, just to cite a few. In traffic surveillance, segmenting road information brings special benefits: to automatically wrap regions of traffic analysis (consequently, speeding up flow analysis in videos), to help with the detection of driving violations (to improve contextual information in videos of traffic), and so forth. Methods and techniques can be used interchangeably for both types of application. Particularly, we are interested in segmenting road regions from the remaining of an image, aiming to support traffic flow analysis tasks. In our proposed method, road segmentation relies on a superpixel detection based on a novel edge density estimation method; in each superpixel, priors are extracted from features of gray-amount, texture homogeneity, traffic motion and horizon line. A feature vector with all those priors feeds a support vector machine classifier, which ultimately takes the superpixel-wise decision of being a road or not. A dataset of challenging scenes was gathered from traffic video surveillance cameras, in our city, to demonstrate the effectiveness of the method.

The cascade classifier is a usual approach in object detection based on vision, since it successively rejects negative occurrences, e.g., background images, in a cascade structure, keeping the processing time suitable for on-the-fly applications. On the other hand, similar to other classifier ensembles, cascade classifiers are likely to have high Vapnik-Chervonenkis (VC) dimension, which may lead to overfitting the training data. Therefore, this work aims at improving the generalization capacity of the cascade classifier by controlling its complexity, which depends on the model of their classifier stages, the number of stages, and the feature space dimension of each stage, which can be controlled by integrating the parameter setting of the feature extractor (in our case an image descriptor) into the maximum-margin framework of support vector machine training, as will be shown in this paper. Moreover, to set the number of cascade stages, bounds on the false positive rate (FP) and on the true positive rate (TP) of cascade classifiers are derived based on a VC-style analysis. These bounds are applied to compose an enveloping receiver operating curve (EROC), i.e., a new curve in the TP–FP space in which each point is an ordered pair of upper bound on the FP and lower bound on the TP. The optimal number of cascade stages is forecasted by comparing EROCs of cascades with different numbers of stages.

The field of robotics has experienced a strong growth worldwide in 2011, with record sales in all areas. Over 166.000 industrial robots has been sold to a variety of industries to improve productivity as well as quality control, being able to operate in dangerous environments for human operators. Over 16.000 professional service robots were sold with defense (unmanned aerial vehicles), agriculture (milking), logistics and medicine (robotic assisted surgery and therapy) being the most relevant application areas. The area of personal domestic service robotics is also booming, with over 2.5 million robots being sold for the applications (e.g. vacuum and floor cleaning, lawn mowing, and entertainment).

Many robotic systems combine cameras with Laser Rangefinders (LRF) for simultaneously achieving multi-purpose visual sensing and accurate depth recovery. Employing a single sensor modality for accomplishing both goals is an appealing proposition because it enables substantial savings in equipment, and tends to decrease the overall complexity of the system. This article explores the possibility of replacing LRF by passive stereo vision for reconstructing the scene along a 2D scan plane. We present a new stereo algorithm that is specifically tailored for the purpose. The algorithm recovers the depth along the scan plane using a symmetry-based matching cost (SymStereo), and refines the raw estimates by applying dynamic programming, followed by a Markov Random Field (MRF) that decides if the reconstructed contour is a line or not. We report for the first time comparative experiments between Stereo Rangefinding (SRF) and LRF. The results are encouraging by showing that SRF can be a plausible alternative to LRF in several application scenarios. Moreover, since SRF also enables independent depth estimates along multiple scan planes with arbitrary orientation, being the only constraint that the scan plane intersects the stereo baseline, it is an important benefit that can be decisive for many robotic applications.

Experiments with a semi-autonomous wheelchair controlled by means of a Brain-Computer Interface (BCI) are presented. The navigation system, having at its core a collaborative controller, performs smooth and safe manoeuvres following sparse steering commands provided by the user. The user intents are decoded from electroencephalographic signals evoked by a visual P300-based paradigm. Experiments have been performed by several able-bodied users and motor disabled participants, showing the effectiveness of the approach.

The video shows an overview of the ISRobotCar, an experimental autonomous electric vehicle that integrates the robot operating system (ROS.org) and several sensors such as an IBEO laser scanner, inertial measurement units (IMU), RTK-GPS, vision cameras, and magnet detectors for magnetic guidance. The main purpose of this vehicle is to serve as a platform for experimental testing of algorithms required for autonomous navigation and cooperative navigation in urban environments. In the video, we present the ISRobotCar moving autonomously in a parking-like area as well as its main hardware and control modules.

Platooning with IVC-enabled autonomous vehicles may enable a significant increase in lane capacity, if performed with constant spacing policies. However, to be effective, such system is very demanding with respect to communication performance and reliability. Dedicated short range communications (DSRC) is the prominent intervehicle communication (IVC) technology. However, its reliability rises concerns when operating under platooning scenarios. In this paper we identify some specific problems that platooning pose to DSRC, through the simulation of several scenarios implemented in the NS-3 network simulator. Moreover, we propose a new concept of IVC when applied to platoons, using simultaneously two different communication technologies: DSRC and infrared (IR). New guidelines toward more efficient use of IVC transmission media are suggested, e.g., by broadcasting the event-driven type of messages through DSRC, whereas periodic vehicle control-based messages use the IR channel, in unicast. Furthermore, the base architecture of the IVC proposed system is presented.

This paper presents a new algorithm for the extrinsic calibration of a perspective camera and an invisible 2D laser-rangefinder (LRF). The calibration is achieved by freely moving a checkerboard pattern in order to obtain plane poses in camera coordinates and depth readings in the LRF reference frame. The problem of estimating the rigid displacement between the two sensors is formulated as one of registering a set of planes and lines in the 3D space. It is proven for the first time that the alignment of three plane-line correspondences has at most eight solutions that can be determined by solving a standard p3p problem and a linear system of equations. This leads to a minimal closed-form solution for the extrinsic calibration that can be used as hypothesis generator in a RANSAC paradigm. Our calibration approach is validated through simulation and real experiments that show the superiority with respect to the current state-of-the-art method requiring a minimum of five input planes.

Intraplatoon information management strategies for dealing with safe and stable operation are proposed in this paper. New algorithms to mitigate communication delays are presented, and Matlab/Simulink-based simulation results are reported. We argue that using anticipatory information from both the platoon's leader and the followers significantly impacts platoon string stability. The obtained simulation results suggest that the effects of communication delays may be almost completely canceled out. The platoon presents a very stable behavior, even when subjected to strong acceleration patterns. When the communication channel is subjected to a strong load, proper algorithms may be selected, lowering network load and maintaining string stability. Upon emergency occurrences, the platoon's timely response may be ensured by dynamically increasing the weight of the platoons' leaders data over the behavior of their followers. The simulation results suggest that the algorithms are robust under several demanding scenarios. To assess if current intervehicle communication technology can cope with the proposed information-updating schemes, research into its operation was conducted through a network simulator.

The ten papers in this special issue cover the full range of cooperative technologies in Intelligent Transportation Systems, from V2V and V2I, including cooperative traffic management to vehicle-to-driver cooperation. These papers are summarized here.

This paper discusses the adaptation of a novel bionic approach of AI to an Assisted Powered Wheelchair (APW) controller, in order to facilitate its driving by people with severe impairments. The control unit offers three modes of operation and this proposal refers to the second control that assists the wheelchair user in driving tasks like passing a door or avoiding obstacles. The user's psychic condition is considered in the APW navigation decision and it is determined by a stress detection system composed by electrocardiogram and skin conductance sensors. The control unit has to deal with environmental and stress data and must trigger actuating commands. The prototype implementation will be done within the platform STAGE and afterwards it will be embedded in a real powered wheelchair.

This paper presents a new shared-control approach for assistive mobile robots, using Brain Computer Interface (BCI) as the Human-Machine Interface (HMI). A P300-based paradigm that allows the selection of brain-actuated commands to steer a Robotic Wheelchair (RW), is proposed. At least one specific motor skill, such as the control of arms, legs, head or voice, is required to operate a conventional HMI. Due to this reason, they are not suited for people suffering from severe motor disorders. BCI may open a new communication channel to these users, since it does not require any muscular activity. The number of decoded symbols per minute (SPM) in a BCI is still very low, which means that users can only provide sparse, and discrete commands. The RW must rely on the navigation system to validate user commands effectively. A two-layer shared-control approach is proposed. The first, a virtual-constraint layer, is responsible for enabling/disabling the user commands, based on certain context restrictions. The second layer is an user-intent matching responsible for determining the suitable steering command, that better fits the user command, taking the user competence on steering the wheelchair into account. Experimental results using Robchair, the RW platform developed at ISR-UC [1], [2] are presented, showing the effectiveness of the proposed methodologies.

LIDAR-based object detection usually relies on geometric feature extraction, followed by a generative or discriminative classification approach. Instead, we propose to change the way of detecting objects using LIDAR by means of not only a featureless approach, but also inferring context-aware relations of object parts. For the first feature, a coarse-to-fine segmentation based on β-skeleton random graph is proposed; after segmentation, each segment is labeled, and scored by a Procrustes analysis. For the second feature, after defining the sub-segments of each object, a contextual analysis is in charge of assessing levels of intra-object or inter-object relationship, ultimately integrated into a Markov logic network. This way, we contribute with a system which deals with partial segmentation, also embodying contextual information. The system proof-of-concept is in pedestrian detection, but the rationale of the approach can be applied to any other object after the definition of its physical structure. The effectiveness of the proposed method was assessed over a data set gathered in challenging scenarios, with a significant gain in accuracy over a full segmentation version of the system.

A single feature extractor-classifier is not usually able to deal with the diversity of multiple image scenarios. Therefore, integration of features and classifiers can bring benefits to cope with this problem, particularly when the parts are carefully chosen and synergistically combined. In this paper, we address the problem of pedestrian detection by a novel ensemble method. Initially, histograms of oriented gradients (HOGs) and local receptive fields (LRFs), which are provided by a convolutional neural network, have been both classified by multilayer perceptrons (MLPs) and support vector machines (SVMs). A diversity measure is used to refine the initial set of feature extractors and classifiers. A final classifier ensemble was then structured by an HOG and an LRF as features, classified by two SVMs and one MLP. We have analyzed the following two classes of fusion methods of combining the outputs of the component classifiers: (1) majority vote and (2) fuzzy integral. The first part of the performance evaluation consisted of running the final proposed ensemble over the DaimlerChrysler cropwise data set, which was also artificially modified to simulate sunny and shadowy illumination conditions, which is typical of outdoor scenarios. Then, a window-wise study has been performed over a collected video sequence. Experiments have highlighted a state-of-the-art classification system, performing consistently better than the component classifiers and other methods.

This paper proposes three novel training methods, two of them based on the backpropagation approach and a third one based on information theory for multilayer perceptron (MLP) binary classifiers. Both backpropagation methods are based on the maximal-margin (MM) principle. The first one, based on the gradient descent with adaptive learning rate algorithm (GDX) and named maximum-margin GDX (MMGDX), directly increases the margin of the MLP output-layer hyperplane. The proposed method jointly optimizes both MLP layers in a single process, backpropagating the gradient of an MM-based objective function, through the output and hidden layers, in order to create a hidden-layer space that enables a higher margin for the output-layer hyperplane, avoiding the testing of many arbitrary kernels, as occurs in case of support vector machine (SVM) training. The proposed MM-based objective function aims to stretch out the margin to its limit. An objective function based on Lp-norm is also proposed in order to take into account the idea of support vectors, however, overcoming the complexity involved in solving a constrained optimization problem, usually in SVM training. In fact, all the training methods proposed in this paper have time and space complexities O(N) while usual SVM training methods have time complexity O(N (3)) and space complexity O(N (2)) , where N is the training-data-set size. The second approach, named minimization of interclass interference (MICI), has an objective function inspired on the Fisher discriminant analysis. Such algorithm aims to create an MLP hidden output where the patterns have a desirable statistical distribution. In both training methods, the maximum area under ROC curve (AUC) is applied as stop criterion. The third approach offers a robust training framework able to take the best of each proposed training method. The main idea is to compose a neural model by using neurons extracted from three other neural networks, each one previously trained by MICI, MMGDX, and Levenberg-Marquard (LM), respectively. The resulting neural network was named assembled neural network (ASNN). Benchmark data sets of real-world problems have been used in experiments that enable a comparison with other state-of-the-art classifiers. The results provide evidence of the effectiveness of our methods regarding accuracy, AUC, and balanced error rate.

Abstract-Brain Computer Interface (BCI) systems based on electroencephalography (EEG) open a new communication channel for people with severe motor disabilities, without recurring to the conventional motor output pathways. The very low signal-to-noise ratio and low spatial resolution still limits severely BCIs communication bandwidth. This paper presents the ongoing work toward the development of a BCI system for wheelchair steering. A full system based on a visual P300 oddball paradigm is proposed. The signal processing algorithms are computationally efficient and require a short phase training. Temporal features and EEG channels are selected through a Fisher criteria. For enhancement of signal-to-noise ratio and data dimensionality reduction, a spatial filter named Common Spatial Patterns is applied. This method is widely used for classification of motor imagery events, however it is not very often used for classification of event related potentials such as P300. In this paper we show that Common Spatial Patterns is an effective approach to improve P300 classification rates. In our approach, the input features for classification are the projections of the filtered data instead of the variance of the projections as typically used in motor imagery. Offline classification results, obtained with a Bayesian classifier, are presented showing the effectiveness of the overall methodology.

The nine papers in this special section examine perception, planning, and navigation for intelligent vehicles.

This paper discusses the introduction of mechanisms to adapt commercial powered wheelchairs in order to facilitate its driving by quadriplegic people with reduced strength. Several models of operation are proposed and the most promising, at the moment, called legacy adapted mode, is detailed. A part of the formal operation model is presented. The model is then used in the STAGE simulator, not only for its evaluation, but also to tune operational parameters that will be specific of each patient. A brief description of the hardware architecture proposed to adapt the commercial wheelchair will also be included.

This special issue contains revised versions of selected papers originally presented at the 9th IEEE International Conference on Intelligent Transportation Systems (ITSC 2006) held in Toronto, Canada, on September 17-20, 2006.

This paper presents an outdoor guidepath navigation system for autonomous mobile robots (AMR) that use permanent magnetic markers embedded in the ground. The odometric data provided by the wheel encoders is fused with the data from magnetic markers. The extended Kalman filter (EKF) was chosen for the fusion process. The AMR is equipped with a magnetic sensing ruler (MSR) developed at ISR-UC that is able to perform a robust detection of magnetic markers. The detection is based on a 3-D algorithm that includes longitudinal-fitting detection (LFD), and cross-fitting detection (CFD). Both, the LFD and the CFD are based on the least squares fitting (LSF) of the measurement data with the 3-D model of the vertical magnetic field. The experimental results with Robchair (intelligent wheelchair being developed at ISR-UC) primarily show that the detection system is robust, since it is able to detect true magnetic markers, and to eliminate noisy magnetic distortions and false markers. The design, and implementation of the navigation algorithm in the Robchair were carried out, and results are presented.

Mobile robotics is a challenging research area, with produced results that were unthinkable several years ago. There exist algorithms and methods capable of performing difficult tasks such as detect/classify objects, skill learning and SLAM. From the initial design steps, the real-time software architecture of a robotic platform requires great attention. The problem is difficult, because various components, such as sensing, perception, localization, and motor control, are required to operate and interact in real-time. This makes the system a very complex one. This paper presents a real-time control architecture designed for mobile robots and intelligent vehicles. Moreover, an example of application of the control structure consisting on a system for learning to classify places, using laser range data, is reported.

A feature detection system has been developed for real-time identification of lines, circles and people legs from laser range data. A new method suitable for arc/circle detection is proposed: the Inscribed Angle Variance (IAV). Lines are detected using a recursive line fitting method. The people leg detection is based on geometrical relations. The system was implemented as a plugin driver in Player, a mobile robot server. Real results are presented to verify the effectiveness of the proposed algorithms in indoor environment with moving objects.

This paper presents, in detail, the implementation of a new control strategy, Kalman-based active observer controller (AOB), for the path following of wheeled mobile robots (WMRs) subject to nonholonomic constraints. This control strategy presents some particularities as being used in discrete mode, and being robust against uncertainties and disturbances such as the ones due to the use of the input-output feedback-linearization method in discrete mode, while it was developed to be used in continuous mode. The performance of the proposed control algorithm is verified via computer simulation, and is compared with other control strategies, such as pole placement controller (PPC) and PPC with a Kalman filter observer (CKF).

This paper addresses the development of an anti-collision system (ACS) based on a laserscanner, for low speed vehicles running in cybercars scenarios. The ACS core is a multi-target detection and tracking system (MTDATS), which is able to classify several kind of objects and can be easily expanded to detect new ones. The MTDATS is composed by five modules: 1) scan segmentation; 2) situation based information integration; 3) object classification using a suitable voting scheme of several object properties; 4) object tracking using a Kalman filter that takes the object type to increase the tracking performance into account; 5) and a database with the objects being tracked at each interval of data processing. For each database object, the time to collision with the vehicle is computed. The worst case time-to-collision and the correspondent predicted impact point on the vehicle are sent to the path-following controller, which using this information provides collision avoidance behaviour.

This work describes a data fusion architecture for robotic assembly tasks based on human sensory-motor skills. These skills are transferred to the robot through geometric and dynamic perception signals. Artificial neural networks are used in the learning process. The data fusion paradigm is addressed. It consists of two independent modules for optimal fusion and filtering. Kalman techniques linked to stochastic signal evolutions are used in the fusion algorithm. Compliant motion signals obtained from vision and pose sense are fused, enhancing the task performance. Simulations and peg-in-hole experiments are reported.

This paper presents a path-tracking controller of a bi-steerable cybernetic car with an anti-collision behaviour. The velocity planner and the anti-collision system are fundamental modules in the architecture. The path tracking implementation uses fuzzy logic. The smoothness of the acceleration profile was one of requirements taken into account in the controller design. The anti-collision system based on Laser Range Data consists of estimating the trajectories and behaviour of surrounding objects. Simulation and experimental results are presented showing the effectiveness of the overall navigation control system.

The paper discusses new developments of the data fusion paradigm due to Cortesao and Koeppe (1999, 2000). A bank of Kalman filters is analyzed in the fusion process. Experiments for a robotic compliant motion task (peg-in-hole) emerged from human skills are reported. Stereo vision and pose sense are fused to execute the task. Feedforward artificial neural networks (ANNs) are trained to transfer human skills to robotic manipulators.

The theory of active observers was initially described by Cortesao et al. (2000). This paper introduces properties of the Kalman gains used in the active observer (AOB) design. An important result of the state-space design is demonstrated, which allows stiffness adaptation without changing the control structure. Experiments with a human-robot skill transfer system to perform the peg-in-hole compliant motion task are described, showing the importance of the AOB.

The article describes a systematic procedure to design a force controller with active observers (AOB). The design is based on pole placement using discrete state space theory. The concept of AOB is introduced as a starting point to perform robust estimates of the system state. The robustness of the system is accomplished through optimal noise processing embedded in the control strategy. The design was tested as a force controller in a Manutec R2 Robot at the DLR.

Many industrial operations can be automated using robot arms but, however, require the use of robots being able to perform surface following in an adaptive way. Contour-following in real-time based on sensor information is a delicate task, yet very useful in operations like painting, welding, glues administration and surface polishing. The contour-following can be accomplished either by using previous knowledge of the shape of the surface, by executing a sequence of movements point-by-point, or in an adaptive way. In the last case a sensor or a set of sensors for collecting data about the environment are used. Furthermore, the contour-following can occur with or without contact. The paper describes a 3-D contour-following behavior controller. The contour-following operation is performed in real-time without contact using proximity information. Results of simulations and real-time implementation are presented and discussed.>

Flexible robotic systems, for example systems to be used in parts assembly automation, require robot manipulators featuring among other capabilities, the capacity of executing reliable fine motions. This requirement leads to the need of using sensors and so to the need of searching for techniques for efficient processing and integration of sensory data. The paper addresses issues regarding the integration of sensors for flexible robotics, namely, force/torque, tactile and proximity range sensors. The authors are concerned with sensory integration for fine motion control, the following subjects being addressed: the architecture of the control system integrating force/torque, tactile and distance sensors and algorithms for position and force control using task-space sensory feedback.>