This paper proposes a novel data-driven technique, namely Online Learning for sensor Fault diagnosis Analysis (OLFA), to perform real-time fault analysis for autonomous cars. Considering the non-stationary properties of real-time sensor faults and the mapping relationship between sensors and feature variables, the proposed method decomposes the sensor fault diagnosis analysis problem into an online data stream classification and feature ranking problems. To detect and identify faults, a clustering-based data stream classification approach is developed to continuously capture and classify non-stationary sensor faults for autonomous cars with little intervention from human experts. An effective active learning method is extended and embedded into the proposed framework to minimize the need for prior knowledge about faults and enable the continual learning capability to adapt to and handle the non-stationary properties of sensor faults. Moreover, the proposed framework addresses the parameter optimization issue of existing machine learning based fault analysis techniques and employs feature ranking analysis to systematically analyze the possible source(s) of sensor faults. CAR Learning to Act (CARLA), a well-known realistic autonomous driving simulator, is used as the benchmark to perform the sensor fault injection and online data stream collection to evaluate the efficacy of OLFA. Analysis of the collected faulty datasets and experimental results, and comparison between OLFA and several state-of-the-art clustering-based approaches for fault classification, demonstrated the efficacy of the proposed framework in the domain of autonomous cars.

Utilizing perception for feedback control in combination with dynamic movement primitive (DMP)-based motion generation for a robot’s end-effector control is a useful solution for many robotic manufacturing tasks. For instance, while performing an insertion task when the hole or the recipient part is not visible in the eye-in-hand camera, a learning-based movement primitive method can be used to generate the end-effector path. Once the recipient part is in the field of view (FOV), image-based visual servo (IBVS) can be used to control the motion of the robot. Inspired by such applications, this article presents a generalized control scheme that switches between motion generation using DMP and IBVS control. To facilitate the design, a common state-space representation for the DMP and the IBVS systems is first established. The stability analysis of the switched system using multiple Lyapunov functions shows that the state trajectories converge to a bound asymptotically. The developed method is validated by three real-world experiments using the eye-in-hand configuration of a Baxter research robot.

As complex autonomous systems become increasingly ubiquitous, their deployment and integration into our daily lives will become a significant endeavor. Human–machine trust relationship is now acknowledged as one of the primary aspects that characterize a successful integration. In the context of human–machine interaction (HMI), proper use of machines and autonomous systems depends both on the human and machine counterparts. On one hand, it depends on how well the human relies on the machine regarding the situation or task at hand based on willingness and experience. On the other hand, it depends on how well the machine carries out the task and how well it conveys important information on how the job is done. Furthermore, proper calibration of trust for effective HMI requires the factors affecting trust to be properly accounted for and their relative importance to be rightly quantified. In this article, the functional understanding of human–machine trust is viewed from two perspectives—human-centric and machine- centric. The human aspect of the discussion outlines factors, scales, and approaches, which are available to measure and calibrate human trust. The discussion on the machine aspect spans trustworthy artificial intelligence, built-in machine assurances, and ethical frameworks of trustworthy machines.

The non-stationary nature of data streams strongly challenges traditional machine learning techniques. Although some solutions have been proposed to extend traditional machine learning techniques for handling data streams, these approaches either require an initial label set or rely on specialized design parameters. The overlap among classes and the labeling of data streams constitute other major challenges for classifying data streams. In this paper, we proposed a clustering-based data stream classification framework to handle non-stationary data streams without utilizing an initial label set. A density-based stream clustering procedure is used to capture novel concepts with a dynamic threshold and an effective active label querying strategy is introduced to continuously learn the new concepts from the data streams. The sub-cluster structure of each cluster is explored to handle the overlap among classes. Experimental results and quantitative comparison studies reveal that the proposed method provides statistically better or comparable performance than the existing methods.

To commemorate the 50th anniversary of the IEEE Transactions on Systems, Man, and Cybernetics: Systems, this article examines and reports on its past to current topical coverage of systems science and engineering toward exploring the evolving focus of the research community. Results of a systematic bibliometric analysis are presented with associated conclusions, implications, and summary of topical areas. In addition, respective views regarding the current state of the field and where it is headed are offered by recent leaders of the IEEE Systems, Man, and Cybernetics Society, including its continued relevance and role in the advancement of systems technology.

It is recognized that system trustworthiness is a hyperstructure embodied by the structural, behavioral, and system dimensions with a set of coherent attributes. We explore a theoretical framework of tripartite trustworthiness that can be applied to real-world autonomous systems. We present a formal study of the essences and mathematical models of system trustworthiness and their quantitative measurements in the contexts of autonomous and mission-critical intelligent systems where humans and machines interact in a hybrid environment.

For a variety of use cases, the utility of autonomous mobile robots is strengthened by a capacity to explore spatial environments with particular focus on objects or phenomena of interest. To generally support such a capacity, an agile, application-agnostic interest operator is constructed to visually guide focused autonomous exploration. An interest operator is proposed that serves to identify unique features in a scene in an application-agnostic manner by maintaining a sort of “short-term memory” and identifying regions that are unique within the context of what has already been ascertained about the scene, not necessarily what is unique or of interest for a given application. The proposed interest operator is particularly suited to render-based mapping and perception, taking advantage of the natural characterization of spaces comprising an environment as free, occupied, and unknown, and the rich, highly-detailed texture information supported in graphical renderings of visual scenes. With focus on two scales of image features, one being driven by discontinuities between surfaces and the other based on variations in texture of surfaces in a region, the proposed interest operator can inform robot sensing and navigation decisions and resource allocation during autonomous exploration. Results from numerical experiments demonstrate the efficacy of the approach.

Human-robot collaboration towards achieving a common goal is most effective when the robot has the capability to estimate the intentions and needs of its human partner, and to plan complementary actions accordingly. To this end, synergistic coupling between inference engines and task-planning algorithms is essential: the earlier the robot can anticipate the actions performed by its partner, the safer and more seamless the interaction between the two parties will be.In this work, we propose a perception-based analytics framework that incorporates discriminative and generative models, which together estimate the current and future class of an action being performed by a human. The analytics leverage a sequence of human skeletal joint locations extracted from a depth map video stream of the human partner. The generative model ingests current and previous joint positions and outputs a sequence of predicted future positions. The discriminative model produces a vector of probabilities indicating the likelihood that the future action belongs to each class within a set of action classes being considered. The information on current and future actions is fed to a task planning module which selects the robot collaborative action that better suits the estimated present and future human states.

A prototype conversational interface has been developed to facilitate spoken interaction between humans and robots. The focus is on end-users such as security officers including military personnel who would conduct missions with robot partners in close proximity. The focal group is familiar with Battle Management Language, a variation of natural language that is restricted to avoid ambiguity. This language is chosen, given its familiarity, to inspire natural spoken interaction with robots and to avoid the greater challenge of full natural language processing. To achieve this, an open-source framework for building applications with conversational interfaces, Dorset, was interfaced to a research mobile robot enabling spoken voice commands and responses. The completed prototype successfully demonstrated feasibility and potential for advancing interaction with robots from current head-down, hands-on approaches toward a more natural head-up, hands-off paradigm suitable for humans and robots working in close proximity to one another.

This paper presents a nested marsupial robotic system and its execution of a notional disaster response task. Human supervised autonomy is facilitated by tightly-coupled, high-level user feedback enabling command and control of a bimanual mobile manipulator carrying a quadrotor unmanned aerial vehicle that carries a miniature ground robot. Each robot performs a portion of a mock hazardous chemical spill investigation and sampling task within a shipping container. This work offers an example application for a heterogeneous team of robots that could directly support first responder activities using complementary capabilities of autonomous dexterous manipulation and mobility, autonomous planning and control, and teleoperation. The task was successfully executed during multiple live trials at the DARPA Robotics Challenge Technology Expo in June 2015. A key contribution of the work is the application of a unified algorithmic approach to autonomous planning, control, and estimation supporting vision-based manipulation and non-GPS-based ground and aerial mobility, thus reducing algorithmic complexity across this capability set. The unified algorithmic approach is described along with the robot capabilities, hardware implementations, and human interface, followed by discussion of live demonstration execution and results.

Advanced driver assistance systems are required to detect latent hazards posed by surrounding vehicles and generate an appropriate response to enhance safety. Lane changes constitute potentially risky maneuvers, as drivers involved encounter latent hazards due to surrounding vehicles. A careful study of lane change behavior is therefore essential in identifying potential abnormalities that may lead to various hazards, during the process of a lane change. In this study, an anomaly detection technique is used to compare snapshots of normal and dangerous lane change maneuvers, to identify the abnormal instances. A one-class support vector machine is used and tested for novelty identification of naturalistic driving study data. The results show that the technique is able to detect dangerous lane changes with high accuracy. In addition, results suggest that dangerous behavior could occur before, after or during a lane change maneuver.

This study considers offline identification of switched linear MIMO systems using measurements from their inputs and outputs. This is a class of non-convex optimization and ill-posed problems. To convert this optimization into a binary integer programming problem, the proposed approach assumes that the number of switches among the subsystems is upper-bounded. The state-space realization of each sub-system is found by the subspace identification. The proposed approach does not need the tuning of the moving window size or any penalization factor. The algorithm efficiency is evaluated through numerical simulations. The results indicate that the error of identification is small and the eigenvalues of sub-systems are estimated successfully.

The ability of robotic systems to effectively address disaster scenarios that are potentially dangerous for human operators is continuing to grow as a research and development field. This leverages research from areas such as bimanual manipulation, dexterous grasping, bipedal locomotion, computer vision, sensing, object segmentation, varying degrees of autonomy, and operator control/feedback. This paper describes the development of a semi-autonomous bimanual dexterous robotic system that comes to the aid of a mannequin simulating an injured victim by operating a fire extinguisher, affixing a cervical collar, cooperatively placing the victim on a spineboard with another bimanual robot, and relocating the victim. This system accomplishes these tasks through a series of control modalities that range from supervised autonomy to full teleoperation and allows the control model to be chosen and optimized for a specific subtask. We present a description of the hardware platform, the software control architecture, a human-in-the-loop computer vision algorithm, and an infrastructure to use a variety of user input devices in combination with autonomous control to compete several dexterous tasks. The effectiveness of the system was demonstrated in both laboratory and live outdoor demonstrations.

This paper presents a Radio Frequency based system for handling long-term drift of pose estimates for a robot performing odometer-based navigation in an indoor environment. The indoor environment is augmented with RFID tags and their associated door-markers to form a partially structured environment. To enhance the performance of the odometry, we adopted a Least Squares calibration approach to mitigate the effect of the systematic errors. The residual errors, mainly non-systematic, are handled by intermittent resetting of the robot's pose based on the global positioning references designed with the RFID tags and their associated door-markers. The results reveal that the long-term confidence in the estimated position improves about six times with this approach.

Waqaa! And welcome to Anchorage, Alaska USA. It is our distinct pleasure to greet you on behalf of the IEEE Systems, Man, and Cybernetics Society and its annual flagship conference, the IEEE International Conference on Systems, Man, and Cybernetics. This year the conference brings us all to the City of Lights and Flowers where we invite your enthusiastic participation in our yearly international forum on latest innovations, state-of-the-art, ideas, and advances in all aspects of systems science & engineering, human-machine systems, and cybernetics. With the conference fully contained within the Hilton Anchorage you are in great hands of Alaskan hospitality. While in Anchorage, we encourage you to also partake in the “Big Wild Life,” that it offers, both within the city and in the vast wilderness outside.

Teleoperated robotic systems provide a valuable solution for the exploration of hazardous environments. The ability to explore dangerous environments from the safety of a remote location represents an important progression towards the preservation of human safety in the inevitable response to such a threat. While the benefits of removing physical human presence are clear, challenges associated with remote operation of a robotic system need to be addressed. Removing direct human presence from the robot's operating environment introduces telepresence as an important consideration in achieving the desired objective. The introduction of the haptic modality represents one approach towards improving operator performance subject to reduced telepresence. When operating in an urban environment, teleoperative stair climbing is not an uncommon scenario. This work investigates the operation of an articulated track mobile robot designed for ascending stairs under teleoperative control. In order to assist the teleoperator in improved navigational capabilities, a fuzzy expert system is utilised to provide the teleoperator with intelligent haptic augmentation with the aim of improving task performance.

The search for signs of past or present life on planetary surfaces such as Mars is a high priority objective of international space agencies. At present and in the near future, the most promising means to conduct such searches employ robotic systems that operate from planetary orbits or on planetary surfaces. This paper presents a robotic system concept for mobile search and surveying on planetary surfaces. It employs an open-path absorption spectrometer with components distributed on a rover and terrain to measure biogenic gases in the near-surface atmosphere. Rover motion control algorithms are presented and demonstrated in simulation for surveying wide areas autonomously using the distributed spectrometer for in situ remote sensing.

In recent years there has been growing evidence that Mars once had a warm, wet environment that was capable of supporting life. This leads to questions as to whether life ever arose on the planet. Space agencies are actively supporting projects to answer this question. As part of that effort, this paper proposes a robotic system to rapidly scan large areas of the Martian surface for specific biogenic gases in the atmosphere. The long-term goal of the project is to develop a system that can locate local emissions of water vapor and biogases on Mars. The paper presents preliminary proof of concept experiments for a practical robotic measurement system to search for biogenic gases on Mars. In this work, an open path spectrometer to measure the biogenic gas ammonia was developed and deployed on a rover. Initial tests in an outdoor setting indicate that the system is effective in measuring ammonia. Difficulties in taking measurements outdoors include compensating for wind gusts and changes in ambient lighting. The paper proposes improvements in the current system that will compensate for current limitations and allow measurements to be taken over longer distances.

Current and future NASA robotic missions to planetary surfaces are tending toward longer duration and are becoming more ambitious for rough terrain access. For a higher level of autonomy in such missions, the rovers will require behavior that must also adapt to declining rover health and unknown environmental conditions. The MER (Mars Exploration Rovers) called Spirit and Opportunity have both passed 350 days of life on the Martian surface, with possible extensions to 450 days and beyond depending on rover health. Changes in navigational planning due to degradation of the drive motors as they reach their lifetime are currently done on Earth for the Spirit rover. The upcoming 2009 MSL (Mars Science Laboratory) and 2013 AFL (Astrobiology Field Laboratory) missions are planned to last 300-500 days, and will possibly involve traverses on the order of multiple kilometers over challenging terrain. This paper presents an adaptive control algorithm for onboard learning of weights within a free flow hierarchy (FFH) behavior framework for autonomous control of planetary surface rovers that explicitly addresses the issues of rover health and rough terrain access. We also present the results of some laboratory and field studies.

On January 24, 2004, the Mars Exploration Rover named Opportunity successfully landed in the region of Mars known as Meridiani Planum, a vast plain dotted with craters where orbiting spacecraft had detected the signatures of minerals believed to have formed in liquid water. The first pictures back from Opportunity revealed that the rover had landed in a crater roughly 20 meters in diameter - the only sizeable crater within hundreds of meters - which became known as Eagle Crater. And in the walls of this crater just meters away was the bedrock MER scientists had been hoping to find, which would ultimately prove that this region of Mars did indeed have a watery past. Opportunity explored Eagle Crater for almost two months, then drove more than 700 meters in one month to its next destination, the much larger Endurance Crater. After surveying the outside of Endurance Crater, Opportunity drove into the crater and meticulously studied it for six months. Then it went to examine the heat shield that had protected Opportunity during its descent through the Martian atmosphere. More than a year since landing, Opportunity is still going strong and is currently en route to Victoria Crater - more than six kilometers from Endurance Crater. Opportunity has driven more than four kilometers, examined more than eighty patches of rock and soil with instruments on the robotic arm, excavated four trenches for subsurface sampling, and sent back well over thirty thousand images of Mars - ranging from grand panoramas to up close microscopic views. This paper details the experience of driving Opportunity through this alien landscape from the point of view of the Rover Planners, the people who tell the rover where to drive and how to use its robotic arm.

This paper applies the biofunctional model of human learning to the implementation of a learning machine that is effective in navigating complex environments. The target model is rule-based and is highly flexible in establishing the relation between any state-action pair. The learning machine is designed using X classifier systems and a fuzzy logic controller (FLC). A learning machine is built in simulation that closely approximates the learning characteristics of the human brain as described by the theory of biofunctional cognition. The methodology is tested with experiments using both single and multiple agents. We also investigated the effectiveness of biofunctionality using competitive and cooperative modes. Furthermore, we studied the robustness of our approach. Our results show that the integration of a FLC and an X classifier system, realizing a biofunctional model, provides a methodology for constructing very effective learning machines.

A biologically inspired approach to sensory fusion and decision-making in a network of interacting autonomous agents is outlined. The underlying biological model (KIV) explores the hierarchy of dynamically interacting units, i.e., sensory cortices. Multi-sensory percept formation in vertebrates is used for modeling multi-agent cooperation in robot networks. Each agent autonomously performs its task, e.g., classification and pattern recognition. The autonomous units weakly interact to produce a coherent, goal-oriented behavior at the level of the overall network. High-level decision-making is manifested through the sequence of intermittent phase transitions in the network coordination unit, which is modeled based on the operation of the entorhinal cortex

This paper discusses technical challenges and navigational skill requirements of mobile robots for traversable path planning in natural environments similar to Mars surface terrains. Different methods for detecting salient terrain features based on imaging texture analysis techniques are described. In particular, three competing soft computing techniques are presented for terrain traversability assessment: a rule-based terrain classifier, a neural network-based terrain classifier, and a fuzzy-logic terrain classifier. Each terrain classifier divides a region of natural terrain into finite sub-terrain regions and classifies terrain condition exclusively within each sub-terrain region based on terrain visual clues. Image processing techniques are applied for aggregative fusion of sub-terrain assessment results. Results of a comparative performance evaluation of all three terrain classifiers are presented. The last two terrain classifiers are shown to have remarkable capability for traversability assessment, which facilitates navigation in unstructured natural terrain environments.

Increased attention has been focused in recent years on human-machine systems, how they are architected, and how they should operate. The purpose of this paper is to describe an actual instance of a practical human-robot system used on a NASA Mars rover mission that has been underway since January 2004 involving daily interaction between humans on Earth and mobile robots on Mars. The emphasis is on the human-robot collaborative arrangement and the performance enabled by mobility and robotic arm software functionality during the first 90 days of the mission. Mobile traverse distance, accuracy, and rate as well as robotic arm operational accuracy achieved by the system is presented.

The aim of This work is to demonstrate that the dynamic KIV architecture can be used to integrate various sensory signals to achieve an efficient goal oriented navigation, when the robot has no a priori information about the simulated Martian environment. Navigation through space commonly involves goal-seeking and obstacle-avoidance. We show how a robot equipped with landmark detectors and eight infrared sensors can accomplish this task using a biologically inspired artificial brain. KIV demonstrates robust multisensory fusion with fast learning of goal-oriented behavior.

This paper describes recent work undertaken at the Jet Propulsion Laboratory in Pasadena, CA in the area of increased rover autonomy for planetary surface operations. The primary vehicle for this work is the Field Integrated, Design and Operations (FIDO) rover. The FIDO rover is an advanced technology prototype that is a terrestrial analog of the Mars Exploration Rovers (MER) being sent to Mars in 2003. We address the autonomy issue through improved integration of rover based sensing and higher level onboard planning capabilities. The sensors. include an inertial navigation unit (INU) with 3D gyros and accelerometers, a sun sensor, mast and body mounted imagery, and wheel encoders. Multisensor fusion using an Extended Kalman Filter (EKF) approach coupled with pattern recognition and tracking algorithms has enabled the autonomy that is necessary for maximizing science data return while minimizing the number of ground loop interactions. These algorithms are coupled with a long range navigation algorithm called ROAMAN (Road Map Navigation) for an integrated approach to rover autonomy. We also report the results of algorithm validation studies in remote field trials at Black Rock Summit in Central Nevada, California's Mojave Desert, and the Arroyo Seco at JPL.

Operational safety and health monitoring are critical matters for autonomous field mobile robots such as planetary rovers operating on challenging terrain. This paper describes relevant rover safety and health issues and presents an approach to maintaining vehicle safety in a mobility and navigation context. The proposed rover safety module is composed of two distinct components: safe attitude (pitch and roll) management and safe traction management. Fuzzy logic approaches to reasoning about safe attitude and traction management are presented, wherein inertial sensing of safety status and vision–based neural network perception of terrain quality are used to infer safe speeds of traversal. Results of initial field tests and laboratory experiments are also described. The approach provides an intrinsic safety cognizance and a capacity for reactive mitigation of robot mobility and navigation risks.

Hierarchical approaches and methodologies are commonly used for control system design and synthesis. Well-known model-based techniques are often applied to solve problems of complex and large-scale control systems. The general philosophy of decomposing control problems into modular and more manageable subsystem control problems applies equally to the growing domain of intelligent and autonomous systems. However, for this class of systems, new techniques for subsystem coordination and overall system control are often required. This article presents an approach to hierarchical control design and synthesis for the case where the collection of subsystems is comprised of fuzzy logic controllers and fuzzy knowledge-based decision systems. The approach is used to implement hierarchical behavior-based controllers for autonomous navigation of one or more mobile robots. Theoretical details of the approach are presented, followed by discussions of practical design and implementation issues. Example implementations realized on various physical mobile robots are described to demonstrate how the techniques may be applied in practical applications involving homogeneous and heterogeneous robot teams. © 2002 Wiley Periodicals, Inc.

This paper presents a rule-based fuzzy traversability index that quantifies the ease-of-traversal of a terrain by a mobile robot based on real-time measurements of terrain characteristics retrieved from imagery data. These characteristics include, but are not limited to slope, roughness, hardness, and discontinuity. The proposed representation of terrain traversability incorporates an intuitive, linguistic approach for expressing terrain characteristics that is robust with respect to imprecision and uncertainty in the terrain measurements. The terrain assessment method is tested and validated with a set of real-world imagery data. These tests demonstrate the capability of the terrain classification algorithm for perceiving hazards associated with terrain traversal.

Presents a strategy for autonomous navigation of field mobile robots on hazardous natural terrain using a fuzzy logic approach and a measure of terrain traversability. The navigation strategy comprises three simple, independent behaviors: seek-goal, traverse-terrain, and avoid-obstacle. The recommendations from these three behaviors are combined through appropriate weighting factors to generate the final steering and speed commands that are executed by the robot. The weighting factors are produced by fuzzy logic rules that take into account the current status of the robot. This navigation strategy requires no a priori information about the environment, and uses the on-board traversability analysis to enable the robot to select relatively easy-to-traverse paths autonomously. Field test results obtained from implementation of the proposed algorithms on the commercial Pioneer All Terrain rover are presented. These results demonstrate the real-time capabilities of the terrain assessment and fuzzy logic navigation algorithms.

Operational safety and health monitoring are critical matters for autonomous field mobile robots such as planetary rovers operating on challenging terrain. The paper describes relevant rover safety and health issues and presents an approach to maintaining vehicle safety in a navigational context. The proposed rover safety module is composed of two distinct components: safe attitude (pitch and roll) management and safe traction management. Fuzzy logic approaches to reasoning about safe attitude and traction management are presented, wherein sensing of safety status and perception of terrain quality are used to infer safe speeds of traversal. Results of field tests and laboratory experiments are also described. The approach provides an intrinsic safety cognizance and a capacity for reactive mitigation of navigation risks.

Adaptive behavioral capabilities are necessary for robust navigation in non-engineered environments. A control approach to adaptive behavior is described which exploits the approximate reasoning facility of fuzzy logic. In particular, a behavior-based architecture for hierarchical fuzzy control of mobile robots is presented. Its structure is described as well as mechanisms of control decision-making which give rise to adaptive behavior. Control decisions result from a consensus of recommendations offered only by behaviors that are applicable to current situations. Indoor navigation examples demonstrate the practicality of the approach and reveals characteristics of multiple behavior interaction.