Generic object tracking remains an important yet challenging task in computer vision due to complex spatio-temporal dynamics, especially in the presence of occlusions, similar distractors, and appearance variations. Over the past two decades, a wide range of tracking paradigms, including Siamese-based trackers, discriminative trackers, and, more recently, prominent transformer-based approaches, have been introduced to address these challenges. While a few existing survey papers in this field have either concentrated on a single category or widely covered multiple ones to capture progress, our paper presents a comprehensive review of all three categories, with particular emphasis on the rapidly evolving transformer-based methods. We analyze the core design principles, innovations, and limitations of each approach through both qualitative and quantitative comparisons. Our study introduces a novel categorization and offers a unified visual and tabular comparison of representative methods. Additionally, we organize existing trackers from multiple perspectives and summarize the major evaluation benchmarks, highlighting the fast-paced advancements in transformer-based tracking driven by their robust spatio-temporal modeling capabilities.

Simplicity bias is a critical challenge in neural net-works since it often leads to favoring simpler solutions and learning unintended decision rules captured by spuri-ous correlations, causing models to be biased and dimin-ishing their generalizability. While existing solutions rely on human supervision, obtaining annotations of the dif-ferent bias attributes is often impractical. To tackle this, we present Debiasify, a novel self-distillation approach that works without any prior information about the nature of biases. Our method leverages a new distillation loss to distill knowledge within a network; from a deep layer where complex, highly-predictive features reside, to a shallow layer where simpler yet attribute-conditioned features are found in an unsupervised manner. In this way, Debiasify learns robust, debiased representations that generalize well across various biases and datasets, enhancing worst-group performance and overall accuracy. Extensive experiments on computer vision and medical imaging benchmarks show the efficacy of our method, significantly outperforming the previous unsupervised debiasing methods (e.g., a 10.13% improvement in worst-group accuracy on Wavy Hair classi-fication in CelebA) while achieving comparable or superior performance to supervised methods. Our code is publicly available at the following link:Debiasify.

Safe reinforcement learning (Safe RL) refers to a class of techniques that aim to prevent RL algorithms from violating constraints in the process of decision-making and exploration during trial and error. In this paper, a novel model-free Safe RL algorithm, formulated based on the multi-objective policy optimization framework is introduced where the policy is optimized towards optimality and safety, simultaneously. The optimality is achieved by the environment reward function that is subsequently shaped using a safety critic. The advantage of the Safety Optimized RL (SORL) algorithm compared to the traditional Safe RL algorithms is that it omits the need to constrain the policy search space. This allows SORL to find a natural tradeoff between safety and optimality without compromising the performance in terms of either safety or optimality due to strict search space constraints. Through our theoretical analysis of SORL, we propose a condition for SORL’s converged policy to guarantee safety and then use it to introduce an aggressiveness parameter that allows for fine-tuning the mentioned tradeoff. The experimental results obtained in seven different robotic environments indicate a considerable reduction in the number of safety violations along with higher, or competitive, policy returns, in comparison to six different state-of-the-art Safe RL methods. The results demonstrate the significant superiority of the proposed SORL algorithm in safety-critical applications.

Multi-agent pathfinding plays a crucial role in various robot applications. Recently, deep reinforcement learning methods have been adopted to solve large-scale planning problems in a decentralized manner. Nonetheless, such approaches pose challenges such as non-stationarity and partial observability. In this paper, we address these challenges by integrating a state representation model into a multi-agent proximal policy optimization framework. To do so, we propose to utilize a state representation model which extracts representation features from the global map and leverages this information to enhance the training process. Our approach involves decoupling the feature extractor from the agent training process, enabling a more accurate representation of the global state that remains unbiased by the actions of the agents. Furthermore, our modularized approach offers the flexibility to replace the representation model with another model or modify tasks within the global map, without the retraining of the agents. We demonstrated the effectiveness of our approach by comparing three multi-agent proximal policy optimization frameworks. Our experimental results demonstrate that our approach improves the average episode reward compared to the other approaches.

Safe Reinforcement Learning (Safe RL) is one of the prevalently studied subcategories of trial-and-error-based methods with the intention to be deployed on real-world systems. In safe RL, the goal is to maximize reward performance while minimizing constraints, often achieved by setting bounds on constraint functions and utilizing the Lagrangian method. However, deploying Lagrangian-based safe RL in real-world scenarios is challenging due to the necessity of threshold fine-tuning, as imprecise adjustments may lead to suboptimal policy convergence. To mitigate this challenge, we propose a unified Lagrangian-based model-free architecture called Meta Soft Actor-Critic Lagrangian (Meta SAC-Lag). Meta SAC-Lag uses meta-gradient optimization to automatically update the safety-related hyperparameters. The proposed method is designed to address safe exploration and threshold adjustment with minimal hyperparameter tuning requirement. In our pipeline, the inner parameters are updated through the conventional formulation and the hyperparameters are adjusted using the meta-objectives which are defined based on the updated parameters. Our results show that the agent can reliably adjust the safety performance due to the relatively fast convergence rate of the safety threshold. We evaluate the performance of Meta SAC-Lag in five simulated environments against Lagrangian baselines, and the results demonstrate its capability to create synergy between parameters, yielding better or competitive results. Furthermore, we conduct a real-world experiment involving a robotic arm tasked with pouring coffee into a cup without spillage. Meta SAC-Lag is successfully trained to execute the task, while minimizing effort constraints. The success of Meta SAC-Lag in performing the experiment is intended to be a step toward practical deployment of safe RL algorithms to learn the control process of safety-critical real-world systems without explicit engineering.

The rise of automation has provided an opportunity to achieve higher efficiency in manufacturing processes, yet it often compromises the flexibility required to promptly respond to evolving market needs and meet the demand for customization. Human-robot collaboration attempts to tackle these challenges by combining the strength and precision of machines with human ingenuity and perceptual understanding. In this paper, we conceptualize and propose an implementation framework for an autonomous, machine learning-based manipulator that incorporates human-in-the-loop principles and leverages Extended Reality (XR) to facilitate intuitive communication and programming between humans and robots. Furthermore, the conceptual framework foresees human involvement directly in the robot learning process, resulting in higher adaptability and task generalization. The paper highlights key technologies enabling the proposed framework, emphasizing the importance of developing the digital ecosystem as a whole. Additionally, we review the existent implementation approaches of XR in human-robot collaboration, showcasing diverse perspectives and methodologies. The challenges and future outlooks are discussed, delving into the major obstacles and potential research avenues of XR for more natural human-robot interaction and integration in the industrial landscape.

Deformation detection is vital for enabling accurate assessment and prediction of structural changes in materials, ensuring timely and effective interventions to maintain safety and integrity. Automating deformation detection through computer vision is crucial for efficient monitoring, but it faces significant challenges in creating a comprehensive dataset of both deformed and non-deformed objects, which can be difficult to obtain in many scenarios. In this paper, we introduce a novel framework for generating controlled synthetic data that simulates deformed objects. This approach allows for the realistic modeling of object deformations under various conditions. Our framework integrates an intelligent adapter network that facilitates sim-to-real domain adaptation, enhancing classification results requiring limited real data from deformed objects. We conduct experiments on domain adaptation and classification tasks and demonstrate that our framework improves sim-to-real classification results compared to the simulation baseline. Our code is available here.

Reinforcement learning (RL) for motion planning of multi-degree-of-freedom robots still suffers from low efficiency in terms of slow training speed and poor generalizability. In this article, we propose a novel RL-based robot motion planning framework that uses implicit behavior cloning (IBC) and dynamic movement primitive (DMP) to improve the training speed and generalizability of an off-policy RL agent. IBC utilizes human demonstration data to leverage the training speed of RL, and DMP serves as a heuristic model that transfers motion planning into a simpler planning space. To support this, we also create a human demonstration dataset using a pick-and-place experiment that can be used for similar studies. Comparison studies reveal the advantage of the proposed method over the conventional RL agents with faster training speed and higher scores. A real-robot experiment indicates the applicability of the proposed method to a simple assembly task. Our work provides a novel perspective on using motion primitives and human demonstration to leverage the performance of RL for robot applications.

Depth completion and object detection are two crucial tasks often used for aerial 3D mapping, path planning, and collision avoidance of Uncrewed Aerial Vehicles (UAVs). Common solutions include using measurements from a LiDAR sensor; however, the generated point cloud is often sparse and irregular and limits the system's capabilities in 3D rendering and safety-critical decision-making. To mitigate this challenge, information from other sensors on the UAV (viz., a camera used for object detection) is utilized to help the depth completion process generate denser 3D models. Performing both aerial depth completion and object detection tasks while fusing the data from the two sensors poses a challenge to resource efficiency. We address this challenge by proposing a novel approach to jointly execute the two tasks in a single pass. The proposed method is based on an encoder-focused multi-task learning model that exposes the two tasks to jointly learned features. We demonstrate how semantic expectations of the objects in the scene learned by the object detection pathway can boost the performance of the depth completion pathway while regressing the missing depth values. Experimental results show that the proposed multi-task network outperforms its single-task counterpart, particularly when exposed to defective inputs.

Reinforcement Learning is rapidly establishing itself as the foremost choice for optimization of sequential autonomous decision-making problems. Encumbered by its sample inefficiency, the extension of the field to large state space and dynamic environments remains an open problem. We present a novel concept that exploits abstract spatial symmetry in complex environments for extending the skills of naïvely trained agents in local abstractions of the environment. The concept of EASE (Exploitation of Abstract Symmetry of Environments), when incorporated, improves the sample efficiency of traditional reinforcement learning algorithms. The presented work exemplifies the concept of EASE by presenting three distinct settings; EASE with heuristics-based planning, EASE with learning from demonstrations and EASE with state-space abstraction and proposes a novel algorithm for each setting.

Cohesive human-robot collaboration (HRC) for carrying out an industrial task requires an intelligent robot capable of functioning in uncertain and noisy environments. This can be achieved through seamless and natural communication between human and robot partners. Introducing naturalness in communication is highly complex due to both aleatoric variability and epistemic uncertainty originating from the components of the HRC system including the human, sensors, robot(s), and the environment. The presented work proposes the artificial intelligence (AI)-powered multimodal, robust fusion (AI-MRF) architecture that combines communication modalities from the human for a more natural communication. The proposed architecture utilizes fuzzy inferencing and Dempster-Shafer theory for deal with different manifestations of uncertainty. AI-MRF is scalable and modular. The evaluation of AI-MRF for safety and robustness under real-world mimicking case studies is showcased. While the architecture has been evaluated for HRC in industrial settings, it can be readily implemented into any human and machine communication scenarios.

The presented work is guided with the motivation of understanding the deep-learning based 3D reconstruction process for applications in aerial close-range photogrammetry. Given the highly dynamic nature of such a setting, the accuracy and understanding of traditional reconstruction methods as well as the generalization capabilities of deep learning-based methods is required. However, the state-of-the-art methods are typically inadequate. The presented work demonstrates a two-part machine learning-based approach that rely on autoencoder-like models. The first is a Sparse AutoEncoder (SAE) that takes a single image as input and reconstructs a 3D voxel grid. The input images are then sorted based on the reconstruction quality of the SAE output. The second is a Variational AutoEncoder (VAE) that processes multiple images sampled from the ordered set to generate an enhanced 3D voxel grid. The work highlights a novel approach to 3D model reconstruction and presents insights to the process of 3D reconstruction from single image inputs. The autoencoders are trained on a dataset comprised of multiple objects with images captured from different zenith and azimuth angles, simulating an aerial vehicle viewpoint. We show the efficacy of the proposed approach by reconstructing a 3D voxel grid on a ModelNet40 dataset class.

Deployed AI applications are often heavily con-strained in computational resources; to this end, a method of producing miniature, well-performing neural networks for 2D image segmentation is devised and applied to the Severstal Steel Defect Detection and Kidney Tumor Segmentation (KiTS19) Challenges. By limiting the width of U-Net and employing a full-domain activation function, a network of aggregated weak learners is able to achieve a mean F1score within 93% of the EfficientNetB0 baseline using only 0.5% of the trainable parameters. A similar network of aggregated strong learners matches the mean F1score of the baseline on the Severstal dataset using only 10% of the trainable parameters. Gradient boosting is then applied to the weak learners, achieving a mean F1score within 98% of the strong learner network on the Severstal dataset with approximately 20% of the FLOPS; the key insight is in constraining the O(n2) relationship between network width and FLOPS. The same approach is applied to the KiTS19 dataset with good success in kidney detection. Interestingly, the method does not perform as well on the much harder to isolate tumor class, and the authors explore some possible reasons. In analyzing the impact of the full-domain activation function, the authors show that density of information is promoted by significantly reduced peaks in layer outputs and a wider range of output values. The method has significant implications in constrained deployments, as many small devices could be used to compute the overall network.

In this paper, we propose 3DCADFusion, an end-to-end object reconstruction pipeline. Our system is capable of producing dense high-fidelity reconstructions of objects by simultaneously tracking and reconstructing their incrementally improving 3D geometry in a sequence of frames captured by a depth-enabled camera setup. In this process, no external information about the object motion is required as long as the camera is stationary. This is accomplished by removing irrelevant scene elements such as the background or the operator hands holding the object to track and fuse its shape over time only using pixels associated with the object mask. Previous approaches either target static scene reconstruction therefore ignoring dynamic elements, or they make strong assumptions about the scanning platform or required scanning sessions. In contrast, we propose a fast and flexible scanning method similar to how humans visually inspect objects, enabling users to easily acquire a true-to-scale 3D CAD models ready for use in other computer vision tasks such as object tracking or pose estimation. We demonstrate the performance of our approach on a real-world dataset and quantitatively analyze the resulting reconstruction accuracy. Demo is available at github.com/4ri4Slr/3DCADFusion

This Research to Practice Category Full Paper presents a teaching tool for users to design and practice different features and parameters of the longitudinal control of autonomous vehicles. The proposed platform is composed of four modules: vision-based perception, decision-making, speed control and simulation. The teaching tool provides the ability for users to work with individual modules and then integrate these modules to visualize their effect on the vehicle performance in simulation. Each module has its own MATLAB graphical user interface for users to simplify the learning procedure and intuitively update system parameters. The output of the teaching tool, autonomous vehicle longitudinal control, is visualized using the MATLAB 3D World Simulator. The proposed teaching tool was presented in a fourth-year engineering control system class and the students’ feedback was used to improve the teaching tool for a better learning experience.

This article considers the problem of high-level decision process for autonomous vehicles on highways. The goal is to select a predictive reference trajectory among a set of candidate ones, issued from a trajectory generator. This selection aims at optimizing multi-criteria functions, such as safety, legal rules, preferences and comfort of passengers, or energy consumption. This work introduces a new framework for Multi-Criteria Decision Making (MCDM). The proposed approach adopts fuzzy logic theory to deal with heterogeneous criteria and arbitrary functions. Moreover, the consideration of uncertain vehicle's sensors data is done using the Dempster-Shafer Theory with fuzzy sets in order to provide a risk assessment. Simulation results using datasets collected under the NGSIM program are presented on car following cases, and extended to lane changing situations.

There has been an increasing interest in the use of correlation filters for visual object tracking due to their impressive tracking performance. However, existing correlation filter based tracking methods, such as Struck and Kernelized Correlation Filter (KCF), cannot always solve tracking problems in complicated conditions such as heavy occlusion and aggressive motion. In this paper, we proposed a real-time visual tracker via a robust KCF. We start by implementing a search window alignment, based on a motion model with uncertainty, which increases the tracking accuracy for fast moving targets and reduces the padding value to accelerate tracking speed. Next, we establish a combined confidence measurement including occlusion information, which is utilized for robust updating. Then we apply an adaptive Kalman filter to improve the tracking accuracy. Qualitative and quantitative experimental results show that the proposed algorithm outperforms the state-of-the-art methods such as KCF and Struck.

Recognizing objects from the point cloud captured by modern 3D sensors is an important task for robots operating autonomously in real-world environments. However, the existing well-performing approaches typically suffer from a trade-off between resolution of representation and computational efficiency. In this paper, raw point cloud normals are fed into the Point Convolution Network (PCN) without any other representation converts. The point cloud set disordered and unstructured problems are tackled by Kd-tree-based local permutation and spatial commutative pooling strategies proposed in this paper. Experiments on ModelNet illustrate that our method has two orders of magnitude less floating point computation in each non-linear mapping layer while it contributes to significant classification accuracy improvement. Compared to some of the state-of-the-art methods using the 3D volumetric image convolution, the PCN method also yields comparable classification accuracy.

This paper presents a new approach for navigating a quadrotor over undulated terrains that can be of great importance for the use of unmanned aerial vehicles in civilian applications such as monitoring of pipes, bridges and buildings. The proposed approach involves the use of a single-beam LiDAR to estimate the terrain profile under uncertainty. The LiDAR is installed at the base of a quadrotor and can be set at different angles to send information to the quadrotor about undulations of the terrain ahead. This strategy helps the quadrotor to build a smooth trajectory for the UAV and allows the controller to follow it closely. In turn, maneuverability of UAVs over and around ground-based obstacles is improved significantly in comparison to typical default autopilot controllers programmed to maintain the UAV at a given altitude. Through simulation, the result shows how this prospective technique with motion planning algorithms improves the performance of the quadrotor.

Quadrotors, like any dynamical system, are subjected to model uncertainties which may cause instability and inaccuracy in navigation. In addition, the presence of input constraints violates the affine property which, as a result, adds additional challenges and complicates designing the control system for quadrotors. In this paper, an adaptive nonlinear control algorithm is designed to overcome these difficulties for small size quadrotors. The proposed algorithm consists of an approximation technique using radial basis function neural network and a modified reference model. This allows the quadrotor to follow a defined path under the presence of external disturbances, actuator saturation and system uncertainties associated with system parameters including mass, inertia and force coefficients. Finally, the stability of the proposed algorithm is proven within the region of interest and validated using simulation.

Autonomous vehicle technology has been rapidly expanding through the incorporation of advanced driver assistance systems in many new vehicles. The integration of autonomous vehicle technology to assist and alert drivers is essential to increase driver safety. The main aim of this paper is to (1) compare real driving data from the Next Generation SIMulation I-80 dataset to an "ideal" driving scenario and (2) develop a tool that can be used to prescreen large datasets and filter the data points according to specific study parameters. This proposed tool uses a fuzzy inference system which outputs a warning level based on three inputs including relative velocity between the host and the preceding vehicle, velocity of the host vehicle and time headway. The warning level is used as a measure for initial analysis of real-life driving data to categorize the data and identity "unsafe" driving patterns. The "ideal" driving scenario and real driving data are compared and visualized using a graphical simulation in MATLAB. This visual comparison clearly highlights the importance of the integration of autonomous vehicle technology to increase driver safety.

Collision warning and avoidance is a well-established area of research for the automotive industry. However, there is little research towards vitally important collision warning systems for cyclists, who are increasingly jeopardized by motorists on urban roads, especially as quiet, fast electric vehicles become more popular. This paper describes the hardware and software of a low-cost collision warning system for cyclists. Installed on the back of a bike seat, the system consists of a single-beam laser rangefinder and two ultrasonic sensors that detect oncoming vehicles from behind, two handlebar eccentric mass vibrators that provide left and right haptic feedback to the cyclist, and a taillight that warns oncoming vehicles. Executed by an Arduino microcontroller, its software consists of a fuzzy rule-based inference system (FIS), which computes the collision risk and generates appropriate warning signals in a similar way to how a cyclist would assess collision risk based on the distance, velocity and direction of an approaching vehicle. The device was prototyped and statistically evaluated by a survey taken from a pool of seven participants. The participants tested the system before and after receiving initial training. The experimental results demonstrate the efficacy of the proposed system in warning cyclists in an intuitive manner, without distracting them.

A switching nonlinear control system consisting of multiple Lyapunov functions and an adaptive mechanism is considered for small quadrotors. A typical quadrotor has only four actuators and six degrees of freedom (DOF) which makes it an under-actuated system. Underactuation adds constraints to the control input and complicates designing the control system. In addition, quadrotor control systems suffer from the presence of parameter uncertainty. In this work, the quadrotor's parameters are considered to be unknown but bounded. The proposed control algorithm combines a switching technique with an adaptation mechanism to overcome the difficulties associated with underactuation and uncertainty. Specifically, the switching controller is designed based on multiple Lyapunov functions to deal with the underactuation problem; and the adaptive mechanism is used to overcome the parametric uncertainties. The stability of the combined controller is proven within the region of interest, and simulation with real UAV model parameters was used to show that the proposed controller allows the system to follow a designed path closely.

This paper demonstrates a new approach for model-based hybrid control of an underactuated system. The balancing of an underactuated inverted pendulum system is achieved using a reaction wheel. Specifically, a two-layer hybrid controller is proposed with control algorithms implemented into prioritized states from the calculated dynamic equations. The first layer controller uses the reaction wheel with feedback linearization to control the first priority parameter at the non-actuated joint. The second layer controller then governs the first layer controller to guide the actuated joint to reach the set point. The effectiveness of the hybrid controller on the underactuated dynamic nonlinear system is illustrated by simulations in MATLAB.

This paper demonstrates a new approach for the use of multiple strain sensors on a wearable flexible finger band to measure the posture and movement of a human finger accurately. The system is further developed to repeat the human finger motion on a robotic finger. Here, we used adaptive network-based fuzzy interface system (ANFIS) to relate the strain sensor readings to human finger posture and motion. The input and output measurements used to train ANFIS are obtained from the strain sensors of the wearable platform and a 3 degree of freedom (DOF) exoskeleton testbed, respectively. The ANFIS model is then used to predict human finger posture and motion directly from the strain sensors installed on the finger band. We made additional experiments and generated testing data using the exoskeleton testbed to verify the ANFIS model. Finally, we demonstrate that the robotic finger closely follows the human finger motion by reading the wearable finger band output and calculating the posture and motion parameters in real time.

This paper presents a model predictive control (MPC) motion planning and control scheme for an automated firefighting system. The proposed automated firefighting system consists of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) cooperating to detect, localize, and handle the wildfire. In this automated cooperative framework, the UGVs have to reach to the fire (target) in an optimal way, optimal in the sense that the operation time and fuel consumption are minimized. Meanwhile, the UAVs gather and report the localization data to the system. Furthermore, the UAVs motions are intended to reduce the system uncertainty. The proposed motion planning scheme is designed to handle various sources of uncertainty as well as environment constraints through incorporating them to the conventional nonlinear model predictive control. Numerical simulations have been carried out to study the performance of the proposed MPC motion planning and control scheme. Simulation results demonstrate that the proposed algorithm is able to design an optimal trajectory while handling the system uncertainties and constraints.

Formation control of nonholonomic robots in dynamic unstructured environments is a challenging task yet to be met. This paper presents the unscented model predictive control (UMPC) approach to tackle the formation control of multiple nonholonomic robots in unstructured environments. In unscented predictive control, the uncertainty propagation in the nonholonomic nonlinear motion model is approximated using the unscented transform. The collision avoidance constraints have been introduced as the chance constraints to model predictive control. The UMPC approach enables us to find a closed form of the collision avoidance probabilistic constraints. The desired pose of each robot in the formation is introduced through the local objective function of UMPC of each robot. The simulation results indicate the effective and robust performance of UMPC in unstructured environment in the presence of action disturbance and communication signal noise.

This paper proposes a novel fuzzy evidential rule-based (FERB) system to represent uncertain expert knowledge. The fuzzy evidential reasoning framework is introduced to model epistemic uncertainties including nonspecificity, vagueness, as well as local and global ignorance in the knowledge base. A computationally efficient formulation of the FERB system using an uncertain IF-THEN rule matrix is presented. Inference is performed through determining the fired rules followed by fuzzy Dempster-Shafer combination of activated belief structures. The application of the proposed FERB system is investigated using a case study of risk assessment for drinking water. Finally, the proposed FERB system is tested through data that were available for a water distribution network.

In this paper an efficient object correspondence algorithm is presented for tracking multiple objects in dynamic environments is proposed. It is assumed that objects can be added to the scene, removed from the environment, or occluded by other objects. The proposed algorithm benefits from two key features including a confidence measure and a two-way matching mechanism to improve the correspondence accuracy. Unlike the traditional methods which solve the correspondence problem by matching new objects and then removing the incorrect ones, our algorithm avoids establishing invalid correspondences considering a confidence measure. Also, we introduce a two-way correspondence algorithm consisting of forward matching and backward matching. As a result, the track of objects is expanded both from the head and tail using new objects and previous unmatched objects respectively. The proposed method has been applied to synthetic data, and the results show the efficiency and reliability of the method against a large number of objects.

Empathy is a key component of interpersonal interactions that is often neglected by modern communication technologies. This paper presents the theoretical basis, prototype, and preliminary user testing for an application of affective computing in mediating live human-to-human interactions. The proposed system uses facial expression recognition to identify the emotional state of a user's conversation partner. It algorithmically generates emotional music to match the expressive state of the partner and plays the music to the user in a non-disruptive manner. Preliminary user studies indicate that the prototype system can generate music that users reliably associate with the emotions of anger, happiness, fear, and sadness and that the presence of emotional music augments the emotional response generated by visual cues.

An unscented predictive motion planning algorithm of a nonholonomic system using output feedback is presented. A two-wheeled robot is selected as the example nonholonomic system. A predictive motion planning scheme is used to find the suboptimal control inputs. In addition to the nonholonomic constraint, state estimation and collision avoidance chance constraints are incorporated to the predictive scheme. The closed form of the probabilistic constraints is solved by utilizing the unscented transform of the motion model. Numerical simulation results demonstrate a high level of robustness and effectiveness of the proposed algorithm in the presence of disturbances, measurement noise and chance constraints.

This paper proposes a new belief structure fuzzy inference system (FIS) that can model vagueness, ambiguity and interval uncertainties in the knowledge base. The interval belief structure is introduced to define the rules of an FIS and build uncertain knowledge. Fuzzy implication followed by extended fuzzy Dempster-Shafer combination is presented for inference in the proposed rule-based system. The application of the interval belief structure FIS is investigated through the implementation of a rule-based system for microbial water quality risk assessment. Finally, the proposed FIS is tested through historical datasets that were available for a water distribution network.

This work investigates the problem of combining deficient evidence for the purpose of quality assessment. The main focus of the work is modeling vagueness, ambiguity, and local nonspecificity in information within a unified approach. We introduce an extended fuzzy Dempster–Shafer scheme based on the simultaneous use of fuzzy interval-grade and interval-valued belief degree (IGIB). The latter facilitates modeling of uncertainties in terms of local ignorance associated with expert knowledge, whereas the former allows for handling the lack of information on belief degree assignments. Also, generalized fuzzy sets can be readily transformed into the proposed fuzzy IGIB structure. The reasoning for quality assessment is performed by solving nonlinear optimization problems on fuzzy Dempster–Shafer paradigm for the fuzzy IGIB structure. The application of the proposed inference method is investigated by designing a reasoning scheme for water quality monitoring and validated through the experimental data available for different sampling points in a water distribution network. © 2011 Wiley Periodicals, Inc.

In this paper the design and development of a crawling robot for inspection of live water pipes are addressed. The mechanical design of the robot is described in detail. The governing dynamics equations of the robot moving against water flow as well as gravity in a straight pipe are also derived. Specifically, the hydrodynamic forces exerted on the robot when moving in a live pressurized pipe are taken into account. Two fuzzy-logic based control strategies are adopted. The first one is to maintain a constant translational speed in robot's motion when subjected to flow disturbances that are numerically modeled using step changes in flow velocity within a human-in-the-loop real-time simulation environment, and the second is to steer the real robot inside the pipe while following a numerically modeled time-varying velocity set point with no fluid present in the pipe. The controller parameters were tuned based on data obtained from a human-in-the-loop control system via an artificial neural network.

In this paper the design and development of a crawling robot for inspection of live water pipes is addressed. The mechanical design of the robot is described briefly. A fuzzy-logic based control strategy is adopted to maintain a constant translational speed in robotpsilas motion when subjected to flow disturbances that are numerically modeled using step changes in flow velocity within a simulation environment. The controller has been synthesized in real-time through a human-in-the-loop setting within a virtual reality environment.

This paper presents a sensor fusion framework that incorporates a monocular vision system, laser range finder and an inertial navigation sensor (INS) for localization of mobile robots and manipulators. The proposed method is particularly useful for applications in which there is a featured wall or floor in proximity to the robots. Examples include unmanned ground vehicles (UGV), unmanned aerial vehicles (UAV), and field inspection robots. In essence, the proposed method fuses image mosaicing and dead reckoning where the sequential images of a digital camera are stitched together with the help of an inertial navigation system (INS). In the proposed model, the processing load of image mosaicing is reduced significantly, and at the same time accumulation of error of the INS is prevented. The localization method was developed as a standalone module and tested using a hardware-in-the-loop simulator explained in this paper. This module will eventually be used for autonomous navigation of a pipe inspection robot capable of carrying nondestructive testing (NDT) sensors and visual inspection instruments into large water mains.

Site seismic hazard assessment entails the integration of ground shaking, landslide and liquefaction potential. Liquefaction induced damage has highlighted the importance of considering liquefaction in earthquake risk assessment. Discernment of liquefaction vulnerability is a complex and nonlinear procedure that is influenced by model and parameter uncertainty. Inherent uncertainties encountered in the model can be handled with fuzzy based methods, which are capable of incorporating information obtained from expert knowledge and datasets. In this paper, the use of fuzzy template based (FTB) modeling is proposed for assessing earthquake-induced liquefaction. The efficiency of the proposed method is verified using actual liquefaction data reported in the literature. Overall, the correct liquefaction classifications obtained are 98% and 92% for training validation datasets, respectively.

This paper describes the development of a medical diagnosis expert system that can be used by physicians in their daily practices. Differential artificial intelligence techniques are incorporated into the expert system to best represent the various stages of the diagnosis process. A linear scoring system is used to represent the initial subjective analysis stage, while a rule-based fuzzy expert system is used to interpret lab tests and imaging studies to confirm final diagnosis. An actual example of patient walkthrough is used to demonstrate various computation steps from embedding the patient information to reaching the final diagnosis.

This paper describes a method of combining fuzzy inference and evidential reasoning to quantify the corrosion rate of buried metallic pipes, i.e., cast-iron and ductile-iron. The method relies on two bodies of evidence: the corrosivity of surrounding soil and the corrosion rate estimated from measured maximum pit depth. Fuzzy inference is used to deduce a corrosivity criterion, viz., corrosivity potential (CoP), for each body of evidence. The two CoPs are then fused using evidential reasoning to obtain a CoP that is expected to be more reliable than that obtained from any one individual body of evidence. This type of criterion can help utility managers make informed decisions on how to protect their pipes exposed to different soil conditions. The proposed reasoning framework is demonstrated through a case study based on soil properties and corrosion rate data.

Presents the application of an image registration method for a mobile manipulator. The robot is used for scanning natural terrain and detecting metal objects hidden beneath the terrain surface (e.g., landmines) using a metal detector. The range image may be interpreted for visual servoing, map building and path planning, or object recognition. In the work, the image is used to build a terrain map for obstacle free path planning. Because the working area of the robot is extremely dynamic (i.e., not only the robot travels but also the environment is also subject to change) an active range sensing method is selected to provide the range image. The range values are acquired using a laser range finder with a rotating mirror for scanning so that sensor fusion in the form of collecting sensor readings over an extended period of time is required. In addition, range readings of two ultrasonic range finders are fused at signal level to tackle both sensor imperfection and environmental illumination that induce uncertainty at the system. We explain the use of a real-time programming platform that executes an online map-building process in parallel for robot manipulation and control.