This paper presents a comprehensive study of robotic manipulation tasks, focusing on the movement planning and task-handling capabilities of robots. We introduce a novel approach that employs Dynamic Movement Primitives (DMP) for movement planning, coupled with an evolutionary algorithm, specifically the Genetic Algorithm (GA), for hyperparameter tuning of the DMP. Our method significantly enhances the system's precision and control, thereby facilitating a more accurate output. Furthermore, we conduct a comparative analysis of two optimization techniques - user-based and GA-based. Our findings indicate that the GA-based technique offers superior precision, underscoring its potential in advancing robotic manipulation tasks.

Channel pruning approaches for convolutional neural networks (ConvNets) deactivate the channels, statically or dynamically, and require special implementation. In addition, channel squeezing in representative ConvNets is carried out via$1\times 1$convolutions which dominates a large portion of computations and network parameters. Given these chal-lenges, we propose an effective multi-purpose module for dynamic channel sampling, namely Pick-or-Mix (PiX), which does not require special implementation. PiX divides a set of channels into subsets and then picks from them, where the picking decision is dynamically made per each pixel based on the input activations. We plug PiX into prominent ConvNet architectures and verify its multi-purpose utilities. After replacing$1\times 1$channel squeezing layers in ResNet with PiX, the network becomes 25% faster without losing accuracy. We show that PiX allows ConvNets to learn bet-ter data representation than widely adopted approaches to enhance networks' representation power (e.g., SE, CBAM, AFF, SKNet, and DWP). We also show that PiX achieves state-of-the-art performance on network downscaling and dynamic channel pruning applications. Code: https://github.com/ashishkumar822/PiX

Emotional stability, awareness, and attention may likely be enhanced by meditation and related techniques. Since meditation practitioners may need focus and engagement, virtual reality (VR) may be helpful. Even though there has been some research on the usefulness of VR for meditation, very few studies have looked at the effectiveness of VR on audible mantra repetition (AuMR). Our research addresses this limitation by investigating the efficacy of AuMR, which is assigned to promote better cognitive health and overall brain well-being in VR. Fortyone individuals were randomly divided into two groups, test and control. The test group was engaged in a ten-minute VR-based AuMR session, while the control group did nothing in the same virtual reality setting for ten minutes. Both groups completed self-reported questionnaires before and after the intervention and electroencephalography (EEG) and heart rate variability (HRV) measurements. We evaluated EEG band power ratios such as alpha-to-beta (AB) ratio and frontal-alpha-to-temporal-theta (FATT) ratio to find the effects of VR-aided meditation. The findings of the ANOVA test demonstrated a substantial decrease in the self-reported stress, anxiety, and depression parameters. Furthermore, comparing the test group to the control group revealed a significant increase in the FATT ratio and a significant decrease in the AB ratio. We also observed significant changes in the HRV values of the test group. The study offers sufficient evidence to suggest the feasibility of AuMR in VR for cognitive wellness.

This paper proposes a novel approach to recognizing dynamic hand gestures facilitating seamless interaction between humans and robots. Here, each robot manipulator task is assigned a specific gesture. There may be several such tasks, hence, several gestures. These gestures may be prone to several dynamic variations. All such variations for different gestures shown to the robot are accurately recognized in real-time using the proposed unsupervised model based on the Gaussian Mixture model. The accuracy during training and real-time testing prove the efficacy of this methodology.

This paper proposes a novel optimal control scheme for constrained image based visual servoing of a robot manipulator. For a robot manipulator with an eye-on-hand configuration, visibility constraint is an essential requirement to avoid servo failure, while robot’s actuator limits must also be satisfied. To ensure this, the constraints are modelled implicitly via learning the task and defining safe regions using expert human demonstrations via mixture of Dynamic Movement Primitives (DMPs). The visual servoing problem is then formulated as a closed-loop optimal control problem using these constraint model where a desired target (possibly time-varying) is obtained by acting upon the feedback from the real-time visual sensors. The visual servo control loop consists of a single network adaptive critic optimal tracking control scheme whose weights are tuned using Lyapunov stability criteria. The stability and the performance of the proposed control scheme is shown theoretically via Lyapunov approach and also verified experimentally using a seven degree of freedom (DOF) Franka Emika and six DOF Universal Robot (UR) 10 manipulator. The approach is also demonstrated on a use case scenarios in mock-up convenience store and warehouse setup.Note to Practitioners—The applications of robots in busy warehouses, healthcare sectors and convenience store, has a big societal impact. The scenarios in these real-world problems often consist of a dynamic environment. Therefore, sensor-based localization and planning, along with satisfying robot and task constraints are needed. These naturally adds up to the programming costs in addition to the robot platform and actuation. However, modern robots need intuitive and easy programming for more pervasive in a practical societal application. In our proposed method, this sensor-based planning with environmental constraints is modelled implicitly using the Programming by Demonstration framework. The user needs to catch the robot arm by his hand and teach the task at hand, and the framework captures the task and robot constraints while generalizing to new goals. This modelling, along with cost optimal controller, generates real-time constraint aware robot manipulation trajectories for the demonstrated task in dynamic environments.

The task of fully automated picking of novel bin objects that are placed in a densely cluttered pile poses a significant challenge. It becomes even more challenging if the objects are of various shapes, sizes, colors, and textures. Generally, grasp planning for a given scene begins with sampling several grasp poses, which are then evaluated to determine the final grasp pose for the robot action. With the increment in clutter level in the bin, the fraction of graspable locations becomes smaller. Hence, for a scalable grasp pose planning strategy, the pose sampling method should be intelligent enough to find suitable grasp regions in a time-efficient manner, irrespective of the amount of clutter and the workspace size. In this paper, we present a scalable robotic bin-picking method (SE-RoB) that performs equally well amidst the increasing level of clutter in the scene. In a real-world challenging bin-picking setup, our proposed method has shown significantly better performance ($13\%$improvement) compared to the state-of-the-art methods in terms of grasp success rate in a time-efficient manner (6 Hz inference speed).Note to Practitioners—Picking objects from a cluttered pile can be challenging, especially when dealing with objects of different shapes, sizes, colors, and textures. We have created a new method called SE-RoB that can help robots pick up unknown objects from a pile reliably and in a time-efficient manner. In our experiments, we tested this method in real-world situations where the pile of objects was densely cluttered and had many different types of objects. SE-RoB worked much better compared to the state-of-the-art methods. Additionally, this method works at around 6 Hz inference speed, which means it is suitable for real-life scenarios such as warehouse automation. We believe that SE-RoB can be a valuable tool for practitioners who are looking to automate their bin-picking processes using robots.

This paper presents a singularity free fast terminal sliding mode control (SFFTSMC) for position and attitude tracking of a quadrotor with unknown dynamics. The contribution of this work is threefold, namely: devising an SFFTSMC strategy using an auxiliary function to eliminate the singularity in the control law, a cost-saving minimum parameter update scheme using a fully connected recurrent neural network (FCRNN) to handle the unknown dynamics of quadrotor and development of an adaptive control law to compensate for the loss of effectiveness of the actuator and saturation effects, considered explicitly, without the requirement of any fault detection and diagnosis (FDD) unit. The proposed approach offers a direct singularity free control action in the presence of unknown dynamics, actuator faults, and saturation. Overall closed-loop stability is guaranteed in finite-time using Lyapunov stability theory and update laws for minimum parameters of FCRNN and fault compensator are derived using the same. Extensive simulations are presented for the trajectory tracking tasks using the proposed method and compared with the existing piece-wise fast terminal sliding mode controller (PFTSMC) and Radial basis function network (RBFN) based fault compensation approach. The proposed method is also validated in the Gazebo simulator via Pixhawk autopilot to demonstrate the feasibility in real-time. Note to Practitioners—This research is motivated by a cost-effective control scheme for quadrotors in cases of unknown/uncertain dynamics, external disturbances, actuator faults, and saturation. Precise quadrotor dynamics are difficult to obtain in real time. Also, there may be parametric uncertainties in quadrotor parameters like mass, inertia, and aerodynamic coefficients. External disturbances like wind gusts are always present in outdoor environments, and it may be the case that quadrotor motors lose their effectiveness due to defective motors or propeller damage while flying. To address these issues, first, it requires a precise control scheme to control the quadrotor while guaranteeing stability to ensure the safety of the quadrotor itself and people in the surrounding environment. Second, the approach should not be computationally heavy, which may sluggish the overall response of the quadrotor. Thus, we propose a control scheme to address these two issues simultaneously using a new fast terminal sliding mode control scheme augmented with FCRNN by updating minimum parameters instead of all the weight parameters of FCRNN. A compensator is introduced to mitigate the effect of unexpected actuator fault and saturation. The proposed approach can be helpful in various quadrotor applications, like pesticide spraying in agriculture, payload transportation, search and rescue, construction, etc., where unknown quadrotor dynamics and unknown payload situations occur.

In this work, we focus on low-resource dependency parsing for multiple languages.Several strategies are tailored to enhance performance in low-resource scenarios.While these are well-known to the community, it is not trivial to select the best-performing combination of these strategies for a low-resource language that we are interested in, and not much attention has been given to measuring the efficacy of these strategies.We experiment with 5 lowresource strategies for our ensembled approach on 7 Universal Dependency (UD) low-resource languages.Our exhaustive experimentation on these languages supports the effective improvements for languages not covered in pretrained models.We show a successful application of the ensembled system on a truly low-resource language Sanskrit. 1

Deep neural networks have demonstrated excellent object detection and segmentation performance from RGB data. However, these models can only recognize and predict segmentation masks with great accuracy when RGB data have sufficient information about the objects of interest. In this paper, we suggest an intelligent, active perception system that can adjust its 3D position to improve signal acquisition. The segmentation score of cluttered scene is improved a lot due to this proposed system, which can also enhance grasp pose detection for the robotic manipulator. The ResNet-50 backbone of the proposed perception system is initialized using pre-trained weights to extract a latent state from an RGB image of the cluttered scene. A Reinforcement Learning (RL) agent uses these retrieved states to reposition the visual perception system for enhancement of the underlying computer vision tasks such as segmentation of the cluttered scene. Our trained RL agent can anticipate the better position of the visual perception system, which ensures enhanced signal recovery. The effectiveness of the proposed approach is tested in a pybullet simulation environment.

Machine Learning (ML) classifiers have been made more robust in recent years by leveraging the graph structure between the inputs using Neural Structured Learning (NSL). However, researchers have not taken full advantage of it for the Brain-Computer Interface (BCI) classification tasks. While the traditional NSL faces the issues of a very minimized use of graph structural properties and optimal similarity metric, in this paper, we propose a Node Impact Multi Metric Threshold NSL (NI-MT-NSL) to overcome these issues. For the first time, the node-influence properties from graph theory are incorporated to alter the way different EEG samples influence the training, while an ensemble of semantic graphs is used in the NSL module to capture different semantic relations between the EEG trial data. The proposed model is assessed on the standard BCI IV 2a dataset. On comparing its test accuracies with the traditional Riemannian classifiers and the baseline NSL, we have found improved accuracies over all subjects. We have found a tremendous improvement in classification, with a mean gain of 7% for the subjects having very poor accuracy even with state-of-the-art methods.

In this article, we propose a novel stochastic event-driven near-optimal sliding-mode controller design for addressing the consensus of a multiagent system in a network. The system is prone to external disturbances and network uncertainties, such as losses and delays of data packets. The randomness of network uncertainties introduces stochasticity in the system. The design starts with the formulation of control-affine dynamics based on a single integrator robot model, formation error, and sliding surface dynamics. An event-triggering condition is then derived for an update of control input for each agent. These input updates guarantee desired consensus in finite time with reaching time of each agent's sliding surface having an upper bound. The admissibility of event-driven near-optimal control updates is also ensured for each agent. The near-optimal control design for each agent has achieved through neural-network-based actor-critic architecture. The implementation of Pioneer P3-DX mobile robots illustrates threefold efficacy of the proposed design: 1) advantages of event-driven approach and higher order sliding mode controller; 2) robustness to network uncertainties; and 3) near-optimality in system performance.

The phenomenon of compounding is ubiquitous in Sanskrit. It serves for achieving brevity in expressing thoughts, while simultaneously enriching the lexical and structural formation of the language. In this work, we focus on the Sanskrit Compound Type Identification (SaCTI) task, where we consider the problem of identifying semantic relations between the components of a compound word. Earlier approaches solely rely on the lexical information obtained from the components and ignore the most crucial contextual and syntactic information useful for SaCTI. However, the SaCTI task is challenging primarily due to the implicitly encoded context-sensitive semantic relation between the compound components. Thus, we propose a novel multi-task learning architecture which incorporates the contextual information and enriches the complementary syntactic information using morphological tagging and dependency parsing as two auxiliary tasks. Experiments on the benchmark datasets for SaCTI show 6.1 points (Accuracy) and 7.7 points (F1-score) absolute gain compared to the state-of-the-art system. Further, our multi-lingual experiments demonstrate the efficacy of the proposed architecture in English and Marathi languages.

Picking unseen objects from clutter is a difficult problem because of the variability in objects (shape, size, and material) and occlusion due to clutter. As a result, it becomes difficult for grasping methods to segment the objects properly and they fail to singulate the object to be picked. This may result in grasp failure or picking of multiple objects together in a single attempt. A push-to-move action by the robot will be beneficial to disperse the objects in the workspace and thus assist the grasping and vision algorithm. We propose a disperse and pick method for domain-independent robotic grasping in a highly cluttered heap of objects. The novel contribution of our framework is the introduction of a heuristic clutter removal method that does not require deep learning and can work on unseen objects. At each iteration of the algorithm, the robot either performs a push-to-move action or a grasp action based on the estimated clutter profile. For grasp planning, we present an improved and adaptive version of a recent domain-independent grasping method. The efficacy of the integrated system is demonstrated in simulation as well as in the real-world.

In this work, we present a Programming by imitation for a robotic manipulation system, which can be programmed for various tasks from only a single demonstration. The system is primarily based on the three components: i) scene parsing, ii) action classification, and iii) dynamic primitive shape fitting. All the above modules are developed by leveraging state-of-the-art techniques in 2D and 3D visual perception. The primary contribution of this system paper is an imitation-based robotic system that can replicate highly complex tasks by executing elementary task-specific program templates, thus avoiding extensive and exhaustive manual coding. In addition, we contribute by introducing a primitive shape fitting module by which it becomes easier to grasp objects of various shapes and sizes. To evaluate the system performance, the proposed robotic system has been tested on the task of multiple object sorting and reports 91.8% accuracy in human demonstrated action detection, 76.1% accuracy in action execution, and overall accuracy of 80%. We also examine the proposed system's component-wise performance to demonstrate the efficacy and deployability in industrial and household scenarios.

Meditation methods, which have their origins in ancient traditions are gaining popularity as a result of their potential mental and physical health advantages. EEG neural correlates underlying enhanced cognitive abilities such as sustained attention and working memory need to be analyzed scrutinizingly to evaluate the effects of meditation practices. In this article, we thus provide an analysis of EEG features such as various band powers and connectivity based features to evaluate the meditation effects. Also, we provide a classification framework to classify the meditation states from the baseline EEG states. We report our results on an in house dataset of 20 participants(10 experienced and 10 novice) who underwent a two-week long mantra meditation practice. Strikingly we have found out that, as the novice participants practice meditation overtime, the accuracies of machine learning classification between the baseline EEG versus meditative EEG of the novice increase significantly, as an indication of their enhanced meditation experiences. Also, we found out that even such short and regular meditation practices, the cognitive abilities of novice meditators get enhanced which are evaluated through Brain-Based Intelligence Test (BBIT) psychometric tests and the results that got reflected in their EEG correlates.

The goal of this work is to develop the next generation of coordinated optimal planning and control schemes for real-world robotic applications, with cost-effective intelligent robots that can safely and robustly perform the tasks at hand. This article presents a dual-loop optimal hierarchical control scheme for robotic manipulators consisting of outer and inner loops. The outer kinematic control loop in the operational space provides a joint velocity reference signal to the inner one. The kinematic control is formulated as a closed-loop optimal control approach to trajectory generation where the desired target (possibly time-varying) is obtained by acting upon the feedback from the actual state of the robot. The kinematic control is obtained using a closed-form analytic solution of the Hamilton–Jacobi–Bellman (HJB) equation. The proposed methodology defines the task in terms of the integral cost function, which results in a global optimal solution. The inner dynamic control loop consists of a neural network (NN)-based adaptive critic (AC) optimal tracking control scheme. The online NN approximator-based dynamic controller learns the infinite-horizon cost function related to inner loop error dynamics in continuous time and calculates the corresponding optimal control input to minimize the cost function forward in time. The stability and the performance of the proposed control scheme are shown theoretically via the Lyapunov approach and also verified experimentally using a 7-DOF Barrett WAM robot manipulator. The warehousing applications of the proposed dual-loop control scheme have been demonstrated experimentally in an exact warehouse setting.Note to Practitioners—This article was motivated by our previous experiences in the Amazon Robotics Challenge (ARC) competitions as a participant. Although the major lessons from those events revealed many important issues toward warehouse automation both from robotic vision and grasping aspects, the application aspects of control engineering to solution for these real-world problems could significantly benefit the today’s highly automated society. The intention of this article is to address this situation by designing a cost-effective robot manipulator control scheme by optimizing robot manipulation trajectories in the outer loop along with the actuator input in the inner loop. Therefore, we propose a dual-loop optimal control scheme for robotic manipulations under a cost-optimal framework with guaranteed stability proof via the Lyapunov approach. This results in smoother trajectories with cost-effective control actuator inputs than the state-of-the-art methods. We believe that this work can be beneficial for academic and industrial research to design more advanced and optimized solutions in this domain.

This work proposes a novel event-triggered exponential supertwisting algorithm (ESTA) for path tracking of a mobile robot. The proposed work is divided into three parts. In the first part, a fractional-order sliding surface-based exponential supertwisting event-triggered controller has been proposed. Fractional-order sliding surface improves the transient response, and the exponential supertwisting reaching law reduces the reaching phase time and eliminates the chattering. The event-triggering condition is derived using the Lipschitz method for minimum actuator utilization, and the interexecution time between two events is derived. In the second part, a fault estimator is designed to estimate the actuator fault using the Lyapunov stability theory. Furthermore, it is shown that in the presence of matched and unmatched uncertainty, event-trigger-based controller performance degrades. Hence, in the third part, an integral sliding-mode controller (ISMC) has been clubbed with the event-trigger ESTA for filtering of the uncertainties. It is also shown that when fault estimator-based ESTA is clubbed with ISMC, then the robustness of the controller increases, and the tracking performance improves. This novel technique is robust toward uncertainty and fault, offers finite-time convergence, reduces chattering, and offers minimum resource utilization. Simulations and experimental studies are carried out to validate the advantages of the proposed controller over the existing methods.

This article proposes an online stochastic dynamic event-based near-optimal controller for formation in the networked multirobot system. The system is prone to network uncertainties, such as packet loss and transmission delay, that introduce stochasticity in the system. The multirobot formation problem poses a nonzero-sum game scenario. The near-optimal control inputs/policies based on proposed event-based methodology attain a Nash equilibrium achieving the desired formation in the system. These policies are generated online only at events using actor-critic neural network architecture whose weights are updated too at the same instants. The approach ensures system stability by deriving the ultimate boundedness of estimation errors of actor-critic weights and the event-based closed-loop formation error. The efficacy of the proposed approach has been validated in real-time using three Pioneer P3-Dx mobile robots in a multirobot framework. The control update instants are minimized to as low as 20% and 18% for the two follower robots.

Passive electroencephalogram (EEG) brain–computer interfaces (BCI) have common usage in the area of Driver Drowsiness Detection. The approach presented herein identifies the cognitive state of the user while no mental action is required. Data recorded in EEG-based BCI experiments are generally noisy, nonstationary, and contaminated with artifacts that can deteriorate any analyzer’s performance. Recently, common spatial patterns (CSPs) have been adapted with EEG state-space incorporating spatiospectral optimization using fuzzy time delay (FTD-CSSP). Temporal phase disparity sequence (TPDS) is used to measure synchrony between EEG signals. The output of Linear transforms operating on the TPDS constitute useful features for EEG regression problems. On similar lines, this article proposes spatiospectral optimized fuzzy-independent phase-locking value (SSO-FIPLV) representations (exploiting the spatiospectral information from TPDS) for EEG signals to monitor a user’s cognitive states. Specifically, we analyze changes in EEG synchronization for a car driver as she/he drifts between alert and drowsy states. We use neural networks (NNs) for prediction. This article also proposes a cutting-edge method for training NN using the Euler–Lagrangian formulation. A stability proof is provided for the intended training approach alongside, and the performance is corroborated on the EEG reaction time prediction task, both within and across subjects, using a publicly available dataset. The NN trained by the proposed approach performs better than other competitive approaches in terms of minimizing root-mean-squared error and maximizing correlation coefficient. Channelwise feature importance in terms of average relevance values calculated from NN feature representations is visualized in the form of Topoplots using layerwise relevance propagation for regression.

This article focuses on developing an adaptive fast terminal sliding-mode controller (AFTSMC) with power rate proportional reaching law for the position and altitude tracking of a quadrotor in the presence of parametric uncertainties and bounded external disturbance. A nonlinear fast terminal sliding surface is proposed for the fast and finite-time convergence of the tracking error despite having the system states far away from the equilibrium point. Also, a power rate proportional reaching law has been proposed that ensures fast and finite-time convergence of the sliding manifold while attenuating the chattering phenomena in the sliding phase. To avoid the problem associated with over-estimation of the unknown disturbance bound, which eventually leads to chattering, an adaptive tuning law for gain adaptation is developed based on the Lyapunov’s stability theory that completely eradicates the necessity of knowing the upper bound of the disturbancea priori. The finite-time stability of a closed-loop system is investigated using the Lyapunov theory. The effectiveness of the proposed scheme is compared with an adaptive sliding-mode controller (ASMC) using extensive simulation and validated on the DJI Matrice 100 quadrotor as a proof of concept on the hardware platform.

This paper proposes a novel 3 DoF vision augmented Quadrotor model for visual servoing. The proposed model eliminates the necessity of deploying a separate visual-servoing controller and a robot controller, thereby reducing the on-board computational load drastically. The proposed model, as opposed to the conventional PBVS and IBVS approaches, helps in the use of torque control strategies. To utilize this feature of the model, a non-singular fast-terminal sliding mode modified super-twisting controller (NSFTSM-MSTC) is proposed. The non-singular fast-terminal sliding manifold ensures the fast and finite time convergence of the error between the desired and actual points of interest, while ensuring smoother transitions in the quadrotor states. The modified super-twisting reaching law ensures that the control input is continuous thereby ensuring chattering attenuation. The overall system stability is presented using Lyapunov’s stability criteria and an expression for convergence time is also derived. The proposed theory is validated using numerical simulations and is compared with the existing conventional sliding mode based visual servoing approach (CSMVS).

In this work, we present a robotic solution to automate the task of wall construction. To that end, we present an end-to-end visual perception framework that can quickly detect and localize bricks in a clutter. Further, we present a light computational method of brick pose estimation that incorporates the above information. The proposed detection network predicts a rotated box compared to YOLO and SSD, thereby maximizing the object’s region in the predicted box regions. In addition, precision (P), recall (R), and mean-average-precision (mAP) scores are reported to evaluate the proposed framework. We observed that for our task, the proposed scheme outperforms the upright bounding box detectors. Further, we deploy the proposed visual perception framework on a robotic system endowed with a UR5 robot manipulator and demonstrate that the system can successfully replicate a simplified version of the wall-building task in an autonomous mode.

This paper presents an intelligent robust controller design scheme for a quadrotor where system dynamics of quadrotor are not known and quadrotor actuators may suffer from partial loss of effectiveness (PLOE) along with the actuator saturation. Robust controller design employs an integral fast terminal sliding mode controller (IFTSMC) which provides faster response and singularity free control action along with the finite-time convergence. Actuator saturation is accommodated by employing an auxiliary variable. To cope with the unknown quadrotor dynamics, a radial basis function neural network (RBFN) along with the robust compensator has been introduced to directly approximate control law instead of approximating system dynamics separately. The letter requires more computational effort and that is greatly reduced by the proposed methodology. Weights of RBFN are updated in real-time where update laws of these weights are obtained using the principle of Lyapunov stability and overall stability of the system is guaranteed using the same. To show the effectiveness of the proposed approach, simulation results have been provided for ∞-shaped trajectory tracking.

The displacement-based formation control for perturbed multirobot systems, with practical issues like collision avoidance and connectivity assurance, is a challenging problem. This article presents and implements a two-step design process consisting of both holonomic and nonholonomic frameworks along with a process of demonstration to solve this problem. We use the process of demonstration to obtain the parameters of the desired trajectory. In a holonomic framework, each virtual robot is described as double integrator system. This framework generates a set of reference trajectories. The idea is to feed these generated reference points for the mobile robots to track under nonholonomic framework. This article formulates control laws under which multiple mobile robots simply connected are stable while ensuring collision avoidance and connectivity. Both holonomic and nonholonomic models are subjected to external disturbances. In the holonomic framework, the proposed controller ensures collision avoidance and connectivity while maintaining the desired formation. The proposed controller in the nonholonomic framework tracks the reference trajectories while guaranteeing Lyapunov stability. The proposed approach is scalable to any n-robot systems which are simply connected. Both simulation and experimental results prove the efficacy of the proposed approach.

This article proposes an adaptive integral sliding mode control (ISMC) strategy for quadrotor control that ensures faster and finite-time convergence along with chattering attenuation. Quadrotor dynamics are assumed to be unknown because of the high degree of parametric uncertainties, including external disturbances. The equivalent control law obtained by ISMC consists of quadrotor dynamics and, thus, cannot be applied to the quadrotor. A new fully connected recurrent neural network (FCRNN) controller has been proposed to mimic the equivalent control instead of estimating the Quadrotor dynamics separately. The proposed FCRNN architecture consists of output feedback to the input layer and the hidden layer, which enhances the approximation capability of FCRNN. All hidden layer neurons receive self-feedback and feedback from other hidden layer neurons, which further strengthens FCRNN's potential to capture complex dynamic characteristics. As learning should happen in finite time, the finite-time stability of the overall system has been guaranteed using the Lyapunov stability theory, and the update laws for FCRNN weights in real time are derived using the same. To show the effectiveness of the proposed approach, a comprehensive analysis has been done against existing SMC strategy and against well-known function approximation techniques, e.g., the radial basis function network (RBFN) and RNN.

This paper is concerned with the motor skill learning from human demonstrations using the framework of dynamic regressive models (DRMs). The DRM-based motion planner is preferred as it generates the end-effector trajectory dynamically based on the current state of the end-effector. Within existing frameworks, a single DRM can learn a single motion profile. In addition, such learned DRMs from the data may not be stable. This paper addresses both these issues in a comprehensive manner. In this paper a single DRM has been used to encode human demonstrations involving multitask profiles and multiple task-equilibriums which is novel. We have introduced the idea that the learned DRM will generate human-like stable motion if the energy dissipation rate (EDR) of the generated trajectory follows that of the human demonstration. Thus, the DRM structure has been modified by adding a continuous guiding signal which can be called as the control signal. This signal has been derived using control theoretic principle to ensure asymptotic stability while maintaining the EDR equivalent to that of the human demonstration. The asymptotic stability of the learned DRM has been established by involving a nonmonotonic Lyapunov function consisting of first derivative of a quadratic function and the energy function associated with the DRM. The proposed framework can be learned using many existing regression techniques in this paper Gaussian mixture regression, locally weighted projection regression, and support vector regression techniques have been used successfully. During the pick and place tasks, human demonstrations involving multiple task profiles and multiple task-equilibriums are generated using a 7 DOF commercial robot manipulator. Experimental validations show that the DRMs learned using these three regression schemes are able to guide the robot along the multitask profiles in a stable manner.

EEG signals can be processed and classified into commands for brain-computer interface (BCI). Stable deciphering of EEG is one of the leading challenges in BCI design owing to low signal to noise ratio and non-stationarities. Presence of non-stationarities in the EEG signals significantly perturb the feature distribution thus deteriorating the performance of Brain Computer Interface. Stationary Subspace methods discover subspaces in which data distribution remains steady over time. In this paper, we develop novel spatial filtering based feature extraction methods for dealing with nonstationarity in EEG signals from a drowsiness detection problem (a machine learning regression problem). The proposed method: DivOVR-FuzzyCSP-WS based features clearly outperformed fuzzy CSP based baseline features in terms of both RMSE and CC performance metrics. It is hoped that the proposed feature extraction method based on DivOVR-FuzzyCSP-WS will bring in a lot of interest in researchers working in developing algorithms for signal processing, in general, for BCI regression problems.

The ubiquitous presence of non-stationarities in the EEG signals significantly perturb the feature distribution thus deteriorating the performance of Brain Computer Interface. In this work, a novel method is proposed based on Joint Approximate Diagonalization (JAD) to optimize stationarity for multiclass motor imagery Brain Computer Interface (BCI) in an information theoretic framework. Specifically, in the proposed method, we estimate the subspace which optimizes the discriminability between the classes and simultaneously preserve stationarity within the motor imagery classes. We determine the subspace for the proposed approach through optimization using gradient descent on an orthogonal manifold. The performance of the proposed stationarity enforcing algorithm is compared to that of baseline One-Versus-Rest (OVR)-CSP and JAD on publicly available BCI competition IV dataset IIa. Results show that an improvement in average classification accuracies across the subjects over the baseline algorithms and thus essence of alleviating within session non-stationarities.

In a DC microgrid, energy management among distributed renewable energy sources, energy storage and load have been a major challenge. For the proper management of energy, a control scheme for bidirectional Quasi Z-Source DC-DC Converter (BQZSDC) has been presented in this paper. BQZSDC is buck-boost type of bidirectional DC-DC converter. It helps to integrate energy storage systems with energy sources and load. This paper presents the performance analysis of BQZSDC with dual-loop proportional integral control for regulating battery current and the load voltage. The controller parameters are tuned using stability boundary locus method based on desired phase margin. The performance of the controller has been validated in both discharging (Boost) and charging (Buck) mode. The MATLAB simulation results shows that the load voltage and battery current is satisfactorily regulated under load current variation in both discharging and charging modes.

Standalone photovoltaic distributed generation (PVDG) systems have found their way into many popular off-grid applications like deserts, military and rural development. Generation of power at the consumption site and supplying power in DC make these systems more efficient due to minimization of losses during transmission and conversion. But, these systems suffer from lower inertia and this factor added with uncertainties in power generation and load consumption, greatly affects system stability. Nonlinear control techniques like backstepping although effective, are expensive to implement since they are model-based and demand information from many sophisticated sensors. In this paper, a disturbance observer based on back-stepping control strategy is proposed for grid voltage control and MPPT of an isolated PVDG system with storage consisting of PV array, battery and load. The effects of irradiation and temperature on PV arrays, the variations in loads and battery voltage are modeled in the form of disturbances. Instead of measuring these entities with sensors, the update laws designed in this paper based on Lyapunov stability theory estimate their values which are further utilized for effective control during intermittencies. It can be seen from the MATLAB simulation results that adoption of this technique contributes towards faster and cheaper control of the PVDG system for a greater range of operating conditions.

Stability and optimality are the two foremost re-quirements for robotic systems that are deployed in critical operations and are to work for long hours or under limited energy resources. To address these, in this work we present a novel Lyapunov stability based discrete-time optimal kinematic control of a robot manipulator using actor-critic (AC) framework. The robot is actuated using optimal joint-space velocity control input to track a time-varying end-effector trajectory in its task space. In comparison to the existing near-optimal kinematic control solutions for robot manipulator under AC framework, proposed controller exhibits guaranteed analytical stability. We derive a novel critic weight update law based on Lyapunov stability, thus ensuring that the weights are updated along the negative gradient of Lyapunov function. This eventually ensures closed-loop system stability and convergence to the optimal control in discrete-time. Extensive simulations are performed on a 3D model of 6-DoF Universal Robot (UR) 10 in Gazebo, followed by implementation on real UR 10 robot manipulator to show the efficacy of the proposed scheme.

Due to the approximation errors in dynamic model, changing payloads and dynamic disturbances acting on the system, the model based tracking is not satisfactory. Hence the application of real-time machine learning techniques in inverse dynamics learning has gained prominence for collaborative human robot interaction. In this paper, we propose a novel combined online and offline Neural Network based learning technique in conjunction with an acceleration tracker for inverse dynamics learning of a robot manipulator. This eliminates the need for explicit reliance on the approximate analytical robot model while controlling the robotic systems. The proposed approach can even capture the system dynamics accurately at higher acceleration where non-linear forces such as non-linear friction and damping play a prominent role. The performance of the proposed inverse dynamic model has been verified using extensive simulations on control of a 6 DOF UR5 robot manipulator in an accurate physics based Pybullet Simulator. The efficacy of the proposed method has been validated by comparing the control performance using model based backstepping controller. The results show that the inverse dynamics learning based controller outperforms significantly its counterpart in real world scenarios where uncertainties are ubiquitous.

This paper presents an approach for position and attitude control of a quadrotor using adaptive backstepping technique along with an uncertainty observer via Recurrent Neural Network (RNN). The quadrotor dynamics are expressed as two subsystems, namely translational and rotational, on which the backstepping control law has been developed. In comparison with feedforward neural networks, RNN has better dynamic characteristics and approximation capabilities. Therefore, an RNN based uncertainty observer has been employed to accommodate the system uncertainties as well as the unknown external disturbances. The proposed controller consists of two parts - an adaptive backstepping based controller that contains an RNN observer and a robust controller to deal with the approximation error induced by the RNN. The RNN parameters have been updated via an update law based on Lyapunov stability theory in an online manner where the overall system stability is also guaranteed. The proposed approach has been implemented in simulations for trajectory tracking of the quadrotor in the presence of parametric uncertainties and external disturbances. Also, the hardware results are presented to show the effectiveness of the proposed approach on DJI Matrice 100.

In this work, we present a pragmatic approach to enable unmanned aerial vehicle (UAVs) to autonomously perform highly complicated tasks of object pick and place. This paper is largely inspired by challenge-2 of MBZIRC 2020 and is primarily focused on the task of assembling large 3D structures in outdoors and GPS-denied environments. Primary contributions of this system are: (i) a novel computationally efficient deep learning based unified multi-task visual perception system for target localization, part segmentation, and tracking, (ii) a novel deep learning based grasp state estimation, (iii) a retracting electromagnetic gripper design, (iv) a remote computing approach which exploits state-of-the-art MIMO based high speed (5000Mb/s) wireless links to allow the UAVs to execute compute intensive tasks on remote high end compute servers, and (v) system integration in which several system components are weaved together in order to develop an optimized software stack. We use DJI Matrice-600 Pro, a hexrotor UAV and interface it with the custom designed gripper. Our framework is deployed on the specified UAV in order to report the performance analysis of the individual modules. Apart from the manipulation system, we also highlight several hidden challenges associated with the UAVs in this context.

Convolutional Neural Networks (CNNs) have been successfully applied for relative camera pose estimation from labeled image-pair data, without requiring any handengineered features, camera intrinsic parameters or depth information. The trained CNN can be utilized for performing pose based visual servo control (PBVS). One of the ways to improve the quality of visual servo output is to improve the accuracy of the CNN for estimating the relative pose estimation. With a given state-of-the-art CNN for relative pose regression, how can we achieve an improved performance for visual servo control? In this paper, we explore switching of CNNs to improve the precision of visual servo control. The idea of switching a CNN is due to the fact that the dataset for training a relative camera pose regressor for visual servo control must contain variations in relative pose ranging from a very small scale to eventually a larger scale. We found that, training two different instances of the CNN, one for large-scale-displacements (LSD) and another for small-scale-displacements (SSD) and switching them during the visual servo execution yields better results than training a single CNN with the combined LSD+SSD data. However, it causes extra storage overhead and switching decision is taken by a manually set threshold which may not be optimal for all the scenes. To eliminate these drawbacks, we propose an efficient switching strategy based on model agnostic meta learning (MAML) algorithm. In this, a single model is trained to learn parameters which are simultaneously good for multiple tasks, namely a binary classification for switching decision, a 6DOF pose regression for LSD data and also a 6DOF pose regression for SSD data. The proposed approach performs far better than the naive approach, while storage and run-time overheads are almost negligible.

This article proposes a real-time event-triggered near-optimal controller for the nonlinear discrete-time interconnected system. The interconnected system has a number of subsystems/agents, which pose a nonzero-sum game scenario. The control inputs/policies based on proposed event-based controller methodology attain a Nash equilibrium fulfilling the desired goal of the system. The near-optimal control policies are generated online only at events using actor-critic neural network architecture whose weights are updated too at the same instants. The approach ensures stability as the event-triggering condition for agents is derived using Lyapunov stability analysis. The lower bound on interevent time, boundedness of closed-loop parameters, and optimality of the proposed controller are also guaranteed. The efficacy of the proposed approach has been validated on a practical heating, ventilation, and air-conditioning system for achieving the desired temperature set in four zones of a building. The control update instants are minimized to as low as 27% for the desired temperature set.

In this paper, we present a semi supervised deep quick learning framework for instance detection and pixelwise semantic segmentation of images in a dense clutter of items. The framework can quickly and incrementally learn novel items in an online manner by real-time data acquisition and generating corresponding ground truths on its own. To learn various combinations of items, it can synthesize cluttered scenes, in real time. The overall approach is based on the tutor-child analogy in which a deep network (tutor) is pretrained for class-agnostic object detection which generates labeled data for another deep network (child). The child utilizes a customized convolutional neural network head for the purpose of quick learning. There are broadly four key components of the proposed framework: semi supervised labeling, occlusion aware clutter synthesis, a customized convolutional neural network head, and instance detection. The initial version of this framework was implemented during our participation in Amazon Robotics Challenge (ARC), 2017. Our system was ranked 3rd rd, 4th and 5 th worldwide in pick, stow-pick and stow task respectively. The proposed framework is an improved version over ARC'17 where novel features such as instance detection and online learning has been added.

This paper is concerned with the optimal kinematic control of a robot manipulator where the robot end effector position follows a task space trajectory. The joints are actuated with the desired velocity profile to achieve this task. This problem has been solved using a single network adaptive critic (SNAC) by expressing the forward kinematics as input affine system. Usually in SNAC, the critic weights are updated using back propagation algorithm while little attention is given to convergence to the optimal cost. In this paper, we propose a critic weight update law that ensures convergence to the desired optimal cost while guaranteeing the stability of the closed loop kinematic control. In kinematic control, the robot is required to reach a specific target position. This has been solved as an optimal regulation problem in the context of SNAC based kinematic control. When the robot is required to follow a time varying task space trajectory, then the kinematic control has been framed as an optimal tracking problem. For tracking, an augmented system consisting of tracking error and reference trajectory is constructed and the optimal control policy is derived using SNAC framework. The stability and performance of the system under the proposed novel weight tuning law is guaranteed using Lyapunov approach. The proposed kinematic control scheme has been validated in simulations and experimentally executed using a real six degrees of freedom (DOF) Universal Robot (UR) 10 manipulator.

Dynamic Movement Primitive (DMP) which is popular for motion planning of a robot manipulator, has been adapted for a nonholonomic mobile robot to track the desired trajectory. DMP is a simple damped spring model with a forcing function, which learns the trajectory. The damped spring model attracts the robot towards the goal position, and the forcing function forces the robot to follow the given trajectory. Two Radial Basis Function Networks (RBFNs) have been used to learn the forcing function associated with the DMP model. Weight update laws are derived using the gradient descent approach to train the RBFNs. Fuzzy logic based steering angle dynamics is proposed to handle the asymmetric nature of an obstacle. The proposed scheme is capable enough to generate a smooth trajectory in the presence of an obstacle even when start and goal positions are altered, without losing the spatial information embedded while training. The convergence of the robot goal position has been shown using Lyapunov stability theory-based analysis. The approach has been extended to multiple static and dynamic obstacles for the successful convergence of the robot at the goal position. Both simulation and experimental results are provided to confirm the efficacy of the proposed scheme.

This paper proposes a fast terminal sliding mode super twisting controller (FTSMSTC) design for quadrotor position and altitude tracking in the presence of bounded disturbances. A nonlinear fast terminal sliding manifold has been proposed for fast convergence of the tracking error to zero in finite time unlike the conventional sliding mode control (CSMC) that guarantee only asymptotic convergence of the tracking error. The super twisting reaching law has been proposed to deal with the chattering phenomena, which is inherent in the CSMC. The finite time stability of the complete closed loop system is investigated using Lyapunov stability theory and an analytical expression for the convergence time has also been derived. The effectiveness of the designed controller is checked against the CSMC using MATLAB simulation. The controller has been experimentally validated using the DJI Matrice M100 as a proof of utility in real time applications.

One of the main challenges in the vision-based grasping is the selection of feasible grasp regions while interacting with novel objects. Recent approaches exploit the power of convolutional neural network (CNN) to achieve accurate grasping at the cost of high computational power and time. In this paper, we present a novel unsupervised learning based algorithm for the selection of feasible grasp regions. Unsupervised learning infers the pattern in dataset without any external labels. We applied k-means clustering at every sampling stage in image plane to identify the grasp regions, followed by axes assignment method. We define a novel concept of Grasp Decide Index (GDI) to select the best grasp pose in the image plane. We have conducted several experiments in clutter or isolated environment on standard objects of Amazon Robotics Challenge 2017 and Amazon Picking Challenge 2016. We compared the results with prior learning based approaches to validate the robustness and adaptive nature of our algorithm for a variety of novel objects in different domains.

This paper proposes a vision-based sliding mode control technique for autonomous landing of a quadrotor over the static platform. The proposed vision algorithm estimates the quadrotor's position relative to an ArUco marker placed on a static platform using an on-board monocular camera. The relative position is provided as an input to a Fast-terminal Sliding Mode Super Twisting Controller (FTSMSTC) which ensures finite time convergence of the relative position between the landing pad marker and the quadrotor. In addition, the proposed controller attenuates chattering phenomena and guarantees robustness towards bounded external disturbances and modelling uncertainties. The proposed vision-based control scheme is implemented using numerical simulations and validated in real-time on the DJI Matrice 100.

This paper proposes a vision-based sliding mode control technique for autonomous tracking of a moving vehicle by a quadrotor. The proposed vision algorithm estimates the quadrotor's position relative to moving vehicle using an on-board monocular camera. The relative position is provided as an input to a Fractional Order Sliding mode Controller (FOSMC) which ensures the convergence of the relative position between the moving vehicle and the quadrotor thereby enabling it to track the vehicle effectively. In addition, the proposed controller guarantees robustness towards bounded external disturbances and modelling uncertainties. The proposed vision-based control scheme is implemented using numerical simulations and validated in real-time on the DJI Matrice 100. Theses validations help in gaining into the maximum allowable speed of the moving target for the quadrotor to successfully track the object. This plays a vital role in surveillance operations and intruder chase.

Complex and interactive robot manipulation skills such as playing a game of table tennis against a human opponent is a multifaceted challenge and a novel problem. Accurate dynamic trajectory generation in such dynamic situations and an appropriate controller in order to respond to the incoming table tennis ball from the opponent is only a prerequisite to win the game. Decision making is a major part of an intelligent robot and a policy is needed to choose and execute the action which receives highest reward. In this paper, we address this very important problem on how to learn the higher level optimal strategies that enable competitive behaviour with humans in such an interactive game setting. This paper presents a novel technique to learn a higher level strategy for the game of table tennis using P-Q Learning (a mixture of Pavlovian learning and Q-learning) to learn a parameterized policy. The cooperative learning framework of Kohenon Self Organizing Map (KSOM) along with Replay Memory is employed for faster strategy learning in this short horizon problem. The strategy is learnt in simulation, using a simulated human opponent and an ideal robot that can perform hitting motion in its workspace accurately. We show that our method is able to improve the average received reward significantly in comparison to the other state-of-the-art methods.

Grasping of novel objects in pick and place applications is a fundamental and challenging problem in robotics, specifically for complex-shaped objects. It is observed that the well-known strategies like i) grasping from the centroid of object and ii) grasping along the major axis of the object often fails for complex-shaped objects. In this paper, a realtime grasp pose estimation strategy for novel objects in robotic pick and place applications is proposed. The proposed technique estimates the object contour in the point cloud and predicts the grasp pose along with the object skeleton in the image plane. The technique is tested for the objects like ball container, hand weight, tennis ball and even for complex shape objects like blower (non-convex shape). It is observed that the proposed strategy performs very well for complex shaped objects and predicts the valid grasp configurations in comparison with the above strategies. The experimental validation of the proposed grasping technique is tested in two scenarios, when the objects are placed distinctly and when the objects are placed in dense clutter. A grasp accuracy of 88.16% and 77.03% respectively are reported. All the experiments are performed with a real UR10 robot manipulator along with WSG-50 two-finger gripper for grasping of objects.

This paper demonstrates an hybrid methodology of quadrotor navigation and control in an environment with obstacles by combining a Q-learning strategy for navigation with a non-linear sliding mode control scheme for position and altitude control of the quadrotor. In an unknown environment, an optimal safe path is estimated using the Q-learning scheme by considering the environment as a 3D grid world. Furthermore, a non-singular terminal sliding mode control (NTSMC) is employed to navigate the quadrotor through the planned trajectories. The NTSMC that is employed for trajectory tracking ensures robustness towards bounded disturbances as well as parametric uncertainties. In addition, it ensures finite time convergence of the tracking error and avoids issues that arise due to singularities in the dynamics. The effectiveness of the proposed navigation and control scheme are validated using numerical simulations wherein a quadrotor is required to pass through a window.

Electroencephalogram (EEG) signals are one of the most widely used noninvasive signals in brain-computer interfaces. Large dimensional EEG recordings suffer from poor signal-tonoise ratio. These signals are very much prone to artifacts and noise, so sufficient preprocessing is done on raw EEG signals before using them for classification or regression. Properly selected spatial filters enhance the signal quality and subsequently improve the rate and accuracy of classifiers, but their applicability to solve regression problems is quite an unexplored objective. This paper extends common spatial patterns (CSP) to EEG state space using fuzzy time delay and thereby proposes a novel approach for spatial filtering. The approach also employs a novel fuzzy information theoretic framework for filter selection. Experimental performance on EEG-based reaction time (RT) prediction from a lane-keeping task data from 12 subjects demonstrated that the proposed spatial filters can significantly increase the EEG signal quality. A comparison based on root-mean-squared error (RMSE), mean absolute percentage error (MAPE), and correlation to true responses is made for all the subjects. In comparison to the baseline fuzzy CSP regression one versus rest, the proposed Fuzzy Time-delay Common Spatio-Spectral filters reduced the RMSE on an average by 9.94%, increased the correlation to true RT on an average by 7.38%, and reduced the MAPE by 7.09%.

This paper proposes a vision-based guidance strategy for safe navigation of a nonholonomic mobile robot in unknown indoor environments. The proposed switching-based sliding mode control (SMC) law makes the robot follow the desired trajectory as given by the guidance law. The guidance strategy uses the centroid of the depth map of an obstacle as obtained from the Red Green Blue -Depth (RGB-D) sensor to generate the desired angular velocity. The fuzzy rule-based guidance is developed to generate the desired linear velocity command. The analysis of guidance strategy is done for an infinite length obstacle. The proposed SMC is shown to be asymptotically stable using Krasovskii Method. The finite time convergence of robot navigation has been shown using Poincare Map method. The stability of the proposed SMC under burst losses has also been established. Experiments on the Pioneer P3-DX robot in different obstacle scenarios show that the robot safely navigates in presence of communication channel burst losses.

Transferring skills to roUots through human demonstrations is an interesting problem. Locally generated demonstrations of reaching motion, given by a human teacher are generally encoded in a dynamical model. Stability of this encoding system demands great attention while learning the model parameters. In that context, we present a new architecture of dynamical system to learn movement primitives from multiple demonstrations exploiting a fuzzy Lyapunov function (FLF). We assume that there exists a natural Lyapunov function (LF) that associates the demonstrations. The proposed FLF tries to approximate that LF. First, the dynamics of the demonstrations are encoded in a regressive model, learnt using Gaussian mixture regression with EM algorithm. Then the FLF is searched involving the learnt dynamics in an optimization process. The FLF in turn helps to learn a fuzzy controller. Our architecture is new in a sense that it combines the probabilistic model with a fuzzy controller to create a globally asymptotically stable motion model. The proposed algorithm can simultaneously learn position and orientation profiles in a single model. The algorithm is experimentally validated on a commercially available manipulator and also compared with a state-of-the-art technique.

The paper presents a deep learning based fully automatic object annotation technique for warehouse application usecase. One of the main challenges that is addressed in this paper is the large amount of manual labour involved in generating datasets for training a deep network. The proposed annotation model is developed by fine-tuning a deep network based object detection framework with ImageNet pre-trained models. We have used Faster RCNN network with pre-trained model VGG-16 and RFCN with ResNet-101. A small set of manually annotated images of single objects are used to automatically generate a dataset of significantly large size within a very short time duration (in real-time). The model also has the competence of precisely localizing the region of any new object that comes into the familiar background. Incorporation of techniques like color augmentation and affine transformation enables the network invariant to rotation, scale and brightness. Augmentation also enables the model to performs well even if the background is different. A clutter generation technique is introduced in the framework which makes the system capable of annotating objects even in a densely populated real-world environment. This work has another significant contribution in detection of objects those are used in Amazon Robotic Challenge (ARC) 2017 where our team was among the four finalist in both picking and stowing task. The automatically generated big dataset is further used to train multi-class detectors using Faster RCNN and RFCN networks to validate the performance of the proposed annotation model. The efficacy of the proposed model is hence demonstrated through various experimental results. The dataset is shared online for the convenience of the reader.

Brain-Computer Interfaces (BCI) are systems that enable users to use neural signals, typically Electroencephalogram (EEG) to direct an application or an external device. Motor imagery (MI) based BCI detects subject motor intentions which could be further used as control signals. Due to spatial lateralization of different MI tasks, spatial filtering followed by band power extraction is the most commonly used algorithm for classification tasks in MI-based BCIs. Unfortunately, the spatial filtering approach significantly incorporates Amplitude characteristics when compared to Phase characteristics of the EEG signal. Single trial Phase locking value (sPLV) has been a popular statistics to extract phase based information for classification task in MI-based BCI. To utilize the phase characteristics for MI classification, this paper proposes a novel approach based on instantaneous phase difference (IPD) sequence to extract phase features that explicitly use the phase synchronization information between EEG sensors. We maximized the discriminability of IPD sequence using linear transformation calculated from common spatial pattern algorithm (CSP) on the IPD sequence. An evaluation of our method on BCI competition dataset led to around 15% increase in mean classification accuracies compared to sPLV approach and comparable accuracies to power feature based CSP algorithm. Furthermore, incorporating phase features from our method and power features from traditional algorithms using sparse feature selection technique increased the classification accuracy over both CSP and CSP on the IPD sequence.

This paper presents a novel automatic facial expressions recognition system (AFERS) using the deep network framework. The proposed AFERS consists of four steps: 1) geometric features extraction; 2) regional local binary pattern (LBP) features extraction; 3) fusion of both the features using autoencoders; and 4) classification using Kohonen self-organizing map (SOM)-based classifier. This paper makes three distinct contributions. The proposed deep network consisting of autoencoders and the SOM-based classifier is computationally more efficient and performance wise more accurate. The fusion of geometric features with LBP features using autoencoders provides better representation of facial expression. The SOM-based classifier proposed in this paper has been improved by making use of a soft-threshold logic and a better learning algorithm. The performance of the proposed approach is validated on two widely used databases (DBs): 1) MMI and 2) extended Cohn-Kanade (CK+). An average recognition accuracy of 97.55% in MMI DB and 98.95% in CK+ DB are obtained using the proposed algorithm. The recognition results obtained from fused features are found to be distinctly superior to both recognition using individual features as well as recognition with a direct concatenation of the individual feature vectors. Simulation results validate that the proposed AFERS is more efficient as compared to the existing approaches.

The heating, ventilation, and air conditioning system is an important component for achieving desired thermal condition in rooms or spaces in buildings, office complex, or airports. This paper proposes a real-time event-triggered adaptive critic controller for generating near optimal control actions to achieve desired temperatures. The desired temperatures may have variable or fixed values over time. The real-time controller is designed in two phases. Initially event-triggered control actions are generated by linear quadratic regulator for small period while the actor-critic network of controller is trained. Later, adaptive critic controller takes over for event-based actions. Hence, the event triggering conditions for both general linear and nonlinear discrete time systems using Lyapunov stability analysis are derived in this paper. The event-based actor-critic network weight update formulation and ultimate boundedness of parameters are also presented in this paper. The proposed approach has been validated for different and common temperature sets for four zones, where the control execution events are minimized to 20% and 26%, respectively.

A control theoretic approach is presented in this paper for both batch and instantaneous updates of weights in feed-forward neural networks. The popular Hamilton-Jacobi-Bellman (HJB) equation has been used to generate an optimal weight update law. The main contribution in this paper is that a closed form solutions for both optimal cost and weight update can be achieved for any feed-forward network using HJB equation. The proposed approach has been compared with some of the existing best performing learning algorithms. It is found as expected that the proposed approach is faster in convergence in terms of computational time. Some of the benchmark test data such as 8-bit parity, breast cancer and credit approval, as well as 2D Gabor function have been used to validate our claims.

The ability to follow a human is an important requirement for a service robot designed to work along side humans in homes or in work places. This paper describes the development and implementation of a novel robust visual controller for the human-following robot. This visual controller consists of two parts: 1) a robust algorithm that tracks a human visible in its camera view and 2) a servo controller that generates necessary motion commands so that the robot can follow the target human. The tracking algorithm uses point-based features, like speeded up robust feature, to detect human under challenging conditions, such as, variation in illumination, pose change, full or partial occlusion, and abrupt camera motion. The novel contributions in the tracking algorithm include the following: 1) a dynamic object model that evolves over time to deal with short-term changes, while maintaining stability over long run; 2) an online K-D tree-based classifier along with a Kalman filter is used to differentiate a case of pose change from a case of partial or full occlusion; and 3) a method is proposed to detect pose change due to out-of-plane rotations, which is a difficult problem that leads to frequent tracking failures in a human following robot. An improved version of a visual servo controller is proposed that uses feedback linearization to overcome the chattering phenomenon present in sliding mode-based controllers used previously. The efficacy of the proposed approach is demonstrated through various simulations and real-life experiments with an actual mobile robot platform.

Heating, ventilation and air-conditioning(HVAC) system is a very important component in designing a Smart Home. The HVAC system itself is a Cyber-Physical system(CPS) and comes under Industry 4.0. Being connected to network it requires an integrated architecture of communication and control for its smooth operation. Heterogeneous nature of control and cyber domains is a great challenge in dealing with CPS development. An intelligent controller design coupled in a communication framework is presented in this paper for performance improvements in HVAC system. The control and communication architecture considers relevant system objectives based on system states and actuator actions. The HVAC system regulates the flow of conditioned air for various desired temperatures in different thermal zones. The formulated problem has been solved through real time optimization approach using learning based Proportional-Integral(PI) controller methodology following the communication protocol. The gradient descent algorithm updates the parameters of PI controller which in turn generates online control actions for achieving desired states. The algorithm helps in obtaining optimal control actions for the actuators and shows a fast convergence to the different desired temperature sets.

This paper is concerned about flocking control of a group of mobile agents having leader-follower configuration where the number of leaders is lesser in number than followers. The leaders have the navigation information where the followers only know who the leaders are. Artificial potential function is used to design the distributed control law of the agents. The leaders move as per the tracking information while avoiding collision with other agents. The followers estimate the position of the flocking center and tend to move towards it while aligning velocities and avoiding collisions with neighboring agents. A novel algorithm for estimation of flocking center is introduced in this paper. This algorithm uses consensus concept that makes the followers reach an agreement regarding the flocking center position. The stability analysis of this algorithm shows that the estimation error is asymptotically ε-stable. The agents do not need to know their own positions in global coordinates. The leaders do not access any information from other agents while the followers communicate with its neighboring agents to estimate the flocking center and also to control their motion, the neighborhood being defined by the limited communication range. Simulation results show the effectiveness of the algorithm.

In this work, we approach the problem of maximization of delay constraint utility function for a group of robots using a Cyber-Physical System (CPS) framework. For any task involving coordination of robots, a reliable communication link is required to be established. The maintenance of this link reliability highly depends on trajectories of individual robots. This needs a hybrid approach to be adopted, in which mobility and routing control are taken care of simultaneously. In this paper, a hybrid approach has been proposed based on bidirectional optimization i.e. network utility maximization and energy minimization in the presence of delay constraint. Network Utility function allocates network resources to the robots based on physical constraints involved. The result of this optimization problem would empower the hybrid controller to control both robot position as well as communication link strength.

The Disturbance observer is becoming very popular and is being mainly used in high speed motion control applications where high precision is required. This paper focus on the problem of designing a nonlinear disturbance observer for good estimation of external disturbances acting on the body of quadcopter during flying for attitude, altitude as well as position regulation control problem. The proposed controller is based on composite controller scheme, consists of a nonlinear disturbance observer and a Backstepping controller.The stability analysis of the nonlinear disturbance observer is successfully done using Lyapunov stability theory. The effectiveness of the proposed disturbance observer is investigated by MATLAB simulation. The simulation results shows that Backstepping controller with nonlinear disturbance observer has good tracking ability in the presence of external disturbance.

Complex robotics task such as biped walking, tennis-like swing, object grasping etc, depend on state prediction, complex motion generation and stable execution of motion command. Predictions of states get more accurate over time, hence the robot behavior need to be updated continuously. Such state updates cannot be incorporated straight forwardly in most trajectory generation solutions. dynamic movement primitives (DMP) provides such a flexible formulation which can adapt to spacial and temporal variations. In this paper, we present a novel algorithm for adaptive learning of DMP that can be used to generate complex trajectories required by the robot to perform a complex task. The proposed technique uses a piece-wise linear canonical system (PLCS) instead of the standard exponential canonical system (ECS) for learning the DMP parameters. We show that the proposed PLCS learns the parameters faster and has a smaller mean squared error (MSE) as compared to ECS. Proposed learning technique coupled with PLCS uniquely learns DMP parameters as compared to other state of art techniques. The proposed technique is used to learn the primitive trajectories via imitation (learning from demonstration) and an unseen primitive is generated by a kernel based mixture model. In this paper, we present results from a real 4 degree-of-freedom (DOF) Barrett WAM robotic arm and show that the proposed system is able to hit a ball randomly thrown at it.

This paper is concerned with the problem of tracking a random target by a circular pattern of mobile agents using leader-follower approach. There is a single leader in the group which is located at the centre of the circle. Other agents, i.e., the followers control their motions to locate themselves evenly on the arc of the circle. The problem considered here is two fold - formation of a circular pattern with a static leader and tracking of a randomly moving target by this pattern. A novel distributed control law has been designed based on artificial potential function approach to achieve the formation and maintenance of the circle with the leader being static. The follower agents are allowed to communicate with the leader and only one neighboring follower. For the tracking problem, the leader estimates state of the target using Kalman filter. The leader is then made to track the target as the estimated target position works as the tracking destination. The control law of the followers is modified by adding a navigation term to make them move along with the leader while maintaining the circular pattern. Both the algorithms - circular formation and tracking - have been shown to be Lyapunov stable. Effectiveness of the proposed scheme is demonstrated through extensive simulation.

In the past decade, improvements in the production of in-expensive PC equipment and software has permitted more refined real-time signal processing in BCI systems. In the literature, Deep learning concepts have not been applied to EEG data analysis in a systematic manner. This paper applies various existing Deep learning architectures and algorithms for the classification of EEG data applied to eye state detection. The deep learning based classifier systems presented in this work are comparable to the state of the art classifiers devised by Roesler and Suenderman (2013), and Cameron et al. (2015). The goal of this work is to construct a system producing accuracies comparable to Roesler's K* classifier, Cameron et al.'s (RRF+K*) classifiers and at the same time providing enough speed to be used in an online BCI framework. In order to meet the constraints, following architectures were designed: A Multi layered neural network with ReLU and drop-out, deep belief networks based unsupervised learning, drop-out masks on deep neural networks. Specifically, we compare our results with K*, RRF, (K*+RRF), ada(RJ48F) classifiers. Also an in-depth analysis of binary class features has been done using t-SNE based visualizations while fitting elliptical contours to the features. Prior research suggests that instance-based/lazy learners like the K* algorithm are likely to be too slow to be used in a BCI framework, with ada(RJ48F) model performing decently well. But our chosen deep neural network architectures produce higher classification accuracies and have lower convergence times making them even faster within the time specifications of real-time classification and control applications.

This paper presents a single-network adaptive critic-based controller for continuous-time systems with unknown dynamics in a policy iteration (PI) framework. It is assumed that the unknown dynamics can be estimated using the Takagi-Sugeno-Kang fuzzy model with arbitrary precision. The successful implementation of a PI scheme depends on the effective learning of critic network parameters. Network parameters must stabilize the system in each iteration in addition to approximating the critic and the cost. It is found that the critic updates according to the Hamilton-Jacobi-Bellman formulation sometimes lead to the instability of the closed-loop systems. In the proposed work, a novel critic network parameter update scheme is adopted, which not only approximates the critic at current iteration but also provides feasible solutions that keep the policy stable in the next step of training by combining a Lyapunov-based linear matrix inequalities approach with PI. The critic modeling technique presented here is the first of its kind to address this issue. Though multiple literature exists discussing the convergence of PI, however, to the best of our knowledge, there exists no literature, which focuses on the effect of critic network parameters on the convergence. Computational complexity in the proposed algorithm is reduced to the order of (Fz)(n-1) , where n is the fuzzy state dimensionality and Fz is the number of fuzzy zones in the states space. A genetic algorithm toolbox of MATLAB is used for searching stable parameters while minimizing the training error. The proposed algorithm also provides a way to solve for the initial stable control policy in the PI scheme. The algorithm is validated through real-time experiment on a commercial robotic manipulator. Results show that the algorithm successfully finds stable critic network parameters in real time for a highly nonlinear system.

Kinematic motor behaviour of robots can be encoded using dynamical systems. These dynamical systems are learnt from human demonstrations to generalize human like motions. The size of the demonstration space involving a robot usually being large, it is not possible to provide all the demonstrations for robot's learning. Hence, it requires an efficient learning architecture that is able to generalize for unseen contexts. In the proposed algorithm, we model the movements, shown in the human demonstrations, as non-linear multivariate dynamics using mixture of Gaussians. Generally, in non-linear multivariate modelling approach pertaining to programming by demonstration, requires structured demonstrations i.e. it always requires to have a fixed and unique equilibrium point during the learning phase. The proposed method can relax these constraints and has the following advantage over the existing work: first, it would be possible to learn from any demonstration which is not constrained to have always the same equilibrium point; second, it would be possible to capture the variations in movement patterns depending upon the position of the equilibrium point. The proposed algorithm has been implemented using Barrett WAM and experimental results have been compared with existing approach.

Limited availability of resources increases the importance of decentralized control through optimal networking of sensors and controllers in practical MIMO systems. Design of this network entails rigorous consideration of constraints in sensors, communication relays as well as controllers. Given that each of these elements has different capacity, a framework for optimal allocation of communication resources for enhanced system performance is necessary. In this paper, these two challenges have been incorporated to find out the set of optimal possible routes from sensors to controllers while ensuring stability. The major contribution of this paper is the development of a generalized algorithm to find optimal combination of sensors and controllers to be connected so as to make the system highly stable. The proposed algorithm minimizes a suitable cost or enhances reliability while guaranteeing stability using Lyapunov stability theory and linear matrix inequalities (LMI). The efficacy of the algorithm has been demonstrated through application on a cyber physical smart grid system. The multi-cast sensor — controller routing for decentralized voltage control in a 4-bus smart-grid system operating in islanded mode has been successfully simulated in MATLAB environment.

The concept of using a Magnetic Resonance Imaging (MRI) device for chemotherapy, by employing a micro robot, consisting of a polymer bound aggregate of ferromagnetic particles, is explored in this paper. We primarily contribute towards the design of a Fuzzy Sliding Mode Controller (FSMC) for trajectory tracking of the micro robot in the human vasculature considering a highly non-linear model available in literature. An adaptive algorithm based on Lyapunov stability theory is used to estimate the parameters associated with the FSMC. The proposed FSMC is able to eliminate the chattering phenomenon completely which is present in conventional sliding mode control. Since the system in consideration is a biological one, many parameters are difficult to estimate resulting in parametric uncertainties. A significant merit of the proposed framework is its ability to estimate the dielectric density of blood on-line with great accuracy. Simulation results also indicate perfect tracking with very fast dynamical response. To illustrate the efficacy of our controller, a detailed comparison is made between the performances of a state-of-the-art adaptive backstepping control and our proposed control action in the presence of bounded model uncertainties for micro-robots made up of different ferromagnetic materials.

Source transcription of pitched polyphonic music entails providing the pitch (F0) values corresponding to each source in a separate channel. This problem is an important step towards many important problems in music and speech processing. It involves 1) estimating the multiple F0 values in each short time frame, and 2) clustering the F0 values into streams corresponding to different sources. We address the problem in an unsupervised way, with only the total number of sources given beforehand. The framework of probabilistic latent component analysis (PLCA) is used to decompose the polyphonic short-time magnitude spectra for multiple F0 estimation and source-specific feature extraction. It is further embedded into the structure of hidden Markov random fields (HMRF) for clustering the F0s into different sources. This clustering is constrained by the cognitive grouping of continuous F0 contours as well as segregation of simultaneous F0s into different source streams. Such constraints are effectively and elegantly modeled by the HMRF's. Simulated annealing varies the degree of constraints for better clustering. The paper also proposes a novel strategy using the trade-off between precision and recall of multiple F0 estimation for better clustering. Evaluations over a variety of datasets show the efficacy of the proposed algorithm and its robustness to the presence of spurious F0s while clustering. It also outperforms a state-of-the-art unsupervised source streaming algorithm in a set of comparative experiments.

A near optimal control technique for robotic manipulator with completely unknown dynamics is described in this work. Obtaining the optimal control law u* depends on solving Hamilton Jacobi Bellman equation but getting an analytic solution is not possible for unknown models. It is shown that instead of solving HJB equation analytically, the optimal control law can be obtained through learning of a Single Network Adaptive Critic (SNAC). The generic nonlinear model of manipulator dynamics is represented as Takagi-Sugeno-Kang fuzzy combination of local linear models. A stabilizing fixed gain controller is designed for the TSK fuzzy system using an unconventional Lyapunov function that is used to represent the value function. Stable Lyapunov P(i)matrices are selected using the Genetic Algorithm (GA) Toolbox in Matlab. This approach avoids the learning of initial cost that can be accumulated by an existing controller. The critic is trained to approximate the optimal cost J* by renewing the policy in iterations. Validation of the proposed technique is done through simulation on a robotic manipulator model. Results show the effectiveness of the presented work.

Detecting and tracking a human from a mobile robot platform has several applications in service robotics where a robot is expected to assist humans. In this paper, we propose a novel interest point-based algorithm that can track a human reliably under several challenging situations like variation in illumination, pose change, scaling, camera motion and occlusion. The limitations of point-based methods are overcome using colour information and imposing a structure on the colour blobs. Whenever sufficient number of SURF matching points are not available for a given frame, the presence of human is detected using Markov random field based graph matching algorithm. Imposition of structure on coloured blobs helps in eliminating background objects having similar colour distribution. The stability-versus-plasticity dilemma inherent in tracking over long run is resolved by selecting new templates on-line and maintaining a tree of templates which is updated with new information. The performance of the algorithm is demonstrated through simulation on standard datasets and the computation time is found to be comparable with existing SURF-based tracking methods.

This paper presents an appearance feature based facial expression recognition system using Kohonen Self-Organizing Map (KSOM). Appearance features are extracted using uniform Local binary patterns (LBPs) from equally sub-divided blocks applied over face image. The dimensionality of the LBP feature vector is further reduced using principal component analysis (PCA) to remove the redundant data that leads to unnecessary computation cost. Using our proposed KSOM based classification approach, we train only 59 dimensional LBP features extracted from whole facial region. The classifier is designed to categorize six basic facial expressions (happiness, sadness, disgust, anger, surprise and fear). To validate the performance of the reduced 59 dimensional LBP feature vector, we also train the original data of dimension 944 using the KSOM. The results demonstrates, that with marginal degradation in overall recognition performance, the reduced 59 dimensional data obtains very good classification results. The paper also presents three more comparative studies based on widely used classifiers like; Support vector machine (SVM), Radial basis functions network (RBFN) and Multi-layer perceptron (MLP3). Our KSOM based approach outperforms all other classification methods with average recognition accuracy 69.18%. Whereas, the average recognition rated obtained by SVM, RBFN and MLP3 are 65.78%, 68.09% and 62.73% respectively.

The paper presents a facial expressions recognition system using Bayesian network. We train the network using probabilistic modeling that draws relationship between facial features, action units and finally recognizes six basic emotions. We propose features extraction methods to get geometric feature vector containing angular informations and appearance feature vector containing moments extracted after applying gabor filter over certain facial regions. Both the feature vectors are further used to draw relationships among Action Units (AUs). The angular informations are directly extracted from the facial landmark points. The geometric features extraction approach contains only 22 dimensional angular informations against direct facial landmarks based approach that contains 136 dimensional feature vector. Facial activities are represented by three distinct layers. Bottom level contains landmark measurement data with angular features. Middle level has facial AUs those are coded in facial action coding system (FACS) and the top level, represents emotion node. We also propose a method using k-means clustering to automatically define the states of nodes in anatomical layer that draws relationship among AUs and measurement data. Extended Cohn Kanade Database is being used for our experimental purposes. An average emotion recognition accuracy of 95.7% is achieved using proposed Bayesian network based approach for 22 dimensional angular feature vector. To verify the performance of the proposed approach we apply three different classifiers such as, Support vector machine, Decision tree and Radial basis functions network. The confusion matrices show that the Bayesian network based classification approach outperforms all other applied approaches. The experimental results illustrates the effectiveness of the proposed model.

For music transcription or musical source separation, apart from knowing the multi-F0 contours, it is also important to know which F0 has been played by which instrument. This paper focuses on this aspect, i.e. given the polyphonic audio along with its multiple F0 contours, the proposed system clusters them so as to decide `which instrument played when.' For the task of identifying the instrument or singers in the polyphonic audio, there are many supervised methods available. But many times individual source audio is not available for training. To address this problem, this paper proposes novel schemes using semi-supervised as well as unsupervised approach to source clustering. The proposed theoretical framework is based on auditory perception theory and is implemented using various tools like probabilistic latent component analysis and graph clustering, while taking into account various perceptual cues for characterizing a source. Experiments have been carried out over a wide variety of datasets - ranging from vocal to instrumental as well as from synthetic to real world music. The proposed scheme significantly outperforms a state of the art unsupervised scheme, which does not make use of the given F0 contours. The proposed semi-supervised approach also performs better than another semi-supervised scheme, which makes use of the given F0 information, in terms of computations as well as accuracy.

A novel neural information processing architecture inspired by quantum mechanics and incorporating the well-known Schrodinger wave equation is proposed in this paper. The proposed architecture referred to as recurrent quantum neural network (RQNN) can characterize a nonstationary stochastic signal as time-varying wave packets. A robust unsupervised learning algorithm enables the RQNN to effectively capture the statistical behavior of the input signal and facilitates the estimation of signal embedded in noise with unknown characteristics. The results from a number of benchmark tests show that simple signals such as dc, staircase dc, and sinusoidal signals embedded within high noise can be accurately filtered and particle swarm optimization can be employed to select model parameters. The RQNN filtering procedure is applied in a two-class motor imagery-based brain-computer interface where the objective was to filter electroencephalogram (EEG) signals before feature extraction and classification to increase signal separability. A two-step inner-outer fivefold cross-validation approach is utilized to select the algorithm parameters subject-specifically for nine subjects. It is shown that the subject-specific RQNN EEG filtering significantly improves brain-computer interface performance compared to using only the raw EEG or Savitzky-Golay filtered EEG across multiple sessions.

A major challenge in two-class brain-computer interface (BCI) systems is the low bandwidth of the communication channel, especially while communicating and controlling assistive devices, such as a smart wheelchair or a telepresence mobile robot, which requires multiple motion command options in the form of forward, left, right, backward, and start/stop. To address this, an adaptive user-centric graphical user interface referred to as the intelligent adaptive user interface (iAUI) based on an adaptive shared control mechanism is proposed. The iAUI offers multiple degrees-of-freedom control of a robotic device by providing a continuously updated prioritized list of all the options for selection to the BCI user, thereby improving the information transfer rate. Results have been verified with multiple participants controlling a simulated as well as physical pioneer robot.

This work aims at building a robust controller using Maximum Power Point Tracking (MPPT) strategy for a solar power generation system by implementing Takagi-Sugeno (T-S) Fuzzy model of the power system. A Dc-Dc buck converter is used to control the power output from the Photovoltaic (PV) Array. We propose a method to design a state feedback controller to regulate power output by controlling the duty cycle of the converter while maintaining the system Lyapunov stable. Both Fixed gain and variable gain state feedback controllers are compared for the purpose of stability. In addition, a tracking controller is designed which searches for Maximum Power Point (MPP) to optimize systems performance without actually calculating the MPP or measuring the solar radiation. The controller is also robust to disturbances in atmospheric conditions. The proposed system is found to be extremely efficient even in rapidly changing weather conditions. The system is found to reach optimal operation point within few milliseconds. The stability analysis is shown wherever appropriate. All the results are shown in the form of simulations.

The present paper considers the problem of relative motion control for spacecraft formation flying. Using Fuzzy sliding mode control (FSMC) technique, a relative position/velocity tracking control based on the non-linear model is developed. The reference trajectory is generated by the force-free linearized equations of the relative motion (known as Hill's equations) [1]. The fuzzy parameters associated with the FSMC are estimated using an adaptive algorithm derived using Lyapunov stability theory. The adaptive FSMC thus proposed is intended to compensate for the modeling uncertainties existing in practical applications. It is shown that the proposed FSMC is able to eliminate the chattering phenomenon completely as observed in conventional sliding mode control. The simulation results confirms the stability and robustness of the present scheme. A nonlinear model with additive external disturbances and bounded uncertainties is chosen for simulation studies.

The interest point based tracking methods suffer from the limitation of unavailability of sufficient number of matching key points for the target in all frames of a running video. In this paper, a dynamic model is proposed for describing the object model which is used for tracking a human in a non-stationary video. This dynamic model takes into account the change in the pose as well as the motion of the human. A simple autoregression based predictor is used for dealing with the case of full occlusion. Simulation results are provided to show the efficacy of the algorithm.

Extraction of predominant melody from the musical performances containing various instruments is one of the most challenging task in the field of music information retrieval and computational musicology. This paper presents a novel framework which estimates predominant vocal melody in real-time by tracking various sources with the help of harmonic clusters (combs) and then determining the predominant vocal source by using the harmonic strength of the source. The novel on-line harmonic comb tracking approach complies with both structural as well as temporal constraints simultaneously. It relies upon the strong higher harmonics for robustness against distortion of the first harmonic due to low frequency accompaniments, in contrast to the existing methods which track the pitch values. The predominant vocal source identification depends upon the novel idea of source dependant filtering of recognition score, which allows the algorithm to be implemented on-line. The proposed method, although on-line, is shown to significantly outperform our implementation of a state-of-the-art offline method for vocal melody extraction. Evaluations also show the reduction in octave error and the effectiveness of novel score filtering technique in enhancing the performance.

Existing models for document summarization mostly use the similarity between sentences in the document to extract the most salient sentences. The documents as well as the sentences are indexed using traditional term indexing measures, which do not take the context into consideration. Therefore, the sentence similarity values remain independent of the context. In this paper, we propose a context sensitive document indexing model based on the Bernoulli model of randomness. The Bernoulli model of randomness has been used to find the probability of the cooccurrences of two terms in a large corpus. A new approach using the lexical association between terms to give a context sensitive weight to the document terms has been proposed. The resulting indexing weights are used to compute the sentence similarity matrix. The proposed sentence similarity measure has been used with the baseline graph-based ranking models for sentence extraction. Experiments have been conducted over the benchmark DUC data sets and it has been shown that the proposed Bernoulli-based sentence similarity model provides consistent improvements over the baseline IntraLink and UniformLink methods [1].

This paper presents a position based visual tracking system of a redundant manipulator using a Kinect camera. Kinect camera provides 3-D information of a target object, therefore the control algorithm of the position-based visual servoing (PBVS) can be simplified, as there is no requirement to estimate a 3-D feature point position from the extracted image and the camera model. The Kalman filter is used to predict the target position and velocity. This control method is applied to a calibrated robotic system with eye-to-hand configuration. The stability analysis has been derived and real-time experiments have been carried out using a 7 DOF PowerCube manipulator from Amtec Robotic. The experimental results of both static and moving targets are presented to demonstrate and to verify the proposed position based visual tracking system performance.

A user query for information retrieval (IR) applications may not contain the most appropriate terms (words) as actually intended by the user. This is usually referred to as the term mismatch problem and is a crucial research issue in IR. Using the notion of relevance, we provide a comprehensive theoretical analysis of a parametric query vector, which is assumed to represent the information needs of the user. A lexical association function has been derived analytically using the system relevance criteria. The derivation is further justified using an empirical evidence from the user relevance criteria. Such analytical derivation as presented in this paper provides a proper mathematical framework to the query expansion techniques, which have largely been heuristic in the existing literature. By using the generalized retrieval framework, the proposed query representation model is equally applicable to the vector space model (VSM), Okapi best matching 25 (Okapi BM25), and Language Model (LM). Experiments over various data sets from TREC show that the proposed query representation gives statistically significant improvements over the baseline Okapi BM25 and LM as well as other well-known global query expansion techniques. Empirical results along with the theoretical foundations of the query representation confirm that the proposed model extends the state of the art in global query expansion.

Brain-computer interface (BCI) technology is a means of communication that allows individuals with severe movement disability to communicate with external assistive devices using the electroencephalogram (EEG) or other brain signals. This paper presents an alternative neural information processing architecture using the Schrödinger wave equation (SWE) for enhancement of the raw EEG signal. The raw EEG signal obtained during the motor imagery (MI) of a BCI user is intrinsically embedded with non-Gaussian noise while the actual signal is still a mystery. The proposed work in the field of recurrent quantum neural network (RQNN) is designed to filter such non-Gaussian noise using an unsupervised learning scheme without making any assumption about the signal type. The proposed learning architecture has been modified to do away with the Hebbian learning associated with the existing RQNN architecture as this learning scheme was found to be unstable for complex signals such as EEG. Besides, this the soliton behaviour of the non-linear SWE was not properly preserved in the existing scheme. The unsupervised learning algorithm proposed in this paper is able to efficiently capture the statistical behaviour of the input signal while making the algorithm robust to parametric sensitivity. This denoised EEG signal is then fed as an input to the feature extractor to obtain the Hjorth features. These features are then used to train a Linear Discriminant Analysis (LDA) classifier. It is shown that the accuracy of the classifier output over the training and the evaluation datasets using the filtered EEG is much higher compared to that using the raw EEG signal. The improvement in classification accuracy computed over nine subjects is found to be statistically significant.

In a multi-robot system, the coordination and cooperation among the robots determine the effectiveness of task execution. Different centralised and distributed task allocation algorithms have been proposed by researchers. Recently consensus based task allocation has been extensively researched because of its robustness in handling large teams of robots. We propose a new auction and consensus based algorithm for fast task allocation in parallel with task execution. The performance of the proposed algorithm under different conditions is analyzed and compared with other distributed consensus algorithms.

Diversity improvement methods generally implement niching and fitness sharing schemes. In this work we propose a general principle based on using the inverse mapping from objective space to decision space that allows for the creation of diverse solutions in a direct manner. When analytical forms of objective functions are known, we propose a method of generating set-valued inverse maps, in functional or algorithmic form, which when restricted to the feasible search range yield pseudo inverses of objectives. In the absence of analytical functional forms we propose the use of artificial neural networks (ANNs) in a novel configuration to directly learn the inverse map without network inversion procedures. We implement two diversity creation operators and use them in a standard binary multi-objective genetic algorithm (MOGA) to solve standard bi-objective optimization problems. We also propose a parameter less approach of fixing the number and desirable locations of solutions in sparse regions. Proposed algorithms are compared with NSGA-II and it is shown that the proposed algorithms achieve desired level of diversity in fewer function evaluations compared to NSGA-II.

This paper presents a distributed fuzzy proportional control system for a vision guided redundant robot manipulator. Firstly, the Takagi Sugeno (TS) fuzzy algorithm is used to model analytical Image Based Visual Servoing (IBVS) using shape moments by offline learning. This control method is applied to an uncalibrated robotic system with eye-in-hand visual feedback. The system is able to track a moving object through a variety of motions and maintain the object's image features in a desired position in the image plane without a priori knowledge of the robot kinematic, camera calibration and inverse Jacobian. The experimental results of both static and moving targets using the 7 DOF PowerCube manipulator from Amtec Robotic show and verify its performance in a realtime application.

In this paper, we have presented a GPS and sonar based area mapping and navigation scheme for a mobile robot. A mapping is achieved between the GPS space and the world coordinates of the mobile robot which enables us to generate direct motion commands for it. This mapping enables the robot to navigate among different GPS locations within the mapped area. The GPS data is extracted online to get the latitude and longitude information of a particular location. In the training phase, a 2-D axis transformation is used to relate local robot frame with the robot world coordinates and then the actual world coordinates are mapped from the GPS data using a RBFN (radial basis function network) based Neural Network. In the second phase, direct GPS data is used to get the mapping into the world coordinates of mobile robot using the trained network and the motion commands are generated accordingly. The physical placement of sonar devices, their ranging limits and beam opening angles are considered during navigation for possible collision detection and obstacle avoidance. This scheme is successfully implemented in real time with Pioneer mobile robot from ActivMedia Robotics and GPS receiver. The scheme is also tested in the simulation to justify its application in the real world.

A novel adaptive critic based kinematic control scheme for a redundant manipulator has been proposed in this paper. The redundancy resolution has been formulated as a discrete-time optimal control problem. A Takagi-Sugeno (T-S) fuzzy based critic network is proposed to predict the global costate dynamics as fuzzy average of local costate dynamics. The integral cost function is used in the literature earlier, to achieve global optimum in contrast to instantaneous cost functions which gives local optimum. But both the approaches require the computation of pseudo-inverse which is computationally complex and suffers from numerical instability. In contrast, the proposed scheme does not require computation of pseudo-inverse of the Jacobian which makes the method computationally efficient. The proposed scheme is tested on 7 degree of freedom (7DOF) PowerCube manipulator from Amtec Robotics.

This paper proposes a novel scheme of a Takagi-Sugeno (T-S) fuzzy based adaptive critic for the optimal control of the continuous-time input affine nonlinear system. A novel learning strategy is proposed to update the weights of critic network which resolves the issue of under-determined weight update equations discussed in [1]. The T-S Fuzzy based critic network approximates the global optimal cost as fuzzy average of local costs associated with local linear subsystems. This work clearly demonstrates that the optimal cost of a nonlinear system can be represented as the fuzzy cluster of optimal costs of locally valid linear models in a T-S framework. The proposed scheme has been simulated for four different dynamic systems. Simulation results clearly demonstrate that the T-S Fuzzy approximates the optimal cost, with subsystems in each fuzzy zone represents the optimal cost of locally valid linear model.

We report the design and implementation of three different multiobjective optimization based control strategies for the cart pole system: l) a multiobjective version of the classic quadratic regulator problem, 2) a multiobjective formulation of a standard Hinfincontroller and 3) a mixed norm H2/Hinfincontroller design problem in a multiobjective setting. The optimization problems have been solved using an elitist Pareto dominance based multiobjective genetic algorithm developed by the authors. Input saturation and bounds on state variables have been incorporated in the problem. It is shown by way of an example that the solution to the scalarized version of multiobjective linear regulator design problem is contained in the set of solutions of the vector objective formulation of the same multiobjective design problem. Finally the validity of the solutions was tested on a real cart pole regulator system.

This paper is concerned with the inverse kinematic control of a 6DOF robot manipulator using visual feedback. Two different frameworks have been proposed to learn the inverse kinematics of the manipulator. In the first framework, the robot work-space has been discretized using a priori fixed number of fuzzy regions. Within each fuzzy region, the inverse kinematic relationship from image plane as observed by two fixed cameras to joint space of the manipulator is expressed as a linear map using first order approximation. This proposed framework allows the inverse kinematics to be represented by a Takagi-Sugeno (T-S) fuzzy model whose parameters are learned on-line using gradient descent algorithm. In the second framework, the robot workspace in image plane is discretized into a number of clusters whose centers are determined using Fuzzy C Mean (FCM) clustering algorithm. The FCM algorithm allows each data vector to belong to every cluster with a fuzzy truth value between 0 and 1. The inverse kinematics problem is solved without using any knowledge about orientation of the manipulator. This leads to redundant solutions in the joint angle space for a given target position. This redundancy in the joint angle space is achieved using the concept of sub clustering in the joint space. Inclusion of sub-clustering also improves the position tracking accuracy. The proposed algorithms have been successfully implemented on a 6 DOF PowerCube manipulator from Amtec robotics with a reasonable position tracking accuracy.

This paper deals with visual-motor coordination of a 7 dof robot manipulator for pick and place applications. Three issues are dealt with in this paper - finding a feasible inverse kinematic solution without using any orientation information, resolving redundancy at position level and finally maintaining the fidelity of information during clustering process thereby increasing accuracy of inverse kinematic solution. A 3-dimensional KSOM lattice is used to locally linearize the inverse kinematic relationship. The joint angle vector is divided into two groups and their effect on end-effector position is decoupled using a concept called function decomposition. It is shown that function decomposition leads to significant improvement in accuracy of inverse kinematic solution. However, this method yields a unique inverse kinematic solution for a given target point. A concept called sub-clustering in configuration space is suggested to preserve redundancy during learning process and redundancy is resolved at position level using several criteria. Even though the training is carried out off-line, the trained network is used online to compute the required joint angle vector in only one step. The accuracy attained is better than the current state of art. The experiment is implemented in real-time and the results are found to corroborate theoretical findings.

In the above titled paper (ibid., vol. 17, no. 5, pp. 1116-1125), there were a few errors. Corrections are presented here.

This paper investigates new learning algorithms (LF I and LF II) based on Lyapunov function for the training of feedforward neural networks. It is observed that such algorithms have interesting parallel with the popular backpropagation (BP) algorithm where the fixed learning rate is replaced by an adaptive learning rate computed using convergence theorem based on Lyapunov stability theory. LF II, a modified version of LF I, has been introduced with an aim to avoid local minima. This modification also helps in improving the convergence speed in some cases. Conditions for achieving global minimum for these kind of algorithms have been studied in detail. The performances of the proposed algorithms are compared with BP algorithm and extended Kalman filtering (EKF) on three bench-mark function approximation problems: XOR, 3-bit parity, and 8-3 encoder. The comparisons are made in terms of number of learning iterations and computational time required for convergence. It is found that the proposed algorithms (LF I and II) are much faster in convergence than other two algorithms to attain same accuracy. Finally, the comparison is made on a complex two-dimensional (2-D) Gabor function and effect of adaptive learning rate for faster convergence is verified. In a nutshell, the investigations made in this paper help us better understand the learning procedure of feedforward neural networks in terms of adaptive learning rate, convergence speed, and local minima.

This correspondence proposes two novel control schemes with variable state-feedback gain to stabilize a Takagi-Sugeno (T-S) fuzzy system. The T-S fuzzy model is expressed as a linear plant with nonlinear disturbance terms in both schemes. In controller I, the T-S fuzzy model is expressed as a linear plant around a nominal plant arbitrarily selected from the set of linear subsystems that the T-S fuzzy model consists of. The variable gain then becomes a function of a gain parameter that is computed to neutralize the effect of disturbance term, which is, in essence, the deviation of the actual system dynamics from the nominal plant as the system traverses a specific trajectory. This controller is shown to stabilize the T-S fuzzy model. In controller II, individual linear subsystems are locally stabilized. Fuzzy blending of individual control actions is shown to make the T-S fuzzy system Lyapunov stable. Although applicability of both control schemes depends on the norm bound of unmatched state disturbance, this constraint is relaxed further in controller II. The efficacy of controllers I and II has been tested on two nonlinear systems.

This paper presents a new paradigm for stochastic filtering by modeling the unified response of a neural lattice using the Schroedinger wave equation. The model is based on a novel concept that a quantum object mediates the collective response of a neural lattice. The model is referred as recurrent quantum neural network (RQNN). The RQNN model has been simulated in two different ways. In one case the potential field of the Schroedinger wave equation is linearly modulated and in the other case the potential field of the Schroedinger wave equation is nonlinearly modulated. It is shown that the proposed quantum stochastic filter can efficiently denoise signals such as DC, sinusoid, amplitude modulated sinusoid and speech signals embedded in very high Gaussian and non-Gaussian noises. Performance of linearly modulated RQNN compares well with traditional techniques such as Kalman filter and wavelet filter. However, preliminary results show that nonlinearly modulated RQNN performs much better when compared with traditional techniques. For example, nonlinearly modulated RQNN model denoises a DC signal 1000 times more accurately in comparison to a traditional Kalman filter. The most important fact is that the proposed quantum stochastic filter does not make any assumption about the shape and nature of the signal and noise when denoising a signal. In a sense, the proposed quantum stochastic filter is a step forward towards intelligent filtering.

This paper is concerned with the stable tracking of a nonlinear plant using fuzzy clustering of ARMA models. The nonlinear input-output model is identified as fuzzy clusters of local ARMA models using a fuzzy neural net (FNN). It is thoroughly investigated whether the overall system can be made stable by stabilizing the individual linear subsystems. Two nonlinear systems have been taken and the controller is designed based on stabilizing each fuzzy region. The overall controller is determined by fuzzy blending of the individual controllers and it is shown that the systems are stable. Also, the overall system tracks a desired trajectory if all the individual subsystems track the same.

This paper presents an unconventional approach to adaptive fuzzy logic controller (FLC) design wherein a new evolution strategy, Differential Evolution (DE) is used in the simultaneous design of membership functions and rule sets for fuzzy logic cont

This paper is concerned with the design of a neuro-adaptive trajectory tracking controller. The paper presents a new control scheme based on inversion of a feedforward neural model of a robot arm. The proposed control scheme requires two modules. The first module consists of an appropriate feedforward neural model of forward dynamics of the robot arm that continuously accounts for the changes in the robot dynamics. The second module implements an efficient network inversion algorithm that computes the control action by inverting the neural model. In this paper, a new extended Kalman filter (EKF) based network inversion scheme is proposed. The scheme is evaluated through comparison with two other schemes of network inversion: gradient search in input space and Lyapunov function approach. Using these three inversion schemes the proposed controller was implemented for trajectory tracking control of a two-link manipulator. Simulation results in all cases confirm the efficacy of control input prediction using network inversion. Comparison of the inversion algorithms in terms of tracking accuracy showed the superior performance of the EKF based inversion scheme over others.