Mayur Patidar, Riya Sawhney, Avinash Singh, Biswajit Chatterjee, Mausam ., Indrajit Bhattacharya. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Mayur Patidar, Prayushi Faldu, Avinash Singh, Lovekesh Vig, Indrajit Bhattacharya, Mausam -. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

In physical human-robot collaboration (pHRC), singularity avoidance strategies are often critical to obtain stable interaction dynamics. It is hypothesised a predictable singularity avoidance strategy is preferred in pHRC as humans tend to maximise predictability when using complex systems. By using an electroencephalogram (EEG), it is possible to assess the predictability of a task through a feature found in event-related potentials (ERP) and called prediction-error negativity (PEN). In this paper, two research questions are addressed. Can a complex pHRC singularity avoidance strategy generate a detectable PEN? Are PEN and human preferences related when comparing different control settings in a singularity avoidance strategy? Fourteen participants compared two different sets of parameters (modes) in a singularity avoidance strategy based on the exponentially damped least-squared (EDLS) method. ERP results are presented in terms of power spectral density (PSD). ERP results were then compared with human preferences to see whether they are related. Results show that the mode that causes PEN is also the one that participants did not like, suggesting that a lack of predictability might have an impact on human preference.

Driver mental fatigue leads to thousands of traffic accidents. The increasing quality and availability of low-cost electroencephalogram (EEG) systems offer possibilities for practical fatigue monitoring. However, non-data-driven methods, designed for practical, complex situations, usually rely on handcrafted data statistics of EEG signals. To reduce human involvement, we introduce a data-driven methodology for online mental fatigue detection: self-weight ordinal regression (SWORE). Reaction time (RT), referring to the length of time people take to react to an emergency, is widely considered an objective behavioral measure for mental fatigue state. Since regression methods are sensitive to extreme RTs, we propose an indirect RT estimation based on preferences to explore the relationship between EEG and RT, which generalizes to any scenario when an objective fatigue indicator is available. In particular, SWORE evaluates the noisy EEG signals from multiple channels in terms of two states: shaking state and steady state. Modeling the shaking state can discriminate the reliable channels from the uninformative ones, while modeling the steady state can suppress the task-nonrelevant fluctuation within each channel. In addition, an online generalized Bayesian moment matching (online GBMM) algorithm is proposed to online-calibrate SWORE efficiently per participant. Experimental results with 40 participants show that SWORE can maximally achieve consistent with RT, demonstrating the feasibility and adaptability of our proposed framework in practical mental fatigue estimation.

In order for robots to interact with humans on real-world scenarios or objects, these robots need to construct a representation ('state of mind') of these scenarios that a) are grounded in the robots' perception and b) ideally should match human understanding and concepts.Using table-top settings as scenario, we propose a framework that generates a robot's ''state of mind' by extracting the objects on the table along with their properties (color, shape and texture) and spatial relations to each other.The scene as perceived by the robot is represented in a dynamic graph in which object attributes are encoded as fuzzy linguistic variables that match human spatial concepts.In particular, this paper details the construction of such graph representations by combining low-level neural networkbased feature recognition and a high-level fuzzy inference system.Using fuzzy representations allows for easily adapting the robot's original scene representation to deviations in properties or relations that emerge in language descriptions given by humans viewing the same scene.The framework is implemented on a Pepper humanoid robot and has been evaluated using a data set collected in-house.

A driver's cognitive state of mental fatigue significantly affects his or her driving performance and more important, public safety. Previous studies have leveraged reaction time (RT) as the metric for mental fatigue and aim at estimating the exact value of RT using electroencephalogram (EEG) signals within a regression model. However, due to the easily corrupted and also nonsmooth properties of RTs during data collection, methods focusing on predicting the exact value of a noisy measurement, RT generally suffer from poor generalization performance. Considering that human RT is the reflection of brain dynamics preference (BDP) rather than a single regression output of EEG signals, we propose a novel channel-reliability-aware ranking (CArank) model for the multichannel ranking problem. CArank learns from BDPs using EEG data robustly and aims at preserving the ordering corresponding to RTs. In particular, we introduce a transition matrix to characterize the reliability of each channel used in the EEG data, which helps in learning with BDPs only from informative EEG channels. To handle large-scale EEG signals, we propose a stochastic-generalized expectation maximum (SGEM) algorithm to update CArank in an online fashion. Comprehensive empirical analysis on EEG signals from 40 participants shows that our CArank achieves substantial improvements in reliability while simultaneously detecting noisy or less informative EEG channels.

Designing immersion is the key challenge in virtual reality; this challenge has driven advancements in displays, rendering and recently, haptics. To increase our sense of physical immersion, for instance, vibrotactile gloves render the sense of touching, while electrical muscle stimulation (EMS) renders forces. Unfortunately, the established metric to assess the effectiveness of haptic devices relies on the user's subjective interpretation of unspecific, yet standardized, questions.

In verbal human-robot interaction natural language utterances have to be grounded in visual scenes by the robot. Visual language grounding is a challenging task that includes identifying a primary object among several objects, together with the object properties and spatial relations among the objects. In this paper we focus on extracting this information from sentences only. We propose two language modelling techniques, one uses regular expressions and the other one utilizes Euclidian distance. We compare these two proposed techniques with two other techniques that utilize tree structures, namely an extended Hobb’s algorithm and an algorithm that utilizes a Stanford parse tree. A comparative analysis between all language modelling techniques shows that our proposed two approaches require less computational time than the tree-based approaches. All approaches perform good identifying the primary object and its property, but for spatial relation extraction the Stanford parse tree algorithm performs better than the other language modelling techniques. Time elapsed for the Stanford parse tree algorithm is higher than for the other techniques.

Physical Human-Robot Collaboration (pHRC) is about the interaction between one or more human operator(s) and one or more robot(s) in direct contact and voluntarily exchanging forces to accomplish a common task. In any pHRC, the intuitiveness of the interaction has always been a priority, so that the operator can comfortably and safely interact with the robot. So far, the intuitiveness has always been described in a qualitative way. In this paper, we suggest an objective way to evaluate intuitiveness, known as prediction error negativity (PEN) using electroencephalogram (EEG). PEN is defined as a negative deflection in event related potential (ERP) due to cognitive conflict, as a consequence of a mismatch between perception and reality. Experimental results showed that the forces exchanged between robot and human during pHRC modulate the amplitude of PEN, representing different levels of cognitive conflict. We also found that PEN amplitude significantly decreases (p <; 0.05) when a mechanical resistance is being applied smoothly and more time in advance before an invisible obstacle, when compared to a scenario in which the resistance is applied abruptly before the obstacle. These results indicate that an earlier and smoother resistance reduces the conflict level. Consequently, this suggests that smoother changes in resistance make the interaction more intuitive.

© 2017 IEEE. Brain-Computer interface (BCI) which aims at enabling users to perform tasks through their brain waves has been a feasible and worth developing solution for growing demand of healthcare. Current proposed BCI systems are often with lower applicability and do not provide much help for reducing burdens of users because of the time-consuming preparation required by adopted wet sensors and the shortage of provided interactive functions. Here, by integrating a state visually evoked potential (SSVEP)-based BCI system and a robotic eating assistive system, we propose a non-invasive wireless steady state visually evoked potential (SSVEP)-based BCI eating assistive system that enables users with physical disabilities to have meals independently. The analysis compared different methods of classification and indicated the best method. The applicability of the integrated eating assistive system was tested by an Amyotrophic Lateral Sclerosis (ALS) patient, and a questionnaire reply and some suggestion are provided. Fifteen healthy subjects engaged the experiment, and an average accuracy of 91.35%, and information transfer rate (ITR) of 20.69 bit per min are achieved. For online performance evaluation, the ALS patient gave basic affirmation and provided suggestions for further improvement. In summary, we proposed a usable SSVEP-based BCI system enabling users to have meals independently. With additional adjustment of movement design of the robotic arm and classification algorithm, the system may offer users with physical disabilities a new way to take care of themselves.

Exploring correct patterns from low‐frequency time‐series data is challenging. For resolving this problem, the concept of possibility theory–based hidden Markov model (PTBHMM) has been proposed. In this article, all three fundamental problems (evaluation, decoding, and learning) of conventional HMM have been addressed using possibility theory. For handling uncertainty, we have used an axiomatic approach of possibility theory proposed by Zadeh. The time complexity of existing solutions of HMM (forward, backward, Viterbi, and Baum Welch) and proposed possibility‐based solutions has been calculated and compared. From the comparison result, it has been found that PTBHMM has lesser time complexity and hence will be more suitable for real‐time gesture–based communication.

The visual perception of eyewitness plays a vital role in criminal identification scenario. It helps law enforcement authorities in searching particular criminal from their previous record. It has been reported that searching a criminal record manually requires too much time to get the accurate result. We have proposed a query-based approach which minimises the computational cost along with the reduction of search space. A symbolic database has been created to perform a stringent analysis on 150 public (Bollywood celebrities and Indian cricketers) and 90 local faces (our data-set). An expert knowledge has been captured to encapsulate every criminal’s anatomical and facial attributes in the form of symbolic representation. A fast query-based searching strategy has been implemented using dynamic decision tree data structure which allows four levels of decomposition to fetch respective criminal records. Two types of case studies - viewed and forensic sketches have been considered to evaluate the strength of our proposed approach. We have derived 1200 views of the entire population by taking into consideration 80 participants as eyewitness. The system demonstrates an accuracy level of 98.6% for test case I and 97.8% for test case II. It has also been reported that experimental results reduce the search space up to 30 most relevant records.