In this paper, a novel Multimodal Smartphone based IoT framework is proposed that interconnects human user with robotic system in performing SLAM based Human Robot Interactive tasks. Through this system, humans are added in the loop to build and update the SLAM map where they send commands to add and change locations on top of the 2D SLAM map and navigate to locations using the SLAM map when users issue a navigation command. Preliminary experiments conducted in a lab environment showed that the system can build and update SLAM maps, using user voice and text commands. Adding or changing a location in the SLAM map through user speech and text commands took an average time of 9.7 seconds while the robotic system took an average time of 40.96 seconds to complete a navigation task based on user commands using the SLAM map with an average distance error of 10.4 cm.

In this paper, a novel Edge-IoT framework is proposed which interconnects the user's smartphones with the robotic system for human-robot interaction. The user sends speech commands using mobile application which is processed by the framework and sent to the robotic system. The robotic system executes the task and sends a completion message to the user through the framework. Preliminary experiments conducted by sending multiple requests to the framework to perform a robotic task showed that the framework is able to handle multiple requests, send commands to the robotic system to perform the robotic task and send completion messages to the user efficiently.

Robots are increasingly present in environments shared with humans. Robots can cooperate with their human teammates to achieve common goals and complete tasks. This paper focuses on developing a real-time framework that assesses the cognitive load of a human while cooperating with a robot to complete a collaborative assembly task. The framework uses multi-modal sensory data from Electrocardiography (ECG) and Electrodermal Activity (EDA) sensors, extracts novel features from the data, and utilizes machine learning methodologies to detect high or low cognitive load. The developed framework was evaluated on a collaborative assembly scenario with a user study. The results show that the framework is able to reliably recognize high cognitive load and it is a first step in enabling robots to understand better about their human teammates.