EDBT 2026 Demo / reviewers in the wild / expert
Yunyi Jia
dblp:07/7748
· DBLP profile ↗
35ranked-venue papers
7as first author
13since 2021 · last 2026
0000-0003-1334-1384ORCID · corroborated
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 22 · 7 first-author · 3 since 2021Systems, architecture and hardware · 18 · 7 first-author · 3 since 2021Applied, interdisciplinary, general and emerging computing · 12 · 9 since 2021Human-computer interaction and ubiquitous computing · 4 · 1 since 2021Graphics, computer vision, multimedia, augmented reality and games · 1
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | MR-HVIL: A Mixed-Reality-Based Human-Vehicle-In-the-Loop On-Road Validation Platform for Mixed Traffic Testing
Rongyao Wang, Prakhar Gupta, Tyler Ard, Dominik Karbowski, Ardalan Vahidi, Yunyi Jia |
IEEE Trans. Intell. Transp. Syst. | 7 |
| 2025 | Actor-Critic Cooperative Compensation to Model Predictive Control for Off-Road Autonomous Vehicles Under Unknown DynamicsabstractThis study presents an Actor-Critic Cooperative Compensated Model Predictive Controller$(\text{AC}^3 \text{MPC})$designed to address unknown system dynamics. To avoid the difficulty of modeling highly complex dynamics and ensuring real-time control feasibility and performance, this work uses deep reinforcement learning with a model predictive controller in a cooperative framework to handle unknown dynamics. The model-based controller takes on the primary role as both controllers are provided with predictive information about the other. This improves tracking performance and retention of inherent robustness of the model predictive controller. We evaluate this framework for off-road autonomous driving on unknown deformable terrains that represent sandy deformable soil, sandy and rocky soil, and cohesive clay-like deformable soil. Our findings demonstrate that our controller statistically outperforms standalone model-based and learning-based controllers by upto 29.2% and 10.2%. This framework generalized well over varied and previously unseen terrain characteristics to track longitudinal reference speeds with lower errors. Furthermore, this required significantly less training data compared to purely learning-based controller, while delivering better performance even when under-trained. Prakhar Gupta, Jonathon M. Smereka, Yunyi Jia |
ICRA | 3 |
| 2025 | Proactive Assignment Strategy With Human Choice Models for Boosting Pooled Rideshare ServiceabstractThis study analyzes various human factors considerations in estimating discounts for pooled rideshare trips. The discounts are utilized in an optimization-based rideshare assignment strategy (proactive strategy) and compared against each other, as well as a heuristic strategy attempting to replicate current real-world pooling rates. Simulations within Austin, Texas and Greenville, South Carolina, reveal the proactive strategy’s ability to increase average vehicle occupancy by 0.23 persons/mile in Austin and 0.52 persons/mile in Greenville. A significant ability to decrease trip rejections and increase profitability is also observed. Finally, the strengths of particular combinations of factors are discussed relative to their effectiveness in each region. Joseph Paul, Krishna M. Gurumurthy, Taner Cokyasar, Haotian Su, Nazmul Arefin Khan, Joshua Auld, Yunyi Jia |
IEEE Trans. Intell. Transp. Syst. | 7 |
| 2023 | Estimating Human Comfort Levels in Autonomous Vehicles Based on Vehicular Behaviors and Physiological SignalsabstractIn this study, we proposed a dynamic model that could quantify human comfort in autonomous vehicles (AVs) based on vehicular behaviors and a Kalman filter (KF) based approach to further refine comfort level estimation by leveraging physiological signals. The dynamic model could capture the dynamics in human comfort when the passenger was exposed to a continuous sequence of vehicular behaviors during an AV journey. The KF-based comfort estimation approach could fuse comfort level estimations based on physiological signals and the dynamic model. A simulator-based user study was conducted to evaluate the comfort estimation approaches in which the participants experienced a set of virtual AV journeys on a high-fidelity driving simulator with 6-degree-of-freedom motions. Experimental results show that the proposed approaches could quantify human comfort levels and the KF-based approach outperforms the others. Haotian Su, Yunyi Jia |
IROS | 2 |
| 2023 | Bilateral Adaptation of Longitudinal Control of Automated Vehicles and Human DriversabstractAutomated vehicles have great potential to transform our existing transportation systems by improving driving safety, comfort, congestion, and emissions. Despite the tremendous efforts that have been spent on the development of various automated driving technologies, user acceptance of automated driving technologies is still low, which is largely caused by the gaps between automated driving controllers and human preferences. Recent research efforts have been focusing on adapting automated driving behaviors to human demonstrations. However, most existing methods assume that human demonstration is perfect and only focus on mimicking human driving behaviors. In reality, the human demonstration will not be ideal and will include some over-aggressive or over-conservative actions that compromise the safety or efficiency of the trained automated driving controller. In this paper, an Inverse Model Predictive Control (IMPC) based bilateral adaptation method for automated vehicles and human drivers is proposed. The method can adapt automated longitudinal driving behaviors to human preferences based on human interventions during automated driving. Meanwhile, it can also reject improper interventions and send warnings to the human driver such that he/she can realize the irrationality in his/her behaviors. Eventually, the automated driving controller will adapt to the human driver’s preferences and the human driver will get rid of his/her bad driving habits. Human-in-the-loop experiments were conducted using a driving simulator to demonstrate the effectiveness of the proposed approach. Longxiang Guo, Yunyi Jia |
IEEE Trans. Intell. Transp. Syst. | 2 |
| 2023 | Tracking Occupant Activities in Autonomous Vehicles Using Capacitive SensingabstractAutonomous vehicles (AV) are a promising contemporary engineering innovation. Since they have no human drivers, their vehicle controllers must be fully informed of the activities of their occupants in vehicle seats so that occupant safety and comfort can be guaranteed. This paper introduces a system that employs capacitive sensing and machine learning to inform the controller of occupant activities, like actions and posture changes. The system facilitates capacitive sensing by deploying a capacitance-sensing mat on the vehicle seat. A sensing circuitry connected to the mat measures all its capacitances continuously. Since an occupant’s body induces variations in these capacitances, temporal sequences of capacitance measurements represent various occupant activities in the vehicle seat. Subsequently, the system converts capacitance measurements corresponding to every time step to two grayscale capacitance-sensing images (CSIs). The CSIs, in turn, yield a feature vector corresponding to every time step, thereby building a dataset of temporal sequences of features. The system’s machine-learning unit tracks and recognizes occupant actions and posture changes using an action-recognition block, which is essentially a long short-term memory (LSTM) network trained on a dataset of temporal sequences of either features or capacitance measurements corresponding to various occupant actions. If the occupant remains stationary, a switching block in the unit disables the action-recognition block and enables a posture-recognition block. The latter is a${k}$-nearest-neighbor (${k}$NN) classifier trained on a dataset of features to recognize stationary occupant postures. This paper validates the system’s performance and investigates its deployment for real-time tracking of occupant activities in AVs. Rahul P. Kumar, David Melcher, Pietro Buttolo, Yunyi Jia |
IEEE Trans. Intell. Transp. Syst. | 4 |
| 2022 | Predicting Human Intentions in Human-Robot Hand-Over Tasks Through Multimodal LearningabstractIn human–robot shared manufacturing contexts, product parts or tools hand-over between the robot and the human is an important collaborative task. Facilitating the robot to figure out and predict human hand-over intentions correctly to improve the task efficiency in human–robot collaboration is therefore a necessary issue to be addressed. In this study, a teaching-learning-prediction (TLP) framework is proposed for the robot to learn from its human partner’s multimodal demonstrations and predict human hand-over intentions. In this approach, the robot can be programmed by the human through demonstrations utilizing natural language and wearable sensors according to task requirements and the human’s working preferences. Then the robot learns from human hand-over demonstrations online via extreme learning machine (ELM) algorithms to update its cognition capacity, allowing the robot to use its learned policy to predict human intentions actively and assist its human companion in hand-over tasks. Experimental results and evaluations suggest that the human may program the robot easily by the proposed approach when the task changes, as the robot can effectively predict hand-over intentions with competitive accuracy to complete the hand-over tasks.Note to Practitioners—This article is motivated by human–robot hand-over problems in smart manufacturing contexts. Product parts or tools delivery in worker–robot partnerships is an important collaborative task. We develop a teaching-learning-prediction (TLP) framework for the robot to learn from its human partner’s multimodal demonstrations and predict human hand-over intentions. The robot can be taught by human through natural language and wearable sensing information. The extreme learning machine (ELM) approach is employed for the robot to build its cognition capacity to predict human intentions actively and assist its human companion in hand-over tasks. We demonstrate that the proposed approach presents distinct and effective advantages to facilitate human–robot hand-over tasks in collaborative manufacturing contexts. Weitian Wang, Rui Li 0021, Yi Chen 0030, Yunyi Jia |
IEEE Trans Autom. Sci. Eng. | 5 |
| 2022 | Anticipative and Predictive Control of Automated Vehicles in Communication-Constrained Connected Mixed TrafficabstractConnected automated driving technologies have shown substantial benefits to improve the safety and efficiency of traffic. However, connected mixed traffic, which involves both connected automated vehicles and connected human-driven vehicles, is more foreseen for the realistic case in the near future. This brings new challenges because of the complexity of human elements in the system. In addition, the communication constraints in realistic connectivity such as random delays and packet losses bring even more challenges to the system. Therefore, this paper proposes a new anticipative and predictive automated vehicle control approach in connected mixed traffic. The approach first anticipates the states of surrounding vehicles including human-driven vehicles, and then integrates the anticipation into the predictive control of automated vehicles, which can help improve the control performance and also handle the communication constraints. An inverse model predictive control (IMPC) based anticipation approach has been proposed. The proposed approach, together with constant speed (CS), intelligent driver model (IDM) and artificial neural network (ANN) based anticipation methods are integrated with model predictive control (MPC) for automated vehicle control. The approaches have been tested in human-in-the-loop experiments and the results show that the integration with a newly proposed IMPC based anticipation has shown the best performance in terms of accuracy, efficiency and scalability in connected mixed traffic with both ideal and constrained communications. Longxiang Guo, Yunyi Jia |
IEEE Trans. Intell. Transp. Syst. | 2 |
| 2022 | Robust Target Recognition and Tracking of Self-Driving Cars With Radar and Camera Information Fusion Under Severe Weather ConditionsabstractRadar and camera information fusion sensing methods are used to solve the inherent shortcomings of the single sensor in severe weather. Our fusion scheme uses radar as the main hardware and camera as the auxiliary hardware framework. At the same time, the Mahalanobis distance is used to match the observed values of the target sequence. Data fusion based on the joint probability function method. Moreover, the algorithm was tested using actual sensor data collected from a vehicle, performing real-time environment perception. The test results show that radar and camera fusion algorithms perform better than single sensor environmental perception in severe weather, which can effectively reduce the missed detection rate of autonomous vehicle environment perception in severe weather. The fusion algorithm improves the robustness of the environment perception system and provides accurate environment perception information for the decision-making system and control system of autonomous vehicles. Yingfeng Cai, Hai Wang 0003, Long Chen 0003, Hongbo Gao 0001, Yunyi Jia, Yicheng Li 0001 |
IEEE Trans. Intell. Transp. Syst. | 6 |
| 2022 | Study of Human Comfort in Autonomous Vehicles Using Wearable SensorsabstractThe rapid development of autonomous vehicles (AVs) has depicted a promising future of a safer and more efficient transportation system. To better induce this revolution, massive efforts have been spent on the technical competence of AVs. However, the human comfort in AVs has been an under-discussed yet important topic to the user acceptance of AVs. For the detection of human comfort, existing studies focus more on the physical influential factors of comfort such as sitting posture, vibration, and noise. With the introduction of AVs, psychological factors also have gained greater influence on human comfort. Despite existing studies of exploring correlations between human comfort and some physiological signals in automated driving contexts, there is few study on how human comfort level in AVs can be detected with these physiological signals. In this paper, we developed effective human comfort study approaches in autonomous vehicles with wearable sensors. We also proposed a machine learning based approach with adaptive feature selection to detect human comfort levels based on the wearable sensing data. The experimental results illustrated the effectiveness of the proposed approaches in studying human comfort in AVs. Haotian Su, Yunyi Jia |
IEEE Trans. Intell. Transp. Syst. | 2 |
| 2022 | Dynamic Modeling and Control of Deformable Linear Objects for Single-Arm and Dual-Arm Robot ManipulationsabstractRobotic manipulation of deformable linear objects (DLOs) is important in many applications such as the assembly of deformable wire harnesses and cables in manufacturing. Despite some relevant work in modeling, few studies have studied the comprehensive dynamic modeling and precise automatic control of DLOs using robots due to their high degrees of freedom and high flexibility. To fill this gap, the precise single-arm and dual-arm robot manipulation control of DLOs is studied in this article. First, a more comprehensive dynamic model of DLOs is established based on a discrete elastic rod model, which takes into account the twisting deformation of DLOs. The collisions, contacts, and frictions between the DLOs and the plane, as well as the kinematic constraints of both ends of the DLOs, are also considered in the model. Second, practical dynamic control schemes of robots are proposed to realize the precise control of DLOs for both the single-arm control and dual-arm control. For validations, we built an experimental platform using an ABB Yumi robot to implement and validate the proposed approaches in addition to simulations. Finally, various DLO manipulation tasks are conducted and the results for both single-arm and dual-arm manipulations validate the proposed modeling and control approaches. Naijing Lv, Jianhua Liu 0005, Yunyi Jia |
IEEE Trans. Robotics | 3 |
| 2022 | Learn How to Assist Humans Through Human Teaching and Robot Learning in Human-Robot Collaborative AssemblyabstractHuman–robot collaborative assembly has been one of the next-generation manufacturing paradigms in which superiorities of humans and robots can be fully leveraged. To enable robots effectively collaborate with humans, similar to human–human collaboration, robot learning from human demonstrations has been adopted to learn the assembly tasks. However, existing feature-based approaches require critical feature design and extraction process and are usually complex to incorporate task contexts. Existing learning-based approaches usually require a large amount of manual effort for data labeling and also rarely consider task contexts. This article proposes a dual-input deep learning approach to incorporate task contexts into the robot learning from human demonstration process to assist human in assembly. In addition, online automated data labeling during human demonstration is proposed to reduce the training efforts for learning. The experimental validations on a realistic human–robot model car assembly task with safety-concerned execution designs demonstrate the effectiveness and advantages of the proposed approaches. Weitian Wang, Yi Chen 0030, Yunyi Jia |
IEEE Trans. Syst. Man Cybern. Syst. | 4 |
| 2021 | Adaptive Optimization of Autonomous Vehicle Computational Resources for Performance and Energy ImprovementabstractAutonomous vehicles usually consume a large amount of computational power for their operations, especially for the tasks of sensing and perception with artificial intelligence algorithms. Such a computation may not only cost a significant amount of energy but also cause performance issues when the onboard computational resources are limited. To address this issue, this paper proposes an adaptive optimization method to online allocate the onboard computational resources of an autonomous vehicle amongst multiple vehicular subsystems depending on the contexts of the situations that the vehicle is facing. Different autonomous driving scenarios were designed to validate the proposed approach and the results showed that it could help improve the overall performance and energy consumption of autonomous vehicles compared to existing computational arrangement. Saurabh Jambotkar, Longxiang Guo, Yunyi Jia |
IROS | 3 |
| 2020 | Real-Time Adaptive Assembly Scheduling in Human-Multi-Robot Collaboration According to Human Capability*abstractHuman-multi-robot collaboration is becoming more and more common in intelligent manufacturing. Optimal assembly scheduling of such systems plays a critical role in their production efficiency. Existing approaches mostly consider humans as agents with assumed or known capabilities, which leads to suboptimal performance in realistic applications where human capabilities usually change. In addition, most robot adaptation focuses on human-single-robot interaction and the adaptation in human-multi-robot interaction with changing human capability still remains challenging due to the complexity of the heterogeneous multi-agent interactions. This paper proposes a real-time adaptive assembly scheduling approach for human-multi-robot collaboration by modeling and incorporating changing human capability. A genetic algorithm is also designed to derive implementable solutions for the formulated adaptive assembly scheduling problem. The proposed approaches are validated through different simulated human-multi-robot assembly tasks and the results demonstrate the effectiveness and advantages of the proposed approaches. Yunyi Jia |
ICRA | 4 |
| 2020 | Enabling Robot to Assist Human in Collaborative Assembly using Convolutional Neural NetworksabstractHuman-robot collaborative assembly consists of humans and automated robots, who cooperate with each other to accomplish complex assembly tasks, which are difficult for either humans or robots to accomplish alone. There has been some success in statistics-based and optimization-based approaches to realize human-robot collaboration. However, they usually need a set of complex modeling and setup efforts and the robots usually need to be programmed by a well-trained expert. In this paper, we take a new approach by introducing convolutional neural networks (CNN) into the teaching- learning-collaboration (TLC) model for collaborative assembly tasks. The proposed approach can alleviate the need for complex modeling and setup compared to the existing approaches. It can collect and automatically label the data from human demonstrations and then train a CNN-based robot assistance model to make the robot assist humans in the assembly process in real-time. We have experimentally verified our proposed approach on a human-robot collaborative assembly platform and the results suggest that the robot can successfully learn from human demonstrations to automatically generate right actions to assist human in accomplishing assembly tasks. Weitian Wang, Venkat N. Krovi, Yunyi Jia |
IROS | 4 |
| 2020 | Predictive Control of Connected Mixed Traffic under Random Communication ConstraintsabstractFully connected and automated vehicles have been envisioned to help improve the driving safety and efficiency of the transportation system. However, human-driven vehicles will still be present in the near future, which will lead to connected mixed traffic instead of fully connected and automated traffic. This is challenging because of the complexity of human-driving vehicles and the potential communication constraints in the connectivity. To address this issue, this paper models the connected mixed traffic and proposes model predictive control approaches with various prediction approaches including a new inverse model predictive control (IMPC) based approach to handle random communication delays and packet losses in connectivity. The human-in-the-loop experimental results for connected mixed traffic demonstrated the effectiveness and advantages of the proposed approaches, especially the predictive control with IMPC in handling communication constraints in mixed traffic. Longxiang Guo, Yunyi Jia |
IROS | 2 |
| 2019 | Modeling, Learning and Prediction of Longitudinal Behaviors of Human-Driven Vehicles by Incorporating Internal Human DecisionMaking Process using Inverse Model Predictive ControlabstractUnderstanding the behaviors of human-driven vehicles such as acceleration and braking are critical for the safety of the near-future mixed transportation systems which involve both automated and human-driven vehicles. Existing approaches in modeling human driving behaviors including driver-model-based approaches and heuristic approaches have issues in either model accuracy or scalability limitation to new situations. To address these issues, this paper proposes a new inverse model predictive control (IMPC) based approach to model longitudinal human driving behaviors. The approach incorporates the internal decision making process of humans, and achieves better predicting accuracy and improved scalability to different situations. The modeling, learning, and prediction of longitudinal human driving behaviors using the proposed IMPC approach are presented. Experimental results validate the effectiveness and advantages of the approach. Longxiang Guo, Yunyi Jia |
IROS | 2 |
| 2019 | Facilitating Human-Robot Collaborative Tasks by Teaching-Learning-Collaboration From Human DemonstrationsabstractCollaborative robots are widely employed in strict hybrid assembly tasks involved in intelligent manufacturing. In this paper, we develop a teaching-learning-collaboration (TLC) model for the collaborative robot to learn from human demonstrations and assist its human partner in shared working situations. The human could program the robot using natural language instructions according to his/her personal working preferences via this approach. Afterward, the robot learns from human assembly demonstrations by taking advantage of the maximum entropy inverse reinforcement learning algorithm and updates its task-based knowledge using the optimal assembly strategy. In the collaboration process, the robot is able to leverage its learned knowledge to actively assist the human in the collaborative assembly task. Experimental results and analysis demonstrate that the proposed approach presents considerable robustness and applicability in human-robot collaborative tasks. Note to Practitioners-This paper is motivated by the human-robot collaborative assembly problem in the context of advanced manufacturing. Collaborative robotics makes a huge shift from the traditional robot-in-a-cage model to robots interacting with people in an open working environment. When the human works with the robot in the shared workspace, it is significant to lessen human programming effort and improve the human-robot collaboration efficiency once the task is updated. We develop a TLC model for the robot to learn from human demonstrations and assist its human partner in collaborative tasks. Once the task is changed, the human may code the robot via natural language instructions according to his/her personal working preferences. The robot can learn from human assembly demonstrations to update its task-based knowledge, which can be leveraged by the robot to actively assist the human to accomplish the collaborative task. We demonstrate the advantages of the proposed approach via a set of experiments in realistic human-robot collaboration contexts. Weitian Wang, Rui Li 0021, Yi Chen 0030, Zachary Max Diekel, Yunyi Jia |
IEEE Trans Autom. Sci. Eng. | 5 |
| 2019 | Controlling Object Hand-Over in Human-Robot Collaboration Via Natural Wearable SensingabstractWith the deployment of collaborative robots in intelligent manufacturing, object hand-over between humans and robots plays a significant role in human-robot collaborations. In most collaboration studies, human hand-over intentions were usually assumed to be known by the robot, and the research mainly focused on robot motion planning and control during the hand-over process. Several approaches have been developed to control the human-robot hand-over, such as vision-based approach and physical contact-based approach, but their applications in manufacturing environments are limited due to various constraints, such as limited human working ranges and safety concerns. In this paper, we develop a practical approach using a wearable sensory system, which has a natural and simple configuration and can be easily utilized by humans. This approach could make a robot recognize a human's hand-over intentions and enable the human to effectively and naturally control the hand-over process. In addition, the approach could recognize the attribute classes of the objects in the human's hand using the wearable sensing and enable the robot to actively make decisions to ensure that graspable objects are handed over from the human to the robot. Results and evaluations illustrate the effectiveness and advantages of the proposed approach in human-robot hand-over control. Weitian Wang, Rui Li 0021, Zachary Max Diekel, Yi Chen 0030, Zhujun Zhang, Yunyi Jia |
IEEE Trans. Hum. Mach. Syst. | 6 |
| 2018 | Personalize Vison-based Human Following for Mobile Robots by Learning from Human-Driven DemonstrationsabstractHuman following is an important feature in various human-mobile-robot collaboration applications. Vision-based human following approaches employing visual servoing controls to achieve human following are commonly adopted. Such approaches, however, require to pre-define the desired human-following parameters and need to online extract features from the acquired images to calculate the human-following parameters which serve as the feedback of visual servoing controls. This paper proposes a novel visual servoing control using the non-vector space control theory, which makes the robot be able to personalize its desired human-following parameters as a desired image learnt from human-driven demonstrations. The approach provides an easy and intuitive way for humans to personalize mobile robots to complete human-following tasks in the manners that humans prefer. Experimental results demonstrate the effectiveness and advantage of the proposed approach. Lihua Jiang, Weitian Wang, Yunyi Jia |
RO-MAN | 4 |
| 2018 | Cascade heterogeneous face sketch-photo synthesis via dual-scale Markov NetworkabstractHeterogeneous face sketch-photo synthesis is an important and challenging task in computer vision, which has widely applied in law enforcement and digital entertainment. According to the different synthesis results based on different scales, this paper proposes a cascade sketch-photo synthesis method via dual-scale Markov Network. Firstly, Markov Network with larger scale is used to synthesise the initial sketches and the local vertical and horizontal neighbour search (LVHNS) method is used to search for the neighbour patches of test patches in training set. Then, the initial sketches and test photos are jointly entered into smaller scale Markov Network. Finally, the fine sketches are obtained after cascade synthesis process. Extensive experimental results on various databases demonstrate the superiority of the proposed method compared with several state-of-the-art methods. Saisai Yao, Zhenxue Chen, Yunyi Jia, Chengyun Liu |
J. Exp. Theor. Artif. Intell. | 3 |
| 2018 | Face sketch-photo synthesis and recognition: Dual-scale Markov Network and multi-information fusion
Zhenxue Chen, Saisai Yao, Yunyi Jia, Chengyun Liu |
J. Vis. Commun. Image Represent. | 3 |
| 2015 | Data correlation approach for slippage detection in robotic manipulations using tactile sensor arrayabstractIn this paper, two techniques have been presented for slippage detection. They are independent of sensor signal type and are promising for general use on tactile array sensors. The first method is based on frequency analysis of the correlation coefficient sequence of sensor array data sampled as time evolves. The main idea is that a slippage causes heavier fluctuation in the sensor signal distribution and values than a static case. The second approach employs 2-D cross correlation to detect displacements of the sliding object from tactile images, which makes it possible to estimate the slippage velocity using commercially available sensors rather than custom hardware. Experiments have been implemented to evaluate the proposed approaches. It can be seen that the first method is capable of detecting both translational and rotational slippage. Also, it works well in dynamic environments. Additionally, the ability of the second method to detect slippage velocity has been confirmed in the experiment. Yu Cheng 0006, Chengzhi Su, Yunyi Jia, Ning Xi 0001 |
IROS | 3 |
| 2014 | Perceptive feedback for natural language control of robotic operationsabstractA new planning and control scheme for natural language control of robotic operations using the perceptive feedback is presented. Different from the traditional open-loop natural language control, the scheme incorporates the high-level planning and low-level control of the robotic systems and makes the high-level planning become a closed-loop process such that it is able to handle some unexpected events in the robotics system and the environment. The experimental results on a natural language controlled mobile manipulator clearly demonstrate the advantages of the proposed method. Yunyi Jia, Ning Xi 0001, Joyce Y. Chai, Yu Cheng 0006, Lanbo She |
ICRA | 1 |
| 2014 | Coordination of a nonholonomic mobile platform and an on-board manipulatorabstractMobile manipulators provide more advantages and flexibility in a wide range of applications than standard manipulators by introducing mobility. However, adding mobile platforms to standard manipulators, especially nonholonomic mobile platforms, introduces new challenges to the system modeling and control. Most existing methods for mobile manipulators do not consider the performance difference between the mobile platform and the manipulator and therefore cannot handle the uncertain and unexpected events happened in both the mobile platform and the manipulator. This paper introduces a planning and control method in a perceptive reference frame for a nonholonomic mobile manipulator to efficiently handle uncertain and unexpected events. The experimental results on a nonholonomic mobile manipulator demonstrate the effectiveness and advantages of the designed method. Yunyi Jia, Ning Xi 0001, Erick Nieves-Rivera |
ICRA | 1 |
| 2014 | Coordinated motion control of a nonholonomic mobile manipulator for accurate motion trackingabstractStandard manipulators are restrained in many applications due to their limited working ranges. Adding mobile platforms, in particular nonholonomic mobile platforms, can expediently enlarge their working ranges but also introduces new challenges. The problem of the existing control methods for nonholonomic mobile manipulators is that they leave out the consideration of the differences between the mobile platform and the manipulator such as the dynamics differences and working condition differences. This may consequently result in some unnecessarily large errors for the motion tracking in the implementation. To address this problem, this paper proposes a new practical control method using the adaptive motion distribution and coordination between the mobile platform and manipulator to minimize the errors of the motion control and also automatically handle some unexpected events. The effectiveness and advantages of the proposed method were demonstrated through both simulation and experimental results. Yunyi Jia, Ning Xi 0001, Yu Cheng 0006, Siyang Liang |
IROS | 1 |
| 2014 | Teaching Robots New Actions through Natural Language InstructionsabstractRobots often have limited knowledge and need to continuously acquire new knowledge and skills in order to collaborate with its human partners. To address this issue, this paper describes an approach which allows human partners to teach a robot (i.e., a robotic arm) new high-level actions through natural language instructions. In particular, built upon the traditional planning framework, we propose a representation of high-level actions that only consists of the desired goal states rather than step-by-step operations (although these operations may be specified by the human in their instructions). Our empirical results have shown that, given this representation, the robot can reply on automated planning and immediately apply the newly learned action knowledge to perform actions under novel situations. Lanbo She, Yu Cheng 0006, Joyce Y. Chai, Yunyi Jia, Ning Xi 0001 |
RO-MAN | 4 |
| 2014 | Back to the Blocks World: Learning New Actions through Situated Human-Robot DialogueabstractThis paper describes an approach for a robotic arm to learn new actions through dialogue in a simplified blocks world. In particular, we have developed a three-tier action knowledge representation that on one hand, supports the connection be-tween symbolic representations of lan-guage and continuous sensorimotor repre-sentations of the robot; and on the other hand, supports the application of existing planning algorithms to address novel situ-ations. Our empirical studies have shown that, based on this representation the robot was able to learn and execute basic actions in the blocks world. When a human is engaged in a dialogue to teach the robot new actions, step-by-step instructions lead to better learning performance compared to one-shot instructions. 1 Lanbo She, Yu Cheng 0006, Yunyi Jia, Joyce Y. Chai, Ning Xi 0001 |
SIGDIAL Conference | 4 |
| 2012 | Online identification of quality of teleoperator (QoT) for performance improvement of telerobotic operationsabstractIn teleoperation studies, most researchers have been researching on the stability and telepresence. Few of them have studied the influence of the operational status of the teleoperator on the telerobotic systems. As a matter of fact, improper and incorrect operations of the teleoperator may decrease the teleoperation efficiency and even result in some serious safety problems even if the stability and telepresence are both guaranteed. Thus, this paper investigates a method to online identify the quality of the teleoperator and then integrate it into the planning and control of the telerobotic system. The method can help improve the performance of the system including efficiency and safety. It is also implemented on a mobile manipulator and the experimental results illustrate the effectiveness of the designed method. Yunyi Jia, Ning Xi 0001 |
ICRA | 1 |
| 2012 | Multi-objective optimization for telerobotic operations via the InternetabstractTeleoperation systems extend the ability of human power to manipulate objects in distant locations and have a wide range of applications in many areas. For decades, most researchers focused on the stability and telepresence of the teleoperation systems. Few of them have studied the influence of the teleoperation condition variables on the telerobotic operations, such as the quality of teleoperator, task dexterity and network quality. In fact, these variables may seriously affect the telerobotic operations and system performance even if the system stability and telepresence are both perfectly guaranteed. Thus, this paper investigates the method to online identify these condition variables and then employs them to enhance the telerobotic operations with multiple objectives. The designed method was implemented on a mobile manipulator and the experimental results demonstrated its effectiveness. Yunyi Jia, Ning Xi 0001, Huatao Zhang |
IROS | 1 |
| 2012 | Sensor-based redundancy resolution for a nonholonomic mobile manipulatorabstractRedundant nonholonomic mobile manipulators have wide range applications in civilian and military areas. The high redundancy provides high operation flexibility but also introduces redundancy resolution problems. The existing methods mainly focus on the off-line redundancy resolution, and most of them are based on single objective optimization. However, in order to efficiently accomplish a specific task, the dynamic environment information, task constraints and requirements should be considered. This paper investigates a new method which employs the onboard sensor information to resolve the redundancy online by realizing multiple objectives optimization. The effectiveness of this method is demonstrated by simulation results. Huatao Zhang, Yunyi Jia, Ning Xi 0001 |
IROS | 2 |
| 2011 | Design of single-operator-multi-robot teleoperation systems with random communication delayabstractMultiple robots can be teleoperated by a single operator to provide enhanced capacity and efficiency on accomplishing complicated tasks. The design of this kind of systems is challenging because simultaneously tele-controlling multiple robots exceeds the ability of one single operator. The intelligence and autonomy of the robots need to be integrated into the system. Besides, the communication between the operator and the multi-robot system and the communication among the multiple robots are both subject to communication constraints like time delays and packet losses. This paper designs a non-time based method to realize the single-operator-multi-robot system with random communication delay. The system is designed based on the non-time based teleoperation method and a proposed perceptive coordination method. The random delay problem and the problem of simultaneously tele-controlling multiple robots by a single operator are resolved. Experiments implemented on a multi-robot system illustrated the effectiveness of the design. Yunyi Jia, Ning Xi 0001, John Buether |
IROS | 1 |
| 2011 | Controlling telerobotic operations adaptive to quality of teleoperator and task dexterityabstractTelerobotic systems have been researched for decades due to their extensive applications in many civilian and military areas. Most research mainly focused on either the stability or telepresence of the telerobotic systems. Few studies have investigated the effects of the confidence of the teleoperator on the performance of the teleoperation. The confidence of the teleoperator is of significant importance to the efficiency and safety of the telerobotic systems. This paper proposes a concept named quality of teleoperator (QoT) to represent the confidence of the decisions and commands generated by the teleoperator. The value of QoT is computed based on a set of mental states of the teleoperator. Based on the QoT, a control adjustment mechanism is designed to enhance the efficiency and safety of the telerobotic systems. Experiments were implemented on a manipulator to demonstrate the effectiveness of the proposed method. Yunyi Jia, Ning Xi 0001 |
IROS | 1 |
| 2010 | Dynamic model and adaptive tracking controller for 4-Powered Caster VehicleabstractA new approach for adaptive torque distribution of 4-Powered Caster Vehicle (4-PCV) is presented on complex terrain without any additional sensor. The objective is that torques applied to wheels are dynamically redistributed based on the real time conditions of the whole wheel-ground interactions in order to track the desired trajectory. A novel approach based on the redundant actuated wheels is proposed to identify the status of the vehicle and the wheel slip ratio by only observing the velocity feedback from motors encoders. A dynamic model considering the wheel-ground interaction is described. Based on the slip ratio of the wheel joints and the null space of the operational space, control strategies are employed to redistribute the torques applied to the wheel joints so that each wheel can be self-adapted to meet a complex wheel-ground condition to eliminate slippage with high rate. Simulation results show the effectiveness of the proposed estimation approach and the performance of the torque distribution schemes. Yunyi Jia, Ning Xi 0001 |
ICRA | 2 |
| 2009 | Development and sensitivity analysis of a portable calibration system for joint offset of industrial robotabstractThis paper describes our updated system for industrial robot joint offset calibration. The system consists of an IRB1600 industrial robot, a laser tool attached to the robot's end-effector, a portable position-sensitive device (PPD), and a PC based controller. By aiming the laser spot to the center of position-sensitive-detector (PSD) on the PPD with different robot configurations, the developed system ideally implements our proposed calibration method called the virtual line-based single-point constraint approach. However, unlike our previous approach, the calibration method is extended to identify the offset parameters with an uncalibrated laser tool. The position errors of the PPD and the sensitivities of error in the PSD plane to the variation of joint angles are analyzed. Two different robot configuration patterns are compared by implementing the calibration method. Both simulation and real experimental results are consistent with the mathematical analysis. Experimental results with small (10−3−10−2) mean and standard deviation of parameters error verify the effectiveness of both the sensitivity analysis and the developed system. Ning Xi 0001, Erick Nieves-Rivera, Yunyi Jia, Bingtuan Gao |
IROS | 5 |