Developing robot software is difficult for most software engineers as it requires multi-discipline knowledge such as robotics, AI, and software engineering.Robot Operating Systems (ROS) provides a software development framework and lots of reusable ROS Nodes that encapsulate various robotics functions, which can simplify robot software development in terms of software reuse.However, searching and reusing required ROS Nodes from thousands of ROS Nodes is still challenging due to the scattered distribution of ROS Node information and the need for adequate search methods.In this paper, we present an effective method to construct a ROS Node knowledge graph in support of searching and reusing ROS Nodes.Our method uses multiple data sources, including open-source ROS software in Github and ROS wiki community.We extract two-tuple functional information and task-related noun phrases from the ROS Node description and ROS communication interactions from the ROS Node source code.The constructed ROS Node knowledge graph (RNKG) contains 14,065 entities and 15,767 relations.It provides rich semantic information to comprehensively and precisely describe ROS Nodes, their services, and related interaction topics and messages.

The Robot Operating System (ROS) has become the most popular robot development framework in the last few years, which has loosely coupled structure and provides remote communications between different component nodes. The ROS messages are critical to bridge the communication channels and clearly define the data structures. The developers can use the standardized or user-customized ROS message types to construct a communication channel between two component nodes uniquely. However, it becomes increasingly difficult for developers to find the required ROS message type from thousands of diverse ROS message types in ROS-based robotic software development. Finding the proper ROS message type is a non-trivial task because developers may hardly know the exact names of required ROS messages but only has a rough knowledge of the task domain features. To tackle this challenge, we construct a novel ROS Message Knowledge Graph (RMKG) with 4543 entities and 14320 relationships, including all ROS message types and message packages. We take the shortest path algorithm to search ROS message in RMKG by searching with ROS message feature or ROS message package and visualize the subgraph structure of the search results. Moreover, we develop a ROS message package library that supports fuzzy queries to find the required message package. A comprehensive evaluation of RMKG shows the high accuracy of our knowledge construction approach. A user study indicates that RMKG is promising in helping developers find suitable ROS message types for robotics software development tasks. An effect evaluation of message package fuzzy query shows the good effects of our fuzzy query method under different situations.

Robots operating in open environments expect to have robust plans to achieve tasks successfully under environment uncertainties. However, both partial observability and dynamics of environment states have significantly decreased the robustness of task achievement, making robot task planning much more challenging. The partially observable states require the robot to obtain observations for optimally acting of the task goal. Also, state dynamics expects the robot to continuously observe surroundings for acting safely. Both challenges practically demand the purposeful and tight interactions between robot state-changing actuating actions and sensor-based observation actions. This paper proposes a novel model of Adjoint Sensing and Acting (ASA) that explicitly defines two parallel and sequential interaction schemes between actuating and observation actions, as well as an extended Behavior Tree for a concrete implementation of above schemes. We further propose an interleaving task planning approach for planning ASA-style plans, which integrates a deliberative POMDP planner for pursuing task goals, and a reactive Behavior Tree executive for fast responding to unexpected events. We experimentally demonstrate that ASA interaction schemes are practical and applicable to model and plan the open environment robot tasks. The plans from the interleaving task planning approach are both reactive in run-time response and efficient in task achievement.

Over the past several years, the Robot Operating System (ROS), has grown from a small research project into the most popular framework for robotics development.It offers a core set of software for operating robots that can be extended by creating or using existing packages, making it possible to program robotic software that can be reused on different hardware platforms.With thousands of packages available per stable distribution, encapsulating algorithms, sensor drivers, etc., it is the de facto middleware for robotics.However, finding the proper ROS package is a nontrivial task because ROS packages involve different functions and even with the same function, there are different ROS packages for different tasks.So it is timeconsuming for developers to find suitable ROS packages for given task, especially for newcomers.To tackle this challenge, we build a ROS package knowledge graph, ROSKG, including the basic information of ROS packages and ROS package characteristics extracted from text descriptions, to comprehensively and precisely characterize ROS packages.Based on ROSKG, we support ROS packages search with specific task description or attributes as input.A comprehensive evaluation of ROSKG shows the high accuracy of our knowledge construction approach.A user study shows that ROSKG is promising in helping developers find suitable ROS packages for robotics software development tasks.

In real-world environments, robotic task planning is expected to handle both partial observability and unexpected dynamics of the environment. A robust plan for the task requires the robot's observation actions to concurrently run with the task actions, to observe and adapt to environmental changes. The Partially Observable Markov Decision Process (POMDP) has been widely applied for planning under partially observable domains. For realistic robotic tasks, however, the POMDP model and planning algorithm are quite restrictive and unrealistic. One limitation is that task actions are modelled as atomic entities that only have endpoint effects, with no conditions specified at arbitrary points during task action execution. Also, the observation is obtained only after each task action execution, with no intermediate observations and decision-making during task action execution. To mitigate the limitations of POMDP planning, this paper first proposes an Adjoint Action Model (AAM) that explicitly defines the continuous interaction between robot's observation and task actions. Then we extend the POMDP task action model with intermediate invariant conditions which specifies the runtime properties of action execution. Finally, we propose the AAM-extended POMDP planning approach which handles observation action planning and task replanning for task action execution. We experimentally demonstrate that the plan from our proposed approach is more effective and robust to cope with the environment dynamics, comparing with the standard POMDP planning approach.

To support robust task execution in open environment, autonomous robots (AR) should include reactive capabilities to cope with the dynamics and uncertainties from real-world environment.The uncertainties pose great challenges for robots being sensitive to the environmental changes and flexible to adjust self-behaviors.To this end, this paper aims to improve the sensing and acting capabilities of autonomous robots by novel behavioral theories, observation modes and software architectures.Specifically, this paper has three main contribution: (1) presents an accompanying model that specifies a novel accompanying pattern for interacting robot behaviors;(2) proposes four types of accompanying observation modes that coordinate multiple robot sensing behaviors; (3) proposes a concrete multi-agent software architecture that implements aforementioned accompanying model and accompanying observation modes.To demonstrate the applicability and validity of our accompanying modes and MAS-based software architecture, this paper conducts a case study to implement a domestic service example, which requires the robot to run in a highly dynamic environment and can adapt its behaviors to unexpected situations.

Autonomous robot software is an extremely complex system that drives robots to operate in an open and dynamic environment to accomplish tasks. This requires such software to be designed with effective control models to support continuous and flexible feedback. To this end, this paper presents a dualloop control model called D-SMPA and corresponding software framework that adopts several design principles to support the development and running of autonomous robot. Our proposed model explicitly abstracts the robot behaviors into observation-oriented and task-oriented types, and provides a dual-loop control to coordinate these behaviors. Control mechanisms in D-SMPA are proposed to support a tight integration and accompanying cooperation between observation and task behaviors so that the task accomplishment can get required feedback of sensors. Our approach aims to enrich the capabilities of autonomous robot software by obtaining on-demand feedback from multiple sources, and at the same time to simplify software development by separating complex behaviors of robots. We present a multi-agent software framework called AutoRobot to implement the proposed control model and provide tools to support the development of autonomous robot software. Our case study validates the effectiveness and applicability of our proposed approach and framework.

Autonomous robots are increasingly applied in realworld environments, and expect to execute plans robustly to accomplish the assigned tasks in the presence of dynamics and uncertainties of the changing environment. The robustness of plan execution requires the robot to keep aware of plan execution status and to adapt the plan towards possible execution contingencies. Such requirements pose a great challenge for autonomous robot software in terms of the abstraction model over robot behavior patterns. Conventional abstraction models for robot behaviors generally follow the sense-model-planact and behavior-based paradigms, which show limitations in tight integration with sensory inputs and tracking execution traces of robot plans. This paper proposes an accompanying behavior model that considers robot behaviors as task-oriented and observation-based types with diverse aims, and develops the run-time mechanisms to facilitate collaboration between two types of behaviors. Additionally, we implement the model by the multi-agent approach which develops the robot software as a multi-agent system. To demonstrate the feasibility and applicability of proposed model, we conduct a case study by implementing a typical example of service scenarios, e.g., a robot that autonomously picks up and drops off dishes for remote guests in the open and dynamic environment.

Multi-robot systems (MRS) working in open and dynamic environments are expected to deal with uncertain arrival of new tasks and environment changes, by repeatedly adapting the current task allocation and schedule, in order to maintain its performance (e.g., total utility, balance, etc.). This paper presents an adaptive approach to multi-robot task allocation (MRTA), which combines two adaptive measures corresponding to different levels of a MRS: (1) re-allocating at inter-robot level, for balancing task allocation, and improving total utility of the MRS, and (2) re-scheduling at intra-robot level, for maintaining each robot's utility against the influence of both re-allocating and environment changes. Our approach is expected to have significantly higher adaptation power than both re-allocating only and re-scheduling only cases. An experiment is conducted to evaluate our approach's capability of improving balance and total utility of the MRS, under different environment settings and different combinations of re-allocating and re-scheduling.

Great attentions have been put on the programming technologies to construct robot software in both academic research and industry due to the increasingly wide applications of robots in various areas, and potential challenges resulting from the complexity of robot software. In the past years, diverse programming technologies and languages have been designed to support the development of robot software. However, with robot applications and requirements change, to develop robot software remains a great challenge, especially when robots are widely used in open environment and expected to provide better and friendly services for human beings. This paper aims at analyzing the technical requirements for designing robot programming language, presenting the roadmap of the robot programming language and discussing its trends and potential challenges.