Event extraction is an important task in natural language processing that focuses on mining event-related information from unstructured text. Despite considerable advancements, it is still challenging to achieve satisfactory performance in this task, and issues like data scarcity and imbalance obstruct progress. In this paper, we introduce an innovative approach where we employ Large Language Models (LLMs) as expert annotators for event extraction. We strategically include sample data from the training dataset in the prompt as a reference, ensuring alignment between the data distribution of LLM-generated samples and that of the benchmark dataset. This enables us to craft an augmented dataset that complements existing benchmarks, alleviating the challenges of data imbalance and scarcity and thereby enhancing the performance of fine-tuned models. We conducted extensive experiments to validate the efficacy of our proposed method, and we believe that this approach holds great potential for propelling the development and application of more advanced and reliable event extraction systems in real-world scenarios.

Grasping and moving objects in a large cluster is a common real scenario. In such scenarios, tens of objects are adjacent to each other, even stacked layer by layer, so that simple grasp would not work due to obstruction. In this paper, we propose a well-designed strategy to use synergy of pushing and grasping to automatically push and grasp objects in a large tightly packed cluster of objects. Our strategy is to detect and grasp isolated graspable objects first before other actions. We then use a smart strategy that pushes objects at the narrowest edge of the clusters. For push action, the robot pushes the edge at the perpendicular direction relative to the cluster, thus improving the performance of isolation and minimizing collisions. We have conducted experiments in both simulation and real-world environments with more than 20 cluttered objects and demonstrated that our solution outperforms existing deep learning based methods, especially in challenging cases, and achieves significantly higher completion rate, grasp success rate, picked rate and efficiency.

The deep learning community has made rapid progress in low-level visual perception tasks such as object localization, detection and segmentation. However, for tasks such as Visual Question Answering (VQA) and visual language grounding that require high-level reasoning abilities, huge gaps still exist between artificial systems and human intelligence. In this work, we perform a diagnostic study on recent popular VQA in terms of analogical reasoning. We term it as Analogical VQA, where a system needs to reason on a group of images to find analogical relations among them in order to correctly answer a natural language question. To study the task in depth, we propose an initial diagnostic synthetic dataset CLEVR-Analogy, which tests a range of analogical reasoning abilities (e.g. reasoning on object attributes, spatial relationships, existence, and arithmetic analogies). We benchmark various recent state-of-the-art methods on our dataset and compare the results against human performance, and discover that existing systems fall shorts when facing analogical reasoning involving spatial relationships. The dataset and code will be publicly available to facilitate future research.

Reliable perception is essential for robots that interact with the world. But sensors alone are often insufficient to provide this capability, and they are prone to errors due to various conditions in the environment. Furthermore, there is a need for robots to maintain a model of its surroundings even when objects go out of view and are no longer visible. This requires anchoring perceptual information onto symbols that represent the objects in the environment. In this paper, we present a model for action-aware perceptual anchoring that enables robots to track objects in a persistent manner. Our rule-based approach considers inductive biases to perform high-level reasoning over the results from low-level object detection, and it improves the robot’s perceptual capability for complex tasks. We evaluate our model against existing baseline models for object permanence and show that it outperforms these on a snitch localisation task using a dataset of 1,371 videos. We also integrate our action-aware perceptual anchoring in the context of a cognitive architecture and demonstrate its benefits in a realistic gearbox assembly task on a Universal Robot.

Object permanence in psychology means knowing that objects still exist even if they are no longer visible. It is a crucial concept for robots to operate autonomously in uncontrolled environments. Existing approaches learn object permanence from low-level perception, but perform poorly on more complex scenarios, like when objects are contained and carried by others. Knowledge about manipulation actions performed on an object prior to its disappearance allows us to reason about its location, e.g., that the object has been placed in a carrier. In this paper we argue that object permanence can be improved when the robot uses knowledge about executed actions and describe an approach to infer hidden object states from agent actions. We show that considering agent actions not only improves rule-based reasoning models but also purely neural approaches, showing its general applicability. Then, we conduct quantitative experiments on a snitch localization task using a dataset of 1,371 synthesized videos, where we compare the performance of different object permanence models with and without action annotations. We demonstrate that models with action annotations can significantly increase performance of both neural and rule-based approaches. Finally, we evaluate the usability of our approach in real-world applications by conducting qualitative experiments with two Universal Robots (UR5 and UR16e) in both lab and industrial settings. The robots complete benchmark tasks for a gearbox assembly and demonstrate the object permanence capabilities with real sensor data in an industrial environment.

This paper presents a new mobile robotic platform (R-Mo) which can reduce unexpected variations in height as well as pitch angle of its main body while traversing rough terrains. As a measure for the smooth movement of mobile platform, the variations in height and pitch angle are chosen in this study. Then, the kinematic analysis on the Rocker-Bogie mechanism is carried out to investigate its variations in height and pitch angle on rough terrains. Based on this result, a new mobile platform is systematically designed by combining the Rocker-Bogie with the inverse four bar linkage. The extensive experiments are carried out by using the Rocker-Bogie mechanism and the proposed R-Mo against rough terrain, which validate that in comparison with the Rocker-Bogie mechanism, the average and maximum variations in height of proposed R-Mo are reduced by 12.72% and 5.96%, respectively, and the average and maximum variations in pitch angle of proposed R-Mo are considerably reduced by 65.87 % and 60.53 %, respectively.

The ability to overcome rough terrain is a main issue of mobile robots. However, as the speed of a robot increases, stability becomes another issue because it is directly related to the mobility of the robot. Without stability, the robot is exposed to risks of overturn. Therefore, a mobile robot needs to be not only maneuverable but also stable. We present a new mobile platform named “Rocker-Pillar” which enables a robot to overcome rough terrains with stability. The robot is composed of two sets of caterpillar tracks, four wheels, and a pair of two linkages. The robot can travel a rugged terrain at a speed 50m/min while maintaining stability, and can overcome complex terrains such as holes, steps without frontal sides, and stairs. The robot can overcome a hole 1.5 times larger than the wheel diameter of the robot and a step without the frontal side that is 1.25 times higher than wheel diameter.

Today, there are increased interest and various efforts in using cognitive architectures to control robotic platforms. Recent advances to essential capabilities in robots contributed to this trend, which tries to meet the increased demand for high-level control mechanisms. The tradition of cognitive architectures aims for general intelligence, and they have some great potential for use in robots that are now increasingly more capable of complex tasks. In this paper, we introduce one such architecture, ICARUS, used in a robotic environment to provide knowledge-based control for a humanoid robot, MAHRU. We show some experimental observations in the Blocks World domain.

In this paper, we propose a new representation for physical control -- teleoreactive logic programs -- along with an interpreter that uses them to achieve goals. In addition, we present a new learning method that acquires recursive forms of these structures from traces of successful problem solving. We report experiments in three different domains that demonstrate the generality of this approach. In closing, we review related work on learning complex skills and discuss directions for future research on this topic.