Tangram assembly, the art of human intelligence and manipulation dexterity, is a new challenge for robotics and reveals the limitations of state-of-the-arts. Here, we describe our initial exploration and highlight key problems in reasoning, planning, and manipulation for robotic tangram assembly. We present MRChaos (Master Rules from Chaos), a robust and general solution for learning assembly policies that can generalize to novel objects. In contrast to conventional methods based on prior geometric and kinematic models, MRChaos learns to assemble randomly generated objects through self-exploration in simulation without prior experience in assembling target objects. The reward signal is obtained from the visual observation change without manually designed models or annotations. MRChaos retains its robustness in assembling various novel tangram objects that have never been encountered during training, with only silhouette prompts. We show the potential of MRChaos in wider applications such as cutlery combinations. The presented work indicates that radical generalization in robotic assembly can be achieved by learning in much simpler domains. The code will be available https://robotll.github.io/MasterRulesFromChaos/.

Object transfer through throwing is a classic dynamic manipulation task that necessitates precise control and perception capabilities. However, developing dynamic models for unstructured environments using analytical methods presents challenges. In this study, we present DartBot, a robot that integrates tactile exploration and reinforcement learning to achieve robust throwing skills for nonrigid relatively small objects under the influence of moment of inertia which cause the object to spin in the air. Unlike traditional sim-to-real transfer methods, our approach involves direct training of the agent on a real hardware robot equipped with a high-resolution tactile sensor, enabling reinforced learning in a realistic and dynamic environment. By leveraging tactile perception, we incorporate pseudo-embeddings of the physical properties of objects into the learning process through tilting actions at two distinct angles. This tactile information enables the agent to infer and adapt its throwing strategy, resulting in improved accuracy when handling various objects and targeting distant locations. Furthermore, we demonstrate that the quality of a grasp significantly impacts the success rate of the throwing task. We evaluate the effectiveness of our method through extensive experiments, demonstrating superior performance and generalization capabilities in real-world throwing scenarios. We achieved a success rate of 95% for unseen objects with a mean error of 3.15 cm from the goal. A high-resolution video demo of our work is available athttps://youtu.be/KNFgDeLt-0g. Note to Practitioners—The industrial demand for precise and accurate object transfer beyond the robot’s maximum kinematic range is rapidly growing, necessitating advancements in robotic throwing manipulation to efficiently utilize resources. In this context, this paper contributes to the field by presenting a method for the transfer of deformable objects through overhand throwing, addressing the challenges of achieving precise control and perception capabilities by learning the complex physics of the task using raw data. The approach offers valuable insights and techniques for enhancing throwing manipulation tasks. The integration of high-resolution tactile sensing and reinforcement learning, while considering the influence of moment of inertia, opens up new possibilities for handling deformable objects in throwing tasks and developing dynamic models for such unstructured environments. The learned throwing policy is applied to a variety of previously unseen objects, demonstrating its effectiveness and ability to generalize in real-world throwing scenarios. The presented work holds applicability in various domains such as medical rehabilitation, logistic warehouses for packaging, and handling of urban waste. It shows promise in improving the precision, accuracy and efficiency of the throwing task. In the future, we aim to expand the current overhand throwing framework by incorporating rapid grasping in cluttered environments and enabling throwing to far distances with diverse target locations.

Detecting moving events produced by moving objects is a crucial task in the realms of autonomous driving and mobile robots. Moving objects have the potential to create ghost artifacts in mapped environments and pose risks to autonomous navigation. LiDAR serves as a vital sensor for autonomous systems due to its ability to provide dense and precise range measurements. However, existing LiDAR datasets often lack sufficient discussion on the motion labeling of moving objects, containing only a limited representation of moving entities within a single scene. Furthermore, the methodologies for Moving Event Detection (MED) on LiDAR sensors have not been comprehensively explored or evaluated. To address these gaps, this study focuses on constructing a diverse LiDAR moving event dataset encompassing multiple scenes with a high density of moving objects. A thorough review of current MED techniques is conducted, followed by the establishment of a performance benchmark based on evaluating these methods using our dataset. Additionally, part sequences of the dataset are utilized to host an online MED competition, aimed at fostering collaboration within the research community and advancing related studies.

As automation technologies advance, the need for compact and multi-modal sensors in robotic applications is growing. To address this demand, we introduce CompdVision, a novel sensor that employs a compound-eye imaging system to combine near-field 3D visual and tactile sensing within a compact form factor. CompdVision utilizes two types of vision units to address diverse sensing needs, eliminating the need for complex modality conversion. Stereo units with far-focus lenses can see through the transparent elastomer for depth estimation beyond the contact surface. Simultaneously, tactile units with near-focus lenses track the movement of markers embedded in the elastomer to obtain contact deformation. Experimental results validate the sensor’s superior performance in 3D visual and tactile sensing, proving its capability for reliable external object depth estimation and precise measurement of tangential and normal contact forces. The dual modalities and compact design make the sensor a versatile tool for robotic manipulation.

Predicting the strength of promoters and guiding their directed evolution is a crucial task in synthetic biology. This approach significantly reduces the experimental costs in conventional promoter engineering. Previous studies employing machine learning or deep learning methods have shown some success in this task, but their outcomes were not satisfactory enough, primarily due to the neglect of evolutionary information. In this paper, we introduce the Chaos-Attention net for Promoter Evolution (CAPE) to address the limitations of existing methods. We comprehensively extract evolutionary information within promoters using merged chaos game representation and process the overall information with modified DenseNet and Transformer structures. Our model achieves state-of-the-art results on two kinds of distinct tasks related to prokaryotic promoter strength prediction. The incorporation of evolutionary information enhances the model's accuracy, with transfer learning further extending its adaptability. Furthermore, experimental results confirm CAPE's efficacy in simulating in silico directed evolution of promoters, marking a significant advancement in predictive modeling for prokaryotic promoter strength. Our paper also presents a user-friendly website for the practical implementation of in silico directed evolution on promoters. The source code implemented in this study and the instructions on accessing the website can be found in our GitHub repository https://github.com/BobYHY/CAPE.

This paper tackles the task of singulating and grasping paper-like deformable objects. We refer to such tasks as paper-flipping. In contrast to manipulating deformable objects that lack compression strength (such as shirts and ropes), minor variations in the physical properties of the paper-like deformable objects significantly impact the results, making manipulation highly challenging. Here, we present Flipbot, a novel solution for flipping paper-like deformable objects. Flipbot allows the robot to capture object physical properties by integrating exteroceptive and proprioceptive perceptions that are indispensable for manipulating deformable objects. Furthermore, by incorporating a proposed coarse-to-fine exploration process, the system is capable of learning the optimal control parameters for effective paper-flipping through proprioceptive and exteroceptive inputs. We deploy our method on a real-world robot with a soft gripper and learn in a self-supervised manner. The resulting policy demonstrates the effectiveness of Flipbot on paper-flipping tasks with various settings beyond the reach of prior studies, including but not limited to flipping pages throughout a book and emptying paper sheets in a box. The code is available here: https://robotll.github.io/Flipbot/.

Single-cell high-throughput chromatin conformation capture technologies (scHi-C) has been used to map chromatin spatial organization in complex tissues. However, computational tools to detect differential chromatin contacts (DCCs) from scHi-C datasets in development and through disease pathogenesis are still lacking. Here, we present SnapHiC-D, a computational pipeline to identify DCCs between two scHi-C datasets. Compared to methods designed for bulk Hi-C data, SnapHiC-D detects DCCs with high sensitivity and accuracy. We used SnapHiC-D to identify cell-type-specific chromatin contacts at 10 Kb resolution in mouse hippocampal and human prefrontal cortical tissues, demonstrating that DCCs detected in the hippocampal and cortical cell types are generally associated with cell-type-specific gene expression patterns and epigenomic features. SnapHiC-D is freely available at https://github.com/HuMingLab/SnapHiC-D.