In lunar exploration missions, tasks such as target recognition or obstacle detection always encounter significant challenges due to the quality of images and the limited availability of training samples. Typically, existing methodologies can only recognize and locate a limited number of targets within simple scenarios, and models developed within simulated environments lack practical applicability. To alleviate above issues, an image fragments-based twofold learning strategy has been conceptualized. This strategy facilitates the decoupling of target features from environmental attributes through processes such as image fragmentation, confidence evaluation, and domain transfer, thereby achieving enhanced recognition accuracy across diverse lighting conditions. To provide more accurate localization and identification of a wider variety of targets,$\mathrm {\omega }$-CIoU loss is introduced to address the anomalies in prediction box scale and target count induced by flare and shadow features. Moreover, the establishment of the Real Chang’e Lunar Landscape Dataset, the most extensive public dataset for lunar surface target recognition in terms of sample quantity and diversity, provides an invaluable experimental foundation for future research in this field. Comprehensive experimental results on the RCLLD and public dataset ALLD demonstrate that the FTLS can significantly boosts recognition precision and generalization capabilities without escalating model complexity, outperforming prevailing target recognition methodologies for lunar exploration rovers.Note to Practitioners—This research is inspired by the need for high-precision, robust target recognition and avoidance systems for lunar rovers during lunar surface exploration missions. These systems are crucial for deployment on rovers with limited computational resources and energy, providing a convenient platform for future lunar terrain surveys, lunar base site selection, and mineral collection tasks. However, existing target recognition systems usually require training on extensive actual lunar datasets and often suffer from significantly reduced accuracy in areas with severe lighting changes, which decreases reliability on the lunar rover platform—an unacceptable risk for the cost-intensive lunar exploration missions. To address these challenges, this paper proposes a twofold learning strategy based on image fragments that trains on limited actual lunar data, offers resistance to flare and shadow interference, and its effectiveness is validated on both actual lunar and simulated datasets.

Surface 3-D mapping of small bodies is crucial for planning in situ deep-space explorations, yet current approaches rely heavily on manual processing and significantly increase time costs. To achieve autonomous and efficient 3-D mapping of small bodies, we propose a visual-based autonomous 3-D mapping framework that integrates neural radiance fields (NeRFs) with deep reinforcement learning (DRL). We introduce a NeRF variant, which is built within the optimization loop of DRL, for instant 3-D mapping and quantitative uncertainty estimation of the global mapping. We employ the uncertainty estimation to optimize the DRL policy, thereby predicting the next best state that minimizes uncertainty in surface mapping. Moreover, we introduce a new six degrees of freedom (6 DoF) simulation environment that simulates the visual-based 3-D mapping process of small bodies, and demonstrate that our framework significantly enhance 3-D mapping quality. Our framework provides a new technological direction for small body exploration missions, potentially enhancing efficiency while also advancing operational reliability.

Efficient visual localization of lunar rovers is essential for long-range autonomous exploration missions and the construction of the International Lunar Research Station. Given the communication delay and bandwidth limitations between the Earth and the Moon, we propose a novel cross-view localization framework that autonomously registers a single rover image to an orbital image. We developed a bird’s-eye-view (BEV) feature synthesis method that integrates geometric projection, cross-scale feature transfer (CSFT), and contour guidance mechanism (CGM). The basic principle involves first projecting rover features into the BEV perspective through geometric projection, then reconstructing BEV features in reference to orbital features using CSFT. This process is guided by CGM, enhancing the expression of rich terrain contours in BEV and orbital features, thereby improving the viewpoint and scale consistency of cross-view features. By conducting dense spatial correlation searches between BEV and orbital features, we can accurately estimate the position of the lunar rover. Additionally, we introduce a lunar surface simulation environment and construct the lunar cross-view localization (LCVL) simulation dataset based on this environment to demonstrate the framework’s effectiveness. Our research offers a new solution for rover localization, potentially improving the efficiency of future exploration missions.

Tracking-and-inserting a moving peg using a robot manipulator is a challenging task in manufacturing. In the past decades, various visual-based methods have been proposed for robotic manipulating static targets, which usually ignore the time delay in robot command transmission and image processing. However, for tracking and inserting a moving peg, time delays cannot be overlooked because they can reduce the tracking performance and even cause manipulations to fail. In this article, a robot visual-based control with a time-delay compensator is presented to solve the problem of inserting a moving peg. The time-delay compensator was designed using radial basis function neural network, a feedback compensator aimed at eliminating the tracking errors caused by the time delays. Thus, we could manipulate a moving object using a commercial industry robot, even with the time-variant delays in the control loop. Furthermore, the visual-based controller with the pseudoinverse image Jacobian matrix was designed using a linearization model. Thus, the matrix could be efficiently updated using the model. In the experiment, we inserted a peg into a moving hole using an eye-in-hand robot with precision.

Object grasping or localization is an essential stage in automatic manufacturing processes. In general, stable grasping of 3-D objects is achieved by a multifinger hand. Different from the existing approaches, in this paper, we propose a novel vision-based method for grasping 3-D objects with a simple 2-D gripper. We describe the grasping strategy, the range of 3-D objects which can be grasped, and the region of grasping orientations which can guarantee successful actions. The proposed method facilitates the freedom of grasping orientation, which endows the strategy with a larger application range. Furthermore, we also explain the potential applications of the proposed method in grasping 3-D objects with other 2-D grippers. Although the proposed method uses a 2-D gripper and 2-D vision information, it can be regarded as a 3-D grasping approach due to the following facts. 1) The objects to be grasped are 3-D. The grasping process and the grasping orientations are analyzed and given in a 3-D space. 2) The contact points between the object and the gripper are not always in a 2-D plane. In the proposed method, the concept attractive region in configuration space, which was proposed in the previous work, is introduced to analyze the whole grasping process. Compared with other traditional methods, the proposed method directly gives the range of objects which can be grasped as well as the region of grasping orientations. By establishing the relationship between our method and traditional ones, it is proved that the bottom of the attractive region corresponds to a stable grasp.

In general, localization is a very important step in the manufacturing process, which can be considered as a prior process of assembly, machining, transportation, etc. Localization can be achieved with or without sensors. Compared with localization with sensors, localization without sensors can be more reliable, cheaper, and can have lower requirements on the environment. For example, localization without sensors can be achieved in a dark environment. However, it is more restrictive to the condition of the system. Localization of 2-D objects without sensors has been deeply investigated. Some theoretical results on 3-D-object localization without sensors have been given. On the other hand, some work has been carried out that tries to find methods to reduce uncertainties in the 3-D orientation of a polyhedron (i.e., the orientation of the polyhedron in a 3-D space). However, to the best of our knowledge, no practical and effective methods have been proposed, so far, to localize a polyhedron from any initial 3-D orientation to a unique 3-D orientation without sensors. This paper aims to find conditions and strategies for 3-D objects to be rotated from an unknown initial stable state to a unique stable state without sensors. The main contributions of this paper are given as follows: 1) It is discovered and proved that there are two classes of 3-D objects that can be rotated into a unique state from an arbitrary initial state without sensory feedback. 2) For these two classes of objects, the practical strategies are presented, and one example for each class is given to show the validity of the strategy. 3) Based on the above results, the robotic system and localization operations are illustrated, and some experimental results are given.

Sensorless localization of 3D objects has been a significant research topic for many years. Researchers have focused on this problem from both theoretical and practical perspective where the goal is to reduce uncertainties in the orientation of a 3D object. However, to the best of our knowledge, no effective practical methods have been proposed so far to localize a polyhedron from any initial orientation to a unique orientation without sensors. In our previous work [1], two broad classes of 3D objects have been introduced, which can be localized from an arbitrary state to a unique state on a flat plane (the surface resting on the flat plane is established) without sensors. In this paper, a much broader class of polyhedra is introduced, which can be localized to a unique state without sensors. The main contributions of this paper are given as follows: • It is found that a polyhedron with an arbitrary initial state on the flat plane can be rotated to a fixed orientation (the orientation of the surface resting on the flat plane is fixed), provided that the polygon corresponding to each surface of the polyhedron can be oriented to a unique orientation in a 2D space. The method of rotating the polyhedron to a fixed orientation is given. • Base on the above result, both conditions and the strategy are given for a polyhedron to be localized to a unique state. • An example is given to show the validity of the strategy.