Language-Conditioned Manipulation (LCM) facilitates human-robot interaction via Behavioral Cloning (BC), which learns control policies from human demonstrations and serves as a cornerstone of embodied AI. Overcoming compounding errors in sequential action decisions remains a central challenge to improving BC performance. Existing approaches mitigate compounding errors through data augmentation, expressive representation, or temporal abstraction. However, they suffer from physical discontinuities and semantic-physical misalignment, leading to inaccurate action cloning and intermittent execution. In this paper, we present Continuous vision-language-action Co-Learning with Semantic-Physical Alignment (CCoL), a novel BC framework that ensures temporally consistent execution and fine-grained semantic grounding. It generates robust and smooth action execution trajectories through continuous co-learning across vision, language, and proprioceptive inputs (i.e., robot internal states). Meanwhile, we anchor language semantics to visuomotor representations by a bidirectional cross-attention to learn contextual information for action generation, successfully overcoming the problem of semantic-physical misalignment. Extensive experiments show that CCoL achieves an average 8.0% relative improvement across three simulation suites, with up to 19.2% relative gain in human-demonstrated bimanual insertion tasks. Real-world tests on a 7-DoF robot further confirm CCoL’s generalization under unseen and noisy object states.

The unique tilt-servo mechanism of the tilt-rotor unmanned aerial vehicle (UAV) facilitates seamless transitions between multi-rotor and fixed-wing modes, enhancing both flexibility and maneuverability. However, traditional modeling methods, which treat each flight mode independently, fail to provide a unified dynamic representation, limiting the accurate description of aerobatic maneuvers during mode transitions. This paper introduces a novel modeling approach based on transient Computational Fluid Dynamics (CFD) to capture the aerodynamics of the transition mode, resulting in a multimodal, consistent dynamics model. This model simplifies the mathematical representation for specific tilt angles, ensuring compatibility with both multi-rotor and fixed-wing dynamics, and accurately describes aerobatic maneuvers. An autonomous feedback motion planning method, utilizing third-order Bézier curves for angular velocity planning, is applied, along with a modal switching strategy to address the limitations of traditional fixed-wing UAVs. The feasibility of this method was validated through numerical simulations, hardware-in-the-loop simulations, and outdoor flight experiments of a tilt-rotor UAV performing the Cobra maneuver in transition mode.

Open-environment machine learning is crucial for category discovery in synthetic aperture radar automatic target recognition (SAR ATR). However, SAR ATR toward intelligent applications requires addressing not only open-world distributions but also data imbalance. In this letter, we first propose the distribution-unknown generalized category discovery (DUGCD) problem and introduce the balanced knowledge distillation-contrastive learning (BKD-CL) framework, which includes the frequency attention ViT (FAViT) module and a multilayer perceptron (MLP) projection head. Second, we optimize the loss function using both supervised and self-supervised contrastive learning methods to learn feature representations from labeled and unlabeled data. We also implement self-distillation and entropy regularization to facilitate knowledge training for a parameterized classifier aimed at classification learning. Finally, to tackle the issue of data imbalance, we introduce balanced knowledge distillation, which selectively transfers knowledge using weighted coefficients to address the poor recognition performance caused by imbalanced data distributions. Extensive experiments conducted on the MSTAR dataset demonstrate the superiority of our proposed method.

In robotic orthopedic surgery, the optical tracking system (OTS) is typically placed in a fixed location. In surgery, the OTS’s line of sight is likely to be blocked. This will interrupt the navigation and affect surgical safety. To solve this occlusion problem, an RGB-D camera is used to detect possible occluders and a navigation robot is utilized to actively adjust the OTS viewpoint before occlusion occurs. To guarantee the applicability, the occluder is enveloped using a convex polytope, and a two-phase optimization method is proposed based on duality of convex optimization. The effectiveness of the proposed method is verified via simulations and experiments. Experimental results show that the proposed method can avoid occlusion between the OTS and the occluder, and the targets are located near the center of the measurement volume. This active navigation guarantees the continuity of intraoperative navigation, and thus helps to improve the safety in robotic orthopedic surgery.

Autonomous, accurate, and dynamic 3-D reconstruction for wide-area environments is crucial for unmanned aerial vehicle monitoring and rescue tasks, however, when conducted in an unknown complex terrain, the reconstruction result obtained from a single flight suffers poor quality. In this article, we present an Active Iterative Optimization framework for trajectory planning and visual reconstruction. Firstly, the trajectory is planned under the photogrammetric constraints based on rough terrain. Due to the visual field deviation caused by pose error during actual flight, the view loss evaluation is established and keyframes are selected to conduct 3-D reconstruction. A comprehensive metric is designed to quantitatively evaluate reconstruction effect without ground truth. The point cloud is then rasterized and divided into normal or low-scoring region according to the evaluation metric. In the next iteration, trajectory is replanned in low-scoring region to purposefully optimize the point cloud of local area. Thus the reconstruction result can be iteratively optimized. We validated the effectiveness of the proposed framework in simulation and physical experiments.

Hysteresis has posed significant challenges to the modeling and control of flexible endoscopic robots, which impedes the advancement of automated endoscopic operation. Despite numerous hysteresis modeling approaches aimed at improving accuracy, there are still several unresolved issues, such as inappropriate model selection and non-ideal assumption of noise. Focusing on these challenges, a novel reinforcement active modeling (RAM) scheme is proposed in this paper. By incorporating reinforcement learning, this method augments an Extended Kalman Filter (EKF)-based active modeling strategy, which improves the insensitivity and generalization ability to non-Gaussian noise that is not introduced in training. Finally, a series of comparative experiments are conducted on the self-built flexible endoscopic robot to validate the improvement achieved by the proposed scheme. Compared with some widely-applied methods, the proposed scheme achieved at least 63.8% improvement in the root mean square error (RMSE) in modeling accuracy under Gaussian noise conditions, and at least 36.5% improvement in RMSE under Poisson noise conditions.

Intraoperative optical navigation is widely utilized in robotic surgery systems. Typically, the observation pose of the optical tracking system (OTS) is manually adjusted and then fixed throughout the surgery. However, fixed OTS suffers from limited measurement volume (MV) and visual interferences, making consistent navigation challenging in clinics. In this paper, an operation-navigation dual-robot collaborative system is proposed for orthopedic surgeries. An extra navigation robot is introduced to actively adjust the observation pose of the OTS. A collaborative preoperative planning method is proposed for this dual-robot system, including osteotomy path planning of the operation robot and collaborative planning of the navigation robot. Firstly, osteotomy paths of the operation robot are generated according to the surgery regulations and the geometric features of the vertebral foramen. Secondly, based on the generated osteotomy paths, the collaborative planning of the navigation robot is formulated into a multi-objective optimization problem to find the optimal poses of the OTS for each osteotomy plane. Compared with fixed OTS, active navigation is capable of keeping all the targets within the MV of the OTS throughout the surgery. Semi-laminectomy on a human spine phantom is adopted as an example to experimentally evaluate the effectiveness of the proposed method.Note to Practitioners—As the demand for robot-assisted surgery is increasing, the precision of operation has become one of the key safety requirements. Preoperative planning provides guidance for the surgeon, and intraoperative navigation monitor the status of the lesion and the surgical tool in real-time. In conventional intraoperative optical navigation, the OTS is manually adjusted and remains stationary. However, the limited MV and the visual interferences introduce risks and uncertainties to the surgical system. In order to address the limitations of fixed OTS, an operation-navigation dual-robot collaborative system is proposed for orthopedic surgeries, which is composed of a surgical operation module and an active navigation module. The navigation robot is used to actively adjust the pose of the OTS. A collaborative preoperative planning for the operation-navigation dual-robot orthopedic surgery system is proposed in this paper. The effectiveness of the proposed method is verified on a human spine phantom. Experimental results show that the active navigation provides more freedom to the overall system by freely adjusting the OTS, which ensures the stability of the surgical navigation. In future work, efforts will be directed toward the identification and avoidance of the obstacle.

In this paper, we address the multiple quadrotor UAVs trajectory planning optimization problem for large-scale, persistent, high-depth visual coverage tasks in three-dimensional (3-D) terrain environment. To minimize the overall energy expenditure of the UAVs for accomplishing a task, we set up an air-to-ground collaborative system which introduces base stations to hold and recharge UAVs. The system is formulated as an integer programming, and solved by a novel hierarchical reinforcement learning trajectory planning algorithm (RL-TP), in which the paths are obtained by reinforcement learning method, and then the trajectories are obtained by Bézier curve method. Both simulation and physical experiments show that RL-TP can effectively improve the efficiency and persistence of aerial visual coverage task.Note to Practitioners—While the multi-rotor UAV has been an important means for field monitoring, it suffers the problem of short battery life a lot. To make it more efficient and persistent, we use multiple UAVs and introduce ground base stations to charge the UAVs. The scenario is formulated as an air-to-ground collaborative system, and the motion planning strategy is to minimize the energy consumption. We propose a hierarchical collaborative coverage reinforcement learning trajectory planning algorithm (RL-TP) to solve it. We carry out both simulation and physical field experiments, and compare RL-TP with other popular methods. The experimental results show that the system is feasible and RL-TP performs well in both time efficiency and energy consumption. In future research, we will introduce unmanned ground vehicles to replace the stationary ground base stations to make the air-to-ground collaborative system more powerful and flexible.

Prior methodologies have disregarded the diversities among distinct degradation types during image reconstruction, employing a uniform network model to handle multiple deteriorations. Nevertheless, we discover that prevalent degradation modalities, including sampling, blurring, and noise, can be roughly categorized into two classes. We classify the first class as spatial-agnostic dominant degradations, less affected by regional changes in image space, such as downsampling and noise degradation. The second class degradation type is intimately associated with the spatial position of the image, such as blurring, and we identify them as spatial-specific dominant degradations. We introduce a dynamic filter network integrating global and local branches to address these two degradation types. This network can greatly alleviate the practical degradation problem. Specifically, the global dynamic filtering layer can perceive the spatial-agnostic dominant degradation in different images by applying weights generated by the attention mechanism to multiple parallel standard convolution kernels, enhancing the network's representation ability. Meanwhile, the local dynamic filtering layer converts feature maps of the image into a spatially specific dynamic filtering operator, which performs spatially specific convolution operations on the image features to handle spatial-specific dominant degradations. By effectively integrating both global and local dynamic filtering operators, our proposed method outperforms state-of-the-art blind super-resolution algorithms in both synthetic and real image datasets.

Medical segmentation of cochlear images aims to identify the area of the cochlea in a set of CT slices. The shape of cochlea will vary a quite in different CT slicing levels, and the relevant dataset has a higher labeling cost. This will lead to segmentation results with edge discontinuity when we implement supervised algorithm under few samples. In order to solve the problem of a small number of labeled images, this paper proposes a semi-supervised model called GSDNet which is based on GAN, which captures the features of the cochlear image without labels, so as to achieve high performance for processing fewer sampled data. To further improve the generalization of the model, we adopt a training method that allows the model to gradually distinguish between real images and fake images. In addition, in order to solve the problem of local noise interference and discontinuous segmentation results, we introduce a label discrimination network to force the distribution of generated results from segmentation network to align with the true label distribution, so that the edges of the segmentation results are continuous and the shape is more accurate. Finally, we conduct a segmentation experiment of the cochlear region containing 30 slices about cochlea data, and compare different cutting-edge methods. The method proposed in this paper achieves higher performance on the dice index.

Most reference-based image super-resolution (RefSR) methods directly leverage the raw features extracted from a pretrained VGG encoder to transfer the matched texture information from a reference image to a low-resolution image. We argue that simply operating on these raw features neglects the influence of irrelevant and redundant information and the importance of abundant high-frequency representations, leading to undesirable texture matching and transfer results. Taking the advantages of wavelet transformation, which represents the contextual and textural information of features at different scales, we propose a Wavelet-based Texture Reformation Network (WTRN) for RefSR. We first decompose the extracted texture features into low-frequency and high-frequency sub-bands and conduct feature matching on the low-frequency component. Based on the correlation map obtained from the feature matching process, we then separately swap and transfer wavelet-domain features at different stages of the network. Furthermore, a wavelet-based texture adversarial loss is proposed to make the network generate more visually plausible textures. Experiments on four benchmark datasets demonstrate that our proposed method outperforms previous RefSR methods both quantitatively and qualitatively. The source code is available at https://github.com/zskuang58/WTRN-TIP.