To ensure future high-quality and reliable massive communication during 6G uplink transmissions, we propose a conditional diffusion model-driven massive MIMO detector, which can iteratively estimate channels and detect data for the uplink multiuser access. This approach utilizes a generative diffusion model to learn the score function of the joint posterior by integrating the prior distribution with the likelihood derived from the transmission model. The prior distribution is obtained either by learning from channel statistics or through analytical derivation from the symbol constellation. By employing noise matching initialization and an asynchronous annealed Langevin dynamics (ALD) sampling scheme, the receiver alternates efficiently between score-based channel estimation and data detection, thus avoiding traps of local minima. Simulation results demonstrate that this iterative diffusion process outperforms Bayesian-based and existing synchronous ALD channel estimation and data detection schemes in multiuser uplink scenarios.

The co-design problem of dual event-triggered (DET) mechanism and polynomial dynamic output-feedback (PDOF) controller is investigated for positive polynomial fuzzy systems (PPFSs) with uncertainty and disturbance constraints. Specifically, a 1-norm DET mechanism compatible with the positivity of PPFSs is proposed to asynchronously update measurement outputs and PDOF control signals. However, synthesizing this DET-PDOF controller proves challenging due to the coupling of multiple unknown PDOF controller gain matrices within the positivity and stability conditions, which results in complex nonconvex terms. By introducing auxiliary variables and constraints, sufficient conditions for DET-PDOF controller solution are given to ensure both the $L_{1}$ -gain performance and strict positivity of PPFSs with uncertainty and disturbance. Moreover, existing stability analysis results that ignore membership functions (MFs) tend to be conservative, implying that the obtained DET-PDOF controller is effective only within a limited triggered threshold range, leading to worse transmission performance. Therefore, a multivariate optimization method based on an improved genetic algorithm (IGA), which accounts for the system states and PDOF controller variables, is developed to substantially expand the admissible DET threshold range while effectively suppressing dual-triggering frequencies. Finally, a numerical example and a two-linked tank system with parameter uncertainty are provided to validate the feasibility of the proposed scheme.

Mobile manipulation is the fundamental challenge for robotics to assist humans with diverse tasks and environments in everyday life. However, conventional mobile manipulation approaches often struggle to generalize across different tasks and environments because of the lack of large-scale training. In contrast, recent advances in vision-language-action (VLA) models have shown impressive generalization capabilities, but these foundation models are developed for fixed-base manipulation tasks. Therefore, we propose an efficient policy adaptation framework named MoManipVLA to transfer pre-trained VLA models of fix-base manipulation to mobile manipulation, so that high generalization ability across tasks and environments can be achieved in mobile manipulation policy. Specifically, we utilize pre-trained VLA models to generate waypoints of the end-effector with high generalization ability. We design motion planning objectives for the mobile base and the robot arm, which aim at maximizing the physical feasibility of the trajectory. Finally, we present an efficient bi-level objective optimization framework for trajectory generation, where the upper-level optimization predicts way-points for base movement to enhance the manipulator policy space, and the lower-level optimization selects the optimal end-effector trajectory to complete the manipulation task. Extensive experimental results on OVMM and the real world demonstrate that MoManipVLA achieves a 4.2% higher success rate than the state-of-the-art mobile manipulation, and only requires 50 training cost for real world deployment due to the strong generalization ability in the pre-trained VLA models. Our project page can be found here.

Performing general language-conditioned bimanual manipulation tasks is of great importance for many applications ranging from household service to industrial assembly. However, collecting bimanual manipulation data is expensive due to the high-dimensional action space, which poses challenges for conventional methods to handle general bimanual manipulation tasks. In contrast, unimanual policy has recently demonstrated impressive generalizability across a wide range of tasks because of scaled model parameters and training data, which can provide sharable manipulation knowledge for bimanual systems. To this end, we propose a plug-and-play method named AnyBimanual, which transfers pre-trained unimanual policy to general bimanual manipulation policy with few bimanual demonstrations. Specifically, we first introduce a skill manager to dynamically schedule the skill representations discovered from pre-trained unimanual policy for bimanual manipulation tasks, which linearly combines skill primitives with task-oriented compensation to represent the bimanual manipulation instruction. To mitigate the observation discrepancy between unimanual and bimanual systems, we present a visual aligner to generate soft masks for visual embedding of the workspace, which aims to align visual input of unimanual policy model for each arm with those during pretraining stage. AnyBimanual shows superiority on 12 simulated tasks from RLBench2 with a sizable 12.67% improvement in success rate over previous methods. Experiments on 9 real-world tasks further verify its practicality with an average success rate of 84.62%.

Grip control interfaces impose a cognitive load on novices in teleoperation tasks. This paper introduces the Grip-Modulated Velocity Control (GMVC) framework, a novel teleoperation paradigm leveraging grip force modulation to control robot velocity intuitively. GMVC enables operators to modulate grip intensity to regulate robotic motion velocity, establishing a mapping that aligns with humans’ natural sensorimotor expectations. The framework features two variants: basic GMVC and enhanced GMVC with grip memory and context adaptation capabilities. Experimental evaluation across virtual path following, point navigation, and obstacle avoidance demonstrated that while performance was comparable for path following tasks, GMVC significantly reduced completion times for point navigation (39.7% improvement) and obstacle avoidance tasks (28.3% improvement). Subjective measures indicated a strong preference for grip-based control in tasks requiring transitions between coarse and fine movements. Real-world robot experiment through a plug-in-hole task shows that enhanced GMVC reduces completion time by 17.4% compared to conventional joystick control.

Enabling embodied agents to complete complex human instructions from natural language is crucial to autonomous systems in household services. Conventional methods can only accomplish human instructions in the known environment where all interactive objects are provided to the embodied agent, and directly deploying the existing approaches for the unknown environment usually generates infeasible plans that manipulate non-existing objects. On the contrary, we propose an embodied instruction following (EIF) method for complex tasks in the unknown environment, where the agent efficiently explores the unknown environment to generate feasible plans with existing objects to accomplish abstract instructions. Specifically, we build a hierarchical embodied instruction following framework including the high-level task planner and the low-level exploration controller with multimodal large language models. We then construct a semantic representation map of the scene with dynamic region attention to demonstrate the known visual clues, where the goal of task planning and scene exploration is aligned for human instruction. For the task planner, we generate the feasible step-by-step plans for human goal accomplishment according to the task completion process and the known visual clues. For the exploration controller, the optimal navigation or object interaction policy is predicted based on the generated step-wise plans and the known visual clues. The experimental results demonstrate that our method can achieve 45.09% success rate in 204 complex human instructions such as making breakfast and tidying rooms in large house-level scenes. Code and supplementary are available at https://gary3410.github.io/eif_unknown/.

This paper investigates two distinct paradigms for gaze-based control in human-robot collaboration (HRC). While gaze tracking offers a promising hands-free interaction method, the optimal mapping between eye movements and robot control remains an open research question. We examine two fundamentally different control approaches: (1) position-based control, which utilizes fiducial markers for spatial referencing and maps gaze positions directly to physical target locations; and (2) velocity-based control, whose functions are similar to a joystick where gaze position relative to camera frame centers determines movement direction and speed. Participants completed standardized pick-and-place tasks with both control methods. Performance was assessed through objective metrics including task completion time, trajectory efficiency, and error rates. Subjective experiences were evaluated using NASA Task Load Index questionnaires. Both systems incorporate a blink detection mechanism for gripper activation, enabling completely hands-free operation. This research addresses fundamental questions in eye-based robotic control for HRC, with applications spanning assistive technologies for mobility-impaired users, industrial settings that require hands-free operation, and medical environments where maintaining sterility is crucial. Results indicate significant differences between control paradigms, providing design insights for more intuitive and effective gaze-based interfaces in human-robot systems.

The utilization of shared control in the realm of surgical robotics augments precision and safety by amalgamating human expertise with autonomous assistance. This paper proposes a user-centered shared control framework enabling a robot to learn from expert demonstration, predict operators’ intent and modulate control authority to provide natural assistance when needed. We employ deep inverse reinforcement learning (IRL) to enable the robot to learn path planning from expert demonstrations with fast convergence, subsequently enhancing the policy with a potential field method. The control authority is allocated seamlessly between the human operator and the autonomous agent based on the prediction of operators’ movement from an adaptive filter and fuzzy logic inference. The proposed method is executed using the da Vinci Research Kit (dVRK) robot in a simulation environment, and its effectiveness is assessed through user performance evaluation in a trajectory tracking task. Compared to direct control and simple shared control, the proposed shared control scheme exhibits superior tracking accuracy and trajectory smoothness under external disturbances. Subjective responses underscore users’ perception of the method’s efficacy in enhancing their performance.

Robotic teleoperation has enabled remote interaction with hazardous environments, overcoming spatial constraints on human perception and manipulation. Most teleoperation systems rely on task-dependent interfaces to generate human instructions. This can lead to barriers in familiarizing the robot’s workspace and thus increase the training time for less experienced users. In order to address these problems, we introduce a novel hand gestures based robot teleoperation method, eliminating the need for specialized controlling devices. Leveraging hand landmark detection and a neural network-based decoding algorithm, the system interprets hand movements to control robot velocity, offering a user-friendly solution to communicating with the robot. Our trained model achieves an F2 score of 0.994 and outperforms algorithms in the collected dataset. Furthermore, the proposed method has been validated on a real-world Franka robot, achieving success rates of 100%, 80%, and 86.7% across three manipulation tasks.

A scarcity of frequencies arises from the increased demand for the Industrial Internet of Things (IIoT) and networked systems in warehouse operations. This puts an additional burden on available bandwidth in cellular networks. This problem can be tackled by cognitive radio networks (CRNs), which use spectrum sensing to track and access unused frequencies, increasing spectrum efficiency. However CRNs have been extensively studied in several IoT projects, this study is the initial to examine the utilisation of CRN to intelligently manage the use of available radio frequencies. Two macro base stations are the central hubs of a network, communicating with IIoT devices in warehouse settings. This paper investigates the utilisation of CRN with machine-learning algorithms to intelligently manage the spectrum for connected IoT devices for intelligent Warehouse settings. A range of machine learning methods, including Support Vector Machine (SVM), k-nearest Neighbors (KNN), Decision Tree, Random Forest, and Naive Bayes, are provided to identify accessible bands for the best possible spectrum allocation and to recognize key users. Based on criteria like accuracy, precision, recall, and score for all ML techniques, the system’s performance is assessed. The numerical results demonstrate a noteworthy 20% reduction in false positives and a substantial improvement in cooperative spectrum sensing accuracy, which in turn improves the effectiveness of IIoT operations in warehouse environments.

Radio Access Network (RAN) deployments are evolving quickly owing to the innovative approaches of the Open Radio Access Network (O-RAN) Alliance. Specifically, they are moving away from closed, customized hardware implementations and toward virtualized instances operating on shared platforms. Future successful and affordable RAN deployments are made possible by this paradigm change, which is characterised by the separation of radio software components from hardware. Real-time network parameter configuration, sufficient computing resources for virtualized RAN (vRAN) deployment, and dependable processing unit sharing among numerous vRAN instances are some of the obstacles still standing in the way of successful O-RAN network implementations. Thus, this paper explored and compared the effectiveness of diverse optimization algorithms for minimising the number of resource blocks (nRBS), including machine learning (RandomForestRegressor), heuristic, and mathematical methods. Moreover, it investigates the lessons learned and the limitations of the proposed system. It demonstrates the practical success of a heuristic approach in O-RAN optimization, achieving significant reductions in resource blocks based on the Throughput-to-Bandwidth Ratio. It also provided insights into challenges with the RandomForestRegressor model, highlighted the importance of considering real-world network dynamics, and offered valuable lessons for future research, emphasizing the need for adaptive solutions and exploring hybrid optimization approaches, ultimately contributing to an enhanced understanding of O-RAN optimization.

This paper proposes a distributed on-orbit spacecraft assembly algorithm, where future spacecraft can assemble modules with different functions on orbit to form a spacecraft structure with specific functions. This form of spacecraft organization has the advantages of reconfigurability, fast mission response and easy maintenance. Reasonable and efficient on-orbit self-reconfiguration algorithms play a crucial role in realizing the benefits of distributed spacecraft. This paper adopts the framework of imitation learning combined with reinforcement learning for strategy learning of module handling order. A robot arm motion algorithm is then designed to execute the handling sequence. We achieve the self-reconfiguration handling task by creating a map on the surface of the module, completing the path point planning of the robotic arm using A*. The joint planning of the robotic arm is then accomplished through forward and reverse kinematics. Finally, the results are presented in Unity3D.

Many surgical tasks require three or more tools working together, where a hands-free interface could extend a surgeon's actions to control a third surgical tool. However, most current interfaces do not allow skilled control of grasping critical to robotic manipulation. Here we first present a systematic study to identify efficient and intuitive interaction strategies to control grasping of a surgical tool. A series of experiments were conducted to evaluate six foot pressure-based gestures. Based on the results, three modular novel foot-machine interfaces were developed, which can be integrated with other motion control interfaces. The identified interaction strategies were implemented to control a laparoscopic tool in a surgical simulator, and evaluated in a user study. The results illustrate how naive participants can operate grasping yielding smooth and pick & place operation.

Accurately estimating the shape of objects in dense clutters makes important contribution to robotic packing, because the optimal object arrangement requires the robot planner to acquire shape information of all existed objects. However, the objects for packing are usually piled in dense clutters with severe occlusion, and the object shape varies significantly across different instances for the same category. They respectively cause large object segmentation errors and inaccurate shape recovery on unseen instances, which both degrade the performance of shape estimation during deployment. In this paper, we propose a category-level shape estimation method for densely cluttered objects. Our framework partitions each object in the clutter via the multi-view visual information fusion to achieve high segmentation accuracy, and the instance shape is recovered by deforming the category templates with diverse geometric transformations to obtain strengthened generalization ability. Specifically, we first collect the multi-view RGB-D images of the object clutters for point cloud reconstruction. Then we fuse the feature maps representing the visual information of multi-view RGB images and the pixel affinity learned from the clutter point cloud, where the acquired instance segmentation masks of multi-view RGB images are projected to partition the clutter point cloud. Finally, the instance geometry information is obtained from the partially observed instance point cloud and the corresponding category template, and the deformation parameters regarding the template are predicted for shape estimation. Experiments in the simulated environment and real world show that our method achieves high shape estimation accuracy for densely cluttered everyday objects with various shapes.

Efficient and accurate object detection and localization play a crucial role in enabling robots to understand and interact with their environment. To this end, this paper presents a novel hybrid approach that combines deep learning and feature-based methods to address category-agnostic object detection and localization using image queries. By leveraging the strengths of both approaches, our method achieves superior performance in accurately localizing and segmenting objects, surpassing traditional feature-based template matching methods and the widely-used YoLov3. The proposed method utilizes a category-agnostic semantic segmentation framework, where objects are segmented based on their presence rather than their specific categories. Through quantitative evaluations on both synthetic and real-world datasets, our approach demonstrates remarkable accuracy and robustness in various scenarios, including objects with arbitrary shapes. The results demonstrate that the proposed approach provides an effective object detection and localization tool for visual servoing augmentation.

It is well-known that the tendon-driven robotic manipulator plays an important role in robotic-assisted minimally invasive surgery (MIS). However, due to the intrinsic nonlinearities, uncertainties, slack and hysteresis introduced by the tendon-driven actuation, the tendon-driven robotic manipulator is difficult to model and control when compared with the traditional actuation styles. To serve the modeling purpose, in this paper, the deep-learning-based intelligent modeling of inverse kinematics in the snake-like tendon-driven surgical instrument is presented. In the proposed approach the Deep Recurrent Neural Network (DRNN) with Long Short-Term Memory (LSTM) architecture is adopted to memorize and identify the nonlinear inverse kinematics of the tendon-driven surgical instrument through the history of the motor and tip positions. To collect highly reliable data to train the DRNN, the experiment to generate training data is carefully designed with the consideration of the stainless tendon characters and motor limitations. During the designed controller movements, the kinematics data is obtained by recording the motor positions and the tip positions. Besides, it is noticed that there are correlations of the sequential data samples, which could significantly reduce the modeling accuracy. To remove the correlations and improve the modeling performance, the correlations of the sequential data samples are removed by modifying the training processes. Modeling results and detailed discussions verified the effectiveness of the proposed approach.

The traditional master-slave teleoperation relies on human expertise without correction mechanisms, resulting in excessive physical and mental workloads. To address these issues, a co-pilot-in-the-loop control framework is investigated for cooperative teleoperation. A deep deterministic policy gradient (DDPG) based agent is realised to effectively restore the master operators' intents without prior knowledge on time delay. The proposed framework allows for introducing an operator (i.e., copilot) to generate commands at the slave side, whose weights are optimally assigned online through DDPG-based arbitration, thereby enhancing the command robustness in the case of possible human operational errors. With the help of interval type-2 (IT2) Takagi-Sugeno (T-S) fuzzy identification, force feedback can be reconstructed at the master side without a sense of delay, thus ensuring the telepresence performance in the force-sensor-free scenarios. Two experimental applications validate the effectiveness of the proposed framework.

Robotic dual-arm twisting is a common but very challenging task in both industrial production and daily services, as it often requires dexterous collaboration, a large scale of end-effector rotating, and good adaptivity for object manipulation. Meanwhile, safety and efficiency are primary concerns for robotic dual-arm coordinated manipulation. Thus, the normally adopted fully automated task execution approaches based on environmental perception and motion planning techniques are still inadequate and problematic for the arduous twisting tasks. To this end, this paper presents a novel strategy of the dual-arm coordinated control for twisting manipulation based on the combination of optimized motion planning for one arm and real-time telecontrol with human intelligence for the other. The analysis and simulation results showed it can achieve collision and singularity free for dual arms with enhanced dexterity, safety, and efficiency.

The hip assistance robotic exoskeleton has been demonstrated as an effective device to assist elderly and disabled people with gait disorders. The assistance efficiency of these devices, however, is less optimized because the parameters in the active impedance control are manually designated. This paper presented a novel assistance control scheme to address the sub-optimal issue. This study poses that the assistance efficiency can be maximized by modifying the mechanical impedance to resonate with the muscle driving force, in which the human-exoskeleton coupling system is approximated with a second-order dynamical system. Based on this, the exoskeleton virtual stiffness is adaptively tuned to make the system intrinsic frequency align with the intended swing frequency. The proposed assistance control scheme demonstrated an increased assistance efficiency than the conventional active impedance control in a simulated study. Experiments that were managed on a newly custom-made hip assistance robotic exoskeleton also demonstrated strong evidence of improved gait kinematics with decreased muscle-skeleton efforts.

Limited by the operation time window and working space, space teleoperation tasks need to be completed within an expected time while ensuring that the end effector meets the physical constraints. Meanwhile, the interaction with unknown environments would cause uncertainty in the closed-loop system, which brings great challenges to the control design. To solve the above problems, the control performance issue for a class of space teleoperation systems subject to multiple constraints and interaction uncertainties is investigated in this article. The force interaction with the human operator/space environment is represented by interval type-2 (IT2) Takagi-Sugeno (T-S) fuzzy systems, where the uncertain equivalent mass and damping parameters can be effectively described and captured by IT2 membership functions. In order to reduce the communication burden and satisfy the constraints of settling time, transient-state performance and operating space, a time-varying threshold event-triggered control scheme together with exponential-type Lyapunov function is developed for the first time. We show that, with the proposed controller, the synchronization tracking errors are guaranteed to converge to a user-defined residual set within preassigned settling time, and never exceed the prescribed range despite unknown control direction and actuator faults, which solves the long-standing constraint issue with more flexibility due to the fact that the related constraints can be arbitrarily specific within the physically available range. Moreover, the convergence set is only dependent on fewer user-defined parameters rather than approximation errors, which provides an effective analysis technique to deal with the difficulty that the convergence accuracy is difficult to calculate quantitatively in the presence of unknown disturbance. Detailed simulation results are provided to show the effectiveness and merit of the proposed control strategy.

This paper is concerned with interval type-2 (IT2) fuzzy control design for a class of nonlinear space teleoperation systems with external disturbances and time-varying delays. IT2 fuzzy model based (FMB) control design with exponential-type Barrier Lyapunov function (EBLF) is presented to address state constraints, communication burden from ground stations to satellites (space-robot), and uncertain human/environment interaction parameters in a unified event-triggered control structure. We show that, with the proposed adaptive event-triggered control scheme, the exponential convergence performance of the synchronization tracking errors is guaranteed, while the prescribed constraint requirement is satisfied. Simulation results are provided to validate the effectiveness of the proposed controller.

To obtain synchronized gait assistance, this paper presents a new delayless adaptive dual-oscillator (ADO) scheme to address the inherent delay issue. In the ADO structure, a new oscillator is coupled with the primitive one but the phase is adaptively feed-forward compensated. It's remarkable that the compensated phase is determined by the proposed extended phase lag observer, in which both the phase lag and phase leading can be properly estimated and eliminated in the steady and non-steady gait. Moreover, a unified exoskeleton control scheme based on ADO is further proposed to improve the gait segmentation, velocity/acceleration estimation, intention estimation, and assistance generation performances, which further enhances the assistance synergy and reduces the safety risks. Experimental results demonstrate better alignment assistance and consequently reduced muscle efforts with ADO-based assistance control.

Accurate acceleration acquisition is a critical issue in the robotic exoskeleton system, but it is difficult to directly obtain the acceleration via the existing sensing systems. The existing algorithm-based acceleration acquisition methods put more attention on finite-time convergence and disturbance suppression but ignore the error constraint and initial state irrelevant techniques. To this end, a novel radical bias function neural network (RBFNN) based fixed-time reconstruction scheme with error constraints is designed to realize high-performance acceleration estimation. In this scheme, a novel exponential-type barrier Lyapunov function is proposed to handle the error constraints. It also provides a unified and concise Lyapunov stability-proof template for constrained and non-constrained systems. Moreover, a fractional power sliding mode control law is designed to realize fixed-time convergence, where the convergence time is irrelevant to initial states or external disturbance, and depends only on the chosen parameters. To further enhance observer robustness, an RBFNN with the adaptive weight matrix is proposed to approximate and attenuate the completely unknown disturbances. Numerical simulation and human subject experimental results validate the unique properties and practical robustness.

In this article, we present an adaptive prescribed-time control method for a class of nonlinear telerobotic systems with actuator faults and position error constraints. Extended from prescribed-time stability, practically prescribed-time stability (PPTS) is proposed for the first time aiming at stability analysis and control synthesis of nonlinear systems with disturbance and uncertainty. We show that, under the control scheme in the framework of PPTS, the system states are guaranteed to converge to a user-defined set (physically realizable) within user-defined settling time (physically realizable). Based on PPTS, an adaptive fault-tolerant controller is developed by integrating a novel exponential-type barrier Lyapunov function. Rigorous stability analysis based on back-stepping approach proves that, under the proposed control strategy, synchronization errors converge to a user-defined residual-set within predefined settling time and never exceed the prescribed range. Universal performance indexes, including the settling time, residual-set, accuracy, and overshoot, can be user-defined and only dependent on fewer user-defined parameters. Simulation results illustrate the effectiveness of the developed control scheme.