Sequential-Horizon Vision-and-Language Navigation (SH-VLN) presents a challenging scenario where agents should sequentially execute multi-task trajectory navigation guided by complex, long-horizon natural language instructions. Current vision-and-language navigation models exhibit significant performance degradation with such instructions, as information overload impairs the agent's ability to attend to observationally relevant details. To address this problem, we propose SeqWalker, a novel navigation model built on a hierarchical planning framework. Our SeqWalker features: (1) A High-Level Planner that dynamically selects global instructions into contextually relevant sub-instructions based on the agent's current visual observations, thus reducing cognitive load; (2) A Low-Level Planner incorporating an Exploration-Verification strategy that leverages the inherent logical structure of instructions for trajectory error correction. To evaluate SH-VLN performance, we also extend the IVLN dataset and establish a new benchmark. Extensive experiments are performed to demonstrate the effectiveness and superiority of SeqWalker.

Mainstream multimodal large language models (MLLMs) rely on patch-based tokenization methods, which compromise the integrity of objects and thereby limit the model's perception capabilities while triggering object-related hallucinations. To address this issue, we propose ObjecTok, an innovative object tokenization framework. ObjecTok generates a single, holistic object token for each object in an image. This token is produced by a specially trained object encoder that embeds the object's semantic, positional, and shape information into a single compact representation, thereby preserving the object's integrity. To mitigate the imperfections of upstream object proposer models, we introduce learnable confidence embeddings. These embeddings enable the MLLM to learn the reliability of each object's information, significantly enhancing the model's robustness. Additionally, ObjecTok employs a hybrid input strategy, combining object tokens with traditional image patch tokens, allowing the model to leverage both object-level information and global scene context. By integrating ObjecTok into the LLaVA architecture, we achieve notable performance improvements on multiple object-centric benchmarks, effectively reducing object hallucinations and enhancing perception capabilities. Experimental results robustly demonstrate that the object tokens generated by our ObjecTok framework hold great potential for building more powerful and reliable MLLMs.

Exoskeletons have been shown to effectively assist humans during steady locomotion. However, their effects on non-steady locomotion, characterized by nonlinear phase progression within a gait cycle, remain insufficiently explored, particularly across diverse activities. This work presents a shank angle-based control system that enables the exoskeleton to maintain real-time coordination with human gait, even under phase perturbations, while dynamically shaping assistance profiles to match the biological ankle moment patterns across walking, running, stair negotiation tasks. The control system consists of an assistance profile online generation method and a model-based feedforward control method. The assistance profile is formulated as a dual-Gaussian model with the shank angle as the independent variable. Leveraging only IMU measurements, the model parameters are updated online each stride to adapt to inter- and intra-individual biomechanical variability. The profile tracking control employs a human-exoskeleton kinematics and stiffness model as a feedforward component, reducing reliance on historical control data due to the lack of clear and consistent periodicity in non-steady locomotion. Three experiments were conducted using a lightweight soft exoskeleton with multiple subjects. The results validated the effectiveness of each individual method, demonstrated the robustness of the control system against gait perturbations across various activities, and revealed positive biomechanical and physiological responses of human users to the exoskeleton's mechanical assistance.

Continual video instance segmentation (CVIS) requires the plasticity to absorb new categories while maintaining the stability to retain previously learned knowledge. Crucially, the model must also preserve temporal consistency of instances across video frames. In this work, we introduce Contrastive Residual Injection and Semantic Prompting (CRISP), a framework tailored to address instance-wise, category-wise, and task-wise confusion in CVIS. For instance-wise learning, we model instance tracking and construct instance correlation loss, which emphasizes the correlation with the prior query space while strengthening the specificity of the current task query. For category-wise learning, we build an adaptive residual semantic prompt (ARSP) learning framework, which constructs a learnable semantic residual prompt pool generated by category text and uses an adjustive query-prompt matching mechanism to build a mapping relationship between the query of the current task and the semantic residual prompt. Meanwhile, a semantic consistency loss based on the contrastive learning is introduced to maintain semantic coherence between object queries and residual prompts during incremental training. For task-wise learning, to ensure the correlation at the inter-task level within the query space, we introduce a concise yet powerful initialization strategy for incremental prompts. Extensive experiments on YouTube-VIS-2019 and YouTube-VIS-2021 datasets demonstrate that CRISP significantly outperforms existing continual segmentation methods in the long-term continual video instance segmentation task, avoiding catastrophic forgetting and effectively improving segmentation and classification performance. The code is available at https://github.com/LyuQi127/CRISP.

Most LiDAR odometry and SLAM systems construct maps in point clouds, which are discrete and sparse when zoomed in, making them not directly suitable for navigation. Mesh maps represent a dense and continuous map format with low memory consumption, which can approximate complex structures with simple elements, attracting significant attention of researchers in recent years. However, most existing methods operate under a static environment assumption. In effect, moving objects cause ghosting, degrading the quality of meshing. To address these issues, we propose a plug-and-play meshing module adapting to dynamic environments, which can easily integrate with various LiDAR odometry to generally improve the pose estimation accuracy of odometry. In our meshing module, a novel two-stage coarse-to-fine dynamic removal method is designed to effectively filter dynamic objects, generating consistent, accurate, and dense mesh maps. To the best of our knowledge, this is the first mesh construction method with explicit dynamic removal. Additionally, sliding window-based keyframe aggregation and adaptive downsampling strategies are used to ensure the uniformity of point cloud, benefiting for Gaussian process in mesh construction. We evaluate the localization and mapping accuracy on six publicly available datasets. Extensive experiments demonstrate the superiority of our method compared with the state-of-the-art algorithms. The code and introduction video are publicly available at https://yaepiii.github.io/CAD-Mesher/.

Soft artificial muscle actuators have gained attention in robotics for their remote control, fast response, and high compliance. However, replicating the intricate and efficient motions of natural muscles remains a challenge. Existing designs often lack the hierarchical and anisotropic properties of muscle sarcomeres, limiting their ability to achieve biomimetic movements. We developed a novel Biomimetic Magnetic Artificial Actuator (BMAA) inspired by muscle sarcomeres. Using a soft magnetic composite material arranged in a hierarchical structure, the actuator mimics the arrangement of actin and myosin filaments. External magnetic fields enable precise control of contraction and relaxation, emulating natural muscle motion. The driver can achieve the motion performance of muscle like motion characteristics, and its driving ability is verified by reptile experiment and elbow experiment. The actuator demonstrates significant deformation, fast response, and excellent controllability, enabling complex and precise movements. This research advances the development of biomimetic soft actuators, offering potential applications in soft robotics, biomedical devices, and artificial muscles, and paving the way for more versatile and intelligent machines.

The rock-climbing fish is a benthic organism that can move rapidly and flexibly on rock surfaces in complex underwater environments. Studies have shown that this unique adhesion-sliding movement mechanism of the rock-climbing fish relies on the anisotropic friction exhibited by its sucker structure, which helps to reduce friction in the forward direction and defend against the impact of the flow field. In this work, inspired by the anisotropic friction phenomenon of the sucker of the rock-climbing fish, we designed the absorption module and pectoral and pelvic fin flapping module of the robotic fish to realize the contact switching from low friction to high friction. Meanwhile the propulsion module adopts a novel design of wire-driven caudal fin that can oscillate at high frequency (exceeding ~5 Hz). Resulting robotic fish can realize different motion modes such as adhesion-sliding movement (~0.5 BL/s) and wall-stabilized adsorption. This work will provide a new solution for the design pattern of underwater wall robots.

Engineered skeletal muscle tissue (SMT) is the ideal driving units for achieving fine movements in biosyncretic robots due to their excellent controllability and potentially large driving force. However, the selective stimulation of SMTs continues to pose a significant technical challenge. In this study, we propose a method for the selective stimulation of 3D SMT using thin-film interdigitated electrodes (IDEs). By optimizing the IDEs geometry through finite element simulations, electrical field intensity of fingertip is effectively reduced. The thin-film IDEs are fabricated on a Polyester (PET) substrate using screen printing technology and successfully enable selective activation and controlled contraction of the SMTs. Compared to conventional parallel-plate electrodes (PPEs) and rod-shaped electrodes (RSEs), the IDEs significantly improve the electrical field distribution and enhance spatial resolution. This advancement provides a promising new approach for achieving high-precision motion control in biosyncretic robots (or biohybrid robots).

It remains a formidable challenge to accurately recognize motion intentions of patients thus to control hand exoskeletons according to their volition. Current methods primarily focus on recognition of limited patient’s motion intentions, with the purpose of controlling preconfigured gestures of a hand exoskeleton for grasping objects. These methods exhibit a marked shortfall when encountering scenarios that are unexpected or not designed in advance, such as non-preprogrammed hand movements and object manipulation tasks. To tackle this issue, large language model (LLM) and speech recognition technology are employed in this study to allow the patient to control a hand exoskeleton at will. In particular, two LLMs are tailored to formulate codes of either generating non-preprogrammed gestures or dealing with unencountered objects. Additionally, an incremental learning framework is proposed to enable patients to perform both predefined and non-predefined operation tasks by integrating a natural language parser with the two LLM-based learners. The natural language parser can directly control the hand exoskeleton to perform predefined operations tasks from prestored command set, while the LLM-based learners can incrementally expand the control command set so as to enhance adaptability of the hand exoskeleton to complex activities over daily use. This study is a pioneering work in the field of hand exoskeletons, which will revolutionize the way to control hand exoskeletons. Furthermore, the proposed framework can be easily generalized to any other robots by modifying the prompt of customized LLMs, which provides a new idea to achieve autonomous learning in robotics.Note to Practitioners—The motivation of this article is to tackle the challenge of intention recognition for performing activities of daily living (ADLs) by stroke patients using a multi-degree of freedom hand exoskeleton. Existing methods for intention recognition so far can only be used for several tasks that are predefined in advance, thus none of them allow patients to control the hand exoskeleton completely at will. To surpass this limitation, an LLM-enabled incremental learning framework that integrates a hand exoskeleton controller with Large Language Model (LLM) is proposed and validated in this study. The framework offers patients an intuitive interface via voice interaction and enables patients to perform not only predefined operation tasks by the hand exoskeleton controller but also non-predefined ones that can be learned from the LLM. As a result, the hand exoskeleton controller continues to learn from the LLM, therefore is gradually able to perform all tasks in daily life. This pioneering study paves a new way in building patient-controlled hand exoskeletons with autonomous intelligence that can deal with non-predefined operation tasks in unstructured environments.

Accurate and prompt speed grasp intention recognition is crucial in online human-robot interaction (HRI). However, dynamic grasping relying on complete motion for high recognition accuracy will lead to an unavoidable delay in real-time prediction. To address this issue, we propose a Spatial Gating and Temporal Weighting Early Grasp Prediction (STEGP) method that utilizes incomplete dynamic grasping data from sliding windows to reduce the time delay for reliable robot teleoperation. The proposed method comprises a synergy-based feature extraction module, a spatial gating classification module, and a time-decay weighting fusion prediction module. The spatial-temporal mechanism with gating units effectively classifies sequential movements, achieving performance comparable to that of Transformers but being much easier to implement. Integrating a time-decay weighting frame enables reliable early prediction even with incomplete data. gMLP is chosen for the classification of hand dynamic grasping because of its high accuracy, realizing 93.83% accuracy for 33 grasping categories. The prediction tests demonstrated 85.4% accuracy, with the first 25% grasp completion across 28 subjects. Online robotic teleoperation grasp experiments achieved a 57.4% reduction in time delay and a 93.3% success rate.

Accurately measuring both in-plane and out-of-plane mechanical properties is essential for understanding material behavior at the nanoscale. Conventional atomic force microscopy (AFM) techniques often struggle with the coupling between vertical and lateral forces, which can distort lateral mechanical property measurements. To address this limitation, we propose a novel Peak Force Torsional Resonance (PFTR) method that combines Peak Force Tapping for precise height control with torsional resonance for in-plane property extraction. This approach effectively decouples vertical and lateral forces, significantly reducing the influence of topography and vertical interactions on lateral measurements. Additionally, we introduce a mathematical model to accurately quantify lateral properties, such as shear modulus and lateral viscosity, independent of height variations. Experimental results confirm that the PFTR method achieves high-resolution imaging of both in-plane and out-of-plane mechanical properties, offering a more accurate and reliable solution for nanoscale mechanical characterization compared to conventional AFM-based methods. Note to Practitioners—This work addresses a common challenge in nanoscale mechanical property measurement: the interference from surface topography that affects accurate imaging of in-plane (lateral) properties. Traditional AFM methods struggle with this coupling effect, making it difficult to reliably measure both in-plane and out-of-plane mechanical characteristics—measurements that are vital in fields such as semiconductor manufacturing and materials development. This paper introduces PFTR Microscopy, a new AFM-based approach designed to decouple these forces and accurately resolve nanoscale mechanical properties. PFTR combines PFT for height control with torsional resonance to extract in-plane mechanical data without distortions from surface topography. By providing clear imaging of lateral properties and reducing topographical artifacts, PFTR enables more reliable assessments of materials such as semiconductor coatings, thin films, and composite compounds used in high-precision industries. PFTR is validated through experimental comparisons with conventional AFM methods, demonstrating that it achieves higher resolution and accuracy in in-plane measurements. While this approach provides robust data for characterizing homogeneous materials, further research is needed to adapt it for complex, heterogeneous surfaces and to simplify calibration for broader industrial application. The ability to distinguish mechanical properties at the nanoscale also suggests potential for applications in MEMS design, failure analysis, and the characterization of soft or biological materials.

While rehabilitation exoskeletons have been extensively studied, systematic design principles for effectively addressing heterogeneous bilateral locomotion in hemiplegia patients are poorly understood. In this article, a multijoint lower exoskeleton driven by series elastic actuators (SEAs) is developed, and the design philosophy of rehabilitation robots for hemiplegia patients is systematically explored. The exoskeleton has six powered joints for both lower limbs in a hip–knee–ankle configuration, and each joint incorporates a custom, lightweight SEA module. A unified interaction-oriented control framework is designed for exoskeleton-assisted walking, including gait generation, task scheduling, and advanced joint-level control. The closed-loop design provides methodical solutions to address hemiplegia rehabilitation needs and provides walking assistance for bilateral lower limbs. Moreover, a multitemplate gait generation approach is proposed to address the altered kinematics induced by exoskeleton-assisted walking and enhance the exoskeleton's adaptability to patient-specific kinematic variations in an iterative manner. Experiments are conducted with both healthy individuals and hemiplegia patients to verify the effectiveness of the exoskeleton system. The clinical outcomes demonstrate that the exoskeleton can achieve mechanical transparency, facilitate movement, and enable coordinated interjoint locomotion for bilateral gait assistance.

Biohybrid microrobots with autonomous movement capabilities have broad application prospects in targeted delivery, attracting researchers to study their movement characteristics. However, its automatic control is still challenging, and exploring real-time detection of its environment for path planning to achieve stable closed-loop control is highly important for its practical application. Here, we applied deep learning for the detection of biohybrid microrobots and their targets and obstacles, followed by real-time path planning and trajectory tracking of biohybrid microrobots for targeted delivery. The proposed detection algorithm introduces attention and multi-scale feature fusion mechanisms in YOLOv7 algorithm (AM-YOLOv7) with the aim of enhancing the precision of detecting small-scale targets when robots, obstacles and targets are displayed globally, and the detection capabilities are verified through simulations and experiments. The proposed planning algorithm introduces a turning penalty function and a path smoothing strategy into A* algorithm (PS-A*) to make the planned path short and smooth, which has been verified through simulation and experiments. The adaptive fuzzy PID method is used to track the robot's trajectory, and experiments and simulations show that the biohybrid microrobot can move according to the preset trajectory better. The final cell scene experimental results show that the biohybrid microrobot using this system can effectively avoid obstacle cells and be delivered to target cells. The system can detect biohybrid microrobots, obstacle cells and target cells, plan short and smooth trajectories, and track them accurately. The proposed method has certain generalizability and broad application prospects in targeted delivery.

Fish can adaptively adjust their body kinematics and swimming modes by sensing to realize optimal propulsion. However, most soft robotic fish have an unchangeable swimming mode through simple structure design, making them difficult to adapt to dynamic and complex fluid environments. Here, inspired by the multiple muscle synergy and lateral line sensing function of fish, we developed a soft robotic fish with multiple actuating units and embedded sensing elements. By collaboratively controlling the amplitude and phase of excitation from the multiple flexible actuating units, the soft robotic fish can successfully realize various swimming modes very similar to those of natural fish. Additionally, the embedded flexible sensing elements enable the robotic fish to sense the swimming state and the surrounding fluid environment in real time. The multiple actuation and embedded sensing allow the soft robotic fish to adaptively switch to an optimal swimming mode in a certain fluid environment. The multimode swimming and perception capabilities proposed in this work not only make soft robotic fish more intelligent and adaptable to complex fluid environments, but also contribute to the future implementation of autonomous control capabilities for robotic fish.

Rehabilitation exoskeletons have been demonstrated to benefit mobility-limited patients; however, accessibility is hindered by several challenges, and interaction evaluation of coupled human–exoskeleton systems remains critically understudied. In this pioneering study“, assistive normality (AN)” is introduced to characterize the cross-stage spatiotemporal features of exoskeleton-assisted gait in hemiplegic patients, and a multidimensional, low-data-cost metric framework is developed to quantify AN. Three subdivided metrics, including gait restoration (GR), phase-deviation weighting (PDW) and multijoint coordination (MJC), are proposed to assess coupled system interaction behavior across different intervention stages during rehabilitation. A homologous difference evaluation paradigm (HDEP) is introduced to capture pathological differences between healthy and hemiplegic subjects on the basis of the reusability of experimental data, providing an approach for the nominal assistance calibration and assessment of specific exoskeleton devices. In a pilot study with eight healthy individuals and nine hemiplegic patients, between-group metric differences were analyzed to determine the calibrated AN of an exoskeleton. The results provide calibration references for quantitative metrics and demonstrate the ability of framework to characterize the temporal dynamics of human–exoskeleton interactions. The proposed AN framework offers a generalizable approach to assistive robotics, potentially enhancing evaluation of human-robot interaction and advancing clinical rehabilitation applications.

Magnetic microrobots can be navigated by an external magnetic field to autonomously move within living organisms with complex and unstructured environments. Potential applications include drug delivery, diagnostics, and therapeutic interventions. Existing techniques commonly impart magnetic properties to the target object, or drive the robot to contact and then manipulate the object, both probably inducing physical damage. This paper considers a non-contact formulation, where the robot spins to generate a repulsive field to push the object without physical contact. Under such a formulation, the main challenge is that the motion model between the input of the magnetic field and the output velocity of the target object is commonly unknown and difficult to analyze. To deal with it, this paper proposes a data-driven-based solution. A neural network is constructed to efficiently estimate the motion model. Then, an approximate model-based optimal control scheme is developed to push the object to track a time-varying trajectory, maintaining the non-contact with distance constraints. Furthermore, a straightforward planner is introduced to assess the adaptability of non-contact manipulation in a cluttered unstructured environment. Experimental results are presented to show the tracking and navigation performance of the proposed scheme.

In recent years, bio-syncretic robots actuated by living materials have received widespread attention. Among the common living materials, engineered skeletal muscle tissue (eSKT) has been the focus of researchers due to its high contraction force and good controllability. However, the current performance of eSKT is far from that of natural skeletal muscle tissue. In this paper, an optimized design method for eSKTs has been proposed. By combining simulation analysis with experiments, the eSKTs with multiple strips have been developed. The results show that under a specific volume (250 μL), the optimized strip structures can enhance the stability of eSKT and facilitate the penetration of nutrients and oxygen, leading to improved fusion of myoblasts and the directional arrangement of myotubes, thus improving the performance of eSKT. The eSKT with multiple strips exhibits a significant contraction force and has been successfully utilized in a bio-syncretic robot to demonstrate its actuation capability. This work may provide insights into the development of the field of bio-syncretic robots and even tissue engineering.

Assist-as-needed (AAN) assistance can promote active voluntary participation in rehabilitation and motor function recovery of post-stroke patients. However, different patients have personalized damaged regions and recovery states, causing difficulties to obtain adaptive and customized assistance in robot-assisted rehabilitation. This paper presents a mirror Adaptive Assist-As-Needed (AAAN) scheme, including two modules of Multi-Sensors Fused Estimation (MSFE) and Online Incremental Mirror Adaptation (OIMA), to encourage the subjects to actively participate in rehabilitation. Specifically, the first MSFE module can obtain the needed assistance based on the functional capability of the post-stroke patients via the data fusion of biological and motional signals using Kalman Filter. The second OIMA module fine-tunes the control torques estimated by MSFE to adapt the muscle fatigue and stiffness varieties of the affected limb based on the motion and physiological reference of the mirror healthy limb. The results demonstrate that the AAAN strategy can realize the transparent mode for healthy subjects and promote post-stroke patients to rehabilitate the affected limb with active participation using EMG signals 90.5% similar to those of the mirror healthy limb. The proposed method can be expected to greatly enhance power assistance and rehabilitation outcome of post-stroke patients using exoskeletons by provoking active participation. Note to Practitioners—For robotic rehabilitation, it is crucial to provide suitable assistances that can maximize the participation of post-stroke patients, which can promote the recovery outcome of therapies. The main purpose of this work is to achieve the adaptive assist-as-needed control strategy for upper limb rehabilitation tasks in two steps. Firstly, the elbow joint torques of a post-stroke patient are estimated by data fusion of motion and electromyography (EMG) signals using Kalman Filter, which can make up for the shortcomings of the individual signals, such as poor reliability and low sensitivity. Secondly, the joint motion and EMG signals of the mirror healthy limb are used as the reference to calculate the adaptive needed assistance to rehabilitate the affected limb. The preliminary experiments with healthy and post-stroke subjects demonstrate that this approach can obtain stable motion with the natural physiological states of subjects and enhance active voluntary participation in rehabilitation. In the future study, it will be investigated how to accelerate the adaptation of new patients based on the knowledge of the learned individuals using machine learning methods, such as lifelong learning and incremental learning.

Magnetic catheter robots have great potential for application in minimally invasive surgical interventions that require submillimeter soft catheters to be inserted into the body. Compared to other medical continuum robots, the kinematic characteristics of the magnetic continuum robot are required to be improved, including operating dexterity. The opposite magnetic continuum robot (OMCR), which has oppositely-magnetized magnets on the robot, is capable of deforming in large and high-order curvatures and shows improved dexterity. However, unstable kinematic behaviors emerge owing to solution multiplicity and affect the actual operation. For example, snapping, which indicates the sudden jump from one state to another, induces danger for medical operations. In addition, the motion sequence of the external mobile magnet affects the final state of OMCR. Therefore, in this study, we first analyze the reason for these emerging behaviors by understanding the solution multiplicity of an OMCR. We then modified the OMCR kinematic model with instability analysis and additional constraints, to predict the critical point for snapping and avoid other unexpected behavior for path planning. Herein, our study aims to analyze the stability of the OMCR with relatively unstable configurations in their actuation space to ensure stable and dexterous motions for medical applications. Note to Practitioners—The motivation of this work is to analyze, predict, and avoid the unexpected behaviors of the medical magnetic catheter robots, and ensure stable and dexterous operation. Submillimeter magnetic catheter robots have received considerable attention in the past ten years owing to their small size and dexterous motion. They have great potential for minimally invasive surgical interventions that require the navigation of “tighter corners” inside the human body. However, there may exist multiple local solutions under an external field, especially when the robot deforms in high-order curvature for dexterous operation. This situation induces unexpected behaviors and danger for medical use, such as a sudden jump from one state to another. Herein, we analyze the solution evolution as the external field varies and predict the unexpected behaviors using a modified kinematic model with instability analysis. Preliminary experiments demonstrated the effectiveness of the modified model, which can predict unexpected behaviors, and evaluate and enhance the stability during path planning. This method can be implemented for the dexterous and stable operation of magnetic catheter robots, which is significant for medical applications.

Due to high biological adaptability and flexibility, pneumatic artificial muscle (PAM) systems are widely employed in exoskeleton robots to accomplish rehabilitation training with repetitive motions. However, some intrinsic characteristics of PAMs and inevitable practical factors, e.g., high nonlinearity, hysteresis, uncertain dynamics, and limited working space, may badly degrade tracking performance and safety. Hence, this paper designs a new learning-based motion controller for PAMs, to simultaneously compensate for model uncertainties, eliminate tracking errors, and satisfy preset motion constraints. Particularly, when PAMs suffer from periodically non-parametric uncertainties, the elaborately designed continuous update algorithm can repetitively learn them online to enhance tracking accuracy, without employing upper/lower bounds of unknown parts for controller design and gain selections. Meanwhile, some non-periodic uncertainties are handled by a robust term, whose value is only related to the initial states of PAMs, instead of exact upper bounds of unknown dynamics. From safety concerns, we introduce error-related saturation terms to limit initial amplitudes of control inputs within saturation constraints and avoid overlarge errors inducing overlarge acceleration. Meanwhile, the constraint-related auxiliary term is utilized to keep tracking errors within allowable ranges. To the best of our knowledge, this paper presents the first learning-based error-constrained controller for uncertain PAM-actuated exoskeleton robots, to realize high-precision tracking control and improve safety without additional gain conditions. Moreover, the asymptotic convergence of tracking errors is strictly proven by Lyapunov-based stability analysis. Finally, based on a self-built exoskeleton robot, the effectiveness of the proposed controller is verified by hardware experiments. Note to Practitioners—This work is motivated by the practical requirements of exoskeleton robots in rehabilitation training and exploration fields. Currently, PAM systems, as a kind of new flexible actuator equipment, are playing increasingly important roles in the development of exoskeleton robot control. However, uncertain (or time-varying) parameters/structures and highly nonlinear dynamics, such as creep and hysteresis, may badly increase the control difficulty of PAMs. Moreover, higher and higher tracking accuracy and safety requirements also induce urgently solved problems to practical PAM-actuated exoskeleton robots, e.g., smooth start, motion constraints, and rapid error elimination. To this end, this paper proposes a new learning-based adaptive controller, which realizes accurate tracking control for PAM-actuated exoskeleton robots by utilizing an elaborately designed repetitive learning algorithm and a robust term to handle periodic and non-periodic uncertainties, respectively. More importantly, the proposed controller simultaneously enhances transient performance of PAMs, including gradually improved tracking accuracy, effective constraints for startup acceleration and tracking errors. Additionally, it is not required to consider the upper bounds of unknown dynamics and additional gain selection conditions, which is theoretically and practically important for PAM systems. Some hardware experiments further verify the effectiveness and robustness of the suggested controller. In our future work, we intend to design more effective methods for PAMs with unmeasurable states and time-delay.

It has been clinically proven that exoskeletons are effective self-training rehabilitation or daily living assistance devices for patients with hand dysfunctions. However, exoskeleton-assisted hand exercises with high degrees-of-freedom are considered as challenging tasks because the digit space, especially the thumb, cannot accommodate enough actuators. In this article, we report a tendon-driven soft hand exoskeleton with a hybrid configuration for thumb actuation. The soft hand exoskeleton system uses the least number of actuators to realize full degrees-of-freedom actuation for all digits. It is tested on a stroke patient with hemiplegia and a healthy subject. The experimental results show that the hand exoskeleton could assist the stroke patient to accomplish various training tasks, such as thumb encircling, grasping, pinching, releasing, and writing. It was found that digit trajectories and joint angle changes of the stroke patient were close to those of the healthy subject. Especially, the range of motion of the stroke patient shows significant improvement with the hand exoskeleton assistance compared to that without the hand exoskeleton assistance. The research in this article paves the way to develop fully actuated soft hand exoskeleton that can be eventually integrated with an electroencephalogram or electromyography for self-training rehabilitation or daily living assistance.

BACKGROUND: Combinatorial drug therapy for complex diseases, such as HSV infection and cancers, has a more significant efficacy than single-drug treatment. However, one key challenge is how to effectively and efficiently determine the optimal concentrations of combinatorial drugs because the number of drug combinations increases exponentially with the types of drugs. RESULTS: In this study, a searching method based on Markov chain is presented to optimize the combinatorial drug concentrations. In this method, the searching process of the optimal drug concentrations is converted into a Markov chain process with state variables representing all possible combinations of discretized drug concentrations. The transition probability matrix is updated by comparing the drug responses of the adjacent states in the network of the Markov chain and the drug concentration optimization is turned to seek the state with maximum value in the stationary distribution vector. Its performance is compared with five stochastic optimization algorithms as benchmark methods by simulation and biological experiments. Both simulation results and experimental data demonstrate that the Markov chain-based approach is more reliable and efficient in seeking global optimum than the benchmark algorithms. Furthermore, the Markov chain-based approach allows parallel implementation of all drug testing experiments, and largely reduces the times in the biological experiments. CONCLUSION: This article provides a versatile method for combinatorial drug screening, which is of great significance for clinical drug combination therapy.

Reinforcement learning (RL) is an efficient learning approach to solving control problems for a robot by interacting with the environment to acquire the optimal control policy. However, there are many challenges for RL to execute continuous control tasks. In this article, without the need to know and learn the dynamic model of a robotic manipulator, a kernel-based dynamic model for RL is proposed. In addition, a new tuple is formed through kernel function sampling to describe a robotic RL control problem. In this algorithm, a reward function is defined according to the features of tracking control in order to speed up the learning process, and then an RL tracking controller with a kernel-based transition dynamic model is proposed. Finally, a critic system is presented to evaluate the policy whether it is good or bad to the RL control tasks. The simulation results illustrate that the proposed method can fulfill the robotic tracking tasks effectively and achieve similar and even better tracking performance with much smaller inputs of force/torque compared with other learning algorithms, demonstrating the effectiveness and efficiency of the proposed RL algorithm.

A manipulation technique based on optothermally generated surface bubbles is proposed in this paper. The manipulation and assembly of microstructures are completed by using bubbles. In addition, the hydrogel microstructures are also used as microrobots driven by the bubble to operate and pattern the microspheres. Considering that many materials and lasers with different wavelength have been used for generating bubbles by optothermal effects, absorptivity and transmissivity are used as indicators of selections. Besides, the size of the bubble can be controlled by the frequency and time of the laser. This technique is supposed to be applied for manipulation of cells, microparticles and microstructures.

Bio-syncretic robots made up of living biological systems and electromechanical systems may have the potential excellent performance of natural biological entities. Therefore, the study of the bio-syncretic robots has got lots of attention in recent years. The 3D skeletal muscles have been used widely, due to the considerable contraction force and the controllability. However, the low differentiation quality of the C2C12 in the tissues hinders the broad application in the development of the skeleton muscle actuated bio-syncretic robots. In this work, an approach based on circular mould and rotary electrical stimulation to build high-quality muscle rings, which can be used to actuate various bio-syncretic robots, has been proposed. Firstly, the advantage of the proposed circular mould for the muscle rings culture has been shown by simulation. Then, the muscle rings have been fabricated with different moulds using the experiment-optimized compositions of the biological mixture. After that, the muscle rings in the circular moulds with different electrical stimulations have been cultured, to show the superiority of the proposed rotary electrical stimulation. Moreover, the contractility of the muscle rings have been measured under the different electrical pulses stimulation, for the study of the control property of the muscle rings. This work may be meaningful not only the development of bio-syncretic robots actuated by 3D muscle tissues but also the muscle tissue engineering.

In recent years, reinforcement learning has been developed dramatically and is widely used to solve control problems, e.g., playing games. However, there are still some problems for reinforcement learning to perform robotic control tasks. Fortunately, the kernel trick-based methods provide a chance to deal with those challenges. This work aims at developing a kernel trick-based learning control method to carry out robotic tracking control tasks. A reward system, in this work, is presented in order to speed up the learning processes. And then, a kernel trick-based reinforcement learning tracking controller is presented to perform tracking control tasks on a robotic manipulator system. To evaluate the policy and assist the reward system to accelerate the speed of finding the optimal control policy, a critic system is introduced. Finally, from the comparison with the benchmark, the simulation results illustrate that our algorithm has faster convergence rate and can execute tracking control tasks effectively, the reward function and the critic system proposed in this work is efficient.

The state-of-the-art online learning approaches are only capable of learning the metric for predefined tasks. In this paper, we consider a lifelong learning problem to mimic "human learning," i.e., endowing a new capability to the learned metric for a new task from new online samples and incorporating the previous experiences. Therefore, we propose a new metric learning framework: lifelong metric learning (LML), which only utilizes the data of the new task to train the metric model while preserving the original capabilities. More specifically, the proposed LML maintains a common subspace for all learned metrics, named lifelong dictionary, transfers knowledge from the common subspace to learn each new metric learning task with task-specific idiosyncrasy, and redefines the common subspace over time to maximize performance across all metric tasks. For model optimization, we apply online passive aggressive optimization algorithm to achieve lifelong metric task learning, where the lifelong dictionary and task-specific partition are optimized alternatively and consecutively. Finally, we evaluate our approach by analyzing several multitask metric learning datasets. Extensive experimental results demonstrate effectiveness and efficiency of the proposed framework.

Micro-robots have a great application prospect in the biomedical field due to the feature of small size. To solve the issues of energy supply and bio-compatibility of micro-robots, bio-syncretic micro-robots composed of biological materials and electromechanical systems have been studied widely. The skeletal muscle is a potential material to develop bio-actuator for the bio-syncretic robots on account of the great contraction force and the controllability. However, the low differentiation quality of C2C12s and the control of the bio-syncretic robots are the two of the main challenges for the development of the bio-syncretic robots based on the skeleton muscle. In this paper, an approach based on circularly distributed multiple electrodes (CDMEs) was proposed to improve the differentiation of C2C12 myoblast cells and characterize the electro-responsive beating behavior of myotubes for the development of bio-syncretic robots. Three groups of C2C12 blasts were used to fulfill the differentiation experiments without electrical stimulation and with electrical stimulation using parallel electrodes and CDMEs respectively, for evaluating the effect of CDMEs on C2C12 differentiation. It was demonstrated that electrical field through CDMEs can improve the differentiation quality of C2C12 blasts into myotubes in terms of intensity, length, and widths. Then, the effect of electrical stimulation on the beating behaviors of myotubes was also investigated with CDMEs, and it was shown that the beating amplitudes of myotubes were significantly affected by the frequencies, amplitude and direction of electrical stimulation with respect to the myotubes, which is fundamental for the control of the micro-robot based on skeletal muscle cells. The proposed approach is useful for not only the development of the bio-syncretic robots, but also the study of muscle tissue engineering.

Bio-syncretic robots, consisting of living biological materials and traditional electromechanical systems, have attracted lots of attention due to the potentialities of self-sensing, self-actuation and self-repairing with intrinsic safety and high energy conversion efficiency. However, most of current researches focus on the movement of the devices, and have ignored the study on the control of actuation unit “cells”, which is as important as motors for traditional electromechanical robots. In this work, the effects of cell culturing time, seeding concentration and functional drugs (cytochalasin and adrenalin) on contractile frequency and force strength of cardiomyocytes have been studied using scanning ion conductance microscope (SICM) and arrays of micro-pillars made of PDMS. This work will lay the foundation for the further study of quantitatively control of bio-syncretic robots actuated by cardiomyocytes and is also meaningful for the development of cytology, medicine, and clinical science.

In atomic force microscopy (AFM)-based nanomanipulation, the tip position uncertainties still exist due to the parameter inaccuracies in the open-loop compensation of the piezo scanner, the noise in the closed-loop control and thermal drift. These spatial uncertainties are very challenging to be directly estimated owing to the lack of real-time feedback, and its effects are more significant in performing an automatic nanomanipulation/assembly task than macro world manipulations. In this paper, we propose a stochastic framework for feature-based localization and planning in nanomanipulations to cope with these uncertainties. In the proposed framework, some features in the sample surface are identified to calculate their positions in statistics, and detected by using the AFM tip as the sensor itself through a local scan-based motion. In the localization, the Kalman filter is used through incorporating the tip motion model and the local scan-based observation model to estimate the on-line tip position in the task space. The simulation and experiments about tip positioning are carried out to illustrate the validity and feasibility of the proposed algorithm. Then, positioning tip for effective nanomanipulation is presented by using several experiments. Finally, a carbon nanotube is followed to show that the proposed method can provide a great potential for improving the position accuracy.

Successful computer-aided diagnosis systems typically rely on training datasets containing sufficient and richly annotated images. However, detailed image annotation is often time consuming and subjective, especially for medical images, which becomes the bottleneck for the collection of large datasets and then building computer-aided diagnosis systems. In this article, we design a novel computer-aided endoscopy diagnosis system to deal with the multi-classification problem of electronic endoscopy medical records (EEMRs) containing sets of frames, while labels of EEMRs can be mined from the corresponding text records using an automatic text-matching strategy without human special labeling. With unambiguous EEMR labels and ambiguous frame labels, we propose a simple but effective pooling scheme called Multi-class Latent Concept Pooling, which learns a codebook from EEMRs with different classes step by step and encodes EEMRs based on a soft weighting strategy. In our method, a computer-aided diagnosis system can be extended to new unseen classes with ease and applied to the standard single-instance classification problem even though detailed annotated images are unavailable. In order to validate our system, we collect 1,889 EEMRs with more than 59K frames and successfully mine labels for 348 of them. The experimental results show that our proposed system significantly outperforms the state-of-the-art methods. Moreover, we apply the learned latent concept codebook to detect the abnormalities in endoscopy images and compare it with a supervised learning classifier, and the evaluation shows that our codebook learning method can effectively extract the true prototypes related to different classes from the ambiguous data.

This paper proposed a phase-closed-loop nanomachining control method to realize the directly control of machining depth based on ultrasonic vibration-assisted AFM. By using applied force to control the machining depth, conventional AFM machining approaches unable to machining a nanostructure with specified machined depth. With the proposed method, the vibration phase of micro-cantilever has a specific relationship with machining depth. Therefore, the nano-grooves with desired depth can be machined by using phase value as feedback of PID control. In this paper, the theoretical analysis and simulation are carried out, and the experiments of phase-closed-loop control method are conducted. The experimental results verify the primary feasibility of the proposed method. The present method also demonstrates the potential on the fabrication of three-dimension nanostructures and nanoelectronic device.

Cellular mechanical properties as the main physical performance characteristics have been actively studied in the past years for the study of cytobiology and the development of medicine. In this study, by combining Hertz model, a novel strategy is proposed to simultaneously measure the cellular mechanical properties including cellular mass, elasticity and viscosity, based on the principle of forced vibration stimulated by simple harmonic force, with piezoelectric transducer (PZT) as vibrator and Atomic Force Microscope (AFM) as detector. The corresponding theoretical model was derived and the simulation was realized based on the proposed model. The experiments of indentations and vibrations with myoblasts and myotubes were implemented to calculate the three mechanical parameters of cells according to the proposed strategy. The results validated the proposed approach. This work would be useful for the development of cytology, medicine, previously diagnose, specific therapy and so on.

Nanoscale vibration, a critical nanomechanical property of cell membranes/walls, is a crucial aspect of cell physiology. However, limitations of current nanoscale vibration detecting methods remain the major obstacle for scientific study and cell vibration experiments. Due to the absence of effective method of feeble nanoscale vibration detecting, most sorts of quantitative and dynamic cell vibrations cannot be observed. Therefore, a real-time tracking detection method is vital for the study of cell physiology. In this paper, a real-time tracking detection of nanoscale vibrations based on sweep frequency (SF) - amplitude modulation (AM) method using cantilever sweep frequency as a carrier frequency was proposed. Furthermore, the process of tip-sample vibration coupling is analyzed by using the idea of amplitude modulation model. The nanoscle vibration detecting experiments were carried out on a piezoceramic disc, which can mimic cell vibrations. The experiment results show that the SF-AM AFM real-time vibration tracking and detecting approach can accurately detect and track feeble sample vibration within few nanometers amplitude.

The determination of the mechanical properties of cells plays an important role in biological studies and has gained acceptance recently as a possible label-free biomarker for cell status determination or diseases detection. Investigations on how external cellular properties affect cell mechanics are helpful in understanding cell disease processes and cell morphogenesis, which are of large significance in medical science. Although most researchers have focused on individual cell mechanics, or the effect of substrate stiffness on cells, cell mechanical response due to interactions among cells is yet to be examined. A reason for this is that the study of cell mechanical response to cell shape requires one to use a cell patterning process. However, existing cell patterning methods are very complex and time-consuming. In this paper, we describe a practical and rapid technique that can easily pattern cells into desired shapes, which allows investigations of the effect of external environment on cell stiffness. In the new technique, Poly-(ethylene) glycol diacrylate (PEGDA) hydrogel film with thickness 70–100 nm is controllably patterned on a hydrogenated amorphous silicon (a-Si:H) substrate by polymerizing PEGDA molecules in-situ using programmable visual light patterns. The idea is to enable the confinement of cells cultured on the hydrogels into special areas. The elastic modulus of the patterned cells is measured using an atomic force microscope. Experimental results have demonstrated the versatility of the technique as a tool for cell pattering and exploration of cell mechanics under external mechanical stimuli.

The nonlinearities such as hysteresis and creep are the major factors inherent in PZT actuation that affect the tip positioning precision and manipulation performance of the AFM system. In this study, an extended PI model is generalized by introducing a creep model to the basic hysteretic operator of the PI model at the inflexion point of the hysteresis loop. Unified compensation for hysteresis and creep can be implemented with the extended PI model. Experiment results demonstrate the validity and effectiveness of the extended PI model and it is implied that the inflexion creep compensation not only improves the tip positioning precision at the inflexion points on the hysteresis loops, but also the localization effectiveness during the whole process of PZT actuation.

One of the major limitations for Atomic Force Microscopy (AFM)-based nanomanipulation is that AFM only has one sharp tip as the end-effector, and can only apply a point force to the nanoobject, which makes it extremely difficult to achieve a stable manipulation. For example, the AFM tip tends to slip-away during nanoparticle manipulation due to its small touch area, and there is no available strategy to manipulate a nanorod in a constant posture with a single tip since the applied point force can make the nanorod rotate more easily. In this paper, a robotic nano-hand method is proposed to solve these problems. The basic idea is using a single tip to mimic the manipulation effect that multi-AFM tip can achieve through the planned high speed sequential tip pushing. The theoretical behavior models of nanoparticle and nanorod are developed, based on which the moving speed and trajectory of the AFM tip are planned artfully to form a nano-hand. In this way, the slip-away problem during nanoparticle manipulation can be get rid of efficiently, and a posture constant manipulation for nanorod can be achieved. The simulation and experimental results demonstrate the effectiveness and advantages of the proposed method.

The drift distorts the atomic force microscopy (AFM) images as the time taken to acquire a complete AFM image is relatively long (a few minutes). As the AFM image is used as a reference for most manipulation mechanisms, the image distorted by drift will cause problems for AFM-based manipulation because the displayed positions of the objects under nanomanipulation do not match their actual locations. The drift during manipulation, similarly, will further exacerbate the mismatch between the displayed positions and the actual locations. Such mismatch is a major hurdle to achieve automation in AFM-based nanomanipulation. Without proper compensation, manipulation based on a wrong displayed location of the object often fails. In this paper, we present an algorithm to identify and eliminate the drift-induced distortion in the AFM image by applying a strategic local scan method. Briefly, after an AFM image is captured, the entire image is divided into several parts along vertical direction. A quick local scan is performed in each part of the image to measure the drift value in that very part. In this manner, the drift value is calculated in a small local area instead of the global image. Thus, the drift can be more precisely estimated and the actual position of the objects can be more accurately identified. In this paper, we also present the strategy to constantly compensate the drift during manipulation. By applying local scan on a single fixed feature in the AFM image frequently, the most current positions of all objects can be displayed in the augmented reality for real-time visual feedback.

One of the prerequisite conditions for making a successful manipulation is that the relative position between the AFM tip and the objects can be sensed and controlled accurately. While this prerequisite is grandly hampered by the PZT nonlinearity and thermal drift. Although the PZT nonlinearity can be compensated to a certain extent through mounting a position sensor on the PZT scanner, this method leads to a higher system noise and a higher cost. In addition, this method can not handle the positioning error caused by thermal drift due to the lack of sensing ability to the displacement between the AFM tip and the sample stage. This paper propose a newly developed strategy to solve these problems. Its pivotal idea is the tip position is localized based on the sensing information to sample features, not PZT driving voltage or sensor signal. In this way, the positioning error aroused from PZT nonlinearity and thermal drift can be effectively suppressed. Experimental results demonstrate the advantage and effectiveness of the proposed method.

Because of the presence of thermal drift, AFM (atomic force microscopy) images are always contaminated. Such contamination is one of the major hampers to achieve accurate and efficient AFM based nanomanipulation. Based on contaminated images, the manipulation operations often fail. In this paper, we apply a local scan method to identify and compensate the thermal drift contamination in the AFM image. After an AFM image is captured, the entire image is divided into several parts along y direction. A local scan is immediately performed in each part of the image to calculate the drift value at that very part. In this manner, the drift value is calculated in a small local area instead of the global image. Thus, the drift can be more precisely estimated and the image can be more accurately recovered, which lead to improved accuracy for AFM imaging and enhanced productivity for AFM based nanomanipulation.

One of the main roadblocks to Atomic Force Microscope (AFM) based nanomanipulation is lack of real time visual feedback. Although the model based visual feedback can partly solve this problem, its unguaranteed reliability due to the inaccurate models in nano-environment still limits the efficiency of AFM based nanomanipulation. This paper introduce a Real-time Fault Detection and Correction (RFDC) method to improve the reliability of the visual feedback. By utilizing Kalman filter and local scan technologies, the RFDC method not only can real-time detect the fault display caused by the modeling error, but also can on-line correct it without interrupting manipulation. In this way, the visual feedback keeps consistent with the true environment changes during manipulation, which makes several operations being finished without a image scanning in between. The theoretical study and the implementation of the RFDC method are elaborated in this paper. Experiments of manipulating nano-particles have been carried out to demonstrate the effectiveness and efficiency of the proposed method.

Atomic Force Microscope (AFM) has been used as a manipulation tool for a decade. The problem of lacking real time visual feedback still limits its efficiency and hinders its wide application. Although the model based visual feedback can partly solve this problem, due to the complexity of nano environment, it is difficult to use a model to accurately describe the object’s behavior. The modeling error will give the operator a false feedback and lead to a failed manipulation. In this paper, a strategy for visual feedback error on-line detection and correction is proposed to solve this problem. As the real time force information is a key factor for this strategy, an adaptable end effector is employed to accurately measure the interaction force between the probe and the nano-objects, and the system error is also compensated to improve the accuracy of interaction force measurement. Based on the true real time force information, an extended Kalman filter is developed to online detect whether there is a false feedback. Once a false feedback is detected, an optimal searching pattern is generated to get the real manipulation result in a short time. With the assistance of this strategy, the false visual feedback can be realtime detected and corrected without interrupting manipulation. Complex manipulation task can be finished without being interrupted by a new image scan. Experiments of manipulating nano-particles are performed to verify the effectiveness of this strategy, which demonstrated the improved efficiency of the AFM based nano-assembly system.