Aphasia following stroke affects millions globally, yet rehabilitation remains severely limited by speech therapist shortages. Existing digital systems rely on static video demonstrations, single-modality assessment, and rule-based feedback, failing to address authentic clinical needs. Through formative investigation with five therapists, three patients, and their caregivers, we identified concrete clinical challenges: therapists spending 30-40% of time on repetitive demonstrations, existing tools providing only speech scores without articulatory evaluation, and patients struggling with complex interfaces and monotonous content. To address these challenges, we developed an integrated rehabilitation system combining an embodied digital therapist for Action Observation Therapy, tri-dimensional assessment coordinating speech quality, lip movement accuracy, and semantic understanding, and large language model-driven personalization for content generation and adaptive training. We conducted a proof-of-concept evaluation across two in-situ hospital training sessions with six patients, six caregivers, and three therapists. Results demonstrated substantial efficiency gains, with therapists spending 69-78% less time per patient. Patient acceptance improved 42.6% across sessions, and low digital literacy patients showed steepest gains (+69.0%). However, human intervention remained necessary for 24-30% of session time to provide emotional support. These findings empirically characterize human-AI collaboration boundaries in clinical rehabilitation, revealing both automation’s potential to enhance efficiency and bridge digital divides, and the persistent necessity of human therapeutic presence—providing evidence for responsible deployment of AI-assisted healthcare systems.

The proliferation of low-cost, portable sensors has made wearable human activity recognition (HAR) a cornerstone for real-time health monitoring and behavior analysis. However, deploying accurate yet lightweight deep learning models on resource-constrained wearable devices poses a significant challenge for on-device activity recognition. While channel pruning is a common solution to accelerate deep Convolutional Neural Networks (CNNs), existing works often require specialized implementations or pre-trained models, which potentially degrade performance by simply removing an entire channel, limiting their ability to handle complex multimodal sensor inputs. Moreover, lightweight CNN design, particularly the heavy use of 1×1 convolution layers for channel squeezing, remain inefficient for sensor-based HAR, which consume resources without expanding the receptive field due to their pointwise nature. To address these issues, we propose a novel dynamic channel sampling module, Select-and-Fuse (SaF), specifically designed for sensor-based HAR. SaF divides channels into subsets and performs a dynamic, input-dependent selection from them, with the picking decision being made per-time-step based on the input sensor signal activations, allowing for fine-grained feature adaptation to multi-modal sensor signals. While integrated into compact backbones, SaF significantly reduces model size and inference latency while maintaining high accuracy. Extensive evaluations on public UCI-HAR, OPPORTUNITY, WISDM, and UniMiB-SHAR benchmarks confirm a favorable performance-cost trade-off. Crucially, we measure actual inference latency on a Raspberry Pi, proving its practicality for resource-constrained HAR applications. Code will be released.

Lightweight CNNs and Transformers have shown great promise in sensor-based human activity recognition (HAR), yet their structural synergies remain underexplored. This paper bridges this gap by integrating the MetaFormer paradigm—a general architecture abstracted from Transformers that structurally separates token mixing (i.e., self-attention) and channel mixing (i.e., feed-forward networks)—into efficient CNN design. While MetaFormer offers a powerful inductive bias, its standard self-attention mechanism is often computationally intensive for resource-constrained HAR. To address this, we revolutionize the classic MobileNetV3 architecture from a MetaFormer perspective, introducing Rep-MMB, a new family of pure lightweight CNNs. By leveraging structural reparameterization, Rep-MMB decouples multi-branch training-time complexity from efficient single-branch inference, enabling high accuracy with low latency. Evaluations on four public HAR benchmarks show that Rep-MMB outperforms state-of-the-art lightweight models in accuracy and efficiency, with practical validation on embedded devices. We hope that Rep-MMB may serve as a strong baseline to inspire future edge-deployed HAR research.

Human Activity Recognition (HAR) plays a pivotal role in ubiquitous computing. However, it remains constrained by the challenge of balancing fine-grained temporal modeling with real-time efficiency on resource-limited devices. While Transformer-based models excel at capturing long-range dependencies, they suffer from high computational costs, limiting their applicability on resource-constrained devices. Linear attention mechanisms improve efficiency but often discard negative signals and produce overly smooth, high-entropy attention distributions, impairing the extraction of fine-grained patterns and degrading classification accuracy in complex scenarios. In this work, we present ActiFormer, a novel sign-aware linear attention framework tailored for sensor-based HAR to overcome these limitations. To preserve bidirectional signal dynamics, we introduce Sign-Aware Attention, which explicitly models both same-sign and cross-sign interactions between queries and keys, effectively retaining negative signals crucial for accurate recognition. Furthermore, we propose a learnable entropy-scaling function that compensates for the exponential scaling effect lost in linear attention, originally provided by softmax, solving the high-entropy attention weight issue by amplifying the importance of critical temporal points. Extensive experiments on four benchmark HAR datasets demonstrate that ActiFormer consistently outperforms CNNs, standard Transformers, and state-of-the-art linear attention models, both in accuracy and efficiency. Its lightweight design supports real-time inference on edge devices such as the Raspberry Pi 5, highlighting its practical deployability in real-world applications.

Transformer models have demonstrated superior capability in capturing long-range temporal dependencies crucial for Sensor-Based Human Activity Recognition (HAR). However, the quadratic computational complexity inherent to the Softmax-Attention mechanism significantly impedes their deployment on resource-constrained wearable devices and real-time streaming tasks. To address this, we propose a novel Linear Transformer with Taylor Series Attention specifically tailored for the HAR domain, named TSA-Former. It leverages the first-order Taylor expansion to approximate the Softmax-Attention and utilizes the norm-preserving mapping to approximate the high-order non-linear information, resulting in a linear computational complexity. In addition, TSA-Former integrates a multi-branch architecture featuring multi-scale patch embedding, which enables the model to dynamically capture multi-scale temporal features while minimizing overhead. Experimental results across four public HAR benchmarks, namely UniMiB-SHAR, UCI-HAR, WISDM, and OPPORTUNITY, demonstrate that TSA-Former achieves state-of-the-art (SOTA) accuracy and efficiency, outperforming conventional Transformers and existing linear-attention models. Deployment experiments conducted on the Raspberry Pi 5 platform further validate the model’s superior low-latency and minimal power consumption profile, confirming its robust suitability for real-world embedded HAR applications. Code will be released.

Human Activity Recognition (HAR) aims to classify human behaviors from large-scale sensor data. A key challenge is to achieve high recognition accuracy while maintaining low computational cost. Recent advances such as Mamba address this by enabling long-range dependency modeling with subquadratic computational complexity, thus achieving strong representational capacity at reduced cost. However, when directly applied to HAR tasks, lightweight Mamba-based backbones often underperform compared to conventional CNN and Transformer architectures. To investigate this gap, we perform detailed temporal and spectral analyses, revealing that Mamba exhibits an inherent bias towards low-frequency components. In contrast, HAR sensor signals typically comprise a mixture of both high- and low-frequency information, both of which are crucial for accurate activity recognition. To address this limitation, we propose Machar, a novel lightweight MAmba-Convolution Hybrid ARchitecture specifically designed for HAR. Instead of relying solely on global modeling, Machar introduces a dedicated FreqDecoupler that decomposes sensor signals into high- and low-frequency components, enabling each to be processed by the most appropriate mechanism. Furthermore, we propose a frequency scheduling strategy that dynamically adjusts channel capacity allocation across network stages, effectively combining the local feature extraction capability of CNNs with Mamba’s global modeling strength. Extensive experiments on three widely used HAR benchmarks, namely USC-HAD, UCI-HAR, and UniMiB-SHAR, show that Machar consistently outperforms existing methods, achieving impressive accuracy while preserving a favorable computational footprint, which underscore the effectiveness and scalability of Machar for real-world HAR applications.

Attributing to the strong reasoning capability, behavior models in artificial intelligence for games play a crucial role in creating gaming experiences. For further enhancing the reasoning capability of behavior models, we propose a novel Switch, Reason, and Revise (SRR) framework, which integrates them with Large Language Model (LLM). The SRR framework contains three core components. The component of Dual-Track Experiential Reasoning fully utilizes the agent experiences for reasoning. The component of Block-Retrieval-Augmented Thoughts adaptively determines the granularity of information retrieval for external sources. The component of Self-Reliant Thinking System Switch increases the reasoning speed by model switching and performs the model switching automatically upon the LLM. Together, these three components can strengthen the agent reasoning capability in complex tasks. Experimental results in the Pokémon battle environment demonstrate the effectiveness and efficiency superiority of SRR over the rival methods. Furthermore, we conduct an exploratory study to reveal the potential of the SRR-empowered agent for guiding new players in Pokémon battle games.

Multiagent reinforcement learning (MARL) methods have good application performances and prospects in cooperative tasks. To improve the capability of agent policy learning in new scenarios, some methods transfer the learned policy knowledge to new scenarios. However, most methods only focus on the knowledge transfer of individual agent policies, neglecting the credit assignment among agents in cooperative tasks, which results in a transfer bias of cooperative policies. In this paper, we propose a novel method, transferable policy generation and credit assignment (TPGCA) network for cooperative MARL. TPGCA can transfer the entire MARL model by the constructed transferable$Q$-value network and mixing network. Specifically, in TPGCA, to enhance the effectivity and transferability of agent policies, we design the correspondence network between observations and actions (COA) on the basis of transformer and gated recurrent unit (GRU). To implement the reliable credit assignment and diminish the transfer bias, we devise the role-based joint$Q$-value decomposition network (RVD) that can evaluate the contributions of agents from different observation perspectives. Experimental results in various micro-management scenarios on StarCraft multiagent challenge (SMAC) and multiagent particle environment (MPE) sufficiently demonstrate the effectiveness and transferability of TPGCA.

Despite the progress in human–computer interaction technology, the interaction methods that visually impaired individuals possess are still rudimentary. The widely used Text-to-Speech technology has issues such as privacy leaks in practical applications, and most of the new interaction designs proposed in recent years have limited application scenarios. Geared toward enriching interaction methods for users with visual impairments, this article explores the potential for translating Braille into vibrotactile cues as a way of conveying universal information. We designed a set of schemes and implemented them based on the vibration motor of a mobile phone. These schemes convert a single Braille character into several highly distinguishable vibration combinations, thereby conveying any information that Braille can express. Experiments were conducted on both sighted and visually impaired participants to evaluate the accuracy and efficiency. With a brief learning period of just 10 minutes, individuals can attain an accuracy rate greater than 95%, and the accuracy degradation remains minimal when playback speeds increase. By employing vibration motors to deliver comprehensive information, this framework shows promise for application in a wider range of technological devices.

The efficient integration of visual and tactile information is essential for slip detection and evaluation of grasping stability. However, existing research generally combines visual or tactile modalities as prior information, without fully exploring mechanisms for fusing complementary modalities. In this article, we propose a visual–tactile fusion Transformer (VTF-Trans) for slip detection, designed to handle unaligned data in different formats and facilitate cross-modal information exchange. The main advantages of the proposed method are summarized as follows: first, VTF-Trans employs an improved dual-stream transformer for feature extraction. In addition, we introduce a gated modal attention module to further refine cross-modal fusion. Compared with the existing methods, VTF-Trans effectively integrates useful information from different modalities across multiple scales. Second, to extract deep multimodal information, we propose a cross-modal attention (CMA) mechanism. By defining cross-affinity based on single-modality affinities (token metrics), CMA naturally alleviates the gap between different modalities and domains. Third, we evaluate VTF-trans on three datasets and conduct unknown object grasping experiments. Compared with the state-of-the-art methods, VTF-Trans achieves the highest accuracy for robotic grasping and slip detection, highlighting its superior performance and practical applicability.

Lightweight convolution-based neural networks (CNNs) are well suited for sensor-based human activity recognition (HAR) applications on resource-constrained edge devices with faster inference speed. However, the convolutional kernels are often limited to a small window range, which can only capture local details in time series sensor data, thus preventing further performance boost. Though Introducing self-attention into convolution can help to handle long-range dependence well, it might significantly slow down actual activity inference speed, due to high computational cost. In this paper, we introduce a new learning paradigm (star operation) and then present a lightweight Dual-Branch High-Order Interactions (DbHoi) block, which is computationally friendly for mobile HAR deployment. The proposed DbHoi block may implicitly transform raw sensor inputs into high-dimensional non-linear features, but actually operate in a low-dimensional feature space (analogs to the design principle of polynomial kernel tricks), without incurring extra computational overhead. Extensive experiments are conducted on three public HAR benchmarks including UCI-HAR, UniMiB-SHAR, and OPPORTUNITY, which demonstrate that our suggested DbHoi can consistently surpass various meticulously designed lightweight networks such as MobileNet, ShuffleNet, and GhostNet. Detailed ablation studies, visualizing representations, and on-device latency analyses further validate our insights with regards to the star operation, while underscoring its practical merit in real-world HAR deployment.

Inspired by recent success of foundation models in vision and language domains, time series foundation models (TSFMs) have garnered increasing attention in general time series analysis tasks like finance, weather, healthcare, and power. However, given high heterogeneity and severe annotation scarcity in time series sensor data, how to unlock the potential of large-scale general-purpose TSFMs for downstream activity recognition tasks remains yet unexplored? This paper makes the first attempt to address this timely challenge by adapting the self-supervised pre-trained TSFM (i.e., MOMENT) to few-shot activity recognition. We introduce a simple and efficient Sensor-Prompt Tuning (SPT) strategy, which employs multiple convolution-based sensor-friendly filters with a gating mechanism to act as learnable soft prompts, which can dynamically adapt sensor input space to the frozen TSFM backbone, effectively bridging domain gap between pre-training general time series data with wearable sensor stream. Extensive experiments across three public activity recognition benchmarks demonstrate that our SPT achieves up to 15.5% performance gains over existing state-of-the-art baselines under few-shot scenarios, while considerably outperforming other mainstream fine-tuning strategies with smaller than 1% of backbone parameters. Practical cloud-edge inference latencies are measured. This work offers a new prompt-tuning perspective on how to adapt pre-trained TSFMs for wearable activity recognition tasks. Code will be released.

Image-text retrieval aims to align image regions with textual words for semantic matching, facilitating bidirectional retrieval between images and texts. While significant progress has been made in modeling both coarse-grained image-sentence and fine-grained region-word relationships, fully capturing multi granularity correspondences remains a challenge. Many existing methods predominantly depend on region-level segmentation or recognition, which tends to introduce noise, compromise semantic consistency, and increase computational complexity, ultimately limiting retrieval performance. To address these issues, we propose a Multi-constraint Relational Semantic Alignment (McRSA) method, which incorporates three complementary loss-based constraints to enhance multi-granularity alignment while preserving complete information. Specifically, the method includes Posterior Probability Estimation (PPE), which utilizes Bayesian analysis to model causal relationships between image-text feature pairs and labels, reducing intra-class variations for fine-grained alignment. Additionally, a Momentum-driven Centroid Update (MCU) mechanism is introduced to mitigate oscillations and improve modal consistency in coarse-grained representations. A dynamic Feature Scale Adaptation (FSA) module is also employed, adjusting feature scales across modalities to alleviate granularity discrepancies and improve alignment robustness. Extensive experiments on five public datasets (Flickr30K, MS-COCO, RSTPReid, CUHK-PEDES, and ICFG-PEDES) demonstrate that McRSA achieves competitive retrieval performance compared to existing methods. Code and pre-trained models are available at https://github.com/xiaoyiseu/McRSA.

Amplitude tuning is an important strategy in animals employing traveling wave patterns, enhancing their adaptability to unstructured environments. This paper proposes, for the first time, an amplitude tuning method that leverages the compliance of a soft corkscrew by twisting its ends. An amplitude tunable traveling wave robot (ATWBot) is developed, consisting of a soft corkscrew housed in a high DOF cage and driven by only two servos. The soft corkscrew and cage are monolithically 3D-printed. ATWBot achieves a wide range of active amplitude tuning with passive compliance adaptation, and can extend its morphology to a coiled configuration, enabling clamping and rolling. A comprehensive model is built for the twisted soft corkscrew geometry, proving that the robot's speed is decoupled from amplitude variations during twisting. A genetic algorithm is used to optimize the soft corkscrew for achieving the fastest speed while matching the cage geometry. Experiments demonstrate that the combination of active amplitude tuning and passive body compliance enables the robot to adapt to unstructured terrains including slits, steps, gaps, converging tunnels, slopes, and swimming.

In bilateral teleoperation systems, achieving fast position synchronization while maintaining high-communication efficiency represents an essential control objective. This article proposes a simplified control scheme with a finite-time proportional damping (FTP+d) injection method and an event-driven communication mechanism to address the challenge of position synchronization in time-delayed telerobotic systems. The system takes into account the asymmetric time-varying delays and external forces. First, a novel event-triggered strategy based on noninteger power is designed. With a designed error-velocity mixed variable, a new FTP+d controller is developed by employing a continuous nonsmooth function, a damping term, an adaptive term, and noninteger power. With the Lyapunov–Krasovskii (LK) method and the finite-time stability theory, the boundedness of the telerobot’s state is established, and the finite-time synchronization performance is analytically guaranteed. Furthermore, the relationships between controller parameters, time-varying delays, and system stability are also analyzed based on linear matrix inequality (LMI). Finally, both simulation and experimental investigations are conducted to evaluate the efficacy and performance of the proposed control approach.

In order to obtain a good tactile sensing, traditional dexterous hands always enable all the sensing units installed on them all the time, even if just a few sensor units are actually used, which make the tactile sensing system resource-wasting and energy consuming. In order to reduce their complexities by placing the tactile sensing units only at critical locations, this work proposes an embodied tactile dexterous hand (ET-Hand) and a novel multimodal sensor placement framework that learns multiple tasks to generate optimal placement proposal. Furthermore, our ET-Hand can dynamically adjust the perceived tactile sensor positions, types and numbers during robotic manipulation, providing novel tools and methods for investigating the tactile channels and placement scale required for robot exploration. In the object recognition and slip detection tasks, the results show that our proposed method performs close to or even better than traditional sensing way with large-scale placement.

Vision-based tactile sensors have recently gained prominence due to their superior resolution and ability to capture multi-dimensional contact information. However, even when sensors share the same sensing principle, variations in production factors can lead to differences in the color patterns of tactile signals. Unlike common vision tasks, vision-based tactile perception depends on tracking light variation in colorful signals, making it more susceptible to lighting conditions and thus more prone to domain gaps. In this paper, we propose an Omni-hardness perception framework that enables adaptation across various vision-based tactile sensors. Firstly, in-depth analyses of the factors influencing the generalization of hardness perception are presented. Furthermore, the light balance module and the force scale module are coupled to regulate network learning of generalized representations. Experimental results across multiple sensors demonstrate the transferability of learned representations. Additionally, downstream tasks in natural object perception, tumor detection, and grasping stability prediction, are proposed to evaluate the potential applications. The framework's performance shows promise for advancing general tactile sensing and embodied tactile perception.

Modular robotic structures simplify robot design and manufacturing by using standardized modules, enhancing flexibility and adaptability. However, the need for manual input in design and assembly limit their potential. Current methods to automate this process still require significant human effort and technical expertise. This paper introduces a novel approach that employs Large Language Models (LLMs) as intelligent agents to automate the creation of modular robotic structures. We decompose the modular robot creation task and develop two agents based on LLM to plan and assemble the modular robots from text prompts. By inputting a textual description, users can generate robot designs that are validated in both simulated and real-world environments. This method reduces the need for manual intervention and lowers the technical barrier to creating complex robotic systems.

Slow-speed animals can also exhibit remarkable capabilities, as seen in snails that crawl while maintaining adhesion. Snails have inspired researchers to develop traveling wave-based robots and suction robots; however, the combination of traveling wave propulsion with suction ability remains a challenge. In this paper, we propose a snail-inspired robot that integrates a corkscrew propulsion mechanism with distributed suction cups, enabling it to crawl upside down on the ceiling. The propulsion model of the corkscrew generating the traveling wave is derived, and a temporal-spatial decomposition method is applied to validate the high efficiency of traveling wave generation. The trade-off between wave amplitude and suction cup depth is investigated to determine an optimized configuration. The results show that the robot’s speed aligns well with the propulsion model. The traveling wave ratio calculated from experiments is 0.938. The optimized configuration consists of a corkscrew with a 14 mm diameter and suction cups with a 2.5 mm depth, achieving a crawling speed of 3.02 ± 0.28 mm/s while moving upside down. The combination of the proposed smooth traveling wave generation method and distributed suction cups enables the robot to crawl upside down while carrying a 200 g load and to climb a vertical wall, like a natural snail.

In order to provide natural and immersive interactive experience for teleoperation in the context of human-robot collaboration and interaction, this work introduces a natural human-robot interaction system for teleoperation based on ultrasonic haptic feedback. Specifically, our system can accurately capture an operator's hand movements and replicate these actions on the remote robot with low latency and high fidelity. It utilizes an ultrasonic phased array to achieve non-contact haptic feedback. We propose a dynamic ultrasonic array acoustic field customization method based on interactive feature information image. This method can dynamically adjust the acoustic field according to the operator's hand characteristics, focus on multiple target points in real time, and project them onto an operator's fingertips, thereby providing force-controllable non-contact haptic feedback to the operator. The operator is integrated into the feedback loop of our system, controlling the system through multimodal feedback to form a high-quality human-in-the-loop closed control system. The system's performance is validated in two classic robotic tasks: block pick-and-place and nut-tightening. The experimental results show that the system exhibits excellent accuracy and dexterity, and can efficiently complete tasks with high accuracy while providing great interactive experience for operators.

Traditional magnetic tactile sensors are highly susceptible to external magnetic field interference, limiting their reliability in practical applications. To address this challenge, we propose a dual-modal soft magnetic skin capable of simultaneously acquiring magnetic and force tactile information across spatiotemporal domains, inspired by the sensory mechanisms of human skin. The system integrates a Convolutional Neural Network-Convolutional Neural Network-Multilayer Perceptron (CNN-CNN-MLP) architecture to fuse these dual-modal signals effectively. Furthermore, we introduce a novel Dynamic Weighting Coefficient Layer (DWCL) to dynamically optimize fusion weights for each modality based on real-time input characteristics, thereby enhancing robustness against magnetic interference. The DWCL leverages temporal discrepancies between modalities during pre-contact sensing and quantifies the magnetic field strength of target objects to autonomously adjust fusion ratios, prioritizing the more reliable modality under varying interference conditions. Extensive experimental evaluations demonstrate that the proposed DWCL significantly improves interference resistance compared to conventional fusion methods, advancing the feasibility of magnetic tactile sensing in real-world environments.

Chinese acupuncture practitioners primarily depend on muscle memory and tactile feedback to insert needles and accurately target acupuncture points, as the current workflow lacks imaging modalities and visual aids. Consequently, new practitioners often learn through trial and error, requiring years of experience to become proficient and earn the trust of patients. Medical students face similar challenges in mastering this skill. To address these challenges, we developed an innovative system, MRUCT, that integrates ultrasonic computed tomography (UCT) with mixed reality (MR) technology to visualize acupuncture points in real-time. This system offers offline image registration and real-time guidance during needle insertion, enabling them to accurately position needles based on anatomical structures such as bones, muscles, and auto-generated reference points, with the potential for clinical implementation. In this paper, we outline the non-rigid registration methods used to reconstruct anatomical structures from UCT data, as well as the key design considerations of the MR system. We evaluated two different 3D user interface (3DUI) designs and compared the performance of our system to traditional workflows for both new practitioners and medical students. The results highlight the potential of MR to enhance therapeutic medical practices and demonstrate the effectiveness of the system we developed.

Recently, deep convolutional neural networks (CNNs) have achieved outstanding success in sensor-based human activity recognition (HAR) scenario, but at the cost of huge computational complexity, thereby restricting their practical deployment on resource-limited wearable devices. This may be partly attributed to static nature of most existing CNNs, which process all activity samples uniformly, resulting in structural and data redundancy. Comparing to static networks, one promising strategy is to accelerate activity inference by exploiting structural redundancy within deep CNNs, which selectively activates computation units such as convolution channels while handling different samples. The other promising strategy is to explore spatiotemporal redundancy by concentrating computational effort on the most informative regions of sensor data. How to simultaneously leverage structural and data redundancy still remains largely overlooked. In this article, from a new perspective of exploring both structural and spatiotemporal redundancy, we introduce an efficient spatiotemporal-structural masker network (SSMNet) for activity recognition. It utilizes a dual-mask mechanism to make dynamic, sample-specific decisions, thereby accelerating activity inference. The spatiotemporal-structural masker integrates spatiotemporal and structural decisions through masks, dynamically allocating computational resources based on input with minimal overhead. Extensive experiments on three public HAR benchmark datasets, namely, WISDM, UniMiB-SHAR, and PAMAP2. SSMNet is guided by a high-accuracy static model, allowing it to reduce computational costs while maintaining state-of-the-art performance. For example, comparing to static baselines, it may reduce nearly 40% FLOPs with an accuracy drop smaller than 1%, across all three datasets The detailed analyses affirm that our method can strike an optimal tradeoff between accuracy and efficiency.

It requires terrain classification for unmanned Mars Rover to identify the safe areas. The current deep learning-based semantic segmentation and object recognition suffer from a large number of parameters and long training time. In this paper, a lightweight segmentation framework called Mobile-DeepRFB is proposed for the Martian terrain classification. It improves from the DeepLabV3$+$by taking the MobileNetV3 as the backbone module to decrease the parameters and the Receptive Field Block (RFB) module to strengthen the feature extraction capability as well as to enlarge the receptive field. Experimental results on the NASA Mars terrain dataset AI4MARS show that the presented method reduces 94% about the parameter number and improves the mean pixel accuracy by 2% compared to the existing ResNet101 and Xception backbone networks. The deployment of this framework on a low-computing power embedded platform (NVIDIA Jetson Xavier) demonstrates its great potential to apply to Mars rovers.Note to Practitioners—This paper was motivated by the problem of terrain classification of planetary rovers. Existing methods are typically based on semantic segmentation technology to recognize various terrains while suffering from the drawback of a large number of parameters. We propose a lightweight segmentation framework to address this issue. In particular, the lightweight backbone network is applied to significantly reduce the number of parameters. The receptive field module is substantially improved to enhance the feature extraction capability. Eventually, we deploy the framework on a low-computing platform. Experimental tests show that the framework can significantly reduce the number of network parameters and it can be used for planetary rovers with limited computational resources.

This study investigates the trajectory tracking strategy of a snake robot with sideslip disturbance and unknown model parameters. To guide the robot to track the ideal trajectory faster and more accurately, an adaptive anti-sideslip strategy for a snake robot with the Integral Line-of-Sight (ILOS) function is reported. This technique eliminates direction sideslip and error fluctuation by using auxiliary integral terms and shortens the convergence time of state variables. Following the position and angle control objectives, the proposed controller considers the negative effects caused by the uncertainty and time variability of environmental parameters and compensates for the joint input using the adaptive update laws. The environment adaptability and tracking efficiency are improved. The stability analysis indicates that the state errors converge to the origin. The simulation and experiment data verifies the effectiveness and strength of the work.Note to Practitioners—This article was motivated by the problem of robust trajectory tracking for a snake robot in an environment with sideslip disturbance and unknown model parameters. In this environment, information of the motion space (for example, the coefficient of ground friction) cannot be obtained. In addition, there may be other system limitations (for example, motion sideslip limitations) and other operational limitations. These limitations are caused by the requirements of various common trajectory tracking objectives. These cases should also be considered in the control strategy. However, based on the existing methods of tracking control for snake robots, there is still a lack of a complete and reliable autonomous control scheme that can consider the above problems. On this basis, we present a reliable control strategy, which considers the above problems and the dynamic uncertainty of the model. In the future, we will extend the proposed method to the field of formation tracking control for multiple robots.

Accurate prediction of grasp stability is crucial for reliable and precise operations with multi-fingered robotic hands. Traditional methods tend to oversimplify tactile information and pay equal attention to all regions of the data. This can obscure subtle yet critical variations and introduce noise, increasing the risk of stability assessment errors. To address these challenges, a novel self-supervised method, Adversarial Subgraph Contrastive Learning (ASCL), is proposed. It constructs an instance graph from the spatial distribution and features of perceptual nodes. It employs a bi-level adversarial strategy to enhance latent data representations by maximizing the mutual information between the instance graph and its semantic subgraphs, while minimizing it with its noisy subgraphs. To prevent trivial solutions and continuous relaxation of semantic subgraphs, node confidence and edge connection terms are incorporated to ensure stabilization. From an information-theoretic perspective, ASCL exhibits notable advantages on unlabeled or sparsely labeled data, well outperforming existing methods in empirical tests with robotic hands.

Aerial manipulator (AM) systems are significantly more effective than both conventional manipulators and flying robots, especially in disaster rescue settings, because they can perform human-like interaction tasks in flight. However, controlling the contact force of an AM is difficult due to the coupling between its flying platform and manipulator. This paper proposes a contact force control framework to address this problem. First, the UAV/AM position response under an external force is studied, and it is theoretically shown that a closed-loop UAV/AM behaves as a spring-mass-damper system. Second, a contact force controller is designed using an inverse-dynamics method. Third, an attitude feed-forward approach is employed to improve the force tracking performance. Then, the real-time contact position is introduced into the control system to achieve interaction with an actively moving environment. Finally, an innovative AM is developed and subjected to flight experiments, validating the proposed framework. An AM’s characteristics in interaction operations are summarized, and general conclusions are drawn. This study is novel in that 1) the contact force control is implemented without relying on force sensors, 2) the whole framework is applicable for controlling a constant/variable contact force with high closed-loop performance, and 3) it can also perform reliable interaction with a dynamic and unknown environment.Note to Practitioners—This study is motivated by the contact force control problem of an aerial manipulator (AM) during interaction operations. Previous approaches have explored the feasibility of controlling contact forces to some extent, but they lack universality and assume a static environment. Ensuring sufficient safety and providing reliable solutions in real-world applications remain challenging. This paper aims to investigate the position response of a closed-loop aircraft system under external forces, without disrupting its existing steady flight. The proposed method transforms contact force control into position control, eliminating the need for a force sensor. A series of aerial experiments were conducted to validate the effectiveness and applicability of the method in various scenarios. Our ongoing work will focus on migrating such an AM system from a laboratory scenario to the real world.

In this study, for the musculoskeletal robot system, we aim at optimizing the angle tracking control based on the system model. First, we analyze the driving principle of muscles and establish a bionic muscle dynamics model. Further, based on the geometric relationship between muscles and skeletons, we establish the kinematics model and dynamics model of the musculoskeletal robot system. The adaptive dynamic programming (ADP) algorithm is used for solving the Hamilton-Jacobi-Bellman (HJB) equation. Based on the proposed updating law, the neural network is trained to approximate the solution. Using the Lyapunov’s direct method, we can prove that the system is stable with the proposed controller. Furthermore, simulation indicates that this presented control law is feasible.

Existing works on controlling a concentric tube robot (CTR) mostly focus on the trajectory of its tip position or pose. In order to safely send CTRs in a confined lumen space, we propose to continuously steer the CTRs so that its entire shape will always attempt to approximate target curves over time. We focus on stiffness-dominant CTRs. Considering the differential geometry of such CTR shapes, we propose to work on the curvature domain to reduce the computational cost in searching the configuration of the CTRs. With our formulation, we model the curvature control of the CTR to find the optimal translation of each tube and then search for the rotation of the tubes to fit the target shapes. We demonstrate our method using sets of different target paths. The computational time per frame, ranging between 0.1 to 0.3 seconds across all experiments, highlights the efficiency of our approach in aligning the complete shape of the CTR with specified paths. Notably, for time-varying trajectories that could be reproduced by the CTR with its maximum deployment length reaching 150 mm, the root mean square error and median error were 0.98mm and 0.46mm, respectively.

The supernumerary robotic limb (SRL) mounted on the shoulder has been demonstrated to be able to serve as a third arm to assist human in overhead support task. However, the mechanical connection between the wearer and the SRL means that human operator’s movement will continually disturb the SRL and may lead to instability. Moreover, there may be physical conflicts between SRL and human operator due to their different intentions, which may potentially increase the load on human operator. Therefore, it’s necessary to control SRL to ensure stable supporting and transparent interaction. Here, we propose an adaptive controller for human movement compensation. Firstly, we model the human-SRL coordinative behavior based on non-zero-sum differential game theory aiming to enhance the support stability and reduce the operator’s load, which is a framework capable of dynamically regulating the control strategies between two interacting agents. We then implement an adaptive control strategy that adjusts the SRL’s input optimally responsive to the human operator’s input in the sense of Nash equilibrium to meet predefined control objectives. From experimental results during overhead support task, the proposed controller reduces the peak reaction force from 15.59 N to 4.50 N compared to the state-of-the-art controller that disregards human input while ensuring the stability of support, thereby providing advantage for human-SRL coordination in overhead support task.

In reinforcement learning, the agents need to sufficiently explore the environment and efficiently exploit the existing experiences before finding the solution to the tasks, particularly in cooperative multi-agent scenarios where the state and action spaces grow exponentially with the number of agents. Hence, enhancing the exploration ability of agents and improving the utilization efficiency of experiences are two critical issues in cooperative multi-agent reinforcement learning. We propose a novel method called Generative Adversarial Imitation Learning Policy Gradients (GAILPG). The contributions of GAILPG are as follows: (a) we integrate generative adversarial self-imitation learning into the multi-agent actor-critic framework to improve the utilization efficiency of experiences, thus further assisting the policy training; (b) we design a new curiosity module to enhance the exploration ability of the agents. Experimental results on the StarCraft II micromanagement benchmark demonstrate that GAILPG surpasses state-of-the-art policy-based methods and is even on par with the value-based methods. And the ablation experiments validate the reasonability of the discriminator module and the curiosity module encapsulated in our method.

Robot-assisted vascular interventional surgery is a hot topic in the interdisciplinary field of medical science and engineering, it has important research significance. Compared with traditional manual operation, it shows obvious advantages, such as avoiding the radiation exposure to the interventionist, reducing the workload of the interventionist, improving the precision of the operation, and guaranteeing the safety of the operation. However, the human–robot interaction ability has always been a major challenge that cannot be ignored. In this article, we designed and developed a leader device that enables the interventionists to experience the state of surgical instruments naturally and realistically when operating it, as if they were operating actual surgical instruments. The innovations can be summarized as follows: the structure of the leader device is highly consistent with the interventionist’s operating habit, which greatly reduces the interventionist’s operating difficulty. Haptic feedback in both translational direction and circumferential direction is achieved to provide haptic guidance to the interventionists and improve their surgical presence. The leader device is optimized in both structural design and functional realization compared to existing leader devices. This study has great potential in improving human-robot interaction ability, and could provide a reference for the design and evaluation of the leader device.

During recent years, dynamic early-exit has provided a promising paradigm to improve the computational efficiency of deep neural networks by constructing multiple classifiers to let easy samples exit at shallow layers while avoiding redundant computations at deep exits, which has been seldom explored in the context of latency-aware human activity recognition (HAR) deployed on wearable devices. Particularly, most existing early-exit strategies have always treated all activity samples equally at each exit during training, which ignore such dynamic early-exit behavior at test-time, causing a potential mismatch between training and test. Intuitively, easy activity samples that often exit earlier at test-time should place more emphasis on the training loss of shallow classifiers, while hard activity samples should contribute more to the training loss of deep classifiers. To bridge this gap, this paper introduces a sample-reweighting approach for efficient activity inference, which employs a weight-predicting network to reweight the training loss of different activity samples at every exit. From a perspective of meta learning, a new optimization objective function is designed to jointly optimize both weight-predicting network and backbone network. We perform extensive experiments on three popular HAR benchmarks including UCI-HAR, WISDM, and UniMiB-SHAR, which demonstrate that while incorporating such test-time early-exit behavior into conventional training pipeline, it can consistently improve the accuracy-efficiency trade-offs under budgeted batch classification and anytime prediction patterns. Moreover, our approach has a natural advantage in handing class-imbalance HAR problem. Detailed ablation studies, visualized illustrations, and real hardware deployment are provided to support our statement.

For the creation of wireless network applications, node locations are frequently necessary. However, communication effectiveness, measurement accuracy, and localization stability will be low in irregular multi-hop networks when locating nodes using conventional algorithms. To this end, a novel cooperative localization algorithm using expected minimum segments (LEMS, for short) is proposed in this paper. LEMS begins by measuring the distance between paired nodes, which is completed along with network initialization. Then, each unlocated node constructs its own sub-network, including it, based on the error characteristics among anchor nodes. Finally, each unlocated node searches for its estimated location in its sub-region based on the objective function generated by the chaotic mapping. Simulation results demonstrate that the proposed algorithm significantly outperforms the state-of-the-art regarding efficiency, accuracy, and stability for various irregular networks. Specifically, our proposed algorithm achieves a median improvement in localization accuracy of 0.62 to 29.57 times and a reduction in the range of localization errors of 0.06 to 16.8 times.

Accurate geographical information of nodes is crucial for network applications. However, many existing positioning algorithms face challenges in achieving efficient, accurate, and robust performance when applied to irregular networks with holes or obstacles. Therefore, we introduce a new algorithm, named Network Division Positioning (NDP), to tackle this issue. In NDP, we use a similarity function to derive the distance between neighboring nodes and explore routing paths concurrently, facilitating efficient distance measurement. Next, we analyze measurement errors between landmark nodes to define a threshold that filters out incorrect distances, ensuring measuring and positioning accuracy. To enhance robustness, we first identify collinearity issues by examining the positional relationship between unpositioned nodes and their nearest landmark. Subsequently, we addressed the poor positioning results and built the subnetwork utilizing the nearest landmark node and its associated measurement distance, seeking the most accurate and robust estimated position within this subnetwork. The simulation results demonstrate that NDP outperforms state-of-the-art algorithms in terms of efficiency, accuracy, and robustness when dealing with various irregular networks. Specifically, NDP enhances positioning accuracy by at least 40.82% in terms of the median.

Elasticity is one of therepresentative parameters that reflect the mechanical properties of soft materials. Detecting the underneath elasticity distribution called elastography is a key step for understanding and interacting with objects. Existing solutions for capturing the interior elasticity distribution typically rely on expensive apparatus. In this work, the dense tactile signal captured by the high-resolution vision-based tactile sensor is introduced as a new modality for reconstructing 3D elasticity distribution. We propose a model-based method, which exploits the tactile maps from active pressing trials for the elastography task. The interior elasticity distribution for non-rigid objects is reconstructed from an inverse physics model. We analyze the credibility of the estimated elasticity distribution obtained from our method. Varying design factors are also discussed. We experiment our method on a set of synthesized 3D models and physical models in robot-assisted scenes. Various experimental results have been gathered, demonstrating the efficacy of our approach in perceiving elasticity distribution.

This article proposes a distributed passivity-based bilateral teleoperation control for optimizing the velocity/force manipulability of the coordinated remote redundant manipulators during the task execution. Following the leader–follower paradigm, the control connects a local haptic device with a leader remote manipulator and coordinates all the leader and follower remote manipulators. The approach is novel in reconciling the potential conflicts between the pose synchronization task and the manipulability optimization task for the remote manipulators by two-layer auxiliary systems. The first layer decouples the pose synchronization constraints into separable position and orientation constraints, and the second layer optimizes the manipulability under the position and orientation constraints. The approach is robust by designing smooth controls for the manipulators without knowing their dynamic parameters. Finally, the control renders the bilateral teleoperator output strictly passive for stable physical interactions with the human user and the environment. Comparative experiments verify the effectiveness of the proposed control in the presence of time-varying communication delays.

This article presents a passivity-based shared control for a multirobot teleoperation system to perform extravehicular assembly tasks. At the remote site, an eye-in-hand camera is integrated with three manipulators that cooperatively drive a customized tool. A haptic device at the local site allows a human operator to intervene when necessary via bilateral teleoperation. To enable intuitive user control, a homography-based method seamlessly integrates visual servoing and operator input to guide the remote camera. To accommodate different peg geometries and enable adaptive tool actuation, a partial pose synchronization method decouples roll from the other pose dimensions. It allows each manipulator to independently actuate a gripper by rotating its end-effector frame around the roll axis, without interfering with cooperative motion in position, pitch, and yaw. In addition, to minimize undesired internal forces, we formulate and solve a passivity-constrained interaction wrench optimization problem that enhances stability. The proposed control ensures smooth transitions between autonomous and teleoperated modes, supporting flexible human intervention. Theoretical analysis and experimental validation confirm the system’s stability and effectiveness in achieving precise, robust, and responsive multirobot teleoperation for complex space assembly tasks.

Handheld haptic devices often face challenges in delivering stiffness feedback with both high force output and good backdrivability, especially under practical constraints on power consumption, size, and weight. These difficulties stem from the inherent performance limitations of conventional actuation mechanisms. To address this issue, we propose a lightweight, low-power handheld device that provides wide-range stiffness feedback through a novel dual actuation design composed of two key components. A programmable spring (PS), implemented via an adjustable lever arm, enables tunable physical stiffness. Two electrostatic clutches (ECs) are integrated to compensate for the inherent limitations of PS-based interactions in stiffness display range, rendered object size, and free motion capability. The feedback force arises passively from the reaction of the PS and ECs to user input, effectively lowering both power consumption and actuator torque demands. A fully integrated prototype was developed, incorporating wireless communication, control, and power modules. The results of the evaluation experiments and user studies demonstrate that the device effectively renders stiffness across the full range, from free motion to full rigidity, and delivers more realistic elastic feedback compared to conventional electric motor-based systems.

As virtual objects contain increasingly rich attribute information, small wearable fingertip devices need to have higher degrees of freedom (DoFs) to convey the haptic sensation of virtual objects. In order to effectively display the shape features of virtual objects to users through curvature, we designed a 6-DoF wearable fingertip device (WFD). This WFD combines a 6-DoF Stewart parallel mechanism, consisting of a static platform and a mobile platform connected by six revolute-spherical-spherical kinematic chains. The translation and rotation of the mobile platform are driven by six miniature servo motors, which can simulate haptic sensations such as making and breaking contact, sliding, and skin stretch when the fingertip interacts with a virtual surface. The WFD is fixed at the user's dominant index finger using hook-and-loop fasteners, with a size of 68 × 59 × 56 mm$^{3}$3 and a mass of 45.5 g. We analyzed and validated the kinematic model of the WFD and tested its force output capability. Finally, we invited 15 adults to conduct three subjective perception experiments to evaluate the performance of the WFD in curvature perception and shape display. The experimental results show that: (1) The just noticeable difference (JND) for curvature identification using the WFD is 3.02$\pm$±0.23 m$^{-1}$-1; (2) The 6-DoF haptic feedback provided by the WFD improves the accuracy of curved surface recognition from 53.4$\pm$±7.1% in 3-DoF to 72.0$\pm$±5.9%; (3) Even without visual feedback, the shape recognition accuracy of the WFD when combined with the Touch device reaches 82.3$\pm$±8.2% . Experimental results show that the WFD has good performance and potential in curvature perception and shape display.

Autonomous takeoff of flapping-wing robots (FWRs) is crucial for accelerating response speed and reducing costs in executing tasks. Jumping-aided takeoff is an effective method adopted by birds. However, the limited power density of motors poses challenges in achieving this type of takeoff for FWRs. In this study, we introduce a ground-based FWR ejecting system that utilizes a symmetric slider-crank mechanism (S-SCM) to store energy in spring. This stored energy is then converted into the takeoff speed of the FWR. The dynamic model of each working stage is established, and the design parameters are optimized according to simulation results. The prototype of the FWR ejecting system is fabricated for experimental validations. The results indicate that the system can provide a takeoff speed of 4 m/s for the 270 g FWR. Notably, the system is deployable on rough terrains and only adds a 3.2 g payload to the robot. Our work advances the autonomous takeoff of FWRs, promoting the application of such robots.

Elastic rods are commonly seen in our daily life. Although humans are sensitive to the shape change of rods, it is not intuitive to estimate the external forces applied to generate the deformation. We propose a method to interactively track the elastic linear objects by using the Cosserat rod model to regulate the captured noisy points. We develop a framework based on particle filters to work with the physics-based model, turning the inverse physics problem into a forward simulation and search problem. We show that with the proposed method, we can simultaneously digitalize the shape as well as the external forces on real-world elastic rods. With these capabilities, we demonstrate virtual and augmented reality applications to facilitate the interaction with elastic linear objects. The tracking performance is also validated with experiments.

Knowledge distillation stands as a capable technique for transferring knowledge from a larger to a smaller model, thereby notably enhancing the smaller model’s performance. In the recent past, data augmentation has been employed in contrastive learning based knowledge distillation techniques yielding superior results. Despite the significant role of data augmentation, its value remains underappreciated within the domain of knowledge distillation, with no in-depth analysis in the literature thus far. To make up for this oversight, we conduct a multi-perspective theoretical and experimental analysis on the role that data augmentation can play in knowledge distillation. We summarize the properties of data augmentation and list the core findings as follows. (a) Our investigations validate that data augmentation significantly boosts the performance of knowledge distillation on the tasks of image classification and object detection. And this holds true even if the teacher model lacks comprehensive information about the augmented samples. Moreover, our novel J oint D ata A ugmentation (JDA) approach outperforms single data augmentation in knowledge distillation. (b) The pivotal role of data augmentation in knowledge distillation can be theoretically explained via Sharpness-Aware Minimization. (c) The compatibility of data augmentation with various knowledge distillation methods can enhance their performance. In light of these observations, we propose a new method called C osine C onfidence D istillation (CCD) for more reasonable knowledge transfer from augmented samples. Experimental results not only demonstrate that CCD becomes the state-of-the-art method with less storage requirement on CIFAR-100 and ImageNet-1k, but also validate the superiority of CCD over DIST on the object detection benchmark dataset, MS-COCO.

During recent years, deep neural networks have achieved outstanding success in sensor-based human activity recognition (HAR). Particularly, dynamic convolution has emerged as a promising solution to accelerate activity inference of deep networks on mobile devices. Exploiting spatial redundancy, such a dynamic strategy can adaptively sample the salient areas of interest over sensor feature maps while skipping unimportant locations to avoid computational expenditure on activity-irrelevant disturbing areas. Despite theoretic efficiency, it has to rely on a binary-valued mask combined with element-wise multiplication, which potentially incurs noncontiguous memory access while performed at the finest granularity. To the best of our knowledge, most existing HAR literatures have always adopted hardware-agnostic FLOPs as an indicator to guide the algorithm design, lacking delay-aware considerations about scheduling strategy and specific hardware characteristic. In this article, we propose a delay-aware coarse-grained dynamic convolutional network called DACDNet to bridge the gap between theoretical FLOPs and realistic delay, which is highly challenging but less explored in ubiquitous HAR environments. Instead of theoretic FLOPs, we introduce a novel delay prediction model to guide the HAR algorithm design while simultaneously considering the scheduling strategy on various hardware platforms, especially multicore processors like the edge GPU devices. Experiments on multiple HAR benchmarks, including WISDM, UniMiB-SHAR, and PAMAP2 demonstrate that our approach can significantly accelerate activity inference without sacrificing accuracy.

Credit assignment is a critical problem in cooperative Multi-Agent Reinforcement Learning (MARL). To address this problem, current studies mainly rely on the intrinsic reward, which is directly summed with the global reward to generate a total reward. However, such kinds of intrinsic reward functions ignore the dependence among agents and inevitably limit the adaptivity and effectiveness of MARL methods. In this paper, we propose a novel method, Attention-based Intrinsic Reward Mixing Network (AIRMN), for credit assignment in MARL. Specifically, we design a new intrinsic reward network on the basis of the attention mechanism, in order to enhance the effectiveness of teamwork. Besides, we devise a new mixing network that combines the intrinsic and extrinsic rewards in a nonlinear and dynamic manner, so as to adapt the total reward to the variation of the environment. Experimental results on the battle games of StarCraft II demonstrate that AIRMN outperforms the state-of-the-art methods in terms of the average test win rate, and also validate that AIRMN can dynamically return the precise intrinsic reward to each agent based on their contributions to the team cooperation, thereby better dealing with the credit assignment problem.

Multiagent reinforcement learning (MARL) has made remarkable progress in recent years. However, in most MARL methods, agents share a policy or value network, which is easy to result in similar behaviors of agents, and thus, limits the flexibility of the method to handle complex tasks. To enhance the diversity of agent behaviors, we propose a novel method, making decisions from different perspectives (MDDP). This method enables agents to switch flexibly between different policy roles and make decisions from different perspectives, which can improve the adaptability of policy learning in complex scenarios. Specifically, in MDDP, we design a new self-attention and gated recurrent unit (GRU)-based dueling architecture network (SG-DAN) to estimate the individual$Q$-values. SG-DAN contains two components: 1) the new self-attention-based role-switching network (SAR) and the capable GRU-based state value estimation network (GSE). SAR takes charge of action advantage estimation and GSE is responsible for state value estimation. Experimental results on the challengingStarCraftII micromanagement benchmark not only verify the modeling reasonability of MDDP but also demonstrate its performance superiority over the related advanced approaches.

Self-supervised Human Activity Recognition (HAR) has been gradually gaining a lot of attention in ubiquitous computing community. Its current focus primarily lies in how to overcome the challenge of manually labeling complicated and intricate sensor data from wearable devices, which is often hard to interpret. However, current self-supervised algorithms encounter three main challenges: performance variability caused by data augmentations in contrastive learning paradigm, limitations imposed by traditional self-supervised models, and the computational load deployed on wearable devices by current mainstream transformer encoders. To comprehensively tackle these challenges, this paper proposes a powerful self-supervised approach for HAR from a novel perspective of denoising autoencoder, the first of its kind to explore how to reconstruct masked sensor data built on a commonly employed, well-designed, and computationally efficient fully convolutional network. Extensive experiments demonstrate that our proposed Masked Convolutional AutoEncoder (MaskCAE) outperforms current state-of-the-art algorithms in self-supervised, fully supervised, and semi-supervised situations without relying on any data augmentations, which fills the gap of masked sensor data modeling in HAR area. Visualization analyses show that our MaskCAE could effectively capture temporal semantics in time series sensor data, indicating its great potential in modeling abstracted sensor data. An actual implementation is evaluated on an embedded platform.

The multi-dimensional force/torque decoupling and calibration is extremely crucial to increase the accuracy of the Wheel Force Transducer/Sensor (WFT). A novel interpretable nonlinear decoupling and calibration approach to WFT is presented. A physical interpretable prime-error framework is developed such that the linear prime part accounts for most force-voltage responses while the nonlinear error part accounts for the gross error deviation. The conventional least-square decoupling is improved with the delicate nonlinear error modeling using a polynomial base module and a hyperbolic activation function. The developed framework is proved to be mathematically solvable and physically feasible by a two-step calibration scheme. A two-axis WFT is tested and compared with the proposed interpretable nonlinear decoupling model (IND), the least-square-based method (LSM), and the error-based neural network model (eNN). Results demonstrate that the proposed IND provides an accurate, practical, and effective scheme for modeling and calibrating WFTs and maintains a good balance among accuracy, generalization ability, and computational efficiency for real applications.

Despite strong representation ability, deep convolutional neural networks (CNNs) are largely hindered in practical human activity recognition (HAR) deployment due to high computational cost, which is often unaffordable on resource-limited wearable devices. In this article, to bridge the gap between on-device HAR and deep learning, we present a collaborative compression scheme to reduce the runtime of HAR with an acceptable performance degradation, which combines channel pruning and tensor decomposition to simultaneously handle sparsity and low-rankness when fully considering mutual interference in one network consisting of efficient 1-dimensional convolutional kernels. Our method includes two main stages. Concretely, given a target compression ratio, a global compression ratio decision optimization is first performed to automatically decide per-layer compression ratio by measuring compression sensitivity, without requiring labor-exhaustive human intervention. Then a multi-step collaborative compression is iteratively implemented to remove the least important compression unit based on an improved importance metric until the per-layer target compression ratio is attained. Extensive experiments on multiple HAR benchmarks show that our approach considerably outperforms previous compression strategies. For example, it can achieve around 50% FLOPs reduction with only an accuracy drop of 0.25% and 0.15% on UCI-HAR and PAMAP2, respectively. Actual implementation is evaluated on an embedded platform.

Node positioning is an essential function of wireless networks and serves as the foundation for many applications. In the existing works, the cooperative positioning approaches have been extensively studied and are shown to be effective for scenarios with energy and cost constraints. However, these approaches may not perform well in concave networks with holes or obstacles. To address this issue, this paper proposes adistributed cooperative positioning model with adaptive hop-range selection(DCP-AHS for short) for concave networks. DCP-AHS first uses a low-complexity and fast convergent distance estimation method based on the local neighbor nodes. It then uses an adaptive hop-range selection method based on the residual analysis between pairs of anchors. Within the hop range, an unknown node uses multi-lateration with the optimal weight function to determine its estimated position. Finally, a weighted Bounding-Box method with the virtual anchor is employed to avoid significant position estimation errors caused by the collinearity issues. Simulation results demonstrated that the proposed DCP-AHS significantly outperformed the existing algorithms regarding efficiency, accuracy, and stability in various concave networks. Specifically, our proposed model achieved a median improvement of 16.62% to 81.65% in positioning accuracy compared to the comparison algorithms.

This study focuses on reachability problems in differential games. An improved level set (LS) method for computing reachable tubes (RTs) is proposed in this article. The RT is described as a sub-LS of a value function, which is the viscosity solution of a Hamilton–Jacobi (HJ) equation with running cost. We generalize the concept of RTs and propose a new class of RTs, which are referred to as cost-limited one. In particular, a performance index can be specified for the system, and A set of initial states of the system’s evolutions that can reach the target set before the performance index grows to a given allowable cost is referred to as a cost-limited RT (CRT). Such an RT can be obtained by specifying the corresponding running cost function for the HJ equation. Different nonzero sub-LSs of the viscosity solution of the HJ equation at a certain time point can be used to characterize the CRTs with different allowable costs (or the RTs with different time horizons), thus reducing the storage space consumption. The validity and accuracy of the suggested technique are demonstrated via some examples.

For upper limb rehabilitation, the robot-assisted technique in combination with serious games requires well-specified training plans. For the best quality of the rehabilitation process, customized game levels for each user are desired, while it is labor-intensive to design and adjust game levels for different individuals. We work on generating training content for a desktop end-effector rehabilitation robot and propose a method to automatically generate individualized training plans. By modeling the search of the training motions as finding optimal hand paths and trajectories, we introduce solving the design problem with a multi-objective optimization (MO) solver. We further improve the MO solver to enhance the diversity of the solutions. With the proposed approach, our system is capable of automatically generating various training plans considering the training intensity and dexterity of each joint in the upper limb. In addition, the enhanced diversity avoids repeated training plans, which helps motivate the user in the rehabilitation. We test our method with different requirements on the training plans and validate the solutions.

Robot-assisted eye-hand coordination rehabilitation training system is extremely urgent to study since recent evidence suggests that eye-hand coordination can be brutally disturbed by stroke with critical consequences on motor behavior. In this paper, we develop a robot-assisted eye-hand coordination training system by estimating motion direction using smooth-pursuit eye movements. Firstly, we design a Pong Game, which requires users to extrapolate the direction of a linearly moving ball and to predict whether this ball would be hit. Secondly, the motion direction of the ball is estimated via smooth-pursuit eye movements, allowing the robot quickly establish an assistive force field to hit the ball. Thirdly, adding haptic feedback technology into this training system to make users more immersive. Finally, we conduct a feasibility study with eight healthy subjects to verify the effectiveness of the proposed system. The experimental results show that the mean success rate for hitting the pong ball of the experiment group (assistance turn-on) is 28.33% higher than that of the control group (assistance turn-off), and the mean interception time of the experiment group is 0.35s shorter than that of the control group. Therefore, the developed system may be promising for transferring to the robot-assisted eye-hand coordination rehabilitation training for post-stroke patients.

Teleoperated techniques enable remote human-robot interaction and have been widely accepted in robot-assisted surgeries. However, it is still hard to guarantee the safety of teleoperated surgery due to the imperfect input commands limited by remote perception, preventing teleoperated surgery from being widely used. We propose a new framework to avoid the collision of surgery robots and human tissue caused by inaccurate inputs. We directly take the medical volume data and propose to use the implicit neural field to guide teleoperated robot-assisted surgery. With guidance, the trajectory of the robot manipulator is optimized to safely work inside a narrow workspace. We evaluated our method in several aspects and conducted a real-world experiment on a head phantom. Experimental results show that our proposed method can effectively avoid the collision between the surgical tool and the human tissue during teleoperation.

This paper studies a novel application of an aerial manipulator (AM)-the contact-based landing on a mobile platform. An AM is inherently unstable, under-actuated, and usually loses some DOFs while contacting environments. Meanwhile, the AM's flight state is susceptible to uncertain movements of the mobile platform, such as acceleration, sudden stopping, and reversing. To accomplish the contact-based landing mission, a robust controller is first designed to maintain a steady contact-based flight. Then a hierarchical control framework is applied, integrating the controllers in free-flight and restricted-flight stages. An AM and a mobile platform are developed for contact-based flight experiments. The proposed scheme is reliable and has good repeatability in experiments. To the best of our knowledge, this is the first time an AM has been implemented to conduct a contact-based landing, which is also an innovative landing approach for rotorcraft UAVs.

The neural radiance field (NeRF) is attracting increasing attentions from researchers in various fields. While NeRF has produced visually plausible results and found its potential applications in virtual reality, users are only allowed to rotate the camera to observe the scene represented as NeRF. We study the haptic interaction with NeRF models in this paper to enable the experience of touching objects reconstructed by NeRF. Existing haptic rendering algorithms do not work well for NeRF-represented models because NeRF is often noisy. We propose a stochastic haptic rendering method to deal with the collision response between the haptic proxy and NeRF. We validate our method with complex NeRF models and experimental results show the efficacy of our proposed algorithm.

During the past decade, human activity recognition ( HAR ) using wearable sensors has become a new research hot spot due to its extensive use in various application domains such as healthcare, fitness, smart homes, and eldercare. Deep neural networks, especially convolutional neural networks ( CNNs ), have gained a lot of attention in HAR scenario. Despite exceptional performance, CNNs with heavy overhead is not the best option for HAR task due to the limitation of computing resource on embedded devices. As far as we know, there are many invalid filters in CNN that contribute very little to output. Simply pruning these invalid filters could effectively accelerate CNNs , but it inevitably hurts performance. In this article, we first propose a novel CNN for HAR that uses filter activation. In comparison with filter pruning that is motivated for efficient consideration, filter activation aims to activate these invalid filters from an accuracy boosting perspective. We perform extensive experiments on several public HAR datasets, namely, UCI-HAR ( UCI ), OPPORTUNITY ( OPPO ), UniMiB-SHAR ( Uni ), PAMAP2 ( PAM2 ), WISDM ( WIS ), and USC-HAD ( USC ), which show the superiority of the proposed method against existing state-of-the-art ( SOTA ) approaches. Ablation studies are conducted to analyze its internal mechanism. Finally, the inference speed and power consumption are evaluated on an embedded Raspberry Pi Model 3 B plus platform.

We propose a novel approach for enabling comic-style conversation in mixed reality to assist face-to-face conversation on-site or remotely. Our approach brings word balloons of comic-style conversation to the real world. The word balloons can adapt to mixed reality scenes, such as the 3-D head motion of the speaker, the comic styles, and the speech. During the conversation, our approach updates the word balloons continuously in the object space and discretely in the image space, guided by a field learned from comics. Quantitative experiments and perceptual studies were conducted to evaluate and compare our approach with alternatives. The results from the user study and ablation study demonstrated that our approach turns out to be practical for assisting face-to-face conversation.

In order to help robots understand and perceive an object's properties during noncontact robot-object interaction, this article proposes a deeply supervised subspace learning method. In contrast to previous work, it takes the advantages of low noise and fast response of noncontact sensors and extracts novel contactless feature information to retrieve cross-modal information, so as to estimate and infer material properties of known as well as unknown objects. Specifically, a depth-supervised subspace cross-modal material retrieval model is trained to learn a common low-dimensional feature representation to capture the clustering structure among different modal features of the same class of objects. Meanwhile, all of unknown objects are accurately perceived by an energy-based model, which forces an unlabeled novel object's features to be mapped beyond the common low-dimensional features. The experimental results show that our approach is effective in comparison with other advanced methods.

Federated Learning (FL) has recently attracted great interest in sensor-based human activity recognition (HAR) tasks. However, in real-world environment, sensor data on devices is non-independently and identically distributed (Non-IID), e.g., activity data recorded by most devices is sparse, and sensor data distribution for each client may be inconsistent. As a result, the traditional FL methods in the heterogeneous environment may incur a drifted global model that causes slow convergence and a heavy communication burden. Although some FL methods are gradually being applied to HAR, they are designed for overly ideal scenarios and do not address such Non-IID problem in the real-world setting. It is still a question whether they can be applied to cross-device FL. To tackle this challenge, we propose ProtoHAR, a prototype-guided FL framework for HAR, which aims to decouple the representation and classifier in the heterogeneous FL setting efficiently. It leverages the global prototype to correct the activity feature representation to make the prototype knowledge flow among clients without leaking privacy while solving a better classifier to avoid excessive drift of the local model in personalized training. Extensive experiments are conducted on four publicly available datasets: USC-HAD, UNIMIB-SHAR, PAMAP2, and HARBOX, which are collected in both controlled environments and real-world scenarios. The results show that compared with the state-of-the-art FL algorithms, ProtoHAR achieves the best performance and faster convergence speed in HAR datasets.

Electroencephalogram (EEG)-based emotion recognition has gradually become a research hotspot. However, the large distribution differences of EEG signals across subjects make the current research stuck in a dilemma. To resolve this problem, in this article, we propose a novel and effective method, Multi-Source Feature Representation and Alignment Network (MS-FRAN). The effectiveness of proposed method mainly comes from three new modules: Wide Feature Extractor (WFE) for feature learning, Random Matching Operation (RMO) for model training, and Top- h ranked domain classifier selection (TOP) for emotion classification. MS-FRAN is not only effective in aligning the distributions of each pair of source and target domains, but also capable of reducing the distributional differences among the multiple source domains. Experimental results on the public benchmark datasets SEED and DEAP have demonstrated the advantage of our method over the related competitive approaches for cross-subject EEG-based emotion recognition.

Recent years have witnessed great success of deep convolutional networks in sensor-based human activity recognition (HAR), yet their practical deployment remains a challenge due to the varying computational budgets required to obtain a reliable prediction. This article focuses on adaptive inference from a novel perspective of signal frequency, which is motivated by an intuition that low-frequency features are enough for recognizing "easy" activity samples, while only "hard" activity samples need temporally detailed information. We propose an adaptive resolution network by combining a simple subsampling strategy with conditional early-exit. Specifically, it is comprised of multiple subnetworks with different resolutions, where "easy" activity samples are first classified by lightweight subnetwork using the lowest sampling rate, while the subsequent subnetworks in higher resolution would be sequentially applied once the former one fails to reach a confidence threshold. Such dynamical decision process could adaptively select a proper sampling rate for each activity sample conditioned on an input if the budget varies, which will be terminated until enough confidence is obtained, hence avoiding excessive computations. Comprehensive experiments on four diverse HAR benchmark datasets demonstrate the effectiveness of our method in terms of accuracy-cost tradeoff. We benchmark the average latency on a real hardware.

To respond to the rapid growth of shipping container throughput, terminals urgently need to improve the efficiency of thier operations and reduce operational costs through automation and intellectualization upgrades, thereby improving service levels and enhancing market competitiveness. Due to the advantages of reliable transportation, efficient operation, and environmental friendliness, AGV-based automated container terminal (ACT) has become the development trend of container terminals. To help ACT improve its operational management capabilities, plenty of scholars have explored the transportation system of ACT. Through the analysis of operational management issues, the paper defines the four main research topics in vehicle transportation of the ACT including equipment scheduling, path planning, exception handling, and vehicle management. Then, in each topic, the works in the recent 25 years are summarized and several research opportunities for possible follow-up research directions in different fields are proposed. We expect our survey could not only provide references for more scholars on the research of operation and management of terminals, but also provide guidance for system evaluation and improvement for terminal system engineers and operation managers.

During recent years, channel attention has attracted great interest in deep learning community. Despite significant success, it has been rarely exploited in ubiquitous human activity recognition (HAR) scenario. To decrease computational overhead, the channel attention often uses global averaging pooling (GAP) to compress each channel into a simple scalar. It is well known that GAP is equal to the lowest frequency component. Despite obvious lightweight advantage, such compression process inevitably causes severe information loss. In this paper, we propose a novel multi-frequency channel attention framework for activity recognition tasks. Considering various sensing frequencies of human activities, an intuition solution is to convert the time series from time domain to frequency domain. Instead of GAP, the discrete cosine transform (DCT) is used to compress channels. We prove that GAP can be seen as a special case of DCT, which uses the lowest frequency component only and leaves out all other frequency components unused. DCT is able to better compress channels by fully exploiting other frequency components discarded by GAP. Despite multiple frequency components used, each channel will still be represented by a scalar in order to maintain the same computational overhead. Using two frequency screening criteria, our method is able to achieve state-of-the-art results on four benchmark HAR datasets. Extensive ablation studies are conducted, which provides a better interpretability of deep model behaviors. Finally, actual inference is evaluated on an embedded platform.

Recently, human activity recognition (HAR) that uses wearable sensors has become a research hotspot because its wide applications in real-world scenarios. Essentially, HAR can be treated as multi-channel time series classification problem, where different channels may come from heterogeneous sensor modalities. Deep learning, especially convolutional neural networks (CNNs) have made breakthroughs in ubiquitous HAR scenario. Various normalization methods enable layers of networks to learn more independently by normalizing hybrid sensor features. However, normalization tends to produce a channel collapse phenomenon, where many channels generates tiny values. Most channels are inhibited and contribute very little to output. As a result, the network has to rely on only a few valid channels, which inevitably impair the generality ability. In this paper, we provide an alternative called Channel Equalization to reactivate these inhibited channels by performing whitening or decorrelation operation, which compels all channels to contribute more or less to feature representation. Extensive experiments are conducted on several public HAR benchmarks, which indicate that the proposed method significantly surpasses recent SOTA at negligible computational overhead. To our knowledge, the Channel Equalization is for the first time to be applied in multimodal HAR scenario. Finally, the actual operation is evaluated on an embedded platform.

This work reports an adaptive path-following controller for a multijoint snake robot (MSR) to improve the adaptability of the robot to the environment. The new strategy estimates the time-varying parameters of the system and the external interference to adjust the motion state of the robot in real time. Estimations are used to compensate for the joint torque of an MSR, thus reducing the fluctuation peak of path-following errors. In addition, this work designs an anti-sideslip line-of-sight (LOS) guidance strategy to avoid the deviation of the direction angle. The method can improve the tracking accuracy of an MSR, and the position errors enable the system to achieve uniformly ultimate boundedness (UUB). The angle errors converge to the origin to achieve stability. Experimental results demonstrate that the novel method can accurately estimate the time-dependent parameters, sideslip, and interference, raise the convergent speed of errors, and reduce the fluctuation peak.

In order to enable intelligent robots to recognize unknown objects as accurately as human beings, object perception research is of great significance in service and industrial robot application scenarios. However, object perception using spectral measurements under few-shot learning usually leads to a poor result because of inadequate training samples. To overcome this problem, this work proposes a novel few-shot learning with coupled dictionary learning (FSL-CDL) framework. First, a hybrid feature fusion method is developed to extract the multiple dimension-reduced features of original spectral measurements to build the hybrid features. Then, based on the hybrid features, a multitask coupled learning method is developed to effectively recognize unknown objects under few-shot learning. In this method, two coupling patterns, i.e., interspectroscopy coupling and intraspectroscopy coupling, effectively bridge the gap between two spectral measurements. Finally, the proposed FSL-CDL is compared with other advanced algorithms on the SMM50 dataset, and reaches 97.5% and 98.4% recognition accuracy under one-shot and five-shot learning, respectively, which are better than other algorithms. Besides, FSL-CDL can be extended to other perception tasks which contains multiple heterogeneous measurements.

With the development of virtual reality, the practical requirements of the wearable haptic interface have been greatly emphasized. While passive haptic devices are commonly used in virtual reality, they lack generality and are difficult to precisely generate continuous force feedback to users. In this work, we present SmartSpring, a new solution for passive haptics, which is inexpensive, lightweight and capable of providing controllable force feedback in virtual reality. We propose a hybrid spring-linkage structure as the proxy and flexibly control the mechanism for adjustable system stiffness. By analyzing the structure and force model, we enable a smart transform of the structure for producing continuous force signals. We quantitatively examine the real-world performance of SmartSpring to verify our model. By asymmetrically moving or actively pressing the end-effector, we show that our design can further support rendering torque and stiffness. Finally, we demonstrate the SmartSpring in a series of scenarios with user studies and a just noticeable difference analysis. Experimental results show the potential of the developed haptic display in virtual reality.

This work proposes a scalable gamma non-negative matrix network (SGNMN), which uses a Poisson randomized Gamma factor analysis to obtain the neurons of the first layer of a network. These neurons obey Gamma distribution whose shape parameter infers the neurons of the next layer of the network and their related weights. Upsampling the connection weights follows a Dirichlet distribution. Downsampling hidden units obey Gamma distribution. This work performs up-down sampling on each layer to learn the parameters of SGNMN. Experimental results indicate that the width and depth of SGNMN are closely related, and a reasonable network structure for accurately detecting brain fatigue through functional near-infrared spectroscopy can be obtained by considering network width, depth, and parameters.

The popularity of convolutional architecture has made sensor-based human activity recognition (HAR) become one primary beneficiary. By simply superimposing multiple convolution layers, the local features can be effectively captured from multi-channel time series sensor data, which could output high-performance activity prediction results. On the other hand, recent years have witnessed great success of Transformer model, which uses powerful self-attention mechanism to handle long-range sequence modeling tasks, hence avoiding the shortcoming of local feature representations caused by convolutional neural networks (CNNs). In this paper, we seek to combine the merits of CNN and Transformer to model multi-channel time series sensor data, which might provide compelling recognition performance with fewer parameters and FLOPs based on lightweight wearable devices. To this end, we propose a new Dual-branch Interactive Network (DIN) that inherits the advantages from both CNN and Transformer to handle multi-channel time series for HAR. Specifically, the proposed framework utilizes two-stream architecture to disentangle local and global features by performing conv-embedding and patch-embedding, where a co-attention mechanism is used to adaptively fuse global-to-local and local-to-global feature representations. We perform extensive experiments on three mainstream HAR benchmark datasets including PAMAP2, WISDM, and OPPORTUNITY, which verify that our method consistently outperforms several state-of-the-art baselines, reaching an F1-score of 92.05%, 98.17%, and 91.55% respectively with fewer parameters and FLOPs. In addition, the practical execution time is validated on an embedded Raspberry Pi P3 system, which demonstrates that our approach is adequately efficient for real-time HAR implementations and deserves as a better alternative in ubiquitous HAR computing scenario. Our model code will be released soon.

For the purpose of avoiding injury and realizing precise operations, the multilateral teleoperation system is the most efficient way to transport trace toxic or radioactive substances, to perform minimally invasive surgery, etc. It is essential to enhance the transparency of a multilateral teleoperation system including multiple masters and the single slave manipulator. However, there are few researchers focus on the allocation of the contact force of the single slave manipulator to different master manipulators. In this paper, we firstly introduce the concept of force translation for teleoperation systems consisting of dual hand master (left and right hands) manipulators and a single slave manipulator. Force translation reflects how the impedance on the single slave side is translated or allocated to contact forces on different master sides. To maintain the stability of the system and to improve transparency, we elucidate the mechanism of the force translation and analyze the relation among masters and the slave. Furthermore, the force translation mechanism is analyzed through numerical simulations and CHAI 3D virtual physical simulations. It is used to propose a multilateral impedance control, and the Lyapunov function is used to analyze the system stability. The results of numerical simulations and real robot experiments verify the effectiveness of the proposed control methods based on the proposed force translation mechanism.

Many discriminative correlation filter (DCF)-based methods have successfully leveraged the guidance for solving two problems (i.e., the boundary effect and temporal filtering degradation) as a model prior to visual tracking. The intuitive motivation of these methods is to control the degeneration of the updating loss of the objective function with a structural framework. While these methods rely mostly on various regularization items, they always ignore the loss from data fidelity term. Therefore, we propose a bilateral weighted regression ranking model termed as BWRR. Here, we resort to two procedures for solving the above problems. First, BWRR introduces a bilateral constraint into the data fidelity term to control the loss of rows and columns of the filter learning data term. The weighted matrices could impose an adaptive penalty for large data loss during the learning process to avoid the model degradation problem. Second, the data of the updated weighted matrices is not directly applied to the calculation of the filter during each iteration. Instead, a new weighted product matrix is obtained by ranking and numerical transformation for updating the filter. We show that the proposed model converts the original correlation filter regression problem into a regression-with-ranking problem, thus avoiding the problem of positive and negative sample imbalance. Overall, the BWRR model is iteratively solved by the alternating direction method of multipliers(ADMM). Qualitative and quantitative evaluations demonstrate the effectiveness and superiority of our proposed method by extensive and quantitative experiments on the OTB, VOT, and UAV datasets.

An unmanned swarm system (UWS) is a multiagent system that can fulfill task requirements through autonomous and cooperative behavior strategy learning. However, learning instability is inevitable in a dynamic mission setting, as the agents continuously adapt to an evolving mission objective. This article proposes several knowledge enhancement mechanisms to improve the training efficiency and learning stability of a UWS in a confined-space confrontation mission. Specifically, a punishment for transcending action-space boundary and a reward for satisfying agent space-time distance constraints are introduced as training reward enhancements. Meanwhile, experience sharing among agents is optimized for unanimous behavior. We apply these novel mechanisms to several representative single-agent and multiagent reinforcement learning algorithms and verify their effectiveness on our proprietary,SwarmFlow, simulation system. Simulations show that the proposed mechanisms improve existing algorithms’ convergence speed and performance stability. The increase is more prominent for multiagent reinforcement learning algorithms than single-agent algorithms where the convergence time is halved, and the mission success rates increase by 3–4%.

This article proposes a four-dimensional brain mapping method, which can represent the continuous process of a person’s fatigue state in the form of image frames in a space-time range. This work couples 3-D-convolutional neural networks and long-short-term memory networks to form a cognitive detection model of brain fatigue dynamics, which can simulate the continuous process of a person’s brain fatigue dynamics and accurately identify different cognitive fatigue states. Our approach can be applied to any type of brain fatigue detection.

The communication time delay and uncertain models of robotic manipulators are the major problem in the teleoperation system, which can reduce the performance and stability of the system. This article proposed a novel finite-time adaptive control scheme for position and force tracking performances of the teleoperation system. First, a combined auxiliary error system with position and force tracking errors is designed. Second, a velocity feedback filter is introduced, and a new auxiliary variable function with finite-time structure is designed for controller design. The radial basis function neural network (RBFNN) is applied to estimate the uncertain parts. Then the finite-time adaptive control scheme and adaptive laws are given. Third, based on the Lyapunov method, stability and finite-time performance are demonstrated. And finally, the simulation and experimental studies (with Phantom Ommi devices) are performed and demonstrate the effectiveness of the proposed control scheme on teleoperation position/force tracking.

Haptic feedback is widely used to enhance realism in virtual reality (VR). Shape and softness are two common factors perceived by the users in the haptic rendering of soft objects. To integrate these factors, we propose a new handheld shape-changing device, TapeTouch, to provide various shapes and softness in real time. TapeTouch is based on a controllable shape-changing tape, which is mainly composed of four motors and a section of brass tape. We design a structure of the components to fit a portable controller and allow to flexibly adjust the shape of the brass tape. After decoding desired shapes into the signals to control the motor, we automatically reproduce varying shapes and levels of softness to the finger or palm touching the shape-changing tape. We conducted two user studies to understand the capability of TapeTouch to render shape and softness, and the results showed that TapeTouch could provide a variety of distinguishable shapes as well as multiple levels of softness. Based on the results, we performed two VR experience studies to verify that the haptic feedback from TapeTouch enhances VR realism.

This paper proposes a new path planning strategy - the Rapid Visible Tree (RVT) algorithm to guide a robot to its goal in a complex environment without dangerous collisions. By fusing the visibility information with the classic tree-based searching method, RVT only takes the noisy points locally acquired from the environment as input and computes the visible region at each location to decide the growing direction of the path tree. Compared with traditional methods, RVT is more efficient, lightweight, and robust. We demonstrate that the RVT algorithm can not only complete the path planning task in real-time but also explore the unknown environment in simulated or real scenes.

Conventional coverage path planning algorithms are mainly based on the zigzag and spiral patterns or their combinations. The traversal order is limited by the linear or inside-outside manner. We propose a new set of coverage patterns induced from geometric folding operations, called the geometric folding pattern, to make coverage paths with more flexible traversal order. We study the modeling and parameterization of the geometric folding patterns. Then, a sampling operator is introduced. Based on the computational tools, we demonstrate the application of the proposed patterns in designing coverage paths. We show that the simple geometric folding patterns are flexible and controllable, which enables more choices for the coverage path planning problem.

We propose a new design of a hardware accelerator for processing regular expression to speedup text search inside SSD storage (Processing in Storage: PIS). The unique features include parallel processing of 32 streams to quickly identify the first matched character under scan mode and match four characters concurrently under matching mode. In addition, we present a new approach of combining forward and backward scan to accomplish the first character search efficiently. Our experimental results show that the new parallel algorithm reduces the depth of logic circuit and the hybrid architecture performs as well as the Linux Grep algorithm does.

In haptic interaction technology, texture haptic display is an important part. In order to perceive color image texture better, the haptic display methods of color texture based on force feedback and tactile feedback are proposed in this work. On the one hand, through the study of the physiological and psychological perception characteristics of color information, a new force rendering method of color image texture based on force feedback device is presented. The experimental results of color texture force perception show that the color texture force rendering algorithm in this paper works well. On the other hand, a color texture vibration tactile model is established based on the designed vibrotactile device. Its effectiveness is verified by vibration tactile perception experiment. Finally, for the color texture image of the real object surface, the force and vibrotactile display methods of color texture in this paper are used to conduct perception experiments and compared.

In order to enhance the sense of reality haptic display based on image, it is widely expected to express various characteristics of the objects in the image using different kinds of haptic feedback. To this end, a multi-mode haptic display method of image was proposed in this paper, including the multi-feature extraction of image and the image expression with various types of haptic rendering. First, the device structure integrating force and vibrotactile feedbacks was designed for multi-mode haptic display. Meanwhile, the three-dimensional geometric shape, detail texture and outline of the object in the image were extracted by various image processing algorithms. Then, a rendering method for the object in the image was proposed based on the psychophysical experiments on the piezoelectric ceramic actuator. The 3D geometric shape, detail texture and outline of the object were rendered by force and vibration tactile feedbacks, respectively. Finally, these three features of the image were haptic expressed simultaneously by the integrated device. Haptic perception experiment results show that the multi-mode haptic display method can effectively improve the authenticity of haptic perception.

In this paper, two novel adaptive finite-time control schemes are proposed for position tracking of nonlinear teleoperation system, which dynamic uncertainties, actuator saturation, and time-varying communication delays are considered. First, a novel auxiliary variable is designed to provide more stable performance. The radial basis function (RBF) neural network is introduced to estimate dynamic uncertainties. Second, two adaptive finite-time control schemes are investigated. In control scheme I, the RBF neural network and the gain switching strategy are applied to compensate the actuator saturation. In control scheme II, an auxiliary compensation filter and the compensation adaptive update laws, which contain the finite-time structure, are developed for dealing with saturation. Third, the finite-time adaptive controller is designed in each of these two control schemes. Based on the multiple Lyapunov function method, the closed-loop teleoperation system with these two control methods is proved to be bounded and finite-time stability. Finally, the simulation experiments are performed and the comparisons with other control methods are shown. The effectiveness of the proposed control schemes is demonstrated.

In security e-health brain neurosurgery, one of the important processes is to move the electrocoagulation to the appropriate position in order to excavate the diseased tissue. 1 However, it has been problematic for surgeons to freely operate the electrocoagulation, as the workspace is very narrow in the brain. Due to the precision, vulnerability, and important function of brain tissues, it is essential to ensure the precision and safety of brain tissues surrounding the diseased part. The present study proposes the use of a robot-assisted tele-surgery system to accomplish the process. With the aim to achieve accuracy, an output-bounded and RBF neural network–based bilateral position control method was designed to guarantee the stability and accuracy of the operation process. For the purpose of accomplishing a minimal amount of bleeding and damage, an adaptive force control of the slave manipulator was proposed, allowing it to be appropriate to contact the susceptible vessels, nerves, and brain tissues. The stability was analyzed, and the numerical simulation results revealed the high performance of the proposed controls.

In this paper an automated data labeling (ADL) neural network is proposed to streamline dataset collecting for real-time predicting the continuous motion of hand and wrist, these gestures are only decoded from a surface electromyography (sEMG) array of eight channels. Unlike collecting both the bio-signals and hand motion signals as samples and labels in supervised learning, this algorithm only collects unlabeled sEMG into an unsupervised neural network, in which the hand motion labels are auto-generated. The coefficient of determination (R2) for three DOFs, i.e. wrist flex/extension, wrist pro/supination, hand open/close, was 0.86, 0.89 and 0.87 respectively. The comparison between real motion labels and auto-generated labels shows that the latter has earlier response than former. The results of Fitts’ law test indicate that ADL has capability of controlling multi-DOFs simultaneously even though the training set only contains sEMG data from single DOF gesture. Moreover, no more hand motion measurement needed which greatly helps upper limb amputee imagine the gesture of residual limb to control a dexterous prosthesis.

Successful sampling on the surface of asteroids is difficult because of their weightless environment and unknown material mechanical property. This work presents an asteroid sampler based on sweeping and grinding methods to improve the success rate of sampling. The sampler uses two brushes rotating clockwise and counter-clockwise to collect sample particles on the surface of asteroids. When encountering the hard rock or sample particles with large cohesion, the sampler adopts a drill bit to grind them to loose samples suitable for collecting by the brushes. The interaction between the brushes and the regolith is modeled and the sweeping mechanism is designed. A simple grinding mechanism is also designed. Numerical simulation and prototype experiments, at different parameters including blades number, rotational speed, and feeding speed of the brushes, mechanical property of the sample, and gravity, were conducted for validating the proposed methods. The 280g sampler prototype with 8 blades of brushes could collect about 19g regolith simulant in 25s in earth environment. The drill bits could work together with the brushes to improve the sampling efficiency through DEM simulation in case of large cohesion among sample particles. The sampler will be a good choice for installing into an asteroid rover in exploration.

This paper developed a 4-degree-of-freedom cable-driven upper limb rehabilitation robot and proposed a control algorithm of the passive training for this robot. Comparing with the conventional cable-driven rehabilitation robot, the workspace of this robot is increased by optimizing the distribution of the cable attachment points and by improving the mechanical design. The rotation structure of the upper arm module can change the distribution of the attachment points as needed, by which the cable tension planner can be satisfied in almost all cases. At the meantime, the internal/external rotation of shoulder joint can be achieved without the change of the cables configuration, which is also important for increasing the workspace and comfortability of utilization. The activities of daily living (ADLs) training can be achieved well without any manual adjustment. The related controller for passive training is designed, which includes a higher controller for trajectory tracking and a lower controller for keeping cable tension as the output of the tension planner in real-time. The passive training experiments are conducted on five healthy subjects of different body size. The results demonstrated that the passive training can be achieved well on different subjects and the cable tension controller is also working effectively.

The hop‐based localisation algorithm uses hop‐by‐hop propagation to establish node‐to‐anchor distance estimation, which does not require costly and complicated ranging hardware. This helps boost system performance, while minimising the cost of localising the nodes within the network. However, the application of hop‐based localisation algorithms is restricted due to their dramatic accuracy degradation in irregular network, which is mainly caused by the large error of distance estimation. The authors find that the error variance of the estimated distance increases as the hop count increases, i.e. there is a heteroscedasticity problem in the distance estimation process, which will affect the location estimation. In this study, by exploring the error during the location estimation, they aim to find and employ the optimal weighted function to improve localisation accuracy. A geometric constraint algorithm is also devised to correct the incorrectly estimated location by mitigating the adverse effects from flip ambiguity. By combining the optimal weighted function and the geometric constraint algorithm, a novel hop‐based localisation algorithm is proposed in this study. Both the theoretical analysis and experimental results show that the proposed method has not only maintained the economic characteristics of hop‐based localisation, but also has the high localisation accuracy where it can be adapted to various networks with different node distributions.

The multilayer perceptrons (MLPs) are widely used in many fields, however, singularities in the parameter space may seriously influence the learning dynamics of MLPs and cause strange learning behaviors. Given that the singularities are the subspaces of the parameter space where the Fisher information matrix (FIM) degenerates, the FIM plays a key role in the study of the singular learning dynamics of the MLPs. In this paper, we obtain the analytical form of the FIM for unipolar activation function-based MLPs where the input subjects to the Gaussian distribution with general covariance matrix and the unipolar error function is chosen as the activation function. Then three simulation experiments are taken to verify the validity of the obtained results.

Haptic interaction plays an important role in the virtual reality technology, which let a person not only view the 3D virtual environment but also realistically touch the virtual environment. As a key part of haptic interaction, force feedback has become an essential function for the haptic interaction. Therefore, multi-dimensional force sensors are widely used in the fields of virtual reality and augmented reality. In this paper, some conventional multi-dimensional force sensors based on different measurement principles, such as resistive, capacitive, piezoelectric, are briefly introduced. Then the mechanical structures of the elastic body of multi-dimensional force sensors are reviewed. It is obvious that the performance of the multi-dimensional force sensor is mainly dependent upon the mechanical structure of elastic body. Furthermore, the calibration process of the force sensor is analyzed, and problems in calibration are discussed. Interdimensional coupling error is one of the main factors affecting the measurement precision of the multi-dimensional force sensors. Therefore, reducing or even eliminating dimensional coupling error becomes a fundamental requirement in the design of multi-dimensional force sensors, and the decoupling state-of-art of the multi-dimensional force sensors are introduced in this paper. At last, the trends and current challenges of multi-dimensional force sensing technology are proposed.

Photonic Network-on-Chip (PNoC) has been a new trend for next generation multi-processor system. However, components, such as Micro-Ring Resonators (MRRs), in PNoC are fault prone and would not provide reliable operation unless fault components in the routers are detected and masked properly. There is very little work done to analyze and detect fault in PNoC. An approach based on fault check graph is proposed. An n-port photonic router is modelled as a complete weighted directed graph, which is called pre-Fault Check Graph, and MRR model is created. By the complete weighted directed graph and fault simulation, the proposed method is established with fault check graph and MRR model. The experimental results prove that the proposed approach is effective with corresponding fault simulation.

The brain-machine interface (BMI) has been reported to offer the potential for controlling the assistive robot for the motor impaired people, using the non-invasively obtained electroencephalogram (EEG) signals. However, the EEG based BMI may not be sufficient and stable to drive the robot moving freely in its 2D or 3D workspace. The robot autonomy may provide assistance for the BMI users with the shared control paradigm. Nevertheless, users suffers from several limitations of the current shared control paradigms applied on BMI, e.g., loss of sense of control, high mental workload due to unintuitive control with the human-robot interface and fixed level of assistance. To overcome these drawbacks, we propose a new control paradigm for the robotic arm reaching task where the robot autonomy is dynamically blended with the gaze-BMI control from a user. In this paradigm, the hybrid gaze-BMI constitutes an intuitive and effective input to continuously control the robotic arm end-effector moving freely in its 2D workspace, with an adjustable speed proportional to the motion intention strength. Furthermore, the adjustable level of assistance by our paradigm allows the system to balance the user's capabilities and feelings of control while compensating for the reaching task's difficulty. The proposed paradigm is verified in the task where a healthy subject utilizes the hybrid gaze-BMI to control the robotic arm end-effector reaching for a target object while avoiding the obstacle in the path. The experimental results demonstrate that the movements with our shared control paradigm are safer, more efficient and less difficult than those without shared control.

In this paper, a home Internet-of-Things system is analyzed by dividing it into four layers, i.e., the node layer, gateway layer, service layer, and open layer. The gateway layer, which supports a variety of wireless technologies and is the core of home wireless networks access unit, together with the node layer constitutes the home wireless network. A gateway prototype following the proposed architecture has been implemented. A testbed of an interference-aware wireless network which includes the gateway prototype has also been created for testing its user interaction performances. The experimental results show that both Wi-Fi and Bluetooth have an impact on the ZigBee communication. Considering the complex scene of home and building, ZigBee multihop communications are set to reduce the packet loss probability. In addition, an event-level-based transmission control strategy is proposed, in which the packet loss probability of wireless network is reduced by controlling the transmission priority of different levels of monitoring events, and optimizing the ZigBee wireless network channel occupancy.

The utility to decode hand movement parameters is significant to the control of artificial limb in the BCI fields. Most previous studies have adopted amplitude features of the low-frequency EEG signals to decode hand movement parameters. In this study, we have investigated the instantaneous phase of the low-frequency EEG signals attained by Hilbert transform for such a task for the first time, and compared its decoding accuracy with that of the amplitude features. An experiment was carried out that 5 subjects executed a center-out reaching task in two sessions. Then the Multiple Linear Regression (MLR) model is used to decode hand movement parameters based on the amplitude feature and the phase feature, respectively. The performance of the proposed approach is evaluated by calculating the correlation coefficients between the recorded parameters and the reconstructed parameters. The experiments results show that compared to the decoder with the amplitude feature, the correlation coefficients obtained by the decoder with the phase feature have increased 27.8% (X-position), 24.1% (Y-position), 27.9% (X-velocity), 20.9% (Y-velocity).

This paper presents a deployable boom which combines a rigid telescopic frame and a flexible tape spring. The front end of the spring is fixed on the rear end of the innermost segment of the frame. The spring spreads and rolls up inside the frame to drive the segments to move one by one to realize the boom deployment and retraction. The driving forces needed to deploy and retract the frame are modeled and simulated. The feasibility of the frame driving by only one spring is also studied. A 1.6 kg prototype system with 2.1 m total deployment length is implemented. Experimental results show the maximum driving forces for deployment and retraction of the frame are about 9.1 N and 6.8 N respectively. The boom is able to deploy in 76 s with energy consumption of 315 J. The boom can resist at least 15 N force axially and 31.5 N·m bending moment when the forces are acted on its front end. Advantages of this kind of boom enable it to be applied for instruments deployment, walking and sampling assists, and robotic arms design in space exploration.

Image feature extraction is one of the key technologies of image haptic display. In this paper, multi-feature extraction method of the object in image is proposed to improve image-based haptic perception. The multi-feature extraction includes contour shape extraction, pattern extraction and detail texture extraction. Firstly, we use an intrinsic decomposition method to decompose an image into shading image and reflectance image. The reflectance image describes nonillumination affected color patterns spread on the surface. Then, the shading image is utilized in contour shape and detail texture extraction. Contour shape extraction is based on partial differential equation (PDE), to reconstruct three-dimensional (3D) surface model in virtual environments. Detailed texture extraction is based on fractional differential method simultaneously. Finally, the various features extracted above are haptic rendered by different methods. The experimental results show the effectiveness and potentiality of the proposed method for improving the ability of haptic perception and recognition of human in virtual environments.

In this brief, the concept of logical stochastic resonance is applied to implement the Set-Reset latch in a synthetic gene network derived from a bacteriophage λ . Clear Set-Reset latch operation is obtained when the network is only subjected to periodic forcing. The correct probability of obtaining the desired logic operation first increases to unity and then decreases as the amplitude of the periodic forcing increases. In addition, the output logic operation can be easily morphed by tuning the frequency and the amplitude of the periodic forcing. At the same time, we indicate that adding moderate periodic forcing to the background Gaussian noise may increase the length of the optimal plateau of getting the desired logic operation in genetic regulatory network. We also point out that robust Set-Reset latch operation can be obtained using the interplay of periodic forcing and background noise when the noise strength is lower than what is required.

In addition to the noisy and limited spatial resolution characteristics of the electroencephalography (EEG) signal, the intrinsic nonstationarity in the EEG data makes the single-trial EEG classification an even more challenging problem in brain-computer interface (BCI). Variations of the signal properties within a session often result in deteriorated classification performance. This is mainly attributed to the reason that the routine feature extraction or classification method does not take the changes in the signal into account. Although several extensions to the standard feature extraction method have been proposed to reduce the sensitivity to nonstationarity in data, they optimize different objective functions from that of the subsequent classification model, and thereby, the extracted features may not be optimized for the classification. In this paper, we propose an approach that directly optimizes the classifier's discriminativity and robustness against the within-session nonstationarity of the EEG data through a single optimization paradigm, and show that it can greatly improve the performance, in particular for the subjects who have difficulty in controlling a BCI. Moreover, the experimental results on two benchmark data sets demonstrate that our approach significantly outperforms the compared approaches in reducing classification error rates.

The system center of gravity (SCG) is a critical element for the stability of a robot when it undergoes locomotion. In this paper, we propose a new algorithm for enhancing such stability by manipulating the location of the SCG. Specifically, we can prevent the robot from tipping over, rolling over, and tumbling over. The tipover stability criteria for a tracked mobile manipulator are discussed and the velocity kinematic model of the manipulator for SCG adjustment is also presented in this paper. The embedded 3-axial gyroscope provides us the data necessary for the SCG computation. The algorithm outputs the adjustments needed on the joint angles in order to maintain the SCG within a body-fixed safety zone. The experimental results verified the effectiveness of the proposed algorithm.

This paper presents a 10 cm × 5 cm × 5 cm, 52 g one-motor driven robot. One DC motor with a driving gear drives two driven gears to implement the functions of jumping and walking. Two one-way bearings mounted on the inner races of the two driven gears are used to switch between jumping and walking when the motor rotates clockwise and anticlockwise respectively. The jumping energy is obtained by compressing and releasing two torsion springs using a cylindrical cam with quick return characteristics. Two disk cams drive two forelegs with elastic joints to step forward one after another to implement the walking locomotion pattern. Two connecting rods link the forelegs and the rear legs on the left and right sides of the robot to transmit motions from forelegs to rear legs. The jumping and walking performances of the robot are tested. Experimental results show that the proposed robot can jump more than 33 cm high at a takeoff angle of 71.2° and it can walk forward at 1.43 mm/s.

In this brief, by employing an improved Lyapunov-Krasovskii functional (LKF) and combining the reciprocal convex technique with the convex one, a new sufficient condition is derived to guarantee a class of delayed neural networks (DNNs) to be globally asymptotically stable. Since some previously ignored terms can be considered during the estimation of the derivative of LKF, a less conservative stability criterion is derived in the forms of linear matrix inequalities, whose solvability heavily depends on the information of addressed DNNs. Finally, we demonstrate by two numerical examples that our results reduce the conservatism more efficiently than some currently used methods.

This paper presents a 9 cm × 7 cm × 12 cm, 154 g jumping robot with self-righting, steering, and takeoff angle adjusting capabilities. The quick energy releasing function of the jumping mechanism is implemented by using an eccentric cam. The self-righting, steering, and takeoff angle adjusting capabilities are achieved by adding a rotatable pole leg. The pole leg can prop up the body of the robot when it falls down. The pole leg can also steer the robot to turn at a step of about 24°. By adjusting the center of mass (COM), the robot can jump at different takeoff angles. Experimental results show that the constructed robot can jump more than 88 cm high at a takeoff angle of 82.7° and it can continuously jump to overcome stairs.

In this brief, based on Lyapunov-Krasovskii functional approach and appropriate integral inequality, a new sufficient condition is derived to guarantee the global stability for delayed neural networks with unbounded distributed delay, in which the improved delay-partitioning technique and general convex combination are employed. The LMI-based criterion heavily depends on both the upper and lower bounds on time delay and its derivative, which is different from the existent ones and has wider application fields than some present results. Finally, three numerical examples can illustrate the efficiency of the new method based on the reduced conservatism which can be achieved by thinning the delay interval.

In this paper, a new method to model the global deformation between a rigid object and an elastic object with a hole is presented. This method extends the idea of beam-skeletons [10] by introducing curved cantilever beams for efficient modeling of global deformation of elastic objects with holes. The method is implemented and tested on different examples. Results from three examples are given to demonstrate the efficiency and effectiveness of the approach.

This paper investigates robust exponential stability for discrete-time recurrent neural networks with both time-varying delay (0 < or = tau(m) < or = tau(k) < or = tau(M)) and distributed one. Through partitioning delay intervals [0, tau(m)] and [tau(m), tau(M)], respectively, and choosing an augmented Lyapunov-Krasovskii functional, the delay-dependent sufficient conditions are obtained by using free-weighting matrix and convex combination methods. These criteria are presented in terms of linear matrix inequalities (LMIs) and their feasibility can be easily checked by resorting to LMI in Matlab Toolbox in Ref. 1. The activation functions are not required to be differentiable or strictly monotonic, which generalizes those earlier forms. As an extension, we further consider the robust stability of discrete-time delayed Cohen-Grossberg neural networks. Finally, the effectiveness of the proposed results is further illustrated by three numerical examples in comparison with the reported ones.

This paper addresses a largely open problem in haptic simulation and rendering: contact force and deformation modeling for haptic simulation of grasping a deformable object with a realistic virtual human hand, especially in power grasps. The virtual hand model consists of meshes of realistic shapes for the finger links and palm of a hand. We tackle the problem by adopting the non-linear contact force model and the beam-skeleton model for global shape deformation introduced in [5]. The results verify the efficiency of contact force and deformation modeling for both power grasp and precision grasp of deformable objects with reasonable realism.

Many applications in virtual reality and telerobot call for the implementation of displaying to the human softness haptic on the object being touched. Although there are lots of literatures on discrimination thresholds for displacement, force magnitude, shape, and viscosity, there is still a lack of research on discrimination and remembrance of softness perception of human fingertip. In this paper, a novel stiffness display device based on deformable length of elastic element control is presented firstly. Then the experiments of human fingertip perception of softness haptic by using the device that we developed are carried out. The haptic perception of human finger which includes resolution, times, frequency and remembrance are discussed. At last, some important conclusions are drawn. The results can aid in the process of designing haptic devices.

A new softness display system has been developed which is based on the principle of deformable length of elastic element control (DLEEC). In this system, operator can feel the softness of the virtual objects with the softness display device. Compared with other haptic device, the device is passive and exert the react force only when the operator "actively touch" the virtual objects. The stability of the softness display system is analyzed, and some experimental results are presented to show the validity of the proposed approach

A novel four degree-of-freedom (4 DOF) wrist force/torque sensor is developed, which is mechanically decoupled. This type of wrist force/torque sensor is different from ordinary mechanically decoupled ones. It has a simple structure and small coupled interference or noise. It is easy to process and calibrate, and low in cost. This paper introduces the elastic body structure of the wrist force/torque sensor, and analyses the mechanically decoupled principle in detail

In the research of reconnaissance robot to respond events involving hazardous materials, a novel hybrid behavior coordination mechanism based on priority and FSA is proposed. It uses a behavior group which combines several elementary behaviors based on priority to perform simple scout tasks. And then uses one of specified FSAs designed for each more complex task respectively as the behavior group selector. The key feature is that a hybrid behavior group coordinator can be structured dynamically once the corresponding task is required to be performed. Thus, such a behavior-based robot is capable of performing a goal-oriented task by this method. The implementation of a hybrid behavior coordinator used to perform the task of moving to goal is presented in detail. Simulations and experiments show the validity, robustness, and simplicity of the hybrid behavior mechanism

A novel method based on deformable length of elastic element control (DLEEC) to realize the stiffness display for perception of virtual soft object is proposed, and the stiffness display interface device has been developed and is presented. The stiffness display interface device is composed of a thin elastic beam and an actuator to adjust the length of the beam. The deformation of the beam under a force is proportional to the third power of the beam length. By controlling the beam length, the stiffness display device can reproduce the stiffness of the virtual object from very soft to hard, so that the human fingertip can feel it as if he directly touches with the virtual object by interacting with the device. A real time position control algorithm is employed to guarantee the real time stiffness display.

This paper describes a novel distributed architecture for building Web-enabled remote robotic laboratories. The solution presented here focuses mainly on three aspects: easy and efficient network communication in client-server applications, Internet access of networked sensors, portable architecture based on Java 2 platform. These goals have been achieved by the employment of Java 2 platform and other Web tools. The proposed architecture has been successfully implemented by a demonstration project to run a remote robotic laboratory. Experiment results of the demonstration project prove that the architecture works well and functions to be open, extensible, Web-enabled and platform independent. The original Web site of this demonstration project is offline now. But a video of the demonstration project can be downloaded from http://robot.seu.edu.cn/websensor/.

Telerobotics, the body of science and technology which bridge human control and purely autonomous machines, is expected to be a merging point of modern developments in robotics, control theory, cognitive science and computer science. A telerobot helps the human operator feel and control the remote pre-unknown environment as if he/she is at the remote site, which is usually called "telepresence". However, the transmission time delay between master manipulator and slave manipulator causes instability of the telerobot system and bad "telepresence" of the human operator. In this paper, a new adaptive and passive control scheme based on active impedance matching is proposed to guarantee stability and transparency of the bi-lateral telerobot system. An adaptive impedance term which is active and should adapt to the impedance of environment is defined in this new control scheme. A genetic algorithm (GA) is used to directly identify parameters of the environment impedance, because the GA can success in the search for global optimum when the search space is not differentiable or linear in the parameters.