Advanced Planning and Scheduling (APS) systems have become indispensable for modern manufacturing operations, enabling optimized resource allocation and production efficiency in increasingly complex and dynamic environments. While algorithms for solving abstracted scheduling problems have been extensively investigated, the critical prerequisite of specifying manufacturing requirements into formal constraints remains manual and labor-intensive. Although recent advances of generative models, particularly Large Language Models (LLMs), show promise in automating constraint specification from heterogeneous raw manufacturing data, their direct application faces challenges due to natural language ambiguity, non-deterministic outputs, and limited domain-specific knowledge. This paper presents a constraint-centric architecture that regulates LLMs to perform reliable automated constraint specification for production scheduling. The architecture defines a hierarchical structural space organized across three levels, implemented through domain-specific representation to ensure precision and reliability while maintaining flexibility. Furthermore, an automated production scenario adaptation algorithm is designed and deployed to efficiently customize the architecture for specific manufacturing configurations. Experimental results demonstrate that the proposed approach successfully balances the generative capabilities of LLMs with the reliability requirements of manufacturing systems, significantly outperforming pure LLM-based approaches in constraint specification tasks.

Near-eye gaze estimation has been a core technology for natural interaction in smart head-mounted devices. Existing gaze estimation approaches have the following limitations: (1) Data-driven methods learn nonlinear mappings directly from images to gaze direction but ignore the physiological characteristics of the iris–pupil system. (2) Model-driven methods rely on hand-designed modeling and perform poorly in complex, noisy scenes. (3) Hybrid-driven methods attempt to combine both paradigms but are restricted by simplified model designs. To address these issues, we propose KDB-Gaze, aKeypoint-GuidedDual-Branch learning for near-eyeGazeestimation. First, a backbone and a 3-Pass path aggregation feature pyramid network effectively capture multi-scale ocular features. Then, two parallel branches refine the features: an implicit spatial modeling branch employs deformable DETR to capture nonlinear features, while an explicit topological constraint branch uses a graph attention network to provide stable physiological guidance. Finally, the dual-branch features are fused and optimized jointly with a multi-task loss for precise prediction. Extensive experiments on the TEyeD, LPW, and NVGaze datasets verify the state-of-the-art performance of the proposed method.

Self-driving laboratories have begun to replace human experimenters in performing single experimental skills or predetermined experimental protocols. However, as the pace of idea iteration in scientific research has been intensified by Artificial Intelligence, the demand for rapid design of new protocols for new discoveries become evident. Efforts to automate protocol design have been initiated, but the capabilities of knowledge-based machine designers, such as Large Language Models, have not been fully elicited, probably for the absence of a systematic representation of experimental knowledge, as opposed to isolated, flatten pieces of information. To tackle this issue, we propose a multi-faceted, multi-scale representation, where instance actions, generalized operations, and product flow models are hierarchically encapsulated using Domain-Specific Languages. We further develop a data-driven algorithm based on non-parametric modeling that autonomously customizes these representations for specific domains. The proposed representation is equipped with various machine designers to manage protocol design tasks, including planning, modification, and adjustment. The results demonstrate that the proposed method could effectively complement Large Language Models in the protocol design process, serving as an auxiliary module in the realm of machine-assisted scientific exploration.

Industrial designers have long sought a natural and intuitive way to achieve the targeted control of prototype models---using simple natural language instructions to configure and adjust the models seamlessly according to their intentions, without relying on complex modeling commands. While Large Language Models have shown promise in this area, their potential for controlling prototype models through language remains partially underutilized. This limitation stems from gaps between designers' languages and modeling languages, including mismatch in abstraction levels, fluctuation in semantic precision, and divergence in lexical scopes. To bridge these gaps, we propose an interface architecture that serves as a medium between the two languages. Grounded in design principles derived from a systematic investigation of fast prototyping practices, we devise the interface's operational mechanism and develop an algorithm for its automated domain specification. Both machine-based evaluations and human studies on fast prototyping across various product design domains demonstrate the interface's potential to function as an auxiliary module for Large Language Models, enabling precise and effective targeted control of prototype models.

Knee pain is prevalent in over 20% of the population, limiting the mobility of those affected. In turn, isokinetic dynamometers and robots have been used to facilitate rehabilitation for those still capable of ambulation. However, there are at most only a few wearable robots capable of delivering isokinetic training for bedridden patients. Here, we developed a wearable robot that provides bedside isokinetic training by utilizing a variable stiffness actuator and dynamic energy regeneration. The efficacy of this device was validated in a study involving six subjects with debilitating knee injuries. During two courses of rehabilitation over a total of three weeks, the average peak torque, average torque, and average work produced by their affected knees increased significantly by 81.0%, 101.4%, and 117.6%, respectively. Furthermore, the device's energy regeneration features were found capable of extending its operating time to 198 days under normal usage, representing a 57.8% increase over the same device without regeneration. These results suggest potential methodologies for delivering isokinetic joint rehabilitation to bedridden patients in areas with limited infrastructure.

Evolutionary pressures have pushed humans to become efficient walkers, but inefficient divers. People consume more energy to travel the same distance underwater than on land. In diverse overground locomotion, emerging exoskeletons have reduced the metabolic cost of humans. Can we also improve the energy economy in underwater locomotion via exoskeletons? Here, we propose an underwater exoskeleton to assist scuba diving using flutter kick, by applying assistive knee extension torque during the strike phase of the diving kick cycle. When divers wore the powered exoskeleton, the average net air cost across six experienced divers was reduced by 22.7$\pm$10.0%, and the peak quadriceps activation was decreased by 20.9$\pm$7.5%, compared with normal diving without the exoskeleton. The average gastrocnemius activation also decreased by 20.6$\pm$5.3%, suggesting that the divers sufficiently utilized the exoskeleton assistance. These results indicate that applying exoskeleton assistance is conducive to improving the endurance of human underwater diving and enhancing our ability to explore the underwater world. Our study extends the application boundary of wearable robots, and provides a reference for the design and assessment of future underwater assistive devices, with the potential to strengthen the connection between humans and the ocean.

Representing human arm dynamic intent is essential for efective human-robot interaction. Accurately and robustly decoding these intentions through mathematical modeling of neuromuscular processes poses signifcant challenges. This study introduces an EIT-driven musculoskeletal model which integrates an EIT sensing system with methods for muscle identifcation, parameter estimation, and musculoskeletal system modeling. Unlike existing muscle-signal techniques, EIT captures muscle activities from the anatomical cross-sectional plane, providing both activation dynamics and morphological features. We validated our method through multi-DoF wrist kinematics estimation under varying contraction intensities, arm endpoint stifness estimation, and robotic variable admittance control. Our approach achieves accuracy comparable to state-of-the-art methods while requiring fewer training samples and a more compact sensing system. The model incorporates physiological constraints, minimizing decoding errors and ensuring interaction safety. This method enables reliable intent decoding with practical training demands. Future work will enhance the EIT system for complex tasks.

Eye tracking technology, essential for enhancing user experience in virtual reality (VR) and augmented reality (AR) devices, has been widely incorporated into advanced head-mounted devices like the Apple Vision Pro and PICO 4 Pro, becoming a standard feature. However, dedicated eye tracking datasets for such devices are severely lacking, with existing datasets commonly facing issues like camera skew and low resolution, particularly failing to adequately consider the diversity in wearing postures. To address this gap, we have developed the Posture-Variant Eye Tracking Dataset (PVEye), which includes 11,044,800 high-resolution near-eye images from 104 participants, showcasing a rich variety of wearing postures. This dataset aims to advance the development and application of appearance-based eye tracking methods. Utilizing this dataset, our evaluations demonstrate that the appearance-based method, particularly the NVGaze model, provides improved accuracy and robustness compared to the traditional feature-based method. Crucially, our experiments indicate that variations in wearing posture can significantly impact eye tracking performance, with posture-related errors contributing approximately 45% to the overall error variance. Moreover, the study delves into the specific impact of calibration and other critical factors on eye tracking performance, offering insights for further optimization of tracking effectiveness.

Yu-Zhe Shi, Haofei Hou, Zhangqian Bi, Fanxu Meng, Xiang Wei, Lecheng Ruan, Qining Wang. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Recent development in Artificial Intelligence (AI) models has propelled their application in scientific discovery, but the validation and exploration of these discoveries require subsequent empirical experimentation. The concept of self-driving laboratories promises to automate and thus boost the experimental process following AI-driven discoveries. However, the transition of experimental protocols, originally crafted for human comprehension, into formats interpretable by machines presents significant challenges, which, within the context of specific expert domain, encompass the necessity for structured as opposed to natural language, the imperative for explicit rather than tacit knowledge, and the preservation of causality and consistency throughout protocol steps. Presently, the task of protocol translation predominantly requires the manual and labor-intensive involvement of domain experts and information technology specialists, rendering the process time-intensive. To address these issues, we propose a framework that automates the protocol translation process through a three-stage workflow, which incrementally constructs Protocol Dependence Graphs (PDGs) that approach structured on the syntax level, completed on the semantics level, and linked on the execution level. Quantitative and qualitative evaluations have demonstrated its performance at par with that of human experts, underscoring its potential to significantly expedite and democratize the process of scientific discovery by elevating the automation capabilities within self-driving laboratories.

With the advances in microfluidics, electrowettingon-dielectric (EWOD) chips have widely been applied to various biological and chemical laboratory protocols. Glass-based EWOD chips with non-regular electrodes are proposed, which allow more reliable droplet operations and facilitate the integration of optical sensors for many biochemical applications. Furthermore, non-regular electrode designs are utilized in EWOD chips, e.g., interdigitated electrodes for more reliable droplet manipulation, custom shaped electrodes for specific applications like concentric heating, etc. However, due to the technical challenges of fabricating multi-layer interconnection on the glass substrate, e.g., unreliable process and high cost, both control electrodes and wires are fabricated with a single-layer configuration, which poses significant challenges to pin selection for non-regular electrodes. In this paper, we propose a minimum-cost flow-based routing algorithm called NR-Router+ that features efficient and robust routing for single-layer EWOD chips with non-regular electrodes. To the best of our knowledge, this is the first work that overcomes the aforementioned challenges. We construct a minimum-cost flow algorithm to generate optimal routing paths followed by a light-weight model to handle flow capacity. A grid reduction strategy is proposed to reduce the computational overhead. Additionally, a flow collocation algorithm based on integer linear programming is presented to efficiently prevent wire overlapping. Experimental results show that NR-Router+ achieves 100% routability while minimizing wirelength with shorter run time. Moreover, NR-Router+ can generate mask files feasible for manufacturing via adjustments of design parameters, thus demonstrating its robustness and efficiency.

Wearable robotic devices interact with human by applying assistive force in parallel with muscle-tendon systems. Designing actuations in mimicking the natural activation patterns of human muscles is a promising way to optimize the performance of wearable robots. In this paper, we propose a bio-inspired cable-driven actuation system capable of providing anisometric contractions (including concentric and eccentric contraction) assistance or nearly acting as a transparent device in an efficient manner. A novel clutch-spring mechanism is employed to accomplish switches between assistive modes and the transparent mode. Corresponding control strategies coordinating with the mechanical design were presented and described in detail. Multiple evaluations were conducted on a test bench to characterize the system performance. The closed-loop bandwidth of the system running concentric assistance control was 18.2 Hz. The R-squared values of linear fitting under eccentric assistance control were above 0.99. The engagement time of the proposed clutch was about 90 ms. Applying the actuation to an ankle exoskeleton, multiple walking experiments with electromyography measurement were performed on five subjects to show its application potential in existing wearable robots. Experimental results revealed that the proposed design could reduce soleus muscle activity by 27.32% compared with normal walking. This study highlights the importance of functional bionic design in human-assistance-related devices and introduces a general actuation system that could be directly applied to existing cable-driven wearable robots.

With the advances in microfluidics, electrowetting-on-dielectric (EWOD) chips have widely been applied to various laboratory procedures. Glass-based EWOD chips with non-regular electrodes are proposed, which allow more reliable droplet operations and facilitating integration of optical sensors for many biochemical applications. Besides, non-regular electrode designs (e.g., interdigitated electrodes) are utilized in EWOD chips to precisely control droplet volume, and electrodes with a specific shape become necessary for certain applications. However, due to the technical barriers of fabricating multi-layer interconnection on the glass substrate (e.g., unreliable process and high cost), both control electrodes and wires are fabri-cated with a single-layer configuration, which poses significant challenges to pin selection for non-regular electrodes under the limited routing resource. In this paper, we propose a minimum cost flow-based routing algorithm called NR-Router that features efficient and robust routing for single-layer EWOD chips with non-regular electrodes, which overcomes the challenges mentioned above. NR- Router is the first algorithm that can accurately route in single-layer EWOD chips with non-regular electrodes to the best of our knowledge. We construct a minimum cost flow algorithm to generate optimal routing paths followed by a light-weight model to handle flow capacity. NR-Router achieves 100% routability while minimizing wirelength at shorter run time, and generates mask files feasible for manufacturing via adjustments of design parameters. Experimental results demonstrate the robustness and efficiency of our proposed algorithm.

In this paper, we present a low-damping active knee prosthesis (LDKP) with low noise and high backdrivability. The proposed prosthesis is driven by a motor and then decelerated by a four-stage synchronous belt. High backdrivability given by this structure accelerates the prosthetic response. A control system containing several sensors are embedded in the proposed prosthesis to recognize different modes and provide corresponding strategies. Preliminary experiments were carried out on a transfemoral amputee subject, demonstrating the features of low noise, high backdrivability and ability to reproduce natural walking gaits.

Maximum dorsiflexion (MDF) is an important gait event corresponding to the maximum ankle dorsiflexion angle in each gait cycle. MDF timing plays an important role in the control of robotic prosthesis. This article puts forward an on-board adaptive algorithm to detect MDF timing of robotic transtibial prosthesis in different walking conditions (at different speeds and on different ramps) and for different users. Based on the adaptive algorithm, we can get a time-variant detection model. The framework of the adaptive algorithm is composed of: 1) training data collecting and labeling; 2) model training and real-time detection; and 3) model updating according to the detection results. Based on the adaptive algorithm, we conducted speed and ramp experiments to detect MDF timings at slow, normal, and fast speeds, and on ramps with different inclination angles (10°, 5°, 0°, -5°, and -10°). Three transtibial amputee participated in the experiments. The model training/updating time ranges from 3.6 to 4.1 s and the detection time ranges from 0.95 to 1.17 ms for different speeds and ramps. In real-time detection, there is false detection (1.67%) at normal walking speed. In addition, all MDF timings are detected correctly (accuracy: 100%) based on the adaptive algorithm. The mean detection delays are 7.23, 18.27, and 7.5 ms corresponding to slow, normal and fast speeds and 10.60, 10.30, 18.27, 10.27, and 15.63 ms for ramps of different inclination angles (10°, 5°, 0°, -5°, and -10°). Compared with the proposed adaptive algorithm, both the nonadaptive and adaptive threshold decision methods cause more false detections. The results show that the proposed approach for MDF timing detection has adaptations to different walking conditions (speeds and ramps) and prosthesis users, which indicates that the adaptive algorithm is effective and shows the potential in robotic prosthesis control in the future. Note to Practitioners-This article proposes an on-board adaptive algorithm to detect the maximum dorsiflexion (MDF) timing based on inertial measurement unit (IMU) and ankle angle sensor for robotic transtibial prosthesis users in each gait cycle. IMU and angle sensor are integrated in the prosthesis, and the adaptive algorithm is embedded in the control circuit of prosthesis. The adaptive algorithm can realize the model updating continuously for real-time MDF timing detection with collected and labeled training data. The proposed adaptive algorithm shows satisfactory adaptation for MDF timing detection in different walking speed and ramp conditions. In addition, the adaptive algorithm also shows some generalizations for prosthesis users, which are useful to improve prosthesis control.

Wearable robotic devices often need electrical energy. An interesting idea is to collect mechanical energy during walking and convert it into electrical energy to recharge these devices directly. In this article, we built a light-weight robotic prosthesis (1.3 kg) with the feature of self-charging. During stance phase, the prosthetic ankle joint with damping, is driven by human dynamics. The rotated ankle joint backdrives the motor, and the motor works as a generator according to the electromagnetic induction theory. Five subjects participated in experiments to verify the feasibility and five speeds walking were studied (0.7, 0.9, 1.1, and 1.3 m/s treadmill speeds and one self-selected outdoor walking speed). Experimental results demonstrate that the electrical regenerative energy per step is 1.53 ± 0.29 J on average. Meanwhile, an average consumed energy per step of the robotic prosthesis is 4.64 ± 0.15 J, which means 33 ± 5% energy can be returned to the active prosthesis (battery, 24 V, 2.6 Ah).

Locomotion mode recognition based on the off-line trained model brings difficulties in integration and application to wearable robots. In this paper, we put forward an on-board training based on back propagation (BP) neural network and developed the real-time locomotion mode recognition research in robotic transtibial prosthesis. Three transtibial amputees participated in the study to finish the designed six experimental tasks (standing, level ground walking, stair ascending and descending, ramp ascending and descending) with robotic transtibial prostheses. Data of six locomotion modes were collected under normal speed condition as training data set to train model on board. Based on the on-board trained models, real-time recognition experiments were developed under three different speeds conditions. The total recognition accuracies were 91.54%, 96.72% and 95.35% corresponding to slow, normal and fast speeds, respectively. The results showed some adaptation of recognition for the six locomotion modes at different speeds. The on-board training strategy was feasible and effective with satisfactory performance.

In this study, we proposed a novel human-machine interface (HMI) for robotic manipulator control. The specific target was to adjust the impedance coefficients of the robot controller in real time by measuring the human forearm muscle contractions. We firstly designed a HMI system. Different from the frequently used sEMG technologies, the interface in our study could detect muscle morphological changes within the skin by the electrical impedance tomography (EIT). The sensing front-end was a soft elastic fabric band which was compatible to different arm shapes. With the specific designed sensing hardware and the re-construction algorithms, EIT images indicating forearm muscle shapes were obtained. We then designed a hybrid positon/impedance controller on a UR5 with the impedance coefficients being tuned in real time by the grasp force estimation. A sigmoid regression algorithm was used to map the EIT images to the grasp forces. After implementation of the whole system, two experiments were carried out. The first experiment was the off-line grasp force estimation. With the 1:1 cross validation, an average R2of 0.83±0.04 and an average of the relative root mean square error (RRMSE) of 0.31±0.10 across 5 subjects were yielded. The second experiment was the real-time robot control. Trajectory tracking task with dynamic uncertainties were investigated and grasp forces were estimated in real-time. With higher muscle contraction levels, smaller position errors were observed and shorter time was needed to return to the expected trajectory when there were external disturbances. The results proved the feasibility of the new approach on human-robot interaction tasks. Future endeavours will be made to get more promising results.

This paper focuses on chaos control of a seven-link torque-stiffness-controlled dynamic walking model, actuated by a bio-inspired control system. The biped consists of compliant hip, knee and ankle joints and flat feet. We employed Ott-Grebogi-Yorke and delayed feedback control methods, responsible for small errors around the equilibrium solution and large errors far away, respectively. In simulation, we study the stabilization of bifurcations and chaotic behaviors under diverse actuation parameters, and the convergence speed to 1-period gaits. The results of this paper may provide insights into motion control of dynamic walking robots and principles of human locomotion.

During early and middle stance of level-ground walking, the ankle joint usually rotates passively due to the locomotion of human body. Based on this passive dynamics of human locomotion, we develop an energy-efficient torque controller with hierarchical structure for a robotic prosthesis. The low-level controller generates motor current from the human locomotion and controls the motor current/torque according to the high-level command. The high-level controller consists of a forward estimator and a feedback compensator. The forward estimator estimates the motor torque according to the desired ankle torque based on the prosthesis model that represents the transmission gain between the motor torque and the ankle torque. The feedback compensator compensates for the model and low-level control errors based on the torque measurements. Step response and frequency response show that the low-level controller could reach the desired value within 0.3 seconds and had a bandwidth of 4.2 Hz. Experiments of constant and linear torque tracking achieve small RMS tracking errors, which confirm the effectiveness of the proposed high-level controller.

This paper presents a realtime locomotion mode recognition method for an active pelvis orthosis. Five locomotion modes, including sitting, standing still, level-ground walking, ascending stairs, and descending stairs, are taken into consideration. The recognition is performed with locomotion information measured by the onboard hip angle sensors and the pressure insoles. These five modes are firstly divided into static modes and dynamic modes, and the two kinds are classified by monitoring the variation of the relative hip angles of the two legs within a pre-defined period. Static states are further classified into sitting and standing still based on the absolute hip angle. As for dynamic modes, a fuzzy-logic based method is proposed for the recognition. Two event-based locomotion features, including the hip joint angle at the first foot-strike and the center of foot pressure at the first foot-strike are used to calculate the membership of different modes based on the membership function, and the mode with the maximal membership is selected as the target mode. Experimental results with three subjects achieve an average recognition accuracy of 99.87% and average recognition delay of 18.12% of one gait cycle.

Ankle joint with contracture and/or spasticity can severely disable the mobility and the independence of stroke survivors. Robot-assisted rehabilitation has been proposed to support physicians in providing effective therapies. In this paper, we propose a robot-assisted ankle rehabilitation system integrated with human-computer interaction interface and acquisition of Electromyography signals, joint torque and joint angle. Furthermore, we investigate the effects of proprioceptive neuromuscular facilitation (PNF) rehabilitation method applied in our robotic system. The proposed robot-assisted system has been used in real experiments and provides PNF rehabilitation to five stroke patients for six weeks. Preliminary experimental results suggest that PNF is effective in increasing ankle range of motion(ROM), decreasing ankle resistance torque, and alleviating joint stiffness.

This paper presents a fuzzy logic based terrain identification method using multi-sensor fusion for powered prosthesis control. Five locomotion features including rising time of ground reaction force, sequence of foot strike on ground, foot inclination angle during stance, shank inclination angle at toe-off and maximal shank inclination angle during swing are selected to identify different terrains. These features are measured by fusion of two gyroscopes, two accelerometers, two force sensitive resistors and a timer. Based on the features, a fuzzy logic identification method is developed to identify level-ground, stair ascent, stair descent, upslope and downslope online in real time. Average identification accuracy higher than 97.5% is obtained in experiments of five able-bodied subjects and a transtibial amputee. Continuous identification results show the prospect of using the proposed method to realize real-time terrain identification of powered prostheses.

This paper presents a bio-inspired below-knee exoskeleton to assist human walking. Different from the passive orthotic devices, the proposed exoskeleton includes powered compliant ankle and toe joints, which can output sufficient power to help the one with exoskeleton relearn normal walking gaits. We first propose a passivity-based dynamic bipedal model to analyze the effects of segmented foot and compliant joints on energetic efficiency and stability of bipedal walking. Starting from the results of theoretical analysis, we construct a below-knee exoskeleton prototype with ankle and toe joints driven by two series-elastic actuators. Experimental results validate the effectiveness of the proposed exoskeleton.

This paper presents a seven-link dynamic walking model that is more close to human beings. We add hip actuation, upper body, flat feet and compliant ankle joints to the model. Walking sequence of the flat-foot walker has several sub-streams that form bipedal walking with dynamic series of phases, which is different with the motion of round-foot and point-foot models. We investigate the characteristics of three different walking gaits with different step lengths. Comparison of these walking gaits in walking velocity, efficiency and stability reveals the relation between step length and walking performance. Experimental results indicate that the gait which is more close to human normal walking achieves higher stability and energetic efficiency.

This paper presents a novel approach for a group of vision-based autonomous robots to localize in dynamic environments. We propose a hybrid system method for localization consisted of on-line and off-line subsystems inspired by human cognition. For the on-line subsystem, we use the landmark based Markov localization method to estimate the position. When the robot does not update the probability of current position through landmarks for a certain period, we use the off-line experience subsystem to help. In addition to the hybrid system for individual localization, we propose a method of dynamic reference object for collaborative localization. By using this method, an autonomous robot can estimate and correct its position perception more accurately and effectively, taking the odometry error and other negative influence into consideration. Satisfactory experimental results are obtained in the RoboCup Four-Legged League environment