Precision orthodontics relies heavily on timely precision measurement during an orthodontic treatment. However, the measurement done by orthodontists currently (i.e., calipers) cannot provide a fast and precise measurement. The study introduces a novel real-time automatic image-based system for precision tooth measurement, characterized by its high accuracy and efficiency. The video input of the dentures from a head-mounted camera was taken one frame at a time. Since 70% of orthodontal treatment is done using fixed brackets, we used an object detection algorithm to locate the position of the brackets within the frame. A novel multi-object tracking algorithm was developed to track and update the positions based on the long short-time memory information collected on the previous frames. A measurement module then measures and updates the distance between each bracket and saves it making sure the measurements are available even when the bracket is not visible in the current frame. The real-time tracking and measurement improved the efficiency and accuracy of tooth measurement. In clinical cases, the teeth were recognized with 97% accuracy and tracked without any loss within a video input of 60 FPS. This measurement achieves a resolution on the order of 0.01 mm, with a data error less than 50 μm. This real-time automatic tooth measurement proved to be efficient and easy to use compared to existing methods while improving accuracy. It can be used in aiding precision orthodontics and in the process of automating dental treatments.

Automated surgical workflow recognition is a key enabler for context-aware computer-assisted surgery. Due to the scarcity of labeled surgical videos, training models from scratch or fully fine-tuning large vision architectures is often resourceintensive and can lead to overfitting. To alleviate this bottleneck, we insert a lightweight spatiotemporal adapter module into an image-pretrained backbone to inject temporal reasoning capability into the backbone and fine-tune the entire network. This not only preserves the backbone's original strong spatial representation capability, but also enables the model to learn such issues as the characteristic long-range temporal dependencies in surgical scenarios and the fine-grained interactions between instruments and organs, thereby achieving robust transfer to surgical video understanding tasks. Specifically, we propose a parameter-efficient Gated Dual-Path Spatiotemporal Adapter (GDPA) for surgical workflow recognition. GDPA is a lightweight module inserted into a frozen image-pretrained backbone to enable parallel spatial and temporal modeling. Within each GDPA block, the spatial and temporal branches operate in parallel, and their outputs are fused by an adaptive gating mechanism. This gating mechanism adaptively allocates the contributions of the two branches according to phase-dependent requirements for fine-grained spatial semantics and long-range temporal cues. Compared with previous methods, GDPA requires only a small number of trainable parameters and low computational cost. By retaining the backbone's pretrained visual knowledge and training only lightweight adapters in an end-to-end manner, it achieves higher accuracy even in data-limited scenarios. We evaluate our method on the Cholec80 and AutoLaparo surgical video benchmarks. The results show that even with a parameter budget far below that of full fine-tuning, GDPA still achieves competitive or even superior performance.

Graph based models have significantly advanced skeleton-based action recognition, yet most methods rely on pairwise connections and struggle to capture higher order, rhythm consistent dependencies. We introduce the Spectral Adaptive Hypergraph Convolution Network (SA-HyperGCN), which constructs hyperedges in the frequency domain to model coordinated multi joint relations. By transforming joint trajectories into spectral representations and adaptively grouping joints via low-frequency similarity, SA-HyperGCN discovers actiondriven structures beyond spatial adjacency. The resulting spectral hypergraph is fused with the static skeleton topology through multi-head hypergraph convolution for expressive message passing. We further incorporate virtual joints into the hypergraph and apply a topology-aware regularization that discourages overly similar geometric configurations. This constraint helps maintain diversity among the learned virtual joint structures and stabilizes training. Extensive experiments on NTU RGB+D 60, NTU RGB+D 120 and NW-UCLA show that SA-HyperGCN consistently outperforms strong GCN and hypergraph baselines, validating the effectiveness of spectral modeling and topologyguided regularization.

Hand gesture and movement analysis is a crucial learning task in Human-computer interaction (HCI) applications. Sensor-based HCI systems simultaneously capture the information with multiple locations to track the coordination of different regions of muscles. Based on the fact that there exists a temporal correlation between the regions, the connectivity analysis of sensor signals builds a network. The graph-based approach for analyzing the sensor network has provided novel insight into the learning in HCI, which has not been broadly investigated in hand gesture recognition tasks. This work proposes a topological representation learning scheme as a graph-based approach for sensor network analysis. Through investigation of the topological properties with persistent homology, the spatial-temporal characteristics are well described to build recognition models. Experiments on the NinaPro DB-2, DB-4, DB-5, and DB-7 datasets with sensor networks built with sEMG signal and IMU signal demonstrate exceptional performance of the proposed topological approach. The topological features are effective in graph representation learning with sensor networks used in hand gesture recognition. The proposed work provides a novel learning scheme in HCI systems and human-in-the-loop studies.

This work presents a topological nonlinear analysis approach for dynamical system measurements, frequently appearing in sensor-based inference tasks in human physical activity analysis. Traditional approaches to dynamical modeling included linear and nonlinear methods with specific representational abilities and some drawbacks. A novel approach we investigate is using topological descriptors of the shape of the dynamical attractor to represent the nature of dynamics. The proposed framework has three essential advantages compared to previous approaches: 1) with nonlinear phase space reconstruction, the dynamics descriptor is derived from the observation time series without any statistical assumption; 2) with the topological data analysis technique, the phase space topological properties are described in an intrinsic multiresolution analytical way, which brings novel information compared to traditional phase-space modeling techniques; 3) with different types of measurement sensing signals, the proposed approach shows stability in activities state inference. We illustrate our idea with the physical activity recognition tasks with wearable sensors, where the topological characteristics of reconstructed phase state space show strong representational ability for activity type inference.

Detecting pupil from the image is critical in human-machine interaction and biomedical computing applications, which is supposed to be an actual image segmentation problem. Recently developed deep learning models provide a variety of novel approaches to the pupil segmentation task. However, dataset preparation and annotation acquirement to build pupil image datasets are labor-intensive and time-consuming. The shortage of labeled samples restricted the improvement of deep learning models. In this work, we use a mask image modeling mechanism to learn the latent representation from limited data samples, which significantly helps train deep models. Further, we propose a novel pupil segmentation model based on the recently proposed Swin-Transformer to validate the improvement validity of the mask mechanism. The proposed computational framework achieves better performance on the pupil segmentation tasks based on the LPW dataset through comparison experiments with other related deep learning models. The proposed framework is a promising solution for pupil segmentation and detection in small-sample learning applications.

The sensor-based human activity recognition (HAR) in mobile application scenarios is often confronted with variation in sensing modalities and deficiencies in annotated samples. To address these two challenging problems, we devised a graph-inspired deep learning approach that uses data from human-body mounted wearable sensors. As a step toward a complete HAR solution, the proposed method was further used to build a deep transfer learning model. Specifically, we present a multi-layer residual structure involving graph convolutional neural network (ResGCNN) toward the sensor-based HAR tasks, namely the HAR-ResGCNN approach. Experimental results on the PAMAP2 and mHealth data sets demonstrate that our ResGCNN is effective at capturing the characteristics of actions with comparable results compared to other sensor-based HAR models (with an average accuracy of 98.18% and 99.07%, respectively). More importantly, the parameter-based transfer learning experiments using the ResGCNN model show excellent transferability and small sample learning ability, which is a promising solution in sensor-based HAR applications.

Since the tumor moves with the patient's breathing movement in clinical surgery, the real-time prediction of respiratory movement is required to improve the efficacy of radiotherapy. Some RNN-based respiratory management methods have been proposed for this purpose. However, these existing RNN-based methods often suffer from the degradation of generalization performance for a long-term window (such as 600 ms) because of the structural consistency constraints. In this paper, we propose an innovative Long Short-term Transformer (LSTformer) for long-term real-time accurate respiratory prediction. Specifically, a novel Long-term Information Enhancement module (LIE) is proposed to solve the performance degradation under a long window by increasing the long-term memory of latent variables. A lightweight Transformer Encoder (LTE) is proposed to satisfy the real-time requirement via simplifying the architecture and limiting the number of layers. In addition, we propose an application-oriented data augmentation strategy to generalize our LSTformer to practical application scenarios, especially robotic radiotherapy. Extensive experiments on our augmented dataset and publicly available dataset demonstrate the state-of-the-art performance of our method on the premise of satisfying the real-time demand.

Recently, flexible robotic systems are developed to enhance minimally invasive interventions on internal organs located in confined areas of human body. These surgical devices are designed to navigate anatomical pathways via single-port access, such as natural orifices or minimal incisions and intraluminal interventions. With improved precision, spatial flexibility and dexterity, the robotic technology can enhance surgery such that minimally invasive flexible access would become a faster, safer, and more convenient method for intra-body interventions without multiple or wide incisions. However, a lot of works are still required for global acceptance of existing flexible robotic surgical platforms. This review provides extended insights on the design details of two types of flexible robotic systems used for endoscopic and endovascular procedures. As of today, several prototypes of both platforms have been proposed; however, their global acceptability and applicability remains very low. To address these, we present an extensive review on design constraints and control methods which are vital for safer, faster, and better operation of the flexible robotic systems in minimally invasive surgery (MIS). Finally, research trends of flexible robotic systems and their clinical application status in MIS are discussed along with some of the technical and technological challenges hindering their prominence.

Three-dimensional (3D) models of tooth-alveolar bone complex are needed in treatment planning and simulation for computer-aided orthodontics. Tooth and alveolar bone segmentation from computed tomography (CT) images is a fundamental step in reconstructing their models. Due to less application of alveolar bone in conventional orthodontic treatment which may cause undesired side effects, the previous studies mainly focused on tooth segmentation and reconstruction, and did not consider the alveolar bone. In this study, we proposed a method to implement both tooth and alveolar bone segmentation from dental CT images for reconstructing their 3D models. First, the proposed method extracted the connected region of tooth and alveolar bone from CT images using a global convex level set model. Then, individual tooth and alveolar bone are separated from the connected region based on Radon transform and a local level set model. The experimental results showed that the proposed method could successfully complete both the tooth and alveolar bone segmentation from CT images, and outperformed the state of the art tooth segmentation methods in terms of accuracy. This suggests that the proposed method can be used in reconstructing the 3D models of tooth-alveolar bone complex for precise treatment.

Customized archwires are demanded in the lingual orthodontic treatment for patients suffering from malocclusion. Traditionally, these archwires could only be bent by experienced orthodontists manually. This pattern requires a specialized skills training and occupies long charside time, but still cannot ensure the accuracy of appliances. Therefore, a novel robotic system was developed for automatic and accurate preparation in our study. First, the implementation of hardware system was designed. Second, a modular and ROS-integrated control system was developed to control automatic bending. Third, an adaptive sampling-based bending planner with collision checker in a time-varying environment was established and realized in control system architecture. Preliminary validation of the developed robot system and its control system have been conducted both in simulation and physical robotic system. Experimental results have shown that the developed robotic system with its ROS-integrated control system was able to accomplish automatic and accurate orthodontic archwire preparation.

Tooth segmentation from dental computed tomography (CT) images with metal artifacts is challenging as metal artifacts make some of the crown boundaries unrecognizable. This letter proposes a semiautomatic method for crown segmentation from CT images with metal artifacts. A user manually selects a starting slice and initializes this slice. Then crown contours are segmented automatically from volumetric CT images slice by slice. In the segmentation of each slice, the Radon transform is used to extract a line to separate neighboring crowns into independent ones. A statistical shape prior-based level set model is then applied to segment each crown from the mesial or distal side of the line. The proposed method was tested on 15 set of volumetric images. Experimental results validated that it is effective to extract crown contours from CT images with metal artifacts.

In clinics, customized archwires are demanded for lingual orthodontic treatment. However, only very experienced orthodontists can handle the manual appliance preparation. This pattern not only occupies lots of the orthodontist's labor time, but also can not ensure the accuracy of the appliances. Therefore, a robotic system was developed for automatic and accurate orthodontic archwire bending in our study. First, a method for customized archwire parameterization was developed. Second, an adaptive sampling-based bending planner with collision checker in time-varying environment was designed. Finally, a bending control strategy was used to eliminate the springback effect of the archwires and bending point shift during the bending process. A self-developed simulation platform based on Robot Operating System with MoveIt was used for preliminary validation of the proposed method. Physical experiments for multi-functional orthodontic bends on the robotic system were conducted as well. The results have shown that the developed robotic system using the proposed planning and control method was able to accomplish the automatic and accurate orthodontic archwire bending.