Dynamic optimization focuses on solving problems where the search space changes over time. The multi-population method is the most widely used approach for addressing such problems. Traditional multi-population methods often lack a deep understanding of the problem’s structural characteristics, such as the boundaries of basins of attraction (BoAs), which leads to redundant searches in less promising regions. Without guidance from these structural features, most populations are regenerated randomly, resulting in inefficient exploration. Furthermore, the search range for each population remains fixed and does not adapt to the BoAs, leading to the loss of tracking for certain peaks. To address these challenges, this paper proposes a solution space partitioning based multi-population method. The algorithm partitions the solution space into subspaces and leverages historical population data to assign an uncertainty property to each subspace. It further learns the problem’s BoAs to guide populations in exploiting within the BoAs while exploring outside them. A dual-layer exclusion mechanism dynamically adjusts the search and exclusion ranges based on the BoAs, ensuring precise control, preventing overlaps, and preserving diversity. Experimental results demonstrate that the proposed algorithm significantly outperforms state-of-the-art algorithms on moving peaks benchmark, generalized moving peaks benchmark, and a real-world problem: marine magnetic compensation problem.

Computer-aided surgical simulation is a critical component of orthognathic surgical planning, where accurately simulating face-bone shape transformations is significant. The traditional biomechanical simulation methods are limited by their computational time consumption levels, labor-intensive data processing strategies and low accuracy. Recently, deep learning-based simulation methods have been proposed to view this problem as a point-to-point transformation between skeletal and facial point clouds. However, these approaches cannot process large-scale points, have limited receptive fields that lead to noisy points, and employ complex preprocessing and postprocessing operations based on registration. These shortcomings limit the performance and widespread applicability of such methods. Therefore, we propose a Transformer-based coarse-to-fine point movement network (TCFNet) to learn unique, complicated correspondences at the patch and point levels for dense face-bone point cloud transformations. This end-to-end framework adopts a Transformer-based network and a local information aggregation network (LIA-Net) in the first and second stages, respectively, which reinforce each other to generate precise point movement paths. LIA-Net can effectively compensate for the neighborhood precision loss of the Transformer-based network by modeling local geometric structures (edges, orientations and relative position features). The previous global features are employed to guide the local displacement using a gated recurrent unit. Inspired by deformable medical image registration, we propose an auxiliary loss that can utilize expert knowledge for reconstructing critical organs. Our framework is an unsupervised algorithm, and this loss is optional. Compared with the existing state-of-the-art (SOTA) methods on gathered datasets, TCFNet achieves outstanding evaluation metrics and visualization results. The code is available at https://github.com/Runshi-Zhang/TCFNet.

Dual-arm robots (e.g., humanoid robots) possess substantial potential for executing collaborative tasks in universal scenarios. However, the workspaces of the individual arms frequently overlap, rendering self-collision avoidance crucial for maintaining safe robotic operations. Existing motion planning-based methods demonstrate inadequate real-time efficacy, and learning-based distance proxy methods are subject to elevated false positive rates. To address these challenges, we present a novel minimum distance prediction neural network for reactive collision avoidance of dual-arm robots, which considers the continuous motion of the robotic arms and the interrelation of joint configurations. The temporal joint configurations are encoded and divided into historical and current features. A state-space model is utilized to capture the temporal dependency of historical features. A self-attention mechanism is employed to model the hidden relationships among current features. The integration of historical and current features via a cross-attention mechanism followed by a gated fusion module allows for precise prediction of the minimum distance between the dual arms. Simulations and real-world experiments, including human-robot interaction and autonomous grasping tasks, were conducted using two redundant robotic arms. The proposed method achieves an average error of 1.804 cm in minimum distance prediction. In dual-arm autonomous grasping experiments, an average error of 1.471 cm is attained. Our approach has improved accuracy by 38.08% over the state-of-the-art methods. No collisions occurred throughout all real-world experiments. This method holds promise for extensive applications of dual-arm robots. The code is accessible at https://github.com/XuejinLuo/SelfCollision.

Oral and maxillofacial surgery (OMS) imposes an increasing workload on even the most experienced surgeons due to long operation time, high skill requirements, limited observation field, constrained workspace, and fast-growing patient population. Robot-assisted OMS is particularly challenging, requiring technological advancements to replicate complex surgical workflows executed by human surgeons and novel working concepts to properly address human-machine relationships. We introduced a Surgeon Supervised Autonomous Surgical System (SSASS) aiming to solve emerging bottlenecks in OMS. SSASS custom develops a deep-learning-assisted virtual planning module, a teeth-based monocular camera navigation module, and a six-degree-of-freedom compact robot module to function as surgeons’ auxiliary brain, eye, and hand, respectively. These three modules are further seamlessly integrated to autonomously complete most labor-intensive procedures, while prioritizing surgeons to supervise and be responsible for the overall procedure. Le Fort I experiments on five human head models demonstrated that the surgical results of SSASS closely matched the preoperative plan, with high drilling accuracy and acceptable cutting accuracy under a fundamentally new and significantly simplified surgical workflow. Compared to its existing OMS counterparts, SSASS integrates the latest technologies such as deep learning, medical 3D printing, markerless navigation, virtual reality, and collaborative robotics, providing a comprehensive surgical solution for encompassing the entire OMS loop.

Complicated image registration is a key issue in medical image analysis, and deep learning-based methods have achieved better results than traditional methods. The methods include ConvNet-based and Transformer-based methods. Although ConvNets can effectively utilize local information to reduce redundancy via small neighborhood convolution, the limited receptive field results in the inability to capture global dependencies. Transformers can establish long-distance dependencies via a self-attention mechanism; however, the intense calculation of the relationships among all tokens leads to high redundancy. We propose a novel unsupervised image registration method named the unified Transformer and superresolution (UTSRMorph) network, which can enhance feature representation learning in the encoder and generate detailed displacement fields in the decoder to overcome these problems. We first propose a fusion attention block to integrate the advantages of ConvNets and Transformers, which inserts a ConvNet-based channel attention module into a multihead self-attention module. The overlapping attention block, a novel cross-attention method, uses overlapping windows to obtain abundant correlations with match information of a pair of images. Then, the blocks are flexibly stacked into a new powerful encoder. The decoder generation process of a high-resolution deformation displacement field from low-resolution features is considered as a superresolution process. Specifically, the superresolution module was employed to replace interpolation upsampling, which can overcome feature degradation. UTSRMorph was compared to state-of-the-art registration methods in the 3D brain MR (OASIS, IXI) and MR-CT datasets (abdomen, craniomaxillofacial). The qualitative and quantitative results indicate that UTSRMorph achieves relatively better performance. The code and datasets are publicly available at https://github.com/Runshi-Zhang/UTSRMorph.

Osteotomy holds a pivotal position among the fundamental procedures in craniomaxillofacial (CMF) surgery. However, there are inherent challenges and risks associated with ensuring the recuperation of occlusion, safeguarding the facial nerves and blood vessels, as well as preserving facial aesthetics. In this study, a non-invasive image-to-patient registration method for navigation/robotic CMF surgery based on intraoperative freehand ultrasound (US) 3D reconstruction is proposed. Building upon this, a CMF osteotomy robotic system with compliant human-robot interaction and osteotomy trajectory planning was devised. In the freehand US 3D reconstruction and registration experiments, the registration errors for human volunteers and phantoms were consistently less than 1 mm. In robot osteotomy experiments based on the resulting registration, the average osteotomy error was below 1.5 mm. The proposed US 3D reconstruction based registration method is non-invasive and radiation-free, and shows the promising accuracy which is suitable for CMF robotic or navigation systems.

Objective:We propose a general control framework for task space compliant motion and six-dimensional (6-D) force regulation towards automated robotic ultrasound (US) imaging. The framework endows a position-controlled robotic manipulator with the capability of accurate compliant motion in free space and accurate force control in motion-constrained environment.Methods:An intuitive six degree-of-freedom (6-DoF) admittance control model expressed in an arbitrary Cartesian body frame is mathematically derived with closed-form task space error mapping. Its practical implementation on widely-used collaborative manipulators is proposed to achieve full task space compliant behaviors and accurate 6-D force control. A hybrid control law is presented to achieve good motion accuracy in free space and improved coupled stability in motion-constrained environment. The coupled model of physical human-robot interaction is established and the reason for the improved coupled stability is analyzed through simulation.Results:Evaluation experiments on the proposed control framework were performed to show the effectiveness. The mean error of compliant trajectory following was less than 0.30 mm in free space. The mean relative force and moment control accuracy in three orthogonal directions was better than 0.5% and 0.8%, respectively. The improved coupled stability under the same model parameters was also confirmed by human-robot interaction experiments. Finally, an automated robotic US imaging experiment on a human volunteer in a real clinical scenario was carried out to show the potential application of our proposed framework.Conclusion:Experimental results have shown the advantages of the control framework, including satisfied force control accuracy, high accuracy of compliant motion, improved coupled stability, and system effectiveness on a human volunteer.Note to Practitioners—This paper was motivated by the increasing needs of automated ultrasound (US) scanning for both diagnostic and interventional purpose. Clinical sonographers suffer from repeated workload when performing diagnostic US imaging, which could benefit from automated robotic scanning. Robotic US imaging involves physical interaction between the robot end-effector (i.e., US transducer) and the human body. The dynamics of the interaction is regulated by the control law to guarantee the contact of the US transducer and the safety of the procedure. Most existing works have focused on regulating in-plane contact force in terms of the position without considering the compliance in other dimensions. However, it is not a trivial work to extend the positional compliance to six degree-of-freedom (6-DoF) compliance. As the prevalence of low-cost collaborative robotic arms in medical scenarios, how to perform 6-DoF compliant trajectory following and accurate six-dimensional (6-D) force control on these robotic arms becomes increasingly important. This paper gives a complete general solution to achieve 6-DoF compliant control and 6-D force regulation with accurate kinematics on a position-controlled robotic arm. A hybrid control law is proposed to switch the government of “instantaneous model” and “theoretical model” to achieve compliant motion accuracy in free space and improved coupled stability in motion-constrained environment. No expensive torque sensors and torque control interface are required. And no prior geometric knowledge about the scanning object is needed. We have demonstrated the application for robotic US imaging in a real clinical scenario.

Craniomaxillofacial (CMF) surgery always relies on accurate preoperative planning to assist surgeons, and automatically generating bone structures and digitizing landmarks for CMF preoperative planning is crucial. Since the soft and hard tissues of the CMF regions possess complicated attachment, segmenting the CMF bones and detecting the CMF landmarks are challenging problems. In this study, we proposed a semantic segmentation network to segment the maxilla, mandible, zygoma, zygomatic arch, and frontal bones. Then, we obtained the minimum bounding box around the CMF bones. After cropping, we used the top-down heatmap landmark detection network, similar to the segmentation module, to identify 18 CMF landmarks from the cropping patch. In addition, an unbiased heatmap encoding method was proposed to generate actual landmark coordinates in the heatmap. To overcome quantization effects in the heatmap-based landmark detection networks, the distribution-prior coordinate representation of medical landmarks (DCRML) was proposed to utilize the prior distribution of the encoding heatmap, approximating the accurate landmark coordinates in heatmap decoding by Taylor's theorem. The encoding and decoding method can easily contribute to other existing landmark detection frameworks based on heatmaps; consequently, these approaches can readily benefit without changing model structure. We used prior segmentation knowledge to enhance the semantic information around the landmarks, increasing landmark detection accuracy. The proposed framework was evaluated by 100 healthy persons and 86 patients from multicenter cooperation. The mean Dice score of our proposed segmentation network achieved over 88 %; in particular, the mandible accuracy was approximately 95%. The mean error of landmarks was 1.84 ±1.32 mm.

Abstract Temporal heterogeneous graphs can model lots of complex systems in the real world, such as social networks and e-commerce applications, which are naturally time-varying and heterogeneous. As most existing graph representation learning methods cannot efficiently handle both of these characteristics, we propose a Transformer-like representation learning model, named THAN, to learn low-dimensional node embeddings preserving the topological structure features, heterogeneous semantics, and dynamic patterns of temporal heterogeneous graphs, simultaneously. Specifically, THAN first samples heterogeneous neighbors with temporal constraints and projects node features into the same vector space, then encodes time information and aggregates the neighborhood influence in different weights via type-aware self-attention. To capture long-term dependencies and evolutionary patterns, we design an optional memory module for storing and evolving dynamic node representations. Experiments on three real-world datasets demonstrate that THAN outperforms the state-of-the-arts in terms of effectiveness with respect to the temporal link prediction task.

Although evolutionary computing (EC) methods are stochastic optimization methods, it is usually difficult to find the global optimum by restarting the methods when the population converges to a local optimum. A major reason is that many optimization problems have basins of attraction (BoAs) that differ widely in shape and size, and the population always prefers to converge toward BoAs that are easy to search. Although heuristic restart based on tabu search is a theoretically feasible idea to solve this problem, existing EC methods with heuristic restart are difficult to avoid repetitive search results while maintaining search efficiency. This article tries to overcome the dilemma by online learning the BoAs and proposes a search mode called history-guided hill exploration (HGHE). In the search mode, evaluated solutions are used to help separate the search space into hill regions which correspond to the BoAs, and a classical EC method is used to locate the optimum in each hill region. An instance algorithm for continuous optimization named HGHE differential evolution (HGHE-DE) is proposed to verify the effectiveness of HGHE. Experimental results prove that HGHE-DE can continuously discover unidentified BoAs and locate optima in identified BoAs.

In complex optimization problems, learning where to search is a difficult but critical decision for all search algorithms. Evolutionary computation methods also encounter a dilemma about where to explore or exploit. In this paper, a Monte-Carlo tree is constructed to guide evolutionary algorithms to search multiple promising regions simultaneously. In the Monte-Carlo tree model, a root node that contains all historical solutions represents the whole solution space. In each node of the tree, with k-means clustering method to partition solutions into different groups, group labels of the solutions are used to train support vector regression, which can learn a boundary to partition a region into different sub-regions. According to state values of nodes, reproduction operators of evolutionary algorithms are strengthened by selecting solutions in the most promising regions. From experimental results on multimodal problems, the proposed algorithm shows a competitive performance, which also indicates a great potential for applications to other kinds of optimization problems.

In evolutionary algorithms, how to effectively select interactive solutions for generating offspring is a challenging problem. Though many operators are proposed, most of them select interactive solutions (parents) randomly, having no specificity for the features of landscapes in various problems. To address this issue, this paper proposes a reinforcement-learning-based evolutionary algorithm to select solutions within the approximated basin of attraction. In the algorithm, the solution space is partitioned by the k-dimensional tree, and features of subspaces are approximated with respect to two aspects: objective values and uncertainties. Accordingly, two reinforcement learning (RL) systems are constructed to determine where to search: the objective-based RL exploits basins of attraction (clustered subspaces) and the uncertainty-based RL explores subspaces that have been searched comparatively less. Experiments are conducted on widely used benchmark functions, demonstrating that the algorithm outperforms three other popular multimodal optimization algorithms.

For computationally expensive constrained optimization problems, one crucial issue is that the existing expected improvement (EI) criteria are no longer applicable when a feasible point is not initially provided. To address this challenge, this paper uses the expected improvement of constraint violation to reach feasible region. A new constrained expected improvement criterion is proposed to select sample solutions for the update of Gaussian process (GP) surrogate models. The validity of the proposed constrained expected improvement criterion is proved theoretically. It is also verified by experimental studies and results show that it performs better than or competitive to compared criteria.