Matrix placement machines improve production efficiency of printed circuit board assembly (PCBA), addressing critical needs for flexible and intelligent electronics manufacturing. However, their complex head structure renders solutions for traditional beam-head placement machines inefficient for matrix placement machines. This article proposes a phased hybrid algorithm with adaptive hyper-nondominated sorting genetic algorithm II (NSGA-II) for PCBA optimization. A bidirectional search mechanism is applied to derive feeder distributions and nozzle configurations, and iteratively tighten the solution space using priority-based search strategies. The softmax, max greatest common divisor, and max matching mechanisms are proposed for placement and pickup sequences, which facilitates construction of solution pools. Initial solutions are extracted from the pool and, subsequently, hyperheuristic mechanisms dynamically adjust genetic operators within NSGA-II to minimize placement, pickup, and recognition times with better convergence speed. Experimental validation with real-world production data demonstrates that the proposed algorithm achieves 6.05%–38.18% performance improvements compared to state-of-the-art solutions.

Operational efficiency of placement machines constrains the overall production capacity of printed circuit board (PCB) assembly lines. Existing state-of-the-art algorithms face challenges, such as conflicts between multiple objectives and coupling within different problems. This article proposes a multiobjective hybrid evolutionary multitasking algorithm (MOHEMTA) to address PCB assembly optimization in beam-head placement machines. The algorithm divides the problem into pickup and placement tasks, leveraging implicit parallelism to enhance solution efficiency. A nozzle block encoding method and heuristic decoding strategies with domain knowledge are introduced to reduce encoding complexity and accelerate algorithm convergence. MOHEMTA enhances offspring population diversity and quality through an elitist strategy, evolutionary operators, and knowledge transfer mechanisms, while incorporating safeguards against negative transfer. Experiments demonstrate that the multiobjective solution performance and practical results of MOHEMTA are better than those of other state-of-the-art algorithms.

In surface mount technology, solder paste printing quality critically affects product reliability. Accurate segmentation of solder paste regions is essential for defect detection, quantitative analysis, and process optimization. Traditional threshold-based methods lack robustness under varying surface textures and lighting, while deep learning approaches require large annotated datasets and expensive hardware, limiting their use in cost-sensitive manufacturing. We propose a fast, annotation-free segmentation framework based on parameteric multimodal learning, integrating RGB color with 3-D height data. Height priors generate an initial mask, followed by a lookup table–based parameteric color model that adapts to different printed circuit board types and batches. A convolutional feature fusion operator then constructs a joint height–color probability space, suppressing interference from substrate variations and uneven illumination, yielding a refined probability map for final segmentation. Tests on a 3D-solder paste inspection industrial dataset achieve 96.0% mean intersection over union and 98.8% pixel accuracy, matching state-of-the-art deep learning performance while greatly improving efficiency and suitability for real-world deployment without annotated data.

In printed circuit board assembly (PCBA), placement route schedule (PRS) significantly affects assembly efficiency of the beam head placement machine. The PRS is typically solved by decomposing it into placement point assignment problem (PPAP) and beam heads sequencing problem (BHSP). This article first proposes a deep reinforcement learning framework to tackle PPAP, which is a key determinant of overall process quality. Then, to mitigate the impact of placement position and angle on assembly efficiency, a dynamic programming-based beam head sequencing algorithm is introduced to solve BHSP. Since component types and placement point assignment states vary across different pick-and-place cycles, a dynamic combinatorial mask encoding method is proposed to effectively extract feature information between placement points. Inspired by the beam head placement process, a decoder that combines gated recurrent units and an attention mechanism is finally introduced, which fully utilizes historical node information to predict the next node. Experimental results demonstrate that the proposed method reduces PCBA routing distance by an average of 4.62%, outperforming other State-of-the-Art approaches.

Time-optimal path generation is critical for maximizing throughput in high-speed PCB assembly, yet existing approaches predominantly focus on geometric distance minimization, overlooking the fundamental impact of acceleration dynamics and multi-axis coordination on temporal efficiency. This study addresses this gap by introducing a physics-based time estimator that explicitly models trapezoidal acceleration profiles for synchronized X/Y/Z/R-axis motions, enabling precise performance evaluation under realistic kinematic constraints. Experimental validation on production PCBs demonstrates that the proposed estimator achieves much higher estimation accuracy, outperforming conventional methods. When integrated as the objective function in multi-chromosome genetic algorithm optimization, time-optimal solutions effectively reduce actual movement times compared to distance-optimal baselines, despite requiring longer travel paths. These findings confirm the time-optimal estimator’s superiority over pure distance minimization, proving that peak efficiency is achieved by balancing travel distance with movement speed.

In surface mount technology (SMT) assembly, the increasing complexity and miniaturization of electronic components pose critical challenges to balancing placement precision and production efficiency. This paper presents a quality-efficiency driven scheduling and process co-optimization (SPCO) methodology for the pick-and-place (PAP) process in SMT production lines. Leveraging a cyber-physical system framework integrated with automated optical inspection, the proposed approach dynamically couples offline scheduling with online process capability feedback to achieve adaptive allocation of components. A precision-aware SPCO model is formulated to assign components to placement heads based on real-time process capability indices, ensuring compliance with stringent precision constraints. To enable real-time deployment, a precision-prioritized allocation heuristic (PPAH) is introduced, supporting component-head assignments under heterogeneous head capabilities. Experiments on industrial datasets demonstrate that PPAH completely eliminates precision violations while improving the overall process capability margin by 2.6-fold compared to state-of-the-art benchmarks, with only a moderate increase in total PAP time. These results validate the effectiveness of the proposed co-optimization strategy in improving first-pass yield and robustness in high-mix SMT environments.

Organoids are cell clusters cultured in vitro that maintain the structure and function of the donor organs. They have found important applications in biomedicine, such as drug screening and personalized therapy. However, conventional organoid culture methods lack control of physical properties like size and distribution, leading to increased heterogeneity and very low batch-to-batch reproducibility, which significantly limits their widespread use. Controlling these properties at the microscale is challenging, particularly for fragile fragments, which are the main source for culturing organoids. To address this issue, we present a robotic micromanipulation system that allows operators to select fragments of particular sizes and automatically transfer them into a customized in-situ organoid chip (IOC) for culture. The chip was designed with microwell arrays to uniform the culture environment and facilitate imaging analysis. The transfer of fragments is modeled based on computational fluid dynamics (CFD) and is enabled by designing a robust model predictive control (RMPC) framework. Simulation and experiment results demonstrated the effectiveness of the model and controller. In colorectal cancer organoid culture experiments, our system significantly improved the morphological homogeneity of organoids. Note to Practitioners—Organoids have been demonstrated to be one of the most promising in vitro models. Lacking control of its size and distribution results in significant heterogeneity and low batch-to-batch reproducibility, which limits its wide uses. Here, we report a robotic micromanipulation system that allows operators to select fragments of particular sizes and morphologies and automatically transfer them into a customized organoid chip for culture. The results of colorectal cancer organoids culture experiments verified the effectiveness of our system in reducing the morphological heterogeneity among organoids.

When a manipulator moves in a highly dynamic scenario with a large range of rapid motion, the coupling disturbances between the manipulator and the UAV in the aerial manipulator system (AMS) become very strong, which directly affects the ability of the AMS to perform aerial manipulation and even poses a threat to the safety of the system. The aim of this article is to address the strong coupling disturbance problem in the AMS through precise coupling disturbance modeling and compensation. First, considering the rapid changes in the center of mass (CoM) and the moment of inertia (MoI) of the system under a highly dynamic scenario, this article delves into the generation mechanism of the coupling disturbances and models them based on the variable inertia parameters. The proposed precise coupling disturbance model (CDM) makes good use of the state information of the system, which enables one to achieve accurate estimation of the coupling disturbances without the aid of external force and torque sensors. With the proposed model, the strong coupling disturbances in the AMS are compensated in a feedforward way during the controller design process. An indoor AMS experimental platform is developed for validation purposes. The experiments and simulation are conducted in a highly dynamic scenario, involving rapid movements of the manipulator across a large range. The experimental and simulation results demonstrate the effectiveness and advantages of the proposed method for suppressing the strong coupling disturbances.

In the field of smart surface mount technology (SMT) production, integrating machines through a cyber-physical system (CPS) architecture holds significant potential for improving assembly quality and efficiency. However, fully unifying inspection and production systems to effectively address assembly-related quality issues remains a challenge. This study seeks to close these gaps by introducing collaborative optimization methods to ensure seamless operations. The research is driven by the need for precise control of key assembly parameters, such as placement height, x-offset, y-offset, rotation angle deviations, and blowing durations, all of which are major contributors to defects. To address these challenges, we propose a self-adaptive collaborative optimization (SACO) framework that prioritizes enhancements based on their impact on both quality and efficiency. The SACO framework combines customized Bayesian optimization and particle swarm optimization techniques, allowing for dynamic adjustments to process parameters, guided by real-time data from automatic optical inspection (AOI) systems. The primary goal of this study is to reduce defects and improve efficiency in the SMT assembly process through these targeted improvements. Experimental results validate the effectiveness of the proposed methods, demonstrating significant advancements in placement accuracy and overall assembly efficiency. Our findings confirm that the SACO framework provides a robust solution to persistent challenges in SMT production, addressing critical gaps in quality control and process optimization.

Printed circuit board assembly line scheduling (PCBALS) is a difficult task in the electronic industry for assembly lines using surface mounters, which is critical for production efficiency. This is a special type of line optimization problem that uses different allocation techniques, resulting in wide differences in assembly times between machines. This article proposes a hyper-heuristic optimizer embedded with a multifeature fusion ensemble estimator (HHO-MFEE) for PCBALS using linear-aligned-heads surface mounters. The objective and constraints of the problem are discussed, and a min-max integer model for small-scale problems is built. At the hyper-heuristic low level, seven data- and target-driven heuristics are presented for allocating components to different machines. Strategies for duplicated conditions with component types and placement points allocation are proposed to improve the applicability of the algorithm and the quality of the solution. An ensemble assembly time estimator that incorporates the coding of multifeatures, including estimated subobjectives, is proposed for evaluating the quality of the solution. Experimental results show that: 1) the gaps between the solution from HHO-MFEE and the optimal solution of the model are 3.44%~7.28% for small-scale data; 2) the proposed time estimator has higher accuracy than regression and heuristic-based ones, with mean absolute error of 2.01% and 3.43% for training and testing data, respectively; and 3) HHO-MFEE is better than other state-of-the-art algorithms, with average improvement of 7.21%~9.47%.

This brief presents an adaptive neural zeta-backstepping control strategy for a class of uncertain nonlinear systems, which allows these systems to be practically stabilized with predefined damping ratios. By introducing the zeta-backstepping technique, system damping ratios can be predetermined based on specific parameter selection rules. To reduce the impact of unknown nonlinearities, neural networks (NNs) with gradient descent training are applied to compensate such nonlinearities online. A new filter, called dynamic command filter, is used to construct the gradient of the NNs. By resorting to second-order Lyapunov stability criteria, it is proved that the closed-loop system is practically stable and has predefined damping ratio. Finally, experiments on a perturbed direct current (DC) motor system demonstrate the advantages of the proposed method.

Route scheduling for automatic optical inspection (AOI) of printed circuit boards (PCBs) impacts the productivity of surface mount production lines. Current state-of-the-art mathematical models in the area are not rigorous enough and neglect significant practical constraints, such as component geometric constraints. This article proposes a hierarchical mixed integer programming model to describe the route scheduling problem for AOI of PCBs. The model allows theoretical optimal solutions to be obtained for small-scale problems. In addition, a two-stage heuristic framework, consisting of clustering and path planning stages, is proposed to improve efficiency in solving large-scale problems, achieving near-optimal solutions. Taking into account that component distribution affects clustering results, the clustering stage is developed with a hierarchical heuristic algorithm based on block density with an aggregation strategy. The Lin–Kernighan algorithm is first used to quickly generate the scheduling sequence in the path planning stage. Image acquisition centers are initially adjusted with a customized heuristic. After that, a hybrid evolutionary multitask algorithm is proposed to further reduce path distance by dividing the image acquisition center adjustment task into several subtasks using heuristic rules. The algorithm obtains better quality results and is faster than traditional evolutionary algorithms. Experiments on an actual industrial AOI platform demonstrate that the proposed two-stage heuristic route scheduling algorithm outperforms state-of-the-art research in the area.

Structured light imaging is a typical technique for industrial 3-D microscopic measurement. Extensive research on structured light codecs has been conducted to accurately correlate camera and projector pixels. However, these methods suffer significant degradation when measuring low-reflectivity and complex surfaces. This article introduces a deep correlation-based cascade structured light network (CasSLNet) that utilizes deep phase and column features to calculate correspondences at the subpixel scale. To mitigate the huge computational cost of full correlation, a coarse-to-fine approach is proposed. Specifically, multiscale features from the camera observation sequence and the 1-D encoding pattern are extracted through a pseudosiamese network, and cascade cost volumes are constructed. An initial column map is then regressed from the low-resolution column cost volume. Based on this, an iterative update operator is introduced to refine initial estimates, resulting in a full-resolution column map. Furthermore, a structured light dataset has been collected and experiments have been conducted on a typical structured light imaging platform. Experimental results demonstrate that CasSLNet outperforms both traditional and state-of-the-art deep learning-based methods.

This article proposes a novel safety-critical control (SCC) framework based on embedded control barrier functions (EMB-CBF-SCC) for high-order strict-feedback nonlinear affine control systems. It is aimed at reconciling the potential conflict between predesigned desired trajectory and multiple safety constraints that could have different high relative degrees. Compared with existing CBF-based SCCs, our method can significantly reduce differential order and computational burden. Specifically, we first propose a novel concept of embedded control barrier function (EMB-CBF), which can reduce an arbitrary high-relative-degree safety constraint to relative degree one, and ensure safety of high-relative-degree systems. Further, EMB-CBF-SCC divides the original system into a top-level and a bottom-level subsystem. Then, it embeds between the two subsystems a quadratic program based on EMB-CBF, and introduces command filters to smooth virtual control inputs and obtain differential signals. Coordination performance of safety and stability is analyzed, considering the impact of filter errors. Finally, we present two real safety-critical robotic application scenarios with different safety constraint settings, namely, multiple state constraints for a single-link manipulator numerical model and dynamic obstacle avoidance constraints for a self-developed micro mobile robot experimental platform, respectively. The effectiveness of the proposed framework is demonstrated in both scenarios.

This article introduces a novel composite aerial vehicle configuration called hybrid quadrotor with all-moving wings (HQWAW), consisting of a conventional quadrotor combined with two independently all-moving wings. A nonlinear geometric controller in the special Euclidean group SE(3) is proposed as the basic controller for the HQWAW, achieving high maneuverability and energy-efficient flight. Lyapunov stability criterion is used to prove that the proposed control scheme can track the reference trajectory almost globally ultimately uniformly bounded. A deep reinforcement learning compensator, based on the twin delayed deep deterministic policy gradient algorithm, is designed to fine-tune all-moving wing angles, ensuring that wing surfaces remain at optimal angles, thereby maximizing aerodynamic efficiency and reducing rotor consumption. Tracking results for a trajectory involving high-speed dive followed by spiral ascent demonstrate that the proposed algorithm achieves both high maneuverability and improved energy efficiency of the HQWAW.

In mounting machines, vision measurement of surface mount devices (SMDs) is a crucial task, widely used for size calculation and defect detection. However, current vision measurement methods focus on specific SMDs, such as quad flat packages. There is no unified method for measuring all kinds of rectangular-pin components, which account for more than half of the SMDs processed in mounting machines. This article presents an automatic and universal method for measuring with subpixel accuracy the parameters of rectangular-pin SMDs, which can be applied to different types of SMDs. First, an edge gradient model is proposed in the form of asymmetric Gaussian function. After that, line segments of SMDs are extracted and a line adjacency matrix is obtained that describes the relationship between lines. Then, segments are combined into long lines by searching through the line adjacency matrix. Finally, pixel line pairs are refined to subpixel level. Experimental results on the surface mount hardware platform for different components under different angles and light conditions demonstrate the high accuracy and strong robustness of the proposed method.

Dual-gantry surface mount optimization effectively improves the productivity of printed circuit board assembly (PCBA), but also brings new challenges. Optimizing workload allocation to balance the front and rear gantry placement completion time is a significant challenge for improving PCBA productivity. This study proposes a finite potential game heuristic algorithm (FPGHA) to solve the workload allocation problem. The algorithm generates game agents by analyzing the feeding characteristics of the dual-gantry placement machine and using an improved bisection K-means clustering method. Agent utility is calculated based on metrics affecting productivity of the pick-and-place process, including the number of simultaneous pickups, nozzle changes, cycles, and mounting points. Nash equilibrium of FPGHA is obtained by a best-response dynamics and heuristic algorithm. Then, the effectiveness of FPGHA in solving the workload allocation problem is first demonstrated in simulated experiments with different nozzle and feeder configurations. Finally, FPGHA is compared with the hierarchical restricted balance algorithm, adaptive clustering algorithm, and the popular industrial optimizer software in actual placement experiments using real-world industrial printed circuit boards. The effectiveness and accuracy of FPGHA are verified by analyzing the correlation between three variables: The FPGHA estimated value, the actual assembly value, and the PCB assembly time.

In this article, a practical finite-time command-filtered adaptive backstepping (PFTCFAB) control method is presented for a class of uncertain nonlinear systems with nonparametric unknown nonlinearities and external disturbances. Unlike PFTCFAB control techniques that use neural networks (NNs) or fuzzy-logic systems (FLSs) to deal with system uncertainties, the proposed method is capable of handling such uncertainties without the need for NNs or FLSs, thus reducing complexity and increasing reliability. In the proposed approach, novel function adaptive laws are designed to directly estimate unknown nonparametric nonlinearities and external disturbances by means of command filter techniques, and a type of practical finite-time command filters is proposed to obtain such laws. Moreover, the PFTCFAB controllers and finite-time command filters are designed with practical finite-time Lyapunov stability, which ensures finite-time stability of system tracking and filter estimation errors. Experimental results with a quadrotor hover system are presented and discussed to demonstrate the advantages and effectiveness of the proposed control strategy.

The optimization of printed circuit board assembly (PCBA) for a beam head placement machine is a multivariable and multiconstraint combinatorial problem. Current techniques falter in solving a variety of PCBA problems since heuristic algorithms lack theoretical guarantees of optimality, and mathematical modeling methods have high computational complexity for the whole problem. This article proposes a novel two-phase optimization for PCBA, integrating the advantages of mathematical modeling with heuristic algorithms. We divide the problem into the head task assignment and the placement route schedule. For the former, an effective integer linear programming model with component partition is proposed, encompassing key efficiency-influencing factors. A recursive heuristic-based initial solution speeds up the solving convergence, while the reduction strategies enhance model solvability. For the placement route schedule, a tailored greedy algorithm yields high-quality solutions, leveraging the results of the model, and an aggregated route relink heuristic does further optimization. In addition, we propose a selection criterion for the solution pool of the model to pre-evaluate the placement movement, which builds the connection between the two phases. Finally, we validate the performance of the two-phase optimization, which provides an average efficiency improvement of 8.66%–21.83% compared to other mainstream research.

Surface mount technology is essential to the development of the electronic manufacturing industry. This article studies optimizing the surface mount process for the beam-head placement machine. A mixed-integer programming (MIP) model is proposed for this problem, which is decomposed into three interconnected hierarchical parts: feeder allocation; component assignment; and pick-and-place (PAP) sequence problems. This article proposes an efficient hierarchical framework with three elaborately designed heuristics to solve the above problem. The design of the scan-based algorithms optimizes the subobjectives of feeder allocation and component assignment. First, the allocation heuristic arranges the feeders into slots as a prerequisite for other problems. Then, the component assignment heuristic determines the component type for each head with a variety of criteria and long short-term objectives. Finally, the PAP sequence problem is solved using a modified beam search algorithm. The proposed algorithm offers advantages in terms of effectiveness, efficiency, and extension, which can satisfy various customization demands. Experiments are conducted on our self-designed placement machine using industrial and randomly generated data. Computational experiments show that the scan-based heuristic algorithm obtains near-optimal solutions with a gap of 9.93% averagely compared with the proposed MIP model and provides efficiency improvement over the mainstream studies.

In this article, a velocity-free adaptive controller is proposed for the tracking control of servo mechanisms with friction compensation. A continuously differentiable friction model is employed to compensate for the dominant friction nonlinearity of servo mechanisms. Besides, a projection-type adaptive law is applied to handle parameter uncertainties in the system model. Since only the output position signal is directly measurable, an adaptive extended state observer (AESO) is constructed to estimate the indeterminate velocity state, which can also provide an estimation of unmodeled dynamics. Moreover, the dynamic gain switching of AESO can effectively suppress the peaking phenomenon at the motion beginning. Specifically, the parameter adaptation law utilizes the desired velocity state instead of the estimated value, avoiding the coupling problem between parameter and state estimation. The proposed control strategy theoretically demonstrates the transient performance and boundedness of the error in output tracking. Asymptotic stabilization of the system can also be implemented when only parameter uncertainty exists. Comparative experiments are conducted on a linear motor platform to demonstrate the effectiveness of the proposed control scheme.

This article presents a novel compound aerial vehicle configuration called the hybrid quadplane with all-moving wings (HQWAWs), and provides a six-degrees-of-freedom (6-DOF) flight dynamics modeling and nonlinear controller design for this new configuration. Compared to traditional quadrotors, the HQWAW add two wings on left and right sides of the quadrotor. Each wing is a single structure that can rotate to any angle of attack independently, which is referred to as all-moving wing (AW) in this article. The dynamic modeling of this configuration takes into account a complete description of flight dynamics, including wing aerodynamics and the dynamics of motors and propellers. A comprehensive nonlinear flight controller is proposed using model feedforward and state feedback for the HQWAW that supports the whole flight envelope. The proposed HQWAW configuration is compared with a traditional quadrotor with all parameters being the same except for the absence of the AWs. A set of numerical results demonstrate that the proposed configuration can obtain flight flexibility beyond quadrotors, while effectively reducing energy consumption.

With the continuous development of the SMT (surface mounting technology), equipment circuit boards are gradually moving towards the direction of miniaturization and precision. Surface mounting technology is playing a more critical role in manufacturing. The main task of the core mounting machine is to realize the detection and identification of different chips, to ensure that the specified chips can be mounted stably with high speed and precision. With the development of patch chips, there are more detection methods for component pins. If a specific detection method is proposed for each non-standard component, the component detection will become more complicated. To solve such problems, this paper proposes an algorithm for extracting pins of non-standard chips based on curve detection, which aims to help users to teach different types of pins of non-standard chips by curve detection and use the teaching results as template matching to detect all non-standard patch chips. By mounting experiments on non-standard component,the test speed and production stability of the proposed algorithm have reached the leading level.

With the increasing miniaturization of electronic devices, the demand for smaller and more precise components has led to the increasing popularity of surface mount technology (SMT) in the electronics manufacturing industry. This paper proposes an industrial chip positioning method in surface mount technology, which has good robustness to illumination changes. Firstly, the image processing algorithm is used to extract the edge of the chip, and the discrete Fourier transform method is used to obtain the frequency domain features of the image. Based on the frequency domain features, the angle of the chip is detected using the radial projection method. Then, the threshold image is obtained based on OTSU and the chip region is segmented using a projection method to achieve chip positioning. Experimental results conducted under different lighting conditions demonstrate that the proposed method exhibits high accuracy and robustness, making it effective for industrial applications.

Compared with traditional rigid objects' dynamic throwing and catching by the robot, the in-flight trajectory of nonrigid objects (incredibly variable centroid objects) throwing is more challenging to predict and track. This article proposes a variable centroid trajectory tracking network (VCTTN) with the fusion of vision and force information by introducing force data of throw processing to the vision neural network. The VCTTN-based model-free robot control system is developed to perform highly precise prediction and tracking with a part of the in-flight vision. The flight trajectories dataset of variable centroid objects generated by the robot arm is collected to train VCTTN. The experimental results show that trajectory prediction and tracking with the vision-force VCTTN is superior to the ones with the traditional vision perception and has an excellent tracking performance.

Vision-based location of industrial chips is crucial for high-speed and high-precision mounting in surface mount technology (SMT) applications. The conventional general location method requires numerous details of the chips’ physical characteristics, making it unsuitable for unknown chip locations and offline learning. In this article, we propose a general chip detection algorithm based on subpixel features from accelerated segment test (FAST) points that offers strong applicability, high precision, and high speed. The core part of our method is the extraction of subpixel boundary FAST points. The conventional subpixel calculation method uses a single-edge model that results in large deviations between calculated and actual positions when applied to actual FAST points. We propose a dual-edge subpixel model containing two groups of edges to reduce this error. Compared with the iterative calculation method in OpenCV, this method has a closing solution and faster performance. We determine model parameters using spatial moments, and present the relationship between the model and subpixel positions. Our experiments on the SMT hardware platform demonstrate that our method is robust to noise, illumination, position, and chip type, and is faster, more accurate, and more reliable than the Hanwha SM481-Plus placement machine and point registration method.

In this study, a data-augmentation method is proposed to narrow the significant difference between the distribution of training and test sets when small sample sizes are concerned. Two major obstacles exist in the process of defect detection on sanitary ceramics. The first results from the high cost of sample collection, namely, the difficulty in obtaining a large number of training images required by deep-learning algorithms, which limits the application of existing algorithms in sanitary-ceramic defect detection. Second, due to the limitation of production processes, the collected defect images are often marked, thereby resulting in great differences in distribution compared with the images of test sets, which further affects the performance of detect-detection algorithms. The lack of training data and the differences in distribution between training and test sets lead to the fact that existing deep learning-based algorithms cannot be used directly in the defect detection of sanitary ceramics. The method proposed in this study, which is based on a generative adversarial network and the Gaussian mixture model, can effectively increase the number of training samples and reduce distribution differences between training and test sets, and the features of the generated images can be controlled to a certain extent. By applying this method, the accuracy is improved from approximately 75% to nearly 90% in almost all experiments on different classification networks.

The capability to obtain detailed motility information of model organisms is fundamental to reveal their functional and social behavior characteristics. Zebrafish is a powerful vertebrate model organism. Despite recent success in the automatic quantification of adult zebrafish movement, it remains a laborious task for group zebrafish larval tracking due to their similar appearance, frequent occlusions, and highly discontinuous kinematics. This article presents DanioSense (DS), an automatic tracker for group larval zebrafish, to overcome these tracking challenges. The integration of a light convolutional neural network and a centerline extraction algorithm enables the tracker to localize individuals even in occlusion cases where objects’ identities are prone to switch. With reliable detections, an adaptive Kalman filter is designed to optimally estimate locomotive parameters, which is also used for object reidentification accomplished by a two-stage data association protocol. Experimental results demonstrated a tracking accuracy of over 97%, median errors of$102~{\mathrm{\mu m}}$, and 8.8° for the position and orientation measurement, and a processing speed of over 30 frames/s with a normal computer configuration. DS provides detailed quantitative data for a large-scale larvae group in nearly real time, highly boosting the efficiency of characterizing individual phenotypes and analyzing social interactions.Note to Practitioners—This article aimed to tackle the problem of automated tracking groups of zebrafish larvae, an ideal vertebrate model organism for large-scale chemical and genetic screens. The task of group tracking is to record each individual’s movement and calculate their position, velocity, direction, and other parameters for further analysis, where the correct identity of each individual must be maintained. Existing algorithms either switch larvae’ identities easily or are unable to achieve online tracking due to the limitations of their methods to address individuals’ intersections. DanioSense (DS) adopts a convolutional neural network to identify larval heads whenever they intersect and uses an adaptive Kalman filter to calculate the movement parameters optimally. Besides, a range of visualization options is designed to bring insight into underlying patterns through massive amounts of data. Theoretically, this algorithm’s approach to solving intersections and calculating movement statistics can also apply to other fish-like animals. Its visualization options are applicable to other tracking systems. The key advantage of Daniosense over existing trackers is the capability to track each larva within a group and output detailed quantitative data in nearly real time. The tracking performance of DS is based on the quality of image segmentation and the success rate of classifying samples. Many state-of-the-art image segmentation and classification neural networks can be adopted to extend this system’s applications to more complex environments but at a higher computation and time cost, which is a tradeoff between efficiency and capability. Some applications require a higher video sampling rate, so the system’s processing speed needs to be further improved with better hardware and software framework optimization. The next steps include improving the processing efficiency, providing more tracking modules and visualization options, and extending its application fields.

This article investigates the adaptive fuzzy control algorithm for a class of large-scale switched fractional-order nonlinear nonstrict feedback systems. In this algorithm, we utilize fuzzy-logic systems (FLSs) to approximate the complicated unknown nonlinear functions. Based on the fractional Lyapunov stability rules, a virtual control law is presented. A fuzzy adaptive decentralized control method is developed under the technique of the Lyapunov function. Under the operation of the proposed algorithm, the stability of the proposed systems and the control performance can be guaranteed. Finally, simulation results are presented to illustrate the feasibility and effectiveness of the proposed method.

This article proposes a hierarchical multiobjective heuristic (HMOH) to optimize printed-circuit board assembly (PCBA) in a single beam-head surface mounter. The beam-head surface mounter is the core facility in a high-mix and low-volume PCBA line. However, as a large-scale, complex, and multiobjective combinatorial optimization problem, the PCBA optimization of the beam-head surface mounter is still a challenge. This article provides a framework for optimizing all the interrelated objectives, which has not been achieved in the existing studies. A novel decomposition strategy is applied. This helps to closely model the real-world problem as the head task assignment problem (HTAP) and the pickup-and-place sequencing problem (PAPSP). These two models consider all the factors affecting the assembly time, including the number of pickup-and-place (PAP) cycles, nozzle changes, simultaneous pickups, and the PAP distances. Specifically, HTAP consists of the nozzle assignment and component allocation, while PAPSP comprises place allocation, feeder set assignment, and place sequencing problems. Adhering strictly to the lexicographic method, the HMOH solves these subproblems in a descending order of importance of their involved objectives. Exploiting the expert knowledge, each subproblem is solved by an elaborately designed heuristic. Finally, the proposed HMOH realizes the complete and optimal PCBA decision making in real time. Using industrial PCB datasets, the superiority of HMOH is elucidated through comparison with the built-in optimizer of the widely used Samsung SM482.

In this article, an adaptive event-triggered fault-tolerant asymptotic tracking control problem guaranteeing prescribed performance is addressed for a class of block-triangular multi-input and multioutput uncertain nonlinear systems with unknown nonlinearities, unknown control directions, and actuator faults. Through a systematic co-design of the adaptive control law and the event-triggered mechanism, including fixed and relative threshold strategies, a control scheme with low structure and calculation complexity is designed to conserve system communication and computation resources. In this design, the output asymptotic tracking is achieved. The Nussbaum gain technique is incorporated to overcome unknown control directions with a new adaptive law, and a type of barrier Lyapunov function is adopted to handle the prescribed performance control problem, which contributes to a novel control law with strong robustness. The robust controller can address the uncertainties and couplings derived from the system structure, actuator faults, and event-triggered rules, without using approximating structures or compensators. Besides, the explosion of complexity is avoided. It is proved that all signals of the closed-loop system remain bounded, and system tracking errors asymptotically approach 0 with the prescribed performance, while the Zeno behavior is prevented. Finally, the effectiveness of the proposed control scheme is evaluated via an application example of the half-car active suspension system.

In this article, an adaptive sensor fault accommodation scheme is proposed for uncertain vehicle active suspensions via output-feedback control where vehicle body displacement is the only measurable output signal corrupted by sensor bias. An adaptive observer with variable gains is constructed to obtain state estimates whose design procedure involves parameter adaption of the uncertain system parameters and sensor bias, and an output-feedback controller is designed to attenuate the vehicle body displacement based on the partial measurement information, estimates of the states, and unknown parameters. Compensation for measurement error is made both in the design process of the adaptive observer and output-feedback controller in order to weaken the influence brought about by sensor bias fault. In order to guarantee system stability, the variable observer gains are determined in real time using a switching strategy where their values can be modified in finite times by monitoring the state estimates generated by the observer itself. It is proved that the vehicle body displacement will converge to a small neighborhood around zero, and all the signals of the closed-loop system are ensured to be bounded through selecting suitable control parameters. Simulation is carried out to show the effectiveness of the proposed method and results indicate that better stabilization of the suspension vertical motion can be achieved through adaptive compensation for sensor bias.

Optimizing all the objectives of the printed circuit board assembly (PCBA) optimization in a multifunctional placer remains a formidable challenge till now. This article converts the original PCBA optimization problem to a newly defined component allocation problem, which decides the component-type handled by each head per pickup-and-place (PAP) cycle. The component allocation problem is a quadratic 3-D assignment problem (Q3AP) and effectively combines the optimization of all the main objectives. It is possible that one head stays idle, so the assigning 2-D locations are uncertain. We propose the cell division genetic algorithm (CDGA) to solve such a complex Q3AP. The CDGA allocates a component cell as the basic unit. Each of the first-generation component cells contains the mounting points of the same type. A cell chromosome decoding heuristic is designed to determine the next assigning head. By doing so, the problem dimension is reduced, so the conventional GA can be used for searching the optimal component allocation formed by the current-generation cells. When a better allocation can no longer be found by allocating the current cells, the cell division operation is performed to divide each cell into two new cells. The new cells are used in the next round of GA searching, which further optimizes the allocation from two perspectives: better balancing the minimization of nozzle changes and PAP cycles, more flexibly maximizing the simultaneous pickups with the uncertain locations. The CDGA works continuously until the current cells cannot bring any improvement. In simulations and experiments using the industrial samples, the proposed algorithm significantly reduces the PCBA time compared to two recent studies and the built-in optimizer of the widely used multifunctional placer, Hanwha SM482 PLUS, which demonstrates its effectiveness and superiority.

Predators in nature grip their prey in different ways, which give innovational ideas of gripping approaches in industrial applications. Octopus performs flexible gripping with the help of vacuum grippers, suction cups, which inspired a new type of microgripper for biological sample micromanipulation. The proposed gripper consists of a glass pipette and a pump driven by a step-motor. The step-motor is controlled with adaptive robust control to adjust the gripping pressure applied on the biological sample. A dynamic model is developed for the biological sample aiming for better deformation control performance. A visual detection algorithm is developed for data processing to identify the parameters in the dynamic model and the detection result of visual algorithm is also used as feedback of adaptive robust control, which diminishes the negative influence of parameter and model uncertainties. Zebrafish larva was used as the testing sample for experiment and the corresponding parameters were identified experimentally. The experimental results correlated well with the model predicted deformation curve and visual detection algorithm provided promising accuracy, which is less than [Formula: see text]. Adaptive robust control provides fast and accuracy response in point-to-point deformation testing, and the average responding time is less than 30 s and the average error is no larger than 1 pixel.

Multi-target tracking application for fast-moving workpieces has drawn increasing attention in the industrial field. For the dense, fast moving workpieces with few texture features, traditional cameras get poor quality images with dynamic blur and object adhesion, which makes the detection and tracking of workpieces unreliable. However, the event camera outputs events asynchronously at a microsecond speed when the pixel intensity changes, which can capture the contours of fast-moving workpieces well. In this paper, we propose a parallel two-pipe multi-target tracking algorithm based on the event camera for fast-moving workpieces. RGB-E image obtained by fusing the RGB image and the event solves the unreliable detection caused by dynamic blur and object adhesion. The parallel mechanism ensures that the low-speed detection pipeline does not have much impact on the speed of the high-speed tracking pipeline. Hungarian algorithm is used to associate the detection results obtained by the YOLOv4-tiny detector with the tracking results obtained by the KCF tracker. A correction algorithm based on pixel speed is proposed to synchronize detection results and tracking results. Experimental results prove the proposed algorithm can achieve reliable detection and tracking performance for fast-moving workpieces.

This article presents an automated strategy to touch the injection site on zebrafish larva skin with the injection pipette tip accurately in the presence of water-depth variation, which is a crucial problem to automate zebrafish larva microinjection. The presented method consists of two parts: adaptive coordinate transformation and curve evolution for edge detection. In the first part, the impact of refraction is taken into consideration. An adaptive calibration method is developed, which enables the coordinate transformation matrix to adapt to the changing water depth. In the second part, the abovementioned calibration result is used to keep the injection pipette tip descending along the desired route. A curve-evolution-based edge detection algorithm is introduced to detect the deformation of larva skin caused by contact with the injection pipette tip. Experimental results demonstrate that high accuracy and success rates are achieved. The effect of uncertainties caused by water-depth variation and the skill requirement in manual manipulation are eliminated. The proposed contact detection strategy can be extended to microinjection for other organisms.Note to Practitioners—As a typical multicellular model organism, the zebrafish has been increasingly used in biological research. For studying drug toxicity and disease models, exogenous substances need to be injected into zebrafish larvae. However, for both manual and automated injection, a fatal problem is that the camera on the microscope only provides 2-D positional information. It is laborious to align the pipette tip with the injection site along the$z$-axis. Moreover, due to the characteristic of stereomicroscopes, the impact of refraction at the water surface cannot be ignored. In order to address these issues, in this article, we present an adaptive calibration method and an edge detection algorithm for zebrafish larva heart injection to avoid contact failure in practical implementations.

This article investigates the adaptive fuzzy fault-tolerant control problem for a class of strict-feedback stochastic nonlinear systems with quantized input signal. A hysteretic quantizer is utilized to avoid chattering caused by quantized input signals. The fuzzy-logic systems are utilized to approximate the unknown nonlinear functions and also to construct the fuzzy state observer, which is used to estimate the immeasurable state vector. The actuator faults considered in this article are loss of effectiveness and lock-in-place faults. By using the Lyapunov stability theory, the closed-loop stochastic nonlinear system is guaranteed to be stable in probability, and all the signals of the closed-loop system are bounded in probability in the presence of quantized input and actuator faults. Finally, a simulation example is given to verify the validity of the proposed control strategy.

In the image based visual servo, image Jacobian is vital to the system performance because it is the bridge that warps the velocity in feature space to camera velocity in Cartesian space. However, image Jacobian is sensitive to the camera intrinsic parameters, while the camera intrinsic parameter calibration error is almost unavoidable, which heavily affects the image Jacobian and visual servo process. In this paper, we find the camera intrinsic parameter invariance of the image Jacobian in our 4 DOF visual servo system. The image Jacobian of some geometry features is invariant to the camera intrinsic parameters, indicating that the disturbance in camera intrinsic parameters will not affect the convergence trajectory of those features. To further analyse camera intrinsic invariance, we proposed camera intrinsic parameter Jacobian of Image Jacobian, which can fully describe the camera intrinsic parameter invariance of the image Jacobian. The work in this paper can be used to analyse the system sensitivity to the camera intrinsic parameters. The camera intrinsic invariance is also significant for choosing the visual servo features when designing the visual servo system.

Computer image processing technology has been widely used in ceramic product defect detection. However, there are few studies on automatic defect detection methods for ceramic products with complex surface geometry, such as ceramic wash basins. The main difficulty is that this type of sanitary ceramic product has a complex surface, strong light reflection, and very little texture information. It is hard to find a perfect illumination to make the defects obvious. In view of the above problems, this paper proposes a defect detection method based on the neighborhood pixel gray threshold, and conducts defect detection tests on the imaging results of ceramic wash basins with cracks on the surface under general light conditions, and obtains good results.

In this paper, we propose a modified CenterNet to complete the defect detection of Sanitary Ceramics. Generally, visual quality inspection is rather important during the productive process of Sanitary Ceramics and it is nearly impossible to inspect the massive images by hand. Consequently, it is necessary to devise an accurate and real-time system to process the data. However, due to the varied shapes and backgrounds of ceramics, conventional computer vision methods are usually not robust to all those variables. Detectors based on Deep Learning start to be adopted in recent years, but most algorithms require some carefully devised anchor boxes and post-processing methods, which also bring more computational costs. Here we decide to take advantage of the anchor-free model, CenterNet. We change the main structure to fit our own data and introduce an extra branch with shallow layers to strengthen the feature representation. The results have shown the great power of this model. Without even any post-processing methods, our model achieves a result of 96.16 AP on the established dataset.

This paper presents an identification method which is based on genetic algorithm (GA) and its improved method to estimate dynamic parameters of industrial robots without load. The procedure consists of the following steps: 1) derivation of the linear form of the dynamic model of the robot according to the Lagrange equation; 2) designing of the excitation trajectory in the form of fifth order Fourier series as exciting trajectory; 3) identification, where genetic algorithm is used to find the global optimal parameters through the genetic exchange between the groups and the survival of the fittest mechanism with the minimum variance between the theoretical torque and the actual torque as the optimization criteria; 4) model validation; 5) analysis of the factors influencing the accuracy of the results in the identification process; 6) proposal of improved method. The experimental results show that the predicted torque and the measured torque obtained by the identification algorithm have a high matching degree, and the model can reflect the actual dynamic characteristics of the robot.

With the development of intelligent technologies, intelligent railway vehicles are playing an important role in modern railway transportation. To provide an accurate estimation of speed and position as feedback for the speed regulation system of intelligent railway vehicles, we propose a robust odometry algorithm based on data fusion of a low-resolution photoelectric encoder and an IMU. Firstly, the train pose measured from IMU is used to transform the gravity vector from the world frame to IMU frame, which is used to offset the ground inclination and acquire longitudinal acceleration measurements. Then, the speed measurements of the encoder are fused with the acceleration measurements in a joint data-fusion framework. Criteria based on acceleration difference is set to judge whether wheel sliding happens, and dynamically configure the weights of IMU and encoder to compensate for the deviation caused by sliding. The performance of our algorithm is verified in MATLAB simulation tests, and the experimental results demonstrate that our method outperforms traditional encoder-based and IMU-based methods in both accuracy and robustness.

In this paper, we propose four offline data augmentation methods to improve the performance of convolutional neural network(CNN) on defect detection of sanitary ceramics. In recent years, based on big data, deep learning has begun to become a popular way for sanitary ceramics defect detection. Comparing with traditional vision inspection system, deep learning method is more robust and convenient without manual design of feature extraction. As a data-driven detection way, data plays a vital roll, however, sometimes we could not obtain a high-quality and large dataset. Consequently, we consider data augmentation to improve the quality of original dataset. Here, we use image generation, image mosaic, image fusion and image rotation mosaic. According to the experiment results, with these methods, the enhanced datasets perform well compared with the original one.

Aerial manipulator is a kind of robot with broad application prospects, which is suitable for high altitude operation and other dangerous application scenarios. This paper presents a trajectory tracking control algorithm for the aerial parallel manipulator based on Stewart platform. By modeling the overall dynamics of the aerial parallel manipulator, the expression of the influence of Stewart platform is given, and it is proved that this type of influence can be combined with the unmodeled error and external disturbance into the comprehensive disturbance of the flight platform. The flight platform trajectory tracking control is carried out by using the backstepping method, and the nonlinear disturbance observer is used for disturbance estimation and compensation in the control output. Numerical experiments show that the proposed control method can realize the trajectory tracking control of the flight platform of the aerial parallel manipulator.

Computer vision based detection approaches are widely employed to detect or calibrate different types of meters nowadays. However, traditional detection algorithms suffer drawbacks in accuracy and adaptability upon detecting various types of automobile dashboards. Plenty of parameters of these algorithms need to be tuned to suit certain types of dashboards. Besides, theses algorithms cannot automatically read the speed value, which requires manual setting operations. In this paper, a novel approach is presented to adaptively detect different types of automobile dashboards. The contour analysis based method is first implemented to extract the connected component of the pointer. A robust character segmentation classifier, which is designed by cascading histogram of oriented gradients (HOG)/support vector machine (SVM) binary classifier, character filter as well as HOG/multiclass SVM digit classifier, is then proposed to recognize digit characters on the dashboard. Simultaneously, tick marks are then extracted based on recognition results. Finally, Newton interpolation linear relationship is established to diagnose the potential responding errors of the pointer. The experimental results show that the pointer extraction method is robust to interferences caused by connected components of digits and also that the established character segmentation classifier has a more accurate detection result. Furthermore, compared with similar algorithms, it has a significant advantage in detecting a vast majority of different dashboards without manual tuning of the parameters.

In this paper, we propose a force feedback scheme for Delta device to improve precision and stability in master-slave teleoperation. A simple and exact dynamical equation is created with principle of virtual work, which is easy to calculate in real-time. After analysing three items deep in dynamical equation, a reasonable strategy is designed to identify the mass of Delta mechanism. The identified parameters and dynamical equation are verified correct in ADAMS and MATLAB softwares. Simultaneously, following previous work on haptic interface, a suitable Kalman Filter algorithm is proposed to attain smooth contact force in real-time based on the impedance of environment, and simplified due to the short period of a cycle. Finally, a whole master-slave system is set up with CHAI3D toolkit, which consists of Phantom Omni, Computer and Delta device. With a force sensor mounted on the end, the contact force is measured in practice and then filtered with proposed algorithm. The final result shows that the estimated curve followed measured data well and lied at the center of original curve.