In this letter, the novel asymmetrict-GARCH(1,1) (ATGARCH(1,1)) model is proposed. By calculating the tail exponent (TE), it is discovered that the TE of the ATGARCH(1,1) distribution is smaller than the degree of freedom (dof) of its corresponding driving Student-t distribution. Actually, we get that the TE of the ATGARCH(1,1) distribution is jointly determined by the corresponding dof and its model parameters. Since the closed-form density functions of the ATGARCH(1,1) noises are unknown, the corresponding filtering problems are analyzed by the heuristic method. In the ATGARCH(1,1) noise cases, the simulation results show that our heuristic algorithm is superior to the standard Kalman filter (KF), the particle filter, and those robust KFs designed by the Student-t distribution models.

Accurate hand-eye and robot-world calibration remains crucial for vision-guided robotic applications. Existing methods predominantly focus on deriving closed-form or nonlinear optimization solutions for hand-eye equationAX=XBor hand-eye-robot-world calibration equationAX=YBunder various rotation parameterizations, while the critical role of input data refinement on calibration accuracy has been largely overlooked. This paper addresses this gap by proposing a pose graph optimization model that strategically enhances the precision of key pose chains in robotic vision systems, particularly under significant discrepancies between robotic positioning and visual measurement accuracies. Building upon the optimized pose chains associated with the hand-eye and robot-world parameters, a pair of dual equations forXandYis then constructed and solved via our proposed Kronecker product-based least squares algorithm, allowing simultaneous and accurate estimation on rotation matrices and translation components ofXandY. Simulation and real-world experimental results have validated the superiority of the proposed pose graph optimization model, under significant robot-vision precision disparities. Moreover, even when the error magnitudes between the robot and visual measurement system are comparable, the proposed method achieves superior accuracy for both the hand-eye and robot-world parameters compared to nine state-of-the-art approaches.

This study investigates the problem of large-scale swarm robots shape self-assembly problem under conditions of information asymmetry. Existing methods assume complete sharing of global information; however, this assumption has significant limitations in terms of resource consumption and swarm emergence. On the other hand, strategies that rely entirely on local information struggling to achieve the self-assembly of complex shapes, especially disconnected shapes. To address these challenges, this study proposes a novel bio-inspired distributed self-assembly strategy specifically designed for information asymmetric swarm. The strategy draws inspiration from the task specialization mechanism between scout ants and worker ants in social insects, guiding individuals to efficiently complete shape-assembly under information asymmetry through local perception, neighborhood interactions, and dynamic rule adjustments. Experimental results show that the proposed strategy successfully achieves the self-assembly of various shapes, including simple shapes (e.g., circle, rectangle), complex shapes (e.g., human, flower, and letter "A"), and disconnected shapes (e.g., letter "IO"). This demonstrates the strategy’s adaptability to shape complexity. Furthermore, experiments with varying swarm sizes validate the strategy’s robustness and scalability across different scales. During the experiments, we unexpectedly observed emergent behaviors within the swarm, further confirming that the proposed strategy not only significantly enhances task flexibility but also strengthens swarm emergence. These results indicate that the proposed method provides an efficient, scalable, and innovative solution for swarm robots self-assembly under information asymmetry.

This paper presents an online multi-population evolution path planning (OMEPP) algorithm to address the flexible path planning problem for mobile manipulators in complex dynamic environments. The OMEPP algorithm treats the mobile manipulator as a high-dimensional system to utilize its flexibility. The OMEPP algorithm is based on random sampling and evolutionary concepts: Optimization and passive obstacle avoidance operations are performed on the path at runtime, with superior paths replacing inferior ones within the same population. A novel path population partitioning approach is proposed to maintain diverse switchable paths, thereby improving robustness. This paper also proposes an efficient manipulator collision detection method and several other mechanisms to enhance the algorithm’s effectiveness. The experimental results demonstrate the algorithm’s ability to swiftly adapt and optimize paths in response to dynamic environmental changes. Note to Practitioners—This paper presents OMEPP, an online evolutionary algorithm for real-time path planning of mobile manipulators in dynamic environments. OMEPP employs novel techniques including path population partitioning, random sampling, and evolution to efficiently generate collision-free paths among moving obstacles. A novel path population partitioning approach is proposed to maintain diverse switchable paths, thereby improving robustness. Simulations have demonstrated that the OMEPP algorithm is effective for real-time path planning of mobile manipulators in complex dynamic environments. Future work will focus on trajectory generation respecting dynamics limits.

This paper presents a novel output-feedback direct adaptive controller with a decoupling design to completely attenuate periodic disturbances in the linear multi-input multi-output (MIMO) system subject to input delay. First, the family of strictly causal stabilizing controllers for this system is derived by using the Youla parameterization method. The stabilizing controller consists of a fixed base controller J and an adjustable block Q. Periodic disturbances can be completely attenuated by adjusting the block Q. Then, to improve the attenuation rate of the periodic disturbance, a diagonal decoupling strategy is proposed. Based on the internal model principle (IMP), the interpolation condition for the complete attenuation of periodic disturbance is established. Next, to attenuate unknown time-varying periodic disturbances, a parameter adaptive algorithm is designed to update online the Q parameters. Finally, the effectiveness of the proposed controller is validated by simulation results. Note to Practitioners—The simultaneous presence of input delays and periodic disturbances in real systems not only limits control effectiveness but also leads to instability in the closed-loop system. Therefore, it is crucial to attenuate the periodic disturbances in input delay systems. However, existing adaptive controllers cannot completely attenuate unknown time-varying periodic disturbances if the system state is unavailable. Moreover, the multi-channel cross-coupling inherent in MIMO systems further constrains the controller performance. Motivated by this challenge, this paper proposes an output-feedback direct adaptive controller with decoupling design to completely attenuate periodic disturbances in linear MIMO input delay systems. It is hoped that the proposed method can provide valuable theoretical and technical guidance for the design of controllers aimed at compensating for periodic disturbances in linear MIMO systems subject to input delay.

In the field of no-reference image quality assessment (NR-IQA), the visual masking effect has long been a challenging issue. Although existing methods attempt to alleviate the interference caused by masking by generating pseudoreference images, the quality of these images is often constrained by the accuracy and reconstruction capabilities of image restoration algorithms. This can introduce additional biases, thereby affecting the reliability of the evaluation results. To address this problem, we propose a novel generative “noise” estimation framework (GNE-Vim) that eliminates the need for pseudoreference images. Instead, it deeply decouples the distortion components from degraded images and performs quality-aware modelling of these components. During the training phase, the model leverages both reference images and distortion components to guide the learning of the true distortion distribution. In the inference phase, quality prediction is conducted directly on the basis of the decoupled distortion components, making the evaluation results more aligned with human subjective perception. The experimental results demonstrate that the proposed method achieves strong performance across datasets containing various types of distortions. The source code is publicly available at the following website: https://github.com/opencodelxt/GNE-Vim.

This paper develops a bilayer predefined time fuzzy reinforcement learning (PT-FRL) control strategy to improve the efficiency of autonomous vehicle formation execution and reduce energy consumption. First, the fixed constraints of communication connectivity and collision avoidance are reconstructed into performance constraints, and normalized error mapping techniques are used to transform them into a new unconstrained error system. Then, based on the system, a cost function was constructed that balances cost control and performance. The control strategy adopts a bilayer architecture: In the first layer, a feedforward controller is designed to provide a more concise control object for subsequent PT-FRL controllers by compensating for known nonlinear coupling terms in advance, and can significantly reduce fuzzy logic systems computational load. and in the second layer, a PT optimal formation controller is designed using FRL to ensure that the autonomous vehicles complete the formation task within the predefined time. Finally, the effectiveness of the proposed method was verified through simulation and experiments.

Detecting small objects in remote sensing images poses significant challenges to the field of computer vision, primarily stemming from the complexity of backgrounds, limitations in pixel resolution, and information loss during the feature fusion process. While general object detection has significantly advanced in recent years, remote sensing small object detection remains an unsolved problem, with existing frameworks struggling to achieve high performance at small scales. In this article, we propose a novel framework called the boundary-aware feature fusion network (BAFNet), which significantly enhances the model’s ability to represent and locate small objects precisely within complex remote sensing scenarios. First, a dual-stream attention fusion module captures complementary foreground and background cues through bidirectional context modeling. Jointly attending to objects and their surroundings enhances discriminative power for distinguishing small objects. Additionally, we incorporate a boundary-aware branch to better preserve crucial detailed information vital for small-scale objects. This auxiliary component supervises the fusion of contextual semantics and spatial information, aiding in retaining critical boundary details that are prone to loss during cross-layer feature fusion. We conducted experiments on the challenging AI-TOD, VisDrone, DIOR, and LEVIR-Ship datasets. The results demonstrate the superiority of our approach over other state-of-the-art (SOTA) object detection methods, particularly in terms of precisely identifying small objects within remote sensing images. The code is available athttps://github.com/ooo1128/BAFNet.

Vibration isolators are essential for sensitive and precise industrial applications working on instrumentation and control systems since they isolate microvibration, prevent error propagation, and enhance processing/image quality. This article proposes a novel gravity compensator (GC) for vibration isolation in satellite applications based on permanent magnets. Owing to the new magnetic circuit topology, the proposed system features a lower natural frequency and higher force density, enabling superior low-frequency isolation performance and bearing capacity compared with traditional isolators. Owing to its low natural frequency, the proposed system is superior to conventional GCs in isolating low-frequency disturbances. Moreover, the high force density enables the GC to exploit a heavier payload with less magnetic material, which is economical and space-saving. In addition, well-designed electromagnets are utilized to adjust the static levitation force, which can improve the ability to satisfy the requirements of different loads without drastically changing the stiffness of the entire system. The results suggest that the proposed GC could be utilized for satellite applications with heavy loads and limited mounting space.

Multispectral object detection has shown great promise in security and industrial applications. RGB images offer rich texture but are limited by lighting, whereas IR images excel in low light but lack texture. Current methods face challenges in accurately capturing information differences and achieving effective feature fusion across modalities. To address these issues, we propose a graph aggregation alignment network (GAANet) for multispectral object detection. GAANet consists of two key modules: the graph interaction fusion module (GIFM) and the information alignment module (IAM). GIFM uses graph representation learning to effectively process single-modality features, and the direct connection information flow mechanism guides and references low-level multimodal features, ensuring the global and comprehensive fusion of node information in the graph space. The results are then refined through the IAM for secondary calibration and alignment of corresponding local regions, ensuring accurate fusion. We also introduce an information reconstruction path (IRP) and reconstruction loss to prevent the loss of single-modality information due to multiple IAM calculations. GAANet achieves excellent fusion detection capability and significantly reduces the number of parameters, reducing the model size by 61.2% compared with that of representative baselines such as CALNet. GAANet achieves state-of-the-art results on the DroneVehicle, LLVIP, and FLIR datasets, with superior object detection accuracy. It also performs well on the unaligned DVTOD dataset, effectively capturing feature offsets across modalities through global graph perception.

Infrared small target detection has been extensively studied due to its wide range of applications. Most studies treat infrared small target detection as an independent task, either as a detection-based or a segmentation-based, failing to fully leverage the supervisory information from different annotation forms. To address this issue, we propose a multi-task mutual learning network (MTMLNet) specifically designed for infrared small targets, aiming to enhance both detection and segmentation performance by effectively utilizing various forms of supervisory information. Specifically, we design a multi-stage feature aggregation (MFA) module capable of capturing features with varying gradients and receptive fields simultaneously. Additionally, a hybrid pooling down-sampling (HPDown) module is proposed to mitigate information loss during the down-sampling process of infrared small targets. Finally, the hierarchical feature fusion (HFF) module is designed to adaptively select and fuse features from different semantic layers, learning the optimal way to fuse features across semantic layers. The results on IRSTD-1k and SIRST-V2 datasets show that our proposed MTMLNet achieves state-of-the-art (SOTA) performance in both detection-based and segmentation-based methods. The codes are available at https://github.com/YangBo0411/MTMLNet.

Representation learning of road networks is essential for various downstream traffic-related tasks, as road network contain multi-modal data with rich information, and the learned embeddings can be directly used in machine learning models. However, due to the dynamic changes in road networks with respect to topology and associated data, as well as the local and long-range dependency caused by complex mobility semantics, learning robust and effective representations remains challenging. To this end, we exploit the properties of the road network and the mobility semantics embedded in trajectories, and propose a novel S emantic- E nhanced G raph C ontrastive L earning (SE-GCL) framework, for learning general-purpose embeddings of road networks. Specifically, in this framework, we propose (1) a multi-modal feature embedding module to capture both the attribute and visual information of road segments, (2) a semantic-enhanced graph augmentation strategy to simulate topological changes and data missing in the road network, and (3) a semantic-enhanced contrastive optimization module that leverages geo-locality and mobility semantics to guide representation learning. Extensive experiments are conducted on two real-world road networks with three representative downstream tasks. The result demonstrate that SE-GCL yields more robust and effective representations, outperforming the state-of-the-art baselines. The source code is available at https://github.com/csjiezhao/SE-GCL .

Region representations (also called embeddings) are useful for various urban computing tasks. While graph-based region representation learning methods have shown outstanding performance, they encounter two major challenges: 1) the pervasive data noise and missing data can affect the quality of the constructed region graphs; and 2) high-order relationships (i.e., group-wise relationships) among regions are often insufficiently modeled and sometimes entirely overlooked. To this end, we proposeHyperRegion, an unsupervised region representation learning framework that integrates graph and hypergraph contrastive learning to learn comprehensive region embeddings from multi-modal data. Built upon a region hybrid graph network, this framework models both pair-wise and group-wise dependencies involving POI semantics, mobility patterns, geographic neighbors, and visual semantics. To mitigate the impact of data noise and missing data, graph and hypergraph contrastive learning are performed in parallel, and a cross-module contrast is further introduced to facilitate information exchange and collaboration. Extensive experiments on real-world datasets across three downstream tasks demonstrate thatHyperRegionoutperforms all baselines, particularly improving check-in prediction by reducing MAE and RMSE by approximately 8.5% and 8.2%, respectively, and increasing$R^{2}$by about 7%.

Multispectral object detection, which combines RGB visible light and thermal infrared spectral information, has broad applications in complex environments and varying illumination conditions. However, existing methods face challenges in processing multispectral data, such as inconspicuous object features in spectral images and significant discrepancies between input modality spaces and output detection spaces. To address these issues, we propose an innovative multispectral object detection method that combines contrastive learning and a new cross-modal feature fusion module. We introduce a mask feature contrastive loss that maximizes the similarity between the box-level mask features and modal features while suppressing background responses, enabling effective representative alignment between the input and output spaces. Additionally, we propose a mask-guided attention fusion module that uses a predicted pseudo mask to guide the fusion of different modal features, enhancing object responses and reducing background noise interference. Our extensive experiments on several challenging multispectral datasets demonstrate that our proposed COFNet achieves state-of-the-art performance.

Graphs offer a flexible structure for vision tasks, with CNNs and Transformers conditioned as two specific cases of graph structures. In CNNs, the input images are treated as graphs where only neighboring patches are connected, whereas Transformers view images as fully connected graphs. To leverage the potential of graphs in video representation learning, effective graph generation and training methods are crucial. To this end, we propose VideoGNN, which represents the video as a discrete time dynamic graph and learns the dynamic graph efficiently. Given the multitude of frames in videos, we introduce an efficient graph generation module characterized by low complexity and high quality, facilitating the transformation of videos into dynamic graphs. Additionally, we introduce a dual-view graph neural network to capture spatial and temporal information from the generated dynamic graphs. Then, a sequential model is applied to capture the long-term temporal information and generate the final frame embeddings. Experiments demonstrate that VideoGNN can achieve competitive results in terms of graph quality assessment and video downstream tasks. The codes are available at https://github.com/Dodo-D-Caster/VideoGNN .

Methods based on transfer learning have achieved rich research results in the field of intelligent diagnosis of mechanical devices. However, current transfer learning methods typically require the source and target domains to be known in advance, heavily relying on historical data, which fails to meet the requirements of practical applications. Therefore, this study proposes a multisource online transfer learning with joint adaptive distribution selection (MSOTL-JADS) algorithm for real-time diagnosis of online samples. First, during the offline phase, a metric function is designed to extract sub-domain feature information in the multisource domain scenario, facilitating accurate knowledge transfer. Second, the multisource weight distribution parameters obtained based on the distribution distance are dynamically selected for the multisource domains to achieve differentiated transfer in the domain. Third, in the online phase, the pretrained offline model is integrated using online input target samples to construct an online diagnostic model and fine-tuning the weight parameters to match the model accuracy. Finally, several online transfer diagnostic tasks were constructed using two public datasets and one self-constructed dataset. The experimental results demonstrate that the proposed MSOTL-JADS outperforms the comparison methods in terms of performance.

Rigid robots can achieve precise motions but expose shortcomings in system complexity, fabrication cost, and humanrobot interaction, which motivates researchers to develop various soft robots to fill these gaps. Electro-hydraulic actuators (EHAs) have received widespread attention and been used in many soft robots due to impressive high-strain, fast-speed and rapidresponse characteristics. However, existing EHAs face challenges in achieving large-deformation, high-robustness, and low-weight simultaneously. This limits the application of EHAs in robotic systems that are weight-sensitive or require fail-safe and faulttolerant behavior. Here, we present a lightweight (0.98 g) electropneumatic actuator (EPA) filled with air and only 0.1-mL liquid dielectric, which achieves high-speed bending from 11° to 93.5° in 60 ms, large-angle bending from 11° to 104° in 2 s (the largest in current EHAs), and high-frequency swing at 20 Hz. The EPA is ultrarobust and can operate properly after being punctured by four needles or crushed twice by a 1500-kg vehicle. Furthermore, to validate the above features of EPAs, three applications are demonstrated at a voltage of 6 kV, including four-finger grippers, fast-crawling robots, and water-walking robots. This work pushes the boundaries of robustness and lightweight for EHAs, providing a foundation for the application of electro-pneumatic actuation in soft robotics.

Visual Simultaneous Localization and Mapping (SLAM) is crucial to many applications such as self-driving vehicles and robot tasks. However, it is still challenging for existing visual SLAM approaches to achieve good performance in low-texture or illumination-changing scenes. In recent years, some researchers have turned to edge-based SLAM approaches to deal with the challenging scenes, which are more robust than feature-based and direct SLAM methods. Nevertheless, existing edge-based methods are computationally expensive and inferior than other visual SLAM systems in terms of accuracy. In this study, we propose EdgeSLAM, a novel RGB-D edge-based SLAM approach to deal with challenging scenarios that is efficient, accurate, and robust. EdgeSLAM is built on two innovative modules: efficient edge selection and adaptive robust motion estimation. The edge selection module can efficiently select a small set of edge pixels, which significantly improves the computational efficiency without sacrificing the accuracy. The motion estimation module improves the system's accuracy and robustness by adaptively handling outliers in motion estimation. Extensive experiments were conducted on TUM RGBD, ICL-NUIM and ETH3D datasets, and experimental results show that EdgeSLAM significantly outperforms five state-of-the-art (SOTA) methods in terms of efficiency, accuracy, and robustness, which achieves 29.17% accuracy improvements with a high processing speed of up to 120 FPS and a high positioning success rate of 97.06%.

This article investigates the prescribed-time formation control in the task space of multirobot systems (MRSs), which is subject to the uncertain nonlinear dynamics and the position requirements. The strategy constructs a cascade system consisting of control and reference layers by connections of coupled prescribed-time control units, which separately guarantee the convergence of coordination errors, accuracy of states, and synchronization between layers. Meanwhile, the transformation from task space to joint space is built based on the pseudo-inverse of the Jacobi matrix, which avoids the singular value problem brought by computing the inverse kinematics. The adaptive control method utilizes parameter estimation to eliminate the inaccuracy problem brought by the pseudo-inverse of Jacobi matrix transformation. Then, this article provides solutions to the formation control and the formation along the trajectory control. Correspondingly, the Lyapunov analysis process proves the system’s stability and the parameter estimation’s boundedness, which confirms the sufficient conditions for realizing the prescribed-time formation control of the MRSs. Finally, this article presents examples of time-varying formation and along-trajectories formation, thereby demonstrating the effect of the controller.

In complex unstructured environments, accurate terrain traversability analysis is a fundamental requirement for the successful execution of any movements of ground robots, especially given that terrain traversability often exhibits anisotropy. However, the difficulty in obtaining multi-directional terrain labels hinders the emergence of end-to-end multi-directional traversability network. This paper introduces a framework for real-time multi-directional traversability maps (MTraMap) generation tailored for unstructured environments. It involves pre-training a uni-directional traversability classifier, termed UniTraT, through self-supervised learning using ground robot travel simulation. Furthermore, it employs Uni-directional to Multi-directional Traversability Distillation (UMTraDistill) to distill a multi-directional traversability network, termed MultiTCNN, which is capable of directly generating MTraMap. We evaluated both networks on our traversability dataset, achieving an 89% accuracy in terrain traversability classification with the UniTraT. Compared to UniTraT, the accuracy of the MultiTCNN distilled via UMTraDistill only decreases by 1.8%, and it can process 10 m × 10 m elevation map at a speed of 74 fps. Field robotics experiments were also conducted and showed that MultiTCNN can generate MTraMap of the surrounding 20 m × 20 m environment at a rate of 9.39 fps, with a slight reduction of 0.61 fps compared to the lidar data publishing rate, and the generated MTraMap can clearly delineate the multi-directional traversability of the surrounding environments.

The quality of a robot’s environmental perception determines whether it can achieve more intelligent applications, such as semantic interaction with humans. SLAM, on the other hand, is one of the crucial capabilities for a robot to perceive its environment. However, when only a monocular image is provided, dynamic objects in the environment significantly impact the accuracy of map construction by the robot, leading to erroneous perception results. To address this issue, we propose a Visual SLAM framework based on BEV perception results, named BE-SLAM. With this framework, we can handle dynamic objects, occlusions, and incompletely observed objects. It can construct a stable static map by strengthening trust in static objects. Considering that object-level semantic maps can enhance a robot’s perception abilities, we also reconstruct static objects in the map and use them to optimize the pose. Through experiments on existing publicly available dataset, we compare BE-SLAM with several existing methods that have shown good performance. The experimental results demonstrate that BE-SLAM performs exceptionally well on high-dynamic sequences and achieves comparable results on static or low-dynamic sequences.

Versatile video coding (VVC) aims to achieve high compression but also issues like varying content/network conditions. Existing rate control (RC) methods struggle to achieve optimal quality under these complex scenarios. This paper proposes a novel RC scheme for VVC based on a linear model. The Lagrange minimization multiplier is introduced under bit budget constraints, allowing optimized bit allocation. RC optimization is formulated as a convex solution, and is derived into the optimal quantization parameter (QP) for RC. Experimental analysis demonstrates the proposed linear model-based RC algorithm performances are better compared to other state-of-the-art methods due to their use of a linear model and optimal QP determination.

In this paper, a parallel decoder and a word group prediction module are proposed to speed up decoding and improve the effect of captions. The features of the image extracted by the encoder are linearly projected to different word groups, and then a unique relaxed mask matrix is designed to improve the decoding speed and the caption effect. First, since image captioning is composed of many words, sentences can also be broken down into word groups or words according to their syntactic structure, and we achieve this function through constituency parsing. Second, we make full use of the extracted features to predict the size of word groups. Then, a new embedding representing the information of the word is proposed based on word embedding. Finally, with the help of word groups, we design a mask matrix to modify the decoding process so that each step of the model can produce one or more words in parallel. Experiments on public datasets demonstrate that our method can reduce the time complexity while maintaining competitive performance.

Public computer education systems provide students essential opportunities to enhance computer literacy and information skills. However, the widespread adoption of online education technology exposes the field to several critical security risks. Threats, such as malware infections, data breaches, and other network intrusions, are all challenging the security of education systems, posing potential hazards to students’ personal information and even the entire teaching environment. To spur further work into specialized anomaly detection techniques for computer education, this paper presents an anomaly detection framework tailored for network services in computer education environments to safeguard these systems. Specifically, the proposed approach learns from large-scale online educational traffic data to classify the security state into five alert levels, enabling more granular anomaly detection and analysis. To assess their detection performance, deep learning and traditional machine learning algorithms are implemented and compared for multi-class intrusion classification. The results show that the proposed framework provides an effective security solution to bolster the integrity and stability of computer education systems against evolving network threats, enhancing threat intelligence to inform proactive security by detecting and characterizing anomalies through multilevel classification.

Deep learning-based video coding methods have demonstrated superior performance compared to classical video coding standards in recent years. The vast majority of the existing deep video coding (DVC) networks are based on convolutional neural networks (CNNs), and their main drawback is that since CNNs are affected by the size of the receptive field, they cannot effectively handle long-range dependencies and local detail recovery. Therefore, how to better capture and process the overall structure as well as local texture information in the video coding task is the core issue. Notably, the transformer employs a self-attention mechanism that captures dependencies between any two positions in the input sequence without being constrained by distance limitations. This is an effective solution to the problem described above. In this paper, we propose end-to-end transformer-based adaptive video coding (TAVC). First, we compress the motion vector and residuals through a compression network built on the vision transformer (ViT) and design the motion compensation network based on ViT. Second, based on the requirement of video coding to adapt to different resolution inputs, we introduce a position encoding generator (PEG) as adaptive position encoding (APE) to maintain its translation invariance across different resolution video coding tasks. The experiment shows that for multiscale structural similarity index measurement (MS-SSIM) metrics, this method exhibits significant performance gaps compared to conventional engineering codecs, such as [Formula: see text], [Formula: see text], and VTM-15.2. We also achieved a good performance improvement compared to the CNN-based DVC methods. In the case of peak signal-to-noise ratio (PSNR) evaluation metrics, TAVC also achieves good performance.

Single-frame infrared small target detection is considered to be a challenging task, due to the extreme imbalance between target and background, bounding box regression is extremely sensitive to infrared small target, and target information is easy to lose in the high-level semantic layer. In this article, we propose an enhancing feature learning network (EFLNet) to address these problems. First, we notice that there is an extremely imbalance between the target and the background in the infrared image, which makes the model pay more attention to the background features rather than target features. To address this problem, we propose a new adaptive threshold focal loss (ATFL) function that decouples the target and the background, and utilizes the adaptive mechanism to adjust the loss weight to force the model to allocate more attention to target features. Second, we introduce the normalized Gaussian Wasserstein distance (NWD) to alleviate the difficulty of convergence caused by the extreme sensitivity of the bounding box regression to infrared small target. Finally, we incorporate a dynamic head mechanism into the network to enable adaptive learning of the relative importance of each semantic layer. Experimental results demonstrate our method can achieve better performance in the detection performance of infrared small target compared to the state-of-the-art (SOTA) deep-learning-based methods. The source codes and bounding box annotated datasets are available athttps://github.com/YangBo0411/infrared-small-target.

Aircraft detection holds significant importance in both civilian and military fields, such as air transportation control and battlefield situational awareness. Deep learning-based methods for aircraft detection can obtain promising detection performance with sufficient and labeled samples. However, current public aircraft datasets are mainly from top-down perspective images, lacking multi-perspectives samples, which limits their applicability. Therefore, we construct a multi-perspectives aircraft dataset (MAD), encompassing 10 distinct classes, and making up of 13,205 images and 18,908 instances. Moreover, the imbalanced and similar classes are common issues in aircraft detection. We design an adaptive threshold focal loss (ATFL) function to alleviate the class imbalance through dynamically adjusting the loss weights between different classes. Meanwhile, the dynamic visual center (DVC) module which can effectively capture both local and global information of target is proposed to distinguish the aircraft classes that share similar feature. Finally, we evaluated several state-of-the-art methods on the constructed MAD dataset as baselines for future research.The experimental results verify that the recognition performance of the current state-of-the-art model can be improved after the application of ATFL and DVC. The dataset and code are available athttps://github.com/YangBo0411/aircraft-detection.

While a variety of innovative travel modes, such as taxi service and ridesourcing service, have been launched to improve the transportation efficiency, people still encounter travel problems in real life. The major cause is the imbalance between transportation supply and demand. To strike a balance, it is well-recognized that an accurate and timely passenger demand prediction model is the foundation to enable high-level human intelligence (i.e., taxi drivers) or machine intelligence (i.e., ride-hailing platforms) to allocate resources in advance. Although quite a lot of deep models have been designed to model the complicated spatial and temporal dependencies in a data-driven way, they focus on the demand prediction of a single mode and ignore the fact that passengers may shift between different modes, especially between taxis and ridesourcing cars. In this paper, we target a co-prediction problem that considers the prediction of taxi and ridesourcing as two coupled and associated tasks, and propose a novel Temporal and Spatial Intertwined Network (TSIN) that consists of two twin components and an intertwined component. Each twin in the TSIN model is able to extract spatial and temporal dependencies from its corresponding travel mode separately (i.e., intra-mode features), and the in-between intertwined component is designed to bridge the twins and allow them to exchange information (i.e., inter-mode features), thus enabling better prediction. We first evaluate our model on four real-world datasets. Results demonstrate the outstanding performance of our model and the necessity to take into account the influence between modes. Based on an additional demand data from bike in NYC, we then discuss the generalizability in coupling more transportation modes. Further results demonstrate that our proposed intertwined neural network is highly flexible and extendable, and can yield better prediction performance.

Foggy days limit the functionality of outdoor surveillance systems. However, it is still a challenge for existing methods to maintain the uniformity of defogging between image regions with a similar depth of field and large differences in appearance. To address above problem, this article proposes a regional gradient constrained prior (RGCP) for defogging that uses the piecewise smoothing characteristic of the scene structure to achieve accurate estimation and reliable constraint of the transmission. RGCP first derives that when adjacent similar pixels in the fog image are aggregated and spatially divided into regions, clusters of region pixels in RGB space conform to a chi-square distribution. The offset of the confidence boundary of the clusters can be regarded as the initial transmission of each region. RGCP further uses a gradient distribution to distinguish different regional appearances and formulate an interregional constraint function to constrain the overestimation of the transmission in the flat region, thereby maintaining the consistency between the estimated transmission map and the depth map. The experimental results demonstrate that the proposed method can achieve natural defogging performance in terms of various foggy conditions.

RGBT tracking is gaining popularity due to its ability to provide effective tracking results in a variety of weather conditions. However, feature specificity and complementarity have not been fully used in existing models that directly fuse the correlation filtering response, which leads to poor tracker performance. In this article, we propose correlation filters with adaptive modality weight and cross-modality learning (AWCM) ability to solve multimodality tracking tasks. First, we use weighted activation to fuse thermal infrared and visible modalities, and the fusion modality is used as an auxiliary modality to suppress noise and increase the learning ability of shared modal features. Second, we design modal weights through average peak-to-correlation energy coefficients to improve model reliability. Third, we propose consistency in using the fusion modality as an intermediate variable for joint learning consistency, thereby increasing tracker robustness via interactive cross-modal learning. Finally, we use the alternating direction method of multipliers algorithm to produce a closed solution and conduct extensive experiments on the RGBT234, VOT-TIR2019, and GTOT tracking benchmark datasets to demonstrate the superior performance of the proposed AWCM against compared to existing tracking algorithms. The code developed in this study is available at the following website. 1

Multidirectional jumping is commonly found in living creatures and desirable to be integrated into mobile robots for enhanced agility. Existing jumping robots mostly employ complex or cumbersome structures and modular designs to achieve multidirectional jumping. There is a lack of a simple, lightweight, and compact actuator design for multidirectional jumping robots. Here, we present a multidirectional jumping soft robot (MDJSR) driven by a biaxial electrohydraulic actuator (BEHA). The BEHA has a simple structure, i.e., a thin plastic frame-guided film pouch with four pairs of distributed electrodes and enclosed with a dielectric liquid. Inspired by gall midge larvae, the MDJSR exhibits two switchable locomotion strategies, including continuous non-energy-storing jumping to move fast and energy-storing jumping to cross obstacles. Its multidirectional jumping capability was demonstrated in the navigation through a labyrinth with two ways of obstaclecrossing and obstacle-circumventing in different terrain environments. In addition, the robot can be deployed to detect unknown space and collect environmental factors. This work provides an enabling solution to miniature and lightweight multimodal jumping soft robots for various robotic tasks

In this paper, we propose a lightweight multi-scale based attention network (MBAN) for single-image super-resolution (SISR). First, a deep feature transform block (DFTB) is designed for multi-scale feature extraction; this block combines group convolution and improved channel attention (ICA) for performance purposes while remaining sufficiently lightweight. Second, a dual multi-scale attention block (DMAB) is proposed for long-range information interaction; this block employs different window sizes for self-attention (SA) and short connections between different branches to achieve multiscale attention interaction. Finally, our MBAN is constructed by cascaded multi-scale based attention blocks (MBABs) that perform detail restoration; these blocks simultaneously extract multi-scale local features and integrate multi-scale global features with the DFTBs and DMABs. Extensive experiments suggest the superiority of our MBAN over the state-of-the-art (SOTA) lightweight SR methods in terms of both quantitative metrics and visual quality.

Adaptively reconstruction of the skin surface covering the skeletons of aircrafts, ships, high-speed trains, buildings with steel frames, etc., based on on-site measured point clouds is an important issue in both industry and construction. However, the skeleton has the characteristics of long and narrow shape and variable structure, resulting in a narrow and sparse distribution of measured point cloud on its surface with variable shape in different areas, which brings great challenges to the reconstruction of skin surface. In this article, a method for accurately reconstructing the skin surface covering the skeleton structures based on the arbitrary distributed sparse on-site measured points of the skeleton surface is proposed. At first, a robust B-spline curve fitting method based on the least square principle is proposed to construct the boundary curves of the point cloud. Then, a Coons-B-spline surface fitting method based on the generated boundary curves is proposed to generate an initial skin surface. Next, the initial skin surface is considered as a curved thin plate with stiffness, and a method of surface deformation considering the target points of deformation and the tensile and shear stiffness of the surface is proposed to obtain the surface with high accuracy and good smoothness. To show the feasibility of the proposed method, simulations and experiments are carried out. It is proved that the proposed method can achieve better accuracy while maintaining smoothness.

Remaining useful life (RUL) prediction can provide a foundation for the operation and maintenance of industrial equipment. In order to improve the predictive ability for the complex degradation trajectory, a new dual-thread gated recurrent unit (DTGRU) is explored. It uses a dual-thread learning strategy to mine the stationary and nonstationary information from the input data and the difference of hidden states at two adjacent time steps. Then the state transition updating formulas of DTGRU are derived. Using the collected gear vibration signals and degradation-trend-constrained variational autoencoder, the gear health indicator (HI) is constructed. Based on the constructed HI and DTGRU, a novel RUL prediction method is developed. Via multiple gear life-cycle datasets, the effectiveness of the DTGRU-based RUL prediction approach is verified. Furthermore, compared with the existing typical prediction methods, the experimental results show that DTGRU has higher predictive ability in terms of HI fitting precision and RUL prediction performance.

Facing the gap in the unsupervised construction of health indicator (HI) with a uniform failure threshold, a new unsupervised HI construction approach is developed. First, the distribution of the raw vibration signal is estimated by the Gaussian mixture model, then a distribution contact ratio metric (DCRM) is designed to compute the distance between two arbitrary distributions. With DCRM, a distribution contact ratio metric health indicator (DCRHI) is innovatively constructed for well representing the degradation process and obtaining a uniform failure threshold. Next, aiming at the challenge of prediction under limited samples, a novel consolidated memory gated recurrent unit (CMGRU) is proposed by making full use of the historical state information, and it can effectively slow down the forgetting speed of important trend information. Combing the proposed DCRHI and CMGRU, a novel remaining useful life (RUL) prediction methodology is put forward for enhancing the predictive performance. Via two public bearing datasets, several contrast experiments are implemented, and the comparative results show that DCRHI can better describe the degradation process of bearing than other typical unsupervised HIs, and CMGRU has a stronger prediction ability than other classical time series processing networks. Thus, the proposed methodology has great application value in the RUL prediction.

Accurate traffic prediction is a critical yet challenging task in Intelligent Transportation Systems, benefiting a variety of smart services, e.g., route planning and traffic management. Although extensive efforts have been devoted to this problem, it is still not well solved due to the flexible dependency within traffic data along both spatial and temporal dimensions. In this paper, we explore the flexibility from three aspects, namely the time-varying local spatial dependency, the dynamic temporal dependency, and the global spatial dependency. Then we propose a novel Dual Graph Gated Recurrent Neural Network (DG2RNN) to effectively model all these dependencies and offer flexible (multi-step) predictions for future traffic flow. Specifically, we design a Dual Graph Convolution Module to capture the local spatial dependency from two perspectives, namely road distance and adaptive correlation. To model the dynamic temporal dependency, we firstly develop a Bidirectional Gated Recurrent Layer to capture the forward and backward sequential contexts of historical traffic flow, then combine the derived hidden states with their various contributions learned by a temporal attention mechanism. Besides, we further design a spatial attention mechanism to learn the latent global spatial dependency among all locations to facilitate the prediction. Extensive experiments on three types of real-world traffic datasets demonstrate that our model outperforms state-of-the-arts. Results also show our model has more stable performance for the flexible prediction with varying prediction horizons.

Soft wall-climbing robots have been limited in their ability to perform complex locomotion in diverse environments due to their structure and weight. Thus far, soft wall-climbing robots with integrated functions that can locomote in complex 3-D environments are yet to be developed. This article addresses this challenge by presenting a lightweight (2.57 g) soft wall-climbing robot with integrated linear, turning, and transitioning motion capabilities. The soft robot employs three pneumatic bending actuators and two adaptive electroadhesion pads, which enable it to flip forward, transition between two walls, turn in two directions, and adhere to various surfaces. Different motion and control strategies are proposed based on a theoretical model. The experimental results demonstrate that the robot can move at an average speed of 3.85 mm/s (0.08 body length/s) on horizontal, vertical, and inverted walls and make transitions between walls with different pinch angles within 180°. Additionally, the soft robot can carry a miniature camera on vertical walls to perform detection and surveillance tasks. This article provides a reliable structure and control strategy to enhance the multifunctionality of soft wall-climbing robots and enable their applications in unstructured environments.

The input/output (I/O) pins of an industry-level fluorescent optical fiber temperature sensor readout circuit need on-chip integrated high-performance electro-static discharge (ESD) protection devices. It is difficult for the failure level of basic N-type buried layer gate-controlled silicon controlled rectifier (NBL-GCSCR) manufactured by the 0.18 µm standard bipolar-CMOS-DMOS (BCD) process to meet this need. Therefore, we propose an on-chip integrated novel deep N-well gate-controlled SCR (DNW-GCSCR) with a high failure level to effectively solve the problems based on the same semiconductor process. Technology computer-aided design (TCAD) simulation is used to analyze the device characteristics. SCRs are tested by transmission line pulses (TLP) to obtain accurate ESD parameters. The holding voltage (24.03 V) of NBL-GCSCR with the longitudinal bipolar junction transistor (BJT) path is significantly higher than the holding voltage (5.15 V) of DNW-GCSCR with the lateral SCR path of the same size. However, the failure current of the NBL-GCSCR device is 1.71 A, and the failure current of the DNW-GCSCR device is 20.99 A. When the gate size of DNW-GCSCR is increased from 2 µm to 6 µm, the holding voltage is increased from 3.50 V to 8.38 V. The optimized DNW-GCSCR (6 µm) can be stably applied on target readout circuits for on-chip electrostatic discharge protection.

This article proposes a new multisensor fusion fault diagnosis method for gearbox, namely residual gated dynamic sparse network, to improve the multisensor feature learning and fusion ability. Considering that the fault sensitivity of the sensor varies with mounted location and complex transfer path modulation causes information from multisensor redundant, the lightweight channel attention unit is designed to strengthen the feature extraction ability of the network. The developed gated dynamic sparse unit is inserted into the deep architecture to eliminate ineffective components caused by high noise interference. Besides, the loss function is improved with multiple activation criteria to enhance convergence ability. The results of experiments and the engineering application show that the proposed method is more effective than other methods under varying degrees of noise interference.

The digital twin of a life-cycle rolling bearing is significant for its degradation performance analysis and health management. This article proposes a digital twin model of life-cycle rolling bearing driven by the data-model combination. With the measured signals and the bearing fault dynamic model, the time-varying defect size is estimated, and the evolution law of bearing defect during the life cycle is revealed by a back propagation neural network. Then, the excitations of evolutionary defects are introduced into the bearing dynamic model, so as to form a life-cycle bearing dynamic model in the virtual space. Finally, the simulation data in the virtual space is mapped into the corresponding data in the physical space via an improved CycleGAN neural network with the smooth cycle consistency loss. By comparing the obtained digital twin result with the measured signal in the time-domain and frequency-domain, the effectiveness of the proposed model is verified.

In this article, facing the gaps that the traditional long short-term memory (LSTM) and convolution neural network (CNN) cannot differentially deal with the input data based on the corresponding trend and stage information in remaining useful life (RUL) prediction, a more accurate and robust RUL prediction model is constructed. First, a temporally multidifferential LSTM (TMLSTM) with the multitrend division unit and multicellular unit is proposed, and a spatially multidifferential CNN (SMCNN) with the multistage division unit and differentiated convolutions is designed. Then, by combining TMLSTM and SMCNN, a spatiotemporally multidifferential deep neural network is developed for predicting the equipment RUL, which enhances the ability of feature extraction from the spatiotemporal perspective by using the multitrend and multistage information. Via several evaluation indexes, the commercial modular aero propulsion system simulation dataset and the wind turbine gearbox bearing dataset are used to validate the superiority of the proposed method over several existing prediction methods.

This paper investigates the issue of event-triggered adaptive finite-time state-constrained control for multi-input multi-output uncertain nonlinear systems. To prevent asymmetric time-varying state constraints from being violated, a tan-type nonlinear mapping is established to transform the considered system into an equivalent “non-constrained” system. By employing a smooth switch function in the virtual control signals, the singularity in the traditional finite-time dynamic surface control can be avoided. Fuzzy logic systems are used to compensate for the unknown functions. A suitable event-triggering rule is introduced to determine when to transmit the control laws. Through Lyapunov analysis, the closed-loop system is proved to be semi-globally practical finite-time stable, and the state constraints are never violated. Simulations are provided to evaluate the effectiveness of the proposed approach.

Domain adaption models are widely applied to fault transfer diagnosis. However, the traditional domain adaption models can output only one high-dimensional transfer feature (TF); thus, it is difficult to capture domain-invariant information. Besides, using only one fully connected top classifier probably causes overfitting. Considering these two problems, in this article, we propose a multiscale transfer voting mechanism (MSTVM) to improve the classical domain adaption models and it can be universally applicable to any one of most domain adaption models. MSTVM consists of two substrategies: multiscale transfer mechanism (MSTM) and multiple transfer voting mechanisms (MTVM). The MSTM block includes several branches with multiscale convolutional and pooling operations, and it can output several multiscale TFs to strengthen the domain confusion. The MTVM block consists of multiple top classifiers and a plurality voting operation; thus, MTVM can effectively avoid overfitting and improve generalization ability. MSTVM has the advantages of MSTM and MTVM. Via two transfer diagnosis experiments, the advantage of MSTVM for improving various domain adaption models is verified.

The current maritime applications have yielded strong demands for the development of advanced unmanned surface vehicles (USVs) with more reliable path following capabilities to greatly extend mission durations and enhance accommodative capabilities of USVs to more hazardous and dynamic environments. This paper presents an adaptive path following control method using a retrofit adaptive tracking control technique with application to a USV with consideration of environmental disturbances (like winds, waves, and currents), while taking into account of the system constraints of USVs, including both turning features (turning rate limit and turning dynamics) and rudder operation constraints (rudder deflection and rate saturation, and its dynamics). In order to guarantee the satisfactory performance of the USV operating in a calm environment, a baseline state feedback tracking controller considering the characteristics of yaw rate and rudder operations, and USV steering and actuator dynamics is first designed. In the presence of time-varying environmental disturbances, a retrofit adaptive disturbance compensating control mechanism is then developed based on the disturbance amplitude estimated from an indirect adaptive disturbance estimator. Finally, a reconfigurable adaptive path following controller is synthesized by combining the baseline controller and the adaptive disturbance compensating mechanism for the proper operation of the USV in the presence of environmental disturbances, while the desired path is successfully followed by the USV within an acceptable deviation boundary and without violating constraints of turning rates as well as amplitude and rate of rudder deflections. To evaluate the effectiveness of the proposed path following control methodology, both numerical simulations on a nonlinear USV model and field experiments on a real-size USV are conducted.

This paper is concerned with the synchronization control problem for a class of discrete time-delay complex dynamical networks under a dynamic event-triggered mechanism. For the efficiency of energy utilization, we make the first attempt to introduce a dynamic event-triggering strategy into the design of synchronization controllers for complex dynamical networks. A new discrete-time version of the dynamic event-triggering mechanism is proposed in terms of the absolute errors between control input updates. By constructing an appropriate Lyapunov functional, the dynamics of each network node combined with the introduced event-triggering mechanism are first analyzed, and a sufficient condition is then provided under which the synchronization error dynamics is exponentially ultimately bounded. Subsequently, a set of the desired synchronization controllers is designed by solving a matrix inequality. Finally, a simulation example is provided to verify the effectiveness of the proposed dynamic event-triggered synchronization control scheme.

Research on screen content images (SCIs) attracts more attention as they are highly applied to image- and video-centric applications on mobile and other devices. It is important to develop an efficient image-quality assessment (IQA) method for SCIs because IQA can guide and optimize various image-processing methods for SCIs and improve user experience. In this paper, we propose a no-reference objective assessment model for SCIs including SCIs segmentation and the analysis of local and global perceptual feature representations. Since the human visual system is highly sensitive to sharp edges that are commonly encountered in SCIs, we utilize the variance of local standard deviation, which is a noise robust index to distinguish the sharp edge patches (SEPes) and non-SEPes of SCIs. For SEPes, we perform two kinds of feature extractions. First, the entropy and contrast features are extracted with a gray-level co-occurrence matrix, which are highly perceptive of microstructural change. Second, the local phase coherence is utilized to capture the loss in sharpness. Then, average pooling is adopted to fuse features obtained from all of the SEPes to represent the local features. We further combine local features with global features that are derived using the BRISQUE method as the hybrid region (HR)-based features. Finally, a regression module is learned using support vector regression to train the mapping function that maps HR-based features to subjective quality scores. Experimental results on the screen image-quality assessment database show that the proposed method can achieve better performance in visual-quality prediction for SCIs than the performance achieved by state-of-the-art methods.

This paper studies the compound learning control of disturbed uncertain strict-feedback systems. The design is using the dynamic surface control equipped with a novel learning scheme. This paper integrates the recently developed online recorded data-based neural learning with the nonlinear disturbance observer (DOB) to achieve good "understanding" of the system uncertainty including unknown dynamics and time-varying disturbance. With the proposed method to show how the neural networks and DOB are cooperating with each other, one indicator is constructed and included into the update law. The closed-loop system stability analysis is rigorously presented. Different kinds of disturbances are considered in a third-order system as simulation examples and the results confirm that the proposed method achieves higher tracking accuracy while the compound estimation is much more precise. The design is applied to the flexible hypersonic flight dynamics and a better tracking performance is obtained.

Caching content with fog computing at the edge nodes has been a promising alternative to mitigate burdens of backbone networks and improve mobile users' quality of experience in mobile social networks (MSNs). However, as edge node may be vulnerable due to the attacks from malicious users, the design of secure caching schemes for the fog/edge enabled MSNs becomes a new challenge. In this paper, to tackle the above problem, we propose a secure caching scheme for disaster backup in MSNs with fog computing. Specifically, to protect the privacy, a partitioning and scrambling method is first designed to encrypt the contents. Then, the encrypted contents are replicated to multiple replicates, where these replicates are delivered and stored in different servers. Based on the recovery time objective and content delivery latency, an auction game model is developed to determine the optimal servers, where both edge nodes and cloud servers can obtain the maximum utilities. Extensive simulations are conducted to show the effectiveness and reliability of the proposed scheme.

The importance of mobile robots for the nuclear disaster response has been realized after Fukushima Dai-ichi nuclear power plant accident. In this paper, we propose a tracked robot for rescue and search in nuclear environment. A gamma-ray irradiation test of the robot's control system is conducted, in order to evaluate the performance of the robot in nuclear environment. The parallel test method is described and the test result is reported.

To improve the mobility of locomotive devices on loose, sandy terrain, protrusions or convex patterns called lugs (i.e., grousers) are attached to the surface of a locomotive modulus. Following our previous study, in which the effects of angular speed, lug sinkage length, and soil cumulative deformation on lug-soil interaction forces during the fixed-axis rotational motion were experimentally confirmed, this study proposed an approximation equation to formulize the relationship among the normal force, lug sinkage length, and lug rotational angle. Moreover, the measured tangential force is compared with values calculated from a conventional tangential force model for discussing its accuracy of predicting the tangential force. Conclusions from this study present the fundamental principles for understanding the lug-soil interaction mechanics for a lug that is performing arbitrary planar motion on sandy terrain.

Mammalian sperms reorient against fluid flow in the female reproductive tract, known as rheotaxis. Compared to chemotaxis that provides short-distance guidance, rheotaxis provides long-distance guidance for a sperm to find the egg cell. However, only a low number of sperms are capable of rheotaxis and their tail behavior during reorientation is not yet known. We have developed an automated system to manipulate human sperm orientation in fluid flow and quantitatively reveal sperm behavior changes during rheotaxis. The system automatically detects multiple sperms, selects the sperm for analysis, controls fluid flow, and quantifies sperm tail behavior. Sperm head angle is used as feedback to control fluid flow and select reorienting sperms. High accuracy of head angle tracking and automated sperm selection enables the capturing of dynamic sperm turning behavior in a large sample size. Algorithms are developed to track sperm tail skeletons and quantify tail beating amplitude and asymmetry, based on which the first quantitative analysis of sperm tail behavior in rheotaxis is obtained. Experimental results reveal, for the first time, that the sperms that are capable of reorienting against fluid flow beat their tails more asymmetrically than those sperms that are unable to reorient against fluid flow while no significant difference was found in their tail beating amplitudes.

This paper contributes to the understanding of the fundamental properties of rectilinearly locomotion of an one-dimensional system travelling on the horizontal plane, where dry Coulomb friction acting between it and surface. We discuss an approximate steady-state motion on a simplified two-segment system, which is propelled by a periodic internal excitation. First, the analysis is presented of the sufficient and necessary conditions for the system to move from the state of rest. Then, the explicit equations to calculate the constant average velocity of the steady-state are found. In addition, the influences of different parameters on the average velocity of the steady-state motion are discussed. Finally, the obtained theoretical results are verified by the numerical simulations.

Central catadioptric omnidirectional images exhibit serious nonlinear distortions due to the involved quadratic mirrors. Therefore, features based on the conventional pin-hole model are hard to achieve satisfactory performances when directly applied to the distorted omnidirectional images. This paper analyzes the catadioptric geometry to facilitate modeling the nonlinear distortions of omnidirectional images. Different to the conventional imaging model, the prior information is considered in catadioptric system. A parameterized neighborhood mapping model is proposed to efficiently calculate the neighborhood of an object based on its measurable radial distance in the image plane. On the basis of the parameterized nonlinear model, a distortion-invariant fragment-based joint-feature mixture model of Gaussian is presented for human target tracking in omnidirectional vision. Under the framework of Gaussian Mixture Model, the problem of feature matching is converted into the feature clustering. The joint probability distribution of a joint-feature class is modeled by a mixture of Gaussian. A weight contribution mechanism is designed to flexibly weight the fragments contribution based on their responses, which leads to a robust tracking even under serious partial occlusion. Finally, experiments validate the advantage of the proposed algorithm over other conventional approaches. Catadioptric omnidirectional cameras have been widely used in robotics and surveillance fields for visual sensing due to its big field-of-view. However, conventional visual models use large-scale statistical sampling for feature extraction in catadioptric sensor, which may consume lot of computational cost. For practical applications, a parameterized model that can accurately and efficiently formulate distortion of catadioptric image is desirable. Integrating of the priori of system, a parameterized neighborhood model is presented to directly extract distorted image content in image, which can significantly improve the efficiency of algorithm. To robustly handle challenging occlusion in the distorted image, a flexible fragment-based joint-feature framework is presented for robust non-rigid human target tracking. Compared with the conventional tracking methods applied to catadioptric vision, the proposed tracking approaches leads to much better performance from the perspective of efficiency and robustness.

Central catadioptric omnidirectional images exhibit serious nonlinear distortions due to quadratic mirrors involved. Conventional visual features developed based on the perspective model are hard to achieve a satisfactory performance when directly applied to the distorted omnidirectional image. This paper presents a parameterized neighborhood model to efficiently calculate the adaptive neighborhood of an object based on the measurable radial distance in image plane. On the basis of the parameterized neighborhood model, a distortion invariant joint-feature framework implemented with contour-color fragment mixture model of Gaussian is proposed for visual tracking in catadioptric omnidirectional camera system. Under the framework of Gaussian Mixture Model, the problem of feature matching is converted into feature clustering. A weight contribution mechanism is presented to flexibly weight the fragments based on their responses, which makes the system robustly guided by limited visible fragments even when serious partial occlusion happens. The experiments validate the performance of the proposed algorithm.

Most mammalian cells (e.g., cancer cells and cardiomyocytes) adhere to a culturing surface. Compared to robotic injection of suspended cells (e.g., embryos and oocytes), fewer attempts were made to automate the injection of adherent cells due to their smaller size, highly irregular morphology, small thickness (a few micrometers thick), and large variations in thickness across cells. This paper presents a recently developed robotic system for automated microinjection of adherent cells. The system is embedded with several new capabilities: automatically locating micropipette tips; robustly detecting the contact of micropipette tip with cell culturing surface and directly with cell membrane; and precisely compensating for accumulative positioning errors. These new capabilities make it practical to perform adherent cell microinjection truly via computer mouse clicking in front of a computer monitor, on hundreds and thousands of cells per experiment (vs. a few to tens of cells as state-of-the-art). System operation speed, success rate, and cell viability rate were quantitatively evaluated based on robotic microinjection of over 4,000 cells. This paper also reports the use of the new robotic system to perform cell-cell communication studies using large sample sizes. The gap junction function in a cardiac muscle cell line (HL-1 cells), for the first time, was quantified with the system.

In robotic micromanipulation, end-effector tips must be first located under microscopy imaging before manipulation is performed. The tip of micromanipulation tools is typically a few micrometers in size and highly delicate. In all existing micromanipulation systems, the process of locating the end-effector tip is conducted by a skilled operator, and the automation of this task has not been attempted. This paper presents a technique to automatically locate end-effector tips. The technique consists of programmed sweeping patterns, motion history image end-effector detection, active contour to estimate end-effector positions, autofocusing and quad-tree search to locate an end-effector tip, and, finally, visual servoing to position the tip to the center of the field of view. Two types of micromanipulation tools (micropipette that represents single-ended tools and microgripper that represents multiended tools) were used in experiments for testing. Quantitative results are reported in the speed and success rate of the autolocating technique, based on over 500 trials. Furthermore, the effect of factors such as imaging mode and image processing parameter selections was also quantitatively discussed. Guidelines are provided for the implementation of the technique in order to achieve high efficiency and success rates.

Pick-place of single nanowires inside scanning electron microscopes (SEM) is useful for prototyping functional devices and characterizing nanowires's properties. Nanowire pick-place has been typically performed via teleoperation, which is time-consuming and highly skill-dependent. This paper presents an automated approach to the pick-place of single nanowires. Through SEM visual detection and vision-based motion control, the system automatically transferred individual silicon nanowires from their growth substrate to a microelectromechanical systems (MEMS) device that characterized the nanowires's electromechanical properties. The performance of the nanorobotic pick-up and placement procedures was experimentally quantified.

A novel control strategy of pan-tilt-zoom camera is described. Because the active camera is mounted on a moving autonomous helicopter in visual tracking system, and the tracked object is moving at same time, and there exists the vibration influence of the helicopter, image stabilization becomes poor, and all pixels are running. Therefore, a biomimetic control strategy of on-board pan-tilt-zoom camera is presented. In this paper, the biomimetic oculomotor control model is obtained based on physiological neural path of eye movement control. In order to validate the functions of the biomimetic control model, simulation experiments were done under the same condition as the physiological experiments in physiological researches. Then the biomimetic controller of onboard pan-tilt-zoom camera is developed. The results of flight tracking experiments show that the biomimetic controller can compensate the deflection caused by the flight platform, and enhance the visual tracking system performance.