The eyes are the windows for humans to interact with the outside world and a prerequisite for the promotion of many applications. In gaze estimation, although the full-face image encompasses the eyes, when extracting gaze features, the network often comprehensively considers the information characteristics of the entire face and its interrelationships. This may lead to neglecting the delicate gaze-related information contained in the eye images. To improve the accuracy of gaze direction prediction, this study proposes a novel multi-stream binocular gaze estimation model named CO-Net, which takes facial and binocular images as input. First, we use facial and eye features to preliminarily estimate a primary gaze direction. Considering the differences that arise when observing objects with both eyes in an unconstrained environment, we utilize this characteristic to purify the gaze features. Finally, we use the purified features to linearly optimize the primary gaze direction. Experimental results demonstrate that our method reduces the mean angular error to 3.63 ∘ on MPIIGaze and 5.9 ∘ on RT-GENE, representing improvements of 7% and 10%, respectively, over the previous state-of-the-art.

Retinal microaneurysms (MA) are the earliest clinical indicators of diabetic retinopathy (DR), and their detection is crucial for early diagnosis of DR and preventing vision loss. However, accurately detecting MA is extremely challenging. Existing object detection models fail to accurately detect MA and suffer from severe under-detection, because MA are characterized by tiny size and low contrast with the fundus background. To address this challenge, we have developed a two-stage learning model, MADET, which is proposed. MADET consists of two models: an enhanced feature attention fusion model (EFU-Net) for segmentation and a residual multi-scale model (RMSNet) for classification. EFU-Net integrates multi-attention gate fusion (MAGF) modules and residual RFCBAM convolution (ResRF-conv) into the standard U-Net architecture to improve the encoder-decoder connection and enhance the model’s ability to acquire strong context features, resulting in better segmenting of the MA regions. RMSNet considers the difference of feature information between different scales to eliminate non-MA regions, thereby achieving more accurate MA detection. MADET achieved a Free-response Receiver Operating Characteristic (FROC) score of 0.625 on the E-Ophtha-MA dataset (148 images), showing an improvement of 0.034 over recent methods. On the IDRiD dataset (81 images), it achieved a score of 0.497, which is 0.021 higher than the latest comparable approach. These results indicate that MADET achieves competitive or near-best performance compared to existing state-of-the-art techniques.

This article proposes a novel distributed neuro-adaptive 3-D formation tracking control framework of multiple autonomous underwater vehicles (multi-AUVs) subject to marine environmental disturbances. On the one hand, we assume that all AUVs can obtain the real-time states. By introducing a series of variable transformations, the multi-AUV system is transformed into an underactuated nonlinear system with virtual control input. Radial basis function neural networks (RBFNNs), whose weights are updated online, are utilized to approximate nonlinear functions. Considering environmental disturbances, a virtual controller is designed such that all AUVs track the leader while maintaining the desired formation geometry. Then, the actual controller is given as an adaptive form according to the virtual control signals. On the other hand, we assume that all AUVs can only obtain the sampling states of themselves and their neighbors under the predefined event-triggered conditions. Multi-AUV system is transformed into a second-order system with complex nonlinear dynamics, then their states are reconstructed via a neuro-adaptive state observer using sampling states, and a virtual controller is proposed such that all AUVs track the leader while maintaining the desired formation geometry under local communication with no Zeno behavior. Finally, numerical simulations are carried out to demonstrate the effectiveness of the proposed control design.

Pre-trained Vision-Language Models (VLMs) such as CLIP have shown excellent generalization abilities. However, adapting these large-scale models to downstream tasks while preserving their generalization capabilities remains challenging. Although prompt learning methods have shown promise, they suffer from two fundamental bottlenecks that limit generalization: (a) modality isolation, and (b) hierarchical semantic decay. To address these limitations, we propose HiCroPL, a Hierarchical Cross-modal Prompt Learning framework that establishes bidirectional knowledge flow between text and vision modalities, enabling them to refine their semantics mutually. HiCroPL routes knowledge flows by leveraging the complementary strengths of text and vision. In early layers, text prompts inject relatively clear semantics into visual prompts through a hierarchical knowledge mapper, enhancing the representation of low-level visual semantics. In later layers, visual prompts encoding specific task-relevant objects flow back to refine text prompts, enabling deeper alignment. Crucially, our hierarchical knowledge mapper allows representations at multi-scales to be fused, ensuring that deeper representations retain transferable shallow semantics thereby enhancing generalization. We further introduce a lightweight layer-specific knowledge proxy to enable efficient cross-modal interactions. Extensive evaluations across four tasks demonstrate HiCroPL's superior performance, achieving state-of-the-art results on 11 benchmarks with significant improvements. Code is available at: https://github.com/zzeoZheng/HiCroPL.

Live fish possess the ability to modulate their body stiffness to achieve diverse swimming characteristics, a feature that is largely absent in existing robotic fish designs. Most robotic fish are constrained by rigid or fixed-stiffness bodies, which limit their flexibility, axial modulation capabilities, and ability to navigate confined spaces. This paper presents a novel soft robotic fish capable of stiffness modulation and earthworm-inspired wriggling locomotion. The design incorporates a pneumatic stiffness modulation mechanism, a cable-driven actuation system, and two passive flapping foils near the caudal fin. Two distinct swimming modes are demonstrated: a body and/or caudal fin (BCF) mode with active stiffness modulation and an earthworm-inspired wriggling mode. Experimental results validate the effectiveness of the proposed design in both swimming modes. This work advances the development of soft robotic fish by introducing innovative structural and actuation mechanisms, enhancing flexibility and adaptability for future applications in underwater exploration and related fields.

Aviation accidents are frequently related to pilots’ operations, especially during a landing phase. Therefore, accurately evaluating a pilot’s performance during this phase is crucial for minimizing landing risks. Traditional assessment methods, however, primarily focus on discrete monitoring points, failing to capture the continuous and dynamic nature of a pilot’s performance throughout the entire landing phase. To address this issue, we propose a Flight Process Importance (FPI) assessment framework that precisely determines accurate landing timing and captures the diverse operational characteristics of pilots. It consists of two components: Time-varying Importance Coefficient (TIC) and Pilot Characteristics Matrix (PCM). TIC develops a Spatio-Temporally Consistent Attention Network (STCAN) to classify Quick Access Recorder (QAR) data for anomalous event detection. It then determines the importance of different periods during the landing process by analyzing the STCAN model’s response to the data in an interpretable manner. PCM generates a parameter matrix for each flight by deriving the ideal intervals of various parameters through the interquartile range. This matrix is used to identify the duration and intensity of anomalies in operations across different pilots. By integrating TIC and PCM, our framework computes an evaluation matrix for each flight, quantifying the operational risk factors associated with pilots. Experimental results indicate that STCAN significantly surpasses other algorithms on QAR data. FPI provides a more precise and comprehensive assessment of a pilot’s performance. In particular, our findings highlight that the 10 seconds before landing to the touchdown are the most critical period of airplane landing.

Current multicast protocols assume prior knowledge of destination locations, leading to excessive message transmissions, especially in sensor networks with mobile sinks. Quorum-based systems, known for efficient information dissemination, have not been explored in the context of multicast. To address this gap, we propose ORM (Orthogonal Rendezvous Multicast), a quorum-based multicast protocol that solves location dissemination and data multicasting as a whole. The protocol utilizes a simple geometric principle to disseminate the information from sinks and sources. By exploiting the footprint of disseminated information, ORM transforms the underlying infrastructure to reveal the hidden multicasting structure. Furthermore, two efficient algorithms are proposed to fetch the optimal efficient multicasting paths based on the built infrastructure. ORM was implemented in the NS-2 simulator. Our detailed simulations demonstrate that ORM achieves significantly enhanced scalability in terms of message overhead and low cost without the burden of complex state maintenance.

Detecting objects with inherent orientations has numerous applications in the context of livestock reproduction. In this new scenario, the inherent orientations in the range [0, 2π) of target objects are detected alongside their bounding boxes to produce real-oriented bounding boxes. Due to the 0-to-2π orientation angle, however, traditional oriented IOU algorithms and mean-squared error on orientation are incapable of determining the similarity between two real-oriented bounding boxes. Therefore, we propose an orientation-sensitive pseudo-IOU algorithm and periodic loss of orientation to adapt to the new scenario of oriented object detection. The detection workflow is implemented by extending the YOLO head. Experimental results indicate that the detector equipped with the minor-angle-based pseudo-IOU (MinorAngle) and cosine form of orientation-related loss (LDirectCOS) presents the best performance and outperforms the state-of-the-art techniques. We hope that these discoveries will stimulate the community.

Abstract This paper focuses on the intelligent detection of prohibited items in X‐ray images during the security checking process. An intelligent semantic segmentation model of prohibited items in X‐ray images is proposed based on the attention‐based object localization method. Based on the pre‐trained CNN classification framework, the attention mechanism can map the high‐layer semantic information of objects into the input space, while generating energy saliency maps to locate the prohibited items. In order to make the obtained attention maps discriminative, the lateral and contrastive inhibition strategies are introduced and combined together which can highlight the responses of activated neurons. Under the guidance of attention responses, two traditional image segmentation algorithms are employed to achieve the semantic segmentation results for the prohibited items detection in X‐ray images. The proposed semantic segmentation model relies on weakly supervised learning mechanism, and only depends on the category labels of prohibited items, which greatly avoids the work cost of data semantic annotation. The experimental results based on the public SIXray baseline and the self‐built X‐ray image database demonstrate the proposed method can achieve about 65% IoU localization precise averagely. In addition, comparison experiments were carried out with the state‐of‐the‐arts and ablation experiments to verify the effectiveness of the proposed model.

Existing studies on knowledge distillation typically focus on teacher-centered methods, in which the teacher network is trained according to its own standards before transferring the learned knowledge to a student one. However, due to differences in network structure between the teacher and the student, the knowledge learned by the former may not be desired by the latter. Inspired by human educational wisdom, this paper proposes a Student-Centered Distillation (SCD) method that enables the teacher network to adjust its knowledge transfer according to the student network's needs. We implemented SCD based on various human educational wisdom, e.g., the teacher network identified and learned the knowledge desired by the student network on the validation set, and then transferred it to the latter through the training set. To address the problems of current deficiency knowledge, hard sample learning and knowledge forgetting faced by a student network in the learning process, we introduce and improve Proportional-Integral-Derivative (PID) algorithms from automation fields to make them effective in identifying the current knowledge required by the student network. Furthermore, we propose a curriculum learning-based fuzzy strategy and apply it to the proposed PID control algorithm, such that the student network in SCD can actively pay attention to the learning of challenging samples after with certain knowledge. The overall performance of SCD is verified in multiple tasks by comparing it with state-of-the-art ones. Experimental results show that our student-centered distillation method outperforms existing teacher-centered ones.

Knowledge distillation is a deep learning method that mimics the way that humans teach, i.e., a teacher network is used to guide the training of a student one. Knowledge distillation can generate an efficient student network to facilitate deployment in resource-constrained edge computing devices. Existing studies have typically mined knowledge from a teacher network and transferred it to a student one. The latter can only passively receive knowledge but cannot understand how the former acquires the knowledge, thus limiting the latter’s performance improvement. Inspired by the old Chinese saying “Give a man a fish and you feed him for a day; teach a man how to fish and you feed him for a lifetime,” this work proposes a Skill-transferring Knowledge Distillation (SKD) method to boost a student network’s ability to create new valuable knowledge. SKD consists of two main meta-learning networks: Teacher Behavior Teaching and Teacher Experience Teaching. The former captures the process of a teacher network’s learning behavior in the hidden layers and can predict the teacher network’s subsequent behavior based on previous ones. The latter models the optimal empirical knowledge of a teacher network’s output layer at each learning stage. With their help, a teacher network can provide its actions to a student one in the subsequent behavior and its optimal empirical knowledge in the current stage. SKD’s performance is verified through its application to multiple object recognition tasks and comparison with the state of the art.

As emerging memory technologies (e.g., non-volatile memory (NVM)) coming out and machine learning algorithms successfully applying to different fields, the potentials of cache replacement policy for NVM-based systems with the integration of machine learning algorithms are worthy of being exploited to improve the performance of computer systems. In this work, we proposed a machine learning based cache replacement algorithm, named ExpCache, to improve the system performance with NVM as the main memory. By considering the non-volatility characteristic of the NVM devices, we split the whole NVM into two caches, including a read cache and a write cache, for retaining different types of requests. The pages in each cache are managed by both LRU and LFU policies for balancing the recency and frequency of workloads. The online Expert machine learning algorithm is responsible for selecting a proper policy to evict a page from one of the caches based on the access patterns of workloads. In experimental results, the proposed ExpCache outperforms previous studies in terms of hit ratio and the number of dirty pages written back to storage.

The distribution of feature vectors plays a critical role in image registration. In this letter, we propose a novel approach for remote sensing image registration based on marginal distribution adaptation. First, we map the feature vectors of reference and sensed images into a latent space. Transfer component analysis (TCA) is employed to compute the transformation matrix by minimizing the maximum mean discrepancy (MMD). Then, we match feature vectors in the latent space where their marginal distributions are similar, which can increase correct correspondences and enhance registration accuracy. Finally, we test the proposed algorithm on ten real image pairs. The effectiveness and efficiency of our approach are verified by experimental results.

Hybrid storage systems are widely used in big data fields to balance system performance and cost. However, due to a poor understanding of the characteristics of database block requests, past studies in this area cannot fully utilize the performance gain from emerging storage devices. This study presents a hybrid storage database system, called HeuristicDB, which uses an emerging non-volatile memory (NVM) block device as an extension of the database buffer pool. To consider the unique performance behaviors of NVM block devices and the block-level characteristics of database requests, a set of heuristic rules that associate database (block) requests with the appropriate quality of service for the purpose of caching priority are proposed. Using online analytical processing (OLAP) and online transactional processing (OLTP) benchmarks, both trace-based examination and system implementation on MySQL are carried out to evaluate the effectiveness of the proposed design. The experimental results indicate that HeuristicDB provides up to 75% higher performance and migrates 18X fewer data between storage and the NVM block device than existing systems.

Solid state drive (SSD) as a fast storage device has been playing an important role across many applications from mobile computing to large distributed systems in recent years. However, the performance of the SSD can be degraded tremendously due to the intrinsic properties of NAND-based flash memory including limited erase cycles and asymmetric write and erase operations. Previous works separated hot/cold data into different blocks in order to improve SSD performance. “Hotness” is typically defined as the cumulative update frequencies of pages. However, we believe that an additional new parameter, average update time interval, should also be considered into the “hotness” definition associated with the update frequency. Moreover, to adaptively classify hot/cold data, a machine learning algorithm is applied to better accommodate the dynamically changed I/O access patterns of traces. In this paper, a machine learning (ML) based SSD management called HAML-SSD is proposed. The purpose of applying the ML algorithm is to dynamically cluster the data with similar “hotness” based on a new definition of “hotness”. Thus, a two-dimension clustering algorithm is used for storing the pages categorized into the same cluster within the same block. Moreover, to obtain reasonable training time, a specific hardware component called HAML-unit is designed in the SSD. Finally, the experimental results indicate that the HAML-SSD decreases the response time around 26.3% - 57.7% compared to previous works with the evaluation of real traces.

Extracting the domain-invariant features provides an important intuition for unsupervised domain adaptation. Due to the unavailable target labels, it is difficult to guarantee that the learned domain-invariant features are good for target instances classification. In this paper, we extend the classic geodesic flow kernel method by leveraging the pseudo labels during the training process to learn a discriminative geodesic flow kernel for unsupervised domain adaptation. Specifically, the proposed method alternately discovers the pseudo target labels and builds the geodesic flow from a discriminative source subspace to another ‘discriminative’ target subspace. More specially, the pseudo target labels are inferred via the learned kernel based on an easy yet effective label propagation strategy. Hence, the proposed method not only holds the property of domain-invariance, but also maximizes the consistency between pseudo label structure and data structure. Experimental results illustrate that the proposed method outperforms the state-of-the-art unsupervised domain adaptation methods for object recognition and sentiment analysis.

BACKGROUND: Electronic Medical Record (EMR) comprises patients' medical information gathered by medical stuff for providing better health care. Named Entity Recognition (NER) is a sub-field of information extraction aimed at identifying specific entity terms such as disease, test, symptom, genes etc. NER can be a relief for healthcare providers and medical specialists to extract useful information automatically and avoid unnecessary and unrelated information in EMR. However, limited resources of available EMR pose a great challenge for mining entity terms. Therefore, a multitask bi-directional RNN model is proposed here as a potential solution of data augmentation to enhance NER performance with limited data. METHODS: A multitask bi-directional RNN model is proposed for extracting entity terms from Chinese EMR. The proposed model can be divided into a shared layer and a task specific layer. Firstly, vector representation of each word is obtained as a concatenation of word embedding and character embedding. Then Bi-directional RNN is used to extract context information from sentence. After that, all these layers are shared by two different task layers, namely the parts-of-speech tagging task layer and the named entity recognition task layer. These two tasks layers are trained alternatively so that the knowledge learned from named entity recognition task can be enhanced by the knowledge gained from parts-of-speech tagging task. RESULTS: The performance of our proposed model has been evaluated in terms of micro average F-score, macro average F-score and accuracy. It is observed that the proposed model outperforms the baseline model in all cases. For instance, experimental results conducted on the discharge summaries show that the micro average F-score and the macro average F-score are improved by 2.41% point and 4.16% point, respectively, and the overall accuracy is improved by 5.66% point. CONCLUSIONS: In this paper, a novel multitask bi-directional RNN model is proposed for improving the performance of named entity recognition in EMR. Evaluation results using real datasets demonstrate the effectiveness of the proposed model.

In this paper, a transfer bi-directional recurrent neural networks (RNN) is proposed for named entity recognition (NER) in Chinese electronic medical records (EMRs) that aims to extract medical knowledge such as phrases recording diseases and treatments automatically. We propose a two-step procedure where the first step is to train a shallow bi-directional RNN in the general domain, and the second step is to transfer knowledge from the general domain to train a deeper bi-directional RNN for recognizing medical concepts from Chinese EMRs. Specifically, this is achieved by initializing the shallow parts of the deeper network in the second step with parameter weights from the bi-directional RNN trained in the first step. Then the deeper networks are re-trained on the Chinese EMRs. Experimental results show that NER performances are improved by the transferred knowledge significantly.

Similarity between words is becoming a generic problem for many applications of computational linguistics, and computing word similarities is determined by word representations. Inspired by the analogies between words and lymphocytes, a lymphocyte-style word representation is proposed. The word representation is built on the basis of dependency syntax of sentences and represent word context as head properties and dependent properties of the word. Lymphocyte-style word representations are evaluated by computing the similarities between words, and experiments are conducted on the Penn Chinese Treebank 5.1. Experimental results indicate that the proposed word representations are effective.

In this paper, a new finger-vein vector field estimation method is proposed for the primitive finger-vein feature representation. First, a set of spatial curve filters (SCFs) is built based on a variable curve model in curvature and orientation. To make SCFs adaptive to vein-width variations, a curve length field (CLF) estimation method is then proposed. Next, with the CLF constrain, a vein vector field is established for finger-vein feature description. Finally, finger-vein matching performance is evaluated using Phase-Only-Correlation (POC) measure. Experimental results show that the proposed method is highly powerful in improving finger-vein matching accuracy.

In this paper, we consider the double-blindness problem in large-scale wireless sensor networks (WSNs) with mobile sinks, where the mobile sink(s) and data do not know the locations of each other a priori. We first propose a random line walk mechanism for message forwarding and based on this forwarding mechanism we further propose an efficient data discovery scheme called Double Cross to address the double-blindness problem. Double Cross exploits a simple geometric property of a planar, i.e., for a couple of pairs of orthogonal lines in a planar, the probability that they intersect within the planar is larger than 99%. However, it does not depend on the geographic location or directional information of a node, which is difficult to obtain in such networks. Instead, each sensor only needs to know the distances between neighbor nodes within its transmission range. Analytical and simulation results show that Double Cross can achieve a high successful discovery rate with low energy consumption.

In order to achieve accurate recognition of human finger vein (FV), this paper addresses the problems of finger vein localization and vein feature extraction. An inherent physical property of human fingers is used to localize the region of interest (ROI) of vein images as well as removing uninformative vein imagery based on the inter-phalangeal joint prior. In addition, vein images are characterized as a series of energy features through steerable filters. Experimental results show the promising performance of the proposed algorithm for human vein identification.

Finger-vein recognition has been considered as one of the most convenient and effective biometric ways for personal identification. Extracting vein characteristics is crucial for finger-vein classification. However, the finger-vein extraction results always are greatly sensitive to noises due to the low contrast of finger-vein images. To robustly exploit real vein information, this paper proposes a novel method of finger-vein enhancement based on multi-channel Gabor filters. Firstly, multi-channel Gabor filters are used to prominently protrude vein vessel information with variances in widths and orientations in images. The vein information in different scales and orientations of Gabor filters is then combined together to generate an enhanced finger-vein image using a reconstruction rule. Experimental results show that the proposed method is capable of enhancing finger-vein images effectively and reliably.

A novel method of exploiting finger-vein features for personal identification is proposed in this paper. First, the circular Gabor filter is used to enhance finger-vein region in an image. Then, image segmentation is implemented for finger-vein network extraction. To obtain the finger-vein skeleton, thinning operation is performed accordingly. Based on the extracted network and skeleton, finger-vein features on local moments, topological structure and statistics are exploited respectively. Finally, a fusion scheme is adopted for decision making. The experimental results show that the proposed method has good performance in personal identification.

Illumination always affects image quality seriously in practice. To weaken illumination effect on image quality, this paper proposes an adaptive gamma correction method. First, a mapping between pixel and gamma values is built. The gamma values are then revised using two non-linear functions to prevent image distortion. Experimental results demonstrate that the proposed method performs better in readjusting image illumination condition and improving image quality.

This paper presents an adaptive approach to skin detection. First, we propose a nonlinear relationship among R, G and B components and use a closed curve to identify the skin cluster region. Then, a split machine is designed that aids the extraction of the pixels with similar low-level features from images. Finally, a nonlinear skin color classifier with an adaptive threshold is developed by analyzing the properties of the extracted pixels in the HSL, YCbCr, YUV and YIQ color spaces. Experimental results show that our proposed method works very well in skin detection.