In visual object tracking via unmanned aerial vehicle (UAV), discriminative correlation filtering (DCF) is one of the major methods owing to circulant samples which can be utilized not only for computing economically but also to hasten the optimization of filters. The universal DCF methods are seen as ridge regression models with some kinds of regularizations, which may likely result in tracking drift. Inspired by structured SVM, a generic framework that combines the squared hinge loss with spatial-temporal regularizations is advocated in this letter to distinguish the feature of targets from surrounding backgrounds and thus improve the robustness in tracking. Meanwhile, the standard squared norm penalty is turned into a group lasso penalty in the multichannel framework which enables filters with modest channel selection. The proposed adaptive spatial-temporal structured correlation filtering (ASTSCF) method has attained competitive results on major UAV tracking benchmarks.

Existing multimodal sentiment analysis models are effective at capturing sentiment commonalities across different modalities and discerning emotions. However, these models still face significant challenges when analyzing samples with sentiment polarity differences across modalities. Neural networks struggle to process such divergent sentiment samples, particularly when they are scarce within datasets. While larger datasets could help address this limitation, collecting and annotating them is resource-intensive. To overcome this challenge, we proposeTriagedMSA, a multimodal sentiment analysis model with triage capability. Our model introduces theSentiment Disagreement Triage Network, which identifies sentiment disagreement between modalities within a sample. This triage mechanism reduces mutual influence by learning to distinguish between samples of sentiment agreement and disagreement. To process these two sample types, we develop theSentiment Selection Attention Networkand theSentiment Commonality Attention Network, both of which enhance modality interaction learning. Furthermore, we propose theAdaptive Polarity Detection (APD) algorithm, which ensures the generalizability of our model across different datasets, regardless of whether unimodal labels are available. The APD algorithm adaptively determines sentiment polarity disagreement or agreement between modalities. We conduct experiments on three multimodal sentiment analysis datasets:CMU-MOSI,CMU-MOSEIandCH-SIMS.v2. The results demonstrate that our proposed methodology outperforms existing state-of-the-art approaches.

Video summarization involves creating a succinct overview by merging the valuable parts of a video. Existing video summarization methods approach this task as a problem of selecting keyframes by frame- and shot-level techniques with unimodal or bimodal information. Besides underestimated inter-relations between various configurations of modality embedding spaces, current methods are also limited in their ability to maintain the integrity of the semantics within the same summary segment. To address these issues, we propose a novel multimodal temporal interest detection network (MTIDNet), to learn multimodal features in the fine- and coarse-grained embedding spaces using the mutual cross fusion layer. Furthermore, we design a temporal interest detection network to predict the importance scores and boundaries of each temporal segment that possesses local and global features across shots. Experimental results demonstrate the effectiveness of our MTIDNet on challenging datasets (SumMe and TVSum).

Currently, discrete memristors are a focal point in the study of chaotic maps. Similar to memristors, memcapacitors-another type of memory circuit component-have not received widespread attention in the design of chaotic maps. In this article, we propose a 4-D memcapacitive hyperchaotic logistic map (4D-MHLM) by integrating memcapacitors with the logistic map. The dynamical behavior of the 4D-MHLM is analyzed using Lyapunov exponent analysis, and the impact of different parameters on system performance is discussed. The complexity of generating pseudo-random sequences with the 4D-MHLM is investigated through complexity analysis, including spectral entropy complexity and C0 complexity. Notably, attractor analysis reveals a unique phenomenon of infinite coexisting attractors within the 4D-MHLM. Finally, the chaotic attractor generated by the 4D-MHLM is successfully implemented on a hardware platform. Theoretical analysis and digital circuit implementation results indicate that the 4D-MHLM exhibits rich dynamical behavior and higher complexity, offering significant value for practical applications.

Chaos systems find extensive applications in cryptography and pseudorandom number generation due to their ability to generate pseudo-random signals. This paper focuses on enhancing the complexity of chaotic systems by introducing the memristor, a nonlinear component. We propose a novel map called the 2D memristive Cubic map (2D-MCM), which integrates the memristor with the Cubic map to create a discrete mapping. The 2D-MCM exhibits rich dynamical behavior and a broad parameter space. Notably, the 2D-MCM displays boosting bifurcation behavior. As the control parameters increase, the 2D-MCM demonstrates an expanded range of values, indicating its ability to generate a larger number of pseudo-random sequences. To validate its performance, we establish a hardware platform to physically capture the attractors of the 2D-MCM. To verify the performance of the 2D-MCM in generating pseudorandom sequences, we designed a video encryption algorithm based on the 2D-MCM. This algorithm selectively encrypts specific areas within the video, with correlation coefficients of the encrypted video in the horizontal, vertical, and diagonal directions being 0.0002, -0.0005, and 0.0004, respectively. Through simulation experiments and security analysis, we demonstrate that the 2D-MCM performs well in video encryption tasks.

To provide robust security measures for audio data during transmission, this article has developed a novel dual-channel audio encryption scheme based on chaos theory. Specifically, a new 2-D chaotic system called 2-D infinite collapse with logistic map (2-D-ICLM) is designed in this article. Compared to traditional 2-D chaotic systems, the 2-D-ICLM exhibits a larger parameter space, complexity, and richness, along with high unpredictability and randomness. These characteristics provide potential advantages and applications in the field of encryption. In the proposed encryption scheme, audio information serves as input to a hash function, which generates the initial values and parameters for the 2-D-ICLM, producing the keystream for the cryptographic system. Considering the correlation between the two channels of audio information, the information from the left and right channels is fused to create a new audio signal for encryption. Scrambling and diffusion processes are performed synchronously in the encryption algorithm, with the ciphertext information from the left channel utilized in the encryption of the right channel audio. The experimental results prove the effectiveness of the suggested audio encryption technique, effectively countering various conventional attack methods and showcasing its robust security features. The correlation of adjacent elements of ciphertext audio is 0.0013, the NSCR and UACI is around 0.9960 and 0.3345, and the efficiency is 0.0003 s/KB.

In the visual object tracking via unmanned aerial vehicle (UAV) the correlation filtering (CF) is one of the mainstream methods for the reason that optimizing filters by circulant samples facilitates the calculation. The prevalent discriminative CF (DCF) frameworks credited to a ridge regression model followed by various regularizations focus on regressing to a fixed Gaussian label, which maybe easily induce over-fitting. To these concerns, we integrate the hinge-squared loss (HSL) of structured SVM with the CF model so as to attain a dynamical label which regresses the difference between target and background samples and thus enhances the robustness in tracking. Moreover, we assume weighted channels to augment the discrimination of hinge-squared loss, thus indicating it has as good extensibility as the ridge regression in usual CF frameworks. The proposed method channel-weighted structured correlation filtering (CWSCF) has achieved excellent performance compared with other methods in mainstream UAV tracking benchmarks.

An asynchronous updating Boolean network is employed to simulate and analyze the gene expression of a particular tissue or species, revealing the life activity process from a system perspective to reveal the disease mechanism and treat the disease. Therefore, to ensure the safe transmission of the asynchronous updating Boolean network in the network, we designed an asynchronous updating Boolean network encryption algorithm based on chaos (ABNEA). First, a novel 2D chaotic system (2D-FPSM) is designed. This system has better performance than the classical 2D chaotic system. It is very suitable for cryptographic systems to generate key streams. Second, an encoding rule is designed to convert the asynchronous updating Boolean network to a Boolean matrix and propagate it on the network as an image. The receiver and sender jointly save the encoding rule. Last, to protect the safe propagation of the Boolean network matrix on the network, the method of synchronous scrambling-diffusion is adapted to encrypt the Boolean network matrix based on the 2D-FPSM. Simulation experiments and security analysis show that the average correlation of adjacent pixels of ciphertext are 0.0010, -0.0010, -0.0020, and the average information entropy is 7.9984. The ABNEA can complete the encryption tasks of asynchronously updating Boolean networks and exhibits good security characteristics.

Many practical applications suffer from the problem of imbalanced classification. The minority class has poor classification performance; on the other hand, its misclassification cost is high. One reason for classification difficulty is the intrinsic complicated distribution characteristics (CDCs) in imbalanced data itself. Classical oversampling method SMOTE generates synthetic minority class examples between neighbors, which is parameter dependent. Furthermore, due to blindness of neighbor selection, SMOTE suffers from overgeneralization in the minority class. To solve such problems, we propose an oversampling method, called extended natural neighbors based SMOTE (ExNN-SMOTE). In ExNN-SMOTE, neighbors are determined adaptively by capturing data distribution characteristics. Extensive experiments over synthetic and real datasets demonstrate the effectiveness of ExNN-SMOTE dealing with CDCs and the superiority of ExNN-SMOTE over other SMOTE-related methods.

Accurate bounding box estimation has recently attracted much attention in the tracking community because traditional multi-scale search strategies cannot estimate tight bounding boxes in many challenging scenarios involving changes to the target. A tracker capable of detecting target corners can flexibly adapt to such changes, but existing corner detection based tracking methods have not achieved adequate success. We analyze the reasons for their failure and propose a state-of-the-art tracker that performs correlation-guided attentional corner detection in two stages. First, a region of interest (RoI) is obtained by employing an efficient Siamese network to distinguish the target from the background. Second, a pixel-wise correlation-guided spatial attention module and a channel-wise correlation-guided channel attention module exploit the relationship between the target template and the RoI to highlight corner regions and enhance features of the RoI for corner detection. The correlation-guided attention modules improve the accuracy of corner detection, thus enabling accurate bounding box estimation. When trained on large-scale datasets using a novel RoI augmentation strategy, the performance of the proposed tracker, running at a high speed of 70 FPS, is comparable with that of state-of-the-art trackers in meeting five challenging performance benchmarks.

Multi-channel discriminative correlation filter (DCF) tracking methods have exhibited superior performance on several benchmarks. However, existing methods usually treat each channel of the features equally, whereas they pay less attention to the contribution of different channels. Different channels exhibit variant properties in the tracking process. A DCF learned with equally important channels is likely to be contaminated by the unreliable ones, which results in model degradation. To address this problem, we propose a new formulation for jointly learning the channel reliability and the correlation filters. The formulation is generic, and it can be combined with existing techniques in the DCF framework to further improve the performance. Our method can adaptively increase the impact of reliable channels and down-weight the corrupted ones. To solve the joint learning problem, we propose an optimization strategy that alternates between the correlation filters and the channel weights. Further, we prove the upper bound of the objective function and solve the channel weights efficiently. The joint learning strategy makes the correlation filters more discriminative and the channel weights more accurate. To verify the joint formulation, we propose a tracker based on the proposed formulation and the techniques used in the ECO tracker. We conduct extensive experiments to evaluate the proposed tracker on three benchmarks. The experimental results show that our formulation is effective and efficient, and that it performs favorably against other state-of-the-art trackers.

In this paper, we propose a novel channel pruning method for convolutional neural network (CNN)-based trackers. Pre-trained CNNs are widely used in visual tracking to obtain high-level representations of targets. However, most pre-trained CNNs are trained for other tasks (e.g., VGGNet is trained for image classification), and they require a considerable amount of time to generate features. First, we introduce a dimensionality reduction method considering the information amount and tracking errors to obtain good low-dimensionality features from the last convolutional layer for tracking. Then, a backward channel selection method is proposed to select representative channels layer by layer. In this process, we aim to minimize the target changes and maximize the loss of the background or other objects. Finally, we reconstruct the neural network weights to reduce the information loss of the target with one-shot learning. Experimental results on challenging benchmarks show that the proposed channel pruning method can enhance the tracking performance and reduce the computational requirements.

Robust object tracking and redetection require stably predicting the trajectory of the target object and recovering from tracking failure by quickly redetecting it when it is lost during long-term tracking. The locations of the target and the background are calculated relative to the region occupied by the object. The effect of tracking can be enhanced by isolating the target and the background, modeling and tracking them, respectively, and integrating their tracking results. In this study, we propose an approach that builds motion models for the target and its context. Tracking results from a target tracker and a context tracker are integrated through linear fusion to predict the position of the target. A kernelized correlation filter tracker is used to track the target in the predicted position. When the target is lost, it can be quickly recovered by searching in the given field of view using a target model built and updated through observation models that are constructed prior to the loss of the target. Our approach is not sensitive to the segmentation of the target and the context. The motion models and observation models of the target and the context work together in the tracking process, whereas the target model alone is involved in redetection. Experiments to test our proposed approach, which simultaneously models the target and its context, showed that it can effectively enhance the robustness of long-term tracking.

In this paper, we aim to construct a more reasonable and effective active sampling model, named as reinforcement uncertainty sampling with bag-of-visual-words (RUSB). Compared with traditional active sampling strategy based on uncertainty, both certainty metric and sample post-processing are introduced for better performance. The certainty metric is measured by the bag-of-visual-words (BoVW) classification model in order to entirely evaluate samples, and the post-processing module is driven by the Q-learning method to construct a compact and efficient training set for the BoVW module. The performance of BoVW is used to initialize and determine the status of the post-processing module during the process of iteration. Meanwhile, the weight of the measurement is associated with each iteration instead of being set manually. Experimental results on real world datasets show the effectiveness of the proposed framework.

In this paper, we address the problem of fast motion and bound effect about the popular high-speed correlation filters-based trackers. Such trackers are facing with the contradiction between extended detection region and reduced precision. To improve the robustness against fast motion, we firstly propose a tracker with extended region. In addition, in order for adapting different region sizes and target sizes, we introduce the target enhancement strategy to increase the effect of the target in learning a discriminative regression. Furthermore, a novel sample selection mechanism is established to drop the error samples generated by the circular structure of correlation filters. Our approach enlarges the detection region, improves the tracking accuracy and preserves the significant kernel structure of the correlation filters. Moreover, extensive experimental results in a recent benchmark datasets show that our proposed method have a promising performance compared to the state-of-art methods.

This paper proposes individualized cleaning for diverse imbalanced data sets. Existing techniques for data cleaning have difficulties with rare cases and outliers in minority class, especially, in highly unbalanced data. The drawback leads incomplete and imprecise examples to removal. In order to enhance the robustness and perform thorough data cleaning, we propose a weighted edited nearest neighbor (WENN), which detects and removes noisy examples from both classes intelligently. It considers individual characteristics of each imbalanced data, involving global class imbalance and local distribution. The main idea of the proposed method is to carefully put more focus on the majority class than the minority class during data cleaning. Extensive experiments over synthetic and real data clearly validate the superiority of our approach against other data cleaning methods.

Indoor scene classification is an important topic in computer vision, which is challenging due to the variability of decoration. Human vision system, on the other hand, is marvelous in adaptively recognizing scene categories with excellent performance and can be used for reference. Although bio-inspired computer vision algorithms have proven their effectiveness in classification applications, nowadays few researches on indoor scene classification algorithms attempt to model human vision system, restricting further improvement of performance and making it difficult to achieve adaptive scene understanding. To deal with this problem, in this paper we attempt to model the human vision system and achieve scene classification according to the cognitive theory, by dividing the problem into low-level objection annotation and high-level knowledge inference respectively on a macro perspective. Inspired by the biotical perception principle, a novel cognitive hybrid motivation framework is proposed, including empirical based annotation and inference over knowledge base, which is a simple yet effective framework based on techniques of object detection and classification. For a given indoor scene, objects of indoor scene are first annotated, then knowledge base is utilized to infer the category, reducing the effect of variable background. Environmental context is also utilized to assist classification. The proposed framework are evaluated on popular indoor scene dataset, and its effectiveness is proved by experimental results.

Uncertainty sampling is one of the most widely used strategy for pool-based active learning, however, there exists the problem that selected images do not reflect the desired training distribution and need additional labeling cost. To deal with this problem, from aspects of image classification and visual perception, we improve the traditional entropy-based sampling strategy by introducing bag-of-visual-words classification method and negative-accelerated learning principle from Rescorla-Wagner perceptive model. Differs from previous researches that treated sampling and classifying process separately, under the unified negative-accelerated learning model, we combine the two processes as a uniform model, named as negative-accelerated uncertainty sampling strategy with BoVW (NUSB) by proposing a new evolving sample selection measure, which takes category distribution into consideration. Classifier is trained to provide category distribution for the sampling process, reducing additional cost of annotation. Also, transfer test is utilized to prevent over-fitting and further evaluate the performance of different sampling strategies. Experimental results on real world datasets show that our active sampling framework outperforms both baseline active sampling strategies and state-of-the-art active learning based image classification method.

To improve the diagnostic accuracy of breast ultrasound classification, a novel computer-aided diagnosis (CAD) system based on B-Mode and color Doppler flow imaging is proposed. Several new features are modeled and extracted from the static images and color Doppler image sequences to study blood flow characteristics. Moreover, we proposed a novel classifier ensemble strategy for obtaining the benefit of mutual compensation of classifiers with different characteristics. Experimental results demonstrate that the proposed CAD system can improve the true-positive and decrease the false positive detection rate, which is useful for reducing the unnecessary biopsy and death rate.

Focusing on the video text detection, which is challenging and with wide potential applications, a novel stroke width feature is proposed and a system which detects text regions based on multi-scale corner detection is implemented in this paper. In our system, candidate text regions are generated by applying morphologic operation based on corner points detected in different scales, and non-text regions are filtered by combining proposed stroke width feature with some simple geometric properties. Moreover, there is a new multi-instance semi-supervised learning strategy being proposed in this paper considering the unknown contrast parameter in stroke width extraction. Experiments taken on video frames from different kinds of video shots prove that the proposed approach is both efficient and accurate for video text detection.

Dynamic weather conditions, which mainly include rain and snow, make prevailing algorithms for many applications of outdoor video analysis and computer vision lapse. To remove dynamic weather conditions, the authors propose a pixel‐wise framework combining a detection method with a removal approach. Dynamic weather conditions are detected by a strategy‐driven state transition, which integrates static initialisation using K ‐means clustering with dynamic maintenance of Gaussian mixture model. Moreover, a variable time window is presented for removal of rain and snow. Each component of the framework is addressed using detailed descriptions of corresponding algorithms. Experiments demonstrate the effectiveness of the method on detection and removal of dynamic weather conditions.

Breast ultrasound (BUS) image segmentation is a very challenge task because of the poor image quality. In this paper, we proposed a probability model-based method for the accurate and robust segmentation for low quality medical images. It combines the spatial priori knowledge with the frequency constraints under the maximum a posteriori probability with markov random field (MAP-MRF) segmentation frameworks. The spatial constraints model the global location, object pose and the appearance, and the objective boundary is constrained in the frequency domain via modeling the phase feature and the zero crossing feature of the wavelet coefficients. The proposed method is applied to a breast ultrasound database with 131 cases, and its performance is evaluated by area error metrics and boundary error metrics. In comparing with the state of the art, our method is more accurate and robust in segmenting breast ultrasound images.

A background estimation system, which integrates pixel-level features with a region-level one and combines short-term and long-term analysis of videos in outdoor illumination variations, is proposed for accurate foreground detection. Firstly, we discuss autocorrelation-based features for identification of the presence of foreground and outdoor illumination variations in short-term sequences, and propose an adaptive threshold learning approach insensitive to inner-pixel fast illumination variation based on histograms of intensity differences between successive frames. Then, we employ a pixel-wise rapid autoregressive model against gradual illumination change for background estimation in long-term sequence. Finally, we devise a texture measure to eliminate the regional effect of fast illumination variation. The effectiveness of our system is demonstrated using experiments on foreground detection in videos with various illumination changes.

Evaluation of different weather conditions provides a first step support for many different applications of outdoor video analysis and computer vision. In this paper, a simple but effective classification method on visual effects of different weather conditions is proposed. Due to the complex manifestations of weather conditions, we firstly provide a two-stage classification scheme. Then, we extract spatio-temporal and chromatic features to represent different weather situations. Using these features, we develop a classifier based on an experiential decision binary tree associated with C-SVM. The experimental results of classification on our newly-built video dataset indicate the effectiveness of our method.

This letter presents a novel banknote image processing system that includes banknote recognition, general attrition evaluation, and feature identification, in which the phenomenon of banknote deterioration is discussed in detail for the first time. To compare the sensed image with its reference, a banknote image registration algorithm based on the free-form deformations model (FFD) is proposed, in which a homogeneity-based banknote deterioration energy (BDE) is used as the cost function. The proposed algorithms lead to a high capacity for low-quality banknote processing and greatly decrease the false reject rate.

This paper presents a computer vision system for tracking high-speed non-rigid skaters over a large playing area in short track speeding skating competitions. The outputs of the tracking system are spatio-temporal trajectories of the players which can be further processed and analyzed by sport experts. Given very fast and non-smooth camera motions to capture highly complex and dynamic scenes of skating, tracking amorphous skaters should be a challenging task. We propose a new method of (1) automatically computing the transformation matrices to map each frame of the imagery to the globally consistent model of the rink and (2) incorporating the hierarchical model based on the contextual knowledge and multiple cues into the unscented Kalman filter to improve the tracking performance when occlusion occurs. Experimental results show that the proposed algorithm is very efficient and effective on video recorded live by the authors in the World Short Track Speed Skating Championships.