High time complexity is one of the biggest challenges faced by k-Nearest Neighbors (kNN). Although current classical and quantum kNN algorithms have made some improvements, they still have a speed bottleneck when facing large amounts of data. To address this issue, we propose an innovative algorithm called Granular-Ball based Quantum kNN(GB-QkNN). This approach achieves higher efficiency by first employing granular-balls, which reduces the data size needed to processed. The search process is then accelerated by adopting a Hierarchical Navigable Small World (HNSW) method. Moreover, we optimize the time-consuming steps, such as distance calculation, of the HNSW via quantization, further reducing the time complexity of the construct and search process. By combining the use of granular-balls and quantization of the HNSW method, our approach manages to take advantage of these treatments and significantly reduces the time complexity of the kNN-like algorithms, as revealed by a comprehensive complexity analysis.

Federated Learning (FL) has emerged as a promising collaborative learning paradigm that enables to train machine learning models across decentralized devices, while keeping the training data localized to preserve user privacy. However, the heterogeneity in both decentralized training data and distributed computing resources has posed significant challenges to the design of effective and efficient FL schemes. Most existing solutions either focus on tackling a single type of heterogeneity, or are unable to fully support model heterogeneity with low communication overhead, fast convergence, and good interpretability. In this paper, we present CloREF, a novel rule-based collaborative learning framework that allows devices in FL to use completely different local learning models to cater to both data and resource heterogeneity. In CloREF, each rule is represented as a linear model, which provides good interpretability. Each participating device chooses a local model and trains it using its local data. The decision boundary of each trained local model is then approximated using a set of rules, which effectively bridges the gap arising from model heterogeneity. All participating devices collaborate to select the optimal set of rules as the global model, employing evolutionary optimization to effectively fuse the knowledge acquired from all local models. Experimental results on both synthesized and real-world datasets demonstrate that the rules generated by our proposed method can mimic the behaviors of various learning models with high fidelity ($\gt $0.95 in most tests), and CloREF gives competitive performance in accuracy, AUC, and communication overhead, compared with both the best-performing model trained centrally and several state-of-the-art model-heterogeneous federated learning schemes.

Semantic segmentation based on deep learning methods can attain appealing accuracy provided large amounts of annotated samples. However, it remains a challenging task when only limited labelled data are available, which is especially common in medical imaging. In this paper, we propose to use Leaking GAN, a GAN-based semi-supervised architecture for retina vessel semantic segmentation. Our key idea is to pollute the discriminator by leaking information from the generator. This leads to more moderate generations that benefit the training of GAN. As a result, the unlabelled examples can be better utilized to boost the learning of the discriminator, which eventually leads to stronger classification performance. In addition, to overcome the variations in medical images, the mean-teacher mechanism is utilized as an auxiliary regularization of the discriminator. Further, we modify the focal loss to fit it as the consistency objective for mean-teacher regularizer. Extensive experiments demonstrate that the Leaking GAN framework achieves competitive performance compared to the state-of-the-art methods when evaluated on benchmark datasets including DRIVE, STARE and CHASE_DB1, using as few as 8 labelled images in the semi-supervised setting. It also outperforms existing algorithms on cross-domain segmentation tasks.

We propose a cross-domain latent modulation mechanism within a variational autoencoders (VAE) framework to enable improved transfer learning. Our key idea is to procure deep representations from one data domain and use it as perturbation to the reparameterization of the latent variable in another domain. Specifically, deep representations of the source and target domains are first extracted by a unified inference model and aligned by employing gradient reversal. Second, the learned deep representations are cross-modulated to the latent encoding of the alternate domain. The consistency between the reconstruction from the modulated latent encoding and the generation using deep representation samples is then enforced in order to produce inter-class alignment in the latent space. We apply the proposed model to a number of transfer learning tasks including unsupervised domain adaptation and image-to-image translation. Experimental results show that our model gives competitive performance.

Surrogate-assisted evolutionary algorithms (SAEAs) have become one popular method to solve complex and computationally expensive optimization problems. However, most existing SAEAs suffer from performance degradation with the dimensionality increasing. To solve this issue, this article proposes a classifier-assisted level-based learning swarm optimizer on the basis of the level-based learning swarm optimizer (LLSO) and the gradient boosting classifier (GBC) to improve the robustness and scalability of SAEAs. Particularly, the level-based learning strategy in LLSO has a tight correspondence with the classification characteristic by setting the number of levels in LLSO to be the same as the number of classes in GBC. Together, the classification results feedback the distribution of promising candidates to accelerate the evolution of the optimizer, while the evolved population helps to improve the accuracy of the classifier. To select informative and valuable candidates for real evaluations, we devise an${L}1$-exploitation strategy to extensively exploit promising areas. Then, the candidate selection is conducted between the predicted${L}1$offspring and the already real-evaluated${L}1$individuals based on their Euclidean distances. Extensive experiments on commonly used benchmark functions demonstrate that the proposed optimizer can achieve competitive or better performance with a very small training dataset compared with three state-of-the-art SAEAs.

Effective analysis of EEG signals remains a challenging task. So far, the analysis and conditioning of EEG have largely remained gender-neutral. This paper explores the evidence of gender effects on EEG signals and confirms the generality of these effects by achieving successful gender prediction through EEG signals. Specifically, we propose a novel statistical feature representation that captures the gender discrepancy, and design a customized classification ensemble framework to overcome the non-stationary characteristics in EEG signals, utilizing findings obtained through several machine learning techniques including clustering, visualization, and metric learning. Apart from gender differentiation, the age effect on EEG gender patterns is also revealed.

The physical constraints in wireless sensor networks can often be a significant compromising factor to their effective performance. To improve energy efficiency, there has been increasing research efforts that investigate employing predictive sensing mechanisms where adaptive clustering schemes and intelligent predictive algorithms are incorporated in data acquisition and transmission. There seems, however, little treatment in the literature on performance modelling of the predictive sensing scenarios. In this paper we examine a few light-weight data prediction algorithms operating in a synchronized cluster setting on some real-world environmental sensing data streams. Besides their capabilities in transmission reduction, we look at the relevant traffic modelling of the sensing scheme. Some empirical results obtained from real-world environment monitoring data are presented and discussed.

Virtual machine placement (VMP) and energy efficiency are significant topics in cloud computing research. In this paper, evolutionary computing is applied to VMP to minimize the number of active physical servers, so as to schedule underutilized servers to save energy. Inspired by the promising performance of the ant colony system (ACS) algorithm for combinatorial problems, an ACS-based approach is developed to achieve the VMP goal. Coupled with order exchange and migration (OEM) local search techniques, the resultant algorithm is termed an OEMACS. It effectively minimizes the number of active servers used for the assignment of virtual machines (VMs) from a global optimization perspective through a novel strategy for pheromone deposition which guides the artificial ants toward promising solutions that group candidate VMs together. The OEMACS is applied to a variety of VMP problems with differing VM sizes in cloud environments of homogenous and heterogeneous servers. The results show that the OEMACS generally outperforms conventional heuristic and other evolutionary-based approaches, especially on VMP with bottleneck resource characteristics, and offers significant savings of energy and more efficient use of different resources.

In pedagogy, teachers usually separate mixed-level students into different levels, treat them differently and teach them in accordance with their cognitive and learning abilities. Inspired from this idea, we consider particles in the swarm as mixed-level students and propose a level-based learning swarm optimizer (LLSO) to settle large-scale optimization, which is still considerably challenging in evolutionary computation. At first, a level-based learning strategy is introduced, which separates particles into a number of levels according to their fitness values and treats particles in different levels differently. Then, a new exemplar selection strategy is designed to randomly select two predominant particles from two different higher levels in the current swarm to guide the learning of particles. The cooperation between these two strategies could afford great diversity enhancement for the optimizer. Further, the exploration and exploitation abilities of the optimizer are analyzed both theoretically and empirically in comparison with two popular particle swarm optimizers. Extensive comparisons with several state-of-the-art algorithms on two widely used sets of large-scale benchmark functions confirm the competitive performance of the proposed optimizer in both solution quality and computational efficiency. Finally, comparison experiments on problems with dimensionality increasing from 200 to 2000 further substantiate the good scalability of the developed optimizer.

Stream multi-class imbalance learning in smart home applications is an evolving learning area that incorporates the challenges of both multi-class imbalance and stream learning. Moreover, another argument in the learning from the imbalanced multi-class distributions that cause misleading classification outcomes, is the imbalanced ratio in a sensor data stream which is vigorously changing. Due to the presence of an inadequate representation of sensor data stream and class distribution skews, learning from such data entails a new algorithm to transform balanced data into a model in a stream fashion. In this paper, we propose a new multi-class stream imbalance ensemble method where the base learner is a Naïve Bayesian classifier. In this approach, each training instance from any of the classes involved in learning based on thresholding on the median prior probability to aid in balancing the classes. Our proposed method diverges from state-of-the-art approaches with regard to being robust to outliers, retains more useful information, and is less sensitive to over-fitting. Also, it has a simple conceptual justification and is easy to implement. We illustrate the effectiveness of the proposed method on two smart home testbed datasets. Our proposed method compares favourably with state-of-the-art approaches.

In recent years, the utilization of sensors equipped with renewable energy sources has attracted the attention of many researchers. However, the unsteady nature of energy harvested from the environment imposes new challenges for resource management in sensors. In this research, we investigate the energy flow behaviour of mobile sensors modelled by coupled data and energy queues. Based on verified Markovian modelling, we derive the probabilities of battery depletion and data overflow of the sensors. In addition, we propose sufficient conditions in which model feasibility is guaranteed, and a closed-form solution for the optimal battery size for a node is obtained. Employing numerical study, we investigate the impact of different depletion and overflow rates on the battery size. We also use simulation to test the applicability of our mathematical model in a real-world scenario and estimate the data buffer size. These results offer useful insights to the optimal design of energy harvesting sensors.

The popular performance profiles and data profiles for benchmarking deterministic optimization algorithms are extended to benchmark stochastic algorithms for global optimization problems. A general confidence interval is employed to replace the significance test, which is popular in traditional benchmarking methods but suffering more and more criticisms. Through computing confidence bounds of the general confidence interval and visualizing them with performance profiles and (or) data profiles, our benchmarking method can be used to compare stochastic optimization algorithms by graphs. Compared with traditional benchmarking methods, our method is synthetic statistically and therefore is suitable for large sets of benchmark problems. Compared with some sample-mean-based benchmarking methods, e.g., the method adopted in black-box-optimization-benchmarking workshop/competition, our method considers not only sample means but also sample variances. The most important property of our method is that it is a distribution-free method, i.e., it does not depend on any distribution assumption of the population. This makes it a promising benchmarking method for stochastic optimization algorithms. Some examples are provided to illustrate how to use our method to compare stochastic optimization algorithms.

Large-scale optimization has become a significant yet challenging area in evolutionary computation. To solve this problem, this paper proposes a novel segment-based predominant learning swarm optimizer (SPLSO) swarm optimizer through letting several predominant particles guide the learning of a particle. First, a segment-based learning strategy is proposed to randomly divide the whole dimensions into segments. During update, variables in different segments are evolved by learning from different exemplars while the ones in the same segment are evolved by the same exemplar. Second, to accelerate search speed and enhance search diversity, a predominant learning strategy is also proposed, which lets several predominant particles guide the update of a particle with each predominant particle responsible for one segment of dimensions. By combining these two learning strategies together, SPLSO evolves all dimensions simultaneously and possesses competitive exploration and exploitation abilities. Extensive experiments are conducted on two large-scale benchmark function sets to investigate the influence of each algorithmic component and comparisons with several state-of-the-art meta-heuristic algorithms dealing with large-scale problems demonstrate the competitive efficiency and effectiveness of the proposed optimizer. Further the scalability of the optimizer to solve problems with dimensionality up to 2000 is also verified.

Detecting abnormal events in video surveillance is a challenging problem due to the large scale, stream fashion video data as well as the real-time constraint. In this paper, we present an online, adaptive, and real-time framework to address this problem. The spatial locations in a frame is partitioned into grids, in each grid the proposed Adaptive Multi-scale Histogram Optical Flow (AMHOF) features are extracted and modelled by an Online Weighted Clustering (OWC) algorithm. The AMHOFs which cannot be fit to a cluster with large weight are regarded as abnormal events. The OWC algorithm is simple to implement and computational efficient. In addition, we improve the detection performance by a Multiple Target Tracking (MTT) algorithm. Experimental results demonstrate our approach outperforms the state-of-the-art approaches in pixel-level rate of detection at a processing speed of 30 FPS.

We propose a novel, online adaptive one-class support vector machines algorithm for anomaly detection in crowd scenes. Integrating incremental and decremental one-class support vector machines with a sliding buffer offers an efficient and effective scheme, which not only updates the model in an online fashion with low computational cost, but also discards obsolete patterns. Our method provides a unified framework to detect both global and local anomalies. Extensive experiments have been carried out on two benchmark datasets and the comparison to the state-of-the-art methods validates the advantages of our approach.

Real-time train localization using wireless sensor networks (WSNs) offers huge benefits in terms of cost reduction and safety enhancement in railway environments. A challenging problem in WSN-based train localization is how to guarantee timely communication between the anchor sensors deployed along the track and the gateway deployed on the train with minimum energy consumption. This paper presents an energy-efficient scheme for timely communication between the gateway and the anchor sensors, in which each anchor sensor runs an asynchronous duty-cycling protocol to conserve energy and wakes up only when it goes into the communication range of the gateway. A beacon-driven wake-up scheme is designed, and we establish the upper bound on the amount of time that an anchor sensor can sleep in one duty cycle to guarantee timely wake-up once a train approaches. We also give a thorough theoretical analysis for the energy efficiency of our scheme and give the optimal amount of time that an anchor sensor should sleep in terms of minimizing the total energy consumption at each anchor sensor. We evaluate the performance of our scheme through simulations, and results show that our scheme can wake up anchor sensors timely at a very low cost on energy consumption.

We propose to use color covariance matrices of superpixels as a feature in addition to colors. A non-Euclidean distance metric is employed for the covariance matrix manifolds. We then introduce three ways of fusing the similarity matrices obtained from both feature spaces for affinity graph generation. Experiments carried out using a benchmark dataset reveals that our approach achieves competitive and even better results compared with the state of the art.

Delay tolerant networks under wireless intermittent settings have gathered growing research interests in recent years. There remains, however, a lacking of performance modelling results for DTN protocols. In this paper, we propose to model the system-level performance of DTN protocols using the Erlang B queueing model, and profile the performance metrics such as message delivery ratios under different traffic conditions using the system capacity measurements. A number of common DTN protocols are evaluated using a scenario based on map-based mobility under different settings.

Data reduction strategy is one of the schemes employed to extend network lifetime. In this paper we present an implementation of a light-weight forecasting algorithm for sensed data which saves packet transmission in the network. The proposed Naive algorithm achieves high energy savings with a limited computational overhead on a node. Simulation results from realistic Building monitoring application of WSN are compared with well-known prediction algorithms such as ARIMA, LMS and WMA models. We implemented a real-world deployment using 32bit mote-class device. Overall, up to 96% transmission reduction is achieved using our Naive method, while still able to maintain a considerable level of accuracy at 0.5°C error bound and it is comparable in performance to the more complex models such as ARIMA, LMS and WMA.

Data reduction strategy is one of the schemes employed to extend network lifetime. In this paper we present an implementation of a light-weight forecasting algorithm for sensed data which saves packet transmission in the network. The proposed Naive algorithm achieves high energy savings with a limited computational overhead on a node. Simulation results from realistic Building monitoring application of WSN are compared with well-known prediction algorithms such as ARIMA, LMS and WMA models. We implemented a real-world deployment using 32bit mote-class device. Overall, up to 96% transmission reduction is achieved using our Naive method, while still able to maintain a considerable level of accuracy at 0.5 °C error bound and it is comparable in performance to the more complex models such as ARIMA, LMS and WMA.

The Mixture of Gaussians (MoG) is a frequently used method for moving objects detection in a video, but its parameter setting is often tricky for dynamic scenes. Therefore, in this paper we propose an adaptive algorithm for the parameters learning by using working set of recent random samples. Furthermore, a novel Spatio-Temporal voting scheme is introduced to refine the foreground map. Also, we propose a components shifting based technique for handling abrupt scene changes. The components shifting based scheme can reutilize most of the already learned models, thus avoids a large number of false alarms by quickly adapting to the changed illumination conditions. The proposed model is rigorously tested and compared with several state-of-the-art methods and has shown significant performance improvements.

Keyframe selection has been crucial for effective and efficient video content analysis. While most of the existing approaches represent individual frames with global features, we, for the first time, propose a keypoint-based framework to address the keyframe selection problem so that local features can be employed in selecting keyframes. In general, the selected keyframes should both be representative of video content and containing minimum redundancy. Therefore, we introduce two criteria, coverage and redundancy, based on keypoint matching in the selection process. Comprehensive experiments demonstrate that our approach outperforms the state of the art.

In recent work on image and video retrieval there seems to be a shift of focus from low-level feature extraction to producing high-level semantic representation of scenes. This paper presents a framework that produces semantic context features from video frames which are then employed for key-frame extraction. Working with wildlife video frames, the framework starts with image segmentation, followed by low-level feature extraction and classification of the image blocks extracted from image segments. Based on the image block labels in the neighbourhood a co-occurrence matrix is then constructed to represent the semantic context of the scene. The semantic co-occurrence matrices then undergo binarization and principal component analysis for dimension reduction, forming the feature vectors used in a one-class classifier that extracts the key-frames. Experiments show that the utilization of high-level semantic features result in better key-frame extraction when compared with methods using low-level features only.

Novelty detection is an important functionality that has found many applications in information retrieval and processing. In this paper we propose a novel framework that deals with novelty detection for multiple-scene image sets. Working with wildlife image data, the framework starts with image segmentation, followed by feature extraction and classification of the image blocks extracted from image segments. The labelled image blocks are then scanned through to generate a co-occurrence matrix of object labels, representing the semantic context within the scene. The semantic co-occurrence matrices then undergo binarization and principal component analysis for dimension reduction, forming the basis for constructing one-class models for each scene category. An algorithm for outlier detection that employs multiple one-class models is proposed. An advantage of our approach is that it can be used for scene classification and novelty detection at the same time. Our experiments show that the proposed approach algorithm gives favourable performance for the task of detecting novel wildlife scenes, and binarization of the label co-occurrence matrices helps to significantly increase the robustness in dealing with the variation of scene statistics.

In tackling data mining and pattern recognition tasks, finding a compact but effective set of features has often been found to be a crucial step in the overall problem-solving process. In this paper, we present an empirical study on feature analysis for recognition of classical instrument, using machine learning techniques to select and evaluate features extracted from a number of different feature schemes. It is revealed that there is significant redundancy between and within feature schemes commonly used in practice. Our results suggest that further feature analysis research is necessary in order to optimize feature selection and achieve better results for the instrument recognition problem.

Content-based image retrieval techniques have been under intensively research, mainly on extracting effective low level visual features for indexing and enabling fast query of individual images by feature matching over the indexing structure. In this paper, we propose to extend the content-based approach towards the problem of multimedia collection profiling and comparison. Our method is to carry out visual feature clustering using self-organised maps, and then apply distance measures on the generated feature maps to evaluate their similarity. A modified Hausdorff distance is defined over the feature maps and further verified in an experiment using four image collections. Some preliminary results are presented with a comparison of different distance measures obtained from profiles generated by two feature schemes.

Although widely studied for many years, colour quantisation remains a practical problem in image processing. Unlike previous works where the image can only be quantised after the whole set of image data is acquired, we propose to use an evolving localised learning model for on-line colour quantisation. This approach is compared with some conventional algorithms.

An algorithm of evolving self-organizing map (ESOM) is proposed as a dynamic version of the Kohonen self-organizing map, where network structure is evolved in an online adaptive mode. Experiments have been carried out on some benchmark data sets as well as on macroeconomic data. Results show that ESOM is a good tool for clustering, data analysis, and visualisation.

So far the bottleneck of Chinese recognition, especially handwritten recognition, still lies in the effectiveness of feature-extraction to cater for various distortions and position shifting. In the paper, a novel method is proposed by applying a set of Gabor spatial filters with different directions and spatial frequencies to character images, in an effort to reach the optimum trade-off between feature stability and feature localization. While a classic self-organizing map is used for unsupervised clustering feature codes, a multi-staged LVQ with a fuzzy judgement unit is applied for the final recognition on the feature mapping result.>