Learning to rank models are broadly applied in ad hoc retrieval for scoring and sorting documents based on their relevance to textual queries. The generalizability of the trained model in the learning to rank approach, however, can have an impact on the retrieval performance, particularly when data includes noise and outliers, or is incorrectly collected or measured. In this paper, we introduce a Self-Distilled Learning to Rank (SDLR) framework for ad hoc retrieval, and analyze its performance over a range of retrieval datasets and also in the presence of features’ noise. SDLR assigns a confidence weight to each training sample, aiming at reducing the impact of noisy and outlier data in the training process. The confidence weight is approximated based on the feature’s distributions derived from the values observed for the features of the documents labeled for a query in a listwise training sample. SDLR includes a distillation process that facilitates passing on the underlying patterns in assigning confidence weights from the teacher model to the student one. We empirically illustrate that SDLR outperforms state-of-the-art learning to rank models in ad hoc retrieval. We thoroughly investigate the SDLR performance in different settings including when no distillation strategy is applied; when different portion of data are used for training the teacher and the student models, and when both teacher and student models are trained over identical data. We show that SDLR is more effective when training data are split between a teacher and a student model. We also show that SDLR’s performance is robust when data features are noisy.

Abstract Support vector data description (SVDD) characterizes a dataset by a spherically shaped boundary around it. Since the complexity of SVDD training is O(N3), its performance decreases for large‐scale datasets. In this paper, we propose an improved SVDD algorithm, called EM‐SVDD, which combines the expectation maximization (EM) algorithm and SVDD to reduce the complexity and accelerate the training phase, while the accuracy of the classifier remains unchanged. First, the dataset is clustered to obtain smaller subsets, and then the boundary of each subset is identified by SVDD. After that, to construct the dataset boundary and get the optimal weighted combination of SVDDs, the EM algorithm is utilized to estimate the parameters and weights of SVDDs. The time complexity of the proposed method is N/i times lower than SVDD, where i is the number of EM iterations. In addition to EM‐SVDD, Sparse EM‐SVDD is proposed to guarantee the sparsity of the iteratively estimated parameters. EM‐SVDD is well compared with several similar methods. Simulation results indicate higher speed and performance of the proposed method in the training and testing phases. Furthermore, the capability of the proposed method is tested on a large image dataset acquired from social networks and our method identifies in‐class and outlier images with 0.71 accuracy rate.

In recent years, many learning systems have been developed for higher level forms of data, such as learning on distributions in which each example itself is a distribution. This article proposes active robust learning on distributions. In learning on distributions, there is no access to distributions themselves but rather access is through a sample drawn from a distribution. Therefore, similar to robust learning, any estimates of examples are inexact. In order to address these difficulties, we provide an upper bound on the risk of the classifier in the next stage of active learning, where the size of the labeled dataset increases. Based on this upper bound, we propose probabilistic minimax active learning (PMAL) as a general multiclass active learning method that is easy to use in many Bayesian settings, which provably selects an example with knowledge of its label minimizing the expected risk. We present an efficient approximation of the objective with a known error bound to deal with the intractability of the proposed method for active robust learning. Here, we face a nonconvex problem, which we solve by means of a related convex problem with a bound on the norm of the difference between their solutions. To utilize the information about the estimates of distributions, we propose active robust learning on the distributions method based on learning the kernel embedding of distributions by a recent Bayesian method. The experiments demonstrate the effectiveness of the resulting method on a set of synthetic and real-world distributional datasets.

Multi-object tracking is a well known problem in the context of vehicle tracking. Kalman filter is a common tool to solve the problem in real world. In a driving enviroment, there are other parameters affecting the behavior of the driver than itself such as other driver’s behavior and the environment including obstacles and possible paths. Interactive Kalman filter (IKF), a generalized from of DKF, was previously introduced to model the interaction between vehicles. To augment KF, DKF, and IKF, we use information extracted from history of traffic in the same environment called situation assessment. In this paper, we proposed SAIKF, a variant of Kalman filter and interactive Kalman filter that employs situation assessment information to enhance the performance of tracking. A graph called Motion History Graph is constructed based on the history of the vehicle motions and is then used to augment the estimation. The results on real world video sequences show effective performance improvement.

The hyperspectral-image (HSI) noise-reduction step is a very significant preprocessing phase of data-quality enhancement. It has been attracting immense research attention in the remote sensing and image processing domains. Many methods have been developed for HSI restoration, the goal of which is to remove noise from the whole HSI cube simultaneously without considering the spectral-spatial similarity. When a noise-removal algorithm is used globally to the entire data set, it would not eliminate all levels of noise, effectively. Furthermore, most of the existing methods remove independent and identically distributed (i.i.d.) Gaussian noise. The real scenarios are much more complicated than this assumption. The complexity created by natural noise that has a non-i.i.d. structure leads to inefficient methods containing underestimation and invalid performance. In this article, we calculated the spatial-spectral similarity criteria by defining a set of clustering-based latent variables (CLVs) in a Bayesian framework to improve the robustness. These criteria can be extracted using the clustering operators. Then, by applying the CLV to the variational Bayesian model, we investigated a new low-rank matrix factorization denoising approach based on the proposed clustering-based latent variable (CLV-LRMF) to remove noise with the non-i.i.d. mixture of Gaussian structures. Finally, we switched to the GPU for MATLAB implementation to reduce the runtime. The experimental results show that the performance has been improved by applying the proposed CLV and demonstrate the effectiveness of the proposed CLV-LRMF over other state-of-the-art methods.

In this paper, an interactive Kalman filter (IKF) is proposed to demonstrate the interaction between targets as to how the behavior of a desired target is affected by the behavior of its neighbors. The IKF utilizes two types of interactions available in multi-agent systems, namely, cooperative and competitive. The IKF is similar to the distributed Kalman filter (DKF) in terms of architecture, method of representation of equations, and use of neighborhood weight matrix while IKF appears to be a general form of DKF. In this method, a network of IKF nodes is constructed such that each node is associated with every target. There are edges between nodes for which the corresponding targets have effect on each other. Time-varying weights are used to control the interaction information exchanged among IKF nodes. The method of calculating interaction weights in the weight matrix plays a key role on the estimation results. The calculation of optimal IKF gain and evaluations on MOTP, MOTA, and MSE metrics illustrate the effectiveness of the proposed filter in vehicle tracking.

While GPUs are meantime omnipresent for many scientific and technical computations, they still continue to evolve as processors. An important recent feature is the ability to execute multiple kernels concurrently via queue streams. However, experiments show that different parameters including the behavior of kernels, the order of kernel launches and other execution configurations, e.g., the number of concurrent thread blocks, may result in different execution time for concurrent kernel execution. Since kernels may have different resource requirements, they can be classified into different classes, which are traditionally assumed as either memory-bound or compute-bound. However, a kernel may belong to the different classes on different hardware according to the hardware resources. In this paper, the definition of kernel mix intensity is introduced. Based on this, a scheduling framework called concurrent CUDA (cCUDA) is proposed to co-schedule the concurrent kernels more efficiently. It first profiles and ranks kernels with different execution behaviors and then takes the kernel resource requirements into account to partition thread blocks of different kernels and overlap them to better utilize the GPU resources. Experimental results on real hardware demonstrate performance improvement in terms of execution time of up to 1.86x, and an average speedup of 1.28x for a wide range of kernels. cCUDA is available at https://github.com/kshekofteh/cCUDA.

Nowadays, several digital forensic tools extract a lot of low‐level information from different parts of the system. Constructing high‐level information from low‐level ones is very challenging. This study reconstructs high‐level events by using the traces of applications that are found in the file system metadata. In this regard, an event reconstruction framework is proposed that determines which applications have been run on a compromised system. The proposed framework works in two phases. In the training phase, the signatures of various applications are constructed. The signature of an application is the temporal pattern of file system modification of the application. In the detection phase, at first, the temporal pattern of file system modification of the hard disk (TPFSM‐D) of the compromised system is constructed. Then in order to determine whether a particular application has been run on the compromised system, the distance between the signature of the application and the TPFSM‐D of the hard disk is calculated by using a proposed distance measure. Finally, a decision engine decides whether the application has been run on the compromised system. The proposed event reconstruction framework has been tested on different scenarios. The empirical results suggest that the framework is effective in reconstructing events.

Graphics Processing Units (GPUs) are vastly used for running massively parallel programs. GPU kernels exhibit different behavior at runtime and can usually be classified in a simple form as either “compute-bound” or “memory-bound.” Recent GPUs are capable of concurrently running multiple kernels, which raises the question of how to most appropriately schedule kernels to achieve higher performance. In particular, co-scheduling of compute-bound and memory-bound kernels seems promising. However, its benefits as well as drawbacks must be determined along with which kernels should be selected for a concurrent execution. Classifying kernels can be performed online by instrumentation based on performance counters. This work conducts a thorough analysis of the metrics collected from various benchmarks from Rodinia and CUDA SDK. The goal is to find the minimum number of effective metrics that enables online classification of kernels with a low overhead. This study employs a wrapper-based feature selection method based on the Fisher feature selection criterion. The results of experiments show that to classify kernels with a high accuracy, only three and five metrics are sufficient on a Kepler and a Pascal GPU, respectively. The proposed method is then utilized for a runtime scheduler. The results show an average speedup of 1.18× and 1.1× compared with a serial and a random scheduler, respectively.

Several objective functions have been proposed in the literature to adjust the input parameters of a node in constructive networks. Furthermore, many researchers have focused on the universal approximation capability of the network based on the existing objective functions. In this brief, we use a correntropy measure based on the sigmoid kernel in the objective function to adjust the input parameters of a newly added node in a cascade network. The proposed network is shown to be capable of approximating any continuous nonlinear mapping with probability one in a compact input sample space. Thus, the convergence is guaranteed. The performance of our method was compared with that of eight different objective functions, as well as with an existing one hidden layer feedforward network on several real regression data sets with and without impulsive noise. The experimental results indicate the benefits of using a correntropy measure in reducing the root mean square error and increasing the robustness to noise.

Signal subspace identification (SSI) is known as an important preprocessing for most remote sensing processes and its applications. Hence, a graph-based method is presented in this letter to improve identification of signal subspace and its dimension. Our proposed method reduces sensitivity to noise through integrating a weighted graph of image pixels in the cost function of the well-known hyperspectral SSI by minimum error (HySime) method to improve the accuracy of SSI. The method proposed in this letter is a very simple and yet very effective in estimating various types of hyperspectral data. The proposed method was implemented on various synthetic and real data. The results of the experiments on both types of hyperspectral data indicated the accuracy of this approach in estimating the signal subspace as compared with other well-known methods.

Semi-supervised learning with a growing self-organizing map (GSOM) is commonly used to cope with the machine learning problems. The performance of semi-supervised GSOM is associated with the structure of clustering layer, the activation level, and the weights of a classifier. Current methods have b een advocated to calibrate the GSOM parameters based on local point approach. The local point approach is associated with structure of dataset. On the other hand, the semi-supervised GSOM output is so closely intertwined with problem inputs. This paper present an analytical semi-supervised learning method based on GSOM and extreme learning machine. Extreme learning machine was used to exploit the substantial classification response. However, the learning of GSOM parameters was eliminated with use of the extreme learning machine. Furthermore, the sequential extreme learning machine was implemented to achieve an online semi-supervised GSOM for streaming dataset. This study showed the proposed method converges to optimum response regardless to structure of dataset. The proposed method was applied on the online and partially labeled dataset. Online semi-supervised GSOM integrated with extreme learning machine achievement implies that the F-measure of proposed method is more precise than the conventional semi-supervised GSOM.

The robust support vector machines (RoSVM) for ellipsoidal data is difficult to solve. To overcome this difficulty, its primal form has been approximated with a second‐order cone programming (SOCP) called approximate primal RoSVM. In this article, we show that the primal RoSVM is equivalent to an SOCP and name it accurate primal RoSVM. The optimal weight vector of this model is not sparse necessarily. The sparser the weight vector, the less time the test phase takes. Hence, to reduce the test time, first, we obtain its dual form and then prove the sparsity of its optimal solution. Second, we show that some parts of the optimal decision function can be computed in the training phase instead of the test phase. This can decrease the test time further. However, training time of the dual model is more than that of the primal model, but the test time is often more critical than the training time because the training is often an off‐line procedure while the test procedure is performed online. Experimental results on benchmark data sets show the superiority of the proposed models.

This paper introduces a nonlinear dynamic model to study spatial and temporal dynamics of epidemics of susceptible-infected-removed type. It involves modeling the respective collections of epidemic states and syndromic observations as random finite sets. Each epidemic state consists of the number of infected individuals in an isolated population system and the corresponding partially known parameters of the epidemic model. The infectious disease could spread between population systems with known probabilities based on prior knowledge of ecological and biological features of the environment. The problem is then formulated in the context of Bayesian framework and estimated via a probability hypothesis density filter. Each population system under surveillance is assumed to be homogenous and fixed, with daily reports on the number of infected people available for monitoring and prediction. When model parameters are partially known, results of numerical studies indicate that the proposed approach can help early prediction of the epidemic in terms of peak and duration.

In this paper, we introduce an incremental version of recently proposed constrained Linear Discriminant Analysis (LDA). In addition of application in constrained LDA problems, our algorithm which we call Online Discriminative Component Analysis (ODCA

Abstract Data may be afflicted with uncertainty. Uncertain data may be shown by an interval value or in general by a fuzzy set. A number of classification methods have considered uncertainty in features of samples. Some of these classification methods are extended version of the support vector machines (SVMs), such as the Interval‐SVM (ISVM), Holder‐ISVM and Distance‐ISVM, which are used to obtain a classifier for separating samples whose features are interval values. In this paper, we extend the SVM for robust classification of linear/non‐linear separable data whose features are fuzzy numbers. The support of such training data is shown by a hypercube. Our proposed method tries to obtain a hyperplane (in the input space or in a high‐dimensional feature space) such that the nearest point of the hypercube of each training sample to the hyperplane is separated with the widest symmetric margin. This strategy can reduce the misclassification probability of our proposed method. Our experimental results on six real data sets show that the classification rate of our novel method is better than or equal to the classification rate of the well‐known SVM, ISVM, Holder‐ISVM and Distance‐ISVM for all of these data sets.

Abstract This paper presents a new model of support vector machines (SVMs) that handle data with tolerance and uncertainty. The constraints of the SVM are converted to fuzzy inequality. Giving more relaxation to the constraints allows us to consider an importance degree for each training samples in the constraints of the SVM. The new method is called relaxed constraints support vector machines (RSVMs). Also, the fuzzy SVM model is improved with more relaxed constraints. The new model is called fuzzy RSVM. With this method, we are able to consider importance degree for training samples both in the cost function and constraints of the SVM, simultaneously. In addition, we extend our method to solve one‐class classification problems. The effectiveness of the proposed method is demonstrated on artificial and real‐life data sets.

In this paper, a new classifier fusion method is introduced based on a decision template structure as an extension to Dempster Shafer method. It employs multilayer neural networks as base classifiers. The idea relies on the fact that in a multilayer neural network, behavior of each layer can be a guide for modeling decision-making process. The new decision template based method constructs decision template for each layer of the neural networks including all hidden layers such that a complete model of the base classifiers decision making process is built. In the combiner part, a new strategy based on extension to Dempster Shafer method is introduced. Efficiency of this method is compared with some known benchmark datasets.

In this paper, an imprecise data classification is considered using new version of Fisher discriminator, namely interval Fisher. In the conventional formulation of Fisher, elements of within-class scatter matrix (related to covariance matrix between clusters) and between-class scatter matrix (related to covariance matrix of centers of clusters) have single values; but in the interval Fisher, the elements of matrices are in the interval form and can vary in a range. The particle swarm optimization search method is used for solving a constrained optimization problem of the interval Fisher discriminator. Unlike conventional Fisher with one optimal hyperplane, interval Fisher gives two optimal hyperplanes thereupon three decision regions are obtained. Two classes with regard to imprecise scatter matrices are derived by decision making using these optimal hyperplanes. Also, fuzzy region lets us help in fuzzy decision over input test samples. Unlike a support vector classifier with two parallel hyperplanes, interval Fisher generally gives us two nonparallel hyperplanes. Experimental results show the suitability of this idea. © 2011 Wiley Periodicals, Inc.

One problem in background estimation is the inherent change in the background such as waving tree branches, water surfaces, camera shakes, and the existence of moving objects in every image. In this paper, a new method for background estimation is proposed based on function approximation in kernel domain. For this purpose, Weighted Kernel-based Learning Algorithm (WKLA) is designed. WKLA includes a weighted type of kernel least mean square algorithm with ability to function approximation in the presence of noise. So, the proposed background estimation method includes two stages: firstly, a novel algorithm for outlier detection namely Fuzzy Outlier Detector (FOD) is applied. Then obtained results are fed to the WKLA. The proposed approach can handle scenes containing moving backgrounds, gradual illumination changes, camera vibrations, and non-empty backgrounds. The qualitative results and quantitative evaluations on various indoor and outdoor sequences relative to existing approaches show the high accuracy and effectiveness of the proposed method in background estimation and foreground detection.