Accurate ship track prediction plays a pivotal role in maritime operations, enabling proactive decision-making, enhancing safety, and optimizing vessel routing. We propose ship trajectory prediction - a threefold technique for facilitating accurate predictions, which involves clustering historical AIS trajectories into maritime de facto routes, classifying new trajectory to one of these routes, and conducting predictions along the identified route. To overcome the challenges of capturing the latent structure in high-dimensional and heterogeneous space imposed by AIS data, we introduce a new similarity technique that automatically determines the number of clusters. Furthermore, we introduce a method to automatically annotate the feature space, enhancing the efficiency of data analysis tasks like clustering. This not only improves performance but also ensures transparency, allowing for effective performance evaluation. Our approach demonstrates an accuracy of over 88% and an accuracy of 78% in predicting routes for tanker vessels.

In the maritime environment, the Automatic Identification System (AIS) is used to monitor vessel activity concerning security and safety ocean-wide. AIS data has been used to detect anomalous behaviors related to suspicious activities and hazardous events. Typically, clustering analysis is used to investigate anomalous events within the AIS data stream. However, the main challenge in this approach is to determine and execute the dissimilarity measure between trajectories since they differ in size and time. In addition, these calculations are computationally expensive and not scalable. To tackle this issue, compression algorithms can be applied to perform clustering analysis since they are typically used to reduce storage and processing time. Therefore, the proposed analysis will assess how compression algorithms affect clustering results with respect to detecting anomalous vessel trajectories. The analysis results show that a suitable compression algorithm can reduce the overall processing time with little impact on the clustering results while supporting the scalability of this type of analysis.

The automated extraction of maritime routes that accurately resembles the real traffic of vessels is crucial for intelligent traffic management systems to understand vessel behaviour in sea areas, identify events, and support decision-making. Available solutions for maritime traffic route extraction utilize traditional clustering algorithms, which have high computational costs. Data reduction methods are proposed for use with these clustering methods to improve clustering performance, which involves a loss in movement pattern quality. Such solutions often result in low quality representations of traffic routes, which poorly estimate sailing distances for long journeys and time of arrivals and poorly identify non-conformities. In this paper, we propose a spatial clustering method (SPTCLUST-II) to extract spatial representations of sailing routes from historical Automatic Identification System (AIS) data. Our method can cluster huge volumes of trajectory data in a minimal amount of time without using any of the traditional clustering algorithms and with no reduction or modification of the spatio-temporal predicates of the original trajectories. A real-world AIS dataset captured in the area of the Gulf of Mexico is used for the evaluation of the proposed method. The results demonstrate that the proposed method extracts tankers maritime traffic routes with an accuracy of 97% and a f1-measure of 98.5% and cargoes maritime traffic routes with an accuracy of 98.1% and a f1-measure of 99%. This method can be utilized by surveillance authorities for stable and sustainable vessel traffic management.

MOTIVATION: There is a plethora of measures to evaluate functional similarity (FS) of genes based on their co-expression, protein-protein interactions and sequence similarity. These measures are typically derived from hand-engineered and application-specific metrics to quantify the degree of shared information between two genes using their Gene Ontology (GO) annotations. RESULTS: We introduce deepSimDEF, a deep learning method to automatically learn FS estimation of gene pairs given a set of genes and their GO annotations. deepSimDEF's key novelty is its ability to learn low-dimensional embedding vector representations of GO terms and gene products and then calculate FS using these learned vectors. We show that deepSimDEF can predict the FS of new genes using their annotations: it outperformed all other FS measures by >5-10% on yeast and human reference datasets on protein-protein interactions, gene co-expression and sequence homology tasks. Thus, deepSimDEF offers a powerful and adaptable deep neural architecture that can benefit a wide range of problems in genomics and proteomics, and its architecture is flexible enough to support its extension to any organism. AVAILABILITY AND IMPLEMENTATION: Source code and data are available at https://github.com/ahmadpgh/deepSimDEF. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Abstract In this paper we model spatio-temporal data describing the fishing activities in the Northern Adriatic Sea over four years. We build, implement and analyze a database based on the fusion of two complementary data sources: trajectories from fishing vessels (obtained from terrestrial Automatic Identification System, or AIS, data feed) and fish catch reports (i.e., the quantity and type of fish caught) of the main fishing market of the area. We present all the phases of the database creation, starting from the raw data and proceeding through data exploration, data cleaning, trajectory reconstruction and semantic enrichment. We implement the database by using MobilityDB, an open source geospatial trajectory data management and analysis platform. Subsequently, we perform various analyses on the resulting spatio-temporal database, with the goal of mapping the fishing activities on some key species, highlighting all the interesting information and inferring new knowledge that will be useful for fishery management. Furthermore, we investigate the use of machine learning methods for predicting the Catch Per Unit Effort (CPUE), an indicator of the fishing resources exploitation in order to drive specific policy design. A variety of prediction methods, taking as input the data in the database and environmental factors such as sea temperature, waves height and Clorophill-a, are put at work in order to assess their prediction ability in this field. To the best of our knowledge, our work represents the first attempt to integrate fishing ships trajectories derived from AIS data, environmental data and catch data for spatio-temporal prediction of CPUE – a challenging task.

Time-series forecasting is one of the most active research topics in artificial intelligence. It has the power to bring light to problems in several areas of knowledge, such as epidemiological studies, healthcare inference, and climate change analysis. Applications in real-world time series should consider two factors for achieving reliable predictions: modeling dynamic dependencies among multiple variables and adjusting the model's intrinsic hyperparameters. An open gap in the literature is that statistical and ensemble learning approaches systematically present lower predictive performance than deep learning methods. The existing applications consistently disregard the data sequence aspect entangled with multivariate data represented in more than one time series. Conversely, this work presents a novel neural network architecture for time-series forecasting that combines the power of graph evolution with deep recurrent learning on distinct data distributions, named after Recurrent Graph Evolution Neural Network ( ReGENN ). The idea is to infer multiple multivariate relationships between co-occurring time-series by assuming that the temporal data depends not only on inner variables and intra-temporal relationships (i.e., observations from itself) but also on outer variables and inter-temporal relationships (i.e., observations from other-selves). An extensive set of experiments was conducted comparing ReGENN with tens of ensemble methods and classical statistical ones. The results outperformed both statistical and ensemble-learning approaches, showing an improvement of 64.87 percent over the competing algorithms on the SARS-CoV-2 dataset of the renowned John Hopkins University for 188 countries simultaneously. For further validation, we tested our architecture in two other public datasets of different domains, the PhysioNet Computing in Cardiology Challenge 2012 and Brazilian Weather datasets. We also analyzed the Evolution Weights arising from the hidden layers of ReGENN to describe how the variables of the dataset interact with each other; and, as a result of looking at inter and intra-temporal relationships simultaneously, we concluded that time-series forecasting is majorly improved if paying attention to how multiple multivariate data synchronously evolve.

Multi-Task Learning (MTL) for text classification takes advantage of the data to train a single shared model with multiple task-specific layers on multiple related classification tasks to improve its generalization performance. We choose pre-trained language models (BERT-family) as the shared part of this architecture. Although they have achieved noticeable performance in different downstream NLP tasks, their performance in an MTL setting for the biomedical domain is not thoroughly investigated. In this work, we investigate the performance of BERT-family models in different MTL settings with Open-I (radiology reports) and OHSUMED (PubMed abstracts) datasets. We introduce the MTLV (Multi-Task Learning Visualizer) library for building Multi-task learning-related architectures which use existing infrastructure (e.g., Hugging Face Transformers and MLflow Tracking). Following previous work in computer vision, we clustered tasks and trained a separate model on each cluster (Grouped Multi-Task Learning (GMTL)). Contextual representation of the class labels (Tasks) and their descriptions was used by the library as features to cluster the tasks. We observed that grouping tasks for training with few models (GMTL) outperforms the MTL also GMTL is computationally more efficient than the STL setting (a separate model is trained for each task).

A plethora of applications and devices reporting their locations generate massive amounts of spatiotemporal data along with other useful information. These data can form trajectories with sequences...

We present an approach to explain the decisions of black box image classifiers through synthetic exemplar and counter-exemplar learnt in the latent feature space. Our explanation method exploits the latent representations learned through an adversarial autoencoder for generating a synthetic neighborhood of the image for which an explanation is required. A decision tree is trained on a set of images represented in the latent space, and its decision rules are used to generate exemplar images showing how the original image can be modified to stay within its class. Counterfactual rules are used to generate counter-exemplars showing how the original image can “morph” into another class. The explanation also comprehends a saliency map highlighting the areas that contribute to its classification, and areas that push it into another class. A wide and deep experimental evaluation proves that the proposed method outperforms existing explainers in terms of fidelity, relevance, coherence, and stability, besides providing the most useful and interpretable explanations.

We present an approach for unsupervised domain adaptation{—}with a strong focus on practical considerations of within-domain class imbalance and between-domain class distribution shift{—}from a class-conditioned domain alignment perspective. Current methods for class-conditioned domain alignment aim to explicitly minimize a loss function based on pseudo-label estimations of the target domain. However, these methods suffer from pseudo-label bias in the form of error accumulation. We propose a method that removes the need for explicit optimization of model parameters from pseudo-labels. Instead, we present a sampling-based implicit alignment approach, where the sample selection is implicitly guided by the pseudo-labels. Theoretical analysis reveals the existence of a domain-discriminator shortcut in misaligned classes, which is addressed by the proposed approach to facilitate domain-adversarial learning. Empirical results and ablation studies confirm the effectiveness of the proposed approach, especially in the presence of within-domain class imbalance and between-domain class distribution shift.

While variational autoencoders (VAE) provide the theoretical basis for deep generative models, they often produce "blurry" images which is linked to their training objective. In this paper, we propose the "Sharpened Adversarial Variational Auto-Encoder" (AVAE-S) which uses an adversarial training mechanism to fine-tune the learned latent code vector of the VAE with a specialized objective function. The loss function is designed to uncover global structure as well as the local and high frequency features in VAE and leading to the smaller variance in the aggregated posterior and hence, reducing the blurriness of their generated samples. AVAE-S leverages the learned representations to the meaningful latent features by enforcing feature consistency between the model distribution and the target distribution leading to the sharpened output with better perceptual quality. Then, AVAE-S starts training a GAN network, which generator has been collapsed on the VAE's decoder, upon that learned latent code vector. Moreover, we augment the standard VAE's evidence lower bound objective function with other element-wise similarity measures. Our experiments show that AVAE-S achieves the state-of-the-art sample quality in the common MNIST and CelebA datasets. AVAE-S shares many of the good properties of the VAE (stable training, encoder-decoder architecture, nice latent manifold structure) while generating more realistic images, as measured by the sharpness score.

Most of the trajectory datasets only record the spatio-temporal position of the moving object, thus lacking semantics and this is due to the fact that this information mainly depends on the domain expert labeling, a time-consuming and complex process. This paper is a contribution in facilitating and supporting the manual annotation of trajectory data thanks to a visual-analytics-based platform named VISTA. VISTA is designed to assist the user in the trajectory annotation process in a multi-role user environment. A session manager creates a tagging session selecting the trajectory data and the semantic contextual information. The VISTA platform also supports the creation of several features that will assist the tagging users in identifying the trajectory segments that will be annotated. A distinctive feature of VISTA is the visual analytics functionalities that support the users in exploring and processing the trajectory data, the associated features and the semantic information for a proper comprehension of how to properly label trajectories.

In this paper, we adapt Recurrent Neural Networks with Stochastic Layers, which are the state-of-the-art for generating text, music and speech, to the problem of acoustic novelty detection. By integrating uncertainty into the hidden states, this type of network is able to learn the distribution of complex sequences. Because the learned distribution can be calculated explicitly in terms of probability, we can evaluate how likely an observation is then detect low-probability events as novel. The model is robust, highly unsupervised, end-to-end and requires minimum preprocessing, feature engineering or hyperparameter tuning. An experiment on a benchmark dataset shows that our model outperforms the state-of-the-art acoustic novelty detectors.

OBJECTIVE: In biomedicine, there is a wealth of information hidden in unstructured narratives such as research articles and clinical reports. To exploit these data properly, a word sense disambiguation (WSD) algorithm prevents downstream difficulties in the natural language processing applications pipeline. Supervised WSD algorithms largely outperform un- or semisupervised and knowledge-based methods; however, they train 1 separate classifier for each ambiguous term, necessitating a large number of expert-labeled training data, an unattainable goal in medical informatics. To alleviate this need, a single model that shares statistical strength across all instances and scales well with the vocabulary size is desirable. MATERIALS AND METHODS: Built on recent advances in deep learning, our deepBioWSD model leverages 1 single bidirectional long short-term memory network that makes sense prediction for any ambiguous term. In the model, first, the Unified Medical Language System sense embeddings will be computed using their text definitions; and then, after initializing the network with these embeddings, it will be trained on all (available) training data collectively. This method also considers a novel technique for automatic collection of training data from PubMed to (pre)train the network in an unsupervised manner. RESULTS: We use the MSH WSD dataset to compare WSD algorithms, with macro and micro accuracies employed as evaluation metrics. deepBioWSD outperforms existing models in biomedical text WSD by achieving the state-of-the-art performance of 96.82% for macro accuracy. CONCLUSIONS: Apart from the disambiguation improvement and unsupervised training, deepBioWSD depends on considerably less number of expert-labeled data as it learns the target and the context terms jointly. These merit deepBioWSD to be conveniently deployable in real-time biomedical applications.

In this work, we investigate word embedding algorithms in the context of natural language processing. In particular, we examine the notion of ``negative examples'', the unobserved or insignificant word-context co-occurrences, in spectral methods. we provide a new formulation for the word embedding problem by proposing a new intuitive objective function that perfectly justifies the use of negative examples. In fact, our algorithm not only learns from the important word-context co-occurrences, but also it learns from the abundance of unobserved or insignificant co-occurrences to improve the distribution of words in the latent embedded space. We analyze the algorithm theoretically and provide an optimal solution for the problem using spectral analysis. We have trained various word embedding algorithms on articles of Wikipedia with 2.1 billion tokens and show that negative sampling can boost the quality of spectral methods. Our algorithm provides results as good as the state-of-the-art but in a much faster and efficient way.

Model interpretability is a requirement in many applications in which crucial decisions are made by users relying on a model's outputs. The recent movement for “algorithmic fairness” also stipulates explainability, and therefore interpretability of learning models. And yet the most successful contemporary Machine Learning approaches, the Deep Neural Networks, produce models that are highly non-interpretable. We attempt to address this challenge by proposing a technique called CNN-INTE to interpret deep Convolutional Neural Networks (CNN) via meta-learning. In this work, we interpret a specific hidden layer of the deep CNN model on the MNIST image dataset. We use a clustering algorithm in a two-level structure to find the meta-level training data and Random Forest as base learning algorithms to generate the meta-level test data. The interpretation results are displayed visually via diagrams, which clearly indicates how a specific test instance is classified. Our method achieves global interpretation for all the test instances on the hidden layers without sacrificing the accuracy obtained by the original deep CNN model. This means our model is faithful to the original deep CNN model, which leads to reliable interpretations.

A first fundamental step in the process of analyzing movement data is trajectory segmentation, i.e., splitting trajectories into homogeneous segments based on some criteria. Although trajectory segmentation has been the object of several approaches in the last decade, a proposal based on a semi-supervised approach remains inexistent. A semi-supervised approach means that a user labels manually a small set of trajectories with meaningful segments and, from this set, the method infers in an unsupervised way the segments of the remaining trajectories. The main advantage of this method compared to pure supervised ones is that it reduces the human effort to label the number of trajectories. In this work, we propose the use of the Minimum Description Length (MDL) principle to measure homogeneity inside segments. We also introduce the Reactive Greedy Randomized Adaptive Search Procedure for semantic Semi-supervised Trajectory Segmentation (RGRASP-SemTS) algorithm that segments trajectories by combining a limited user labeling phase with a low number of input parameters and no predefined segmenting criteria. The approach and the algorithm are presented in detail throughout the paper, and the experiments are carried out on two real-world datasets. The evaluation tests prove how our approach outperforms state-of-the-art competitors when compared to ground truth.

We present Partition-wise Gated Recurrent Units (pGRUs) for point-based trajectory classification to detect real-world trawler fishing activities in the ocean. We propose partition-wise activation functions integrated with Gated Recurrent Units that partitions each feature and uses independent parameters to model distinct regions of the feature space. This approach enables us to leverage the benefits of deep learning on a low-dimensional and heterogeneous feature space by mapping the low-dimensional features into another space that can be better separated with piece-wise learnable parameters. We show that the proposed partition-wise activation functions can approximate a wide array of functions, including conventional activations such as sigmoid and hyperbolic tangent. Our experimental results demonstrate that pGRU learns the probability distribution from data and achieves substantial improvements over the state-of-the-art systems on the task of trawler fishing activity detection.

This paper is an early report from research undertaken to meet the needs of the General Directorate for National Roads and Motorways of Poland. They have defined the task of estimating the annual average Daily Traffic on a class of highways in the country, based on a very small number of daily traffic measurements undertaken throughout the year (typically one or two such measurements). We report the data available to us, and the data preprocessing step, including the generation of additional attributes and generation of synthetic data. We use a deep neural network model of the annual count number, and we report encouraging early result. In the conclusion, we discuss the next steps of this research.

Fishing activity detection is important for fishery management to maintain abundant oceans.This paper presents a novel approach to identifying fishing activities from Automatic Identification System (AIS) data using Conditional Random Fields (CRFs).CRFs are popular for solving structured prediction problems such as sequence labeling in natural language processing.To model the conditional probability distributions that can identify fishing activities of the vessel points, we treat attributes of vessel points as observed variables and the fishing and non-fishing labels as hidden variables.We present three experiments and two comparisons to demonstrate the stability and effectiveness of the resulting models.

IN THIS presentation we will argue that Big Data technologies can contribute in an important way to an unprecedented breakthrough in the understanding of oceans as a factor in climate change, in transportation, and in supplying humanity with its important food component.

In this research we present a remote localization technique as an essential preprocessing step to enable Physical Analytics in the retail and hospitality sector. We studied two crowdsourced Wi-Fi data sources as potential inputs for fingerprinting-based positioning systems. These sources are non intrusively crowdsourced and can be easily acquired at almost any retail store. We evaluated our hypothesis on large, real-world datasets using statistical and machine learning techniques. With the use of these sources, we built a fingerprinting-based positioning system that achieved reliable and accurate physical positioning results. Our method is capable of estimating positions without any prior knowledge about the store plan or the antennas' location with, only one off-the-shelf access point. Unlike other positioning techniques, instead of estimating a relative position of a device from an antenna, we provide an absolute position for a device as inside or outside of a venue without making any assumption about the site nor the positioned devices. To investigate its practicality, we evaluated our method with datasets of five different stores.

The paper presents an unsupervised nonlinear dimensionality reduction algorithm called Unit Ball Embedding (UBE). Many high-dimensional data, such as object or face images, lie on a union of low-dimensional subspaces which are often called manifolds. The proposed method is able to learn the structure of these manifolds by exploiting the local neighborhood arrangement around each point. It tries to preserve the local structure by minimizing a cost function that measures the discrepancy between similarities of points in the high-dimensional data and similarities of points in the low-dimensional embedding. The cost function is proposed in a way that it provides a hyper-spherical representation of points in the low-dimensional embedding. Visualizations of our method on different datasets show that it creates large gaps between the manifolds and maximizes the separability of them. As a result, it notably improves the quality of unsupervised machine learning tasks (e.g. clustering). UBE is successfully applied on image datasets such as faces, handwritten digits, and objects and the results of clustering on the low-dimensional embedding show significant improvement over existing dimensionality reduction methods.

Data stream research has grown rapidly over the last decade. Two major features distinguish data stream from batch learning: stream data are generated on the fly, possibly in a fast and variable rate; and the underlying data distribution can be non-stationary, leading to a phenomenon known as concept drift. Therefore, most of the research on data stream classification focuses on proposing efficient models that can adapt to concept drifts and maintain a stable performance over time. However, specifically for the classification task, the majority of such methods rely on the instantaneous availability of true labels for all already classified instances. This is a strong assumption that is rarely fulfilled in practical applications. Hence there is a clear need for efficient methods that can detect concept drifts in an unsupervised way. One possibility is the well-known Kolmogorov-Smirnov test, a statistical hypothesis test that checks whether two samples differ. This work has two main contributions. The first one is the Incremental Kolmogorov-Smirnov algorithm that allows performing the Kolmogorov-Smirnov hypothesis test instantly using two samples that change over time, where the change is an insertion and/or removal of an observation. Our algorithm employs a randomized tree and is able to perform the insertion and removal operations in O(log N) with high probability and calculate the Kolmogorov-Smirnov test in O(1), where N is the number of sample observations. This is a significant speed-up compared to the O(N log N) cost of the non-incremental implementation. The second contribution is the use of the Incremental Kolmogorov-Smirnov test to detect concept drifts without true labels. Classification algorithms adapted to use the test rely on a limited portion of those labels just to update the classification model after a concept drift is detected.

MOTIVATION: Measures of protein functional similarity are essential tools for function prediction, evaluation of protein-protein interactions (PPIs) and other applications. Several existing methods perform comparisons between proteins based on the semantic similarity of their GO terms; however, these measures are highly sensitive to modifications in the topological structure of GO, tend to be focused on specific analytical tasks and concentrate on the GO terms themselves rather than considering their textual definitions. RESULTS: We introduce simDEF, an efficient method for measuring semantic similarity of GO terms using their GO definitions, which is based on the Gloss Vector measure commonly used in natural language processing. The simDEF approach builds optimized definition vectors for all relevant GO terms, and expresses the similarity of a pair of proteins as the cosine of the angle between their definition vectors. Relative to existing similarity measures, when validated on a yeast reference database, simDEF improves correlation with sequence homology by up to 50%, shows a correlation improvement >4% with gene expression in the biological process hierarchy of GO and increases PPI predictability by > 2.5% in F1 score for molecular function hierarchy. AVAILABILITY AND IMPLEMENTATION: Datasets, results and source code are available at http://kiwi.cs.dal.ca/Software/simDEF CONTACT: [email protected] or [email protected] SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Feature selection is based on the notion that redundant and/or irrelevant variables bring no additional information about the data classes and can be considered noise for the predictor. As a result, the total feature set of a dataset could be minimized to only few features containing maximum discrimination information about the class. Classification accuracy is used as the evaluation measure in guiding the feature selection process. At the same time, such measure does not take into account the privacy of the resulting dataset. In this work, we incorporate privacy considerations into the very evaluation measure that is used to evaluate and select feature subsets. We consider privacy “during” the feature selection process and as such introduce a two-dimensional measure in automatic feature selection that takes into account both objectives of privacy and efficacy (e.g. accuracy) simultaneously and provides the data user with the flexibility of trading-off one for another.

An important problem in the knowledge discovery of trajectories is segmentation in subparts (subtrajectories). Existing algorithms for trajectory segmentation generally use explicit criteria to create segments. In this article, we propose segmenting trajectories using a novel, unsupervised approach, in which no explicit criteria are predetermined. To achieve this, we apply the Minimum Description Length (MDL) principle, which can measure homogeneity in the trajectory data by computing the similarities between landmarks (i.e. representative points of the trajectory) and the points in their neighborhood. Based on the homogeneity measurements, we propose an algorithm named Greedy Randomized Adaptive Search Procedure for Unsupervised Trajectory Segmentation (GRASP-UTS), which is a meta-heuristic that builds segments by modifying the number and positions of landmarks. We perform experiments with GRASP-UTS in two real-world datasets, using segment purity and coverage metrics to evaluate its efficiency. Experimental results demonstrate that GRASP-UTS correctly segmented sample trajectories without predetermined criteria, by computing similarities between landmarks and other trajectory points.

XACML (extensible Access Control Mark-up Language) is a declarative access control policy language that has unique language constructs for factoring out access control logic. These constructs make the specification of access control requirements more compact than decision trees, which can be considered the most natural way to specify access control logic. However, many publications report that performance of XACML policy decision point (PDP) engines is greatly affected by the structure of policy sets. In this paper we first explore the causes of potential inefficiencies of XACML policies, and then propose a procedure to re-structure policy sets vertically by modifying the distribution of access control logic among different configurations of structural elements, in order to remove much of this inefficiency. This is in contrast to horizontal re-ordering of constant structural elements. Our procedure can be applied regardless of the complexity and structure of the original policy set. We also compare the performance of policy sets that take advantage of the expressive power of XACML targets to decision trees.

A large amount of digital information collected and stored in databases creates new opportunities for knowledge discovery and data mining. The datasets, however, may contain personally identifiable information that needs to be protected. With high dimensionality of many large datasets, dimensionality reduction such as feature selection becomes indispensible. In this work, we aim at incorporating privacy into the very process of feature selection and as such, propose a privacy-aware filter-based feature selection method (PF-IFR). Our method enables data custodians to define a trade-off measure for controlling the amount of privacy and efficacy using filter-based feature selection techniques.

Maritime traffic monitoring is an important aspect of safety and security, particularly in close to port operations. While there is a large amount of data with variable quality, decision makers need reliable information about possible situations or threats. To address this requirement, we propose extraction of normal ship trajectory patterns that builds clusters using, besides ship tracing data, the publicly available International Maritime Organization (IMO) rules. The main result of clustering is a set of generated lanes that can be mapped to those defined in the IMO directives. Since the model also takes non-spatial attributes (speed and direction) into account, the results allow decision makers to detect abnormal patterns - vessels that do not obey the normal lanes or sail with higher or lower speeds.

We present a two-level approach to detect abnormal activities for vessels' routes. The data is obtained from the Automatic Identification System (AIS) which is required to be installed on vessels over specific gross tonnage. In the first level, we develope a Clustering algorithm: Density-based Spatial Clustering of Applications with Noise considering Speed and Direction (DBSCAN_SD). This algorithm is applied to pre-cluster the data points. Using domain knowledge in maritime, experts adjust the results produced by DBSCAN_SD with extra features. In this way, we get the optimal labeling result about whether a data point is normal or abnormal. In the second level, we use the labeled data generated in the first level to train the Parallel Meta-Learning (PML) algorithm on Hadoop. The results show that both accuracy and time complexity results are improved when we increase the number of nodes in a cluster.

When huge amounts of data are processed to extract knowledge, the situation becomes a challenge because the data mining techniques are not adapted to the space and time requirements. This challenge is more significant when the data is class imbalanced. Like many other machine learning algorithms, the success of the support vector machine (SVM) is limited when it is applied to the problem of learning from imbalanced datasets, especially on big datasets. In this paper, we are trying to apply an instance-weighted variant of the SVM, with a parallel Meta-learning algorithm using MapReduce, to deal with the big data class imbalance problem. We develop a symmetric weight boosting method to optimize the instance-weighted SVM. Experimental results on benchmark datasets and real application big datasets show that the proposed algorithm not only is effective on big data class imbalanced problem, but also reduces the training computational complexity significantly when the number of computing nodes increases.

Learning with class imbalance is always a challenging task in many real world applications such as the Internet, surveillance, security, and finance. Like many other successful machine learning algorithms, the success of the support vector machine (SVM) is limited when it is applied to the problem of learning from imbalanced datasets. SVM with different error costs has been widely used to deal with the class imbalanced problem. In this paper, we are trying to apply an instance-weighted variant of the SVM with both 1-norm and 2-norm format to deal with the class imbalance problem. We develop an asymmetric boosting method on the weights of the tradeoff parameters to optimize the instance-weighted SVM. The experimental results on the benchmark datasets show that the proposed algorithm is effective on the class imbalanced problem.

Multi-instance (MI) learning is different than standard propositional classification, as it uses a set of bags containing many instances as input. While the instances in each bag are not labeled, the bags themselves are, as positive or negative. In this paper, we present a novel multi-view, two-level classification framework to address the generalized multi-instance problems. We first apply supervised and unsupervised learning methods to transform a MI dataset into a multi-view, single meta-instance dataset. Then we develop a multi-view learning approach that can integrate the information acquired by individual view learners on the meta-instance dataset from the previous step, and construct a final model. Our empirical studies show that the proposed method performs well compared to other popular MI learning methods.

Automatic Target Recognition (ATR) methods have been successfully applied to detect possible objects or regions of interest in sonar imagery. It is anticipated that the additional information obtained from additional views of an object should improve the classification performance over single-aspect classification. In this paper the detection of mine-like objects (MLO) on the seabed from multiple side-scan sonar views is considered. We transform the multiple-aspect classification problem into a multiple-instance learning problem and present a framework based upon the concepts of multiple-instance classifiers. Moreover, we present another framework based upon the Dempster-Shafer (DS) concept of fusion from single-view classifiers. Our experimental results indicate that both the presented frameworks can be successfully used in mine-like object classification.

We have entered the big data age. Knowledge extraction from massive data is becoming more and more rewarding and urgent. MapReduce has provided a feasible framework for programming machine learning algorithms in Map and Reduce functions. The relatively simple programming interface has helped to solve machine learning algorithms' scalability problems. However, this framework suffers from an obvious weakness: it does not support iterations. This makes those algorithms requiring iterations difficult to fully explore the efficiency of MapReduce. In this paper, we propose to apply Meta-learning programmed with MapReduce to avoid parallelizing machine learning algorithms while also improving their scalability to big datasets. The experiments conducted on Hadoop fully distributed mode on Amazon EC2 demonstrate that our algorithm PML reduces the training computational complexity significantly when the number of computing nodes increases while gaining smaller error rates than those on one single node. The comparison of PML with the contemporary parallelized AdaBoost algorithm: AdaBoost.PL shows that PML has lower error rates.

The objective of this study is to develop a privacy-preserving framework for publishing longitudinal health data. Longitudinal health data contain valuable clinical information about patients collected over time and there is an increasing demand to use these data in medical and clinical research. However, since longitudinal health data contain personal information, improper release and usage of such data may violate privacy of patients. In this paper, we study the challenges of publishing longitudinal health data and propose a privacy notion, called (K,C)P-privacy together with a hybrid anonymization algorithm. This work is the first attempt to anonymize multidimensional longitudinal data to prevent both identity disclosure and attribute disclosure. Experimental results on the synthetic data demonstrate the effectiveness of our approach.

Policy administrators increasingly face the challenge of managing large policy bases, and this need becomes more acute with the growing importance of fine-grained access control models, e.g. ABAC. We have shown in previous work that simple policies mostly based on conjunctions of single attribute conditions, can be merged into more complex conditions composed of combinations of conjunctions and disjunctions of attribute/value pairs. Here, we propose an algorithm that uses a recursive process of subsumption applied on the original set of policies that results in a complex and short policy, often significantly compressing the original policy. We present this algorithm, and discuss the advantages of this approach, i.e. its performance when working on the policy structures encountered in real-life policy sets, its scalability, and its ability to deal with large alphabet sets.

Twitter is a micro logging service where worldwide users publish and share their feelings. However, sentiment analysis for Twitter messages ('tweets') is regarded as a challenging problem because tweets are short and informal. In this paper, we apply a novel approach for automatically classifying the sentiment and emotions of Twitter messages. These messages are hierarchically categorized on basis of neutrality, polarity (positive or negative) and presence of various emotions. The hierarchical classification approach (HC) is a specialization of the well-known flat classification task. The main difference between them is that when using HC, examples must be assigned to classes organized in a previously defined class hierarchy, while traditional flat classification does not take into account the hierarchical information. We applied our model to posts collected from Twitter regarding the 2011 season of the Brazilian Soccer League. Our results show that the proposed method outperforms the corresponding flat approach in emotion classification.

Real world data mining applications such as Mine Countermeasure Missions (MCM) involve learning from imbalanced data sets, which contain very few instances of the minority classes and many instances of the majority class. For instance, the number of naturally occurring clutter objects (such as rocks) that are detected typically far outweighs the relatively rare event of detecting a mine. In this paper we propose support vector machine with adaptive asymmetric misclassification costs (instances weighted) to solve the skewed vector spaces problem in mine countermeasure missions. Experimental results show that the given algorithm could be used for imbalanced sonar image data sets and makes an improvement in prediction performance.

Semi-supervised learning algorithms are widely used to build strong learning models when there are not enough labeled instances. Some semi-supervised learning algorithms, including co-training and co-EM, use multiple views to build learning models. P

In 2011 Matwin et al published a letter to JAMIA entitled ‘Performance of SVM and Bayesian classifiers on the systematic review classification task’.1 This letter continued a discussion on the relative benefits of using support vector machine (SVM) and Bayesian techniques for performing systematic reviews.2–4 In particular, it was suggested that the running time of algorithms must be taken into consideration when comparing their performances as it becomes very important for large datasets. Following up on this idea, we attempted to directly compare the performance of a Bayesian method with the SVM algorithm used by Cohen in his original work.4 The same SVM system (SVMlight5) with the same parameters as was …

As applications requiring access control and the environments in which they operate in become more complex, an acute need for better ways to manage access control rules has arisen. Decentralized access control, for example, requires sophisticated techniques for conflict detection and for managing rules across multiple applications with different rule formats. XACML is an OASIS standard whose interoperability qualities help in solving the latter problem. XACML has its own limitations, however. In particular, although it has the expressive power to specify very complex conditions like those needed in the ABAC (Attribute Based Access Control) model, users tend to avoid using its full power because of its verbosity. In this paper, we show how a non-technical notation we have proposed in our earlier work resolves this difficulty and allows users to work with a very compact and readable form of XACML rules, thus allowing them to take advantage of XACML's full expressive power. This expressive power can be exploited to write policies that are better organized. It can be easier, for example, to write a single possibly complex rule to cover a particular aspect of a policy as opposed to distributing the complexity over several rules with simpler conditions. As a result, policies are smaller, more compact, and easier to understand. Policy development becomes more manageable, allowing users to concentrate on the more central issue of choosing the model (RBAC, ABAC, PBAC or other) that is best suited to a particular application and policy. We show that using the full expressive power to better organize policies has a significant positive impact on PDP performance.

MOTIVATION: Annotation Enrichment Analysis (AEA) is a widely used analytical approach to process data generated by high-throughput genomic and proteomic experiments such as gene expression microarrays. The analysis uncovers and summarizes discriminating background information (e.g. GO annotations) for sets of genes identified by experiments (e.g. a set of differentially expressed genes, a cluster). The discovered information is utilized by human experts to find biological interpretations of the experiments. However, AEA isolates and tests for overrepresentation only individual annotation terms or groups of similar terms and is limited in its ability to uncover complex phenomena involving relationship between multiple annotation terms from various knowledge bases. Also, AEA assumes that annotations describe the whole object of interest, which makes it difficult to apply it to sets of compound objects (e.g. sets of protein-protein interactions) and to sets of objects having an internal structure (e.g. protein complexes). RESULTS: We propose a novel logic-based Annotation Concept Synthesis and Enrichment Analysis (ACSEA) approach. ACSEA fuses inductive logic reasoning with statistical inference to uncover more complex phenomena captured by the experiments. We evaluate our approach on large-scale datasets from several microarray experiments and on a clustered genome-wide genetic interaction network using different biological knowledge bases. The discovered interpretations have lower P-values than the interpretations found by AEA, are highly integrative in nature, and include analysis of quantitative and structured information present in the knowledge bases. The results suggest that ACSEA can boost effectiveness of the processing of high-throughput experiments. CONTACT: [email protected].

We are grateful to Professor Cohen for his letter and clarification of the support vector machine algorithm (SVM) results (in press). We agree that the results he supplies fill a gap in our paper. We could not have performed this comparison in our paper, as the first version was written prior to the publication of his own article,1 which in any case, as Dr Cohen points out, did not include the SVM results in terms of within-groups sum of squares (WSS). We would like, however, to be cautious about the broader conclusion from the results in table 1 of Dr Cohen's letter. While they are indeed superior to our factorized version of the complement naïve Bayes (FCNB) approach, they do not necessarily indicate the general superiority of the SVM classifier over the Bayesian methods. Our implementation was an extension of the classical naïve Bayes classifier for the kind of imbalanced data likely to be encountered when classifying abstracts for a systematic review. New exciting developments in the area of Bayesian text classification, such as discriminative multinomial naïve Bayes2 and latent Dirichlet allocation,3 are more than likely to improve our results significantly. This is the topic of our current research. Moreover, we would like to bring up another aspect related to the use of SVM versus Bayesian approaches as regards the running time of the algorithms. It is well known that SVM is significantly slower than the Bayesian methods. While this may be acceptable for the data used in Cohen1 and in Cohen et al4 and in our paper5 which have the order of 103 abstracts, it may not be the case for datasets orders of larger magnitude. However, datasets with the order of 104 abstracts are, as far as we know, common in the practice of systematic review production. When working recently6 with such large datasets we attempted to use SVM, but the running times of the train/test protocols on the computers available to us were unacceptably long. Finally, we want to comment briefly on the performance of FCNB/weight engineering (WE) on the Opioids dataset. As this dataset has a very high imbalance (very low inclusion rate), it is encouraging to see that the FCNB/WE method, which, as we discuss in our paper, has been engineered specifically to work well with such imbalanced data, indeed performs better than the standard SVM. We agree with Professor Cohen that application of the text mining methods developed specifically for imbalanced data is an interesting topic of research (see eg, Zhuang and Dai7). None. Not commissioned; internally peer reviewed.

Managing access control policies is a complex task. We argue that much of the complexity is unnecessary and mostly due to historical reasons. There are number of legacy policy specification languages that all have limitations of some kind. These limitations have forced policy implementers to use certain styles of writing policies, often resulting in inconsistencies. The detection and resolution of these inconsistencies has been widely researched and many solutions have been found. This paper highlights new possibilities for avoiding inconsistencies, drawing on the expressive power allowed in the condition field of rules in modern languages such as XACML. In particular, we show that making use of this expressive power has many advantages-it allows organizations to considerably reduce the number of policies and rules required to protect company assets; it provides improved views and summaries of related policies; and it allows increased scalability of analysis tools, such as tools that detect inconsistencies and tools that perform audits to verify compliance to regulations. Such tools are increasingly important in the current environment where the number of regulations governing company security continues to grow. In addition, we show how our user-friendly representation for the XACML language facilitates the use of complex conditions by increasing their readability. This increased readability has the additional benefit of allowing non-technical users to better understand the implementation of their policies. These factors all contribute to a lower risk of inconsistencies in policies.

OBJECTIVE: To determine whether a factorized version of the complement naïve Bayes (FCNB) classifier can reduce the time spent by experts reviewing journal articles for inclusion in systematic reviews of drug class efficacy for disease treatment. DESIGN: The proposed classifier was evaluated on a test collection built from 15 systematic drug class reviews used in previous work. The FCNB classifier was constructed to classify each article as containing high-quality, drug class-specific evidence or not. Weight engineering (WE) techniques were added to reduce underestimation for Medical Subject Headings (MeSH)-based and Publication Type (PubType)-based features. Cross-validation experiments were performed to evaluate the classifier's parameters and performance. MEASUREMENTS: Work saved over sampling (WSS) at no less than a 95% recall was used as the main measure of performance. RESULTS: The minimum workload reduction for a systematic review for one topic, achieved with a FCNB/WE classifier, was 8.5%; the maximum was 62.2% and the average over the 15 topics was 33.5%. This is 15.0% higher than the average workload reduction obtained using a voting perceptron-based automated citation classification system. CONCLUSION: The FCNB/WE classifier is simple, easy to implement, and produces significantly better results in reducing the workload than previously achieved. The results support it being a useful algorithm for machine-learning-based automation of systematic reviews of drug class efficacy for disease treatment.

Bayesian network classifiers have been widely used for classification problems. Given a fixed Bayesian network structure, parameters learning can take two different approaches: generative and discriminative learning. While generative parameter learning is more efficient, discriminative parameter learning is more effective. In this paper, we propose a simple, efficient, and effective discriminative parameter learning method, called Discriminative Frequency Estimate (DFE), which learns parameters by discriminatively computing frequencies from data. Empirical studies show that the DFE algorithm integrates the advantages of both generative and discriminative learning: it performs as well as the state-of-the-art discriminative parameter learning method ELR in accuracy, but is significantly more efficient.

This paper addresses the problem of data sharing among multiple parties in the following scenario: without disclosing their private data to each other, multiple parties, each having a private data set, want to collaboratively construct support vector machines using a linear, polynomial or sigmoid kernel function. To tackle this problem, we develop a secure protocol for multiple parties to conduct the desired computation. In our solution, multiple parties use homomorphic encryption and digital envelope techniques to exchange the data while keeping it private. All the parties are treated symmetrically: they all participate in the encryption and in the computation involved in learning support vector machines.

Data mining is a process to extract useful knowledge from large amounts of data. To conduct data mining, we often need to collect data. However, sometimes the data are distributed among various parties. Privacy concerns may prevent the parties from directly sharing the data and some types of information about the data. How multiple parties can collaboratively conduct data mining without breaching data privacy presents a grand challenge. In this paper, we propose a randomisation-based scheme for multi-parties to conduct data mining computations without disclosing their actual data sets to each other.

A considerable amount of system maintenance experience can be found in bug tracking and source code configuration management systems. Data mining and machine learning techniques allow one to extract models from past experience that can be used in future predictions. By mining the software change record, one can therefore generate models that can be used in future maintenance activities. In this paper, we present an example of such a model that represents a relation between pairs of files and show how it can be extracted from the software update records of a real world legacy system. We show how different sources of data can be used to extract sets of features useful in describing this model, as well as how results are affected by these different feature sets and their combinations. Our best results were obtained from text-based features, i.e. those extracted from words in the problem reports as opposed to syntactic structures in the source code.

In this paper we have solved the open problem of generating random vectors when the underlying structure obeyed by the dependence graph is a Directed Acyclic Graph (DAG). To the best of our knowledge, our work is of a pioneering sort. We present a formal strategy for the case when the DAG structure and the marginals are given. The paper presents the formal algorithm, proves its correctness, derives its complexity, and presents examples for both artificial data, and for date that is intended to artificially populate a medical database. The method has also been used for testing the ALARM network.

This paper describes the application of inductive methods to data extracted from both source code and software maintenance records. We would like to extract relations that indicate which files in, a legacy system, are relevant to each other in the context of program maintenance. We call these relations maintenance relevance relations. Such a relation could reveal existing complex interconnections among files in the system, which may in turn be useful in comprehending them. We discuss the methodology we employed to extract and evaluate the relations. We also point out some of the problems we encountered and our solutions for them. Finally, we present some of the results that we have obtained.

Abstract Inductive Logic Programming (ILP) is a field of research in which logic programs are synthesized (learned) from examples. There is a need for a variety of large datasets to evaluate ILP learners. The paper describes an example generator program GENEXthat provides the user with a language in which to describe large sets of structured examples. GENEXis available on the WWW. Copyright © 2001 John Wiley & Sons, Ltd.

Evidence for image classification can be considered to come from two sources: traditional statistical information derived algorithmically from image data, and model-based evidence arising from previous expertise and experience in a given application

Presents an approach to software reuse based on automatic analysis of program comments. First, domain terms are extracted from the comments in a semi-automatic procedure. Those terms are then used in an off-the-shelf case-based reasoning (CBR) system as indices for software modules. Noun phrases extracted from comments in LINPACK (a widely distributed linear algebra package) form the basis of simple domain models for linear systems. The process of constructing a reuse system is broken into three steps. A file containing comments from all LINPACK routines is processed to yield a list of technical phrases. The second step involves building domain models based on an analysis of these technical phrases and then indexing cases according to these models. Finally, tools provided by the REMIND CBR shell are used to create a case library incorporating this domain knowledge. Early experiments described in the paper show that noun phrases automatically extracted from the comments can provide useful functional description of the routines. The resulting simple domain models are usually sufficient for software reuse applications. Finally, we found standard CBR technology to be a viable means of constructing compositional software reuse libraries.>

Systems involving remote sensing analysis for airborne and satellite data in combination with geographic information systems are large and complex. The Canada Centre for Remote Sensing (CCRS) has created an expert system shell and several expert systems in order to provide image analysis programs with the necessary knowledge to solve difficult image processing problems, such as updating a forest inventory geographic information system. An interactive task interface (ILTI) provides an expert system with a Prolog module designed to answer queries from the image analysis program by retrieving knowledge from an image analysis knowledge base, the analyst advisor. Image analysis experts currently create ILTI's. They have found this to be a time-consuming task. An incremental/adaptive planner has been developed that will create a plan that emulates the ILTI's behavior by analyzing image processing session dialogues between a human expert and an image analysis program for several cases of forest updates. The planner relies on a knowledge base in order to generalize and modify plans acquired from session dialogues. The planner speeds and simplifies the creation of new expert systems.>

For a software component to be reusable, it must have two characteristics: it must be designed for reuse, and it must be available for reuse. Object-oriented development enables the reuse of designs and code in future projects. However, the extent to which object-oriented development currently lends itself to such reuse is frequently overstaged. The authors seek to improve the specification and retrieval mechanisms for reusable components in object-oriented languages. They describe and prototype a tool that enables programmers to describe a general specification for a function, in a language independent of detailed design constructs. The computer can then identify, using case-based reasoning, an existing code sample or collection of code samples that matches the specification.

It is argued that negotiation rules can be learned and invented by means of genetic algorithms. The work presented introduces a method, a system design, and a prototype implementation that uses genetic-based machine learning to acquire negotiation rules. The learned rules support a party involved in a two-party bargaining problem with multiple issues. It is assumed that both parties work towards a compromise deal. The method provides a framework in which genetic-based learning is applied repetitively on a changing problem representation. System design proposes a problem representation that is adequate to express bargaining processes and that is at the same time conducive to genetic-based learning. The authors report results of experiments with the prototype implementation. These results indicate that genetically learned rules, when used in real negotiations, yield results that are better than results obtained by humans in the same negotiation. The experiments indicate considerable robustness of genetically learned rules with respect to varying parameters defining the genetic operations on which the system relies in modeling negotiations. In terms of user support, experimental results show that in the bargaining process, a good rule is one that advises conceding in small steps and bringing new issues into the negotiation process.>

LEW (learning by watching), a machine learning system, is described. It was designed for knowledge acquisition in cooperation with an expert. LEW learns from examples of problem-solution (or question-answer) pairs by generalizing on differences in those pairs. In this sense, it belongs to the family of inductive learning methods. It provides for using background knowledge through the environment component of problem-solution pairs, thereby making constructive learning possible. The user can control the extent of the generalizations performed by LEW. The learning method is incremental and, to some extent, noise-resistant. The authors give an informal overview of the knowledge representation and the basic learning algorithm of LEW and indicate that the system's design meets the stated criteria and enables it to give helpful assistance, even in situations characterized by noisy or conflicting information and by lack of extensive background knowledge. The theory behind LEW is presented, along with rigorous definitions of its fundamental concepts and a general description of its learning algorithm. LEW's functioning with some larger examples, one from the QUIZ Advisor domain and another from the domain of block-world planning, is illustrated. The authors compare LEW with several other knowledge acquisition tools and introduce a precise characterization of learning from near misses and a near-miss metric. Possible extensions and enhancements to the system are noted.>

This paper describes LEW (learning by watching), an implementation of a novel learning technique, and discusses its application to the learning of plans. LEW is a domain‐independent learning system with user‐limited autonomy that is designed to provide robust performance in realistic knowledge acquisition tasks in a variety of domains. It partly automates the knowledge acquisition process for different knowledge types, such as concepts, rules, and plans. The inputs to the system, which we callcues, consist of an environmental component and of pairs containing a problem and its solution. Unlike traditional forms of “learning from examples”, in which the system uses the teacher's answer to improve the result of a prior generalization of an example, LEW treats the problem‐solution or question‐answer instances, i. e., the cues themselves, as the basic units for generalization.

Abstract Two problems involving the use of dynamic lists in Pascal are (1) the limited size of the heap and (2) the difficulty in creating copies of complex multi‐linked structures. Both these problems can be solved by writing the appropriate segment of memory to a file on disk. The heap segment may then be released. To recover the structure during a later phase of processing, the contents of the file can be transferred back onto the heap. This paper presents an algorithm to implement this solution.

This paper develops a method of mechanical deduction based on a graphical representation of the structure of proofs. Attempts to find a refutation(s) are recorded in the form of plans, corresponding to portions of an AND/OR graph search space and representing a purely deductive structure of derivation. This method can be applied to any initial base (set of nonnecessarily Horn clauses). Unlike the exhaustive (blind) backtracking which treats all the goals deduced in the course of a proof as equally probable sources of failure, his approach detects the exact source of failure. Only a small fragment of the solution space is kept on disk as a collection of pairs, each of which consists of a plan and a graph of constraints. The search strategy and the method of nonredundant processing of individual pairs which leads to a solution (if it exists) is presented. This approach is compared¿on a special case¿with a blind backtracking algorithm for which an exponential improvement is demonstrated. Some important implementation problems are discussed, and toplevel design of a mechanical deduction system implementing our algorithm is presented. It is proven that the algorithm is complete in the following sense: if for a given base a resolution refutation exists, then this refutation is found by the algorithm.