Federated Learning (FL) presents a compelling distributed learning paradigm that enables resource-constrained clients to collaboratively train machine learning models while preserving data privacy. However, inter-client data heterogeneity poses fundamental challenges to federated optimization efficacy. Although Knowledge Distillation (KD) effectively addresses model performance alignment under heterogeneity, its inherent constraints restrict client parameter exploration capacity, thus, inducing confinement to suboptimal basins. To reconcile this trade-off, we introduce FedKDMR, a novel FL paradigm unifying KD constraints with exploration via model recombination. FedKDMR imposes global model consistency and robustness in training through dynamic KD while sufficiently harnessing model recombination-induced perturbations for diverse parameter exploration. We establish convergence guarantees for strongly convex and smooth objectives. Extensive experiments on FL benchmark datasets demonstrate that FedKDMR achieves a superior accuracy-robustness trade-off against state-of-the-art methods when tackling non-independent and identically distributed and heterogeneous data in FL environments.

Open-source software serves as a foundation for the internet and the cyber supply chain, but its exploitation is becoming increasingly prevalent. While advances in vulnerability detection for OSS have been significant, prior research has largely focused on static code analysis, often neglecting runtime indicators. To address this shortfall, we created a comprehensive dataset spanning five ecosystems, capturing features generated during the execution of packages and libraries in isolated environments. The dataset includes 9,461 package reports, of which 1,962 are identified as malicious, and encompasses both static and dynamic features such as files, sockets, commands, and DNS records. Each report is labeled with verified information and detailed sub-labels for attack types, facilitating the identification of malicious indicators when source code is unavailable. This dataset supports runtime detection, enhances detection model training, and enables efficient comparative analysis across ecosystems, contributing to the strengthening of supply chain security.

Abstract In this paper, we investigate Thompson sampling-based sequential block elimination approaches for dynamic assignment problems in a pure-exploration Multi-Armed Bandit (MAB) setting with limited budget constraints. The problem can be considered as a bandit game-play between the environment and a decision-maker in a metric space. Many instances of problems in fields such as e-commerce, logistics, mobility management, data management and operations research can be framed as dynamic assignment problems with budget constraints. Given an l-dimensional action space representing l variants of an entity and a budget for exploring the action space, the optimal dynamic assignment problem refers to the task of identifying the values to be assigned to different variants of the entity that maximizes the total reward by utilizing at most the given budget of rounds of play. We contribute a class of block elimination-based MAB algorithms specifically designed for the dynamic assignment problem with limited budget. Our algorithms begin by discretizing the continuous action space into a finite set of discrete actions, then proceed with a recursive block elimination procedure to remove sub-optimal actions. The elimination is carried out by calculating confidence bounds over blocks of actions. We explore two different confidence bound estimation techniques. We perform comprehensive experiments on two problem instances from distributed data management and logistics. Our results showcase that our approach yields a lower misidentification probability (i.e., the probability of recommending a non-optimal action) compared to state-of-the-art elimination-based pure-exploration MAB algorithms.

Context-aware data selectivity in Edge Computing (EC) requires nodes to efficiently manage the data collected from Internet of Things (IoT) devices, e.g., sensors, for supporting real-time and data-driven pervasive analytics. Data selectivity at the network edge copes with the challenge of deciding which data should be kept at the edge for future analytics tasks under limited computational and storage resources. Our challenge is to efficiently learn the access patterns of data-driven tasks (analytics) and predict which data arerelevant, thus, being stored in nodes’ local datasets. Task patterns directly indicate which data need to be accessed and processed to support end-users’ applications. We introduce a task workload-aware mechanism which adopts one-class classification to learn and predict the relevant data requested by past tasks. The inherent uncertainty in learning task patterns, identifying inliers and eliminating outliers is handled by introducing a lightweight fuzzy inference estimator that dynamically adapts nodes’ local data filters ensuring accurate data relevance prediction. We analytically describe our mechanism and comprehensively evaluate and compare against baselines and approaches found in the literature showcasing its applicability in pervasive EC.

Federated Learning (FL) is used to train machine learning models across distributed data and clients (devices). A key challenge in FL is addressing the complexities posed by heterogeneous clients with varying computational resources. While effective methods like knowledge distillation have been proposed to assist clients with limited resources, prevailing teacher-student models often focus solely on knowledge transfer, neglecting whether student clients can effectively and inclusively assimilate this knowledge. We introduce a leader-driven hierarchical distributed inclusive FL framework (LIFE) to address this challenge. LIFE engages hierarchical leaders across clients with various levels of computational capacities, aiming to optimize the performance of subordinate client models through a novel all-inclusive, leader-driven, level-by-level aggregation. Knowledge distillation is gradually achieved by fine-tuning subordinate models at all levels. Comprehensive experiments and comparisons with state-of-the-art methods across different tasks showcase that LIFE significantly enhances knowledge transfer across a variety of clients in an all-inclusive FL environment.

Distributed analytics involving classification tasks demand robust model training. Real-time arbitrary classification tasks on distributed clients pose challenges due to constraints in data sharing. Federated (Meta)-Learning (FML) has been introduced for global distributed (meta)-model training, which generalizes well over distributed data and classification tasks. Current FML approaches assume fixed labels over unskewed class proportions and data distributions along with uniform task distributions. However, global meta-models can only be used for tasks that do not require addressing arbitrary out-of- distribution label issues. In real-world cases, class imbalance and label shifting are common issues in clients' data. On-demand tasks arriving at clients involve unseen labels. Therefore, 'one (meta)-model-fits-all‘ is not the best option. To address these challenges, we introduce multiple cluster-based meta-models, each one tailored to specific label distribution. Our framework, coined Cluster-based & Label-aware Federated Meta-Learning (CL-FML), involves distributed client clustering based on label shifting and cluster-based FML identifying the most suitable clients to engage per task. CL-FML leverages lightweight data augmentation to deal with arbitrary class-imbalanced tasks. Our comprehensive experiments and comparative assessment against baselines showcase that CL-FML efficiently achieves high accuracy by fast convergence, significantly reducing training rounds and communication load.

Abstract We study the problem of predicting the next query to be recommended in interactive data exploratory analysis to guide users to correct content. Current query prediction approaches are based on sequence-to-sequence learning, exploiting past interaction data. However, due to the resource-hungry training process, such approaches fail to adapt to immediate user feedback. Immediate feedback is essential and considered as a signal of the user’s intent. We contribute with a novel query prediction ensemble mechanism, which adapts to immediate feedback relying on multi-armed bandits framework. Our mechanism, an extension to the popular Exp3 algorithm, augments Transformer-based language models for query predictions by combining predictions from experts, thus dynamically building a candidate set during exploration. Immediate feedback is leveraged to choose the appropriate prediction in a probabilistic fashion. We provide comprehensive large-scale experimental and comparative assessment using a popular online literature discovery service, which showcases that our mechanism (i) improves the per-round regret substantially against state-of-the-art Transformer-based models and (ii) shows the superiority of causal language modelling over masked language modelling for query recommendations.

Federated Learning (FL) has been successfully adopted for distributed training and inference of large-scale Deep Neural Networks (DNNs). However, DNNs are characterized by an extremely large number of parameters, thus, yielding significant challenges in exchanging these parameters among distributed nodes and managing the memory. Although recent DNN compression methods (e.g., sparsification, pruning) tackle such challenges, they do not holistically consider an adaptively controlled reduction of parameter exchange while maintaining high accuracy levels. We, therefore, contribute with a novel FL framework (coined FedDIP), which combines (i) dynamic model pruning with error feedback to eliminate redundant information exchange, which contributes to significant performance improvement, with (ii) incremental regularization that can achieve extreme sparsity of models. We provide convergence analysis of FedDIP and report on a comprehensive performance and comparative assessment against state-of-the-art methods using benchmark data sets and DNN models. Our results showcase that FedDIP not only controls the model sparsity but efficiently achieves similar or better performance compared to other model pruning methods adopting incremental regularization during distributed model training. The code is available at: https://github.com/EricLoong/feddip.

Internet of Things offers the infrastructure for smooth functioning of autonomous context-aware devices being connected towards the Cloud. Edge Computing (EC) relies between the IoT and Cloud providing significant advantages. One advantage is to perform local data processing (limited latency, bandwidth preservation) with real time communication among IoT devices, while multiple nodes become hosts of the collected data (reported by IoT devices). In this work, we provide a mechanism for the exchange of data synopses (summaries of extracted knowledge) among EC nodes that are necessary to give the knowledge on the data present in EC environments. The overarching aim is to intelligently decide on when nodes should exchange data synopses in light of efficient execution of tasks. We enhance such a decision with a stochastic optimization model based on the Theory of Optimal Stopping. We provide the fundamentals of our model and the relevant formulations on the optimal time to disseminate data synopses to network edge nodes. We report a comprehensive experimental evaluation and comparative assessment related to the optimality achieved by our model and the positive effects on EC.

Several data mining tasks focus on repeatedly inspecting multidimensional data regions summarized by a statistic. The value of this statistic (e.g., region-population sizes, order moments) is used to classify the region's interesting-ness. These regions can be naively extracted from the entire dataspace - however, this is extremely time-consuming and compute-resource demanding. This paper studies the reverse problem: analysts provide a cut-off value for a statistic of interest and in turn our proposed framework efficiently identifies multidimensional regions whose statistic exceeds (or is below) the given cut-off value (according to user's needs). However, as data dimensions and size increase, such task inevitably becomes laborious and costly. To alleviate this cost, our solution, coined SuRF (SUrrogate Region Finder), leverages historical region evaluations to train surrogate models that learn to approximate the distribution of the statistic of interest. It then makes use of evolutionary multi-modal optimization to effectively and efficiently identify regions of interest regardless of data size and dimensionality. The accuracy, efficiency, and scalability of our approach are demonstrated with experiments using synthetic and real-world datasets and compared with other methods.

Analysts wishing to explore multivariate data spaces, typically issue queries involving selection operators, i.e., range or equality predicates, which define data subspaces of potential interest. Then, they use aggregation functions, the results of which determine a subspace’s interestingness for further exploration and deeper analysis. However, Aggregate Query (AQ) results are scalars and convey limited information and explainability about the queried subspaces for enhanced exploratory analysis. Analysts have no way of identifying how these results are derived or how they change w.r.t query (input) parameter values. We address this shortcoming by aiding analysts to explore and understand data subspaces by contributing a novel explanation mechanism based on machine learning. We explain AQ results using functions obtained by a three-fold joint optimization problem which assume the form of explainable piecewise-linear regression functions. A key feature of the proposed solution is that the explanation functions are estimated using past executed queries. These queries provide a coarse grained overview of the underlying aggregate function (generating the AQ results) to be learned. Explanations for future, previously unseen AQs can be computed without accessing the underlying data and can be used to further explore the queried data subspaces, without issuing more queries to the backend analytics engine. We evaluate the explanation accuracy and efficiency through theoretically grounded metrics over real-world and synthetic datasets and query workloads.

Large organizations have seamlessly incorporated data-driven decision making in their operations. However, as data volumes increase, expensive big data infrastructures are called to rescue. In this setting, analytics tasks become very costly in terms of query response time, resource consumption, and money in cloud deployments, especially when base data are stored across geographically distributed data centers. Therefore, we introduce an adaptive Machine Learning mechanism which is light-weight, stored client-side, can estimate the answers of a variety of aggregate queries and can avoid the big data backend. The estimations are performed in milliseconds and are inexepensive as the mechanism learns from past analytical-query patterns. However, as analytic queries are ad-hoc and analysts' interests change over time we develop solutions that can swiftly and accurately detect such changes and adapt to new query patterns. The capabilities of our approach are demonstrated using extensive evaluation with real and synthetic datasets.

The state-of-the-art approaches for scalable kNN query processing utilise big data parallel/distributed platforms (e.g., Hadoop and Spark) and storage engines (e.g, HDFS, NoSQL, etc.), upon which they build (tree based) indexing methods for efficient query processing. However, as data sizes continue to increase (nowadays it is not uncommon to reach several Petabytes), the storage cost of tree-based index structures becomes exceptionally high. In this work, we propose a novel perspective to organise multivariate (mv) datasets. The main novel idea relies on data space probabilistic transformations and derives a Space Transformation Organisation Structure (STOS) for mv data organisation. STOS facilitates query processing as if underlying datasets were uniformly distributed. This approach bears significant advantages. First, STOS enjoys a minute memory footprint that is many orders of magnitude smaller than indexes in related work. Second, the required memory, unlike related work, increases very slowly with dataset size and, thus, enjoys significantly higher scalability. Third, the STOS structure is relatively efficient to compute, outperforming traditional index building times. The new approach comes bundled with a distributed coordinator-based query processing method so that, overall, lower query processing times are achieved compared to the state-of-the-art index-based methods. We conducted extensive experimentation with real and synthetic datasets of different sizes to substantiate and quantify the performance advantages of our proposal.

Analysts wishing to explore multivariate data spaces, typically pose queries involving selection operators, i.e., range or radius queries, which define data subspaces of possible interest and then use aggregation functions, the results of which determine their exploratory analytics interests. However, such aggregate query (AQ) results are simple scalars and as such, convey limited information about the queried subspaces for exploratory analysis. We address this shortcoming aiding analysts to explore and understand data subspaces by contributing a novel explanation mechanism coined XAXA: eXplaining Aggregates for eXploratory Analytics. XAXA's novel AQ explanations are represented using functions obtained by a three-fold joint optimization problem. Explanations assume the form of a set of parametric piecewise-linear functions acquired through a statistical learning model. A key feature of the proposed solution is that model training is performed by only monitoring AQs and their answers on-line. In XAXA, explanations for future AQs can be computed without any database (DB) access and can be used to further explore the queried data subspaces, without issuing any more queries to the DB. We evaluate the explanation accuracy and efficiency of XAXA through theoretically grounded metrics over real-world and synthetic datasets and query workloads.

We investigate the quality of aggregation and predictive analytics in edge computing environments. Edge analytics require pushing processing and inference to the edge of a network of sensing & actuator nodes, which enables huge amount of contextual data to be processed in real time that would be prohibitively complex and costly to transfer on centralized locations. We propose a quality-aware, time-optimized edge analytics model that supports communication efficient predictive modeling within the edge network. Our idea rests on the capability of edge nodes to intelligently decide when and which data to deliver and process in light of minimizing the communication overhead and maximizing the quality of analytics results. We provide mathematical modeling, performance and comparative assessment over real datasets showing its benefits in edge computing environments.

Recommender systems have been intensively used to create personalised profiles, which enhance the user experience. In certain areas, such as e-learning, this approach is short-sighted, since each student masters each concept through different means. The progress from one concept to the next, or from one lesson to another, does not necessarily follow a fixed pattern. Given these settings, we can no longer use simple structures (vectors, strings, etc.) to represent each user's interactions with the system, because the sequence of events and their mapping to user's intentions, build up into more complex synergies. As a consequence, we propose a graph-based interpretation of the problem and identify the challenges behind (a) using graphs to model the users' journeys and hence as the input to the recommender system, and (b) producing recommendations in the form of graphs of actions to be taken.

Recent trends aim to incorporate advanced data analytics capabilities within DBMSs. Linear regression queries are fundamental to exploratory analytics and predictive modeling. However, computing their exact answers leaves a lot to be desired in terms of efficiency and scalability. We contribute a novel predictive analytics model and associated regression query processing algorithms, which are efficient, scalable and accurate. We focus on predicting the answers to two key query types that reveal dependencies between the values of different attributes: (i) mean-value queries and (ii) multivariate linear regression queries, both within specific data subspaces defined based on the values of other attributes. Our algorithms achieve many orders of magnitude improvement in query processing efficiency and near-perfect approximations of the underlying relationships among data attributes.

Fundamental to many predictive analytics tasks is the ability to estimate the cardinality (number of data items) of multi-dimensional data subspaces, defined by query selections over datasets. This is crucial for data analysts dealing with, e.g., interactive data subspace explorations, data subspace visualizations, and in query processing optimization. However, in many modern data systems, predictive analytics may be (i) too costly money-wise, e.g., in clouds, (ii) unreliable, e.g., in modern Big Data query engines, where accurate statistics are difficult to obtain/maintain, or (iii) infeasible, e.g., for privacy issues. We contribute a novel, query-driven, function estimation model of analyst-defined data subspace cardinality. The proposed estimation model is highly accurate in terms of prediction and accommodating the well-known selection queries: multi-dimensional range and distance-nearest neighbors (radius) queries. Our function estimation model: (i) quantizes the vectorial query space, by learning the analysts’ access patterns over a data space, (ii) associates query vectors with their corresponding cardinalities of the analyst-defined data subspaces, (iii) abstracts and employs query vectorial similarity to predict the cardinality of an unseen/unexplored data subspace, and (iv) identifies and adapts to possible changes of the query subspaces based on the theory of optimal stopping. The proposed model is decentralized, facilitating the scaling-out of such predictive analytics queries. The research significance of the model lies in that (i) it is an attractive solution when data-driven statistical techniques are undesirable or infeasible, (ii) it offers a scale-out, decentralized training solution, (iii) it is applicable to different selection query types, and (iv) it offers a performance that is superior to that of data-driven approaches.

We study a novel solution to executing aggregation (and specifically COUNT) queries over large-scale data. The proposed solution is generally applicable, in the sense that it can be deployed in environments in which data owners may or may not restrict access to their data and allow only `aggregation operators' to be executed over their data. For this, it is based on predictive analytics, driven by queries and their results. We propose a machine learning (ML) framework for the task (which can be adapted for different aggregates as well). We focus on the widely used set-cardinality (i.e., COUNT) aggregation operator, as it is a fundamental operator for both internal data system optimisations and for aggregation-query analytics. We contribute a novel, query-driven ML model whose goals are to: (i) learn the query space (access patterns), (ii) associate (complex) aggregation queries with the cardinality of their results, (iii) define query similarity and use it to predict the cardinality of the answer set of an ad-hoc incoming query. Our ML model incorporates incremental learning algorithms for ensuring high prediction accuracy even when both the querying patterns and the underlying data change. The significance of contribution lies in that it (i) is the only query-driven solution applicable over general environments which include restricted-access data, (ii) offers incremental learning adjusted for arriving ad-hoc queries, which is well suited for big data analytics, and (iii) offers a performance (in terms of prediction accuracy and time, and memory requirements) that is superior to data-centric approaches. We provide a comprehensive performance evaluation of our model, evaluating its sensitivity and comparative advantages versus acclaimed data-centric methods (self-tuning histograms, sampling, and multidimensional histograms).

Distance-based nearest neighbours (dNN) queries and aggregations over their answer sets are important for exploratory data analytics. We focus on the Set Cardinality Prediction (SCP) problem for the answer set of dNN queries. We contribute a novel, query-driven perspective for this problem, whereby answers to previous dNN queries are used to learn the answers to incoming dNN queries. The proposed novel machine learning (ML) model learns the dynamically changing query patterns space and thus it can focus only on the portion of the data being queried. The model enjoys several comparative advantages in prediction error and space requirements. This is in addition to being applicable in environments with sensitive data and/or environments where data accesses are too costly to execute, where the data-centric state-of-the-art is inapplicable and/or too costly. A comprehensive performance evaluation of our model is conducted, evaluating its comparative advantages versus acclaimed methods (i.e., different self-tuning histograms, sampling, multidimensional histograms, and the power-method).

Solving the missing-value (MV) problem with small estimation errors in big data environments is a notoriously resource-demanding task. As datasets and their user community continuously grow, the problem can only be exacerbated. Assume that it is possible to have a single machine (`Godzilla'), which can store the massive dataset and support an ever-growing community submitting MV imputation requests. Is it possible to replace Godzilla by employing a large number of cohort machines so that imputations can be performed much faster, engaging cohorts in parallel, each of which accesses much smaller partitions of the original dataset? If so, it would be preferable for obvious performance reasons to access only a subset of all cohorts per imputation. In this case, can we decide swiftly which is the desired subset of cohorts to engage per imputation? But efficiency and scalability is just one key concern! Is it possible to do the above while ensuring comparable or even better than Godzilla's imputation estimation errors? In this paper we derive answers to these fundamentals questions and develop principled methods and a framework which offer large performance speed-ups and better, or comparable, errors to that of Godzilla, independently of which missing-value imputation algorithm is used. Our contributions involve Pythia, a framework and algorithms for providing the answers to the above questions and for engaging the appropriate subset of cohorts per MV imputation request. Pythia functionality rests on two pillars: (i) dataset (partition) signatures, one per cohort, and (ii) similarity notions and algorithms, which can identify the appropriate subset of cohorts to engage. Comprehensive experimentation with real and synthetic datasets showcase our efficiency, scalability, and accuracy claims.

We study the performance improvement of Location Based Services through the identification and subsequent use of groups of mobile nodes. In our scheme we exploit the formation of nodes into groups in order to reduce the computation load incurred in back-end systems (e.g., Location Servers) and the associated network overhead. The back-end systems track the position and communicate with the Group Leader (GL). The GL, in turn, passes the received information to the members of the group (e.g., through short-range communications). The formation of mobile groups is validated over time to avoid misinterpreted temporary groupings which could endanger the adoption of the reduced load/overhead scheme. A time scheduling scheme based on the Optimal Stopping Theory assists in the finalization of the group validity. Metrics like group compactness are thoroughly assessed in line with the optimal stopping time scheme to increase confidence on group validity and persistence. Performance assessment reveals significant benefits for the considered location based services system.

Future Web business models involve virtual environments where entities interact in order to sell or buy information goods. Such environments are known as Information Markets (IMs). Intelligent agents are used in IMs for representing buyers or information providers (sellers). We focus on the decisions taken by the buyer in the purchase negotiation process with sellers. We propose a reasoning mechanism on the offers (prices of information goods) issued by sellers based on fuzzy logic. The buyer’s knowledge on the negotiation process is modeled through fuzzy sets. We propose a fuzzy inference engine dealing with the decisions that the buyer takes on each stage of the negotiation process. The outcome of the proposed reasoning method indicates whether the buyer should accept or reject the sellers’ offers. Our findings are very promising for the efficiency of automated transactions undertaken by intelligent agents.

Mobile applications are required to operate in highly dynamic pervasive computing environments of dynamic nature and predict the location of mobile users in order to act proactively. We focus on the location prediction and propose a new model/framework. Our model is used for the classification of the spatial trajectories through the adoption of Machine Learning (ML) techniques. Predicting location is treated as a classification problem through supervised learning. We perform the performance assessment of our model through synthetic and real-world data. We monitor the important metrics of prediction accuracy and training sample size.

In certain settings, like for example a Wireless Sensor Network (WSN), contextual information (context) needs to be disseminated between nodes and then interpreted. Dissemination is typically performed through a wireless network infrastructure where resources are scarce. Our focus is on the design/implementation of a context compression scheme that tries to minimize the pieces of information exchanged over the network. Our scheme heavily relies on the multi-value(vectorial) nature of context dissemination messages that flow throughout the network. We adopt the Principal Component Analysis and determine the statistical dependencies between the context vector components. We manage to reduce (compress)the transmitted contextual information down to the identified principal components. A comparative assessment with other energy efficient models is reported indicating the capability of the proposed model to minimize resource consumption.

Mobile applications are required to operate in ubiquitous environments of dynamic nature. Specifically, the availability of resources and services may vary significantly during a typical session of system operation. As a consequence, mobile applications need to be capable of adapting to these changes to ensure the best possible level of service to the user. Therefore, such adaptive applications may have pre-evaluated the appropriate knowledge of their environment to act efficiently. Such knowledge is not known a priori, so information prediction and proactivity should enhance and extend the functionality of such applications in order to be adaptable to the future changes of their underlying computational environment. In this paper, we discuss and evaluate such a context prediction algorithm.