Chess is a complex logical game involving ongoing strategic forward planning and evaluation. Solving chess puzzles is one of the most common ways of training and developing chess skills. It involves continuing the game from a certain initial chessboard state against a real or AI opponent until defeat or a significant advantage is achieved. To ensure solving chess puzzles is efficient and engaging for the real player, it is important to understand the difficulty of the puzzle and match it with the skill of the solver. Accurate and fast assessment of puzzle difficulty is therefore a critical problem that online chess platforms need to solve at scale to optimally match many real players with adequate puzzles. To solve this challenge, we propose a chess knowledge-agnostic strategy to predict puzzle difficulty based solely on the moves of the players against their opponents trying to solve the puzzles. Specifically designed deep convolutional neural networks (CNN) were deployed as supervised learning predictors, fed with player moves represented as multichannel chessboard images. Extensive testing of our model with almost 4 million training examples against Glicko-2 evaluated puzzle difficulty ratings—considered as ground truth—resulted in good predictive performance. This was acknowledged by our runner-up result as the 7th place in the IEEE Big Data 2024 Cup and highlighted the capability of fast puzzle difficulty prediction based only on players’ moves as evidence, with no prior chess knowledge nor utilization of computationally expensive chess engines.

Food security relies on factors like availability, access, and stability, often assisted by food imports when local production falters. Importantly, these imports stabilize supplies, mitigate shortages and price volatility, and enhance economic stability. Anticipating import requirements is vital for proactive food security planning. In this case study, we employ multiple forecasting models to predict food import for a large number of products from multiple countries. The results highlight varying algorithm performance across datasets. Traditional statistical models remain highly competitive compared to newer alternatives, especially for shorter time series. Our study introduces a multi-model forecasting approach to predict periodic food imports, a pivotal tool for food authorities.

Muon scattering tomography (MST) allows to reconstruct internal structure of the matter based on imaging of muon rays, Coulomb scattering when passing near heavier nuclei of the penetrated matter. While point of closest approach (PoCA) and maximum-likelihood expectation maximization (MLEM) methods offer reasonable MST solutions, they require large number of muons or huge computational cost while still struggling to distinguish between similar materials. As an alternative we propose to utilize deep, fully convolutional neural networks with atrous convolutions designed to deliver robust semantic image segmentation (SIS). The information from muon rays is summarized as image pixels containing path length and momentum weighted statistics of per-voxel muon scattering angles and displacements encoded independently along image rgb channels. Reserving one channel for iterative self-improvement by feeding classification outputs back as inputs delivered significant material detection improvements with nearly perfect localization and improved material differentiation. The presented iterative SIS model has been tested within IEEE BigData 2023 Cup dedicated to muon scattering tomography and scored $2 ^{nd}$ place significantly outperforming PoCA/MLEM and Catboost with 99.9% localization accuracy and IoU score in excess of 0.35, on the very challenging task of detecting nested material layers composed of a mixture of up to 22 elements and compounds.

An accurate evaluation of player-win likelihoods during online games is critical for controlling the attractiveness and immersion of the gameplay, especially if played against an AI bot. Predicting game victories is a very challenging problem heavily depends on the complexity and the stage of the game. With the multitude of ways and formats that the players, the gameplay, individual state and history can be encoded and utilized by machine learning (ML) models, it is fascinating to take part in the IEEE Big Data 2021 Cup and explore which data representations, feature engineering methods and prediction models work better than others and what levels of predictability they can achieve in the specific use case of predicting victories in Tactical Troops: Anthracite Shift online computer game. Our explorations throughout the game and a high 4th place in the Cup’s final indicate that a careful and comprehensive feature engineering methodology numerically capturing the last state of the game paired with the variants of the gradient boosting models offer a robust and competitive solution for this challenge.

At anytime in a tactics game, given the current status of the game players, it’s always interesting to predict who is going to win the game in the end. This is the challenge defined in the IEEE BigData Cup 2021, where the objective is to predict the likelihood of winning the game, Tactical Troops: Anthracite Shift, given information of the players as well as logged data of the game at the time the prediction needs to be made. To address this challenge, we have developed a hybrid model that combines a general model trained on all the data and two specific models trained on particular data of two distinct game modes. Our implementation was realized by LightGBM, one of the most popular tree-based libraries implementing the state-of-the-art Extreme Gradient Boosting algorithm. In addition to combined models, a thoroughly feature engineering and feature selection together with a special handling of missing values and categorical features, a careful hyper-parameter tuning, and a robust cross-validation strategy to avoid overfitting issues helped us win the 2ndplace in the competition.

The rapid expansion of Internet of Things (IoT) quickly evolves towards a connected network of AI-enabled, smart, multi-sensory devices that generate, consume and exchange enormous amounts of data. Stimulated by the omnipresent cloud-services and ever widening data bandwidth they aspire to control every aspect of our lives from transportation to our health, significantly increasing our reliance on the web-based services and their security. Already millions of cyber security alerts are generated every day and trigger increasingly costly investigations of the Security Operations Centres (SOC). In order to make their operation more efficient security warnings need to be reliably detected and classified based on various levels of severity, scale of potential damage or an ability to defend. We have responded to this challenge in the context of IEEE BigData Cup 2019 focused on predicting cyber security threats that require attention based on detailed logs of network activity leading to the security alert. We have developed a hybrid supervised learning ensemble model combining several state-of-the-art Extreme Gradient Boosting algorithms. Specifically, xGBoost and LightGBM model versions have been built on separate sets of features extracted from the raw logs of network events preceding the alerts and then synergically aggregated. Models' diversity imposed by algorithmic differences, complementary feature subsets, and individually optimized hyperparameters, combined with robust stratified cross-validation scheme resulted with the best true alerts detection rate yielding the AUC score in excess of 0.93, that outperformed all other 248 competing teams.

Sequential, predominantly temporal nature of the vast amounts of big data released every day from many different sources could potentially be linked, aligned along the time and deliver new evidence for the next generation predictive systems or knowledge discovery engines. However, big data owners are reluctant to share their data due to legally binding privacy and identity protection concerns, thereby posing a major hurdle preventing shared exploitation of big data on a massive scale. Data anonymization is expected to solve this problem, yet the current approaches are limited predominantly to univariate time series generalized by aggregation or clustering to eliminate identifiable uniqueness of individual data points or patterns. For multivariate time series, uniqueness among of the combination of values or patterns across multiple dimensions is much harder to eliminate due the to exponentially growing number of unique configurations of point values across multiple dimensions. Our method implements linearly scalable asynchronous summarization of multivariate time series independently at every dimension. As a result the series retain only a small subset of defining points at different times along multiple dimensions effectively breaking up the multivariate time series into a collection of summarized univariate time series that are perturbed from the original series in terms of actual points and pattern shapes. Current implementation of the anonymizing summarization involves shape preserving greedy elimination and aggregation that supports parallel cluster processing for big data implementation.

Rapidly evolving businesses generate massive amounts of time-stamped data sequences and cause a demand for both univariate and multivariate time series forecasting. For such data, traditional predictive models based on autoregression are often not sufficient to capture complex nonlinear relationships between multidimensional features and the time series outputs. In order to exploit these relationships for improved time series forecasting while also better dealing with a wider variety of prediction scenarios, a forecasting system requires a flexible and generic architecture to accommodate and tune various individual predictors as well as combination methods. In reply to this challenge, an architecture for combined, multilevel time series prediction is proposed, which is suitable for many different universal regressors and combination methods. The key strength of this architecture is its ability to build a diversified ensemble of individual predictors that form an input to a multilevel selection and fusion process before the final optimized output is obtained. Excellent generalization ability is achieved due to the highly boosted complementarity of individual models further enforced through cross-validation-linked training on exclusive data subsets and ensemble output postprocessing. In a sample configuration with basic neural network predictors and a mean combiner, the proposed system has been evaluated in different scenarios and showed a clear prediction performance gain.