Behavioral modeling is an emerging machine learning area which aims to predict user actions, especially in commercial settings. Companies actively gather data such as clicks, likes, page views, card transactions, add-to-basket, or purchase events. However, the large size of data combined with hardships in applying sophisticated graph-based machine learning often leads to data not being used for modeling at all. In response to this, we propose the Monad platform. The primary focus of Monad is to train a large, private behavioral foundation model for each client company. The foundation model is then fine-tuned to any user behavioral prediction task, such as recommendations or churn. The Monad foundation models are based on our algorithms – Cleora and EMDE – which are award winning solutions (KDD Cup and other anonymized AI contests). Cleora and EMDE allow to process real-life datasets composed of billions of events in record time. In this paper, we present and analyze the temporal side of Monad. Time is a crucial aspect of behavioral training, because many target tasks such as propensity to buy rely on seasonal aspects and predictions are usually required to include a time frame (e.g. “Will the user buy brand X within a week from now?”). To tackle this problem, we present the concept of time-based data splits in training, as well as approaches towards time-based feature encoding, which do not require normalization or any feature statistics.

The complexity of industry-grade event-based datalakes grows dynamically each passing hour. Companies actively gather behavioral information on their customers, recording multiple types of events, such as clicks, likes, page views, card transactions, add-to-basket, or purchase events. In response to this, the Synerise Monad platform has been proposed. The primary focus of Monad is to produce Universal Behavioral Representations (UBRs) - large vectors encapsulating the behavioral patterns of each user. UBRs do not lose knowledge about individual events, in contrast to aggregated features or averaged embeddings. They are based on award-winning algorithms developed at Synerise - Cleora and EMDE - and allow to process real-life datasets composed of billions of events in record time. In this paper, we introduce a new aspect of Monad: private foundation models for behavioral data, trained on top of UBRs. The foundation models are trained in purely self-supervised manner and allow to exploit general knowledge about human behavior, which proves especially useful when multiple downstream models must be trained and time constraints are tight, or when labeled data is scarce. Experimental results show that the Monad foundation models can cut training time in half and require 3x less data to reach optimal results, often achieving state-of-the-art results.

The growth of time-sensitive heterogeneous data in industry-grade datalakes has recently reached unprecedented momentum. In response to this, we propose Synerise Monad - a prototype of a real-time behavioral modeling platform for event-based data streams. It automates representation learning and model training on massive data sources with arbitrary data structures. With Monad we showcase how to automatically process various data modalities, such as temporal, graph, categorical, decimal, and textual data types, in a time-sensitive way allowing for real-time time feature creation and predictions. Monad's distributed and scalable architecture coupled with efficient award-winning algorithms developed at Synerise - Cleora and EMDE - allows to process real-life datasets composed of billions of events in record time. The Monad ecosystem showcases a viable path towards real-time event-based AutoML.

Recently introduced self-supervised methods for image representation learning provide on par or superior results to their fully supervised competitors, yet the corresponding efforts to explain the self-supervised approaches lag behind. Motivated by this observation, we introduce a novel visual probing framework for explaining the self-supervised models by leveraging probing tasks employed previously in natural language processing. The probing tasks require knowledge about semantic relationships between image parts. Hence, we propose a systematic approach to obtain analogs of natural language in vision, such as visual words, context, and taxonomy. We show the effectiveness and applicability of those analogs in the context of explaining self-supervised representations. Our key findings emphasize that relations between language and vision can serve as an effective yet intuitive tool for discovering how machine learning models work, independently of data modality. Our work opens a plethora of research pathways towards more explainable and transparent AI.