The rapid advancement of deep learning and computer vision technologies is transforming sports analytics, enabling more precise performance analysis and motion tracking. However, accurately estimating hammer throw distances without physical measurements remains a challenge due to the complexity of the motion and the high speed of the projectile. The ability to accurately predict hammer throw distances would be particularly useful in indoor training settings, where the hammer’s trajectory is stopped by safety nets or mattresses from a short distance. To address this challenge, we introduce a deep learning-based method that combines object detection with physics-based modeling to estimate motion outcomes. Our methodology employs a dual-camera setup to capture side and back views of the throw, applies advanced object detection to track the hammer’s position frame by frame, and reconstructs 3D trajectory points to estimate the release speed, angle, and height that allow predicting the throw distance. By enabling quantitative assessment of performance without relying on physical landing measurements, the proposed approach supports objective training feedback on the release parameters and distance estimation in typical training environments where traditional distance-based evaluation is not feasible. Our approach enables accurate performance evaluation in spatially constrained settings. Experimental results demonstrate that our approach achieves an average error of less than three meters ( ∼ 4 %) in estimating the distances compared to ground truth measurements. Our codes and trained models will be made publicly available once the paper is published at https://github.com/AhmedEH28/Hammer-Throw-Distance-Estimation .

High speed and efficient accelerators such as neuromorphic photonics are becoming increasingly popular, as they can significantly improve computation speed and energy efficiency of matrix-based calculations, leading to femtojoule per MAC efficiency. However, deploying existing DL models on such platforms is not trivial, as a wide range of photonic neural network (PNN) architectures relies on incoherent setups and power-adding operational schemes that cannot natively represent negative quantities. This results in additional hardware complexity that increases cost and reduces energy efficiency. To overcome this, we can train non-negative neural networks and potentially exploit the full range of incoherent neuromorphic photonic capabilities. However, the existing non-negative training approaches cannot achieve the same level of accuracy as their regular counterparts. To this end, we introduce a methodology to obtain the non-negative isomorphic equivalents of regular artificial neural networks (ANNs) that meet the requirements of neuromorphic hardware as it allows one to perform inference with non-negative only parameters. Furthermore, we also introduce a sign-preserving optimization approach that enables the training of such isomorphic networks in a non-negative manner. The source code implementation of the proposed numerical framework is publicly available at: https://github.com/eakirtas/nn_isomorphics_24 .

• A deep learning method that models data uncertainty for uncertainty estimation is proposed. • A method that generates soft labels in an adaptive manner is used for the optimization. • The uncertainty-aware learning process improves state-of-the-art methodologies in out-of-distribution detection. In the pursuit of enhancing the trustworthiness of deep learning models, there has been a growing interest in improving their uncertainty estimation for more reliable decision-making. Recent methodologies have concentrated on refining uncertainty estimation particularly in the context of identifying out-of-distribution input samples. This work introduces and explores two distinct categories of uncertainty: class uncertainty and instance uncertainty . The former highlights uncertainty stemming from inter-class resemblances, while the latter addresses analogous uncertainty within individual instances. We propose a novel framework, Adaptive Similarity Labeling (ASL), that captures both class-level and instance-level uncertainty by adaptively assigning soft labels based on semantic similarity and instance difficulty. ASL mitigates feature collapse and improves uncertainty estimation without adding inference-time overhead, and can be used regardless of the uncertainty metric used. The proposed approach is architecture-agnostic and can be combined with recent state-of-the-art architectures for uncertainty estimation. Experiments conducted on several datasets demonstrate the effectiveness of ASL in improving uncertainty estimation and out-of-distribution detection across diverse tasks.

In the field of self-supervised learning, the Image-based Joint-Embedding Predictive Architecture (I-JEPA) has shown impressive results in capturing semantic information from images without the need for strong augmentations between views. I-JEPA learns to predict the representations of multiple masked target blocks within the same image, using the representation of a single unmasked context block. To achieve this, I-JEPA reconstructs the masked patches in the representation space rather than at the pixel level, as done in previous works. However, accurately reconstructing abstract representations of patches is a challenging task, as pointwise matching focuses on exact reconstruction of individual target representations rather than the preservation of their local geometric structure. In this work, we present a probabilistic matching approach that can be applied to I-JEPA’s learning process to encourage the output of semantically richer representations. Specifically, we propose a probabilistic formulation of I-JEPA, which involves efficiently modeling and matching the probability distributions of the missing patches between the target block representations and the network’s predictions based solely on the context block. Through rigorous evaluations, we demonstrate that the proposed methodology provides consistent improvements over I-JEPA in downstream classification and transfer learning tasks.

While Reinforcement Learning (RL) has demonstrated success in numerous applications, ensuring the consistency and safety of learned policies remains a significant challenge, particularly in safety-critical domains. To enable the reliable and safe operation of RL agents, it is essential to quantify the agent’s uncertainty regarding its action selection, commonly referred to as Policy Uncertainty. In this work, we propose a novel approach for estimating policy uncertainty in Deep Q-learning (DQN) agents by leveraging Dropout Distillation Networks. Our methodology consists of three key steps: (1) training a DQN agent with dropout applied to its network layers, (2) using the trained agent to interact with the RL environment and collect a distillation dataset, and (3) training a student network (DDN) using the collected dataset. Furthermore, we identify a gap in the literature, where existing works primarily introduce uncertainty estimation algorithms without assessing their estimation accuracy. To address this, we propose a novel RL policy uncertainty evaluation algorithm capable of assessing the calibration of any agent that provides policy uncertainty estimates, utilizing a simulator of the RL environment. We compare our proposed policy uncertainty estimation approach against Monte Carlo Dropout, a widely used technique in the literature. Experimental results indicate that our method consistently achieves better calibration while requiring only a single forward pass, a crucial advantage for real-time RL applications.

Even recent Deep Learning (DL) architectures are highly sensitive to training hyperparameters, initial weights, and data distributions, making the development of fast and stable optimization methods a challenging task. To this end, generations of researchers have pursued the development of robust methods to train DL architectures. Multiplicative updates naturally take into account the magnitude of parameters and despite their significant contributions on the early development of Machine Learning (ML) and the strong theoretical claims, they have been merely studied in the context of DL. To this end, in this work, we propose two alternative Stochastic Gradient Descent (SGD) optimization methods that are based on a novel multiplicative update term, which proportionally scales the parameters using the normalized gradients, decoupling in this way the magnitude of gradients from the update. More specifically, we formulate two SGD alternatives, one multiplicative and another hybrid multiplicative-additive, which overcomes the limitation of purely multiplicative updates that constraints the change of parameter’s sign. The proposed methods accelerate training while leading to more robust models in contrast to traditionally used SGD, and we experimentally demonstrate its effectiveness in a wide range of tasks. Such tasks range from convex and non-convex optimization to difficult image classification benchmarks applying a wide range of Artificial Neural Network (ANN) architectures, providing quantitative and qualitative experimental results.

Machine learning is becoming increasingly popular across various applications and has led to state-of-the-art results, but it faces challenges related to its trustworthiness. One aspect of making deep learning models more trustworthy is improving their ability to estimate the uncertainty of whether a sample is from the in-domain (ID) data distribution or not. Especially, neural networks have a tendency to make overly confident extrapolations and struggle to convey their uncertainty, which can limit their trustworthiness. Recent approaches have employed Radial Basis Function (RBF)-based models with great success in improving uncertainty estimation in Deep Learning. However, such models assume a unimodal distribution of the data for each class, which we show is critical for out-of-distribution sample detection, but can be limiting in many real world cases. To overcome these limitations, in this paper, we propose a method for training a deep model utilizing the inherent different modalities that naturally arise in a class in real data, which we call subclasses , leading to improved uncertainty quantification. The proposed method leverages a variance-preserving reconstruction-based representation learning approach that prevents feature collapse and enables robust discovery of subclasses, further improving the effectiveness of the proposed approach. The improvement of the approach is demonstrated using extensive experiments on several datasets.

Existing knowledge distillation methods typically do not adequately account for the representativeness of sampled training data or the adverse effects of missing classes within mini-batches, both of which can lead to suboptimal knowledge transfer. In this paper, we introduce A nchor-based K nowledge D istillation (AKD), a method that leverages the most informative and representative samples, referred to as anchors , during the transfer process, and matches the representation distribution of the data in the feature space rather than their actual representations. Anchors are strategically chosen based on their ability to encapsulate critical features of the data distribution, ensuring that the student model focuses on the most informative aspects of the teacher’s knowledge. The proposed method enables the introduction of information from a wider variety of classes at each mini-batch iteration, ensuring a more balanced data distribution and thereby improving the knowledge transfer process. Furthermore, by leveraging the static nature of anchors, we enhance the student model’s representation learning through attention maps, improving both convergence and generalization. The proposed approach is extensively evaluated across various datasets and tasks, demonstrating that the incorporation of anchors into the knowledge distillation process improving the accuracy and trustworthiness of the resulting models.

Being able to accurately predict the generation of Renewable Energy Sources (RES), such as photovoltaic and wind generation, can provide valuable information for various stakeholders, including grid operator, energy producers, and consumers. Deep Learning (DL) provided powerful tools for this goal. However, it is not trivial to leverage data from multiple countries to improve forecasting accuracy due to distribution shift phenomena that are often involved. Adaptive normalization approaches have recently emerged as an effective tool for tackling such difficulties. These formulations, despite their very promising results in classification tasks, often face challenges in (auto)regressive tasks, especially when combined with recent powerful DL architectures. To overcome this limitation we introduce a novel residual-based adaptive normalization layer that is capable of re-introducing the information discarded during the normalization process to the forecasting model. The proposed method enables the use of data that are generated by different distributions but expresses the same phenomenon, allowing for exploiting additional data sources, e.g., multiple countries, when training DL models, leading to significant improvements in forecasting accuracy.

Financial time series are sequences of price observations related to financial assets collected over time. Deep Learning (DL) is currently standing as the predominant approach for addressing various time series tasks, including problems in finance, such as the development of trading agents using Deep Reinforcement Learning (DRL). However, the noisy and temporal nature of such data as well as their non-stationarity pose substantial challenges to current methodologies. DL models suffer from overfitting noise, frequently arising from the absence of strong priors. In this paper, we address the instability of trading DRL agents due to noise by proposing an end-to-end hybrid trainable filtering and feature extraction approach. The proposed method employs Gaussian filters as priors and can be attached at the beginning of any DL architecture forming a hybrid model-based and data-driven model that can directly process the raw input data. The bandwidth of the filters is determined through the learning process, ultimately allowing the agent to autonomously determine the optimal bandwidth for the task and data at hand, without requiring any additional supervision. Moreover, the proposed method leverages high-order derivatives to address the non-stationarity of financial data and provides multiple views of the input signal efficiently utilized by the subsequent model. We conduct experiments with a plethora of financial assets from the Foreign Exchange Market (FOREX) and demonstrate the method's efficiency when compared to alternative processing pipelines.

While softmax cross-entropy (CE) loss is the standard objective for supervised classification, it primarily focuses on the ground-truth classes, ignoring the relationships between the nontarget, complementary classes. This leaves valuable information unexploited during optimization. In this work, we propose a novel loss function, one-cold CE (OCCE) loss, which addresses this limitation by structuring the activations of these complementary classes. Specifically, for each class, we define an anticlass, which consists of everything that is not part of the target class-this includes all complementary classes as well as out-of-distribution (OOD) samples, noise, or in general any instance that does not belong to the true class. By setting a uniform one-cold encoded distribution over the complementary classes as a target for each anticlass, we encourage the model to equally distribute activations across all nontarget classes. This approach promotes a symmetric geometric structure of classes in the final feature space, increases the degree of neural collapse (NC) during training, addresses the independence deficit problem of neural networks, and improves generalization. Our extensive evaluation shows that incorporating OCCE loss in the optimization objective consistently enhances performance across multiple settings, including classification, open-set recognition, and OOD detection.

Abstract In this paper, we propose a novel voxel-based 3D single object tracking (3D SOT) method called Voxel Pseudo Image Tracking (VPIT). VPIT is the first method that uses voxel pseudo images for 3D SOT. The input point cloud is structured by pillar-based voxelization, and the resulting pseudo image is used as an input to a 2D-like Siamese SOT method. The pseudo image is created in the Bird’s-eye View (BEV) coordinates; and therefore, the objects in it have constant size. Thus, only the object rotation can change in the new coordinate system and not the object scale. For this reason, we replace multi-scale search with a multi-rotation search, where differently rotated search regions are compared against a single target representation to predict both position and rotation of the object. Experiments on KITTI [1] Tracking dataset show that VPIT is the fastest 3D SOT method and maintains competitive Success and Precision values. Application of a SOT method in a real-world scenario meets with limitations such as lower computational capabilities of embedded devices and a latency-unforgiving environment, where the method is forced to skip certain data frames if the inference speed is not high enough. We implement a real-time evaluation protocol and show that other methods lose most of their performance on embedded devices; while, VPIT maintains its ability to track the object.

Deep Learning (DL) models have enabled very accurate pose estimation. However, most of the existing approaches require images of relatively high resolution, since locating body parts and joints accurately is challenging, which increases the computational cost of these approaches. To overcome this limitation in this paper we propose an active perception method for high-resolution pose estimation that enables efficiently selecting the most appropriate image region for analysis and then employing a bottom-up pose estimator on the corresponding region. This allows for significantly improving the efficiency of pose estimation by selectively analyzing in high resolution only the parts of the image that contain humans. To ensure the computational efficiency of the proposed method we propose using low-resolution heat maps extracted using the same pose estimation model in order to guide the active perception process. The proposed method is model agnostic since it can be combined with any bottom-up pose estimation model in order to enable high-resolution analysis. We have experimentally evaluated the proposed method using a well-known pose estimation model, Lightweight OpenPose, demonstrating its effectiveness on three high-resolution variants of the COCO2017 dataset.

Active vision aims to enhance the efficiency of computer vision methods by enabling the capturing sensor, usually placed on a robot or, more generally, an autonomous system, to dynamically adjust its viewpoint, position or parameters in real-time. This capability allows for more precise decision-making by the model. However, training and evaluating an active vision model often necessitates a substantial number of annotated images, which must be captured under various sensor and environmental settings. These diverse images enable the model to learn the underlying dynamics of the active perception process. Unfortunately, collecting and annotating such datasets is a challenging and expensive task. It involves not only providing hand-crafted ground truth annotations but also ensuring that actions, such as moving around / towards / away from a person, are properly “imitated” to enable active vision approaches to model the corresponding active perception dynamics. To address these limitations, in this paper we propose a synthetic facial image generation pipeline specifically designed to support active face recognition, developed using a highly realistic simulation framework based on Unity. The developed pipeline allows for the generation of facial images for a set of persons at various view angles, distances, illumination conditions, and backgrounds. We demonstrate the effectiveness of our approach by training and evaluating a recently proposed embedding-based active face recognizer, as well as extending it to perform 2 axis control, leveraging the additional information provided by the generated dataset. To facilitate replication and encourage the use of the generated dataset for training and evaluating other active vision approaches, we also provide the associated assets and the developed dataset generation pipeline.

Face recognition systems play a vital role in various applications by providing identification and verification based on facial features. However, these systems face challenges in large-scale in-the-wild applications, where the current static pipelines are usually unable to cope with the high velocity, variety, and volume of the data, negatively affecting their accuracy and reliability. To overcome these challenges, in this paper, we propose departing from traditional static face recognition pipelines and moving towards dynamic and adaptable approaches. To this end, we propose a combination of an active and continual learning approach that can automatically augment the model with informative samples gathered during the inference, provided that the model is already confident enough, significantly improving its recognition accuracy. Furthermore, the proposed pipeline also natively incorporates active learning, allowing for using human feedback, when available, to further improve its performance. The effectiveness of the proposed method is validated on a challenging setup using two large-scale face recognition datasets.

Abstract The energy demanding nature of deep learning (DL) has fueled the immense attention for neuromorphic architectures due to their ability to operate in a very high frequencies in a very low energy consumption. To this end, neuromorphic photonics are among the most promising research directions, since they are able to achieve femtojoule per MAC efficiency. Although electrooptical substances provide a fast and efficient platform for DL, they also introduce various noise sources that impact the effective bit resolution, introducing new challenges to DL quantization. In this work, we propose a quantization-aware training method that gradually performs bit reduction to layers in a mixed-precision manner, enabling us to operate lower-precision networks during deployment and further increase the computational rate of the developed accelerators while keeping the energy consumption low. Exploiting the observation that intermediate layers have lower-precision requirements, we propose to gradually reduce layers’ bit resolutions, by normally distributing the reduction probability of each layer. We experimentally demonstrate the advantages of mixed-precision quantization in both performance and inference time. Furthermore, we experimentally evaluate the proposed method in different tasks, architectures, and photonic configurations, highlighting its immense capabilities to reduce the average bit resolution of DL models while significantly outperforming the evaluated baselines.

Deep Learning (DL) models are increasingly used for financial forecasting problems, such as price or trend prediction of a financial asset. However, most methods either rely solely on price information or require difficult to implement data harvesting pipelines, e.g., from social media, to deploy them. The main contribution of this paper is a method that exploits sentiment information as a source of additional supervision during the training process, allowing for improving the profitability of the developed strategies compared to baseline agents, while also allowing for operating the agent under partial observability, i.e., without requiring sentiment information as input during inference. As demonstrated in the conducted experiments on the Bitcoin-USD currency pair, this approach can indeed lead to significant improvements in the performance of DL agents, as well as help reduce the overfitting phenomena that often occur when training such agents.

Knowledge distillation enables us to transfer the knowledge from a large and complex neural network into a smaller and faster one. This allows for improving the accuracy of the smaller network. However, directly transferring the knowledge between enormous feature maps, as they are extracted from convolutional layers, is not straightforward. In this work, we propose an efficient mutual information-based approach for transferring the knowledge between feature maps extracted from different networks. The proposed method employs an efficient Neural Bag-of-Features formulation to estimate the joint and marginal probabilities and then optimizes the whole pipeline in an end-to-end manner. The effectiveness of the proposed method is demonstrated using a lightweight, fully convolutional neural network architecture, which aims toward high-resolution analysis and targets photonic neural network accelerators.

Recent advances in Deep Neural Networks (DNNs) led to enormous progress in many different fields, covering a wide range of applications, including financial time-series analysis. Many different financial time-series forecasting tasks have been successfully tackled using such approaches, including predicting the next day’s return of FOREX currency pairs. However, using DNNs for such tasks is not always straightforward due to training stability issues that often arise. Indeed, the noisy nature of the data can often cause considerable different behaviors between DL models, despite following the same training process, model architecture, and hyper-parameters. At the same time, the methods proposed for generating the training labels can sometimes further reinforce such issues. All these phenomena can reduce the reliability of training DL models for financial forecasting tasks, while also making the training process especially time-consuming, requiring several validation and back-testing runs. To overcome these limitations, we propose an ensemble-based online distillation method that can significantly reduce this behavior. The proposed method is efficient since, in contrast to offline distillation approaches, it works in a single step, while it also allows for reducing the number of hyper-parameters to be tuned, e.g., the number of epochs for training the teachers. As demonstrated in the conducted experiments, the soft labels extracted through the proposed approach can mitigate the effect of noisy annotation that often exists in FOREX data, leading to significant performance improvements.

Existing Deep Learning (DL) frameworks typically do not provide ready-to-use solutions for robotics, where very specific learning, reasoning, and embodiment problems exist. Their relatively steep learning curve and the different methodologies employed by DL compared to traditional approaches, along with the high complexity of DL models, which often leads to the need of employing specialized hardware accelerators, further increase the effort and cost needed to employ DL models in robotics. Also, most of the existing DL methods follow a static inference paradigm, as inherited by the traditional computer vision pipelines, ignoring active perception, which can be employed to actively interact with the environment in order to increase perception accuracy. In this paper, we present the Open Deep Learning Toolkit for Robotics (OpenDR). OpenDR aims at developing an open, non-proprietary, efficient, and modular toolkit that can be easily used by robotics companies and research institutions to efficiently develop and deploy AI and cognition technologies to robotics applications, providing a solid step towards addressing the aforementioned challenges. We also detail the design choices, along with an abstract interface that was created to overcome these challenges. This interface can describe various robotic tasks, spanning beyond traditional DL cognition and inference, as known by existing frameworks, incorporating openness, homogeneity and robotics-oriented perception e.g., through active perception, as its core design principles.

Active vision approaches hold the credentials for improving the accuracy of Deep Learning (DL) models for many challenging visual analysis tasks and varying environmental conditions. However, active vision approaches are typically closely tied to the underlying hardware, slowing down their adoption, while they typically increase the latency of perception systems, since sensory data must be recaptured. In this work, we propose a pseudo-active data refinement method that works by appropriately refining the sensory input, without having to reacquire the sensor data through traditional camera control approaches. The proposed method is fully differentiable and can be trained for the task at hand in an end-to-end fashion, while it can be directly deployed in a wide variety of systems, tasks and conditions. The effectiveness and robustness of the proposed method is demonstrated across a variety of tasks using two challenging datasets.

Recent advances in intelligent surveillance systems have enabled a new era of smart monitoring in a wide range of applications from health monitoring to homeland security. However, this boom in data gathering, analyzing and sharing brings in also significant privacy concerns. We propose a Compressive Sensing (CS) based data encryption that is capable of both obfuscating selected sensitive parts of documents and compressively sampling, hence encrypting both sensitive and non-sensitive parts of the document. The scheme uses a data hiding technique on CS-encrypted signal to preserve the one-time use obfuscation matrix. The proposed privacy-preserving approach offers a low-cost multi-tier encryption system that provides different levels of reconstruction quality for different classes of users, e.g., semi-authorized, full-authorized. As a case study, we develop a secure video surveillance system and analyze its performance.

Weight imprinting (WI) was recently introduced as a way to perform gradient descent-free few-shot learning. Due to this, WI was almost immediately adapted for performing few-shot learning on embedded neural network accelerators that do not support back-propagation, e.g., edge tensor processing units. However, WI suffers from many limitations, e.g., it cannot handle novel categories with multimodal distributions and special care should be given to avoid overfitting the learned embeddings on the training classes since this can have a devastating effect on classification accuracy (for the novel categories). In this article, we propose a novel hypersphere-based WI approach that is capable of training neural networks in a regularized, imprinting-aware way effectively overcoming the aforementioned limitations. The effectiveness of the proposed method is demonstrated using extensive experiments on three image data sets.

Knowledge-transfer (KT) methods allow for transferring the knowledge contained in a large deep learning model into a more lightweight and faster model. However, the vast majority of existing KT approaches are designed to handle mainly classification and detection tasks. This limits their performance on other tasks, such as representation/metric learning. To overcome this limitation, a novel probabilistic KT (PKT) method is proposed in this article. PKT is capable of transferring the knowledge into a smaller student model by keeping as much information as possible, as expressed through the teacher model. The ability of the proposed method to use different kernels for estimating the probability distribution of the teacher and student models, along with the different divergence metrics that can be used for transferring the knowledge, allows for easily adapting the proposed method to different applications. PKT outperforms several existing state-of-the-art KT techniques, while it is capable of providing new insights into KT by enabling several novel applications, as it is demonstrated through extensive experiments on several challenging data sets.

Machine learning methods have recently seen a growing number of applications in financial trading. Being able to automatically extract patterns from past price data and consistently apply them in the future has been the focus of many quantitative trading applications. However, developing machine learning-based methods for financial trading is not straightforward, requiring carefully designed targets/rewards, hyperparameter fine-tuning, and so on. Furthermore, most of the existing methods are unable to effectively exploit the information available across various financial instruments. In this article, we propose a deep reinforcement learning-based approach, which ensures that consistent rewards are provided to the trading agent, mitigating the noisy nature of profit-and-loss rewards that are usually used. To this end, we employ a novel price trailing-based reward shaping approach, significantly improving the performance of the agent in terms of profit, Sharpe ratio, and maximum drawdown. Furthermore, we carefully designed a data preprocessing method that allows for training the agent on different FOREX currency pairs, providing a way for developing market-wide RL agents and allowing, at the same time, to exploit more powerful recurrent deep learning models without the risk of overfitting. The ability of the proposed methods to improve various performance metrics is demonstrated using a challenging large-scale data set, containing 28 instruments, provided by Speedlab AG.

Knowledge Distillation (KD) methods are capable of transferring the knowledge encoded in a large and complex teacher into a smaller and faster student. Early methods were usually limited to transferring the knowledge only between the last layers of the networks, while latter approaches were capable of performing multi-layer KD, further increasing the accuracy of the student. However, despite their improved performance, these methods still suffer from several limitations that restrict both their efficiency and flexibility. First, existing KD methods typically ignore that neural networks undergo through different learning phases during the training process, which often requires different types of supervision for each one. Furthermore, existing multi-layer KD methods are usually unable to effectively handle networks with significantly different architectures (heterogeneous KD). In this paper we propose a novel KD method that works by modeling the information flow through the various layers of the teacher model and then train a student model to mimic this information flow. The proposed method is capable of overcoming the aforementioned limitations by using an appropriate supervision scheme during the different phases of the training process, as well as by designing and training an appropriate auxiliary teacher model that acts as a proxy model capable of “explaining” the way the teacher works to the student. The effectiveness of the proposed method is demonstrated using four image datasets and several different evaluation setups.

Deep learning models are capable of achieving state-of-the-art performance on a wide range of time series analysis tasks. However, their performance crucially depends on the employed normalization scheme, while they are usually unable to efficiently handle non-stationary features without first appropriately pre-processing them. These limitations impact the performance of deep learning models, especially when used for forecasting financial time series, due to their non-stationary and multimodal nature. In this paper we propose a data-driven adaptive normalization layer which is capable of learning the most appropriate normalization scheme that should be applied on the data. To this end, the proposed method first identifies the distribution from which the data were generated and then it dynamically shifts and scales them in order to facilitate the task at hand. The proposed nor-malization scheme is fully differentiable and it is trained in an end-to-end fashion along with the rest of the parameters of the model. The proposed method leads to significant performance improvements over several competitive normalization approaches, as demonstrated using a large-scale limit order book dataset.

For autonomous unmanned surface vehicles (USV) operations, it is important to be able to observe the surroundings using visual information. Water segmentation is a task where the water surface is recognized and separated from everything else. The algorithm performing the segmentation must be robust, because safety is the most important feature of autonomous USVs. This is especially challenging in many USV applications, where the rapidly changing weather and lighting conditions can cause significant distribution shifts. In this study, we analyze the robustness of different color spaces (e.g., RGB and HSV) for water segmentation and consider how to use different color channels in training and testing to maximize the robustness. We evaluate the segmentation performance on a challenging completely unseen test dataset, recorded in vastly different conditions and with different equipment.

Recent advances in Deep Reinforcement Learning (DRL) led to the development of powerful agents that can learn how to perform complicated tasks in an end-to-end fashion operating directly on raw unstructured data, e.g., images. However, the real world performance of such methods critically relies on the quality of the simulation environments used for training them. The main contribution of this paper is the development of a realistic simulation environment, by employing a state-of-the-art graphics engine, for training DRL agents that are able to control a drone for performing active shooting. In contrast with previous approaches, that solely relied on simplistic constrained datasets, the environment employed in this work supports a challenging open-world setting, providing a solid step towards developing effective RL methods for various drone control tasks. An appropriate reward shaping approach is also introduced in this work, ensuring that the agent will behave as expected, avoiding erratic movements, as demonstrated through the conducted experiments.

Visual Question Answering (VQA) is one of the most challenging emerging applications of deep learning. Providing powerful attention mechanisms is crucial for VQA, since the model must correctly identify the region of an image that is relevant to the question at hand. However, existing models analyze the input images at a fixed and typically small resolution, often leading to discarding valuable fine-grained details. To overcome this limitation, in this work we propose a reinforcement learning-based active perception approach that works by applying a series of transformation operations on the images (translation, zoom) in order to facilitate answering the question at hand. This allows for performing fine-grained analysis, effectively increasing the resolution at which the models process information. The proposed method is orthogonal to existing attention mechanisms and it can be combined with most existing VQA methods. The effectiveness of the proposed method is experimentally demonstrated on a challenging VQA dataset.

Several deep supervised hashing techniques have been proposed to allow for querying large image databases. However, it is often overlooked that the process of information retrieval can be modeled using information-theoretic metrics, leading to optimizing various proxies for the problem at hand instead. Contrary to this, we propose a deep supervised hashing algorithm that optimizes the learned codes using an information-theoretic measure, the Quadratic Mutual Information (QMI). The proposed method is adapted to the needs of large-scale hashing and information retrieval leading to a novel information-theoretic measure, the Quadratic Spherical Mutual Information (QSMI), that is inspired by QMI, but leads to significant better retrieval precision. Indeed, the effectiveness of the proposed method is demonstrated under several different scenarios, using different datasets and network architectures, outperforming existing deep supervised image hashing techniques.

Deploying state-of-the-art deep learning models on embedded systems dictates certain storage and computation limitations. During the recent few years Knowledge Distillation (KD) has been recognized as a prominent approach to address this issue. That is, KD has been effectively proposed for training fast and compact deep learning models by transferring knowledge from more complex and powerful models. However, knowledge distillation, in its conventional form, involves multiple stages of training, rendering it a computationally and memory demanding procedure. In this paper, a novel single-stage self knowledge distillation method is proposed, namely Online Subclass Knowledge Distillation (OSKD), that aims at revealing the similarities inside classes, so as to improve the performance of any deep neural model in an online manner. Hence, as opposed to existing online distillation methods, we are able to acquire further knowledge from the model itself, without building multiple identical models or using multiple models to teach each other, rendering the proposed OSKD approach more efficient. The experimental evaluation on two datasets validates that the proposed method improves the classification performance.

Photonic Deep Learning (DL) accelerators are among the most promising approaches for providing fast and energy efficient neural network implementations for several applications. However, photonic accelerators require using different activation functions compared to those typically used in DL. This renders the training process especially difficult to tune, often requiring several trials just for selecting the appropriate initialization hyper-parameters for the network. This process becomes even more difficult for recurrent networks, where exploding gradient phenomena can further destabilize the training process. In this paper, we propose an adaptive data-driven initialization approach for recurrent photonic neural networks. The proposed method is activation-agnostic, while it takes into account the actual distribution of the data used to train the network, overcoming a number of significant limitations of existing approaches. The proposed method is simple and easy to implement, yet it leads to significant improvements in the performance of DL models, as it was experimentally demonstrated using two large-scale challenging time-series datasets.

Probabilistic Knowledge Transfer (PKT) aims to transfer the knowledge encoded in the representations extracted from a layer of a large and complex neural network (teacher) into a smaller and faster one (student). However, PKT only transfers the knowledge between two layers of the networks, ignoring the potentially useful information encoded by the previous ones, reducing in this way the efficiency of PKT and the performance of the student model. In this paper, we propose a novel efficient multilayer PKT method that is capable of transferring the knowledge between the student and teacher networks by employing the representations extracted from multiple layers. The ability of the proposed multilayer PKT method to improve the knowledge transfer and increase the performance of the student model over other state-of-the-art methods is demonstrated using two image datasets.

Early exits provide an effective way of implementing adaptive computational graphs over deep learning models. In this way it is possible to adapt them on-the-fly to the available computational resources or even to the difficulty of each input sample, reducing the energy and computational power requirements in many embedded and mobile applications. However, performing this kind of adaptive inference also comes with several challenges, since the difficulty of each sample must be estimated and the most appropriate early exit must be selected. It is worth noting that existing approaches often lead to highly unbalanced distributions over the selected early exits, reducing the efficiency of the adaptive inference process. At the same time, only a few resources can be devoted to the aforementioned process, in order to ensure that an adequate speedup will be obtained. The main contribution of this work is to provide an easy to use and tune adaptive inference approach for early exits that can overcome some of these limitations. In this way, the proposed method allows for a) obtaining a more balanced inference distribution among the early exits, b) relying on a single and interpretable hyperparameter for tuning its behavior (ranging from faster inference to higher accuracy), and c) improving the performance of the networks (increasing the accuracy and reducing the time needed for inference). Indeed, the effectiveness of the proposed method over existing approaches is demonstrated using four different image datasets.

Even though recent advances in deep learning (DL) led to tremendous improvements for various computer and robotic vision tasks, existing DL approaches suffer from a significant limitation: they typically ignore that robots and cyber-physical systems are capable of interacting with the environment in order to better sense their surroundings. In this work we argue that perceiving the world through physical interaction, i.e., employing active perception, allows for both increasing the accuracy of DL models, as well as for deploying smaller and faster models. To this end, we propose an active perception-based face recognition approach, which is capable of simultaneously extracting discriminative embeddings, as well as predicting in which direction the robot must move in order to get a more discriminative view. To the best of our knowledge, we provide the first embedding-based active perception method for deep face recognition. As we experimentally demonstrate, the proposed method can indeed lead to significant improvements, increasing the face recognition accuracy up to 9%, as well as allowing for using overall smaller and faster models, reducing the number of parameters by over one order of magnitude.

Deep Learning (DL) has provided powerful tools for visual information analysis. For example, Convolutional Neural Networks (CNNs) are excelling in complex and challenging image analysis tasks by extracting meaningful feature vectors with high discriminative power . However, these powerful feature vectors are crushed through the pooling layers of the network, that usually implement the pooling operation in a less sophisticated manner. This can lead to significant information loss, especially in cases where the informative content of the data is sequentially distributed over the spatial or temporal dimension, e.g., videos, which often require extracting fine-grained temporal information. A novel stateful recurrent pooling approach, that can overcome the aforementioned limitations, is proposed in this paper. The proposed method is inspired by the well-known Bag-of-Features (BoF) model, but employs a stateful trainable recurrent quantizer, instead of plain static quantization, allowing for efficiently processing sequential data and encoding both their temporal, as well as their spatial aspects. The effectiveness of the proposed Recurrent BoF model to enclose spatio-temporal information compared to other competitive methods is demonstrated using six different datasets and two different tasks.

Supervised deep metric learning led to spectacular results for several Content-based Information Retrieval (CBIR) applications. The success of these approaches slowly led to the belief that image retrieval and classification are just slightly different variations of the same problem. However, recent evidence suggests that learning highly discriminative representation for a (limited) set of training classes removes valuable information from the representation, potentially harming both the in-domain, as well as the out-of-domain retrieval precision. In this paper, we propose a regularized discriminative deep metric learning method that aims to not only learn a representation that allows for discriminating between different classes, but it is also capable of encoding the latent generative factors separately for each class, overcoming this limitation. This allows for modeling the in-class variance and, as a result, maintaining the ability to represent both sub-classes of the in-domain data, as well as objects that belong to classes outside the training domain. The effectiveness of the proposed method, over existing supervised and unsupervised representation/metric learning approaches , is demonstrated under different in-domain and out-of-domain setups and three challenging image datasets.

• Logistic Neural Bag-of-Features are employed for financial time series analysis. • The proposed method can be efficiently used in deep learning architectures. • An adaptive scaling method is proposed to ensure the smooth flow of information. • A logistic kernel is used to estimate the feature vector densities. • The proposed method outperforms the competitive methods on a large-scale dataset. Time series forecasting is a crucial component of many important applications, ranging from forecasting the stock markets to energy load prediction. The high-dimensionality, velocity and variety of the data collected in many of these applications pose significant and unique challenges that must be carefully addressed for each of them. In this work, a novel Temporal Logistic Neural Bag-of-Features approach, that can be used to tackle these challenges, is proposed. The proposed method can be effectively combined with deep neural networks , leading to powerful deep learning models for time series analysis. However, combining existing BoF formulations with deep feature extractors pose significant challenges: the distribution of the input features is not stationary, tuning the hyper-parameters of the model can be especially difficult and the normalizations involved in the BoF model can cause significant instabilities during the training process. The proposed method is capable of overcoming these limitations by a employing a novel adaptive scaling mechanism and replacing the classical Gaussian-based density estimation involved in the regular BoF model with a logistic kernel. The effectiveness of the proposed approach is demonstrated using extensive experiments on a large-scale limit order book dataset that consists of more than 4 million limit orders.

In this paper, we propose a novel color constancy approach, called Bag of Color Features (BoCF), building upon Bag-of-Features pooling. The proposed method substantially reduces the number of parameters needed for illumination estimation. At the same time, the proposed method is consistent with the color constancy assumption stating that global spatial information is not relevant for illumination estimation and local information (edges, etc.) is sufficient. Furthermore, BoCF is consistent with color constancy statistical approaches and can be interpreted as a learning-based extension of many statistical approaches. To further improve the illumination estimation accuracy, we propose a novel attention mechanism for the BoCF model with two variants based on self-attention. BoCF approach and its variants achieve competitive, compared to the state of the art, results while requiring much fewer parameters on three benchmark datasets: ColorChecker RECommended, INTEL-TUT version 2, and NUS8.

Deep learning (DL) models can be used to tackle time series analysis tasks with great success. However, the performance of DL models can degenerate rapidly if the data are not appropriately normalized. This issue is even more apparent when DL is used for financial time series forecasting tasks, where the nonstationary and multimodal nature of the data pose significant challenges and severely affect the performance of DL models. In this brief, a simple, yet effective, neural layer that is capable of adaptively normalizing the input time series, while taking into account the distribution of the data, is proposed. The proposed layer is trained in an end-to-end fashion using backpropagation and leads to significant performance improvements compared to other evaluated normalization schemes. The proposed method differs from traditional normalization methods since it learns how to perform normalization for a given task instead of using a fixed normalization scheme. At the same time, it can be directly applied to any new time series without requiring retraining. The effectiveness of the proposed method is demonstrated using a large-scale limit order book data set, as well as a load forecasting data set.

Photonic neuromorphic hardware can provide significant performance benefits for Deep Learning (DL) applications by accelerating and reducing the energy requirements of DL models. However, photonic neuromorphic architectures employ different activation elements than those traditionally used in DL, slowing down the convergence of the training process for such architectures. An initialization scheme that can be used to efficiently train deep photonic networks that employ quadratic sinusoidal activation functions is proposed in this paper. The proposed initialization scheme can overcome these limitations, leading to faster and more stable training of deep photonic neural networks. The ability of the proposed method to improve the convergence of the training process is experimentally demonstrated using two different DL architectures and two datasets.

Forecasting time series has several applications in various domains. The vast amount of data that are available nowadays provide the opportunity to use powerful deep learning approaches, but at the same time pose significant challenges of high-dimensionality, velocity and variety. In this paper, a novel logistic formulation of the well-known Bag-of-Features model is proposed to tackle these challenges. The proposed method is combined with deep convolutional feature extractors and is capable of accurately modeling the temporal behavior of time series, forming powerful forecasting models that can be trained in an end-to-end fashion. The proposed method was extensively evaluated using a large-scale financial time series dataset, that consists of more than 4 million limit orders, outperforming other competitive methods.

Developing accurate financial analysis tools can be useful both for speculative trading, as well as for analyzing the behavior of markets and promptly responding to unstable conditions ensuring the smooth operation of the financial markets. This led to the development of various methods for analyzing and forecasting the behaviour of financial assets, ranging from traditional quantitative finance to more modern machine learning approaches. However, the volatile and unstable behavior of financial markets forbids the accurate prediction of future prices, reducing the performance of these approaches. In contrast, in this paper we propose a novel price trailing method that goes beyond traditional price forecasting by reformulating trading as a control problem, effectively overcoming the aforementioned limitations. The proposed method leads to developing robust agents that can withstand large amounts of noise, while still capturing the price trends and allowing for taking profitable decisions.

Using early exits provide a straightforward way to implement models that can adapt on-the-fly to the available computational resources. However, early exits in many cases suffer from significant limitations, which often prohibit their practical application, especially when placed on convolutional layers with narrow receptive fields. In this work, we propose a method capable of overcoming these limitations by a) using a Bag-of-Features (BoF)-based pooling approach, that allows for keeping more information regarding the distribution of the extracted feature vectors, while also maintaining more spatial information and b) employing a simple, yet effective, hierarchical approach for designing the exits, allowing for efficiently re-using the information that was already extracted by the previous layers. It is experimentally demonstrated that the proposed approach leads to significant performance improvements, allowing early exits to be a more practical tool that can be used in many real-world embedded applications.

Supervised learning for Content-based Information Retrieval allows for obtaining discriminative representations that often excel within the training domain. However, recent evidence suggests that these representations can actually harm the retrieval precision for queries that do not belong to the domain of the training set compared to other, less discriminative representations. To avoid this behavior, we propose to learn discriminative representations which also encode the latent generative factors for each class. In this way, the proposed method is capable of maintaining (part of) the in-class variance, as well as being able to represent data that belong to classes that were not seen during the training by better learning the structure of the input space. The proposed method is evaluated under different in-domain and out-of-domain setups, significantly outperforming existing supervised and unsupervised representation learning approaches.

With the advent of deep neural networks, there is a growing interest in transferring the knowledge from a large and complex model to a smaller and faster one. In this brief, a method for unsupervised knowledge transfer (KT) between neural networks is proposed. To the best of our knowledge, the proposed method is the first method that utilizes similarity-induced embeddings to transfer the knowledge between any two layers of neural networks, regardless of the number of neurons in each of them. By this way, the knowledge is transferred without using any lossy dimensionality reduction transformations or requiring any information about the complex model, except for the activations of the layer used for KT. This is in contrast with most existing approaches that only generate soft-targets for training the smaller neural network or directly use the weights of the larger model. The proposed method is evaluated using six image data sets and it is demonstrated, through extensive experiments, that the knowledge of a neural network can be successfully transferred using different kinds of (synthetic or not) data, ranging from cross-domain data to just randomly generated data.

Convolutional neural networks (CNNs) are predominantly used for several challenging computer vision tasks achieving state-of-the-art performance. However, CNNs are complex models that require the use of powerful hardware, both for training and deploying them. To this end, a quantization-based pooling method is proposed in this paper. The proposed method is inspired from the bag-of-features model and can be used for learning more lightweight deep neural networks. Trainable radial basis function neurons are used to quantize the activations of the final convolutional layer, reducing the number of parameters in the network and allowing for natively classifying images of various sizes. The proposed method employs differentiable quantization and aggregation layers leading to an end-to-end trainable CNN architecture. Furthermore, a fast linear variant of the proposed method is introduced and discussed, providing new insight for understanding convolutional neural architectures. The ability of the proposed method to reduce the size of CNNs and increase the performance over other competitive methods is demonstrated using seven data sets and three different learning tasks (classification, regression, and retrieval).

Reducing the dimensionality of data is among the most common preprocessing steps for a number of machine learning tasks. A linear similarity-based discriminant analysis (S-LDA) algorithm was recently proposed, significantly improving the discriminative ability of the learned low-dimensional representation over classical techniques, such as LDA. S-LDA builds upon the notion of similarity and allows for exploiting higher-order statistics, while being notably robust to outliers. However, optimizing S-LDA`s objective requires the calculation of the pairwise similarities between all the training data. This quickly becomes a significant burden and limits the ability of the method to scale to larger datasets. In this paper we propose using prototypes, that act as similarity anchors, to reduce the number of required similarity calculations. We experimentally demonstrate that the proposed technique is capable of reducing the optimization time by two orders of magnitude without harming the discriminative ability of the learned representations.

Transferring the knowledge from a large and complex neural network to a smaller and faster one allows for deploying more lightweight and accurate networks. In this paper, we propose a novel method that is capable of transferring the knowledge between any two layers of two neural networks by matching the similarity between the extracted representations. The proposed method is model-agnostic overcoming several limitations of existing knowledge transfer techniques, since the knowledge is transferred between layers that can have different architecture and no information about the complex model is required, apart from the output of the layers employed for the knowledge transfer. Three image datasets are used to demonstrate the effectiveness of the proposed approach, including a large-scale dataset for learning a light-weight model for facial pose estimation that can be directly deployed on devices with limited computational resources, such as embedded systems for drones.

One of the most complex multi-model problems faced today is Visual Question Answering (VQA), which requires a machine to properly understand a question about a reference visual input, expressed in natural language, and then produce the answer to that question. In order to solve this problem and increase the probability of producing the correct answer, it is crucial to provide reliable attention information. However, existing methods only use implicitly trained attention models that are often unable to attend to the appropriate image region the question refers to, limiting their ability to provide the correct answer. To address this issue, we propose an explicitly trained attention model that is inspired by the theory of pictorial superiority effect. In this model, we use attention-oriented word embeddings that increase the efficiency of learning common representation spaces. The dataset that we use, the Visual7W dataset, is the only dataset that provides visual attention ground truth information. In this paper, we demonstrate the effectiveness of the proposed method over both implicit attention models and other state-of-art VQA techniques.

Using Unmanned Aerial Vehicles (UAVs), also known as drones, for covering public sport events, such as bicycle races, is becoming increasingly popular. Even though the problem of controlling the flight path of a drone is well studied in the literature, little work has been done on controlling the shooting camera for producing professional grade video footage. In this work we propose a fast and efficient proportional-integral-derivative (PID) based control algorithm that rely solely on 2D visual information and we demonstrate that it is possible to accurately control the camera without inferring the 3D position of the target. To ensure that the proposed method will not exhibit undesired behavior, a genetic algorithm is used to tune its parameters using a properly defined fitness function. The proposed method is evaluated using two datasets that contain actual drone footage: a dataset that contains videos of a single cyclist, and a dataset that contains actually footage from a bicycle race event, the Giro D'Italia bicycle race.

In this paper, a manifold-based dictionary learning method for the bag-of-features (BoF) representation optimized toward information clustering is proposed. First, the spectral representation, which unwraps the manifolds of the data and provides better clustering solutions, is formed. Then, a new dictionary is learned in order to make the histogram space, i.e., the space where the BoF historgrams exist, as similar as possible to the spectral space. The ability of the proposed method to improve the clustering solutions is demonstrated using a wide range of datasets: two image datasets, the 15-scene dataset and the Corel image dataset, one video dataset, the KTH dataset, and one text dataset, the RT-2k dataset. The proposed method improves both the internal and the external clustering criteria for two different clustering algorithms: 1) the -means and 2) the spectral clustering. Also, the optimized histogram space can be used to directly assign a new object to its cluster, instead of using the spectral space (which requires reapplying the spectral clustering algorithm or using incremental spectral clustering techniques). Finally, the learned representation is also evaluated using an information retrieval setup and it is demonstrated that improves the retrieval precision over the baseline BoF representation.

The vast majority of dimensionality reduction (DR) techniques rely on the second-order statistics to define their optimization objective. Even though this provides adequate results in most cases, it comes with several shortcomings. The methods require carefully designed regularizers and they are usually prone to outliers. In this paper, a new DR framework that can directly model the target distribution using the notion of similarity instead of distance is introduced. The proposed framework, called similarity embedding framework (SEF), can overcome the aforementioned limitations and provides a conceptually simpler way to express optimization targets similar to existing DR techniques. Deriving a new DR technique using the SEF becomes simply a matter of choosing an appropriate target similarity matrix. A variety of classical tasks, such as performing supervised DR and providing out-of-sample extensions, as well as, new novel techniques, such as providing fast linear embeddings for complex techniques, are demonstrated in this paper using the proposed framework. Six data sets from a diverse range of domains are used to evaluate the proposed method and it is demonstrated that it can outperform many existing DR techniques.

Convolutional Neural Networks (CNNs) are well established models capable of achieving state-of-the-art classification accuracy for various computer vision tasks. However, they are becoming increasingly larger, using millions of parameters, while they are restricted to handling images of fixed size. In this paper, a quantization-based approach, inspired from the well-known Bag-of-Features model, is proposed to overcome these limitations. The proposed approach, called Convolutional BoF (CBoF), uses RBF neurons to quantize the information extracted from the convolutional layers and it is able to natively classify images of various sizes as well as to significantly reduce the number of parameters in the network. In contrast to other global pooling operators and CNN compression techniques the proposed method utilizes a trainable pooling layer that it is end-to-end differentiable, allowing the network to be trained using regular back-propagation and to achieve greater distribution shift invariance than competitive methods. The ability of the proposed method to reduce the parameters of the network and increase the classification accuracy over other state-of-the-art techniques is demonstrated using three image datasets.

In this paper, the well-known bag-of-features (BoFs) model is generalized and formulated as a neural network that is composed of three layers: 1) a radial basis function (RBF) layer; 2) an accumulation layer; and 3) a fully connected layer. This formulation allows for decoupling the representation size from the number of used codewords, as well as for better modeling the feature distribution using a separate trainable scaling parameter for each RBF neuron. The resulting network, called retrieval-oriented neural BoF (RN-BoF), is trained using regular back propagation and allows for fast extraction of compact image representations. It is demonstrated that the RN-BoF model is capable of: 1) increasing the object encoding and retrieval speed; 2) reducing the extracted representation size; and 3) increasing the retrieval precision. A symmetry-aware spatial segmentation technique is also proposed to further reduce the encoding time and the storage requirements and allows the method to efficiently scale to large datasets. The proposed method is evaluated and compared to other state-of-the-art techniques using five different image datasets, including the large-scale YouTube Faces database.

The word embedding models are capable of capturing the semantic content of the textual words. The process of extracting a set of word embedding vectors from a text document is similar to the feature extraction step of the Bag-of-Features pipeline, which is usually used in computer vision tasks. That gives rise to the Bag-of-Embedded Words (BoEW) model. In this paper a novel learning technique that optimizes both the word embedding and the codebook of the BoEW model towards text retrieval is proposed. The proposed method adheres to the cluster hypothesis that states that points in the same cluster are likely to fulfill the same information need and it is demonstrated, using two text datasets, that can significantly increase the retrieval precision. Finally, the proposed technique uses smaller representations than the competitive representation methods, that allows to reduce both the retrieval time and the storage requirements.

In this paper, we present a supervised dictionary learning method for optimizing the feature-based Bag-of-Words (BoW) representation towards Information Retrieval. Following the cluster hypothesis, which states that points in the same cluster are likely to fulfill the same information need, we propose the use of an entropy-based optimization criterion that is better suited for retrieval instead of classification. We demonstrate the ability of the proposed method, abbreviated as EO-BoW, to improve the retrieval performance by providing extensive experiments on two multi-class image datasets. The BoW model can be applied to other domains as well, so we also evaluate our approach using a collection of 45 time-series datasets, a text dataset, and a video dataset. The gains are three-fold since the EO-BoW can improve the mean Average Precision, while reducing the encoding time and the database storage requirements. Finally, we provide evidence that the EO-BoW maintains its representation ability even when used to retrieve objects from classes that were not seen during the training.