In this paper, we aim to improve the performance of a deep learning model towards image classification tasks, proposing a novel anchor-based training methodology, named Online Anchor-based Training (OAT). The OAT method, guided by the insights provided in the anchor-based object detection methodologies, instead of learning directly the class labels, proposes to train a model to learn percentage changes of the class labels with respect to defined anchors. We define as anchors the batch centers at the output of the model. Then, during the test phase, the predictions are converted back to the original class label space, and the performance is evaluated. The effectiveness of the OAT method is validated on four datasets.

In this paper we deal with image classification tasks using the powerful CLIP vision-language model. Our goal is to advance the classification performance using the CLIP’s image encoder, by proposing a novel Large Multimodal Model (LMM) based regularization method. The proposed method uses an LMM to extract semantic descriptions for the images of the dataset. Then, it uses the CLIP’s text encoder, frozen, in order to obtain the corresponding text embeddings and compute the mean semantic class descriptions. Subsequently, we adapt the CLIP’s image encoder by adding a classification head, and we train it along with the image encoder output, apart from the main classification objective, with an additional auxiliary objective. The additional objective forces the embeddings at the image encoder’s output to become similar to their corresponding LMM-generated mean semantic class descriptions. In this way, it produces embeddings with enhanced discrimination ability, leading to improved classification performance. The effectiveness of the proposed regularization method is validated through extensive experiments on three image classification datasets.

In this paper we deal with the problem of Electric Load Demand Forecasting (ELDF) considering the Greek Energy Market. Motivated by the anchored-based object detection methods, we argue that considering the ELDF task we can define an anchor and transform the problem into predicting the offset instead of predicting the actual load values. The experimental evaluation considering the one-day-ahead forecasting task, validated the effectiveness of the proposed Anchor-based FOREcasting (AFORE) method. The AFORE method achieved significant improvements in terms of mean absolute percentage error under various setups, using different loss functions and model architectures.

In this work we deal with the one-day-ahead electric load demand forecasting problem, namely the task of predicting the electricity demand a day ahead. We draw inspiration from objection detection methods, where the use of anchors facilitates the task by providing the network with some bias resembling the expected output. Based on the same principles, we propose the use of anchors and encode the groundtruth load demand. Furthermore, we propose the use of anchor-encoded input features to match the encoded output. We perform experiments on a dataset collected from the Greek Energy Market and show that the proposed method consistently outperforms the baseline methods. Our best model has a 17.5% relative improvement in terms of Mean Absolute Percentage Error.

Logo recognition describes the challenging task of detecting and classifying logos in digital images and videos. Former works approach logo recognition as a closed-set problem. However, this approach is accompanied by several shortcomings linked with its incapability of recognising new classes. In this paper, we propose an open-set logo recognition method, named REtrieval-based methodology For FEw-sample LOgo Recognition (REFELOR). REFELOR is composed by a generic logo detector and a feature extractor, allowing the generalization on unseen classes, using only a few samples per logo. That is, a single-stage generic logo detector is trained to detect logos in an input image. Then, feature representations for the detected logos are extracted, using the feature extractor, while the feature representations of a database containing only a few samples per class are also extracted. Finally, the detected logo representations are classified to the corresponding class based a similarity search in the representations of the aforementioned database. In addition, a regularization technique is applied to the feature extractor, providing further improvements. The experimental evaluation validates the effectiveness of the proposed method, outperforming current state-of-the-art logo recognition methods.

Online knowledge distillation has been proposed as an auspicious approach for circumventing the flaws of the conventional offline distillation (i.e., complex, and computationally and memory demanding process). In this work, a novel online self-distillation method, named Multilayer Online Self-Acquired Knowledge Distillation (MOSAKD), is proposed, aiming to develop fast-to-execute and effective models that can comply with applications with memory and computational restrictions, e.g., robotics applications. The MOSAKD method is able to mine additional knowledge both from the intermediate and the output layers of a deep neural model in an online fashion. To achieve this goal, k-nn non-parametric density estimation for estimating the unknown probability distributions of the data samples in the feature space generated by any neural layer is used. This enables us to compute the soft labels that explicitly express the similarities of the data with the classes, by directly estimating the posterior class probabilities of the data samples. The experimental evaluation on four datasets, including a dataset of synthetic images, indicates the effectiveness of the MOSAKD method and the superiority over existing online distillation methods.

Existing Deep Learning (DL) frameworks typically do not provide ready-to-use solutions for robotics, where very specific learning, reasoning, and embodiment problems exist. Their relatively steep learning curve and the different methodologies employed by DL compared to traditional approaches, along with the high complexity of DL models, which often leads to the need of employing specialized hardware accelerators, further increase the effort and cost needed to employ DL models in robotics. Also, most of the existing DL methods follow a static inference paradigm, as inherited by the traditional computer vision pipelines, ignoring active perception, which can be employed to actively interact with the environment in order to increase perception accuracy. In this paper, we present the Open Deep Learning Toolkit for Robotics (OpenDR). OpenDR aims at developing an open, non-proprietary, efficient, and modular toolkit that can be easily used by robotics companies and research institutions to efficiently develop and deploy AI and cognition technologies to robotics applications, providing a solid step towards addressing the aforementioned challenges. We also detail the design choices, along with an abstract interface that was created to overcome these challenges. This interface can describe various robotic tasks, spanning beyond traditional DL cognition and inference, as known by existing frameworks, incorporating openness, homogeneity and robotics-oriented perception e.g., through active perception, as its core design principles.

Knowledge Distillation has been established as a highly promising approach for training compact and faster models by transferring knowledge from heavyweight and powerful models. However, KD in its conventional version constitutes an enduring, computationally and memory demanding process. In this paper, Online Self-Acquired Knowledge Distillation (OSAKD) is proposed, aiming to improve the performance of any deep neural model in an online manner. We utilize k-nn non-parametric density estimation technique for estimating the unknown probability distributions of the data samples in the output feature space. This allows us for directly estimating the posterior class probabilities of the data samples, and we use them as soft labels that encode explicit information about the similarities of the data with the classes, negligibly affecting the computational cost. The experimental evaluation on four datasets validates the effectiveness of proposed method.

Knowledge-transfer (KT) methods allow for transferring the knowledge contained in a large deep learning model into a more lightweight and faster model. However, the vast majority of existing KT approaches are designed to handle mainly classification and detection tasks. This limits their performance on other tasks, such as representation/metric learning. To overcome this limitation, a novel probabilistic KT (PKT) method is proposed in this article. PKT is capable of transferring the knowledge into a smaller student model by keeping as much information as possible, as expressed through the teacher model. The ability of the proposed method to use different kernels for estimating the probability distribution of the teacher and student models, along with the different divergence metrics that can be used for transferring the knowledge, allows for easily adapting the proposed method to different applications. PKT outperforms several existing state-of-the-art KT techniques, while it is capable of providing new insights into KT by enabling several novel applications, as it is demonstrated through extensive experiments on several challenging data sets.

Knowledge Distillation (KD) methods are capable of transferring the knowledge encoded in a large and complex teacher into a smaller and faster student. Early methods were usually limited to transferring the knowledge only between the last layers of the networks, while latter approaches were capable of performing multi-layer KD, further increasing the accuracy of the student. However, despite their improved performance, these methods still suffer from several limitations that restrict both their efficiency and flexibility. First, existing KD methods typically ignore that neural networks undergo through different learning phases during the training process, which often requires different types of supervision for each one. Furthermore, existing multi-layer KD methods are usually unable to effectively handle networks with significantly different architectures (heterogeneous KD). In this paper we propose a novel KD method that works by modeling the information flow through the various layers of the teacher model and then train a student model to mimic this information flow. The proposed method is capable of overcoming the aforementioned limitations by using an appropriate supervision scheme during the different phases of the training process, as well as by designing and training an appropriate auxiliary teacher model that acts as a proxy model capable of “explaining” the way the teacher works to the student. The effectiveness of the proposed method is demonstrated using four image datasets and several different evaluation setups.

New affordances for online discussion forums are increasingly explored. In this regard, gamification elements such as badges, progress bars and progress visualizations are used in order to increase students' engagement and participation. This study uses the Community of Inquiry model to explore students' critical thinking under the effect of a) a community and b) individual gamification elements, over two discussions. Reaching the last discussion of the course, the students, who were awarded individual gamification elements, attained higher levels of critical thinking compared to the students that were awarded community gamification elements. Teaching presence has been influenced by the gamification elements provided, regardless of their type. The results highlight the value of gamification on the student's conscious awareness of their own thinking and learning.

Deploying state-of-the-art deep learning models on embedded systems dictates certain storage and computation limitations. During the recent few years Knowledge Distillation (KD) has been recognized as a prominent approach to address this issue. That is, KD has been effectively proposed for training fast and compact deep learning models by transferring knowledge from more complex and powerful models. However, knowledge distillation, in its conventional form, involves multiple stages of training, rendering it a computationally and memory demanding procedure. In this paper, a novel single-stage self knowledge distillation method is proposed, namely Online Subclass Knowledge Distillation (OSKD), that aims at revealing the similarities inside classes, so as to improve the performance of any deep neural model in an online manner. Hence, as opposed to existing online distillation methods, we are able to acquire further knowledge from the model itself, without building multiple identical models or using multiple models to teach each other, rendering the proposed OSKD approach more efficient. The experimental evaluation on two datasets validates that the proposed method improves the classification performance.

Probabilistic Knowledge Transfer (PKT) aims to transfer the knowledge encoded in the representations extracted from a layer of a large and complex neural network (teacher) into a smaller and faster one (student). However, PKT only transfers the knowledge between two layers of the networks, ignoring the potentially useful information encoded by the previous ones, reducing in this way the efficiency of PKT and the performance of the student model. In this paper, we propose a novel efficient multilayer PKT method that is capable of transferring the knowledge between the student and teacher networks by employing the representations extracted from multiple layers. The ability of the proposed multilayer PKT method to improve the knowledge transfer and increase the performance of the student model over other state-of-the-art methods is demonstrated using two image datasets.

In this paper, we first propose lightweight deep CNN models, capable of effectively operating on-drone, in order to address various classification problems, i.e. crowd, football player, and bicycle detection, in the context of media coverage of specific sport events by drones with increased decisional autonomy. Subsequently, we propose a regularization technique, namely Discriminant Analysis regularization, aiming to enhance the generalization ability of the proposed models. The experimental evaluation validates the enhanced performance of the proposed regularizer.

This paper describes a study that investigates the relation between critical thinking and the metacognitive element of cognition monitoring in asynchronous discussions adopting the Community of Inquiry framework. The aim of this study is to propose measurements reflecting the cognitive development of the community through an asynchronous discussion. These measurements will be integrated in an adaptable visualization learning analytics tool to enhance personalized interaction. The results of the study show moderate positive significant relation between learners' critical thinking and learners' monitoring of the inquiry process whilst the linear regression analysis indicated critical thinking as a significant predicting measurement for learners' monitoring of the inquiry process.

In this paper a novel CNN-based approach in the Content Based Image Retrieval domain that exploits supervised learning is proposed. We employ a deep CNN model to derive feature representations from the activations of the deepest layers and we refine the weights of the utilized layers in order to produce better image descriptors using information obtained from the available data labels. To this end, we adapt the pretrained model and we retrain it on the dataset so that each image representation comes closer in terms of Euclidean distance to its nearest relevant representations and moves away from the irrelevant ones. Experimental results on four publicly available datasets for image retrieval denote the effectiveness of the proposed method in enhancing the retrieval performance, outperforming other CNN-based retrieval techniques in three out of four datasets, as well as traditional handcrafted approaches.

Although learning through personalized representations has been shown to be effective in various domains, little is known about the mechanisms that are effecting it. We focus on a CoI based visualisation tool design, to feature community dynamics effect in a learning design context which promotes interaction with peers and self reflection through personalization.