The rapid evolution of autonomous driving technology hinges on advanced perception systems, which are integral for safe and efficient vehicle navigation. While effective, traditional approaches often lack semantics, preventing their application in real use cases. To address this challenge, we propose an innovative approach combining semantic representations and integration of semantic artefacts with distillation techniques. We integrate domain semantic artefacts to create a knowledge graph that provides a structured framework of the driving environment, including dynamic elements like pedestrian behavior. This integration enables the vehicle to accurately interpret and react to various events, particularly in complex urban scenarios. Further enhancing this system, we distill knowledge from a complex, pre-trained segmentation model into a more lightweight counterpart to have a more robust approach, achieving a streamlined segmentation process required in real-time autonomous driving systems. A focal point of our study is a case analysis of pedestrian detection at crosswalks, a critical aspect of urban driving. The results showcase the efficacy of our framework, elevating the decision-making process in autonomous vehicles.

Abstract In the current landscape of sports data analysis, particularly in football, the automatic extraction of information from match videos is a key challenge. Traditional video annotation techniques rely on the generation of natural language descriptions from videos or detection analysis of static frames. The first approach has limitations in the level of detail of the information extracted, while the second does not analyze the information at a high semantic level. This work presents an efficient framework for the automatic construction of Temporal Knowledge Graphs from unlabeled football match videos, using computer vision techniques and multimodal large-language models. The constructed graph refers to the entire video of the match and contains information on the positional values of the players on the field, jersey number identification, team recognition, interactions, and ball possession. The framework implements a solution for handling failures without keyframe loss, as well as support for parallel processing. In addition, we propose two metrics to assess the quality of the resulting graph in terms of overall consistency and missing information.

Agriculture is a multidisciplinary domain that spans a wide range of sectors, including horticulture, animal husbandry, and water and land resource management. The intricate complexity of this field arises from the interconnectivity of its diverse sub-sectors, requiring advanced knowledge-specific expertise to address interdisciplinary challenges effectively. Traditional methods for integrating knowledge across sub-sectors are often limited in their ability to deliver precise and context-specific insights, particularly for complex, multi-sectors inquiries.In this paper, we present a multi-sectors framework for question-answering in agriculture. Our framework utilizes multiple Large Language Models, each of which is specialized in a sector of agriculture. By adopting the Mixture of Experts paradigm, our framework synthesizes the expertise of multiple Large Language Models to produce accurate and interdisciplinary answers to questions spanning multiple agricultural sectors. We have evaluated the performance of our approach using qualitative metrics. Our results demonstrate the effectiveness of our approach compared with a baseline Large Language Model question-answering strategy.

Hematoxylin and Eosin (H&E) staining remains a standard of histopathological diagnostics due to its efficiency, affordability, and ubiquity. However, its lack of molecular specificity often necessitates complementary immunohistochemistry (IHC) to detect protein-level biomarkers critical for diagnosis and treatment planning. In scenarios where IHC is impractical, due to cost, limited tissue, or logistical constraints, computational alternatives may provide viable solutions. In this study, we investigate the feasibility of predicting CAF-1/p60 protein expression directly from H&E-stained slides using deep learning. Leveraging a curated dataset of oral squamous cell carcinoma samples with nucleus-level annotations, we classify nuclei into three categories: (i) Tumor-CAF-1/p60-positive, (ii) Tumor-CAF1/p 6 0-negative, and (iii) stromal. Our results demonstrate that Convolutional Neural Networks (CNNs) can learn morphological cues indicative of protein expression, achieving promising classification performance at the single-cell level. This approach may enable more accessible molecular profiling from routine H&E slides, reducing dependency on specialized staining procedures in resource-constrained or time-sensitive settings.

OBJECTIVE: The integration of multi-omics data is essential for understanding complex biological systems, providing insights beyond single-omics approaches. However, challenges related to data heterogeneity, standardization, and computational scalability persist. This study explores the interdisciplinary application of semantic technologies to enhance data integration, standardization, and analysis in multi-omics research. METHODS: We performed a systematic mapping study assessing literature from 2014 to 2024, focusing on the utilization of ontologies, knowledge graphs, and graph-based methods for multi-omics integration. RESULTS: Our findings indicate a growing number of publications in this field, predominantly appearing in high-impact journals. The deployment of semantic technologies has notably improved data visualization, querying, and management, thus enhancing gene and pathway discovery, and providing deeper disease insights and more accurate predictive modeling. CONCLUSION: The study underscores the significance of semantic technologies in overcoming multi-omics integration challenges. Future research should focus on integrating diverse data types, developing advanced computational tools, and incorporating AI and machine learning to foster personalized medicine applications.

Player scouting in soccer is witnessing a surge of interest from the research community. Traditional scouting methods are often limited by subjectivity and biases in evaluation. Moreover, the lack of structured data and models hinders the progress of the field. To overcome these limitations, we introduce a novel player recommendation system which integrates similarity techniques and generative artificial intelligence. It aims to support player recruitment by providing a data-driven and inclusive approach. The novelty of our work lies in its use of advanced machine learning and artificial intelligence to accurately predict player potential and performance by similarity measures, thereby mitigating the influence of subjective biases that often affect talent identification. Our contributions represent a significant advancement in the field of sports analytics and talent identification, offering a more equitable and efficient approach to scouting and recruitment. The results obtained underscore the effectiveness of the proposed system, demonstrating the transformative potential of artificial intelligence in revolutionizing talent scouting.

The domain of nutrigenetics investigates the complex relationship between genetic variations and individual dietary responses, encompassing a wide array of disciplines, including genomics, nutrition science, bioinformatics, and personalized medicine. This field is marked by its intricate data landscape, necessitating innovative approaches to effectively manage and interpret the vast volumes of information involved. Given nutrigenetic data sheer volume and complexity, traditional AI models often struggle to maintain comprehensive and up-to-date knowledge. In this paper, we propose an implementation of the Retrieval-Augmented Generation (RAG) strategy to address the question-answering task in nutrigenetic domain. This framework enhances the accuracy and relevancy of outputs produced by an advanced Large Language Model, circumventing the exhaustive model fine-tuning process. As a result, our RAG approach not only alleviates the computational demand but also fortifies against data leakage concerns, particularly critical in the sensitive area of nutrigenetics. The implementation of RAG in the nutrigenetic domain not only addresses the existing challenges but also paves the way for more advanced and efficient exploration of nutrigenetic data. Our proposed workflow could advance the understanding of nutrigenetic interactions and personalized nutrition.

Abstract The detection of violations of intellectual properties on multimedia files is a critical problem for the current infrastructure of the Internet, especially within very large document collections. To contrast such a problem, either proactive or reactive methods are used. The first category prevents the upload of infringing files themselves by comparing illegal files with a reference collection, while the second one responds to reports made by third parties or artificial intelligence systems in order to delete files deemed illegal. In this article we propose an approach that is both reactive and proactive at the same time, with the aim of preventing the deletion of legal uploads of files (or modifications of such files, such as remixes, parodies and other edits) due to the presence of illegal uploads on a platform. We developed a rule-based obfuscating focused crawler able to work with audio files in the Audio Information Retrieval (AIR) domain, but its use can be easily extended to other multimedia file types, such as videos or textual documents. Our proposed model automatically scans multimedia files uploaded to the public collection only when a user query is submitted to it. We will also show experimental results obtained during tests on a known musical collection. Several combinations of specific Neural Network-Similarity Scorer solutions are shown, and we will discuss the strength and efficiency of each combination.

Abstract Identifying cases of intellectual property violation in multimedia files poses significant challenges for the Internet infrastructure, especially when dealing with extensive document collections. Typically, techniques used to tackle such issues can be categorized into either of two groups: proactive and reactive approaches. This article introduces an approach combining both proactive and reactive solutions to remove illegal uploads on a platform while preventing legal uploads or modified versions of audio tracks, such as parodies, remixes or further types of edits. To achieve this, we have developed a rule-based focused crawler specifically designed to detect copyright infringement on audio files coupled with a visualization environment that maps the retrieved data on a knowledge graph to represent information extracted from audio files. Our system automatically scans multimedia files that are uploaded to a public collection when a user submits a search query, performing an audio information retrieval task only on files deemed legal. We present experimental results obtained from tests conducted by performing user queries on a large music collection, a subset of 25,000 songs and audio snippets obtained from the Free Music Archive library. The returned audio tracks have an associated Similarity Score, a metric we use to determine the quality of the adversarial searches executed by the system. We then proceed with discussing the effectiveness and efficiency of different settings of our proposed system. Graphical abstract

Abstract Recent technologies advancements promise to change our lives dramatically in the near future. A new different living society is progressively emerging, witnessed from the conception of novel digital ecosystems, where humans are expected to share their own spaces and habits with machines. Humanoid robots are more and more being developed and provided with enriched functionalities; however, they are still lacking in many ways. One important goal in this sense is to enrich their cognitive capabilities, to make them more “intelligent” in order to better support humans in both daily and special activities. The goal of this research is to set a step in bridging the gap between symbolic AI and connectionist approaches in the context of knowledge acquisition and conceptualization. Hence, we present a combined approach based on semantics and machine learning techniques for improving robots cognitive capabilities. This is part of a wider framework that covers several aspects of knowledge management, from representation and conceptualization, to acquisition, sharing and interaction with humans. Our focus in this work is in particular on the development and implementation of techniques for knowledge acquisition. Such techniques are discussed and validated through experiments, carried out on a real robotic platform, showing the effectiveness of our approach. The results obtained confirmed that the combination of the approaches gives superior performance with respect to when they are considered individually.

Abstract Automatic storytelling is a broad challenge in research contexts such as Natural Language Processing and Contend Based Image Analysis. Despite the considerable achievements of machine learning techniques in these research fields, combining different approaches to fill the gap between an automatic generated story and human handwriting is hard. This work proposes a novel storytelling framework in the Cultural Heritage domain. We developed our framework based on a Multimedia Knowledge Graph (MKG), a crucial point of our work. Furthermore, we populated our Multimedia Knowledge Graph with a focused crawler that employs deep learning techniques to recognise a multimedia object from web resources. Furthermore, we used a combined approach of deep learning techniques and Linked Open Data (LOD) to retrieve information about images and depicted figures using Instance Segmentation. The system has a dynamic, user-friendly interface that guides the user during the storytelling process. Finally, we evaluated the system from a qualitative and quantitative point of view.

In the last decade, many digital slides have been available in the pathological field thanks to the spreading of new technologies for computerized acquisition. Often hardware and software tools and devices are different among biomedical analysis centers; consequently, the digital slides do not have the same representation using different colorization, exposition, contrast, brightness, and other distortions. Many computer vision algorithms are sensitive to these differences, and, in specific tasks such as image retrieval, color stain normalization can be a helpful technique to mitigate this misunderstanding. In this paper, we explored the effects of color stain normalization in the patches based on Hematoxylin and Eosin (H&E) image retrieval to measure how and how much it impacts the accuracy of this task providing an exhaustive analysis employing a standard dataset.

The growth of data in volume and complexity needs automatic tools to manage and process information. Semantic Web Technologies are a silver bullet in this context due to their capacity to transform human-readable contents into machine-readable ones. Knowledge graphs and the related ontologies represent essential tools for managing very large knowledge bases. The population process of these knowledge structures is composed of expensive and time-consuming tasks, and we propose a novel approach to automate the population step. Our approach is based on novel techniques based on semantic analysis and deep learning using NoSQL technologies. Several results to show the effectiveness of our approach is also reported.

Abstract The rapid increase of available data in different complex contexts needs automatic tasks to manage and process contents. Semantic Web technologies represent the silver bullet in the digital Internet ecosystem to allow human and machine cooperation in achieving these goals. Specific technologies as ontologies are standard conceptual representations of this view. It aims to transform data into an interoperability format providing a common vocabulary for a given domain and defining, with different levels of formality, the meaning of informative objects and their possible relationships. In this work, we focus our attention on Ontology Population in the multimedia realm. An automatic and multi-modality framework for images ontology population is proposed and implemented. It allows the enrichment of a multimedia ontology with new informative content. Our multi-modality approach combines textual and visual information through natural language processing techniques, and convolutional neural network used the features extraction task. It is based on a hierarchical methodology using images descriptors and semantic ontology levels. The results evaluation shows the effectiveness of our proposed approach.

The current trends in society evolution are showing rapid changes in our habitual environments and consequently affecting human interactions with them. Among the key factors of this process we can certainly cite the growth of BigData favored by the knowledge digitalization, the dissemination of sensors in environments and the advancements of connectivity capabilities. At the same time, the progress in artificial intelligence and cognitive robotics has lead to the production of sophisticated humanoid robots, which are progressively spreading to a wide public. In this way, research efforts are needed for a proper knowledge management and knowledge acquisition by machines, in order to have more natural and friendly human-robot interactions in daily tasks, usually performed by human beings. In this paper we show an approach related to a human-robot interaction in cultural heritage context, simulating a digital ecosystem where a robot plays the role of a guide for tourists and it is able to proactively interact with its interlocutors by combining both semantic and visual information. The proposed approach, its implementation and experimental results on a real robotic platform are shown and discussed.

Abstract The synthesis process of document content and its visualization play a basic role in the context of knowledge representation and retrieval. Existing methods for tag-clouds generations are mostly based on text content of documents, others also consider statistical or semantic information to enrich the document summary, while precious information deriving from multimedia content is often neglected. In this paper we present a document summarization and visualization technique based on both statistical and semantic analysis of textual and visual contents. The result of our framework is a Visual Semantic Tag Cloud based on the highlighting of relevant terms in a document using some features (font size, color, etc.) showing the importance of a term compared to other ones. The semantic information is derived from a knowledge base where concepts are represented through several multimedia items. The Visual Semantic Tag Cloud can be used not only to synthesize a document but also to represent a set of documents grouped by categories using a topic detection technique based on textual and visual analysis of multimedia features. Our work aims at demonstrating that with the help of semantic analysis and the combination of textual and visual features it is possible to improve the user knowledge acquisition by means of a synthesized visualization. The whole strategy has been evaluated by means of a ground truth and compared with similar approaches. Experimental results show the effectiveness of our approach, which outperforms state-of-art algorithms in topic detection combining both visual and semantic information.

Nowaday the massive and growing production of digital data is the natural consequence of the use of ICT technologies in every aspects of human life. The benefits carried out by the digital revolution are innumerable both for users involved with the retrieval of useful and rich information and for companies which tries to have profits from data processing knowledge discovery. For this reason, data have a crucial role both from technology and economic-financial point of view. In fact, it is often not trivial to extract meaningful and reusable information from the large amount of available data for different reasons (e.g. unstructured data, lack of ability to process the size of data, standardless representation of data leading to heterogeneity). One of the most serious issues to consider in this context is heterogeneity, which can arise at different levels of abstraction, preventing a richer and more exhaustive retrieval of information. In our opinion, the matching and merging of various information sources become crucial process in order to create a comprehensive knowledge base smart applications. The use of pure formalisms, i.e. ontologies, is a good strategy to limit the gap originating from different sources. In this paper we propose a novel technique for matching and merging ontologies, which aims to create an integrated ontology based on standard representations of abstract concepts related to a particular domain. Then we present a real use of our methodology by means a use case implementing a graph-based knowledge base related to a complex domain related to cultural heritage.

The development of personal robots or service robots has aroused much interest in recent times, witnessing an amazing growth of robotics in different domains. Although sophisticated humanoid robots have been developed, much more effort is needed for improving their cognitive capabilities. Interactions with humans and / or with other agents are still limited and not considered satisfactory. So, the way we store and represent knowledge in a cognitive architecture(CA) is fundamental in order to overcome these limitations and improve human-machine and machine-machine interactions. In this paper we propose a multimedia graph knowledge base using pure formalisms, which can be used by personal robots to represent and share the acquired knowledge, linking it to abstract concepts gifted with semantic relations. Moreover we propose a prototype architecture for sharing the knowledge between two agents following the Knowledge-as-a-service (KaaS) paradigm.

The user is a basic component in the whole information management process and for this reason s/he has to be taken into account in the design and implementation of frameworks and systems for information retrieval. For this reason, the problem of defining efficient techniques for knowledge representation in a user's prospective is becoming a challenging topic in both academic and industrial community. Moreover, the large amount of available data induces several problems from different points of view and it is stressed in the bigdata vision. In this latter some approaches related to the semantic web could be used to improve the data models which underlie the implementation of user-based information retrieval applications. In this paper we propose the use of a semantic approach to design the structure of a multimedia BigData. In addition, the recognition of multimodal features to represent concepts and its attributes together with linguistic properties to relate them are an effective way to bridge the gap between the target semantic classes and the available low-level multimedia descriptors. Information about users (i.e. position) is also used to recognize his/her behaviour and improve the quality of the information retrieval process. Our framework has been implemented using a NoSQL graphdb populated from very large knowledge sources and mobile technologies. Extended experiments are presented to show the effectiveness of our approach.

The use of formal representation is a key task in the era of big data. In the context of multimedia big data this issue is stressed due to the intrinsic complexity nature of this kind of data. Moreover, the relations among objects should be clearly expressed and formalized to give the right meaning of data correlation. For this reason the design of formal models to represent and manage information is a necessary task to implement intelligent information systems. In this latter some approaches related to the semantic web could be used to improve the data models which underlie the implementation of big data applications. Using these models the visualization of data and information become an intrinsic and strategic task for the analysis and exploration of multimedia BigData. In this paper we propose the use of a semantic approach to formalize the model structure of multimedia BigData. In addition, the recognition of multimodal features to represent concepts and linguistic properties to relate them are an effective way to bridge the gap between the target semantic classes and the available low-level multimedia descriptors. The proposed model has been implemented in a NoSQL graph database populated from different knowledge sources and a visualization of this very large knowledge base has been presented and discussed as a case study.