Nowadays, billions of phones, internet-of-things (IoT), and edge devices around the world generate data continuously, enabling many machine-learning (ML)-based products and applications. However, due to increasing privacy concerns and regulations, these data tend to reside on devices (clients) instead of being centralized for performing traditional ML model training. Federated learning (FL) is a distributed approach in which a single server and multiple clients collaboratively build an ML model without moving data away from clients. Whereas existing studies on FL have their own experimental evaluations, most experiments were conducted using a simulation setting or a small-scale testbed. This might limit the understanding of FL implementation in realistic environments. In this empirical study, we systematically conduct extensive experiments on a large network of IoT and edge devices (called IoT-Edge devices) to present FL real-world characteristics, including learning performance and operation (computation and communication) costs. Moreover, we mainly concentrate on heterogeneous scenarios, which is the most challenging issue of FL. By investigating the feasibility of on-device implementation, our study provides valuable insights for researchers and practitioners, promoting the practicality of FL and assisting in improving the current design of real FL systems.

In this paper, we propose a new dataset, ReasonVQA, for the Visual Question Answering (VQA) task. Our dataset is automatically integrated with structured encyclopedic knowledge and constructed using a low-cost framework, which is capable of generating complex, multi-hop questions. We evaluated state-of-the-art VQA models on ReasonVQA, and the empirical results demonstrate that ReasonVQA poses significant challenges to these models, highlighting its potential for benchmarking and advancing the field of VQA. Additionally, our dataset can be easily scaled with respect to input images; the current version surpasses the largest existing datasets requiring external knowledge by more than an order of magnitude.

Domain shift is a formidable issue in Machine Learning that causes a model to suffer from performance degradation when tested on unseen domains. Federated Domain Gener-alization (FedDG) attempts to train a global model using collaborative clients in a privacy-preserving manner that can generalize well to unseen clients possibly with domain shift. However, most existing FedDG methods either cause additional privacy risks of data leakage or induce signifi-cant costs in client communication and computation, which are major concerns in the Federated Learning paradigm. To circumvent these challenges, here we introduce a novel architectural method for FedDG, namely gPerXAN11https://github.com/lhkhiem28/gPerXAN, which relies on a normalization scheme working with a guiding regularizer: In particular, we carefully design Personalized eXplicitly Assembled Normalization to enforce client mod-els selectively filtering domain-specific features that are bi-ased towards local data while retaining discrimination of those features. Then, we incorporate a simple yet effec-tive regularizer to guide these models in directly capturing domain-invariant representations that the global model's classifier can leverage. Extensive experimental results on two benchmark datasets, i.e., PACS and Office-Home, and a real-world medical dataset, Camelyon17, indicate that our proposed method outperforms other existing methods in ad-dressing this particular problem.

The availability of vast amounts of visual data with diverse and fruitful features is a key factor for developing, verifying, and benchmarking advanced computer vision (CV) algorithms and architectures. Most visual datasets are created and curated for specific tasks or with limited data distribution for very specific fields of interest, and there is no unified approach to manage and access them across diverse sources, tasks, and taxonomies. This not only creates unnecessary overheads when building robust visual recognition systems, but also introduces biases into learning systems and limits the capabilities of data-centric AI. To address these problems, we propose the Vision Knowledge Graph (VisionKG), a novel resource that interlinks, organizes and manages visual datasets via knowledge graphs and Semantic Web technologies. It can serve as a unified framework facilitating simple access and querying of state-of-the-art visual datasets, regardless of their heterogeneous formats and taxonomies. One of the key differences between our approach and existing methods is that VisionKG is not only based on metadata but also utilizes a unified data schema and external knowledge bases to integrate, interlink, and align visual datasets. It enhances the enrichment of the semantic descriptions and interpretation at both image and instance levels and offers data retrieval and exploratory services via SPARQL and natural language empowered by Large Language Models (LLMs). VisionKG currently contains 617 million RDF triples that describe approximately 61 million entities, which can be accessed at https://vision.semkg.org and through APIs. With the integration of 37 datasets and four popular computer vision tasks, we demonstrate its usefulness across various scenarios when working with computer vision pipelines.

Unsupervised Domain Adaptation (UDA) has shown significant advancements in object detection under well-lit conditions; however, its performance degrades notably in low-visibility scenarios, especially at night, posing challenges not only for its adaptability in low signal-to-noise ratio (SNR) conditions but also for the reliability and efficiency of automated vehicles. To address this problem, we propose a Cooperative Students (CoS) framework that innovatively employs global-local transformations (GLT) and a proxy-based target consistency (PTC) mechanism to capture the spatial consistency in day- and nighttime scenarios effectively, and thus bridge the significant domain shift across contexts. Building upon this, we further devise an adaptive IoU-informed thresholding (AIT) module to gradually avoid overlooking potential true positives and enrich the latent information in the target domain. Comprehensive experiments show that CoS essentially enhanced UDA performance in low-visibility conditions and surpasses current state-of-the-art techniques, achieving an increase in mAP of 3.0%, 1.9%, and 2.5% on BDD100K, SHIFT, and ACDC datasets, respectively.

This position paper presents ThothSP, a Semantic Programming framework with the aim of lowering the coding effort in building smart applications on the Device-Edge-Cloud continuum by leveraging semantic knowledge. It introduces a novel neural-symbolic stream fusion mechanism, which enables the specification of data fusion pipelines via declarative rules, with degrees of learnable probabilistic weights. Moreover, it includes an adaptive federator that allows the Thoth⊳runtime to be distributed across multiple compute nodes in a network, and to coordinate their resources to collaboratively process tasks by delegating partial workloads to their peers. To demonstrate ThothSP's capability, we report a case study on a distributed camera network to show ThothSP's behaviour against a traditional edge-cloud setup.

Driven by deep neural networks (DNN), the recent development of computer vision makes vision sensors such as stereo cameras and Lidars ubiquitous in autonomous cars, robotics and traffic monitoring. However, a traditional DNN-based data fusion pipeline like object tracking has to hard-wire an engineered set of DNN models to a fixed processing logic, which makes it difficult to infuse new models to that pipeline. To overcome this, we propose a novel neural-symbolic stream reasoning approach realised by semantic stream reasoning programs which specify DNN-based data fusion pipelines via logic rules with learnable probabilistic degrees as weights. The reasoning task over this program is governed by a novel incremental reasoning algorithm, which lends itself also as a core building block for a scalable and parallel algorithm to learn the weights for such program. Extensive experiments with our first prototype on multi-object tracking benchmarks for autonomous driving and traffic monitoring show that our flexible approach can considerably improve both accuracy and processing throughput compared to the DNN-based counterparts.

Sensor technology and sensor networks have evolved so rapidly that they are now considered a core driver of the Internet of Things (IoT), however data analytics on IoT streams is still in its infancy. This paper introduces an approach to sensor data analytics by using the OpenIoT1middleware; real time event processing and clustering algorithms have been used for this purpose. The OpenIoT platform has been extended to support stream processing and thus we demonstrate its flexibility in enabling real time on-demand application domain analytics. We use mobile crowd-sensed data, provided in real time from wearable sensors, to analyse and infer air quality conditions. This experimental evaluation has been implemented using the design principles and methods for IoT data interoperability specified by the OpenIoT project. We describe an event and clustering analytics server that acts as an interface for novel analytical IoT services. The approach presented in this paper also demonstrates how sensor data acquired from mobile devices can be integrated within IoT platforms to enable analytics on data streams. It can be regarded as a valuable tool to understand complex phenomena, e.g., air pollution dynamics and its impact on human health.

This paper introduces the stack for service delivery models and interoperability in the Internet of Things. The main characteristics and functional layers of the IoT stack are described. The applicability of the IoT stack is described based on particular use cases and deployed pilots. The validation of the IoT stack in terms of functionality and adaptation at different IoT particular areas is based on the Virtual Development Kit (VDK) developed and implemented within the framework of the OpenIoT project-OpenIoT project is the awarded Internet of Things open-source rookie of the year by BlackDuck Software Co. (www.github.com/OpenIotOrg). The methods and standards that boosted OpenIoT-VDK implementation are described in this paper. An instance of the OpenIoT-VDK process is described as the practical use case demonstrating being an IoT platform with autonomic behavior. OpenIoT-VDK creates IoT instances, analyzes the IoT stack dependence, and resolves them following interoperability principles. The OpenIoT-VDK instance deploys IoT service delivery models facilitating the validation of use cases by using the OpenIoT platform. As proof of concept, a delivered IoT service using open data from OpenIoT local instantiation is described.

The W3C Semantic Sensor Network Incubator group (the SSN-XG) produced an OWL 2 ontology to describe sensors and observations — the SSN ontology, available at http://purl.oclc.org/NET/ssnx/ssn. The SSN ontology can describe sensors in terms of capabilities, measurement processes, observations and deployments. This article describes the SSN ontology. It further gives an example and describes the use of the ontology in recent research projects.

The use of RDF data published on the Web for applications is still a cumbersome and resource-intensive task due to the limited software support and the lack of standard programming paradigms to deal with everyday problems such as combination of RDF data from dierent sources, object identifier consolidation, ontology alignment and mediation, or plain querying and filtering tasks. In this paper we present a framework, Semantic Web Pipes, that supports fast implementation of Semantic data mash-ups while preserving desirable properties such as abstraction, encapsulation, component-orientation, code re-usability and maintainability which are common and well supported in other application areas.

Within the vision of ubiquitous multimedia, using multimedia sensor nodes, such as audio, image, and video sensor nodes, in wireless sensor networks can dramatically enhance the capability of sensor networks for event description. Different kinds of holes can easily appear in sensor networks. For example, tens of sensor nodes can consist of a hole in a small area because of overload. Efficiently and reliably transmit these multimedia streaming data back to the base station while bypassing these static or dynamic holes is one of the key challenge issues. In this paper, we propose a new Two Phase geographical Greedy Forwarding (TPGF) routing algorithm for exploring one or multiple optimized hole-bypassing transmission paths. We demonstrate the execution of TPGF routing algorithm by using our implemented simulator NetTopo. Comparison and simulation results show that our algorithm can effectively solve the identified problems and is better than the famous GPSR routing algorithm in both functionality and performance aspects.