Despite recent progress in integrating Large Language Models (LLMs) to enhance robotic capabilities, key challenges remain—particularly in interpreting perceptions, interacting with humans, and executing tasks in real-world environments. These challenges are largely due to LLM hallucinations and lack of formal semantic knowledge that hinder contextual understanding. To address this issue, we propose a semantic framework that combines an LLM and a commonsense knowledge graph to align the robot’s perceptions with a suitable task execution plan. The framework integrates pre-trained deep learning models for object detection, face identification, speech recognition, an LLM, and a local knowledge graph based on Resource Description Framework (RDF). The knowledge graph including scene and object description, as well spatial knowledge is used to select and execute the suitable task plan for the given context. Experimental results demonstrate the effectiveness of framework’s components in improving the robot’s interaction and and actuation in its environment.

Current deep learning-based activity recognition and anticipation approaches struggle with unlabeled datasets and the lack of commonsense knowledge about previously unseen objects, i.e., unknown objects. This paper proposes a neurosymbolic framework that combines context recognition, incremental learning, and commonsense reasoning to anticipate activities from egocentric vision, while accounting for perceptual uncertainty and the incompleteness of real-world commonsense knowledge. The incremental learning component leverages language models, while Bayesian reasoning enables continuous updates to an ontological knowledge graph with class and affordance descriptions of these unknown objects. These descriptions are derived from visual feature embeddings extracted using a transformer and enriched with knowledge from various multi-modal large language models (MLLMs). Event calculus and answer set programming are used to formalize domain knowledge, enabling probabilistic symbolic reasoning over contextual events to support the prediction of user intentions and the anticipation of both simple and complex activities. A prototype has been tested in a domestic environment using images captured from a head-mounted camera and real-time object and context event detection.

The Internet of Behaviors (IoB) provides powerful tools for collecting, analyzing, and interpreting data on human actions and decision-making patterns. However, its application in education remains limited, despite the critical importance of being able to monitor and assess student focus, engagement, and participation in real time to improve learning outcomes and provide timely interventions. This study proposes an IoB-based system that employs deep learning and computer vision to assess and enhance student engagement in real-time. The system's architecture integrates two YOLOv8-based models: a multiclassifier for recognizing six classroom behaviors, and a binary classifier for distinguishing between focused and unfocused states. Anonymized identifiers are used to ensure a consistent and ethical tracking of individual behavior over time. These classifications trigger adaptive interventions designed to maintain or restore students' attention. Given the lack of standard benchmarks for assessing focus in classroom settings, especially under realistic conditions that include ambient noise and peer influence, a dedicated dataset was collected and used to train and evaluate the proposed system's performance in a university classroom context. The experimental results showed that the multi-class model achieved 97% accuracy, while the binary classification model reached 99% accuracy, both outperforming three state-of-the-art baseline methods. In a real-world classroom deployment, 75% of the students who received interventions subsequently re-engaged, underscoring the potential of the framework to improve learning outcomes through timely data-driven behavioral feedback.

This paper presents a novel framework for Human Activity Recognition (HAR) by unifying all sensor streams, visual, audio, and inertial, into a single textual domain, enabling the direct application of GPT-3 for multimodal data classification. Unlike traditional pipelines that use dedicated encoders for each modality, we show that converting sensor outputs into text tokens offers both simplicity and a powerful proof-of-concept for large language models (LLMs). To further boost performance, we introduce a composite loss function combining cross-entropy, Kullback-Leibler divergence, total variation, and multimodal consistency terms, ensuring both temporal smoothness and cross-modal alignment. We conduct extensive experiments on the CMU-MMAC dataset, achieving up to 98% accuracy and significantly outperforming baseline methods. We also demonstrate robustness under missing sensor streams via partial tokenization, maintaining strong performance despite sensor failures. These results highlight the potential of LLM-driven HAR for enhanced human-robot interaction in real-world scenarios, and pave the way for broader multimodal applications of next-generation language models.

Human Activity Recognition (HAR) plays a crucial role in applications such as healthcare, smart environments, and human-robot interaction. This study proposes a novel Hyperbolic optimization strategy to improve model generalization by leveraging the geometric structure of the parameter space. To evaluate its effectiveness while also benefiting from the capabilities of modern sequence models in capturing long-range dependencies and multimodal interactions, the loss function is integrated into Transformer and GPT-2 models, fine-tuned on both unimodal (UCI-HAR, Opportunity) and multimodal (UTD-MHAD, NTU RGB+D) datasets. Unlike prior work that typically leverages only one to three modalities, this study utilizes all available modalities—RGB, depth, skeleton, and inertial—for multimodal evaluation. The Transformer achieves 98.26% and 93.40% accuracy on UCI-HAR and Opportunity, respectively, and 99.08% and 89.93% on UTD-MHAD and NTU RGB+D. GPT-2 also performs competitively, achieving 86.33% and 83.57% on the unimodal datasets, and 83.23% and 86.51% on the multimodal ones. These results highlight the potential of hyperbolic optimization for HAR across diverse sensor modalities and architectures.

This paper presents an innovative approach to enhancing Generative Adversarial Networks (GANs) for human face image generation. Generative tasks are traditionally hindered by issues like mode collapse and unstable training. We introduce a groundbreaking integration of Nash equilibrium principles into GAN architectures, featuring a tailored loss function that significantly stabilizes the training process. Our method not only bolsters GAN stability but also substantially improves the quality and variety of generated expressions. Rigorous experimentation across benchmark datasets like CIFAR-10, CelebA, and ImageNet confirms the exceptional performance of our model, particularly evidenced by significant reductions in the Fréchet Inception Distance, a testament to the method’s efficacy in producing more realistic and diverse facial expressions. The implications of this research are far-reaching, particularly in the realm of human-machine interaction where expressive and diverse facial expressions are crucial. We will extend the application of the proposed method to other domains beyond facial expression generation, such as medical imaging or autonomous vehicle perception systems, to assess its adaptability and effectiveness.

Human activity anticipation is a pillar of new interactive multimedia technologies. It enables better human-computer interaction and creative sensory experience in all types of virtual reality-based multimedia applications. Human behaviors are hierarchically constituted in terms of low-level and high-level activities. Fine-grained activities play a substantial role in the early recognition of simple and complex human movements in real-time with low latency, allowing longer predictions in the future. In this paper, a new objective function is proposed to anticipate fine-grained human activities using IMU data, which still suffers from an imbalance problem. Four customized objective surrogate functions are applied in unidirectional and bidirectional recurrent neural networks and compared to efficiently optimize the model considering the loss of individual classes. The experiments on five datasets across proposed loss functions show that the proposed model significantly outperforms the counterpart methods and advances the state-of-the-art. The proposed model shows accuracy scores of up to 98% and 96% for high-level and low-level activities, respectively.Note to Practitioners—There are a large number of methods that can be used for the recognition of human activity in assisted living, smart home technologies, and pervasive healthcare applications. However, studies on these methods tend to focus on recognition rather than anticipation of human activity. Furthermore, they use either vision-based or egocentric video data, which can make it challenging to collect and evaluate in virtual reality-based scenarios. To enable more natural interaction with physical and virtual reality environments, virtual reality-based technologies require built-in algorithms capable of consistently identifying complex and varied human movements. Fine-grained activities are paramount in predicting activities at an early stage to adequately characterize human behaviors. In this paper, a study is conducted to anticipate fine-grained human activity based on IMU (Inertial Measurement Unit) data, which are still subject to the negative effects of an imbalanced sample distribution. The proposed model is evaluated on different benchmarks and compared with baseline methods. The results show that the proposed model outperforms all baselines. These results have important implications for human activity prediction problems.

In recent years, semantic segmentation has become an increasingly important task in computer vision, with numerous applications in image analysis and object recognition. One popular approach for semantic segmentation is the use of convolutional neural networks (CNNs), such as the U-net architecture, which is known for its ability to capture both local and global contexts in images. However, the U-Net model can be computationally intensive and time-consuming, especially for large or high-resolution images. This paper proposes a depth-residual separable U-net (DRSU-net) model to improve the U-net model for semantic segmentation tasks. The proposed approach introduces an additional regularisation technique instead of dropout and L2 regularisation and modifies the network's structure to reduce the risk of overfitting and the number of model parameters. This reduces training time without losing model information, especially for large datasets. The effectiveness of the DRSU-net method is demonstrated through extensive experiments on several semantic segmentation benchmarks. The results show improved performance and lower temporal complexity compared to the basic U-net and state-of-the-art architectures.

In this work, the results of an exploratory study into the use of the publicly available GPT-2 model for simultaneous conditional human activity signal synthesis and classification are presented. To accomplish this, the small variant of a pre-trained GPT-2 model is fine-tuned on quantized and windowed human activity signal sequences. The conditional generation is achieved by appending and prepending class labels to each window to introduce class information. During the generation phase, the model is prompted either with a class label for synthesizing signals, or a signal sequence for classification by synthesizing a class label. The study is conducted on the publicly available WISDM and UCI-HAR datasets, and the generated signals are evaluated using an LSTM-CNN model. As a classifier, the fine-tuned GPT-2 model achieved an overall accuracy of 90.47% on the WISDM, and 82.29% on the UCI-HAR dataset, which is 4% and 6% lower than the LSTM-CNN model evaluated on the same test subsets. When used as a multi-class generator, on average, 86.33% of the generated data are classified as valid samples by the LSTM-CNN model. While there is room for improvement, these results demonstrate that GPT family architectures can be used for simultaneous conditional signal synthesis and classification, opening up new opportunities for human activity recognition.

Time is an important dimension in sound event detection (SED) systems. However, evaluating the performance of SED systems is directly taken from the classical machine learning domain, and they are not well adapted to the needs of these systems such as recognizing the time, duration, detection, and uniformity of sound events. Despite its importance, it is not well-developed yet. Current methods are highly biased by their assumptions and may misleadingly present convincible results. This paper presents a novel multimodal method to evaluate SED systems from multiple perspectives such as detection, total duration, relative duration, and uniformity. Furthermore, the proposed method is simple, time-efficient, visualizable, extensible, open-source, and overcomes the limitations of existing methods. The benefits of the proposed approach are demonstrated by re-evaluating the best systems presented in a known challenge on sound event detection.

The coronavirus disease 2019 (COVID-19) has been stated as a global pandemic, and the BA.4 and BA.5 variants are anticipated to drive the next wave of COVID-19 infection. Early diagnosis of this infection reduces its viral excretion. In this paper, after a large study of existing algorithms for pre-symptomatic COVID-19 detection in the state-of-the-art, we discovered a notable flaw in most models related to the choice of the evaluation function, such that, all the tested algorithms perform worse (from the evaluation function perspective) than an algorithm that generates alarms randomly from a binomial distribution. Therefore, we propose a simple and less biased evaluation function to better compare the quality of different algorithms. Comprehensive experimental evaluations of the state-of-the-art algorithms over the real-world dataset published by Nature Medicine journal contains 84 COVID-19 patients and 2,000 healthy participants show the effectiveness and the relevance of our evaluation method. Moreover, the proposed framework is released as an open-source library.

The advent of the internet of things (IoT) and artificial intelligence (AI) technologies enable continuous data generation, also known as data streams. Considering data streams is required for ambient intelligence (AmI) systems for context-aware assistance services. This paper proposes a hybrid approach combining deep learning models and probabilistic commonsense reasoning over data streams to anticipate human activities. The reasoning is performed using the event calculus formulated in answer set programming (ECASP); the latter allows for abductive and temporal reasoning, which enables an eXplainable AI (XAI) approach. An activity context ontology is exploited for the reasoning axiomatisation. Several experiments were conducted to demonstrate the effectiveness of the proposed approach in terms of accuracy and computation time.

Endowing ubiquitous robots with cognitive capabilities for recognizing emotions, sentiments, affects, and moods of humans in their context is an important challenge, which requires sophisticated and novel approaches of emotion recognition. Most studies explore data-driven pattern recognition techniques that are generally highly dependent on learning data and insufficiently effective for emotion contextual recognition. In this article, a hybrid model-based emotion contextual recognition approach for cognitive assistance services in ubiquitous environments is proposed. This model is based on: 1) a hybrid-level fusion exploiting a multilayer perceptron (MLP) neural-network model and the possibilistic logic and 2) an expressive emotional knowledge representation and reasoning model to recognize nondirectly observable emotions; this model exploits jointly the emotion upper ontology (EmUO) and the n-ary ontology of events HTemp supported by the NKRL language. For validation purposes of the proposed approach, experiments were carried out using a YouTube dataset, and in a real-world scenario dedicated to the cognitive assistance of visitors in a smart devices showroom. Results demonstrated that the proposed multimodal emotion recognition model outperforms all baseline models. The real-world scenario corroborates the effectiveness of the proposed approach in terms of emotion contextual recognition and management and in the creation of emotion-based assistance services.

With the proliferation of the Internet of Things (IoT), employing Received Signal Strength (RSS) as a metric to determine the location of a target (e.g., person or mobile device) is of great interest in terms of cost and ease of implementation. Indeed, RSS measurements can be easily obtained for most off-the-shelf devices, such as WiFi- or ZigBee compatible devices or sensors. This paper deals with the indoor localization problem in wireless sensor networks (WSNs) and proposes a new approach for radio signal propagation modelling and localization estimation, that accounts for the imperfection of RSS measurements and the reliability of RSS sources by using the Dempster-Shafer Theory (DST). In the signal propagation modelling, key information regarding the geometry of indoor environment that is divided into zones separated by walls (zoning), are considered. Based on the number of walls, the RSS irregularities are estimated using different distance intervals, which are weighted by a probability density determined experimentally. To estimate the location of a target node, the PCR6 rule is used to combine the belief masses of the positions obtained from the probability density. In order to evaluate the performance of the proposed approach, an experimental WSN has been deployed in a living apartment. The obtained results demonstrate that the proposed approach improves the localization accuracy compared to the case without zoning. Moreover, the obtained localization mean error proves the feasibility of a precise localization of humans in indoor environments in the case of Ambient Assisted Living and Social Robotics applications.

Nowadays, a healthy lifestyle is an essential requirement in people's daily life. Although well-being recommendation systems have been extensively explored in different domains, there are still some challenges for developing efficient recommendation systems dealing with the limitations of content-based recommendation approaches. In this paper, a context-aware adaptive recommendation system is proposed to provide personal wellbeing services intended to help people to have a healthy lifestyle in Ambient Assisted Living (AAL) systems. The recommendations are based on people's behaviors. Machine-learning models are firstly used to recognize human activities, locations, and objects. The different contexts of human behaviors, including location, object, frequency, duration, and sequences of frequent activities, are then extracted. An ontology, called Human ActiVity ONtology (HAVON) ontology, is used to conceptualize human activities and their contexts. Finally, a probabilistic version of Answer set Programming (ASP), a high-level expressive logic-based formalism, is proposed to provide adaptive recommendations through a set of probabilistic rules based on human behaviors. A companion robot, called Pepper, is used for the evaluation of the proposed recommendation system. The evaluation results demonstrate the ability of the proposed system to help people to have a healthy lifestyle.

In Ambient Assisted Living (AAL) systems and personal assistive robots, human behavior analysis is essential to provide intelligent services intended to improve people's quality of lives in terms of autonomy, well-being, and safety. Human behavior analysis allows discovering people's preferences, activities, and habits. While human behavior analysis has been explored in several domains, there are still some challenges for developing efficient approaches dealing with the limitations of data-driven approach to analyze human behaviors. In this paper, a framework is proposed to better characterize the human context by inferring new knowledge about his/her behaviors using commonsense reasoning and exploiting contextual information. Human activities are firstly recognized using a CNN-LSTM model. Different contexts of human activities are then extracted to analyze human behaviors. The obtained activity contexts are mapped to an ontology, called Human AcTivity (HAT) ontology, conceptualizing the human activities and their contexts. Answer Set Programming (ASP), a high-level expressive logic-based formalism, is then used to represent human behaviors and carry out commonsense reasoning to infer new knowledge about these behaviors. The proposed framework is evaluated using the Orange4Home dataset. Moreover, two quantitative experiments are carried out to demonstrate the ability of the proposed framework to better characterize human behaviors.

In Ambient Assisted Living (AAL) systems, one of the main objectives is to provide intelligent services to enhance the quality of people's lives in terms of safety, well-being, and autonomy. One of the challenges in designing these systems is abnormal human behavior detection, which is critically important to prevent users, especially elderlies, from dangerous situations. Abnormality detection has been widely explored in various fields; however, challenges remain in developing effective approaches that take into account the limitations of data-driven and knowledge-driven approaches in detecting abnormal human behaviors in AAL systems. In this paper, a hybrid context-aware framework combining a machine-learning model and probabilistic reasoning is proposed to detect abnormal human behavior. An LSTM model is firstly used to classify input data into a set of labels describing human activities. Different human activity contexts, including the duration, frequency, time of the day, locations, used objects, and sequences of the frequent activities, are then extracted to analyze human behaviors. The obtained human activities and behaviors are mapped to the proposed ontology called Human AcTivity (HAT) ontology, which conceptualizes human behavior contexts. Afterward, the abnormal human behaviors are detected using Markov Logic Network (MLN), which combines logic and probability. The concepts and relationships defined in HAT ontology are exploited in defining the FOL rules used in MLN. The proposed framework is evaluated on the Orange4Home dataset and HAR dataset using smartphones. The obtained results demonstrate the ability of the proposed framework to detect abnormal human daily living behavior with high accuracy.

Analysis of human daily living activities, particularly walking activity, is essential for health-care applications such as fall prevention, physical rehabilitation exercises, and gait monitoring. Studying the evolution of the gait cycle using wearable sensors is beneficial for the detection of any abnormal walking pattern. This paper proposes a novel discrete/continuous unsupervised Hidden Markov Model method that is able to recognize six gait phases of a typical human walking cycle through the use of two wearable Inertial Measurement Units (IMUs) mounted at both feet of the subject. The results obtained with the proposed approach were compared to those of well-known supervised and unsupervised segmentation approaches. The obtained results show the efficiency of the proposed approach in accurately recognizing the different gait phases of a human gait cycle. The proposed model allows the consideration of the sequential aspect of the walking gait phases while operating in an unsupervised context that avoids the process of data labeling, which is often tedious and time-consuming, particularly within a massive-data context.

Patients with chronic conditions require medical care at their home. To this end, a smart follow-up and monitoring system is proposed, called Hapicare; which applies ontology-based uncertain reasoning over IoT sensors data and self-assessment. While similar approaches rely on certain events and rules, the proposed monitoring system is based on probabilistic reasoning that interleaves Bayesian and non-monotonic inference. The latter is defined by using rule-based on concepts of the Semantic Sensor Network (SSN) and the SNOMED-CT ontologies. This system also considers uncertain contextual information captured from sensors and the history of patients in order to better diagnose the current situation and trigger suitable reactions. It allows also handling overlaps between symptoms, the possibility of errors and hidden facts. Hapicare is developed in the context of Medolution EU project.

Received signal strength (RSS) is a simple and low-cost method of localization in wireless sensor networks (WSNs) and is of significant interest in ambient intelligence technologies. However, RSS-based indoor localization poses important challenges due to the intrinsic characteristics of RSS measurements. This paper proposes a localization approach that accounts for the imperfection of RSS measurements and the reliability of RSS sources to estimate the target node position in an indoor WSN environment. Non-Gaussian probability density functions are used to model RSS deviations more realistically in the context of indoor environments. In addition, the proposed approach uses the Dempster-Shafer theory to represent and combine separate pieces of information (evidence) provided by more or less reliable or conflicting RSS sources (anchor nodes) on the same hypotheses regarding the target node position. Experiments conducted in two different indoor environments demonstrate the effectiveness of the proposed approach in terms of its accuracy, robustness, and computation time and its superiority compared with state-of-the-art methods. Note to Practitioners-This paper was motivated by the problem of indoor localization in the context of ambient intelligence applications. The localization technique proposed in this paper exploits RSS measurements to estimate the target node position. This technology is very attractive to system designers, due to its simplicity and low cost. This paper also suggests a new approach using, on the one hand, the belief function theory to represent and manage the imperfection of RSS measurements and the reliability of the RSS sources, and, on the other hand, a more realistic modeling of the variability of RSS measurements due to interference and attenuation phenomena that strongly affect signal propagation in indoor environments. Experimental results obtained in two different indoor environments (a residential apartment and a laboratory) are provided to demonstrate the effectiveness of the proposed approach and its superiority compared to state-of-the-art localization techniques.

Building context-aware pervasive computing systems - such as ambient intelligent spaces or ubiquitous robots - needs to take into account the quality of contextual information collected from sensors. Such information are often inaccurate, uncertain or subject to noise due to environment and user dynamics. Dempster-Shafer theory has been extensively adopted to handle uncertainty in situation and activity recognition. This theory is used to represent, manipulate and decide under uncertainty. However, combining information using Dempster's rule may produce counterintuitive decision in highly conflicting evidences due to sources failure. Recently, a variety of rules were proposed to overcome such drawback. Inspired by Murphy's rule, we propose in this paper a new rule called “Weighted Average Combination Rule” (WACR) to deal with context recognition in highly dynamic environment such as ambient intelligence spaces. The proposed WACR rule is based on evidence arithmetic average and cardinality. WACR rule was applied to some conflictual evidence examples and has been shown to reap more appropriate decisions than other alternative rules for decision-making in activity-aware systems. To demonstrate the applicability and performance of our approach, we have studied a scenario of context recognition in an ambient intelligent environment. In this scenario, we simulated a smart kitchen composed of status devices and RFID sensors that allow determining what is the artifact in use by the inhabitant and for which activity.

One of the main objectives of ubiquitous robots is to proactively provide context-aware intelligent services to assist humans in their professional or daily living activities. One of the main challenges is how to automatically obtain a consistent and correct description of human context such as location, activities, emotions, etc. In this paper, a new hybrid approach for reasoning on the context is proposed. This approach focuses on human activity recognition and consists of machine-learning algorithms, an expressive ontology representation, and a reasoning system. The latter allows detecting the inconsistencies that may appear during the machine learning phase. The proposed approach can also correct automatically these inconsistencies by considering the context of the ongoing activity. The obtained results on the Opportunity dataset demonstrate the feasibility of the proposed method to enhance the performance of human activity recognition.

In recent years, a plethora of different studies for design of traditional ensemble classifiers has been proposed in order to improve final recognition accuracy. However, among the ensemble classifiers, combination methods are focused on building independent classifiers of the same or different algorithms using majority voting methods. In this paper, we present a new fusion scheme for ensemble classifiers based on a new concept called Generalized Fuzzy Soft Set (GFSS), which we apply in activity classification. Essentially, we apply a weighted aggregate operator to the output of each classifier in order to fuse the GFSS into a more reliable classifier. The proposed fusion method is based on a new ranking algorithm to classify activities. We show that the proposed method produces more accurate results than the best single classifier and its effectiveness is demonstrated by comparing it with single classifier in terms of activity recognition accuracy.

In the present paper, a method based on a new concept called power fuzzy soft set is proposed for multi-observer decision making problems under uncertain information. The new method applies a weighted conjunctive operator to aggregate these sets into a reliable resultant power fuzzy soft set from the input data set. To decide among the alternatives, a new ranking algorithm is introduced. The effectiveness and feasibility of this method are demonstrated by comparing it to algorithms based on the maximum score in decision making.

Infant's cry is an innate response to express several situations including pain, disturbance and discomfort. Therefore the automatic recognition of infant cries patterns is the key factor to develop successful ambient intelligence applications to enhance the quality of life of both infants and parents. This paper proposes a complete machine learning process including consistent dataset generation from infant cries and selecting appropriate sound features, with promising experimental results for enhancing the monitoring of infants in real world settings. The originality of the proposed approach lies in its ability to detect and analyze automatically discomfort signals, which recurrently affects 20 to 25% of newborns. The machine learning process includes low-level audio features selection methods from labeled infant pre-cry recordings as well as high-level features characterizing the envelop of the crying. The classification is performed using ensemble learning methods after a stage of features selection. The exploitation of pre-crying signals to improve the quality of the recognition is another important aspect of the proposed approach, which optimizes the accuracy of the learning step as it is shown by the obtained results on a real dataset. This result gives the opportunity to develop new baby monitors able to anticipate the infants needs.

Creating a standard for knowledge representation and reasoning in autonomous robotics is an urgent task if we consider recent advances in robotics as well as predictions about the insertion of robots in human daily life. Indeed, this will impact the way information is exchanged between multiple robots or between robots and humans and how they can all understand it without ambiguity. Indeed, Human Robot Interaction (HRI) represents the interaction of at least two cognition models (Human and Robot). Such interaction informs task composition, task assignment, communication, cooperation and coordination in a dynamic environment, requiring a flexible representation. Hence, this paper presents the IEEE RAS Autonomous Robotics (AuR) Study Group, which is a spin-off of the IEEE Ontologies for Robotics and Automation (ORA) Working Group, and and its ongoing work to develop the first IEEE-RAS ontology standard for autonomous robotics. In particular, this paper reports on the current version of the ontology for autonomous robotics as well as on its first implementation successfully validated for a human-robot interaction scenario, demonstrating the developed ontology's strengths which include semantic interoperability and capability to relate ontologies from different fields for knowledge sharing and interactions.

An important challenge to be addressed in the domain of Internet of Things (IoT) is the development of efficient services selection algorithms for an optimal management of both energy and Quality of Service (QoS) in the context of IoT services composition. This issue becomes crucial in the case of large-scale IoT environments composed of thousands of distributed entities. In this paper, an energy-centered and QoS-aware services selection algorithm (EQSA) is proposed for IoT services composition. The proposed selection approach consists of preselecting the services offering the QoS level required for user's satisfaction using a lexicographic optimization strategy and QoS constraints relaxation technique. In order to reduce the energy consumption of a composite service without affecting the user's satisfaction, the most suitable services among the preselected ones are then selected using the concept of relative dominance of services in the sense of Pareto. The relative dominance of a candidate service depends on its energy profile and QoS attributes, and user's preferences. The proposed algorithm has been evaluated through several simulation scenarios. The obtained results show clearly the good performances of the EQSA algorithm in terms of selection time, energy efficiency, composition lifetime, and optimality and its added value in comparison with algorithms dealing separately with QoS and energy consumption.

The Internet of Things is the natural continuity of the Ambient Intelligence where smart and ambient environments are built mainly by integrating a large number of interconnected smart objects (sensors, actuators, Smartphone, appliances, etc.) with heterogeneous capabilities abstracted as software services. These services can be composed on the fly and provided, all the time and everywhere, to assist users in their daily activities. A key issue in user-centered services composition is to intelligently and effectively discover and select the most relevant services that best match the users' requirements and closely meet the specified quality-of-service level. Monitoring seamlessly the provided services and enhancing their quality, is still a challenging issue due mainly to the dynamicity and uncertainty characterizing ambient environments. In this paper, we propose a new service-oriented, user-centered and event-aware Framework capable of performing services monitoring to handle automatically events that may occur in ambient environments. This monitoring is based on a dynamic services discovery and selection process to enhance self-adaptation to unpredicted changes, and ensure services continuity with best quality. The overall proposed Framework has been implemented and validated through a scenario dedicated to daily activity recognition in an Ambient-Assisted Living environment. In addition, the obtained performances from extensive tests show clearly the efficiency and feasibility of the proposed approach in the case of a large-scale environment.

Abstract Action languages have emerged as an important field of knowledge representation for reasoning about change and causality in dynamic domains. This paper presents Cerbere, a production system designed to perform online causal, temporal and epistemic reasoning based on the Event Calculus. The framework implements the declarative semantics of the underlying logic theories in a forward-chaining rule-based reasoning system, coupling the high expressiveness of its formalisms with the efficiency of rule-based systems. To illustrate its applicability, we present both the modeling of benchmark problems in the field, as well as its utilization in the challenging domain of smart spaces. A hybrid framework that combines logic-based with probabilistic reasoning has been developed, that aims to accommodate activity recognition and monitoring tasks in smart spaces.

The next generation of ambient assisted living services will be based on eco-systems or organizations of intelligent artificial agents embodied in companion robots and smart objects. To provide, anywhere and anytime, smart assistance services to people, these agents need to be endowed with advanced knowledge representation, reasoning and communication capabilities. In this paper, we propose a distributed cognitive architecture allowing to integrate seamlessly the actors of the ambient system and an expressive model for commonsense knowledge representation and reasoning on events. This model allows a common description of the actors in an open world and a management of interactions with humans using natural language. A scenario dedicated to the cognitive assistance of frail people is implemented and analyzed for validation purposes of the proposed approach.

Fault diagnosis is a critical task for operators in the context of e-TOM (enhanced Telecom Operations Map) assurance process. Its purpose is to reduce network maintenance costs and to improve availability, reliability and performance of network services. Although necessary, this operation is complex and requires significant involvement of human expertise. The study of the fundamental properties of fault diagnosis shows that the diagnosis process complexity needs to be addressed using more intelligent and efficient approaches. In this paper, we present a hybrid approach that combines Bayesian networks and case-based reasoning in order to overcome the usual limits of fault diagnosis techniques and to reduce human intervention in this process. The proposed mechanism allows the identification of the root cause with a finer precision and a higher reliability. At the same time, it helps to reduce computation time while taking into account the network dynamicity. Furthermore, a study case is presented to show the feasibility and performance of the proposed approach based on a real-world use case: a virtual private network topology.

The capability to diagnose the root cause of an observed problem precisely and quickly is a desirable feature for large communication networks. However, the design of a technique that is at the same time fast, scalable and accurate is a challenging task. In this paper, we propose a novel method based on inter-cluster inference to overcome the usual limits of fault diagnosis techniques. The approach is based on two important concepts: a cluster decomposition of the dependency graph in order to ensure scalability, and the introduction of duplicated nodes aiming at preserving the end-to-end network view. The evaluation of the proposed approach has demonstrated a significant reduction in the complexity and the computation time of the root cause analysis, since it is based on a set of small-scale dependency graphs.

In this paper, we present a semantic framework that is intended to enable natural interactions between ubiquitous robots and humans in Ambient Intelligence (AmI) environments. The main contribution of this paper is the extension of the core of the Narrative Knowledge Representation Language (NKRL) framework with semantic modules to allow on one hand, converting robot interactions into formal n-ary semantic annotations, and on the other hand, making semantic inferences for: (i) driving the human-robot dialogue, (ii) inferring the spatio-temporal context of the overall dialogue and (iii) mapping the inferred context with the actions that should be triggered in the AmI environment. A scenario dedicated to the monitoring and cognitive assistance of elderly people is implemented and discussed for validation purposes of the proposed framework.

Ambient intelligence, ubiquitous and networked robots, cloud robotics, are new research hot topics that start to gain popularity among the robotics community. They enable robots to acquire richer functionalities and open the way for the composition of a variety of robotic services with three functions: semantic perception, reasoning and actuation. This paper introduces the recent challenges and future trends of these topics.

Ubiquitous robotics, ambient Intelligence and cloud robotics are new research topics that start to gain popularity among the robotics community. The overlap that exists now between ubirobots and AmI makes their integration together within cloud computing valuable to create a hybrid physical-digital space rich with myriad of proactive intelligent services that enhance the quality and the way of our living and working. Indeed, there is no visible cooperation or synergies between ubiquitous computing and robots communities. This paper presents a workshop that is intended to facilitate discussions and build a bridge between these communities. Moreover we hope that satellite topics such as affective computing, semantic reasoning, and humancomputer interaction will enhance such discussions and foster the research in this area. We believe that both communities can learn and benefit from their mutual experiences.

We present a rule-based architecture (CIDA) for the provision of relevant information tailored to the user's current context, which consists of an ontology-based context model (ConAD) and a rule engine (RARE). ConAD is extensible to characterize the contextual situations in urban environments, and RARE supports the reconfiguration of the behavior of systems in different situations. Our preliminary evaluation results based on emergency scenarios show that CIDA is feasible for providing context-aware urban information services.

In this article, we present the ongoing efforts within the newly formed IEEE-RAS Working Group named Ontologies for Robotics and Automation. We focus in particular on one of the four subgroups that compose this working group, called the Upper Ontology/Methodology (UpOM) subgroup. As the name indicates, the aim of this subgroup is to develop an upper ontology and a methodology for ontology building and evaluation. This methodology will be used to coordinate the distributed development of domain specific ontologies by the other three subgroups, and to generate a global ontology that we hope will contribute to the standardization process of robotics and automation. This paper presents the composition, the envisioned methodology, and the future work of the UpOM subgroup. It also discusses some general aspects related to ontologies for the robotics and automation field, and to the efforts in standardizing them.

In ambient intelligence environments, the information provided by robot's embedded sensors and physical or logical entities may be inaccurate and uncertain. The Dempster-Shafer evidence Theory (DST) gives a mathematical convenient framework for the evidential fusion representation and inference of uncertain information. However, DST yields counterintuitive results in high conflicting ambient intelligence situations. This paper aims to provide a new strategy to manage conflict in activity recognition process in the ambient intelligence applications. It addresses the challenge of uncertainty and proposes an evidential fusion model based on the management of conflicting situation to optimize decision making in activity recognition. The proposed approach gives intuitive interpretation for combining multiple sources in conflicting situations and avoids the problems of using The Dempster-Shafer rule of combination.

Fault diagnosis is one of the most important tasks in fault management. The main objective of the fault management system is to detect and localize failures as soon as they occur to minimize their effects on the network performance and therefore on the service quality perceived by users. In this paper, we present a new hybrid approach that combines Bayesian Networks and Case-Based Reasoning to overcome the usual limits of fault diagnosis techniques and reduce human intervention in this process. The proposed mechanism allows identifying the root cause failure with a finer precision and high reliability while reducing the process computation time and taking into account the network dynamicity.

Enabling cross-organizational cooperation in ubiquitous computing environments poses new security challenges that concern particularly the interoperability of security management systems and the security policies of each organization. In this paper, we present a semantic framework for cooperative security management processes design in a cross-organizational context. Our framework is based on a hybrid approach that caters between the advantages of bottom-up and top down approaches. The cooperation model of our framework is based on the composition of atomic security management processes, by using speech acts and ontologies, and their mapping with internal processes views. The establishment of an e-contract between partners allows specifying common terminology for exchanging messages, cooperation security policy and process control flows. A scenario of cooperative process is presented to demonstrate the feasibility of the proposed framework.

This paper deals with some conceptual and practical problems that become apparent when trying to make use of up-to-date ontological and semantic techniques within a concrete industrial context. More precisely, it describes the compromises adopted in the European SEMbySEM project in order to deal with specific industrial problems - like checking rail transport of hazardous materials - by preserving a general compatibility with the W3C (World Wide Web Committee) suggestions.

The service composition aims to provide a variety of high level services. Recent approaches cannot fully satisfy the requirement raised by ubiquitous environment. In this paper, we propose a layered design framework which aims at being flexible and robust to failure service composition. It adopts an abstract way of generating plan using rule-based techniques in order to adapt to the changes occurring on the services and the context of use. The approach optimizes the number of services and the recomposition time in large-scale environment by removing the phase of rediscovery. The framework for service composition and monitoring includes learning mechanism for the service selection, based on an estimation of the reputation for abstract services and the quality (QoS) for concrete services. The proposed approach is tested, under USARSim simulator, on a set of ubiquitous services for assisting elderly or dependant person in a residential environment. The obtained results show the feasibility and the scalability of the approach and a better reactivity to the dynamic and uncertain nature of the ubiquitous environment.

With the growing emergence of ubiquitous computing and networked systems, ubiquitous robotics is becoming an active research domain. The issue of services composition to offer seamless access to a variety of complex services has received widespread attention in recent years. The majority of the proposed approaches have been inspired from the research undertaken jointly on Workflow and AI-based classical planning techniques. However, the traditional AI-based methods assume that the environment is static and the invocation of the services is deterministic. In ubiquitous robotics, services composition is a challenging issue when the execution environment and services are dynamic and the knowledge about their state and context is uncertain. The services composition requires taking into account the parameters of quality of service (QoS) to adapt the composed service to context of the user and the environment, in particular, dealing with failures such as: service invocation failures, network disconnection, sensor failures, context change due to mobility of objects (robots, sensors, etc.), service discovery failures and service execution failures. In this paper, we present a framework which gives ubiquitous robotic system the ability to dynamically compose and deliver ubiquitous services, and to monitor their execution. The main motivation behind the use of services composition is to decrease time and costs to develop integrated complex applications using robots by transforming them from a single task issuer to smart services provider and human companion, without rebuilding each time the robotic system. To address these new challenges, we propose in this paper a new framework for services composition and monitoring, including QoS estimation and Bayesian learning model to deal with the dynamic and uncertain nature of the environment. This framework includes three levels: abstract plan construction, plan execution, and services discovery and re-composition. This approach is tested under USARSim simulator on a prototype of ubiquitous robotic services for assisting an elderly person at home. The obtained results from extensive tests demonstrate clearly the feasibility and efficiency of our approach.

The field of ubiquitous robotics is becoming an active research domain. One of the more challenging of this domain is providing services composition in a seamless manner. Several recent research efforts have dealt with the services composition problem in ubiquitous environment. However, most of them assume that the composition plan was already constructed despite of the major challenge and complexity that involves this task. In this paper, we propose an approach which generates automatically a flexible plan for the services composition by optimising both the number of services and parameters which appear in this composition. Our plan is constructed in an abstract way in order to be adaptable to the changes which occur on the services and the context of use. This approach is tested under USARSim simulator on a set of ubiquitous robotic services which assist an elderly person. We have also tested our approach when the complexity of services composition increases. Obtained results show clearly the feasibility and scalability of our approach in ubiquitous environment.