Abstract Dynamic Treatment Regimes (DTRs) are adaptive treatment strategies that allow clinicians to personalize dynamically the treatment for each patient based on their step-by-step response to their treatment. There are a series of predefined alternative treatments for each disease and any patient may associate with one of these treatments according to his/her demographics. DTRs for a certain disease are studied and evaluated by means of statistical approaches where patients are randomized at each step of the treatment and their responses are observed. Recently, the Reinforcement Learning (RL) paradigm has also been applied to determine DTRs. However, such approaches may be limited by the need to design a true reward function, which may be difficult to formalize when the expert knowledge is not well assessed, as when the DTR is in the design phase. To address this limitation, an extension of the RL paradigm, namely Inverse Reinforcement Learning (IRL), has been adopted to learn the reward function from data, such as those derived from DTR trials. In this paper, we define a Projection Based Inverse Reinforcement Learning (PB-IRL) approach to learn the true underlying reward function for given demonstrations (DTR trials). Such a reward function can be used both to evaluate the set of DTRs determined for a certain disease, as well as to enable an RL-based intelligent agent to self-learn the best way and then act as a decision support system for the clinician.

Modern Medical Information Systems very often comprise Medical Devices and governed by regulations which require stringent Risk Management activities to be implemented to minimize the occurrence of safety risks. Currently, the reference standard adopted by manufacturers for Risk Management is ISO 14971, which, however, was devised for traditional (mostly hardware) Medical Devices and does not either take into account the peculiarities of modern Medical Information Systems, or define a formal methodology to conduct Risk Assessment. Moreover, the approaches currently implemented by manufacturers typically aims at obtaining qualitative Risk Assessment results. Within the so-delineated application scenario, this paper proposes a methodology for the Dynamic Probabilistic Risk Assessment of Medical Information Systems, by specifically looking at medical devices that are intended as one of the most relevant components in such systems. The methodology complies with ISO 14971 and improves current practices because it allows the analyst to conduct a quantitative analysis, also taking into account the temporal dimension. It relies on a Probabilistic Risk Model, defined as a set of Markov Models, which is model-checked to obtain quantitative information about the risks. The proposed methodology is also adopted to improve definitively the Medical Device post-market surveillance, which is currently implemented as a ”wait for an incident” activity. In other words, currently a manufacturer sets up a service that has to ”react” to an incident by starting an investigation activity. Instead, the methodology proposes the adoption of risk models defined during the development phase also to re-assess periodically the risks related to the product during the post-market surveillance. This may prevent some incidents because risks are assessed using data collected in the field (no longer guesstimated as during the development phase) and taking into account the temporal effects on probability distributions (such as the deterioration of hardware/software components over the time).

In this paper, we propose and empirically assess Markovian Probabilistic Model for Risk Analysis and Forecasting in Big Healthcare Data Settings. Our proposed model introduce several aspects of research innovation, within the broader context of so-called Medical Information Systems. The proposed model is critically assessed on top of a reference case study coming from a real-life research project in the context of hospital and healthcare management.

This article presents a patient assistance system-based on home sensors, Ambient Intelligence, and Artificial Intelligence-that helps the elderly during their medical treatment at home to reduce medication errors. The patient may have some physical and/or cognitive disabilities.The system consists of a Tutor and a Checker. The Tutor is an intelligent medication reminder that relies on Reinforcement Learning. The Checker, instead, integrates three different intelligent technologies(Deep Neural Network, Optical Character Recognition, and Barcode Reading) to check the adherence to the therapy. The final objective is both to remind effectively when and what medication is to be taken, and check that the patient will take the correct medication.

Typical cruise-line itineraries are planned more than a year in advance, so it is possible to know how many passengers arrive in advance months ahead, as well as when they leave and for how long they stay. Nevertheless, cruise passengers are often responsible of what is called “tourism crowding” that has a beneficial economic outcome for a destination, but often at the cost of increment of city traffic, overloading of the public transportation means and discomfort generated for residents. So, cruise tourism requires planning and management that takes into account also port city destinations factors. In this paper, we present a Deep Reinforcement Learning based planner for the onshore touristic itineraries and the intelligent distribution of cruise passengers in a city. The aim is to maximise the number of touristic attraction locations visited during a tour by avoiding the overcrowding of the touristic attraction locations. The planner is able to compose onshore touristic itineraries meeting the constraint given within the time window available for the cruise passengers, by taking into account the number of attraction locations available in the city together with dynamic parameters as their reception capacity and the time needed to go from one attraction to another.

One of the most common objective for an Intelligent Environment (IE), especially in case of applications of Ambient Assisted Living, is to support a user in everyday tasks. A complex task can be effectively represented as a workflow; i.e., a composition of simpler activities. This research activity aims at developing methodologies and tools able to make a robotic system learns a task from demonstrations in an IE. The large number of accessible sensors in the intelligent environment, as well as the availability of advanced services such as those for activity recognition and situation awareness, can facilitate either the recognition of the action to be executed at the ithstep of the workflow and the verification of the correct execution of the task. In this paper, we present an hybrid approach based on Inverse Reinforcement Learning (IRL) to learn from observations of the behavior of an expert and on forward Reinforcement Learning to correct and improve the learned behavior. We also present the high-level architecture of the IE that supports the learning process.

Huntington disease (HD) is a progressive disorder of motor, cognitive, and psychiatric disturbances. A general lack of coordination and an unsteady gait often follow motor speed, fine motor control, and gait are affected. Gait disturbance is one of the main factors contributing to a negative impact on quality of life of patients. The state-of-the-art of assessment approaches for the evaluation and recognition of this type of disease are expensive ambulation-based performed under the supervision of clinicians. Our research aim at overcoming these issues by defining an in-house self-test mobile solution able to detect anomalies in the gait dynamics of elderly. In this paper, we present the preliminary results of our research exploring a deep learning-based model for the automatic assessment of the gaits dynamics of elderly people. The gait dynamics signal is measured by means of a temporal time series of the acceleration values of the patient's acceleration movements along the (x,y,z) axes. Our experiments show classification results reaching a good accuracy rate at 0.75% with a recall an precision rate at 0.70% and 0.75%.

In recent decades the aging of the population has led to a change in the health held with special attention to the issue of home care and e-health. The aim is to provide different types of health care services to the patient's home rather than in hospital so improving the quality of life of patients by allowing them to stay in their own home environment. The healthcare sector is undergoing profound transformation thanks to the possibilities offered by the Internet of Things and new technologies, the mobile and wearable. The new model is oriented to the overall health of the patient, stimulated and implemented through a strong pro activities of the patient and realized using mobile devices and multi-channel technology. The idea is to introduce a set of functionalities in order to support the family or the patient, to act independently or to support existing treatments coaching systems. With this aim the paper proposes an architecture, based on a cognitive model, to realize a coaching system for daily living activities which relies on solving a problem as an human so modeling human cognition. The idea is to handle the process of acquiring skills as a transition from use of declarative knowledge of procedures that can be applied quickly and automatically in specific situations.

PeopleN EURODEGENERATIVE diseases are progressive and difficult to diagnose in their early stages.Progressive impairment in the activities of daily living, as well as cognitive deterioration, leads to an increase in patient's dependence.Neuropsychiatric symptoms are common features and include psychosis (delusions and hallucinations), depressive mood, anxiety, irritability/lability, apathy, euphoria, disinhibition, agitation/aggression, aberrant motor activities, sleep disturbance, and eating disorders.Additionally, patients have a tendency to wander and misplace objects in their environment.Therefore, cognitive impaired patients need constant monitoring to ensure their wellbeing, and that they do not harm themselves.Ambient intelligence (AmI) is the computing paradigm that can provide support both for cognitive impaired people within their domestic environments and for clinicians to collect data needed to assess the disease.A novel class of applications can potentially emerge from the specialization of AmI technologies to the special domain of cognitive impaired users.However, such applications must be able to deal with peculiarities due to irrational and abnormal behaviors of cognitive impaired people within their environments, and their general attitude to not accepting wearable technologies.Several technical challenges remain to be solved before effective, intelligent, secure, and reliable AmI applications for the support of cognitive impaired people can be deployed trustfully into real environments.In this special section, the paper "The role of Virtual Motor Rehabilitation.A quantitative analysis between Acute and Chronic Patients with acquired brain injury," concerning acquired brain injury, compares the evolution of chronic patients with acute patients in a virtual rehabilitation program.For this study authors have developed a "Vestibular Virtual Rehabilitation" system.Results have shown a similar recovery for chronic and acute patients during the period of intervention.It has also been shown that chronic patients stop their improvement when they finish the training encouraging the migration of such rehabilitation technologies to home.In the contribution "An Adaptable and Flexible Framework for Assistive Living of Cognitively Impaired People" describes guidelines and an adaptable framework for the dynamic integration of assistive services with their related sensing technologies and interaction devices.The paper on "Statistical Anomaly Detection for Individuals with Cognitive Impairments" focuses on the identification of anomalies for users' routes, which may be useful to assist the impaired in case of losing one's way or to alert caregivers in case of wandering.The proposed approach relies on the exploitation Digital Object

Ubiquitous and pervasive applications may present critical requirements from the point of view of functional correctness, reliability, availability, security, and safety. Unlike traditional safety-critical applications, the behavior of ubiquitous and pervasive applications is affected by the movements and location of users and resources. In this article, we first present emerging formal methods for the description of both entities and their behavior in pervasive computing environments; then, we introduce this special issue. Despite many previous works that have focused on modeling the entities, relatively few have concentrated on modeling or verifying behaviors; and almost none has dealt with combining techniques proposed in these two aspects. The articles accepted in this special issue cover some of the topics aforementioned and constitute a representative sample of the latest development of formal methods in pervasive computing environments.

Ambient Intelligence technologies have not yet been widely adopted in safety critical scenarios. This principally has been due to fact that acceptable degrees of dependability have not been reached for the applications that rely on such technologies. However, the new critical application domains, like Ambient Assisted Living and Smart Hospitals, which are currently emerging, are increasing the need for methodologies and tools that can improve the reliability of the final systems. This paper presents a middleware architecture for safety critical Ambient Intelligence applications which provides the developer with services for runtime verification. It is now possible to continuously monitor and check the running system against correctness properties defined at design time. Moreover, a visual tool which allows the formal design of several of the characteristics of an Ambient Intelligence application and the automatic generation of setting up parameters and code for the middleware infrastructure is also presented.

This article presents a methodology to formally express requirements in safety-critical ubiquitous and pervasive applications in order to achieve a higher degree of dependability. In particular, it will be shown how it is possible to formalize and constrict mobility characteristics by combining and extending several formal methods. The article also discusses some issues concerning both static and dynamic verification.

Technological advances in wireless networks and mobile devices are driving towards a new concept of grid computing, in which mobile users play an active role. For this reason, classical wired grid infrastructures are being extended in order to provide facilities which enable mobile users to exploit services available in the environment. Since grid computing borrows such facilities from pervasive computing, the new computing paradigm is named pervasive grid computing. In this paper, we identify a number of characteristics, like context-awareness and handling of mobile sessions, that all pervasive grid environments should provide. Accordingly, we define an architectural model based on a set of services. A running implementation of such an architecture and an application scenario are presented as well.

Wireless and pervasive healthcare applications typically present critical requirements from the point of view of functional correctness, reliability, availability, security, and safety. In contrast to the case of classic safety critical applications, the behavior of wireless and pervasive applications is affected by the movements and location of users and resources. This article presents a methodology to formally express requirements in safety critical wireless and pervasive healthcare applications in order to achieve a higher degree of dependability. In particular, it will be shown how it is possible to formalize and constrict mobility characteristics by combining, and in some cases extending, several formal methods. The article also describes a rigorous specification process. Finally, it concludes with a case study of a real safety critical pervasive healthcare application that is going to be deployed in a city hospital.

Wireless and pervasive applications typically present critical requirements from the point of view of functional correctness, reliability, availability, security and safety. In contrast to the case of classic safety critical applications, the behavior of such applications is affected by the movements and location of users and resources. This paper presents some formal tools that enable designers to specify the software requirements of a pervasive application in an unambiguous and verifiable way. A real safety critical pervasive application for the department of nuclear medicine of a city hospital is also presented as a case study.

Immersive Virtual Reality environments are suitable to support activities related to medicine and medical practice. The immersive visualization of information-rich 3D objects, coming from patient scanned data, provides clinicians with a clear perception of depth and shapes. However, to benefit from immersive visualization in medical imaging, where inspection and manipulation of volumetric data are fundamental tasks, medical experts have to be able to act in the virtual environment by exploiting their real life abilities. In order to reach this goal, it is necessary to take into account user skills and needs so as to design and implement usable and accessible human-computer interaction interfaces. In this paper we present a natural interface for a semi-immersive virtual environment. Such interface is based on an off-the-shelf handheld wireless device and a speech recognition component, and provides clinicians with intuitive interaction modes for inspecting volumetric medical data.

The advancements of wireless and mobile computing technologies and the diffusion of pervasive healthcare technologies are changing our perception of healthcare. In this paper, we describe how the pervasive computing technologies can be used to build a Smart Hospital. In particular, we propose a concrete implementation of a smart hospital and discuss how e-Health services and applications can be enhanced by location information. As a solution, we present semantic models, mechanisms and a system to locate diverse kinds of mobile entities in Smart Hospitals. The key feature of the system is the semantic integration of different positioning systems that not only enables the hospital to transparently handle such physical positioning systems, but also to reason on location information coming from different systems and to combine them in order to get higher context information or to resolve inconsistencies or conflicts due to sensing errors or limitations of the positioning systems.

Smart and intelligent environments can proficiently benefit from a new generation of multimodal dialog systems, which enable users to interact in natural ways like voice, gesture, facial motions, and so on. However, in order to make this possible, it is necessary to integrate such systems and services, possibly in a transparent way. In the present paper, we propose (i) an architectural model for multimodal smart environments; (ii) an agent-based middleware infrastructure that provides basic components for the integration of multimodal interaction systems; and, (iii) a tool for rapid-prototyping, which automatically customizes some middleware components to implement multimodal interaction mechanisms for the environment's application services.

Grid computing and pervasive computing have rapidly emerged and affirmed respectively as the paradigm for high performance computing and the paradigm for user-friendly computing. The conjunction of such paradigms are now generating a new one, the pervasive grid computing, which aims at extending classic grids with characteristics of pervasive computing like spontaneous and transparent integration of mobile devices, context-awareness, pro-activity, and so on. In this paper, we present a software infrastructure for integrating mobile devices as active resources in a grid. In particular, it has been developed a service able to dynamically distribute and execute user's tasks on a grid of mobile and fixed devices. Mobile devices participate to the grid in a way completely transparent to their owners, hence the computational power is given without any configuration operation and in a reliable way. As well, users willing of executing their applications directly submit their code without caring of choosing and allocating resources of the grid

This paper presents some redundancy-based transmission strategies, suitable for MPEG (Moving Picture Experts Group) coded multimedia streams. We focus on version 2 of the standard, which is a major compression scheme for distributed VoD (Video on Demand) systems. In such systems, multimedia data transmitted from a server to a client, is to be delivered and rendered within pre-defined time constraints. Typical values range between 100 to 200 ms. Due to these constraints, developing effective redundancy and transmission mechanisms is not a simple task. The effectiveness of the suggested schemes is evaluated, under varying fault rates and network load conditions. The evaluation is done via simulated fault-injection, i.e. by simulating the behavior of the system while faults are injected to its individual components. In particular, we concentrated on the impact-on the quality of the delivered service-of transient faults in the communication infrastructure.