Advances in camera-based physiological monitoring have enabled the robust, non-contact measurement of res-piration and the cardiac pulse, which are known to be in-dicative of the sleep stage. This has led to research into camera-based sleep monitoring as a promising alternative to “gold-standard” polysomnography, which is cumbersome, expensive to administer, and hence unsuitable for longer-term clinical studies. In this paper, we introduce SleepVST, a transformer model which enables state-of-the-art performance in camera-based sleep stage classification (sleep staging). After pretraining on contact sensor data, SleepVST outperforms existing methods for cardio-respiratory sleep staging on the SHHS and MESA datasets, achieving total Cohen's kappa scores of 0.75 and 0.77 respectively. We then show that SleepVST can be successfully transferred to cardio-respiratory waveforms extracted from video, enabling fully contact-free sleep staging. Using a video dataset of 50 nights, we achieve a total accuracy of 78.8% and a Cohen's κ of 0.71 in four-class video-based sleep staging, setting a new state-of-the-art in the domain.

The papers in this special section focus on camera-based monitoring for pervasive healthcare informatics. Measuring physiological signals from the human face and body using video cameras is an emerging research topic that has grown rapidly in the last decade. Remote cameras (in both visible and infrared wavelengths) can be used to measure vital signs from a human body based on skin optics or body movements thereby avoiding mechanical contact with the skin. Camera-based health monitoring will bring a rich set of compelling healthcare applications that directly improve upon contact-based monitoring solutions and impact people’s care experience and quality of life in various scenarios, such as in hospital care units, sleep/senior centers, assisted-living homes, telemedicine and e-health, baby/elderly care at home, fitness and sports, driver monitoring in automotive applications, cardiac/ respiratory gating for MRI/CT, AR/VR based therapy and clinical training, e

Collecting data from smartphones can provide useful information for health research, but developing mobile health (mHealth) apps requires extensive resources and implies often overlooked regulatory and privacy-related requirements. Mobile health "aggregators" like HealthKit or Google Fit and specialised platforms may allow reducing costs in mHealth research. Current platforms, however, do not efficiently exploit mHealth aggregators and provide little attention to regulations. We propose an open-source platform that can be used by health researchers without the need to develop custom apps. The platform has a strong focus on regulatory compliance, patient consent and transparency, and allows collecting data through electronic surveys and querying aggregators. We tested the feasibility of our platform in a pilot study with 18 healthy volunteers. The results show that the participants' app is usable and well-accepted and is able to frequently collect data about physical activity from both phones and connected wearables. Some limitations were identified regarding data loss because of insufficient connectivity and the impossibility of extracting data from wearables that are not compatible with aggregators.

Knowledge of the pathological instabilities in the breathing pattern can provide valuable insights into the cardiorespiratory status of the critically-ill infant as well as their maturation level. This paper is concerned with the measurement of respiratory rate in premature infants. We compare the rates estimated from the chest impedance pneumogram, the ECG-derived respiratory rhythms, and the PPG-derived respiratory rhythms against those measured in the reference standard of breath detection provided by attending clinical staff during 165 manual breath counts. We demonstrate that accurate RR estimates can be produced from all sources for RR in the 40-80 bpm (breaths per min) range. We also conclude that the use of indirect methods based on the ECG or the PPG poses a fundamental challenge in this population due to their poor behavior at fast breathing rates (upward of 80 bpm).

A visualisation method for automatically clustering heart beats in single channel electrocardiography (ECG) signals was developed and applied to a dataset of post-intensive care patients who received long-term continuous monitoring. We first segmented the ECG signal into individual beats using an R-peak detection algorithm. A matrix was constructed by storing the segmented ECG beats in row-wise format. Singular value decomposition (SVD) was applied to remove sparse invalid detected R peaks, thus smoothing the matrix. Treating the matrix of ECG beat values as an image, an edge detection algorithm was applied, resulting in a binary matrix containing traces of heart beats with the contiguous and discontiguous components extracted. We considered each component to be a cluster of heart beats. This method was robust to signal noise by exploiting detected R peaks and ECG raw cycles represented in a matrix format for estimation of heart beats. The algorithm also eliminated the effect of underestimated R peaks in the estimation of heart beats, and minimised the effects of overestimated R peaks using the SVD algorithm. This method allows clusters of beats to be visualized, which may assist clinicians in estimating the components of long-term ECG signals.

We describe an evaluation of photoplethysmography (PPG) signals with two wavelengths channels (infrared and red) using a wrist-worn sensor for the estimation of heart rate variability (HRV) and oxygen saturation (SpO2). Five healthy subjects were equipped with a commercial wrist-worn pulse oximeter (Wavelet Health, USA) on the right hand, and both a commercial smart watch for fitness use (Huawei Watch, Series 2) and a clinically-validated transmission-mode pulse oximeter (Creative Medical PC-68B) on their left hand as a reference. Synchronised PPG signals from the Wavelet Health, the Huawei watch, and the PC-68B were recorded for approximately 10 minutes. Subjects were asked to leave the left hand in a resting state, while moving the right hand with two types of movement (periodic and random). A method is proposed to incorporate coupling information between the two wavelengths of PPGs based on the bivariate empirical mode decomposition algorithm. Our method is shown to improve the quality of red PPG allowing improved estimation of SpO2. A comparison of average heart rate (HR), HRV, and SpO2 from all devices is provided.

Patient detection and skin segmentation are important steps in non-contact vital sign monitoring as skin regions contain pulsatile information required for the estimation of vital signs such as heart rate, respiratory rate and peripheral oxygen saturation (SpO2). Previous methods based on face detection or colour-based image segmentation are less reliable in a hospital setting. In this paper, we develop a multi-task convolutional neural network (CNN) for detecting the presence of a patient and segmenting the patient's skin regions. The multi-task model has a shared core network with two branches: a segmentation branch which was implemented using a fully convolutional network, and a classification branch which was implemented using global average pooling. The whole network was trained using images from a clinical study conducted in the neonatal intensive care unit (NICU) of the John Radcliffe hospital, Oxford, UK. Our model can produce accurate results and is robust to changes in different skin tones, pose variations, lighting variations, and routine interaction of clinical staff.

An abnormal respiratory rhythm is an early indicator of physiological deterioration. It is of critical importance in the clinical management of critically-ill or premature infants, for whom apnoea of prematurity is a major concern. Nevertheless, respiratory signals are still largely disregarded in neonatal intensive care units due to the high prevalence of noise and high false alarm rates in conventional monitoring. To address this, we present a novel method for the extraction of respiration from camera-based measurements taken from the top-view of an incubator. A total of 107 events from 30 neonatal admissions were annotated by three clinical reviewers as either true cessations of breathing (physiologically relevant) or false (artefact-related). The events were divided into two independent groups for training and validation and our algorithm was trained to classify true cessations. We achieved a good classification performance with 9 out of 10 cessations and 7 out of 10 artefactual events correctly identified in the training set, and with 7 out of 10 cessations and 34 out of 44 artefactual events correctly identified in the out-of-sample test set. A reduction in false alarm rate of 77.3% was achieved.

Current monitoring systems available to track changes in the vital signs of patients (such as heart rate, respiratory rate or peripheral oxygen saturation) require contact with the subject. Most patients requiring regular monitoring find the probes difficult to wear over prolonged periods of time. Research in non-contact vital sign monitoring has recently expanded through the use of off-the-shelf video cameras; nevertheless, most of the current work in video-based non-contact vital sign monitoring has so far been performed over short time periods (typically up to a couple of minutes), under tightly controlled conditions with relatively still and healthy volunteer subjects. Using an off-the-shelf camera, we have been able to compute estimates of heart rate and respiratory rate, and also detect changes in peripheral oxygen saturation in a real hospital scenario, without interfering with regular patient care. Videos were recorded for 369.1 hours from 40 patients undergoing haemodialysis treatment in the Renal Unit of the Churchill Hospital in Oxford, UK. The mean absolute error between the heart rate estimates from the camera and the average from two reference pulse oximeters (positioned at the finger and earlobe respectively) was 2.8 beats per minute for over 65% of the time, which was comparable to the error between the two reference pulse oximeters. The mean absolute error between the respiratory rate estimates from the camera and the reference values (computed from the Electrocardiogram and a thoracic expansion sensor - chest belt) was 2.1 breaths per minute for over 69% of the time for which the reference signals were valid. By calibrating the camera data with the reference pulse oximeters, changes in peripheral oxygen saturation could also be tracked during time periods with minimal patient motion.

The current institution-based model for healthcare service delivery faces enormous challenges posed by an aging population and the prevalence of chronic diseases. For this reason, pervasive healthcare, i.e., the provision of healthcare services to individuals anytime anywhere, has become a major focus for the research community. In this paper, we map out the current state of pervasive healthcare research by presenting an overview of three emerging areas in personalized health monitoring, namely: 1) mobile phone sensing via in-built or external sensors, 2) self-reporting for manually captured health information, such as symptoms and behaviors, and 3) social sharing of health information within the individual's community. Systems deployed in a real-life setting as well as proofs-of-concept for achieving pervasive health are presented, in order to identify shortcomings and increase our understanding of the requirements for the next generation of pervasive healthcare systems addressing these three areas.

The identification of invalid data in recordings obtained using wearable sensors is of particular importance since data obtained from mobile patients is, in general, noisier than data obtained from nonmobile patients. In this paper, we present a signal quality index (SQI), which is intended to assess whether reliable heart rates (HRs) can be obtained from electrocardiogram (ECG) and photoplethysmogram (PPG) signals collected using wearable sensors. The algorithms were validated on manually labeled data. Sensitivities and specificities of 94% and 97% were achieved for the ECG and 91% and 95% for the PPG. Additionally, we propose two applications of the SQI. First, we demonstrate that, by using the SQI as a trigger for a power-saving strategy, it is possible to reduce the recording time by up to 94% for the ECG and 93% for the PPG with only minimal loss of valid vital-sign data. Second, we demonstrate how an SQI can be used to reduce the error in the estimation of respiratory rate (RR) from the PPG. The performance of the two applications was assessed on data collected from a clinical study on hospital patients who were able to walk unassisted.

There is potential for the use of mobile phones to remotely identify patients with a high risk of heart conditions using automated auscultation. However, accurate heart sound analysis is dependent on the quality of heart sound recordings. This paper investigates the signal quality classification of phonocardiograms (PCGs) recorded on two devices (a 3M Littmann 3200 electronic stethoscope and an iPhone 3G). These recordings were professionally annotated and classified using a support vector machine (SVM) and a combination of ten signal quality metrics computed from each recording as input features. One third of all mobile phone-recorded PCGs were found to be of high quality. The classifier was able to distinguish good and bad-quality iPhone recordings with 87.0% accuracy, the Littmann recordings with accuracy of 76.4% and the combined set with accuracy of 85.6% on unseen test data. Therefore, the quality of PCGs made with a range of stethoscopes can be accurately classified using this technique.

The majority of patients in the hospital are ambulatory and would benefit significantly from predictive and personalized monitoring systems. Such patients are well suited to having their physiological condition monitored using low-power, minimally intrusive wearable sensors. Despite data-collection systems now being manufactured commercially, allowing physiological data to be acquired from mobile patients, little work has been undertaken on the use of the resultant data in a principled manner for robust patient care, including predictive monitoring. Most current devices generate so many false-positive alerts that devices cannot be used for routine clinical practice. This paper explores principled machine learning approaches to interpreting large quantities of continuously acquired, multivariate physiological data, using wearable patient monitors, where the goal is to provide early warning of serious physiological determination, such that a degree of predictive care may be provided. We adopt a one-class support vector machine formulation, proposing a formulation for determining the free parameters of the model using partial area under the ROC curve, a method arising from the unique requirements of performing online analysis with data from patient-worn sensors. There are few clinical evaluations of machine learning techniques in the literature, so we present results from a study at the Oxford University Hospitals NHS Trust devised to investigate the large-scale clinical use of patient-worn sensors for predictive monitoring in a ward with a high incidence of patient mortality. We show that our system can combine routine manual observations made by clinical staff with the continuous data acquired from wearable sensors. Practical considerations and recommendations based on our experiences of this clinical study are discussed, in the context of a framework for personalized monitoring.

Novelty detection, or one-class classification, is of particular use in the analysis of high-integrity systems, in which examples of failure are rare in comparison with the number of examples of stable behaviour, such that a conventional multi-class classification approach cannot be taken. Support Vector Machines (SVMs) are a popular means of performing novelty detection, and it is conventional practice to use a train-validate-test approach, often involving cross-validation, to train the one-class SVM, and then select appropriate values for its parameters. An alternative method, used with multi-class SVMs, is to calibrate the SVM output into conditional class probabilities. A probabilistic approach offers many advantages over the conventional method, including the facility to select automatically a probabilistic novelty threshold. The contributions of this paper are (i) the development of a probabilistic calibration technique for one-class SVMs, such that on-line novelty detection may be performed in a probabilistic manner; and (ii) the demonstration of the advantages of the proposed method (in comparison to the conventional one-class SVM methodology) using case studies, in which one-class probabilistic SVMs are used to perform condition monitoring of a high-integrity industrial combustion plant, and in detecting deterioration in patient physiological condition during patient vital-sign monitoring.

This paper details the hardware and software infrastructure used to collect data and deliver a novel intervention in an on-going individually randomised three-arm parallel group trial in a resource-limited setting. The SMS-text Adherence support trial (StAR) tests the efficacy of a behavioural intervention delivered by SMS-text to support hypertension treatment adherence compared to usual care. The intervention is a structured program of clinic appointment and medication pick-up reminders, medication adherence support and hypertension-related education delivered remotely using an automated system of semi-tailored informational or interactive SMS-text messages delivered to the 1372 recruited participants. The technical infrastructure includes mobile device-based electronic data collection at the point of care with secure remote upload of the data; the intervention delivery system; real-time query identification for screening, enrolment and participant management procedures; and monitoring and evaluation of trial processes. We are using several open-source software platforms in conjunction with off-the-shelf hardware to set-up and run a randomised clinical trial of a novel intervention in a low-resource setting.

We consider an integrated patient monitoring system, combining electronic patient records with high-rate acquisition of patient physiological data. There remain many challenges in increasing the robustness of "e-health" applications to a level at which they are clinically useful, particularly in the use of automated algorithms used to detect and cope with artifact in data contained within the electronic patient record, and in analyzing and communicating the resultant data for reporting to clinicians. There is a consequential "plague of pilots," in which engineering prototype systems do not enter into clinical use. This paper describes an approach in which, for the first time, the Emergency Department (ED) of a major research hospital has adopted such systems for use during a large clinical trial. We describe the disadvantages of existing evaluation metrics when applied to such large trials, and propose a solution suitable for large-scale validation. We demonstrate that machine learning technologies embedded within healthcare information systems can provide clinical benefit, with the potential to improve patient outcomes in the busy environment of a major ED and other high-dependence areas of patient care.

In this paper we present a method of assessing the reliability of heart rates (HRs) obtained from ambulatory ECGs. Our method assigns a Reliability Index (RI) to ECG segments based on a set of physiologically relevant rules prior to using a template matching approach. We validated the algorithm on 1500 manually annotated samples of ECG taken from two different studies and using three different sensors at different sampling rates. The sensitivity of our method was 98% and the specificity was 94%. Our method matched or was more conservative than the human annotations in 99.4% of the samples, making it a promising tool for inclusion in next-generation wearable sensors.

Vital-sign monitoring of patients within a hospital setting is a big component in the recognition and treatment of early signs of deterioration. Current vital-sign monitoring systems, including both manual early warning systems, and more sophisticated data fusion systems, typically make use of the most recently recorded data, and are unable to deal with missing data in a principled manner. The latter is particularly pertinent in the field of ambulatory monitoring, in which patient movement can result in sensor disconnections and other artefact. This paper presents a Gaussian process regression technique for estimating missing data and how it can be incorporated within an automated data fusion monitoring system. The technique is then demonstrated using vital-sign data from a recent clinical study conducted at the John Radcliffe Hospital, Oxford, showing an improvement over an existing data fusion algorithm by providing both an estimate of missing vital sign data and the uncertainty in the estimated value.

If wearable sensors are to play a significant role in monitoring the vital signs of hospitalised patients they need to be accepted by doctors and other healthcare workers. To gain this acceptance, evidence of their effectiveness needs to be demonstrated in clinical trials. In this pragmatic feasibility study four commercially-available, CE-marked sensors were combined into three monitoring systems and used to record the electrocardiograms (ECGs) and photoplethysmograms (PPGs) of 31 hospitalised patients, to determine whether the sensors could collect vital sign data reliably enough for use in larger clinical trials. Patients were asked to wear the sensors for 24 hours. Out of the 31 studies, on only 3 occasions did any of the monitoring systems manage to record both ECG and PPG data for the full 24-hour duration. The causes for the failure of sensors to record data from in-hospital patients consistently are discussed and a clinical perspective is given on the design features needed for a sensor to be usable in a hospital setting.

Hospital patient outcomes can be improved by the early identification of physiological deterioration. Automatic methods of detecting patient deterioration in vital-sign data typically attempt to identify deviations from assumed normal physiological conditions, which is a one-class approach to classification. This paper investigates the use of a two-class approach, in which abnormal physiology is modelled explicitly. The success of such a method relies on the accuracy of data labels provided by clinical experts, which may be incomplete (due to large dataset size) or imprecise (due to clinical labels covering intervals, rather than each data point within those intervals). We propose a novel method of refining clinical labels such that the two-class classification approach may be adopted for identifying patient deterioration. We demonstrate the effectiveness of the proposed methods using a large dataset acquired in a 24-bed hospital step-down unit.

This article is concerned with the analysis of the mastoid electroencephalogram (EEG) using parametric modeling and neural network techniques to assess the vigilance of an individual. One possible application of this work would be the design of a monitoring system for tracking the transitions within the vigilance continuum. The strategy presented consists of training neural networks with spectral features extracted from the mastoid EEG. The results are validated against the expert scoring of the vigilance level performed by visual inspection of the central EEG, electrooculographic (EOG), and electromyography (EMG) signals, and against the results obtained by training similar networks with information extracted from the central EEG (widely recognized as providing useful information for vigilance level assessment). To improve the performance of the neural networks, a Kohonen map-based technique for filtering the training data is proposed (this allows labels assigned by an expert to 15-sec epochs to be transcripted reliably to 1-sec segments). The results presented demonstrate conclusively that the tracking of fluctuations from alertness to drowsiness within the vigilance continuum is possible by neural network analysis of a single channel of EEG recorded from the mastoid site.

We examine the use of hidden Markov and hidden semi-Markov mod- els for automatically segmenting an electrocardiogram waveform into its constituent waveform features. An undecimated wavelet transform is used to generate an overcomplete representation of the signal that is more appropriate for subsequent modelling. We show that the state dura- tions implicit in a standard hidden Markov model are ill-suited to those of real ECG features, and we investigate the use of hidden semi-Markov models for improved state duration modelling.

A system has been developed to extract diagnostic information from jet engine carcass vibration data. Support Vector Machines applied to nov(cid:173) elty detection provide a measure of how unusual the shape of a vibra(cid:173) tion signature is, by learning a representation of normality. We describe a novel method for Support Vector Machines of including information from a second class for novelty detection and give results from the appli(cid:173) cation to Jet Engine vibration analysis.

Neural networks are being increasingly used for problems involving function approximation. However, a key limitation of neural methods is the lack of a measure of how much confidence can be placed in output estimates. In the last few years many authors have addressed this shortcoming from various angles, focusing primarily on predicting output bounds as a function of the trained network's characteristics, typically as defined by the Hessian matrix. In this paper the problem of the effect of errors or noise in the presented, input, vector is examined and a method based on perturbation analysis of determining output bounds based on both the error in the input vector and the imperfections in the weight values after training is presented and demonstrated.

Autonomous vehicles could benefit greatly from visual-motion sensors of sufficiently low cost that 10 or 20 of them could be distributed around the vehicle's periphery. A design is outlined here that builds on well-known schemes using gratings. The grating principle is illustrated by the fact that a man with a torch, walking at night behind a railing, seems to flash. The frequency of flashing is proportional to his velocity. A major drawback is that backward and forward motion are not distinguished. The key development here is the use of commutation as a means of modulating the grating output signal. This is equivalent, in the illustration above, to simulating stroboscopic motion of the railing. Thus, when the man is stationary, there is flashing at a resting frequency. When he moves one way the frequency increases, and for motion the opposite way, frequency decreases. Results from an analogue implementation of the visual motion transducer are presented. The current transducer measures translational motion across the grating. The design is also shown to be capable of extension to direct measurement of the divergence and curl of the flow field.>

The detection of novel or abnormal input vectors is of importance in many monitoring tasks, such as fault detection in complex systems and detection of abnormal patterns in medical diagnostics. We have developed a robust method for novelty detection, which aims to minimize the number of heuristically chosen thresholds in the novelty decision process. We achieve this by growing a gaussian mixture model to form a representation of a training set of “normal” system states. When previously unseen data are to be screened for novelty we use the same threshold as was used during training to define a novelty decision boundary. We show on a sample problem of medical signal processing that this method is capable of providing robust novelty decision boundaries and apply the technique to the detection of epileptic seizures within a data record.

A design that builds on well-known schemes using gratings is outlined. The key development here is the use of commutation as a means of modulating the grating output signal. This enables backwards and forwards motion to be distinguished. Results from an analog implementation of the visual motion transducer are presented. The current transducer measures translational motion across the grating. The design is also shown to be capable of extension to direct measurement of the divergence and curl of the flow field.>

Results from simulations of weight perturbation as an on-chip learning scheme for analogue VLSI neural networks are presented. The limitations of analogue hardware are modelled as realistically as possible. Thus synaptic weight precision is defined according to the smallest change in the weight setting voltage which gives a measurable change at the output of the corresponding neuron. Tests are carried out on a hard classification problem constructed from mobile robot navigation data. The simulations show that the degradation in classification performance on a 500-pattern test set caused by the introduction of realistic hardware constraints is acceptable: with 8-bit weights, updated probabilistically and with a simplified output error criterion, the error rate increases by no more than 7% when compared with weight perturbation implemented with full 32-bit precision.

Neural networks have recently been applied to real-world speech recognition problems with a great deal of success. This paper develops a strategy for optimising a neural network known as the Radial Basis Function classifier (RBF) on a large spoken letter recognition problem designed by British Telecom Research Laboratories. The strategy developed can be viewed as a compromise between a fully adaptive approach involving prohibitively large amounts of computation and a heuristic approach resulting in poor generalisation. A value for the optimal number of kernel functions is suggested and methods for determining the positions of the centres and the values of the kernel function widths are provided. During the evolution of the optimisation strategy, it was demonstrated that spatial organisation of the centres does not adversely affect the ability of the classifier to generalise. An RBF employing the optimisation strategy achieved a lower error rate than Woodland’s multilayer perceptron26 and two traditional static pattern classifiers on the same problem. The error rate of the RBF was very close to the estimated minimum error rate obtainable with an optimal Bayesian classifier. An examination of the computational requirements of the classifiers illustrated a significant trade-off between the computational investment in training and level of generalisation achieved.

Results from working analog VLSI implementations of two different pulse stream neural network forms are reported. The circuits are rendered relatively invariant to processing variations, and the problem of cascadability of synapses to form large systems is addressed. A strategy for interchip communication of large numbers of neural states has been implemented in silicon and results are presented. The circuits demonstrated confront many of the issues that blight massively parallel analog systems, and offer solutions.

A method for robot path planning that uses a 2D scalar electric potential subject to Neumann boundary conditions is presented. Obstacles are modeled as nonconducting solids in a conducting medium. The starting point is modeled as a current source and the goal as an equal and opposite current sink. It is shown that this formulation is considerably more powerful than the recent potential-field algorithm of C.I. Connolly et al. (1990), particularly when navigating long, narrow corridors. Feasible paths for navigation are current streamlines, as demonstrated by the results of software simulations in a 2D Euclidean plane. One of the principal advantages of the method is that it can be implemented with parallel analog hardware in the form of a resistive grid. With analog VLSI chips, it will be possible to plan paths for realistic environments in real time.>

The pulse-stream technique, which represents neural states as sequences of pulses, is reviewed. Several general issues are raised, and generic methods appraised, for pulsed encoding, arithmetic, and intercommunication schemes. Two contrasting synapse designs are presented and compared. The first is based on a fully analog computational form in which the only digital component is the signaling mechanism itself-asynchronous, pulse-rate encoded digital voltage pulses. In this circuit, multiplication occurs in the voltage/current domain. The second design uses more conventional digital memory for weight storage, with synapse circuits based on pulse stretching. Integrated circuits implementing up to 15000 analog, fully programmable synaptic connections are described. A demonstrator project is described in which a small robot localization network is implemented using asynchronous, analog, pulse-stream devices.