Consensus mechanisms are the core of any blockchain system. However, the majority of these mechanisms do not target federated learning directly nor do they aid in the aggregation step. This paper introduces Proof of Reasoning (PoR), a novel consensus mechanism specifically designed for federated learning using blockchain, aimed at preserving data privacy, defending against malicious attacks, and enhancing the validation of participating networks. Unlike generic blockchain consensus mechanisms commonly found in the literature, PoR integrates three distinct processes tailored for federated learning. Firstly, a masked autoencoder (MAE) is trained to generate an encoder that functions as a feature map and obfuscates input data, rendering it resistant to human reconstruction and model inversion attacks. Secondly, a downstream classifier is trained at the edge, receiving input from the trained encoder. The downstream network’s weights, a single encoded datapoint, the network’s output and the ground truth are then added to a block for federated aggregation. Lastly, this data facilitates the aggregation of all participating networks, enabling more complex and verifiable aggregation methods than previously possible. This three-stage process results in more robust networks with significantly reduced computational complexity, maintaining high accuracy by training only the downstream classifier at the edge. PoR scales to large IoT networks with low latency and storage growth, and adapts to evolving data, regulations, and network conditions.

In recent years, the area of Robot-Assisted Minimally Invasive Surgery (RAMIS) is standing on the the verge of a new wave of innovations. However, autonomy in RAMIS is still in a primitive stage. Therefore, most surgeries still require manual control of the endoscope and the robotic instruments, resulting in surgeons needing to switch attention between performing surgical procedures and moving endoscope camera. Automation may reduce the complexity of surgical operations and consequently reduce the cognitive load on the surgeon while speeding up the surgical process. In this paper, a hybrid robotic endoscope control system based on fusion model of natural language processing (NLP) and modified YOLO-V8 vision model is proposed. This proposed system can analyze the current surgical workflow and generate logs to summarize the procedure for teaching and providing feedback to junior surgeons. The user study of this system indicated a significant reduction of the number of clutching actions and mean task time, which effectively enhanced the surgical training.

Effective early screenings for gastrointestinal diseases are crucial for reducing mortality through timely interventions and improving life expectancy. In this paper, a strain effect-based biomimetic artificial whisker system is proposed to extract the structural and textural information of the tissues in the lumen based on interactive tactile perception data and an end-to-end screening algorithm. Benchmark experiment of the proposed method and an ex-vivo pilot study are conducted to characterize the baseline performance and feasibility of detecting several common tissue structures in surgical application scenarios. Our method shows promising results, with a test accuracy of up to 97.27% and a kappa value of 0.9590. This integrated hardware-software, end-to-end solution is promising to become an emerging human-machine interaction paradigm, empowering traditional healthcare applications.

Camera-based passive dietary intake monitoring is able to continuously capture the eating episodes of a subject, recording rich visual information, such as the type and volume of food being consumed, as well as the eating behaviors of the subject. However, there currently is no method that is able to incorporate these visual clues and provide a comprehensive context of dietary intake from passive recording (e.g., is the subject sharing food with others, what food the subject is eating, and how much food is left in the bowl). On the other hand, privacy is a major concern while egocentric wearable cameras are used for capturing. In this article, we propose a privacy-preserved secure solution (i.e., egocentric image captioning) for dietary assessment with passive monitoring, which unifies food recognition, volume estimation, and scene understanding. By converting images into rich text descriptions, nutritionists can assess individual dietary intake based on the captions instead of the original images, reducing the risk of privacy leakage from images. To this end, an egocentric dietary image captioning dataset has been built, which consists of in-the-wild images captured by head-worn and chest-worn cameras in field studies in Ghana. A novel transformer-based architecture is designed to caption egocentric dietary images. Comprehensive experiments have been conducted to evaluate the effectiveness and to justify the design of the proposed architecture for egocentric dietary image captioning. To the best of our knowledge, this is the first work that applies image captioning for dietary intake assessment in real-life settings.

Motor Imagery (MI) Electroencephalography (EEG) is one of the most common Brain-Computer Interface (BCI) paradigms that has been widely used in neural rehabilitation and gaming. Although considerable research efforts have been dedicated to developing MI EEG classification algorithms, they are mostly limited in handling scenarios where the training and testing data are not from the same subject or session. Such poor generalization capability significantly limits the realization of BCI in real-world applications. In this paper, we proposed a novel framework to disentangle the representation of raw EEG data into three components, subject/session-specific, MI-task-specific, and random noises, so that the subject/session-specific feature extends the generalization capability of the system. This is realized by a joint discriminative and generative framework, supported by a series of fundamental training losses and training strategies. We evaluated our framework on three public MI EEG datasets, and detailed experimental results show that our method can achieve superior performance by a large margin compared to current state-of-the-art benchmark algorithms.

Conventional approaches to dietary assessment are primarily grounded in self-reporting methods or structured interviews conducted under the supervision of dietitians. These methods, however, are often subjective, inaccurate, and time-intensive. Although artificial intelligence (AI)-based solutions have been devised to automate the dietary assessment process, prior AI methodologies tackle dietary assessment in a fragmented landscape (e.g., merely recognizing food types or estimating portion size) and encounter challenges in their ability to generalize across a diverse range of food categories, dietary behaviors, and cultural contexts. Recently, the emergence of multimodal foundation models, such as GPT-4V, has exhibited transformative potential across a wide range of tasks in various research domains. These models have demonstrated remarkable generalist intelligence and accuracy, owing to their large-scale pre-training on broad datasets and substantially scaled model size. In this study, we explore the application of GPT-4V powering multimodal ChatGPT for dietary assessment, along with prompt engineering and passive monitoring techniques. We evaluated the proposed pipeline using a self-collected, semi free-living dietary intake dataset, captured through wearable cameras. Our findings reveal that GPT-4V excels in food detection under challenging conditions without any fine-tuning or adaptation using food-specific datasets. By guiding the model with specific language prompts (e.g., African cuisine), it shifts from recognizing common staples like rice and bread to accurately identifying regional dishes like banku and ugali. Another standout feature of GPT-4V is its contextual awareness. GPT-4V can leverage surrounding objects as scale references to deduce the portion sizes of food items, further facilitating the process of dietary assessment.

Monitoring of volatile organic compounds (VOCs) in body fluids (blood, urine, sweat and saliva) or exhaled breath is a recent trend in medical research with the potential to unveil information about internal body processes or metabolic pathways dysregulated due to bacterial infection, tissue cancer, inflammation, and injury. Typically, these low-weight chemical compounds are analyzed from collected samples by expensive methods involving gas chromatography-mass spectrometry, proton-transfer-reaction mass spectrometry or ion mobility-spectrometry, which makes it difficult to translate into wearable technology for day-to-day use by patients. Recently, E-noses have been proposed to sense chemicals and/or odors from gas exchanges taking place in the upper body part, in an attempt to replace the human nose, with different degrees of success. In this paper, we propose a prototype for an E-nose device with sensing modules for VOCs detection by graphene-field effect transistor (GFET) technology, combined with modules for the detection of body temperature, motion and sounds produced by the cardiovascular system. We successfully tested the prototype in the neck region (carotid artery) for monitorization of the latter variables, whereas 12 clinically relevant VOCs were monitored inside a controlled setup for metrics such as the change of graphene’s resistance and spectral noise upon exposure to these vapors. This can then constitute the basis for development of a fully integrated system that directly correlates physiological variables with disease biomarkers sensed from gas exchanges.

Human movement intention recognition is important for human-robot interaction. Existing work based on motor imagery electroencephalogram (EEG) provides a non-invasive and portable solution for intention detection. However, the data-driven methods may suffer from the limited scale and diversity of the training datasets, which result in poor generalization performance on new test subjects. It is practically difficult to directly aggregate data from multiple datasets for training, since they often employ different channels and collected data suffers from significant domain shifts caused by different devices, experiment setup, etc. On the other hand, the inter-subject heterogeneity is also substantial due to individual differences in EEG representations. In this work, we developed two networks to learn from both the shared and the complete channels across datasets, handling inter-subject and inter-dataset heterogeneity respectively. Based on both networks, we further developed an online knowledge co-distillation framework to collaboratively learn from both networks, achieving coherent performance boosts. Experimental results have shown that our proposed method can effectively aggregate knowledge from multiple datasets, demonstrating better generalization in the context of cross-subject validation.

The continuing increase in the incidence and recognition of children's sleep disorders has heightened the demand for automatic pediatric sleep staging. Supervised sleep stage recognition algorithms, however, are often faced with challenges such as limited availability of pediatric sleep physicians and data heterogeneity. Drawing upon two quickly advancing fields, i.e., semi-supervised learning and self-supervised contrastive learning, we propose a multi-task contrastive learning strategy for semi-supervised pediatric sleep stage recognition, abbreviated as MtCLSS. Specifically, signal-adapted transformations are applied to electroencephalogram (EEG) recordings of the full night polysomnogram, which facilitates the network to improve its representation ability through identifying the transformations. We also introduce an extension of contrastive loss function, thus adapting contrastive learning to the semi-supervised setting. In this way, the proposed framework learns not only task-specific features from a small amount of supervised data, but also extracts general features from signal transformations, improving the model robustness. MtCLSS is evaluated on a real-world pediatric sleep dataset with promising performance (0.80 accuracy, 0.78 F1-score and 0.74 kappa). We also examine its generality on a well-known public dataset. The experimental results demonstrate the effectiveness of the MtCLSS framework for EEG based automatic pediatric sleep staging in very limited labeled data scenarios.

Large AI models, or foundation models, are models recently emerging with massive scales both parameter-wise and data-wise, the magnitudes of which can reach beyond billions. Once pretrained, large AI models demonstrate impressive performance in various downstream tasks. A prime example is ChatGPT, whose capability has compelled people's imagination about the far-reaching influence that large AI models can have and their potential to transform different domains of our lives. In health informatics, the advent of large AI models has brought new paradigms for the design of methodologies. The scale of multi-modal data in the biomedical and health domain has been ever-expanding especially since the community embraced the era of deep learning, which provides the ground to develop, validate, and advance large AI models for breakthroughs in health-related areas. This article presents a comprehensive review of large AI models, from background to their applications. We identify seven key sectors in which large AI models are applicable and might have substantial influence, including: 1) bioinformatics; 2) medical diagnosis; 3) medical imaging; 4) medical informatics; 5) medical education; 6) public health; and 7) medical robotics. We examine their challenges, followed by a critical discussion about potential future directions and pitfalls of large AI models in transforming the field of health informatics.

With the development of modern cameras, more physiological signals can be obtained from portable devices like smartphone. Some hemodynamically based non-invasive video processing applications have been applied for blood pressure classification and blood glucose prediction objectives for unobtrusive physiological monitoring at home. However, this approach is still under development with very few publications. In this paper, we propose an end-to-end framework, entitled cocktail causal container, to fuse multiple physiological representations and to reconstruct the correlation between frequency and temporal information during multi-task learning. Cocktail causal container processes hematologic reflex information to classify blood pressure and blood glucose. Since the learning of discriminative features from video physiological representations is quite challenging, we propose a token feature fusion block to fuse the multi-view fine-grained representations to a union discrete frequency space. A causal net is used to analyze the fused higher-order information, so that the framework can be enforced to disentangle the latent factors into the related endogenous association that corresponds to down-stream fusion information to improve the semantic interpretation. Moreover, a pair-wise temporal frequency map is developed to provide valuable insights into extraction of salient photoplethysmograph (PPG) information from fingertip videos obtained by a standard smartphone camera. Extensive comparisons have been implemented for the validation of cocktail causal container using a Clinical dataset and PPG-BP benchmark. The root mean square error of 1.329±0.167 for blood glucose prediction and precision of 0.89±0.03 for blood pressure classification are achieved in Clinical dataset.

Dietary assessment has proven to be effective to evaluate the dietary intake of patients with diabetes and obesity. The traditional approach of accessing the dietary intake is to conduct a 24-hour dietary recall, a structured interview designed to obtain information on food categories and volume consumed by the participants. Due to unconscious biases in this kind of self-reporting approaches, many research studies have explored the use of vision-based approaches to provide accurate and objective assessments. Despite the promising results of food recognition by deep neural networks, there still exist several hurdles in deep learning-based food volume estimation ranging from domain shift between synthetic and raw 3D models, shape completion ambiguity and lack of large-scale paired training dataset. Therefore, this paper proposed an intelligent nutritional assessment approach via weakly-supervised point cloud completion, which aims to close the reality gap in 3D point cloud completion tasks and address the targeted challenges. Then the volume can be easily estimated from the completed representation of the food. Another major merit of our system is that it can be used to estimate the volume of handheld food items without requiring the constraints including placing the food items on a table or next to fiducial markers, which facilitates the implementation on both wearable and handheld cameras. Comprehensive experiments have been carried out on major benchmark datasets and self-constructed volume-annotated dataset respectively, in which the proposed method demonstrates comparable results with several strong fully-supervised baseline methods and shows superior completion ability in handling food volume estimation.

Road surface condition monitoring and bike riding event detection are crucial in densely populated cities for travel efficiency and rider safety. However, most current approaches are either costly, unreliable in different scenarios, or not adaptable in new environments. This article proposes a novel automated approach leveraging widely used shared bikes to intelligently detect road surface conditions and riding events suitable for interactive Internet of Things (IoT) cities. We propose a novel dual stream meta learning approach to solve the reliability problem when bike types for the training and testing are different with a limited set of new samples and the self-adaptive problem when classifying new classes without retraining the model, both via dual stream meta learning. Results demonstrate the feasibility of the proposed IoT-based solution with 98.9% accuracy for road surface conditions and 99.6% accuracy for riding events via the proposed dual stream deep learning method in the conventional scenario. With few samples per class, the proposed method is more reliable than other commonly used approaches in the different-bike scenario (e.g., proposed 92.4% versus random forest 74.6%). In cases of predicting new classes, the algorithm is 95.6% accurate using only one sample per class without explicit training (compared to 78.0% for$K $-nearest neighbor). This article proposes a robust IoT framework for smart cities involving road surface conditions and rider events which could be critical for many applications, including city mapping, shared bike rental maintenance and rider performance, and city maintenance services.

Wearable technology in the form of wristwatches, armbands, or fit monitors has fast widespread lately among technology enthusiasts that are eager for a quick hands-on experience with their own body parameters. Nonetheless, the accuracy, replicability and reproducibility of the measurements collected by these monitors is still highly debatable outside laboratory settings, thus resulting in their nonacceptance as valid medical diagnostic tools. Furthermore, the inability to collect temporally detailed physiological variables like heartrate, pulse plethysmography, skin temperature and galvanic skin response for extended periods of time has also been appointed as a factor contributing to wearables’ nonacceptance within the biomedical research community. Even more so if the monitoring is to be performed in remote places, usually involving prolonged and arduous physical tasks performed by the participant. In this paper, we propose an inexpensive prototype smartwatch for prolonged physiological monitoring in remote environments. Equipped with sensing channels that monitor the aforementioned body variables, the device can also be instructed to operate in an asynchronous recording mode, thereby saving battery life and memory while recording some ambient variables (humidity, temperature, luminescence, and atmospheric pressure) in order to provide descriptive context awareness to the physiological processes taking place inside the human body at the same time.

Hearing aid devices have been around for decades, while most of them focus on sound amplification and SNR improvement. This paper proposes an artificial ear based on the vibrotactile feedback. The speech signal is converted into the vibrotactile devices placed around the subject’s ear through the speech recognition algorithm and pattern coding method. Preliminary experiments on the prototype consisting of six motors which has shown that the recognition accuracy of letters and daily sentences reached 90%. The learning time of interpreting the vibrotactile signals could be less than four times that in real-time conversation, proving the feasibility of the proposed device for real-life application.

Human-robot shared control, which integrates the advantages of both humans and robots, is an effective approach to facilitate efficient surgical operation. Learning from demonstration (LfD) techniques can be used to automate some of the surgical sub tasks for the construction of the shared control mechanism. However, a sufficient amount of data is required for the robot to learn the manoeuvres. Using a surgical simulator to collect data is a less resource-demanding approach. With sim-to-real adaptation, the manoeuvres learned from a simulator can be transferred to a physical robot. To this end, we propose a sim-to-real adaptation method to construct a human-robot shared control framework for robotic surgery. In this paper, a desired trajectory is generated from a simulator using LfD method, while dynamic motion primitives (DMP) is used to transfer the desired trajectory from the simulator to the physical robotic platform. Moreover, a role adaptation mechanism is developed such that the robot can adjust its role according to the surgical operation contexts predicted by a neural network model. The effectiveness of the proposed framework is validated on the da Vinci Research Kit (dVRK). Results of the user studies indicated that with the adaptive human-robot shared control framework, the path length of the remote controller, the total clutching number and the task completion time can be reduced significantly. The proposed method outperformed the traditional manual control via teleoperation.

In endoluminal surgery, the miniature instruments shall be of high accuracy and flexibility for minimal invasive diagnosis and surgical intervention. To this end, continuum robots with flexible joints have been proposed as the mechanism of endoscopic instruments. The compliance and deformability of the continuum joints enable access into the curved lumen. However, the manufacturing tolerances are normally not considered in the design procedure, and led to inaccuracy in the robotic control. To improve the control accuracy and flexibility of endoluminal surgical robots, we propose a novel design of a metal printed continuum joint in this paper, which incorporates a variable pitch design into the spring-like structure. The design can reduce the position errors accumulated on the distal tip of the joint, especially at large bending angles. The specification of variable pitch is investigated and determined with a friction model. In addition, to eliminate the distortion of the joint induced during the metal printing process, an extensive experiment was conducted to access the effect of the variables in the design (pitch, thickness, width and number of coils), with the aim of determining optimal parameters for reducing discrepancy caused by manufacturing variations. The final results indicated that the bending error of a single joint can be reduced from 18.10% to 4.63%, and a multi-segment prototype was developed to verify its effectiveness for potential surgical applications.

Sleep posture, which affects the quality of sleep and could lead to medical conditions, such as pressure ulcers, is a key metric for sleep analysis in Internet of Medical Things (IoMT). In this article, a real-time and low-cost smart mat system for sleep posture recognition based on frequency channel selection is proposed. The system can recognize postures unobtrusively with a dense flexible sensor array. In addition, to enable real-time recognition with a relatively low-cost STM32 processor system, a lightweight algorithm that includes frequency channel selection, model pretraining, and real-time classification is proposed. Through a series of short-term and overnight experiments with 21 subjects, the feasibility and reliability of the proposed system were evaluated. Experimental results show that the accuracy of the short-term experiment is up to 95.43% and of the overnight experiment is up to 86.80% for four posture categories (supine, prone, right, and left) classification. The model size is just 56 kB which is much smaller than other methods. The runtime of the complete algorithm is about 6 ms with a low-power STM32 embedded system, which shows the system’s ability to provide real-time posture recognition. As an edge device, the proposed system could lead to the development of fast, convenient, and low-cost sleep posture recognition products for IoMT.

Device authentication, encryption, and key distribution are of vital importance to any Internet of Things (IoT) systems, such as the new smart city infrastructures. This is due to the concern that attackers could easily exploit the lack of strong security in IoT devices to gain unauthorized access to the system or to hijack IoT devices to perform denial-of-service attacks on other networks. With the rise of fog and edge computing in IoT systems, increasing numbers of IoT devices have been equipped with computing capabilities to perform data analysis with deep learning technologies. Deep learning on edge devices can be deployed in numerous applications, such as local cardiac arrhythmia detection on a smart sensing patch, but it is rarely applied to device authentication and wireless communication encryption. In this article, we propose a novel lightweight IoT device authentication, encryption, and key distribution approach using neural cryptosystems and binary latent space. The neural cryptosystems adopt three types of end-to-end encryption schemes: 1) symmetric; 2) public-key; and 3) without keys. A series of experiments was conducted to test the performance and security strength of the proposed neural cryptosystems. The experimental results demonstrate the potential of this novel approach as a promising security and privacy solution for the next-generation of IoT systems.

Accurate lower-limb pose estimation is aprerequisite of skeleton based pathological gait analysis. To achieve this goal in free-living environments for long-term monitoring, single depth sensor has been proposed in research. However, the depth map acquired from a single viewpoint encodes only partial geometric information of the lower limbs and exhibits large variations across different viewpoints. Existing off-the-shelf 3D pose tracking algorithms and public datasets for depth based human pose estimation are mainly targeted at activity recognition applications. They are relatively insensitive to skeleton estimation accuracy, especially at the foot segments. Furthermore, acquiring ground truth skeleton data for detailed biomechanics analysis also requires considerable efforts. To address these issues, we propose a novel cross-domain self-supervised complete geometric representation learning framework, with knowledge transfer from the unlabelled synthetic point clouds of full lower-limb surfaces. The proposed method can significantly reduce the number of ground truth skeletons (with only 1%) in the training phase, meanwhile ensuring accurate and precise pose estimation and capturing discriminative features across different pathological gait patterns compared to other methods.

Electroencephalography (EEG) is one of the most widely used brain-activity recording methods in non-invasive brain-machine interfaces (BCIs). However, EEG data is highly nonlinear, and its datasets often suffer from issues such as data heterogeneity, label uncertainty and data/label scarcity. To address these, we propose a domain independent, end-to-end semi-supervised learning framework with contrastive learning and adversarial training strategies. Our method was evaluated in experiments with different amounts of labels and an ablation study in a motor imagery EEG dataset. The experiments demonstrate that the proposed framework with two different backbone deep neural networks show improved performance over their supervised counterparts under the same condition.

Hearing aid devices have been around for decades, and one of the most recent approaches is the cochlear implant which is designed for patients with severe hearing loss. This paper introduces the design of a haptic-signal based hearing aid targeting patients suffering from inner ear malfunction, for which the conventional assistive hearing devices will not suffice. This device is designed to record the incoming sound, filter and analyze it into its harmonics, and classify it into the phonemes. The output is transfer into tactile feedback with vibrating motors and each phoneme will activate the respective combination of them.

Sound monitoring from sources inside the human body can have important diagnostic relevance in medicine. Cardiac sounds originated from the pumping activity of the heart structure is such an example, with valuable cardiovascular parameters being extracted from the signal, including heart rate (HR) and the systolic intervals. Novel non-invasive methods for early detection of potential life-threatening risks convoyed by unbalanced cardiovascular parameters are essential to reduce the mortality rates associated with cardiac diseases nowadays. In this paper, we propose a small-form wearable device for longterm monitoring of the cardiac sounds through a miniaturized microphone in contact with the body surface at specific locations, which extend from the chest region to the upper and lower body parts. Powered by battery, the device can measure signals for a consecutive period of 28 h in continuous recording mode that is extensive up to 7 days in discontinuous mode, achieving signal amplitude resolution of 0.81 μV and optimal bandwidth between 5 to 20 Hz (infrasound range). The proposed device was able to detect cardiac sound patterns in locations as distant as the forehead, wrist, or ankle, thus paving the way to the use of acoustic signals for wearable heartbeat estimators still relying on optical or bio-potential methods, while replacing the obtrusive and expensive cardiography equipment dedicated to the estimation of the systolic intervals directly from the chest.

Cerebral palsy can severely impair children's motor function and leading to permanent disability. Compared to adults, children are more vulnerable and susceptible to external harm. Wearable robotics gained much attention in rehabilitation, and has shown its potential in supporting the recovery of people with motor dysfunctions. Conventional adult-oriented wearable assistive robots are tendon-driven whereas the force and inertia generated is too large for children, which could injure children. To address this issue, this paper proposes a novel soft inflatable robot that can aid children in elbow movement whilst minimising the risk of harm. Thermoplastic Polyurethane (TPU) and pneumatic actuation were used in developing the soft robot. From the experiment, the maximum bending angle is 142.2°, with the maximum moment generated being 0.784 Nm, which is suitable for the needed elbow support for young children with cerebral palsy.

Research on generating or processing point clouds has become an increasingly popular domain in robotic research due to its extensive applications, such as robotic grasping, augmented reality and autonomous vehicle navigation. In this paper, we explore a new research area on point clouds - Learning to rank 3D models captured from a single depth image. In the Learning To Rank (LTR) task, we aim at optimizing the order of a list of 3D models according to the given query. Inspired by the recent advances in Natural Language Processing (NLP), we propose a novel framework, namely Deep3DRanker, for ranking 3D models by leveraging graph-based encoding and self-attention mechanisms. Comprehensive experiments are conducted to validate our methods on publicly available YCB synthetic and YCB video datasets. The promising results have shown that our proposed framework is generic enough to be applicable with any combinations of randomly positioned, oriented, and unseen object items with accuracy ranging from 59.2% to 94.9%, which shows great potentials of the proposed framework for robotic applications, in particular, for making decisions under different circumstances.

In Robot-Assisted Minimally Invasive Surgery (RAMIS), a camera assistant is normally required to control the position and the zooming ratio of the laparoscope, following the surgeon’s instructions. However, moving the laparoscope frequently may lead to unstable and suboptimal views, while the adjustment of zooming ratio may interrupt the workflow of the surgical operation. To this end, we propose a multi-scale Generative Adversarial Network (GAN)-based video super-resolution method to construct a framework for automatic zooming ratio adjustment. It can provide automatic real-time zooming for high-quality visualization of the Region of Interest (ROI) during the surgical operation. In the pipeline of the framework, the Kernel Correlation Filter (KCF) tracker is used for tracking the tips of the surgical tools, while the Semi-Global Block Matching (SGBM)-based depth estimation and Recurrent Neural Network (RNN)-based context-awareness are employed to determine the upscaling ratio for zooming. The framework is validated with the JIGSAW dataset and Hamlyn Centre Laparoscopic/Endoscopic Video Datasets, with results demonstrating its practicability.

Minimally invasive surgery mainly consists of a series of sub-tasks, which can be decomposed into basic gestures or contexts. As a prerequisite of autonomic operation, surgical gesture recognition can assist motion planning and decision-making, and build up context-aware knowledge to improve the surgical robot control quality. In this work, we aim to develop an effective surgical gesture recognition approach with an explainable feature extraction process.A Bidirectional Multi-Layer independently RNN (BMLindRNN) model is proposed in this paper, while spatial feature extraction is implemented via fine-tuning of a Deep Convolutional Neural Network (DCNN) model constructed based on the VGG architecture. To eliminate the black-box effects of DCNN, Gradient-weighted Class Activation Mapping (Grad-CAM) is employed. It can provide explainable results by showing the regions of the surgical images that have a strong relationship with the surgical gesture classification results.The proposed method was evaluated based on the suturing task with data obtained from the public available JIGSAWS database. Comparative studies were conducted to verify the proposed framework. Results indicated that the testing accuracy for the suturing task based on our proposed method is 87.13%, which outperforms most of the state-of-the-art algorithms.

Accurate prediction of future person location and movement trajectory from an egocentric wearable camera can benefit a wide range of applications, such as assisting visually impaired people in navigation, and the development of mobility assistance for people with disability. In this work, a new egocentric dataset was constructed using a wearable camera, with 8,250 short clips of a targeted person either walking 1) toward, 2) away, or 3) across the camera wearer in indoor environments, or 4) staying still in the scene, and 13,817 person bounding boxes were manually labelled. Apart from the bounding boxes, the dataset also contains the estimated pose of the targeted person as well as the IMU signal of the wearable camera at each time point. An LSTM-based encoder-decoder framework was designed to predict the future location and movement trajectory of the targeted person in this egocentric setting. Extensive experiments have been conducted on the new dataset, and have shown that the proposed method is able to reliably and better predict future person location and trajectory in egocentric videos captured by the wearable camera compared to three baselines.

Assessing dietary intake in epidemiological studies are predominantly based on self-reports, which are subjective, inefficient, and also prone to error. Technological approaches are therefore emerging to provide objective dietary assessments. Using only egocentric dietary intake videos, this work aims to provide accurate estimation on individual dietary intake through recognizing consumed food items and counting the number of bites taken. This is different from previous studies that rely on inertial sensing to count bites, and also previous studies that only recognize visible food items but not consumed ones. As a subject may not consume all food items visible in a meal, recognizing those consumed food items is more valuable. A new dataset that has 1,022 dietary intake video clips was constructed to validate our concept of bite counting and consumed food item recognition from egocentric videos. 12 subjects participated and 52 meals were captured. A total of 66 unique food items, including food ingredients and drinks, were labelled in the dataset along with a total of 2,039 labelled bites. Deep neural networks were used to perform bite counting and food item recognition in an end-to-end manner. Experiments have shown that counting bites directly from video clips can reach 74.15% top-1 accuracy (classifying between 0-4 bites in 20-second clips), and a MSE value of 0.312 (when using regression). Our experiments on video-based food recognition also show that recognizing consumed food items is indeed harder than recognizing visible ones, with a drop of 25% in F1 score.

With the development of medical artificial intelligence, automatic magnetic resonance image (MRI) segmentation method is quite desirable. Inspired by the power of deep neural networks, a novel deep adversarial network, dilated block adversarial network (DBAN), is proposed to perform left ventricle, right ventricle, and myocardium segmentation in short-axis cardiac MRI. DBAN contains a segmentor along with a discriminator. In the segmentor, the dilated block (DB) is proposed to capture, and aggregate multi-scale features. The segmentor can produce segmentation probability maps while the discriminator can differentiate the segmentation probability map, and the ground truth at the pixel level. In addition, confidence probability maps generated by the discriminator can guide the segmentor to modify segmentation probability maps. Extensive experiments demonstrate that DBAN has achieved the state-of-the-art performance on the ACDC dataset. Quantitative analyses indicate that cardiac function indices from DBAN are similar to those from clinical experts. Therefore, DBAN can be a potential candidate for short-axis cardiac MRI segmentation in clinical applications.

For abnormal gait recognition, pattern-specific features indicating abnormalities are interleaved with the subject-specific differences representing biometric traits. Deep representations are, therefore, prone to overfitting, and the models derived cannot generalize well to new subjects. Furthermore, there is limited availability of abnormal gait data obtained from precise Motion Capture (Mocap) systems because of regulatory issues and slow adaptation of new technologies in health care. On the other hand, data captured from markerless vision sensors or wearable sensors can be obtained in home environments, but noises from such devices may prevent the effective extraction of relevant features. To address these challenges, we propose a cascade of deep architectures that can encode cross-modal and cross-subject transfer for abnormal gait recognition. Cross-modal transfer maps noisy data obtained from RGBD and wearable sensors to accurate 4-D representations of the lower limb and joints obtained from the Mocap system. Subsequently, cross-subject transfer allows disentangling subject-specific from abnormal pattern-specific gait features based on a multiencoder autoencoder architecture. To validate the proposed methodology, we obtained multimodal gait data based on a multicamera motion capture system along with synchronized recordings of electromyography (EMG) data and 4-D skeleton data extracted from a single RGBD camera. Classification accuracy was improved significantly in both Mocap and noisy modalities.

The recent development of Robot-Assisted Minimally Invasive Surgery (RAMIS) has brought much benefit to ease the performance of complex Minimally Invasive Surgery (MIS) tasks and lead to more clinical outcomes. Compared to direct master-slave manipulation, semi-autonomous control for the surgical robot can enhance the efficiency of the operation, particularly for repetitive tasks. However, operating in a highly dynamic in-vivo environment is complex. Supervisory control functions should be included to ensure flexibility and safety during the autonomous control phase. This paper presents a haptic rendering interface to enable supervised semi-autonomous control for a surgical robot. Bayesian optimization is used to tune user-specific parameters during the surgical training process. User studies were conducted on a customized simulator for validation. Detailed comparisons are made between with and without the supervised semi-autonomous control mode in terms of the number of clutching events, task completion time, master robot end-effector trajectory and average control speed of the slave robot. The effectiveness of the Bayesian optimization is also evaluated, demonstrating that the optimized parameters can significantly improve users' performance. Results indicate that the proposed control method can reduce the operator's workload and enhance operation efficiency.

Gastrointestinal (GI) endoscopy is a conventional and prevalent procedure used to diagnose and treat diseases in the digestive tract. This procedure requires inserting an endoscope equipped with a camera and instruments inside a patient to the target of interest. To manoeuvre the endoscope, an endoscopist would rotate the knob at the handle to change the direction of the distal tip and apply the feeding force to advance the endoscope. However, due to the nature of the design, this often causes a looping problem during insertion making it difficult to be further advanced to the deeper section of the tract such as the transverse and ascending colon. To this end, in this paper, we propose a novel robotic endoscope which is covered by a rotating screw-like sheath and uses a spiral insertion technique to generate 'pull' forces at the distal tip of the endoscope to facilitate insertion. The whole shaft of the endoscope can be actively rotated, providing the crawling ability from the attached spiral sheath. With the redundant control on a spring-like continuum joint, the bending tip is capable of maintaining its orientation to assist endoscope navigation. To test its functions and feasibility to address the looping problem, three experiments were carried out. The first two experiments were to analyse the kinematic of the device and test the ability of the device to hold its distal tip at different orientation angles during spiral insertion. In the third experiment, we inserted the device in the bent colon phantom to evaluate the effectiveness of the proposed design against looping when advancing through a curved section of a colon. Results show the moving ability using spiral technique and verify its potential of clinical application.

Abstract Classification of electrocardiogram (ECG) signals is obligatory for the automatic diagnosis of cardiovascular disease. With the recent advancement of low‐cost wearable ECG device, it becomes more feasible to utilize ECG for cardiac arrhythmia classification in daily life. In this paper, we propose a lightweight approach to classify five types of cardiac arrhythmia, namely, normal beat (N), atrial premature contraction (A), premature ventricular contraction (V), left bundle branch block beat (L), and right bundle branch block beat (R). The combined method of frequency analysis and Shannon entropy is applied to extract appropriate statistical features. Information gain criterion is employed to select features that the results show that 10 highly effective features can obtain performance measures comparable to those obtained by using the complete features. The selected features are then fed to the input of Random Forest, K‐Nearest Neighbour, and J48 for classification. To evaluate classification performance, tenfold cross validation is used to verify the effectiveness of our method. Experimental results show that Random Forest classifier demonstrates significant performance with the highest sensitivity of 98.1%, the specificity of 99.5%, the precision of 98.1%, and the accuracy of 98.08%, outperforming other representative approaches for automated cardiac arrhythmia classification.

Dietary assessment is an important tool for nutritional epidemiology studies. To assess the dietary intake, the common approach is to carry out 24-h dietary recall (24HR), a structured interview conducted by experienced dietitians. Due to the unconscious biases in such self-reporting methods, many research works have proposed the use of vision-based approaches to provide accurate and objective assessments. In this article, a novel vision-based method based on real-time three-dimensional (3-D) reconstruction and deep learning view synthesis is proposed to enable accurate portion size estimation of food items consumed. A point completion neural network is developed to complete partial point cloud of food items based on a single depth image or video captured from any convenient viewing position. Once 3-D models of food items are reconstructed, the food volume can be estimated through meshing. Compared to previous methods, our method has addressed several major challenges in vision-based dietary assessment, such as view occlusion and scale ambiguity, and it outperforms previous approaches in accurate portion size estimation.

Blood glucose level needs to be monitored regularly to manage the health condition of hyperglycemic patients. The current glucose measurement approaches still rely on invasive techniques which are uncomfortable and raise the risk of infection. To facilitate daily care at home, in this article, we propose an intelligent, noninvasive blood glucose monitoring system which can differentiate a user's blood glucose level into normal, borderline, and warning based on smartphone photoplethysmography (PPG) signals. The main implementation processes of the proposed system include 1) a novel algorithm for acquiring PPG signals using only smartphone camera videos; 2) a fitting-based sliding window algorithm to remove varying degrees of baseline drifts and segment the signal into single periods; 3) extracting characteristic features from the Gaussian functions by comparing PPG signals at different blood glucose levels; 4) categorizing the valid samples into three glucose levels by applying machine learning algorithms. Our proposed system was evaluated on a data set of 80 subjects. Experimental results demonstrate that the system can separate valid signals from invalid ones at an accuracy of 97.54% and the overall accuracy of estimating the blood glucose levels reaches 81.49%. The proposed system provides a reference for the introduction of noninvasive blood glucose technology into daily or clinical applications. This article also indicates that smartphone-based PPG signals have great potential to assess an individual's blood glucose level.

The fifteen articles in this special section focus on data science used in smart healthcare applications. A shift toward a data-driven socio-economic health model is occurring. This is the result of the increased volume, velocity and variety of data collected from the public and private sector in healthcare, and biology in general. In the past five-years, there has been an impressive development of computational intelligence and informatics methods for application to health and biomedical science. However, the effective use of data to address the scale and scope of human health problems has yet to realize its full potential. The barriers limiting the impact of practical application of standard data mining and machine learning methods have been inherent to the characteristics of health data. Besides the volume of the data (‘big data’), these are challenging due to their heterogeneity, complexity, variability and dynamic nature. Finally, data management and interpretability of the results have been limited by practical challenges in implementing new and also existing standards across the different health providers and research institutions. The scope of this Special issue is to discuss some of these challenges and opportunities in health and biological data science, with particular focus on the infrastructure, software, methods and algorithms needed to analyze large datasets in biological and clinical research.

Excessive stress is one of the main causes of mental illness. Long-term exposure of stress could affect one's physiological wellbeing (such as hypertension) and psychological condition (such as depression). Multisensory information such as heart rate variability (HRV) and pH can provide suitable information about mental and physical stress. This paper proposes a novel approach for stress condition monitoring using disposable flexible sensors. By integrating flexible amplifiers with a commercially available flexible polyvinylidene difluoride (PVDF) mechanical deformation sensor and a pH-type chemical sensor, the proposed system can detect arterial pulses from the neck and pH levels from sweat located in the back of the body. The system uses organic thin film transistor (OTFT)-based signal amplification front-end circuits with modifications to accommodate the dynamic signal ranges obtained from the sensors. The OTFTs were manufactured on a low-cost flexible polyethylene naphthalate (PEN) substrate using a coater capable of Roll-to-Roll (R2R) deposition. The proposed system can capture physiological indicators with data interrogated by Near Field Communication (NFC). The device has been successfully tested with healthy subjects, demonstrating its feasibility for real-time stress monitoring.

A daily dietary assessment method named 24-hour dietary recall has commonly been used in nutritional epidemiology studies to capture detailed information of the food eaten by the participants to help understand their dietary behaviour. However, in this self-reporting technique, the food types and the portion size reported highly depends on users' subjective judgement which may lead to a biased and inaccurate dietary analysis result. As a result, a variety of visual-based dietary assessment approaches have been proposed recently. While these methods show promises in tackling issues in nutritional epidemiology studies, several challenges and forthcoming opportunities, as detailed in this study, still exist. This study provides an overview of computing algorithms, mathematical models and methodologies used in the field of image-based dietary assessment. It also provides a comprehensive comparison of the state of the art approaches in food recognition and volume/weight estimation in terms of their processing speed, model accuracy, efficiency and constraints. It will be followed by a discussion on deep learning method and its efficacy in dietary assessment. After a comprehensive exploration, we found that integrated dietary assessment systems combining with different approaches could be the potential solution to tackling the challenges in accurate dietary intake assessment.

Epilepsy seizure prediction paves the way of timely warning for patients to take more active and effective intervention measures. Compared to seizure detection that only identifies the inter-ictal state and the ictal state, far fewer researches have been conducted on seizure prediction because the high similarity makes it challenging to distinguish between the pre-ictal state and the inter-ictal state. In this paper, a novel solution on seizure prediction is proposed using common spatial pattern (CSP) and convolutional neural network (CNN). Firstly, artificial pre-ictal EEG signals based on the original ones are generated by combining the segmented pre-ictal signals to solve the trial imbalance problem between the two states. Secondly, a feature extractor employing wavelet packet decomposition and CSP is designed to extract the distinguishing features in both the time domain and the frequency domain. It can improve overall accuracy while reducing the training time. Finally, a shallow CNN is applied to discriminate between the pre-ictal state and the inter-ictal state. Our proposed solution is evaluated on 23 patients' data from Boston Children's Hospital-MIT scalp EEG dataset by employing a leave-one-out cross-validation, and it achieves a sensitivity of 92.2% and false prediction rate of 0.12/h. Experimental result demonstrates that the proposed approach outperforms most state-of-the-art methods.

Wearable inertial sensors have demonstrated their potential to screen for various neuropathies and neurological disorders. Most such research has been based on classification algorithms that differentiate the control group from the pathological group, using biomarkers extracted from wearable data as predictors. However, such methods often lack quantitative evaluation of how much information provided by the wearable biomarkers contributes to the overall prediction. Despite promising results from internal cross validation, their utility in clinical practice remains unclear. In this paper, we highlight in a case study - early screening for diabetic peripheral neuropathy (DPN) - evaluation methods for quantifying the contribution of wearable inertial sensors. Using a quick-to-deploy wearable sensor system, we collected 106 in-hospital diabetic patients' gait data and developed logistic regression models to predict the risk of a diabetic patient having DPN. Adopting various metrics, we evaluated the discriminative information added by gait biomarkers and how much it improved screening. The results show that the proposed wearable system added useful information significantly to the existing clinical standards, and boosted the C-index significantly from 0.75 to 0.84, surpassing the current survey-based screening methods used in clinics.

Dietary assessment system has proven as an effective tool to evaluate the eating behavior of patients suffering from diabetes and obesity. To assess the dietary intake, the traditional method is to carry out a 24-hour dietary recall (24HR), a structured interview aimed at capturing information on food items and portion size consumed by participants. However, unconscious biases are developed easily due to individual's subjective perception in this self-reporting technique which may lead to inaccuracy. Thus, this paper proposed a novel vision-based approach for estimating the volume of food items based on deep learning view synthesis and depth sensing techniques. In this paper, a point completion network is applied to perform 3D reconstruction of food items using a single depth image captured from any convenient viewing angle. Compared to previous approaches, the proposed method has addressed several key challenges in vision-based dietary assessment, such as view occlusion and scale ambiguity. Experiments have been carried out to examine this approach and showed the feasibility of the algorithm in accurate estimation of food volume.

A novel vision-based approach for estimating individual dietary intake in food sharing scenarios is proposed in this paper, which incorporates food detection, face recognition and hand tracking techniques. The method is validated using panoramic videos which capture subjects' eating episodes. The results demonstrate that the proposed approach is able to reliably estimate food intake of each individual as well as the food eating sequence. To identify the food items ingested by the subject, a transfer learning approach is designed. 4, 200 food images with segmentation masks, among which 1,500 are newly annotated, are used to fine-tune the deep neural network for the targeted food intake application. In addition, a method for associating detected hands with subjects is developed and the outcomes of face recognition are refined to enable the quantification of individual dietary intake in communal eating settings.

Food intake is a major healthcare issue in developed countries that has become an economic and social burden across all sectors of society. Bad food intake habits lead to increased risk for development of obesity in children, young people and adults, with the latter more prone to suffer from health diseases such as diabetes, shortening the life expectancy. Environmental, cultural and behavioural factors have been appointed to be responsible for altering the balance between energy intake and expenditure, resulting in excess body weight. Methods to counteract the food intake problem are vast and include self-reported food questionnaires, body-worn sensors that record the sound, pressure or movements in the mouth and GI tract or image-based approaches that recognize the different types of food being ingested. In this paper we present an ear-worn device to track food intake habits by recording the acoustic signal produced by the chewing movements as well as the glucose level amperiometrically. Combined with a small camera on a future version of the device, we hope to deliver a complete system to control dietary habits with caloric intake estimation during satiation and deficit during satiety periods, which can be adapted to the physiology of each user.

Extracting human attributes, such as gender and age, from biometrics have received much attention in recent years. Gender and age recognition can provide crucial information for applications such as security, healthcare, and gaming. In this paper, a novel deep learning approach on gender and age recognition using a single inertial sensors is proposed. The proposed approach is tested using the largest available inertial sensor-based gait database with data collected from more than 700 subjects. To demonstrate the robustness and effectiveness of the proposed approach, 10 trials of inter-subject Monte-Carlo cross validation were conducted, and the results show that the proposed approach can achieve an averaged accuracy of 86.6%±2.4% for distinguishing two age groups: teen and adult, and recognizing gender with averaged accuracies of 88.6%±2.5% and 73.9%±2.8% for adults and teens respectively.

The prospect of roll-to-roll (R2R) processable Organic Thin Film Transistors (OTFTs) and circuits has attracted attention due to their mechanical flexibility and low cost of manufacture. This work will present a flexible electronics application for pH sensing with flexible and wearable signal processing circuits. A transimpedance amplifier was designed and fabricated on a polyethylene naphthalate (PEN) substrate prototype sheet that consists of 54 transistors. Different types and current ratios of current mirrors were initially created and then a suitable simple 1:3 current mirror (200nA) was selected to present the best performance of the proposed OTFT based transimpedance amplifier (TIA). Finally, this transimpedance amplifier was connected to a customized needle-based pH sensor that was induced as microfluidic collector for potential disease diagnosis and healthcare monitoring.

Personalized stent graft is designed to treat Abdominal Aortic Aneurysms (AAA). Due to the individual difference in arterial structures, stent graft has to be custom made for each AAA patient. Robotic platforms for autonomous personalized stent graft manufacturing have been proposed in recently which rely upon stereo vision systems for coordinating multiple robots for fabricating customized stent grafts. This paper proposes a novel hybrid vision system for real-time visual-sevoing for personalized stent-graft manufacturing. To coordinate the robotic arms, this system is based on projecting a dynamic stereo microscope coordinate system onto a static wide angle view stereo webcam coordinate system. The multiple stereo camera configuration enables accurate localization of the needle in 3D during the sewing process. The scale-invariant feature transform (SIFT) method and color filtering are implemented for stereo matching and feature identifications for object localization. To maintain the clear view of the sewing process, a visual-servoing system is developed for guiding the stereo microscopes for tracking the needle movements. The deep deterministic policy gradient (DDPG) reinforcement learning algorithm is developed for real-time intelligent robotic control. Experimental results have shown that the robotic arm can learn to reach the desired targets autonomously.

The seven papers included in this special section focus on machine learning applications for the mental health industry. Mental health is one of the major global health issues affecting substantially more people than other noncommunicable diseases. Much research has been focused on developing novel technologies for tackling this global health challenge, including the development of advanced analytical techniques based on extensive datasets and multimodal acquisition for early detection and treatment of mental illnesses. The papers in this issue are dedicated to cover the related topics on technological advancements for mental health care and diagnosis with a focus on pervasive sensing and machine learning.

As the popularity of wearable and the implantable body sensor network (BSN) devices increases, there is a growing concern regarding the data security of such power-constrained miniaturized medical devices. With limited computational power, BSN devices are often not able to provide strong security mechanisms to protect sensitive personal and health information, such as one's physiological data. Consequently, many new methods of securing wireless body area networks have been proposed recently. One effective solution is the biometric cryptosystem (BCS) approach. BCS exploits physiological and behavioral biometric traits, including face, iris, fingerprints, electrocardiogram, and photoplethysmography. In this paper, we propose a new BCS approach for securing wireless communications for wearable and implantable healthcare devices using gait signal energy variations and an artificial neural network framework. By simultaneously extracting similar features from BSN sensors using our approach, binary keys can be generated on demand without user intervention. Through an extensive analysis on our BCS approach using a gait dataset, the results have shown that the binary keys generated using our approach have high entropy for all subjects. The keys can pass both National Institute of Standards and Technology and Dieharder statistical tests with high efficiency. The experimental results also show the robustness of the proposed approach in terms of the similarity of intraclass keys and the discriminability of the interclass keys.

Continuous buried soft tissue free flap postoperative monitoring is crucial to detect flap failure and enable early intervention. In this case, clinical assessment is challenging as the flap is buried and only implantable or hand held devices can be used for regular monitoring. These devices have limitations in their price, usability and specificity. Near-infrared spectroscopy (NIRS) has shown promising results for superficial free flap postoperative monitoring, but it has not been considered for buried free flap, mainly due to the limited penetration depth of conventional approaches. A wearable wireless tomographic probe has been developed for continuous monitoring of tissue perfusion at different depths. Using the NIRS method, blood flow can be continuously measured at different tissue depths. This device has been designed following conclusions of extensive computerised simulations and it has been validated using a vascular phantom.

A novel food volume measurement technique is proposed in this paper for accurate quantification of the daily dietary intake of the user. The technique is based on simultaneous localisation and mapping (SLAM), a modified version of convex hull algorithm, and a 3D mesh object reconstruction technique. This paper explores the feasibility of applying SLAM techniques for continuous food volume measurement with a monocular wearable camera. A sparse map will be generated by SLAM after capturing the images of the food item with the camera and the multiple convex hull algorithm is applied to form a 3D mesh object. The volume of the target object can then be computed based on the mesh object. Compared to previous volume measurement techniques, the proposed method can measure the food volume continuously with no prior information such as pre-defined food shape model. Experiments have been carried out to evaluate this new technique and showed the feasibility and accuracy of the proposed algorithm in measuring food volume.

Gait analysis is an important tool for monitoring and preventing injuries as well as to quantify functional decline in neurological diseases and elderly people. In most cases, it is more meaningful to monitor patients in natural living environments with low-end equipment such as cameras and wearable sensors. However, inertial sensors cannot provide enough details on angular dynamics. This paper presents a method that uses a single RGB camera to track the 2D joint coordinates with state-of-the-art vision algorithms. Reconstruction of the 3D trajectories uses sparse representation of an active shape model. Subsequently, we extract gait features and validate our results in comparison with a state-of-the-art commercial multi-camera tracking system. Our results are comparable to those from the current literature based on depth cameras and optical markers to extract gait characteristics.

Electroencephalogram (EEG) that measures the electrical activity of the brain has been widely employed for diagnosing epilepsy which is one kind of brain abnormalities. With the advancement of low-cost wearable brain-computer interface devices, it is possible to monitor EEG for epileptic seizure detection in daily use. However, it is still challenging to develop seizure classification algorithms with a considerable higher accuracy and lower complexity. In this study, we propose a lightweight method which can reduce the number of features for a multiclass classification to identify three different seizure statuses (i.e., Healthy, Interictal and Epileptic seizure) through EEG signals with a wearable EEG sensors using Extended Correlation-Based Feature Selection (ECFS). More specifically, there are three steps in our proposed approach. Firstly, the EEG signals were segmented into five frequency bands and secondly, we extract the features while the unnecessary feature space was eliminated by developing the ECFS method. Finally, the features were fed into five different classification algorithms, including Random Forest, Support Vector Machine, Logistic Model Trees, RBF Network and Multilayer Perceptron. Experimental results have shown that Logistic Model Trees provides the highest accuracy of 97.6% comparing to other classifiers.

This paper proposes a novel artificial neural network based method for real-time gait analysis with minimal number of Inertial Measurement Units (IMUs). Accurate lower limb attitude estimation has great potential for clinical gait diagnosis for orthopaedic patients and patients with neurological diseases. However, the use of multiple wearable sensors hinder the ubiquitous use of inertial sensors for detailed gait analysis. This paper proposes the use of two IMUs mounted on the shoes to estimate the IMU signals at the shin, thigh and waist for accurate attitude estimation of the lower limbs. By using the artificial neural network framework, the gait parameters, such as angle, velocity and displacements of the IMUs can be estimated. The experimental results have shown that the proposed method can accurately estimate the IMUs signals on the lower limbs based only on the IMU signals on the shoes, which demonstrates its potential for lower limb motion tracking and real-time gait analysis.

In fasciocutaneous free flap surgery, close postoperative monitoring is crucial for detecting flap failure, as around 10% of cases require additional surgery due to compromised anastomosis. Different biochemical and biophysical techniques have been developed for continuous flap monitoring, however, they all have shortcoming in terms of reliability, elevated cost, potential risks to the patient, and inability to adapt to the patient's phenotype. A wearable wireless device based on near infrared spectroscopy has been developed for continuous blood flow and perfusion monitoring by quantifying tissue oxygen saturation (). This miniaturized and low-cost device is designed for postoperative monitoring of flap viability. With self-calibration, the device can adapt itself to the characteristics of the patients' skin such as tone and thickness. An extensive study was conducted with 32 volunteers. The experimental results show that the device can obtain reliable measurements across different phenotypes (age, sex, skin tone, and thickness). To assess its ability to detect flap failure, the sensor was tested in a pilot animal study. Free groin flaps were performed on 16 Sprague Dawley rats. Results demonstrate the accuracy of the sensor in assessing flap viability and identifying the origin of failure (venous or arterial thrombosis).

Tthe BSN 2016 meeting in the University of California at San Francisco (UCSF) Mission Bay conference center focused on neuroscience applications including stress and behavior monitoring and chronic disease management. Keynote presentations highlighted the use of pattern analysis and "smart shoes" to manage Parkinson’s disease, and bioengineering advances in detecting electrodermal activity at the wrist associated with epilepsy and psychologically stressful events. The themes of the conference are described. A workshop sponsored and organized by Friedrich Alexander University, Erlangen, previewed this theme of automated sensor-based mobility analysis for chronic disease management.

Blood flow, posture and phenotype (such as age, sex, smoking habit or physical activity) are closely related to vascular health. Episodic monitoring of the vascular system in clinical setting can lead to late diagnose. Inexpensive wearable devices for continuous monitoring of vascular parameters have been widely used, however, they often have limitations in data interpretation: changes in the environment setting can significantly affect the meaning of the results. This paper proposes a low cost networked body worn sensors for real-time analysis of hemodynamics and reports preliminary results on the relation between blood flow (measured through pulse arrival time (PAT)), the effect of postures and age ranges based on experiments with 13 volunteers of different age ranges (50 years old). Standing, supine and sitting postures were investigated while photoplethysmograph (PPG) sensors were placed at different locations (ear, wrist and ankle). Results show the PAT changes according to the investigated locations and postures for both age group. Also, the average PAT values of the older group are generally higher than those of the younger group. In the older group, the average PAT value is higher for the supine posture than that of the sitting posture which is itself higher than that of the standing posture. In the younger group, the average PAT is higher in supine than that of the sitting and standing postures which have similar average PAT values. This indicates that hemodynamics vary with posture and age.

With increasing popularity of wearable and Body Sensor Networks technologies, there is a growing concern on the security and data protection of such low-power pervasive devices. With very limited computational power, BSN sensors often cannot provide the necessary data protection to collect and process sensitive personal information. Since conventional network security schemes are too computationally demanding for miniaturized BSN sensors, new methods of securing BSNs have proposed, in which Biometric Cryptosystem (BCS) appears to be an effective solution. With regards to BCS security solutions, physiological traits, such as an individual's face, iris, fingerprint, electrocardiogram (ECG), and photoplethysmogram (PPG) have been widely exploited. However, behavioural traits such as gait are rarely studied. In this paper, a novel lightweight symmetric key generation scheme based on the timing information of gait is proposed. By extracting similar timing information from gait acceleration signals simultaneously from body worn sensors, symmetric keys can be generated on all the sensor nodes at the same time. Based on the characteristics of generated keys and BSNs, a fuzzy commitment based key distribution scheme is also developed to distribute the keys amongst the sensor nodes.

Recent studies have shown that air pollution has a negative impact on people's health, especially for patients with respiratory and cardiac diseases (e.g. COPD, asthma, ischemic heart disease). Although there are already many air quality monitoring stations in major cities, such as London, these stations are sparsely located, and the periodic collection of information is insufficient to provide the granularity needed to assess the environmental risk for an individual (e.g. to avoid exacerbation). Wearable devices, on the other hand, are more suitable in this context, providing a better estimation of the air quality in the proximity of the person. Therefore, relevant warnings and information on health risks can be provided in real-time. As a proof of concept, we have developed a wearable sensor for continuous monitoring of air quality around the user, and a preliminary study was conducted to validate the sensor and assess the air quality in London underground stations. Based on the PM2.5 (particulate matter with a diameter of 2.5 μm), temperature and location information, a model is generated for predicting the air quality of each station at different times. Our preliminary results have shown that there are significant differences in air quality among stations and metro lines. It also demonstrates that wearable sensors can provide necessary information for users to make travel arrangements that minimize their exposure to polluted air.

With a massive influx of multimodality data, the role of data analytics in health informatics has grown rapidly in the last decade. This has also prompted increasing interests in the generation of analytical, data driven models based on machine learning in health informatics. Deep learning, a technique with its foundation in artificial neural networks, is emerging in recent years as a powerful tool for machine learning, promising to reshape the future of artificial intelligence. Rapid improvements in computational power, fast data storage, and parallelization have also contributed to the rapid uptake of the technology in addition to its predictive power and ability to generate automatically optimized high-level features and semantic interpretation from the input data. This article presents a comprehensive up-to-date review of research employing deep learning in health informatics, providing a critical analysis of the relative merit, and potential pitfalls of the technique as well as its future outlook. The paper mainly focuses on key applications of deep learning in the fields of translational bioinformatics, medical imaging, pervasive sensing, medical informatics, and public health.

The increasing popularity of wearable devices in recent years means that a diverse range of physiological and functional data can now be captured continuously for applications in sports, wellbeing, and healthcare. This wealth of information requires efficient methods of classification and analysis where deep learning is a promising technique for large-scale data analytics. While deep learning has been successful in implementations that utilize high-performance computing platforms, its use on low-power wearable devices is limited by resource constraints. In this paper, we propose a deep learning methodology, which combines features learned from inertial sensor data together with complementary information from a set of shallow features to enable accurate and real-time activity classification. The design of this combined method aims to overcome some of the limitations present in a typical deep learning framework where on-node computation is required. To optimize the proposed method for real-time on-node computation, spectral domain preprocessing is used before the data are passed onto the deep learning framework. The classification accuracy of our proposed deep learning approach is evaluated against state-of-the-art methods using both laboratory and real world activity datasets. Our results show the validity of the approach on different human activity datasets, outperforming other methods, including the two methods used within our combined pipeline. We also demonstrate that the computation times for the proposed method are consistent with the constraints of real-time on-node processing on smartphones and a wearable sensor platform.

Blood flow and perfusion monitoring are critical appraisal to ensure survival of tissue flap after reconstructive surgery. Many techniques have been developed over the years: from optical to chemical, invasive or not, they all have limitations in their price, risks and adaptiveness to the patient. A wireless wearable self-calibrated device, based on near infrared spectroscopy (NIRS) was developed for blood flow and perfusion monitoring contingent on tissue oxygen saturation (StO2). The use of such device is particularly relevant in the case of free flap myocutaneous reconstructive surgery; postoperative monitoring of the flap is crucial for a prompt intervention in case of thrombosis. Although failure rate is low, the rate of additional surgery following anastomosis problem is about 50%. NIRS has shown promising results for the monitoring of free flap, however lack of adaptation to its environment (ambient light) and users (body mass index (BMI), skin tone, alcohol and smoking habits or physical activity level) hinders the practical use of this technique. To overcome those limitations, a self-calibrated approach is introduced. Tested with is chaemia and cold water experiments on healthy subjects of different skin tones, its ability to personalize its calibration is demonstrated. Furthermore, using a vascular phantom, it is also able to detect pulses, differentiate venous and arterial coloured-like fluids with distinct clusters and detect significant changes in simulated partial venous occlusion. Placed in the trained classifier, partial occlusion data showed similar results between predicted and true classification. Further analysis from partial occlusion data showed that distinct clusters for 75% and 100% occlusion emerged.

Tremor is a neurological disorder which can significantly impede the daily functions of patients. The available treatments for patients with tremor are mainly pharmacotherapy and neurosurgery, but these treatments often have side effects. A wearable exoskeleton can potentially provide the assistance needed for patients with Parkinsonian or essential tremor to carry out daily activities and enable independent living. This paper presents the design and development of a 3D printed lightweight tremor suppression wearable exoskeleton. One of the major technical challenges for wearable robot is to maintain long battery life meanwhile miniature in size for practical use. This paper proposes an integrated approach where context aware Body Sensor Networks (BSN) sensors are incorporated to characterize voluntary and tremor movement, and detect activities of daily life (ADL). With the contextual information, the system can determine the intention of the user, optimize its control and minimize its power consumption by providing the necessary suppression only when needed. The preliminary result has shown that the wearable robot prototype can reduce the amplitude of simulated tremor by around 77%, and accurately identify different ADL with accuracy above 70%.

Human Activity Recognition provides valuable contextual information for wellbeing, healthcare, and sport applications. Over the past decades, many machine learning approaches have been proposed to identify activities from inertial sensor data for specific applications. Most methods, however, are designed for offline processing rather than processing on the sensor node. In this paper, a human activity recognition technique based on a deep learning methodology is designed to enable accurate and real-time classification for low-power wearable devices. To obtain invariance against changes in sensor orientation, sensor placement, and in sensor acquisition rates, we design a feature generation process that is applied to the spectral domain of the inertial data. Specifically, the proposed method uses sums of temporal convolutions of the transformed input. Accuracy of the proposed approach is evaluated against the current state-of-the-art methods using both laboratory and real world activity datasets. A systematic analysis of the feature generation parameters and a comparison of activity recognition computation times on mobile devices and sensor nodes are also presented.

The papers in this special section were presented at three well-known conferences organized in 2014: EAI Mobihealth, IEEE HealthCom, and IEEE Biomedical and Health Informatics. EAI Mobihealth is an annually organized conference, which started in 2010, to address the demands of the rapidly evolving disciplines of wireless communications, mobile computing, and sensing technologies in healthcare. The IEEE-Healthcom is held every year since 1999 in different countries in Asia, Europe, and in America. It aims at bringing together interested parties working in the field of healthcare to exchange ideas, discuss innovative and emerging solutions, and develop collaborations. The IEEE Biomedical Health Informatics Conference started in 2013 and is organized every year providing the forum to showcase enabling technologies of computing, devices, imaging, sensors, and systems that optimize the acquisition, transmission, processing, storage, retrieval, visualization, and analysis of medical data. The aim of this special section is to present an overview of recent advances in sensing technologies, monitoring of patients, security and privacy of data transfer, provision of collaborative environments, data gathering and analysis from various sources, and predictive models, which all finally target the best strategy for patient monitoring and treatment.

After decades of evolution, measuring instruments for quantitative gait analysis have become an important clinical tool for assessing pathologies manifested by gait abnormalities. However, such instruments tend to be expensive and require expert operation and maintenance besides their high cost, thus limiting them to only a small number of specialized centers. Consequently, gait analysis in most clinics today still relies on observation-based assessment. Recent advances in wearable sensors, especially inertial body sensors, have opened up a promising future for gait analysis. Not only can these sensors be more easily adopted in clinical diagnosis and treatment procedures than their current counterparts, but they can also monitor gait continuously outside clinics - hence providing seamless patient analysis from clinics to free-living environments. The purpose of this paper is to provide a systematic review of current techniques for quantitative gait analysis and to propose key metrics for evaluating both existing and emerging methods for qualifying the gait features extracted from wearable sensors. It aims to highlight key advances in this rapidly evolving research field and outline potential future directions for both research and clinical applications.

The year 2016 marks the 200th birth anniversary of Carl Friedrich Wilhelm Ludwig (1816-1895). As one of the most remarkable scientists, Ludwig invented the kymograph, which for the first time enabled the recording of continuous blood pressure (BP), opening the door to the modern study of physiology. Almost a century later, intraarterial BP monitoring through an arterial line has been used clinically. Subsequently, arterial tonometry and volume clamp method were developed and applied in continuous BP measurement in a noninvasive way. In the last two decades, additional efforts have been made to transform the method of unobtrusive continuous BP monitoring without the use of a cuff. This review summarizes the key milestones in continuous BP measurement; that is, kymograph, intraarterial BP monitoring, arterial tonometry, volume clamp method, and cuffless BP technologies. Our emphasis is on recent studies of unobtrusive BP measurements as well as on challenges and future directions.

This paper proposes a novel concept of using a multiple PPG and ECG based sensing platform aimed for monitoring the progress of diabetic peripheral neuropathy (DPN). It explores the use of PPG sensor to capture pulse arrival time (PAT). Based on the same principal of using Brachial-ankle pulse wave velocity (baPWV) to assess DPN, this paper proposes a platform which integrated two PPG sensors and one 2-lead ECG sensor to detect the difference in PAT (pulse arrive time on the finger compare to the time when the pulse reaches the ankle) as a surrogate measure for evaluating the progression of DPN. Preliminary results show that PAT increases when a pressure was applied onto upper leg using a blood pressure cuff simulating arterial stiffness/DPN. It shows that PDN can potentially be quantified by measuring PAT by using the proposed platform.

Similar to fingerprint and iris pattern, everyone's gait is unique, and gait has been proposed as a biometric feature for security applications. This paper presents a lightweight accelerometer-based technique for user authentication on smart wearable devices. Designed as an unsupervised classification approach, the proposed authentication technique can learn the user's gait pattern automatically when the user first starts wearing the device. Anomaly detection is then used to verify the device owner. The technique has been evaluated both in controlled and uncontrolled environments, with 20 and 6 healthy volunteers respectively. The Equal Error Rate (EER) in the controlled environments ranged from 5.7% (waist-mounted sensor) to 8.0% (trouser pocket). In the uncontrolled experiment, the device was put in the subject's trouser pocket, and the results were similar to the respective supervised experiment (EER=9.7%).

Recent trends in wearable applications demand flexible architectures being able to monitor people while they move in free-living environments. Current solutions use either store-download-offline processing or simple communication schemes with real-time streaming of sensor data. This limits the applicability of wearable applications to controlled environments (e.g, clinics, homes, or laboratories), because they need to maintain connectivity with the base station throughout the monitoring process. In this paper, we present the design and implementation of an opportunistic communication framework that simplifies the general use of wearable devices in free-living environments. It relies on a low-power data collection protocol that allows the end user to opportunistically, yet seamlessly manage the transmission of sensor data. We validate the feasibility of the framework by demonstrating its use for swimming, where the normal wireless communication is constantly interfered by the environment.

This paper analyses gait patterns of patients with Parkinson;s Disease (PD) based on the acceleration data given by an e-AR sensor. Ten PD patients wearing the e-AR sensor walked along a 7m walkway and each session contained 16 repeated trials. An iterative algorithm has been proposed to produce robust estimations in the case of measurement noise and short-duration of gait signals. Step-frequency as a gait parameter derived from the estimated heel-contacts is calculated and validated using the CODA motion-capture system. Intersession variability of step-frequency for each patient and the overall variability across patients demonstrate a good agreement between estimations from the e-AR and CODA systems.

Assessment of food intake has a wide range of applications in public health and life-style related chronic disease management. In this paper, we propose a real-time food recognition platform combined with daily activity and energy expenditure estimation. In the proposed method, food recognition is based on hierarchical classification using multiple visual cues, supported by efficient software implementation suitable for realtime mobile device execution. A Fischer Vector representation together with a set of linear classifiers are used to categorize food intake. Daily energy expenditure estimation is achieved by using the built-in inertial motion sensors of the mobile device. The performance of the vision-based food recognition algorithm is compared to the current state-of-the-art, showing improved accuracy and high computational efficiency suitable for realtime feedback. Detailed user studies have also been performed to demonstrate the practical value of the software environment.

Humanoid robots have been used in a wide range of applications including entertainment, healthcare, and assistive living. In these applications, the robots are expected to perform a range of natural body motions, which can be either preprogrammed or learnt from human demonstration. This paper proposes a strategy for imitating dynamic walking gait for a humanoid robot by formulating the problem as an optimization process. The human motion data are recorded with an inertial sensor-based motion tracking system (Biomotion+). Joint angle trajectories are obtained from the transformation of the estimated posture. Key locomotion frames corresponding to gait events are chosen from the trajectories. Due to differences in joint structures of the human and robot, the joint angles at these frames need to be optimized to satisfy the physical constraints of the robot while preserving robot stability. Interpolation among the optimized angles is needed to generate continuous angle trajectories. The method is validated using a NAO humanoid robot, with results demonstrating the effectiveness of the proposed strategy for dynamic walking.

Photoplethysmography (PPG) is a well established method of measuring Heart Rate Variability (HRV) and blood oxygen saturation (SpO2) at the fingers, forehead or other areas of the body where pulsatile flow is present. However, obtaining reliable tissue oxygen saturation (StO2) from optical devices is more challenging due to a number of factors including motion and signal-to-noise ratio. Instrumentation of such devices as miniaturised wearable platforms would allow the device to be worn freely by patients in hospitals or at home. The purposes of this paper are to present: 1) a bespoke, low power StO2 sensor, 2) preliminary comparison to a commercially available photospectroscopy and laser Doppler machine (Oxygen 2 See, Medizintechnik, LEA, Germany) using a pressure cuff forearm ischaemia model, and 3) validation of fluid (blood) flow/relative haemoglobin measurements using a novel forearm phantom.

The use of e-AR (ear-worn activity recognition) sensorfor gait pattern estimation has shown promise for a range of health and wellbeing applications. To establish its more detailed quantitative accuracy, an in-shoe pressure measurement system (Parotec) has been used to validate the estimated gait events from the e-AR sensor. Ten healthy adults equipped with Parotec and e-AR systems walked in acorridor of about 15m. The sampling frequency of both systems was set at 100Hz and a manual synchronisation has been performed for subsequent error measurements. The gait events from the e-AR sensor are estimated by using a recently developed method based on singular spectrum analysis and longest common subsequence algorithms [1]. Thecorresponding gait events from the Parotec system are estimated using the ground reaction forces. The upper and lower limits of absolute errors using 95% confidence intervals for heel contact and toe off events obtained as 35.38±3.22ms and 73.05±7.24ms respectively. We furtherprovide a preliminary patient study to demonstrate how the estimated gait events and the gait analysis platform can be used for assessing patients recovering after orthopaedic surgery inside the clinic.

In activity recognition and behaviour profiling studies, wearable inertial sensors are commonly used to monitor the subjects' daily activities. However, the need of carrying the sensing devices in addition to personal belongings may prohibit the widespread use of the technologies. On the other hand, smartphones have become ubiquitous and most smartphones are already equipped with similar inertial sensors. Recent studies have proposed the use of smartphone for quantifying the activity and behaviour of the users. A smartphone based long-term routine profiling system is proposed. To simplify the user interface and facilitate the ubiquitous use of the system, unsupervised and optimized techniques have been developed and integrated into a mobile phone application. By running the application continuously in the background of the phone, the system captures and processes the sensing information to infer the activities of the users, and the results are forwarded to the server for profiling the routines using pattern mining techniques. The proposed system is validated through a study of six users over two weeks. The ability of the proposed system in capturing routine behavior is demonstrated in the results of the study.

The prevalence of obesity worldwide presents a great challenge to existing healthcare systems. There is a general need for pervasive monitoring of the dietary behaviour of those who are at risk of co-morbidities. Currently, however, there is no accurate method of assessing the nutritional intake of people in their home environment. Traditional methods require subjects to manually respond to questionnaires for analysis, which is subjective, prone to errors, and difficult to ensure consistency and compliance. In this paper, we present a wearable sensor platform that autonomously provides detailed information regarding a subject's dietary habits. The sensor consists of a microphone and a camera and is worn discretely on the ear. Sound features are extracted in real-time and if a chewing activity is classified, the camera captures a video sequence for further analysis. From this sequence, a number of key frames are extracted to represent important episodes during the course of a meal. Results show a high classification rate of chewing activities, and the visual log demonstrates a detailed overview of the subject's food intake that is difficult to quantify from manually-acquired food records.

Activity monitoring is important for assessing daily living conditions for elderly patients and those with chronic diseases. Transitions between activities can present characteristic patterns that may be indicative of quality of movement. To detect and analyze transitional activities, a manifold-based approach is proposed in this paper. The proposed method uses a recursive spectral graph-partitioning algorithm to segment transitions in activity. These segments are subsequently mapped to a reference manifold space. Categorization of transitions is performed with the corresponding features in the manifold space. The practical value of the work is demonstrated through data collected under laboratory conditions, as well as patients recovering from total knee replacement operations, demonstrating specific transitions and motion impairment compared to normal subjects.

Abstract Wireless sensor networks enable continuous and reliable data acquisition for real‐time monitoring in a variety of application areas. Due to the large amount of data collected and the potential complexity of emergent patterns, scalable and distributed reasoning is preferable when compared to centralised inference as this allows network wide decisions to be reached robustly without specific reliance on particular network components. In this paper, we provide an overview of distributed inference for both wearable and ambient sensing with specific focus on graphical models—illustrating their ability to be mapped to the topology of a physical network. Examples of research conducted by the authors in the use of ambient and wearable sensors are provided, demonstrating the possibility for distributed, real‐time activity monitoring within a home healthcare environment. Copyright © 2010 John Wiley & Sons, Ltd.

A progressive improvement in gait following knee arthroplasty surgery can be observed during walking and transitional activities such as sitting/standing. Accurate assessment of such changes traditionally requires the use of a gait lab, which is often impractical, expensive, and labour intensive. Quantifying gait impairment following knee arthroplasty by employing wearable sensors allows for continuous monitoring of recovery. This study employed a recognised protocol of activities both pre-operatively, and at regular intervals up to twenty-four weeks post-total knee arthroplasty. The results suggest that a wearable miniaturised ear-worn sensor is potentially useful in monitoring post-operative recovery, and in identifying patients who fail to improve as expected, thus facilitating early clinical review and intervention.

Human back movement estimation is clinically important for assessing patients with back pain. Most current techniques are limited to simple spinal movement angles without consideration of surrounding muscle movement and backplane rotation and torsion. These three dimensional analysis is fraught with difficulties due to the complex nature of the movement and sensor placement. In this paper, a consistent method based on multiple Body Sensor Network (BSN) nodes for the measurement of 3D bending and twist of the back is proposed. In our method, five BSN nodes, each consisting of a three axis accelerometer, a gyroscope and a magnetometer, are placed at the human back. Euler angles are then defined to represent the orientation for human back segments, kinematics analysis is then derived. An unscented Kalman filter (UKF) is deployed to estimate the defined Euler angles. Detailed experimental results have shown the feasibility and effectiveness of the proposed measurement and analysis framework.

Patients' functional recovery at home following surgery may be evaluated by monitoring their activities of daily living. Existing tools for assessing these activities are labor-intensive to administer and rely heavily on recall. This study describes the use of a wireless ear-worn activity recognition sensor to monitor postoperative activity levels continuously using a Bayesian activity classification framework. The device was used to monitor the postoperative recovery of five patients following abdominal surgery. Activity was classified into four groups ranging from very low (level 0) to high (level 3). Overall, patients were found to be undertaking a higher proportion of level 0 activities on postoperative day 1 which was gradually replaced by higher-level activities over the next 3 days. This study demonstrates how a pervasive healthcare technology can objectively monitor functional recovery in the unsupervised home setting. This may be a useful adjunct to existing postoperative monitoring systems.

Activities of daily living are important for assessing changes in physical and behavioural profiles of the general population over time, particularly for the elderly and patients with chronic diseases. Although accelerometers are widely integrated with wearable sensors for activity classification, the positioning of the sensors and the selection of relevant features for different activity groups still pose interesting research challenges. This paper investigates wearable sensor placement at different body positions and aims to provide a framework that can answer the following questions: (i) What is the ideal sensor location for a given group of activities? (ii) Of the different time-frequency features that can be extracted from wearable accelerometers, which ones are most relevant for discriminating different activity types?

The recent maturity of body sensor networks has enabled a wide range of applications in sports, well-being and healthcare. In this paper, we hypothesise that a single unobtrusive head-worn inertial sensor can be used to infer certain biomotion details of specific swimming techniques. The sensor, weighing only seven grams is mounted on the swimmer's goggles, limiting the disturbance to a minimum. Features extracted from the recorded acceleration such as the pitch and roll angles allow to recognise the type of stroke, as well as basic biomotion indices. The system proposed represents a non-intrusive, practical deployment of wearable sensors for swimming performance monitoring.

Determining human intention is a challenging task for establishing affective human robot interaction. The aim of this paper is to provide a vision based framework to achieve a level of understanding about people in an environment before engaging in active communication or interaction. The proposed method combines multiple cues in a Bayesian framework to identify people in the scene and determine potential intentions. To improve the system performance, contextual feedback is used, which allows the Bayesian network to evolve and adjust itself according to the surrounding environment. Our results demonstrate the effectiveness of the technique in dealing with human-robot interaction in a relatively crowded environment.

New approaches to chronic disease management within a home or community setting offer patients the prospect of more individually focused care and improved quality of life. This paper investigates the use of a light-weight ear worn activity recognition device combined with wireless ambient sensors for identifying common activities of daily living. A two-stage Bayesian classifier that uses information from both types of sensors is presented. Detailed experimental validation is provided for datasets collected in a laboratory setting as well as in a home environment. Issues concerning the effective use of the relatively limited discriminative power of the ambient sensors are discussed. The proposed framework bodes well for a multi-dwelling environment, and offers a pervasive sensing environment for both patients and care-takers.

Laparoscopic surgery is a challenging task in minimally invasive surgery, which involves complex instrument control, extensive manual dexterity, and hand-eye coordination. This requires a greater attention to training and skills evaluation. In order to provide a more objective skills assessment method, this paper presents a wireless sensor platform for the capture of laparoscopic hand gesture data and a hidden-Markov-model-based analysis framework for optimal sensor selection and placement. Detailed experimental validation is provided to illustrate how the proposed method can be used to assess surgical performance improvement over repeated training.

On-line surveillance to improve safety and security is a major requirement for the management of public transport networks and other public places. The surveillance task is a complex one involving people, management procedures, and technology. This work describes an architecture that takes into account the distributed nature of the detection processes and the need to allow for different types of devices and actuators. This was part of a major European initiative on intelligent transport systems. Because of the dominant nature of closed circuit television in surveillance, This work describes in detail a computer-vision module used in the system and its particular ability to detect situations of interest in busy conditions. The system components have been implemented, integrated, and tested in real metropolitan railway environments and are considered to be the first step toward providing ambient intelligence in such complex scenarios. Results are presented that not only deal with detection performance, but also on the perception of people who used the system on its effectiveness and potential impact.

Due to its static nature, the inference capability of Bayesian networks (BNs) often deteriorates when the basis of input data varies, especially in video processing applications where the environment often changes constantly. This paper presents an adaptive BN where the network parameters are adjusted in accordance to input variations. An efficient retraining method is introduced for updating the parameters and the proposed network is applied to shadow removal in video sequence processing with quantitative results demonstrating the significance of adapting the network with environmental changes.

In surgery, virtual and augmented reality are increasingly being used as new ways of training, preoperative planning, diagnosis and surgical navigation. Further development of virtual and augmented reality in medicine is moving towards photorealistic rendering and patient specific modeling, permitting high fidelity visual examination and user interaction. This coincides with the current development in computer vision and graphics where image information is used directly to render novel views of a scene. These techniques require extensive use of geometric information about the scene and provide a comprehensive review of the underlying techniques required for building patient specific models with photorealstic rendering. It also highlights some of the opportunities that image based modeling and rendering techniques can offer in the context of minimally invasive surgery.