Vital signs such as heart rate, oxygen saturation, and blood pressure are crucial information for healthcare workers to identify clinical deterioration of ward patients. Currently, medical devices monitor these vital signs and trigger alarms when the vital signs are not in the normal ranges based on predefined thresholds, which suggests the presence of clinical deterioration. However, such threshold-based approach is not robust for patient monitoring. This is because vital signs differ among patients due to human physiology and change across time based on the action performed by a patient. In this work, we want to tackle these problems by building adaptable action-aware vital models. These models can understand the changes in vital signs caused by patient's actions and can be adapted to the normal vital sign ranges of individual patients. Our experimental results show that general vital sign patterns for different actions exist and can be personalized to new patients. Additionally, we investigate the possibility of estimating the initial vital model for an unobserved action using models of observed actions for model personalization. The resulting adaptable action-aware vital models have the potential to improve patient monitoring by reducing false clinical alarms.

Traffic sign detection is crucial for enabling autonomous vehicles to navigate in real-world streets, which must be carried out with high accuracy and in real-time. CNNs have become one of the standard approaches for traffic sign detection research in recent years. The use of CNNs has allowed the development of traffic sign detectors that are capable of achieving prediction accuracies similar to those of human drivers. However, most CNNs do not run in real-time due to the high number of computational operations involved during the inference phase. This hinders the deployment of CNNs in autonomous vehicles despite their high prediction accuracy. In this paper, we explore BNNs to tackle this problem. BNNs binarize the full-precision weights and activations of a CNN, drastically reducing the complexity of the computational operations required for inference, while at the same time maintaining the architectural parameters, as well as spatial dimensions of the input image. This reduces the memory required to run the model and enables faster inference time. We carry out in-depth studies on applying BNNs for traffic sign detection using real-world datasets. We observe an improvement of 11.63 × for normalized compute complexity, while suffering only 3.93 pp in detection accuracy on GTSDB dataset.

Autonomous vehicles need to be able to perceive both the presence and motion of objects in the surrounding environment to navigate in the real world. In this work, we propose to solve the tasks of identifying objects and estimating the corresponding motion by viewing them as a single unified task known as instance flow. Instance flow provides the pixel-wise instance mask of an object and the dense optical flow within it. To achieve this, we extended the state of the art object detection model to include a dense optical flow estimator. The estimator is used to estimate the optical flow for each region of interest only, instead of the entire image. We tested the approach by carrying out experiments on publicly available datasets for autonomous driving research, VKITTI, KITTI and HD1K. Furthermore, we also introduced a new instance flow quality metric to evaluate the instance flow estimation.

We propose an object-centric framework to label and predict human manipulation actions from observations of the object trajectories in 3D space. The goal is to lift the low-level sensor observation to a context specific human vocabulary. The low-level visual sensory input from a depth camera is processed into high-level descriptive action labels using a directed action graph representation. It is built based on the concepts of pre-computed Location Areas (LA), regions within a scene where an action typically occur, and Sector-Maps (SM), reference trajectories between the LAs. The framework consists of two stages, an offline teaching phase for graph generation, and an online action recognition phase that maps the current observations to the generated graph. This graph representation allows the framework to predict the most probable action from the observed motion in real-time and to adapt its structure whenever a new LA appears. Furthermore, the descriptive action labels enable not only a better exchange of information between a human and a robot but they allow also the robots to perform high-level reasoning. We present experimental results on real human manipulation actions using a system designed with this framework to show the performance of prediction and labeling that can be achieved.