The advent of machine learning (ML) has significantly advanced the recognition and translation of sign languages, bridging communication gaps for hearing-impaired communities. At the heart of these technologies is data labeling, crucial for training ML algorithms on a huge amount of consistently labeled data to achieve models that generalize well. The adoption of language-agnostic annotations is essential to connect different sign languages, as single-language databases often provide limited lexicon examples, insufficient for training robust ML algorithms. This study critically examines the Hamburg Sign Language Notation System (HamNoSys), which describes the signer’s initial position and body movements, in contrary to the meanings of glosses. Despite HamNoSys’s utility in standardizing transcriptions across various sign languages, our investigation uncovers inconsistencies within HamNoSys that may negatively impact the development of accurate and reliable ML models. By analyzing HamNoSys labels across five sign languages, we identified a lack of standardized annotation procedures and the complexities within HamNoSys that introduce biases and errors. Our findings underscore the urgent need for unified, standardized data annotation guidelines to enhance the accuracy and efficiency of sign language recognition technologies. This research highlights the importance of addressing annotation challenges and advocates for a comprehensive, diversified database to improve the generalization of ML models.

Automatic liver segmentation of Computed Tomography (CT) images is becoming increasingly important. Although there are many publications in this field there is little explanation why certain pre-processing methods were utilised. This paper presents a comparison of the commonly used approach of Hounsfield Units (HU) windowing, histogram equalisation, and a combination of these methods to try to ascertain what are the differences between them and how big the differences are. All experiments were conducted on the LiTS dataset. To achieve comparable and reliable results only one architecture of neural network is used which is U-Net with ResNet34 blocks.

The rate of progress in the field of Artificial Intelligence (AI) and Machine Learning (ML) has significantly increased over the past ten years and continues to accelerate. Since then, AI has made the leap from research case studies to real production ready applications. The significance of this growth cannot be undermined as it catalyzed the very nature of computing. Conventional platforms struggle to achieve greater performance and efficiency, what causes a surging demand for innovative AI accelerators, specialized platforms and purpose-built computes. At the same time, it is required to provide solutions for assessment of ML platform performance in a reproducible and unbiased manner to be able to provide a fair comparison of different products. This is especially valid for Human System Interaction (HSI) systems that require specific data handling for low latency responses in emergency situations or to improve user experience, as well as for preserving data privacy and security by processing it locally. Taking it into account, this work presents a comprehensive guideline on preferred benchmarking criteria for evaluation of ML platforms that include both lower level analysis of ML models and system-level evaluation of the entire pipeline. In addition, we propose a Systematic Taxonomy of Embedded Platforms (STEP) that can be used by the community and customers for better selection of specific ML hardware consistent with their needs for better design of ML-based HSI solutions.

Increased interest in non-contact evaluation of the health state has led to higher expectations for delivering automated and reliable solutions that can be conveniently used during daily activities. Although some solutions for cough detection exist, they suffer from a series of limitations. Some of them rely on gesture or body pose recognition, which might not be possible in cases of occlusions, closer camera distances or impediments that prevent users from performing such movements at all. Others focus on analyzing breath using audio recordings, which cannot be easily applied in crowded or loud spaces. Many of them utilize visible light data which is prone to changing lighting conditions and can lead to various privacy concerns. Taking these into account, we propose to make use of the temporal pixel value changes occurring within specific facial areas. Due to the use of a combination of object detection and signal classification models, our system allows for fully automated classification of breathing anomalies. The benchmark evaluation performed on the newly created thermal cough data set proved the reliability of the introduced solution (precision of cough detection equals 94%). Due to the use of a lightweight deep learning model, the proposed system also has huge practical value, as it can potentially be deployed on edge devices frequently sought out in markets such as autonomous vehicles, drones, smart home or military applications.

Human-system interactions frequently require a retrieval of the key context information about the user and the environment. Image processing techniques have been widely applied in this area, providing details about recognized objects, people and actions. Considering remote diagnostics solutions, e.g. non-contact vital signs estimation and smart home monitoring systems that utilize person's identity, security is a very important factor. Thus, thermal imaging has become more and more popular, as it does not reveal features that are often used for person recognition, i.e. sharp edges, clear changes of pixel values between areas, etc. On the other hand, there are much more visible light data available for deep model training. Taking it into account, person recognition from thermography is much more challenging due to specific characteristics (blurring and smooth representation of features) and small amount of training data. Moreover, when low resolution data is used, features become even less visible, so this problem may become more difficult. This study focuses on verifying whether model trained to extract important facial embedding from RGB images can perform equally well if applied to thermal domain, without additional re-training. We also perform a set of experiments aim at evaluating the influence of resolution degradation by down-scaling images on the recognition accuracy. In addition, we present deep super-resolution (SR) model that by enhancing donw-scaled images can improve results for data acquired in scenarios that simulate real-life environment, i.e. mimicking facial expressions and performing head motions. Preliminary results proved that in such cases SR helps to increase accuracy by 6.5% for data 8 times smaller than original images. It has also been shown that it is possible to accurately recognize even 40 volunteers using only 4 images per person as a reference embedding. Thus, the initial profiles can be easily created in a real time, what is an additional advantage considering a solution setup in a new environment.

This paper focuses on convolution neural network quantization problem. The quantization has a distinct stage of data conversion from floating-point into integer-point numbers. In general, the process of quantization is associated with the reduction of the matrix dimension via limited precision of the numbers. However, the training and inference stages of deep learning neural network are limited by the space of the memory and a variety of factors including programming complexity and even reliability of the system. On the whole the process of quantization becomes more and more popular due to significant impact on performance and minimal accuracy loss. Various techniques for networks quantization have been already proposed, including quantization aware training and integer arithmetic-only inference. Yet, a detailed comparison of various quantization configurations, combining all proposed methods haven't been presented yet. This comparison is important to understand selection of quantization hyperparameters during training to optimize networks for inference while preserving their robustness. In this work, we perform in-depth analysis of parameters in the quantization aware training, the process of simulating precision loss in the forward pass by quantizing and dequantizing tensors. Specifically, we modify rounding modes, input preprocessing, output data signedness, bitwidth of the quantization and locations of precision loss simulation to evaluate how they affect accuracy of deep neural network aimed at performing efficient calculations on resource-constrained devices.

In this paper the implementation of recurrent neural network models for hand gesture recognition on edge devices was performed. The models were trained with 27 hand gestures recorded with the use of a linear optical sensor consisting of 8 photodiodes and 4 LEDs. Different models, trained off-line, were tested in terms of inference time on several mobile devices. The impact of different network topologies (different number of neurons and layers) and different effective sampling frequency of a recorded gesture on the inference time were evaluated. Inference time tests performed on the most effective of available platforms, Samsung Galaxy S8 smartphone, present that recurrent neural networks trained on unprocessed (raw) data from the gesture sensor (1-layer model) execute faster than models trained on differently processed features (2 and 3-layer models): 374ms vs 765ms and 1048ms for data recorded with 100Hz. Also, the reduction of effective sampling frequency allows to shorten the inference time, e.g., for raw data from 374ms at 100Hz to 115ms at 25Hz. Presented results confirm the usability of low complexity interfaces like linear gesture sensor in mobile devices with limited computational capabilities, which allows to consider such sensors in wearable electronic devices e.g. smart glasses.

With the technology advancements in smart home sector, voice control and automation are key components that can make a real difference in people's lives. The voice recognition technology market continues to involve rapidly as almost all smart home devices are providing speaker recognition capability today. However, most of them provide cloud-based solutions or use very deep Neural Networks for speaker recognition task, which are not suitable models to run on smart home devices. In this paper, we compare relatively small Convolutional Neural Networks (CNN) and evaluate effectiveness of speaker recognition using these models on edge devices. In addition, we also apply transfer learning technique to deal with a problem of limited training data. By developing solution suitable for running inference locally on edge devices, we eliminate the well-known cloud computing issues, such as data privacy and network latency, etc. The preliminary results proved that the chosen model adapts the benefit of computer vision task by using CNN and spectrograms to perform speaker classification with precision and recall ~84 % in time less than 60 ms on mobile device with Atom Cherry Trail processor.

The aim of this work was to evaluate the usability of low-cost and low-power components for road condition classification and pedestrian detection in challenging environments. Therefore, deep learning based system dedicated for resource constrained environments is proposed to classify possible road hazards. In the performed experiments we investigated the influence of various factors (lightning conditions, moisture of the road surface and ambient temperature) on the system ability to accurately detect a person on the road. The implemented system was tested on images acquired in different climate zones. Preliminary results show that CNN models with optimized number of parameters can classify thermal road conditions with precision and recall above 70% in most cases, with exception of differentiating lightning conditions. This, in turn, proved system insensitivity to the poor lightning on the road and allows for evaluating road hazards regardless of the time of the day. Pedestrian detection using the model optimized for mobile environments proved to be highly efficient (precision and recall above 95%) both when ambient temperature is lower and higher than human body temperature. System was deployed on Movidius Neural Compute Stick, that allowed for increasing processing throughput (whole pipeline took 0.287±0.028s to process) while conserving energy, what indicates suitability of the solution for implementation in real-time automotive systems.

There is a lack of efficient video data labeling mechanism for real-time applications. Most of the labeling solutions are designed for big data offline video analytics, however emerging real-time applications with video analytics and the data-centric global trend require real-time video analytics with real-time labeling/annotation. In this paper, we present a solution that provides a per frame custom metadata that can be easily encoded and decoded and overlaid with the video frame for labeling the relevant objects/scenes. The presented solution is implemented and tested in a pilot and is leveraging edge computing capabilities to minimize the cost of using the cloud (in terms of latency and additional network resources).

Recently, capabilities of many computer vision tasks have significantly improved due to advances in Convolutional Neural Networks. In our research, we demonstrate that it can be also used for face detection from low resolution thermal images, acquired with a portable camera. The physical size of the camera used in our research allows for embedding it in a wearable device or indoor remote monitoring solution for elderly and disabled people. The benefits of the proposed architecture were experimentally verified on the thermal video sequences, acquired in various scenarios to address possible limitations of remote diagnostics: movements of the person performing a diagnose and movements of the examined person. The achieved short processing time (42.05±0.21ms) along with high model accuracy (false positives -0.43%; true positives for the patient focused on a certain task -89.2%) clearly indicates that the current state of the art in the area of image classification and face tracking in thermography was significantly outperformed.

Recently, facial feature tracking systems have become more and more popular because of many possible use cases. Especially in medical applications location of the face and facial features are very useful. Many researches have presented methods to detect and track facial features in visible light. However, facial feature analysis in thermography may also be very advantageous. Some examples of using infrared imagery in medicine include the estimation of the respiration rate using an analysis of temperature changes in the area below nose region. Moreover, due to technological development small thermal cameras may be embedded into wearable devices, like smart glasses and used to support remote patient monitoring. Therefore, in this paper, we focused on face tracking in low quality thermal images. Especially, we compared four interest points detectors for facial feature tracking in thermal images. All methods were tested for processing time, displacement of detected areas and errors of calculated mean value of pixel intensities in the detected nose region. Finally, we presented a fully automatic system for facial features tracking, which allows to process one frame in about 27.7ms (Harris), 23.9ms (ORB), 19.7ms (SIFT), 27.6ms (SURF) with acceptable accuracy (Harris - 7.2±4.3%, ORB 9.9±2.2%, SIFT 7.0±1.9%, SURF 8.9±2.7%).

In this paper we propose to cast the problem of identification of people, objects or places into an application for smart glasses that decodes information from graphical markers. We focus on analyzing different factors that can have influence on the processes of the automatic recognition of information from a code. The research we present aims at reviewing recognition performances in function of: size of a marker, distance from/to a marker, position of a marker and hardware used to decode information. Additionally, we describe a colorful graphical marker which was created for comparative analysis with existing monochrome codes. We also analyze accuracy of the detection of markers colors under different illumination conditions and different scanning distances. Moreover, considering possible usages of graphical markers, we present a prototype application, which can potentially provide faster access to patients' information. We believe that it may be very useful, because health care have to deal with a lot of various assets that could be identify or classify by using eyewear devices. Our first conclusions are the facts that following factors may have influence on monochrome code recognition (i) distance from the camera, (ii) rotation and slope of code, (iii) quality and size of the code. Distance from the camera should be adapted to the device that is used to scan the code. Under preliminary studies with colorful codes, tested light conditions and distances did not affect the classification of colors.