Efficiency is paramount in industry and can be vastly improved by driving improvements in procedure design. This study explores the feasibility of using electroencephalography (EEG) and gaze tracking for assessing the quality of procedural design. The study hypothesises that EEG and gaze information can be indicative of the difficulties workers face during procedural tasks and therefore used to identify areas for procedural design improvements. Fifteen participants completed a number of origami tasks, designed to contain problem points predicted to stimulate detectable emotional responses. The analysis of the fixation rate and pupil diameter revealed that participants fixated on steps either directly preceding or following the identified problem points, and pupil size increased during the execution of these steps. EEG analysis included power spectral densities (PSD) and event related potentials (ERP), though ERP was found not to be indicative enough for the purpose of this study . Participants provided feedback on challenging steps, which aligned with predictions. Brain activity patterns while undertaking problematic steps compared to base unstimulated brain activity showed that theta activity increased across the whole brain in 77% of recordings, most prominently in the left temporal region; delta activity increased in 65% of recordings, most prominently in the left temporal region; alpha activity decreased in in the occipital region in 65% of recordings but increased in the left temporal region in 70%; and beta activity increased in the left frontal region in 74% of recordings. These results validated the hypothesis, as they showed clear trends in the reactions to problem points. Finally, a framework is proposed for a procedure problem point identification using EEG and gaze tracking, and recommendations for further research have been outlined.

The growing need for standardised and automated cardiac ultrasound (US) acquisition has driven the integration of deep learning into echocardiographic workflows. While existing deep learning (DL) models have shown promising results in tasks such as view classification and image quality assessment, most of these approaches focus either on differentiating among standard views or grading image quality within a standard view. However, these methods lack the capacity to model the sequential spatial transitions that occur during the acquisition process, limiting their applicability to real-time probe guidance and robotic control. To address this gap, we propose a classification framework designed for the process of acquiring the parasternal long-axis (PLAX) view under a fixed scanning protocol. Based on extensive probe movement experiments across multiple patients, we identified four representative echocardiographic views that appear during the search for the optimal PLAX position. These views correspond to distinct probe positions and orientations and reflect varying levels of image completeness. A dataset of 7,200 annotated images was used to train a ResNet50-based deep network for multi-class classification. The model achieved robust performance with accuracy, sensitivity, specificity, and F1 scores above 89%, and AUC exceeding 97% in patient-level cross-validation. It effectively captures spatially relevant features, distinguishes subtle view differences, and generalizes well to unseen data. The outputs provide interpretable feedback correlating image quality with probe position, enabling real-time scanning assessment. Furthermore, this work introduces a novel problem formulation and multi-class view classification under a fixed acquisition protocol. It provides a foundation for developing the next generation of intelligent US systems. By linking image classification to probe position and orientation, the proposed framework enables real-time feedback that can ultimately support autonomous scanning agents in locating diagnostically optimal cardiac views.

Protecting intellectual property (IP) in the digital age presents significant challenges due to rapid technological advancements and industrial growth. Traditional methods of registering and securing IP are becoming increasingly ineffective. To address these challenges, a more robust system is needed to control access, prevent unauthorized use, and safeguard digital rights. Despite efforts to transition from central registries to encrypted systems, vulnerabilities still exist that can compromise IP security. Therefore, a comprehensive solution must ensure legal use, prevent misuse, and enhance overall IP protection. This study introduces a robust framework designed to prioritize IP security and protection while addressing financial considerations. Our tiered Blockchain-based approach features logically segregated layers governed by smart contracts, which control access based on predefined agreements set by the IP owner. A common application interface (CAI) via smart contracts simplifies common operation with regard to an IP. The decentralized nature of Blockchain technology ensures unassailable trust, availability, and security. Additionally, we employ a flexible off-chain identity verification and storage mechanism for quick access and improved processing capabilities. Financial aspects tied to digital rights are managed through Blockchain's oracle services, ensuring seamless integration and management. Our integrated solution provides a reliable platform for IP protection, validated through thorough performance evaluations across diverse real-world scenarios. This framework demonstrates significant improvements in efficiency, security, and cost-effectiveness compared to traditional IP protection methods. By leveraging Blockchain's immutable ledger and decentralized network, we enhance the traceability and accountability of IP transactions, reinforcing legal compliance and reducing disputes. Ultimately, this approach ensures that IP is safeguarded, valued, and shared in a manner that benefits creators, consumers, and society as a whole. The rigorous analysis showed significant enhancements in process optimization, technology adoption, efficiency, and cost reduction compared to traditional IP rights protection practices.

Recognizing human intentions is a key challenge in human-robot interaction research. Much of the current work in this area centers on identifying human intentions within specific activities, often relying on a limited set of features. In contrast, this paper introduces a more versatile framework for intention recognition and introduces a novel model: the Spatial-Temporal Graph Attention Informer Neural Network (STGAIN). To recognize intentions, this model leverages spatial relationships between humans and objects in different scenes, along with their temporal evolution. In addition, to address an existing research gap, this research developed a new dataset called Dynamic Scene Graph (DSG) with representative dynamic relationships, derived from 471 videos covering 20 categories of human intentions. This dataset represents people and objects in different scenes, and the relationships between them. The model was tested rigorously at different points in the videos to track how the scenes evolved and to assess prediction accuracy, comparing the results to a range of advanced algorithms. Our findings clearly demonstrate that STGAIN outperforms these models, showcasing its potential for advanced human intention recognition applications. This model represents a significant advance toward creating more human-centered robots, capable of understanding and adapting to human intentions in real-world situations.

Over the past decade, automating ultrasound scanning has been the subject of intense research. However, training a robot to perform automated ultrasound examinations requires a substantial corpus of training data. Traditionally, researchers have sought to obtain such data either through publicly available datasets or by engaging professional sonographers in experiments aimed at dataset generation. The former approach often yields incomplete datasets insufficient for specialized research objectives, while the latter entails logistical challenges, including the necessity for frequent manual experimentation and ready access to medical professionals. Therefore, the acquisition of a comprehensive and suitable dataset remains an essential yet formidable challenge. Here, we propose a novel framework for achieving the automated acquisition of cardiac ultrasound datasets by controlling robot behaviour using a digital twin within a simulated environment. This framework consists of two modules: physical and virtual. Within the virtual simulation module, diverse body models of varying dimensions can be inputted, enabling the planning of robot arm path and ultrasound scanning manoeuvrers. Then the physical robot arm clones the actions of the robot in the simulation environment and updates its current state in the virtual module. The proposed framework was used to collect 43,000 cardiac ultrasound images from 8 patients with different pathologies and 1 healthy individual using a KUKA LBR Med robot and Intelligent Ultrasound Simulator. It is also expected to be feasible for a real-person dataset collection.

Human activity recognition, as a significant branch of artificial intelligence, requires increasingly generalized and precise methodologies due to growing demands. Therefore, this paper proposes a context-based method for recognising indoor human activities, which interlinks indoor human activities with interactions with objects, making the contextual relationship between humans and objects particularly crucial. In addition, this research has developed a new dynamic graph dataset based on publicly available video datasets and their associated descriptive scripts, instantiating the relationships between humans and objects. A novel architecture for human activity recognition is developed in this research. This architecture utilizes graph neural networks and self-attention mechanisms to learn the significance of the interactions between humans and objects and capture the relationships between video frames on a temporal level. The results demonstrate that the classification accuracy reaches 0.86 and it also performs better than other current advanced algorithms STGAT and STGCN. It is noteworthy that the approach also effectively reduces ambiguity in activity recognition.

Electroencephalography (EEG) and eye tracking devices are used in this study to assess the capability of such systems to measure emotional responses in a healthcare-related environment. Experiments are conducted in which positive, negative and neutral stimuli are presented to participants and data is captured from both systems simultaneously. Images from the International Affective Picture System (IAPS) are employed to trigger standardised emotion states and calibrate the experiment, whilst images from a medical drama are used to provide hospital-based stimuli. It is found that EEG and eye tracking can successfully indicate emotion features, with the EEG data providing better visualisation, whilst eye metrics are more meaningful with statistics. Both devices show that the emotional responses to hospital-based images differ to the responses from standardised images. Greater variation between participants in the hospital-based stimuli indicates that personal experiences from healthcare related events can influence emotional responses to related stimuli.

This paper proposes Context-Sensitive Behaviors for Robots (CSBR), a method for generating diverse behaviors for robots in indoor environments based on five personality traits. This method is based on a novel model developed in this work that reacts to a synthetic genome that defines the personality of the robot. The model functions return different answers and reactions, depending on a given spoken request. The responses of the robot included spoken answers, facial animations, gestures, and actions. The novelty of this method lies in its capacity to adapt the behavior of the robot according to the context of the request. Moreover, the model is scalable since its functions not only return spoken answers but also physical responses, such as opening a gripper, saying hello with gestures, or animating a face that represents an emotion according to the context. Changes in the parameters of the synthetic genome produce different behaviors. By defining different synthetic genomes, robots can adapt to different people's moods. In this work, we introduce two scenarios for human–robot interaction in two domestic environments (house and office) through spoken requests from a human user. We implemented our method in Care-O-Bot 4 and defined three synthetic genomes to produce three behaviors: friendly, detached, and hostile. In the considered scenarios, we asked the robot the same set of requests for every synthetic genome. Not only did Care-O-Bot 4 answer according to its personality, but it also proved that our method produces different behaviors. For these scenarios, we assume that the given request includes its connotation. Since our method has characteristics influenced by context, we show that the robot's behavior changed according to the human mood and the environment.

In human-robot interaction, understanding human intention is important to smooth interaction between humans and robots. Proactive human-robot interactions are the trend. They rely on recognising human intentions to complete tasks. The reasoning is accomplished based on the current human state, environment and context, and human intention recognition and prediction. Many factors may affect human intention, including clues which are difficult to recognise directly from the action but may be perceived from the change in the environment or context. The changes that affect human intention are the triggers and serve as strong evidence for identifying human intention. Therefore, detecting such changes and identifying such triggers are the promising approach to assist in human intention recognition. This paper discusses the current state of art in human intention recognition in human-computer interaction and illustrates the importance of context change and triggers for human intention recognition in a variety of examples.

This paper presents the details of a sub-system developed to address coordination between a serial manipulator robot (machining) and SwarmItFIX robot (fixturing) for a sheet metal drilling process. A heterogeneous multi-robot coordination methodology that has already been demonstrated to be successful in a milling process has been further enhanced here to make it suitable for a drilling process. For the convergence of joint angles in the trajectory planning of the serial manipulator robot, an optimisation-based approach is proposed. The velocity of the tool center point (TCP) is considered to be constant throughout, as it improves the quality of the machining. The SwarmItFIX robot abides by a revised five-step locomotion strategy to traverse between any two support locations. A new time plan that ensures multi-robot coordination has also been proposed in this work. The proposed method has been tested with three different drilling patterns, and the results show that the proposed method computes the trajectory of the serial manipulator, support locations of the SwarmItFIX and locomotion sequence of the base agent accurately.

With the rise of Industry 4.0 technological trends, there is a growing tendency in manufacturing automation towards collaborative robots. Human-robot collaboration (HRC) is motivated by the combination of complementary human and robot skills and intelligence, which can increase productivity, flexibility and adaptability. However, it is still challenging to achieve safe and efficient human-robot collaborative systems due to the dynamics of human presence, uncertainties in the dynamic environment, and the need for adaptability. Such uncertainties could relate to the human-robot capabilities and availability, parts positioning, unexpected obstacles, etc. This paper develops time-based simulations and event-based simulations to model and analyse the dynamic factors in human-robot collaboration systems. The novelty of this work is the systematic modelling and analysis of dynamic factors in HRC manufacturing scenarios through the development of digital simulations of human-robot collaboration scenarios while considering the dynamic nature of humans and environments. A real-world industrial case study was redesigned into a collaborative workstation. The simulated scenario is developed using the software called Tecnomatix Process Simulate, which can help to visualise the dynamic factors and analyse the impact of the factors on the HRC. The simulation illustrates and analyses possible uncertainties in human-robot industrial collaborative workstations, which can contribute to the future design of HRC industrial workstations and the optimisation of productivity.

Multistep tasks, such as block stacking or parts (dis)assembly, are complex for autonomous robotic manipulation. A robotic system for such tasks would need to hierarchically combine motion control at a lower level and symbolic planning at a higher level. Recently, reinforcement learning (RL)-based methods have been shown to handle robotic motion control with better flexibility and generalizability. However, these methods have limited capability to handle such complex tasks involving planning and control with many intermediate steps over a long time horizon. First, current RL systems cannot achieve varied outcomes by planning over intermediate steps (e.g., stacking blocks in different orders). Second, the exploration efficiency of learning multistep tasks is low, especially when rewards are sparse. To address these limitations, we develop a unified hierarchical reinforcement learning framework, named Universal Option Framework (UOF), to enable the agent to learn varied outcomes in multistep tasks. To improve learning efficiency, we train both symbolic planning and kinematic control policies in parallel, aided by two proposed techniques: 1) an auto-adjusting exploration strategy (AAES) at the low level to stabilize the parallel training, and 2) abstract demonstrations at the high level to accelerate convergence. To evaluate its performance, we performed experiments on various multistep block-stacking tasks with blocks of different shapes and combinations and with different degrees of freedom for robot control. The results demonstrate that our method can accomplish multistep manipulation tasks more efficiently and stably, and with significantly less memory consumption.

In human-robot teaming, one of the crucial keys for the team’s success is that the robot and human teammates can collaborate accordingly in a coordinated manner. Each teammate should be aware of what the other teammate is going to perform and likely to need. In this context, a robot is expected to understand human teammate intention and performance and explain its actions and decisions and its rationale to its teammate. In addition, the capability to model the expectation of a human teammate empowers the robot to collaborate with human understandably and expectedly, leading to effective teaming. Through forming mental modelling, the robot can understand the impact of its own behaviour on the mental model of the human. In addition, the desirable traits in human-robot teaming, including fluent behaviour, adaptability, trust-building, effective communication, and explainability, can be achieved through mental modelling. In this work, we introduce a scenario for human-robot teaming considering all the five desirable traits in teaming with the main focus on explainability and effective communication. Using a general model reconciliation, the expectation of the human teammate of the robot can be modelled, and the explanation can be generated. In a considered scenario including Care-O-bot 4 service robot and a human teammate, we assume that the robot detects the human’s current task (analysing his body gesture) and predicts his following action and his expectation from the robot. In a reciprocal interdependence task, the robot coordinates his behaviour and acts accordingly by picking up the relevant tool. Through explanation and communication robot further offers the outcome of his decision to the human teammate and adapts its action by handing the tool to the human upon his desire.

Human activity recognition using wearable and mobile devices is used for decades to monitor humans’ daily behaviours. In recent years as smartphones being widely integrated into our daily lives, the use of smartphone’s built-in sensors in human activity recognition has been receiving more attention, in which smartphone accelerometer plays the main role. However, in comparison to the standard machine, when developing human activity recognition using a smartphone, the limitations such as processing capability and energy consumption should be taken into consideration, and therefore, a trade-off between performance and computational complexity should be considered. In this paper, we shed light on the importance of feature selection and its impact on simplifying the activity classification process, which enhances the computational complexity of the system. The novelty of this work is related to identifying the most efficient features for the detection of each individual activity uniquely. In an experimental study with human users and using different smartphones, we investigated how to achieve an optimal feature set, using which the system complexity can be decreased while the activity recognition accuracy remains high. For that, in the considered scenario, we instructed the participants to perform different activities, including static, dynamic, going up and down the stairs, and walking fast and slow while freely holding a smartphone in their hands. To evaluate the obtained optimal feature set implementing two major classification algorithms, the decision tree and the Bayesian network, we investigated activity recognition accuracy for different activities. We further evaluated the optimal feature set by comparing the performance of the activity recognition system using the optimal feature set and three feature sets taken from the state-of-the-art. The experimental results demonstrated that replacing a large number of conventional features with an optimal feature set has only a negligible impact on the overall activity recognition system performance while it can significantly decrease the system’s complexity, which is essential for smartphone-based systems.

The increasing number of dementia patients is one of the major social problems in Japan. Early detection and prevention of dementia is important. Many welfare facilities use check tests to measure the progression of dementia. However, some elderly people can be very nervous about assessment tests. In addition, the evaluation tests should be conducted regularly to assess the cognitive function of the patient over time, which can be a huge burden for medical and care providers. On the other hand, research papers have recently reported that it is possible to measure cognitive functions by focusing on brain functions and eye movements. In this paper, the authors aimed to develop a new dementia evaluation system to reduce the burden on medical staff and elderly persons. As the first step of this project, we focused on eye movement and employed a simple puzzle game to collect a patient’s eye movement. Because of COVID-19, we could not conduct experiments at care houses; instead, we conducted a preliminary experiment with healthy subjects and collected eye movement data during the puzzle game.

Activities of daily living (ADLs) refer to the activities performed by individuals on a daily basis and are the indicators of a person's habits, lifestyle, and wellbeing. Consequently, learning an individual's ADL daily routines has significant value in the healthcare domain. Specifically, ADL recognition and inter-ADL pattern learning problems have been studied extensively in the past couple of decades. However, discovering the patterns of ADLs performed in a day and clustering them into ADL daily routines has been a relatively unexplored research area. In this paper, a self-organizing neural network model, called the Spatiotemporal ADL Adaptive Resonance Theory (STADLART), is proposed for learning ADL daily routines. STADLART integrates multimodal contextual information that involves the time and space wherein the ADLs are performed. By encoding spatiotemporal information explicitly as input features, STADLART enables the learning of time-sensitive knowledge. Moreover, a STADLART variation named STADLART-NC is proposed to normalize and customize ADL weighting for daily routine learning. A weighting assignment scheme is presented that facilitates the assignment of weighting according to ADL importance in specific domains. Empirical experiments using both synthetic and real-world public data sets validate the performance of STADLART and STADLART-NC when compared with alternative pattern discovery methods. The results show that STADLART could cluster ADL routines with better performance than baseline algorithms.

Nowadays, smartphones are gradually being integrated in our daily lives, and they can be considered powerful tools for monitoring human activities. However, due to the limitations of processing capability and energy consumption of smartphones compared to standard machines, a trade-off between performance and computational complexity must be considered when developing smartphone-based systems. In this paper, we shed light on the importance of feature selection and its impact on simplifying the activity classification process which enhances the computational complexity of the system. Through an in-depth survey on the features that are widely used in state-of-the-art studies, we selected the most common features for sensor-based activity classification, namely conventional features. Then, in an experimental study with 10 participants and using 2 different smartphones, we investigated how to reduce system complexity while maintaining classification performance by replacing the conventional feature set with an optimal set. For this reason, in the considered scenario, the users were instructed to perform different static and dynamic activities, while freely holding a smartphone in their hands. In our comparison to the state-of-the-art approaches, we implemented and evaluated major classification algorithms, including the decision tree and Bayesian network. We demonstrated that replacing the conventional feature set with an optimal set can significantly reduce the complexity of the activity recognition system with only a negligible impact on the overall system performance.

This paper introduces a new research area called Explainable Robotics, which studies explainability in the context of human-robot interactions. The focus is on developing novel computational models, methods and algorithms for generating explanations that allow robots to operate at different levels of autonomy and communicate with humans in a trustworthy and human-friendly way. Individuals may need explanations during human-robot interactions for different reasons, which depend heavily on the context and human users involved. Therefore, the research challenge is identifying what needs to be explained at each level of autonomy and how these issues should be explained to different individuals. The paper presents the case for Explainable Robotics using a scenario involving the provision of medical health care to elderly patients with dementia with the help of technology. The paper highlights the main research challenges of Explainable Robotics. The first challenge is the need for new algorithms for generating explanations that use past experiences, analogies and real-time data to adapt to particular audiences and purposes. The second research challenge is developing novel computational models of situational and learned trust and new algorithms for the real-time sensing of trust. Finally, more research is needed to understand whether trust can be used as a control variable in Explainable Robotics.

Increasing the number of dementia patients is one of the big social problems in Japan. Early detection and prevention of dementia are essential. Medical specialists and care workers usually use some dementia evaluation methods. However, some elderly persons often become very nervous about the evaluation tests. Also, these evaluation tests should be conducted continually to assess a patient’s cognitive function. On the other hand, recently, research papers on the relationship between simple human actions and a patient’s cognitive function have been reported. The authors focused on hand dexterity and movement of both hands because these factors have some relationships to cognitive functions. We developed a new recreational system to evaluate hand dexterity and activity of both hands to assess a subject’s cognitive functions. Also, the authors discussed the relationship between the extracted features and a subject’s cognitive functions. As a result of experiments, we determined the feature descriptor(s) about hand motion to estimate and evaluate a patient’s cognitive function.

Although the concept of user experience includes two key aspects, experience of meaning (usability) and experience of emotion (affect), the empirical work that measures both the usability and affective aspects of user experience is currently limited. This is particularly important considering that affect could significantly influence a user's perception of usability. This paper uses image schemas to quantitatively and systematically evaluate both these aspects. It proposes a method for evaluating user experience that is based on using image schemas, sentiment analysis, and computational semantics. The aim is to link the sentiments expressed by users during their interactions with a product to the specific image schemas used in the designs. The method involves semantic and sentiment analysis of the verbal responses of the users to identify (i) task-related words linked to the task for which a certain image schema has been used and (ii) affect-related words associated with the image schema employed in the interaction. The main contribution is in linking image schemas with interaction and affect. The originality of the method is twofold. First, it uses a domain-specific ontology of image schemas specifically developed for the needs of this study. Second, it employs a novel ontology-based algorithm that extracts the image schemas employed by the user to complete a specific task and identifies and links the sentiments expressed by the user with the specific image schemas used in the task. The proposed method is evaluated using a case study involving 40 participants who completed a set task with two different products. The results show that the method successfully links the users' experiences to the specific image schemas employed to complete the task. This method facilitates significant improvements in product design practices and usability studies in particular, as it allows qualitative and quantitative evaluation of designs by identifying specific image schemas and product design features that have been positively or negatively received by the users. This allows user experience to be assessed in a systematic way, which leads to a better understanding of the value associated with particular design features.

A clock drawing test (CDT) is a neurological test used for the assessment of cognitive impairment based on sketches of a clock completed by a patient. Usually, a medical expert assesses the sketches to discover any deficiencies in the cognitive processes of the patient. More recently, automatic tools for assessing such tests have been developed. However, the problem of automatic interpretation of clock drawings, especially those sketched by people with cognitive impairment, is not fully solved, and in more difficult cases, the automatic systems have to revert to the help of human assessors in labelling the sketched objects forming the clock drawing. Moreover, the labelling of the sketched objects could be more reliable if prior knowledge of the expected CDT sketch structure and human reasoning could be integrated into the drawing interpretation system. This paper proposes a novel CDT sketch interpretation system, which represents the prior knowledge of the CDT structure by using ontology and integrating human reasoning through a fuzzy inference engine. The combination of the above technologies fuses multiple sources of information concerning the sketch structure and the visual appearance of the sketched objects whilst dealing with the interpretation uncertainty inherent to CDT sketches. The proposed CDT interpretation system is evaluated using two CDT data sets. The first data set consists of 65 drawings made by healthy people, while the second set contains 100 drawings reproduced from the drawings of dementia patients to simulate the kind of challenging sketches the system may have to work with. The evaluation analysis shows an improved interpretation performance of the proposed system in comparison with the classical approach, which does not receive benefits from the prior knowledge of the CDT sketch structure or simulated human reasoning.

Early detection and diagnosis of Traumatic Brain Injury (TBI) could reduce significantly the death rate and improve the quality of life of the people affected if emergency services are equipped with tools for TBI diagnosis at the place of the accident. This problem is addressed here by proposing a portable decision support system called EmerEEG, which is based on Quantitative Electroencephalography (qEEG). The contributions of the paper are the proposed system concept, architecture and decision support for TBI diagnosis. By the virtue of its easily operable mobile system, the proposed solution for emergency TBI diagnosis provides valuable decision support at a very early stage after an accident, thereby enabling a short response time in critical situations and better prospects for the people affected.

Traumatic Brain Injury (TBI) is recognized as an important cause of death and disabilities after an accident. The availability a tool for the early diagnosis of brain dysfunctions could greatly improve the quality of life of people affected by TBI and even prevent deaths. The contribution of the paper is a process including several methods for the automatic processing of electroencephalography (EEG) data, in order to provide a fast and reliable diagnosis of TBI. Integrated in a portable decision support system called EmerEEG, the TBI diagnosis is obtained using discriminant analysis based on quantitative EEG (qEEG) features extracted from data recordings after the automatic removal of artifacts. The proposed algorithm computes the TBI diagnosis on the basis of a model extracted from clinically-labelled EEG records. The system evaluations have confirmed the speed and reliability of the processing algorithms as well as the system's ability to deliver accurate diagnosis. The developed algorithms have achieved 79.1% accuracy in removing artifacts, and 87.85% accuracy in TBI diagnosis. Therefore, the developed system enables a short response time in emergency situations and provides a tool the emergency services could base their decision upon, thus preventing possibly miss-diagnosed injuries.

The clock drawing test (CDT) is a standard neurological test for detection of cognitive impairment. A computerised version of the test promises to improve the accessibility of the test in addition to obtaining more detailed data about the subject's performance. Automatic handwriting recognition is one of the first stages in the analysis of the computerised test, which produces a set of recognized digits and symbols together with their positions on the clock face. Subsequently, these are used in the test scoring. This is a challenging problem because the average CDT taker has a high likelihood of cognitive impairment, and writing is one of the first functional activities to be affected. Current handwritten digit recognition system perform less well on this kind of data due to its unintelligibility. In this paper, a new system for numeral handwriting recognition in the CDT is proposed. The system is based on two complementary sources of data, namely static and dynamic features extracted from handwritten data. The main novelty of this paper is the new handwriting digit recognition system, which combines two classifiers—fuzzy k-nearest neighbour for dynamic stroke-based features and convolutional neural network for static image- based features, which can take advantage of both static and dynamic data. The proposed digit recognition system is tested on two sets of data: first, Pendigits online handwriting digits; and second, digits from the actual CDTs. The latter data set came from 65 drawings made by healthy people and 100 drawings reproduced from the drawings by dementia patients. The test on both data sets shows that the proposed combination system can outperform each classifier individually in terms of recognition accuracy, especially when assessing the handwriting of people with dementia.

Additive layer manufacturing is emerging as the next generation in part manufacture. It is being adopted by aerospace, tool making, dental and medical industries to produce and develop new conceptual designs and products due to its speed and flexibility. It has been noted that parts produced using additive layer manufacturing are not to a consistent quality. Variations have been recorded showing inadequate control over dimensional tolerances, surface roughness, porosity, and other defects in built parts. It is, however, possible to control these variables using real-time processes that currently lack adequate process measurement methods. This paper identifies process variation and lists parameters currently being recorded during a commercial additive manufacture (AM) machine build process. Furthermore, it examines correlations between manufactured parts and real time build variations.

Increasing the number of elderly persons who have dementia is one of the severe social problems. In Japan, the Ministry of Health, Labor and Welfare expects that the number of dementia patients will be around 5 million in 2025. It is also easily estimated that they require various living supports. Therefore, early detection and prevention of dementia are important. The authors have been developing a new system for quantitative and accurate evaluation of dementia. The basic concept of our system is evaluating a patient's dementia types and progression without awareness. To realize this, we are now developing the system using daily conversations, drawings, facial expressions and so on. In this paper, we focused on Clock Drawing Test (CDT) and proposed a dementia evaluation method for CDT. In the proposed method, Weighted Direction Index Histogram Method was used to extract features from given images, and Support Vector Machine (SVM) detected dementia cases from them. As a result of evaluation experiments, the proposed method could detect 97.1% of dementia cases correctly.

Trademarks are signs of high reputational value. Thus, they require protection. This paper studies conceptual similarities between trademarks, which occurs when two or more trademarks evoke identical or analogous semantic content. This paper advances the state-of-the-art by proposing a computational approach based on semantics that can be used to compare trademarks for conceptual similarity. A trademark retrieval algorithm is developed that employs natural language processing techniques and an external knowledge source in the form of a lexical ontology. The search and indexing technique developed uses similarity distance, which is derived using Tversky's theory of similarity. The proposed retrieval algorithm is validated using two resources: a trademark database of 1400 disputed cases and a database of 378 943 company names. The accuracy of the algorithm is estimated using measures from two different domains: the R-precision score, which is commonly used in information retrieval and human judgment/collective human opinion, which is used in human-machine systems.

The Internet is used by more than two billion customers around the world and is expected to serve as a global platform for interconnecting cyber-physical objects that form the Internet of Things (IoT). Within the next decade, traffic demands are expected to increase a thousand-fold. This challenge can be addressed by introducing and expanding heterogeneous wireless technologies, which provide higher network capacity, wider coverage and higher quality of service (QoS). However, the heterogeneity and complexity of these networks are a major challenge for traditional control and management systems. Therefore, there is a need for self-manageable and self-configurable networks that support the data produced by the different IoT devices and provide opportunities for data analytics. In this work, a cognitive network framework is proposed, in which the network protocol stack is integrated with a semantic system. The proposed framework provides the bases for building smart networks that observe data from different layers in the network protocol stack and represents it in a hierarchical structure in a knowledge base. The framework employs an ontology that provides an abstraction model for the different heterogeneous wireless devices. The ontology determines the relationships between technology-dependent parameters in the network protocol stack and enables, through the use of inferences, the utilization of the observed data from the network. The use of a cognitive network framework with the network protocol stack allows adding ontologies to describe the data, a solution which could solve the problem of analysing, searching or visualising data.

The Clock Drawing Test (CDT) is an inexpensive and effective measure for early detection of cognitive impairment in the elderly, which is important for timely diagnosis and initiation of appropriate treatment. Currently, medical experts assess the drawings based on their judgement and a number of available scoring systems. An automatic system for assessment of CDT drawings would simultaneously decrease the waiting time for a specialist appointment and improve accessibility of the test to the patients. Published research has only started to address the problem of automatic assessment of CDT drawings and existing systems require user intervention during the segmentation of the CDT drawing into its composing parts, such as numbers and clock hands. In this paper, a new set of temporal and spatial features automatically extracted from the CDT data acquired using a graphics tablet is proposed. Consequently, a Support Vector Machine (SVM) classifier is employed to segment the CDT drawings into their elements, such as numbers and clock hands, on the basis of the extracted features. The proposed algorithm is tested on two data sets, the first set consisting of 65 drawings made by healthy people, and the second consisting of 100 drawings reproduced from actual drawings of dementia patients. The test on both data sets shows that the proposed method outperforms the current state-of-the-art method for CDT drawing segmentation.

Recent advances in autonomous vehicle technology pose an important problem of automatic risk assessment in road scenes. This article addresses the problem by proposing a novel ontology tool for assessment of risk in unpredictable road traffic environment, as it does not assume that the road users always obey the traffic rules. A framework for video-based assessment of the risk in a road scene encompassing the above ontology is also presented in the paper. The framework uses as input the video from a monocular video camera only, avoiding the need for additional sometimes expensive sensors. The key entities in the road scene (vehicles, pedestrians, environment objects etc.) are organised into an ontology which encodes their hierarchy, relations and interactions. The ontology tool infers the degree of risk in a given scene using as knowledge video-based features, related to the key entities. The evaluation of the proposed framework focuses on scenarios in which risk results from pedestrian behaviour. A dataset consisting of real-world videos illustrating pedestrian movement is built. Features related to the key entities in the road scene are extracted and fed to the ontology, which evaluates the degree of risk in the scene. The experimental results indicate that the proposed framework is capable of assessing risk resulting from pedestrian behaviour in various road scenes accurately.

Recently, the number of elderly persons with dementia has been increasing. In the past, we proposed a dementia evaluation system using daily conversations and developed the system with a conversational robot. However, the current system is not ready for practical use because it can only evaluate time/geographical orientation and short-term memory, and some methods to evaluate other orientations and functions is required as well. In this paper, we discuss a new dementia evaluation system using not only daily conversations but also drawing tests. The authors employed a Clock Drawing Test (CDT) as a new dementia evaluation test and implemented it in a tablet device. This paper discusses a feature extraction and recognition method to distinguish normal cases from dementia cases. After evaluation experiments, the proposed method could recognize 87.6% of the clock drawing images.

Feature selection is used in many application areas relevant to expert and intelligent systems, such as data mining and machine learning, image processing, anomaly detection, bioinformatics and natural language processing. Feature selection based on information theory is a popular approach due its computational efficiency, scalability in terms of the dataset dimensionality, and independence from the classifier. Common drawbacks of this approach are the lack of information about the interaction between the features and the classifier, and the selection of redundant and irrelevant features. The latter is due to the limitations of the employed goal functions leading to overestimation of the feature significance. To address this problem, this article introduces two new nonlinear feature selection methods, namely Joint Mutual Information Maximisation (JMIM) and Normalised Joint Mutual Information Maximisation (NJMIM); both these methods use mutual information and the ‘maximum of the minimum’ criterion, which alleviates the problem of overestimation of the feature significance as demonstrated both theoretically and experimentally. The proposed methods are compared using eleven publically available datasets with five competing methods. The results demonstrate that the JMIM method outperforms the other methods on most tested public datasets, reducing the relative average classification error by almost 6% in comparison to the next best performing method. The statistical significance of the results is confirmed by the ANOVA test. Moreover, this method produces the best trade-off between accuracy and stability.

Intuitive interaction is an important aspect of usability in interface design. This paper contributes to the research in this area by proposing a conceptual framework for evaluating intuitive interaction based on image schemas. The framework comprises four phases: goal identification, image schemas extraction, analysis and assessment. It quantifies intuitive interaction by comparing the image schemas envisaged by the designer of a product with those used by its users. The proposed framework is evaluated through a study involving 42 participants completing a set task with a product. The study identified the image schemas, which were correctly used in accordance with the designer's intent and those that were incorrectly used and contributed to the difficulties that many participants experienced. The inter-rater reliability and empirical validity were examined. The proposed framework provides a structured approach to usability testing by enabling both quantitative and qualitative evaluation of intuitive interaction.

Generating plans in real world environments by mobile robot planner is a challenging task due to the uncertainty and environment dynamics. Therefore, task-planning should take in its consideration these issues when generating plans. Semantic knowledge domain has been proposed as a source of information for deriving implicit information and generating semantic plans. This paper extends the Semantic-Knowledge Based (SKB) plan generation to take into account the uncertainty in existing of objects, with their types and properties, and proposes a new approach to construct plans based on probabilistic values which are derived from Markov Logic Networks (MLN). An MLN module is established for probabilistic learning and inferencing together with semantic information to provide a basis for plausible learning and reasoning services in supporting of robot task-planning. In addition, an algorithm has been devised to construct MLN from semantic knowledge. By providing a means of modeling uncertainty in system architecture, task-planning serves as a supporting tool for robotic applications that can benefit from probabilistic inference within a semantic domain. This approach is illustrated using test scenarios run in a domestic environment using a mobile robot.

Many wireless devices employ multi-rate techniques to improve network performance. However, despite the significant amount of research aimed at dynamically adjusting the transmission rate, the majority of this effort considers neither the competing nodes in wireless mesh networks nor the congestion in the nodes. This work employs distributed intelligent agents to observe the surrounding environment in order to dynamically adjust the individual node transmission rates. Reinforcement learning is employed to control the way each node updates its transmission rate based on the transmission rate of the adjacent node as well as the traffic load. This work is validated through extensive simulations that compare the proposed model with three of the most widely cited schemes. The results indicate significant improvement in system throughput.

Trademarks are visual symbols with high reputational value, which requires protection. This paper proposes an algorithm to retrieve phonetically similar trademarks that can be used as a means for supporting trademark examination during the registration process. The algorithm employs a phonology based string similarity algorithm together with a typography mapping and token rearrangement to compute a phonetic similarity between trademarks. The trademark phonetic similarity score is then computed from the employed phonetic similarity algorithm. The proposed algorithm advances the state-of-the-art in trademark retrieval by providing a mechanism to compare trademarks with special characters or symbols phonetically. The proposed algorithm is tested on 1,400 trademarks obtained from real court cases between 1999 and 2012. The proposed algorithm improves the R-precision score by 14% and 17% compared with two state-of-the-art methods.

Dementia is a syndrome caused by a chronic or progressive disease of the brain, which affects memory, orientation, thinking, calculation, learning ability and language. The Clock Drawing Test (CDT) and Mini Mental State Examination (MMSE) are well-known cognitive assessment tests. A known obstacle to the wider usage of the CDT assessments is the scoring and interpretation of the results. This paper introduces a novel cascade CDT classifier, which can help in the diagnosis of three stages of dementia. The data used in this research are 604 clock drawings produced by patients and healthy individuals. The study employs 47 visual features, which are selected following a comprehensive analysis of the available data and the most common CDT scoring systems reported in the medical literature. These features are used to build a new digitized dataset needed to train and validate the proposed classifier. The results show significant improvement of 6.8% in differentiating between three levels of dementia (normal/functional, mild cognitive impairment/mild dementia, and moderate/severe dementia) when compared to a single stage classifier. In particular, the results show classification accuracy of over 89% when discriminating between normal and abnormal conditions only.

Due to its high efficiency, the Iterative Closest Point (ICP) algorithm has become a popular choice in computer vision and robotics for the registration of point cloud data sets when the point-to-point correspondences are unknown. Its generalization for articulated structures, although possible through a joint optimization of all pose parameters, is challenging as it is necessary to solve a non-closed form. It also suffers heavily from the local minima problem. A number of proposed Articulated ICP (AICP) algorithms circumvent the problem of the non-closed form solution and offer an efficient alternative. However, they still exhibit an increased tendency, caused by the local minima, to converge to an incorrect pose. Typically, the above problem manifests itself after a transient disturbance in the convergence, such as an occlusion which causes an increase in the point-to-point association distances between the model and the data. In this paper, we propose an extension to the AICP algorithm that benefits from the efficiency of ICP as well as avoids its problems by using global pose optimization elements to guide the convergence process to the correct pose. The proposed approach is to merge adaptively the joint adjustments computed by AICP with the adjustments needed for a number of key points to reach their respective target positions, identified by a local feature descriptor search. Experiments show that the proposed Global-Local Articulated ICP algorithm exhibits improved robustness to transient disturbances, like occlusions, in comparison with the AICP algorithm.

Although Traumatic Brain Injury (TBI) is recognized as a major public health concern, there is currently no efficient method of fast and reliable detection of mild TBI at the point of need, where the injury has occurred. This paper addresses this problem by proposing a portable system for emergency TBI diagnosis and monitored personalized treatment based on quantitative electroencephalography (qEEG) and High Definition transcranial Electrical Stimulation (HD-tES). The paper highlights three innovative elements of the proposed system: its architecture, communication framework, and diagnostic process for detecting TBI in emergency situations.

Electroencephalography (EEG) offers a rich representation of human brain activity in the time domain. EEG would in many circumstances be the preferred technique for analysing brain activity, as it is less expensive and more practical to use than other modalities like functional Magnetic Resonance Imaging (fMRI), notably due to its size. However, its spatial resolution is limited, hampering its ability to characterise activity across spatially distributed brain networks. In comparison, functional Magnetic Resonance Imaging (fMRI) offers very good spatial resolution but the haemodynamic nature of the signal limits its temporal resolution to the order of seconds. A possible solution to this problem is to use both EEG and fMRI signals, but this approach would lead to the loss of convenience of EEG alone. We would like to bring in the advantages of fMRI signal into EEG assessment of brain state and brain responses without the necessity for the presence of the fMRI equipment on site. In this article, we propose a joint statistical model of fMRI/EEG signals and then exploit the learnt correlations to improve the results of signal processing of EEG on its own. We compare the performance of a Blind Source Separation (BSS) method on its own with one, which uses our joint EEG-fMRI model, and show the improvement in the precision of the source separation.

This paper presents a developed approach for intelligently generating symbolic plans by mobile robots acting in domestic environments, such as offices and houses. The significance of the approach lies in developing a new framework that consists of the new modeling of high-level robot actions and then their integration with common-sense knowledge in order to support a robotic task planner. This framework will enable interactions between the task planner and the semantic knowledge base directly. By using common-sense domain knowledge, the task planner will take into consideration the properties and relations of objects and places in its environment, before creating semantically related actions that will represent a plan. This plan will accomplish the user order. The robot task planner will use the available domain knowledge to check the next related actions to the current one and the action's conditions met will be chosen. Then the robot will use the immediately available knowledge information to check whether the plan outcomes are met or violated.

The rapid expansion of e-commerce at the beginning of 21st century has had a significant impact on intellectual property management. A particular area of concern is the misuse of trademarks and trademark protection. Trademarks are proprietary words and images with high reputational value; they are important assets, often used as a marketing tool, which require infringement protection. One of the issues considered during infringement litigation is the visual, conceptual and phonetic similarity of different trademarks. In particular, the conceptual similarity of trademarks is an area never previously studied in information retrieval. This paper focuses on this important aspect by proposing a conceptual model of the comparison process, aimed at retrieving conceptually similar trademarks. The proposed model employs natural language processing and semantic technology to compute the conceptual similarity between trademarks.

The Clock Drawing Test is one of the most widely used screening tools for cognitive impairment and dementia. Since its introduction, more than fifteen scoring systems have been developed to assess the clock drawings. However, very little research has been conducted to study the significance of the elements (features) of the clock drawings for the correct diagnosis of dementia. This paper employs a feature selection method called Feature Interaction Maximization (FIM) to identify the most significant visual features of the test, which can be associated with dementia. The proposed approach is tested with a dataset of 648 clock drawings produced by dementia patients and healthy individuals. The results are compared with other methods used by medical experts. Furthermore, the paper compares the FIM method with an alternative feature selection method based on Information Gain. The results show that the FIM method selects features with higher discriminative power which leads to a deeper understanding of the Clock Drawing Test.

A set of 418 Mini-Mental State Examinations (MMSE) is analysed with the goal of identifying linguistic markers for discriminating between Alzheimer's Disease and Vascular Dementia. The markers are identified by automatically annotating the sentence writing question of the MMSE with syntactic information. 101 variables are extracted from the annotations which are compared with the MMSE questions for information about the diagnosis. Out of the 101 variables, 14 are identified which are within the top 10 of the MMSE questions. Words per sentence, maximal word length, use of adjectives, nouns, verbs and number of subject clauses are discussed in detail.

Automated task planning for service robots faces great challenges in handling dynamic domestic environments. Classical methods in the Artificial Intelligence (AI) area mostly focus on relatively structured environments with fewer uncertainties. This work proposes a method to combine semantic knowledge representation with classical approaches in AI to build a flexible framework that can assist service robots in task planning at the high symbolic level. A semantic knowledge ontology is constructed for representing two main types of information: environmental description and robot primitive actions. Environmental knowledge is used to handle spatial uncertainties of particular objects. Primitive actions, which the robot can execute, are constructed based on a STRIPS-style structure, allowing a feasible solution (an action sequence) for a particular task to be created. With the Care-O-Bot (CoB) robot as the platform, we explain this work with a simple, but still challenging, scenario named “get a milk box”. A recursive back-trace search algorithm is introduced for task planning, where three main components are involved, namely primitive actions, world states, and mental actions. The feasibility of the work is demonstrated with the CoB in a simulated environment.

SRS is a European research project for building robust personal assistant robots using ROS (Robotic Operating System) and Care-O-bot (COB) 3 as the initial demonstration platform. In this paper, experience gained while building the SRS system is presented. A main contribution of the paper is the SRS autonomous control framework. The framework is divided into two parts. First, it has an automatic task planner, which initialises actions on the symbolic level. The planner produces proactive robotic behaviours based on updated semantic knowledge. Second, it has an action executive for coordination actions at the level of sensing and actuation. The executive produces reactive behaviours in well-defined domains. The two parts are integrated by fuzzy logic based symbolic grounding. As a whole, they represent the framework for autonomous control. Based on the framework, several new components and user interfaces are integrated on top of COB's existing capabilities to enable robust fetch and carry in unstructured environments. The implementation strategy and results are discussed at the end of the paper.

Discretization is a process applied to transform continuous data into data with discrete attributes. It makes the learning step of many classification algorithms more accurate and faster. Although many efficient supervised discretization methods have been proposed, unsupervised methods such as Equal Width Discretization (EWD) and Equal Frequency Discretization (EFD) are still in use especially with datasets when classification is not available. Each of these algorithms has its drawbacks. To improve the classification accuracy of EWD, a new method based on adjustable intervals is proposed in this paper. The new method is tested using benchmarking datasets from the UCI repository of machine learning databases; the C4.5 classification algorithm is then used to test the classification accuracy. The experimental results show that the method improves the classification accuracy by about 5% compared to the conventional EWD and EFD methods, and is as good as the supervised Entropy Minimization Discretization (EMD) method.

Autosomal recessive ataxias are a heterogeneous group of rare neurodegenerative diseases characterized by early onset cerebellar ataxia associated with various neurologic, ophthalmologic and systemic signs. In comparison with autosomal dominant ataxias, the group of recessive ataxias is less extensively characterized. In fact, only a few conditions have been genetically characterized. The pathogenesis of these forms is associated with a "loss of function" of specific cellular proteins involved in metabolic homeostasis, cell cycle, and DNA repair/protection processing. The two most common autosomal recessive ataxias, in European countries, are Friedreich's ataxia and ataxia telangiectasia. Other forms are much less frequent, and include ataxia with vitamin E deficiency, abetalipoproteinemia. Refsum's disease, spastic ataxia, infantile onset spinocerebellar ataxia, and ataxia with oculomotor apraxia. These pathological conditions, although extremely rare, have nevertheless to be carefully considered in differential diagnosis, not only for correct nosographical classification, but particularly, for specific prognostic and therapeutic implications. Some of these diseases exhibit a peculiar regional distribution. An updated review of the clinical, genetic, and pathogenic aspects of recessive ataxias is presented. Specific management problems with respect to diagnosis and genetic counseling are discussed.

The aim of document clustering is to produce coherent clusters of similar documents. Although most document clustering algorithms perform well in specific knowledge domains, processing cross-domain document repositories is still a challenge. This difficulty can be attributed to word ambiguity and explained by the observation that monosemic words are more domainoriented than polysemic ones. Document clustering algorithms normally employ text normalization techniques, such as the Porter stemming algorithm. This paper describes a semantically enhanced text normalization algorithm developed for the purpose of improved document clustering. Corpus consistency achieved by the proposed algorithm is compared with the consistency produced by the Porter stemmer. The experimental evidence shows that semantic disambiguation improves clustering performance compared to traditional normalization methods.

A set of 479 Mini-Mental State Examinations (MMSE) is analysed with the goal of discriminating between Alzheimer’s Disease and Vascular Dementia. The patient’s gender has been considered as a predictor in addition to answers of MMSE questions. While similar work has been previously reported, fewer patients were studied and methods inappropriate for 0/1 data were used. The study identifies entropic measures as best suited to analysing this type of data. Performance of five such methods at ordering MMSE questions by decreasing order of information contributed to diagnosis is compared. The analysis uses a novel feature selection method based on parallel estimation of conditional mutual information. The newly introduced method performs demonstrably better than classical and state of the art methods. Good predictors are temporal orientation, language recall and abstract thinking however patient gender is a stronger predictor than any of the MMSE questions.