Abstract Expedient design choices in software development can lead to Technical Debt (TD), with development teams documenting such decisions as Self-Admitted TD (SATD). Algorithm Debt (AD) is a type of TD resulting from the suboptimal implementation of algorithms, which impacts system performance. Given the impact of AD, its automated detection is crucial in Deep Learning (DL) frameworks due to their complexity and evolution. Early detection of AD in DL frameworks can help mitigate model degradation and scalability issues. Despite previous studies on the automated detection of TD from SATD using Machine Learning (ML)/DL models, research on AD detection in DL frameworks remains underexplored. In this study, we empirically investigated the performance of ML/DL models for the automated detection of AD using a dataset of 38, 881 SATD comments from seven DL frameworks. We trained, evaluated, and tested ML/DL models, used embeddings from both DL and large language models, and explored an approach to enrich the dataset with handcrafted features based on AD-related keywords. Our findings reveal that AD is frequently misclassified as Design or Implementation Debt. Logistic Regression (an ML model) with Custom AD Features, achieved an F1-score of 54% for AD, outperforming other ML/DL models (42% to 52%), highlighting the importance of tailored feature engineering. Our research advances automated AD detection in DL frameworks by providing insights into the strengths and limitations of ML/DL models, serving as a first step to guide future tool development. This could help developers using DL frameworks to identify AD issues during development, thereby enhancing system reliability by mitigating model degradation and scalability challenges.

The development of continuous glucose monitoring (CGM) devices has facilitated research of data-driven glucose prediction, which is useful in managing type 1 diabetes mellitus (T1DM). Existing research has investigated using historical-value based machine learning approaches to predict short-term glucose excursion and adapting pre-trained models to unseen patients via meta-learning. This study aims to investigate if time-index based modeling could improve the effectiveness and efficiency to the same objective. Different to existing methods that take the past glucose sequence as the input and output the future sequence, the method we apply — referred to as the meta-optimised time-index model (MOTIM) — takes time-index features as the input and predicts the glucose value at that time. We argue that the reduction of modeling complexity from sequence-level to value-level could ease the learning difficulty and increase the efficiency. In our experiments on the OhioT1DM and UMT1DM datasets, the MOTIM achieves comparable performance to the state-of-the-art models (e.g., 1-hour excursion averaged mean absolute percentage error of 10.25 on OhioT1DM and 12.50 on UMT1DM) while consuming a substantially lower cost (e.g., only 1.3% model parameters and more than 10 times faster inference). These promising findings encourage further work on time-index modeling in glucose prediction for T1DM, and to encourage it, we have released our codes at https://github.com/r-cui/MOTIMGluPred under the MIT license.

The prevalence of neurological disorders poses a challenge to modern healthcare, requiring advancements in diagnostic and prognostic methodologies. This study introduces a deep generative model that retroactively reconstructs magnetic resonance imaging data of human brains into their longitudinal counterparts, creating valuable methods for facilitating meticulous analyses of disease progression. The lack of imaging data on healthy individuals compared to those with brain degenerative disorders, coupled with the time-sensitive nature of some diseases, makes their early diagnosis and effective treatment complex. We demonstrate the model’s efficacy in generating anatomically accurate brain scans to aid in comprehending the dynamic nature of brain pathology, as evidenced by our mixed-method study: Our quantitative evaluation resulted in an outstanding Fréchet Inception Distance score of 5.801 and competitive performance in other key metrics compared to other state-of-the-art inpainting models. Our qualitative evaluation, conducted by two general radiologists and two neuroradiologists, yielded a Discrimination Success Rate of 51.67%, indicating the model’s success in generating realistic images. By integrating this methodology into clinical practice, we anticipate enhanced patient outcomes by personalizing precision medicine and emphasizing preventive strategies such as early and tailored therapeutic interventions.

Although exercising at home has benefits, it is not always engaging or motivating. Augmented Reality (AR) head-mounted displays (HMDs) offer the potential to make in-home exercising and exergaming more inclusive and immersive, but there is limited research investigating how such systems can be designed. We employed a participatory design approach involving semi-structured interviews to investigate how homes can be augmented to facilitate exercising experiences. We developed 10 recommendations for developing home-based exercising experiences using AR HMDs. Our results further contribute to the existing body of research on the use of AR for exercising, home applications, and everyday objects by presenting the first foundational study investigating the wide range of exercises that can be supported through AR HMDs in home environments and the different ways home elements may support these exercises, and laying the groundwork for future work developing home-based exergaming through AR HMDs to increase people’s physical activity levels.

Adoption of artificial intelligence (AI) by the medical community has long been anticipated, endorsed by a stream of machine learning literature showcasing AI systems that yield extraordinary performance. However, many of these systems are likely over-promising and will under-deliver in practice. One key reason is the community's failure to acknowledge and address the presence of inflationary effects in the data. These simultaneously inflate evaluation performance and prevent a model from learning the underlying task, thus severely misrepresenting how that model would perform in the real world. This paper investigated the impact of these inflationary effects on healthcare tasks, as well as how these effects can be addressed. Specifically, we defined three inflationary effects that occur in medical data sets and allow models to easily reach small training losses and prevent skillful learning. We investigated two data sets of sustained vowel phonation from participants with and without Parkinson's disease, and revealed that published models which have achieved high classification performances on these were artificially enhanced due to the inflationary effects. Our experiments showed that removing each inflationary effect corresponded with a decrease in classification accuracy, and that removing all inflationary effects reduced the evaluated performance by up to 30%. Additionally, the performance on a more realistic test set increased, suggesting that the removal of these inflationary effects enabled the model to better learn the underlying task and generalize. Source code is available at https://github.com/Wenbo-G/pd-phonation-analysis under the MIT license.

OBJECTIVE: Ovarian cancer is a significant health issue with lasting impacts on the community. Despite recent advances in surgical, chemotherapeutic and radiotherapeutic interventions, they have had only marginal impacts due to an inability to identify biomarkers at an early stage. Biomarker discovery is challenging, yet essential for improving drug discovery and clinical care. Machine learning (ML) techniques are invaluable for recognising complex patterns in biomarkers compared to conventional methods, yet they can lack physical insights into diagnosis. eXplainable Artificial Intelligence (XAI) is capable of providing deeper insights into the decision-making of complex ML algorithms increasing their applicability. We aim to introduce best practice for combining ML and XAI techniques for biomarker validation tasks. METHODS: We focused on classification tasks and a game theoretic approach based on Shapley values to build and evaluate models and visualise results. We described the workflow and apply the pipeline in a case study using the CDAS PLCO Ovarian Biomarkers dataset to demonstrate the potential for accuracy and utility. RESULTS: The case study results demonstrate the efficacy of the ML pipeline, its consistency, and advantages compared to conventional statistical approaches. CONCLUSION: The resulting guidelines provide a general framework for practical application of XAI in medical research that can inform clinicians and validate and explain cancer biomarkers.

Lexical substitution, which aims to generate substitutes for a target word given a context, is an important natural language processing task useful in many applications. Due to the paucity of annotated data, existing methods for lexical substitution tend to rely on manually curated lexical resources and contextual word embedding models. Methods based on lexical resources are likely to miss relevant substitutes whereas relying only on contextual word embedding models fails to provide adequate information on the impact of a substitute in the entire context and the overall meaning of the input. We proposed CILex, which uses contextual sentence embeddings along with methods that capture additional context information complimenting contextual word embeddings for lexical substitution. This ensured the semantic consistency of a substitute with the target word while maintaining the overall meaning of the sentence. Our experimental comparisons with previously proposed methods indicated that our solution is now the state-of-the-art on both the widely used LS07 and CoInCo datasets with P@1 scores of 55.96% and 57.25% for lexical substitution. The implementation of the proposed approach is available at https://github.com/sandaruSen/CILex under the MIT license.

Computer-assisted quantification and analysis of postural sway may support identifying individuals affected by Parkinson’s disease (PD). Balancing, and its associated postural sway, is a complex process that requires the cooperation of several sensory systems in the brain. Unsurprisingly, a neurodegenerative disease can affect such processes, manifesting itself in the postural sway of affected individuals. Different aspects of postural sway can be quantified and represented as features, which can be used to distinguish between patients and controls. Our aim, inspired by a recent systematic literature review, was to experimentally determine whether sampling frequency and visual state had a meaningful impact on the effectiveness of features in distinguishing between the two groups, and whether overall discriminability could be improved using machine learning. We extracted 102 unique features from 78 postural sway recordings and found that the effectiveness (quantified by an effect size and the average area under the receiver operating characteristic curve) with a sampling frequency of 10 Hz was superior to 20, 40, and 100 Hz, though not with high confidence (quantified through Bayesian analysis). We also concluded that effectiveness under the eyes closed condition was higher than the eyes open condition (confirmed through Bayesian analysis), though combining features from both conditions was superior. Finally, we showed that using machine learning to analyse multiple features through feature selection resulted in higher discriminability in almost all cases. The code for these experiments have been released at https://github.com/Wenbo-G/pd-sway-analysis under the MIT license. When using our code, please cite this paper.

People with type 1 diabetes mellitus (T1DM) must continuously monitor their blood glucose levels and regulate them by insulin dosing to stay in a safe range. A reliable glucose prediction technique could pre-alert the risk of abnormal glycaemia in the near future. In this paper, we apply an attention-based deep network for glucose prediction, which models both the temporal dependencies and the physiological relations among glucose and glucose-related life events, including insulin and carbohydrate intake. We also propose to make use of the knowledge learned from non-target subjects with the same disease to improve the personalised prediction by applying parameters transfer. Our approach was evaluated on the standard benchmark OhioT1DM dataset, where the experiments achieved average root mean square errors over the 12 subjects of 17.82 mg/dL for 30 minutes and 28.54 mg/dL for 60 minutes. Additionally, our ablation experiments indicated that the use of transfer learning constantly improved the prediction. On this basis, we conclude that our approach achieves state-of-the-art performance with statistical significance, and data from other people with T1DM could help on improving personalised predictions. We release our codes at https://github.com/r-cui/GluPred under the MIT license.

Video deblurring models exploit consecutive frames to remove blurs from camera shakes and object motions. In order to utilize neighboring sharp patches, typical methods rely mainly on homography or optical flows to spatially align neighboring blurry frames. However, such explicit approaches are less effective in the presence of fast motions with large pixel displacements. In this work, we propose a novel implicit method to learn spatial correspondence among blurry frames in the feature space. To construct distant pixel correspondences, our model builds a correlation volume pyramid among all the pixel-pairs between neigh-boring frames. To enhance the features of the reference frame, we design a correlative aggregation module that maximizes the pixel-pair correlations with its neighbors based on the volume pyramid. Finally, we feed the aggregated features into a reconstruction module to obtain the restored frame. We design a generative adversarial paradigm to optimize the model progressively. Our proposed method is evaluated on the widely-adopted DVD dataset, along with a newly collected High-Frame-Rate (1000 fps) Dataset for Video Deblurring (HFR-DVD). Quantitative and qualitative experiments show that our model performs favorably on both datasets against previous state-of-the-art methods, confirming the benefit of modeling all-range spatial correspondence for video deblurring.

Gated Recurrent Neural Networks (GRNNs) are important models that continue to push the state-of-the-art solutions across different machine learning problems. However, they are composed of intricate components that are generally not well understood. We increase GRNN interpretability by linking the canonical Gated Recurrent Unit (GRU) design to the well-studied Hopfield network. This connection allowed us to identify network redundancies, which we simplified with an Input Residual Connection (IRC). We tested GRNNs against their IRC counterparts on language modelling. In addition, we proposed an Input Highway Connection (IHC) as an advance application of the IRC and then evaluated the most widely applied GRNN of the Long Short-Term Memory (LSTM) and IHC-LSTM on tasks of i) image generation and ii) learning to learn to update another learner-network. Despite parameter reductions, all IRC-GRNNs showed either comparative or superior generalisation than their baseline models. Furthermore, compared to LSTM, the IHC-LSTM removed 85.4% parameters on image generation. In conclusion, the IRC is applicable, but not limited, to the GRNN designs of GRUs and LSTMs but also to FastGRNNs, Simple Recurrent Units (SRUs), and Strongly-Typed Recurrent Neural Networks (T-RNNs).

Sign language translation (SLT) aims to interpret sign video sequences into text-based natural language sentences. Sign videos consist of continuous sequences of sign gestures with no clear boundaries in between. Existing SLT models usually represent sign visual features in a frame-wise manner so as to avoid needing to explicitly segmenting the videos into isolated signs. However, these methods neglect the temporal information of signs and lead to substantial ambiguity in translation. In this paper, we explore the temporal semantic structures of sign videos to learn more discriminative features. To this end, we first present a novel sign video segment representation which takes into account multiple temporal granularities, thus alleviating the need for accurate video segmentation. Taking advantage of the proposed segment representation, we develop a novel hierarchical sign video feature learning method via a temporal semantic pyramid network, called TSPNet. Specifically, TSPNet introduces an inter-scale attention to evaluate and enhance local semantic consistency of sign segments and an intra-scale attention to resolve semantic ambiguity by using non-local video context. Experiments show that our TSPNet outperforms the state-of-the-art with significant improvements on the BLEU score (from 9.58 to 13.41) and ROUGE score (from 31.80 to 34.96) on the largest commonly used SLT dataset. Our implementation is available at https://github.com/verashira/TSPNet.

The importance of incorporating Natural Language Processing (NLP) methods in clinical informatics research has been increasingly recognized over the past years, and has led to transformative advances. Typically, clinical NLP systems are developed and evaluated on word, sentence, or document level annotations that model specific attributes and features, such as document content (e.g., patient status, or report type), document section types (e.g., current medications, past medical history, or discharge summary), named entities and concepts (e.g., diagnoses, symptoms, or treatments) or semantic attributes (e.g., negation, severity, or temporality). From a clinical perspective, on the other hand, research studies are typically modelled and evaluated on a patient- or population-level, such as predicting how a patient group might respond to specific treatments or patient monitoring over time. While some NLP tasks consider predictions at the individual or group user level, these tasks still constitute a minority. Owing to the discrepancy between scientific objectives of each field, and because of differences in methodological evaluation priorities, there is no clear alignment between these evaluation approaches. Here we provide a broad summary and outline of the challenging issues involved in defining appropriate intrinsic and extrinsic evaluation methods for NLP research that is to be used for clinical outcomes research, and vice versa. A particular focus is placed on mental health research, an area still relatively understudied by the clinical NLP research community, but where NLP methods are of notable relevance. Recent advances in clinical NLP method development have been significant, but we propose more emphasis needs to be placed on rigorous evaluation for the field to advance further. To enable this, we provide actionable suggestions, including a minimal protocol that could be used when reporting clinical NLP method development and its evaluation.

Postural sway is a reflection of brain signals that are generated to control a person’s balance. During the process of ageing, the postural sway changes, which increases the likelihood of a fall. Thus far, expensive specialist equipment is required, such as a force plate, in order to detect such changes over time, which makes the process costly and impractical. Our long-term goal is to investigate the use of inexpensive, everyday video technology as an alternative. This paper describes a study that establishes a 3-way correlation between the clinical gold standard (force plate), a highly accurate multi-camera 3D video tracking system (Vicon) and a standard RGB video camera. To this end, a dataset of 18 subjects performing the BESS balance test on the force plate was recorded, while simultaneously recording the 3D Vicon data, and the RGB video camera data. Then, using Gaussian process regression and a recurrent neural network, models were built to predict the lateral postural sway in the force plate data from the RGB video data. The predicted results show high correlation with the actual force plate signals, which supports the hypothesis that lateral postural sway can be accurately predicted from video data alone. Detecting changes to a person’s postural sway can be used to improve elderly people’s life by monitoring the likelihood of a fall and detecting its increase well before a fall occurs, so that countermeasures (e.g. exercises) can be put in place to prevent falls occurring.

OBJECTIVE: The ShARe/CLEF eHealth 2013 Evaluation Lab Task 1 was organized to evaluate the state of the art on the clinical text in (i) disorder mention identification/recognition based on Unified Medical Language System (UMLS) definition (Task 1a) and (ii) disorder mention normalization to an ontology (Task 1b). Such a community evaluation has not been previously executed. Task 1a included a total of 22 system submissions, and Task 1b included 17. Most of the systems employed a combination of rules and machine learners. MATERIALS AND METHODS: We used a subset of the Shared Annotated Resources (ShARe) corpus of annotated clinical text--199 clinical notes for training and 99 for testing (roughly 180 K words in total). We provided the community with the annotated gold standard training documents to build systems to identify and normalize disorder mentions. The systems were tested on a held-out gold standard test set to measure their performance. RESULTS: For Task 1a, the best-performing system achieved an F1 score of 0.75 (0.80 precision; 0.71 recall). For Task 1b, another system performed best with an accuracy of 0.59. DISCUSSION: Most of the participating systems used a hybrid approach by supplementing machine-learning algorithms with features generated by rules and gazetteers created from the training data and from external resources. CONCLUSIONS: The task of disorder normalization is more challenging than that of identification. The ShARe corpus is available to the community as a reference standard for future studies.

OBJECTIVE: We study the use of speech recognition and information extraction to generate drafts of Australian nursing-handover documents. METHODS: Speech recognition correctness and clinicians' preferences were evaluated using 15 recorder-microphone combinations, six documents, three speakers, Dragon Medical 11, and five survey/interview participants. Information extraction correctness evaluation used 260 documents, six-class classification for each word, two annotators, and the CRF++ conditional random field toolkit. RESULTS: A noise-cancelling lapel-microphone with a digital voice recorder gave the best correctness (79%). This microphone was also the most preferred option by all but one participant. Although the participants liked the small size of this recorder, their preference was for tablets that can also be used for document proofing and sign-off, among other tasks. Accented speech was harder to recognize than native language and a male speaker was detected better than a female speaker. Information extraction was excellent in filtering out irrelevant text (85% F1) and identifying text relevant to two classes (87% and 70% F1). Similarly to the annotators' disagreements, there was confusion between the remaining three classes, which explains the modest 62% macro-averaged F1. DISCUSSION: We present evidence for the feasibility of speech recognition and information extraction to support clinicians' in entering text and unlock its content for computerized decision-making and surveillance in healthcare. CONCLUSIONS: The benefits of this automation include storing all information; making the drafts available and accessible almost instantly to everyone with authorized access; and avoiding information loss, delays, and misinterpretations inherent to using a ward clerk or transcription services.