Given the inherent non-deterministic nature of machine learning (ML) systems, their behavior in production environments can lead to unforeseen and potentially dangerous outcomes. For a timely detection of unwanted behavior and to prevent organizations from financial and reputational damage, monitoring these systems is essential. This paper explores the strategies, challenges, and improvement opportunities for monitoring ML systems from the practitioners' perspective. We conducted a global survey of 91 ML practitioners to collect diverse insights into current monitoring practices for ML systems. We aim to complement existing research through our qualitative and quantitative analyses, focusing on prevalent runtime issues, industrial monitoring and mitigation practices, key challenges, and desired enhancements in future monitoring tools. Our findings reveal that practitioners frequently struggle with runtime issues related to declining model performance, exceeding latency, and security violations. While most prefer automated monitoring for its increased efficiency, many still rely on manual approaches due to the complexity or lack of appropriate automation solutions. Practitioners report that the initial setup and configuration of monitoring tools is often complicated and challenging, particularly when integrating with ML systems and setting alert thresholds. Moreover, practitioners find that monitoring adds extra workload, strains resources, and causes alert fatigue. The desired improvements from the practitioners' perspective are: automated generation and deployment of monitors, improved support for performance and fairness monitoring, and recommendations for resolving runtime issues. These insights offer valuable guidance for the future development of ML monitoring tools that are better aligned with practitioners' needs.

Abstract The use of diverse mobile applications among senior users is becoming increasingly widespread. However, many of these apps contain accessibility problems that result in negative user experiences for seniors. A key reason is that software practitioners often lack the time or resources to address the broad spectrum of age-related accessibility and personalisation needs. As current developer tools and practices encourage one-size-fits-all interfaces with limited potential to address the diversity of senior needs, there is a growing demand for approaches that support the systematic creation of adaptive, accessible app experiences. To this end, we present AdaptForge , a novel model-driven engineering (MDE) approach that enables advanced design-time adaptations of mobile application interfaces and behaviours tailored to the accessibility needs of senior users. AdaptForge uses two domain-specific languages (DSLs) to address age-related accessibility needs. The first model defines users’ context-of-use parameters, while the second defines conditional accessibility scenarios and corresponding UI adaptation rules. These rules are interpreted by an MDE workflow to transform an app’s original source code into personalised instances. We also report evaluations with professional software developers and senior end-users, demonstrating the feasibility and practical utility of AdaptForge .

Context: Financial applications handle sensitive data, including personal details, banking information, and transaction histories, making them prime targets for cyber-attacks. As privacy concerns grow, users and regulators are increasingly analyzing how these apps manage data in different legal contexts. Objective: This study examines user privacy concerns and assesses the impact of privacy regulations on mobile financial applications in Germany, Australia, and the United States. It aims to evaluate how laws such as the GDPR in the EU, the Privacy Act in Australia, and various U.S. state and federal laws shape app privacy policies. Additionally, the study explores the readability and accessibility of privacy policies. Methods: User reviews from app stores were analyzed to identify recurring privacy issues and regional differences in concerns. The study also reviewed privacy laws in the EU, Australia, and the U.S. to assess their influence on financial app policies. To analyze the user-friendliness of privacy documents, a readability analysis was conducted using the Flesch Reading Ease score and estimated reading times. Results: The findings revealed that users are highly concerned about the handling of their data, with significant demand for greater transparency and more robust privacy protections. Regional differences in privacy concerns were identified, with varying levels of engagement with privacy issues in each region. The study also found significant discrepancies in the readability of privacy policies, with many policies proving too complex for the average user to understand. Conclusion: The study concludes that financial app developers need to simplify their privacy policies and improve transparency to build user trust. It also emphasizes the need for stronger regulatory frameworks to address evolving privacy challenges. Recommendations are made for developers and policymakers to enhance data protection and improve user experience in financial services.

Privacy regulations mandate that developers must provide authentic and comprehensive privacy notices, e.g., privacy policies or labels, to inform users of their apps’ privacy practices. However, due to a lack of knowledge of privacy requirements, developers often struggle to create accurate privacy notices, especially for sophisticated mobile apps with complex features and in crowded development teams. To address these challenges, we introduce PriBOM (Privacy Bills of Materials), a systematic software engineering approach that leverages different development team roles to better capture and coordinate mobile app privacy information. PriBOM facilitates transparency-centric privacy documentation and specific privacy notice creation, enabling traceability and trackability of privacy practices. We present a pre-fill of PriBOM based on static analysis and privacy notice analysis techniques. We explore the perceived usefulness of PriBOM through a human evaluation with 150 diverse participants. The role of PriBOM in enhancing privacy-related communication is well received with 83.33% agreement, suggesting that PriBOM could serve as a significant solution for providing privacy support in DevOps for mobile apps.

An Analysis of Privacy Issues and Policies of eHealth Apps

We propose an approach to address the shortcomings of current mobile app review systems on platforms such as the Apple App Store and Google Play. Currently, these platforms lack review categorisation and authentication of genuine user feedback, posing significant barriers for app developers and users. We propose an approach combining socio-technical grounded theory (STGT) and advanced natural language processing (NLP) tools such as GPT-4 to analyse user reviews, providing deeper insights into app functionalities, problems, and ultimately, user satisfaction. An interactive UI prototype is presented to demonstrate the use of structured, verified feedback. This includes a novel review submission process with categorisation/tagging and a”verified download” tag to ensure review authenticity. The goal of our approach is to enhance the app ecosystem by assisting developers in prioritising improvements and enabling users to make informed choices, encouraging a more robust and user-centric digital marketplace.

Machine learning (ML) components are used heavily in many current software systems, but developing them responsibly in practice remains challenging. 'Responsible ML' refers to developing, deploying and maintaining ML-based systems that adhere to human-centric requirements, such as fairness, privacy, transparency, safety, accessibility, and human values. Meeting these requirements is essential for maintaining public trust and ensuring the success of ML-based systems. However, as changes are likely in production environments and requirements often evolve, design-time quality assurance practices are insufficient to ensure such systems' responsible behavior. Runtime monitoring approaches for ML-based systems can potentially offer valuable solutions to address this problem. Many currently available ML monitoring solutions overlook human-centric requirements due to a lack of awareness and tool support, the complexity of monitoring human-centric requirements, and the effort required to develop and manage monitors for changing requirements. We believe that many of these challenges can be addressed by model-driven engineering. In this new ideas paper, we present an initial meta-model, model-driven approach, and proof of concept prototype for runtime monitoring of human-centric requirements violations, thereby ensuring responsible ML behavior. We discuss our prototype, current limitations and propose some directions for future work.

Machine Learning (ML) has become widely adopted as a component in many modern software applications. Due to the large volumes of data available, organizations want to increasingly leverage their data to extract meaningful insights and enhance business profitability. ML components enable predictive capabilities, anomaly detection, recommendation, accurate image and text processing, and informed decision-making. However, developing systems with ML components is not trivial; it requires time, effort, knowledge, and expertise in ML, data processing, and software engineering. There have been several studies on the use of model-driven engineering (MDE) techniques to address these challenges when developing traditional software and cyber–physical systems. Recently, there has been a growing interest in applying MDE for systems with ML components. The goal of this study is to further explore the promising intersection of MDE with ML (MDE4ML) through a systematic literature review (SLR). Through this SLR, we wanted to analyze existing studies, including their motivations, MDE solutions, evaluation techniques, key benefits and limitations. Our SLR is conducted following the well-established guidelines by Kitchenham. We started by devising a protocol and systematically searching seven databases, which resulted in 3,934 papers. After iterative filtering, we selected 46 highly relevant primary studies for data extraction, synthesis, and reporting. We analyzed selected studies with respect to several areas of interest and identified the following: 1) the key motivations behind using MDE4ML; 2) a variety of MDE solutions applied, such as modeling languages, model transformations, tool support, targeted ML aspects, contributions and more; 3) the evaluation techniques and metrics used; and 4) the limitations and directions for future work. We also discuss the gaps in existing literature and provide recommendations for future research. This SLR highlights current trends, gaps and future research directions in the field of MDE4ML, benefiting both researchers and practitioners.

The COVID-19 pandemic has changed the way we live, leading to a rapid expansion of mHealth apps usage. The pandemic also led to the introduction of a large number of ”emerging apps” to the mobile app market. mHealth and emerging app users have reported a range of serious issues in their user reviews, which we identified and better understood after extracting, translating, analysing and classifying over 6 millions user reviews of these apps into different aspects. As evidenced by user reviews, many mHealth and emerging apps are plagued by major issues and problems. App developers could improve the quality and adoption of their apps if they had a better grasp of the major concerns raised by their users. We also link the findings from our user review analysis to the app version history release notes to better understand and identify what issues the developers of mHealth/emerging apps managed to solve or not. Investigating the association between user reviews and app updates will allow us to design a model for mHealth/emerging app developers to follow. We identified that our recommendation models and tools can assist mHealth/emerging app developers and designers in proactively identifying and preventing software and design issues before their final apps are deployed to mobile users. A proactive evaluation model and discovering mHealth/emerging issues early can save billions of dollars and avert millions of deaths a year. As a result, more people will download these apps when they believe that the updates are actually addressing and solving their problems, which will result in saving lives and improving the quality of life for people with disabilities or those who use these apps.

The global mHealth app market is rapidly expanding, especially since the COVID-19 pandemic. However, many of these mHealth apps have serious issues, as reported in their user reviews. Better understanding their key user concerns would help app developers improve their apps' quality and uptake. While app reviews have been used to study user feedback in many prior studies, many are limited in scope, size and/or analysis. In this paper, we introduce a very large-scale study and analysis of mHealth app reviews. We extracted and translated over 5 million user reviews for 278 mHealth apps. These reviews were then classified into 14 different aspects/categories of issues reported. Several mHealth app subcategories were examined to reveal differences in significant areas of user concerns, and to investigate the impact of different aspects of mhealth apps on their ratings. Based on our findings, women's health apps had the highest satisfaction ratings. Fitness activity tracking apps received the lowest and most unfavourable ratings from users. Over half of users who reported troubles leading them to uninstall mHealth apps gave a 1-star rating. Half of users gave the account and logging aspect only one star due to faults and issues encountered while registering or logging in. Over a third of users who expressed privacy concerns gave the app a 1-star rating. However, only 6% of users gave apps a one-star rating due to UI/UX concerns. 20% of users reported issues with handling of user requests and internationalisation concerns. We validated our findings by manually analysing a sample of 1,000 user reviews from each investigated aspect/category. We developed a list of recommendations for mHealth apps developers based on our user review analysis.

One of the biggest challenges of RGB-D posture tracking is separating appendages such as briefcases, trolleys, and backpacks from the human body. Markerless motion tracking relies on segmenting each depth frame to a finite set of body parts. This is achieved via supervised learning by assigning each pixel to a certain body part. The training image set for the supervised learning are usually synthesised using popular motion capture databases and an ensemble of 3D models covering a wide range of anthropometric characteristics. In this paper, we propose a novel method for generating training data of human postures with attached objects. The results have shown a significant increase in body-part classification accuracy for subjects with props from 60% to 94% using the generated image set.