Recently, various risks have been pointed out in artificial intelligence (AI) systems. In particular, AI security, AI ethics, and AI quality are considerable risks. To make AI systems trustworthy against these risks, risk assessment technology is needed to identify potential AI risks and decide which risks should be dealt with in priority. We propose a risk assessment technology that assesses three kinds of risks—AI security, AI ethics, and AI quality—which have been considered separately. Our technology follows the ISO 31000 framework, consisting of four phases: risk candidate identification, impact assessment, likelihood assessment, and priority derivation for countermeasures. To realize this technology, risk candidate identification is achieved by extending AI ethics impact assessment— an identification method of AI ethics risk—to AI security and AI quality. Impact and likelihood assessments are conducted by extending assessment methods for AI security to AI ethics and AI quality. We conducted a case study using our technology and confirmed that the risks were appropriately extracted, and the priority of the risks to be dealt with was derived.

Abstract Nowadays, systems containing components based on machine learning (ML) methods are becoming more widespread. In order to ensure the intended behavior of a software system, there are standards that define necessary qualities of the system and its components (such as ISO/IEC 25010). Due to the different nature of ML, we have to re-interpret existing qualities for ML systems or add new ones (such as trustworthiness). We have to be very precise about which quality property is relevant for which entity of interest (such as completeness of training data or correctness of trained model), and how to objectively evaluate adherence to quality requirements. In this article, we present how to systematically construct quality models for ML systems based on an industrial use case. This quality model enables practitioners to specify and assess qualities for ML systems objectively. In addition to the overall construction process described, the main outcomes include a meta-model for specifying quality models for ML systems, reference elements regarding relevant views, entities, quality properties, and measures for ML systems based on existing research, an example instantiation of a quality model for a concrete industrial use case, and lessons learned from applying the construction process. We found that it is crucial to follow a systematic process in order to come up with measurable quality properties that can be evaluated in practice. In the future, we want to learn how the term quality differs between different types of ML systems and come up with reference quality models for evaluating qualities of ML systems.

As the applications of machine learning algorithms in various fields are widely demanded, the development of machine learning software systems (MLS) is rapidly increasing. The quality of MLS is different from that of conventional software systems, in the sense that it depends on the amount and distribution of training data in a model learning and input data during operation. This is a major challenge in quality assurance of MLS development for the enterprise. In this paper, we propose a requirements-driven method to determine the quality characteristics of the MLS. Major contributions of this paper include: (1) Extending the quality characteristics of ISO 25010, which defines the conventional software quality, to those unique to MLS; this paper also defines its measuring method. (2) A method to identify requirements, i.e., issues to be determined in the requirements definition, in order to derive the quality characteristics and measurement methods for MLS, since the quality characteristics and the measurement method depend on the goals of the system under development. In order to evaluate the proposed method, we carried out an empirical study of the quality characteristics and measurement methods related to functional correctness and the maturity of the MLS for the enterprise. Based on the study, we compare the quality characteristics and measurement methods derived by the proposed method with those suggested by developers, and demonstrate the effectiveness of the proposed method.

As Representational State Transfer (REST)-based Web APIs are spreading to enterprise information systems, software development for the use and provision of Web APIs is rapidly increasing. The quality of the Web APIs significantly influences the application quality and development productivity. However, no quality model for Web APIs has been established yet, because Web APIs differ from conventional APIs in that they execute remotely on different servers and may be changed independently of their users. These unique characteristics introduce new problems in the software engineering of Web APIs, and impose risks to the users, especially those using enterprise Web APIs, whose numbers are increasing. To solve these problems, in this paper, we propose a quality model for Web APIs that reflects their unique characteristics. As the main characteristics of this quality model, we propose the concept of Web API learnability to use and stability to change, from the perspective of Web API users. Based on this quality model, we also propose a set of measures and a quantitative evaluation method. In this study, we applied the proposed quality model and evaluation method to four types of actual Web APIs, including Uber, WordPress, OpenStack, and Media Processing. To validate the proposed model, we also conducted an empirical study of the usability of the Web APIs. Our comparison of the proposed quality statistics with those from the empirical study validates the effectiveness of the proposed quality model and its associated measures of the learnability and stability of Web APIs.

Many User Experience (UX) activities are carried out during requirements engineering phases, e.g. understanding and assessing the UX of existing systems, and eliciting functional and non-functional requirements that improve UX. These activities are typically performed by requirements engineers who are non-UX experts. It is necessary to provide a good UX in order to ensure long-term motivation of users, especially in business applications. UX has various characteristics of differing importance; it can be difficult for RE engineers to grasp all characteristics of UX and to judge which characteristics are important and which need to be improved. We propose a two-step approach to solve these difficulties. The first step is the definition of a UX quality model and corresponding metrics. We propose an approach to calculate the UX score of a business application using the value of these metrics. The second step is a process to identify insufficient characteristics within the calculated UX score. In this paper we present the aforementioned approach to collect and calculate the UX score of a product, show how to identify serious UX-related problems as part of requirements engineering activities, and present the results obtained from an initial validation of our quality model and related questionnaire. With our approach, we enable RE experts who are non-UX experts to find the necessary requirements to improve UX.

This paper describes an easy means of setting traceability in business application artifacts. The traceability is established for completeness verification and impact analysis. There are thousands of functions and screens in real business applications. It is difficult to establish suitable links from these many elements. In addition, there is a constraint that the workload of setting traceability links should be small. This paper proposes a two-step approach to establish traceability in artifacts. In the first step, the software model defines traceability. In the second step, a developer him/herself sets up traceability links during the design phase. The purpose of the first step is to restrict of the set of linkable elements, while the purpose of the second step is to establish links according to the developer's intentions. We also devise the second step in two points. The first point is to make a developer set up links while he/she is working on the design. The second point is appropriate categorization to reduce the number of candidate links. We also propose a support tool for setting up links. The approach and tool decrease the developer's workload. This paper provides details and evaluation on this model-based approach and tool.

Sharing know-how regarding business process modeling activities and best practices can save time in the early phases of business system development. To help developers share know-how and best practices, we have developed a methodology and a tool for business process modeling and have applied it to a system development project. Furthermore, we have developed process-template development techniques to realize granularity-unification of business processes so that business process models can be reused, and have clarified the practical divisions of process models and the design issues concerning each division. These techniques enable the preparation of reuse-oriented business process templates before the business process modeling activity. In this paper, we describe the methodology and tool, including a meta-model and notations we have constructed for business modeling. We then consider the effectiveness of the methodology and tool: in an actual development project, the tool and business process templates decreased development time by 46% or more.