Most organizations still face the common challenge of clearly establishing measurement and evaluation (ME) approaches for quality assurance programs. They could be successful if the resulting measurements and evaluations were well specified to ensure more consistent analysis and decision-making. A well-established ME approach might rely on sound conceptual bases. This paper discusses the main concepts, relationships, and axioms for the Measurement and Evaluation Top-Domain Ontology (MEvalTDO), which is located at the high-domain level in the context of a five-level ontological architecture. The scope of MEvalTDO is primarily intended to represent extensible concepts and relationships for ME activities, methods, and work products useful for carrying out evaluation projects and strategies. To illustrate the applicability of the MEvalTDO conceptualization, the reuse of its terms in specifications of activities, methods, artifacts, and outcomes included in evaluation strategies is shown. Reference is made to a set of evaluation strategies that can help achieve different goal purposes in projects already implemented.

This paper discusses the main concepts and relationships for the Non-Functional Requirements (NFRs) domain ontology, which is placed at the top-domain level in the framework of a five-level ontological architecture. In this (ontological) context, the term Non-Functional Requirement (NFR) does not model a particular thing (concrete entity), but rather a quality-, constraint-related assertion of a certain aspect of the thing to be evaluated. Also, an NFR as a human assertion can deal with a thing category as a universal abstraction. Thus, the quality aspect attributed, for example, to a particular thing (as a class or its individuals) can refer to its parts, such as the properties and/or capabilities of the entity, or the entity as a whole. To better understand the concepts and relationships of NFRs, a basic background in the semantics of the foundational concepts such as Thing, Thing Category, and Assertion is needed. Furthermore, to validate the applicability of the ontological conceptualization of NFRs, a set of typical cases for different practical situations is showcased.

In 2014, Rey Juan Carlos University (from Spain) has implemented the Service Science, Management and Engineering career. From the study conducted in 2016, the formulated hypothesis that the students' evidenced performance in the different modules of the career (i.e., Fundamentals, Technology, Personal Skills, Services and Companies) is affected by the secondary school modality of provenance. Aimed at monitoring the performance of first-year students of this career, in the present work we propose the use of a Holistic Evaluation Approach for Quality. This approach has a family of integrated evaluation strategies which allows achieving goals for diverse purposes such as to understand, compare, monitor, among others. In a nutshell, a strategy integrates a conceptual base with process and method specifications, which ensures that evaluation projects' results are consistent and comparable over time. Particularly, in this work, we use the monitoring strategy for understanding the performance of students, in the first year of the abovementioned career, from 2014 to 2017 period of time. The yielded outcomes will serve as a basis for the development of a predictive model. This will allow to the University to propose instruments aimed at improving the performance of students regarding the predominant modality of provenance.

Companies commonly establish and pursue business goals for different purposes. Many purposes, such as to understand, improve, compare, monitor and control, among others, may be driven by measurement, evaluation and analysis activities. Different strategies can be used to help achieving goal purposes. A strategy defines a set of activities and methods related to nonfunctional requirements, measurement, evaluation, analysis and eventually change. The analysis activity can use qualitative and quantitative data and information yielded by measurement and evaluation tasks. Regarding the intended purpose, a strategy should be tuned, particularly for the analysis activity in order to be useful as information need support for the given business goal. In this paper, we discuss the analysis activity for strategies regarding different evaluation purposes. Specifically, we illustrate the comparing and adopting strategy applied to four social network mobile apps. In addition, we highlight the particularities of its analysis activity in contrast to the one embraced in the monitoring and controlling strategy.

All organizations, both public and private should set goals or objectives at different organizational levels. So higher education institutions should not be the exception. A holistic evaluation approach can be useful for analyzing whether the pursued strategic or tactical objectives are achieved regarding different information needs at operational level. While teachers may aim at facilitating the learning process of all their students, department directors may need to understand the students' performance with the objective of planning extracurricular courses for helping them to improve their skills in specific areas. This paper illustrates a systematic strategy for evaluation and analysis which includes activities and methods for understanding the current situation of an entity in order to foster better decision making. Particularly, it is used to understand and analyze the students' performance in the Service Science, Management and Engineering career of a Spanish university considering the profile of provenance, i.e., the secondary school modality.

In order to assess Web applications in a more consistent way we have to deal not only with non-functional requirement specification, measurement and evaluation (M&E) information but also with the context information about the evaluation project. When organizations record the collected data from M&E projects, the context information is very often neglected. This can jeopardize the validity of comparisons among similar evaluation projects. We highlight this concern by introducing a quality in use assessment scenario. Then, we propose a solution by representing the context information as a new add-in to the INCAMI M&E framework. Finally, we show how context information can improve Web application evaluations, particularly, data analysis and recommendation processes.

A common challenge faced by many software and Web organizations is to have a clear establishment of a measurement and evaluation framework for quality assurance programs. A well-established measurement and evaluation framework might rely on a sound conceptual (ontological) base. Besides, organizations could succeed if resulting measurements and evaluations are tailored to their information needs for specific purposes, contexts, and user viewpoints. In this paper, we discuss our measurement and evaluation framework so-called INCAMI that stands for information need, concept model, attribute, metric and indicator, which is also based on our proposal of the metrics and indicators ontology. Without appropriate metadata of metrics and indicators, it is difficult to ensure that measure and indicator values are repeatable and comparable among organization's projects. Moreover, analyses and comparisons could be performed in an inconsistent way. The present work tries to highlight about the usefulness of this framework and strategy as well as to discuss why INCAMI can be a more robust and engineered framework than others. Finally, given the aim of this special issue on aggregation operators and models, where they fit into the framework to model and compute, partial/global indicators are highlighted. © 2008 Wiley Periodicals, Inc.

In the present paper we discuss a systematic approach in order to specify, measure, and evaluate quality in use, i.e., the perceived quality of software or web applications by actual users in real con-texts of use. The methodological proposal is grounded on our measurement and evaluation framework which in turn is based on the metrics and indicators ontology. In order to illustrate the pro-posal a case study for assessing the quality in use of an e-Learning application was conducted. The final results are analyzed in the light of both the consistency of the methodological support and the challenges of assessing quality in use with actual users.