In our prior work, we presented a novel approach to the evaluation of quality in use of corporate web sites based on an original quality model (QM-U) and a related methodology (EQ-EVAL). This article focuses on two research questions. The first one aims at investigating whether expected quality obtained through the application of EQ-EVAL methodology by employing a small panel of evaluators is a good approximation of actual quality obtained through experimentation with real users. To answer this research question, a comparative study has been carried out involving 5 evaluators and 50 real users. The second research question aims at demonstrating that the adoption of the EQ-EVAL methodology can provide useful information for web site improvement. Three original indicators, namely coherence, coverage and ranking have been defined to answer this question, and an additional study comparing the assessments of two panels of 5 and 10 evaluators, respectively, has been carried out. The results obtained in both studies are largely positive and provide a rational support for the adoption of the EQ-EVAL methodology.

This paper addresses the important issue of identifying the requirements for advanced human-system interface (HSI) in the context of a disciplined requirements engineering process. This is essential if we are to have effective, well-designed HSIs as part of large software development projects. A complete methodology, called AMMETH (a mixed methodology), that can substantially help carry out the requirements analysis of HSIs in a disciplined and effective way, is proposed. The methodology integrates several proven analysis techniques within a structured framework, where they can be exploited effectively, on their own, and orchestrated as a whole. It includes seven steps, namely: 1) analyzing the context, 2) stating interaction goals, 3) eliciting user needs and expectations, 4) identifying and rating interaction features, 5) defining interaction patterns, 6) collecting usability feedback, and 7) defining requirements. The methodology has been applied successfully to the design of a real advanced HSI toward a complex process supervision and diagnostic system of an electric power plant.

Uncertainty calculus overcomes many of the drawbacks affecting other well-known approaches to uncertain reasoning. Nevertheless, it suffers from other severe limitations and gives rise to counter-intuitive results. We show that common sense inconsistencies of uncertainty calculus originate from a scarce cognitive plausibility in the definition of propagation and aggregation operators, and we propose an improved version of Driankov's uncertainty calculus (1986) where the definition of such operators is substantially revised according to common sense criteria. Revised uncertainty calculus allows overcoming in a general and uniform way all the problems identified in uncertainty calculus and provides a substantial enhancement of cognitive plausibility. It has been successfully tested in the design and development of a real application concerning preventive diagnosis of power transformers.

A survey of knowledge-based system (KBS) evaluation methods is presented. The authors argue that these methods are partial, poorly systematic, and not easily applicable. An approach to KBS evaluation that comprises a precise definition of the concepts of performance and quality, a general evaluation methodology, and a set of criteria to support its practical application is presented. The proposed approach has been tried only partially and with rather simple test cases.>

The basic concepts of the multimodeling approach to the representation of physical systems are presented. Emphasis is placed on the exploitation of many, diverse models of a system for the execution of complex problem solving tasks, such as interpretation, diagnosis, design, simulation, etc. The considered models are based on different ontologies, representational assumptions, epistemological types, and aggregation levels. After a brief survey of the techniques adopted for representing structural and behavioral knowledge, attention is focused on function and teleology. A novel approach is proposed for defining, representing, and using these two types of knowledge which play a fundamental role both from the representation and reasoning perspectives. The fundamental claim is that while teleological knowledge concerns the specific purposes for which the system has been designed, functional knowledge is devoted to bridge the gap between such abstract purposes and the actual structure and behavior of the system, through the concepts of phenomena, processes, and functional roles. A clear definition is provided of all the various epistemological and ontological links existing between the different models.>

The requirements of a user modeling component for an expert interface are analyzed, and the main points of a proposed approach to user modeling are stated. The authors focus on a knowledge-based system, called UM-tool, devoted to creating, maintaining, and using explicit user models within an expert interface. UM-tool supports a novel approach to user modeling, which is based both on the use of stereotypes and on a dynamic reclassification scheme. The architecture of the system is described, the organization and content of its knowledge bases are illustrated, and the modeling mechanisms utilized are presented in detail. An example of the use of UM-tool in the frame of the information-retrieval-natural-language-interface (IR-NLI II) expert interface devoted to supporting end users in accessing online information retrieval services is discussed, focusing on the specific role of the user modeling component. An evaluation of the proposed approach and a critical comparison with related work are presented. Future research directions are outlined.>

This paper addresses the problem of building expert interfaces to information retrieval systems. In particular, the problem of augmenting the capabilities of such interfaces with user modeling features is discussed and the main benefits of this approach are outlined. The paper presents a prototype system called IR-NLI II, devoted to model by means of artificial intelligence techniques the human intermediary to information retrieval systems. The overall organization of the IR-NLI II system is presented, together with a short description of the two main modules implemented so far, namely the Information Retrieval Expert Subsystem and the User Modeling Subsystem. An example of interaction with IR-NLI II is described. Perspectives and future research directions are finally outlined.

This paper encompasses two main topics: a broad and general analysis of the issue of performance evaluation of NLP systems and a report on a specific approach developed by the authors and experimented on a sample test case. More precisely, it first presents a brief survey of the major works in the area of NLP systems evaluation. Then, after introducing the notion of the life cycle of an NLP system, it focuses on the concept of performance evaluation and analyzes the scope and the major problems of the investigation. The tools generally used within computer science to assess the quality of a software system are briefly reviewed, and their applicability to the task of evaluation of NLP systems is discussed. Particular attention is devoted to the concepts of efficiency, correctness, reliability, and adequacy, and how all of them basically fail in capturing the peculiar features of performance evaluation of an NLP system is discussed. Two main approaches to performance evaluation are later introduced; namely, black-box- and model-based, and their most important characteristics are presented. Finally, a specific model for performance evaluation proposed by the authors is illustrated, and the results of an experiment with a sample application are reported. The paper concludes with a discussion on research perspectives, open problems, and importance of performance evaluation to industrial applications.

The notion of noncounting language, initially introduced for regular languages recognized by counter-free finite machines, and recently extended to parenthesized context-free languages, is here further studied for general (i.e., nonparenthesized) context-free languages. While weakly equivalent context-free grammars do not, in general, fall in the same class with respect to the noncounting property, it is shown by a complex proof that weakly equivalent operator precedence grammars are all counting or all noncounting (a property which distinguishes the operator precedence languages from classical deterministically parsable families).

The class of noncountlng (aperiodic) context-free parenthesis languages is introduced here and is found to extend the classical theory of noncountmg regular languages It Is proved that it is possible to decide whether or not a context-free parenthesis grammar Is noncountmg The class of k-distract-homogeneous grammars (previously introduced in connection with studies on grammatical mference or language acqmsitmn) is rigorously defined and proved to be noncountlng It as argued that the noncountmg model fits the syntactic aspects of natural or araficial languages more closely than the context-free model