Abstract Science applications preparing for the exascale era are increasingly exploring in situ computations comprising of simulation‐analysis‐reduction pipelines coupled in‐memory. Efficient composition and execution of such complex pipelines for a target platform is a codesign process that evaluates the impact and tradeoffs of various application‐ and system‐specific parameters. In this article, we describe a toolset for automating performance studies of composed HPC applications that perform online data reduction and analysis. We describe Cheetah, a new framework for composing parametric studies on coupled applications, and Savanna, a runtime engine for orchestrating and executing campaigns of codesign experiments. This toolset facilitates understanding the impact of various factors such as process placement, synchronicity of algorithms, and storage versus compute requirements for online analysis of large data. Ultimately, we aim to create a catalog of performance results that can help scientists understand tradeoffs when designing next‐generation simulations that make use of online processing techniques. We illustrate the design of Cheetah and Savanna, and present application examples that use this framework to conduct codesign studies on small clusters as well as leadership class supercomputers.

With the growing computational complexity of science and the complexity of new and emerging hardware, it is time to re-evaluate the traditional monolithic design of computational codes. One new paradigm is constructing larger scientific computational experiments from the coupling of multiple individual scientific applications, each targeting their own physics, characteristic lengths, and/or scales. We present a framework constructed by leveraging capabilities such as in-memory communications, workflow scheduling on HPC resources, and continuous performance monitoring. This code coupling capability is demonstrated by a fusion science scenario, where differences between the plasma at the edges and at the core of a device have different physical descriptions. This infrastructure not only enables the coupling of the physics components, but it also connects in situ or online analysis, compression, and visualization that accelerate the time between a run and the analysis of the science content. Results from runs on Titan and Cori are presented as a demonstration.

Abstract Users' individual differences and tasks are important factors that influence the use of information systems. Two independent investigations were conducted to study the impact of differences in users' cognition and search tasks on Web search activities and outcomes. Strong task effects were found on search activities and outcomes, whereas interactions between cognitive and task variables were found on search activities only. These results imply that the flexibility of the Web and Web search engines allows different users to complete different search tasks successfully. However, the search techniques used and the efficiency of the searches appear to depend on how well the individual searcher fits with the specific task.

Individual differences between users of information systems can influence search performance. In user-centered design it is important to match users with system configurations that will optimize their performance. Two matching strategies were explored in the first experiment: the capitalization match, and the compensatory match. Findings suggest that a compensatory match is likely to be encountered more frequently in designing information systems. Having determined an optimal match between users and system configurations, it is necessary to find ways to ensure that users are guided to the appropriate configuration. The second experiment examined user selection of system configurations, and concluded that users do not act to optimize system configuration when they select features. This result suggests that information systems must have mechanisms such as user models to direct users to optimal configurations. These experiments suggest some of the complexities and problems encountered in applying individual differences research to user-centered design of information systems.

It is common today for scientists to conduct research in collaboration with their colleagues from different institutions and disciplines. This study collected a sample of 846 scientific research papers published in 1992 and tested three hypotheses on the relationship between research collaboration and interdisciplinarity. Collaboration was measured by the number of authors, number of institutional affiliations, number of affiliation disciplines, and type of collaboration. Interdisciplinarity was measured by the number of disciplines represented in the journals cited. The results showed significant differences in degrees of interdisciplinarity among different levels of collaboration and among different disciplines. Some disciplines were shown to be highly collaborative, while others were not. This analysis led to the conclusion that collaboration contributed significantly to the degree of interdisciplinarity in some disciplines and not in others. In addition to an analysis of publications, this investigation used a survey that asked authors about their forms of collaboration, channels of communication, and use of information. The survey provided some qualitative explanation for the bibliometric findings. Findings are discussed from the perspective of scientist-scientist interaction, scientist-information interaction, and information-information interaction. © 1997 John Wiley & Sons, Inc.

An overview of levels and approaches in the evaluation of knowledge-based systems is presented. There is a need for empirical studies using objective criteria in advance of completing the technical evaluation of such systems. A methodology for this type of evaluation developed for a particular knowledge-based indexing system is presented. It is suggested that the proposed study may serve as a model for the design of any evaluation in which the results of existing intellectual procedures are compared with results achieved when these procedures are aided by use of an appropriate expert system. © 1996 John Wiley & Sons, Inc.

Cognitive abilities of fifty university students were tested using eight tests from the Kit of Factor-Referenced Cognitive Tests. All students searched for references on the same topic using a standard computerized index, and performance in the searches was analyzed using a variety of measures. Effects for cognitive differences, as well as for differences in demographic characteristics and knowledge, were identified using multiple regression. Perceptual speed had an effect on the quality of searches, and logical reasoning, verbal comprehension, and spatial scanning abilities influenced search tactics. It is suggested that information retrieval systems can be made more accessible to users with different levels of cognitive abilities through improvements that will assist users to scan lists of terms, choose appropriate vocabulary for searching, and select useful references.