Intelligence analysis involves unpredictable processes and decision making about complex domains where analysts rely upon expertise. Artificial Intelligence (AI) systems could support analysts as they perform analysis tasks, to enhance their expertise. However, systems must also be cognisant about how expertise is gained and designed so that this is not impinged. In this paper, we describe the results of Cognitive Task Analysis interviews with 6 experienced intelligence analysts. We capture themes, in terms of their decision making paths during an analysis task, and highlight how each theme is both influenced by expertise and an influence upon expertise. We also identify important interdependencies between themes. We propose that our findings can be used to help design Human-Centered AI (HCAI) systems for supporting intelligence analysts.

The adoption of artificial intelligence (AI) systems in environments that involve high risk and high consequence decision-making is severely hampered by critical design issues. These issues include system transparency and brittleness, where transparency relates to (i) the explainability of results and (ii) the ability of a user to inspect and verify system goals and constraints; and brittleness, (iii) the ability of a system to adapt to new user demands. Transparency is a particular concern for criminal intelligence analysis, where there are significant ethical and trust issues that arise when algorithmic and system processes are not adequately understood by a user. This prevents adoption of potentially useful technologies in policing environments. In this article, we present a novel approach to designing a conversational agent (CA) AI system for intelligence analysis that tackles these issues. We discuss the results and implications of three different studies; a Cognitive Task Analysis to understand analyst thinking when retrieving information in an investigation, Emergent Themes Analysis to understand the explanation needs of different system components, and an interactive experiment with a prototype conversational agent. Our prototype conversational agent, named Pan, demonstrates transparency provision and mitigates brittleness by evolving new CA intentions. We encode interactions with the CA with human factors principles for situation recognition and use interactive visual analytics to support analyst reasoning. Our approach enables complex AI systems, such as Pan, to be used in sensitive environments, and our research has broader application than the use case discussed.

We present and report on Design Exposition Discussion Documents (DExDs), a new means of fostering collaboration between visualization designers and domain experts in applied visualization research. DExDs are a collection of semi-interactive web-based documents used to promote design discourse: to communicate new visualization designs, and their underlying rationale, and to elicit feedback and new design ideas. Developed and applied during a four-year visual data analysis project in criminal intelligence, these documents enabled a series of visualization re-designs to be explored by crime analysts remotely - in a flexible and authentic way. The DExDs were found to engender a level of engagement that is qualitatively distinct from more traditional methods of feedback elicitation, supporting the kind of informed, iterative and design-led feedback that is core to applied visualization research. They also offered a solution to limited and intermittent contact between analyst and visualization researcher and began to address more intractable deficiencies, such as social desirability-bias, common to applied visualization projects. Crucially, DExDs conferred to domain experts greater agency over the design process - collaborators proposed design suggestions, justified with design knowledge, that directly influenced the re-redesigns. We provide context that allows the contributions to be transferred to a range of settings.

Visualising how social networks evolve is important in intelligence analysis in order to detect and monitor issues, such as emerging crime patterns or rapidly growing groups of offenders. It remains an open research question how this type of information should be presented for visual exploration. To get a sense of how users work with different types of visualisations, we evaluate a matrix and a node-link diagram in a controlled thinking aloud study. We describe the sense-making strategies that users adopted during explorative and realistic tasks. Thereby, we focus on the user behaviour in switching between the two visualisations and propose a set of nine strategies. Based on a qualitative and quantitative content analysis we show which visualisation supports which strategy better. We find that the two visualisations clearly support intelligence tasks and that for some tasks the combined use is more advantageous than the use of an individual visualisation.

Large display and touch screens are becoming ubiquitous within the work place including multiple display screens. There is limited evidence on what configurations and arrangements of the display screens are most effective for data analysis. We conducted two user studies to understand the effectiveness of the display angle, physical size, resolution, and touch precision for data analysis activities. Our results indicated that touch interaction for data analysis sitting at a workstation was most effective with medium sized screens at 27", high precision touch accuracy (not 4K resolution), and display angle titled at 30°. The results from our studies can guide other researchers and developers who want to integrate large touch display screens into their work place environments.

Abstract Visual analytics systems combine machine learning or other analytic techniques with interactive data visualization to promote sensemaking and analytical reasoning. It is through such techniques that people can make sense of large, complex data. While progress has been made, the tactful combination of machine learning and data visualization is still under‐explored. This state‐of‐the‐art report presents a summary of the progress that has been made by highlighting and synthesizing select research advances. Further, it presents opportunities and challenges to enhance the synergy between machine learning and visual analytics for impactful future research directions.

Criminal Intelligence Analysis often requires a search different from the semantic and keyword based searching to reveal the associations among semantically and operationally connected objects within a crime knowledge base. In this paper we introduce associative search as a search along the networks of association between objects like people, places, other organizations, products, events, services, and so on. We also propose an associative search model based on the 5WH associated concepts of a crime, i.e. WHAT (what has happened), WHO (who was involved in the crime), WHEN (the temporal information of the crime), WHERE (the geo-spatial information of the crime) HOW (the modus-operandi used in committing a crime). We have employed Formal Concept Analysis theory to reveal the associations, highlighting Hot Spots, offender's profile and its associated offenders in a criminal activity.

Abstract Many datasets have multiple perspectives – for example space, time and description – and often analysts are required to study these multiple perspectives concurrently. This concurrent analysis becomes difficult when data are grouped and split into small multiples for comparison. A design challenge is thus to provide representations that enable multiple perspectives, split into small multiples, to be viewed simultaneously in ways that neither clutter nor overload. We present a design framework that allows us to do this. We claim that multi‐perspective comparison across small multiples may be possible by superimposing perspectives on one another rather than juxtaposing those perspectives side‐by‐side. This approach defies conventional wisdom and likely results in visual and informational clutter. For this reason we propose designs at three levels of abstraction for each perspective. By flexibly varying the abstraction level, certain perspectives can be brought into, or out of, focus. We evaluate our framework through laboratory‐style user tests. We find that superimposing, rather than juxtaposing, perspective views has little effect on performance of a low‐level comparison task. We reflect on the user study and its design to further identify analysis situations for which our framework may be desirable. Although the user study findings were insufficiently discriminating, we believe our framework opens up a new design space for multi‐perspective visual analysis.

Sensemaking is described as the process of comprehension, finding meaning and gaining insight from information, producing new knowledge and informing further action. Understanding the sensemaking process allows building effective visual analytics tools to make sense of large and complex datasets. Currently, it is often a manual and time-consuming undertaking to comprehend this: researchers collect observation data, transcribe screen capture videos and think-aloud recordings, identify recurring patterns, and eventually abstract the sensemaking process into a general model. In this paper, we propose a general approach to facilitate such a qualitative analysis process, and introduce a prototype, SensePath, to demonstrate the application of this approach with a focus on browser-based online sensemaking. The approach is based on a study of a number of qualitative research sessions including observations of users performing sensemaking tasks and post hoc analyses to uncover their sensemaking processes. Based on the study results and a follow-up participatory design session with HCI researchers, we decided to focus on the transcription and coding stages of thematic analysis. SensePath automatically captures user's sensemaking actions, i.e., analytic provenance, and provides multi-linked views to support their further analysis. A number of other requirements elicited from the design session are also implemented in SensePath, such as easy integration with existing qualitative analysis workflow and non-intrusive for participants. The tool was used by an experienced HCI researcher to analyze two sensemaking sessions. The researcher found the tool intuitive and considerably reduced analysis time, allowing better understanding of the sensemaking process.

Timeline visualization is an important tool for sense making. It allows analysts to examine information in chronological order and to identify temporal patterns and relationships. However, many existing timeline visualization methods are not designed for the dynamic and iterative nature of the sense making process and the various analysis activities it involves. In this paper, we introduce a novel timeline visualization, Schema Line, to address these deficiencies. Schema Line is designed to group notes into analyst-determined schema, using a layout algorithm to produce compact but aesthetically pleasing timeline visualization, and includes fluid user interactions to support sense making activities. It enables interactive temporal schemata construction with seamless integration with visual data exploration and note taking. Our preliminary evaluation results show that the participants found the new method easy to learn and use, and its features effective for the sense making activities for which it was designed.

This article reports an exploratory user study in which a group of civil servants with experience of, or involvement in, intelligence analysis used the tool INVISQUE to address a problem using the 2011 VAST data set. INVISQUE uses a visual metaphor that combines searching, clustering, and sorting of document surrogates with free-form manipulation on an infinite canvas. The study looks into exposing the behaviors and related cognitive strategies that users would employ to better understand how this and similar environments might better support intelligence type work. The results include the observation that the search and spatial features of the system supported participants in establishing, elaborating, and systematically evaluating explanatory narratives that accounted for the data. Also, visual persistence at the interface allowed them to keep track of searches and to re-find documents when their importance became apparent. The article concludes with reflections on our findings and propose a set of guidelines for developing systems that support sensemaking.

This research aims to develop design guidelines for systems that support investigators and analysts in the exploration and assembly of evidence and inferences. We focus here on the problem of identifying candidate 'influencers' within a community of practice. To better understand this problem and its related cognitive and interaction needs, we conducted a user study using a system called INVISQUE (INteractive Visual Search and QUery Environment) loaded with content from the ACM Digital Library. INVISQUE supports search and manipulation of results over a freeform infinite 'canvas'. The study focuses on the representations user create and their reasoning process. It also draws on some pre-established theories and frameworks related to sense-making and cognitive work in general, which we apply as a 'theoretical lenses' to consider findings and articulate solutions. Analysing the user-study data in the light of these provides some understanding of how the high-level problem of identifying key players within a domain can translate into lower-level questions and interactions. This, in turn, has informed our understanding of representation and functionality needs at a level of description which abstracts away from the specifics of the problem at hand to the class of problems of interest. We consider the study outcomes from the perspective of implications for design.

We describe and demonstrate an extensible framework that supports data exploration and provenance in the context of Human Terrain Analysis (HTA). Working closely with defence analysts we extract requirements and a list of features that characterise data analysed at the end of the HTA chain. From these, we select an appropriate non-classified data source with analogous features, and model it as a set of facets. We develop ProveML, an XML-based extension of the Open Provenance Model, using these facets and augment it with the structures necessary to record the provenance of data, analytical process and interpretations. Through an iterative process, we develop and refine a prototype system for Human Terrain Visual Analytics (HTVA), and demonstrate means of storing, browsing and recalling analytical provenance and process through analytic bookmarks in ProveML. We show how these bookmarks can be combined to form narratives that link back to the live data. Throughout the process, we demonstrate that through structured workshops, rapid prototyping and structured communication with intelligence analysts we are able to establish requirements, and design schema, techniques and tools that meet the requirements of the intelligence community. We use the needs and reactions of defence analysts in defining and steering the methods to validate the framework.

This paper aims to address the problems low literacy (LL) users face when searching for information online. The first part of this paper summarizes the problems that LL user's face, and establishes a set of design principles for interfaces suitable for LL users. This is followed by a description of how these design principles are mapped to a novel interface for interactive data retrieval. The interface was realized into a working system and evaluated against a traditional web interface for both high literacy (HL) and LL users. The suitability of the designs was analyzed using performance data, subjective feedback and an observational analysis. The findings from the study suggest that LL users perform better and prefer the proposed designs over a traditional web interface.

Naturalistic Decision Making (NDM) has a special place in the hearts of researchers because of its exploratory nature, and the emphasis on investigating how experts make decisions in “real” situati...

Many authors have argued the need for abroader understanding of context and the situatedness of activity when approaching the evaluation of systems. However, prevailing practice often still tends t...

We describe a work domain analysis of a hydropower plant that operates in the two independent markets, the electricity market and the irrigation market. Based on the results of a work domain analysis, the authors develop configural displays that represent abstract functions of the hydropower system and that are intended as part of an ecological interface. The displays developed cover three critical segments of hydropower system operations: real time power supply operations, water storage management, and electricity trading. We provide detailed descriptions of the displays and of the expected improvements in control quality caused by the each of the displays.

This paper describes ongoing research directed at formulating a set of appropriate measures for assessing and ultimately predicting effort requirements for multimedia systems development. Whilst significant advances have been made in the determination of measures for both transaction-based and process-intensive systems, very little work has been undertaken in relation to measures for multimedia systems. A small preliminary empirical study is reviewed as a precursor to a more exploratory investigation of the factors that are considered by industry to be influential in determining development effort. This work incorporates the development and use of a goal-based framework to assist the measure selection process from a literature basis, followed by an industry questionnaire. The results provide a number of preliminary but nevertheless useful insights into contemporary project management practices with respect to multimedia systems.