Analyzing large quantities of real‐world textual data has the potential to provide new insights for researchers. However, such data present challenges for both human and computational methods, requiring a diverse range of specialist skills, often shared across a number of individuals. In this paper we use the analysis of a real‐world data set as our case study, and use this exploration as a demonstration of our “insight workflow,” which we present for use and adaptation by other researchers. The data we use are impact case study documents collected as part of the UK Research Excellence Framework (REF), consisting of 6,679 documents and 6.25 million words; the analysis was commissioned by the Higher Education Funding Council for England (published as report HEFCE 2015). In our exploration and analysis we used a variety of techniques, ranging from keyword in context and frequency information to more sophisticated methods (topic modeling), with these automated techniques providing an empirical point of entry for in‐depth and intensive human analysis. We present the 60 topics to demonstrate the output of our methods, and illustrate how the variety of analysis techniques can be combined to provide insights. We note potential limitations and propose future work.

In this paper we propose a methodology to understand complex concepts, and which captures aspects of the contextual —and collaboratively constructed — meaning of these concepts with considerably less effort than manual coding. We use the word "quality" as one such concept to exemplify our methodology. By using unsupervised topic models along with a small corpus of human labeled data we explore the different uses of the concept "quality" in a large number of blogs. Our methodology is validated, qualitatively, by comparing our results to previous research. Finally, we note limitations and future directions of this work.

Blogs are a widely growing form of computer-mediated communication used to achieve various personal and professional communicative goals. In the present study, we examined previously posted entries from 71 regular bloggers. We examined the blogs for the use of five forms of verbal irony: hyperbole, understatement, rhetorical question, sarcasm and jocularity. In addition, topic and emotional valence of the ironic utterances were examined. Results showed that hyperbole and understatement were more frequently used than the other forms of ironic language. Discussion of hobbies and social outings was the most commonly occurring topic of ironic language, and bloggers used verbal irony to convey both positive and negative intent. The results of this study demonstrated that adult bloggers do use a variety of forms of verbal irony in their personal blogs, despite the potential risk of being misunderstood.

Privacy is frequently a key concern relating to technology and central to HCI research, yet it is notoriously difficult to study in a naturalistic way. In this paper we describe and evaluate a dictionary of privacy designed for content analysis, derived using prototype theory and informed by traditional theoretical approaches to privacy. We evaluate our dictionary categories alongside privacy-related categories from an existing content analysis tool, LIWC, using verbal discussions of privacy issues from a variety of technology and non-technology contexts. We find that our privacy dictionary is better able to distinguish between privacy and non-privacy language, and is less context-dependent than LIWC. However, the more general LIWC categories are able to describe a greater amount of variation in our data. We discuss possible improvements to the privacy dictionary and note future work.

Abstract This article presents the privacy dictionary, a new linguistic resource for automated content analysis on privacy‐related texts. To overcome the definitional challenges inherent in privacy research, the dictionary was informed by an inclusive set of relevant theoretical perspectives. Using methods from corpus linguistics, we constructed and validated eight dictionary categories on empirical material from a wide range of privacy‐sensitive contexts. It was shown that the dictionary categories are able to measure unique linguistic patterns within privacy discussions. At a time when privacy considerations are increasing and online resources provide ever‐growing quantities of textual data, the privacy dictionary can play a significant role not only for research in the social sciences but also in technology design and policymaking.

This paper examines how different forms of linguistic similarity in a text-chat environment relate to the establishment of interpersonal trust. Sixty-two pairs played an iterative social dilemma investment game and periodically communicated via Instant Messenger (IM). Novel automated and manual analysis techniques identify linguistic similarity at content, structural and stylistic levels. Results reveal that certain types of content (some positive emotion words, task-related words), structural (verb tense, phrasal entrainment), and stylistic (emoticons) similarity characterize high trusting pairs while other types of similarity (e.g., negative emotion words) characterize low trusting pairs. Contrary to previous literature, this suggests that not all similarity is good similarity.

Being able to automatically perceive a variety of emotions from text alone has potentially important applications in CMC and HCI that range from identifying mood from online posts to enabling dynamically adaptive interfaces. However, such ability has not been proven in human raters or computational systems. Here we examine the ability of naive raters of emotion to detect one of eight emotional categories from 50 and 200 word samples of real blog text. Using expert raters as a 'gold standard', naive-expert rater agreement increased with longer texts, and was high for ratings of joy, disgust, anger and anticipation, but low for acceptance and 'neutral' texts. We discuss these findings in light of theories of CMC and potential applications in HCI.

Emotion is central to human interactions, and automatic detection could enhance our experience with technologies. We investigate the linguistic expression of fine-grained emotion in 50 and 200 word samples of real blog texts previously coded by expert and naive raters. Content analysis (LIWC) reveals angry authors use more affective language and negative affect words, and that joyful authors use more positive affect words. Additionally, a co-occurrence semantic space approach (LSA) was able to identify fear (which naive human emotion raters could not do). We relate our findings to human emotion perception and note potential computational applications.

This study examines the relationship between linguistic mimicry and trust establishment in a text-chat environment. Twenty-six participant pairs engaged in a social dilemma investment game and chatted via Instant Messenger (IM) after every five rounds of investment. Results revealed that, within chat sessions, lexical mimicry (repetition of words or word phrases by both partners) was significantly higher for high-trusting pairs than for low-trusting pairs, but that lexical mimicry across chat sessions was significantly higher for low-trusting pairs than for high-trusting pairs. Theoretical and applied implications are discussed.