Large Language Model (LLM)-powered web GUI agents are increasingly automating everyday online tasks. Despite their popularity, little is known about how users’ preferences and values impact agents’ reasoning and behavior. In this work, we investigate how both explicit and implicit user preferences, as well as the underlying user values, influence agent decision-making and action trajectories. We built a controlled testbed of 14 common interactive web tasks, spanning shopping, travel, dining, and housing, each replicated from real websites and integrated with a low-fidelity LLM-based recommender system. We injected 12 human preferences and values as personas into four state-of-the-art agents and systematically analyzed their task behaviors. Our results show that preference and value-infused prompts consistently guided agents toward outcomes that reflected these preferences and values. While the absence of user preference or value guidance led agents to exhibit a strong efficiency bias and employ shortest-path strategies, their presence steered agents’ behavior trajectories through the greater use of corresponding filters and interactive web features. Despite their influence, dominant interface cues, such as discounts and advertisements, frequently overrode these effects, shortening the agents’ action trajectories and inducing rationalizations that masked rather than reflected value-consistent reasoning. The contributions of this paper are twofold: (1) an open-source testbed for studying the influence of values in agent behaviors, and (2) an empirical investigation of how user preferences and values shape web agent behaviors.

We present EvalAssist, a framework that simplifies the LLM- as-a-judge workflow. The system provides an online criteria development environment, where users can interactively build, test, and share custom evaluation criteria in a structured and portable format. A library of LLM based evaluators is made available that incorporates various algorithmic innovations such as token-probability based judgement, positional bias checking, and certainty estimation that help to engender trust in the evaluation process. We have computed extensive benchmarks and also deployed the system internally in our organization with several hundreds of users.

Lucas Monteiro Paes, Dennis Wei, Hyo Jin Do, Hendrik Strobelt, Ronny Luss, Amit Dhurandhar, Manish Nagireddy, Karthikeyan Natesan Ramamurthy, Prasanna Sattigeri, Werner Geyer, Soumya Ghosh. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Zheyuan Zhang, Yiyang Li, Nhi Ha Lan Le, Zehong Wang, Tianyi Ma, Vincent Galassi, Keerthiram Murugesan, Nuno Moniz, Werner Geyer, Nitesh V Chawla, Chuxu Zhang, Yanfang Ye. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Figure 1: Our system supports nonlinear AI-assisted hypothesis exploration that balances breadth and depth of exploration.Using the node-link diagram shared representation and integrated information hint panels that display preliminary results and related work, a participant in our user study deeply explored a branch of hypotheses around gender income gaps, including a nuanced hypothesis about field-specific gender disparities in income (A); but also simultaneously kept track of the overall hypothesis space, and backtracked to explore other hypothesis branches around variations in income by marital status (B), such as post-divorce impacts on income.A more detailed version Figure 10 can be found in Appendix B.

LLM-as-a-Judge has been widely utilized as an evaluation method in various benchmarks and served as supervised rewards in model training. However, despite their excellence in many domains, potential issues are under-explored, undermining their reliability and the scope of their utility. Therefore, we identify 12 key potential biases and propose a new automated bias quantification framework—CALM—which systematically quantifies and analyzes each type of bias in LLM-as-a-Judge by using automated and principle-guided modification. Our experiments cover multiple popular language models, and the results indicate that while advanced models have achieved commendable overall performance, significant biases persist in certain specific tasks. Empirical results suggest that there remains room for improvement in the reliability of LLM-as-a-Judge. Moreover, we also discuss the explicit and implicit influence of these biases and give some suggestions for the reliable application of LLM-as-a-Judge. Our work highlights the need for stakeholders to address these issues and remind users to exercise caution in LLM-as-a-Judge applications.

User flow diagram for EvalAssist in the direct assessment evaluation, illustrating criteria definition, test data input, annotation, AI evaluator selection, result review, iterative adjustments, and criteria export for dataset-wide evaluation via SDK.

Understanding the dynamics of human-AI interaction in question answering is crucial for enhancing collaborative efficiency. Extending from our initial formative study, which revealed challenges in human utilization of conversational AI support, we designed two configurations for prompt guidance: a Nudging approach, where the AI suggests potential responses for human agents, and a Highlight strategy, emphasizing crucial parts of reference documents to aid human responses. Through two controlled experiments, the first involving 31 participants and the second involving 106 participants, we compared these configurations against traditional human-only approaches, both with and without AI assistance. Our findings suggest that effective human-AI collaboration can enhance response quality, though merely combining human and AI efforts does not ensure improved outcomes. In particular, the Nudging configuration was shown to help improve the quality of the output when compared to AI alone. This paper delves into the development of these prompt guidance paradigms, offering insights for refining human-AI collaborations in conversational question-answering contexts and contributing to a broader understanding of human perceptions and expectations in AI partnerships.

Large Language Models (LLMs) have introduced a paradigm shift in interaction with AI technology, enabling knowledge workers to complete tasks by specifying their desired outcome in natural language. LLMs have the potential to increase productivity and reduce tedious tasks in an unprecedented way. A systematic study of LLM adoption for work can provide insight into how LLMs can best support these workers. To explore knowledge workers' current and desired usage of LLMs, we ran a survey (n=216). Workers described tasks they already used LLMs for, like generating code or improving text, but imagined a future with LLMs integrated into their workflows and data. We ran a second survey (n=107) a year later that validated our initial findings and provides insight into up-to-date LLM use by knowledge workers. We discuss implications for adoption and design of generative AI technologies for knowledge work.

Millions of users come to online peer counseling platforms to seek support. However, studies show that online peer support groups are not always as effective as expected, largely due to users' negative experiences with unhelpful counselors. Peer counselors are key to the success of online peer counseling platforms, but most often do not receive appropriate training. Hence, we introduce CARE: an AI-based tool to empower and train peer counselors through practice and feedback. Concretely, CARE helps diagnose which counseling strategies are needed in a given situation and suggests example responses to counselors during their practice sessions. Building upon the Motivational Interviewing framework, CARE utilizes large-scale counseling conversation data with text generation techniques to enable these functionalities. We demonstrate the efficacy of CARE by performing quantitative evaluations and qualitative user studies through simulated chats and semi-structured interviews, finding that CARE especially helps novice counselors in challenging situations. The code is available at https://github.com/SALT-NLP/CARE.

Generative AI applications present unique design challenges. As generative AI technologies are increasingly being incorporated into mainstream applications, there is an urgent need for guidance on how to design user experiences that foster effective and safe use. We present six principles for the design of generative AI applications that address unique characteristics of generative AI UX and offer new interpretations and extensions of known issues in the design of AI applications. Each principle is coupled with a set of design strategies for implementing that principle via UX capabilities or through the design process. The principles and strategies were developed through an iterative process involving literature review, feedback from design practitioners, validation against real-world generative AI applications, and incorporation into the design process of two generative AI applications. We anticipate the principles to usefully inform the design of generative AI applications by driving actionable design recommendations.

Mitigating algorithmic bias is a critical task in the development and deployment of machine learning models. While several toolkits exist to aid machine learning practitioners in addressing fairness issues, little is known about the strategies practitioners employ to evaluate model fairness and what factors influence their assessment, particularly in the context of text classification. Two common approaches of evaluating the fairness of a model are group fairness and individual fairness. We run a study with Machine Learning practitioners (n=24) to understand the strategies used to evaluate models. Metrics presented to practitioners (group vs. individual fairness) impact which models they consider fair. Participants focused on risks associated with underpredicting / overpredicting and model sensitivity relative to identity token manipulations. We discover fairness assessment strategies involving personal experiences or how users form groups of identity tokens to test model fairness. We provide recommendations for interactive tools for evaluating fairness in text classification.

We present AutoDOViz, an interactive user interface for automated decision optimization (AutoDO) using reinforcement learning (RL). Decision optimization (DO) has classically being practiced by dedicated DO researchers [43] where experts need to spend long periods of time fine tuning a solution through trial-and-error. AutoML pipeline search has sought to make it easier for a data scientist to find the best machine learning pipeline by leveraging automation to search and tune the solution. More recently, these advances have been applied to the domain of AutoDO [36], with a similar goal to find the best reinforcement learning pipeline through algorithm selection and parameter tuning. However, Decision Optimization requires significantly more complex problem specification when compared to an ML problem. AutoDOViz seeks to lower the barrier of entry for data scientists in problem specification for reinforcement learning problems, leverage the benefits of AutoDO algorithms for RL pipeline search and finally, create visualizations and policy insights in order to facilitate the typical interactive nature when communicating problem formulation and solution proposals between DO experts and domain experts. In this paper, we report our findings from semi-structured expert interviews with DO practitioners as well as business consultants, leading to design requirements for human-centered automation for DO with RL. We evaluate a system implementation with data scientists and find that they are significantly more open to engage in DO after using our proposed solution. AutoDOViz further increases trust in RL agent models and makes the automated training and evaluation process more comprehensible. As shown for other automation in ML tasks [33, 59], we also conclude automation of RL for DO can benefit from user and vice-versa when the interface promotes human-in-the-loop.

As more and more forms of AI become prevalent, it becomes increasingly important to understand how people develop mental models of these systems. In this work we study people's mental models of an AI agent in a cooperative word guessing game. We run a study in which people play the game with an AI agent while ``thinking out loud''; through thematic analysis we identify features of the mental models developed by participants. In a large-scale study we have participants play the game with the AI agent online and use a post-game survey to probe their mental model. We find that those who win more often have better estimates of the AI agent's abilities. We present three components---global knowledge, local knowledge, and knowledge distribution---for modeling AI systems and propose that understanding the underlying technology is insufficient for developing appropriate conceptual models---analysis of behavior is also necessary.

AI Security researchers have identified a new way crowdsourced data can be intentionally compromised. Backdoor attacks are a process through which an adversary creates a vulnerability in a machine learning model by ?poisoning?’ the training set by selectively mislabelling images containing a backdoor object. The model continues to perform well on standard testing data but misclassifies on the inputs that contain the backdoor chosen by the adversary. In this paper, we present the design and development of the Backdoor Game, the first game in which users can interact with different poisoned classifiers and upload their own images containing backdoor objects in an engaging way. We conduct semi-structured interviews with eight different participants who interacted with a first version of the Backdoor Game and deploy the game to Mechanical Turk users (N=68) to demonstrate how users interacted with the backdoor objects. We present results including novel types of interactions that emerged as a result of game play and design recommendations for the improvement of the system. The combined design, development and deployment of our system can help AI Security researchers to study this emerging concept, from determining the effectiveness of different backdoor objects to help compiling a collection of diverse and unique backdoor objects from the public, increasing the safety of future AI systems.

Human labeling of training data is often a time-consuming, expensive part of machine learning. In this paper, we study "batch labeling", an AI-assisted UX paradigm, that aids data labelers by allowing a single labeling action to apply to multiple records. We ran a large scale study on Mechanical Turk with 156 participants to investigate labeler-AI-batching system interaction. We investigate the efficacy of the system when compared to a single-item labeling interface (i.e., labeling one record at-a-time), and evaluate the impact of batch labeling on accuracy and time. We further investigate the impact of AI algorithm quality and its effects on the labelers' overreliance, as well as potential mechanisms for mitigating it. Our work offers implications for the design of batch labeling systems and for work practices focusing on labeler-AI-batching system interaction.

As more and more forms of AI become prevalent, it becomes increasingly important to understand how people develop mental models of these systems. In this work we study people's mental models of AI in a cooperative word guessing game. We run think-aloud studies in which people play the game with an AI agent; through thematic analysis we identify features of the mental models developed by participants. In a large-scale study we have participants play the game with the AI agent online and use a post-game survey to probe their mental model. We find that those who win more often have better estimates of the AI agent's abilities. We present three components for modeling AI systems, propose that understanding the underlying technology is insufficient for developing appropriate conceptual models (analysis of behavior is also necessary), and suggest future work for studying the revision of mental models over time.

The rapid advancement of artificial intelligence (AI) is changing our lives in many ways. One application domain is data science. New techniques in automating the creation of AI, known as AutoAI or AutoML, aim to automate the work practices of data scientists. AutoAI systems are capable of autonomously ingesting and pre-processing data, engineering new features, and creating and scoring models based on a target objectives (e.g. accuracy or run-time efficiency). Though not yet widely adopted, we are interested in understanding how AutoAI will impact the practice of data science. We conducted interviews with 20 data scientists who work at a large, multinational technology company and practice data science in various business settings. Our goal is to understand their current work practices and how these practices might change with AutoAI. Reactions were mixed: while informants expressed concerns about the trend of automating their jobs, they also strongly felt it was inevitable. Despite these concerns, they remained optimistic about their future job security due to a view that the future of data science work will be a collaboration between humans and AI systems, in which both automation and human expertise are indispensable.

Many conversational agents (CAs) are developed to answer users' questions in a specialized domain. In everyday use of CAs, user experience may extend beyond satisfying information needs to the enjoyment of conversations with CAs, some of which represent playful interactions. By studying a field deployment of a Human Resource chatbot, we report on users' interest areas in conversational interactions to inform the development of CAs. Through the lens of statistical modeling, we also highlight rich signals in conversational interactions for inferring user satisfaction with the instrumental usage and playful interactions with the agent. These signals can be utilized to develop agents that adapt functionality and interaction styles. By contrasting these signals, we shed light on the varying functions of conversational interactions. We discuss design implications for CAs, and directions for developing adaptive agents based on users' conversational behaviors.

When employees participate in organizational crowdfunding, they seek partial funding from their existing social networks. Among proposers of projects, teams with larger social networks tend to be more successful in reaching their funding goals. However, little is known about the consequences of participation on employees' social networks, during and after the crowdfunding campaign. In a study of activity logs and social networks from a very large scale organizational crowdfunding campaign, we found that people in different crowdfunding roles experienced different degrees of growth in their social networks, during and after the crowdfunding campaign, as compared with baseline nonparticipants. These findings contribute to previous work on the strongly social nature of crowdfunding. Organizations can use these results to increase the density of their internal social networks. Employees can use these results to strategize their participation in workplace social networks and in organizational innovation.

Enterprise social networking (ESN) platforms have been implemented by organizations to support employees' collaboration. However, there has been a reported lack of user participation on these platforms. This study focuses on interventions that can be used to overcome barriers of usage. A case study was conducted at IBM to investigate the effect of several interventions to positively influence ESN acceptance within IBM's Human Resources (HR) function. In this study we combined two rounds of interview data with activity log data. We interviewed the initiators of interventions, and we interviewed the employees targeted by those interventions. The activity logs allowed us to contrast actual usage of the ESN by employees from HR vs. other organizational functions. Compared to other ESN studies this study explicitly identifies factors driving or inhibiting ESN use, and shows how these factors can be addressed by interventions to significantly increase ESN use. Our results can inform and enable other organizations interested in the design of interventions to increase the use of ESN.

Request management occurs at an intersection of CSCW and personal information management (i.e., to-do management), with particular emphasis on social relationships and organizational accountability. We explore diverse work practices and representations for person-to-person requests in organizations, detailing the scatter of communications channels, difficulty of aggregation and prioritization, differences due to context, and the sometimes extraordinary effort and passion that knowledge workers expend on this part of their work, as well as the compassion with which they view their colleagues' efforts. We close with a proposal for the study of socially implicated objects in physical and/or virtual organizations, and implications for the design of services to support request management.

Personal agent software is now in daily use in personal devices and in some organizational settings. While many advocate an agent sociality design paradigm that incorporates human-like features and social dialogues, it is unclear whether this is a good match for professionals who seek productivity instead of leisurely use. We conducted a 17-day field study of a prototype of a personal AI agent that helps employees find work-related information. Using log data, surveys, and interviews, we found individual differences in the preference for humanized social interactions (social-agent orientation), which led to different user needs and requirements for agent design. We also explored the effect of agent proactive interactions and found that they carried the risk of interruption, especially for users who were generally averse to interruptions at work. Further, we found that user differences in social-agent orientation and aversion to agent proactive interactions can be inferred from behavioral signals. Our results inform research into social agent design, proactive agent interaction, and personalization of AI agents.

We present a photo co-creation activity, called "Stitched Groupies," in a photo-taking and sharing platform deployed inside IBM. "Stitched Groupies" allow employees to take and combine photos with peers asynchronously across physical boundaries. In a 25-day exploratory field study with 50 users taking 68 half-photos (of which 52 were completed by others), we categorized themes such as Spliced Faces, Composed Scenes, Body Modifications, Inanimate Objects and Doppelgangers. Our results suggest that photo co-creation can stimulate playfulness and fun in the workplace.

Social ties have been hypothesized to help people to gain support in achieving collaborative goals. We test this hypothesis in a study of organizational crowdfunding (or 'crowdfunding behind the firewall'). 201 projects were proposed for peer-crowdfunding in a large international corporation. The crowdfunding website allowed people to join a project as Co-Proposers. We analyzed the funding success of 114 projects as a function of the number of (Co-)Proposers. Projects that had more co-proposers were more likely to reach their funding targets. Using data from an organizational social-networking service, we show how employees' social ties were associated with these success patterns. Our results have implications for theories of collaboration in social networks, and the design of crowdfunding websites.

Employee engagement is a reflection of an employee's experience of work. Previous research has analyzed each employee's experience in terms of individual factors. We provide the first report of the influence on engagement of peers (who report to the same manager) and friends (who share social ties in an internal social network), using linear regression to model employee engagement in a sample of more than 44,000 employees. We show that an employee's engagement is associated with the engagement of her/his peers, friends, and manager. Our results contribute to analyses of social factors at work, and argue for revisions to existing theories of employee engagement.

Employees increasingly are expressing ideas and feelings through enterprise social media. Recent work in CHI and CSCW has applied linguistic analysis towards understanding employee experiences. In this paper, we apply dictionary based linguistic analysis to measure 'Employee Engagement'. Employee engagement is a measure of employee willingness to apply discretionary effort towards organizational goals, and plays an important role in organizational outcomes such as financial or operational results. Organizations typically use surveys to measure engagement. This paper describes an approach to model employee engagement based on word choice in social media. This method can potentially complement surveys, thus providing more real-time insights into engagement and allowing organizations to address engagement issues faster. Our results predicting engagement scores on a survey by combining demographics with social media text demonstrate that social media text has significant predictive power compared to demographic data alone. We also find that engagement may be a state than a stable trait since social media posts closer to the administration of the survey had the most predictive power. We further identify the minimum number of social media posts required per employee for the best prediction.

Enterprise crowdfunding offers a series of opportunities for voluntary or unplanned collaborations within organizations. In an enterprise crowdfunding experiment, we study the influence of interpersonal attributes-in-common on collaborations. Using ideas from Homophily Theory and Social Identity Theory, we analyze attributes-in-common in terms of multiple identity facets: of geography, of formal corporate structure, and of working groups/teams. We combine quantitative and self-report data to show how each identity facet has an influence on the likelihood of voluntary collaborations, and we show their "superadditive" combination. We propose new questions for theory, and we consider how our results can lead to new features and technologies to enhance voluntary collaborations in organizations.

The rise of social media in the enterprise has enabled new ways for employees to speak up and communicate openly with colleagues. This rich textual data can potentially be mined to better understand the opinions and sentiment of employees for the benefit of the organization. In this paper, we introduce Enterprise Social Pulse (ESP) -- a tool designed to support analysts whose job involves understanding employee chatter. ESP aggregates and analyzes data from internal and external social media sources while respecting employee privacy. It surfaces the data through a user interface that supports organic results and keyword search, data segmentation and filtering, and several analytics and visualization features. An evaluation of ESP was conducted with 19 Human Resources professionals. Results from a survey and interviews with participants revealed the value and willingness to use ESP, but also surfaced challenges around deploying an employee social media listening solution in an organization.

In recent years, with the proliferation of the social web, users are increasingly exposed to social overload and the designers of social web sites are challenged to attract and retain their user basis. Social recommender systems are becoming an integral part of virtually any leading website, playing a key factor in its success: First, they aim to address the overload problem by helping users to find relevant content. Second, they can provide recommendations for content creation, increasing participation and user retention. In this tutorial, we will review the broad domain of social recommender systems, their application for the social web, the underlying techniques and methodologies; the data in use, recommended entities, and target population; evaluation techniques; and open issues and challenges.

The emergence of what is called the social web and the continuing stream of new applications and community-based platforms including Facebook, Twitter, LinkedIn and others had a substantial impact on recommender systems research and practice over the last years in different ways.

We describe a first experiment in enterprise crowdfunding - i.e., employees allocating money for employee-initiated proposals at an Intranet site, including a trial of this system with 511 employees in IBM Research. Major outcomes include: employee proposals that addressed diverse individual and organizational needs; high participation rates; extensive inter-departmental collaboration, including the discovery of large numbers of previously unknown collaborators; and the development of goals and motivations based on collective concerns at multiple levels of project groups, communities of practice, and the organization as a whole. We recommend further, comparative research into crowd-funding and other forms of employee-initiated innovations.

No abstract available.

In this paper, we propose to improve time management in the enterprise by providing users interactive visualizations of how they are spending their time. Through an interview study (n=21) in a multi-national corporation, we were able to determine the data available for visualizations and the value of a number of general visualizations of employees' calendar data. We develop implications for design in improving personal time management.

No abstract available.

We present the system design and rational for a novel social microcalendar called Timely. Our system has been inspired by previous research on calendaring and popular social network applications, in particular microblogging. Timely provides an open, social space for enterprise users to share their events, socialize, and discover what else is going on in their network and beyond. A detailed analysis of the events shared by users during the site's first 47 days reveals that users willingly share their time commitments despite an existing culture of restricted calendars.

The ever blurring line between online interactions and physical encounters presents an interesting challenge when recommending events. Events created on social networking sites may have ambiguous location scope. The location information provided may be fuzzy or non existent and additionally the reach and radius of interest in the event can vary greatly. In this work, we identify four categories of events: global, location dependent and socially independent, socially dependent and location independent, and location and socially dependent. We classify events from an organizations internal event management service where the location of the event is unknown, but the location of the attendees are known in order to improve scoping of event recommendations. Our results, investigate the impact of ignoring location properties when recommending events using classic collaborative filtering techniques. Additionally, once global and socially independent events are identified, they can be used to provide recommendations to new users, addressing the cold-start problem.

Existing enterprise calendaring systems have suffered from problems like rigidity, lack of transparency, and poor integration with social networks. We present the system design and rationale for a novel social coordination mechanism, called "Suggestions," that addresses these issues. Our system integrates ideas drawn from designs of lightweight polling systems and one's social network into an open calendar tool, providing a space for users to coordinate, socialize around, or negotiate the "what" and the "when" of their events. Suggestions was released inside a large enterprise setting, where initial interviews revealed users' thoughts on transparent scheduling, reaching wider audiences and task appropriateness, and suggested ways to improve our design.

Blogging in the enterprise is increasingly popular and recent research has shown that there are numerous benefits for both individuals and the organization, e.g. developing reputation or sharing knowledge. However, participation is very low, blogs are often abandoned and few users realize those benefits. We have designed and implemented a novel system -- called Blog Muse -- whose goal is to inspire potential blog writers by connecting them with their audience through a topic-suggestion system. We describe our system design and report results from a 4-week study with 1004 users who installed our tool. Our data indicate that topics requested by users are effective at inspiring bloggers to write and lead to more social interactions around the resulting entries.

Employee blogging has benefits both for individuals and the organization. In order to inspire the creation of blog posts, we developed a novel topic suggestion system that connects blog readers with blog writers through sharing topics of interest. We describe our system and the results from an employee survey that informed its design.

Social networking sites have begun to be used in the enterprise as a method of connecting employees. Recommender systems may be used to recommend social contacts in order to increase user engagement, encourage collaboration and facilitate expertise discovery. This paper evaluates the effects of four recommendation algorithms on the network as a whole and the social structure. We demonstrate that depending on the basis of the recommendation algorithm the effects on the network vary greatly and their potential impact should be understood. It is hoped this research can be used as guidance for future recommendation algorithms.

The success of social media has resulted in an information overload problem, where users are faced with hundreds of new contributions, edits and communications at every visit. A prime example of this in social networks is the news or activity feeds, where the actions (friending, commenting, photo sharing, etc) of friends on the network are presented to users in order to inform them of the network activity. In this work we endeavour to reduce the burden on individuals of identifying interesting updates in social network news feeds by automatically identifying and recommending relevant items to individuals where item relevance is based on the observed interactions of the individual with the social network. The results of our offline study show that combining short term interest models, exploiting previous viewing behavior of users, and long-term models, exploiting previous viewing of network actions, was the best predictor of feed item relevance.

The exponential growth of the Social Web both poses challenges, and presents opportunities for Recommender System research. The Social Web has turned information consumers into active contributors who generate large volumes of rapidly changing online data. Recommender Systems strive to identify relevant content for users at the right time and in the right context but achieving this goal has become more difficult, in part due to the volume and nature of information contributed through the Social Web.

This paper studies people recommendations designed to help users find known, offline contacts and discover new friends on social networking sites. We evaluated four recommender algorithms in an enterprise social networking site using a personalized survey of 500 users and a field study of 3,000 users. We found all algorithms effective in expanding users' friend lists. Algorithms based on social network information were able to produce better-received recommendations and find more known contacts for users, while algorithms using similarity of user-created content were stronger in discovering new friends. We also collected qualitative feedback from our survey users and draw several meaningful design implications.

Social network sites rely on the contributions of their members to create a lively and enjoyable space. Recent research has focused on using personalization and recommender technologies to encourage participation of existing members. In this work we present an early-intervention approach to encouraging participation and engagement, which makes recommendations to new users during their sign-up process. Our recommender system exploits external social media to produce people and profile entry recommendations for new users. We present results of a live user study, showing that users who received recommendations at sign-up created more social connections, contributed more content, and were on the whole more engaged with the system, contributing more without prompt and returning more often. We further show that recommendations for multiple content types yield significantly better results, in terms of user contribution and consumption; and that recommendations of more active users yield a higher return rate.

No abstract available.

Success and sustainability of social networking sites is highly dependent on user participation. To encourage contribution to an opt-in social networking site designed for employees, we have designed and implemented a feature that rewards contribution with points. In our evaluation of the impact of the system, we found that employees are initially motivated to add more content to the site. This paper presents the analysis and design of the point system, the results of our experiment, and our insights regarding future directions derived from our post-experiment user interviews.

Social networking sites support a variety of shared content types such as photos, videos, or music. More structured or form-based social content types are not mainstream but we have started seeing sites evolve that support them. This paper describes the design and use of structured lists in an enterprise social networking system. As a major feature of our shared lists, we introduced the ability to reuse someone else's list. We report the results on the use and reuse of shared lists based on three months of usage data from 285 users and interviews with 9 users. Our findings suggest that despite the structured nature of lists, our users socialize more around lists than photos, and use lists as a medium for self-representation.

This paper investigates using social tags for the purpose of making personalized content recommendations. Our tag-based recommender creates a personalized bookmark recommendation model for each user based on current and general interest tags, defined by different time intervals.

The introduction of a social networking site inside of a large enterprise enables a new method of communication between colleagues, encouraging both personal and professional sharing inside the protected walls of a company intranet. Our analysis of user behavior and interviews presents the case that professionals use internal social networking to build stronger bonds with their weak ties and to reach out to employees they do not know. Their motivations in doing this include connecting on a personal level with coworkers, advancing their career with the company, and campaigning for their projects.

User profiles on today's social networking sites support only a small set of predefined questions. We report on an alternative way for users to richly describe themselves, by entering not only responses, but their own questions as well. Data from 10 months of usage shows that users of a social networking site created thousands of diverse questions and reused existing questions from other users. Our findings suggest that those with highly diverse user profiles have a higher number of friends.

Knowledge workers perform many different activities daily. Each activity defines a distinct work context with different information needs. In this paper we leverage users' activity representations, stored in an activity management system, to automatically recommend resources to support knowledge workers in their current activity. We developed a collaborative activity predictor to both predict the current work activity and measure a resource's relevance to a specific activity. Relevant resources are then displayed in a contextual side bar on the desktop. We describe the system, our new activity-centric search algorithm, and experimental results based on the data from 50 real users.

Traditional social networking sites allow users to enter responses to a set of predefined fields when populating their personal profiles. In the system discussed in this work, freeform 'About You' entries allow users to craft their own questions / topics. We found that this kind of flexibility often leads to low content contributions and infrequent updates. The 'About You' recommender system described in this paper differs from many recommender systems in that it recommends content for users to create, rather than consume. We present empirical data from an experiment with 2,000 users of a social networking site during a one month period. Our findings suggest that users who receive recommendations create more entries and update them more over time. Further, using articulated social network information for recommendations performed better than content-based matching.

Activity-centric collaboration environments help knowledge workers to manage the context of their shared work activities by providing a representation for an activity and its resources. Activity management systems provide more structure and organization than email to execute the shared activity but, as the number of shared activities increases, it becomes more and more difficult for users to focus on important activities that need their attention. This paper describes a personalized activity prioriti-zation approach implemented on top of the Lotus Connections Activities management system. Our prototype implementation allows each user to view activities ordered by her/his predicted priorities. The predictions are made using a ranking Support Vector Machine model trained with the user’s past interactions with the activities system. We describe the prioritization interface and the results of an offline experiment based on data from 13 users over 6-months. Our results show that our feature set derived from shared activity structures can significantly increase prediction accuracy compared to a recency baseline.

We describe the Dogear Game, which works with an enterprise social bookmarking system. The game is designed to accomplish individual, collaborative, and organization goals. Individual players receive entertainment and learn about their colleagues' bookmarks. The player's colleagues receive recommendations of websites and documents of potential interest to them. And the organization benefits from a richer knowledge-base of bookmarks as recommendations are accepted. The Dogear Game builds on von Ahn's "serious games," useful in motivating and distributing game-like entertaining "work" to a large group of game players. This note presents the design and implementation of a working prototype and some initial user feedback.

As the number of alerts generated by collaborative applications grows, users receive more unwanted alerts. FeedMe is a general alert management system based on XML feed protocols such as RSS and ATOM. In addition to traditional rule-based alert filtering, FeedMe uses techniques from machine-learning to infer alert preferences based on user feedback. In this paper, we present and evaluate a new collaborative naive Bayes filtering algorithm. Using FeedMe, we collected alert ratings from 33 users over 29 days. We used the data to design and verify the accuracy of the filtering algorithm and provide insights into alert prediction.

This paper describes a new collaboration technology that is based on the support of lightweight, informally structured, opportunistic activities featuring heterogeneous threads of shared items with dynamic membership. We introduce our design concepts, and we provide a detailed analysis of user behavior during a five month field study. We present the patterns of media use that we observed, using a variety of analytical methods including thread clustering and analysis. Major findings include four patterns of media use: communicating, exchanging mixed objects, coordinating, (e.g., of status reports), and semi-archival filing. We observed differential use of various media including highly variable use of chats and surprisingly informal uses of files. We discuss the implications for the design of mixed media collaborative tools to support the work activities of small to medium sized work teams.

Chat Spaces are rich persistent chats that provide light-weight shared workspaces for small to medium-scale group activities. Chat Spaces can accommodate brief, informal interactions (similar to Instant Messaging), and can also support longer-term complex threaded conversations including large numbers of people and shared resources. Our design maps a hierarchical thread representation onto a time-ordered two-column user interface. This mapping allows a user to follow the global dynamics of the entire thread in the chronological column on the left while being able to participate in a selected topical branch in a second column on the right. We also present a dynamic thread map that provides an overview of the entire conversation and supports quick navigation of topical branches in the thread.

This paper describes a new collaboration technology that is carefully poised between informal, ad hoc, easy-to-initiate collaborative tools, vs. more formal, structured, and high-overhead collaborative applications. Our approach focuses on the support of lightweight, informally structured, opportunistic activities featuring heterogeneous threads of shared objects with dynamic membership. We introduce our design concepts, and we provide a detailed first look at data from the first 100 days of usage by 20 researchers and 13 interns, who both confirmed our hypotheses and surprised us by reinventing the technology in several ways.

We describe a new collaborative technology that is mid-way between the informality of email and the formality of shared workspaces. Email and other ad hoc collaboration systems are typically lightweight and flexible, but build up an unmanageable clutter of copied objects. At the other extreme, shared workspaces provide formal, structured collaboration, but are too heavyweight for users to set up. To bridge this gap between the ad hoc and formal, this paper introduces the notion of "object-centric sharing", where users collaborate in a lightweight manner but aggregate and organize different types of shared artifacts into semi-structured activities with dynamic membership, hierarchical object relationships, as well as real-time and asynchronous collaboration. We present a working prototype implemented with a replicated peer-to-peer architecture, which we describe in detail, and demonstrate its performance in synchronous and asynchronous modes.

Meetings contain a large amount of rich project information that is often not documented. Capturing audio and video of a meeting can provide a comprehensive meeting record. However, finding detailed information in that record can be challenging because there is no structural information other than time to help the user navigate. This paper surveys various ways of creating indices into meeting records and introduces the notion of creating indices based upon user interaction with domain-specific artifacts. As an example, we present a prototype system that uses general team artifacts to provide meaningful pointers into the meeting record.

In this paper, we address the design issues of a collaborative workspace system, called TeamSpace, that supports geographically distributed teams by managing shared work processes and maintaining shared artifacts in a project. TeamSpace attempts to integrate both synchronous and asynchronous types of team interaction into a task-oriented environment. Since meetings are an integral part of teamwork, our current work focuses on supporting virtual meetings as part of a larger collaborative work process. We present an initial TeamSpace prototype that supports asynchronous meeting management seamlessly integrated with capture and access of synchronous distributed meetings. The captured synchronous data is integrated with other related information in TeamSpace, enabling users to efficiently gain knowledge of both current and past team activities.

We propose a novel late join algorithm for distributed applications with a fully replicated architecture (e.g. shared whiteboards). The term 'late join algorithm' is used to denote a mechanism that allows a late-coming participant rejoin an ongoing session. Generally, this requires that participants in the session provide the latecomer with the current state of the shared application. We identify the key issues of late join algorithms and propose a set of requirements which a 'good' late join approach should satisfy. Based on these requirements, we evaluate existing late join algorithms and explain why we opted instead to develop a new, advanced late join algorithm for our own shared whiteboard. This late join approach is general enough to be used for arbitrary distributed applications.

In this paper we present a generic late-join service for distributed interactive media, i.e, networked media which involve user interactions. Examples for distributed interactive media are shared whiteboards, networked computer games and distributed virtual environments. The generic late-join service allows a latecomer to join an ongoing session. This requires that the shared state of the medium is transmitted from the old participants of the session to the latecomer in an efficient and scalable way. In order to be generic and useful for a broad range of distributed interactive media, we have implemented the late-join service based on the Real Time Application Level Protocol for Distributed Interactive Media (RTP/I). All applications which employ this protocol can also use the generic late-join service. Furthermore the late-join service can be adapted to the specific needs of a given application by specifying policies for the late-join process. Applications which do use a different application level protocol than RTP/I may still use the concepts presented in this work. However, they will not be able to profit from our RTP/I based implementation.