Hackathons are time-bound collaborative events that often target software creation. Although hackathons have been studied in the past, existing work focused on in-depth case studies limiting our understanding of hackathons as a software engineering activity.

Hackathons have become popular collaborative events for accelerating the development of creative ideas and prototypes. There are several case studies showcasing creative outcomes across domains such as industry, education, and research. However, there are no large-scale studies on creativity in hackathons which can advance theory on how hackathon formats lead to creative outcomes. We conducted a computational analysis of 193,353 hackathon projects. By operationalizing creativity through usefulness and novelty, we refined our dataset to 10,363 projects, allowing us to analyze how participant characteristics, collaboration patterns, and hackathon setups influence the development of creative projects. The contribution of our paper is twofold: We identified means for organizers to foster creativity in hackathons. We also explore the use of large language models (LLMs) to augment the evaluation of creative outcomes and discuss challenges and opportunities of doing this, which has implications for creativity research at large.

Prescriptive Process Monitoring (PrPM) systems recommend interventions in ongoing business process cases to improve performance. However, performance gains only materialize if users follow the recommendations. Prior research has shown that users are more likely to follow recommendations when they understand them. In this paper, we explore the use of Large Language Models (LLMs) to generate explanations for PrPM recommendations. We developed a prompting method based on typical user questions and integrated it into an existing PrPM system. Our evaluation indicates that LLMs can help users of PrPM systems to better understand the recommendations. The results indicate that LLMs can help users of PrPM systems to better understand the recommendations, and to produce recommendations that have sufficient detail and fulfill their expectations. However, the explanations fall short in addressing the underlying "why" and do not always support users in assessing the trustworthiness of the recommendations.

Hybrid hackathons, which combine in-person and online participation, present unique challenges for organizers and participants. Although now widely practiced, research on them remains fragmented, with limited integration between hackathon studies and hybrid collaboration theories. Existing strategies for in-person or online-only events often fail to address hybrid-specific issues, such as managing communication across physical and virtual spaces and ensuring balanced participation. Our work examines hybrid hackathons through the lens of hybrid collaboration theories, focusing on how organizers structure these events and how participants navigate associated challenges. We frame our analysis using established theories of hybrid collaboration, focusing on four key dimensions - synchronicity, physical distribution, dynamic transitions, and technological infrastructure - that shape collaboration in hybrid events. Using an exploratory case study of three hybrid hackathons, involving observations and interviews with organizers and participants, we investigate how these dimensions are implemented and how they shape participant experiences. Our findings show differing organizer approaches to the hybrid dimensions, leading to varied participant experiences in access to resources, communication, and coordination. Implementation styles, favoring in-person, online, or balanced participation, produced disparities in engagement. Organizers often relied on technology to bridge physical and virtual spaces, but overlooked critical aspects such as time-zone management, dynamic transitions, and targeted hybrid team support. Participants, in turn, adapted collaboration tactics in response to gaps in event scaffolding, sometimes enabling collaboration, but at other times creating new challenges, underscoring that considerations for the hybrid format are still not fully integrated into hackathon planning. Learning from our findings, we offer practical recommendations for both organizers and participants to improve planning, participation, and collaboration in hybrid hackathon events.

Abstract Prescriptive process monitoring (PrPM) systems analyze ongoing business process instances to recommend real-time interventions that optimize performance. The usefulness of these systems hinges on users applying the generated recommendations. Thus, users need to understand the rationale behind these recommendations. One way to build this understanding is to enhance each recommendation with explanations. Existing approaches generate explanations consisting of static text or plots, which users often struggle to understand. Previous work has shown that dialogue systems enhance the effectiveness of explanations in recommender systems. Large Language Models (LLMs) are an emerging technology that facilitates the construction of dialogue systems. In this paper, we investigate the applicability of LLMs for generating explanations in PrPM systems. Following a design science approach, we elicit explainability questions that users may have for PrPM outputs, we design a prompting method on this basis, and we conduct an evaluation with potential users to assess their perception of the explanations and their approach to interact with the system. The results indicate that LLMs can help users of PrPM systems to better understand the origin of the recommendations, and to produce recommendations that have sufficient detail and fulfill their expectations. On the other hand, users find that the explanations do not always address the “why” of a recommendation and do not let them judge if they can trust the recommendation.

Multimodal Learning Analytics (MMLA) is an evolving sector within learning analytics that has become increasingly useful for examining complex learning and collaboration dynamics for group work across all educational levels. The availability of low-cost sensors and affordable computational power allows researchers to investigate different modes of group work. However, the field faces challenges stemming from the complexity and specialization of the systems required for capturing diverse interaction modalities, with commercial systems often being expensive or narrow in scope and researcher-developed systems needing to be more specialized and difficult to deploy. Therefore, more user-friendly, adaptable, affordable, open-source, and easy-to-deploy systems are needed to advance research and application in the MMLA field. The paper presents a field report on the design of mBox that aims to support group work across different contexts. We share the progress of mBox, a low-cost, easy-to-use platform grounded on learning theories to investigate collaborative learning settings. Our approach has been guided by iterative design processes that let us rapidly prototype different solutions for these settings.

Abstract Prescriptive process monitoring methods recommend interventions during the execution of a process to maximize its success rate. Current research in this field focuses on algorithms to learn intervention policies that maximize the expected payoff of the interventions under certain statistical assumptions. In contrast, there has been limited attention on how to aid process stakeholders in understanding the outputs of these algorithms. In this research, we set to develop an interface to provide end users with relevant information to guide the decision on where and when to trigger interventions in a process. We draw upon an analysis of existing solutions and a review of the literature to elicit information items for a user interface for prescriptive process monitoring. Thereon, we develop a user interface concept and evaluate it with experts. The evaluation confirms the informational needs covered by the user interface concept. In addition, the evaluation shows that different end-user groups (operational users, tactical managers, and process analysts) can benefit from the information items included in the interface.

Abstract Prescriptive process monitoring methods recommend interventions during the execution of a case that, if followed, can improve performance. Research on prescriptive process monitoring so far has focused mainly on improving the underlying algorithms and providing suitable explanations for recommendations. Empirical works indicate, though, that process workers often do not follow recommendations even if they understand them. Drawing inspiration from the field of persuasive technology, we developed and evaluated a visualization that nudges process workers towards accepting a recommendation, following a design science approach. Our evaluation points towards the feasibility of the visualization and provides insights into how users perceive different persuasive elements, thus providing a basis for the design of future systems.

We explore the impact of Casual Affective Triggers (CAT) on response rates of online surveys. As CAT, we refer to objects that can be included in survey participation invitations and trigger participants’ affect. The hypothesis is that participants who receive CAT-enriched invitations are more likely to respond to a survey. We conducted a study where the control condition received invitations without affective triggers, and the experimental condition received CAT-enriched invitations. We differentiated the triggers within the experimental condition: one-third of the population received a personalized invitation, one-third received a picture of the surveyor’s cat, and one-third received both. We followed up with a survey to validate our findings. Our results suggest that CATs have a positive impact on response rates. We did not find CATs to induce response bias.

Abstract Organizations have a vital interest in continuously improving their business processes. Process analysts can use process mining tools that provide data-driven discovery and analysis of business processes to achieve this. Current research has mainly focused on creating and evaluating new tools or reporting process mining case studies from different domains. Although usage of process mining has increased in industry, insights into how analysts work with such methods to identify improvement opportunities have consequently been limited. To reduce this gap, we conducted an exploratory interview study of seven process analysts from different domains. Our findings indicate that process analysts assess improvement opportunities by their impact, the feasibility of required implementation, and stakeholders’ input. Furthermore, our results indicate that process mining tools, when used to identify improvement opportunities, do not provide sufficient support for analysis, requiring process analysts to use additional tools. Lastly, analysts use storytelling to frame and communicate their findings to various stakeholders.

With ubiquitous, mobile computing, health care systems, and smart factories, socio-technical phenomena continue to emerge that challenge traditional design and evaluation methods. We perceive such phenomena as the intertwinement of technical artifacts and social practices. Previous work shows that there is no sufficient method to evaluate the quality of this socio-technical intertwinement. Hence, our goal was to develop socio-technical heuristics, in short ST-heuristics, that can be applied by individuals to detect issues. Drawing inspiration from the success of usability heuristics in the field of human–computer interaction, we first applied a literature review to develop an initial set of ST-heuristics derived from six domains comprising groupware/computer-support cooperative work, job design, usability, socio-technical design principles, privacy, and process design. We then conducted two studies to evaluate and improve this set using empirical data from 13 cases from health care, industry, and engineering education fields. In total, we analysed 306 problems. The results substantiate a final set of eight ST-heuristics which allow for evaluating the socio-technical intertwinement in situ. We perceive the contribution of this work as a starting point for evaluators to uncover crucial issues and to improve current practice. We discuss the developed set of ST-heuristics within existing literature.

Context: Hackathons have become popular events for teams to collaborate on projects and develop software prototypes. Most existing research focuses on activities during an event with limited attention to the evolution of the hackathon code. Objective: We aim to understand the evolution of code used in and created during hackathon events, with a particular focus on the code blobs, specifically, how frequently hackathon teams reuse pre-existing code, how much new code they develop, if that code gets reused afterwards, and what factors affect reuse. Method: We collected information about 22,183 hackathon projects from Devpost and obtained related code blobs, authors, project characteristics, original author, code creation time, language, and size information from World of Code. We tracked the reuse of code blobs by identifying all commits containing blobs created during hackathons and identifying all projects that contain those commits. We also conducted a series of surveys in order to gain a deeper understanding of hackathon code evolution that we sent out to hackathon participants whose code was reused, whose code was not reused, and developers who reused some hackathon code. Result: 9.14% of the code blobs in hackathon repositories and 8% of the lines of code (LOC) are created during hackathons and around a third of the hackathon code gets reused in other projects by both blob count and LOC. The number of associated technologies and the number of participants in hackathons increase reuse probability. Conclusion: The results of our study demonstrates hackathons are not always "one-off" events as the common knowledge dictates and it can serve as a starting point for further studies in this area.

Time-bounded events such as hackathons, data dives, codefests, hack-days, sprints or edit-a-thons have increasingly gained attention from practitioners and researchers. Yet there is a paucity of research on corporate hackathons, which are nearly ubiquitous and present significant organizational, cultural, and managerial challenges. To provide a comprehensive understanding of team processes and broad array of outcomes of corporate hackathons, we conducted a mixed-methods, multiple case study of five teams that participated in a large scale corporate hackathon. Two teams were “pre-existing” teams (PETs) and three were newly-formed “flash” teams (FTs). Our analysis revealed that PETs coordinated almost as if it was just another day at the office while creating innovations within the boundary of their regular work, whereas FTs adopted role-based coordination adapted to the hackathon context while creating innovations beyond the boundary of their regular work. Project sustainability depended on how much effort the team put into finding a home for their projects and whether their project was a good fit with existing products in the organization’s product portfolio. Moreover, hackathon participation had perceived positive effects on participants’ skills, careers, and social networks.

Hackathons and similar time-bounded events have become a popular form of collaboration in various domains. They are commonly organized as in-person events during which teams engage in intense collaboration over a short period of time to complete a project that is of interest to them. Most research to date has thus consequently focused on studying how teams collaborate in a co-located setting, pointing towards the advantages of radical co-location. The global pandemic of 2020, however, has led to many hackathons moving online, which challenges our current understanding of how they function. In this paper, we address this gap by presenting findings from a multiple-case study of 10 hackathon teams that participated in 4 hackathon events across two continents. By analyzing the collected data, we found that teams merged synchronous and asynchronous means of communication to maintain a common understanding of work progress as well as to maintain awareness of each other's tasks. Task division was self-assigned based on individual skills or interests, while leaders emerged from different strategies (e.g., participant experience, the responsibility of registering the team in an event). Some of the affordances of in-person hackathons, such as the radical co-location of team members, could be partially reproduced in teams that kept open synchronous communication channels while working (i.e., shared audio territories), in a sort of "radical virtual co-location". However, others, such as interactions with other teams, easy access to mentors, and networking with other participants, decreased. In addition, the technical constraints of the different communication tools and platforms brought technical problems and were overwhelming to participants. Our work contributes to understanding the virtual collaboration of small teams in the context of online hackathons and how technologies and event structures proposed by organizers imply this collaboration.

With the imminent addition of autonomous vehicles to traffic, it is becoming more vital to look at different alternatives to non-verbal communication between the driver and the pedestrian, so that the pedestrian would understand the intentions of autonomous vehicles. The aim of this paper is to evaluate different approaches to communicate the intent of an autonomous vehicle. A survey study was conducted among Estonian people to analyze their understanding of animations of more prominent explicit external interaction modalities. The study revealed that participants may not understand the vehicle intent if they have no prior knowledge about the displayed signals.

For humans and robots to share space and cooperate, communication is essential. In the communication between a human and a robot it is important for the machine to understand the human but also for the human to understand the information conveyed by the robot. This requires, that the robot would be able to convey its intent to humans in an understandable way. The aim of this article is to evaluate different approaches to communicate the intent of a machine. A survey study was conducted to analyze the understanding of pseudo-emotional states and other status expressions of a robot. The study of 171 responses revealed that participants are less likely to correctly interpret the emotions and/or status of a machine if only robot’s eyes are used to express the emotions and system status.

Background: Hackathons have become popular events for teams to collaborate on projects and develop software prototypes. Most existing research focuses on activities during an event with limited attention to the evolution of the code brought to or created during a hackathon. Aim: We aim to understand the evolution of hackathon-related code, specifically, how much hackathon teams rely on pre-existing code or how much new code they develop during a hackathon. Moreover, we aim to understand if and where that code gets reused, and what factors affect reuse. Method: We collected information about 22,183 hackathon projects from Devpost- a hackathon database - and obtained related code (blobs), authors, and project characteristics from the World of Code. We investigated if code blobs in hackathon projects were created before, during, or after an event by identifying the original blob creation date and author, and also checked if the original author was a hackathon project member. We tracked code reuse by first identifying all commits containing blobs created during an event before determining all projects that contain those commits. Result: While only approximately 9.14% of the code blobs are created during hackathons, this amount is still significant considering time and member constraints of such events. Approximately a third of these code blobs get reused in other projects. The number of associated technologies and the number of participants in a project increase reuse probability. Conclusion: Our study demonstrates to what extent pre-existing code is used and new code is created during a hackathon and how much of it is reused elsewhere afterwards. Our findings help to better understand code reuse as a phenomenon and the role of hackathons in this context and can serve as a starting point for further studies in this area.

Declarative process modeling languages are especially suitable to model loosely-structured, unpredictable business processes. One of the most prominent of these languages is Declare. The Declare language can be used for all process mining branches and a plethora of techniques have been implemented to support process mining with Declare. However, using these techniques can become cumbersome in practical situations where different techniques need to be combined for analysis. In addition, the use of Declare constraints in practice is often hampered by the difficulty of modeling them: the formal expression of Declare is difficult to understand for users without a background in temporal logics, whereas its graphical notation has been shown to be unintuitive. In this paper, we present RuM, a novel application for rule mining that addresses the abovementioned issues by integrating multiple Declare-based process mining methods into a single unified application. The process mining techniques provided in RuM strongly rely on the use of Declare models expressed in natural language, which has the potential of mitigating the barriers of the language bias. The application has been evaluated by conducting a qualitative user evaluation with eight process analysts.

Time-based events, such as hackathons and codefests, have become a global phenomenon attracting thousands of participants to hundreds of events every year. While research on hackathons has grown considerably, there is still limited insight into what happens to hackathon projects after the event itself has ended. While case studies have provided rich descriptions of hackathons and their aftermath, we add to this literature a large-scale quantitative study of continuation across hackathons in a variety of domains. Our findings indicate that a considerable number of projects get continued after a hackathon has ended. Our results also suggest that short- and long-term continuation are different phenomena. While short-term continuation is associated with technical preparation, number of technologies used in a project and winning a hackathon, long-term continuation is predicated on skill diversity among team members, their technical capabilities in relationship to the technologies and their intention to expand the reach of a project. Moreover, we found intensive short-term activity to be associated with a lower likelihood of long-term project continuation.

Time-bounded events such as hackathons have become a global phenomenon. Scientific communities in particular show growing interest in organizing them to attract newcomers and develop technical artifacts to expand their code base. Current hackathon approaches presume that participants have sufficient expertise to work on projects on their own. They only provide occasional support by domain experts serving as mentors which might not be sufficient for newcomers. Drawing from work on workplace and educational mentoring, we developed and evaluated an approach where each hackathon team is supported by a community member who serves in a mentor role that goes beyond providing occasional support. Evaluating this approach, we found that teams who took ownership of their projects, set achievable goals early while building social ties with their mentor and receiving learning-oriented support reported positive perceptions related to their project and an increased interest in the scientific community that organized the hackathon. Our work thus contributes to our understanding of mentoring in hackathons, an area which has not been extensively studied. It also proposes a feasible approach for scientific communities to attract and integrate newcomers which is crucial for their long-term survival.

Sustained participation by contributors in opensource software is critical to the survival of open-source projects and can provide career advancement benefits to individual contributors. However, not all contributors reap the benefits of open-source participation fully, with prior work showing that women are particularly underrepresented and at higher risk of disengagement. While many barriers to participation in open-source have been documented in the literature, relatively little is known about how the social networks that open-source contributors form impact their chances of long-term engagement. In this paper we report on a mixed-methods empirical study of the role of social capital (i.e., the resources people can gain from their social connections) for sustained participation by women and men in open-source GitHub projects. After combining survival analysis on a large, longitudinal data set with insights derived from a user survey, we confirm that while social capital is beneficial for prolonged engagement for both genders, women are at disadvantage in teams lacking diversity in expertise.

Time bounded events such as hackathons, data dives, codefests, hack-days, sprints or edit-a-thons have increasingly gained attention from practitioners and researchers. Existing research, however, has mainly focused on the event itself, while potential outcomes of hackathons have received limited attention. Furthermore, most research around hackathons focuses on collegiate or civic events. Research around hackathons internal to tech companies, which are nearly ubiquitous, and present significant organizational, cultural, and managerial challenges, remains scarce. In this paper we address this gap by presenting findings from a case study of five teams which participated in a large scale corporate hackathon. Most team members voiced their intentions to continue the projects their worked on during the hackathon, but those whose projects did get continued were characterized by meticulous preparation, a focus on executing a shared vision during the hackathon, extended dissemination activities afterwards and a fit to existing product lines. Such teams were led by individuals who perceived the hackathon as an opportunity to bring their idea to life and advance their careers, and who recruited teams who had a strong interest in the idea and in learning the skills necessary to contribute efficiently. Our analysis also revealed that individual team members perceived hackathon participation to have positive effects on their career parts, networks and skill development.

Organizations conduct series of face-to-face meetings aiming to improve work practices. In these meetings, participants from different backgrounds collaboratively design artifacts, such as knowledge or process maps. Such meetings are orchestrated and carried out by facilitators and the success of the meetings almost solely depends on the experience of the facilitators. Previous research has mainly focused on approaches that support facilitators and participants in the upfront planning of such events. There is however, little guidance for facilitators and participants once a meeting has started. One critical aspect -- among others -- is that during a meeting, the facilitator and participants need to decide for how long the iterative process of discussion and design should continue. We argue that we can provide support for such decisions based on the evolution of artifacts collaboratively created during such meetings. This paper presents a multi-level, multi-method analysis of artifacts based on experts' observations in combination with network analytics. We study the use of automated analytics to assess the evolution of collaboratively created artifacts and to indicate maturity and established consensus of the collaborative practice. We propose a computational approach to support facilitators and participants in deciding when to stop face-to-face meetings.

Business process modeling has received a lot of attention from practitioners and researchers alike. Organizations make significant investments into process modeling in terms of training, tools, and resources. Yet, having invested into creating large process model collections, process models often fall into disuse, provoking the impression that the initial investment has been lost. While previous work has aimed at exploring model re-use as a design principle, our work examines repeated use of a model as a behavior and thus aims at identifying factors that facilitate or hinder the repeated use of process models by individual users . We develop a conceptual model of factors that can influence an individual's intention to repeatedly use process models. We evaluate this model through a cross-sectional survey of process model users from a large European financial institution. Our results indicate the importance of quality and ease of understanding of process models to repeated use, alongside individual factors, such as motivation and individual expertise. We identify means that help organizations to promote the repeated use of process models, which can assist them to increase the benefits of process modeling.