Disinformation is an escalating global threat, making it essential to understand its content, dissemination, and evolution. To confront this challenge, researchers have begun grouping related false claims into broader disinformation narratives, which can be tracked across cultures, time periods, and media sources. Analyzing these narratives provides critical insights for developing more effective countermeasures. To this end, we introduce DiNO: Disinformation Narrative Observer, a novel method designed to extract disinformation narratives from news articles. We applied DiNO to news articles on the Ukraine War, COVID-19 and Migration, sourced from disinformation-prone outlets as well as a reputable source. We evaluated the narratives extracted by DiNO by measuring how well their topics and stances aligned with a recognized disinformation narratives dataset. DiNO outperforms competitive narrative mining approaches, including Relatio and CaNarEx, achieving a 41%–44% improvement in topical alignment and a 30%–41% improvment in stance alignment.

Clarity and ease of interaction are critical for mobile devices that users rely on daily. As smartphone use time rises, manufacturers are exploring e-paper displays for their advantages, including longer battery life and reduced eye strain. Yet, e-paper technology comes with inherent design limitations, such as reduced responsiveness, constrained color ranges, and ghosting. To address these limitations, we present the first version of our E-Paper Design System comprising a set of design components and guidelines developed specifically for e-paper smartphones, with a particular focus on minimalist devices. The E-Paper Design System is released as a free, open resource on Zeroheight and documented in the Appendix for reference. Our design decisions are grounded in technical constraints, prior research and insights from designers and developers using this pilot version of the design system to create custom mobile applications. We also report findings from an exploratory in-house UX study (N=24) testing developed applications, highlight remaining e-paper-specific design challenges and outline future research directions.

Our goal is to support project-based learning in data science education. We scaffold project solutions for four common data science problems: binary classification, multi-class classification, regression and clustering, using template solutions implemented as Jupyter notebooks that can be run on any dataset provided by a student in line with formatting guidelines. For data-specific steps (like feature engineering), we extend our templates using a custom Python library for integrating an LLM with a Jupyter notebook. The library works both for open-source models and paid APIs, and allows a chatbot to be integrated in any cell of a Jupyter notebook. Together, these form an interactive data science AI tutor (DSAIT). We study the interaction patterns of 24 data science students from two universities who attempted to solve a Kaggle challenge using DSAIT's assistance, and conduct a temporal analysis of how these patterns evolved during the task. We provide the templates and Python library used to implement DSAIT to the computer science education community.

Arkadiusz Modzelewski, Witold Sosnowski, Tiziano Labruna, Adam Wierzbicki, Giovanni Da San Martino. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Disinformation poses a significant threat to democratic societies, public health, and national security.To address this challenge, factchecking experts analyze and track disinformation narratives.However, the process of manually identifying these narratives is highly time-consuming and resource-intensive.In this article, we introduce DiNaM, the first algorithm and structured framework specifically designed for mining disinformation narratives.DiNaM uses a multi-step approach to uncover disinformation narratives.It first leverages Large Language Models (LLMs) to detect false information, then applies clustering techniques to identify underlying disinformation narratives.We evaluated DiNaM's performance using groundtruth disinformation narratives from the EUD-isinfoTest dataset.The evaluation employed the Weighted Chamfer Distance (WCD), which measures the similarity between two sets of embeddings: the ground truth and the predicted disinformation narratives.DiNaM achieved a state-of-the-art WCD score of 0.73, outperforming general-purpose narrative mining methods by a notable margin of 16.4-24.7%.We are releasing DiNaM's codebase and the dataset to the public.

• The foraging paradigm can be applied in research on Multi-List Recommender Interfaces (MLRIs). • Older users find fewer products that match their preferences in MLRIs than younger ones. • The decreased performance of older users in MLRIs is mediated by their reduced exploration, that is inability to switch to new carousels at the right time. • A digital nudge that is activated when a user browses a carousel without interacting with the products on that carousel improves the performance of both older and younger users in MLRIs. User interfaces composed of multiple carousels (Multi-List Recommender Interfaces, MLRIs) are today’s standard for recommender systems used in e-commerce and streaming or music platforms. User behavior in such systems can be compared to foraging, a research paradigm used in the natural sciences and psychology. Research on foraging points out several hypotheses that can apply to MLRI users. Results from the psychology of aging point out possible limitations of older adults in the use of MLRIs. We verify this hypothesis in an experiment that measures users’ objective performance ( N = 441). The experiment results confirm that older users are less effective in a task of searching for products in a MLRI interface. We propose an improvement of the carousel interface through a digital nudge that aims to prompt a user to leave a carousel that does not contain items matching the user’s preferences, and switch to another carousel. Our experimental results confirm the effectiveness of the proposed digital nudge in increasing the performance of both older and younger users.

This study presents a novel corpus of 15,356 Polish web articles, including articles identified as containing disinformation.Our dataset enables a multifaceted understanding of disinformation.We present a distinctive multilayered methodology for annotating disinformation in texts.What sets our corpus apart is its focus on uncovering hidden intent and manipulation in disinformative content.A team of experts annotated each article with multiple labels indicating both disinformation creators' intents and the manipulation techniques employed.Additionally, we set new baselines for binary disinformation detection and two multiclass multilabel classification tasks: manipulation techniques and intention types classification.

Fighting medical disinformation in the era of the pandemic is an increasingly important problem. Today, automatic systems for assessing the credibility of medical information do not offer sufficient precision, so human supervision and the involvement of medical expert annotators are required. Our work aims to optimize the utilization of medical experts' time. We also equip them with tools for semi-automatic initial verification of the credibility of the annotated content. We introduce a general framework for filtering medical statements that do not require manual evaluation by medical experts, thus focusing annotation efforts on non-credible medical statements. Our framework is based on the construction of filtering classifiers adapted to narrow thematic categories. This allows medical experts to fact-check and identify over two times more non-credible medical statements in a given time interval without applying any changes to the annotation flow. We verify our results across a broad spectrum of medical topic areas. We perform quantitative, as well as exploratory analysis on our output data. We also point out how those filtering classifiers can be modified to provide experts with different types of feedback without any loss of performance.

In the Internet era, proliferation of fake news can cause serious social and individual consequences. Little is known about why Web users believe or disbelieve fake news. This article aims to make progress in this area by studying brain activity during credibility evaluation. We conducted an experiment that mimics a condition when fake news are evaluated by a person with expert knowledge. Using advanced EEG equipment, we have identified the brain areas involved in the process of verifying message credibility by using full knowledge. Based on experimental data, we have created a machine learning model using generalized logistic regression that achieved 77% accuracy under 10-fold cross-validation. Our model is a first step towards supporting expert-based fake news debunking using EEG.

Globally observed trends in aging indicate that older adults constitute a growing share of the population and an increasing demographic in the modern technologies marketplace. Therefore, it has become important to address the issue of participation of older adults in the process of developing solutions suitable for their group. In this study, we approached this topic by organizing a hackathon involving teams of young programmers and older adult participants. In our paper we describe a case study of that hackathon, in which our objective was to motivate older adults to participate in software engineering processes. Based on our results from an array of qualitative methods, we propose a set of good practices that may lead to improved older adult participation in similar events and an improved process of developing apps that target older adults.

Real-world optimization problems have been studied in the past, but the work resulted in approaches tailored to individual problems that could not be easily generalized. The reason for this limitation was the lack of appropriate models for the systematic study of salient aspects of real-world problems. The aim of this article is to study one of such aspects: multi-hardness. We propose a variety of decomposition-based algorithms for an abstract multi-hard problem and compare them against the most promising heuristics.

Abstract Globally observed trends in aging indicate that older adults constitute a growing share of the population and an increasing demographic in the modern technologies marketplace. Therefore, it has become important to address the issue of participation of older adults in the process of developing solutions suitable for their group. In this study, we approached this topic by organizing a hackathon involving teams of young programmers and older adults participants. Below we describe a case study of that hackathon, in which our objective was to motivate older adults to participate in software engineering processes. Based on our results from an array of qualitative methods, we propose a set of good practices that may lead to improved older adult participation in similar events and an improved process of developing apps that target older adults.

Context: Better methods of evaluating process performance of OSS projects can benefit decision makers who consider adoption of OSS software in a company. This article studies the closure of issues (bugs and features) in GitHub projects, which is an important measure of OSS development process performance and quality of support that project users receive from the developer team. Objective: The goal of this article is a better understanding of the factors that affect issue closure rates in OSS projects. Methodology: The GHTorrent repository is used to select a large sample of mature, active OSS projects. Using survival analysis, we calculate short-term, and long-term issue closure rates. We formulate several hypotheses regarding the impact of OSS project and team characteristics, such as measures of work centralization, measures that reflect internal project workflows, and developer social networks measures on issue closure rates. Based on the proposed features and several control features, a model is built that can predict issue closure rate. The model allows to test our hypotheses. Results: We find that large teams that have many project members have lower issue closure rates than smaller teams. Similarly, increased work centralization increases issue closure rates. While desirable social network characteristics have a positive impact on the amount of commits in a project, they do not have significant influence on issue closure. Conclusion: Overall, findings from empirical analysis support the classic notion of Brook’s – the “surgical team” – in the context of OSS project development process performance on GitHub. The model of issue closure rates proposed in this article is a first step towards an improved understanding and prediction of this important measure of OSS development process performance.

Controversy is a complex concept that has been attracting attention of scholars from diverse fields. In the era of Internet and social media, detecting controversy and controversial concepts by the means of automatic methods is especially important. Web searchers could be alerted when the contents they consume are controversial or when they attempt to acquire information on disputed topics. Presenting users with the indications and explanations of the controversy should offer them chance to see the “wider picture” rather than letting them obtain one-sided views. In this work we first introduce a formal model of controversy as the basis of computational approaches to detecting controversial concepts. Then we propose a classification based method for automatic detection of controversial articles and categories in Wikipedia. Next, we demonstrate how to use the obtained results for the estimation of the controversy level of search queries. The proposed method can be incorporated into search engines as a component responsible for detection of queries related to controversial topics. The method is independent of the search engine’s retrieval and search results recommendation algorithms, and is therefore unaffected by a possible filter bubble. Our approach can be also applied in Wikipedia or other knowledge bases for supporting the detection of controversy and content maintenance. Finally, we believe that our results could be useful for social science researchers for understanding the complex nature of controversy and in fostering their studies.

In this paper we provide a brief summary of development LivingLab PJAIT as an attempt to establish a comprehensive and sustainable ICT-based solution for empowerment of elderly communities towards better urban participation of seniors. We report on our various endeavors for better involvement and participation of older adults in urban life by lowering ICT barriers, encouraging social inclusion, intergenerational interaction, physical activity and engaging older adults in the process of development of ICT solutions. We report on a model and assumptions of the LivingLab PJAIT as well as a number of activities created and implemented for LivingLab participants: from ICT courses, both traditional and e-learning, through on-line crowdsourcing tasks, to blended activities of different forms and complexity. We also provide conclusions on the lessons learned in the process and some future plans, including solutions for better senior urban participation and citizen science.

Success of Wikipedia would not be possible without the contributions of millions of anonymous Internet users who edit articles, correct mistakes, add links or pictures. At the same time Wikipedia editors are currently overworked and there is always more tasks waiting to be completed than people willing to volunteer. The paper explores the possibility of involving the elderly in the Wikipedia editing process. Older adults were asked to complete various tasks on Wikipedia. Based on the observations made during these activities as well as in-depth interviews, a list of recommendation has been crafted. It turned out that older adults are willing to contribute to Wikiepdia but substantial changes have to be made in the Wikipedia editor.

The goal of our research is to create a predictive model of Web content credibility evaluations, based on human evaluations. The model has to be based on a comprehensive set of independent factors that can be used to guide user’s credibility evaluations in crowdsourced systems like WOT, but also to design machine classifiers of Web content credibility. The factors described in this article are based on empirical data. We have created a dataset obtained from an extensive crowdsourced Web credibility assessment study (over 15 thousand evaluations of over 5000 Web pages from over 2000 participants). First, online participants evaluated a multi-domain corpus of selected Web pages. Using the acquired data and text mining techniques we have prepared a code book and conducted another crowdsourcing round to label textual justifications of the former responses. We have extended the list of significant credibility assessment factors described in previous research and analyzed their relationships to credibility evaluation scores. Discovered factors that affect Web content credibility evaluations are also weakly correlated, which makes them more useful for modeling and predicting credibility evaluations. Based on the newly identified factors, we propose a predictive model for Web content credibility. The model can be used to determine the significance and impact of discovered factors on credibility evaluations. These findings can guide future research on the design of automatic or semi-automatic systems for Web content credibility evaluation support. This study also contributes the largest credibility dataset currently publicly available for research: the Content Credibility Corpus (C3).

Real-world problems are usually composed of two or more (potentially NP-Hard) problems that are interdependent on each other. Such problems have been recently identified as "multi-hard problems" and various strategies for solving them have been proposed. One of the most successful of the strategies is based on a decomposition approach, where each of the components of a multi-hard problem is solved separately (by state-of-the-art solver) and then a negotiation protocol between the sub-solutions is applied to mediate a global solution. Multi-hardness is, however, not the only crucial aspect of real-world problems. Many real-world problems operate in a dynamically-changing, uncertain environment. Special approaches such as risk analysis and minimization may be applied in cases when we know the possible variants of constraints and criteria, as well as their probabilities. On the other hand, adaptive algorithms may be used in the case of uncertainty about criteria variants or probabilities. While such approaches are not new, their application to multi-hard problems has not yet been studied systematically. In this paper we extend the benchmark problem for multi-hardness with the aspect of uncertainty. We adapt the decomposition-based approach to this new setting, and compare it against another promising heuristic (Monte-Carlo Tree Search) on a large publicly available dataset. Our comparisons show that the decomposition-based approach outperforms the other heuristic in most cases.

In this paper we propose the model of signal for objects that are subject to evaluation by crowdsourcing. Such signal, constructed as probability of distribution using Normal Random Utility Model (NRUM), can be used to measure object's performance, create rankings or predict next evaluations. Our model is designed for monadic scale evaluations where evaluators can have different expertise or bias for using scale. Moreover, our model is constructed for situations where we can have a lot of missing evaluations or varying numbers of evaluations for each object and from each evaluator, typical for crowdsourcing data. We have built a model for medical Web pages credibility from real crowdsourcing data and have evaluated the model's predictive ability, proving its superiority to alternative prediction methods.

GitHub is one of the most commonly used web-based code repository hosting service. Majority of projects hosted on GitHub are really small but, on the other hand, developers spend most of their time working in medium to large repositories. Developers can freely join and leave projects following their current needs and interests. Based on real data collected from GitHub we have tried to predict which developer will join which project. A mix of carefully selected list of features and machine learning techniques let us achieve a precision of 0.886, in the best case scenario, where there is quite a long history of a user and a repository in the system. Even when proposed classifier faces a cold start problem, it delivers precision equal to 0.729 which is still acceptable for automatic recommendation of noteworthy projects for developers.

The Wikipedia project has created one of the largest and best-known open knowledge communities. This community is a model for several similar efforts, both public and commercial, and even for the knowledge economy of the future e-society. For these reasons, issues of quality, social processes, and motivation within the Wikipedia knowledge community have attracted attention of researchers. Research has often used Social Network Analysis applied to networks created based on behavioral data available from the edit history of the Wikipedia. This paper asks the following question: are the popular assumptions about the social interpretations of networks created from the edit history valid? We verify commonly assumed interpretations of four types of networks created from discussions on Wikipedia talk pages, co-edits and reverts in Wikipedia articles, and edits of articles in various topics, by comparing these networks with results from a survey of editors of the Polish Wikipedia community. The results indicate that while the behavioral networks are strongly related to the declarations of respondents, only in one case of the network created from talk pages and interpreted as acquaintance we can observe a near equivalence. The article next describes improved definitions of behavioral indicators obtained through machine learning. The improved networks are much closer to their declarative counterparts. The main contribution of the article is a validated model of an acquaintance network among Wikipedia editors that can be derived from behavioral data and validly interpreted as acquaintance. Other contributions are improved versions of behavioral networks based on editing behavior and discussion history on the Wikipedia.

Machine learning algorithms and recommender systems trained on human ratings are widely in use today. However, human ratings may be associated with a high level of uncertainty and are subjective, influenced by demographic or psychological factors. We propose a new approach to the design of object classes from human ratings: the use of entire distributions to construct classes. By avoiding aggregation for class definition, our approach loses no information and can deal with highly volatile or conflicting ratings. The approach is based the concept of the Earth Mover's Distance (EMD), a measure of distance for distributions. We evaluate the proposed approach based on four datasets obtained from diverse Web content or movie quality evaluation services or experiments. We show that clusters discovered in these datasets using the EMD measure are characterized by a consistent and simple interpretation. Quality classes defined using entire rating distributions can be fitted to clusters of distributions in the four datasets using two parameters, resulting in a good overall fit. We also consider the impact of the composition of small samples on the distributions that are the basis of our classification approach. We show that using distributions based on small samples of 10 evaluations is still robust to several demographic and psychological variables. This observation suggests that the proposed approach can be used in practice for quality evaluation, even for highly uncertain and subjective ratings.

By leveraging crowdsourcing, Web credibility evaluation systems (WCESs) have become a promising tool to assess the credibility of Web content, e.g., Web pages. However, existing systems adopt a passive way to collect users' credibility ratings, which incurs two crucial challenges: (1) a considerable fraction of Web content have few or even no ratings, so the coverage (or effectiveness) of the system is low; (2) malicious users may submit fake ratings to damage the reliability of the system. In order to realize a highly effective and robust WCES, we propose to integrate recommendation functionality into the system. On the one hand, by fusing Matrix Factorization and Latent Dirichlet Allocation, a personalized Web content recommendation model is proposed to attract users to rate more Web pages, i.e., the coverage is increased. On the other hand, by analyzing a user's reaction to the recommended Web content, we detect imitating attackers, which have recently been recognized as a particular threat to WCES to make the system more robust. Moreover, an adaptive reputation system is designed to motivate users to more actively interact with the integrated recommendation functionality. We conduct experiments using both real datasets and synthetic data to demonstrate how our proposed recommendation components significantly improve the effectiveness and robustness of existing WCES.

Many real-world problems are composed of two or more problems that are interdependent on each other. The interaction of such problems usually is quite complex and solving each problem separately cannot guarantee the optimal solution for the overall multi-component problem. In this paper we experiment with one particular 2-component problem, namely the Traveling Thief Problem (TTP). TTP is composed of the Traveling Salesman Problem (TSP) and the Knapsack Problem (KP). We investigate two heuristic methods to deal with TTP. In the first approach we decompose TTP into two sub-problems, solve them by separate modules/algorithms (that communicate with each other), and combine the solutions to obtain an overall approximated solution to TTP (this method is called CoSolver ). The second approach is a simple heuristic (called density-based heuristic, DH) method that generates a solution for the TSP component first (a version of Lin-Kernighan algorithm is used) and then, based on the fixed solution for the TSP component found, it generates a solution for the KP component (associated with the given TTP). In fact, this heuristic ignores the interdependency between sub-problems and tries to solve the sub-problems sequentially. These two methods are applied to some generated TTP instances of different sizes. Our comparisons show that CoSolver outperforms DH specially in large instances.

Computational trust propagation is an important method for the establishment of trust in strangers. In ad-hoc or P2P networks, such an approach allows to choose trusted nodes for routing, data storage, or computation, even if the choosing node has not had previous experiences with the considered nodes. Human trust propagation can occur through a variety of phenomena, such as recommendation of trusted strangers (transitive trust propagation) or because of similarity between the trustor and trustee (similarity propagation). Computational trust propagation algorithms aim to reproduce the process of human trust propagation faithfully and to exploit all available information in order to recommend new trust links. Research in this area has established a method of evaluating the correctness of trust propagation algorithms that takes into account the recall of trust recommendation. In this paper, this commonly used evaluation method is criticized, and a new method is proposed that additionally allows to approximate the precision of trust recommendations. The second contribution of the paper is CloseLook, a new trust propagation algorithm that is capable of executing all relevant types of trust propagation in an efficient manner. The efficiency of CloseLook is compared against a well known trust propagation algorithm proposed by Guha et al. CloseLook is much more efficient without sacrificing correctness.

We study two problems directly resulting from organizational decentralization of the grid. Firstly, the problem of fair scheduling in systems in which the grid scheduler has complete control of processors' schedules. Secondly, the problem of fair and feasible scheduling in decentralized case, in which the grid scheduler can only suggest a schedule, which can be later modified by a processor's owner. Using game theory, we show that scheduling in decentralized case is analogous to the prisoner's dilemma game. Moreover, the Nash equilibrium results in significant performance drop. Therefore, a strong community control is required to achieve acceptable performance.

The article describes and gives an economic analysis of a business model for commercial content delivery networks (CDN) based on the peer-to-peer model. The content is stored in the CDN on the hosts of the peers. A user pays for access to the content, and can sell the content to other users as in a direct sales network. The content trade is a free market transaction (including billing and accounting) handled by superpeers, who receive a markup for their services and pay the content provider a gratification for every transaction. The system makes use of reputation mechanisms with a goal contrary to most P2P research: to promote content trading and discourage sharing for free. The article compares the profit obtained by the content provider in a client-server CDN and the P2P CDN, and analyzes the stable-state prices in a P2P CDN.

This paper describes a new protocol for authentication in Peer-to-Peer systems. The protocol has been designed to meet specialized requirements of P2P systems, such as lack of direct communication between peers or requirements for controlled anonymity. At the same time, a P2P authentication protocol must be resistant to spoofing, eavesdropping and playback, and man-in-the-middle attacks. The protocol is studied for a model P2P storage system that needs to implement file access rights.

Peer-to-peer (P2P) file-sharing applications generate a large part if not most of today's Internet traffic. The large volume of this traffic (thus the high potential benefits of caching) and the large cache sizes required (thus nontrivial costs associated with caching) only underline that efficient cache replacement policies are important in this case. P2P file-sharing traffic has several characteristics that distinguish it from well studied Web traffic and that require a focused study of efficient cache management policies. This paper uses trace driven simulations to compare traditional cache replacement policies with new policies that try to exploit characteristics of the P2P file-sharing traffic generated by applications using the FastTrack protocol.

The article considers the design of P2P games without trusted, centralized resources. The main difficulty is how to prevent the possibility of cheating. The article considers scrabble as a case study and attempts to solve issues such as maintenance of public, private, and concealed public state, as well as secret drawing from a finite set of objects. The issues of state replication are considered to allow node leaves. The article presents a fair protocol for secret drawing from a finite state that is resistant to node leaves.

Rhubarb is a platform for building peer-to-peer (P2P) applications. Rhubarb offers an API similar to Berkeley sockets. Using Rhubarb, P2P applications can be developed that are independent of centralized resources and the DNS system. Rhubarb organizes nodes in a virtual network, allowing connections across firewalls/NAT, and efficient broadcasting. The virtual network is scalable due to a hierarchical organization and efficient state management. Rhubarb is securely protected against outside and inside attacks.