Over the last few years, verifying the credibility of information sources has become a fundamental need to combat disinformation. Here, we present a language-agnostic model designed to assess the reliability of web domains as sources in references across multiple language editions of Wikipedia. Utilizing editing activity data, the model evaluates domain reliability within different articles of varying controversiality, such as Climate Change, COVID-19, History, Media, and Biology topics. Crafting features that express domain usage across articles, the model effectively predicts domain reliability, achieving an F1 Macro score of approximately 0.80 for English and other high-resource languages. For mid-resource languages, we achieve 0.65, while the performance of low-resource languages varies. In all cases, the time the domain remains present in the articles (which we dub as permanence ) is one of the most predictive features. We highlight the challenge of maintaining consistent model performance across languages of varying resource levels and demonstrate that adapting models from higher-resource languages can improve performance. We believe these findings can assist Wikipedia editors in their ongoing efforts to verify citations and may offer useful insights for other user-generated content communities.

This paper investigates the extent to which LLMs align with humans when personality shifts are associated with language changes. Based on three experiments, that focus on GPT-4o and the Eysenck Personality Questionnaire-Revised (EPQR-A), our initial results reveal a weak yet significant variation in GPT-4o’s personality across languages, indicating that some stem from a language-switching effect rather than translation. Further analysis across five English-speaking countries shows that GPT-4o, leveraging stereotypes, reflects distinct country-specific personality traits.

This article analyzes how well OpenAI’s LLM GPT-4 can emulate different personalities and simulate populations to answer psychological questionnaires similarly to real population samples. For this purpose, we performed different experiments with the Eysenck Personality Questionnaire-Revised Abbreviated (EPQR-A) in three different languages (Spanish, English, and Slovak). The EPQR-A measures personality on four scales: extraversion (E: sociability), neuroticism (N: emotional stability), psychoticism (P: tendency to break social rules, and not having empathy), and lying (L: social desirability). We perform a comparative analysis of the answers of synthetic populations with those of two real population samples of Spanish students as well as the unconditioned baseline personality of GPT. Furthermore, the impact of time (what year the questionnaire is answered), questionnaire language, and student age and gender are analyzed. To our knowledge, this is the first time the EPQR-A test has been used to assess the GPT´s personality and the impact of different language versions and time are measured. Our analysis reveals that GPT-4 exhibits an extroverted, emotionally stable personality with low psychoticism levels and high social desirability. GPT-4 replicates some differences observed in real populations in terms of gender but only partially replicates the results for real populations.

This study describes a dataset that allows to assess the emerging challenges posed by Generative Artificial Intelligence when doing Active Retrieval Augmented Generation (RAG), especially when summarizing trustworthy sources on the Internet. As a case study, we focus on Microsoft Copilot, an innovative software that integrates Large Language Models (LLMs) and Search Engines (SE) making advanced AI accessible to the general public. The core contribution of this paper is the presentation of the largest public database to date of RAG responses to user prompts, collected during the 2023 electoral campaigns in Switzerland, Bavaria and Hesse. This dataset was compiled with the assistance of a group of experts who posed realistic voter questions and conducted fact-checking of Microsoft Copilot's responses. It contains prompts and answers in English, German, French and Italian. All the collection happened during the electoral campaign, between 21 August 2023 and 2 October 2023. The paper makes available the full set of 5,561 pairs of prompts and answers, including the URLs referenced in the answers. In addition to the dataset itself, we provide 1374 answers labelled by a group of experts who rated the accuracy of the answers in providing factual information, showing that almost one out of three times the chatbot responded with either factually incorrect information or completely nonsensical answers. This resource is intended to facilitate further research into the performance of LLMs in the context of elections, defined as a "high-risk scenario" by the Digital Services Act (DSA) Article 34(1)(c).

Counterfactual explanations are a natural way for humans to gain understanding and trust in the outcomes of complex machine learning algorithms. In the context of natural language processing, generating counterfactuals is particularly challenging as it requires the generated text to be fluent, grammatically correct, and meaningful. In this study, we improve the current state of the art for the generation of such counterfactual explanations for text classifiers. Our approach, named RELITC (Relevance-based Infilling for Textual Counterfactuals), builds on the idea of masking a fraction of text tokens based on their importance in a given prediction task and employs a novel strategy, based on the entropy of their associated probability distributions, to determine the infilling order of these tokens. Our method uses less time than competing methods to generate counterfactuals that require less changes, are closer to the original text and preserve its content better, while being competitive in terms of fluency. We demonstrate the effectiveness of the method on four different datasets and show the quality of its outcomes in a comparison with human generated counterfactuals.

Decide Madrid is the civic technology of Madrid City Council which allows users to create and support online petitions. Despite the initial success, the platform is encountering problems with the growth of petition signing because petitions are far from the minimum number of supporting votes they must gather. Previous analyses have suggested that this problem is produced by the interface: a paginated list of petitions which applies a non-optimal ranking algorithm. For this reason, we present an interactive system for the discovery of topics and petitions. This approach leads us to reflect on the usefulness of data visualization techniques to address relevant societal challenges.

Collaborative content creation inevitably reaches situations where different points of view lead to conflict. We focus on Wikipedia, the free encyclopedia anyone may edit, where disputes about content in controversial articles often reflect larger societal debates. While Wikipedia has a public edit history and discussion section for every article, the substance of these sections is difficult to phantom for Wikipedia users interested in the development of an article and in locating which topics were most controversial. In this paper we present Contropedia, a tool that augments Wikipedia articles and gives insight into the development of controversial topics. Contropedia uses an efficient language agnostic measure based on the edit history that focuses on wiki links to easily identify which topics within a Wikipedia article have been most controversial and when.

In many applications, independence of event occurrences is assumed, even if there is evidence for dependence. Capturing dependence leads to complex models, and even if the complex models were superior, they fail to beat the simplicity and scalability of the independence assumption. Therefore, many models assume independence and apply heuristics to improve results. Theoretical explanations of the heuristics are seldom given or generalizable. This paper reports that some of these heuristics can be explained as encoding dependence in an exponent based on the generalized harmonic sum. Unlike independence, where the probability of subsequent occurrences of an event is the product of the single event probability, harmony is based on a product with decaying exponent. For independence, the sequence probability is |$p^{1+1+ \cdots +1}=p^n$|⁠, whereas for harmony, it is |$p^{1+1/2+ \cdots +1/n}$|⁠. The generalized harmonic sum leads to a spectrum of harmony assumptions. This paper shows that harmony assumptions naturally extend probability theory. An experimental evaluation for information retrieval (IR; term occurrences) and social networks (SN's; user interactions) shows that assuming harmony is more suitable than assuming independence. The potential impact of harmony assumptions lies beyond IR and SN's, since many applications rely on probability theory and apply heuristics to compensate the independence assumption. Given the concept of harmony assumptions, the dependence between multiple occurrences of an event can be reflected in an intuitive and effective way.

Core decomposition has proven to be a useful primitive for a wide range of graph analyses. One of its most appealing features is that, unlike other notions of dense subgraphs, it can be computed linearly in the size of the input graph. In this paper we provide an analogous tool for uncertain graphs, i.e., graphs whose edges are assigned a probability of existence. The fact that core decomposition can be computed efficiently in deterministic graphs does not guarantee efficiency in uncertain graphs, where even the simplest graph operations may become computationally intensive. Here we show that core decomposition of uncertain graphs can be carried out efficiently as well.

Collaborative content creation inevitably reaches situations where different points of view lead to conflict. In Wikipedia, one of the most prominent examples of collaboration online, conflict is mediated by both policy and software, and conflicts often reflect larger societal debates.

Online discussion threads are conversational cascades in the form of posted messages that can be generally found in social systems that comprise many-to-many interaction such as blogs, news aggregators or bulletin board systems. We propose a framework based on generative models of growing trees to analyse the structure and evolution of discussion threads. We consider the growth of a discussion to be determined by an interplay between popularity , novelty and a trend (or bias ) to reply to the thread originator. The relevance of these features is estimated using a full likelihood approach and allows to characterise the habits and communication patterns of a given platform and/or community. We apply the proposed framework on four popular websites: Slashdot , Barrapunto (a Spanish version of Slashdot), Meneame (a Spanish Digg -clone) and the article discussion pages of the English Wikipedia . Our results provide significant insight into understanding how discussion cascades grow and have potential applications in broader contexts such as community management or design of communication platforms.

We introduce a highly customizable social visualization system for exploring online discussions through visual representations of conversation threads. The tool provides users with an interface that allows the navigation and exploration through the often intricate structure of online discussions. Apart from being useful for readers and participants of these forums, the interactive capabilities of our system makes it appealing for social researchers interested in understanding the phenomena and intrinsic structure of online conversations. We also show a use case where we applied the tool to visualize discussions from Slashdot.org, showing its capabilities to represent new, in this case Slashdot specific, metrics. The visual representation of discussion threads has arisen as a complement for supporting investigation,as it helps to understand such large amount of information and contributes to the generation of new research ideas.

Large networks of spiking neurons show abrupt changes in their collective dynamics resembling phase transitions studied in statistical physics. An example of this phenomenon is the transition from irregular, noise-driven dynamics to regular, self-sustained behavior observed in networks of integrate-and-fire neurons as the interaction strength between the neurons increases. In this work we show how a network of spiking neurons is able to self-organize towards a critical state for which the range of possible inter-spike-intervals (dynamic range) is maximized. Self-organization occurs via synaptic dynamics that we analytically derive. The resulting plasticity rule is defined locally so that global homeostasis near the critical state is achieved by local regulation of individual synapses.

In this paper, we describe some preliminary work on using monolingual projections of document collections for performing cross-language information retrieval tasks. The proposed methodology uses multidimensional scaling for projecting the vector-space representations of a given multilingual document collection into spaces of lower dimensionality. An independent projection is computed for each different language, and the structural similarities of the resulting projections are exploited for information retrieval tasks.

We analyze the social network emerging from the user comment activity on the website Slashdot. The network presents common features of traditional social networks such as a giant component, small average path length and high clustering, but differs from them showing moderate reciprocity and neutral assortativity by degree. Using Kolmogorov-Smirnov statistical tests, we show that the degree distributions are better explained by log-normal instead of power-law distributions. We also study the structure of discussion threads using an intuitive radial tree representation. Threads show strong heterogeneity and self-similarity throughout the different nesting levels of a conversation. We use these results to propose a simple measure to evaluate the degree of controversy provoked by a post.

An ensemble of stochastic nonleaky integrate-and-fire neurons with global, delayed, and excitatory coupling and a small refractory period is analyzed. Simulations with adiabatic changes of the coupling strength indicate the presence of a phase transition accompanied by a hysteresis around a critical coupling strength. Below the critical coupling production of spikes in the ensemble is governed by the stochastic dynamics, whereas for coupling greater than the critical value, the stochastic dynamics loses its influence and the units organize into several clusters with self-sustained activity. All units within one cluster spike in unison, and the clusters themselves are phase-locked. Theoretical analysis leads to upper and lower bounds for the average interspike interval of the ensemble valid for all possible coupling strengths. The bounds allow calculating the limit behavior for large ensembles and characterize the phase transition analytically. These results may be extensible to pulse-coupled oscillators.

We propose a modeling framework based on the event-driven paradigm for populations of neurons which interchange messages. Unlike other strategies our approach is focused on the dynamics at the mesoscopic level (spike production and reception) and does not determine the microstates of the neurons. We apply the technique on a discrete model of stochastic ensembles and on extensions of this model to the continuous time domain. Due to the event-driven nature of the method efficient large-scale simulations can be performed without precision errors. The approach uses spike predictions as evidences and a one-step update of the predictions is performed every time an event occurs, resulting in a more efficient solution than the existing strategies.