This paper considers the challenges Large Language Models (LLMs) face when reasoning over text that includes information involving uncertainty explicitly quantified via probability values. This type of reasoning is relevant to a variety of contexts ranging from everyday conversations to medical decision-making. Despite improvements in the mathematical reasoning capabilities of LLMs, they still exhibit significant difficulties when it comes to probabilistic reasoning. To deal with this problem, we introduce the Bayesian Linguistic Inference Dataset (BLInD), a new dataset specifically designed to test the probabilistic reasoning capabilities of LLMs. We use BLInD to find out the limitations of LLMs for tasks involving probabilistic reasoning. In addition, we present several prompting strategies that map the problem to different formal representations, including Python code, probabilistic algorithms, and probabilistic logical programming. We conclude by providing an evaluation of our methods on BLInD and an adaptation of a causal reasoning question-answering dataset. Our empirical results highlight the effectiveness of our proposed strategies for multiple LLMs.

The peer review process is strained by increasing submission volumes, reviewer fatigue, and inconsistent standards. While Large Language Models (LLMs) can aid in reviews, they are often overly optimistic and lack technical depth. We developed an innovative prompting strategy that, when applied to ChatGPT-4 on ICLR 2025 papers, reduced score inflation and generated reviews more closely aligned with human reviewer median scores.

Wildfires are increasing in frequency and severity due to climate change, creating a need for accurate, large-scale damage assessment. In this work, we describe a new multi-temporal benchmark dataset for wildfire segmentation using Sentinel-2 imagery and wildfire polygon data, which we call CalFireSeg-50. For 50 of the largest recent wildfires in California (2019 to 2023), we collect RGB, NBR, and NDVI images. Ground truth masks are, then, generated using a conditional approach that combines spatial fire boundaries with burn-sensitive vegetation indices. Using images from this benchmark, we evaluate four deep learning (DL) segmentation models for detecting wildfire damage: UNet, UNet++, TransUNet, and Seg-Former. Our analysis compares model performance across multiple established quality metrics (e.g., IoU and Dice), training time, and data efficiency using a consistent GPU setup. TransUNet achieves the best performance across most quality metrics, including the highest Dice score (88%) and accuracy (94%), while UNet++ obtains the top IoU score (82%). However, TransUNet requires significantly more time to train than the CNN-based approaches. SegFormer shows promise in few-shot settings, but is outperformed by the other models when training involves larger datasets. These findings provide novel insights into the strengths and limitations of CNN, hybrid, and transformer-based architectures for wildfire segmentation, advancing DL approaches for wildfire damage mapping.

Aliakbar Nafar, K. Brent Venable, Parisa Kordjamshidi. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Many real-life scenarios require humans to make difficult trade-offs: do we always follow all the traffic rules or do we violate the speed limit in an emergency? In general, how should we account for and balance the ethical values, safety recommendations, and societal norms, when we are trying to achieve a certain objective? To enable effective AI-human collaboration, we must equip AI agents with a model of how humans make such trade-offs in environments where there is not only a goal to be reached, but there are also ethical constraints to be considered and to possibly align with. These ethical constraints could be both deontological rules on actions that should not be performed, or also consequentialist policies that recommend avoiding reaching certain states of the world. Our purpose is to build AI agents that can mimic human behavior in these ethically constrained decision environments, with a long term research goal to use AI to help humans in making better moral judgments and actions. To this end, we propose a computational approach where competing objectives and ethical constraints are orchestrated through a method that leverages a cognitive model of human decision making, called multi-alternative decision field theory (MDFT). Using MDFT, we build an orchestrator, called MDFT-Orchestrator (MDFT-O), that is both general and flexible. We also show experimentally that MDFT-O both generates better decisions than using a heuristic that takes a weighted average of competing policies (WA-O), but also performs better in terms of mimicking human decisions as collected through Amazon Mechanical Turk (AMT). Our methodology is therefore able to faithfully model human decision in ethically constrained decision environments.

In many real world situations, collective decisions are made using voting and, in scenarios such as committee or board elections, employing voting rules that return multiple winners. In multi-winner approval voting (AV), an agent submits a ballot consisting of approvals for as many candidates as they wish, and winners are chosen by tallying up the votes and choosing the top-k candidates receiving the most approvals. In many scenarios, an agent may manipulate the ballot they submit in order to achieve a better outcome by voting in a way that does not reflect their true preferences. In complex and uncertain situations, agents may use heuristics instead of incurring the additional effort required to compute the manipulation which most favors them. In this paper, we examine voting behavior in single-winner and multi-winner approval voting scenarios with varying degrees of uncertainty using behavioral data obtained from Mechanical Turk. We find that people generally manipulate their vote to obtain a better outcome, but often do not identify the optimal manipulation. There are a number of predictive models of agent behavior in the social choice and psychology literature that are based on cognitively plausible heuristic strategies. We show that the existing approaches do not adequately model our real-world data. We propose a novel model that takes into account the size of the winning set and human cognitive constraints; and demonstrate that this model is more effective at capturing real-world behaviors in multi-winner approval voting scenarios.

We introduce PCP-nets, a formalism to model qualitative conditional preferences with probabilistic uncertainty. PCP-nets generalise CP-nets by allowing for uncertainty over the preference orderings. We define and study both optimality and dominance queries in PCP-nets, and we propose a tractable approximation of dominance which we show to be very accurate in our experimental setting. Since PCP-nets can be seen as a way to model a collection of weighted CP-nets, we also explore the use of PCP-nets in a multi-agent context, where individual agents submit CP-nets which are then aggregated into a single PCP-net. We consider various ways to perform such aggregation and we compare them via two notions of scores, based on well known voting theory concepts. Experimental results allow us to identify the aggregation method that better represents the given set of CP-nets and the most efficient dominance procedure to be used in the multi-agent context.

If we want people to trust AI systems, we need to provide the systems we create with the ability to discriminate between what humans would consider good and bad decisions. The quality of a decision should not be based only on the preferences or optimization criteria of the decision makers, but also on other properties related to the impact of the decision, such as whether it is ethical, or if it complies to constraints and priorities given by feasibility constraints or safety regulations. The CP-net formalism [2] is a convenient and expressive way to model preferences, providing an effective compact way to qualitatively model preferences over outcomes, i.e., decisions, with a combinatorial structure [3, 7]. If we wish to incorporate ethical, moral, or norms based constraints to a decision context, it means that the subjective preferences of the decision makers are not the only source of information we should consider [1, 8]. Indeed, depending on the context, we may have to consider specific ethical principles derived from an appropriate ethical theory or various laws and norms. While preferences are important, when preferences and ethical principles are in conflict, the principles should override the subjective preferences of the decision maker. Therefore, it is essential to have well founded techniques to evaluate whether preferences are compatible with a set of ethical principles, and to measure how much these preferences deviate from the ethical principles.

The future will see autonomous machines acting in the same environment as humans, in areas as diverse as driving, assistive technology, and health care. Think of self-driving cars, companion robots, and medical diagnosis support systems. We also believe that humans and machines will often need to work together and agree on common decisions. Thus hybrid collective decision making systems will be in great need. In this scenario, both machines and collective decision making systems should follow some form of moral values and ethical principles (appropriate to where they will act but always aligned to humans'), as well as safety constraints. In fact, humans would accept and trust more machines that behave as ethically as other humans in the same environment. Also, these principles would make it easier for machines to determine their actions and explain their behavior in terms understandable by humans. Moreover, often machines and humans will need to make decisions together, either through consensus or by reaching a compromise. This would be facilitated by shared moral values and ethical principles.

NASA and the international aviation community are investing in the development of a commercial transportation infrastructure that includes the increased use of rotorcraft, specifically helicopters and civil tilt rotors. However, there is significant concern over the impact of noise on the communities surrounding the transportation facilities. One way to address the rotorcraft noise problem is by exploiting powerful search techniques coming from artificial intelligence to design low-noise flight profiles that can be then validated though field tests. This article investigates the use of discrete heuristic search methods to design low-noise approach trajectories for rotorcraft. Our work builds on a long research tradition in trajectory optimization using either numerical methods or discrete search. Novel features of our approach include the use of a discrete search space with a resolution that can be varied, and the coupling of search with a robust simulator to evaluate candidates. The article includes a systematic comparison of different search techniques; in particular, in the experiments, we are able to do a trade study that compares complete search algorithms such as A * with faster but approximate methods such as local search.

We consider multi-agent settings where a set of agents want to take a collective decision, based on their preferences over the possible candidate options. While agents have their initial inclination, they may interact and influence each other, and therefore modify their preferences, until hopefully they reach a stable state and declare their final inclination. At that point, a voting rule is used to aggregate the agents’ preferences and generate the collective decision. Recent work has modeled the influence phenomenon in the case of voting over a single issue. Here we generalize this model to account for preferences over combinatorially structured domains including several issues. We propose a way to model influence when agents express their preferences as CP-nets. We define two procedures for aggregating preferences in this scenario, by interleaving voting and influence convergence, and study their resistance to bribery.

We consider a multi-agent scenario where a collection of agents needs to select a common decision from a large set of decisions over which they express their preferences. This decision set has a combinatorial structure, that is, each decision is an element of the Cartesian product of the domains of some variables. Agents express their preferences over the decisions via soft constraints. We consider both sequential preference aggregation methods (they aggregate the preferences over one variable at a time) and one-step methods and we study the computational complexity of influencing them through bribery. We prove that bribery is NPcomplete for the sequential aggregation methods (based on Plurality, Approval, and Borda) for most of the cost schemes we defined, while it is polynomial for one-step Plurality.

We study the computational complexity of finding the next most preferred solution in some common formalisms for representing constraints and preferences. The problem is computationally intractable for CSPs, but is polynomial for tree-shaped CSPs and tree-shaped fuzzy CSPs. On the other hand, it is intractable for weighted CSPs, even under restrictions on the constraint graph. For CP-nets, the problem is polynomial when the CP-net is acyclic. This remains so if we add (soft) constraints that are tree-shaped and topologically compatible with the CP-net.

We consider scenarios where several agents must aggregate their preferences over a large set of candidates with a combinatorial structure. That is, each candidate is an element of the Cartesian product of the domains of some variables. We assume agents compactly express their preferences over the candidates via soft constraints. We consider a sequential procedure that chooses one candidate by asking the agents to vote on one variable at a time. While some properties of this procedure have been already studied, here we focus on independence of irrelevant alternatives, non-dictatorship, and strategy-proofness. Also, we perform an experimental study that shows that the proposed sequential procedure yields a considerable saving in time with respect to a non-sequential approach, while the winners satisfy the agents just as well, independently of the variable ordering and of the presence of coalitions of agents. 1

The stable marriage problem is a well-known problem of matching men to women so that no man and woman, who are not married to each other, both prefer each other. Such a problem has a wide variety of practical applications, ranging from matching resident doctors to hospitals, to matching students to schools or more generally to any two-sided market. In the classical stable marriage problem, both men and women express a strict preference order over the members of the other sex, in a qualitative way. Here we consider stable marriage problems with weighted preferences: each man (resp., woman) provides a score for each woman (resp., man). In this context, we consider the manipulability properties of the procedures that return stable marriages. While we know that all procedures are manipulable by modifying the preference lists or by truncating them, here we consider if manipulation can occur also by just modifying the weights while preserving the ordering and avoiding truncation. It turns out that, by adding weights, we indeed increase the possibility of manipulating and this cannot be avoided by any reasonable restriction on the weights.

While many frameworks for reasoning about time rely on the assumption that, as along as precedences are respected and there are no overlapping activities, any available time is just as good, all of us know that that is really not true! All of us would prefer avoiding heavy meetings right after lunch, waiting for connecting flights for endless hours or going on vacation at a selected destination during its rainy season. Among the many solutions to a temporal problem, being able select most preferred ones is a capability which is desirable in any intelligent system. On the other side, we may be often faced with over-constrained problems where relaxing hard constraints in a smart way can allow us to find a solution representing a better compromise. This contribution takes an AI perspective and shows how an efficient and expressive way to reason about preferences can help to handle time in a more flexible and sophisticated way. Among the many frameworks for temporal reasoning, constraint-based ones (quantitative and qualitative) have provided the most suitable base for the introduction of preferences. We will discuss how temporal constraints have been extended to allow for the representation of preferences, how temporal preferences can coexist with uncertainty and how they can be made conditional. Moreover, we will show how preferences allow for a significant increase in terms of representational power often at a modest additional computational cost.

Preferences and uncertainty are common in many real-life problems. In this article, we focus on bipolar preferences and uncertainty modelled via uncontrollable variables, and we assume that uncontrollable variables are specified by possibility distributions over their domains. To tackle such problems, we concentrate on uncertain bipolar problems with totally ordered preferences, and we eliminate the uncertain part of the problem, while making sure that some desirable properties hold about the robustness of the problem and its relationship with the preference of the optimal solutions. We also consider several semantics to order the solutions according to different attitudes with respect to the notions of preference and robustness.

The stable marriage problem (SM) has a wide variety of practical applications, ranging from matching resident doctors to hospitals, to matching students to schools, or more generally to any two-sided market. In the classical formulation, n men and n women express their preferences over the members of the other sex. Solving an SM means finding a stable marriage: a matching of men to women with no blocking pair. A blocking pair consists of a man and a woman who are not married to each other but both prefer each other to their partners. It is possible to find a male-optimal (resp., female-optimal) stable marriage in polynomial time. However, it is sometimes desirable to find stable marriages without favoring a group at the expenses of the other one. In this paper we present a local search approach to find stable marriages. Our experiments show that the number of steps grows as little as O(nlog(n)). We also show empirically that the proposed algorithm samples very well the set of all stable marriages of a given SM, thus providing a fair and efficient approach to generate stable marriages.

Real-life problems present several kinds of preferences. We focus on problems with both positive and negative preferences, which we call bipolar preference problems. Although seemingly specular notions, these two kinds of preferences should be dealt with differently to obtain the desired natural behaviour. We technically address this by generalising the soft constraint formalism, which is able to model problems with one kind of preference. We show that soft constraints model only negative preferences, and we add to them a new mathematical structure which allows to handle positive preferences as well. We also address the issue of the compensation between positive and negative preferences, studying the properties of this operation. Finally, we extend the notion of arc consistency to bipolar problems, and we show how branch and bound (with or without constraint propagation) can be easily adapted to solve such problems.

Preferences and uncertainty are common in many real-life problems. In this article, we consider preferences modelled via soft constraints that allow for the representation of quantitative preferences. Moreover, we consider uncertainty modelled via uncontrollable variables, that is, variables whose value cannot be decided by us. We also assume that some information is provided for such variables in the form of a possibility distribution over their domains. Possibilities provide a way to model imprecise probabilities, and give us a way to know which values are more possible than others for the uncontrollable variables. For such problems, the aim is to find a solution with a high preference which is also very robust with respect to the uncontrollable part. To tackle such problems, we adopt an existing approach that eliminates the uncertain part of the problem while adding some constraints in the remaining part, and then solves the resulting problem. However, contrarily to the specific methods present in the literature, we formulate several desirable properties, on the robustness of the problem's solutions and its relationship with their preferences, that should be satisfied by any specific method based on this approach. We also define several semantics to order the solutions according to different attitudes to risk, and we discuss which of the desirable properties are satisfied by each of the considered semantics. Finally, we present a solver for this kind of problems, and we show some experimental results of its application over the classes of such problems.

Abstract. Preferences are not always expressible via complete linear orders: sometimes it is more natural to allow for the presence of incomparable outcomes. This may hold both in the agents ’ preference ordering and in the social order. In this paper we consider this scenario and we study what properties it may have. In particular, we show that, despite the added expressivity and ability to resolve conflicts provided by incomparability, classical impossibility results (such as Arrow’s theorem, Muller-Satterthwaite’s theorem, and Gibbard-Satterthwaite’s theorem) still hold. We also prove some possibility results, generalizing Sen’s theorem for majority voting. To prove these results, we define new notions of unanimity, monotonicity, dictator, triple-wise value-restriction, and strategy-proofness, which are suitable and natural generalizations of the classical ones for complete orders. 1

In real-life temporal scenarios, uncertainty and preferences are often essential and coexisting aspects. We present a formalism where quantitative temporal constraints with both preferences and uncertainty can be defined. We show how three classical notions of controllability (that is, strong, weak, and dynamic), which have been developed for uncertain temporal problems, can be generalized to handle preferences as well. After defining this general framework, we focus on problems where preferences follow the fuzzy approach, and with properties that assure tractability. For such problems, we propose algorithms to check the presence of the controllability properties. In particular, we show that in such a setting dealing simultaneously with preferences and uncertainty does not increase the complexity of controllability testing. We also develop a dynamic execution algorithm, of polynomial complexity, that produces temporal plans under uncertainty that are optimal with respect to fuzzy preferences.