Jinggui Liang, Dung Vo, Yap Hong Xian, Hai Leong Chieu, Kian Ming A. Chai, Jing Jiang, Lizi Liao. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

We introduce a novel one-parameter variational objective that lower bounds the data evidence and enables the estimation of approximate fractional posteriors. We extend this framework to hierarchical construction and Bayes posteriors, offering a versatile tool for probabilistic modelling. We demonstrate two cases where gradients can be obtained analytically and a simulation study on mixture models showing that our fractional posteriors can be used to achieve better calibration compared to posteriors from the conventional variational bound. When applied to variational autoencoders (VAEs), our approach attains higher evidence bounds and enables learning of high-performing approximate Bayes posteriors jointly with fractional posteriors. We show that VAEs trained with fractional posteriors produce decoders that are better aligned for generation from the prior.

We propose an extension of Thompson sampling to optimization problems over function spaces where the objective is a known functional of an unknown operator's output. We assume that queries to the operator (such as running a high-fidelity simulator or physical experiment) are costly, while functional evaluations on the operator's output are inexpensive. Our algorithm employs a sample-then-optimize approach using neural operator surrogates. This strategy avoids explicit uncertainty quantification by treating trained neural operators as approximate samples from a Gaussian process (GP) posterior. We derive regret bounds and theoretical results connecting neural operators with GPs in infinite-dimensional settings. Experiments benchmark our method against other Bayesian optimization baselines on functional optimization tasks involving partial differential equations of physical systems, demonstrating better sample efficiency and significant performance gains.

There are recent efforts to "personalize" large language models (LLMs) by assigning them specific personas.This paper explores the writing styles of such persona-assigned LLMs across different socio-demographic groups based on age, profession, location, and political affiliations, using three widely-used LLMs.Leveraging an existing style embedding model that produces detailed style attributes and latent Dirichlet allocation (LDA) for broad style analysis, we measure style differences using Kullback-Leibler divergence to compare LLM-generated and human-written texts.We find significant style differences among personas.This analysis emphasizes the need to consider socio-demographic factors in language modeling to accurately capture diverse writing styles used for communications.The findings also reveal the strengths and limitations of personalized LLMs, their potential uses, and the importance of addressing biases in their design.The code and data are available at: https://github.com/ ra-MANUJ-an/writing-style-persona Example from the training corpus That was the funniest thing so far this season.Sam SCREECHING and stabbin' wights all around in battle fury while more fall on him like throw pillows.Associated 20 Style Descriptors, ordered by score 'The author uses uncommon phrases.','The author uses descriptive words.','The author uses colorful language.','The author uses an energetic style.','The author uses a clever play on words.','The author is vivacious.','The author is using words to create a vivid and engaging atmosphere.','The author is using vivid descriptions.','The author is using punctuation to create a sense of tension and suspense.','The author is using male pronouns.','The author is intense in their writing.','The author is dramatic.','The author is captivating.','The author has a distinct and memorable style.','The author is creating a sense of anticipation and excitement.','The author is using a playful style.','The author is describing a current event.','The author uses victorious language.','The author is using a lighthearted tone.','The author uses singular subjects.'

Algebraic Normal Form (ANF) and Conjunctive Normal Form (CNF) are commonly used to encode problems in Boolean algebra. ANFs are typically solved via Grobner basis algorithms, often using more memory than is feasible; while CNFs are solved using SAT solvers, which cannot exploit the algebra of polynomials naturally. We propose a paradigm that bridges between ANF and CNF solving techniques: the techniques are applied in an iterative manner to learn facts to augment the original problems. Experiments on over 1,100 benchmarks arising from four different applications domains demonstrate that learnt facts can significantly improve runtime and enable more benchmarks to be solved.

Military personnel operating in hot and humid environments are susceptible to heat-related illnesses. As heat-related illnesses are associated with a rise in body core temperature (Tc), a reliable system for real-time assessment of Tc is useful to minimize heat casualties. However, invasive measurement of Tc (such as rectal, intestinal and esophageal temperature) is impractical in the field settings. This paper describes the model construction and qualification results of tracking Tc using an extended Kalman filter (EKF) based on physiological data recorded from wearable sensors. Tc, surface skin temperature (Tsk) and heart rate (HR) data were collected from three studies with different experimental protocols, climatic conditions and soldier volunteers. The predictive performance of the model was evaluated by cross-validation and external validation. The final EKF model was implemented using a nonlinear (cubic) state-space model (Tsk versus Tc) with a stage-wise, autoregressive exogenous model (incorporating HR) as the time update model. Overall, when tested against an independent dataset, the model showed a prediction bias of 0.11°C, a root mean square deviation of 0.29°C, and 87% of all Tc predictions fell within ±0.3°C of the measured Tc values. The results from our study indicate that the derived EKF model is accurate enough to calculate subject-specific Tc for the minimization of heat injuries.

Most of the current automated essay scoring (AES) systems are trained using manually graded essays from a specific prompt.These systems experience a drop in accuracy when used to grade an essay from a different prompt.Obtaining a large number of manually graded essays each time a new prompt is introduced is costly and not viable.We propose domain adaptation as a solution to adapt an AES system from an initial prompt to a new prompt.We also propose a novel domain adaptation technique that uses Bayesian linear ridge regression.We evaluate our domain adaptation technique on the publicly available Automated Student Assessment Prize (ASAP) dataset and show that our proposed technique is a competitive default domain adaptation algorithm for the AES task.

We propose a two-phase framework to adapt existing relation extraction classifiers to extract relations for new target domains. We address two challenges: negative transfer when knowledge in source domains is used without considering the differences in relation distributions; and lack of adequate labeled samples for rarer relations in the new domain, due to a small labeled data set and imbalance relation distributions. Our framework leverages on both labeled and unlabeled data in the target domain. First, we determine the relevance of each source domain to the target domain for each relation type, using the consistency between the clustering given by the target domain labels and the clustering given by the predictors trained for the source domain. To overcome the lack of labeled samples for rarer relations, these clusterings operate on both the labeled and unlabeled data in the target domain. Second, we trade-off between using relevance-weighted sourcedomain predictors and the labeled target data. Again, to overcome the imbalance distribution, the source-domain predictors operate on the unlabeled target data. Our method outperforms numerous baselines and a weakly-supervised relation extraction method on ACE 2004 and YAGO. © 2014 Association for Computational Linguistics.

Sum product networks (SPNs) are a new class of deep probabilistic models. They can contain multiple hidden layers while keeping their inference and training times tractable. An SPN consists of interleaving layers of sum nodes and product nodes. A sum node can be interpreted as a hidden variable, and a product node can be viewed as a feature capturing rich interactions among an SPN’s inputs. We show that the ability of SPN to use hidden layers to model complex dependencies among words, and its tractable inference and learning times, make it a suitable framework for a language model. Even though SPNs have been applied to a variety of vision problems [1, 2], we are the first to use it for language modeling. Our empirical comparisons with six previous language models indicate that our SPN has superior performance.

We introduce a new objective function for pool-based Bayesian active learning with probabilistic hypotheses. This objective function, called the policy Gibbs error, is the expected error rate of a random classifier drawn from the prior distribution on the examples adaptively selected by the active learning policy. Exact maximization of the policy Gibbs error is hard, so we propose a greedy strategy that maximizes the Gibbs error at each iteration, where the Gibbs error on an instance is the expected error of a random classifier selected from the posterior label distribution on that instance. We apply this maximum Gibbs error criterion to three active learning scenarios: non-adaptive, adaptive, and batch active learning. In each scenario, we prove that the criterion achieves near-maximal policy Gibbs error when constrained to a fixed budget. For practical implementations, we provide approximations to the maximum Gibbs error criterion for Bayesian conditional random fields and transductive Naive Bayes. Our experimental results on a named entity recognition task and a text classification task show that the maximum Gibbs error criterion is an effective active learning criterion for noisy models.

We provide some insights into how task correlations in multi-task Gaussian process (GP) regression affect the generalization error and the learning curve. We analyze the asymmetric two-task case, where a secondary task is to help the learning of a primary task. Within this setting, we give bounds on the generalization error and the learning curve of the primary task. Our approach admits intuitive understandings of the multi-task GP by relating it to single-task GPs. For the case of one-dimensional input-space under optimal sampling with data only for the secondary task, the limitations of multi-task GP can be quantified explicitly.

The inverse dynamics problem for a robotic manipulator is to compute the torques needed at the joints to drive it along a given trajectory; it is beneficial to be able to learn this function for adaptive control. A given robot manipulator will often need to be controlled while holding different loads in its end effector, giving rise to a multi-task learning problem. We show how the structure of the inverse dynamics problem gives rise to a multi-task Gaussian process prior over functions, where the inter-task similarity depends on the underlying dynamic parameters. Experiments demonstrate that this multi-task formulation generally improves performance over either learning only on single tasks or pooling the data over all tasks.

In this paper we investigate multi-task learning in the context of Gaussian Pro- cesses (GP). We propose a model that learns a shared covariance function on input-dependent features and a “free-form” covariance matrix over tasks. This al- lows for good flexibility when modelling inter-task dependencies while avoiding the need for large amounts of data for training. We show that under the assump- tion of noise-free observations and a block design, predictions for a given task only depend on its target values and therefore a cancellation of inter-task trans- fer occurs. We evaluate the benefits of our model on two practical applications: a compiler performance prediction problem and an exam score prediction task. Additionally, we make use of GP approximations and properties of our model in order to provide scalability to large data sets.

We derive a tractable and exact computation for the expectation of F-measures. We also demonstrate the non-convexity of this expectation, and investigate errors of approximating the expectation under different settings.

This paper explores the use of Bayesian online classifiers to classify text documents. Empirical results indicate that these classifiers are comparable with the best text classification systems. Furthermore, the online approach offers the advantage of continuous learning in the batch-adaptive text filtering task.