Advanced applied mathematics problems are underrepresented in existing Large Language Model (LLM) benchmark datasets. To address this, we introduce $\textbf{HARDMath}$, a dataset inspired by a graduate course on asymptotic methods, featuring challenging applied mathematics problems that require analytical approximation techniques. These problems demand a combination of mathematical reasoning, computational tools, and subjective judgment, making them difficult for LLMs. Our framework auto-generates a large number of problems with solutions validated against numerical ground truths. We evaluate both open- and closed-source LLMs on $\textbf{HARDMath-mini}$, a sub-sampled test set of 366 problems, as well as on 40 word problems formulated in applied science contexts. Even leading closed-source models like GPT-4 achieve only 43.8% overall accuracy with few-shot Chain-of-Thought prompting, and all models demonstrate significantly lower performance compared to results on existing mathematics benchmark datasets. We additionally conduct a detailed error analysis to gain insights into the failure cases of LLMs. These results demonstrate the limitations of current LLM performance on advanced graduate-level applied math problems and underscore the importance of datasets like $\textbf{HARDMath}$ to advance mathematical abilities of LLMs.

Large language models (LLMs) have shown remarkable progress in mathematical problem-solving, but evaluation has largely focused on problems that have exact analytical solutions or involve formal proofs, often overlooking approximation-based problems ubiquitous in applied science and engineering. To fill this gap, we build on prior work and present $\textbf{HARDMath2}$, a dataset of 211 original problems covering the core topics in an introductory graduate applied math class, including boundary-layer analysis, WKB methods, asymptotic solutions of nonlinear partial differential equations, and the asymptotics of oscillatory integrals. This dataset was designed and verified by the students and instructors of a core graduate applied mathematics course at Harvard. We build the dataset through a novel collaborative environment that challenges students to write and refine difficult problems consistent with the class syllabus, peer-validate solutions, test different models, and automatically check LLM-generated solutions against their own answers and numerical ground truths. Evaluation results show that leading frontier models still struggle with many of the problems in the dataset, highlighting a gap in the mathematical reasoning skills of current LLMs. Importantly, students identified strategies to create increasingly difficult problems by interacting with the models and exploiting common failure modes. This back-and-forth with the models not only resulted in a richer and more challenging benchmark but also led to qualitative improvements in the students' understanding of the course material, which is increasingly important as we enter an age where state-of-the-art language models can solve many challenging problems across a wide domain of fields.

We developed dysarthric speech intelligibility classifiers on 551,176 disordered speech samples contributed by a diverse set of 468 speakers, with a range of self-reported speaking disorders and rated for their overall intelligibility on a five-point scale. We trained three models following different deep learning approaches and evaluated them on ~ 94K utterances from 100 speakers. We further found the models to generalize well (without further training) on the TORGO database[1] (100% accuracy), UASpeech[2] (0.93 correlation), ALS-TDI PMP[3] (0.81 AUC) datasets as well as on a dataset of realistic unprompted speech we gathered (106 dysarthric and 76 control speakers, ~ 2300 samples). We share our model1to advance research in this domain.

Shanqing Cai, Subhashini Venugopalan, Katrin Tomanek, Ajit Narayanan, Meredith Morris, Michael Brenner. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.

Variational data assimilation optimizes for an initial state of a dynamical system such that its evolution fits observational data. The physical model can subsequently be evolved into the future to make predictions. This principle is a cornerstone of large scale forecasting applications such as numerical weather prediction. As such, it is implemented in current operational systems of weather forecasting agencies across the globe. However, finding a good initial state poses a difficult optimization problem in part due to the non-invertible relationship between physical states and their corresponding observations. We learn a mapping from observational data to physical states and show how it can be used to improve optimizability. We employ this mapping in two ways: to better initialize the non-convex optimization problem, and to reformulate the objective function in better behaved physics space instead of observation space. Our experimental results for the Lorenz96 model and a two-dimensional turbulent fluid flow demonstrate that this procedure significantly improves forecast quality for chaotic systems.

Automatic classification of disordered speech can provide an objective tool for identifying the presence and severity of speech impairment. Classification approaches can also help identify hard-to-recognize speech samples to teach ASR systems about the variable manifestations of impaired speech. Here, we develop and compare different deep learning techniques to classify the intelligibility of disordered speech on selected phrases. We collected samples from a diverse set of 661 speakers with a variety of self-reported disorders speaking 29 words or phrases, which were rated by speech-language pathologists for their overall intelligibility using a five-point Likert scale. We then evaluated classifiers developed using 3 approaches: (1) a convolutional neural network (CNN) trained for the task, (2) classifiers trained on non-semantic speech representations from CNNs that used an unsupervised objective [1], and (3) classifiers trained on the acoustic (encoder) embeddings from an ASR system trained on typical speech [2]. We found that the ASR encoder's embeddings considerably outperform the other two on detecting and classifying disordered speech. Further analysis shows that the ASR embeddings cluster speech by the spoken phrase, while the non-semantic embeddings cluster speech by speaker. Also, longer phrases are more indicative of intelligibility deficits than single words.

Advances in genetic engineering technologies have allowed the construction of artificial genetic circuits, which have been used to generate spatial patterns of differential gene expression. However, the question of how cells can be programmed, and how complex the rules need to be, to achieve a desired tissue morphology has received less attention. Here, we address these questions by developing a mathematical model to study how cells can collectively grow into clusters with different structural morphologies by secreting diffusible signals that can influence cellular growth rates. We formulate how growth regulators can be used to control the formation of cellular protrusions and how the range of achievable structures scales with the number of distinct signals. We show that a single growth inhibitor is insufficient for the formation of multiple protrusions but may be achieved with multiple growth inhibitors, and that other types of signals can regulate the shape of protrusion tips. These examples illustrate how our approach could potentially be used to guide the design of regulatory circuits for achieving a desired target structure.

Automatic speech recognition (ASR) systems have dramatically improved over the last few years. ASR systems are most often trained from 'typical' speech, which means that underrepresented groups don't experience the same level of improvement. In this paper, we present and evaluate finetuning techniques to improve ASR for users with non-standard speech. We focus on two types of non-standard speech: speech from people with amyotrophic lateral sclerosis (ALS) and accented speech. We train personalized models that achieve 62% and 35% relative WER improvement on these two groups, bringing the absolute WER for ALS speakers, on a test set of message bank phrases, down to 10% for mild dysarthria and 20% for more serious dysarthria. We show that 71% of the improvement comes from only 5 minutes of training data. Finetuning a particular subset of layers (with many fewer parameters) often gives better results than finetuning the entire model. This is the first step towards building state of the art ASR models for dysarthric speech.

Sphere packing problems have a rich history in both mathematics and physics; yet, relatively few analytical analyses of sphere packings exist, and answers to seemingly simple questions are unknown. Here, we present an analytical method for deriving all packings of n spheres in $\mathbb{R}^3$ satisfying minimal rigidity constraints ($\geq 3$ contacts per sphere and $\geq 3n-6$ total contacts). We derive such packings for $n \leq 10$ and provide a preliminary set of maximum contact packings for $10 < n \leq 20$. The resultant set of packings has some striking features; among them are the following: (i) all minimally rigid packings for $n \leq 9$ have exactly $3n-6$ contacts; (ii) nonrigid packings satisfying minimal rigidity constraints arise for $n \geq 9$; (iii) the number of ground states (i.e., packings with the maximum number of contacts) oscillates with respect to n; (iv) for $10 \leq n \leq 20$ there are only a small number of packings with the maximum number of contacts, and for $10 \leq n < 13$ these are all commensurate with the hexagonal close-packed lattice. The general method presented here may have applications to other related problems in mathematics, such as the Erdös repeated distance problem and Euclidean distance matrix completion problems.

Nuclear pore complexes (NPCs) are highly selective filters that control the exchange of material between nucleus and cytoplasm. The principles that govern selective filtering by NPCs are not fully understood. Previous studies find that cellular proteins capable of fast translocation through NPCs (transport receptors) are characterized by a high proportion of hydrophobic surface regions. Our analysis finds that transport receptors and their complexes are also highly negatively charged. Moreover, NPC components that constitute the permeability barrier are positively charged. We estimate that electrostatic interactions between a transport receptor and the NPC result in an energy gain of several k(B)T, which would enable significantly increased translocation rates of transport receptors relative to other cellular proteins. We suggest that negative charge is an essential criterion for selective passage through the NPC.

A recent paper of B. Naundorf et al. described an intriguing negative correlation between variability of the onset potential at which an action potential occurs (the onset span) and the rapidity of action potential initiation (the onset rapidity). This correlation was demonstrated in numerical simulations of the Hodgkin-Huxley model. Due to this antagonism, it is argued that Hodgkin-Huxley-type models are unable to explain action potential initiation observed in cortical neurons in vivo or in vitro. Here we apply a method from theoretical physics to derive an analytical characterization of this problem. We analytically compute the probability distribution of onset potentials and analytically derive the inverse relationship between onset span and onset rapidity. We find that the relationship between onset span and onset rapidity depends on the level of synaptic background activity. Hence we are able to elucidate the regions of parameter space for which the Hodgkin-Huxley model is able to accurately describe the behavior of this system.

We study how functional constraints bound and shape evolution through an analysis of mammalian voltage-gated sodium channels. The primary function of sodium channels is to allow the propagation of action potentials. Since Hodgkin and Huxley, mathematical models have suggested that sodium channel properties need to be tightly constrained for an action potential to propagate. There are nine mammalian genes encoding voltage-gated sodium channels, many of which are more than approximately 90% identical by sequence. This sequence similarity presumably corresponds to similarity of function, consistent with the idea that these properties must be tightly constrained. However, the multiplicity of genes encoding sodium channels raises the question: why are there so many? We demonstrate that the simplest theoretical constraints bounding sodium channel diversity--the requirements of membrane excitability and the uniqueness of the resting potential--act directly on constraining sodium channel properties. We compare the predicted constraints with functional data on mammalian sodium channel properties collected from the literature, including 172 different sets of measurements from 40 publications, wild-type and mutant, under a variety of conditions. The data from all channel types, including mutants, obeys the excitability constraint; on the other hand, channels expressed in muscle tend to obey the constraint of a unique resting potential, while channels expressed in neuronal tissue do not. The excitability properties alone distinguish the nine sodium channels into four different groups that are consistent with phylogenetic analysis. Our calculations suggest interpretations for the functional differences between these groups.