EDBT 2026 Demo / reviewers in the wild / expert
Mohsen Bayati
dblp:73/6405
· DBLP profile ↗
34ranked-venue papers
21as first author
7since 2021 · last 2026
0000-0002-7280-912XORCID · corroborated
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 15 · 5 first-author · 7 since 2021Theory of computation · 10 · 9 first-authorApplied, interdisciplinary, general and emerging computing · 6 · 5 first-authorComputer networks · 2 · 1 first-authorDatabases, data management, data science and information retrieval · 2 · 2 first-author
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Artificial intelligence
12 papers |
Reinforcement learning · 29% Trustworthy machine learning · 22% Learning theory · 16% | |
| Theoretical computer science
14 papers |
Mathematical optimization · 41% Algorithmic game theory and mechanism design · 40% Graph algorithms and graph theory · 6% | |
| Interdisciplinary, comprehensive, and emerging computing
1 paper |
Computational social science and digital humanities · 100% |
Topics — the 30 heaviest of 66, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Natural language and speech › Language models and text generation
alignment |
1.6 | 2 | 2025 | Conformal Arbitrage: Risk-Controlled Balancing of Competing Objectives in Language Models · NeurIPS 2025 Aligning Model Properties via Conformal Risk Control · NeurIPS 2024 |
Machine learning › Trustworthy machine learning › risk control
conformal risk control |
1.6 | 2 | 2025 | Conformal Arbitrage: Risk-Controlled Balancing of Competing Objectives in Language Models · NeurIPS 2025 Aligning Model Properties via Conformal Risk Control · NeurIPS 2024 |
Machine learning › Trustworthy machine learning
risk control |
1.6 | 2 | 2025 | Conformal Arbitrage: Risk-Controlled Balancing of Competing Objectives in Language Models · NeurIPS 2025 Aligning Model Properties via Conformal Risk Control · NeurIPS 2024 |
Machine learning › Probabilistic and Bayesian machine learning › statistical inference › regression
quantile regression |
1.0 | 1 | 2026 | Text-to-Distribution Prediction with Quantile Tokens and Neighbor Context · ACL (1) 2026 |
Algorithmic game theory and mechanism design › multi-armed bandit
linear bandits |
0.9 | 1 | 2025 | Geometry-Aware Approaches for Balancing Performance and Theoretical Guarantees in Linear Bandits · ICLR 2025 |
Algorithmic game theory and mechanism design
multi-armed bandit |
0.9 | 1 | 2025 | Geometry-Aware Approaches for Balancing Performance and Theoretical Guarantees in Linear Bandits · ICLR 2025 |
Mathematical optimization › online optimization
regret bounds |
0.9 | 1 | 2025 | Geometry-Aware Approaches for Balancing Performance and Theoretical Guarantees in Linear Bandits · ICLR 2025 |
Algorithmic game theory and mechanism design › multi-armed bandit
thompson sampling |
0.9 | 1 | 2025 | Geometry-Aware Approaches for Balancing Performance and Theoretical Guarantees in Linear Bandits · ICLR 2025 |
Machine learning › Graph learning › graph neural network
message passing |
0.8 | 1 | 2024 | Higher-Order Causal Message Passing for Experimentation with Complex Interference · NeurIPS 2024 |
Computational social science and digital humanities
causal inference |
0.8 | 1 | 2024 | Higher-Order Causal Message Passing for Experimentation with Complex Interference · NeurIPS 2024 |
Computational social science and digital humanities › causal inference
interference |
0.8 | 1 | 2024 | Higher-Order Causal Message Passing for Experimentation with Complex Interference · NeurIPS 2024 |
Computational social science and digital humanities › causal inference
treatment effect estimation |
0.8 | 1 | 2024 | Higher-Order Causal Message Passing for Experimentation with Complex Interference · NeurIPS 2024 |
Mathematical optimization › continuous optimization › matrix optimization
matrix recovery |
0.7 | 2 | 2022 | On Low-rank Trace Regression under General Sampling Distribution · J. Mach. Learn. Res. 2022 Personalizing Many Decisions with High-Dimensional Covariates · NeurIPS 2019 |
Machine learning › Reinforcement learning
continuous-time control |
0.6 | 1 | 2022 | Thompson Sampling Efficiently Learns to Control Diffusion Processes · NeurIPS 2022 |
Machine learning › Learning theory › model selection
cross-validation |
0.6 | 1 | 2022 | On Low-rank Trace Regression under General Sampling Distribution · J. Mach. Learn. Res. 2022 |
Machine learning › Reinforcement learning › exploration
exploration-exploitation tradeoff |
0.6 | 1 | 2022 | Thompson Sampling Efficiently Learns to Control Diffusion Processes · NeurIPS 2022 |
Machine learning › Learning theory
statistical learning theory |
0.6 | 1 | 2022 | On Low-rank Trace Regression under General Sampling Distribution · J. Mach. Learn. Res. 2022 |
Machine learning › Reinforcement learning
thompson sampling |
0.6 | 1 | 2022 | Thompson Sampling Efficiently Learns to Control Diffusion Processes · NeurIPS 2022 |
Mathematical optimization › statistical estimation › regression
trace regression |
0.6 | 1 | 2022 | On Low-rank Trace Regression under General Sampling Distribution · J. Mach. Learn. Res. 2022 |
Machine learning › Reinforcement learning › bandit
contextual bandit |
0.5 | 2 | 2020 | Personalizing Many Decisions with High-Dimensional Covariates · NeurIPS 2019 Unreasonable Effectiveness of Greedy Algorithms in Multi-Armed Bandit with Many Arms · NeurIPS 2020 |
Machine learning › Reinforcement learning › multi-armed bandit
greedy algorithm |
0.4 | 1 | 2020 | Unreasonable Effectiveness of Greedy Algorithms in Multi-Armed Bandit with Many Arms · NeurIPS 2020 |
Machine learning › Reinforcement learning
multi-armed bandit |
0.4 | 1 | 2020 | Unreasonable Effectiveness of Greedy Algorithms in Multi-Armed Bandit with Many Arms · NeurIPS 2020 |
Machine learning › Reinforcement learning
regret minimization |
0.4 | 1 | 2020 | Unreasonable Effectiveness of Greedy Algorithms in Multi-Armed Bandit with Many Arms · NeurIPS 2020 |
Machine learning › Reinforcement learning › bandit › contextual bandit
high-dimensional contextual bandit |
0.4 | 1 | 2019 | Personalizing Many Decisions with High-Dimensional Covariates · NeurIPS 2019 |
Machine learning › Reinforcement learning › multi-armed bandit › structured bandit
low-rank bandits |
0.4 | 1 | 2019 | Personalizing Many Decisions with High-Dimensional Covariates · NeurIPS 2019 |
Mathematical optimization › continuous optimization
convex optimization |
0.3 | 2 | 2025 | Geometry-Aware Approaches for Balancing Performance and Theoretical Guarantees in Linear Bandits · ICLR 2025 Scaled Least Squares Estimator for GLMs in Large-Scale Problems · NIPS 2016 |
Machine learning › Learning theory
high-dimensional statistics |
0.3 | 2 | 2013 | Estimating LASSO Risk and Noise Level · NIPS 2013 The LASSO risk: asymptotic results and real world examples · NIPS 2010 |
Mathematical optimization › statistical estimation › regression › sparse regression
lasso |
0.3 | 2 | 2013 | Estimating LASSO Risk and Noise Level · NIPS 2013 The LASSO risk: asymptotic results and real world examples · NIPS 2010 |
Mathematical optimization › statistical estimation › regression
regularized regression |
0.3 | 2 | 2013 | Estimating LASSO Risk and Noise Level · NIPS 2013 The LASSO risk: asymptotic results and real world examples · NIPS 2010 |
Machine learning › Efficient and distributed learning
model deployment |
0.3 | 1 | 2025 | Conformal Arbitrage: Risk-Controlled Balancing of Competing Objectives in Language Models · NeurIPS 2025 |
Methods — techniques the papers use, named apart from their topics
thompson sampling · 1.9conformal risk control · 1.6moment-based feature construction · 1.5function learning · 1.5quantile tokens · 1.0neighbor context · 1.0threshold calibration · 0.9greedy · 0.9frequentist regret analysis · 0.9OFUL · 0.9property testing · 0.8ordinary least squares · 0.6stochastic differential equation · 0.6restricted strong convexity · 0.6non-convex optimization · 0.6cross-validation · 0.6convex relaxation · 0.6row-enhancement subroutine · 0.4
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | Text-to-Distribution Prediction with Quantile Tokens and Neighbor ContextabstractYilun Zhu, Yuan Zhuang, Nikhita Vedula, Dushyanta Dhyani, Shaoyuan Xu, Mohsen Bayati, Bryan Wang, Shervin Malmasi. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026. Yilun Zhu 0003, Nikhita Vedula, Dushyanta Dhyani, Shaoyuan Xu, Mohsen Bayati, Bryan Wang, Shervin Malmasi |
ACL (1) | 6 |
| 2025 | Geometry-Aware Approaches for Balancing Performance and Theoretical Guarantees in Linear BanditsabstractThis paper is motivated by recent research in the $d$-dimensional stochastic linear bandit literature, which has revealed an unsettling discrepancy: algorithms like Thompson sampling and Greedy demonstrate promising empirical performance, yet this contrasts with their pessimistic theoretical regret bounds. The challenge arises from the fact that while these algorithms may perform poorly in certain problem instances, they generally excel in typical instances. To address this, we propose a new data-driven technique that tracks the geometric properties of the uncertainty ellipsoid around the main problem parameter. This methodology enables us to formulate a data-driven frequentist regret bound, which incorporates the geometric information, for a broad class of base algorithms, including Greedy, OFUL, and Thompson sampling. This result allows us to identify and ``course-correct" problem instances in which the base algorithms perform poorly. The course-corrected algorithms achieve the minimax optimal regret of order $\tilde{\mathcal{O}}(d\sqrt{T})$ for a $T$-period decision-making scenario, effectively maintaining the desirable attributes of the base algorithms, including their empirical efficacy. We present simulation results to validate our findings using synthetic and real data. Yuwei Luo, Mohsen Bayati |
ICLR | 2 |
| 2025 | Conformal Arbitrage: Risk-Controlled Balancing of Competing Objectives in Language ModelsabstractModern language‑model deployments must often balance competing objectives—for example, helpfulness versus harmlessness, cost versus accuracy, and reward versus safety. We introduce Conformal Arbitrage, a post‑hoc framework that learns a data‑driven threshold to mediate between a Primary model optimized for a primary objective and a more conservative Guardian—which could be another model or a human domain expert—aligned with a guardrail objective. The threshold is calibrated with conformal risk control, yielding finite‑sample, distribution‑free guarantees that the long‑run frequency of undesirable events (such as factual errors or safety violations) does not exceed a user‑specified quota. Because Conformal Arbitrage operates wholly at the API level—without requiring access to model logits or updating model weights—it complements weight‑based alignment techniques and integrates seamlessly with existing cost‑aware cascades. Empirically, Conformal Arbitrage traces an efficient frontier, allowing users to define an acceptable performance level for one objective while maximizing utility in another. We observe that our method outperforms (in terms of accuracy) cost-matched random routing between models. These properties make Conformal Arbitrage a practical, theoretically grounded tool for trustworthy and economical deployment of large language models across a broad range of potentially competing objectives. William Overman, Mohsen Bayati |
NeurIPS | 2 |
| 2024 | Higher-Order Causal Message Passing for Experimentation with Complex InterferenceabstractAccurate estimation of treatment effects is essential for decision-making across various scientific fields. This task, however, becomes challenging in areas like social sciences and online marketplaces, where treating one experimental unit can influence outcomes for others through direct or indirect interactions. Such interference can lead to biased treatment effect estimates, particularly when the structure of these interactions is unknown. We address this challenge by introducing a new class of estimators based on causal message-passing, specifically designed for settings with pervasive, unknown interference. Our estimator draws on information from the sample mean and variance of unit outcomes and treatments over time, enabling efficient use of observed data to estimate the evolution of the system state. Concretely, we construct non-linear features from the moments of unit outcomes and treatments and then learn a function that maps these features to future mean and variance of unit outcomes. This allows for the estimation of the treatment effect over time. Extensive simulations across multiple domains, using synthetic and real network data, demonstrate the efficacy of our approach in estimating total treatment effect dynamics, even in cases where interference exhibits non-monotonic behavior in the probability of treatment. Mohsen Bayati, Yuwei Luo, William Overman, Mohamad Sadegh Shirani Faradonbeh, Ruoxuan Xiong |
NeurIPS | 1 |
| 2024 | Aligning Model Properties via Conformal Risk ControlabstractAI model alignment is crucial due to inadvertent biases in training data and the underspecified machine learning pipeline, where models with excellent test metrics may not meet end-user requirements. While post-training alignment via human feedback shows promise, these methods are often limited to generative AI settings where humans can interpret and provide feedback on model outputs. In traditional non-generative settings with numerical or categorical outputs, detecting misalignment through single-sample outputs remains challenging, and enforcing alignment during training requires repeating costly training processes.
In this paper we consider an alternative strategy. We propose interpreting model alignment through property testing, defining an aligned model $f$ as one belonging to a subset $\mathcal{P}$ of functions that exhibit specific desired behaviors. We focus on post-processing a pre-trained model $f$ to better align with $\mathcal{P}$ using conformal risk control. Specifically, we develop a general procedure for converting queries for testing a given property $\mathcal{P}$ to a collection of loss functions suitable for use in a conformal risk control algorithm. We prove a probabilistic guarantee that the resulting conformal interval around $f$ contains a function approximately satisfying $\mathcal{P}$. We exhibit applications of our methodology on a collection of supervised learning datasets for (shape-constrained) properties such as monotonicity and concavity. The general procedure is flexible and can be applied to a wide range of desired properties. Finally, we prove that pre-trained models will always require alignment techniques even as model sizes or training data increase, as long as the training data contains even small biases. William Overman, Jacqueline Jil Vallon, Mohsen Bayati |
NeurIPS | 3 |
| 2022 | Thompson Sampling Efficiently Learns to Control Diffusion ProcessesabstractDiffusion processes that evolve according to linear stochastic differential equations are an important family of continuous-time dynamic decision-making models. Optimal policies are well-studied for them, under full certainty about the drift matrices. However, little is known about data-driven control of diffusion processes with uncertain drift matrices as conventional discrete-time analysis techniques are not applicable. In addition, while the task can be viewed as a reinforcement learning problem involving exploration and exploitation trade-off, ensuring system stability is a fundamental component of designing optimal policies. We establish that the popular Thompson sampling algorithm learns optimal actions fast, incurring only a square-root of time regret, and also stabilizes the system in a short time period. To the best of our knowledge, this is the first such result for Thompson sampling in a diffusion process control problem. We validate our theoretical results through empirical simulations with real matrices. Moreover, we observe that Thompson sampling significantly improves (worst-case) regret, compared to the state-of-the-art algorithms, suggesting Thompson sampling explores in a more guarded fashion. Our theoretical analysis involves characterization of a certain \emph{optimality manifold} that ties the local geometry of the drift parameters to the optimal control of the diffusion process. We expect this technique to be of broader interest. Mohamad Kazem Shirani Faradonbeh, Mohamad Sadegh Shirani Faradonbeh, Mohsen Bayati |
NeurIPS | 3 |
| 2022 | On Low-rank Trace Regression under General Sampling DistributionabstractIn this paper, we study the trace regression when a matrix of parameters $\mathbf{B}^\star$ is estimated via the convex relaxation of a rank-regularized regression or via regularized non-convex optimization. It is known that these estimators satisfy near-optimal error bounds under assumptions on the rank, coherence, and spikiness of $\mathbf{B}^\star$. We start by introducing a general notion of spikiness for $\mathbf{B}^\star$ that provides a generic recipe to prove the restricted strong convexity of the sampling operator of the trace regression and obtain near-optimal and non-asymptotic error bounds for the estimation error. Similar to the existing literature, these results require the regularization parameter to be above a certain theory-inspired threshold that depends on observation noise that may be unknown in practice. Next, we extend the error bounds to cases where the regularization parameter is chosen via cross-validation. This result is significant in that existing theoretical results on cross-validated estimators (Kale et al., 2011; Kumar et al., 2013; Abou-Moustafa and Szepesvari, 2017) do not apply to our setting since the estimators we study are not known to satisfy their required notion of stability. Finally, using simulations on synthetic and real data, we show that the cross-validated estimator selects a near-optimal penalty parameter and outperforms the theory-inspired approach of selecting the parameter. Nima Hamidi, Mohsen Bayati |
J. Mach. Learn. Res. | 2 |
| 2020 | Recommendation on a Budget: Column Space Recovery from Partially Observed Entries with Random or Active SamplingabstractWe analyze alternating minimization for column space recovery of a partially observed, approximately low rank matrix with a growing number of columns and a fixed budget of observations per column. We prove that if the budget is greater than the rank of the matrix, column space recovery succeeds – as the number of columns grows, the estimate from alternating minimization converges to the true column space with probability tending to one. From our proof techniques, we naturally formulate an active sampling strategy for choosing entries of a column that is theoretically and empirically (on synthetic and real data) better than the commonly studied uniformly random sampling strategy. Carolyn Kim, Mohsen Bayati |
AISTATS | 2 |
| 2020 | Unreasonable Effectiveness of Greedy Algorithms in Multi-Armed Bandit with Many ArmsabstractWe study the structure of regret-minimizing policies in the {\em many-armed} Bayesian multi-armed bandit problem: in particular, with $k$ the number of arms and $T$ the time horizon, we consider the case where $k \geq \sqrt{T}$. We first show that {\em subsampling} is a critical step for designing optimal policies. In particular, the standard UCB algorithm leads to sub-optimal regret bounds in the many-armed regime. However, a subsampled UCB (SS-UCB), which samples $\Theta(\sqrt{T})$ arms and executes UCB only on that subset, is rate-optimal. Despite theoretically optimal regret, even SS-UCB performs poorly due to excessive exploration of suboptimal arms. In particular, in numerical experiments SS-UCB performs worse than a simple greedy algorithm (and its subsampled version) that pulls the current empirical best arm at every time period. We show that these insights hold even in a contextual setting, using real-world data. These empirical results suggest a novel form of {\em free exploration} in the many-armed regime that benefits greedy algorithms. We theoretically study this new source of free exploration and find that it is deeply connected to the distribution of a certain tail event for the prior distribution of arm rewards. This is a fundamentally distinct phenomenon from free exploration as discussed in the recent literature on contextual bandits, where free exploration arises due to variation in contexts. We use this insight to prove that the subsampled greedy algorithm is rate-optimal for Bernoulli bandits when $k > \sqrt{T}$, and achieves sublinear regret with more general distributions. This is a case where theoretical rate optimality does not tell the whole story: when complemented by the empirical observations of our paper, the power of greedy algorithms becomes quite evident. Taken together, from a practical standpoint, our results suggest that in applications it may be preferable to use a variant of the greedy algorithm in the many-armed regime. Mohsen Bayati, Nima Hamidi, Ramesh Johari, Khashayar Khosravi |
NeurIPS | 1 |
| 2019 | Personalizing Many Decisions with High-Dimensional CovariatesabstractWe consider the k-armed stochastic contextual bandit problem with d dimensional features, when both k and d can be large. To the best of our knowledge, all existing algorithm for this problem have a regret bound that scale as polynomials of degree at least two in k and d. The main contribution of this paper is to introduce and theoretically analyze a new algorithm (REAL Bandit) with a regret that scales by r^2(k+d) when r is rank of the k by d matrix of unknown parameters. REAL Bandit relies on ideas from low-rank matrix estimation literature and a new row-enhancement subroutine that yields sharper bounds for estimating each row of the parameter matrix that may be of independent interest. Nima Hamidi, Mohsen Bayati |
NeurIPS | 2 |
| 2019 | Scalable Approximations for Generalized Linear ProblemsabstractIn stochastic optimization, the population risk is generally approximated by the empirical risk which is in turn minimized by an iterative algorithm. However, in the large-scale setting, empirical risk minimization may be computationally restrictive. In this paper, we design an efficient algorithm to approximate the population risk minimizer in generalized linear problems such as binary classification with surrogate losses and generalized linear regression models. We focus on large-scale problems where the iterative minimization of the empirical risk is computationally intractable, i.e., the number of observations $n$ is much larger than the dimension of the parameter $p$ ($n \gg p \gg 1$). We show that under random sub-Gaussian design, the true minimizer of the population risk is approximately proportional to the corresponding ordinary least squares (OLS) estimator. Using this relation, we design an algorithm that achieves the same accuracy as the empirical risk minimizer through iterations that attain up to a quadratic convergence rate, and that are computationally cheaper than any batch optimization algorithm by at least a factor of $\mathcal{O}(p)$. We provide theoretical guarantees for our algorithm, and analyze the convergence behavior in terms of data dimensions. Finally, we demonstrate the performance of our algorithm on well-known classification and regression problems, through extensive numerical studies on large-scale datasets, and show that it achieves the highest performance compared to several other widely used optimization algorithms. Murat A. Erdogdu, Mohsen Bayati, Lee H. Dicker |
J. Mach. Learn. Res. | 2 |
| 2016 | Scaled Least Squares Estimator for GLMs in Large-Scale ProblemsabstractWe study the problem of efficiently estimating the coefficients of generalized linear models (GLMs) in the large-scale setting where the number of observations $n$ is much larger than the number of predictors $p$, i.e. $n\gg p \gg 1$. We show that in GLMs with random (not necessarily Gaussian) design, the GLM coefficients are approximately proportional to the corresponding ordinary least squares (OLS) coefficients. Using this relation, we design an algorithm that achieves the same accuracy as the maximum likelihood estimator (MLE) through iterations that attain up to a cubic convergence rate, and that are cheaper than any batch optimization algorithm by at least a factor of $\mathcal{O}(p)$. We provide theoretical guarantees for our algorithm, and analyze the convergence behavior in terms of data dimensions. % Finally, we demonstrate the performance of our algorithm through extensive numerical studies on large-scale real and synthetic datasets, and show that it achieves the highest performance compared to several other widely used optimization algorithms. Murat A. Erdogdu, Lee H. Dicker, Mohsen Bayati |
NIPS | 3 |
| 2015 | A Low-Cost Method for Multiple Disease Prediction
Mohsen Bayati, Sonia Bhaskar, Andrea Montanari |
AMIA | 1 |
| 2015 | Matrix Completion Methods and Imputation for EMR-Based Prediction
Erika Strandberg, Mohsen Bayati |
AMIA | 2 |
| 2013 | Estimating LASSO Risk and Noise LevelabstractWe study the fundamental problems of variance and risk estimation in high dimensional statistical modeling. In particular, we consider the problem of learning a coefficient vector $\theta_0\in R^p$ from noisy linear observation $y=X\theta_0+w\in R^n$ and the popular estimation procedure of solving an $\ell_1$-penalized least squares objective known as the LASSO or Basis Pursuit DeNoising (BPDN). In this context, we develop new estimators for the $\ell_2$ estimation risk $\|\hat{\theta}-\theta_0\|_2$ and the variance of the noise. These can be used to select the regularization parameter optimally. Our approach combines Stein unbiased risk estimate (Stein'81) and recent results of (Bayati and Montanari'11-12) on the analysis of approximate message passing and risk of LASSO. We establish high-dimensional consistency of our estimators for sequences of matrices $X$ of increasing dimensions, with independent Gaussian entries. We establish validity for a broader class of Gaussian designs, conditional on the validity of a certain conjecture from statistical physics. Our approach is the first that provides an asymptotically consistent risk estimator. In addition, we demonstrate through simulation that our variance estimation outperforms several existing methods in the literature. Mohsen Bayati, Murat A. Erdogdu, Andrea Montanari |
NIPS | 1 |
| 2013 | Message-Passing Algorithms for Sparse Network AlignmentabstractNetwork alignment generalizes and unifies several approaches for forming a matching or alignment between the vertices of two graphs. We study a mathematical programming framework for network alignment problem and a sparse variation of it where only a small number of matches between the vertices of the two graphs are possible. We propose a new message passing algorithm that allows us to compute, very efficiently, approximate solutions to the sparse network alignment problems with graph sizes as large as hundreds of thousands of vertices. We also provide extensive simulations comparing our algorithms with two of the best solvers for network alignment problems on two synthetic matching problems, two bioinformatics problems, and three large ontology alignment problems including a multilingual problem with a known labeled alignment. Mohsen Bayati, David F. Gleich, Amin Saberi |
ACM Trans. Knowl. Discov. Data | 1 |
| 2012 | Universality in polytope phase transitions and iterative algorithmsabstractWe consider a class of nonlinear mappings FA, Nin RNindexed by symmetric random matrices A ϵ RN×Nwith independent entries. Within spin glass theory, special cases of these mappings correspond to iterating the TAP equations and were studied by Erwin Bolthausen. Within information theory, they are known as `approximate message passing' algorithms. We study the high-dimensional (large N) behavior of the iterates of F for polynomial functions F, and prove that it is universal, i.e. it depends only on the first two moments of the entries of A. As an application, we prove the universality of a certain phase transition arising in polytope geometry and compressed sensing. This solves a conjecture by David Donoho and Jared Tanner. Mohsen Bayati, Marc Lelarge, Andrea Montanari |
ISIT | 1 |
| 2012 | The LASSO Risk for Gaussian MatricesabstractWe consider the problem of learning a coefficient vector xο∈ RNfrom noisy linear observation y = Axo+ ∈ Rn. In many contexts (ranging from model selection to image processing), it is desirable to construct a sparse estimator x̂. In this case, a popular approach consists in solving an ℓ1-penalized least-squares problem known as the LASSO or basis pursuit denoising. For sequences of matrices A of increasing dimensions, with independent Gaussian entries, we prove that the normalized risk of the LASSO converges to a limit, and we obtain an explicit expression for this limit. Our result is the first rigorous derivation of an explicit formula for the asymptotic mean square error of the LASSO for random instances. The proof technique is based on the analysis of AMP, a recently developed efficient algorithm, that is inspired from graphical model ideas. Simulations on real data matrices suggest that our results can be relevant in a broad array of practical applications. Mohsen Bayati, Andrea Montanari |
IEEE Trans. Inf. Theory | 1 |
| 2011 | Fast Convergence of Natural Bargaining Dynamics in Exchange NetworksabstractBargaining networks model the behavior of a set of players who need to reach pairwise agreements for making profits. Nash bargaining solutions in this context correspond to solutions which are stable and balanced. Kleinberg and Tardos [19] proved that, if such solutions exist, then they can by calculated in polynomial time. This left open the question: Are there dynamics which can describe the bargaining process of real-world players, and which converge quickly to a Nash bargaining solution? This paper provides an affirmative answer to that question. The contribution of this paper is threefold: (1) We introduce a single-stage local dynamics which models the way in which actual players could bargain. We show that (approximate) fixed points of our dynamics are in one-to-one correspondence with (approximate) Nash bargaining solutions. (2) We prove that our dynamics converges to an ∊-fixed point in O(1/∊2) iterations independent of the network size when the potential earnings (weights) are uniformly bounded. We use this to prove that an approximate Nash bargaining solution is reached in time polynomial in 1/∊, the network size and 1/g. Here g is the difference between the weights of the two corners of the matching polytope having largest weights, and controls the behavior of fast message passing algorithms for maximum weight matching (matching naturally arises as a subproblem of Nash bargaining). (3) Our proof introduces a new powerful technique from functional analysis to this set of problems. The technique allows us to extend our results in various directions. We believe the tools introduced here will be useful in many related problems. As a corollary, for bipartite graphs we prove polynomial time convergence to an approximate Nash bargaining solution, with probability close to one under small random perturbations. Yashodhan Kanoria, Mohsen Bayati, Christian Borgs, Jennifer T. Chayes, Andrea Montanari |
SODA | 2 |
| 2011 | Belief Propagation for Weighted b-Matchings on Arbitrary Graphs and its Relation to Linear Programs with Integer SolutionsabstractWe consider the general problem of finding the minimum weight [Formula: see text]-matching on arbitrary graphs. We prove that, whenever the linear programming (LP) relaxation of the problem has no fractional solutions, then the belief propagation (BP) algorithm converges to the correct solution. We also show that when the LP relaxation has a fractional solution then the BP algorithm can be used to solve the LP relaxation. Our proof is based on the notion of graph covers and extends the analyses of [M. Bayati, D. Shah and M. Sharma, in Proceedings of the IEEE Int. Symp. Information Theory, 2005] and [B. Huang and T. Jebara, in Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics, 2007]. The result is notable in the following regards: (1) It is one of a very small number of proofs showing correctness of BP without any constraint on the graph structure; (2) Variants of the proof work for both synchronous and asynchronous BP; it is the first proof of convergence and correctness of an asynchronous BP algorithm for a combinatorial optimization problem. Mohsen Bayati, Christian Borgs, Jennifer T. Chayes, Riccardo Zecchina |
SIAM J. Discret. Math. | 1 |
| 2011 | The Dynamics of Message Passing on Dense Graphs, with Applications to Compressed Sensingabstract“Approximate message passing” (AMP) algorithms have proved to be effective in reconstructing sparse signals from a small number of incoherent linear measurements. Extensive numerical experiments further showed that their dynamics is accurately tracked by a simple one-dimensional iteration termed state evolution. In this paper, we provide rigorous foundation to state evolution. We prove that indeed it holds asymptotically in the large system limit for sensing matrices with independent and identically distributed Gaussian entries. While our focus is on message passing algorithms for compressed sensing, the analysis extends beyond this setting, to a general class of algorithms on dense graphs. In this context, state evolution plays the role that density evolution has for sparse graphs. The proof technique is fundamentally different from the standard approach to density evolution, in that it copes with a large number of short cycles in the underlying factor graph. It relies instead on a conditioning technique recently developed by Erwin Bolthausen in the context of spin glass theory. Mohsen Bayati, Andrea Montanari |
IEEE Trans. Inf. Theory | 1 |
| 2010 | The dynamics of message passing on dense graphs, with applications to compressed sensingabstract`Approximate message passing' algorithms proved to be extremely effective in reconstructing sparse signals from a small number of incoherent linear measurements. Extensive numerical experiments further showed that their dynamics is accurately tracked by a simple one-dimensional iteration termed state evolution. In this paper we provide the first rigorous foundation to state evolution. We prove that indeed it holds asymptotically in the large system limit for sensing matrices with iid gaussian entries. While our focus is on message passing algorithms for compressed sensing, the analysis extends beyond this setting, to a general class of algorithms on dense graphs. In this context, state evolution plays the role that density evolution has for sparse graphs. Mohsen Bayati, Andrea Montanari |
ISIT | 1 |
| 2010 | The LASSO risk: asymptotic results and real world examplesabstractWe consider the problem of learning a coefficient vector x0 from noisy linear observation y=Ax0+w. In many contexts (ranging from model selection to image processing) it is desirable to construct a sparse estimator. In this case, a popular approach consists in solving an l1-penalized least squares problem known as the LASSO or BPDN. For sequences of matrices A of increasing dimensions, with iid gaussian entries, we prove that the normalized risk of the LASSO converges to a limit, and we obtain an explicit expression for this limit. Our result is the first rigorous derivation of an explicit formula for the asymptotic risk of the LASSO for random instances. The proof technique is based on the analysis of AMP, a recently developed efficient algorithm, that is inspired from graphical models ideas. Through simulations on real data matrices (gene expression data and hospital medical records) we observe that these results can be relevant in a broad array of practical applications. Mohsen Bayati, José Bento 0001, Andrea Montanari |
NIPS | 1 |
| 2010 | Combinatorial approach to the interpolation method and scaling limits in sparse random graphsabstractWe establish the existence of free energy limits for several sparse random hypergraph models corresponding to certain combinatorial models on Erdos-Renyi (ER) graph G(N,c/N) and random r-regular graph G(N,r). Mohsen Bayati, David Gamarnik, Prasad Tetali |
STOC | 1 |
| 2010 | A Sequential Algorithm for Generating Random Graphs
Mohsen Bayati, Jeong Han Kim, Amin Saberi |
Algorithmica | 1 |
| 2009 | Algorithms for Large, Sparse Network Alignment ProblemsabstractWe propose a new distributed algorithm for sparse variants of the network alignment problem, which occurs in a variety of data mining areas including systems biology, database matching, and computer vision. Our algorithm uses a belief propagation heuristic and provides near optimal solutions for this NP-hard combinatorial optimization problem. We show that our algorithm is faster and outperforms or ties existing algorithms on synthetic problems, a problem in bioinformatics, and a problem in ontology matching. We also provide a unified framework for studying and comparing all network alignment solvers. Mohsen Bayati, Margot Gerritsen, David F. Gleich, Amin Saberi |
ICDM | 1 |
| 2009 | Generating random graphs with large girthabstractWe present a simple and efficient algorithm for randomly generating simple graphs without small cycles. These graphs can be used to design high performance Low-Density Parity-Check (LDPC) codes. For any constant k, α ≤ 1/2k(k + 3) and m = O(n1+α), our algorithm generates an asymptotically uniform random graph with n vertices, m edges, and girth larger than k in polynomial time. To the best of our knowledge this is the first polynomial algorithm for the problem. Our algorithm generates a graph by sequentially adding m edges to an empty graph with n vertices. Recently, this type of sequential process has been very successful for efficiently counting and generating random graphs [35, 18, 11, 7, 5, 6]. Mohsen Bayati, Andrea Montanari, Amin Saberi |
SODA | 1 |
| 2008 | Max-Product for Maximum Weight Matching: Convergence, Correctness, and LP DualityabstractMax-product "belief propagation" (BP) is an iterative, message-passing algorithm for finding the maximum a posteriori (MAP) assignment of a discrete probability distribution specified by a graphical model. Despite the spectacular success of the algorithm in many application areas such as iterative decoding and combinatorial optimization, which involve graphs with many cycles, theoretical results about both the correctness and convergence of the algorithm are known in only a few cases (see section I for references). In this paper, we prove the correctness and convergence of max-product for finding the maximum weight matching (MWM) in bipartite graphs. Even though the underlying graph of the MWM problem has many cycles, somewhat surprisingly we show that the max-product algorithm converges to the correct MWM as long as the MWM is unique. We provide a bound on the number of iterations required and show that for a graph of size n, the computational cost of the algorithm scales as O(n3), which is the same as the computational cost of the best known algorithms for finding the MWM. We also provide an interesting relation between the dynamics of the max-product algorithm and the auction algorithm, which is a well-known distributed algorithm for solving the MWM problem. Mohsen Bayati, Devavrat Shah |
IEEE Trans. Inf. Theory | 1 |
| 2007 | A Sequential Algorithm for Generating Random Graphs
Mohsen Bayati, Jeong Han Kim, Amin Saberi |
APPROX-RANDOM | 1 |
| 2007 | Iterative Scheduling AlgorithmsabstractThe input-queued switch architecture is widely used in Internet routers due to its ability to run at very high line speeds. A central problem in designing an input-queued switch is the scheduling algorithm that decides which packets to transfer from ingress ports to egress ports in a given timeslot. It is desirable that such algorithms be iterative (so as to be pipelineable), distributed (allowing flexibility in hardware implementation) and are able to deliver high performance (in terms of throughput and delay). In practice, implementable algorithms have so far had limited success in combining all of the above properties. For example, the popular iSLIP algorithm is known to perform suboptimally, but it is commercially deployed mainly because it is iterative and distributed. The main contribution of this paper is the design and systematic analysis of two algorithms which, to the best of our knowledge, are the first high-performance iterative and distributed scheduling algorithms with possibility of efficient implementation. We first present an iterative, distributed and low-delay maximal throughput algorithm based on the celebrated "auction algorithm". This algorithm can be seen as a natural extension of iSLIP when queue-size information is allowed to be exchanged. The standard auction algorithm can take an unbounded number of iterations to converge in the worst case. However we show that under admissible Bernoulli i.i.d. traffic, our algorithm takes O(n2) iterations, where n is the number of ingress/egress ports in the switch. Moreover for a switch with finite buffer-size, the algorithm allows for a graceful trade-off between running time and performance, which we verify by representative simulation results. Next, we propose and analyze a throughput-optimal, iterative and distributed scheduling algorithm influenced by Max-product belief propagation. Recently the problem of efficient transmission over multi-hop wireless networks has been formulated as that of finding an appropriate schedule over the grid-graph abstraction of the network. A key feature of the multi-hop wireless transmission problem is that while the communication subgraph is bipartite, the bi-partition is allowed to change in each scheduling epoch. We show that our algorithm can be used to efficiently schedule traffic in multi-hop wireless networks. Mohsen Bayati, Balaji Prabhakar, Devavrat Shah |
INFOCOM | 1 |
| 2007 | Simple deterministic approximation algorithms for counting matchingsabstractWe construct a deterministic fully polynomial time approximationscheme (FPTAS) for computing the total number of matchings in abounded degree graph. Additionally, for an arbitrary graph, weconstruct a deterministic algorithm for computing approximately thenumber of matchings within running time exp(O(√n log2n)),where n is the number of vertices. Mohsen Bayati, David Gamarnik, Dimitriy A. Katz, Chandra Nair, Prasad Tetali |
STOC | 1 |
| 2006 | A Simpler Max-Product Maximum Weight Matching Algorithm and the Auction AlgorithmabstractThe max-product "belief propagation" algorithm has received a lot of attention recently due to its spectacular success in many application areas such as iterative decoding, computer vision and combinatorial optimization. There is a lot of ongoing work investigating the theoretical properties of the algorithm. In our previous work (2005) we showed that the max-product algorithm can be used to solve the problem of finding the maximum weight matching (MWM) in a weighted complete bipartite graph. However, for a graph with n nodes the max-product algorithm requires O(n4) operations to find the MWM compared to O(n3) for best known algorithms such as those proposed by Edmonds and Karp (1972) and Bertsekas (1988). In this paper, we simplify the max-product algorithm to reduce the number of operations required to O(n3). The simplified algorithm has very similar dynamics to the well-known auction algorithm of Bertsekas (1988). To make this connection precise, we show that the max-product and auction algorithms, when slightly modified, are equivalent. We study the correctness of this modified algorithm. There is a tantalizing similarity between this connection and a recently observed connection between the max-product and LP-based algorithms for iterative decoding by Vontobel and Koetter Mohsen Bayati, Devavrat Shah |
ISIT | 1 |
| 2005 | Achieving stability in networks of input-queued switches using a local online scheduling policyabstractIn recent years, several high-throughput low-delay scheduling algorithms have been designed for input-queued (IQ) switches. It has been shown however that scheduling policies such as maximum weight matching, that perform optimally for an isolated switch, fail to provide stability in a network of IQ switches (M. Andrews and L. Zhang, 2001). Although there exist algorithms that ensure stability in networks of switches (M. Andrews and L. Zhang, 2001) (M. Ajmone Marsan et al., 2003), they are either not fully local or require knowledge/estimation of rates, and are thus not desirable. Here we propose a local and online switch-scheduling algorithm and prove that it achieves stability in a network of single-server switches when arriving traffic is admissible and obeys the strong law of large numbers. We then propose its counterpart for networks of crossbar switches and conjecture that this too is stable. Additionally, we prove that our algorithms provide a max-min fair rate allocation for isolated switches even when arriving traffic is inadmissible. We believe that fairness is key to ensuring stability in networks. Shubha U. Nabar, Neha Kumar 0001, Mohsen Bayati, Abtin Keshavarzian |
GLOBECOM | 3 |
| 2005 | Maximum weight matching via max-product belief propagationabstractThe max-product "belief propagation" algorithm is an iterative, local, message passing algorithm for finding the maximum a posteriori (MAP) assignment of a discrete probability distribution specified by a graphical model. Despite the spectacular success of the algorithm in many application areas such as iterative decoding and computer vision which involve graphs with many cycles, theoretical convergence results are only known for graphs which are tree-like or have a single cycle. In this paper, we consider a weighted complete bipartite graph and define a probability distribution on it whose MAP assignment corresponds to the maximum weight matching (MWM) in that graph. We analyze the fixed points of the max-product algorithm when run on this graph and prove the surprising result that even though the underlying graph has many short cycles, the maxproduct assignment converges to the correct MAP assignment. We also provide a bound on the number of iterations required by the algorithm Mohsen Bayati, Devavrat Shah |
ISIT | 1 |