We train Transformer-based language models on ten foundational algorithmic tasks and observe pronounced phase transitions in their loss curves that deviate from established power-law scaling trends. Over large ranges of compute, the validation loss barely improves, then abruptly decreases. Probing the models’ internal representations reveals that quiet features are learned prior to any decrease in task loss. These quiet features represent intermediate algorithmic computations that do not by themselves improve the output loss. Ablation experiments demonstrate that individual quiet features are causally necessary for task performance. Our results demonstrate that substantial representational progress can remain hidden beneath an apparently flat loss curve, challenging the prevailing use of cross‑entropy as a proxy for learning and motivating richer diagnostics for monitoring model training.

We show that Inner Product in 2n variables, IP_n(x, y) = x₁y₁ ⊕ … ⊕ x_ny_n, can be computed by depth-3 bottom fan-in 2 circuits of size poly(n)⋅ (9/5)ⁿ, matching the lower bound of Göös, Guan, and Mosnoi (Inform. Comput.'24). Our construction is obtained via the following steps. 1) We provide a general template for constructing optimal depth-3 circuits with bottom fan-in k for an arbitrary function f. We do this in two steps. First, we partition f^{-1}(1) into orbits of its automorphism group. Second, for each orbit, we construct one k-CNF that (a) accepts the largest number of inputs from that orbit and (b) rejects all inputs rejected by f. 2) We instantiate the template for IP_n and k = 2. Guided by the intuition (which we call modularity principle) that optimal 2-CNFs can be constructed by taking the conjunction of variable-disjoint copies of smaller 2-CNFs, we use computer search to identify a small set of building block 2-CNFs over at most 4 variables. 3) We again use computer search to discover appropriate combinations (disjoint conjunctions) of building blocks to arrive at optimal 2-CNFs and analyze them using techniques from analytic combinatorics. We believe that the approach outlined in this paper can be applied to a wide range of functions to determine their depth-3 complexity.

Jianyou Wang, Weili Cao, Longtian Bao, Youze Zheng, Gil Pasternak, Kaicheng Wang, Xiaoyue Wang, Ramamohan Paturi, Leon Bergen. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Gurumukhani et al. (CCC'24) proposed the local enumeration problem Enum(k, t) as an approach to break the Super Strong Exponential Time Hypothesis (SSETH): for a natural number $k$ and a parameter $t$, given an $n$-variate $k$-CNF with no satisfying assignment of Hamming weight less than $t(n)$, enumerate all satisfying assignments of Hamming weight exactly $t(n)$. Furthermore, they gave a randomized algorithm for Enum(k, t) and employed new ideas to analyze the first non-trivial case, namely $k = 3$. In particular, they solved Enum(3, n/2) in expected $1.598^n$ time. A simple construction shows a lower bound of $6^{\frac{n}{4}} \approx 1.565^n$. In this paper, we show that to break SSETH, it is sufficient to consider a simpler local enumeration problem NAE-Enum(k, t): for a natural number $k$ and a parameter $t$, given an $n$-variate $k$-CNF with no satisfying assignment of Hamming weight less than $t(n)$, enumerate all Not-All-Equal (NAE) solutions of Hamming weight exactly $t(n)$, i.e., those that satisfy and falsify some literal in every clause. We refine the algorithm of Gurumukhani et al. and show that it optimally solves NAE-Enum(3, n/2), namely, in expected time $poly(n) \cdot 6^{\frac{n}{4}}$.

Depth-3 circuit lower bounds and k-SAT algorithms are intimately related; the state-of-the-art Σ^k_3-circuit lower bound (Or-And-Or circuits with bottom fan-in at most k) and the k-SAT algorithm of Paturi, Pudlák, Saks, and Zane (J. ACM'05) are based on the same combinatorial theorem regarding k-CNFs. In this paper we define a problem which reveals new interactions between the two, and suggests a concrete approach to significantly stronger circuit lower bounds and improved k-SAT algorithms. For a natural number k and a parameter t, we consider the Enum(k, t) problem defined as follows: given an n-variable k-CNF and an initial assignment α, output all satisfying assignments at Hamming distance t(n) of α, assuming that there are no satisfying assignments of Hamming distance less than t(n) of α. We observe that an upper bound b(n, k, t) on the complexity of Enum(k, t) simultaneously implies depth-3 circuit lower bounds and k-SAT algorithms: - Depth-3 circuits: Any Σ^k_3 circuit computing the Majority function has size at least binom(n,n/2)/b(n, k, n/2). - k-SAT: There exists an algorithm solving k-SAT in time O(∑_{t=1}^{n/2}b(n, k, t)). A simple construction shows that b(n, k, n/2) ≥ 2^{(1 - O(log(k)/k))n}. Thus, matching upper bounds for b(n, k, n/2) would imply a Σ^k_3-circuit lower bound of 2^Ω(log(k)n/k) and a k-SAT upper bound of 2^{(1 - Ω(log(k)/k))n}. The former yields an unrestricted depth-3 lower bound of 2^ω(√n) solving a long standing open problem, and the latter breaks the Super Strong Exponential Time Hypothesis. In this paper, we propose a randomized algorithm for Enum(k, t) and introduce new ideas to analyze it. We demonstrate the power of our ideas by considering the first non-trivial instance of the problem, i.e., Enum(3, n/2). We show that the expected running time of our algorithm is 1.598ⁿ, substantially improving on the trivial bound of 3^{n/2} ≃ 1.732ⁿ. This already improves Σ^3_3 lower bounds for Majority function to 1.251ⁿ. The previous bound was 1.154ⁿ which follows from the work of Håstad, Jukna, and Pudlák (Comput. Complex.'95). By restricting ourselves to monotone CNFs, Enum(k, t) immediately becomes a hypergraph Turán problem. Therefore our techniques might be of independent interest in extremal combinatorics.

Effective information retrieval (IR) in settings with limited training data, particularly for complex queries, remains a challenging task. This paper introduces IR2, Information Regularization for Information Retrieval, a technique for reducing overfitting during synthetic data generation. This approach, representing a novel application of regularization techniques in synthetic data creation for IR, is tested on three recent IR tasks characterized by complex queries: DORIS-MAE, ArguAna, and WhatsThatBook. Experimental results indicate that our regularization techniques not only outperform previous synthetic query generation methods on the tasks considered but also reduce cost by up to 50%. Furthermore, this paper categorizes and explores three regularization methods at different stages of the query synthesis pipeline—input, prompt, and output—each offering varying degrees of performance improvement compared to models where no regularization is applied. This provides a systematic approach for optimizing synthetic data generation in data-limited, complex-query IR scenarios. All code, prompts and synthetic data are available at https://github.com/Info-Regularization/Information-Regularization.

In scientific research, the ability to effectively retrieve relevant documents based on complex, multifaceted queries is critical. Existing evaluation datasets for this task are limited, primarily due to the high costs and effort required to annotate resources that effectively represent complex queries. To address this, we propose a novel task, $\textbf{S}$cientific $\textbf{Do}$cument $\textbf{R}$etrieval using $\textbf{M}$ulti-level $\textbf{A}$spect-based qu$\textbf{E}$ries (DORIS-MAE), which is designed to handle the complex nature of user queries in scientific research. We developed a benchmark dataset within the field of computer science, consisting of 100 human-authored complex query cases. For each complex query, we assembled a collection of 100 relevant documents and produced annotated relevance scores for ranking them. Recognizing the significant labor of expert annotation, we also introduce Anno-GPT, a scalable framework for evaluating the viability of Large Language Models (LLMs) such as ChatGPT-3.5 for expert-level dataset annotation tasks. The application of Anno-GPT to annotate the DORIS-MAE dataset resulted in a 500x reduction in cost, without compromising quality. Furthermore, due to the multi-tiered structure of these complex queries, our DORIS-MAE dataset can be extended to over 4,000 sub-query test cases without requiring additional annotation. We evaluated 17 recent retrieval methods on DORIS-MAE, observing notable performance drops compared to traditional datasets. This highlights DORIS-MAE's challenges and the need for better approaches to handle complex, multifaceted queries in scientific research. Our dataset and codebase are available at https://github.com/Real-Doris-Mae/Doris-Mae-Dataset .

We investigate the algorithmic properties of circuits of bounded treewidth. Here the treewidth of a circuit C is defined as the treewidth of the underlying undirected graph of C, after the vertices corresponding to input gates have been removed. Thus, boolean formulae correspond to circuits of treewidth 1. Our first main result is an algorithm for counting the number of satisfying assignments of circuits with n input gates, treewidth ω, and at most s · n gates. The running time of our algorithm is , which for formulae instantiates to 2n(1–1/O(s)). This is the first algorithm to achieve exponential speedup over brute force for the satisfiability of linear size circuits with treewidth bounded by a constant greater than 1. For treewidth 1, i.e., boolean formulae, our algorithm significantly outperforms the previously fastest 2n(1-1/O(s2)) time satisfiability algorithm by Santhanam [32]. Our second main result is an algorithm for True Quantified Boolean Circuit Satisfiability for circuits of treewidth ω, in which every input gate has fanout at most s. The running time of our algorithm is . Our algorithm is the first to achieve exponential speed-up over brute force for such circuits. Indeed, even for quantified boolean formulae where every variable appears at most s times, the previously best known algorithm by Santhanam [32] has running time 2n(1–1/O(f(s)log n)). Utilizing the structural properties of low treewidth circuits which helped us obtain improved exponential-time algorithms for satisfiability, we also show that the number of wires of any constant treewidth circuit that computes the majority function must be super-linear.

In the recent years, significant progress has been made in explaining apparent hardness of improving over naive solutions for many fundamental polynomially solvable problems. This came in the form of conditional lower bounds -- reductions from a problem assumed to be hard. These include 3SUM, All-Pairs Shortest Paths, SAT and Orthogonal Vectors, and others. In the (min,+)-convolution problem, the goal is to compute a sequence c, where c[k] = min_i a[i]+b[k-i], given sequences a and b. This can easily be done in O(n^2) time, but no O(n^{2-eps}) algorithm is known for eps > 0. In this paper we undertake a systematic study of the (min,+)-convolution problem as a hardness assumption. As the first step, we establish equivalence of this problem to a group of other problems, including variants of the classic knapsack problem and problems related to subadditive sequences. The (min,+)-convolution has been used as a building block in algorithms for many problems, notably problems in stringology. It has also already appeared as an ad hoc hardness assumption. We investigate some of these connections and provide new reductions and other results.

We consider the problem of solving systems of multivariate polynomial equations of degree k over a finite field. For every integer k ≤ 2 and finite field q where q = pd for a prime p, we give, to the best of our knowledge, the first algorithms that achieve an exponential speedup over the brute force O(qn) time algorithm in the worst case. We present two algorithms, a randomized algorithm with running time qn+o(n) · q−n/O(k) time if q < 24ekd, and otherwise, where e = 2.718… is Napier's constant, and a deterministic algorithm for counting solutions with running time qn+o(n) · q−n/O(kq6/7d). For the important special case of quadratic equations in F2, our randomized algorithm has running time O(20.8765n). For systems over 2 we also consider the case where the input polynomials do not have bounded degree, but instead can be efficiently represented as a ΣΠΣ circuit, i.e., a sum of products of sums of variables. For this case we present a deterministic algorithm running in time 2n-dn for δ = 1/O(log(s/n)) for instances with s product gates in total and n variables. Our algorithms adapt several techniques recently developed via the polynomial method from circuit complexity. The algorithm for systems of ΣΠΣ polynomials also introduces a new degree reduction method that takes an instance of the problem and outputs a subexponential-sized set of instances, in such a way that feasibility is preserved and every polynomial among the output instances has degree O(log(s/n)).

We introduce the Nondeterministic Strong Exponential Time Hypothesis (NSETH) as a natural extension of the Strong Exponential Time Hypothesis (SETH). We show that both refuting and proving NSETH would have interesting consequences.

The field of exact exponential time algorithms for non-deterministic polynomial-time hard problems has thrived since the mid-2000s. While exhaustive search remains asymptotically the fastest known algorithm for some basic problems, non-trivial exponential time algorithms have been found for a myriad of problems, including G raph C oloring , H amiltonian P ath , D ominating S et , and 3-CNF-S at . In some instances, improving these algorithms further seems to be out of reach. The CNF-S at problem is the canonical example of a problem for which the trivial exhaustive search algorithm runs in time O (2 n ), where n is the number of variables in the input formula. While there exist non-trivial algorithms for CNF-S at that run in time o (2 n ), no algorithm was able to improve the growth rate 2 to a smaller constant, and hence it is natural to conjecture that 2 is the optimal growth rate. The strong exponential time hypothesis (SETH) by Impagliazzo and Paturi [JCSS 2001] goes a little bit further and asserts that, for every ϵ < 1, there is a (large) integer k such that k -CNF-S at cannot be computed in time 2 ϵ n . In this article, we show that, for every ϵ < 1, the problems H itting S et , S et S plitting , and NAE-S at cannot be computed in time O (2 ϵ n ) unless SETH fails. Here n is the number of elements or variables in the input. For these problems, we actually get an equivalence to SETH in a certain sense. We conjecture that SETH implies a similar statement for S et C over and prove that, under this assumption, the fastest known algorithms for S teiner T ree , C onnected V ertex C over , S et P artitioning , and the pseudo-polynomial time algorithm for S ubset S um cannot be significantly improved. Finally, we justify our assumption about the hardness of S et C over by showing that the parity of the number of solutions to S et C over cannot be computed in time O (2 ϵ n ) for any ϵ < 1 unless SETH fails.

We give a nontrivial algorithm for the satisfiability problem for threshold circuits of depth two with a linear number of wires which improves over exhaustive search by an exponential factor. The independently interesting problem of the feasibility of sparse 0-1 integer linear programs is a special case. To our knowledge, our algorithm is the first to achieve constant savings even for the special case of Integer Linear Programming. The key idea is to reduce the satisfiability problem to the Vector Domination problem, the problem of checking whether there are two vectors in a given collection of vectors such that one dominates the other component-wise. Our result generalizes to formulas of arbitrary constant depth. We also provide a satisfiability algorithm with constant savings for depth two circuits with symmetric gates where the total weighted fan-in is at most linear in the number of variables. One of our motivations is proving strong lower bounds for TC0 circuits, exploiting the connection (established by Williams) between satisfiability algorithms and lower bounds. Our second motivation is to explore the connection between the expressive power of the circuits and the complexity of the corresponding circuit satisfiability problem.

The field of exact exponential time algorithms for NP-hard problems has thrived over the last decade. While exhaustive search remains asymptotically the fastest known algorithm for some basic problems, difficult and non-trivial exponential time algorithms have been found for a myriad of problems, including GRAPH COLORING, HAMILTONIAN PATH, DOMINATING SET and 3-CNF-SAT. In some instances, improving these algorithms further seems to be out of reach. The CNF-SAT problem is the canonical example of a problem for which the trivial exhaustive search algorithm runs in time O(2n), where n is the number of variables in the input formula. While there exist non-trivial algorithms for CNF-SAT that run in time o(2n), no algorithm was able to improve the growth rate 2 to a smaller constant, and hence it is natural to conjecture that 2 is the optimal growth rate. The strong exponential time hypothesis (SETH) by Impagliazzo and Paturi [JCSS 2001] goes a little bit further and asserts that, for every ϵϵn. In this paper, we show that, for every ϵϵn) unless SETH fails. Here n is the number of elements or variables in the input. For these problems, we actually get an equivalence to SETH in a certain sense. We conjecture that SETH implies a similar statement for SET COVER, and prove that, under this assumption, the fastest known algorithms for STEINTER TREE, CONNECTED VERTEX COVER, SET PARTITIONING, and the pseudo-polynomial time algorithm for SUBSET SUM cannot be significantly improved. Finally, we justify our assumption about the hardness of SET COVER by showing that the parity of the number of set covers.

We consider the problem of efficiently enumerating the satisfying assignments to AC 0 circuits.We give a zeroerror randomized algorithm which takes an AC 0 circuit as input and constructs a set of restrictions which partitions {0, 1} n so that under each restriction the value of the circuit is constant.Let d denote the depth of the circuit and cn denote the number of gates.This algorithm runs in time |C|2 n(1-µ c,d ) where |C| is the size of the circuit for µ c,d ≥ 1/O[lg c + d lg d] d-1 with probability at least 1 -2 -n .As a result, we get improved exponential time algorithms for AC 0 circuit satisfiability and for counting solutions.In addition, we get an improved bound on the correlation of AC 0 circuits with parity.As an important component of our analysis, we extend the Håstad Switching Lemma to handle multiple k-cnfs and k-dnfs.

A growing literature on human networks suggests that the way we are connected influences both individual and group outcomes. Recent experimental studies in the social and computer sciences have claimed that higher network connectivity helps individuals solve coordination problems. However, this is not always the case, especially when we consider complex coordination tasks; we demonstrate that networks can have both constraining edges that inhibit collective action and redundant edges that encourage it. We show that the constraints imposed by additional edges can impede coordination even though these edges also increase communication. By contrast, edges that do not impose additional constraints facilitate coordination, as described in previous work. We explain why the negative effect of constraint trumps the positive effect of communication by analyzing coordination games as a special case of widely-studied constraint satisfaction problems. The results help us to understand the importance of problem complexity and network connections, and how different types of connections can influence real-world coordination.

In this paper, we are concerned with the exponential complexity of the Circuit Satisfiability (CktSat) problem and more generally with the exponential complexity of NP-complete problems. Over the past 15 years or so, researchers have obtained a number of exponential-time algorithms with improved running times for exactly solving a variety of NP-complete problems. The improvements are typically in the form of better exponents compared to exhaustive search. Our goal is to develop techniques to prove specific lower bounds on the exponents under plausible complexity assumptions. We consider natural, though restricted, algorithmic paradigms and prove upper bounds on the success probability. Our approach has the advantage of clarifying the relative power of various algorithmic paradigms. Our main technique is a success probability amplification technique, called the Exponential Amplification Lemma, which shows that for any f(n,m)-size bounded probabilistic circuit family A that decides CktSat with success probability at least 2-α n for α<1 on inputs which are circuits of size m with n variables, there is another probabilistic circuit family B that decides CktSat with size roughly f(α n, f(n,m)) and success probability about 2-α2 n > 2-α n.

In this paper, we present a new link-state routing algorithm called Approximate Link state (XL) aimed at increasing routing efficiency by suppressing updates from parts of the network. We prove that three simple criteria for update propagation are sufficient to guarantee soundness, completeness and bounded optimality for any such algorithm. We show, via simulation, that XL significantly outperforms standard link-state and distance vector algorithms - in some cases reducing overhead by more than an order of magnitude - while having negligible impact on path length. Finally, we argue that existing link-state protocols, such as OSPF, can incorporate XL routing in a backwards compatible and incrementally deployable fashion.

We consider the relationship between the complexities of k-SAT and those of SAT restricted to formulas of constant density. Let skbe the infimum of those c ges 0 such that k-SAT on n variables can be decided in time O(2cn) and dDeltabe the infimum of those c ges 0 such that SAT on n variables and les Deltan clauses can be decided in time O(2cn). We show that limkrarrinfinsk= limDeltararrinfindDelta. So, for any epsi > 0, k-SAT can be solved in 2(1-epsi)ntime independent of k if and only if the same is true for SAT with any fixed density of clauses to variables. We derive some interesting consequences from this. For example, assuming that 3-SAT is exponentially hard (that is, s3> 0), SAT of any fixed density can be solved in time whose exponent is strictly less than that for general SAT. We also give an improvement to the sparsification lemma of Impagliazzo et al. (1998) showing that instances of k-SAT of density slightly more than exponential in k are almost the hardest instances of k-SAT. The previous result showed this for densities doubly exponential in k

We propose and analyze a simple new randomized algorithm, called ResolveSat, for finding satisfying assignments of Boolean formulas in conjunctive normal form. The algorithm consists of two stages: a preprocessing stage in which resolution is applied to enlarge the set of clauses of the formula, followed by a search stage that uses a simple randomized greedy procedure to look for a satisfying assignment. Currently, this is the fastest known probabilistic algorithm for k -CNF satisfiability for k ≥ 4 (with a running time of O (2 0.5625 n ) for 4-CNF). In addition, it is the fastest known probabilistic algorithm for k -CNF, k ≥ 3, that have at most one satisfying assignment (unique k -SAT) (with a running time O (2 (2 ln 2 − 1) n + o ( n ) ) = O (2 0.386 … n ) in the case of 3-CNF). The analysis of the algorithm also gives an upper bound on the number of the codewords of a code defined by a k -CNF. This is applied to prove a lower bounds on depth 3 circuits accepting codes with nonconstant distance. In particular we prove a lower bound Ω(2 1.282…√>i /i< ) for an explicitly given Boolean function of n variables. This is the first such lower bound that is asymptotically bigger than 2 √>i /i< + o (√>i /i<) .

Most network intrusion tools (e.g., Bro) use per-flow state to reassemble TCP connections and fragments in order to detect network attacks (e.g., SYN Flooding or Connection Hijacking) and preliminary reconnaissance (e.g., Port Scans). On the other hand, if network intrusion detection is to be implemented at high speeds at network vantage points, some form of aggregation is necessary. While many security analysts believe that such per-flow state is required for many of these problems, there is no clear proof that this is the case. In fact, a number of problems (such as detecting large traffic footprints or counting identifiers) have scalable solutions. In this paper, we initiate the study of identifying when and how a security attack detection problem can have a scalable solution. We use tools from Communication Complexity to prove that the common formulations of many well-known intrusion detection problems (detecting SYN Flooding, Port Scans, Connection Hijacking, and content matching across fragments) require per-flow state. Our theory exposes assumptions that need to be changed to provide scalable solutions to these problems; we conclude with some systems techniques to circumvent these lower bounds.

We provide some evidence that unique k-SAT is as hard to solve as general k-SAT, where k-SAT denotes the satisfiability problem for k-CNFs and unique k-SAT is the promise version where the given formula has 0 or 1 solutions. Namely, defining for each k/spl ges/1, s/sub k/=inf{/spl delta//spl ges/0|/spl exist/aO(2/sup /spl delta/n/)-time randomized algorithm for k-SAT} and, similarly, /spl sigma//sub k/=inf{/spl delta//spl ges/0|/spl exist/aO(2/sup /spl delta/n/)-time randomized algorithm for Unique k-SAT}, we show that lim/sub k/spl rarr//spl infin//s/sub k/=lim/sub k/spl rarr//spl infin///spl sigma//sub k/. As a corollary, we prove that, if Unique 3-SAT can be solved in time 2/sup /spl epsi/n/ for every /spl epsi/>0, then so can k-SAT for k/spl ges/3. Our main technical result is an isolation lemma for k-CNFs, which shows that a given satisfiable k-CNF can be efficiently probabilistically reduced to a uniquely satisfiable k-CNF, with nontrivial, albeit exponentially small, success probability.

The problem of k-SAT is to determine if the given k-CNF has a satisfying solution. It is a celebrated open question as to whether it requires exponential time to solve k-SAT for k/spl ges/3. Define s/sub k/ (for k/spl ges/3) to be the infimum of {/spl delta/: there exists an O(2/sup /spl delta/n/) algorithm for solving k-SAT}. Define ETH (Exponential-Time Hypothesis) for k-SAT as follows: for k/spl ges/3, s/sub k/>0. In other words, for k/spl ges/3, k-SA does not have a subexponential-time algorithm. In this paper we show that s/sub k/ is an increasing sequence assuming ETH for k-SAT: Let s/sub /spl infin// be the limit of s/sub k/. We in fact show that s/sub k//spl les/(1-d/k) s/sub /spl infin// for some constant d>0.

For several NP-complete problems, there have been a progression of better but still exponential algorithms. In this paper, we address the relative likelihood of sub-exponential algorithms for these problems. We introduce a generalized reduction which we call Sub-Exponential Reduction Family (SERF) that preserves sub-exponential complexity. We show that CircuitSAT is SERF-complete for all NP-search problems, and that for any fixed k, k-SAT, k-Colorability, k-Set Cover, Independent Set, Clique, Vertex Cover, are SERF--complete for the class SNP of search problems expressible by second order existential formulas whose first order part is universal. In particular, sub-exponential complexity for any one of the above problems implies the same for all others. We also look at the issue of proving strongly exponential lower bounds for AC 0 ; that is, bounds of the form 2 \\Omega\\Gamma n) . This problem is even open for depth-3 circuits. In fact, such a bound for depth-3 circuits with even l...

We propose and analyze a simple new algorithm for finding satisfying assignments of Boolean formulae in conjunctive normal form. The algorithm, ResolveSat, is a randomized variant of the DDL procedure by M. Davis et al. (1962) or Davis-Putnam procedure. Rather than applying the DLL procedure to the input formula F, however; ResolveSat enlarges F by adding additional clauses using limited resolution before performing DLL. The basic idea behind our analysis is the same as by R. Paturi (1997): a critical clause for a variable at a satisfying assignment gives rise to a unit clause in the DLL procedure with sufficiently high probability, thus increasing the probability of finding a satisfying assignment. In the current paper, we analyze the effect of multiple critical clauses (obtained through resolution) in producing unit clauses. We show that, for each k, the running time of ResolveSat on a k-CNF formula is significantly better than 2/sup n/, even in the worst case. In particular we show that the algorithm finds a satisfying assignment of a general 3-CNF in time O(2/sup .446n/) with high probability; where the best previous algorithm has running time O(2/sup .582n/). We obtain a better upper bound of O(2/sup (2ln2-1)/n+0(n))=O(2/sup 0.387n/) for 3-CNF that have at most one satisfying assignment (unique k-SAT). For each k, the bounds for general k-CNF are the best known for the worst-case complexity of finding a satisfying solution for k-SAT, the idea of succinctly encoding satisfying solutions can be applied to obtain lower bounds on circuit site. Here, we exhibit a function f such that any depth-3 AND-OR circuit with bottom fan-in bounded by k requires /spl Omega/(2(c/sub k/n/k)) gates (with c/sub k/>1). This is the first such lower bound with c/sub k/>1.

A projection is a subset of {0,1}n given by equations of theform xi = xj, xi = \bar{x}j, xi = 0, and xi = 1, where for $1\leq i \leq n$, xi are Boolean variables and $\bar{x} i are their complements. We study monochromatic projections in 2-colorings of an n-dimensional Boolean cube. We also study the dimension of the largest projection contained in a set specified by its density. We prove almost matching lower and upper bounds on the density of a set required to guarantee the existence of a d-dimensional projection. We also prove almost tight upper and lower bounds on the dimension of monochromatic projections in arbitrary Boolean functions. We then prove almost tight upper and lower bounds on the dimension of monochromatic projections in Boolean functions represented by low degree GF(2) polynomials. It follows from these lower bounds that low-degree GF(2) polynomials can define Boolean functions which are close to being extremal with respect to the property of having no large dimensional monochromatic projections.

We present and analyze two simple algorithms for finding satisfying assignments of /spl kappa/-CNFs (Boolean formulae in conjunctive normal form with at most /spl kappa/ literals per clause). The first is a randomized algorithm which, with probability approaching 1, finds a satisfying assignment of a satisfiable /spl kappa/-CNF formula F in time O(n/sup 2/|F|2/sup n-n//spl kappa//). The second algorithm is deterministic, and its running time approaches 2/sup n-n/2/spl kappa// for large n and /spl kappa/. The randomized algorithm is the best known algorithm for /spl kappa/>3; the deterministic algorithm is the best known deterministic algorithm for /spl kappa/>4. We also show an /spl Omega/(n/sup 1/4/2/sup /spl radic/n/) lower bound on the size of depth 3 circuits of AND and OR gates computing the parity function. This bound is tight up to a constant factor. The key idea used in these upper and lower bounds is what we call the Satisfiability Coding Lemma. This basic lemma shows how to encode satisfying solutions of a /spl kappa/-CNF succinctly.

ExponentialLower Bounds for Depth 3

The following size--depth tradeoff for threshold circuits is obtained: any threshold circuit of depth d that computes the parity function on n variables must have at least $n^{1 + c\theta^{-d }}$ edges, where $c>0$ and $\theta \leq 3$ are constants independent of n and d. Previously known constructions show that up to the choice of c and $\theta$ this bound is best possible. In particular, the lower bound implies an affirmative answer to the conjecture of Paturi and Saks that a bounded-depth threshold circuit that computes parity requires a superlinear number of edges. This is the first superlinear lower bound for an explicit function that holds for any fixed depth and the first that applies to threshold circuits with unrestricted weights. The tradeoff is obtained as a consequence of a general restriction theorem for threshold circuits with a small number of edges: For any threshold circuit with n inputs, depth d, and at most $kn$ edges, there exists a partial assignmentto the inputs that fixes the output of the circuit to a constant while leaving $\lfloor n/(c_1k)^{c_2\theta^{d}} \rfloor$ variables unfixed, where $c_1,c_2 > 0$ and $ \theta \leq 3$ are constants independent of n, k, and d. A tradeoff between the number of gates and depth is also proved: any threshold circuit of depth d that computes the parity of n variables has at least $(n/2)^{1/2(d-1)}$ gates. This tradeoff, which is essentially the best possible, was proved previously (with a better constant in the exponent) for the case of threshold circuits with polynomially bounded weights in [K. Siu, V. Roychowdury, and T. Kailath, IEEE Trans. Inform. Theory, 40 (1994), pp. 455--466]; the result in the present paper holds for unrestricted weights.

In high-performance chip design, the problem of net matching is often critical for achieving correct circuit performance. We adopt a conservative design, to route all matched nets with identical topologies and equal wire lengths to achieve zero skew. The problem is formulated as a variant of the D-dimensional Steiner tree problem. We propose a two-stage solution. The first stage uses an iterative improvement strategy to generate the Steiner tree topology for all the nets. The second stage places the nodes using one of two methods. The first approach expresses the optimal Steiner node positions as a linear programming solution, with average computational complexity O(n/sup 2/m/sup 2/), where n is the number of nets and m is the number of pins. Improved efficiency is achieved under the other approach by transforming the Manhattan metric to an l/sub /spl infin// norm using a 45/spl deg/ rotation of the solution space. The norm is then approximated by either an l/sub /spl lambda// norm, for suitably large values of /spl lambda/, or an exponential "penalty" function. The solution space in both approaches becomes strictly convex, allowing us to apply a greedy approach which converges to an optimal solution with great efficiency, leading to a dramatic speed-up versus the linear programming approach.

In this paper, we consider the problem of correlational learning and present algorithms to determine correlated objects.

Article Size-depth trade-offs for threshold circuits Share on Authors: Russell Impagliazzo View Profile , Ramamohan Paturi View Profile , Michael E. Saks View Profile Authors Info & Claims STOC '93: Proceedings of the twenty-fifth annual ACM symposium on Theory of ComputingJune 1993 Pages 541–550https://doi.org/10.1145/167088.167233Online:01 June 1993Publication History 3citation293DownloadsMetricsTotal Citations3Total Downloads293Last 12 Months6Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

In this paper, we provide matching (up to a constant factor) upper and lower bounds on the degree of polynomials that represent symmetric boolean functions with an error 1/3. Let Γ(f)=min{|2k–n+1|:fk ≠ fk+ 1 and 0 ≤ k ≤ n – 1} where fi is the value of f on inputs with exactly i 1's. We prove that the minimum degree over all the approximating polynomials of f is Θ((n(n-Γ(f))).5). We apply the techniques and tools from approximation theory to derive this result.

Motivated by, the problem of understanding the limitations of neural networks for representing Boolean functions, the authors consider size-depth tradeoffs for threshold circuits that compute the parity function. They give an almost optimal lower bound on the number of edges of any depth-2 threshold circuit that computes the parity function with polynomially bounded weights. The main technique used in the proof, which is based on the theory of rational approximation, appears to be a potentially useful technique for the analysis of such networks. It is conjectured that there are no linear size, bounded-depth threshold circuits for computing parity.>

The authors investigate how good connectivity properties translate into good error-correcting behavior in sparse networks of threshold elements. They determine how the eigenvalues of the interconnection graph (which in turn reflect connectivity properties) relate to the quantities, number of items stored, amount of error-correction, radius of attraction, and rate of convergence in an associative memory model consisting of a sparse network of threshold elements or neurons.>

We study (unbounded error) probabilistic communication complexity. Our new results include -one way and two complexities differ by at most 1 - certain functions like equality and the verification of Hamming distance have upper bounds that are considerably better than their counterparts in deterministic, nondeterministic, or bounded error probabilistic model - there exists a function which requires /spl Omega/(logn) information transfer. As an application, we prove that a certain language requires /spl Omega/(nlogn) time to be recognized by a 1-tape (unbounded error) probabilistic Turing machine. This bound is optimal. (Previous lower bound results [Yao 1] require acceptance by bounded error computation. We believe that this is the first nontrivial lower bound on the time required by unrestricted probabilistic Turing machines.

We study probabilistic on-line simulators for several machine models (or memory structures). The simulators have a more constrained access to data than the virtual machines, but are allowed to use probabilistic means to improve average access time. We show that in many cases coin tosses can not make up for inadequate access.