We introduce a new linear-algebraic tool based on group representation theory, and use it to address three key problems in machine learning. 1. Past researchers have proposed fast attention algorithms for LLMs by approximating or replace softmax attention with other functions, such as low-degree polynomials. The key property of these functions is that, when applied entry-wise to the matrix $QK^{\top}$, the result is a low rank matrix when $Q$ and $K$ are $n \times d$ matrices and $n \gg d$. This suggests a natural question: what are all functions $f$ with this property? If other $f$ exist and are quickly computable, they can be used in place of softmax for fast subquadratic attention algorithms. It was previously known that low-degree polynomials have this property. We prove that low-degree polynomials are the only piecewise continuous functions with this property. This suggests that the low-rank fast attention only works for functions approximable by polynomials. Our work gives a converse to the polynomial method in algorithm design. 2. We prove the first full classification of all positive definite kernels that are functions of Manhattan or $\ell_1$ distance. Our work generalizes an existing theorem at the heart of all kernel methods in machine learning: the classification of all positive definite kernels that are functions of Euclidean distance. 3. The key problem in metric transforms, a mathematical theory used in geometry and machine learning, asks what functions transform pairwise distances in semi-metric space $M$ to semi-metric space $N$ for specified $M$ and $N$. We provide the first full classification of functions that transform Manhattan distances to Manhattan distances. Our work generalizes the foundational work of Schoenberg, which fully classifies functions that transform Euclidean to Euclidean distances. We additionally prove results about stable-rank preserving functions that are potentially useful in algorithmic design, and more. Our core new tool is called the representation theory of the hyperrectangle.

Data-sensitive metrics adapt distances locally based the density of data points with the goal of aligning distances and some notion of similarity. In this paper, we give the first exact algorithm for computing a data-sensitive metric called the nearest neighbor metric. In fact, we prove the surprising result that a previously published 3-approximation is an exact algorithm. The nearest neighbor metric can be viewed as a special case of a density-based distance used in machine learning, or it can be seen as an example of a manifold metric. Previous computational research on such metrics despaired of computing exact distances on account of the apparent difficulty of minimizing over all continuous paths between a pair of points. We leverage the exact computation of the nearest neighbor metric to compute sparse spanners and persistent homology. We also explore the behavior of the metric built from point sets drawn from an underlying distribution and consider the more general case of inputs that are finite collections of path-connected compact sets. The main results connect several classical theories such as the conformal change of Riemannian metrics, the theory of positive definite functions of Schoenberg, and screw function theory of Schoenberg and Von Neumann. We also develop some novel proof techniques based on the combination of screw functions and Lipschitz extensions that may be of independent interest.

We present two graph quantities Psi(G,S) and Psi_2(G) which give constant factor estimates to the Dirichlet and Neumann eigenvalues, lambda(G,S) and lambda_2(G), respectively. Our techniques make use of a discrete Hardy-type inequality due to Muckenhoupt.

Motivated by the study of matrix elimination orderings in combinatorial scientific computing, we utilize graph sketching and local sampling to give a data structure that provides access to approximate fill degrees of a matrix undergoing elimination in polylogarithmic time per elimination and query. We then study the problem of using this data structure in the minimum degree algorithm, which is a widely-used heuristic for producing elimination orderings for sparse matrices by repeatedly eliminating the vertex with (approximate) minimum fill degree. This leads to a nearly-linear time algorithm for generating approximate greedy minimum degree orderings. Despite extensive studies of algorithms for elimination orderings in combinatorial scientific computing, our result is the first rigorous incorporation of randomized tools in this setting, as well as the first nearly-linear time algorithm for producing elimination orderings with provable approximation guarantees. While our sketching data structure readily works in the oblivious adversary model, by repeatedly querying and greedily updating itself, it enters the adaptive adversarial model where the underlying sketches become prone to failure due to dependency issues with their internal randomness. We show how to use an additional sampling procedure to circumvent this problem and to create an independent access sequence. Our technique for decorrelating interleaved queries and updates to this randomized data structure may be of independent interest.

We present a simple spectral approach to the well-studied constrained clustering problem. It captures constrained clustering as a generalized eigenvalue problem with graph Laplacians. The algorithm works in nearly-linear time and provides concrete guarantees for the quality of the clusters, at least for the case of 2-way partitioning. In practice this translates to a very fast implementation that consistently outperforms existing spectral approaches both in speed and quality.

In this paper we provide faster algorithms for solving the geometric median problem: given n points in d compute a point that minimizes the sum of Euclidean distances to the points. This is one of the oldest non-trivial problems in computational geometry yet despite a long history of research the previous fastest running times for computing a (1+є)-approximate geometric median were O(d· n4/3є−8/3) by Chin et. al, Õ(dexpє−4logє−1) by Badoiu et. al, O(nd+poly(d,є−1)) by Feldman and Langberg, and the polynomial running time of O((nd)O(1)log1/є) by Parrilo and Sturmfels and Xue and Ye.

We introduce the notion of balance for directed graphs: a weighted directed graph is α-balanced if for every cut S ⊆ V, the total weight of edges going from S to V∖ S is within factor α of the total weight of edges going from V∖ S to S. Several important families of graphs are nearly balanced, in particular, Eulerian graphs (with α = 1) and residual graphs of (1+є)-approximate undirected maximum flows (with α=O(1/є)).

Over the last several years there have been major breakthroughs in the design of approximation algorithms for such classic problems as finding the maximum flow in a graph. Maximum flow for undirected graphs can now be approximately solved in almost linear time. This result by researchers at Berkeley and MIT, I claim, is only the beginning of a new era in efficient algorithm design.

We use exponential start time clustering to design faster parallel graph algorithms involving distances. Previous algorithms usually rely on graph decomposition routines with strict restrictions on the diameters of the decomposed pieces. We weaken these bounds in favor of stronger local probabilistic guarantees. This allows more direct analyses of the overall process, giving:

We present an efficient algorithm for solving a linear system arising from the 1-Laplacian corresponding to a collapsible simplicial complex with a known collapsing sequence. When combined with a result of Chillingworth, our algorithm is applicable to convex simplicial complexes embedded in ℝ3. The running time of our algorithm is nearly-linear in the size of the complex and is logarithmic on its numerical properties. Our algorithm is based on projection operators and combinatorial steps for transferring between them. The former relies on decomposing flows into circulations and potential flows using fast solvers for graph Laplacians, and the latter relates Gaussian elimination to topological properties of simplicial complexes.

We show an algorithm for solving symmetric diagonally dominant (SDD) linear systems with m non-zero entries to a relative error of ε in O(m log1/2 n logc n log(1/ε)) time. Our approach follows the recursive preconditioning framework, which aims to reduce graphs to trees using iterative methods. We improve two key components of this framework: random sampling and tree embeddings. Both of these components are used in a variety of other algorithms, and our approach also extends to the dual problem of computing electrical flows.

We present an algorithm that on input of an $n$-vertex $m$-edge weighted graph $G$ and a value $k$ produces an incremental sparsifier $\hat{G}$ with $n-1 + m/k$ edges, such that the relative condition number of $G$ with $\hat{G}$ is bounded above by $\tilde{O}(k\log^2 n)$, with probability $1-p$ (we use the $\tilde{O}()$ notation to hide a factor of at most $(\log\log n)^4$). The algorithm runs in time $\tilde{O}((m \log{n} + n\log^2{n})\log(1/p)).$ As a result, we obtain an algorithm that on input of an $n\times n$ symmetric diagonally dominant matrix $A$ with $m$ nonzero entries and a vector $b$ computes a vector ${x}$ satisfying $||{x}-A^{+}b||_A<\epsilon ||A^{+}b||_A $, in expected time $\tilde{O}(m\log^2{n}\log(1/\epsilon)).$ The solver is based on repeated applications of the incremental sparsifier that produces a chain of graphs which is then used as input to the recursive preconditioned Chebyshev iteration.

We describe a new algorithm for computing the Voronoi diagram of a set of n points in constant-dimensional Euclidean space. The running time of our algorithm is O(f log n log Δ) where f is the output complexity of the Voronoi diagram and Δ is the spread of the input, the ratio of largest to smallest pairwise distances. Despite the simplicity of the algorithm and its analysis, it improves on the state of the art for all inputs with polynomial spread and near-linear output size. The key idea is to first build the Voronoi diagram of a superset of the input points using ideas from Voronoi refinement mesh generation. Then, the extra points are removed in a straightforward way that allows the total work to be bounded in terms of the output complexity, yielding the output sensitive bound. The removal only involves local flips and is inspired by kinetic data structures.

We present a new algorithm that produces a well-spaced superset of points conforming to a given input set in any dimension with guaranteed optimal output size. We also provide an approximate Delaunay graph on the output points. Our algorithm runs in expected time O(2O(d)(n log n + m)), where n is the input size, m is the output point set size, and d is the ambient dimension. The constants only depend on the desired element quality bounds.

There has been significant interest and progress recently in algorithms that solve regression problems involving tall and thin matrices in input sparsity time. Given a n * d matrix where n ≥ d, these algorithms find an approximation with fewer rows, allowing one to solve a poly(d) sized problem instead. In practice, the best performances are often obtained by invoking these routines in an iterative fashion. We show these iterative methods can be adapted to give theoretical guarantees comparable to and better than the current state of the art. Our approaches are based on computing the importances of the rows, known as leverage scores, in an iterative manner. We show that alternating between computing a short matrix estimate and finding more accurate approximate leverage scores leads to a series of geometrically smaller instances. This gives an algorithm whose runtime is input sparsity plus an overhead comparable to the cost of solving a regression problem on the smaller approximation. Our results build upon the close connection between randomized matrix algorithms, iterative methods, and graph sparsification.

We study theoretical runtime guarantees for a class of optimization problems that occur in a wide variety of inference problems. These problems are motivated by the LASSO framework and have applications in machine learning and computer vision. Our work shows a close connection between these problems and core questions in algorithmic graph theory. While this connection demonstrates the difficulties of obtaining runtime guarantees, it also suggests an approach of using techniques originally developed for graph algorithms.

We present faster algorithms for approximate maximum flow in undirected graphs with good separator structures, such as bounded genus, minor free, and geometric graphs. Given such a graph with n vertices, m edges along with a recursive -vertex separator structure, our algorithm finds an 1 − ∊ approximate maximum flow in time Õ(m6/5poly(∊−1)), ignoring poly-logarithmic terms. Similar speedups are also achieved for separable graphs with larger size separators albeit with larger run times. These bounds also apply to image problems in two and three dimensions. Key to our algorithm is an intermediate problem that we term grouped L2 flow, which exists between maximum flows and electrical flows. Our algorithm also makes use of spectral vertex sparsifiers in order to remove vertices while preserving the energy dissipation of electrical flows. We also give faster spectral vertex sparsification algorithms on well separated graphs, which may be of independent interest.

We show an improved parallel algorithm for decomposing an undirected unweighted graph into small diameter pieces with a small fraction of the edges in between. These decompositions form critical subroutines in a number of graph algorithms. Our algorithm builds upon the shifted shortest path approach introduced in [Blelloch, Gupta, Koutis, Miller, Peng, Tangwongsan, SPAA 2011]. By combining various stages of the previous algorithm, we obtain a significantly simpler algorithm with the same asymptotic guarantees as the best sequential algorithm.

Spectral Graph Theory is the interplay between linear algebra and combinatorial graph theory. One application of this interplay is a nearly linear time solver for Symmetric Diagonally Dominate systems (SDD). This seemingly restrictive class of systems has received much interest in the last 15 years. Both algorithm design theory and practical implementations have made substantial progress. There is also a growing number of problems that can be efficiently solved using SDD solvers including: image segmentation, image denoising, finding solutions to elliptic equations, computing maximum flow in a graph, graph sparsification, and graphics. All these examples can be viewed as special case of convex optimization problems.

The maximum multicommodity flow problem is a natural generalization of the maximum flow problem to route multiple distinct flows. Obtaining a 1-ε approximation to the multicommodity flow problem on graphs is a well-studied problem. In this paper we present an adaptation of recent advances in single-commodity flow algorithms to this problem. As the underlying linear systems in the electrical problems of multicommodity flow problems are no longer Laplacians, our approach is tailored to generate specialized systems which can be preconditioned and solved efficiently using Laplacians. Given an undirected graph with m edges and k commodities, we give algorithms that find 1-ε approximate solutions to the maximum concurrent flow problem and maximum weighted multicommodity flow problem in time O(m4/3poly(k,ε-1)).

We present NetMesh, a new algorithm that produces a conforming Delaunay mesh for point sets in any fixed dimension with guaranteed optimal mesh size and quality. Our comparison-based algorithm runs in O(n log n + m) time, where n is the input size and m is the output size, and with constants depending only on the dimension and the desired element quality. It can terminate early in O(n log n) time returning a O(n) size Voronoi diagram of a superset of P, which again matches the known lower bounds.

We present an improved algorithm for solving symmetrically diagonally dominant linear systems. On input of an n×n symmetric diagonally dominant matrix A with m non-zero entries and a vector b such that Ax̅ = b for some (unknown) vector x̅, our algorithm computes a vector x such that ∥x-x̅∥A≤ϵ∥x̅∥A1in time Õ (m log n log (1/ϵ))2. The solver utilizes in a standard way a 'preconditioning' chain of progressively sparser graphs. To claim the faster running time we make a two-fold improvement in the algorithm for constructing the chain. The new chain exploits previously unknown properties of the graph sparsification algorithm given in [Koutis,Miller,Peng, FOCS 2010], allowing for stronger preconditioning properties.We also present an algorithm of independent interest that constructs nearly-tight low-stretch spanning trees in time Õ (m log n), a factor of O (log n) faster than the algorithm in [Abraham,Bartal,Neiman, FOCS 2008]. This speedup directly reflects on the construction time of the preconditioning chain.

We consider the problem of approximating a signal P with another signal F consisting of a few piecewise constant segments. This problem arises naturally in applications including databases (e.g., histogram construction), speech recognition, computational biology (e.g., denoising aCGH data) and many more. Specifically, let P = (P1, P2, …, Pn), Pi ∊ ℝ for all i, be a signal and let C be a constant. Our goal is to find a function F : [n] → ℝ which optimizes the following objective function: The above optimization problem reduces to solving the following recurrence, which can be done using dynamic programming in O(n2) time: This recurrence arises naturally in several applications where one wants to approximate a given signal P with a signal F which ideally consists of few piecewise constant segments. Such applications include histogram construction in databases, determining DNA copy numbers in cancer cells from micro-array data, speech recognition, data mining and many others. In this work we present two new techniques for optimizing dynamic programming that can handle cost functions not treated by other standard methods. The basis of our first algorithm is the definition of a constant-shifted variant of the objective function that can be efficiently approximated using state of the art methods for range searching. Our technique approximates the optimal value of our objective function within additive ∊ error and runs in time, where δ is an arbitrarily small positive constant and . The second algorithm we provide solves a similar recurrence that's within a multiplicative factor of (1+∊) and runs in O(n log n/∊). The new technique introduced by our algorithm is the decomposition of the initial problem into a small (logarithmic) number of Monge optimization subproblems which we can speed up using existing techniques.

This paper presents the design and analysis of a near linear-work parallel algorithm for solving symmetric diagonally dominant (SDD) linear systems. On input an SDD n-by-n matrix A with m non-zero entries and a vector b, our algorithm computes a vector x such that Ax - A+b ≤ ε • A+b in O(m logO(1) n log 1/ε) work and O(m1/3+θ log 1/ε) depth for any fixed θ > 0.

We apply ideas from mesh generation to improve the time and space complexities of computing the full persistent homological information associated with a point cloud P in Euclidean space ℜd. Classical approaches rely on the Cech, Rips, ±-complex, or witness complex filtrations of P, whose complexities scale up very badly with d. For instance, the ±-complex filtration incurs the n Ω(d) size of the Delaunay triangulation, where n is the size of P. The common alternative is to truncate the filtrations when the sizes of the complexes become prohibitive, possibly before discovering the most relevant topological features. In this paper we propose a new collection of filtrations, based on the Delaunay triangulation of a carefully-chosen superset of P, whose sizes are reduced to 2O(d2)n. Our filtrations interleave multiplicatively with the family of offsets of P, so that the persistence diagram of P can be approximated in 2O(d2)n3 time in theory, with a near-linear observed running time in practice. Thus, our approach remains tractable in medium dimensions, say 4 to 10.

We present an algorithm that on input of an n-vertex m-edge weighted graph G and a value k, produces an incremental sparsifier G with n-1+m/k edges, such that the condition number of G with G is bounded above by Õ(k log2n), with probability 1-p. The algorithm runs in time Õ((m log n + n log n) log(1/p)). As a result, we obtain an algorithm that on input of an n × n symmetric diagonally dominant matrix A with m non-zero entries and a vector b, computes a vector x satisfying ||x-A+b||A+b||A, in expected time Õ(m log2n log(1/ϵ)). The solver is based on repeated applications of the incremental sparsifier that produces a chain of graphs which is then used as input to a recursive preconditioned Chebyshev iteration.

Memory bandwidth is a major limiting factor in the scalability of parallel iterative algorithms that rely on sparse matrix-vector multiplication (SpMV). This paper introduces Hierarchical Diagonal Blocking (HDB), an approach which we believe captures many of the existing optimization techniques for SpMV in a common representation. Using this representation in conjuction with precision-reduction techniques, we develop and evaluate high-performance SpMV kernels. We also study the implications of using our SpMV kernels in a complete iterative solver. Our method of choice is a Combinatorial Multigrid solver that can fully utilize our fastest reduced-precision SpMV kernel without sacrificing the quality of the solution. We provide extensive empirical evaluation of the effectiveness of the approach on a variety of benchmark matrices, demonstrating substantial speedups on all matrices considered.

We present the Iterated-Tverberg algorithm, the first deterministic algorithm for computing an approximate centerpoint of a set S ∈ Rd with running time sub-exponential in d. The algorithm is a derandomization of the Iterated-Radon algorithm of Clarkson et al and is guaranteed to terminate with an O(1/d2)-center. Moreover, it returns a polynomial-time checkable proof of the approximation guarantee, despite the coNP-Completenes of testing centerpoints in general. We also explore the use of higher order Tverberg partitions to improve the runtime of the deterministic algorithm and improve the approximation guarantee for the randomized algorithm. In particular, we show how to improve the O(1/d2)-center of the Iterated-Radon algorithm to O(1/dr/(r-1)) for a cost of O((rd)d) in time for any integer r.

Counting the number of triangles in a graph is a beautiful algorithmic problem which has gained importance over the last years due to its significant role in complex network analysis. Metrics frequently computed such as the clustering coefficient and the transitivity ratio involve the execution of a triangle counting algorithm. Furthermore, several interesting graph mining applications rely on computing the number of triangles in the graph of interest.

Typical volume meshes in three dimensions are designed to conform to an underlying two-dimensional surface mesh, with volume mesh element size growing larger away from the surface. The surface mesh may be uniformly spaced or highly graded, and may have fine resolution due to extrinsic mesh size concerns. When we desire that such a volume mesh have good aspect ratio, we require that some space-filling scaffold vertices be inserted off the surface. We analyze the number of scaffold vertices in a setting that encompasses many existing volume meshing algorithms. We show that under simple preconditions, the number of scaffold vertices will be linear in the number of surface vertices.

We consider the problem of decomposing a weighted graph with n vertices into a collection P of vertex disjoint clusters such that, for all clusters C ε P, the graph induced by the vertices in C and the edges leaving C, has conductance bounded below by φ. We show that for planar graphs we can compute a decomposition P such that |P| < n/ρ, where ρ is a constant, in O(log n) parallel time with O(n) work. Slightly worse guarantees can be obtained in nearly linear time for graphs that have fixed size minors or bounded genus. We show how these decompositions can be used in the first known linear work parallel construction of provably good preconditioners for the important class of fixed degree graph Laplacians. On a more theoretical note, we present upper bounds on the Euclidean distance of eigenvectors of the normalized Laplacian from the space of vectors which consists of the cluster-wise constant vectors scaled by the square roots of the total incident weights of the vertices.

The authors recently introduced the technique of sparse mesh refinement to produce the first near-optimal sequential time bounds of O(n lg L/s+m) for inputs in any fixed dimension with piecewiselinear constraining (PLC) features. This paper extends that work to the parallel case, refining the same inputs in time O(lg(L/s) lgm) on an EREW PRAM while maintaining the work bound; in practice, this means we expect linear speedup for any practical number of processors. This is faster than the best previously known parallel Delaunay mesh refinement algorithms in two dimensions. It is the first technique with work bounds equal to the sequential case. In higher dimension, it is the first provably fast parallel technique for any kind of quality mesh refinement with PLC inputs. Furthermore, the algorithm's implementation is straightforward enough that it is likely to be extremely fast in practice.

We introduce a family of spectral partitioning methods. Edge separators of a graph are produced by iteratively reweighting the edges until the graph disconnects into the prescribed number of components. At each iteration a small number of eigenvectors with small eigenvalue are computed and used to determine the reweighting. In this way spectral rounding directly produces discrete solutions where as current spectral algorithms must map the continuous eigenvectors to discrete solutions by employing a heuristic geometric separator (e.g. k-means). We show that spectral rounding compares favorably to current spectral approximations on the Normalized Cut criterion (NCut). Results are given for natural image segmentation, medical image segmentation, and clustering. A practical version is shown to converge.

We optimize over the set of corrected Laplacians (CL) associated with a weighted graph to improve the average case normalized cut (NCut) of a graph. Unlike edge-relaxation SDPs, optimizing over the set CL naturally exploits the matrix sparsity by operating solely on the diagonal. This structure is critical to image segmentation applications because the number of vertices is generally proportional to the number of pixels in the image. CL optimization provides a guiding principle for improving the combinatorial solution over the spectral relaxation, which is important because small improvements in the cut cost often result in significant improvements in the perceptual relevance of the segmentation. We develop an optimization procedure to accommodate prior information in the form of statistical shape models, resulting in a segmentation method that produces foreground regions which are consistent with a parameterized family of shapes. We validate our technique with ground truth on MRI medical images, providing a quantitative comparison against results produced by current spectral relaxation approaches to graph partitioning.

In 1995, Gremban, Miller, and Zagha introduced support-tree preconditioners and a parallel algorithm called support-tree conjugate gradient (STCG) for solving linear systems of the form Ax = b, where A is an n × n Laplacian matrix. A Laplacian is a symmetric matrix in which the off-diagonal entries are non-positive, and the row and column sums are zero. A Laplacian A with 2m non-zeros can be interpreted as an undirected positively-weighted graph G with n vertices and m edges, where there is an edge between two nodes i and j with weight c((i, j)) = −Ai,j = −Aj,i if Ai,j = Aj,i < 0. Gremban et al. showed experimentally that STCG performs well on several classes of graphs commonly used in scientific computations. In his thesis, Gremban also proved upper bounds on the number of iterations required for STCG to converge for certain classes of graphs. In this paper, we present an algorithm for finding a preconditioner for an arbitrary graph G = (V, E) with n nodes, m edges, and a weight function c> 0 on the edges, where w.l.o.g., mine∈E c(e) = 1. Equipped with this preconditioner, STCG requires O(log 4 n · � ∆/α) iterations, where α = min U⊂V,|U|≤|V |/2 c(U, V \\U)/|U | is the minimum edge expansion of the graph, and ∆ = maxv∈V c(v) is the maximum incident weight on any vertex. Each iteration requires O(m) work and can be implemented in O(log n) steps in parallel, using only O(m) space. Our results generalize to matrices that are symmetric and diagonally-dominant (SDD). 1

We present a new framework for maintaining the quality of two dimensional triangular moving meshes. The use of curved elements is the key idea that allows us to avoid excessive refinement and still obtain good quality meshes consisting of a low number of well shaped elements. We use B-splines curves to model object boundaries, and objects are meshed with second order Bézier triangles. As the mesh evolves according to a non-uniform flow velocity field, we keep track of object boundaries and, if needed, carefully modify the mesh to keep it well shaped by applying a combination of vertex insertion and deletion, edge flipping, and edge smoothing operations at each time step. Our algorithms for these tasks are extensions of known algorithms for meshes built of straight--sided elements and are designed for any fixed-order Bézier elements and B-splines. Although in this work we have concentrated on quadratic elements, most of the operations are valid for elements of any order and they generalize well to higher dimensions. We present results of our scheme for a set of objects mimicking red blood cells subject to a precomputed flow velocity field.

Given a general graph G, a fundamental problem is to find a spanning tree H that best approximates G by some measure. Often this measure is some combination of the congestion and dilation of an embedding of G into H. One example is the routing time ρ(G, H) ≤ O(congestion + dilation), the number of steps necessary to route pairwise demands G on network links H in the store-and-forward packet routing model. Another is the condition number κf(G, H) ≤ O(congestion·dilation), the square root of which bounds the number of iterations necessary to solve a linear system with coefficient matrix G preconditioned by H using the classical conjugate gradient method. The algorithmic applications of being able to find (efficiently) a good tree approximation H for a graph G are numerous; but what if no good tree exists? In this paper, we seek to identify the class of graphs G which are intrinsically difficult to approximate by a particular measure. It is easily seen that with respect to routing time, G is hardest to approximate by a tree H precisely when it contains either long cycles (which yield high dilation) or large separators (which yield high congestion). We show that with respect to condition number, the existence of long cycles or large separators in G is sufficient but not necessary for it to be hardest to approximate, by demonstrating a nearly-linear lower bound for the case in which G is a square mesh. The proof uses concepts from circuit theory, linear algebra, and geometry, and it generalizes to the case in which H is a spanning subgraph of G of Euler characteristic k. The result has consequences for the design of preconditioners for symmetric M-matrices and perhaps also of communication networks.

Triangulations of a surface are of fundamental importance in computational geometry, computer graphics, and engineering and scientific simulations. Triangulations are ordinarily represented as mutable graph structures for which both adding and traversing edges take constant time per operation. These representations of triangulations make it difficult to support persistence , including ‘multiple futures’, the ability to use a data structure in several unrelated ways in a given computation; ‘time travel’, the ability to move freely among versions of a data structure; or parallel computation, the ability to operate concurrently on a data structure without interference. We present a purely functional interface and representation of triangulated surfaces, and more generally of simplicial complexes in higher dimensions. In addition to being persistent in the strongest sense, the interface more closely matches the mathematical definition of triangulations (simplicial complexes) than do interfaces based on mutable representations. The representation, however, comes at the cost of requiring O (lg n ) time for traversing or adding triangles (simplices), where n is the number of triangles in the surface. We show both analytically and experimentally that for certain important cases, this extra cost does not seriously affect end-to-end running time. Analytically, we present a new randomized algorithm for 3-dimensional Convex Hull based on our representations for which the running time matches the Ω( n lg n ) lower-bound for the problem. This is achieved by using only O ( n ) traversals of the surface. Experimentally, we present results for both an implementation of the 3-dimensional Convex Hull and for a terrain modeling algorithm, which demonstrate that, although there is some cost to persistence, it seems to be a small constant factor.

Many important phenomena in science and engineering, including our motivating problem of microstructural blood flow, can be modeled as flows with dynamic interfaces. The major challenge faced in simulating such flows is resolving the interfacial motion. Lagrangian methods are ideally suited for such problems, since interfaces are naturally represented and propagated. However, the material description of motion results in dynamic meshes, which become hopelessly distorted unless they are regularly regenerated. Lagrangian methods are particularly challenging on parallel computers, because scalable dynamic mesh methods remain elusive. Here, we present a parallel dynamic mesh Lagrangian method for flows with dynamic interfaces. We take an aggressive approach to dynamic meshing by triangulating the propagating grid points at every timestep using a scalable parallel Delaunay algorithm. Contrary to conventional wisdom, we show that the costs of the geometric components (triangulation, coarsening, refinement, and partitioning) can be made small relative to the flow solver.

A sliver is a tetrahedron whose four vertices lie close to a plane and whose perpendicular projection to that plane is a convex quadrilateral with no short edge. Slivers are both undesirable and ubiquitous in 3-dimensional Delaunay triangulations. Even when the point-set is well-spaced, slivers may result. This paper shows that such a point set permits a small perturbation whose Delaunay triangulation contains no slivers. It also gives deterministic algorithms that compute the perturbation of n points in time O(n log n) with one processor and in time O(log n) with O(n) processors. Keywords. Mesh generation, computational geometry, tetrahedral meshes, Delaunay triangulations, slivers, mesh smoothing, mesh clean-up. 1. INTRODUCTION This paper presents a smoothing and clean-up algorithm for 3-dimensional Delaunay triangulations that removes all slivers. A necessary assumption of the algorithm is that the input triangles and tetrahedra have a bounded circumradius to shortest edge length...

In this paper we demonstrate that parallelism and fill can be traded off in orders for Gaussian elimination.While the well-known nested dissection algorithm produces very parallel elimination orders, we show that by reducing the parallelism it is possible to reduce the fill that the orders generate.In particular, we present a new "less parallel nested dissection" algorithm (LPND).We prove that, unlike standard nested dissection, when applied to a chordal graph LPND finds a zero-fill elimination order.Our implementation of LPND generates less fill than state-of-the-art implementations of the nested dissection (METIS), minimum-degree @MD), and hybrid (BEND) algorithms on a large body of test matrices, at the cost of a small reduction in the paralellism in the orders that it produces.We have also implemented a nested dissection algorithm that is different from METIS and that uses the same separator algorithm used by our implementation of LPND.This algorithm, like LPND, generates less fill than METIS, and on large graphs generates significantly less fill than AMD.The latter comparison is notable, because although it is known that, for certain classes of graphs, minimum-degree produces asymptotically more fill than nested dissection, minimumdegree is believed to produce low-fill orderings in practice.Our experiments contradict this belief.

We establish connections between parallel circuit evaluation and uniform algebraic closure properties of unary function classes. We use this connection in the development of time-efficient and processor-efficient parallel algorithms for the evaluation of algebraic circuits. Our algorithm provides a nontrivial upper bound on the parallel complexity of the circuit value problem over $\{{\Bbb R},\min,\max,+\}$ and $\{{\Bbb R}^{+},\min,\max,\times\}$. We partially answer an open question of Miller, Ramachandran, and Kaltofen by showing that circuits over a polynomial-bounded noncommutative semiring and circuits over infinite noncommutative semirings with a polynomial-bounded dimension over a commutative semiring can be evaluated in polylogarithmic time in their size and degree using a polynomial number of processors. We also present an improved parallel algorithm for Boolean circuits.

This paper presents an algorithm for finding parallel elimination orders for Gaussian elimination. Viewing a system of equations as a graph, the algorithm can be applied directly to interval graphs and chordal graphs. For general graphs, the algorithm can be used to parallelize the order produced by some other heuristic such as minimum degree. In this case, the algorithm is applied to the chordal completion that the heuristic generates from the input graph. In general, the input to the algorithm is a chordal graph G with n nodes and m edges. The algorithm produces an order with height at most O(log/sup 3/ n) times optimal, fill at most O(m), and work at most O(W*(G)), where W*(G) is the minimum possible work over all elimination orders for G. Experimental results show that when applied after some other heuristic, the increase in work and fill is usually small. In some instances the algorithm obtains an order that is actually better, in terms of work and fill, than the original one. We also present an algorithm that produces an order with a factor of log n less height, but with a factor of O(/spl radic/log n) more fill.

A collection of n balls in d dimensions forms a k -ply system if no point in the space is covered by more than k balls. We show that for every k -ply system Γ, there is a sphere S that intersects at most O ( k 1/ d n 1−1/ d ) balls of Γ and divides the remainder of Γ into two parts: those in the interior and those in the exterior of the sphere S , respectively, so that the larger part contains at most (1−1/( d +2)) n balls. This bound of ( O ( k 1/ d n 1−1/ d ) is the best possible in both n and k . We also present a simple randomized algorithm to find such a sphere in O(n) time. Our result implies that every k -nearest neighbor graphs of n points in d dimensions has a separator of size O ( k 1/ d n 1−1/ d ). In conjunction with a result of Koebe that every triangulated planar graph is isomorphic to the intersection graph of a disk-packing, our result not only gives a new geometric proof of the planar separator theorem of Lipton and Tarjan, but also generalizes it to higher dimensions. The separator algorithm can be used for point location and geometric divide and conquer in a fixed dimensional space.

In this paper we are concerned with developing a practical parallel algorithm for Delaunay triangulation that works well on general distributions, particularly those that arise in Scientific Computation. Although there have been many theoretical algorithms for the problem, and some implementations based on bucketing that work well for uniform distributions, there has been little work on implementations for general distributions. We use the well known reduction of 2D Delaunay triangulation to 3D convex hull of points on a sphere or paraboloid. A variant of the Edelsbrunner and Shi 3D convex hull is used, but for the special case when the point set lies on either a sphere or a paraboloid. Our variant greatly reduces the constant costs from the 3D convex hull algorithm and seems to be a more promising for a practical implementation than other parallel approaches. We have run experiments on the algorithm using a variety of distributions that are motivated by various problems that use Delau...

Article A Delaunay based numerical method for three dimensions: generation, formulation, and partition Share on Authors: Gary L. Miller School of Computer Science, Carnegie Mellon University, Pittsburgh, Pennsylvania School of Computer Science, Carnegie Mellon University, Pittsburgh, PennsylvaniaView Profile , Dafna Talmor School of Computer Science, Carnegie Mellon University, Pittsburgh, Pennsylvania School of Computer Science, Carnegie Mellon University, Pittsburgh, PennsylvaniaView Profile , Shang-Hua Teng Department of Computer Science, University of Minnesota, Minneapolis, Minnesota Department of Computer Science, University of Minnesota, Minneapolis, MinnesotaView Profile , Noel Walkington Department of Mathematics, Carnegie Mellon University, Pittsburgh, Pennsylvania Department of Mathematics, Carnegie Mellon University, Pittsburgh, PennsylvaniaView Profile Authors Info & Claims STOC '95: Proceedings of the twenty-seventh annual ACM symposium on Theory of computingMay 1995 Pages 683–692https://doi.org/10.1145/225058.225286Online:29 May 1995Publication History 62citation786DownloadsMetricsTotal Citations62Total Downloads786Last 12 Months7Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

We give a deterministic linear time algorithm for finding a “good” sphere separator of a k-ply neighborhood system Φ in any fixed dimension, where a k-ply neighborhood system in $\IR$ d is a collection of n balls such that no points in the space is c

The problem of maximum flow in planar graphs has always been investigated under the assumption that there is only one source and one sink. Here we consider the case where there are many sources and sinks (single commodity) in a directed planar graph. An algorithm for the case when the demands of the sources and sinks are fixed and given in advance is presented. The algorithm can be implemented efficiently sequentially and in parallel, and its complexity is dominated by the complexity of computing all shortest paths from a single source in a planar graph. If the demands are not known, an algorithm for computing the maximum flow is presented for the case where the number of faces that contain sources and sinks is bounded by a slowly growing function, Our result places the problem of computing a perfect matching in a planar bipartite graph in NC and improves a previous parallel algorithm for the case of a single source, single sink in a planar directed (and undirected) graph, both in terms of processor bounds and its simple presentation.

We describe a practical and provably good algorithm for approximating center points in any number of dimensions. Here c is a center point of a point set P in ℝd if every closed halfspace containing c contains at least |P|/(d+1) points of P. Our algorithm has a small constant factor and is the first approximate center point algorithm whose complexity is subexponential in d. Moreover, it can be optimally parallelized to require O(log2 d loglog n) time. Our algorithm has been used in mesh partitioning methods, and has the potential to improve results in practice for constructing weak ε-nets and other geometric algorithms. We derive a variant of our algorithm with a time bound fully polynomial in d, and show how to combine our approach with previous techniques to compute high quality center points more quickly.

We give a deterministic linear time algorithm for finding a small cost sphere separator of a k-ply neighborhood system Φ in any fixed dimension, where a k-ply neighborhood system in Rd is a collection of n balls such that no points in the space is covered by more than k balls. The sphere separator intersects at most O (k1/2 nd-1/d) balls of Φ and it divides the remaining of Φ into two parts: those in the interior and those in the exterior of the sphere, respectively, so that the larger part contains at most δn balls (d+1/d+2 < δ < 1). This result improves the O(n2) time deterministic algorithm of Miller and Teng [29] and answers a major algorithmic open question posed by Mille, Teng,Thurston and Vavasis [23,25].

Article Free Access Share on Separator based parallel divide and conquer in computational geometry Authors: Alan M. Frieze View Profile , Gary L. Miller View Profile , Shang-Hua Teng View Profile Authors Info & Claims SPAA '92: Proceedings of the fourth annual ACM symposium on Parallel algorithms and architecturesJune 1992 Pages 420–429https://doi.org/10.1145/140901.141934Published:01 June 1992Publication History 18citation296DownloadsMetricsTotal Citations18Total Downloads296Last 12 Months13Last 6 weeks6 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

We show that testing reachability in a planar DAG can be performed in parallel in O(log n log n) time (O(log n) time using randomization) using O(n) processors. In general we give a paradigm for contracting a planar DAG to a point and then expanding it back. This paradigm is developed from a property of planar directed graphs we refer to as the Poincar'e index formula. Using this new paradigm we then "overlay" our application in a fashion similar to parallel tree contraction [MR85, MR89]. We also discuss some of the changes needed to extend the reduction procedure to work for general planar digraphs. Using the strongly-connected components algorithm of Kao [Kao91] we can compute multiple-source reachability for general planar digraphs in O(log 3 n) time using O(n) processors. This improves the results of Kao and Klein [KK90] who showed that this problem could be performed in O(log 5 n) time using O(n) processors. This work represents initial results of an effort to develop effi...

A class of graphs called k-overlap graphs is proposed. Special cases of k-overlap graphs include planar graphs, k-nearest neighbor graphs, and earlier classes of graphs associated with finite element methods. A separator bound is proved for k-overlap graphs embedded in d dimensions. The result unifies several earlier separator results. All the arguments are based on geometric properties of embedding. The separator bounds come with randomized linear-time and randomized NC algorithms. Moreover, the bounds are the best possible up to the leading term.>

This paper applies the parallel tree contraction techniques developed in Miller and Reif’s paper [Randomness and Computation, Vol. 5, S. Micali, ed., JAI Press, 1989, pp. 47’72] to a number of fundamental graph problems. The paper presents an $O(\log n)$ time and $n / \log n$ processor, a 0-sided randomized algorithm for testing the isomorphism of trees, and an $O(\log n)$ time, n algorithm for maximal subtree isomorphism and for common subexpression elimination. An O(log n) time, n-processor algorithm for computing the canonical forms of trees and subtrees is given. An Olog n time algorithm for computing the tree of 3-connected components of a graph, an $O(\log ^2 n)$ time algorithm for computing an explicit planar embedding of a planar graph, and an $O(\log ^3 n)$ time algorithm for computing a canonical form for a planar graph are also given. All these latter algorithms use only $n^{O(1)} $ processors on a Parallel Random Access Machine (PRAM) model with concurrent writes and concurrent reads.

We show that every graph that is the 1-skeleton of a simplicial complex K in 3-dimensions has a separator of size O(c 2/3 + ~), where c is the number of 3-simplexes in K and 0 is the number of 0simplexes on the boundary of K, if every 3-simplex has bounded aspect-ratio.This is natural generalization of the separator results for planar graphs, such as the Lipton and Tarjan planar separator theorem.We also show that a family of separators of size O(c 2/3) exists and is constructible.Using this family of separators we get an O(n 2) time algorithm for solving linear systems that arise from the finite element method.In particular, we solve linear systems in O(n 2) time where the underlying graph is the 1-skeleton of a simplicial complex having bounded aspect-ratio and small boundary.All the constructions work in RNC with a reasonably small number of processors.

Given a planar network with many sources and sinks, the problem of computing the maximum flow from the sources to the sinks is investigated. An algorithm that runs in O(log/sup 2/n) time using O(n/sup 1.5/) processors on an exclusive-read-exclusive-write parallel random-access machine (EREW PRAM) is obtained, when the amount of flow (demand) at each source and sink is assumed as input. When the demands are unknown, the problem remains open. However, in the special case in which the sources and sinks are all on one face (and the demands unknown), an algorithm that computes the maximum flow with time complexity O(log/sup 3/n log log n) using O(n/sup 1.5/) processors is given. The results also hold for more general networks, namely, when the edge capacities have both lower and upper bounds.>

An O(log ~ n) time, n2/logn processor as well as an O(log n) time, n3/log n processor CREW deterministic parallel algorithms are presented for constructing Huffman codes from a given list of frequences.The time can be reduced to O(log n(loglog n) 2) on an CRCW model, using only n2/(log log n) 2 processors.Also presented is an optimal O(log n) time, O(n/log n) processor EREW parallel algorithm for constructing a tree given a list of leaf depths when the depths are monotonic.An O(log 2 n) time, n processor parallel algorithm is given for the general tree construction problem.We also give an O(log 2 n) time n2/log2n processor algorithm which finds a nearly optimal binary search tree.An O(log 2 n) time n 2'36 processor algorithm for recognizing linear context free languages is given.A crucial ingredient in achieving those bounds is a formulation of these problems as multiplications of special matrices which we call concave matrices.The structure of these matrices makes their parallel multiplication dramatically more efficient than that of arbitrary matrices.

A new parallel algorithm is given to evaluate a straight-line program. The algorithm evaluates a program over a commutative semi-ring R of degree d and size n in time $O((\log n)(\log nd))$ using $M(n)$ processors, where $M(n)$ is the number of processors required for multiplying $n \times n$ matrices over the semi-ring R in $O(\log n)$ time.

We present a randomized parallel algorithm for finding a simple cycle separator in a planar graph. The size of the separator is O(√n) and it separates the graph so that the largest part contains at most 2/8 · n vertices. Our algorithm takes T = O(log2(n)) time and P = O(n + f1+ε) processors, where n is the number of vertices, f is the number of faces and ε is any positive constant. The algorithm is based on the solution of Lipton and Tarjan [8] for the sequential case which takes O(n) time. Combining our algorithm with the Pan and Reif [12] algorithm, enables us to find a BFS of planar graph in time O(log3(n)) using n1.5/log(n) processors. Using a variation of our algorithm we can construct a simple cycle separator of size O(d · √f) were d is maximum face size.

We present a new algorithm for finding the tri-connected components of an undirected graph. The algorithm is based on ear decomposition and has linear sequential running time. It also has a parallel implementation on a CRCW PRAM with O(log2n) parallel time using a linear number of processors, where n is the number of vertices in the graph. This is the first efficient parallel algorithm for graph tri-connectivity.

The dynamic parallel complexity of general computational circuits (defined in introduction) is discussed. We exhibit some relationships between parallel circuit evaluation and some uniform closure properties of a certain class of unary functions and present a systematic method for the design of processor efficient parallel algorithms for circuit evaluation. Using this method: (1) we improve the algorithm for parallel Boolean circuit evaluation; (2) we give a nontrivial upper bound for parallel min-max-plus circuit evaluation; (3) we partially answer the first open question raised in [MiRK85] by showing that all circuits over finite noncommutative semi-ring and circuits over infinite non-commutative semi-ring which has finite dimension over a commutative semi-ring can be evaluated in polylogarithmic time in its size and degree using M(n) processors. Moreover, we develop a theory for determining closure properties of certain classes of unary functions.

The paper presents a sublinear time parallel algorithm for computing the greatest common divisor of two integers. Its running time on two n bit integers is $O({{n\log \log n} / {\log n}})$ using the weak concurrent read concurrent write model.

Let N be a positive integer, and let $\sigma (N)$ denote the sum of the divisors of N (e.g. $\sigma (6) = 1 + 2 + 3 + 6 = 12$). We show computing $\sigma (N)$ is equivalent to factoring N in the following sense: there is a random polynomial time algorithm that, given $\sigma (N)$, produces the prime factorization of N, and $\sigma (N)$ can be computed in polynomial time given the factorization of N. We show that the same result holds for $\sigma _k (N)$, the sum of the kth powers of divisors of N We give three new examples of problems that are in Gill’s complexity class BPP: perfect numbers, multiply perfect numbers, and amicable pairs. These are the first “natural” sets in BPP that are not obviously in RP.

Abstract : Trees play a fundamental role in many computations, both for sequential as well as parallel problems. The classic paradigm applied to generate parallel algorithms in the presence of trees has been divide-conquer; finding a 1/3 - 2/3 separator and recursively solving the two subproblems. A now classic example is Brent's work on parallel evaluation of arithmetic expressions. This top-down approach has several complications, one of which is finding the separators. We define dynamic expression evaluation as the task of evaluating the expression with no free preprocessing. If we apply Brent's method, finding the separators seems to add a factor of log n to the running time. We give a bottom-up algorithm to handle trees. That is, all modifications to the tree are done locally. This bottom-up approach which we call CONTRACT has two major advantages over the top-down approach: (1) the control structure is straight forward and easier to implement facilitating new algorithms using fewer processors and less time; and (2) problems for which it was too difficult or too complicated to find polylog parallel algorithms are now easy.

The advent of practical parallel processors has caused a reexamination of many existing algorithms with the hope of discovering a parallel implementation. One of the oldest and best known algorithms is Euclid's algorithm for computing the greatest common divisor (GCD). In this paper we present a parallel algorithm to compute the GCD of two integers. The two salient features of the algorithm are: the observation based on the pigeon hole principle that we can easily find an integer combination of the two integers A and B which has fewer bits than n and the idea of working in phases so as to perform arithmetics on n-bit integers only once every phase, the more frequent operations being performed on O(log/sup 2/n)-bit integers. It appears that yet another approach is needed if the GCD is to be computed in poly-log parallel time.

We have obtaincd some results in pebble coordination problems and the diameter of permutation groups.

Let N be a positive integer, and let σ(N) denote the sum of the positive integral divisors of N. We show computing σ(N) is equivalent to factoring N in the following sense: there is a random polynomial time algorithm that, given σ(N), produces the prime factorization of N, and σ(N) can be easily computed given the factorization of N.

We show that every 2-connected triangulated planar graph with n vertices has a simple cycle C of length at most [email protected]@@@n which separates the interior vertices A from the exterior vertices B such that neither A nor B contains more than 2/3n vertices. The method also gives a linear time algorithm for finding the simple cycle. In general, if the maximum face size is d then we exhibit a cycle C as above of size at most [email protected]@@@2d•n.

Every high school student knows how to express the roots of a quadratic equation in terms of radicals; what is less well-known is that this solution was found by the Babylonians a millenia and a half before Christ [Ne]. Three thousand years elapsed before European mathematicians determined how to express the roots of cubic and quartic equations in terms of radicals, and there they stopped, for their techniques did not extend. Lagrange published a treatise which discussed why the methods that worked for polynomials of degree less than five did not work for quintic polynomials [Lag],

In this extended abstract, we present several new layouts for the shuffle-exchange graph, including one which requires only 0(n2/log2n) area. The optimal layout is described and analyzed in section 3. The analysis is heavily dependent on several combinatorial results which we state in section 2 and prove in the appendix. The other layouts are described in section 4. Although these layouts are not asymptotically optimal (most require 0(n2/log3/2n) area), the theory behind their development is interesting and may eventually lead to good practical layouts as well as other asymptotically optimal layouts.

We present an algorithm which determines isomorphism of graphs in vO(g)steps where v is the number of vertices and g is the genus of the graphs. In [FMR 79] an algorithm was presented for embedding graph on surfaces of genus g in vO(g) steps. Here we show how to extend this algorithm to isomorphism testing for graphs of small genus. This result is noteworthy for at least two reasons. First, this extends the polynomial time isomorphism results for the plane [HT 72] and also the projective plane [L 80] to arbitrary surfaces. Second, this gives one of the few known natural decompositions of the isomorphism problem into an infinite hierarchy of problems Po,P1,... such that isomorphism testing of problems in P1 is decidable in time vO(i).

In this paper we present an algorithm which on input a graph G and a positive integer g finds an embedding of G on a surface on genius g, if such an embedding exists. This algorithm runs in (v) O(g) steps where v is the number of vertices of G.

Tarjan has given an algorithm for deciding isomorphism of two groups of order n (given as multiplication tables) which runs in O(n(log2n+O(1)) steps where n is the order of the groups. Tarjan uses the fact that a group of n is generated by log n elements. In this paper, we show that Tarjan's technique generalizes to isomorphism of quasigroups, latin squares, Steiner systems, and many graphs generated from these combinatorial objects.

An open question is the computational complexity of recognizing when two graphs are isomorphic. In an attempt to answer this question we shall analyze the relative computational complexity of generalizations and restrictions of the graph isomorphism problem. In the first section we show graph isomorphism of regular undirected graphs is complete over isomorphism of explicitly given structures (say Tarski models from logic). Then we show that valence seems to be important. Finally we analyze symmetric cubic graphs.

The purpose of this paper is to present new upper bounds on the complexity of algorithms for testing the primality of a number. The first upper bound is 0(n1/7); it improves the previously best known bound of 0(n1/4) due to Pollard [11].