Evaluating the usefulness of data before purchase is essential when obtaining data for high-quality machine learning models, yet both model builders and data providers are often unwilling to reveal their proprietary assets.

Metric magnitude of a point cloud is a measure of its ``size." It has been adapted to various mathematical contexts and recent work suggests that it can enhance machine learning and optimization algorithms. But its usability is limited due to the computational cost when the dataset is large or when the computation must be carried out repeatedly (e.g. in model training). In this paper, we study the magnitude computation problem, and show efficient ways of approximating it. We show that it can be cast as a convex optimization problem, but not as a submodular optimization. The paper describes two new algorithms -- an iterative approximation algorithm that converges fast and is accurate in practice, and a subset selection method that makes the computation even faster. It has previously been proposed that the magnitude of model sequences generated during stochastic gradient descent is correlated to the generalization gap. Extension of this result using our more scalable algorithms shows that longer sequences bear higher correlations. We also describe new applications of magnitude in machine learning -- as an effective regularizer for neural network training, and as a novel clustering criterion.

We present a novel set of rigorous and computationally efficient topology-based complexity notions that exhibit a strong correlation with the generalization gap in modern deep neural networks (DNNs). DNNs show remarkable generalization properties, yet the source of these capabilities remains elusive, defying the established statistical learning theory. Recent studies have revealed that properties of training trajectories can be indicative of generalization. Building on this insight, state-of-the-art methods have leveraged the topology of these trajectories, particularly their fractal dimension, to quantify generalization. Most existing works compute this quantity by assuming continuous- or infinite-time training dynamics, complicating the development of practical estimators capable of accurately predicting generalization without access to test data. In this paper, we respect the discrete-time nature of training trajectories and investigate the underlying topological quantities that can be amenable to topological data analysis tools. This leads to a new family of reliable topological complexity measures that provably bound the generalization error, eliminating the need for restrictive geometric assumptions. These measures are computationally friendly, enabling us to propose simple yet effective algorithms for computing generalization indices. Moreover, our flexible framework can be extended to different domains, tasks, and architectures. Our experimental results demonstrate that our new complexity measures exhibit a strong correlation with generalization error in industry-standard architectures such as transformers and deep graph networks. Our approach consistently outperforms existing topological bounds across a wide range of datasets, models, and optimizers, highlighting the practical relevance and effectiveness of our complexity measures.

The *magnitude* of a metric space is a novel invariant that provides a measure of the 'effective size' of a space across multiple scales, while also capturing numerous geometrical properties, such as curvature, density, or entropy. We develop a family of magnitude-based measures of the intrinsic diversity of latent representations, formalising a novel notion of dissimilarity between magnitude functions of finite metric spaces. Our measures are provably stable under perturbations of the data, can be efficiently calculated, and enable a rigorous multi-scale characterisation and comparison of latent representations. We show their utility and superior performance across different domains and tasks, including the automated estimation of diversity, the detection of mode collapse, and the evaluation of generative models for text, image, and graph data.

We consider the following surveillance problem: Given a set $P$ of $n$ sites in a metric space and a set of $k$ robots with the same maximum speed, compute a patrol schedule of minimum latency for the robots. Here a patrol schedule specifies for each robot an infinite sequence of sites to visit (in the given order) and the latency $L$ of a schedule is the maximum latency of any site, where the latency of a site $s$ is the supremum of the lengths of the time intervals between consecutive visits to $s$. When $k=1$ the problem is equivalent to the travelling salesman problem (TSP) and thus it is NP-hard. We have two main results. We consider cyclic solutions in which the set of sites must be partitioned into $\ell$ groups, for some~$\ell \leq k$, and each group is assigned a subset of the robots that move along the travelling salesman tour of the group at equal distance from each other. Our first main result is that approximating the optimal latency of the class of cyclic solutions can be reduced to approximating the optimal travelling salesman tour on some input, with only a $1+\varepsilon$ factor loss in the approximation factor and an $O\left(\left( k/\varepsilon \right)^k\right)$ factor loss in the runtime, for any $\varepsilon >0$. Our second main result shows that an optimal cyclic solution is a $2(1-1/k)$-approximation of the overall optimal solution. Note that for $k=2$ this implies that an optimal cyclic solution is optimal overall. The results have a number of consequences. For the Euclidean version of the problem, for instance, combining our results with known results on Euclidean TSP, yields a PTAS for approximating an optimal cyclic solution, and it yields a $(2(1-1/k)+\varepsilon)$-approximation of the optimal unrestricted solution. If the conjecture mentioned above is true, then our algorithm is actually a PTAS for the general problem in the Euclidean setting.

Publishing data from IoT devices raises concerns of leaking sensitive information. In this paper we consider the scenario of publishing data on events with timestamps. We formulate three privacy issues, namely, whether one can tell if an event happened or not; whether one can nail down the timestamp of an event within a given time interval; and whether one can infer the relative order of any two nearby events. We show that perturbation of event timestamps or adding fake events following carefully chosen distributions can address these privacy concerns. We present a rigorous study of privately publishing discrete event timestamps with privacy guarantees under the Pufferfish privacy framework. We also conduct extensive experiments to evaluate utility of the modified time series with real world location check-in and app usage data. Our mechanisms preserve the statistical utility of event data which are suitable for aggregate queries.

Over the last few years, the Shapley value, a solution concept from cooperative game theory, has found numerous applications in machine learning. In this paper, we first discuss fundamental concepts of cooperative game theory and axiomatic properties of the Shapley value. Then we give an overview of the most important applications of the Shapley value in machine learning: feature selection, explainability, multi-agent reinforcement learning, ensemble pruning, and data valuation. We examine the most crucial limitations of the Shapley value and point out directions for future research.

What is the value of an individual model in an ensemble of binary classifiers? We answer this question by introducing a class of transferable utility cooperative games called ensemble games. In machine learning ensembles, pre-trained models cooperate to make classification decisions. To quantify the importance of models in these ensemble games, we define Troupe - an efficient algorithm that allocates payoffs based on approximate Shapley values of the classifiers. We argue that the Shapley value of models in these games is an effective decision metric for choosing a high-performing subset of models from the ensemble. Our analytical findings prove that our Shapley value estimation scheme is precise and scalable; its performance increases with the size of the dataset and ensemble. Empirical results on real-world graph classification tasks demonstrate that our algorithm produces high-quality estimates of the Shapley value. We find that Shapley values can be utilized for ensemble pruning and that adversarial models receive a low valuation. Complex classifiers are frequently found to be responsible for both correct and incorrect classification decisions.

We present PyTorch Geometric Temporal, a deep learning framework combining state-of-the-art machine learning algorithms for neural spatiotemporal signal processing. The main goal of the library is to make temporal geometric deep learning available for researchers and machine learning practitioners in a unified easy-to-use framework. PyTorch Geometric Temporal was created with foundations on existing libraries in the PyTorch eco-system, streamlined neural network layer definitions, temporal snapshot generators for batching, and integrated benchmark datasets. These features are illustrated with a tutorial-like case study. Experiments demonstrate the predictive performance of the models implemented in the library on real-world problems such as epidemiological forecasting, ride-hail demand prediction, and web traffic management. Our sensitivity analysis of runtime shows that the framework can potentially operate on web-scale datasets with rich temporal features and spatial structure.

Graphs encode important structural properties of complex systems. Machine learning on graphs has therefore emerged as an important technique in research and applications. We present Karate Club - a Python framework combining more than 30 state-of-the-art graph mining algorithms. These unsupervised techniques make it easy to identify and represent common graph features. The primary goal of the package is to make community detection, node and whole graph embedding available to a wide audience of machine learning researchers and practitioners. Karate Club is designed with an emphasis on a consistent application interface, scalability, ease of use, sensible out of the box model behaviour, standardized dataset ingestion, and output generation. This paper discusses the design principles behind the framework with practical examples. We show Karate Club's efficiency in learning performance on a wide range of real world clustering problems and classification tasks along with supporting evidence of its competitive speed.

Sampling graphs is an important task in data mining. In this paper, we describe Little Ball of Fur a Python library that includes more than twenty graph sampling algorithms. Our goal is to make node, edge, and exploration-based network sampling techniques accessible to a large number of professionals, researchers, and students in a single streamlined framework. We created this framework with a focus on a coherent application public interface which has a convenient design, generic input data requirements, and reasonable baseline settings of algorithms. Here we overview these design foundations of the framework in detail with illustrative code snippets. We show the practical usability of the library by estimating various global statistics of social networks and web graphs. Experiments demonstrate that Little Ball of Fur can speed up node and whole graph embedding techniques considerably with mildly deteriorating the predictive value of distilled features.

In this paper, we propose a flexible notion of characteristic functions defined on graph vertices to describe the distribution of vertex features at multiple scales. We introduce FEATHER, a computationally efficient algorithm to calculate a specific variant of these characteristic functions where the probability weights of the characteristic function are defined as the transition probabilities of random walks. We argue that features extracted by this procedure are useful for node level machine learning tasks. We discuss the pooling of these node representations, resulting in compact descriptors of graphs that can serve as features for graph classification algorithms. We analytically prove that FEATHER describes isomorphic graphs with the same representation and exhibits robustness to data corruption. Using the node feature characteristic functions we define parametric models where evaluation points of the functions are learned parameters of supervised classifiers. Experiments on real world large datasets show that our proposed algorithm creates high quality representations, performs transfer learning efficiently, exhibits robustness to hyperparameter changes and scales linearly with the input size.

This paper considers the problem of privately reporting counts of events recorded by devices in different regions of the plane. Unlike previous range query methods, our approach is not limited to rectangular ranges. We devise novel hierarchical data structures to answer queries over arbitrary planar graphs. This construction relies on balanced planar separators to represent shortest paths using O(logn) number of canonical paths, where n is the number of nodes in the graph. Pre-computed sums along these canonical paths allow efficient computations of 1D counting range queries along any shortest path. We make use of differential forms together with the 1D mechanism to answer 2D queries in which a range is a union of faces in the planar graph. The methods are designed such that the range queries could be answered with differential privacy guarantee on any single event, with only a poly-logarithmic error. They also allow private range queries to be performed in a distributed setup. Theoretical and experimental results confirm that the methods are efficient and accurate on real data and incur less error than competing existing methods.

Abstract Anonymous communication networks like Tor are vulnerable to attackers that control entry and exit nodes. Such attackers can compromise the essential anonymity and privacy properties of the network. In this paper, we consider the path bias attack– where the attacker induces a client to use compromised nodes and thus links the client to their destination. We describe an efficient scheme that detects such attacks in Tor by collecting routing telemetry data from nodes in the network. The data collection is differentially private and thus does not reveal behaviour of individual users even to nodes within the network. We show provable bounds for the sample complexity of the scheme and describe methods to make it resilient to introduction of false data by the attacker to subvert the detection process. Simulations based on real configurations of the Tor network show that the method works accurately in practice.

Modern graph embedding procedures can efficiently process graphs with millions of nodes. In this paper, we propose GEMSEC - a graph embedding algorithm which learns a clustering of the nodes simultaneously with computing their embedding. GEMSEC is a general extension of earlier work in the domain of sequence-based graph embedding. GEMSEC places nodes in an abstract feature space where the vertex features minimize the negative log-likelihood of preserving sampled vertex neighborhoods, and it incorporates known social network properties through a machine learning regularization. We present two new social network datasets and show that by simultaneously considering the embedding and clustering problems with respect to social properties, GEMSEC extracts high-quality clusters competitive with or superior to other community detection algorithms. In experiments, the method is found to be computationally efficient and robust to the choice of hyperparameters.

We consider the problem of finding large scale mobility patterns. A common challenge in mobility tracking systems is that large quantity of data is spread out spatially and temporally across many tracking sensors. We thus devise a spatial sampling and information exchange protocol that provides probabilistic guarantees on detecting prominent patterns. For this purpose, we define a general notion of significant popular paths that can capture many different types of motion. We design a summary sketch for the data at each tracking node, which can be updated efficiently, and then aggregated across devices to reconstruct the prominent paths in the global data. The algorithm is scalable, even with large number of mobile targets. It uses a hierarchic query system that automatically prioritizes important trajectories - those that are long and popular. We show further that this scheme can in fact give good results by sampling relatively few sensors and targets, and works for streaming spatial data. We prove differential privacy guarantees for the randomized algorithm. Extensive experiments on real GPS data show that the method is efficient and accurate, and is useful in predicting motion of travelers even with small samples.

Searching for similar GPS trajectories is a fundamental problem that faces challenges of large data volume and intrinsic complexity of trajectory comparison. In this paper, we present a suite of sketches for trajectory data that drastically reduce the computation costs associated with near neighbor search, distance estimation, clustering and classification, and subtrajectory detection. Apart from summarizing the dataset, our sketches have two uses. First, we obtain simple provable locality sensitive hash families for both the Hausdorff and Fréchet distance measures, useful in near neighbour queries. Second, we build a data structure called MRTS (Multi Resolution Trajectory Sketch), which contains sketches of varying degrees of detail. The MRTS is a user-friendly, compact representation of the dataset that allows to efficiently answer various other types of queries. Moreover, MRTS can be used in a dynamic setting with fast insertions of trajectories into the database.

Analytic methods can be difficult to build and costly to train for mobility data. We show that information about the topology of the space and how mobile objects navigate the obstacles can be used to extract insights about mobility at larger distance scales. The main contribution of this paper is a topological signature that maps each trajectory to a relatively low dimensional Euclidean space, so that now they are amenable to standard analytic techniques. Data mining tasks: nearest neighbor search with locality sensitive hashing, clustering, regression, etc., work more efficiently in this signature space. We define the problem of mobility prediction at different distance scales, and show that with the signatures simple k nearest neighbor based regression perform accurate prediction. Experiments on multiple real datasets show that the framework using topological signatures is accurate on all tasks, and substantially more efficient than machine learning applied to raw data. Theoretical results show that the signatures contain enough topological information to reconstruct non-self-intersecting trajectories upto homotopy type. The construction of signatures is based on a differential form that can be generated in a distributed setting using local communication, and a signature can be locally and inexpensively updated and communicated by a mobile agent.

Periodic phenomena are ubiquitous, but detecting and predicting periodic events can be difficult in noisy environments. We describe a model of periodic events that covers both idealized and realistic scenarios characterized by multiple kinds of noise. The model incorporates false-positive events and the possibility that the underlying period and phase of the events change over time. We then describe a particle filter that can efficiently and accurately estimate the parameters of the process generating periodic events intermingled with independent noise events. The system has a small memory footprint, and, unlike alternative methods, its computational complexity is constant in the number of events that have been observed. As a result, it can be applied in low-resource settings that require real-time performance over long periods of time. In experiments on real and simulated data we find that it outperforms existing methods in accuracy and can track changes in periodicity and other characteristics in dynamic event streams.

We study the r-gather clustering problem in a mobile and distributed setting. In this problem, nodes must be clustered into groups of at least r nodes each, and the goal is to minimize the diameter of the clusters. This notion of clustering is motivated by protecting user anonymity in location-based services or trajectory publication. Prior works on r-gather problems are centralized and cannot be easily adapted to the mobile setting. We describe a distributed algorithm that produces compact clusters, within an approximation factor 4 of the minimum cluster diameter possible. The algorithm can run on the mobile nodes and access points at the network edge locally, and can handle node mobility, rapidly switching cluster memberships as needed. The distributed approach naturally comes with the advantage of greater resilience and stability. Additionally, we show that it achieves local optimality; i.e., from the point of view of any particular node, the solution is nearly as favorable as possible, irrespective of the global configuration. We also show how to cluster trajectories with dynamic re-groupings. Further, we improve the theoretical hardness results for the problem in the Euclidean setting.

Many large-scale machine learning problems--clustering, non- parametric learning, kernel machines, etc.--require selecting a small yet representative subset from a large dataset. Such problems can often be reduced to maximizing a submodular set function subject to various constraints. Classical approaches to submodular optimization require centralized access to the full dataset, which is impractical for truly large-scale problems. In this paper, we consider the problem of submodular function maximization in a distributed fashion. We develop a simple, two- stage protocol GREEDI, that is easily implemented using MapReduce style computations. We theoretically analyze our approach, and show that under certain natural conditions, performance close to the centralized approach can be achieved. We begin with monotone submodular maximization subject to a cardinality constraint, and then extend this approach to obtain approximation guarantees for (not necessarily monotone) submodular maximization subject to more general constraints including matroid or knapsack constraints. In our extensive experiments, we demonstrate the effectiveness of our approach on several applications, including sparse Gaussian process inference and exemplar based clustering on tens of millions of examples using Hadoop.

We describe an online algorithm to simplify large volumes of location and sensor data on the source mobile device, by eliminating redundant data points and saving important ones. Our approach is to use topological persistence to identify large scale sharp features of a data stream.

Homotopic routing asks for a path going around holes according to a given “threading”. Paths of different homo-topy types can be used to improve load balancing and routing resilience. We propose the first lightweight homotopic routing scheme that generates constant bounded stretch compared to the shortest path of the same homotopy type. Our main insight is that in a sequence of triangles to traverse, a message always routed to the nearest point on the next triangle in the sequence travels at most a constant times the length of any shortest path going through the same sequence of triangles. Our routing scheme operates on two levels enabled by a coarse triangulation. The top level is used to specify and represent the requested homotopy type, while the bottom level executes the local greedy routing on a triangle sequence. After a preprocessing step that triangulates the given region and creates a minimum-size auxiliary structure, routing operates greedily at two different resolutions. We also present simulation analysis in a variety of settings and show that the paths indeed have small stretch in practice, considerably shorter than the bounds guaranteed by the theory.

The environmental context of a mobile device determines where/how it is used, which can be exploited for efficient operation and better usability. In this work we describe a general method using only the lightweight sensors on a smartphone to detect if a device is indoor or outdoor. Using semi-supervised machine learning techniques, our method automatically learns characteristics of new environments and devices, thereby achieves detection accuracy of over 90% even in unfamiliar circumstances. Therefore, it easily outperforms existing indoor-outdoor detection techniques based on static algorithms, or relying on energy hungry and unreliable GPS.

The environmental context of a mobile device determines how it is used and how the device can optimize operations for greater efficiency and usability. We consider the problem of detecting if a device is indoor or outdoor. Towards this end, we present a general method employing semi-supervised machine learning and using only the lightweight sensors on a smartphone. We find that a particular semi-supervised learning method called co-training, when suitably engineered, is most effective. It is able to automatically learn characteristics of new environments and devices, and thereby provides a detection accuracy exceeding 90% even in unfamiliar circumstances. It can learn and adapt online, in real time, at modest computational costs. Thus the method is suitable for on-device learning. Implementation of the indoor-outdoor detection service based on our method is lightweight in energy use -- it can sleep when not in use and does not need to track the device state continuously. It is shown to outperform existing indoor-outdoor detection techniques that rely on static algorithms or GPS, in terms of both accuracy and energy-efficiency.

Many large-scale machine learning problems (such as clustering, non-parametric learning, kernel machines, etc.) require selecting, out of a massive data set, a manageable, representative subset. Such problems can often be reduced to maximizing a submodular set function subject to cardinality constraints. Classical approaches require centralized access to the full data set; but for truly large-scale problems, rendering the data centrally is often impractical. In this paper, we consider the problem of submodular function maximization in a distributed fashion. We develop a simple, two-stage protocol GreeDI, that is easily implemented using MapReduce style computations. We theoretically analyze our approach, and show, that under certain natural conditions, performance close to the (impractical) centralized approach can be achieved. In our extensive experiments, we demonstrate the effectiveness of our approach on several applications, including sparse Gaussian process inference on tens of millions of examples using Hadoop.

Consider mobile targets in a plane and their movements being monitored by a network such as a field of sensors. We develop distributed algorithms for in-network tracking and range queries for aggregated data (for example, returning the number of targets within any user given region). Our scheme stores the target detection information locally in the network and answers a query by examining the perimeter of the given range. The cost of updating data about mobile targets is proportional to the target displacement. The key insight is to maintain in the sensor network a function with respect to the target detection data on the graph edges that is a differential form such that the integral of this form along any closed curve C gives the integral within the region bounded by C. The differential form has great flexibility, making it appropriate for tracking mobile targets. The basic range query can be used to find a nearby target or any given identifiable target with cost O(d), where d is the distance to the target in question. Dynamic insertion, deletion, coverage holes, and mobility of sensor nodes can be handled with only local operations, making the scheme suitable for a highly dynamic network. It is extremely robust and capable of tolerating errors in sensing and target localization. Targets do not need to be identified for the tracking, thus user privacy can be preserved. In this paper, we only elaborate the advantages of differential forms in tracking of mobile targets. Similar routines can be applied for organizing many other types of information-for example, streaming scalar sensor data (such as temperature data field)-to support efficient range queries. We demonstrate through analysis and simulations that this scheme compares favorably to existing schemes that use location services for answering aggregate range queries of target detection data.

In traditional routing, the routing tables store shortest paths to all other destinations and have size linear in the size of the network, which is not scalable for resource-constrained networks such as wireless sensor networks. In this article we show that by storing selectively a much smaller set of routing paths in the routing tables one can get low-stretch, compact routing schemes. Our routing scheme includes an approximate distance oracle with which one can obtain approximate shortest path length estimates to destinations. This distance oracle can be obtained, for example, by a landmark-based scheme, or in case of sensor networks, from the geographic distance between node locations. With an approximate distance oracle one can attempt greedy routing by forwarding to the neighbor whose estimate is closer to the destination. But there is no guarantee of delivery nor of the routing path length. We augment the distance oracle by storing, for each node u , routing paths to O (log 2 n ) strategically selected nodes that serve as intermediate destinations. These nodes are selected with probability proportional to 1/ r ρ , where r is the distance to u and ρ is a suitable constant for the network. Then we derive a set of sufficient conditions to select the next step at each stage of routing, such that these conditions can be verified locally and guarantee 1+ε stretch routing on any metric. These conditions serve as the “greedy routing” or local decision rule. On graphs of bounded growth, our scheme guarantees 1+ε stretch routing with high probability, with an average routing table size of O (√n log 2 n ). This scheme is favorable for its simplicity, generality, and blindness to any global state. It demonstrates that global routing properties could emerge from purely distributed and uncoordinated routing table design.

In this paper we address the problem of scalable and load balanced routing for wireless sensor networks. Motivated by the analog of the continuous setting that geodesic routing on a sphere gives perfect load balancing, we embed sensor nodes on a convex polyhedron in 3D and use greedy routing to deliver messages between any pair of nodes with guaranteed success. This embedding is known to exist by the Koebe-Andreev-Thurston Theorem for any 3-connected planar graphs. In our paper we use discrete Ricci flow to develop a distributed algorithm to compute this embedding. Further, such an embedding is not unique and differs from one another by a Möbius transformation. We employ an optimization routine to look for the Möbius transformation such that the nodes are spread on the polyhedron as uniformly as possible. We evaluated the load balancing property of this greedy routing scheme and showed favorable comparison with previous schemes.

A wormhole attack places two radio transceivers connected by a high capacity link and retransmits wireless signals from one antenna at the other. This creates a set of shortcut paths in the network, and may attract a lot of traffic to the wormhole link. The link thus gains control of a large fraction of network traffic which opens the door for more dangerous attacks afterwards. In this paper we introduce a wormhole detection and removal algorithm based on local connectivity tests.

In this article we propose a lightweight algorithm for constructing multiresolution data representations for sensor networks. At each sensor node u , we compute O (log n ) aggregates about exponentially enlarging neighborhoods centered at u . The i th aggregate is the aggregated data from nodes approximately within 2 i hops of u . We present a scheme, named the hierarchical spatial gossip algorithm , to extract and construct these aggregates, for all sensors simultaneously, with a total communication cost of O ( n polylog n ). The hierarchical gossip algorithm adopts atomic communication steps with each node choosing to exchange information with a node distance d away with probability ∼ 1/ d 3 . The attractiveness of the algorithm can be attributed to its simplicity, low communication cost, distributed nature, and robustness to node failures and link failures. We show in addition that computing multiresolution aggregates precisely (i.e., each aggregate uses all and only the nodes within 2 i hops) requires a communication cost of Ω( n √ n ), which does not scale well with network size. An approximate range in aggregate computation like that introduced by the gossip mechanism is therefore necessary in a scalable efficient algorithm. Besides the natural applications of multiresolution data summaries in data validation and information mining, we also demonstrate the application of the precomputed multiresolution data summaries in answering range queries efficiently.

We study how to characterize the families of paths between any two nodes s, t in a sensor network with holes. Two paths that can be deformed to one another through local changes are called homotopy equivalent. Two paths that pass around holes in different ways have different homotopy types. With a distributed algorithm we compute an embedding of the network in hyperbolic space by using Ricci flow such that paths of different homotopy types are mapped naturally to paths connecting s with different images of t. Greedy routing to a particular image is guaranteed with success to find a path with a given homotopy type. This leads to simple greedy routing algorithms that are resilient to both local link dynamics and large scale jamming attacks and improve load balancing over previous greedy routing algorithms.

For in-network storage schemes, one maps data, indexed in a logical space, to the distributed sensor locations. When the physical sensor network has an irregular shape and possibly holes, the mapping of data to sensors often creates unbalanced storage load with high data concentration on nodes near network boundaries. In this paper we propose to map data to a covering space, which is a tiling of the plane with copies of the sensor network, such that the sensors receive uniform storage load and traffic. We propose distributed algorithms to construct the covering space with Ricci flow and Möbius transforms. The use of the covering space improves the performance of many in-network storage and retrieval schemes such as geographical hash tables (GHTs) or the double rulings (quorum based schemes), and provides better load balanced routing.

Consider mobile targets moving in a plane and their movements being monitored by a network such as a field of sensors. We develop distributed algorithms for in-network tracking and range queries for aggregated data (for example returning the number of targets within any user given region). Our scheme stores the target detection information locally in the network, and answers a query by examining the perimeter of the given range. The cost of updating data about mobile targets is proportional to the target displacement. The key insight is to maintain in the sensor network a function with respect to the target detection data on the graph edges that is a differential one-form such that the integral of this one-form along any closed curve C gives the integral within the region bounded by C.

We propose a generic routing table design principle for scalable routing on networks with bounded geometric growth. Given an inaccurate distance oracle that estimates the graph distance of any two nodes with constant factor upper and lower bounds, we augment it by storing the routing paths of pairs of nodes, selected in a spatial distribution, and show that the routing table enables 1 + epsiv stretch routing. In the wireless ad hoc and sensor network scenario, the geographic locations of the nodes serve as such an inaccurate distance oracle. Each node p selects O (log n loglog n) other nodes from a distribution proportional to 1/r2where r is the distance to p and the routing paths to these nodes are stored on the nodes along these paths in the network. The routing algorithm selects links conforming to a set of sufficient conditions and guarantees with high probability 1 + epsiv stretch routing with routing table size O(radicn log n loglog n) on average for each node. This scheme is favorable for its simplicity, generality and blindness to any global state. It is a good example that global routing properties emerge from purely distributed and uncoordinated routing table design.

We are interested in topological analysis and processing of the large-scale distributed data generated by sensor networks. Naturally, a large-scale sensor network is deployed in a geometric region with possibly holes and complex shape, and is used to sample some smooth physical signal field. We are interested in both the topology of the discrete sensor field in terms of the sensing holes (voids without sufficient sensors deployed), as well as the topology of the signal field in terms of its critical points (local maxima, minima and saddles). Towards this end, we develop distributed algorithms to construct the Morse-Smale decomposition, and study the performance benefits obtained by this approach. The sensor field is decomposed into simply-connected pieces, inside each of which the sensor signal is homogeneous, i.e., the data flows uniformly from a local maximum to a local minimum. The Morse-Smale decomposition can be efficiently constructed in the network locally, after which applications such as iso-contour queries, data-guided navigation and routing, data aggregation, and topologically faithful signal reconstructions benefit tremendously from it.

The diversity of the deployment settings of sensor networks is naturally inherited from the diversity of geographical features of the embedded environment, and greatly influences network design. Many sensor network protocols in the literature implicitly assume that sensor nodes are deployed inside a simple geometric region, without considering possible obstacles and holes in the deployment environment. When the real deployment setting deviates from that, we often observe degraded performance. Thus, it is highly desirable to have a generic approach to handle sensor fields with complex shapes. In this article, we propose a segmentation algorithm that partitions an irregular sensor field into nicely shaped pieces such that algorithms and protocols that assume a nice sensor field can be applied inside each piece. Across the segments, problem dependent structures specify how the segments and data collected in these segments are integrated. Our segmentation algorithm does not require any extra knowledge (e.g., sensor locations) and only uses network connectivity information. This unified spatial-partitioning approach makes the protocol design become flexible and independent of deployment specifics. Existing protocols are still reusable with segmentation, and the development of new topology-adaptive protocols becomes much easier. We verified the correctness of the algorithm on various topologies and evaluated the performance improvements by integrating shape segmentation with several fundamental problems in network design.

Abstract—We study the problem of data-driven routing and navigation in a distributed sensor network over a continuous scalar field. Specifically, we address the problem of searching for the collection of sensors with readings within a specified range. This is named the iso-contour query problem. We develop a gradient based routing scheme such that from any query node, the query message follows the signal field gradient or derived quantities and successfully discovers all iso-contours of interest. Due to the existence of local maxima and minima, the guaranteed delivery requires preprocessing of the signal field and the construction of a contour tree in a distributed fashion. Our approach has the following properties: (i) the gradient routing uses only local node information and its message complexity is close to optimal, as shown by simulations; (ii) the preprocessing message complexity is linear in the number of nodes and the storage requirement for each node is a small constant. The same preprocessing also facilitates route computation between any pair of nodes where the the route lies within any user supplied range of values. I.

We study the problem of contour tracking with binary sensors, an important problem for monitoring spatial signals and tracking group targets. In particular, we track the boundaries of the blobs of interest and capture the topological changes as the blobs merge or split. Only the nodes on the boundaries of these deformable blobs stay active and the repair cost is proportional to the size of the contour changes. Our algorithm is completely distributed, requires only local information, and yet captures the global topological properties. The algorithm performs a fundamental monitoring function and is a foundation for further information processing of spatial sensor data.

Many sensor network protocols in the literature implicitly assume that sensor nodes are deployed uniformly inside a simple geometric region. When the real deployment deviates from that, we often observe degraded performance. It is desirable to have a generic approach to handle a sensor field with complex shape. In this paper, we propose a segmentation algorithm that partitions an irregular sensor field into nicely shaped pieces such that algorithms and protocols that assume a nice sensor field can be applied inside each piece. Across the segments, problem dependent structures specify how the segments and data collected in these segments are integrated. This unified topology-adaptive spatial partitioning would benefit many settings that currently assume a nicely shaped sensor field. Our segmentation algorithm does not require sensor locations and only uses network connectivity information. Each node is given a 'flow direction' that directs away from the network boundary. A node with no flow direction becomes a sink, and attracts other nodes in the same segment. We evaluate the performance improvements by integrating shape segmentation with applications such as distributed indices and random sampling.

In this paper we propose a lightweight algorithm for constructing multi-resolution data representations for sensor networks. We compute, at each sensor node u, O(log n) aggregates about exponentially enlarging neighborhoods centered at u. The ith aggregate is the aggregated data among nodes approximately within 2i hops of u. We present a scheme, named the hierarchical spatial gossip algorithm, to extract and construct these aggregates, for all sensors simultaneously, with a total communication cost of O(n polylog n). The hierarchical gossip algorithm adopts atomic communication steps with each node choosing to exchange information with a node distance d away with probability 1 /d3. The attractiveness of the algorithm attributes to its simplicity, low communication cost, distributed nature and robustness to node failures and link failures. Besides the natural applications of multi-resolution data summaries in data validation and information mining, we also demonstrate the application of the pre-computed spatial multi-resolution data summaries in answering range queries efficiently.

We study the problem of information brokerage in sensor networks, where information consumers (sinks,users)search for data acquired by information producers (sources). In-network storage such as geographical hash table (GHTs) has been proposed to store data at rendezvous nodes for consumers to retrieve. In this paper, we propose a double rulings scheme which stores data replica at a curve instead of one or multiple isolated sensors. The consumer travels along another curve which guarantees to intersect with the producer curve. The double rulings is a natural extension of the flat hashing scheme such as GHTs with improved query locality, i.e., consumers close to producers find the data quickly, and structured aggregate queries, i.e., a consumer following a curve is able to retrieve all the data. Further, by the flexibility of retrieval mechanisms we have better routing robustness and data robustness. We show by simulation that the double rulings scheme provide reduced communication costs and more balanced traffic load on the sensors.