GeaFlow is a distributed dataflow system optimized for streaming graph processing, and has been widely adopted at Ant Group, serving various scenarios ranging from risk control of financial activities to analytics on social networks and knowledge graphs. It is built on top of a base with full-fledged stateful stream processing capabilities, extended with a series of graph-aware optimizations to address the space explosion and programming complexity issues of conventional join-based approaches. We propose new state backends and streaming operators that facilitate processing on dynamic graph-structured datasets, reducing space consumed by states. We develop a hybrid domain-specific language that embeds Gremlin into SQL, supporting both table and graph abstractions over streaming data. In addition to streaming workloads, GeaFlow is also extensively used for some batch processing jobs. In the largest deployments to date, GeaFlow is able to process tens of millions of events per second and manage hundreds of terabytes of states.

We consider the crucial problem of maximizing the social welfare of a vehicular participatory sensing system, where the system's social welfare is measured by the amount of sensing data delivered to a central platform through a vehicular ad hoc network. The key to the problem is to control network stability since both network congestion and idleness will slump system social welfare. However, several great challenges exist. First, limited vehicle-to-vehicle (V2V) link capacity and vehicle buffer size will lead to heavy network congestion when each individual vehicle blindly injects too much data into the network hoping to get more rewards. Second, the highly dynamic network topology and stochastic inter-vehicle contacts have a serious impact on the performance of multi-hop data transmission. Third, vehicles need to be practically rewarded based on their sensing and transmission cost, which, however, greatly vary among vehicles. To tackle the aforementioned challenges, we propose a distributed backpressure control approach, the first work to the best of our knowledge, to maximize the social welfare while balancing network stability for a vehicular participatory sensing system. Combining vehicular network properties and Lyapunov optimization techniques, individualized strategies are developed for each participant to control its sensing rate, make its own routing decisions, and set its own price for data relaying. Formally proved by rigorous theoretical analysis, the social welfare achieved by the proposed approach is comparative to the optimum performance. In addition, extensive data-driven simulations based on real taxi GPS traces have been conducted, and the results confirm the efficacy of the proposed algorithm.

With the proliferation of smartphones, participatory sensing using smartphones provides unprecedented opportunities for collecting enormous sensing data. There are two crucial requirements in participatory sensing, fair task allocation and energy efficiency, which are particularly challenging given high combinatorial complexity, trade-off between energy efficiency and fairness, and dynamic and unpredictable task arrivals. In this paper, we present a novel fair energy-efficient allocation framework whose objective is characterized by min–max aggregate sensing time. We rigorously prove that optimizing the min–max aggregate sensing time is NP hard even when the tasks are assumed as a priori. We consider two allocation models: offline allocation and online allocation. For the offline allocation model, we design an efficient approximation algorithm with the approximation ratio of 2−1m⁠, where m is the number of member smartphones in the system. For the online allocation model, we propose two algorithms: greedy algorithm and Robin-Hood algorithm, which achieve the competitive ratio of at most m and m+1⁠, respectively. The results demonstrate that the approximation algorithm reduces over 81% total sensing time, the online greedy algorithm and Robin-Hood algorithms reduce the total sensing time 73% and 37.5%, respectively. The offline approximation algorithm and online greedy algorithm achieve much better min–max fairness compared to other algorithms.

We are interested in delay-constrained data aggregation in delay tolerant vehicular networks, in which contact opportunities are usually scarce and thus become critical resources. Data aggregation has been recognized as an effective technique for reducing communication cost while obtaining useful aggregated information. The unique characteristics of the network present great challenges to the delay-constrained aggregation issue. First, there is no always-connected path from a source node to a destination node. Second, there is an intrinsic tradeoff between communication cost and aggregation quality. In this paper we discuss the delay-constrained data aggregation problem in delay tolerant vehicular networks with the objective of maximizing the amount of information collected, which has not been well studied in the literature. We observe that there are heterogeneity and regularity with node contact patterns. Motivated by this observation, we propose an Efficient tree-based Data Aggregation algorithm called EDA. EDA first constructs a data aggregation tree based on the shortest path tree, and then assigns a waiting time budget to each node on the tree using a dynamic programming algorithm. We have performed extensive simulations on real taxi traces and results show that our EDA scheme incurs much lower transmission overhead while achieving the same performance compared with other schemes.

With the proliferation of smartphones, participatory sensing using smartphones provides unprecedented opportunities for collecting enormous sensing data. There are two crucial requirements in participatory sensing, fair task allocation and energy efficiency, which are particularly challenging given high combinatorial complexity, tradeoff between energy efficiency and fairness, and dynamic and unpredictable task arrivals. In this paper, we present a novel fair energy-efficient allocation framework whose objective is characterized by min-max aggregate sensing time. We rigorously prove that optimizing the min-max aggregate sensing time is NP hard even when the tasks are assumed as a priori. We consider two allocation models: offline allocation and online allocation. For the offline allocation model, we design an efficient approximation algorithm with the approximation ratio of 2 - 1/m, where m is the number of member smartphones in the system. For the online allocation model, we propose a greedy online algorithm which achieves a competitive ratio of at most m. The results demonstrate that the approximation algorithm reduces over 81% total sensing time, the greedy online algorithm reduces more than 73% total sensing time, and both algorithms achieve over 3x better min-max fairness.

We consider the crucial problem of maximizing the social welfare of a vehicular participatory sensing system which consists of mobile source vehicles that collect sensing data on the move and relay vehicles that forward sensing data to the central sensing platform. Three issues should be taken into consideration, i.e., sensing rate control, flow control and transmission, incentives for vehicles to participate. It is highly challenging to achieve the optimal social welfare, given the arbitrary unknown contact events and the dynamic network formed by mobile vehicles. To tackle the challenges, we propose a fully distributed optimal control algorithm DBC to maximize the social welfare while keeping good network stability. Exploiting the stochastic Lyapunov optimization technique, it derives an optimal control strategy for sensing rate control and flow control. Rigorous theoretical analysis shows the gap between the DBC and the optimum is only O(1/V), where V is a tradeoff parameter between social welfare and network stability. In addition, we have conducted extensive simulations based on real taxi GPS traces, and the results show the efficacy of DBC.