Brain-inspired vision sensors (BVSs), drawing inspiration from the human visual system, produce sparse, high-temporal-resolution data stream capable of capturing rapid object motion. However, real-time processing of such data while preserving its inherent sparsity presents a critical challenge for practical deployment. A key challenge is to leverage the spatiotemporal correlations among event stream. Overlooking these correlations leads to prohibitively high query costs that even negate the benefits of sparsity, while exploiting them introduces complications such as input-output order conflicts and trade-offs between memory usage and latency. To address this, we present Espresso, an efficient hardware architecture that leverages the spatiotemporal order of events while explicitly preserving sparsity, achieving low-latency stream processing of event data. We first formalize a spatiotemporal order representation that identifies key features for stream processing on sparse events. Building on this, Espresso decouples output window address from input event address, resolving order conflicts via a dedicated queue mechanism and minimizing memory overhead through an optimized hash table. This enables immediate window-wise processing with minimal latency. To coordinate the pipeline, we design the Event-Scheduler, a streamlined finite state machine that prunes computations on zero values and aligns input-output stream order discrepancies. Integrated together, these modules deliver scalable, high-throughput processing for event-driven vision tasks. Espresso achieves up to 5000 fps, offering a 5.1× performance improvement over embedded GPUs. With parallel instantiations, it exceeds over 7000 fps in structured scenes and maintains over 2000 fps under complex-environment scenarios with minimal hardware overhead. These results establish Espresso as an efficient and scalable solution for real-time event-based vision processing, demonstrating the importance of spatiotemporal ordering in unlocking the full potential of BVSs.

Event-based vision sensors are novel cameras inspired by human eyes, capable of capturing the rapid motion of objects with the high-speed sparse event stream. However, it is challenging to efficiently stream process event data without demolishing its sparsity. In this paper, we design Espresso, an architecture for event-based vision processing that can efficiently stream spatiotemporal events while preserving sparsity. We implement Espresso on the FPGA platform and design experiments to compare the performance with embedded GPU and line-buffer-based accelerator. In real-world scenarios, Espresso achieves throughput up to 5000 fps, which is $5.1 \times$ higher than the embedded GPU.