Recently, 3D Gaussian Splatting (3DGS) has emerged as a significant advancement in 3D scene reconstruction, attracting considerable attention due to its ability to recover high-fidelity details while maintaining low complexity. Despite the promising results achieved by 3DGS, its rendering performance is constrained by its dependence on costly non-commutative alpha-blending operations. These operations mandate complex view dependent sorting operations that introduce computational overhead, especially on the resource-constrained platforms such as mobile phones. In this paper, we propose Weighted Sum Rendering, which approximates alpha blending with weighted sums, thereby removing the need for sorting. This simplifies implementation, delivers superior performance, and eliminates the ``popping'' artifacts caused by sorting. Experimental results show that optimizing a generalized Gaussian splatting formulation to the new differentiable rendering yields competitive image quality. The method was implemented and tested in a mobile device GPU, achieving on average $1.23\times$ faster rendering.

The rise of new video modalities like virtual reality or autonomous driving has increased the demand for efficient multi-view video compression methods, both in terms of rate-distortion (R-D) performance and in terms of delay and runtime. While most recent stereo video compression approaches have shown promising performance, they compress left and right views sequentially, leading to poor parallelization and runtime performance. This work presents Low-Latency neural codec for Stereo video Streaming (LLSS), a novel parallel stereo video coding method designed for fast and efficient low-latency stereo video streaming. Instead of using a sequential cross-view motion compensation like existing methods, LLSS introduces a bidirectional feature shifting module to directly exploit mutual information among views and encode them effectively with a joint cross-view prior model for entropy coding. Thanks to this design, LLSS processes left and right views in parallel, minimizing latency; all while substantially improving R-D performance compared to both existing neural and conventional codecs.

Due to the low computational burden, sequential model predictive control (SMPC) methods have emerged as promising solutions for multilevel inverters (MLIs). Over the years, several SMPC approaches focusing on either load current or load voltage as the primary objective have been explored for MLIs. These methods have adopted philosophies such as cost function with weighting factors, offline switching vector selection or inherent strategies to minimize common-mode voltage (CMV). However, the literature lacks a comprehensive analysis of SMPC methods with CMV minimization and their pros and cons on system performance. To fill the research gap, this paper presents the philosophies behind various SMPC methods for four-level inverters (FLIs), along with the mathematical models utilized in SMPC implementation and case studies to highlight their performances. Finally, a comparative analysis of SMPC methods is provided, aiding researchers in selecting suitable methods for their applications.

Neural video codecs have recently become competitive with standard codecs such as HEVC in the low-delay setting. However, most neural codecs are large floating-point networks that use pixel-dense warping operations for temporal modeling, making them too computationally expensive for deployment on mobile devices. Recent work has demonstrated that running a neural decoder in real time on mobile is feasible, but shows this only for 720p RGB video.This work presents the first neural video codec that decodes 1080p YUV420 video in real time on a mobile device. Our codec relies on two major contributions. First, we design an efficient codec that uses a block-based motion compensation algorithm available on the warping core of the mobile accelerator, and we show how to quantize this model to integer precision. Second, we implement a fast decoder pipeline that concurrently runs neural network components on the neural signal processor, parallel entropy coding on the mobile GPU, and warping on the warping core. Our codec outperforms the previous on-device codec by a large margin with up to 48 % BD-rate savings, while reducing the MAC count on the receiver side by 10×. We perform a careful ablation to demonstrate the effect of the introduced motion compensation scheme, and ablate the effect of model quantization.

Video compression systems must support increasing bandwidth and data throughput at low cost and power, and can be limited by entropy coding bottlenecks. Efficiency can be greatly improved by parallelizing coding, which can be done at much larger scales with new neural-based codecs, but with some compression loss related to data organization. We analyze the bit rate overhead needed to support multiple bitstreams for concurrent decoding, and for its minimization propose a method for compressing parallel-decoding entry points, using bidirectional bitstream packing, and a new form of jointly optimizing arithmetic coding termination. It is shown that those techniques significantly lower the overhead, making it easier to reduce it to a small fraction of the average bitstream size, like, for example, less than 1% and 0.1% when the average number of bitstream bytes is respectively larger than 95 and 1,200 bytes.

In video compression, coding efficiency is improved by reusing pixels from previously decoded frames via motion and residual compensation. We define two levels of hierarchical redundancy in video frames: 1) first-order: redundancy in pixel space, i.e., similarities in pixel values across neighboring frames, which is effectively captured using motion and residual compensation, 2) second-order: redundancy in motion and residual maps due to smooth motion in natural videos. While most of the existing neural video coding literature addresses first-order redundancy, we tackle the problem of capturing second-order redundancy in neural video codecs via predictors. We introduce generic motion and residual predictors that learn to extrapolate from previously decoded data. These predictors are lightweight, and can be employed with most neural video codecs in order to improve their rate-distortion performance. Moreover, while RGB is the dominant colorspace in neural video coding literature, we introduce general modifications for neural video codecs to embrace the YUV420 colorspace and report YUV420 results. Our experiments show that using our predictors with a well-known neural video codec leads to 38% and 34% bitrate savings in RGB and YUV420 colorspaces measured on the UVG dataset.

We introduce a new class of attacks on machine learning models. We show that an adversary who can poison a training dataset can cause models trained on this dataset to leak significant private details of training points belonging to other parties. Our active inference attacks connect two independent lines of work targeting the integrity and privacy of machine learning training data.

Neural-based image and video codecs are significantly more power-efficient when weights and activations are quantized to low-precision integers. While there are general-purpose techniques for reducing quantization effects, large losses can occur when specific entropy coding properties are not considered. This work analyzes how entropy coding is affected by parameter quantizations, and provides a method to minimize losses. It is shown that, by using a certain type of coding parameters to be learned, uniform quantization becomes practically optimal, also simplifying the minimization of code memory requirements. The mathematical properties of the new representation are presented, and its effectiveness is demonstrated by coding experiments, showing that good results can be obtained with precision as low as 4 bits per network output, and practically no loss with 8 bits.

Realizing the potential of neural codecs on real-world mobile devices is a big technological challenge due to the inherent conflict between the computational complexity of deep networks and the power-constrained mobile hardware performance. We demonstrate practical feasibility by leveraging Qualcomm's innovation and technology, bridging the gap from neural network-based model simulations to operation on a mobile device powered by Snapdragon® technology. We show the first-ever inter-frame neural video decoder running on a commercial mobile phone, decompressing high-definition videos in real-time while maintaining a low bitrate and high visual quality, comparable to conventional codecs.

Partitioning algorithms play a key role in many scientific and engineering disciplines. A partitioning algorithm divides a set into a number of disjoint subsets or partitions. Often, the quality of the resulted partitions is measured by the amount of impurity in each partition, the smaller impurity the higher quality of the partitions. Let M be the number of N-dimensional elements in a set and K be the number of desired partitions, then an exhaustive search over all the possible partitions to find a minimum partition has the complexity of O(KM) which quickly becomes impractical for many applications with modest values of K and M. Thus, many approximate algorithms with polynomial time complexity have been proposed, but few provide the bounded guarantee. In this paper, we propose a linear time algorithm with bounded guarantee based on the maximum likelihood principle. Furthermore, the guarantee bound of the proposed algorithm is better than the state-of-the-art method in [1] for many impurity functions, and at the same time, for K ≥ N, the computational complexity is reduced from O(M3) to O(M).

This paper proposes a new four-level multilevel converter for medium-voltage, high-power applications. The proposed topology requires a lesser number of components, eliminates series connection of devices, and suitable for back-to-back operation due to the presence of common DC-link. Furthermore, the proposed topology does not require isolated DC source thereby the phase-shifting transformer can be eliminated to reduce the system cost and size. In addition, a simple capacitor voltage balancing approach based on multicarrier-based pulse width modulation scheme is proposed. The proposed approach selects a right switching state to maintain each flying capacitor voltage at their nominal value by considering the current direction, instantaneous value of capacitor voltage, and voltage level. The performance of the proposed four-level multilevel converter and voltage balancing approach is verified through simulations at steady-state and transient operating conditions.

Homography estimation is an important step in many computer vision problems. Recently, deep neural network methods have shown to be favorable for this problem when compared to traditional methods. However, these new methods do not consider dynamic content in input images. They train neural networks with only image pairs that can be perfectly aligned using homographies. This paper investigates and discusses how to design and train a deep neural network that handles dynamic scenes. We first collect a large video dataset with dynamic content. We then develop a multi-scale neural network and show that when properly trained using our new dataset, this neural network can already handle dynamic scenes to some extent. To estimate a homography of a dynamic scene in a more principled way, we need to identify the dynamic content. Since dynamic content detection and homography estimation are two tightly coupled tasks, we follow the multi-task learning principles and augment our multi-scale network such that it jointly estimates the dynamics masks and homographies. Our experiments show that our method can robustly estimate homography for challenging scenarios with dynamic scenes, blur artifacts, or lack of textures.

Abstract Novel view synthesis from sparse and unstructured input views faces challenges like the difficulty with dense 3D reconstruction and large occlusion. This paper addresses these problems by estimating proper appearance flows from the target to input views to warp and blend the input views. Our method first estimates a sparse set 3D scene points using an off‐the‐shelf 3D reconstruction method and calculates sparse flows from the target to input views. Our method then performs appearance flow completion to estimate the dense flows from the corresponding sparse ones. Specifically, we design a deep fully convolutional neural network that takes sparse flows and input views as input and outputs the dense flows. Furthermore, we estimate the optical flows between input views as references to guide the estimation of dense flows between the target view and input views. Besides the dense flows, our network also estimates the masks to blend multiple warped inputs to render the target view. Experiments on the KITTI benchmark show that our method can generate high quality novel views from sparse and unstructured input views.

A projector is usually coupled with a dedicated projection surface to properly display visual information. This prevents the application of projection in places where a dedicated projection surface is not readily available. This paper presents a method for automatically detecting a good surface in a daily living and working space to support improvisatory projection without a pre-installed projection surface. Our method uses a projector-camera system that scans an environment and evaluates the quality of the environment surface for visual projection in two steps. Our method first excludes non-planar or highly-textured surface through epipolar geometry analysis and texture analysis. For a surface that passes the first test, our method further evaluates its quality for visual projection by quickly projecting the sampled projection content onto the surface and measuring the quality of the projected visual content. Our experiment shows that our method can reliably identify a good surface in a daily environment for high-quality visual projection.

Dynamically updateable and memory-efficient search structures for Internet protocol (IP) lookup have lately attracted a great deal of attention from the researchers. In this paper, we focus on the next-generation IPv6 routing protocol comprising large and sparsely distributed routing tables. The existing data structures either suffer from inefficient resource and memory usage (trie-based algorithms), or require complicated construction processes such as converting routing prefixes into their longer representatives and sorting (tree-based algorithms), or both. We propose a novel data structure denoted value-coded trie (VC-trie) for IP lookup. VC-trie provides significant memory saving in comparison with that of the existing solutions in both IPv4 and IPv6 domains. Thereby, our structure can support longer prefix lengths and larger routing tables. We also design an static random access memory (SRAM)-based pipelined architecture to assist the VC-trie structure to improve the throughput. The architecture is implemented utilizing a state-of-the-art field programmable gate array (FPGA) device and sustainable throughput of 448 million lookups per second (with a routing table consisting of 324 K prefixes) is achieved. Furthermore, the architecture can be enhanced with external SRAMs to relax the limitations of the existing FPGA device in on-chip memory.

In this paper, we revisit the classic Fast Fourier Transform (FFT) for energy efficient designs on FPGAs. A parameterized FFT architecture is proposed to identify the design trade-offs in achieving energy efficiency. We first perform design space exploration by varying the algorithm mapping parameters, such as the degree of vertical and horizontal parallelism, that characterize decomposition based FFT algorithms. Then we explore an energy efficient design by empirical selection on the values of the chosen architecture parameters, including the type of memory elements, the type of interconnection network and the number of pipeline stages. The trade offs between energy, area, and time are analyzed using two performance metrics: the energy efficiency (defined as the number of operations per Joule) and the Energy×Area×Time (EAT) composite metric. From the experimental results, a design space is generated to demonstrate the effect of these parameters on the various performance metrics. For N-point FFT (16 ≤ N ≤ 1024), our designs achieve up to 28% and 38% improvement in the energy efficiency and EAT, respectively, compared with a state-of-the-art design.

Energy efficiency has emerged as one of the key performance metrics. In this work, we first implement a baseline architecture for matrix multiplication, parameterized with the number of processing elements and the types of storage memory. We map this architecture onto a state-of-the-art Field Programmable Gate Array (FPGA). A design space is generated to demonstrate the effect of these parameters on the energy efficiency (defined as number of operations per Joule). We determine that on-chip memory constitutes the largest amount of power consumption among all the components. To improve energy performance, we propose a memory activation schedule. Using this scheme, the proposed optimized design achieves 2.2x and 1.33x improvement with respect to Energy×Area×Time (EAT) and energy efficiency, respectively, compared with the state-of-the-art matrix multiplication core.

Eye blink is a quick action of closing and opening of the eyelids. Eye blink detection has a wide range of applications in human computer interaction and human vision health care research. Existing approaches to eye blink detection often cannot suit well resource-limited eye blink detection platforms like Smart Glasses, which have limited energy supply and typically cannot afford strong imaging and computational capabilities. In this paper, we present an efficient and robust eye blink detection method for Smart Glasses. Our method first employs an eigen-eye approach to detect closing-eye in individual video frames. Our method then learns eye blink patterns based on the closing-eye detection results and detects eye blinks using a Gradient Boosting method. Our method further uses a non-maximum suppression algorithm to remove repeated detection of the same eye-blink action among consecutive video frames. Experiments with our prototyped smart glasses equipped with a low-power camera and an embedded processor show an accurate detection result (with more than 96% accuracy) on video frames of a small size of 16 × 12 at 96 fps, which enables a number of applications in health care, driving safety, and human computer interaction.

In Network Intrusion Detection Systems (NIDSs), string pattern matching demands exceptionally high performance to match the content of network traffic against a predefined database (or dictionary) of malicious patterns. Much work has been done in this field; however, most of the prior work results in low memory efficiency (defined as the ratio of the amount of the required storage in bytes and the size of the dictionary in number of characters). Due to such inefficiency, state-of-the-art designs cannot support large dictionaries without using high-latency external DRAM. We propose an algorithm called "leaf-attaching" to preprocess a given dictionary without increasing the number of patterns. The resulting set of postprocessed patterns can be searched using any tree-search data structure. We also present a scalable, high-throughput, Memory-efficient Architecture for large-scale String Matching (MASM) based on a pipelined binary search tree. The proposed algorithm and architecture achieve a memory efficiency of 0.56 (for the Rogets dictionary) and 1.32 (for the Snort dictionary). As a result, our design scales well to support larger dictionaries. Implementations on 45 nm ASIC and a state-of-the-art FPGA device (for latest Rogets and Snort dictionaries) show that our architecture achieves 24 and 3.2 Gbps, respectively. The MASM module can simply be duplicated to accept multiple characters per cycle, leading to scalable throughput with respect to the number of characters processed in each cycle. Dictionary update involves simply rewriting the content of the memory, which can be done quickly without reconfiguring the chip.

Hierarchical search structures for packet classification offer good memory performance and support quick rule updates when implemented on multi-core network processors. However, pipelined hardware implementation of these algorithms has two disadvantages: (1) backtracking which requires stalling the pipeline and (2) inefficient memory usage due to variation in the size of the trie nodes. We propose a clustering algorithm that can partition a given rule database into a fixed number of clusters to eliminate back-tracking in the state-of-the-art hierarchical search structures. Furthermore, we develop a novel ternary trie data structure (T∈). In T∈structure, the size of the trie nodes is fixed by utilizing ∈-branch property, which overcomes the memory inefficiency problems in the pipelined hardware implementation of hierarchical search structures. We design a two-stage hierarchical search structure consisting of binary search trees in Stage 1, and T∈structures in Stage 2. Our approach demonstrates a substantial reduction in the memory footprint compared with that of the state-of-the-art. For all publicly available databases, the achieved memory efficiency is between 10.37 and 22.81 bytes of memory per rule. State-of-the-art designs can only achieve the memory efficiency of over 23 byte/rule in the best case. We also propose a SRAM-based linear pipelined architecture for packet classification that achieves high throughput. Using a state-of-the-art FPGA, the proposed design can sustain a 418 million packets per second throughput or 134 Gbps (for the minimum packet size of 40 Bytes). Additionally, our design maintains packet input order and supports in-place non-blocking rule updates.

Simplicity refers to one of the most important photography composition rules. Simplicity states that simplifying the image background can draw viewers' attention to the subject of interest in a photograph and help them better comprehend and appreciate it. Understanding whether a photo respects photography rules or not facilitates photo quality assessment. In this paper, we present a method to automatically detect whether a photo is composed according to the rule of simplicity. We design features according to the definition, implementation and effect of the rule. First, we make use of saliency analysis to infer the subject of interest in a photo and measure its compactness. Second, we segment an image into background and foreground and measure the homogeneity within the background as another feature. Third, when looking at an image created with the rule of simplicity, different viewers tend to agree on what the subject of interest is in this photo. We accordingly measure the consistency among various saliency detection results as a feature. We experiment with these features in a range of machine learning methods. Our experiments show that our methods, together with these features, provide an encouraging result in detecting the rule of simplicity in a photo.

Memory efficiency and dynamically updateable data structures for Internet Protocol (IP) lookup have regained much interest in the research community. In this paper, we revisit the classic tree-based approach for solving the longest prefix matching (LPM) problem used in IP lookup. In particular, we target our solutions for a class of large and sparsely distributed routing tables, such as those potentially arising in the next-generation IPv6 routing protocol. Due to longer prefix lengths and much larger address space, preprocessing such routing tables for tree-based LPM can significantly increase the number of prefixes and/or memory stages required for IP lookup. We propose a prefix partitioning algorithm (DPP) to divide a given routing table into k groups of disjoint prefixes (k is given). The algorithm employs dynamic programming to determine the optimal split lengths between the groups to minimize the total memory requirement. Our algorithm demonstrates a substantial reduction in the memory footprint compared with those of the state of the art in both IPv4 and IPv6 cases. Two proposed linear pipelined architectures, which achieve high throughput and support incremental updates, are also presented. The proposed algorithm and architectures achieve a memory efficiency of 1 byte of memory for each byte of prefix for both IPv4 and IPv6. As a result, our design scales well to support either larger routing tables, longer prefix lengths, or both. The total memory requirement depends solely on the number of prefixes. Implementations on 45 nm ASIC and a state-of-the-art FPGA device (for a routing table consisting of 330K prefixes) show that our algorithm achieves 980 and 410 million lookups per second, respectively. These results are well suited for 100 Gbps lookup. The implementations also scale to support larger routing tables and longer prefix length when we go from IPv4 to IPv6. Additionally, the proposed architectures can easily interface with external SRAMs to ease the limitation of on-chip memory of the target devices.

Network router virtualization has recently gained much interest in the research community, as it allows multiple virtual router instances to run on a common physical router platform. The key metrics in designing network virtual routers are (1) number of supported virtual router instances, (2) total number of prefixes, and (3) ability to quickly update the virtual table. Existing merging algorithms use leaf pushing and a shared next hop data structure to eliminate the large memory bandwidth requirement. However, the size of the shared next hop table grows linearly with the number of virtual routers. Due to the limited amount of on-chip memory and the number of I/O pins of Field Programmable Gate Arrays (FPGAs), existing designs cannot support large number of tables and/or large number of prefixes. This paper exploits the abundant parallelism and on-chip memory bandwidth available in the state-of-the-art FPGAs, and proposes a compact trie representation and a hybrid data structure to reduce the memory requirement of virtual routers. The approach does not require leaf-pushing; therefore, reduces the size of each entry of the data structure. Our algorithm demonstrates substantial reduction in the memory footprint compared with the state-of-the-art. Also, it eliminates the shared next hop information data structure and simplifies the table updates in virtual routers. Using a state-of-the-art FPGA, the proposed architecture can support up to 3.1M IPv4 prefixes. Employing the dual-ported memory available in the current FPGAs, we map the proposed data structure on a novel SRAMbased linear pipeline architecture to achieve high throughput. The post place-and-route result shows that our architecture can sustain a throughput of 394 million lookups per second, or 126 Gbps (for the minimum packet size of 40 Bytes).

Virtualization allows heterogeneous networks to share same underlying physical substrate. A single router fulfills the roles of multiple virtual routers by maintaining all the forwarding tables. Therefore, multiple virtual routers can run on a single substrate router, and multiple organizations share this single physical router. A single router plays the role of multiple independent virtual routers while providing the necessary isolation and resource management. Hence, a virtual router should fulfill the following requirements:\begin{itemize} \item \emph{Fair resource usage}: Router resources should be fairly shared among the virtual routers. \item \emph{Fault isolation}: A fault occurring in a particular virtual network should not affect the operation of other virtual networks. \item \emph{Security}: Traffic from one virtual network should not be mixed with the traffic from any other virtual network. \end{itemize}Storing these virtual routing tables separately leads to a large memory requirement and poor resource sharing. Therefore, merging emerges to be a desirable solution. Existing merging algorithms use leaf pushing technique and a shared next hop data structure to eliminate the large memory bandwidth requirement. However, the size of the shared next hop table grows linearly with the number of virtual routers. Due to the limited on-chip memory and the number of I/O pins of Field Programmable Gate Arrays (FPGAs), existing designs cannot support large number of virtual routing tables and/or large number of prefixes. We propose a compact trie representation and a hybrid data structure to reduce the memory consumption of a single virtual router. Our data structure achieves substantial memory compression without the need for backtracking. The proposed hybrid data structure is also used to merge different virtual routing tables. The approach does not require leaf-pushing and reduces the size of each entry of the data structure. Our data structure achieves a substantial memory reduction of $2\times$ over the state-of-the-art solutions. Additionally, it eliminates the shared next hop information structure and simplifies the table updates in virtual routers. Using a state-of-the-art FPGA, the proposed architecture can support up to 3.1M IPv4 prefixes, employing both on-chip BRAM and external SRAM. Our implementation on FPGA shows a sustained throughput of $394$ million lookups per second. In summary, this paper makes the following contributions:\begin{enumerate} \item A compact trie representation and a hybrid data structure for IP lookup that reduces the memory consumption without backtracking. \item A merging algorithm that eliminates leaf pushing and simplifies the table updates in virtual routers. \item A linear pipelined SRAM-based architecture on FPGAs that can support up to $3.1$M prefixes, while achieving a sustained throughput of $394$ million lookups per second.\end{enumerate}

Memory efficiency with compact data structures for Internet Protocol (IP) lookup has recently regained much interest in the research community. In this paper, we revisit the classic trie-based approach for solving the longest prefix matching (LPM) problem used in IP lookup. In particular, we target our solutions for a class of large and sparsely-distributed routing tables, such as those potentially arising in the next generation IPv6 routing protocol. Due to longer prefix and much larger address space, straight-forward implementation of trie based LPM can significantly increase the number of nodes and/or memory required for IP lookup. Additionally, due to the available on-chip memory and the number of I/O pins of in the state-of-the art Field Programmable Gate Arrays (FPGAs), existing designs cannot support large IPv6 routing tables consisting of over 300K prefixes. We propose two algorithms to compress the uni-bit-trie representation of a given routing table: (1) single-prefix distance bounded path compression and (2) multiple-prefix distance bounded path compression. These algorithms determine the optimal maximum slap distance at each node of the trie to minimize the total memory requirement. Our algorithms demonstrate substantial reduction in the memory footprint compared with the uni-bit-trie algorithm (1.86× for IPv4 and 6.16× for IPv6), and with the original path compression algorithm (1.77× for IPv4 and 1.53× for IPv6). Furthermore, implementation on a state of-the-art FPGA shows that our algorithms achieve 466 million lookups per second and are well suited for 100Gbps lookup. The implementation also scales to support larger routing tables and longer prefix length when we go from IPv4 to IPv6.

Router virtualization has recently gained much interest in the research community. It allows multiple virtual router instances to run on a common physical router platform. The key metrics in designing network virtual routers are: (1) number of supported virtual router instances, (2) total number of prefixes, and (3) ability to quickly update the virtual table. Limited on-chip memory in FPGA leads to the need for memory-efficient merging algorithms. On the other hand, due to high frequency of combined updates from all the virtual routers, the merging algorithms must be highly efficient. Hence, the router must support quick updates. In this paper, we propose a simple merging algorithm whose performance is not sensitive to the number of routing tables considered. The performance solely depends on the total number of prefixes. We also propose a novel scalable, high-throughput linear pipeline architecture for IP-lookup that supports large virtual routing tables and quick non-blocking update. Using a state-of-the-art Field Programmable Gate Array (FPGA) along with external SRAM, the proposed architecture can support up to 16M IPv4 and 880K IPv6 prefixes. Our implementation shows a sustained through-put of 400 million lookups per second, even when external SRAM is used.

Most current SRAM-based high-speed Internet Protocol (IP) packet classification implementations use tree traversal and pipelining. However, these approaches result in inefficient memory utilization. Due to the limited amount of on-chip memory of the state-of-the-art Field Programmable Gate Arrays (FPGAs), existing designs cannot support large filter databases arising in backbone routers and intrusion detection systems. Hierarchical search structures for packet classification exhibit good memory performance and support quick rule update. However, pipelined hardware implementation of these algorithms suffer from inefficient resource and memory usage due to variation in the size of the trie nodes and backtracking. We propose a memory efficient organization denoted Clustered Hierarchical Search Structure (CHSS) for packet classification. We present a clustering algorithm that partitions a given filter database to reduce the memory requirement. We show that, using the resulting structure, backtracking is not needed to perform a search. We introduce two parameters (NRtrie, NRtree), which can be chosen based on the given filter database to achieve good memory efficiency. Our algorithm demonstrates substantial reduction in the memory footprint compared with the state-of-the-art. For all publicly available filter databases, the achieved memory efficiency is between 21.54 and 41.25 bytes per rule. We map the proposed data structure onto a linear pipeline architecture to achieve high throughput. Post place and route result using a state-of-the-art FPGA device shows that the design can sustain a throughput of 408 million packets per second, or 130.5 Gbps (for the minimum packet size of 40 Bytes).

Recently, router virtualization has gained much interest in networking community. However, hardware support for router virtualization is still in its primitive stages. One of the major problems in a virtualized router is how to support frequent routing table updates efficiently, without interrupting network traffic. In this paper, we propose a Field Programmable Gate Array (FPGA) based architecture for router virtualization that supports on-the-fly updates, while ensuring scalability and throughput requirements. We introduce a distance-based mapping technique named Fill-In to merge multiple virtual routing tables into a single search tree. Node sharing is avoided by using a uniform data structure that results in a scalable solution for router virtualization. The reconfigurability and abundant parallelism of FPGAs make them a desirable hardware platform for high-performance and cost-effective routers. We leverage the features of modern FPGA devices to implement a parallel-linear-pipelined packet processing engine. Our post place-and route results show that the proposed architecture can support uninterrupted network traffic at 150 Gbps for minimum size (40 Byte) packets. The scalability of the architecture is demonstrated for up to 17 real routing tables. Using the proposed update techniques, our architecture handles an update with a single write bubble.

The rule of thirds is one of the most important composition rules used by photographers to create high-quality photos. The rule of thirds states that placing important objects along the imagery thirds lines or around their intersections often produces highly aesthetic photos. In this paper, we present a method to automatically determine whether a photo respects the rule of thirds. Detecting the rule of thirds from a photo requires semantic content understanding to locate important objects, which is beyond the state of the art. This paper makes use of the recent saliency and generic objectness analysis as an alternative and accordingly designs a range of features. Our experiment with a variety of saliency and generic objectness methods shows that an encouraging performance can be achieved in detecting the rule of thirds from photos.

In Network Intrusion Detection Systems (NIDSs), string matching demands exceptionally high performance to match the content of network traffic against a predefined database of malicious patterns. Much work has been done in this field; however, they result in low memory efficiency. Due to the available on-chip memory and the number of I/O pins of Field Programmable Gate Arrays (FPGAs), state-of-the-art designs cannot support large dictionaries without using high-latency external DRAM. We propose a novel Memory efficient Architecture for large-scale String Matching (MASM), based on pipelined binary search tree. With memory efficiency close to 1 byte/char, MASM can support a dictionary of over 4 MBytes, using a single FPGA device. The architecture can also be easily partitioned, so as to use external SRAM to handle even larger dictionaries of over 8 MBytes. Our implementation results show a sustained throughput of 3.5 Gbps, even when external SRAM is used. The MASM module can be simply duplicated to accept multiple characters per cycle, leading to scalable throughput with respect to the number of characters processed in each cycle. Dictionary update involves only rewriting the memory content, which can be done quickly without reconfiguring the chip.

Network Intrusion Detection Systems (NIDSs) have emerged as powerful tools for detecting and preventing malicious attacks over both the Internet and Intranet. String matching, which is one of the most important functions of NIDS, demands exceptionally high performance to match the content of network traffic against a predefined database of malicious patterns. Much work has been done in this field; however, they result in low memory efficiency\footnote{The memory efficiency (in bytes/char) is defined as the ratio of the amount of the required storage memory (in bytes), and the size of the dictionary (number of characters).}. Due to the available on-chip memory and the number of I/O pins of Field Programmable Gate Arrays (FPGAs), state-of-the-art designs cannot support large dictionaries without using high-latency external DRAM. We propose a novel Memory efficient Architecture for large-scale String Matching, namely MASM, based on pipelined binary search tree. Our design provides a high-throughput matching module, which can be used as the building block to process arbitrary-length patterns. With memory efficiency close to 1 byte/char, MASM can support a dictionary\footnote The size of a dictionary is the total number of characters in all the patterns in the dictionary. of over 4 MB (regardless of the size of the alphabet), using a single state-of-the-art FPGA device. This efficiency is comparable to that of a Ternary Content Addressable Memory (TCAM)-based solution. The architecture can also be easily partitioned, so as to use external SRAM to handle even larger dictionaries of over 8 MB. Our implementation results show a sustained throughput of 3.2 Gbps, even when external SRAM is used. The MASM module can be simply duplicated to accept multiple characters per cycle, leading to scalable throughput with respect to the number of characters processed in each cycle. Dictionary update involves only rewriting the memory content, which can be done quickly without reconfiguring the chip.

Advances in optical networking technology are pushing internet link rates up to 100 Gbps. Such line rates demand a throughput of over 150 million packets per second at core routers. Along with the increase in link speed, the size of the dynamic routing table of these core routers is also increasing at the rate of 25-50 K additional prefixes per year. These dynamic tables require high prefix deletion and insertion rates. Therefore, rapid prefix update without disrupting router operation has also emerged as a critical requirement. Furthermore, IPv6 standard extends the current IPv4 prefix length from 32 to 128 bits. Thus, it is a major challenge to scale the existing solutions to simultaneously support increased throughput, table size, prefix length and rapid update. While the existing solutions can achieve high throughput, they cannot support large routing tables and rapid update at the same time. We propose a novel scalable, high-throughput linear pipeline architecture for IP-lookup that supports large routing tables and single-cycle non-blocking update. Using a state-of-the-art Field Programmable Gate Arrays (FPGA) along with external SRAM, the proposed architecture can support over 2M prefixes. Our implementation shows a throughput of 348 millions lookups per second, even when external SRAM is used.

Dictionary-Based String Matching (DBSM) is used in network Deep Packet Inspection (DPI) applications virus scanning and network intrusion detection. We propose the Pipelined Affix Search with Tail Acceleration (PASTA) architecture for solving DBSM with guaranteed worst-case performance. Our PASTA architecture is composed of a Pipelined Affix Search Relay (PASR) followed by a Tail Acceleration Finite Automaton (TAFA). PASR consists of one or more pipelined Binary Search Tree (pBST) modules arranged in a linear array. TAFA is constructed with the Aho-Corasick goto and failure functions in a compact multi-path and multi-stride tree structure. Both PASR and TAFA achieve good memory efficiency of 1.2 and 2 B/ch (bytes per character) respectively and are pipelined to achieve a high clock rate of 200 MHz on FPGAs. Because PASTA does not depend on the effectiveness of any hash function or the property of the input stream, its performance is guaranteed in the worst case. Our prototype implementation of PASTA on an FPGA with 10 Mb on-chip block RAM achieves 3.2 Gbps matching throughput against a dictionary of over 700 K characters. This level of performance surpasses the requirements of next-generation security gateways for deep packet inspection.

A novel portable hardware architecture of the Elliptic Curve Method of factoring, designed and optimized for application in the relation collection step of the Number Field Sieve, is described and analyzed. A comparison with an earlier proof-of-concept design by Pelzl et al. has been performed, and a substantial improvement has been demonstrated in terms of both the execution time and the area-time product. The ECM architecture has been ported across five different families of FPGA devices in order to select the family with the best performance to cost ratio. A timing comparison with the highly optimized software implementation, GMP-ECM, has been performed. Our results indicate that low-cost families of FPGAs, such as Spartan-3 and Spartan-3E, offer at least an order of magnitude improvement over the same generation of microprocessors in terms of the performance to cost ratio, without the use of embedded FPGA resources, such as embedded multipliers.

Most high-speed Internet Protocol (IP) lookup implementations use tree traversal and pipelining. Due to the available on-chip memory and the number of I/O pins of Field Programmable Gate Arrays (FPGAs), state-of-the-art designs cannot support the current largest routing table(consisting of 257 K prefixes in backbone routers). We propose a novel scalable high-throughput, low-power SRAM-based linear pipeline architecture for IP lookup. Using a single FPGA, the proposed architecture can support the current largest routing table, or even larger tables of up to 400 K prefixes. Our architecture can also be easily partitioned, so as to use external SRAM to handle even larger routing tables (up to 1.7 M prefixes). Our implementation shows a high throughput (340 mega lookups per second or 109 Gbps), even when external SRAM is used. The use of SRAM (instead of TCAM) leads to an order of magnitude reduction in power dissipation. Additionally, the architecture supports power saving by allowing only a portion of the memory to be active on each memory access. Our design also maintains packet input order and supports in-place non-blocking route updates.

Internet Protocol (IP) lookup in routers can be implemented by some form of tree traversal. Pipelining can dramatically improve the search throughput. However, it results in unbalanced memory allocation over the pipeline stages. This has been identified as a major challenge for pipelined solutions. In this paper, an IP lookup rate of 325 MLPS (millions lookups per second) is achieved using a novel SRAM-based bidirectional optimized linear pipeline architecture on Field Programmable Gate Array, named BiOLP, for tree-based search engines in IP routers. BiOLP can also achieve a perfectly balanced memory distribution over the pipeline stages. Moreover, by employing caching to exploit the Internet traffic locality, BiOLP can achieve a high throughput of up to 1.3 GLPS (billion lookups per second). It also maintains packet input order, and supports route updates without blocking subsequent incoming packets.

Most high-speed Internet Protocol (IP) lookup implementations use tree traversal and pipelining. However, this approach results in inefficient memory utilization. Due to available on-chip memory and pin limitations of FPGAs, state-of-the-art designs on FPGAs cannot support large routing tables arising in backbone routers. Therefore, ternary content addressable memory (TCAM) is widely used. We propose a novel SRAM-based linear pipeline architecture, named DuPI. Using a single Virtex-4, DuPI can support a routing table of up to 228 K prefixes, which is 3times the state-of-the-art. Our architecture can also be easily partitioned, so as to use external SRAM to handle even larger routing tables (up to 2 M prefixes), while maintaining a 324 MLPS throughput. The use of SRAM (instead of TCAM) leads to orders of magnitude of reduction in power dissipation. Employing caching to exploit Internet traffic locality, we can achieve a throughput of 1.3 GLPS (billion lookups per second). Our design also maintains packet input order, and supports in-place non-blocking route updates.