Frequency diverse array synthetic aperture radar (FDA-SAR) has been recognized as a potential technique for high-resolution wide-swath imaging, and employs a slight frequency increment to the transmitting sub-apertures/channels to form a range-dependent beam pattern for swath widening. The key challenge of FDA-SAR is echo separation at the receiver. Former studies have employed quasi-orthogonal waveforms with low cross-correlation energies as the transmitted signals, which can not ensure reliable echo separation, especially for distributed scatterers. In this letter an FDA-SAR processing framework is proposed based on the segmented phase code waveform. An encoding scheme and a notch-expanding receiving beamformer based on convex optimization are used for precise echo separation. Then, using the degrees of freedom on transmitting, a beamformer is applied to resolve range ambiguities. The proposed scheme avoids the performance deterioration caused by the echo separation issue, even for distributed scatterers, while allowing other multiple-channel techniques to be integrated. Simulation experiments have validated the effectiveness of the proposed scheme.

Digital beamforming (DBF) with scan-on-receive (SCORE) is a state-of-the-art technique for high-resolution and wide-swath observation in synthetic aperture radar (SAR) imaging. However, this technique has a problem with frequency dispersion. Most existing DBF methods treat the frequency as a constant, and little work has been devoted to the analysis of frequency dispersion. To address this shortcoming, this letter analyzes the dispersion effect using the raw data of two digital beamforming synthetic aperture radars (DBF-SARs). Moreover, this letter demonstrates in practice for the first time the effectiveness of the digital scalloped beamforming (DSBF), one of the candidates for solving the frequency dispersion problem in SCORE DBF, using the X-band sixteen-channel DBF-SAR and the C-band four-channel DBF-SAR. The analyses of the impulse response width extension and the signal-to-noise ratio (SNR) improvement are presented in detail to show the performance of the frequency dispersion compensation. The results show that the DSBF is effective in compensating for the frequency dispersion in the DBF technique. With DSBF, a higher SNR improvement can be obtained in DBF-SARs.

High-resolution wide swath (HRWS) imaging has become a research hotspot in the field of synthetic aperture radars (SARs) in recent years. Digital beamforming (DBF) denotes a powerful technique for obtaining the HRWS images. The DBF can be implemented in a spaceborne SAR in real time but at the cost of massive digital resources. To solve this problem, an intermediate frequency (IF) DBF real-time processing architecture has been proposed, but it has never been validated in practice. In this letter, the X-band 16-channel DBF-SAR and the C-band four-channel DBF-SAR are used to demonstrate the effectiveness of the IFDBF scheme in practice for the first time. The performance of the IFDBF scheme is verified by the detailed analysis of the signal-to-noise ratio (SNR) improvement and system complexity. The result shows that the IFDBF scheme can achieve the same SNR improvement as the DBF scheme. Moreover, by applying the IFDBF scheme, a spaceborne SAR can save more digital resources in real-time processing.

Methods for summarizing and diversifying query results have drawn significant attention recently, because they help present query results with lots of tuples to users in more informative ways. We present QAGView (Quick AGgregate View), which provides a holistic overview of high-valued aggregate query answers to the user in the form of summaries (showing high-level properties that emerge from subsets of answers) with coverage guarantee (for a user-specified number of top-valued answers) that is both diverse (avoiding overlapping or similar summaries) and relevant (focusing on high-valued aggregate answers). QAGView allows users to view the high-level summaries as clusters, and to expand individual clusters for their constituent result tuples. Users can fine-tune the behavior of QAGView by specifying a number of parameters according their preference. To help users choose appropriate parameters interactively, QAGView employ a suite of optimizations that enable quick preview of how the quality of the summaries changes over wide ranges of parameter settings, as well as real-time visualization of how the summaries evolve in response to parameter updates.

We present a system for summarization and interactive exploration of high-valued aggregate query answers to make a large set of possible answers more informative to the user. Our system outputs a set of clusters on the high-valued query answers showing their common properties such that the clusters are diverse as much as possible to avoid repeating information, and cover a certain number of top original answers as indicated by the user. Further, the system facilitates interactive exploration of the query answers by helping the user (i) choose combinations of parameters for clustering, (ii) inspect the clusters as well as the elements they contain, and (iii) visualize how changes in parameters affect clustering. We define optimization problems, study their complexity, explore properties of the solutions investigating the semi-lattice structure on the clusters, and propose efficient algorithms and optimizations to achieve these goals. We evaluate our techniques experimentally and discuss our prototype with a graphical user interface that facilitates this interactive exploration. A user study is conducted to evaluate the usability of our approach.

The fast retrieval in archival traffic data is essential for network security and forensic analysis. A bitmap index is a data structure enabling fast search over large data collections in a limited time, but the space consumption is always a problem. WAH, PLWAH and COMPAX are proposed for compressing bitmap indexes for less storage. In this paper, a new bitmap index encoding scheme, named MASC, is proposed to further improve the compression ratio without impairing the query performance. Instead of being limited to a fixed length (31 bits) in PLWAH and COMPAX, the stride size can be as long as possible to encode consecutive zero bits and nonzero bits in a more compact way. Instead of piggyback used in PLWAH, a new structure in MASC called carrier is introduced as piggyback in PLWAH only carries an individual nonzero bit. We also generalize the traditional literal word concept in PLWAH and COMPAX. The validity of MASC encoding scheme is demonstrated with the application in Internet Traffic Archival system. Based on experiments with real Internet traffic data set from CAIDA, MASC has a better compression ratio than PLWAH and COMPAX2 without the penalty in query performance.

Bitmap index is widely used in archiving and searching of Internet traffic, which is an essential step for analyzing network events in the field of network forensics. However, bitmap index requires a large storage space for fast searching in archival data. As current state-of-the-art bitmap index compression techniques, various encoding algorithms have been proposed, e.g. WAH, PLWAH, COMPAX, etc. With the advantages of fast query speed and easy implementation, PLWAH is an outstanding encoding scheme to encode the sparse dirty bits in bitmap index. Unfortunately, for searching Internet traffic, the constructed bitmap index can be quite dense locally according to the statistics. This is because that Internet traffic are usually composed of the flows with the same five tuple (SrcIP, SrcPort, DstIP, DstPort, proto). In this paper, SPLWAH is proposed to adapt to Internet traffic based on PLWAH. In SPLWAH, a new codebook is introduced to fit the characteristics of Internet traffic. We also conduct several performance evaluation experiments based on real network flow data from CAIDA. The results show that SPLWAH reduces the space consumption with a factor of 20% or more without incurring extra encoding and decoding cost. This work also shows that the design space in bitmap index compression is still a fruitful unknown frontier and worth further exploring to adapt to the emerging data spaces.

Bitmap indexing is flexible to conduct boolean operations in data retrieval. Besides, the query processing based on bitmap indexing is also very fast. Therefore it has been widely used in various big data analytics platforms, such as Druid and Spark etc. However, bitmap index can consume a large amount of memory, which leads to the invention of different kinds of bitmap index compression algorithms without sacrificing temporal performance. In practice, we are often discommoded by choosing a proper algorithm when handling specific problems. Besides, after devising a new algorithm that may outperform existing ones, it is essential to evaluate its performance in theory. Without appropriate theoretical analysis, the deficit of a new algorithm can only be spotted until final experimental results are drawn, thus wasting much time and effort. In this paper, we propose a general analytical model to analyze both the spatial and temporal performance for bitmap index compression algorithms, which can be applied to analyze all kinds of algorithms derived from WAH (word-aligned hybrid). In this model, two types of distributed bitmaps, uniformly distributed bitmaps and clustered bitmaps, are used separately. In order to illustrate this model, several bitmap index compression algorithms are analyzed and compared with each other. Algorithms herein are COMBAT (COMbining Binary And Ternary encoding), SECOMPAX (Scope Extended COMPAX) and CONCISE (Compressed 'n' Composable Integer Set), which are all derived from WAH. Evaluation results by MATLAB simulation about these algorithms are also presented. This paper paves the way for further researches on the performance evaluation of various bitmap index compression algorithms in the future.

Archiving of the Internet traffic is essential for analyzing network events in the field of network security and network forensics. The bitmap index is widely used to achieve fast searching in archival traffic data requiring a large storage space. As current state-of-art, WAH, PLWAH and COMPAX are proposed for compressing bitmap indexes. In this paper, a new bitmap index compression scheme, named PLWAH+ (Position List Word Aligned Hybrid Plus), is introduced, based on PLWAH. With less storage consumption, PLWAH+ is more suitable for indexing in large-scale and high-speed network traffic.

Archiving of Internet traffic is essential for analyzing network events in the field of network security. Currently, bitmap indexing is used to accelerate the indexing and search queries for archival traffic data. However, the generation of bitmap index needs large storage space, which makes bitmap index compression is a must-have function. In this paper, we propose a new bitmap index encoding algorithm named SECOMPAX (Scope-Extended COMPressed Adaptive indeX), which performs better compression ratio and fast encoding speed compared with the state-of-art bitmap index compression algorithm WAH (Word-Aligned-Hybrid), PLWAH(Position list word aligned hybrid) and COMPAX (COMPressed Adaptive indeX). The comparison among WAH, PLWAH, COMPAX and SECOMPAX shows that SECOMAX accomplishes the smallest bitmap index in size and the comparable encoding time with other three methods. We also use real Internet trace from CAIDA to prove the validity of SECOMPAX. SECOMPAX has the best compression ratio in compared with other bitmap index encoding algorithms in our experiments. The encoding time is measured, and statistics of the distribution of codeword used in SECOMPAX is also investigated in experiments. It shows that SECOMPAX's extra time consumption is acceptable as the new designed codebook work effectively in encoding bit sequence which cannot be compressed in other bitmap encoding schemes.