Abstract The payload attribution system has been proposed to analyze network traffic and assist investigators in identifying flows containing specific excerpts to locate criminals and potential victims. However, various attacks or data leakage behaviors can obscure and scatter the crucial portion of flow payloads to evade detection. Although existing payload attribution techniques strive to enhance the data reduction ratio and reduce false positive rates, research on similar payload querying is notably lacking. In this study, we introduce bitmap index table fuzzy matching (BIFM), a method for digesting network traffic to query and trace variants of malicious traffic. Unlike deterministic bitmap-index PAS that require deterministic bit co-occurrence/alignment between the query excerpt and the stored flow bitmap, an assumption violated when payloads are split or jumbled, BIFM overcomes this limitation via progressive relaxation with fuzzy matching and verification. Leveraging the bitmap index table and fuzzy matching, BIFM efficiently identifies flows containing excerpts or their variants (excerpts that change their appearance by splitting or jumbling) by relaxing the matching conditions for candidate malicious flows. To enhance BIFM’s accuracy, we also propose no-shingling and packet caching mechanisms. We extensively evaluate BIFM’s performance using a dataset constructed from real campus network IP-trace data. Our results demonstrate that BIFM outperforms existing state-of-the-art solutions, achieving an accuracy improvement of $\sim $10% without significantly increasing processing time.

Abstract The pervasive interconnection of heterogeneous routing devices forms the fundamental infrastructure of modern Internet communication, making accurate router vendor identification a critical capability for multiple domains including network topology mapping, intelligent traffic engineering, and proactive cybersecurity defense. While Internet Protocol version 6 (IPv6) has achieved widespread global deployment as the next-generation Internet protocol, the opaque nature of its addressing mechanisms and protocol behaviors has created significant challenges in router attribute detection across IPv6 networks, leaving a crucial gap in network visibility and security analytics. To address this pressing challenge, we present IPv6 Router FingerPrinting (6RFP), an innovative lightweight fingerprinting methodology that establishes a new paradigm for IPv6 router vendor identification by systematically combining two complementary analytical dimensions: (i) comprehensive EUI-64 interface identifier analysis that captures vendor-specific hardware encoding patterns embedded in IPv6 addresses, and (ii) sophisticated IPv6 Identification Field characteristic profiling that reveals distinctive vendor implementations. Through extensive evaluation across diverse network environments, 6RFP demonstrates highly effective detection capabilities, achieving 85.79% accuracy—representing a remarkable 86.01% improvement over current state-of-the-art techniques—while maintaining minimal computational overhead suitable for real-time deployment.

Internet-wide scanning is integral to network measurement and security analysis, but the expansive address space of IPv6 limits existing approaches in achieving efficient global-scale scans. This study introduces HMap, an innovative IPv6 scanner that markedly improves scan efficiency and coverage through the implementation of a dynamic search (DS) technique, relying solely on IPv6 routeable BGP prefixes. DS employs a dynamic feedback-driven probing strategy that uses information from previous replies to prioritize more promising address regions in subsequent scans. In Internet-wide scans over IPv6, encompassing both ping-like and traceroute-like scans with DS, HMap has demonstrated its capability to discover 2.29 million non-alias active target addresses, 0.13 million peripheries/middleboxes, and 1.61 million router interfaces, using only million-scale probes. This represents a noteworthy improvement of 1.91 times, 1.63 times, and 12.38 times, respectively, compared to current state-of-the-art alternatives. Additionally, by utilizing an efficient target generation algorithm (TGA) that more effectively leverages seed addresses, HMap expands the non-alias active address count to 44.05 million. This coverage spans 18.97 thousand ASes with a one-hour scan at a limited probing speed of 100 Kpps. The volume of active IPv6 addresses is 4.88 times larger than the currently disclosed largest IPv6 hitlists, providing a more diverse set of IPv6 networks. Unlike prior IPv6 scan studies that preclude their use for Internet-scale security analysis, we also conduct the Internet-wide security scans of IPv6 networks, focusing on the exposed internal IPv6 devices and security-sensitive services in IPv6 routers.

With the rapid development of 5G technology and the Internet of Things, the high-speed, low-latency, and massive connectivity capabilities have made cellular networks a critical infrastructure. However, IPv6 cellular address identification, crucial in cellular network measurement, remains a significant gap. On the one hand, the combination of IPv6 with 5G renders existing methods ineffective; on the other hand, the vast address space of IPv6 makes large-scale network scanning infeasible. To address this gap, we propose a novel IPv6 cellular network identification system, leveraging round-trip time and IPv6 interface identifiers to classify network connection types for / 48 prefixes accurately. Our method achieves an accuracy of over 99% on publicly available global datasets. Furthermore, by incorporating target generation scans, we have first achieved large-scale probing worldwide.

Distributed Denial-of-Service (DDoS) attacks represent a cost-effective and potent threat to network stability. While extensively studied in IPv4 networks, DDoS implications in IPv6 remain underexplored. The vast IPv6 address space renders brute-force scanning and amplifier testing for all active addresses impractical. Innovatively, this work investigates ASlevel vulnerabilities to reflection amplification attacks in IPv6. One prerequisite for amplification presence is that it is located in a vulnerable autonomous system (AS) without inbound source address validation (ISAV) deployment. Hence, the analysis focuses on two critical aspects: global detection of ISAV deployment and identification of amplifiers within vulnerable ASes. Specifically, we develop a methodology combining ICMP Time Exceeded mechanisms for ISAV detection, employ IPv6 address scanning for amplifier identification, and utilize dual vantage points for amplification verification. Experimental results reveal that 4,460 ASes (61.36% of measured networks) lack ISAV deployment. Through scanning approximately 47 M active addresses, we have identified reflection amplifiers in 3,507 ASes. The analysis demonstrates that current IPv6 networks are fertile ground for reflection amplification attacks, alarming network security.

Acquiring router interfaces is crucial for network measurement and security assessment. Established methods are readily available for IPv4 systems; however, efficiently pinpointing IPv6 router interfaces remains an unresolved issue, chiefly due to the vast IPv6 address space. Existing practices in this domain commonly suffer from inefficiencies. Thus, it is imperative to propose a methodology for fast enumeration of IPv6 router interfaces on a massive scale. In this study, we introduce Sweeper, a novel asynchronous IPv6 scanner that excels in discovering router interfaces from scratch, from few to many, on large-scale IPv6 networks. Unlike existing approaches, Sweeper requires merely the readily accessible IPv6 prefixes instead of seed addresses and can strategically optimize its probing direction based on a novel weighted sampling algorithm to increase the discovery rate. Real-world tests prove that Sweeper outperforms state-of-the-art works, discovering$33.2\%\sim 48.8\%$more IPv6 router interface addresses than the baselines, with same computational resources. With Sweeper, we have collected approximately 6 million IPv6 router interface addresses from a single vantage point within less than one hour.

In the global Internet, understanding paths between autonomous systems (ASes) is valuable for improving the Internet routing system and optimizing various applications. However, due to the business and privacy concerns, only a small portion of paths are disclosed. Moreover, limited by the measurement resources, obtaining paths between any two ASes is impossible. Thus, path inference becomes necessary. Recent work proposes training individual model for each AS to infer paths, but it lacks personalization as it uses a shared approach and data for arbitrary ASes. Moreover, training models from scratch for all the ASes is time-consuming and resource-intensive. This paper introduces Personalized and Adaptive Generative Measurable Path Inference (PA-GMPI), a prefix-grained path inference process. PA-GMPI is capable of achieving superior performance and faster model training by fully leveraging the exclusive information of each AS. These improvements come from a personalized path generator, a 3-layer graph kernel based adaptive training warm-starter, and a real-world walks based AS representation learner. In evaluation, PA-GMPI significantly outperforms the state-of-the-art method, achieving a maximal accuracy improvement of 28.72% and ESR (exact same ratio) improvement of 49.95%. Furthermore, PA-GMPI achieves an average reduction of 20.21% in training resource consumption across over two thousand training sessions, using vantage ASes from five snapshots, which included 439 distinct ASes.

Discovering the IPv6 network periphery, i.e., the last-hop router connecting endhosts in the IPv6 Internet, is crucial for network measurement and Internet reconnaissance. However, existing solutions commonly suffer from inefficiency when applied on a global scale due to the vast IPv6 address space. To tackle this challenge, we developed6Seeks, an innovative IPv6 scanning system designed for efficient IPv6 periphery discovery across the global IPv6 Internet without requiring seed IPv6 addresses. Specifically, we proposed to employ a heuristic method for collecting active /48 networks from the global BGP prefixes and then adopt a reinforcement learning-based dynamic probing strategy to optimize resource allocation across these networks and significantly improve efficiency. Real-world tests demonstrate that6Seeksoutperforms all existing methods in global-scale IPv6 periphery measurement experiments. In just a few hours,6Seekscan identify over 128 million IPv6 periphery devices, while using only 37% of the probing resources required by the current state-of-the-art solution. Compared to existing public datasets, the IPv6 addresses identified by6Seeksare more numerous and display unique characteristics, significantly enriching our IPv6 corpus.

Abstract Accurate and current comprehension of network status is crucial for efficient network management. Nevertheless, direct network measurement strategies entail substantial traffic overhead and demand intricate coordination among network entities, making them impractical. Network tomography, an indirect measurement approach, utilizes insights garnered from measured parts to deduce characteristics of the entire network. Past studies frequently depend on acquiring challenging-to-access information, such as the complete network topology or support from specialized protocols. Unfortunately, these constraints pose challenges in non-cooperative scenarios where obtaining such information is difficult. Recent endeavors pursue emancipating tomography from dependence on copious information, striving to predict unmeasured path performance using limited data. Nevertheless, the disparity between the measured data and actual performance has hindered the accuracy. In response, we introduce an innovative tomography framework named DRL-Tomo, designed to alleviate potential biases. DRL-Tomo initiates by generating augmented data through deep reinforcement learning, gradually approximating the genuine performance of unmeasured paths. Subsequently, a neural network model is trained using this augmented data, enabling precise inferences. Our experiments, encompassing both real-world and synthetic datasets, vividly demonstrate DRL-Tomo’s remarkable enhancement. Specifically, it achieves a substantial 10%–67% improvement in path delay prediction and an impressive 30%–98% enhancement in path loss rate prediction.

Abstract Nowadays, with the continuous increase in internet traffic, the demand for real-time and high-speed traffic analysis has grown significantly. However, existing traffic analysis technologies are either limited by specific applications or data, unable to expand for widespread implementation, or in offline mode are unable to keep up with dynamic adjustments required in certain network management scenarios. A promising approach is to utilize sketch technology to enhance real-time traffic analysis. Unfortunately, existing technologies suffer from defects, such as overly coarse-grained statistics that cannot perform precise application-level traffic analysis, and irreversibility, which cannot support real-time queries in a friendly way. To achieve real-time fine-grained application traffic analysis in general scenarios, we propose AppSketch, a real-time network traffic measurement tool. AppSketch adopts a one-pass approach to classify and label the application information of each packet in the network flows. It then hashes the flow, identified with the application tag, into a carefully designed multiple-key sketch, for gathering application-specific statistics. We conducted extensive experiments using a real-world network traffic dataset collected on a university campus. The results showed that AppSketch achieved high accuracy while requiring less update time than other alternatives. Moreover, AppSketch occupies limited memory ($ {\leq }$64KB), making it suitable for online network devices.

Abstract Botnets currently use domain-generation algorithms to produce fast-flux domains that enable them to evade detection. Accurately categorizing these botnet domains is crucial to develop cybersecurity solutions against botnet threats. However, existing methods, requiring labeled data, are ineffective against new botnets. To address this issue, we propose Domain2Vec, a metric learning-based approach that can explore new botnets. Domain2Vec integrates a framework of metric learning, which uses individual domains from known botnets for categorization of unknown botnet domains. The training involves an attention-based encoder, and it includes a constraint to ensure that samples with the same labels are closer in the embedding space. The categorization uses the encoder to project domain names into appropriate representations (numerical vectors), even for domains from new botnets. Finally, Domain2Vec uses numerical vectors to explore botnets. Experiments showed that Domain2Vec performs well on domain retrieval and clustering tasks without labeled data, outperforming the state of the art by 13% and 100%, respectively. Real-world tests demonstrate that Domain2Vec can effectively identify unreported malicious domains and monitor botnet activities.

Global-scale IPv6 scan, critical for network measurement and management, is still a mission to be accomplished due to its vast address space. To tackle this challenge, IPv6 scan generally leverages pre-defined seed addresses to guide search directions. Under this general principle, however, the core problem of effectively using the seeds is largely open. In this work, we propose a novel IPv6 active search strategy, namely HMap6, which significantly improves the use of seeds, w.r.t. the marginal benefit, for large-scale active address discovery in various prefixes. Using a heuristic search strategy for efficient seed collection and alias prefix detection under a wide range of BGP prefixes, HMap6 can greatly expand the scan coverage. Real-world experiments over the Internet in billion-scale scans show that HMap6 can discover 29.39M unique /80 prefixes with active addresses, an 11.88% improvement over the state-of-the-art methods. Furthermore, the IPv6 hitlists from HMap6 include all-responsive IPv6 addresses with rich information. This result sharply differs from existing public IPv6 hitlists, which contain non-responsive and filtered addresses, and pushes the IPv6 hitlists from quantity to quality. To encourage and benefit further IPv6 measurement studies, we released our tool along with our IPv6 hitlists and the detected alias prefixes.

Efficient Internet-wide scanning plays a vital role in network measurement and cybersecurity analysis. While Internet-wide IPv4 scanning is a solved problem, Internet-wide scanning for IPv6 is still a mission yet to be accomplished due to its vast address space. To tackle this challenge, IPv6 scanning generally needs to use pre-defined seed addresses to guide further IPv6 scanning directions. Under this general principle, various solutions have been developed, but all suffer from two primary pitfalls, low hit rate and low probing speed, caused by the inherent sparse distribution of active IPv6 addresses and the high computational complexity of the search algorithms, respectively. We develop 6Scan, a novel asynchronous IPv6 scanner that effectively addresses the above two problems. To increase the hit rate, 6Scan infers the promising search directions by encoding the regional identifiers of the target addresses within the probing packets and recording the regional activities from the asynchronously arrived replies. It then dynamically adjusts the search directions according to the scanning result of the previous steps. To speed up the search algorithm, 6Scan leverages the regional identifier encoding to quickly adjust search direction without excessive computation. Real-world experiments over the IPv6 Internet in a billion-scale probing budget show that compared with the state-of-the-art solutions, on average 6Scan can discover 6% more active addresses with nearly the same scanning time.

IPv6 target generation is the critical step for fast IPv6 scanning for Internet-wide surveys. Existing techniques, however, commonly suffer from low hit rates due to inappropriate space partition caused by the outlier addresses and short-sighted splitting indicators. To address the problem, we propose 6Forest, an ensemble learning-based approach for IPv6 target generation that is from a global perspective and resilient to outlier addresses. Given a set of known addresses, 6Forest first considers it as an initial address region and then iteratively divides the IPv6 address space into smaller regions using a maximum-covering splitting indicator. Before a round of space partition, it builds a forest structure for each region and exploits an enhanced isolation forest algorithm to remove the outlier addresses. Finally, it pre-scans samples from the divided address regions and based on the results generates IPv6 addresses. Experiments on eight large-scale candidate datasets indicate that, compared with the state-of-the-art methods in IPv6 worldwide scanning, 6Forest can achieve up to 116.5% improvement for low-budget scanning and 15× improvement for high-budget scanning.

Fast Internet-wide network measurement plays an important role in cybersecurity analysis and network asset detection. The vast address space of IPv6, however, makes it infeasible to apply a brute-force approach for scanning the entire network. Even worse, the extremely uneven distribution of IPv6 active addresses results in a low hit rate for active scanning. To address the problem, we propose 6Hit, a reinforcement learning-based target generation method for active address discovery in the IPv6 address space. It first divides the IPv6 address space into different regions according to the structural information of a set of known seed addresses. Then, it allocates exploration resources according to the reward of the scanning on each region. Based on the evaluative feedback from existing scanning results, 6Hit optimizes the subsequent search direction to regions that have a higher density of activity addresses. Compared with other state-of-the-art target generation methods, 6Hit achieves better performance on hit rate. Our experiments over real-world networks show that 6Hit achieves 3.5% - 11.5% hit rate for the eight candidate datasets, which is 7.7% - 630% improvement over the state-of-the-art methods.

Adaptive bitrate (ABR) algorithms are often used to optimize the quality of user experience (QoE) during video playback. In the client-side video player, the buffer size and predicted throughput are mainly used to improve user's QoE. However, due to the randomness of mobile network traffic and the heavy-tail effect of the network, it is very difficult to predict throughput. We innovatively use Bayesian neural network to dynamically evaluate video signals. Unlike previous neural network solutions, we use probability distributions instead of point estimates to predict throughput, which can effectively evaluate QoE metrics. Our contributions are to first (i) use of Bayesian neural network to guide video adaptive bitrate adaptation, and then (ii) propose a bitrate adaptive algorithm denoted 2prong, which utilizes high-dimensional contextual information such as buffer occupancy, predicted throughput and video quality to find the most valuable information for quality adaption in real time. We demonstrate the effectiveness of the 2prong algorithm using a simulation testbed. By comparing with other methods, it is demonstrated that 2prong can improve the video quality of video streaming transmission.

Network anomalies, such as wide-area congestion and packet loss, can seriously degrade network performance. To this end, it is critical to accurately identify network anomalies on end-to-end paths for high quality network services in practice. In this work, we propose an unsupervised two-step method for the detection and characterization of general network anomalies. It first finds the change-points in large-scale RTT time series by formalizing an optimization problem in terms of data series segmentation. Then we mark the segments as normal or abnormal on different sides of a change-point through exploitation of their distribution statistics. After detecting an anomaly, a further step is introduced to analyze the relations between links with state changes and localize the entities (nodes or links) that most likely cause the corresponding event. We believe such unsupervised and light-weighed method can provide valuable insights on anomaly mining in large-scale time series data. Extensive experiments on both simulated (artificial time series with ground truth) and real-network (RIPE Atlas traceroute measurements) datasets are performed. The results demonstrate that the proposed method can achieve better performance, w.r.t. accuracy and efficiency, than existing solutions.