The layered architecture of the TCP/IP protocol stack enables protocol layers to be implemented independently and flexibly. However, this layered design introduces potential security risks when shared resources are not properly managed between different layers. This paper investigates a neglected cross-layer shared resource risk, termed SocketFilled, which exploits the insecure usage of the UDP send buffer at the transport layer by the link layer, resulting in the interruption of response packets from the upper application layer. To explore the root causes of cross-layer DoS vulnerabilities resulting from the implementation of the TCP/IP protocol stack, we systematically analyzed the protocol standards of address resolution and reviewed the implementation in mainstream open-source operating systems. Moreover, we conducted a comprehensive experimental evaluation of mainstream operating systems (e.g., Linux and FreeBSD) and UDP services (e.g., DNS and QUIC). The experimental results show that the latest version of Linux and UDP service software (e.g., BIND9, PowerDNS, and Nginx) are affected, causing significant packet loss and even complete service interruption. Then, we estimated the impact range of SocketFilled in the wild and demonstrated that 17.3% of open resolvers,54.3% of authoritative servers of the Tranco Top 100K domains, and 3.8% of these well-known domains' HTTP/3 servers are potentially affected, including Bing, Amazon, and Shopee, after excluding the influence of cloud servers. We have conducted responsible disclosure by reporting the vulnerability to the Linux community. Our research highlights the effectiveness of cross-layer mechanisms in DoS attacks and calls for heightened attention to the layered complexity of protocol stack implementations within the security community.

DNS employs a variety of mechanisms to guarantee availability, protect security, and enhance reliability. In this paper, however, we reveal that these inherent beneficial mechanisms, including timeout, query aggregation, and response fast-returning, can be transformed into malicious attack vectors.We propose a new practical and powerful pulsing DoS attack, dubbed the DNSBomb attack. DNSBomb exploits multiple widely-implemented DNS mechanisms to accumulate DNS queries that are sent at a low rate, amplify queries into large-sized responses, and concentrate all DNS responses into a short, high-volume periodic pulsing burst to simultaneously overwhelm target systems. Through an extensive evaluation on 10 mainstream DNS software, 46 public DNS services, and around 1.8M open DNS resolvers, we demonstrate all DNS resolvers could be exploited to conduct more practical-and-powerful DNSBomb attacks than previous pulsing DoS attacks. Small-scale experiments show the peak pulse magnitude can approach 8.7Gb/s and the bandwidth amplification factor could exceed 20,000x. Our controlled attacks cause complete packet loss or service degradation on both stateless and stateful connections (TCP, UDP, and QUIC). In addition, we present effective mitigation solutions with detailed evaluations. We have responsibly reported our findings to all affected vendors, and received acknowledgement from 24 of them, which are patching their software using our solutions, such as BIND, Unbound, PowerDNS, and Knot. 10 CVE-IDs are assigned.