The traffic bandwidth costs, primarily driven by outbound traffic, comprise a significant amount of operating expenditure in CDNs, induced by the traffic from the end-users to edge servers (edge cost) and from the edge to the center servers (midgress cost). Traffic allocation is the main approach to minimizing the total bandwidth cost. The joint optimization of the total costs is challenging, specifically when the percentile charging mechanism as well as some other practical issues are considered, such as the dynamicity of midgress traffic and the granularity of traffic allocation. In this work, based on our novel miss ratio prediction mechanism, we propose the first online framework, namedIris, jointly optimizing the edge and midgress costs under the 95th percentile charging in commercial CDNs.Iriscan theoretically achieve a competitive ratio of$1+\frac {p_{e}}{\beta \cdot p_{c}}$, when the miss ratio of all domains is set as$\beta $. Here$p_{e}$and$p_{c}$are the unit bandwidth price of the edge and midgress cost, respectively.Irisis tolerant to prediction errors which can be deployed in practical CDN systems. Extensive experiments based on real data indicate thatIriscan dramatically reduce bandwidth costs by about 8.149% compared with the SOTA schemes, potentially saving millions of dollars per month for our large-scale commercial CDN collaborator.

With the rapid growth of data-intensive applications, such as artificial intelligence and the Internet of Things, CDNs, which use persistent storage (e.g., SSDs and HDDs) to cache data at the edge, have become crucial for enhancing network efficiency. Two metrics—hit ratio and processing la tency—are essential for evaluating CDN caching performance. However, CDN caching faces the challenge of write amplification, creating a trade-off between random access for higher hit ratios and sequential writes for reducing processing latency. Existing cache designs struggle to effectively balance these conflicting requirements across diverse workloads. In this paper, we present FOSS, a caching system specifically optimized for CDNs deployed on SSD-based storage and hybrid SSD–HDD storage, which features a streamlined, thin file system that operates independently of the kernel. At its heart, FOSS employs a multi-level FIFO queue to strike a balance between local sequential and global random access on SSDs. Then, FOSS incorporates a learning-based method to dynamically configure the multi-level structure configuration, making the system adaptive to various workload characteristics and caching algorithm requirements. Therefore, FOSS ensures better performance across different scenarios. Our extensive experiments show FOSS improves hit ratios significantly over existing systems, reduces end-to-end response latency by 16.5% and demonstrates a consistent performance improvement in various settings on large-scale commercial CDN traces.

The Large Vision-Language Model (LVLM) integrates computer vision and natural language processing techniques, offering substantial application potential. However, these models demand extensive resources during inference. Adaptive attention techniques can dynamically reduce computational redundancy and thus improve efficiency. Although current adaptive attention methods significantly reduce the memory requirements of Transformer-based language models, they are not tailored for LVLMs. We observe that LVLMs generate responses from both remote image tokens and local text tokens, and different modalities have different attention patterns. This observation inspires us to manage the attention for each modality separately. Specifically, for visual input, we store the cache of potentially useful information but only compute the most critical parts. For language input, we care more about local information. Based on our observation and analysis of vision-language attention patterns, we develop A-VL, a plug-and-play adaptive attention tailored for LVLM inference. Extensive evaluations on three vision-language tasks and five datasets show the effectiveness of our designs. Our approach A-VL outperforms existing adaptive attention methods in reducing memory usage and computational load without compromising performance.

The traffic bandwidth costs comprise a significant amount of operating expenditure in CDNs, induced by the traffic from the end-users to edge servers (edge cost) and from the edge to the center servers (midgress cost). Traffic allocation is the main approach to minimizing the total bandwidth cost. The joint optimization of the total costs is challenging, specifically when the percentile charging mechanism as well as some other practical issues are considered, such as the dynamicity of midgress traffic and the granularity of traffic allocation. In this work, based on our novel miss ratio prediction mechanism, we propose the first online framework, named Iris, jointly optimizing the edge and midgress costs under the 95th percentile charging in commercial CDNs. Iris can theoretically achieve a competitive ratio of$1+\frac{p_{e}}{\beta\cdot p_{c}}$, when the miss ratio of all domains is set as$\beta$. Here$p_{e}$and$p_{c}$are the unit bandwidth price of the edge and midgress cost, respectively. Iris is tolerant to prediction errors which can be deployed in practical CDN systems. Extensive experiments based on real data indicate that Iris can dramatically reduce bandwidth costs by about 8.149% compared with the SOTA schemes, potentially saving millions of dollars per month for our large-scale commercial CDN collaborator.

With the explosion of data transmitted over the Internet, Content Delivery Networks (CDNs) carry massive network traffic globally and suffer an increasingly higher bandwidth cost. A critical issue for CDN service providers is how to allocate network traffic among CDN facilities to reduce the total bandwidth cost without violating the quality of service. This work studies online traffic allocation in CDNs to minimize the bandwidth cost under the 95th percentile charging model. Specifically, we here take into account practical deployment issues in large-scale CDN systems, e.g., allocation granularity and deviation. We first theoretically prove the approximation hardness of the traffic allocation problem. We then propose a novel prediction-based algorithm named OnTPC, which effectively addresses constraints raised in practical deployment. Extensive experiments demonstrate that OnTPC outperforms state-of-the-art baselines and is expected to save over a million dollars per month for our large-scale commercial CDN collaborator. Moreover, the performance of OnTPC is consistently outstanding under various settings, and specifically robust to large allocation deviation.