Recently, deep learning has been introduced to automate network management to reduce human costs. However, the amount of log data obtained from the large-scale network is huge, and conventional centralized deep learning faces communication and computation costs. This paper aims to reduce communication and computation costs by training deep learning models using federated learning on data generated in the network and to deploy deep learning models as soon as possible. In this scheme, data generated at each point in the network are transferred to servers in the network, and deep learning models are trained by federated learning among the servers. In this paper, we first reveal that the training time depends on the transfer routes and the destinations of data and model parameters. Then, we introduce a simultaneous optimization method for (1) to which servers each point transfers the data through which routes and (2) through which routes the servers transfer the parameters to others. In the experiments, we numerically and experimentally compared the proposed method and naive methods in complicated wired network environments. We show that the proposed method reduced the total training time by 34% to 79% compared with the naive methods.

The advent of the Internet of Things (IoT) has led to an exponential increase in data from devices usable for monitoring and managing various systems. Traditional central server data processing is becoming impractical due to the surge in IoT devices and data volume, resulting in high bandwidth and computational costs. In federated learning (FL), servers train models independently using their own data and transfer their models to a central aggregator where the models are aggregated. This process reduces the computational load and bandwidth usage inherent in centralized systems because data transfer can be reduced in FL. The study focuses on a scenario where base stations (BSs), each with a server, receive data from IoT devices, and BSs train models with their own data as participants of FL. Since the accuracy of the model is influenced by the data's amount and distribution across servers, which BS IoT devices transfer their data to is critical, while bandwidth limitations constrain that choice. The paper introduces a conditional optimization problem and solves the problem with a genetic algorithm to maximize FL model accuracy while adhering to bandwidth constraints. In the experiments, we compared the proposed optimization and naive method by numerical simulations and actual training deep learning models. As a result, the accuracies of all deep learning models improved when FL was performed based on the results obtained from our optimization compared to the naive method.

Federated learning is a distributed machine learning technique that addresses the challenges of traditional centralized machine learning, such as high computational expenses and network congestion. In federated learning, a model is distributed to each server from a central server, and then each server trains the model using its own data. After training, the models are integrated at the central server, and the integrated model is re-distributed to the servers for further training. Federated learning can train a model based on all the data without collecting the data in one place, reducing the burden on the network and addressing the problem of training costs for extensive amounts of data. However, federated learning has two unique challenges: prolonged training time depending on the amount of data in each server, the processing capacity of each server, and the bandwidth of links in a network, and the accuracy of the trained model, which is influenced by the heterogeneity of the datasets in each server. We propose an optimization problem for the training time and the convergence behavior in federated learning. Specifically, we present a nonlinear programming problem to minimize the total training time while optimizing the data transfer route and destination of each router, subject to constraints on network congestion avoidance and convergence of the trained model. We evaluate the proposed method in an experiment using a virtual GPU cluster and show that the proposed method improves both the accuracy of the trained models and the training time compared to a prior method.

Federated learning, which can distributedly and efficiently train a deep learning model, attracts much attention since a large amount of data generated on a wide network can be utilized. When data generated in a network are collected, how much data is collected on which servers affects the model training time. However, the bandwidths of the links constrain the amount of data collected at each server and transfer routes of data. In this paper, we propose the optimization formula for data destinations and transfer routes in each generating data site under the constraint of the bandwidth of links in a network to minimize the time consumed by training a model. In the experiments, first, we numerically compared the amount of data collected in each server between the proposed algorithm and a naive method. The experiment shows that the proposed method could conduct the federated learning using all data generated in a network even when the bandwidths of links were less than one-third compared to the naive method. Then, we compared the training time of a deep learning model on the basis of the amount of data calculated in the former numerical experiments. We exhibit that the proposed algorithm reduced the training time by 27% to 47% compared to the naive method.

Machine learning (ML) has been used for various tasks in network operations in recent years. However, since the scale of networks has grown and the amount of data generated has increased, it has been increasingly difficult for network operators to conduct their tasks with a single server using ML. Thus, ML with edge-cloud cooperation has been attracting attention for efficiently processing and analyzing a large amount of data. In the edge-cloud cooperation setting, although transmission latency, bandwidth congestion, and accuracy of tasks using ML depend on the load balance of processing data with edge servers and a cloud server in edge-cloud cooperation, the relationship is too complex to estimate. In this paper, we focus on monitoring anomalous traffic as an example of ML tasks for network operations and formulate transmission latency, bandwidth congestion, and the accuracy of the task with edge-cloud cooperation considering the ratio of the amount of data preprocessed in edge servers to that in a cloud server. Moreover, we formulate an optimization problem under constraints for transmission latency and bandwidth congestion to select the proper ratio by using our formulation. By solving our optimization problem, the optimal load balance between edge servers and a cloud server can be selected, and the accuracy of anomalous traffic monitoring can be estimated. Our formulation and optimization framework can be used for other ML tasks by considering the generating distribution of data and the type of an ML model. In accordance with our formulation, we simulated the optimal load balance of edge-cloud cooperation in a topology that mimicked a Japanese network and conducted an anomalous traffic detection experiment by using real traffic data to compare the estimated accuracy based on our formulation and the actual accuracy based on the experiment.

The environmental load of telecommunication service provision is increasing due to the increase in communication traffic. Virtual networks have recently begun to spread, and flexible virtual network control is expected to reduce the operating costs of telecommunication services. This paper proposes a control method to achieve both reduced power bills and stable network operation when the electricity unit price differs among areas and time periods. To begin with, the problem to minimize power bills is formulated, and to solve it quickly, a heuristic search method is proposed that utilizes network centrality. In addition, we formulate a multi-objective optimization problem characterized by parameter normalization, and simulation results show that the proposed control method can reduce power bills while suppressing the number of network reconfigurations.

Network traffic and computing demand have been changing dramatically due to the growth of various types of network services, e.g., high-quality video delivery and operating system (OS) updates. To maximize the utilization efficiency of limited network resources, network resource control technology is required for smooth and quick operation when network demands change. Therefore, we propose a dynamic virtual network (VN) allocation method based on cooperative multi-agent deep reinforcement learning (Coop-MADRL). This method can quickly optimize network resources even while network demands are drastically changing by learning the relationship between network demand patterns and optimal allocation by using deep reinforcement learning (DRL) in advance. The key idea is to use a multi-agent technique for a reinforcement learning (RL) based dynamic VN allocation method, which can reduce the number of candidate actions per agent and can improve the performance for VN allocation. Moreover, a cooperation technique improves the efficiency of VN allocation. From results of a simulation evaluation, Coop-MADRL can calculate effective allocation within 1 s, which reduces the maximum server and link utilization and drastically reduces the constraint violations compared with that of the static VN allocation method. Furthermore, we revealed that the learning with various mixed traffic models could achieve a high generalization performance for all traffic patterns.

Network traffic and computing demand have been changing dramatically due to the growth of various types of network services, e.g., high-quality video delivery and operating system (OS) updates. To maximize the utilization efficiency of limited network resources, network resource control technology is required for smooth and quick operation when network demands change. We propose a dynamic virtual network (VN) allocation method based on cooperative multi-agent deep reinforcement learning (Coop-MADRL). This method can quickly optimize network resources even while network demands are drastically changing by learning the relationship between network demand patterns and optimal allocation by using deep reinforcement learning (DRL) in advance. The key idea is to use a multi-agent technique for a reinforcement learning (RL) based dynamic VN allocation method, which can reduce the number of candidate actions per agent and can improve the performance for VN allocation. Moreover, a cooperation technique improves the efficiency of VN allocation. From results of a simulation evaluation, Coop-MADRL can calculate effective allocation within 1 s, which reduces the maximum server and link utilization and drastically reduces the average constraint violation compared with that of the static VN allocation method.

This paper presents a novel system architecture for predictive road-traffic information delivery in which computing resources at the network edge and the central cloud are cooperatively used to analyze sensing data collected by vehicles on the road. In this paper, we also present the mathematical problem formulation of the proposed system architecture for ensuring that the system could successfully deliver road-traffic information at realtime without overflowed computational and network loads. The numerical examination using a real dataset and a realistic network emulator validates our system.1

Identifying the root cause of failures in a complicated communication system such as a cloud platform is time-consuming for system operators. Although most current diagnosis methods depend on logs that are observed passively, some failures generate quite similar logs and cannot be distinguished from one another with these methods. To overcome this difficulty, we propose a framework in which operators execute user actions and use logs generated by the actions in root-cause analysis. We focus on the fact that even if we do not see any differences between failures in logs observed passively, logs generated by a particular action may change depending on the failure. We also propose two methods for executing such effective actions in a proper order and obtaining informative logs efficiently. With these methods, we can identify the root cause of failures that are indistinguishable with current methods. We experimentally evaluated the effectiveness of our framework in a cloud system constructed with OpenStack.

We propose a root-cause diagnosis method for finding equipment suffering from rare failures in a communication network. Although many studies have been conducted on root cause diagnosis for finding failed equipment using a Bayesian Network or other methods, there has not been sufficient research into finding rare-failure equipment. Current methods are mainly focused on typical-failure equipment and cannot find rare-failure equipment. This is because rare failures have two features;unexpected causal relations and observation errors. To adapt rare- failure features, we propose a method that consists of an extended causal model and an extended inference algorithm with dynamic modification of the causal relations and observation statuses in a Bayesian Network. We experimentally evaluated its effectiveness.

Network functions virtualization (NFV) enables telecommunications service providers to provide various network services by flexibly combining multiple virtual network functions (VNFs). To provide such services with carrier-grade quality, an NFV controller must optimally allocate such VNFs into physical networks and servers, taking into account combination(s) of objective functions and constraints for each metric defined for each VNF type. The NFV controller should also be extendable, i.e., new metrics should be able to be added. One approach for NFV control to optimize allocations is to construct an algorithm that simultaneously solves the combined optimization problem. However, this algorithm is not extendable because the problem formulation needs to be rebuilt every time, e.g., a new metric is added. Another approach involves using an extendable network-control architecture that coordinates multiple control algorithms specified for individual metrics. However, to the best of our knowledge, no method has been developed to optimize allocations through this kind of coordination. In this paper, we propose an extendable NFV-integrated control method by coordinating multiple control algorithms. We also propose an efficient coordination algorithm based on reinforcement learning. Finally, we evaluate the effectiveness of the proposed method through simulations.

As data collection by mobile devices becomes feasible, network operators expect that the end-to-end measured data widely collected by users' devices will result in a deeper understanding of the current network conditions, identification of the performance-degraded components in a network, and estimation of the degree of their performance degradation. One method of achieving the above with such end-to-end measured data is network tomography. Meanwhile, network virtualization by software-defined networking has advanced, and each end-to-end measurement flow collected by users may pass through different paths even between the same origin-destination node pair. It is difficult and costly to identify through which paths the individual flows have passed, so it is difficult to naively apply conventional network tomography methods to virtualized networks. We propose a novel network tomography for virtualized networks to which the routing paths cannot be uniquely identified. The basic idea of our method is to introduce routing probability in accordance with the aggregated information on measurement flows. Our method involves two proposed algorithms, and we evaluated the performance of these algorithms by comparing them with algorithms of conventional tomography using determined route information and by varying the network conditions through simulation.

This paper proposes a disaster avoidance control method for use against heavy rainfall and discusses its effectiveness through actual weather data. The proposed control method uses geographical information data including weather data, hazard area data, and physical network data. By applying technologies related to meteorology, erosion control, and civil engineering to such data, the proposed method can evaluate the risk of a physical network being disconnected. On the basis of the evaluated risk, the proposed method reconfigures a logical network to reduce service disruption. The proposed method is applied to a cloud computing service network where, in addition to route changes, the relocation of virtual machines is possible, increasing its effectiveness. By using empirical data, we show that the proposed method reduces the probability of service disconnection to almost zero even for heavy rainfall causing landslides. Finally, an experimental system of the proposed method was implemented through software defined network technology and successfully controlled the experimental network.

The offloading of cellular traffic through WLAN APs (wireless local area network access points) distributed in a homogeneous Poisson process (HPP) is theoretically evaluated. The probability Pw that a user can use WLAN and the expected number of vertical handovers Nh are evaluated as the basic performance metrics of WLAN AP coverage. Explicit formulas are derived for the metrics, and the fundamental relationships between the metrics and many parameters such as the shape of each WLAN coverage region D1, D2,... are described. These metrics depend on the size and perimeter length of Di but do not depend on their other shape parameters for a convex Di. In addition, it is proven that a disk-shaped Di minimizes Nh for a fixed WLAN coverage size and that Pw is often insensitive to the perimeter length of Di. It is also proven that the Pw of a user at a random location is equal to that moving along a random straight line or a random bounded curve. One hundred empirical location data sets of WLAN APs in Japan, Korea, and the US were used to confirm the theoretical results. Although these locations do not follow an HPP, many theoretical results are shown to be valid. For example, Nh is minimized by a disk-shaped Di. Simultaneously, we find that Pw slightly increases when a slender Di is used for highly clustered AP locations.

In this paper, we propose a mathematical model for intelligent transportation systems (ITS) in an urban environment, which takes into account both the urban structure and vehicles. Using the stochastic geometry approach, we model the locations of vehicles with a Poisson point process on roads whose interval is a fixed value. We consider typical vehicle-to-infrastructure and vehicle-to-vehicle communication scenarios and derive theoretical values of the probability of successful transmission. Our results from numerical experiments reveal how the urban structure can affect wireless communications in ITS.

Macroscopic diversity (macro-diversity) techniques are attracting much attention in wireless communication. However, the optimal cell planning algorithm for macro-diversity involving sector antennas has not been investigated. We thus propose a metaheuristic algorithm for cell planning with macro-diversity using sector antennas. In the algorithm, the constraint conditions for deploying base stations (BSs) are expressed via homology, which has recently been used in sensor networking. Numerical simulations show that the cell deployment pattern obtained with the proposed algorithm requires about 20% less BSs compared to that obtained with the reference algorithm.

Macroscopic diversity (macro-diversity) techniques, such as coordinated multi-point transmission in LTE networks, are attracting attention for reducing the error rate of wireless transmission. We propose a cell-planning algorithm with macro- diversity when possible locations of cells are given. To define a cell, we focus on the desired received signal power of uplink determined by fractional power control. We also propose a signal-to-interference-plus-noise ratio (SINR) evaluation method under the obtained cell-deployment pattern, which takes into account frequency scheduling used in LTE networks. Numerical results show that by optimally deploying the cells, we can reduce the required number of cells to cover the defined domain by up to 30% compared to when the cells are selected greedily. Moreover, the SINR evaluation results suggest that reducing the required cells to cover the domain improves uplink SINR at the cell edge.

Monitoring flows at routers for flow analysis or deep packet inspection requires the monitors to update monitored flow information at the transmission line rate and needs to use highspeed memory such as SRAM. Therefore, it is difficult to measure all flows, and the monitors need to limit the monitoring target to a part of the flows. However, if monitoring targets are randomly selected, an identical flow will be monitored at multiple routers on its route, or a flow will not be monitored at any routers on its route. To maximize the number of flows monitored in the entire network, the monitors are required to select the monitoring targets while maintaining a balanced load among the monitors. In this paper, we propose an autonomous load balancing method where monitors exchange monitor load information with only adjacent monitors.

Peer-assisted content distribution technologies have been attracting attention. By using not only server resources but also the resources of end hosts (i.e., peers), we can reduce the offered load on servers as well as utilization of the access bandwidth of the servers. However, offered traffi to the network may increase because the traffi exchanged between peers passes across the network. Specificall, if individual peers send traffi disregarding underlay network topology and traffi conditions, the peer-assisted content distribution method may cause excessive traffi offered to the network and poor application performance. We thus investigated the impact of traffi caused by peer-assisted content distribution on the underlay network. We found that although peer-assisted content distribution disregarding underlay network topology causes 80-120% additional traffi compared with the optimal case, i.e., content distribution using cache servers allocated optimally in the network, using underlay network topology enables us to achieve almost the same efficien network resource utilization as the optimal case. We also found that the peer-assisted approach can adaptively cope with change in the traffi demand matrix because uploaders in the network are generated according to the demand matrix in a self-organizing manner. This is because peers that have downloaded the content become uploaders so many uploaders are generated in the area where a large number of content requests exist according to the traffi condition; therefore, the content delivery traffi can be localized.

Due to integrated high-speed networks accommodating various types of services and applications, the quality of service (QoS) requirements for those networks have also become diverse. The network resources are shared by the individual service traffic in the integrated network. Thus, the QoS of all the services may be degraded indiscriminately when the network becomes congested due to a sudden increase in traffic for a particular service if there is no traffic engineering taking into account each service's QoS requirement. To resolve this problem, we present a method of controlling individual service traffic by using an overlay network, which makes it possible to flexibly add various functionalities. The overlay network provides functionalities to control individual service traffic, such as constructing an overlay network topology for each service, calculating the optimal route for the service's QoS, and caching the content to reduce traffic. Specifically, we present a method of overlay routing that is based on the Hedge algorithm, an online learning algorithm to guarantee an upper bound in the difference from the optimal performance. We show the effectiveness of our overlay routing through simulation analysis for various network topologies.

In Video on Demand (VoD) services, the demand for content items greatly changes daily, so reducing the server load at the peak time is an important issue for ISPs to reduce the server cost. To achieve this goal, we proposed to reduce the server load by multicasting popular content items to all users independently of actual requests as well as providing on-demand unicast delivery. In this solution, however, the hit ratio of pre-distributed content items is small, and a large-capacity storage is required at set-top box (STB). We might be able to cope with this problem by limiting the number of pre-distributed content items or clustering users based on the history of viewing. We evaluate the effect of these techniques using actual VoD access log data. We clarify that the required storage capacity at STB can be halved while keeping the effect of server load reduction to about 80% by limiting pre-distributed content items, and user clustering is effective only when the cluster count is about two.

Traffic caused by P2P services dominates a large part of traffic on the Internet and imposes significant loads on the Internet, so reducing P2P traffic within networks is an important issue for ISPs. In particular, a huge amount of traffic is transferred within backbone networks; therefore reducing P2P traffic is important for transit ISPs to improve the efficiency of network resource usage and reduce network capital cost. To reduce P2P traffic, it is effective for ISPs to implement cache devices at some router ports and reduce the hop length of P2P flows by delivering the required content from caches. However, the design problem of cache locations and capacities has not been well investigated, although the effect of caches strongly depends on the cache locations and capacities. We propose an optimum design method of cache capacity and location for minimizing the total amount of P2P traffic based on dynamic programming, assuming that transit ISPs provide caches at transit links to access ISP networks. We apply the proposed design method to 31 actual ISP backbone networks.

Although video-on-demand (VoD) services are provided by many ISPs, the amount of content provided by each VoD service is one order smaller than that provided by rental video services, so the limited content count is one of the obstacles to widespread VoD services. To solve this problem, ISPs can form a coalition with other ISPs and use content owned by other ISPs. However, to form a coalition among multiple ISPs, ISPs need to rationally allocate the profit obtained by the coalition to convince all ISPs participating in the coalition. We propose using the Shapley value of the coalitional game as the rational allocation of profit. Assuming that all but one ISP has the same number of users or (and) the same number of rare content, we derive the Shapley value in closed form and clarify the influence of the numbers of users and rare content on the coalition. We also compare the Shapley value with three general allocation models and show that the Shapley value agrees with the allocation when the profit obtained by each content delivery is equally shared by two ISPs, one that accommodates the receiving user and the other that owns the delivered content.

Video streaming with HDTV or UHDV quality will be provided and widely demanded in the future. However, the transmission bit-rate of high-quality video streaming is quite large, so generated traffic flows will cause link congestion. Therefore, when providing streaming services of rich content, it is important to flatten the link utilization, i.e., reduce the maximum link utilization. To achieve this goal, parallel video streaming in which ISPs use multiple servers to deliver rich content is effective. However, the effect of parallel video streaming depends on the network topology and link capacities. In this paper, we investigate the impact of network topologies on the effect of parallel video streaming using 23 actual commercial ISP networks, when optimally designing server locations and optimally selecting servers.

It is known that there exist Triangle Inequality Violations (TIVs) with respect to network Quality of Service (QoS) metrics such as latency between nodes in the Internet. This motivates the exploitation of QoS-aware routing overlays. To find an optimal overlay route, we would usually need to examine all the possible overlay routes. However, this requires both measuring QoS between all node pairs and investigating all the routes in the full-mesh overlay topology, which poses scalability problem in terms of both measurement cost and route calculation and dissemination cost. We thus propose a method of constructing a QoS overlay network that enables us to find a near optimal route in a cost-effective manner. Our idea is based on the finding that a small number of overlay nodes can provide the optimal routes for a large number of node pairs, which is obtained through measurement data analysis between PlanetLab nodes. Our overlay network has two layers where the upper-layer consists of such small number of overlay nodes that can provide the optimal routes while the lower-layer consists of the other overlay nodes. By allocating such overlay nodes at the upper-layer, we can provide better QoS routes for each node pair with high probability. We construct the overlay network topology where the upper-layer overlay nodes are connected in full-mesh manner while the lower-layer overlay nodes are not connected in full-mesh but only to upper-layer nodes. Through this structure, we can reduce measurement and route calculation costs. Using PlanetLab data, we show that our method can achieve almost the same performance as the optimal solution.

As a promising solution to manage the huge workload of large-scale VoD services, managed peer-assisted CDN systems, such as P4P has attracted attention. Although the approach works well in theory or in a controlled environment, to our best knowledge, there have been no general studies that address how actual peers can be incentivized in the wild Internet; thus, deployablity of the system with respect to incentives to users has been an open issue. With this background in mind, we propose a new business model that aims to make peer-assisted approaches more feasible. The key idea of the model is that users sell their idle resources back to ISPs. In other words, ISPs can leverage resources of cooperative users by giving them explicit incentives, e.g., virtual currency. We show the high-level framework of designing optimal incentive amount to users. We also analyze how incentives and other external factors affect the efficiency of the system through simulation. Finally, we discuss other fundamental factors that are essential for the deployability of managed peer-assisted model. We believe that the new business model and the insights obtained through this work are useful for assessing the practical design and deployment of managed peer-assisted CDNs.

We describe a method of adaptively controlling bandwidth allocation to flows for reducing the file transfer time of short flows without decreasing throughput of long-duration large flows. According to the rapid increase in Internet traffic volume, effective traffic engineering is increasingly required. Specifically, the traffic of long-duration large flows due to the use of peer-to-peer applications, for example, is a problem. Most conventional QoS controls allocate a fair-share bandwidth to each flow regardless of its duration. Thus, a long-duration large flow (such as a P2P flow) is allocated the same bandwidth as a short- duration flow (such as data from a Web page) in which the user is more sensitive to response time, i.e., file transfer time. As a result, long-duration large flows consume bandwidth over a long period and increase response times of short-duration flows, and conventional QoS methods do nothing to prevent this. In this paper, we therefore investigate a different approach, that is, a new form of bandwidth control that enables us to achieve better performance when handling short-duration flows while maintaining performance when handling long-duration flows. The basic idea is to tag packets of long-duration large flows according to traffic conditions and to give temporarily higher priority to non-tagged packets during network congestion. We also show the effectiveness of our method through simulation.

It is known that Internet routing policies for both intra- and inter-domain routing can naturally give rise to triangle inequality violations (TIVs) with respect to quality of service (QoS) network metrics such as latencies between nodes. This motivates the exploitation of such TIVs phenomenon in network metrics to design TlV-aware routing overlay architecture which is capable of choosing quality overlay routing paths to improve end-to-end QoS without changing the underlying network architecture. Our idea is to find quality overlay routes between node pairs based on TIV optimization in terms of the latency and packet loss ratio, and that can offer near optimal routing quality in cost-effective and scalable manner. Our intuition to do this is to choose these overlay routes from a small set of transit nodes. We propose to assign nodes with transit selection frequency scores that are computed based on previous node usage for transit, and consolidate a small set of highly ranked transit nodes. For every node pair, we choose the best transit node in this small set for overlay routing, based on TIV optimization in latency and packet loss ratio. We analyze the quality of our TlV-aware routing overlay algorithm analytically and using real Internet measurements on latency and packet loss ratio. Our results show good quality performance in improving end-to-end QoS routing.

We present a method of detecting network anomalies, such as DDoS (distributed denial of service) attacks and flash crowds, automatically in real time. We evaluated this method using measured traffic data and found that it successfully differentiated suspicious traffic. In this paper, we focus on cyclic traffic, which has a daily and/or weekly cycle, and show that the differentiation accuracy is improved by utilizing such a cyclic tendency in anomaly detection. Our method differentiates suspicious traffic that has different statistical characteristics from normal traffic. At the same time, it learns about cyclic large- volume traffic, such as traffic for network operations, and finally considers it to be legitimate.

We propose a method of identifying anomalous traffic sources using flow statistics. We have investigated a way of detecting whether or not anomalies occur by observing the behavior of several time-series of flow statistics such as the number of flows. After detecting the occurrences of network anomalies, we need to identify the source of the anomalies. In this paper, we describe a method of identifying anomalous traffic sources. For this purpose, we apply data mining approaches such as the K-nearest neighbor method, naive Bayesian classifier, neural network, and support vector machine. We show how to use such approaches to identify anomalous traffic sources by using flow statistics. We also show evaluation results for the effectiveness of our approach using two measurement data sets.

We investigate the detection accuracy of network anomalies when we use flow statistics obtained through packet sampling. We have already shown, through a case study based on measurement data, that network anomalies generating a huge number of small flows, such as network scans or SYN flooding, become hard to detect when we perform packet sampling. In this paper, we first develop an analytical model that enables us to quantitatively evaluate the effect of packet sampling on the detection accuracy and then investigate why detection accuracy worsens when the packet sampling rate decreases. In addition, we show that, even with a low sampling rate, spatially partitioning the monitored traffic into groups makes it possible to increase the detection accuracy. We also develop a method of determining an appropriate number of partitioned groups and show its effectiveness.

Traffic flow measurement is essential to implement QoS control in the Internet. Flow monitoring system collects and stores sampled flow states in a flow table (FT) and the entries are renewed at every packet sampling. Entries in the FT are checked and removed when no packets are sampled within a predetermined timeout. We propose an efficient timeout checking mechanism based on checking a small number of entries selected randomly from the FT. Our proposed method aims to reduce the number of memory accesses dramatically and keep the memory size small. We evaluate our method and compare with the conventional method that checks all flow entries of the FT periodically. Our simulation and comparison results show that our method is able to reduce the number of memory access at a factor of 1000 with a small increase in memory size of approximately 10 percent.

Abusive traffic caused by worms is increasing severely in the Internet. In many cases, worm-infected hosts generate a huge number of flows of small size during a short time. To suppress the abusive traffic and prevent worms from spreading, identifying these "superspreaders" as soon as possible and coping with them, e.g, disconnecting them from the network, is important. This paper proposes a simple and adaptive method of identifying superspreaders by flow sampling. By satisfying the given memory size and the requirement for the processing time, the proposed method can adaptively optimize parameters according to changes in traffic patterns.

A method of estimating TCP flow-rates of sampled flows through packet sampling is described in this paper. We use sequence numbers of sampled packets, which make it possible to improve markedly the accuracy of estimating the flow rates. Using an analytical model, we investigate how to set parameters such as packet sampling probability used in this method of estimation. As a remarkable result, we show that the estimation accuracy improves as the sampling probability decreases. Using measured data, we also show that this method gives accurate estimations. We also show that this estimation method enables us to detect performance degradation at the TCP flow level.

A method of controlling the rate of long-duration large flows and its performance evaluation is described in this paper. Most conventional QoS controls allocate a fair-share bandwidth to each flow regardless of its duration. Thus, a long-duration large flow (such as a P2P flow) is allocated the same bandwidth as a short-duration flow (such as data from a Web page) in which the user is more sensitive to response time. As a result, long-duration flows will occupy the bandwidth over the long period and worsen response times of short-duration flows, and the conventional QoS methods do nothing to prevent this. We have, therefore, proposed a new form of QoS control that takes flow duration into account and assigns higher priority to the acceptance of shorter-duration flows. In this paper, we show through simulation that our method achieves high performance for short-duration flows without degrading the performance of long-duration flows. We also explain how to set parameters used in our method. Furthermore, we discuss the applicability of a packet-sampling technique to improve the method's scalability.

In this paper, we propose a method of detecting TCP performance degradation using only bottleneck-link utilization statistics. The variance of link utilization normally increases as the mean link-utilization increases. However, because link-utilization has a maximum of 100%, as the mean approaches 100%, the variance decreases to zero. In this paper, using the M/G/R processor sharing model, we relate this phenomenon to the behavior of flows. We also show that by using this relationship, we can detect TCP performance degradation using the mean and variance of link utilization. Particularly, with this method, a network operator can determine whether or not the degradation originates from the congestion of his/her own network. Because our method requires us to measure link utilization only, the cost of performance management can be greatly decreased compared with the conventional method, which requires dedicated equipment to measure the network performance.

We develop a method of dimensioning and managing the bandwidth of a link on which TCP flows from access links are aggregated. To do this, we extend the application of the processor-sharing queue model to TCP performance evaluation by using flow statistics. To handle various factors that affect actual TCP behavior, besides the access-link bandwidth, such as round-trip time, window-size, and other bottlenecks, we extend the model by replacing the access-link bandwidth with the actual file-transfer speed of a flow under a low utilization of the aggregation link. We only use the number of active flows and the link utilization to estimate the file-transfer speed. Unlike previous studies, the extended model based on the actual transfer speed does not require any assumptions/predeterminations about file-size, packet-size, and round-trip times, etc. Using the extended model, we predict the TCP performance when the link utilization increases. We also show a method of dimensioning the bandwidth needed to maintain TCP performance. We show the effectiveness of our method through simulation analysis.

Identifying elephant flows is very important in developing effective and efficient traffic engineering schemes. In addition, obtaining the statistics of these flows is also very useful for network operation and management. On the other hand, with the rapid growth of link speed in recent years, packet sampling has become a very attractive and scalable means to measure flow statistics; however, it also makes identifying elephant flows become much more difficult. Based on Bayes' theorem, this paper develops techniques and schemes to identify elephant flows in periodically sampled packets. We show that our basic framework is very flexible in making appropriate trade-offs between false positives (misidentified flows) and false negatives (missed elephant flows) with regard to a given sampling frequency. We further validate and evaluate our approach by using some publicly available traces. Our schemes are generic and require no per-packet processing; hence, they allow a very cost-effective implementation for being deployed in large-scale high-speed networks.

We propose a method of IP traffic management where the quality of TCP performance at a bottleneck link is estimated from monitored data on the behavior of the number of active flows versus utilization of the link, each of which is easy to measure. Our method is based on the characteristics that (i) TCP performance remains constant until the level of link utilization exceeds some threshold value, but becomes degraded when the utilization exceeds this value and (ii) the number of active flows increases linearly with utilization of the link up to the same value, above which the increase becomes nonlinear. Though this threshold may vary from network to network, our method requires neither predetermination of a threshold, on the basis of assumed traffic conditions, nor direct measurement of TCP performance.

The i-mode service has gained rapidly in popularity and it saw the number of users exceed 34 million in July 2002. In addition, the number of packets that the PDC-P (Personal Digital Cellular-mobile Packet data communication system) processes for the i-mode service has increased. It has therefore become important to design the system so that it can cope with the increasing volume of packet data. This paper analyzes the characteristics of i-mode traffic, and discusses an appropriate congestion control mechanism, which has been adopted in the PDC-P system.

We describe how to allocate bandwidth fairly to each user in a differentiated services architecture. Our method estimates the number of active users in each class by simple traffic measurement. Using this estimate, it dynamically changes the weight assigned to each class queue and adaptively updates the target rate used for selective packet discarding. By doing this, it can cope with changes in traffic conditions. We call this method dynamically weighted queueing. In this paper, we evaluated its performance under various heterogeneous conditions, i.e., when there are users who have different numbers of TCP flows, have different access link rates, generate UDP flows, and are bottlenecked elsewhere. Simulation showed that this method can achieve fair bandwidth allocation to each user under any condition.

This paper investigates the characteristics of ABR explicit rate control algorithms in the WAN environment. We also propose two new control algorithms based only on traffic measurement. An ABR service is expected to make it possible to utilize bandwidth effectively by closed-loop congestion control, and several control algorithms have been proposed. However, for WAN environment, a long propagation control delay will affect the effectiveness of control algorithms. Additionally, if a network supports large numbers of connections, some of the algorithms may not work well. Thus, we first categorize the aim and mechanism of control algorithms, including our proposed algorithms. Then, we evaluate their characteristics by simulation in WAN environment from the viewpoint of throughput, robustness, quickness, stability, and fairness. Using this analysis, we discuss their effectiveness in WAN environment.