Big data analytics helps us to find potentially valuable knowledge, but as the size of the dataset increases, the computing cost also grows exponentially. In our previous work, BotCluster, we had designed a pre-processing filtering pipeline, including whitelist filter and flow loss-response rate (FLR) filter, for data reduction, which intended to wipe out irrelative noises and reduce the computing overhead. However, we still face a data redundancy phenomenon in which some of the same feature vectors repeatedly emerged. In this paper, we propose a data compacting approach aimed to reduce the input volume and keep enough representative feature vectors to fit DBSCAN's (Density-based spatial clustering of applications with noise) criteria. It purges the redundant vectors according to a purging threshold and keeps the primary representatives. Experimental results have shown that the average data reduction ratio is about 81.34%, while the precision has only slightly decreased by 1.6% on average, and the results still have 99.88% of IPs overlapped with the previous system.

A MapReduce cloud becomes a key to the success of cloud computing today. However, a MapReduce cloud uses a single Master node as the brain to manage tasks distributed over Slave nodes for controlling the entire progress of the application execution. Accordingly, a MapReduce cloud easily overloads the Master node with reports sent from Slave nodes at run time to harm performance. Besides, a MapReduce cloud makes the Master node a single failure point to suspend the application execution when the Master node cannot work. A MapReduce cloud can use the Distributed Control Framework (DCF) proposed in this paper to improve both performance and fault tolerance, because DCF shifts most works of a Master node to a DCF Master Agent coexisting in each Slave node and allows Slave nodes to join or leave a cloud at run time without interrupting the application execution. According to observations on experiments with various applications in this paper, a MapReduce cloud can use DCF to have better performance and fault tolerance in comparison to a native MapReduce cloud.

MapReduce greatly alleviates the burdens of programmers and gradually becomes an application programming standard on cloud computing nowadays, because the run-time system of cloud computing can automatically handle the issues of paralleled and distributed programming on behalf of programmers at run time. Although MapReduce can strongly benefit programmers on developing cloud computing applications, intermediate data skew inevitably hurts application performances. MapReduce can use the Smart Partitioning Mechanism (SPM) proposed in this paper as an alternative solution to deal with intermediate data skew in Reduce tasks on cloud computing. With the capability of averagely distributing intermediate data over Slave nodes in SPM, MapReduce no longer suffers from the performance penalty resulting from the intermediate data skew problem in Reduce tasks on cloud computing.

MapReduce gradually becomes the de facto programming standard of applications on cloud computing. However, MapReduce needs a cloud administrator to manually configure parameters of the run-time system such as slot numbers for Map and Reduce tasks in order to get the best performance. Because the manual configuration has a risk of performance degradation, MapReduce should utilize the Computation Capability Deduction Architecture (CCDA) proposed in this paper to avoid the risk. MapReduce can use CCDA to help the run-time system to distribute appropriate numbers of tasks over computers in a cloud at run time without any manual configuration made by a cloud administrator. According to experiment observations in this paper, MapReduce can get great performance improvement with the help of CCDA in data-intensive applications such as Inverted Index and Word Count that are usually required to process big data on cloud computing.

IPv6 is a technology that provides enormous address space and end-to-end communication, features that are required in the context of the device automation integration for future network. The transition to IPv6 holds the future of the internet infrastructure. Software-defined networking (SDN) defines a new concept for computer networks that can separate and provide abstract elements of network devices. IPv6 SDN has the potential to revolutionize the network design, construct and operate networks to achieve more efficient business network agility. In this paper, we will discuss the main architectures of SDN and illustrate how IPv6 can be deployed and integrated in SDN technologies using OpenFlow mechanisms. We will also discuss the IPv6 impact on link performance and deployment scenarios.

The challenges of the Big Data era has motivated many organizations to turn towards distributed, large-scale processing platforms to deal with their data. Map Reduce, and its open-source implementation, Hadoop, has grown to be highly popular with its successful programming model for simplified cluster processing. As a result, many organizations deploy their own Map Reduce/Hadoop clusters to store and process large amounts of useful data. This multicluster setting is gradually growing attention. Numerous previous works have researched on how to execute Map Reduce across geographically distributed data in this setting. However, an important class of applications have not been explored for multicluster Map Reduce: iterative computation. In this paper, we propose Fed Loop, a composite system aimed at providing iterative Map Reduce computation for geographically distributed data in multicluster settings. Fed Loop is capable of transparently executing both iterative and non-iterative Map Reduce jobs on either a single cluster or multiple clusters. For our performance evaluation, two well-known iterative algorithms was executed over 4 independent clusters (16 physical nodes in total) using Fed Loop: K-Means and Page Rank. Results helped us discover how different iterative applications may differ in execution efficiency for mutlicluster environments and how iterative multicluster computation systems like Fed Loop can be optimized.

Forward Error Correction (FEC) is one of the most common means of performing packet error recovery in data transmissions. FEC schemes typically tune the FEC rate in accordance with feedback information provided by the receiver. However, the feedback and FEC rate calculation processes inevitably have a finite duration, and thus the FEC rate implemented at the sender may not accurately reflect the current state of the network. Thus, this paper proposes an Enhanced Random Early Detection Forward Error Correction (ERED-FEC) mechanism to improve the quality of video transmissions over Wireless Local Area Networks (WLANs). In contrast to most FEC schemes, the FEC redundancy rate is calculated directly at the Access Point (AP). Moreover, the redundancy rate is tuned in accordance with both the wireless channel condition (as indicated by the number of packet retransmissions) and the network traffic load (as indicated by the AP queue length). The experimental results show that the proposed ERED-FEC mechanism achieves a significant improvement in the video quality compared to existing FEC schemes without introducing an excessive number of redundant packets into the network.

With the increase in the bandwidth needs of network service, the customer requirements for Quality of Service (QoS) are also higher. Although the QoS guarantees can be offered by signing the Service Level Agreement (SLA), the costs of infrastructures are increased as well. This paper proposes a shortest path algorithm based on SLA, combined with the resource allocation for virtual networks (VNs). It can help service providers to enhance the achievement of overall SLA contracts in the case of lack of resources. This architecture was also implemented on NetFPGA platforms applied with the concept of OpenFlow to meet the demands of network performance and flexibility. Finally, the experimental results show that the proposed algorithm can dynamically provide the better path to reach the SLA achievement rates.

With the advancement of network virtualization, physical and virtual network managements become more and more complex and difficult. This article presents a framework, Hierarchical Management System of Virtual Network (HMSVN), including the functions of network virtualization and the features of hierarchical management. The system is implemented on NetFPGA to provide the solution for virtual network management. Experiments show the precise operations of virtual network by network flows and hierarchical management.

With the rapid development of network technology and popularity of virtualization, the importance of developing an adaptive management system for such physical and virtual network environment is obvious increasing. Not simply do the demand systems keep the convenience of virtualization, but conform to the hierarchy of system administration. This article presents the Multilayer Virtual Network Management System, integrated the architecture of network virtualization and the features of multilayer network management. The system is implemented by Net FPGA network development platform and provides a solution of network management for a large scale network. Experiments demonstrate the exact execution of Virtual Network through the policies of network flows and the multilayer management of the proposed system.

MapReduce is an important programming model for large-scale parallel applications. It divides a job into several parallel tasks and completes the job by sequential phases, i.e. map phase and reduce phase. The job completion time will be delayed when a task, called straggler, consumes more time than others. The main reason that a straggler occurs is the imbalance resource distribution among computing nodes in the cloud. Speculative execution is a solution for dealing with stragglers. Duplicate tasks are launched on other nodes to process the same data as the straggler does. Any completion of these tasks implies that this task is finished and other duplicate tasks can be aborted. However, aborting tasks misspends resources. In this paper, we propose an idea of using coworkers to help a straggler. According to the processing rate of the straggler and the coworker, the amount of data parceled out from the straggler to the coworker should be determined. Different from speculative execution, coworkers finish tasks with stragglers and do not misspend computing resources. Experimental results show that coworkers can reduce the task completion time by 37% and the network traffic by 64% when comparing with speculative execution.

Digital multimedia broadcasting services require resiliency against signal fading due to signal encounters with bad weather, terrain obstructions, such as tree, building, etc. There are several solutions that use peer-to-peer network to repair packet losses. But they have some shortcomings such as long lookup latency, control message overhead, flash crowd problem and low chunk availability. In this paper we propose a novel solution that considers load distribution in structured peer-to-peer networks and repairing packet losses in short lookup latency. Our proposed solution doesn't need to modify the codec technology for multimedia stream. Our proposed solution can achieve load evenly distributed, high chunk availability, short lookup latency and low control message overhead by using peer-to-peer network to repair packet losses in digital multimedia broadcasting systems.

The IEEE 802.16j standard supports multihop relay function for WiMAX network. The relay technique could enhance the signal strength and help the devices to compete the shadow fading effect from buildings. How to allocate resource becomes an important issue in recent research. In this paper, we discuss the resource allocation with directional antenna in IEEE 802.16j networks. In prior study, the frequency is equally divided according to the number of sectors in a cell. However, the prior allocation schemes often result in waste resource or insufficient bandwidth if the user distribution is not even. The proposed dynamic resource allocation (DRA) scheme considers the user distribution and allocates resource blocks from users' bandwidth request. The simulation results show the improvements from the DRA scheme in terms of utilization of frequency bands and duration of transmission time.

Abstract Delivering video streaming over wireless Internet is becoming increasingly popular. However, most of the research studies focused on the modeling analysis of system performance such as saturation throughput and channel utilization. Perceived quality of video streaming cannot be assessed solely based on the results of analytical models. In this paper, we propose a model to assess the perceived quality of MPEG‐4 video streaming over IEEE 802.11 distribution coordination function (DCF)‐based wireless local area networks. The analysis of our proposed model considers not only effects of losses such as collision loss from channel access competition but also wireless loss caused by wireless interferences. Moreover, the impact of the loss of specific MPEG‐4 video frames is also taken into account in the performance analysis. The model was validated by comparing our performance results with results obtained from simulation and analytical models. The results show that our proposed model is able to predict the perceived quality of MPEG‐4 video streaming over DCF‐based WLAN more accurately than other models. Copyright © 2010 John Wiley & Sons, Ltd.

Multicore processor provides large computation capability but also involves the complicate parallel programming. One of major considerations in parallel programming is the performance. Traditional design methodologies which usually start a design on a selected platform spend a lot of effort and time on tuning performance and debugging. When platform is changed even with different number of cores, considerable redesign effort is required. Hence a flexible design methodology is necessary. In this paper, a design methodology is presented for video codec, by using MPEG-4 SP decoder as an example, on multicore processor. The parallelisms of MPEG-4 decoder are discussed and exposed with the dataflow model. The dataflow model provides a high-level abstraction of underlying hardware. Computation and communication of MPEG-4 decoder are separated and represented as modules and channels, respectively. It is possible to synthesize the model targeting to either dedicate hardware or software on multiprocessor. To map the high level dataflow model to Cell processor, the mapping flow, including offline profiling, task allocation and runtime libraries, are developed. According to the profiling results, the allocation algorithm could allocate tasks on multiprocessors as balanced as possible. An efficient synchronization mechanism on Cell processor is also proposed. We also discuss the impact of the model and the mapping flow corresponding to decoding speed. The results show that the proposed methodology gets considerable performance boost when the number of cores is increased.

Video streaming applications over wireless networks present many challenges owing to the high packet error rate and the error characteristics of consecutive packets. In order to solve these problems, this paper proposes a cross-layer Unequal Error Protection (UEP) Forward Error Correction (FEC) mechanism with an interleaved strategy for video streaming over wireless networks. The proposed novel UEP mechanism uses a playable frame rate (PFR) model to determine the appropriate FEC redundancy for a varied type video frame. The UEP mechanism requires the video information to be obtained from the application layer in order to allocate FEC redundancy to each video frame in the Internet Protocol (IP) layer. Moreover, this paper aims to find out the appropriate length and depth of interleaving to overcome errors in consecutive packets. Our simulation results demonstrate that the proposed mechanism outperforms a related FEC mechanism with interleaving terms of playable frame rate.

Abstract Conventional remote data access middlewares usually provide client applications with either a pre‐staging scheme or an on‐demand access scheme to fetch data. The pre‐staging scheme uses parallel downloads to fetch a completed input file from multiple data sources, even when only a tiny file fragment is required. Such a transfer scheme consumes unnecessary data transmission time and storage space. In contrast, the on‐demand scheme downloads only the required data blocks from a single data source and does not fully utilize the downstream bandwidth of the computing nodes. This paper presents a middleware called ‘Spigot’ that facilitates legacy (grid‐unaware) applications to transparently access remote data by using native I/O function calls. Spigot uses the on‐demand concept to avoid unnecessary data transfer and adopts a co‐allocation download algorithm to improve the data transfer performance. Moreover, it uses the pre‐fetching strategy to reduce the data waiting time by overlapping data acquisition and data processing. It also provides the client application with its own user‐level cache, which is advantageous since a larger cache space is available in comparison with the kernel‐level cache. Further, it is easy to maintain data consistency between Spigot nodes. The experimental results indicate that Spigot achieves superior performance in reducing the data waiting time than the pre‐staging and the on‐demand access schemes. Copyright © 2010 John Wiley & Sons, Ltd.

These days, a wide variety of wireless interfaces are available to connect to Internet. When coverage area of these different technologies overlap, receiver equipped with multiple interfaces can use them simultaneously to improve the performance of its applications in terms of bandwidth rely on bandwidth aggregation. However, specific conditions such as bottleneck bandwidth and end-to-end delay need to be accounted for before using such techniques. If this problem of end-to-end delay and bottleneck bandwidth are not properly addressed, there may be many packets along multiple paths which can arrive late and can lead to a large number of out-of-order packets at the receiver, which can eventually cause serious degradation of video quality at the receiver. For this reason, in this study, the authors propose a multi-path transmission control scheme (MTCS) combining bandwidth aggregation and packet scheduling for real-time streaming in a multi-path environment. In a bandwidth aggregation scheme, the authors propose a mathematical model to find the transmission rate over each path in order to obtain the optimal total throughput. However, the end-to-end delay of each path is not the same. The out-of-order packets problem will become serious in a multi-path environment. Therefore the authors propose a packet scheduling scheme to arrange the transmission sequence in order to effectively minimise the impact of packet reordering at the receiver. Our proposed control scheme not only aggregates the available bandwidth of multiple paths, but also reduces the time of packet reordering at the receiver. Experimental results show with our proposed scheme, the authors not only obtain the optimal transmission throughput but also reduce packet reordering delays under varying drop and delay conditions caused by the underlying network.

Network Address Translation (NAT) alleviates the shortage of IPv4 addresses but incurs peer-to-peer communication, application functionality and packet integrity problems. To date, no approach has yet been proposed to solve these three problems. By exploiting mobile agent and active networking technologies, we propose a Programmable Network Address Translation (PNAT) implementation that enables peer-to-peer communication while maintaining application functionality and packet integrity. For peer-to-peer communication, our proposed PNAT approach works for various NAT types (including the Symmetric NAT) with simple APIs supported by our proposed NAT design. For application functionality, the PNAT uses the mobile code to update protocol information in packet payloads according to different application needs. For packet integrity, the PNAT allows applications to delay their data encryption until NAT begins to translate addresses and ports in packet headers. To validate our proposed PNAT approach, we implemented the PNAT design on Windows 2000, and we present an empirical performance evaluation of the implemented design.

Conventional data grid systems require the application to access remote data through explicit APIs, consequently sacrificing user transparency. Thus, some systems enable the application to transparently access remote data by copying the entire file to a userpsilas local storage before executing the application, even when only a tiny file fragment is required. Such an approach consumes unnecessary transmission time and storage space, with the additional problem of maintaining replica synchronization. Traditionally, replica consistency treats shared files as read-only, consequently sacrificing guaranteed replica consistency. This paper presents a DSM-based fragment-level file sharing framework called ldquoSpigotrdquo. Spigot allows the users to design their applications with the POSIX I/O interface. Moreover, Spigot transfers only the necessary file fragments on user demand, thereby reducing transmission time, wasted network bandwidth and required storage space. Data waiting time is further reduced by overlapping data transmission and data analysis. The DSM concept maintains replica synchronization. Real experiments show reduced turnaround times in data-intensive applications.

Grid data sharing systems usually provide a data-intensive application with either a prestaging mechanism or an on-demand access mechanism to access shared data. Prestaging systems simultaneously download an entire shared file from multiple data sources even when only a tiny file fragment is required. Such systems consume unnecessary data transmission time and storage space. On demand access systems, on the other hand, download only the required fragments from a single data source. Such systems unfortunately do not fully exploit available network bandwidth. This paper presents a data sharing system, designated as the on demand data coallocation (ODCA). ODCA facilitates an unmodified legacy applications to transparently access shared data by using native I/O system calls. ODCA transfers only the necessary fragments on user demand, thereby reducing data transmission time, avoiding wasted network bandwidth and wasted storage space. Moreover, ODCA reduces data waiting time by downloading the file fragments from multiple data sources. Experimental results show ODCA successfully reduces turnaround time in data-intensive applications.

Active networking introduces a novel programming paradigm to applications by injecting codes into networks in order to program them. Among the approaches to injecting codes into networks, active networking attracts most research communities to explore the capsule-based active network, where the packet named the capsule can carry codes and data together to customize the processing of the packet in network devices with the codes. Because a capsule has space limitations that harm application developments in capsule-based active networks, the just-in-time cache (JITC) is proposed to alleviate capsule space limitations. The JITC is an innovative idea to introduce a cache into a capsule besides into a network device. The JITC automatically tailors codes to the need of a capsule while processing the capsule in a network device. The JITC allows a capsule merely to carry the necessary codes to traverse active networks and makes a network device download codes on behalf of the capsule on demand. The JITC allows applications to disable the cache, control the cache size, and specify the required cache placement policy in a capsule to achieve the optimal performance. Currently, the JITC supports six canonical cache placement policies. The JITC is implemented in Windows 2000 as a proof of concept. In the experiments, the JITC has trivial overheads in contrast to other common overheads in the existing solutions. The JITC outperforms the existing solutions in throughput, processing delay, resource sensitivity, and bandwidth conservation in the cache-policy-free situation. Besides, the JITC has examples for the six cache placement policies to demonstrate the performance improvement with each of them.

Video streaming over wireless networks have many challenges due to the high error rate and burst packet error characteristic. Forward error correction (FEC) is a method commonly used to handle losses in real-time communication. Conventional FEC mechanisms provide redundancy by an averaged packet loss rate and performance decreases by burst packet losses. However, the average packet loss rate cannot give any indication of burst packet loss. Hence, the conventional FEC mechanisms cannot recover original source data over wireless networks. In this paper, we propose a burst-aware adaptive FEC (BAFEC) control mechanism to overcome burst packet losses. We will therefore be able to take account of average burst packet loss length. The sender can rely on this information in order to adjust the FEC redundancy. The experimental result shows that compared to the conventional FEC mechanisms, our proposed mechanism achieved better recovery performance in terms of packet loss rate and PSNR.

The purpose of this paper is to study the packet loss effect on MPEG video transmission quality in wireless networks. First, we consider the distribution of packet losses in wireless network, including distributed and burst packet losses. Besides, we also discuss the additional packet drops by the play-out buffer at the received end, and the effect of the transmission packet size on the video delivered quality. From the results, we find that the effect of burst packet losses on the video delivered quality is less than distributed packet losses in the same packet loss rate. Moreover, the smaller size of the play-out buffer leads to more packet drops and worse video quality. Finally, if there is no video recovery for video transmission, the video delivered quality of the larger packet size will be better than smaller packet size.

Abstract Achieving an appropriate workload distribution is essential if user applications are to achieve a satisfactory performance on software distributed shared memory (SDSM) clusters. To address this problem, the present study develops a novel method for distributing program threads onto the individual computers of SDSM clusters. In contrast to alternative methods, the proposed approach takes account not only of processor speed and data‐sharing aspects, but also memory availability, such that application performance can be enhanced through the implementation of appropriate workload distributions. In addition, when distributing the program threads, the proposed method specifically chooses only those computers that can enhance the performance of the application, rather than simply distributing the threads to all the available nodes in the cluster. This location policy makes it possible to specify the appropriate node combinations that optimize program performance while simultaneously maximizing resource utilization. The proposed method is implemented on a testbed referred to as Teamster. The experimental results demonstrate that, compared to alternative methods, the proposed approach delivers a 20–30% improvement in the performance of the chosen test applications. Importantly, it is shown that the proposed method can efficiently specify proper node combinations for the applications. Copyright © 2006 John Wiley & Sons, Ltd.

This paper is aimed at resolving the reconfiguration problem of software distributed shared memory (DSM) systems in non-dedicated clusters. Different to the past studies focused on dedicated clusters, the goal of this study focuses on prompting system-wide jobs' throughput rather than DSM programs' performance. We invent an approach called progressive multilayer reconfiguration (PMR) for DSM systems. As named, reconfiguration is divided into three different layers, i.e., processor, application, and node in this approach. According to the state transfer of the workload, the three different layer reconfigurations are progressively and respectively performed during the execution of DSM applications. The preliminary results show that PMR can not only utilize abundant CPU cycles available in non-dedicated clusters for DSM applications but also minimize the slowdown of local jobs caused by the disturb from DSM applications.

This paper proposes an improvement to the TCP ramp up behavior in slow-start. Current implementations of the TCP start-up procedure may result in an exponential growth of the congestion window, which results in the transmission of an excessive number of packets. Consequently, the network becomes congested and the self-clocking mechanism of the TCP fails. By estimating the number of packets that which can be in flight in the network from the viewpoint of the TCP source, this study develops an algorithm to suppress congestion before it becomes significant. The simulation results indicate that the proposed algorithm enhances the TCP performance by establishing a smoother transmission rate at the end of the slow-start phase and reducing the number of dropped packets significantly as a result.

DNS redirection and URL modification are two major ways to carry out dynamic replica selections in CDN. However, the two ways may overload DNS servers or dispatchers, and exhaust the network bandwidth while there are a large number of incoming requests. Meanwhile, the modification of DNS servers for DNS redirection harms the compatibility and troubles the management. Moreover, the two ways do not give local administrators the flexibility to customize accesses to foreign CDN applications. In the paper we propose Java Application's Packet Eavesdropper (JAPE) for CDN to improve DNS redirection and URL modification, and for local administrators to control the accesses. JAPE utilizes packet interception technologies for Java objects to process packets. Besides, objects in JAPE are Java application rather than Java applet, which not only keeps the inherent portability but also has the flexibility to use various Java libraries for creating other applications not limited to CDN. We implement JAPE in Windows 2000 and develop a CDN application with an object doing DNS redirection and URL modification. The application with location-aware features can locally resolve clients' requests toward a virtual host name and redirect them to a nearby server customized by local administrators according to the movements of clients. It proves that JAPE for CDN can free DNS servers from modifications, alleviate DNS servers' and dispatchers' loads, conserve the network bandwidth, and permit local administrators to customize the accesses.

Providing users with a familiar programming tool to develop applications is an important issue of grid computing. Currently, much effort has been put on the grid implementation of MPI, Java and RPC. However, a little of work has been done for enabling the use of software distributed shared memory (DSM) system in the grid environment although software DSM systems offer an easier programming user interface than the others. In order to simplify programming in grids, we have developed Teamster-G, a grid-enabled software DSM system that allows users to run their DSM programs on a virtual dedicated homogenous cluster which is the coupling of multiple computers physically distributed at the same or different sites. In this paper, we present the framework of Teamster-G and discuss the preliminary results of performance evaluation in Teamster-G.

Network Address Translator (NAT) is the well-known, transitional method to mitigate the problem of IPv4 address depletion in today's Internet. However, the assignment, translation, and export of address/port in a NAT at run time affect application functions. Accordingly, application servers behind the NAT cannot accept requests directly from public networks. Sensitive applications cannot hold their end-to-end security mechanisms. Applications lose connections after the NAT reboots or changes the binding address/port. However, current proposals for NAT traversal hardly solve the problems. Against the problems, we propose Smart Tunnel Union for NAT Traversal (STUNT) in the paper. STUNT permits applications behind the NAT to be actively contacted by Internet clients, keeps end-to-end security mechanisms, and avoids the risk of exporting binding information of the NAT to connection endpoints. Meanwhile, it permits applications to traverse the NAT and keeps the NAT intact.

A suitable workload distribution is essential for obtaining good performance on software distributed shared memory (DSM) systems. The methods proposed by the past DSM researches only simply distribute program threads in accordance with the CPU power of the individual processors or the data sharing within the DSM applications. These methods expected the minimal program execution time by reducing computation cost and communication cost. However, memory access costs also play a significant role in determining the program performance. If a processor lacks sufficient memory space to cache all of the data required by the working threads, then it will suffer from a series of page faults and will need to execute consecutive page replacements while executing its threads. Although the threads will be able to complete their tasks, their execution will inevitably be delayed by the latency associated with executing the page replacements. To resolve this problem, this paper is dedicated to involving memory resource consideration into workload distribution for software DSM systems. The present results confirm the importance of considering memory resources when addressing the workload distribution of DSM systems. It is shown that the proposed method is more effective than previous methods which considered only CPU resource or memory resource.

IntServ and DiffServ networks are both proposed to promote quality of service (QoS) for multimedia applications in the Internet. However, the complexity of communication between diverse applications and underlying QoS architectures leads to one of the deployment problems which decreases the utility of QoS provisioning. Without modifying legacy applications, the paper presents a transparent QoS mechanism (TQM) to communicate with underlying QoS architectures and also provide swappable modules to support different QoS setups for diverse applications. Currently, TQM is implemented on the UNIX platform, and some important issues of applying TQM in IntServ/DiffServ networks have been examined.

Workload distribution is an important issue for user applications to obtain a good performance on software distributed shared memory (DSM) systems. The workload distribution methods that were previously proposed for DSM systems always considered only CPU power and data sharing but ignored memory availability of processors. However, the cost of memory accesses is also an important factor of determining program performance especially when processors have not sufficient memory space to cache all the data needed by program threads. Therefore, the past methods might make wrong decisions in workload distribution to degrade program performance. To resolve this problem, this paper is dedicated to adding memory resource consideration into workload distribution for software DSM systems. Our experimental result shows that one workload distribution method with memory resource consideration is indeed more effective for minimizing the execution time of DSM applications than another without memory resource consideration.

Local area network (LAN) will be a hybrid network that includes wired and wireless links together. Nonetheless, the wired hosts always take the most bandwidth and bring about the bandwidth allocation unfairness. This problem is caused by the flow control mechanism that is dominated by the round-trip time (RTT). Among the connections on a share link, the ones with shorter RTT tend to acquire bandwidth easier and exhaust the link bandwidth eventually. The RTT of wireless transmission is much longer than wired transmission. Therefore, the protocol that implements flow control mechanism such as TCP suffers a severe low performance problem on wireless links. This paper proposed a new scheme called DFQ that works on gateway. It prevents the wired hosts from grabbing the bandwidth of share link too much and averts the unfairness or starvation of wireless connections. The experiment result shows that DFQ is practicable and makes wireless network to work with wired network seamlessly.

Teamster is a transparent DSM system built on a cluster of symmetric x86 multiprocessors connected with 100 Mb Fast Ethernet. Teamster has a hybrid thread architecture so that a programmer can parallelize the application without concerning the underlying hardware configuration. The Global Memo Image of Teamster provides a truly global and synchronization objects are put into GMI. Because we declare and initialize these data and objects in the beginning, the linker of the operating system helps us forming this and need explicit annotations to propagate the modification of global static variables. With the support of the hybrid thread architecture and Global Memory Image, Teamster can provide a truly transparent DSM in the cluster of SSMP computers. More than that, the overhead of creating more application threads and supporting the Global Memory Image does not affect the performance of Teamster in our measurement.

Load balance is an important issue for the performance of software distributed shared memory (DSM) systems. One solution for addressing this issue is to exploit dynamic thread migration at runtime. In order to reduce the data consistency communication increased by thread migration, an effective load balance scheme must carefully choose the threads and the destination nodes for workload migration. A group-based load balance scheme is proposed to resolve this problem. The main characteristic of this scheme is to classify the overloaded nodes and the lightly loaded nodes into a sender group and a receiver group, and then consider all the threads of the sender group and all the nodes of the receiver group for each thread migration decision. The experimental results show that the group-based load balance scheme reduces more communication than previous methods. Besides, the paper also resolves the problem of the high overhead caused by group-based schemes. Therefore, the performance of the test programs is effectively enhanced after minimizing the communication increased by thread migration.

The paper presents an implementation of QoS library redirection (QLR) for DiffServ in Microsoft Windows systems. In the QLR method, the legacy application obtains packet pre-marking ability without any modification of source codes. We have upgraded the FTP and vic (video conferencing) applications by using QLR in our Microsoft Windows testing platform. Our experimental results show that this approach can use really effectively and provide high performance of the QLR in Microsoft Windows systems.

This paper proposes a transparent packet pre-marking method named QLR (QoS Library Redirection) for DiffServ. This method can upgrade legacy applications to obtain packet pre-marking ability without any modification of source codes. In this case, users are allowed to purchase high or low service quality according to their requirements. The major techniques of QLR include the dynamic linking of shared libraries and the function call redirection. We have upgraded the vic application by using QLR in our FreeBSD testing platform. The experimental results can show the effectiveness of the QLR.

Parallel applications executing on large-sized parallel systems achieve better speedup than on small-sized systems. However, because of the design and implementation of the distributed shared memory (DSM) system, there are some instances where large system size gives no further performance improvement over small system size. It is important to determine what system size will result in the maximum speedup while all kinds of applications are running on DSM systems. In this paper, we describe the design and implementation of the performance prediction mechanism in our DSM system, Proteus, which supports node reconfiguration to adjust the system size at run time. We adopt a simple computation model and combine it with run-time information to predict the performance under different system sizes. With this mechanism, it is possible to provide timely prediction results to adjust the size of the underlying system and thus maximize speedup.

Load balance is an area of current research in software distributed shared memory (DSM) systems. When threads are dynamically migrated from heavily loaded nodes to lightly loaded nodes to achieve load balance, the communication cost of maintaining data consistency is increased if migration threads are carelessly selected. Program performance is degraded when loss from increased communication exceeds the benefit from load balancing. Therefore, load balancing requires careful choice of migration threads. This study addresses the problem with a novel selection policy called Reduce Internode Sharing Cost (RISC). The main characteristic of this thread selection policy is simultaneous consideration of both thread memory access types and global sharing. Experimental application of this policy to a DSM system called Cohesion shows that simultaneous consideration of memory access types and global sharing is necessary for thread selection. RISC can reduce 50% data-consistency communication of benchmark applications during execution of the load balance mechanism.

This paper describes a Distributed Shared Memory (DSM) system called Proteus, which aims to support runtime node reconfiguration. To make the system execute efficiently after node reconfiguration and reduce the overhead of reconfiguration, Proteus employs an adaptive workload redistribution strategy to perform workload redistribution as well as an affinity page movement strategy to redistribute memory pages. Furthermore, a forced update technique and a set of scalable synchronization mechanisms are developed to keep memory consistent and to perform a correct synchronization, respectively.

The purpose of the paper is to demonstrate an efficient thread architecture for a distributed shared memory (DSM) system on symmetric multiprocessor (SMP) clusters. For DSM systems on SMP, how to utilize the processors efficiently without wasting available computational power is a major issue. We discuss three approaches that use the process, the kernel level thread, and the user level thread to map application threads onto execution entities respectively. Considering the advantages and disadvantages of each method, we construct our thread package by combining both the user level thread and the kernel level thread. User level threads correspond to application threads and kernel level threads schedule these user level threads across multiple processors. Threads are light weighted and can be migrated in our thread package. With this thread architecture, our DSM system performs well in elementary experiments.

The main goal of parallel file subsystem on Distributed Shared Memory (DSM) systems is to reduce the network traffic in page based software DSM systems, thereby improving system performance. Our laboratory has built a prototype of the parallel file subsystem on two DSM systems, namely Cohesion and TreadMarks. But these two prototypes have several limitations: users must read/write the whole parallel file in a single access; users cannot modify an existing parallel file; the parallel file request must be issued from the root node. In our new parallel file subsystem on Teamster, a new DSM system developed by our laboratory, we eliminate the limitations revealed in the two previous parallel file subsystems. In addition, we have developed two new mechanisms, the software cache mechanism and the asynchronous file offset mechanism, to lessen the performance degradation caused by the frequent small accesses.

Three basic organizations have been proposed to construct the task queues for a shared memory multiprocessor system: centralized, distributed, and hierarchical structures. The centralized structure is not suitable for massively parallel systems since the centralized queue is a bottleneck. The distributed structure, on the other hand, will end up with load imbalancing due to the random execution time of tasks. The hierarchical structure tends to combine the advantages of the previous two structures and reduces the impact of bottleneck and load imbalancing. However, we found out that the load imbalancing still exists in the hierarchical structure. It also affects system performance, in particular, when the workload becomes heavier. We further identified the cause of this problem. We propose the use of a forest structure to provide load balancing and contention minimization.