Abstract Grid computing promises access to large amounts of computing power, but so far adoption of Grid computing has been limited to highly specialized experts for three reasons. First, users are used to batch systems, and interfaces to Grid software are often complex and different to those in batch systems. Second, users are used to having transparent file access, which Grid software does not conveniently provide. Third, efforts to achieve wide‐spread coordination of computers while solving the first two problems is hampered when clusters are on private networks. Here we bring together a variety of software that allows users to almost transparently use Grid resources as if they were local resources while providing transparent access to files, even when private networks intervene. As a motivating example, the BaBar Monte Carlo production system is deployed on a truly distributed environment, the European DataGrid, without any modification to the application itself. Copyright © 2005 John Wiley & Sons, Ltd.

Firewalls/NATs have brought significant connectivity problems along with their benefits, causing many applications to break or become inefficient. Due to its bi-directional communication, huge scale, and multi-organizational nature, the grid may be one of the areas damaged most by the connectivity problem. Several ideas to deal with the connectivity problem were investigated and many systems are available. However, many issues still remain unanswered. Most systems are firewall/NAT unfriendly and are considered harmful to network security; the tussle between these devices trying to investigate pay loads and applications trying to protect their content from observation and modification must be reconciled. This paper discusses how a simple relay-based system, called XRAY (middlebox traversal by relaying), deals with these issues and provides other benefits such as flexible traffic control. This paper also discusses how relay-based traversal systems can help applications to communicate over firewalls/NATs and also complement firewall/NAT operations to help network security

Firewalls and network address translators (NATs) cause significant connectivity problems along with benefits such as network protection and easy address planning. Connectivity problems make nodes separated by a firewall/NAT unable to communicate with each other. Due to the bidirectional and multi-organizational nature of grids, they are particularly susceptible to connectivity problems. These problems make collaboration difficult or impossible and cause resources to be wasted. This paper presents a system, called CODO, which provides applications end-to-end connectivity over firewalls/NATs in a secure way. CODO allows applications authorized through strong security mechanisms to traverse firewalls/NATs, while blocking unauthorized applications. This paper also formalizes the firewall/NAT traversal problem and clarifies how a traversal system fits in the overall security policy enforcement by a firewall/NAT.

This paper describes two systems to recover the Internet connectivity impaired by private networks and firewalls. These devices cause asymmetry in the Internet, making peer-to-peer computing difficult or even impossible. The Condor system is one of those that are severely impaired by the asymmetry. Compared to normal peer-to-peer computing applications, Condor has stricter requirements, which are representative to any grid computing. To make Condor seamlessly work across private networks and over firewalls, we designed and implemented Dynamic Port Forwarding (DPF) and Generic Connection Brokering (GCB). Both DPF and GCB satisfy the representative requirements. Furthermore DPF supports dedicated large clusters very well because it is simple, efficient, and highly scalable. On the other hand, GCB perfectly supports non-dedicated or personal clusters because it is independent to private network or firewall technologies and does not require airy administrative power to deploy it. In this paper, we describe the implementations of DPF and GCB and analyze them with respect to performance, deployability, security, and scalability.

Access to remote data is one of the principal challenges of Grid computing. While performing I/O, Grid applications must be prepared for server crashes, performance variations and exhausted resources. To achieve high throughput in such a hostile environment, applications need a resilient service that moves data while hiding errors and latencies. We illustrate this idea with Kangaroo, a simple data movement system that makes opportunistic use of disks and networks to keep applications running. We demonstrate that Kangaroo can achieve better end-to-end performance than traditional data movement techniques, even though its individual components do not achieve high performance.