Although distributed threads on distributed shared memory (DSM) provide an easy programming model for distributed computer systems, it is not easy to build a high performance system with them, because a software DSM system is prone to page-thrashing. One way to reduce page-thrashing is to utilize thread migration, which leads to changes in page access patterns on DSM. In this paper, we propose thread scheduling methods based upon page access information and discuss an analytical model for evaluating this information. Then, we describe our implementation of distributed threads, PARSEC (Parallel software environment for workstation cluster). Using user-level threads, PARSEC implements thread migration and thread scheduling based upon the page access information. We also measure the performance of some applications with these thread scheduling methods. These measurements indicate that the thread scheduling methods greatly reduce page-thrashing and improve total system performance.

In this paper, we first propose a class of workstation cluster which utilizes optical wavelength-division multiplexing (WDM) technology to connect nodes (work-stations) of the cluster. The Optical Bus Computer Cluster (OBCC) falls in the class of cache coherent non-uniform memory access (CC-NUMA) multiprocessors. The basic topology of the cluster is star-shaped with an optical star-coupler in the center to enable one-hop simultaneous broadcasting of information packets from one node to all other cluster nodes. WDM technology not only multiplies by N times the network bandwidth using a single optical fiber, where N is the degree of wavelength multiplexing, but also provides independent communication paths between pairs of cluster nodes by properly assigning wavelengths to inter-node communication. Then we identify the cache subsystem requirements for the OBCC and propose or deferred cache coherence protocol suitable for the OBCC. The basic coherence maintenance scheme is to lazy-evaluate the cache coherence transactions among cluster nodes, utilizing the weak consistency memory model. By deferring the transactions, it is possible to combine multiple transaction issues into one transaction by accumulating modified status bits in the enhanced cache status fields. Since not only the remote memory access but also coherence transaction are costly operations in CC-NUMA systems, the deferred invocation of coherence transactions is particularly useful in CC-NUMA systems such as OBCC. We then give a performance evaluation by simulation that the coherence protocol effectively reduces coherence transactions, particularly in situations where false sharing of longer cache lines becomes noticeable.