The authors present a fault injection and monitoring environment (FINE) as a tool to study fault propagation in the UNIX kernel. FINE injects hardware-induced software errors and software faults into the UNIX kernel and traces the execution flow and key variables of the kernel. FINE consists of a fault injector, a software monitor, a workload generator, a controller, and several analysis utilities. Experiments on SunOS 4.1.2 are conducted by applying FINE to investigate fault propagation and to evaluate the impact of various types of faults. Fault propagation models are built for both hardware and software faults. Transient Markov reward analysis is performed to evaluate the loss of performance due to an injected fault. Experimental results show that memory and software faults usually have a very long latency, while bus and CPU faults tend to crash the system immediately. About half of the detected errors are data faults, which are detected when the system is tries to access an unauthorized memory location. Only about 8% of faults propagate to other UNIX subsystems. Markov reward analysis shows that the performance loss incurred by bus faults and CPU faults is much higher than that incurred by software and memory faults. Among software faults, the impact of pointer faults is higher than that of nonpointer faults.>

A user-oriented synthetic workload generator that simulates user file access behavior on the basis of real workload characterizations is described. The workload generator is designed for experiments and simulations related to file system analysis. The model for this workload generator is user-oriented and job-specific, represents file I/O operations at the system call level, allows general distributions for usage measures, and assumes independence in the file I/O operation stream. The workload generator consists of three parts that handle specification of distributions, creation of an initial file system, and selection and execution file I/O operations. Results from experiments on a network file system (NFS) verify that the workload generator can produce realistic workloads and demonstrate the application of the generator.>