4.7 Summary

Modular design techniques are fundamental to good software engineering practice. In this chapter, we have shown how these techniques can be applied to the design of parallel programs. The major points are as follows:

1. The central tenets of modular design, such as simple interfaces and information hiding, apply in parallel programming just as in sequential programming.

2. Data distribution is an important implementation detail that, if abstracted out of a module interface, can facilitate code reuse.

3. It is useful to distinguish between sequential, parallel, and concurrent composition of parallel modules. Sequential composition is simple but inflexible. Parallel composition can be used to improve locality and scalability. Concurrent composition is the most general form.

4. Performance models can be composed, but care must be taken to account for communication costs at interfaces, overlapping of computation and communication, and other factors.

In Part II, we show how the modular design techniques introduced in this chapter can be applied when developing programs using a range of parallel programming tools.