I've been writing a lot recently about Parallel computing and programming and I do notice that there are a lot of patterns that come up when it comes to parallel computing. Noting that Microsoft already has released a library along with the Microsoft Visual C++ 2010 Community Technical Preview (named Parallel Patterns Library) I'm wondering what are the common parallel programming patterns you have been using and encountering that may be worth remembering? Do you have any idioms you follow and patterns that you seem to keep popping up as you write parallel programs with C++?
Parallel Execution Patterns
Structured parallel programming with deterministic patterns is a high-level approach mainly based on a collection of recurrent parallel execution patterns, often referred to as algorithmic skeletons or parallel constructs, which abstract program description and hide low-level multithreading details and many complexities inherent in parallelism from the programmers .
These reusable patterns automate many parallel paradigm-related routines such as synchronization, communication, data partitioning or task scheduling and handle them internally. This high-level approach attempts traditional low-level thread lock model with more abstraction and an easier way to express parallelism and focus productivity and programmability rather than performance.
There's many commonly used patterns such as: Map-Reduce, Fork-Join, Pipeline or Parallel Loop...
"Structured Parallel Programming with Deterministic Patterns" is a paper which discuss these patterns. You can see also "MHPM: Multi-Scale Hybrid Programming Model: A Flexible Parallelization Methodology" which describe a C++ implementation of this approach named XPU.
XPU is a task-based C++ library composed from a set of reusable execution patterns. It allows expression of several types of parallelism at several levels of granularity inside a single homogeneous programming model. It's easy to use and illustrates the intrest in using patterns to design parallel programs.
For example it allows expression of :
Task Parallelism Pattern:
Simple or Hierarchical Fork/Join execution pattern with some features such as automatic detection and protection of shared data.
Data Parallelism Pattern:
Parallel loop pattern with scalable data partitioning.
Temporal Parallelism Pattern:
Pipeline execution pattern.