Parallel Programming and C++


Question

I've been writing a lot recently about Parallel computing and programming and I do notice that there are a lot of patterns that come up when it comes to parallel computing. Noting that Microsoft already has released a library along with the Microsoft Visual C++ 2010 Community Technical Preview (named Parallel Patterns Library) I'm wondering what are the common parallel programming patterns you have been using and encountering that may be worth remembering? Do you have any idioms you follow and patterns that you seem to keep popping up as you write parallel programs with C++?

1
8
11/1/2008 5:51:12 PM

Accepted Answer

Patterns:

  • Produce/Consumer

    • One Thread produces data
    • One Thread consumes the data
  • Loop parallelism

    • If you can show that each loop is independent
      each iteration can be done in a sperate thread
  • Re-Draw Thread

    • Other threads do work and update data structures but one thread re-draws screen.
  • Main-Event Thread

    • Multiple threads can be generating events
    • One thread has to processes the events (as order is important)
    • Should try separate the Event Thread/Re-Draw Thread
      This (helps) prevents the UI from freezing
      But may cause excessive re-draws if not done carefully.
  • Work Group

    • A set of threads waits for jobs on a que.
    • Thread extract one work item from queue (waiting if none is available).
      Thread works on one work item until complete
      Once completed thread returns to queue.
17
11/1/2008 7:05:43 PM

Parallel Execution Patterns

Structured parallel programming with deterministic patterns is a high-level approach mainly based on a collection of recurrent parallel execution patterns, often referred to as algorithmic skeletons or parallel constructs, which abstract program description and hide low-level multithreading details and many complexities inherent in parallelism from the programmers .

These reusable patterns automate many parallel paradigm-related routines such as synchronization, communication, data partitioning or task scheduling and handle them internally. This high-level approach attempts traditional low-level thread lock model with more abstraction and an easier way to express parallelism and focus productivity and programmability rather than performance.

There's many commonly used patterns such as: Map-Reduce, Fork-Join, Pipeline or Parallel Loop...

Papers

"Structured Parallel Programming with Deterministic Patterns" is a paper which discuss these patterns. You can see also "MHPM: Multi-Scale Hybrid Programming Model: A Flexible Parallelization Methodology" which describe a C++ implementation of this approach named XPU.

Library

XPU is a task-based C++ library composed from a set of reusable execution patterns. It allows expression of several types of parallelism at several levels of granularity inside a single homogeneous programming model. It's easy to use and illustrates the intrest in using patterns to design parallel programs.

For example it allows expression of :

  1. Task Parallelism Pattern:

    Simple or Hierarchical Fork/Join execution pattern with some features such as automatic detection and protection of shared data.

  2. Data Parallelism Pattern:

    Parallel loop pattern with scalable data partitioning.

  3. Temporal Parallelism Pattern:

    Pipeline execution pattern.


Licensed under: CC-BY-SA with attribution
Not affiliated with: Stack Overflow
Icon