Session 11 — MPI: Message Passing

Distributed-memory parallelism. These are C + MPI programs plus one browser demo. MPI runs many separate processes that cooperate by passing messages — it cannot run inside a browser. Open the .c files to view the code, then run them with mpicc + mpirun or an online MPI playground (see "How to run" below).

▶ Open the Interactive Slides

5. Collectives, Animated browser demo

Watch Broadcast, Scatter, Gather, and Reduce move data between rank 0 and its workers. The mental model behind every collective call. Runs in-browser — show it live.

1. Hello, MPI view C code

Your first MPI program. Each process prints its rank out of size. Run order is non-deterministic — the first "aha".

2. Send / Recv Ping-Pong view C code

Two processes bounce a number with MPI_Send / MPI_Recv. Includes the deadlock trap and why send/recv ordering matters.

3. Scatter / Compute / Reduce view C code

The workhorse pattern: broadcast a size, scatter an array, sum locally, reduce to a grand total. Fan-out then fan-in (map-reduce).

4. Parallel Sort Lab 2 view C code

Scatter chunks → each process sorts its chunk locally → gather → k-way merge on rank 0. A distributed sort, end to end.

How to run these

The .c files need an MPI implementation (Open MPI or MPICH). Compile with mpicc file.c -o out and launch N processes with mpirun -np 4 ./out.

No MPI installed? Use mpitutorial.com, an online MPI playground, or any Linux box (sudo apt install mpich). MPI cannot run in a browser. Full instructions and the install notes are in README.md.

Unit IV: OpenMP (Session 10, shared memory) · MPI (this session, distributed memory) · CUDA (Session 12, GPU).