Collective operations move data among all processes at once — they are the workhorses of MPI. Here is rank 0 (the root) on top and four worker processes below. Press a button to watch each collective move data. Same four processes, four different communication patterns.
Why collectives beat hand-rolled Send/Recv: the MPI library
implements these in clever tree patterns (often log(P) steps instead of P),
and one readable line replaces a whole loop of point-to-point calls. In real code you reach for
MPI_Scatter / MPI_Gather / MPI_Reduce first, and only drop
down to MPI_Send/MPI_Recv when the pattern doesn't fit.