Revision Notes
2311CSC501J — Parallel Processing. Complete notes for all five units, organised by topic: definitions, hand-drawable diagrams, formulas, comparison tables and worked examples.
Each unit ends with a Quick recall section — every formula and table on one screen — and a set of self-check questions with answers. Revise from these alone and you have the whole subject. Every page prints cleanly.
Introduction to Parallel Processing
The free lunch is over · concurrency vs parallelism · latency vs throughput · Flynn's taxonomy · SPMD · shared, distributed and hybrid memory · UMA vs NUMA · fork-join · race conditions · reduction · data scoping
Parallel Architecture
Static vs dynamic networks · bus, crossbar, Omega · mesh, torus, hypercube · degree, diameter, bisection · XOR routing · memory hierarchy · cache coherence · snooping vs directory · MESI · false sharing · SMP/AMP · CPU vs GPU · SIMT · roofline
Parallel Algorithms & Design
Foster's PCAM · decomposition techniques · granularity · dependency graphs · critical path · work, span & Brent's bound · speedup & efficiency · Amdahl · Gustafson · strong vs weak scaling · static vs dynamic balancing · work stealing
Programming Models & Tools
OpenMP work-sharing, scheduling, scoping, barriers · MPI ranks, Send/Recv, the classic deadlock, collectives · CUDA host/device, thread→block→grid, the five-step pattern, memory spaces, coalescing, reduction
Applications of Parallel Computing
Domain decomposition & halo cells · Monte Carlo · shared-nothing databases · sharding · parallel queries and joins · OLTP vs OLAP · HDFS · MapReduce & the combiner · Spark · real-time systems · RMS and EDF · federated learning · edge computing · distributed AI training
How to use these
- First pass — read a unit end to end. Every diagram is drawable by hand in under two minutes; practise them, because a clear labelled figure earns more marks than extra prose.
- Second pass — work through the self-check questions. Answer out loud before opening each one. Anything you have to peek at is your revision priority.
- Night before — read only the Quick recall block of each unit. Every formula and comparison table is there.
- In the exam — always state the formula, substitute the numbers, then give the answer. Marks are awarded for the substitution step, not only the final value.