Real-Time Systems & Recent Trends
Session 15 • 2311CSC501J — Parallel Processing • The final session + course wrap-up
What You'll Learn
- Real-time & embedded systems: deadlines over throughput
- Federated learning — privacy-preserving parallelism
- Edge computing & distributed AI training
- The whole course, connected into one idea
"Thread → core → GPU → node → cluster → data center is one continuous idea."
— the thesis of this course
When Being On Time IS Being Correct
"A self-driving car that computes the correct braking decision 50 ms too late has computed the wrong decision."
All course we chased throughput. A huge class of systems cares about something else: being on time. A real-time system is one where correctness depends not just on the result, but on when the result arrives.
The mindset flip: Session 01 said parallelism buys throughput, not latency. Real-time systems are the exception — here, guaranteed, predictable latency is the whole point. We now prize determinism over average speed.
Hard vs Soft Real-Time
| Type | A late answer is… | Examples |
|---|---|---|
| Hard real-time | a failure — the deadline is absolute | ABS brakes, airbags, pacemaker, flight control, robot arm |
| Soft real-time | degraded quality, but tolerable | video call, game frame rate, live audio, stock ticker |
Cars (ADAS)
Camera + radar + LiDAR fused in parallel; perception, planning, control run concurrently — each with its own deadline.
Phones (big.LITTLE)
Big power cores + small efficient cores. Asymmetric multiprocessing (Session 05's AMP) — for battery, not just speed.
RTOS scheduling
Priority-based preemptive scheduling. Optimizes worst-case timing, not average. Predictable beats fast.
Recent Trend: Federated Learning
The problem it solves
To train a good ML model — say your phone keyboard's next-word predictor — you need lots of real data. The obvious move: collect everyone's data into one data center and train there.
That's a privacy nightmare. Your messages, a hospital's records, a bank's transactions — nobody wants to ship that to a server, and increasingly the law won't let them (GDPR, India's DPDP Act).
What if we could train a shared model on everyone's data — without the data ever leaving their device?
The core idea: move the model to the data, not the data to the model. Centralize only what the model learned, never the raw data.
One Federated Round
+---------------------------------------------+
| CENTRAL SERVER (model) |
+---------------------------------------------+
| (1) send current model to devices
v v v v
+--------+ +--------+ +--------+ +--------+
|Phone A | |Phone B | |Hospital| |Phone D |
| local | | local | | local | | local | (2) each trains
| data | | data | | data | | data | LOCALLY on its
+--------+ +--------+ +--------+ +--------+ OWN private data
| | | |
+------ (3) send back ONLY the updates -------+
(weight changes / gradients --
never the raw data)
v
+---------------------------------------------+
| SERVER AGGREGATES updates -> new model | (4) Federated
| (Federated Averaging: weighted mean) | Averaging
+---------------------------------------------+
|
+----- repeat for many rounds ---->The raw data never moves. Only anonymous "lessons learned" flow to the center. Devices train in parallel on their own private shards — then the server reduces the updates into one better model.
It's Your Course, Reassembled
| Federated learning step | Course concept it reuses |
|---|---|
| Send model to N devices | Broadcast / scatter (MPI, Session 11) |
| Each device trains locally, in parallel | Data parallelism (S01/S06); embarrassingly parallel (S09) |
| Send back only updates | Communication-minimizing design (Foster's agglomeration, S07) |
| Server averages the updates | Reduction / all-reduce (S10/S11) |
Federated learning is the map → reduce pattern (Session 14) with a privacy twist: map on private data you never see, reduce only the summaries. You already know every piece.
The Honest Catch & Where It's Used
Non-IID data
Every device's data differs. Averaging conflicting updates = heterogeneity / load-imbalance (S09) in disguise.
Stragglers
Phones go offline or are slow. Can't wait for the slowest — the straggler problem (S09). Use whoever reports in time.
Communication cost
Updates over mobile networks are slow. Minimize rounds — the overhead Amdahl (S08) warned eats speedup.
Privacy isn't automatic
Add secure aggregation (server sees only the sum) + differential privacy (add noise) to close the gap.
In the wild: Google Gboard (Android keyboard next-word), healthcare (shared tumor-detection models without sharing patient records), finance (fraud models without exposing transactions).
The Other Frontiers
Edge computing
Do the compute near the data — phone, car, 5G tower — not a distant cloud. Lower latency, better privacy, less bandwidth. Session 14's "move computation to the data," pushed to the edge.
Distributed AI training — three parallelisms at once
Data parallelism
Each GPU gets a different slice of the training data; same model copy.
Model parallelism
Model too big for one GPU — different layers live on different GPUs.
Pipeline parallelism
Factory line: batch 1 at layer 3 while batch 2 is at layer 1.
One arrow of scale: multicore → GPU → clusters → planet-scale. Each step is the same "divide, run at once, combine" at a bigger radius.
The Whole Course, In One Arc
Unit I — Why + Classify + Models
Free lunch over; concurrency vs parallelism; Flynn's SISD/SIMD/MISD/MIMD; shared vs distributed memory; threads & fork-join.
Unit II — Architecture
Interconnects (bus/mesh/torus/hypercube); memory hierarchy, cache coherence (MESI), false sharing; CPUs vs GPUs, SIMT.
Unit III — Design + Measure
Foster's PCAM; Amdahl's Law (the ceiling) & Gustafson; speedup, efficiency; load balancing & work stealing.
Unit IV — Tools
OpenMP (shared, one node) · MPI (distributed, many nodes) · CUDA (the GPU). Labs 1–5.
Unit V — Applications
Scientific computing & parallel databases; big data (MapReduce, Spark); real-time systems & the AI frontier (today).
One Continuous Idea
thread -> core -> GPU -> node -> cluster -> data center -> the planet
| | | | | | |
OpenMP multi- CUDA / MPI MPI / MapReduce / Federated
thread core SIMT ranks clusters Spark learning
"Split the work into independent pieces, run them at once,
keep coordination cheap, and combine the results."
... the SAME idea, at a bigger and bigger scale.The three questions that never change
1. What can run in parallel?
Decomposition — Session 07.
2. What's the ceiling?
Amdahl / the serial fraction — Session 08.
3. Keep coordination cheap?
Communication, load balance, coherence — S04, S05, S09.
The tools change. These three questions never do.
Exam-Prep Pointers
Highest-value topic: Amdahl's Law, numerically. S = 1 / (f + (1−f)/p), ceiling 1/f. Example: f = 0.1, p = 8 → S = 1/(0.1 + 0.9/8) ≈ 4.7×; ceiling = 10×. Do three practice problems.
- Flynn's taxonomy — define all four, one example each.
- Memory & coherence — shared vs distributed; MESI; false sharing.
- Speedup & efficiency — S = T(1)/T(p); E = S/p; where lost efficiency went.
- The three tools & when — OpenMP (one node), MPI (many nodes), CUDA (GPU).
- Foster's PCAM; data vs task decomposition; MapReduce word-count.
- Federated learning — explain in 3–4 sentences.
Exam-answer formula: for any "explain X" — definition → one-line analogy → real example. It earns most of the marks, and it's exactly how we taught every topic.
That's the Course
In Session 01 I told you the free lunch is over — that someone now has to make software fast on purpose. Fifteen sessions later, that someone is you. You now know:
- Why the world went parallel (power wall, more cores).
- How to classify any machine (Flynn, memory models).
- How the hardware works (interconnects, coherence, CPUs vs GPUs).
- How to design & measure (PCAM, Amdahl, speedup, load balancing).
- The three real tools and when to use each (OpenMP, MPI, CUDA).
- Where it all runs (science, data, real-time, the AI frontier).
You now think in parallel.
Go build things that use all the cores. Thank you — it's been a genuine pleasure.