Course catalog description: Parallel and distributed architectures, fundamentals of parallel/distributed data structures, algorithms, programming paradigms, introduction to parallel/distributed application development using current technologies.
Credits and contact hours: 3 credits; 1 hour and 20-minute session twice a week, every week
Pre-Requisite courses: 14:332:331, 14:332:351
Co-Requisite courses: None
Topics Covered:
- Introduction to Parallel and Distributed Programming (definitions, taxonomies, trends)
- Parallel Computing Architectures, Paradigms, Issues, & Technologies (architectures, topologies, organizations)
- Parallel Programming (performance, programming paradigms, applications)
- Parallel Programming Using Shared Memory I (basics of shared memory programming, memory coherence, race conditions and deadlock detection, synchronization)
- Parallel Programming Using Shared Memory II (multithreaded programming, OpenMP, pthreads, Java threads)
- Parallel Programming using Message Passing - I (basics of message passing techniques, synchronous/asynchronous messaging, partitioning and load-balancing)
- Parallel Programming using Message Passing - II (MPI)
- Parallel Programming – Advanced Topics (accelerators, CUDA, OpenCL, PGAS)
- Introduction to Distributed Programming (architectures, programming models)
- Distributed Programming Issues/Algorithms (fundamental issues and concepts - synchronization, mutual exclusion, termination detection, clocks, event ordering, locking)
- Distributed Computing Tools & Technologies I (CORBA, JavaRMI)
- Distributed Computing Tools & Technologies II (Web Services, shared spaces)
- Distributed Computing Tools & Technologies III (Map-Reduce, Hadoop)
- Parallel and Distributed Computing – Trends and Visions (Cloud and Grid Computing, P2P Computing, Autonomic Computing)
Textbook: Peter Pacheco, An Introduction to Parallel Programming, Morgan Kaufmann.
Other supplemental material:
- Hariri and Parashar (Ed.), Tools and Environments for Parallel & Distributed Computing, John Wiley.
- David Kirk, Wen-Mei W. Hwu, Wen-mei Hwu, Programming massively parallel processors: a hands-on approach, Morgan Kaufmann.
- Kay Hwang, Jack Dongarra and Geoffrey C. Fox (Ed.), Distributed and Cloud Computing, Morgan Kaufmann.