This course is about programming multiple processing units for performance. We will discuss two kinds of parallel architectures -- multicore CPU and multi-CPU clusters. For multicore CPU we will discuss Pthread, Open Multi-Processing (OpenMP), and OpenCL. For multi-CPU clusters we will discuss Google's MapReduce. We will discuss the architecture, programming model, programming techniques, and application characteristic for these different architectures. Extensive in-class lab sessions will be conducted.