Distributed Machine-Learning System is an introduction to these system-focused aspects of machine learning, covering guiding principles and commonly used techniques for scaling up to large data sets. That is, we will cover the techniques that lie between a standard machine learning course and an efficient systems implementation. Topics will include stochastic gradient descent, acceleration, variance reduction, methods for choosing metaparameters, parallelization within a chip and across a cluster, popular ML frameworks, and innovations in hardware architectures.