課程資訊
課程名稱
 分散式機器學習系統
Distributed Machine-Learning System 
開課學期
 110-2 
授課對象
 電機資訊學院  資訊工程學研究所  
授課教師
 周承復 
課號
 CSIE5319 
課程識別碼
 922 U4430 
班次
  
學分
 3.0 
全/半年
 半年 
必/選修
 選修 
上課時間
 星期一7,8,9(14:20~17:20) 
上課地點
 資107 
備註
 總人數上限：30人 
  
課程簡介影片
 
核心能力關聯
 核心能力與課程規劃關聯圖
課程大綱
為確保您我的權利,請尊重智慧財產權及不得非法影印
課程概述

Distributed Machine-Learning System is an introduction to these system-focused aspects of machine learning, covering guiding principles and commonly used techniques for scaling up to large data sets. That is, we will cover the techniques that lie between a standard machine learning course and an efficient systems implementation. Topics will include stochastic gradient descent, acceleration, variance reduction, methods for choosing metaparameters, parallelization within a chip and across a cluster, popular ML frameworks, and innovations in hardware architectures. 

課程目標
 We look at the performance as well as design issues of large-scale machine learning application that is deployed in practice. After taking this course, students, who basic models and the basic algorithms, are able to modify those models (or systems) in a bunch of different ways such that the systems could run faster and more efficiently. That is, these modifications are really important—they often are what make the system tractable to run on the data it needs to process. 
課程要求
 待補 
預期每週課後學習時數
  
Office Hours
  
指定閱讀
  
參考書目
 待補 
評量方式
(僅供參考)
    
課程進度
週次
日期
單元主題
第1週
 9/16  Introduction 
第2週
 9/23  Overview 
第3週
 9/30  Techniques for SGD 
第4週
 10/07  Techniques for SGD 
第5週
 10/14  Techniques for SGD 
第6週
 10/21  Variance Reduction in SGD 
第7週
 10/28  Variance Reduction in SGD 
第8週
 11/04  Hyperparameter optimization 
第9週
 11/11  Midterm 
第12週
 12/02  Paper Presentation
1. Federated Learning via Over-the-Air Computation
2. A Survey on Transfer Learning
3. Noise2Noise: Learning Image Restoration without Clean Data
4. Practical Bayesian Optimization of Machine Learning Algorithms 
第13週
 12/09  Paper Presentation
1. LAMBDANETWORKS: MODELING LONG-RANGE INTERACTIONS WITHOUT ATTENTION
2. Dueling Network Architectures for Deep Reinforcement Learning
3. CONTINUOUS CONTROL WITH DEEP REINFORCEMENT LEARNING
4. Multi-Layered Gradient Boosting Decision Trees 
第14週
 12/16  Paper Presentation
1. Human-level control through deep reinforcement learning
2. DARTS: DIFFERENTIABLE ARCHITECTURE SEARCH
3. ADAM: A METHOD FOR STOCHASTIC OPTIMIZATION
4. Who did They Respond to? Conversation Structure Modeling using Masked Hierarchical Transformer (AAAI 2020) 
第15週
 12/23  Paper Presentation
1.DEEP COMPRESSION: COMPRESSING DEEP NEURAL NETWORKS WITH PRUNING, TRAINED QUANTIZATION AND HUFFMAN CODING
2. Adversarial Discriminative Domain Adaptation
3. Applications of Deep Learning and Reinforcement Learning to Biological Data
4. A Simple Framework for Contrastive Learning of Visual Representations
5. 
第16週
 12/30  Paper Presentation
1. Meta-Learning in Neural Networks: A Survey
2. Neural Architecture Search with Reinforcement Learning
3. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks 
第17週
 1/06  Paper Presentation
1. A Distributed Multi-Sensor Machine Learning Approach to Earthquake Early Warning
2. Domain-Adversarial Training of Neural Networks
3. A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients 
第18週
 2021/01/13  Final Project Presentation