1. Mathematical model of neural networks and history of neural networks.
2. Why deep? Universality theorem for shallow/deep neural networks.
3. Optimization algorithms: back-propagation, block-coordinate descent
algorithms, un-rectifying algorithms, and target propagation.
4. Classification problem: dimension reduction methods, support vector machines
and kernel methods. Un-supervised learning and auto-encoder.
5. Generative Adversarial Network.
6. ResNet, Mobile Net, U-nets, and RNN.
7. Using DNN to solve problems: the inverse problem, the forward inference
problem, and the generalization.