OR 610. Deep Learning

Department of Systems Engineering and Operations Research
George Mason University
Fall 2020


Course Material

Piazza

Instructor: Vadim Sokolov (vsokolov(at)gmu.edu)
TA: Ali Seyedmazloom (aseyedma (at) masonlive.gmu.edu)
Office hours: By appointment

Datacamp

If you are rusty on Python, I suggest you refresh your skills using Datacamp. Datacamp gave students in this class a free access to all of the courses. If you follow the link above you can get your free access using masonlive email. I also listed some of the Python courses I suggest there.

Schedule (Homework are due on Mondays at noon)

Week 1 (Aug 24): Introduction

Week 2 (Aug 31): Gradient Descent

Week 3 (Sep 7): Neural Networks

Week 4 (Sep 14): PyTorch

Week 5 (Sep 21): Optimization and regularization

Week 6, 7 (Sep 28, Oct 5): CNN

Week 8,9 (Oct 12, Oct 19): Recurrent Nets

Week 10,11, 12 (Oct 26, Nov 2, Nov 9): Bayesian Neural Networks (Hinton Notes)

Week 13, 14 (Nov 16, Nov 30): Deep Reinforcement Learning

Week 15 (Dec 7): Final Projects

Data analysis projects

You will work in a team of up to 3 people on a Kaggle-like project and will apply deep learning to solve a prediction or data generation problem. By week 8 of the class you should have a team formed and data set + analysis problem identified. You need to submit a 0.5-1 page description of the data and problem you are trying to solve for my feedback and approval. Proposal has to have names and emails of the team members. Description of data set, problem to be solved and proposed architectures. You will post results of your analysis on the class blog post. The final project will be graded on presentation, writing and analysis. All of the posts based on the presentations are posted here vsokolov.org/or610fall2020

Group Work

Both projects and homework can be done in a groups of size of up to 3 people. You can change groups in between. If you do a homework in a gorup, it means that all of the members of the group do it individually and can consult with each other. You can also do 1 submission per group if you prefer. You can use “group” section of the piazza page to find teammates if you need any. If you need help finding a group, please email me.

Grading

Each hw is 10 points, project is 30.

This is a graduate level course focused on developing deep learning predictive models. We will learn both practical and theoretical aspects of deep learning. We will consider applications in engineering, finance and artificial intelligence. It is targeted towards the students who have completed an introductory courses in statistics and optimization. We will make extensive use of computational tools, such as the Python language, both for illustration in class and in homework problems. The class will consist of 9 lectures given by the instructor on several advanced topics in deep learning. At another 5 lectures students will present on a given topic.

Books

Per Topic Resources

Architectures

Optimization

Theory

Reinforcement Learning

Model-Ensemble Trust-Region Policy Optimization Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models

Section 5: Bayesian DL

Practical Tricks

Other Resources

Additional Reading List

Blogs

Videos

Other courses with good web presence

Tools

Misc Links

Previous instances