**Department of Systems Engineering and Operations Research**

**George Mason University**

**Fall 2017**

The class will consist of 7-8 lectures given by the instructor on several advanced topics in data analysis. The rest of the semester (another 6-7 lectures) students will present on the topic of their choice.

** For Lecture notes and homework click here**

Bayesian Optimization for ML and Engineering. By Laura Schultz (11/1)

Adversarial Auto-encoders. By Bill Jeffries on 11/8

RNN for EDU: Deep Knowledge Tracing. By Qian Hu (11/8)

RNN/LSTM. By Darron Fuller.

Hierarchical Bayes and STAN. By Tuan Lee (11/15)

Convolutional Networks: Convolution/pooling layers, network design, theory. By Kathleen Perez-Lopez on 11/15. Materials: Brief info, lecture video, slides

DL for Threat Detection. By James Lee. 11/29

VAE: VAE with a VampPrior. By Hossein Fotouhi on 11/29.

Language Modeling: Exploring the Limits of Language Modeling, Efficient Estimation of Word Representations in Vector Space, Distributed Representations of Sentences and Documents. By Anya Mityushina on 11/29

Reinforcement learning. By Jeff Schneider

10/24: No lecture this Wednesday (10/25)

10/24: HW3 Posted

10/03/2017: Instructions on setting up Google Cloud Machines posted here

9/24/2017: HW2 posted

9/24/2017: Solutions for HW1 posted

9/7/2017: HW1 posted

5/3/2017: First class is on Aug 30 at 4:30pm

Probabilistic models for Machine Learning

Conjugate distributions, exponential family

Model choice

Hierarchical linear and generalize linear models (regression and classification)

Models for missing data (EM-algorithm)

LDA, Normal mixtures, Bayes PCA

Bayes computations (MCMC, Variational Bayes)

Graphical Models

Probabilistic modeling with Stan

Deep learning

Optimization

Architectures (CNN, LSTM, MP, VAE)

Bayesian DL

Generative models (GANs)

Modeling with TensorFlow

Filtering

Kalman Filter

Extended KF and ensemble KF

Particle filter

Modeling with DLM package in R

Probability and Statistics methods for Decision Making

Real-time hypothesis testing

Brownian motion

Bayesian methods for optimal stopping time detection

Bayesian Optimization

Tuning machine learning algorithms

Engineering model calibration

Modeling with spearmint

**Instructor**: Vadim Sokolov

**Office**: Engineering Building, Room 2242

vsokolov(at)gmu.edu

**Tel**: 703-993-4533

The Probability and Statistics Cookbook (book page)

Pattern Recognition and Machine Learning (book page)

Lectures on “Probability in High Dimension” (pdf)

Book on “High-Dimensional Probability” (pdf)

Blog post on SGD (link)

Machine Learning: A Bayesian and Optimization Perspective(safari)

A Bayesian Course with Examples in R and Stan (book page)

Doing Bayesian Data Analysis, Second Edition: A Tutorial with R, JAGS, and Stan (amazon)

Vadim Sokolov: By appointment (at Engineering 2242)

**Location**: Nguyen Engineering Building 1109

**Times**: 4:30-7:10pm on Wednesday

**Grade composition**: No in-class examination. Grade is based entirely on participation in class and homework assignments.

Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks (paper)

Why does Monte Carlo Fail to Work Properly in High-Dimensional Optimization Problems? (paper)

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift (paper)

Auto-Encoding Variational Bayes (paper)

Learning Deep Architectures for AI (monograph)

Representation Learning: A Review and New Perspectives (paper)

Sequence to Sequence Learning with Neural Networks (paper)

Twin Networks: Using the Future as a Regularizer (paper)

Skip RNN (blog and paper)

VAE with a VampPrior (paper)

Bayesian DL (blog)

Generative Adversarial Networks (presentation)

GANs at OpenAI (blog)

Revisiting the Unreasonable Effectiveness of Data (blog)

Learning the Enigma with Recurrent Neural Networks (blog)

Tuning CNN architecture (blog)

Security (blog)

Unsupervised learning (blog)

DL Tuning (blog)

Cybersecurity (paper collection)

LSTM blog

Deep Energy (blog)

DL Summer school 2015 (videos)

DL Representations (blog)

PyData 2017 (videos)

Multivariate Industrial Time Series with Cyber-Attack Simulation: Fault Detection Using an LSTM-based Predictive Data Model (paper)

Signal-based Bayesian Seismic Monitoring (paper)

Introducing practical and robust anomaly detection in a time series (blog)

Anomaly Detection with Twitter in R (blog)

Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems (paper()

Can local particle filters beat the curse of dimensionality? (paper)

The Markov Chain Monte Carlo Revolution (paper)

Graphical Models, Exponential Families, and Variational Inference (monograph)

Variational Inference: A Review for Statisticians (paper)

Neural Block Sampling (paper)

Stochastic Variational Inference (paper)

A stochastic approximation method (paper)

A note on the evidence and Bayesian Occamâ€™s razor (paper)

How to become a Bayesian in eight easy steps: An annotated reading list (paper)

Sparse Bayesian Learning and the Relevance Vector Machine (pdf)

EM Algorithm (blog)

Learning Transferable Architectures for Scalable Image Recognition (paper)

Hyperband (demo)

Calibration at Google (blog)

TF Playground (Google)

SnakeViz (python profiler)

Pytorch resources (a curated list of tutorials, papers, projects)

Prophet (Facebook)

AnomalyDetection (Twitter)

Bayesian Chronological Modeling Workshop site