**Department of Systems Engineering and Operations Research**

**George Mason University**

**Fall 2017**

The class will consist of 7-8 lectures given by the instructor on several advanced topics in data analysis. The rest of the semester (another 6-7 lectures) students will present on the topic of their choice.

** For Lecture notes and homework click here**

5/3/2017: First class is on Aug 30 at 4:30pm

Probabilistic models for Machine Learning

Conjugate distributions, exponential family

Model choice

Hierarchical linear and generalize linear models (regression and classification)

Models for missing data (EM-algorithm)

LDA, Normal mixtures, Bayes PCA

Bayes computations (MCMC, Variational Bayes)

Graphical Models

Probabilistic modeling with Stan

Deep learning

Optimization

Architectures (CNN, LSTM, MP, VAE)

Bayesian DL

Generative models (GANs)

Modeling with TensorFlow

Filtering

Kalman Filter

Extended KF and ensemble KF

Particle filter

Modeling with DLM package in R

Probability and Statistics methods for Decision Making

Real-time hypothesis testing

Brownian motion

Bayesian methods for optimal stopping time detection

Bayesian Optimization

Tuning machine learning algorithms

Engineering model calibration

Modeling with spearmint

**Instructor**: Vadim Sokolov

**Office**: Engineering Building, Room 2242

vsokolov(at)gmu.edu

**Tel**: 703-993-4533

Lectures on “Probability in High Dimension” (pdf)

Book on “High-Dimensional Probability” (pdf)

Blog post on SGD (link)

Machine Learning: A Bayesian and Optimization Perspective(safari)

A Bayesian Course with Examples in R and Stan (book page)

Doing Bayesian Data Analysis, Second Edition: A Tutorial with R, JAGS, and Stan (amazon)

Vadim Sokolov: By appointment (at Engineering 2242)

**Location**: Nguyen Engineering Building 1109

**Times**: 4:30-7:10pm on Wednesday

**Grade composition**: No in-class examination. Grade is based entirely on participation in class and homework assignments.

Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks (paper)

Why does Monte Carlo Fail to Work Properly in High-Dimensional Optimization Problems? (paper)

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift (paper)

Auto-Encoding Variational Bayes (paper)

Learning Deep Architectures for AI (monograph)

Representation Learning: A Review and New Perspectives (paper)

Sequence to Sequence Learning with Neural Networks (paper)

Twin Networks: Using the Future as a Regularizer (paper)

Skip RNN (blog and paper)

VAE with a VampPrior (paper)

Bayesian DL (blog)

Generative Adversarial Networks (presentation)

GANs at OpenAI (blog)

Revisiting the Unreasonable Effectiveness of Data (blog)

Learning the Enigma with Recurrent Neural Networks (blog)

Tuning CNN architecture (blog)

Security (blog)

Unsupervised learning (blog)

DL Tuning (blog)

Multivariate Industrial Time Series with Cyber-Attack Simulation: Fault Detection Using an LSTM-based Predictive Data Model (paper)

Signal-based Bayesian Seismic Monitoring (paper)

Introducing practical and robust anomaly detection in a time series (blog)

Anomaly Detection with Twitter in R (blog)

Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems (paper()

Can local particle filters beat the curse of dimensionality? (paper)

The Markov Chain Monte Carlo Revolution (paper)

Graphical Models, Exponential Families, and Variational Inference (monograph)

Variational Inference: A Review for Statisticians (paper)

Neural Block Sampling (paper)

Stochastic Variational Inference (paper)

A stochastic approximation method (paper)

A note on the evidence and Bayesian Occamâ€™s razor (paper)

How to become a Bayesian in eight easy steps: An annotated reading list (paper)

Sparse Bayesian Learning and the Relevance Vector Machine (pdf)

EM Algorithm (blog)

Learning Transferable Architectures for Scalable Image Recognition (paper)

Hyperband (demo)

Calibration at Google (blog)

TF Playground (Google)

Prophet (Facebook)

AnomalyDetection (Twitter)

Bayesian Chronological Modeling Workshop site