Department of Systems Engineering and Operations Research
George Mason University
Spring 2021
Videos of Lectures
Course Material
Course Number: OR 750
Location: Zoom Thu 4:30-7:10pm
Instructor: Vadim Sokolov (vsokolov(at)gmu.edu)
Office hours: by Appointment
Prerequisites: Undergraduate Calculus, probability theory, statistics, computer programming skills (ideally R).
HW Logistics: You will submit your HW and projects to BlackBoard
This is a graduate course on Bayesain learning. Although basics will be revisited, the pace will be swift so we can get to advanced topics as quickly as possible. This course details classical Bayesain techniques as well as modern approaches from both statistics and machine learning. We will consider some canonical examples of Bayesain analysis but will concentrate on modern Bayesain techniqes, computation and implementation, as well as modern applications. The course material will emphesize deriving and implementing methods over proving theoretical results.
During weeks 11-15, this class will be run in a seminar mode. A student or the instructor will lead the discussion.
Overview: Conditional Probability, Bayes Rule, Bayesain inference, utility theory, distributions and tranformations
Hierarchical models: Bayesian regression, shrinkage (lasso, horseshoe)
Gaussian Process with applications in Bayesian Optimization
Tuning machine learning algorithms
Engineering model calibration
Algorithms
Markov Chain Monte Carlo (MCMC)
Expectation Maximization (EM)
Variational Bayes
Markov and Hidden Markov Models
Kalman Filter
Particle filter
Dynamic Linear Model
Structural Time Series Models
Rubric: 30% HW, 10% Discussion, 60 % Final Project. No in-class examination. Cutoffs: A: 90, B: 80, C: 70, F: < 70
Pattern Recognition and Machine Learning by Bishop (book page)
Machine Learning: a Probabilistic Perspective by Murphy (book page)
An Introduction to Bayesian Thinking by Clyde et al. (book)
Introduction to Statistical Thought by Lavine (book)
The Probability and Statistics Cookbook by Vallentin (book page)
Exchange Paradox by Christensen and Utts (pdf)
Inference for Nonconjugate Bayesian Models Using the Gibbs Sampler by Carlin and Polson (pdf)
Data Augmentation for Support Vector Machines by Polson and Scott (pdf)
The horseshoe estimator for sparse signals by Carvalho, Polson, Scott (pdf)
Deep learning: A Bayesian perspective (pdf)
Bayesian regularization: From Tikhonov to horseshoe (pdf)
Spatial Interaction and the Statistical Analysis of Lattice Systems by Besag (jstor)
Particle learning and smoothing by Carlos M Carvalho, Michael S Johannes, Hedibert F Lopes, Nicholas G Polson (pdf)
Bayesian model assessment in factor analysis by Lopes and West (pdf)
Tracking epidemics with Google flu trends data and a state-space SEIR model by Dukic, Lopes and Polson (pdf)
A statistical paradox by Lindley (pdf)
The philosophy of statistics by Lindley (pdf)
The Relevance Vector Machine by Tipping (pdf)
BART: Bayesian additive regression trees by Hugh A. Chipman, Edward I. George, Robert E. McCulloch (arxiv)
Bayesian methods for hidden Markov models: Recursive computing in the 21st century by Scott (pdf)
A modern Bayesian look at the multi‐armed bandit by Scott (pdf)
Bayesian analysis of computer code outputs: A tutorial by O'Hagan (paper)
Bayesian analysis of stochastic volatility models by Jacquier, Polson and Rossi (paper)
Ockham's razor and Bayesian analysis by Jefferys and Berger (pdf)
Bayesian Learning for Neural Networks by Neal (pdf)
MCMC Using Hamiltonian Dynamics by Neal (pdf)
Slice sampling be Neal (pdf)
Bayesian interpolation by MacKay (pdf)
Bayesian online changepoint detectio by Adams and MacKay (https://arxiv.org/pdf/0710.3742)
The No-U-Turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. by Matthew and Gelman (pdf)
To promote a stronger sense of mutual responsibility, respect, trust, and
fairness among all members of the George Mason University community and
with the desire for greater academic and personal achievement, we, the
student members of the university community, have set forth this honor code:
Student members of the George Mason University community pledge not to
cheat, plagiarize, steal, or lie in matters related to academic work.
Students are responsible for their own work, and students and faculty must take
on the responsibility of dealing with violations. The tenet must be a foundation
of our university culture.
All work performed in this course will be subject to Mason's Honor Code.
Students are expected to do their own work in the course. For the group project,
students are expected to collaborate with their assigned group members.
In papers and project reports, students are expected to write in their own words,
The university is committed to providing equal access to employment and
educational opportunities for people with disabilities.
Mason recognizes that individuals with disabilities may need reasonable
accommodations to have equally effective opportunities to participate in or
benefit from the university educational programs, services, and activities, and
have equal employment opportunities. The university will adhere to all
applicable federal and state laws, regulations, and guidelines with respect to
providing reasonable accommodations as necessary to afford equal
employment opportunity and equal access to programs for qualified people
with disabilities.
Applicants for admission and students requesting reasonable accommodations
for a disability should call the Office of Disability Services at 703-993-2474.
Employees and applicants for employment should call the Office of Equity and
Diversity Services at 703-993-8730. Questions regarding reasonable
accommodations and discrimination on the basis of disability should be
directed to the Americans with Disabilities Act (ADA) coordinator in the Office
of Equity and Diversity Services.