DA 218o Probabilistic Machine Learning: Theory and Applications (January 2023)

Course Instructor: : Punit Rathore, RBCCPS

Topics: Refresher on Probability, Probability Distributions, Concepts of prior, likelihood, and posterior distributions, Conjugate priors, Basics of parameter estimation in probabilistic models (Maximum Likelihood, MAP Estimation etc.), Probabilistic models for Regression and Classification, Introduction to latent variable models, Working with Gaussian models for clustering, Bayesian Model Selection, Probabilistic Graphical Models (PGMs), Statistical Inferencing, Approximate Inferencing, Basics of Markov chains and Markov Models (HMM and MRF), Markov Chain Monte Carlo (MCMC) and Sampling algorithms (Gibbs, Accept/Reject, Metropolis Hastings etc.), Bayesian Optimization, Bayesian Neural networks, with brief overview on deep generative models, Variational Inference, Latent Dirichlet Allocation for topic (language) modelling.

Lab Component

There will be a programming workshop every week in which the machine learning model/concept taught in the previous week will be implemented (from scratch as well as from the in-built libraries in python) , tested and evaluated on various types of real-world datasets/ The objective of these workshops will be to get hands-on experience for each ML concepts taught in the class.

Pre-requisits

Mandatory: Basic knowledge in Probability (Mandatory), Proficiency in python Optional: Some basic knowledge in machine learning through coursework or projects

Course Objective

This course will involve mathematics, practical hands-on workshops in python, and programming tasks that require you to exercise critical thinking. It aims to build a strong foundation in the area of probabilistic/Bayesian techniques for machine learning. I hope that you will find this course and programming tasks intellectually stimulating and relevant to your current program and your careers (research/industry).

Learning Outcomes: On successful completion of this course, students should be able to

(i) Gain an understanding of representative selection of ML techniques (not just limited to probabilistic models)

(ii) Describe and use a range of basic and advances probabilistic models for data mining and machine learning tasks

(iii)Code, apply, and solve the real-world problems using python

(iv) Design, implement, evaluate, and interpret probabilistic models

(v) Identify limitation of the statistical and probabilistic methods covers in the course

(vi) Become a discerning ML consumer

Assessment: Homework (20%), Project through Kaggle Competition (25%), Quizzes (in every 3-4 weeks) – 15% (best 3 out of 4), Presentation (journal, review, advanced topics) – 10%, Final exam – 30%

Textbooks / References

  1. Machine learning: A Probabilistic Perspective, Kevin Murphy, MIT Press, 2012.
  2. Probabilistic Graphical Models, Principles and Techniques, Daphne Koller, Cambridge University Press, 2009
  3. Pattern Recognition and Machine Learning, Christopher Bishop, New York, Springer, 2006
  4. Bayesian Reasoning and Machine Learning, David Barber, Cambridge Univ. Press, 2013.