Machine Learning
Instructor
Dr. Janibul Bashir
Course Overview
In this course, our aim is
(a) to familiarize with/develop the understanding of fundamental concepts of Machine Learning (ML)
(b) To develop the understanding of working of a variety of ML algorithms (both supervised as well as unsupervised)
(c) To learn to apply ML algorithms to real world data/problems
(d) To update with some of the latest advances in the field
Course Code
ITT450 & PHD 14
Lectures
Mon, Thur (L1)
Reference Material for the course:
B1- Probabilistic Machine Learning: An Introduction. Kevin Murphy.
B2- Pattern Recognition and Machine Learning. Christopher Bishop. First Edition, Springer, 2006.
B3- Pattern Classification. Richard Duda, Peter Hart and David Stock. Second Edition, Wiley-Interscience, 2000.
B4- Machine Learning. Tom Mitchell. First Edition, McGraw-Hill, 1997.
B5 - Hands-On Machine Learning with Scikit-Learn and TensorFlow by Aurélien Géron
B6 - Notes from CS229
Video Lectures: 1. Machine Learning using Python by Dr. Janibul Bashirs [Videos]
2. Stanford CS229: Machine Learning by Prof. Andrew NG [Videos]
Announcements:
Classes will be held in Lecture Hall 1 from 02:00 - 04:00 PM
Python will be the default language that will be used to solve the assignments.
You are free to discuss the assignment problems with other students in the class. But all your code should be produced independently without looking at/referring to anyone else's code.
Honor Code: Any cases of copying will be awarded a zero on the assignment and a penalty of -10. More severe penalties may follow.
Course Content
Introduction to Machine Learning, Regression
(06-03-2023) -- [Lecture 1] -- Introduction to Machine learning - [Video Lecture] [B5 Chapter 1]
(07-03-2023) -- [Lecture 2] -- Supervised Learning, Hypothesis Functions, Loss Functions (Zero-One loss and Mean Square Loss) -- [Video Lecture] [B6 Part 1]
(09-03-2023) -- [Lecture 3] -- Linear Regression, Hypothesis Function, Matrix Format Dataset -- [Video Lecture] [B6 Chapter 1][B5 Chapter 4]
(09-03-2023) -- [Lecture 4] -- Gradient Descent -- [Video Lecture] [B6 Chapter 1.1][B5 Chapter 4]
(13-03-2023) -- [Lecture 5] -- Batch GD, Stochastic GD, Mini-Batch GD -- [Video Lecture] [B6 Chapter 1.1 and 1.4][B5 Chapter 4]
(16-03-2023) -- [Lecture 6] -- Normal Equations, Non-linear Hypothesis, Polynomial Regression -- [Video Lecture] [B6 Chapter 1.2][B5 Chapter 4]
(16-03-2023) -- [Lecture 7] -- Generalization, Underfitting, Overfitting, Bias-Variance Tradeoff, Learning Curves -- [Video Lecture] [B6 Chapter 8][B5 Chapter 4]
(17-03-2023) -- [Lecture 8] -- Regularization, Ridge and Lasso, Early stopping, Cross-validation -- [Video Lecture] [B6 Chapter 9][B5 Chapter 4]
Probabilistic Modelling, Classification
(20-03-2023) -- [Lecture 9] -- Introduction to probability and its relation to Machine Learning, PDF, PMF, CDF - [B1 Section 2.1]
(20-03-2023) -- [Lecture 10] -- Random variables, Bayesian Inference and Bayes Theorem - [B1 Section 2.2 - 2.3]
(23-03-2023) -- [Lecture 11] -- Bernoulli Distribution and Binary Logistic Regression using Bernoulli Distribution - [B1 Section 2.4]
(27-03-2023) -- [Lecture 12] -- Categorical Distribution and Multiclass logistic regression using categorical distribution - [B1 Section 2.5]
(27-03-2023) -- [Lecture 13] -- Probabilistic Modelling: Kullback-Leibler Divergence, Maximum likelihood estimation for parameter optimisation - [B1 Section 4.1, 4.2]
(03-04-2023) -- [Lecture 14] -- MLE for Regression and Classification Problems - [B1 Section 4.2]
Data Processing and Discriminant Algorithms
(06-04-2023) -- [Lecture 15] -- Multivariate Data Analysis: Variance, Covariance, and Covariance Matrix - [B1 Section 3.1]
(06-04-2023) -- [Lecture 16] -- Significance of Eigen vectors and Eigen Values of a Covariance Matrix, Contour plots
(10-04-2023) -- [Lecture 17] --Multivariate Models, Multivariate Normal Distribution - [B1 Section 3.2]
(10-04-2023) -- [Lecture 18] -- Gaussian Discriminant Ananlysis [Tutorial With Code] [Master Notes for CS229 Section 4.1] [B1 Section 9.1, 9.2]
Midterm Exam [14-04-2023 to 21-04-2023]
(24-04-2023) -- [Lecture 19] -- Naive Bayes - [Tutorial With Code] [Master Notes for CS229 Section 4.2]
(24-04-2023) -- [Lecture 20] -- Text Classification using Naive Bayes, Laplace Smoothing [Master Notes for CS229 Section 4.2]
Non-Parametric Models
(28-04-2023) -- [Lecture 21] -- Non Parametric Models, K-Nearest Neighbour, Mahalabonis Distance [B1 Section 16.1]
(28-04-2023) -- [Lecture 22] -- Kernels and Kernel Trick [Tutorial] [Python Example][Master Notes for CS229 Section 5.1 - 5.4]
H/W: Support Vector Machines [B1 Section 17.3]
Deep Learning - Neural Networks
(01-05-2023) -- [Lecture 23] Derivation of Feature maps using function composition - [B1 Section 13.1]
(01-05-2023) -- [Lecture 24] Introduction to Neural Networks - [Tutorial][Master Notes for CS229 Section 7.1, 7.2] [B1 Section 13.1]
(04-05-2023) -- [Lecture 25] Backpropagation [Tutorial for creating Neural Networks][Master Notes for CS229 Section 7.3] [B1 Section 13.3]
(04-05-2023) -- [Lecture 26] Training Issues in NN, Regularization in Neural Networks - [B1 Section 13.4 and 13.5]
UnSupervised Learning, Reinforcement Learning, Decision Trees and Ensembling Techniques
(11-05-2023) -- [Lecture 27] K-Means Clustering - [Master Notes for CS229 Section 10] [B1 Section 21.1, 21.3]
(11-05-2023) -- [Lecture 28] Anomaly Detection
(15-05-2023) -- [Lecture 29] Principle Component Analysis and its applications
(18-05-2023) -- [Lecture 30] Recommender Systems, Collaborative filtering
(22-05-2023) -- [Lecture 31] Introduction to Reinforcement Learning
(25-05-2023) -- [Lecture 32] Value Function, Bellman Equation, Policy
(30-05-2023) -- [Lecture 33] Value iteration, Policy Iteration
(01-06-2023) -- [Lecture 34] Decision Trees and their Regularization
(01-06-2023) -- [Lecture 35] Ensembling Techniques - Bagging and Boosting (Adaboost, Gradient Boosting)
Assignments
Homework 1 [Due Date: 10-04-2023]: Implementation of Linear Regression and Polynomial Regression on a housing dataset. All the possible graphs are to be plotted for data visualization on that particular dataset.
Do the regularization of the model and plot the learning curves as well.
Homework 2 [Due Date: 10-04-2023]: Implementation of Binary Logistic Regression and Multiclass Logistic Regression on an IRIS dataset and MNIST Dataset.
Implement Binary logistic regression for a Cat Vs Non-Cat Dataset (available on internet).
Plot all the necessary graphs.
Implement the cross validation.
Confusion Matrix
Homework 3 [Due Date: 20-05-2023]:
Part 1) Build a cat vs non-cat model both using logistic regression and dense neural network and compare their accuracies.
Part 2) Take any dataset of your choice from the internet and optimize the parameters of the neural network for that particular dataset including the architecture of the neural network, activation function to be used, learning rate, and resularization (early stopping and drop out).