CS 675: Introduction to Machine learning
Fall 2015

Instructor: Usman Roshan
Office: GITC 3802
Ph: 973-596-2872
Office hours:W 2:30 - 5:30
Email: usman@cs.njit.edu

TA: Mohammadreza Esfandiari
Office: GITC 3800
Office hours:
Email: me76@cs.njit.edu

Textbook: Introduction to Machine Learning by Ethem Alpaydin
Grading: 25% mid-term, 30% final exam, 15% course project, 30% programming assignments
Course Overview: This course is a hands-on introduction to machine learning and contains both theory and application. We will cover classification and regression algorithms in supervised learning such as naive Bayes, nearest neighbor, decision trees, random forests, hidden Markov models, linear regression, logistic regression, and support vector machines. We will also cover dimensionality reduction, unsupervised learning (clustering), feature selection, and kernel methods. We will apply algorithms to solve problems on real data such as digit recognition, text document classification, and prediction of cancer and molecular activity.

Course plan:

Topic	Date	Notes
Introduction, Bayesian learning, and Perl/Python	09/02/2015	Introduction Linear algebra and probability background Bayesian learning Basic Unix command sheet Instructions for AFS login All of chapter 1, 2.1, 2.4, 2.5, 2.6, 2.7
Bayesian learning	09/04/2015	Textbook reading: 4.1 to 4.5, 5.1, 5.2, 5.4, 5.5
Python	09/09/2015	Perl Python Perl example 1 Perl example 2 Perl example 3 Python example 1 Python example 2 Python example 3
Bayesian learning and Python	09/11/2015	Nearest mean algorithm Naive Bayes algorithm Assignment 1 Assignment 1 submission
Nearest means and naive-bayes	09/16/2015	Datasets
Kernel nearest means	09/18/2015	Balanced error Balanced error in Perl Kernels Kernel nearest means Textbook reading: 13.5, 13.6, 13.7
Linear separators	09/23/2015	Mean balanced cross-validation error on real data Hyperplanes as classifiers Assignment 2 Assignment 2: Implement perceptron algorithm Textbook reading: 10.2, 10.3, 10.6, 11.2, 11.3, 11.5, 11.7
Linear separators	09/25/2015	Hardness of separating hyperplanes Learning Linear and Kernel Predictors with the 01 Loss Function
Support vector machines	09/30/2015	Textbook reading: 13.1 to 13.3 Efficiency of coordinate descent methods on huge-scale optimization problems
Perceptron in Python	10/02/2015
Support vector machines and kernels	10/07/2015	Kernels Multiple kernel learning by Lanckriet et. al. Multiple kernel learning by Gonen and Alpaydin Script to compute average test error
Logistic regression and regularized risk minimization	10/09/2015	Logistic regression Assignment 3: Implement logistic discrimination algorithm Regularized risk minimization Solver for regularized risk minimization Textbook reading: 10.7
Cross-validation and exam review	10/14/2015	Cross validation svm_learn svm_classify run_svm_light.pl linear-bmrm-train linear-bmrm-predict bmrm.pl BMRM training config file BMRM test config file
Mid-term exam	10/16/2015
Mid-term exam solution	10/21/2015
Feature selection	10/23/2015	Feature selection NIPS 2003 feature selection contest Contest website Challenge results Challenge results II A comparison of univariate and multivariate gene selection techniques for classification of cancer datasets Feature selection with SVMs and F-score Ranking genomic causal variants with chi-square and SVM
Dimensionality reduction	10/28/2015	Dimensionality reduction Textbook reading: Chapter 6 sections 6.1, 6.3, and 6.6
Dimensionality reduction	10/30/2015	Dimensionality reduction II Maximum margin criterion Laplacian linear discriminant analysis
Dimensionality reduction	10/30/2015	Dimensionality reduction III
Decision trees, bagging, and boosting	11/04/2015	Decision trees, bagging, and boosting Univariate vs. multivariate trees Survey of decision trees Boosted trees: Slides by Tianqi Chen Textbook reading: Chapters 9 and 17 sections 9.2, 17.4, 17.6, 17.7
Unsupervised learning - clustering	11/06/2015	Clustering Tutorial on spectral clustering K-means via PCA Textbook reading: Chapter 7 sections 7.1, 7.3, 7.7, and 7.8
Clustering	11/11/2015	Assignment 4: Implement k-means clustering in Python Course project Training dataset Training labels Test dataset
Clustering	11/13/2015
Regression	11/18/2015	Regression Textbook reading: Chapter 4 section 4.6, Chapter 10 section 10.8, Chapter 13 section 13.10
Hidden Markov models	11/20/2015	Hidden Markov models Textbook reading: Chapter 15 (all of it)
Feature learning	11/25/2015	Bonus assignment: Implement a decision tree in Python Learning Feature Representations with K-means Analysis of single-layer networks in unsupervised feature learning Feature learning with k-means
Project submission, comparison of classifiers	12/02/2015	Comparison of classifiers Do we Need Hundreds of Classifiers to Solve Real World Classification Problems? An Empirical Comparison of Supervised Learning Algorithms Statistical Comparisons of Classifiers over Multiple Data Sets
Big data	12/04/2015	Big data Mini-batch k-means Stochastic gradient descent Mapreduce for machine learning on multi-core
Review for final, announcement of project winner	12/09/2015