CS 732: Advance Machine learning
Spring 2016

Instructor: Usman Roshan
Office: GITC 3802
Ph: 973-596-2872
Office hours: Tue and Thu 2:30 to 5
Email: usman@cs.njit.edu

TA: Ling Zhong
Office hours: By appointment
Email: lz25@njit.edu

Textbook: Not required
Grading: 20% programming assignments, 10% paper presentation, 70% project
Course Overview: This course will cover advance topics in machine learning. We will begin with the CUDA and OpenCL languages for parallel programming on Graphics Processing Units (GPUs) and OpenMP for multi-core programming. While machine learning and parallel programming are separate topics we include GPU and multi-core computing in this course because of their applications in machine learning on large datasets. We will then discuss recent papers in machine learning throughout the semester such as deep learning, representation learning, optimization algorithms, algorithms for big datasets, and advance Bayesian methods. Students will present recent papers and undertake a machine learning project. The project may be theoretical or experimental in nature. For example students may take on a challenge dataset from Kaggle or study theoretical error bounds of a contemporary classifier.

Course plan:

Topic	Date	Notes
Introduction to GPU computing	01/25/2016	Introduction Basic Unix command sheet Instructions for AFS login
CUDA exercise	02/01/2016	Simulated GWAS Class labels for above data Chi-square 2-df test in parallel on a GPU Assignment 1 Chi8 (Makefile and sample script file include)
OpenCL	02/08/2016	Papers Projects CUDA to OpenCL slides libOpenCL.so (NVIDIA library file for OpenCL code) Chi2 opencl implementation OpenCL files Assignment 2
Papers		Deep learning Learning Feature Representations with K-means Analysis of single-layer networks in unsupervised feature learning Building high-level features using large-scale unsupervised learning Mapreduce Map-Reduce for Machine Learning on Multicore An Iterative MapReduce Approach to Frequent Subgraph Mining in Biological Datasets GPU machine learning Large-scale parallelized sparse principal component analysis Parallel GPU Implementation of Iterative PCA Algorithms Singular value decomposition on GPU using CUDA Fast support vector machine training and classification on graphics processors GPU library for deep learning Comparison of classifiers Do we Need Hundreds of Classifiers to Solve Real World Classification Problems? An Empirical Comparison of Supervised Learning Algorithms Optimization Algorithms for Direct 01 Loss Optimization in Binary Classification Optimizing 0/1 Loss for Perceptrons by Random Coordinate Descent
Projects:		Haifa: Deep learning for sequence alignment regression Ajit: Deep learning vs. another recent classifier Yuan: DNN-HMM based environmental acoustic classifier Min Zhang: Unsupervised neural spike sorting Yang: Neural networks with unreliable feature inputs Mingda: Decision tree in C Anil and Hammad: Predicting vulnerable bugs in programs Anisha and Yogesh: Machine learning based robot navigation Kuang: Machine learning for finance Xiaoyu: Using social media data to understand and assess disasters Jinhe: Conditional random fields Zhangyi Jin: Big data Machine Learning implementation with Spark Angie: Kaggle Yelp Photo Classification Problem Clarissa: CIFAR-10 image classification Haoyue Liu and Keyuan Wu: Deep learning on CIFAR-10 image classification Haodi Jiang: max-min hill-climbing Bayesian network structure learning algorithm Xinge: Gradient boosting
Presentations	02/15/2016	Zahra: Analysis of single-layer networks in unsupervised feature learning Ajit: Building high-level features using large-scale unsupervised learning Kuang: Combining multiple feature selection methods for stock prediction
Presentations	02/22/2016	Mingda: Map-Reduce for Machine Learning on Multicore Min Zhang: (1) Decoding Neural Representational Spaces Using Multivariate Pattern Analysis (2) Method for Unsupervised Classification of Multiunit Neural Signal Recording Under Low Signal-to-Noise Ratio Haifa: SVM sequence alignment
Presentations	02/29/2016	Yogesh: (1) Deep Learning for Multivariate Financial Time Series (2) Support Vector Regression Based Indoor Location in IEEE 802.11 Environments Hammad: Machine Learning Methods for Computer Security Computer security notes by Hammad Yuan: Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups
Presentations	03/07/2016	Xiaoyu: k-means++: The Advantages of Careful Seeding Jinhe: An Empirical Comparison of Supervised Learning Algorithms Roshan: Do we Need Hundreds of Classifiers to Solve Real World Classification Problems
Presentations	03/21/2016	Yang Liu: RANN: A Reliability-Aware Neural Network for Mining Unreliable Multidimensional Data Anisha: Random Forest Based Coarse Locating and KPCA Feature Extraction for Indoor Positioning System Roshan: (1) Optimization I (2) Optimization II (3) Optimizing 0/1 Loss for Perceptrons by Random Coordinate Descent (4) FACM talk on 0/1 loss optimization
Presentations	03/28/2016	Xinge: Algorithms for Direct 01 Loss Optimization in Binary Classification Haoyue Liu and Keyuan Wu: (1) Representation Learning: A Review and New Perspectives (2) Multimodal Learning with Deep Boltzmann Machines
Presentations	04/04/2016	Haodi: (1) Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning (2) The max-min hill-climbing Bayesian network structure learning algorithm Clarissa: Convolutional Deep Belief Networks on CIFAR-10 Image classification benchmark results Angie: Efficient Multi-label Classification with Many Labels
Presentations	04/11/2016	Zhangyi: (1) Mastering the game of Go with deep neural networks and tree search (2) Human-level control through deep reinforcement learning Anil: (1) Mining Bug Databases for Unidentified Software Vulnerabilities (2) Vulnerability Extrapolation: Assisted Discovery of Vulnerabilities using Machine Learning Weiqiang Tong: Optimizing 0/1 Loss for Perceptrons by Random Coordinate Descent
Presentations	04/18/2016	Usman Roshan: Predicting Disease Risk Using Bootstrap Ranking and Classification Algorithms Project presentations Haifa Min Zhang Angie Kuang
Student projects	04/25/2016	Project presentations Yang Yuan Xinge Anil and Hammad Ajit Xiaoyu: Analyzing the Evolution of Rare Events via Social Media Data and k-means Clustering Algorithm
Student projects	05/11/2016	Meeting place and time: CKB 315 at 11:30 Project presentations Clarissa Haoyue Liu and Keyuan Wu Yang Additional papers Dropout: A Simple Way to Prevent Neural Networks from Overfitting Subsemble: an ensemble method for combining subset-specific algorithm fits ImageNet Classification with Deep Convolutional Neural Networks