CS 698: Current topics in data science
Spring 2017

Instructor: Usman Roshan
Office: GITC 3802
Ph: 973-596-2872
Office hours: Wed 2 to 5
Email: usman@njit.edu

Textbook: Not required
Grading: 20% programming assignments, 30% paper presentation, 50% project
Course Overview: This course will cover current topics in data science. We will begin with the CUDA and OpenCL languages for parallel programming on Graphics Processing Units (GPUs) and OpenMP for multi-core programming. While machine learning and parallel programming are separate topics we include GPU and multi-core computing in this course because of their applications in machine learning on large datasets. We will then discuss recent papers in machine learning throughout the semester such as deep learning, representation learning, optimization algorithms, algorithms for big datasets, and advance Bayesian methods. Students will present recent papers and work on GPU assignments. Students have the option of doing a deep learning assignment or a project of their undertaking.

Course plan:

Topic	Date	Notes
Introduction to GPU computing	01/19/2017	Introduction Basic Unix command sheet Instructions for AFS login
CUDA exercise	01/26/2017	Parallel chi-square 2-df test Chi-square 2-df test in parallel on a GPU Simulated GWAS Class labels for above data Assignment 1 Chi8 (Makefile and sample script file include) Stacking and random hyperplanes
Random hyperplanes and stacking	02/02/2017	Student talks: Random hyperplanes by Jay Patel Is stacking better selecting best classifier by Kalyani Papers Projects
Snow day	02/09/2017
No class - to be rescheduled	02/16/2017
OpenCL	02/23/2017	Proof of Johnson-Lindenstrauss lemma for random projections from Foundations of Machine Learning by Mohri, Rostamizadeh, Talwalkar Is margin preserved after random projection? Student talks: Extreme learning machine by Abdulrhman Aljouie (slides) CUDA to OpenCL slides libOpenCL.so (NVIDIA library file for OpenCL code) Chi2 opencl implementation OpenCL files Assignment 2
Student talks	03/03/2017	Student talks: Learning Feature Representations with K-means by Shuai Zhao (slides)
Student talks	03/09/2017	Student talks: Analysis of single-layer networks in unsupervised feature learning by Shuai Zhao (slides)
Student talks	03/10/2017	Student talks: Random Projections for Support Vector Machines by Kshitija Pansare (slides) Random Projections for Support Vector Machines (Full proofs) Support vector machines
Spring break	03/17/2017
Student talks	03/23/2017	Student talks: Decision trees, random forests, boosting, and ensembles by Zhiqi Peng Basic statistics Applied statistics Error bounds
Student talks	04/06/2017	Student talks: Deep learning Deep neural networks are easily fooled: high confidence predictions for unrecognizable images Slides (part I) Slides (part II)
Student talks	04/07/2017	Student talks: Map-Reduce for Machine Learning on Multicore Slides
Student talks	04/13/2017	Student talks: Slides (Coordinate descent talk) Slides (Randomness in neural networks) Slides (SVMs) Assignment 3 Datasets Results on 52 datasets((CSV format) CIFAR CNN in Keras thanks to Girish Sukhwani MNIST CNN in Keras thanks to Girish Sukhwani SGE script for keras jobs
Student talks	04/20/2017	Student talks: Slides (Coordinate descent talk) Slides (Empirical performance of programs) Slides (Bias, variance, 0/1 loss)
Student talks	04/27/2017	Student talks: Slides (Bias, variance, 0/1 loss) Slides (Bayesian learning) Slides (Building high level feautures with unsupervised learning)
Final projects CKB 341 11:30 to 2pm	05/09/2017
Papers		Ensemble methods Ensemble methods Neural network ensembles Generalization bounds for averaged classifiers On Bias, Variance, 0/1Loss, and the Curse-of-Dimensionality Stacking: creating new features by combining different classifiers Bagging, Boosting and the Random Subspace Method for Linear Classifiers A New Fuzzy Stacked Generalization Technique and Analysis of its Performance Comparing Bayes Model Averaging and Stacking When Model Approximation Error Cannot be Ignored Linearly Combining Density Estimators via Stacking Combining Stacking With Bagging To Improve A Learning Algorithm Stacking Bagged and Dagged Models Stacking machine learning classifiers to identify Higgs bosons at the LHC Issues in stacked generalization Stacked regressions Subsemble: an ensemble method for combining subset-specific algorithm fits Super learner How to make stacking faster and better Ensemble-based classifiers Is Combining Classifiers with Stacking Better than Selecting the Best One? Random methods, random hyperplanes, extreme learning machines, randomness in neural networks On Random Weights and Unsupervised Feature Learning Feed forward neural networks with random weights Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy? Random Projections for Support Vector Machines Random features for large-scale kernel machines Randomness in Neural Networks: An Overview Random projection in dimensionality reduction: Applications to image and text data An Elementary Proof of a Theorem of Johnson and Lindenstrauss Database-friendly Random Projections Database-friendly random projections: Johnson-Lindenstrauss with binary coins Projections Extreme Learning Machine: A New Learning Scheme of Feedforward Neural Networks (online discussion of ELM facebook, reddit) Random bits regression: a strong general predictor for big data (also here) Random Bits Forest: a Strong Classifier/Regressor for Big Data Loss functions Are loss functions all the same? Minimizing The Misclassification Error Rate Using a Surrogate Convex Loss Optimization Coordinate descent algorithms Algorithms for Direct 01 Loss Optimization in Binary Classification Optimizing 0/1 Loss for Perceptrons by Random Coordinate Descent Deep learning Intriguing properties of neural networks All you need is a good init STACKING-BASED DEEP NEURAL NETWORK: DEEP ANALYTIC NETWORK ON CONVOLUTIONAL SPECTRAL HISTOGRAM FEATURES Deep learning with SVMs Simplicity of Kmeans versus Deepness of Deep Learning: A Case of Unsupervised Feature Learning with Limited Data Learning Feature Representations with K-means Analysis of single-layer networks in unsupervised feature learning Building high-level features using large-scale unsupervised learning Mapreduce Map-Reduce for Machine Learning on Multicore An Iterative MapReduce Approach to Frequent Subgraph Mining in Biological Datasets GPU machine learning Large-scale parallelized sparse principal component analysis Parallel GPU Implementation of Iterative PCA Algorithms Singular value decomposition on GPU using CUDA Fast support vector machine training and classification on graphics processors GPU library for deep learning Comparison of classifiers Do we Need Hundreds of Classifiers to Solve Real World Classification Problems? An Empirical Comparison of Supervised Learning Algorithms
Projects:		Jay: Random hyperplanes Kalyani: Stacking