Topic
|
Date
|
Notes
|
Basic machine learning and Python scikit-learn
|
|
Introduction
Basic Unix command sheet
Instructions for AFS login
Basic machine learning background with Python scikit-learn
Datasets
|
Introduction to GPU computing
|
|
GPU coding (also see Cuda by Example by Kandrot and Sanders)
Numpy tutorials
Official Numpy tutorial
External Numpy tutorial
CUDA in Python
Numba
CUDAJIT in Anaconda
PyCUDA (PyCUDA slides)
|
CUDA programming
|
|
Parallel chi-square 2-df test
Chi-square 2-df test in parallel on a GPU
Simulated GWAS
Class labels for above data
|
CUDA programming
|
|
Parallel Chi-square 2-df test
Exercise 1
|
OpenCL and OpenMP programming
|
|
CUDA to OpenCL slides
libOpenCL.so (NVIDIA library file for OpenCL code)
Chi2 opencl implementation
OpenCL files
CUDA to OpenMP slides
OpenMP reference
Chi2 openmp implementation
Exercise 2
|
Neural networks
|
|
Multi-layer perceptrons
Scikit-learn MLP code
Approximations by superpositions of sigmoidal functions (Cybenko 1989)
Approximation Capabilities of Multilayer Feedforward Networks (Hornik 1991)
The Power of Depth for Feedforward Neural Networks (Eldan and Shamir 2016)
The expressive power of neural networks: A view from the width
(Lu et. al. 2017)
|
| Back propagation for single layer network with numpy
|
|
Project 1
Project 1 template code as a starting point
Stochastic gradient descent variation
XOR train data
XOR test data
Sample backprop output on xor_train and
xor_test, eta=0.1, epochs=1000, stop=0
Sample backprop output on ion.train.0 and
ion.test.0, eta=0.001, epochs=1000, stop=.001
Train data for convnet
Test data for convnet
Output for convnet with eta=.1, epochs=1000
|
| Image classification
|
|
Image classification code
Python Image Library
|
| Convolutional kernels for images
|
|
Convolutional kernels
|
| Convolutional neural networks
|
|
Convolutional neural network
(Additional slides by Yunzhe Xue)
Flower image classification with CNNs code
|
| Stochastic gradient descent
|
|
Optimization in neural networks
Stochastic gradient descent
Exercise 4
|
| Back propagation for convolutional network with one convolutional layer
followed by global average pooling
|
|
Mid-term review sheet
|
| Introduction to Keras: basic deep convolutional neural networks
|
|
Image classification code v2
Bias variance tradeoff in deep neural networks
|
| Keras, batch normalization, basic deep networks, and mid-term review
|
|
Batch normalization
Batch normalization paper
Group normalization paper
How does batch normalization help optimization
|
| Mid-term exam
|
|
|
| Convolutional neural networks: gradient descent optimization. Large-scale image
classification
|
|
Gradient descent optimization
An overview of gradient descent optimization algorithms
Curriculum learning
|
| More on training convolutional neural networks
|
|
On training deep networks
The Loss Surfaces of Multilayer Networks
Ant colony optimization for deep networks
Simulated Annealing Algorithm for Deep Learning
|
Convolutional neural networks in Keras: pertrained models and transfer learning
Common architectures: ResNet, DenseNet, VGG
|
|
A guide to convolution arithmetic for deep
learning
Common architectures
Transfer learning by Yunzhe Xue
Transfer learning in Keras
Pre-trained models in Keras
Rethinking ImageNet Pre-training
Identity Mappings in Deep Residual Networks
Project 2
Chest X-rays
Project 2 train template
Project 2 test template
Optional datasets for transfer learning exercise
Fruits
Flowers
|
| Data augmentation, transposed convolutions, generative networks, GANs
|
|
Understanding data augmentation
for classification
SMOTE: Synthetic Minority
Over-sampling Technique
Dataset Augmentation in Feature
Space
Improved Regularization of
Convolutional Neural Networks with Cutout
Data Augmentation Using
GANs
|
| Transposed convolutions, generative networks, GANs
|
|
Exercise 9: MNIST GAN
|
| Image localization and segmentation, adversarial attacks, robust
machine learning
|
|
U-Net: Convolutional Networks for Biomedical Image Segmentation
GitHub UNet in Keras
Unsupervised Representation Learning with Deep Convolutional
Generative Adversarial Networks
|
| Adversarial attacks and robust machine learning
|
|
Exercise 10a: White-box adversarial attack
Exercise 10b: Black-box adversarial attack
Keras model to be attacked
Why deep-learning AIs are
so easy to fool
Intriguing properties of neural
networks
Explaining
and Harnessing Adversarial Examples
Adversarial examples in the physical
world
Adversarial Examples Are Not Easily Detected:
Bypassing Ten Detection Methods
Transferability in Machine
Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
Practical Black-Box Attacks against Machine
Learning
Adversarial
Machine Learning at Scale
Adversarial Training Can Hurt
Generalization
Robustness May Be at Odds with
Accuracy
Simple Black-box Adversarial Attacks
GenAttack: Practical Black-box Attacks with
Gradient-Free Optimization
HopSkipJumpAttack: A Query-Efficient Decision-Based
Attack
Adversarial Examples Are a Natural
Consequence of Test Error in Noise
Benchmarking Neural Network Robustness to
Common Corruptions and Perturbations
Exploring the Landscape of Spatial
Robustness
Hidden Cost of Randomized Smoothing
How Robust are Randomized Smoothing based Defenses to
Data Poisoning
The Limitations of Adversarial Training and the
Blind-Spot Attack
|
| Text data
|
|
CNNs for text
Convolutional Neural Networks for
Sentence Classification
A Sensitivity Analysis of (and
Practitioners' Guide to) Convolutional Neural Networks for Sentence Classification
Character-level Convolutional Networks
for Text Classification
Very Deep Convolutional Networks for
Text Classification
Intro to transformers
Python regular
expressions
Perl regular
expressions
Word tagging with nltk
Word2Vec for word representations
Word2Vec paper
Word2Vec
follow-up paper
Word2Vec
illustration
Exercise 11: word2vec exercise
Python Gensim library
Pandas library
Liar Liar dataset
Basic spam filtering
Fake news data
|
| More deep learning applications: Data clean and data quality, and doing
basic science and engineering with deep learning
|
|
Solving problems in Physics, Engineering, and Computer Science with
deep learning
Computer Science
Learning to Repair Software Vulnerabilities with
Generative Adversarial Networks
The Case for Learned Index Structures
How
to sort numbers using Convolutional Neural Network?
An O(N) Sorting Algorithm: Machine
Learning Sort
Shortest path distance approximation using deep
learning techniques
Physics
Newton vs the machine: solving the chaotic
three-body problem using deep neural networks
Learning to predict the cosmological
structure formation
Biology
Deep
learning models for bacteria taxonomic classification of metagenomic data
DeepMicrobes: taxonomic
classification for metagenomics with deep learning
Computational Protein Design with Deep Learning
Neural Networks
Medical diagnosis
Classification of histopathology imnages
A multi-path 2.5 dimensional
convolutional neural network system for segmenting stroke lesions in brain MRI
images
Vessel lumen
segmentation in internal carotid artery ultrasounds with deep convolutional neural
networks
Vessel lumen segmentation in carotid
artery ultrasounds with the U-Net
convolutional neural network
Classifying Histopathology Images with Random Depthwise
Convolutional Neural Networks
Engineering
Deep learning for molecular design - a review of
the state of the art
Airport and runway repair identification
Finance
Stock price prediction
Portfolio management
Partial and ordinary differential equations
Neural Ordinary Differential Equations
|
| Deep learning with Pytorch
|
|
Flower classification in Pytorch
|
| Other topics: Image retrieval, self-designed networks,
self-supervised learning, convolutional temporal kernels vs recurrent networks
|
|
Image retrieval:
Deep Learning for Image Retrieval: What Works and What Doesn't
CNN Features off-the-shelf: an Astounding Baseline for Recognition
Deep Learning for Content-Based Image Retrieval: A
Comprehensive Study
Self learning networks:
Exploring Randomly Wired Neural Networks for Image
Recognition
Automatically Designing CNN Architectures Using Genetic Algorithm
for Image Classification
Unsupervised feature learning:
UNSUPERVISED REPRESENTATION LEARNING BY PREDICTING IMAGE
ROTATIONS
Unsupervised Visual Representation Learning by Context
Prediction
Unsupervised Representation Learning with Deep Convolutional
Generative Adversarial Networks
A Simple Framework for Contrastive Learning of Visual Representations
Time series networks:
Temporal
convolutional kernels
Recurrent neural networks and
LSTMS
Wavenet: A generative model for raw audio
An
Empirical Evaluation of Generic Convolutional and Recurrent
Networks for Sequence Modeling
|
| Extra credit project and final exam review
|
|
Extra credit project
Final exam review sheet
|