Deep
Learning - CS677 - Syllabus
Spring 2023
Course Modality:
This is an online course, which will
be conducted fully online, asynchronously via Canvas. For more information on
using Canvas and other supported learning tools, visit the IST Service Desk Knowledge Base.
|
Instructor |
Email |
Office Hours |
|
Ioannis
Koutis |
ikoutis+677@njit.edu |
TBA |
*I will respond to all emails/Inbox messages within
48 hours. Quizzes, homework, and discussions will be graded weekly.
Grader:
CS675 or instructor permission.
This course covers current topics in data science. The topics include but are not limited to parallel programming on GPU and CPU multi-cores, deep learning, representation learning, optimization algorithms, and algorithms for big datasets. Students will present recent papers in data science, work on programming assignments, and do a machine learning/deep learning/data science project.
Deep Learning (DL) is a
subfield of Machine Learning that has delivered disruptive technologies, and
created AI algorithms that outperform humans in various tasks. It paves the way
for broader advances in science. DL consists of a set of specialized techniques
that exploit the abundant availability of data and computational power to build
models that are composed of multiple processing layers and learn
representations of data at multiple levels of abstraction. Only a few years
back, the development of DL models required significant expertise, but the
introduction of open-source DL libraries like TensorFlow and PyTorch has opened
the area to scientists and professionals with more diverse backgrounds. The
course opens with a review of Artificial Neural Networks that guides you
through PyTorch and enables you to build novel ANN architectures. Then it
presents the evolution of progressively deeper architectures for Convolutional
Neural Networks, that addressed various training difficulties and led to very
successful image classification models. The course then takes you to the emerging
applications of Recurrent Neural Networks in temporal data, including Natural
Language Processing. In this context, you will learn how Attention and
Transformers have led to better language models. You will also learn about
Graph Neural Networks and their applications in the analysis of real-world
networks (e.g., social, or biological networks). The course may also touch upon
selected topics like the ability of deep networks to generalize, techniques for
'pruning' deep networks to make them more computationally efficient, and
successful applications of DL methods in the Sciences.
By the end of the course, students will be able to:
Dive into Deep
Learning
A. Zhang , Z. Lipton, M. Li, A. Smola
The textbook is open, free and available here.
The grading policy is designed to reflect the NJIT Grading Legend
Final grades for all assignments will be based on the
following percentages:
|
Quizzes |
25% |
|
Discussion Forums and
Participation |
15% |
|
Exercises/ Learning Activities |
25% |
|
Projects |
35% |
Raw numerical scores will be converted to letter
grades using the following bounds.
|
A |
B+ |
B |
C+ |
C |
F |
|
|
|
|
|
|
<50 |
In some cases a letter
grade can be upgraded to the next letter to reflect natural clusters of
performance.
Quizzes: (25% of grade) There will be weekly short multiple-choice quizzes, worth 10% of
the total grade. These are meant to help you keep up with the most important
theoretical concepts. These quizzes are not proctored and the two weakest
scores will be dropped. There will be one 90-minutes summary quiz, worth 10%.
This is required and proctored. It is meant to simulate an interview
environment and assess your overall understanding of the material.
Discussion Forums
and Participation Activities: (15% of grade) When all students participate in a discussion, it creates
an active learning environment that will help you better understand the
materials and be more successful in the class. You are expected to participate
in two types of forums: (i) Weekly discussion forums
in Canvas, with Q&A about the week’s material (9%). Your contributions are
due by Sunday, 11:59 pm. (ii) A
permanent discussion forum on successful applications of the material we cover
(5%). Other participation activities are worth 1%.
Exercises/Learning
Activities: (25% of grade) Assignments will
be given bi-weekly (up to week #12) to give you an opportunity to apply course
concepts for that week. These activities are designed to help you practice and
prepare for the project. The weakest grade will be dropped automatically.
Projects: 35% of grade The project will consist of three milestones, with weights [10%, 5%, 20%].
You will have opportunities to iterate
and revise your work based on peer and instructor feedback.
Assignment solutions will be distributed for
each assignment, along with general class-level feedback from the grader.
Occasionally, and when needed, you will also receive individualized comments
directly on your assignment notebook. You can also always directly inquire
about a specific grade item. In that case please email both the instructor and
the grader.
This course has two proctored quizzes. These
will take place in the classroom, and presence is required. The quizzes will be
on LockDown browser, so please make sure you bring
your computers charged. The majority of your grade is based on authentic
assessment, meaning that you will be assessed and graded on your ability to
deliver real-world outputs as well as your participation and feedback to other
students.
2% will be subtracted from the delayed
assignment grade for each hour of delay.
Some homework problems will be challenging. You are advised
to first try and solve all the
problems on your own. For problems that persist you are
welcome to talk to the course assistant or the instructor. You are also allowed
to collaborate with your classmates and search for solutions online. But you
should use such solutions only if you understand them completely (admitting
that you don't understand something is way better than copying things you don't
understand). Also, make sure to give the appropriate credit and citation.
“Academic Integrity is the cornerstone of higher education
and is central to the ideals of this course and the university. Cheating is
strictly prohibited and devalues the degree that you are working on. As a
member of the NJIT community, it is your responsibility to protect your
educational investment by knowing and following the NJIT academic
code of integrity policy.
Please note that it is my professional
obligation and responsibility to report any academic misconduct to the Dean of
Students Office. Any student found in violation of the code by cheating,
plagiarizing or using any online software inappropriately will result in
disciplinary action. This may include a failing grade of F, and/or suspension
or dismissal from the university. If you have any questions about the code of
Academic Integrity, please contact the Dean of Students Office at dos@njit.edu”
The course is organized into modules. Each week consists of 1 or 2
modules. Each week, the students should attend the week’s lecture. Whenever
that is not possible, the students are advised to watch the posted videos. The
students are also expected to read the corresponding sections of the textbook,
and participate in a class discussion forum as prompted by the instructor. The
students must also be aware of any assignments due at the end of each week.
|
Week |
Topic |
Textbook
Reading |
Due Work |
|
1 |
●
Introduction to CS677 ●
Review: Optimization and
PyTorch |
Chapter 1 |
|
|
2 |
●
Softmax Regression ●
Regularization, Dropouts,
Initialization |
Chapters 4.1- 4.5 |
Assignment-1 |
|
3 |
●
Introduction to CNNs |
Chapter 7 |
|
|
4 |
●
Modern CNNs |
Chapter 8 |
Assignment-2 |
|
5 |
●
Introduction to RNNs |
Chapter 9 |
Assignment-3 |
|
6 |
●
Modern RNNs |
Chapter 10 |
|
|
7 |
●
Attention - Transformers |
Chapter 11 |
Project
Milestone-1 |
|
8 |
●
Graph Neural Networks |
Gentle
Introduction to Graph Neural Networks |
Assignment-4 |
|
9 |
●
Review Week |
|
Summary Quiz |
|
10 |
●
Applications in Vision |
Chapter 14 |
Assignment-5 |
|
11 |
●
Word Embeddings |
Chapter 15 |
|
|
12 |
●
NLP applications |
Chapter 16 |
Assignment-6 |
|
13 |
●
Recommender Systems |
Chapter 17 |
Broader Impacts
Discussion |
|
14 |
●
Various Topics in Deep
Learning |
Chapter 20 |
Project
Milestone-3 |
|
15 |
●
Final Week |
|
Broader Impacts
Discussion |
Throughout this course, you are expected to be courteous and
respectful to classmates by being polite, active participants. You should
respond to discussion forum assignments in a timely manner so that your
classmates have adequate time to respond to your posts. Please respect
opinions, even those that differ from your own, and avoid using profanity or
offensive language.
This course is offered through an accessible learning
management system. For more information, please refer to Canvas’s Accessibility
Statement.
Requesting
Accommodations
The Office of Accessibility Resources and Services works in
partnership with administrators, faculty, and staff to provide reasonable
accommodations and support services for students with disabilities who have
provided their office with medical documentation to receive services.
If you are in need
of accommodations due to a disability, please contact the Office of Accessibility Resources and Services to discuss your specific needs.
NJIT is committed to student excellence. To ensure your
success in this course and your program, the university offers a range of
academic support centers and services. To learn more, please review these Resources for NJIT Online Students, which include information related to technical support.