CS 698-108: Advanced Topics in Web/Distributed Systems, Spring '10

Fridays, 6-9:05pm, FMH 106

Tentative Syllabus (some of the details may change before the first day of classes)

Instructor

Cristian Borcea
Office: GITC 4303
Phone: 973 596-3662
Office Hours: TBA
email

Short description

This course covers new architectures and programming techniques for large scale distributed systems and web services. The topics include cloud computing, data processing in large clusters, distributed data-parallel computing, distributed storage systems, virtualization, distributed debugging, secure distributed computing, and transactional memory for multicore architectures. Students will study state-of-the-art solutions for large scale distributed systems developed by Google, Amazon, Microsoft, Yahoo, Sun, Intel, VMWare, etc. Students will also apply what they learn in a semester-long project using the Amazon Web Services platform.

Who should take this course

This class is aimed at all graduate students (both M.S. and Ph.D. students) who want to learn how recent advances in distributed systems have changed the way in which large scale Internet services and applications are designed, implemented, and deployed. By studying real-world systems developed in industry during the past few years, students will acquire cutting-edge knowledge that may be a major advantage when searching for a job. Finally, this class can help students to find topics for research projects and theses.

Prerequisites

CS 656 or CS 633 or instructor's permission. If you didn't take CS 656 or CS 633, but you would like to take this class, you should come and talk with me about your background (if you have a good background, I will give you permission to register). Basic Unix/Linux skills and good programming skills are necessary for the project.

Lectures and Readings

There is no book required for this class. Each lecture is based on recent papers/articles covering a specific topic. Every week, the instructor will introduce the topic (the lecture slides will be posted before each class) and then will moderate the discussions of the papers assigned for that week. Students are required to read the papers before the class and participate in the discussions. Additionally, each lecture will include design reviews and Q/A sessions for the semester-long project.

Project

Students will work in teams of three to design and implement Internet services or systems using Apache's Hadoop and the Amazon Web Services platform. Specific projects ideas will be provided in the first weeks of classes.

Programming Assignment

There will be one individual programming assignment before the project is handed out. This assignment willl consist of a few short programs to help students get used with Hadoop and the Amazon Web Services platform.

Grading

Midterm exam - 20%
Final exam - 25%
Participation in class discussions - 20%
Project - 25%
Programming Assignment - 10%

Schedule

Week	Topic	Readings
1	Introduction. Internet-scale distributed systems. Web services.
2	Cloud Computing. Amazon's EC2 and S3.	M. Armbrust, A. Fox, R. Griffith, A. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, and M. Zaharia. Above the Clouds: A Berkeley View of Cloud Computing , Technical Report No. UCB/EECS-2009-28. Amazon Web Services Documentation.
3	Data Processing in Large Clusters I. Google's MapReduce. Apache's Hadoop. Programming assignment handed out.	J. Dean and S. Ghemawat, MapReduce: Simplified Data Processing in Large Clusters, Communications of the ACM, 2008. Hadoop Documentation
4	Data Processing in Large Clusters II. Yahoo's Pig Latin.	C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins, Pig Latin: A Not-So-Foreign Language for Data Processing, ACM SIGMOD 2008.
5	Distributed Data-Parallel Computing. Microsoft's Dryad and DryadLINQ. Programming assignment due. Projects handed out.	M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly, Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks, ACM EuroSys 2007. Y. Yu, M. Isard, D. Fetterly, M. Budiu, U. Erlingsson, P. K. Gunda, and J. Currey DryadLINQ: A System for General-Purpose Distributed Data-Parallel Computing using a High-Level Language, Usenix OSDI 2008.
6	Distributed Storage Systems I. Google's GFS and BigTable.	S. Ghemawat, H. Gobioff, and S-T. Leung, The Google File System , ACM SOSP 2003. F. Chang, J. Dean, S. Ghemawat, W. Hsieh, D. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. Gruber, Bigtable: A Distributed Storage System for Structured Data, Usenix OSDI 2006.
7	Distributed Storage Systems II. Amazon's Dynamo.	G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, amd W. Vogels, Dynamo: Amazon's Highly Available Key-value Store, ACM SOSP 2007.
8	Midterm. Discussion of midterm solutions.
9	Virtualization I. VMWare and Xen virtual machine monitors.	J. Sugerman, G. Venkitachalam, and B-H. Lin, Virtualizing I/O Devices on VMWare Workstation's Hosted Virtual Machine Monitor, Usenix 2001. P. Barham, B. Dragovic, K. Fraser, S. Hand, T, Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield, Xen and the Art of Virtualization, ACM SOSP 2003.
10	Virtualization II. VMWare and Xen virtual machine migration.	M. Nelson, B-H. Lin, G. Hutchins, Fast Transparent Migration for Virtual Machines, Usenix 2005. C. Clark, K. Fraser, S. Hand, J. G. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield, Live Migration of Virtual Machines, Usenix NSDI 2005.
11	System Debugging and Testing. Sun's DTrace.	D. Engler, D. Y. Chen, S. Hallem, A. Chou, and B. Chelf, Bugs as Deviant Behavior: A General Approach to Inferring Errors in Systems Code, ACM SOSP 2001. B. Cantrill, M. Shapiro, and A. Leventhal, Dynamic Instrumentation of Production Systems, Usenix 2004.
12	Secure Distributed Computing.	X. Qie, R. Pang, and L. Peterson, Defensive Programming: Using an Annotation Toolkit to Build DoS-Resistant Software, Usenix OSDI 2002. E. Shi, A. Perrig, and L. Van Doorn, BIND: A Fine-grained Attestation Service for Secure Distributed Systems, IEEE S&P 2005.
13	Transactional Memory/Multi-core Architectures.	B. Saha, A-R. Adl-Tabatabai, R. Hudson, C. C. Minh, and B. Hertzberg, McRT-STM: A High Performance Software Transactional Memory System for a Multi-Core Runtime, ACM PPoPP 2006.
14	Final project presentations.

Honor Code

The NJIT Honor Code will be upheld, and any violations will be brought to the immediate attention of the Dean of Students.

Modifications to Syllabus

The students will be consulted and must agree to any modifications or deviations from the syllabus throughout the course of the semester.