Official Course Description ---------------------------- CS 750 - High Performance Computing (3 credits) Prerequisite: CS 650. An in-depth study of the state of the art in high performance computing. Topics parallel computer architectures, programming paradigms, and their applications. Parallel architectures include PC clusters, shared-memory multiprocessors, distributed-memory multiprocessors, and multithreaded architectures. Parallel programming paradigms include message passing interface (MPI), its second-generation MPI-2, and multithreaded programming. Applications include computational science and high performance Web and database servers for Internet-based electronic commerce. Students program a parallel machine in class projects. First-hand experience in stable, scalable, high performance computing for Internet-based electronic commerce. Effective From: Fall 2006 Spring Semester 2006: --------------------- 1. What is covered in the course? An in-depth study of the state of the art in high performance computing The application emphasis this semester is on efficient and high-performance web-searching. 2. What traditional topics of parallel computing are covered? Topics covered include parallel computer architectures. Parallel architectures include PC clusters, shared-memory multiprocessors, distributed-memory multiprocessors, and multithreaded architectures. Flynn's taxonomy is discussed with example architectures (including CM-5, CM-200, SGI origin and Power Challenge, Cray T3D/T3E, and Intel multi-core processors, pipelined features and MMX extensions). Other topologies based on address-space organization (share-memory, distributed-memory, distributed-shared memory, NUMA, UMA, SMPs). Taxonomies based on granularity and processor synchronization are also introduced Performance characteristrics are introduced. Fixed Interconnection networks are covered (linear 2d-array, 3d-array, hypercubic networks). and their properties examined (diameter, bisection width, bandwidth etc). PC clusters are also discussed in the context of the platform used for the programming assignments and in terms of Google's cluster architecture. Multithreaded architectures are only discussed in the context of Intel's multi-core systems since past efforts Tera Computers went nowhere. Right now systems such as Cray's [excerpt from Cray's web-page] "combine standard microprocessors (scalar processing), vector processing, multithreading and hardware accelerators in one high-performance computing platform that uses the industry-standard Linux operating system". In addition we cover elementary parallel algorithms and a theoretical parallel programming model based on fully-synchronous shared-memory architecture, the PRAM. 3. What parallel programming paradigms including message passing interface (MPI), its second-generation MPI-2, and multithreaded programming are covered? We cover mostly multiprocessing programming rather than multithreaded though the latter is implicit in certain cases. We cover MPI and MPI-2 and respectively the message-passing programming paradigm (two-sided communication) of both and the remote-memory access paradigm (one-sided communication) of the latter. In addition one-sided communication and multiprocessing/multithreading is covered in the context of the BSPlib programming library implementing the bulk-synchronous parallel programming paradigm. 4. What applications including computational science and high performance Web and database servers for Internet-based electronic commerce are covered? The applications discussed in Spring 2006 all relate to Web Searching. Topics include information retrieval, high performance Web ranking (Google's PageRank and Kleinberg's hub-and-authority model), the Google system, the Google Cluster Architecture, the Google File-System and the Google Map-Reduce model. 5. What exposure do students get in parallel programming? Any first-hand experience in stable, scalable, high performance computing for Internet-based electronic commerce? The CS Department's "parallel machine" ceased operations 10 years ago. We use for the programming component a small pc cluster with 4 processors that can be extended to 8 processors depending on class size. Students write C/C++ code on top of parallel programming libraries such as BSPlib, MPI, MPI-2 code for simple computational science problems. Students write C/C++/Java/Perl code (language is their choice) that implements the components of a high performance Web search engine. Students read papers related to the topics covered. Students design thousand-processor clusters that support stable, scalable, redundant, high-availability web-searching for Internet-based activities. 6. Student Work. Approximately 8 (eight) problem sets. Students are only required to do a subset of them. Some/most problem sets are paper reading, some involve practice problems on material covered in class (PageRanking, Kleinberg's ranking mechanism, data compression methods), and very few involve program installation and execution (a web-crawler, and a parallel programming library installation on a PC cluster plus program execution in such an environment that requires "clusterization" of the environment). Students can accumulate up to [25%] of total credit through any combination of assignments (each one is worth around 4% to 6%). In addition 4 programming assinments are given to students. The emphasis on this semester is on high-performance web-searching. Three of them can utilize any programming language and are essentially sequential programs, the fourth of them deals with a parallel extension (i.e. high-performance parallel search engine component). All 4 assignments can be done individually or in groups of two. Languages of choice range among Java/C/C++/Perl for the first three to C/C++ for the fourth. The total contribution is [45%]. A project report is also expected at the end [5%] Students are also expected to complete an individual work component: either an additional programming component or a paper presentation [25%]. 7. What is available on the web This document, course syllabus, flyer (used for announcing the course in Fall 2005), problem sets and programming assignments are publically available through http://www.cs.njit.edu/~alexg/courses/cis750/handouts.html Other material such as lecture summaries, papers handed out in class, other course related papers and links, code and pointers useful for the programming assignments, and other information (eg documentation) is available through the protected area of the web-page: http://www.cs.njit.edu/~alexg/courses/cis750/solutions/index.html