CS 345 section 001 (assignments, notes, exams, paper links): Fall 2015

The copyrighted material downloadable from this page is to be used only by the students enrolled in CS 345 under Prof. Gerbessiotis. Distribution of this material outside this group is NOT allowed for any reason.

C1. Midterm Performance

How am i doing in class? CS345 Midterm class performance. Edited on 10/30 . √ Note that Exam Statistics are available in section C4.

C2. Solutions to assignments and exams

Homeworks are posted in the handouts section that is section B. Only sample solutions might be posted in this area.

HW1 solutions. Uploaded on 10/05. √
HW2 solutions. Uploaded on 10/16 . √
HW3 solutions. Uploaded on 10/22 . √
HW4 solutions. Uploaded on 10/30 . √
HW5 solutions: No solutions. HW6 is a programming mini-project. √
HW6 solutions: No solutions. HW7 is a programming mini-project. √
HW7 solutions: No solutions. HW7 is a paper presentation. Check Section C6.L0 for the presentation schedule and other information √

C3. Statistics.

Statistics for the class exam will be become available through CS345 exam statistic link. Edited on 10/30 . √

C4. Exams, Statistics, and solutions to selected exams

Exam 1: Exam 1. Exam Uploaded on 11/02 . √ Exam1 Solutions. Exam Uploaded on 11/03 . √
Exam 2: Exam 2in PDF.

C5. Previous Exams

The information available here is as is. There is no guarantee that any future exam will look like previous exams (nor will it be so). Sample solutions are provided for some but not all exams. Ocasionally an exam problem might become another's exam problem or even a homework problem and you might find a solution elsewhere.

Fall 2015 Fall 2015 Exam 1.
Fall 2014 Fall 2014 Exam 1.
Fall 2013 Fall 2013 Exam 1.
Fall 2011 Fall 2011 Exam 1. Fall 2011 Exam 1 solutions. Fall 2011 Exam 2.
Fall 2010 Fall 2010 Exam 1. Fall 2010 Exam 1 solutions. Fall 2010 Exam 2.

C6. Lecture Summaries aka Supplementary Notes and Paper links

DISCLAIMER: The included material DOES NOT substitute the textbook for this class. It should be used in conjunction with the textbook and the material presented in class. If a statement in these "notes" seems to be incorrect, report it to the instructor so that it be fixed immediately. These "notes" are distributed to the students of CS485 offered by Dr. Gerbessiotis at the New Jersey Institute of Technology; distribution outside this group of students is prohibited. The material below will be uploaded in due time; an upload message will appear as soon as the corresponding document is uploaded.

SUBJECTS

Subject 1 (Web Search : Introduction) (Based on material from Chapter 1 of the textbook plus some more) 3Sep2015-copy (retired) Uploaded on 09/04 and modified on 09/09 √

Subject 2 (Web Search Engine Architecture) (Based on material from Chapter 1,2, and Chapter 3 sections 3.1-3.4 of the textbook plus some more on Google) Addendum to Subject 2 (Map-Reduce Operations) Uploaded on 09/17, 09/25 √

Subject 3 (Google Search Engine Architecture) (Based on paper link L1 pdf and html version; typos corrected on Oct 6) Uploaded on 09/17 √

Subject 4 (Hashing,DataCompression and Coding) as revised Based on material from Chapter 3 (section 3.5,3.6, and 3.7) and Chapter 5 (5.4)). It also includes primarily CS 114 material which will be used as background for Web search-related data structure description. Uploaded on 10/09 √

Subject 5 (Google Cluster Architecture) (Based on paper Link L2 below) Uploaded on 10/09 √

Subject 6 (Document Analysis) (Based on Chapters 4 and 6 of the textbook) Uploaded on 10/09 √

Subject 7 (The index and the indexing process) (Based on Chapter 5 of the textbook) Uploaded on 11/05 √

Subject 8 (Retrieval modeling and Retrieval evaluation) (Chapters 1 and 8 (precision and recall),Chapter 4 (4.5), Chapter 7 (7.1,7.5) and Chapter 8) Uploaded on 11/05 √

Subject 9 (Ranking: HITS and PageRank) Chapter 4 (4.5) Uploaded on 11/10 √

Subject 10 (Google File System and BigTable) Uploaded on 11/20 √

Paper and Code Links

L0
Paper Presentation Schedule (Fall 2015) The link on the left includes schedule (text) of the presentations and to the right, the Fall-2015-Presentation-Schedule (Word). The following links will be populated with the presentations and their one page summaries.

SLOT1: 10:00-10:20 'Semantic Web' presentation by Ibrahima JALLOH ; One page summary

SLOT2: 10:20-10:40 'Google's Green Data Centers' presentation by Gabriel LOTERENA; One page summary (12-04:14:24). √

SLOT3: 10:40-11:00 'Google's Map-Reduce' presentation by Mubarak ADAM ; One page summary (12-06:21:24). √

SLOT4: 11:00-11:20 'High Performance Datacenter Networks' presentation by Mohammed RAZA ; One page summary (12-04:14:24). √

SLOT5: 11:30-11:50 'Twitter' presentation by Michael SELSKY ; One page summary (12-05:14:24presentation). √

SLOT6: 11:50-12:10 'Internet Census' presentation by Tyler SHUHNICKI ; One page summary (12-04:14:24). √

SLOT7: 12:10-12:30 'Search Engines' presentation by Drashti SHAH ; One page summary (12-06:10:24). √

SLOT8: 12:30-12:50 'Large-scale cluster management at Google with Borg' presentation by Stephen MORRISON ; One page summary (12-06:10:24). √

Last modified on 12/07 at 09:30. √
L1
Google paper (Brin and Page) in http://www-db.stanford.edu/~backrub/google.html The pdf version of this paper that might included additional material is mirrored locally in Google (Brin and Page).
L2
Google Cluster Architecture (Barroso et al) in http://research.google.com/archive/googlecluster-ieee.pdf Also mirrored locally in pdf in Google Cluster Architecture (Barroso et al).
L3
Who Links to Whom (Bharat et al) in http://theory.lcs.mit.edu/~ruhl/papers/2001-icdm.pdf Also mirrored locally in pdf in Who link to Whom (Bharat et al).
L4
WebCrawlers

WebSphinx (CMU,USA) CMU's WebSPHINX. A local copy is available at this link.

UbiCrawler (Italy) http://ubi.imc.pi.cnr.it/projects/ubicrawler/.

WIRE (Chile) http://www.cwr.cl/projects/WIRE/.

Paper: Parallel Web Crawlers Cho and Garcia-Molina (Stanford).

L5
Code Links

Recursive directory descent recdir.c.

L6
Google Architecture Review (2009) in http://research.google.com/people/jeff/WSDM09-keynote.pdf Also mirrored locally in pdf in Google Architecture Review(Jeff Dean)).
L7
HW 8 papers (selection)

Google Crawler for Deep Web PDF.

PageRank related paper PDF.

Aardvark Social Network search PDF.

L3 paper PDF.

L6 paper PDF.

Google File System PDF.

BigTable PDF.

Kleinberg's HITS algorithm PDF.

Google's MAPREDUCE PDF.

Web as a Graph PDF.

Twitter PDF.

Semantic Web MS Word. PDF.

The Datacenter as a computer by Barroso and Holzle PDF.

PEW Center Survey http://www.pewinternet.org/pdfs/PIP_Internet_and_Daily_Life.pdf (this link might not be active any more).

Host Survey http://news.netcraft.com/archives/web_server_survey.html.

Web size http://www.vldb.org/conf/2001/P069.pdf.

Search Engines (query statistics) http://www.webology.org/2004/v1n2/a4.html.

Zmap Zmap PDF.

Internet Census 2012 Internet Census 2012.

Designs, Lessons and Advice from Building Large Distributed Systems Jeff Dean 2009 LADIS (Large Scale Distributed Systems and Middleware) Keynote talk .

Case Study Google's Green Data Centers.

Case Study Dell: Power Efficiency Comparison.

Case Study High Performance Datacenter Networks.

Collection of Papers on AWS (Data Center centric) Web-page of James Hamilton (AWS Team), papers from 2011-2014.

L8
Paper presentation Schedule available through link L0 above.

L9
NSA Data Center in Wired article on NSA's Utah Data Center (by James Bamford, 2012), another one Wired article on NSA's Utah Data Center Electricity issues (by Klint Finley, Oct 2013), another one Wired article on NSA's Utah Data Center and Taxes (by Klint Finley, May 2013), another one Wired article on NSA's Utah Data Center Water Usage (by Robert McMillan, Mar 2014) Fox13 Salt Lake City's article on NSA's Utah Data Center Water Bill (by Ben Winslow, Apr 2014) Forbes Article (by Kashmir Hill, Jul 2013)
L10
Some notes of the instructors in Notes on parallel and multithreaded computing(dec 2009 print) A more recent compilation (using a more recent version of LaTeX and pdfLaTeX) is this one: Notes on parallel and multithreaded computing (nov 4, 2014 print) There should be no or minimal differences between the two versions; two different versions of LaTeX and PdfLaTeX were used; the more recent version available for the more recent print of the notes. (Note the chapters on web-searching there are from 2008/2009 and obsolete; they have been superseded by newer material including the notes of this CS345 course.)

Last modified Dec 7, 2015 09:31