CS 345 Fall 2016 : assignments, notes, exams, paper links
The copyrighted material downloadable from this page is to be
used only by the students enrolled in CS 345 under Prof. Gerbessiotis.
Distribution of this material outside this group is
NOT allowed for any reason.
C1. Midterm Performance
C2. Solutions to assignments and exams
Homeworks are posted in the handouts section i.e. section B. This is section C: only sample solutions
might be posted in this area, as applicable.
-
HW1 solutions.
Comments on HW1.
Uploaded on 10/03 . √
-
HW2 solutions.
Uploaded on 10/21 . √
-
HW3 solutions.
Uploaded on 11/17 . √
-
HW4 solutions.
Uploaded on 12/03 . √
-
HW5 solutions: No solutions.
HW5 is a paper presentation.
Check Section C6.L0 for the presentation schedule
and other information √
-
HW6 solutions: No solutions.
HW6 is the Mini-Project. √
C3. Exams, Statistics, and solutions to selected exams
Statistics for the class exam will be become available through
CS345 exam statistic link.
Edited on 12/19 . √
Exam 1:
Exam 1.
Exam Uploaded on 10/27 . √
Exam1 Solutions.
Exam Uploaded on 11/01 . √
Exam 2:
Exam 2.
Exam Uploaded on XX/XX . √
C4. Previous Exams
The information available here is as is. There is no guarantee that any future exam will look like
previous exams (nor will it be so). Sample solutions are provided for some but not all exams.
Ocasionally an exam problem might become another's exam problem or
even a homework problem and you might find a solution elsewhere.
Note that starting with Fall 2016 exam schedule will be changed.
C5. Lecture Summaries aka Supplementary Notes and Paper links
DISCLAIMER:
The included material DOES NOT substitute the textbook
for this class. It should be used in conjunction with the textbook and the
material presented in class. If a statement in these "notes" seems to be
incorrect, report it to the instructor so that it be fixed immediately.
These "notes" are distributed to the students of CS485
offered by Dr. Gerbessiotis at the New Jersey Institute of Technology;
distribution outside this group of students is prohibited.
The material below will be uploaded in due time; an upload message will
appear as soon as the corresponding document is uploaded.
SUBJECTS
-
Subject 1 (Web Search : Introduction)
(Based on material from Chapter 1 of the textbook plus some more)
Uploaded on 09/05 √
-
Subject 2 (Web Search Engine Architecture)
(Based on material from Chapters 1, 2, 3.1-3.4, 4.2.2, 4.3-4.4, 5.6.3, 8.4
of the textbook plus some more on Google)
Uploaded on 09/13 and corrected on 09/20 √
-
Subject 3 (Google Search Engine Architecture)
(Based on paper link L1 pdf and html version and 4.2.1 (Heaps' Law))
Uploaded on 09/20 √
-
Subject 4 (Index and Document Properties and Compression)
Chapter 3 (3.5-3.7), Chapter 4 (but not 4.5.2), Chapter 5(5.4-5.5),
Chapter 6)). It also includes primarily CS 114 material which will be used as background for
Web search-related data structure description.
Uploaded on 10/03 √
-
[UPDATED, Add page 29 no other change] Subject 5 (Google Cluster Architecture)
(Based on paper Link L2 below)
Uploaded on 10/17 and Updated on 11/09 √
Previous Copy (without page 29)
-
Subject 6 (The index and the indexing process)
(Based on Chapter 5 of the textbook)
Uploaded on 11/01 √
-
Subject 7 (Retrieval modeling)
(Chapters Chapter 4 (4.5), Chapter 7 (7.1,7.5) and Chapter 8)
Uploaded on 11/18 √
-
Subject 8 (Ranking: HITS and PageRank)
Chapter 4 (4.5)
Uploaded on 11/18 √
-
Subject 9 (Google File System and BigTable)
Uploaded on 11/29 √
Paper and Code Links
- L0
-
Fall 2016 CS 345 Paper Presentation Schedule (Microsoft Word)
-
Fall 2016 CS 345 Paper Presentation Schedule (Text version).
-
The following links will be populated with the presentations and their one page summaries.
-
-
Last modified on 12/12 at 10:00. √
- L1
- Google paper (Brin and Page) in
http://www-db.stanford.edu/~backrub/google.html
The pdf version of this paper that might included additional material is mirrored locally
in Google (Brin and Page).
- L2
- Google Cluster Architecture (Barroso et al) in
http://research.google.com/archive/googlecluster-ieee.pdf
Also mirrored locally in pdf in
Google Cluster Architecture (Barroso et al).
- L3
- Who Links to Whom (Bharat et al) in
http://theory.lcs.mit.edu/~ruhl/papers/2001-icdm.pdf
Also mirrored locally in pdf in
Who link to Whom (Bharat et al).
- L4
- WebCrawlers
- WebSphinx (CMU,USA)
CMU's WebSPHINX. A local copy is available at
this link.
- UbiCrawler (Italy)
http://ubi.imc.pi.cnr.it/projects/ubicrawler/.
- WIRE (Chile)
http://www.cwr.cl/projects/WIRE/.
- Paper: Parallel Web Crawlers
Cho and Garcia-Molina (Stanford).
- L5
- Code Links
- Recursive directory descent
recdir.c.
- L6
- Google Architecture Review (2009) in
http://research.google.com/people/jeff/WSDM09-keynote.pdf
Also mirrored locally in pdf in Google Architecture Review(Jeff Dean)).
- L7
- HW 5 papers (selection)
- Google Crawler for Deep Web
PDF.
- PageRank related paper
PDF.
- Aardvark Social Network search
PDF.
- L3 paper
PDF.
- L6 paper
PDF.
- Google File System
PDF.
- BigTable
PDF.
- Kleinberg's HITS algorithm
PDF.
- Google's MAPREDUCE
PDF.
- Web as a Graph
PDF.
- Twitter
PDF.
- Semantic Web
MS Word.
PDF.
- The Datacenter as a computer by Barroso and Holzle
Book Location (downloadable pdf).
Local PDF of newer edition.
Local PDF of old edition.
- PEW Center Survey
http://www.pewinternet.org/pdfs/PIP_Internet_and_Daily_Life.pdf (this link might not be active any more).
- Host Survey
http://news.netcraft.com/archives/web_server_survey.html.
- Web size
http://www.vldb.org/conf/2001/P069.pdf.
- Search Engines (query statistics)
http://www.webology.org/2004/v1n2/a4.html.
- Zmap
Zmap PDF.
- Internet Census 2012
Internet Census 2012.
- Designs, Lessons and Advice from Building Large
Distributed Systems
Jeff Dean 2009 LADIS (Large Scale Distributed Systems and Middleware) Keynote talk .
- Case Study
Google's Green Data Centers.
- Case Study
Dell: Power Efficiency Comparison.
- Case Study
High Performance Datacenter Networks.
- Collection of Papers on AWS (Data Center centric)
Web-page of James Hamilton (AWS Team), papers from 2011-2014.
- Large-scale cluster management at Google with Borg
Paper at ACM web-site (Proceeding
EuroSys '15 Proceedings of the Tenth European Conference on Computer Systems).
Local copy.
- Availability in Globally Distributed Storage Systems
by D. Ford et al (Proceedings of the 9th USENIX Symposium on Operating Systems Design and Implementation, 2010): Google Labs link.
Local copy.
- AdWords and Generalized On-line Matching
Journal of the ACM (JACM) JACM Homepage archive
Volume 54 Issue 5, October 2007 Article No. 22.
Copy from Stanford.
- Pregel: a system for large-scale graph processing
by G. Malewicz, M. H. Austern, et al in ACM SIGMOD 2010: ACM Link with pdf.
by G. Malewicz, M. H. Austern, et al: github copy.
local copy.
- Spanner: Google's Globally-Distributed Database
James C. Corbett, Jeffrey Dean, et al (10th USENIX OSDI 2012).
USENIX link.
local copy.
- Finding a needle in Haystack: Facebook’s photo storage
Doug Beaver et al (8th USENIX OSDI 2010).
local copy.
- The PageRank Citation Ranking: Bringing Order to the Web
Page, Brin, Motwani, Winograd (Stanford Tech Report 1999) .
local copy.
- Network Traffic Characteristics of Data Centers in the Wild
Benson et al (2010) .
local copy.
- A Survey and Evaluation of Data Center Network Topologies
https://arxiv.org/abs/1605.01701.
local copy.
- Scale and Efficiency in Data Center Networks
A. Vadhat .
local copy.
- A Survey of Data Center Network Architectures
Yang Liu et al .
local copy.
- Inside the Social Network’s (Datacenter) Network
ACM link .
Roy et al (UCSD link) .
local copy.
- Data Center Network Topologies
Raj Jain .
local copy.
- Analyzing the Efficiency of a Green University Data Center
Pegus et al .
Link to paper.
local copy.
- Connecting the World: A look inside Facebook's Networking
A. Moorthy.
local copy.
- L8
- Paper presentation Schedule available through link L0 above.
- L9
- NSA Data Center in
Wired article on NSA's Utah Data Center (by James Bamford, 2012), another one
Wired article on NSA's Utah Data Center Electricity issues (by Klint Finley, Oct 2013), another one
Wired article on NSA's Utah Data Center and Taxes (by Klint Finley, May 2013), another one
Wired article on NSA's Utah Data Center Water Usage (by Robert McMillan, Mar 2014)
Fox13 Salt Lake City's article on NSA's Utah Data Center Water Bill (by Ben Winslow, Apr 2014)
Forbes Article (by Kashmir Hill, Jul 2013)
Last modified Dec 19, 2016 13:59