CIS 475 EVALUATION OF COMPUTER APPLICATIONS

Summer 2006 Syllabus

Mon, Wed, Thu 1:00 - 4:00pm (2nd Session - 7/5/06-8/7/06)

Contact Information

Instructor:        Umar Qasim

Office:              Co-Lab – 4323 GITC Building – 4th Floor

Office Hours:   Monday(12:00 pm - 1:00 pm) Wed ( 12:00 pm - 1:00 pm)

Online Hours:  Thu (11 am - 1 pm)

IM:                  umar.qasim@yahoo.com

Web Site:         http://web.njit.edu/~muq2

Telephone:       (973) 596 5422 (Email is preferred)

E-mail:             muq2@njit.edu

Course Objectives

  1. Learn to evaluate computer applications, including human and organizational aspects of performance, in a scientific and objective way.
  2. To give you enough skills to carry out a case study, design a questionnaire, do interviews, and carry out a controlled experiment.  It introduces methods of analyzing data, and how to analyze an information system, whether a web site, groupware or legacy organizational software. How to identify usability problems, and test the relative merits of alternative designs. Students read journal articles about and practice methods like semi-structured interviews, user surveys, and experiments.

Requirements

        Readings:

  1. Text book: H. Russell Barnard, 2000, Social Research Methods. Sage Publications
  2. A number of recent articles for each week, giving examples of different research methods being applied to analyze and improve software usability, chosen from ACM and other publications that are available online to NJIT students through the digital library facilities.

        Prerequisite: a course in probability and statistics, or social science research methods.

Assessment

    Course assessment (subject to modification) is 65% course work and 35% exams, broken down as follows:

Assessment

Allocation

Graded Out Of
Assignment 1. Research Design

10%

100
Assignment 2. Protocol Analysis

15%

100
Final Project.  

20%

100
Group Presentation

10%

100
Class Participation

10%

100
Mid-Term

10%

100
Final Exam

25%

100
TOTAL           

100%

 

Topics

Chapter numbers refer to the given text book. Papers are in the ACM digital library, unless a link is given.

No

Week

Lesson

Reading

 

 

 

 Part I. Introduction

 

 

1.

   L1 Information systems research.
Qualitative vs. quantitative methods

Epistemology, rationalism, empiricism, positivism, interpretivism, cognitive approach;

Chapter 1

Assignments 1

 

 

2.

   L2. Scientific method concepts.
Variables, measurement, unit of analysis, validity, reliability, hypotheses, causal relationships; Ethics of social research		 Read Chs 2 and 3

Nancy C. Goodwin Functionality and Usability CACM March 1987, 229-233.

 

 

 

 Part II  Qualitative Research    

3

 

 Interviewing. Unstructured and semi-structured interviews Ch 6.
L. Wood, Semi-structured interviewing for user-centered design, Interactions, March-April 1997, 48

 

  

4-5

   Participant observation/ observation/ protocol analysis / case and field studies Chs. 9 and 10

Maloney-Krichmar and Preece, “A multilevel analysis of sociability, usability and community dynamics in an online health community,” ACM TOCHI (find online in ACM digital library), 12, 2 ( June 2005), pp. 201- 232.

Boren, M. Ted and Ramey, Judith.  Thinking Aloud: Reconciling theory and practice. IEEE Transactions on Professional Communication, 43, 3, Sept. 2000, 261- 278.  

Skim: Holzinger, A. Usability engineering methods for software developers. Communications of the ACM 48, 1 (January 2005), 71- 74.

Assignment 1 due; assignment 2 issued

 

6

   Exam 1  

 

 

   PART III  QUANTITATIVE METHODS: DATA COLLECTION  

 

7

    Sampling. Sampling frame, population, sampling methods, sample size, central limit theorem and confidence intervals. Ch 5

Bailey and Pearson  "Development of a Tool for Measuring and Analyzing Computer User Satisfaction"  Management Science, , May 1983, p530-545.

 

8

   Questionnaires and surveys. Question wording, format, response rate problem, survey techniques.

Scales and scaling. Simple, complex, Guttman, Likert, semantic differential, testing uni-dimensionality.

 Ch 7

 

Ch 8
 

Lederer A.L. et. al  "The role of ease of use, usefulness and attitude in the prediction of world wide web usage"  ACM CPR , , undefined 1998, p195-204.

Assignment 2 due: Final Project issued

 

9.

   Experimental design  Ch 4

Ellen Hoadley, Investigating the effects of color," CACM, 33, 2 (February 1990), 120-125.

 

 

      

 

 

 

 Part III.  DATA ANALYSIS

 

 

10.

   Qualitative and quantitative analysis

Univariate analysis and correlation. Raw data, frequency distributions, measures of central tendency and dispersion, graphing variables, z-scores and chi-square test

 Chs  11 and  14

B. J. Fogg et. al “What makes Web sites credible?”: a report on a large quantitative study, Proceedings of the SIGCHI conference on Human factors in computing systems, p.61-68, March 2001, Seattle, Washington, United States

 

11.

   L11. Bivariate Analysis 1. t-test, ANOVA, cross-tabs, Lambda, chi-square, Gamma Ch 15, p545-576

Xue, Y., Sankar, C.S., and Mbarika, V.W., (2003), "Multimedia and Virtual Teams: Results of an Experimental Research", American Society for Engineering Education Southeast Section Conference, Macon, GA, April 2003. http://www.auburn.edu/research/litee/media/pdfs/eval_conf_papers/2003se_lucky.pdf

 

 

12.

   L12. Bivariate Analysis 2. Correlation, Spearman’s and Pearson’s r’s, regression, significance, Eta Ch 15 p576-612

 

13.

   L13. Multi-variate Analysis. Partial correlation, multiple regression, path analysis, factor, cluster and discriminant analysis Ch 16.

 

14.

    review

Final project due

 

15.

    Final exam  

 

 

Copying/Cheating

 

All students are expected to follow published guidelines on academic honesty and integrity. You must acquaint yourself with these policies before submitting any assignments. All written work must be original. Violations of NJIT policies will be reported to the Dean of Students and may result in failure on a particular assignment, failure in the course, failure in the course and probation, or failure in the course and expulsion. Honor Code violations will be pursued immediately and aggressively.

 

Using "TURNITIN.COM"

A copy of all individual assignments, including the final project, will be required to be submitted to the "turnitin.com" web site. This is in addition to paper copies that may be supplied to the professor, or to posting a copy of the assignment for the class, according to detailed instructions that will be given for assignments. This service checks your work against everything available on the world wide web and in the archives for all sections of 675 past and present, and gives a report on exactly where your work is similar to that of other work that can be found. This is an objective way to help determine whether plagiarism has occurred.