Information Systems Department College of Computing Sciences New Jersey Institute of Technology CIS677-102/4: Information Systems Principles, Spring 2005 Professor Bieber
A Public Research University

Quick Links: [677 Home] [Schedule] [Syllabus] [Team-based Learning] [WebBoard access/info] [Links] [Summaries] [Article review guidelines/assignment] [Extra credit participation]

Article Review Guidelines

Notes:

1. See the article review assignment for details on which articles to choose and how to submit your review. Follow the guidelines here on structuring and formatting your review.
2. Include each category in a separate section or subsection. Ensure that you discuss each category! Apply each section and do not skip any. I will be looking for each category when grading.
3. Reference other articles you've read, if appropriate. Just as in the articles you've read, include a reference marker where you refer to each in your review, and provide full bibliographic references in a bibliography section at the end of your report.
4. Your final report will probably be 5-8 pages long (if measured in Times New Roman 12pt single spaced). This does not include your bibliography, diagrams or large gaps between parts of your report. There is no maximum length.
5. Do not copy any part of the article into your review. If you want to use more than 3-4 of the author's words, then use quotation marks, and add a page number from the article (if available) when you cite this quotation.
6. Proof-read your review for grammar and clarity as well as spelling. Spell checkers do not fix grammar. You should be sure that you have caught all mistakes and written clearly. Most word processors have a grammar checker - use it!
7. For the printed copy of your review, turn in just a plain stapled black & white copy of your review. Do not put your review in a folder. Avoid fancy covers, colors or fonts. I want to judge your content, not visual presentation.
8. Please reread the note on plagiarism posted in the syllabus and on the course Web site.

Grading Notes

We have included the grading criteria in these guidelines.

Unless otherwise noted, the categories are:

• great: fully discussed, explained well and well-justified
• ok: partial discussion; not explained well or well-justified
• poor: barely discussed
• zero: not discussed

Length Grading

I take 5 points off for each full page missing (i.e., less than 5 full pages) and 3 points off for every half page missing. Note that large gaps between sections will not count towards the full page length.

Editing Grading

Has the review been prooof-red for both spelling and grammar? If not up to 3 points off for poor grammar and up to 3 points off for poor spelling.

Preparing for Your Review

An article review results from critically examining all aspects of an article. You will have to read your article several times to understand it fully enough to review properly.

Often, comparing your article to others will help you determine its quality.

Also, think about the article and its ideas in terms of each of the different week's concepts and frameworks we study in class. Do the ideas in the article fit all the categories of appropriate class concepts, models and frameworks? This often will help you see things the authors missed, think about things the authors write in a new light, or see that the authors indeed covered a topic thoroughly.

Outline of the Article Review

Please include the following categories in your article review.

1. Full Bibliographic Reference (0 if OK; -3 if missing)
2. Introduction: Objectives, Article Domain and Audience (6)
3. Very Brief Summary (4)
4. Results (8)
5. Contribution (6)
6. Foundation (6)
7. Synthesis with Class Materials (24)
8. Analysis with Additional Analysis (6 + 8 extra credit)
9. General Critique (24)
10. Issues (16)
11. Annotated Bibliography (0 if OK; -5 if missing)

In addition, you can lose points if your review is too short or poorly edited. See the Grading Notes section above.

1. Full Bibliographic Reference

State the full bibliographic reference of the article you are reviewing (authors, title, journal name, volume, issue, year, page numbers, etc.) Important: this is not the bibliography listed at the end of the article, rather the citation of the article itself!

Grading: -3 if missing

2. Introduction: Objectives, Article Domain and Audience

Paragraph 1: State the objectives (goals or purpose) of the article. What is the article's domain (topic area)?

Paragraph 2: State the article's intended audience. At what level is it written, and what general background should the reader have; what general background materials should the reader be familiar with to understand the article? (Knowing the intended readership of the journal could give you a big hint on the intended audience of the article.)

Grading: Objectives: great - 4; ok - 2; poor - 1
Grading: Audience: great - 2; ok/poor - 1

3. Very Brief Summary

For our article reviews, we do not want you to spend much space summarizing the article. Instead we are more interested in your analysis of the article.

Thus, in this section, summarize the article only very briefly (3 paragraphs as described below). Use the IS research paradigm as the format of your summary, but remaining very brief:

• Paragraph 1: what is the problem or opportunity being addressed (that motivates this article)
• Paragraph 2: which solution is proposed (the solution could be a new model or a theory that explains the problem)
• Paragraph 3: what evidence is put forth that this solution is appropriate

If the article does not provide any solutions, state this explicitly. In that case your summary only needs to be 2 paragraphs long. The second paragraph should very briefly describe the content of the other aspects of the article.

Grading: great - 4; ok - 2; poor - 1

4. Results

Briefly summarize the important points (observations, conclusions, findings) and "take home messages" in the article. Please do not repeat lists of items in the articles - just summarize the essence of these if you feel they are necessary to include. Are the article's take-home messages new?

Grading: great - 8; ok - 5; poor - 2

5. Contributions

An article makes a "contribution" by adding to the knowledge of the domain for the article's audience. An article can make a contribution in many ways. Does it explain something really clearly? Does it provide a new way to look at a problem? Does it bring together or "synthesize" several concepts (or frameworks, models, etc.) together in an insightful way that has not been done before? Does it provide new solutions? Does it provide new results? Does it identify new issues? Does it provide a comprehensive survey or review of a domain? Does it provide new insights?

Also, is it salient (relevant and current) to a particular scientific issue or managerial problem? Are the issues addressed introduced in a way that their relevance to practice is evident? Would answers to the questions raised in the article likely to be useful to researchers or managers?

Note: Do not discuss the contributions of the technologies the article describes, but rather the contributions of the article itself!

The article's contributions should be original. To the best of your knowledge, are they?

Describe each contribution clearly in a separate paragraph or bullet point. Discuss why the contribution is important.

Alternatively, if you believe the article makes no contributions, explain why clearly.

Grading: great - 6; ok - 4; poor - 2

6. Foundation

Which basis ideas or concepts does this article build upon? Which ideas, technologies or other concepts lie underneath the article's ideas? And, if it is a research-oriented article, then which research concepts, theories and frameworks lie underneath this work? In other words, what had to come before for this article's topic and any proposed solutions to be important?

Which foundations does this article build on, if any? In what ways? Include references/citations of the foundation work if appropriate. (You can determine this, in part, from the works the article cites, and in part from your own investigations.)

Note, however, that most works cited are not core foundational work, but rather just support certain aspects of the article. Similarly, do not confuse a general discussion of related topics as foundational work.

Justify your answer - be sure to describe both in what ways, and to what degree, the article builds upon these foundations.

Grading: great - 6; ok - 4; poor -2

7. Synthesis with Class Materials

The synthesis section should be at least 3/4 page.

Synthesis means analyzing a particular aspect of the article by comparing and contrasting it with, and thinking about it from the viewpoint of, the class materials from across the semester. These materials include the articles, case studies, models, frameworks, guidelines and other concepts we've covered. (Of course, only certain materials will be relevant for any given article.)

Note: You have to do this synthesis! You need to relate this article to other things we have studied, and you should look for things to relate both that are mentioned and are not mentioned in the article!!!

Discuss the article's research ideas and results in terms of any relevant materials covered in class or which you have found in the readings. Cite these readings, including the text, explicitly, including their source in the bibliography and a bibliographic marker in the text (e.g., [Turoff et al., 1999]).

You also could analyze the approach the author took to the article's analysis and discussion. Discuss the article's approach and results in terms of one or more of the frameworks, etc., from the text or readings, or any you find elsewhere. (For example, if the authors discuss any type of information system, you could use Alter's work system analysis to examine how they approached that information system.) Try to do this for all the models and frameworks, etc., which apply to your article. You could also apply aspects of this article to a case study we have studied.

As part of this analysis, reference other articles you've read, when appropriate. Compare the approach, results and contribution with all articles about similar topics or with a similar approach. Include any articles you cite in the bibliography and use bibliographic markers in the text.

For all of these, do your synthesis comparison in as much depth as you can!

Grading:

• Review #1 - two items up to 24 points total - for each item: great - 12 ok - 8; poor - 3
• Review #2 - four items up to 24 points total - for each item: great - 6 ok - 4; poor - 2

Great: discussed deeply and relating the article in detail with the synthesized models and frameworks.
OK: the synthesized information is only discussed in general

8. Analysis

Note: Many people assume this category is the same as "General Critique". It is not. General Critique is a different category from this, and follows below.

What has changed since the article was written (even though it might have been published recently)? Do its lessons, ideas and theories still apply? If so, how (in your opinion)? If not, why not? To what extent have its issues been resolved? You could also do a "what if" analysis: what if something changes - will the article's lessons still apply? This will tell you how robust the article's ideas are.

Grading: great - 6; ok - 4; poor - 2

Additional Analysis

Optionally, try it yourself! Try applying the article's models, frameworks, guidelines or ideas, etc., to something appropriate. Do you find them useful?

In addition, you may optionally add your own additional analysis in a separate subsection. (Do not repeat the author's analysis in the paper - you could summarize this as part of the results section.)

Grading: this section is extra credit only: great - 8; ok - 5; poor - 2

9. General Critique

In this section you should state your opinions of how well (or poorly) the authors did their research and presented the research results in the article. Your critique can contain both positive and negative comments.

Justify and explain in detail each of your critique points in a separate paragraph of at least 4-5 sentences.

The following are suggestions only:

• Does it build upon the appropriate foundation?
• Did the authors choose the correct approach, and then execute it properly?
• Did the article meet its objectives?
• How confident are you in the article's results, and why?
• Are its ideas really new, or do the authors simply repackage old ideas and perhaps give them a new name?
• Do the authors discuss everything they promise in the article's introduction and outline?
• What are the article's shortcomings (faults) and limitations (boundaries)? Did it discuss all of the important aspects and issues in its domain (topic area)?
• In what way should the article have made a contribution, but then did not?
• Do the authors make appropriate comparisons to similar events, cases or occurrences?
• How complete and thorough a job did the authors do? Do the authors include an adequate discussion, analysis and conclusions? Did they justify everything adequately? Did they provide enough background information for the intended audience to understand it? For you to understand it?
• Were there adequate and appropriate examples and illustrations?

For full credit, ask yourself these questions when justifying your critique points:

• why/why not?
• how?
• what distinguishes the differences/different approaches, and in what ways?

Grading: four items up to 24 points total - for each item: great - 6; ok - 4; poor - 2

10. Issues

In this section you should discuss issues (open questions) raised explicitly or implicitly by the author. You also should discuss issues which remain unresolved in your opinion. Aim for 4 issues total: preferably 2 issues by the author and 2 issues in your opinion.

Note: If you have any critiques in this section, they most likely belong in the General Critique section instead.

Issues (mentioned by the author)

What open questions or issues has the author stated remain unresolved? Discuss each in a separate paragraph of 5-10 sentences. Each issue's paragraph should take the following format:

• what is the issue?
• why do you believe this is an important issue?
• in what way is it unresolved
• suggestions for resolving it - if you give your own suggestions (instead of or in addition to the authors', then precede each with "I would propose ..." If it has been resolved since the article was written, then state how it was resolved.

Issues (in your opinion)

Which open questions or issues remain unresolved in your opinion? For example, what problems remain? How could this work be extended? Is there a need to extend this work? What possible future research questions could arise from this article? Discuss each in a separate paragraph of 5-10 sentences. Each issue's paragraph should take the following format:

• what is the issue?
• why do you believe this is an important issue?
• in what way is it unresolved
• suggestions for resolving it

Grading: 4 items up to 16 points total - for each item: great - 4; ok - 2; poor - 1

11. Annotated Bibliography

For every outside item you have cited in your report, you need a full reference and an annotation explaining it. This includes references to any class materials, as well as the three additional articles, web sites or other materials.

1. List the full bibliographic references (authors, title, journal name, volume, issue, year, page numbers, etc.) for anything you have cited in your review. Use a format similar to that in the course schedule.
   IMPORTANT: This is NOT the bibliography listed at the end of the article. It is the bibliographic references for any readings you yourself referred to inside your review.
2. Write 2-4 sentences describing the article.
3. Write 2-3 sentences describing why you cited it.

Also, be sure that you have included a bibliographic marker to each (such as [Bieber & Smith, 2001]) in the text of your review.

Grading: -5 if missing references; -3 if you mention the authors explicitly in your text and put the references in this bibliography section, but forget to explicitly place citation markers in your text.