2006

Analytic Scoring of Online Portfolios: Sustainable Assessment for the Technical Communication Service Course

Norbert Elliot, Robert Lynch, Carol Johnson, Robert Friedman, Nancy Coppola

This paper was part of a panel entitled "In Search of Meaning: A Community Model for Program Assessment" organized by Norbert Elliot. The panel discussed the Humanities Department’s nine year effort to assess the impact of humanities courses that annually serve 7, 400 students enrolled in undergraduate and graduate courses. The following is the proposal for the panel written by Norbert Elliot:

Recent work in writing assessment by Brian Huot (Re-Articulating Writing Assessment), Bob Broad (What We Really Value), Patricia Lynne (Coming to Terms), and Norbert Elliot (On a Scale) has stressed the important of meaningful assessment practices that are ethical responsible in impact and instructionally oriented in design. This session will present the work of one department’s decade-long efforts to achieve such community-oriented goals while maintaining an empirical orientation toward assessment. (See http://cwa.njit.edu/index.htm.) Under specific review will be the use of portfolios and their ability of yield greater construct representation of both academic and technical writing ability. Influenced by the work of Alan C. Purves (The Web of Text and the Web of God An Essay on the Third Information Transformation) and Nancy J. Hirschmann (Re-Thinking Obligation: A Feminist Method for Political Theory), each presenter will attend to the community-building aspects of assessment.

Presenter 1 will establish the design of the undergraduate assessment model. Variables assessed through analytic scoring methods will be reviewed, and data will be provided from the following undergraduate cohort groups: freshman composition, sophomore cultural history, and senior capstone seminars. Presenter 1 will also focus on the traditional tensions between reliability and validity as they resolved by sampling plan design.

Specific identification of the variables of written communication will be provided in a case study format by Presenter 2. Here, outcomes data will be presented from a junior-level technical communication course, an undergraduate offering taken by incoming transfer students who enroll at our university in numbers equal to first-time, full-time freshmen. Presenter 2 will pay specific attention to the use of on-line portfolios, those promising digital representations of palimpsest identified by Kathleen Blake Yancey (CCC 55.4 [2004]: 738-761).

Presenter 3 will review data on the department’s asynchronous courses offered in an e-learning format. Here, retention issues will be examined in a six-year study of humanities courses compared to distance-learning offerings in the university’s department of computer science. Presenter 3 will provocatively interpret the data using Cleanth Brooks’ classic concept that isolate structure is, in fact, a negation of the empirical (normative) world (“Irony as a Principle of Structure” [1951]).

E-learning at the graduate level will be the focus of Presenter 4’s review of an assessment model for graduate programs that is presently under consideration for national adoption by the Council for Programs in Technical and Scientific Communication. Specifically, Speaker 4 will discuss assessment data from the fusion of imagery and language within a two-semester required course sequence informed, theoretically, by Kristie S. Fleckenstein’s understanding of polymorphic praxis (CE 66.6 [2004]: 612-631) and, pedagogically, by Lee Odell and Susan M. Katz’s advice in Writing in a Visual Age that, although writers still write, they must find ways to use visual information as part of their messages. Those attending the session will be presented with a humanistic frame of reference that advances a unified, community-based vision of assessment dedicated to student achievement.