Papers

Undergraduate Technical Writing Assessment: A Model (with Norbert Elliot) - View Paper

Programmatic Perspectives: Journal of the Council for Programs in Technical and Scientific Communication [forthcoming Sept. 2010]

This paper describes an assessment process developed for an undergraduate technical writing course at a public research university. To document program outcomes, we used a variety of statistical methods. To describe our process, we present longitudinal performance data collected over five years (fall 2004 through spring 2009) on 632 students. After providing a brief overview of the measurement concepts and statistical tools that we employ, we describe our process in five phases: designing the variable model to ensure construct validation; designing the assessment methodology to ensure content validation; designing the sampling plan to address economic constraint; designing the data analysis to articulate the validation argument; and using the assessment results to ensure consequential validation. Our intention is to provide a model that can be applied to other institutional sites and to encourage others to use it, tailoring the model to their unique needs.

A Decade of Research: Assessing Change in the Technical Communication Classroom Using Online Portfolios - View Paper

Journal of Technical Writing and Communication, 2006, (36:4), pp. 413-431.

Over a period of ten years, we have developed a sustainable process of online portfolio assessment that demonstrates both reliability and validity, using both qualitative and quantitative measures. The sustainable cycle is that, each semester, we assess a random sampling of the students’ work that they have posted, as per our instructions, in an online portfolio. During the reading, faculty score the documents for twelve variables, including writing, content, audience awareness and document design. We achieved validity by a modified online Delphi that led to a redefinition of the construct of technical communication itself and we achieved reliability by adjudication resulting in adjacent scores. The results of our assessment meet the requirements of ABET and result in continual cycle of improvement for our technical communication curriculum. It has also resulted in an improving correlation between the course grade and the separate overall portfolio score – we are beginning to grade the variables that we have defined.

The Analytic Assessment of Online Portfolios in Undergraduate Technical Communication: A Model - View Paper

Journal of Engineering Education, October 2006, (95:4), pp. 279-287.

This paper describes an innovative model for assessing the technical communication course by analytically scoring online portfolios, open to the internet, for ten separate (analytic) variables and one overall (holistic) score. The model is a statistically verifiable and sustainable method that strengthens the curriculum and fosters consensus within the teaching community. We achieved construct validity by redefining the elements of the course to incorporate communication in the digital age and then by creating new criteria for evaluation related to that construct. We achieved inter-reader reliability by beginning each assessment with a calibrated reading and by adjudicating non-adjacent scores. After using the model successfully for three semesters, we can see increased consistency in teaching among sections and semesters, more communication among instructors and we are beginning a database with which we can test further change. The theory and method behind this model can be applied to other disciplines as well.

History of New York State Regents Exams - View Paper

Posted on ERIC, 2009

This paper is a brief history of the Regents subject-matter examinations and New York State’s efforts to move towards educational equity. New York State was a leader in integrated curriculum and outcomes assessment in high schools for over a century. The first academic exit exam was administered in 1878 and it evolved into the controversial Regents subject matter exams, a cycle of curriculum building and assessment, run by the state bureaucracy, using the expertise of selected teachers. In the twentieth century, two separate tracks of academic achievement developed: students could earn a Regents Diploma or a Local Diploma. Late in the century, increased reporting revealed a gap in funding and achievement between rural/suburban and city schools. The state is presently attempting to address this problem to provide universal access to a high-quality academic education for all, but the results are unknown.

Impediments to Increasing Diversity in Post-secondary Education - View Paper

Originally published in proceedings of Spring 2007 ASEE Mid-Atlantic Section, posted on ERIC, 2010

Due to the increasing complexity in the financial aid process and the movement of available financial aid up the economic scale, poor people and minorities have less access to college, including engineering programs. Some impediments are lack of access to knowledge about college, increasing complexity and up-front costs in the application process and a gradual legislative and judicial change to provide aid to wealthier families. The latter changes include the weakening of the Pell Grant, increasing reliance on student loans, loans granted regardless of need, tax breaks for college savings, “merit” based aid, early-admissions politics and the anti-trust decision that resulted in a lack of consistency in creating financial-aid formulas. A rededication to need-based financial aid would increase the number of minorities attending and graduating from college.

For more information see:

• Technical Communication Online Portfolio Assessment Website (2009)