Technical Communication Program Assessment - Resources

Validity (is the Model Useful?)

We achieved validity is achieved by carefully defining our goals in the technical communication course and building an assessment model that is specific to those goals. According to the American Psychological Association standards, validity is delineating the knowledge, skills, abilities, processes, or characteristics to be assessed and developing a method of assessing them. In the past, validity in writing assessment has often been sacrificed to reliability (such as in the case of using multiple choice items to assess writing ability). Now, however, inter-reader reliability has become established in writing assessment and we can focus on the assessment’s underlying goals.

We analyzed our model by a multiple regression analysis. In the model we created, all of the components (criteria) were significantly related to the overall portfolio score (Table 2).

The regression analysis of fall 2004 relating the independent variables to the dependent variable (overall portfolio score) revealed a high coefficient of determination (r2 = .734, F(11, 49) = 12.287, p =.000). The regression analysis of spring 2005 was higher (r2 = .785, F(10, 39) = 14.274, p =.000) and that of the fall 2005 was even higher (r2 = .865, F(10, 113) = 72.181, p =.000). Such high coefficients of determination showed that all of our individual criteria acted as predictor variables for the overall portfolio score; this, we had devised a suitable assessment mechanism for our construct of technical communication. Table 2 shows the correlation results for the predictor variables, most of which reached a confidence level of .05 or .01 (95% or 99%).