Reprints of Invited papers for:

1995 International Conference on Computer Assisted Instruction
ICCAI'95

March 7-10, 1995
National Chiao Tung University
Hsinchu, Taiwan

Department of Computer and Information Science
New Jersey Institute of Technology
Newark NJ, 07102
Roxanne@eies.njit.edu

TEACHING IN A VIRTUAL CLASSROOM [TM]

Starr Roxanne Hiltz
Department of Computer and Information Science
New Jersey Institute of Technology, Newark NJ 07102 US
Roxanne@eies.njit.edu

ABSTRACT:

The Virtual Classroom[TM] is a teaching and learning environment constructed in software, which supports collaborative learning among students who participate at times and places of their choosing, through computer networks. This paper describes experiences and results of utilizing Computer Mediated Communications structured to create a Virtual Classroom (VC). Emphasis is placed on guidelines for using the technology successfully by incorporating "collaborative learning" activities.

1 INTRODUCTION

Computer-Mediated Communication systems, especially when enhanced to create what we refer to as a Virtual Classroom,[TM] can make significant improvements in both access to and the quality of education. Currently over 80 programs worldwide are known to be offering courses partially or completely via CMC (See, for example, Harasim, 1990; Harasim, Hiltz, Teles & Turoff, 1995; Hiltz, 1986, 1994; Mason & Kaye, 1989; Paulsen & Rekkedal, 1990; Wells, 1990). The sophistication and flexibility of software structures for supporting distance education vary widely, from simple electronic mail systems to conferencing systems that have been specially enhanced to support classroom-like experiences, particularly group discussions and joint projects. This paper focuses on what we have learned about teaching techniques that are effective online as a result of a decade of experiences at New Jersey Institute of Technology (NJIT). However, most of the themes emphasized are echoed in reports on similar projects.

Generically, the Virtual Classroom is a teaching and learning environment located within a computer-mediated communication system. Rather than being built of steel and concrete, it consists of a set of group communication and work "spaces" and facilities that are constructed in software. Thus it is a "virtual" facility for interaction among the members of a class, rather than a physical space. Specifically, the Virtual Classroom[TM ]is NJIT's trademarked name for a version of its Electronic Information Exchange System (EIES2) with special software structures designed to support collaborative learning. Participation is generally asynchronous; that is, the Virtual Classroom participants may dial in at any time around the clock, and from any location in the world accessible by a reliable telephone system. The fact that the educational process is asynchronous means each student may engage in more reflective thinking before having to answer or discuss issues.

My collaborator, Murray Turoff (1995) is describing desirable features for a Virtual Classroom[TM] in a separate paper for this conference. For the purposes of this paper, let us simply assume that the minimally acceptable technological foundation is a conferencing system that allows the instructor to set up different conferences for different purposes; and also a "reply" or hierarchical structure for a conference, so that responses to root items become attached to them, and the conference "self-organizes" by topic, rather than being a simply a linear time-ordered sequence of items.

It should also be noted that a Virtual Classroom type of environment can be used successfully in many different media mixes:

For example, in a current project, we are delivering an entire undergraduate degree in Information Systems, via Virtual Classroom plus video. Between one and two hours of lecture are made available each week by videotape and/or broadcast; and the discussion and homework portions of the course take place online.

Another interesting observation that should serve as preparation for the results and conclusions to be reported below, is that one cannot segregate the "on-campus" and "distance" markets. If students who take most of their courses on campus are permitted to, they will choose to take a significant portion of their courses via a "distance" mode such as Virtual Classroom. This is because they experience scheduling conflicts with other courses, their jobs, or their family obligations, which mean that they either must take a "distance" course, or take longer to complete their degrees.

2. EXPERIENCES AND RESULTS

Our first extensive experimentation with the Virtual Classroom began in 1986 and involved the comparison of a large number of courses over a period of two years in many different disciplines which used this medium of communication, including undergraduate courses in Sociology, Communication, English Composition, Management, Computer Science, and Statistics. For some of the courses, there was a "matched" section of the same course offered by the same instructor in a traditional classroom and using the Virtual Classroom (as the sole means of delivery, or in combination with a reduced number of face-to-face meetings). For other courses, there was no "match," and the comparison was subjectively made by the students and instructors to previous, traditional courses.

For this study and the projects which have followed, we used a "multi-method" approach to evaluation. This includes pre and post-course questionnaires completed by students, direct observation of online activities, interviews with selected students, comparison of test or course grades or other "objective" measures of performance, and regular reports by faculty, which follow a common outline.

Despite a far-from-perfect implementation, the results of the first extensive field trial were generally positive, in terms of supporting the conclusion that the Virtual Classroom mode of delivery can increase access to and the effectiveness of college-level education. Among the hypotheses and findings for this field trial and for a subsequent field experiment involving 14 sections of a management course taught via a variety of modes are the following (see Hiltz, 1994 for more details):

  • H1: Mastery of course material in the Virtual Classroom will be equal or superior to that in the traditional classroom.
  • Finding:

    Supported. For an initial quasi-experimental design comparing five matched courses, VC final grades were significantly better for one course (Computer Science) and no different for the other courses. In a subsequent study of a management course (Hsu, ), students in a mixed video/VC section scored significantly better on a common final examination than students in traditional sections.
  • H2: VC students will report higher subjective satisfaction with the VC than the TC on a number of dimensions.
  • Findings:

  • 2.1 Convenient access to educational experiences (supported).
  • 2.2 Improved access to their professor (supported).
  • 2.3 Increased participation in a course (supported).
  • 2.4 Improved ability to apply the material of the course in new contexts and express their own independent ideas relating to the material.
  • Finding:

    Questionnaire data showed that increased confidence in expressing ideas was most likely to occur in the mixed modes courses. Qualitative judgments by faculty are that this is generally characteristic of VC courses.
  • 2.5 Increased level of interest and involvement in the subject matter, which may carry beyond the end of the course.
  • Finding:

    This was course dependent. Though the averages for measures of increased interest are higher for both the VC and Mixed modes, the overall scores are not significantly different.
  • 2.6 Improved ability to synthesize or "see connection among diverse ideas and information."
  • Finding: No significant differences overall in scores for items which formed this index from the postcourse questionnaire; mode interacts with course. However, instructors report observing improved ability of students to synthesize diverse information and ideas and deal with complex issues.
  • 2.7 Computer comfort- improved attitudes toward the use of computers and greater knowledge of the use of computers (supported).
  • 2.8 Improved overall quality, whereby the student assesses the experience as being "better" than the TC in some way, involving learning more on the whole or getting more out of the course (supported).
  • H3: Those students who experience "group learning" in the virtual classroom are most likely to judge the outcomes of online courses to be superior to the outcomes of traditional courses.
  • Finding:

    Supported by both correlational analysis of survey data and qualitative data from individual interviews. Those students who experienced high levels of communication with other students and with their professor (who participated in a "group learning" approach to their coursework) were most likely to judge the outcomes of VC courses to be superior to those of traditionally delivered courses.
  • H4: High ability students will report more positive outcomes than low ability students.
  • Finding:

    Supported for Math SAT scores. Results for Verbal SAT scores much more mixed and inconsistent. However, even students with the lowest SAT scores were able to achieve passing grades, if they participated regularly.
  • H5: Students with more positive precourse attitudes towards computers in general and towards the specific system to be used will be more likely to participate actively online and to perceive greater benefits from the VC mode (supported).
  • Although the "average" results supported most of the above predictions, there was a great deal of variation, particularly among courses. Generally, whether or not the above outcomes occurred was dependent more on variations among courses than on variations among modes of delivery. Instructors differed a great deal in terms of the amount and effectiveness of the efforts they put into organizing and conducting their online classes. Observations about what made some courses much more successful than others, will be presented in the following section.

    We are currently half way through a three year project to produce and offer an entire degree program, the BA in Information Systems, via video plus Virtual Classroom[TM]. The first year evaluations by students and faculty are available, though they must be considered very preliminary, since they were based on only the initial offerings of a few of the courses. However, the results are encouraging. For example, Table 1 indicates that the self-pacing available through this mode, may make difficult technical material seem less difficult for many students. And Table 2 continues the trend of positive subjective evaluations by students.

    Table 1: PERCEIVED LEVEL/DIFFICULTY OF COURSE, BY MODE OF DELIVERY

    Mode          Very Easy    Easy   Just Right   Difficult      Very         Total    
                                                                Difficult               
    VC+Vid.           0        11%        44%         37%          7%          N= 54    
                                                                               100%     
    VC+FtF            0         6%        35%         57%          2%          N= 51    
                                                                               100%     
    Traditional      1%         4%        23%         49%          23%         N= 70    
                                                                               100%     
    All              1%       7%          33%         47%          12%        N= 175    
    
    Chi Square = 23.4, p= <.01

    Question: How appropriate is the level of this course? (check one) Very easy, easy, etc.

    Table 2: OVERALL COURSE RATING, BY MODE OF DELIVERY

    Mode         Excellent    Very Good    Good   Fair     Poor    Total    
    VC+Vid.         22%          44%       18%     13%      2%     N= 54    
                                                                    100%    
    VC+FtF          12%          41%       35%     10%      2%      N=51    
                                                                    100%    
    Traditional     13%          40%       24%     14%      9%     N= 70    
                                                                    100%    
    All             15%          42%       26%    13%       5%     N= 175   
                                                                    100%    
    
    Chi Square = 10.03 p= .045

    Question: How would you rate this course over-all?

    2. GUIDELINES FOR THE ONLINE INSTRUCTOR

    This section looks at the Virtual Classroom from the teacher's point of view. Its purpose is to present the lessons learned by the faculty members who have participated in the project, particularly in terms of guidelines or generalizations that may be of use to future instructors who are considering using computer-mediated communication as a mode of educational delivery.

    Just as there is no single recipe for successful teaching in the traditional classroom, there are diverse techniques which can be successful in the computer-mediated environment. Whatever strategy is initially chosen will have to be modified on the basis of observations about the level of ability and motivation of the students; this is true in any medium. However, there are some fundamental differences in teaching between the VC and the TC, stemming from the differences between the communication channels.

    There are four basic principles to keep in mind for successful teaching in the Virtual Classroom, dealing with media richness, timely responsiveness, organization, and interaction:

    2.1 Media richness:

    In the traditional classroom, a pleasing voice, occasional jokes, dramatic gestures, and interspersed questions can help to enliven a long lecture. In CMC, there is only the screen and/or the printed pages. Even if the work station supports color graphics and sound, long segments of lecture-type materials are boring. To maintain interest, the instructor should use written language in a skillful way (including the use of humor and metaphor), orchestrate active participation by the students, and stimulate collaborative assignments that involve both social and task-oriented activities. Do not try to deliver a long "lecture" in written format. Instead, deliver small segments accompanied with opportunities for participation, and use print or pre-recorded materials for purely "lecture" type materials.

    2.2 Timely responsiveness:

    Unlike the face-to-face medium, students will not receive an immediate response to their questions and comments. This can be very frustrating, especially if they are "stuck" in the middle of a mathematics problem or other assignment. However, whereas they may have only a limited opportunity to ask questions during a few set hours a week for a traditional class, they can participate and receive feedback on a daily basis in the Virtual Classroom. But this depends on the instructor, who needs to sign on every day except for occasional announced exceptions to this policy, and respond to all waiting material. Most faculty report that this takes them about 30 minutes to an hour a day, depending on the number of students and the level of their activity. The conscientious instructor becomes a "perpetual professor"; teaching is continuous, like parenthood, rather than being confined to a few specific hours during the week.

    2.3 Organization:

    With 20 to 30 students each expected to enter at least two comments a week, plus material entered by the instructor, a typical course has at least 100 new entries a week. Unless this mass of material is organized, students will become very confused. The instructor must establish regular rhythms and schedules, based on dividing the course into modules which last a week, a week or a half, or two weeks each, so that the participants can plan ahead in terms of when they will need to sign online and when work will be due, and so that the group moves through the topics in an orderly manner.

    One basic strategy to segregate and organized different modules and activities is to use several conferences for different types of activities, and to have the class move from one to another as they progress through topics. A second strategy is for the instructor to enter the stimulus materials for each week's work on a regular basis, with new material predictably appearing at least twice a week. For example, online "electures" are regularly posted for one of my courses each Tuesday, and assignments are posted on Thursdays (due nine days later, to give students plenty of time to plan their work ahead). This generally encourages students to sign on at least three times a week; once to read the electure and respond; once to receive the new assignment to be able to prepare to complete it; and once to upload their completed assignment. During each of these sessions, hopefully they will read and respond to others. In posting assignments, deadlines must be set and penalties for lateness enforced, lest students fall behind and confuse the class discussion by talking about an assignment weeks after it is due. The medium is self-paced in the sense that each student can read and write at their own pace; however, the class has to move through the modules together, in order for interaction to be meaningful.

    2.4 Interaction:

    Probably the greatest determinant of the extent to which students feel that the online mode of delivery is better or worse than traditional modes is the amount and quality of interaction between the instructor and the students, and/or among the students. This is not always easy, but if you can cajole or coerce the students into this collaborative approach to learning, they will share ideas with each other in a way that is seldom or never seen in a traditional classroom. Herein lies both the key and the challenge for being an effective teacher in the Virtual Classroom environment. An expansion of this theme and examples of "collaborative learning" approaches follows.

    3. COLLABORATIVE LEARNING

    Collaborative learning is defined as a learning process that emphasizes group or cooperative efforts among faculty and students. It stresses active participation and interaction on the part of both students and instructors. Knowledge is viewed as a social construct, and therefore the educational process is facilitated by social interaction in an environment that facilitates peer interaction, evaluation and cooperation (Bouton & Garth, 1983; Bruffee, 1986; Johnson, 1981; Johnson & Johnson, 1975; Whipple, 1987). The "teacher" becomes primarily a facilitator who structures learning opportunities, serves as a resource, and encourages the students to work together to build a common body of knowledge.

    The Virtual Classroom is an environment that facilitates collaborative learning -- among students, between students and instructors, among teachers, and between a class and wider academic and nonacademic communities. It also supports independent learning and generative, active learning techniques that are self-paced by each participant. For distance education students, the increased ability to be in constant communication with other learners is obvious. But even for campus-based courses, the technology provides a means for a rich, collaborative learning environment that exceeds the traditional classroom in its ability to "connect" students and course materials on a round-the-clock basis.

    An example of a collaborative learning strategy applied in the VC that is included in most courses is the "seminar" type of interchange in which the students become the teachers. Individuals or small groups of students are responsible for making a selection of a topic (usually from a list provided by the instructor as a Selection Activity); reading material not assigned to the rest of the class; preparing a written summary for the class of the most important ideas in the material; and leading a discussion on the topic or material for which they are responsible.

    Seminar-style presentations and discussions are thus an example of a collaborative learning activity that is often difficult in the Traditional Classroom (TC), but which tends to work very well in the Virtual Classroom environment, even with fairly large classes of undergraduates. Other examples of collaborative learning strategy in the VC include debates, group projects, case study discussions, simulation and role-playing exercises, sharing of solutions to homework problems and/or answers to review questions for exams; and collaborative composition of essays, stories, or research plans.

    For some very technical courses, it is hard to think of how to apply the most interesting of the collaborative learning structures, such as debates and role-playing games. However, no matter how technical the course, there is at least one type of collaborative assignment that can be used, and repeated several times during the course. Each student is responsible for making up one or two questions that test mastery of material in a set of readings and videos or other materials in a course, that would be suitable for an examination. As a homework assignment, students are asked to answer at least one question suggested by other students, in addition to making up their questions. If at all appropriate, students might also be responsible for "grading" or "correcting" and commenting on answers to their question, showing the nature of a correct solution and pointing out how the problem tests a particular concept or technique or skill. The students are graded on the quality and quantity of their questions and their responses. Then, the actual examinations should include some items chosen from those created by the students. This is practically guaranteed as a strategy for involving students in thinking about what is most important to know about material in a unit, and in interacting about the knowledge content of the unit. For American students at least, simply encouraging participation as an optional activity will not work; they put their efforts where the grading points are.

    It is important to establish collaborative learning through substantive contributions by students to the class discussion from the very beginning of a course. However, with distance students, there are often problems in obtaining the books, videotapes, or other materials by the first week of the course. (Students are told to order the materials ahead of time, but they often do not get around to it until the day before the course is supposed to start!) Therefore, the first week's assignment, in particular, should draw upon the students' own experiences and general knowledge, rather than requiring the reading and synthesis of specific assigned materials.

    Figure 1: Example of a Collaborative First Assignment


    C 120.1 CC 1.4 Roxanne Hiltz (Roxanne, 120) 1/11/95 12:19 PM 58 lines

    Subject: assignment 1/response activity/negative experiences.

    Why do many people "hate" computers? Most of you probably consider this irrational, but in fact people often have negative experiences with computers. If we are to build computers and software which people "love," we need to understand some of these negative experiences, and what can be done to prevent them.

    PLEASE ENTER A BRIEF DESCRIPTION OF A NEGATIVE EXPERIENCE WHICH YOU HAVE HAD WITH COMPUTERS, OR WHICH YOU HAVE HEARD ABOUT OR READ ABOUT.

    Try to make this as "important" a problem as you can think of having encountered... e.g., a few people have actually been KILLED by faulty computer systems, and that is certainly an important negative experience for us to learn from! If you can, try to analyze or explain the reason why this negative impact occurred. (e.g., hardware failure, inadequate software design, user failure to follow recommended procedures...)

    This is being entered as a "special" kind of comment. It has an activity attached. This is a question/response activity. When you are ready to enter your response to this homework question, "do" this activity. After you have entered your response, you may view the responses of others. After doing the activity (entering your response), you can make additional responses, either addendum's to your initial reply, or as replies to other people's contributions. At the end, we will try to build a composite list of the types of reasons why people have negative experiences with computers.

    Most assignments will be put in "activities". A separate list is kept by the system of which ones you have done and which you have not done.

    The expected length of response is about 30-50 lines (50 lines is two screens full).


    End of Figure

    Figure 1 shows an example of a first "collaborative learning" assignment for a course in Computers and Society required of all Computer Science majors. Before this course and this assignment, majors are unlikely to have thought of computers as sometimes causing problems for their users or negative consequences for the society. By remembering, finding, and sharing many specific examples, this new way of thinking becomes grounded by the class discussion.

    Note that the instructions given for this first assignment are very explicit. Because students cannot ask you a question about the instructions and receive an immediate reply, it is important to make everything as clear and explicit as possible, so there will be no misunderstandings about what is being requested. In fact, let us add this to our list of teaching hints: when in doubt, err on the side of extremely detailed, step-by-step instructions, rather on the side of vagueness. Students who are new to the online environment are unsure about how they are supposed to behave, and need very clear guidelines about what is expected of them.

    4. CONCLUSIONS

    A Virtual Classroom is not the proper mode for all faculty or all students. Faculty must feel comfortable with computers and with writing, and must genuinely enjoy spirited interaction and "expecting the unexpected" to happen. (No two sections of an online course are ever the same, and one is never quite sure what a group of students will do). It does not work to have very small classes; a minimum of about ten active participants seems to be necessary in order to establish and maintain a lively interchange. In addition, though on the average students like this medium better than traditional courses, some drop out or dislike it very strongly. In order to succeed, students must have convenient access to a computer and modem, reasonable reading and writing skills, and the time, motivation, and self-discipline to participate regularly, every week.

    In many cases, results of the quantitative analyses are inconclusive in determining which is "better," the traditional classroom (TC) or modes employing the VC. The overall answer is, "it depends." Results are superior in the VC for well-motivated and well-prepared students who have adequate access to the necessary equipment and who take advantage of the opportunities provided for increased interaction with their professor and with other students, and for active participation in a course. Students lacking the necessary basic skills and self-discipline may do better in a traditionally delivered course. Whether or not the VC mode is "better" also depends crucially on the extent to which the instructor is able to build and sustain a cooperative, collaborative learning group; it takes new types of skills to teach in this new way .

    In many ways, teaching a course online is merely a variation of moderating any computer conference. As in any computerized conference, the outcomes are dependent upon both the skills and hard work of the moderator, and the skills and level of motivation of the members of the conference.

    We have observed in all studies that there are significant differences among courses in grade distributions, and in all other outcome measures. Underlying these differences among courses are differences in the number and types of online activities required or facilitated by the instructor, and in the frequency and style of online interaction between the instructor and the students. Probably the single most important behavioral practice which produces relatively good results in online courses is the timely and "personal" (in tone) response by instructors to questions and contributions of students online.

    This does pose a problem for the economics of online teaching. Currently, it is not possible for an instructor to handle more than about 30 students in an online section, because of the daily attention that each student requires. It has been suggested that larger class enrollments could be handled by delegating much of this personal attention to paid student teaching assistants. However, thus far, I personally have not been able to find students who have the knowledge of the subject matter, the mastery of the system, and the motivation to take a job like this and perform it satisfactorily. Most students who are qualified, already have paid positions that are higher paid and more secure than that of a teaching assistant.

    We have seen that a Virtual Classroom is a teaching and learning environment located within a computer-mediated communication system. As a teaching environment, it provides a set of tools, strengths, and limitations which are available to an instructor for delivering course materials and structuring learning experiences. Its characteristics are merely potentials, just as the empty classroom with its chalkboards and desks awaits the efforts and creativity of the instructor and the students to make it "come alive."

    Acknowledgments

    The Virtual Classroom project has received substantial support from the Annenberg/CPB Project, the Department of Higher Education of the state of New Jersey, and several industrial partners, including IBM, Hewlett Packard, and Apple Computer. Development and research on the B.A.I.S. degree via a combination of video and VC is supported by the Alfred P. Sloan Foundation.

    References

    Bouton, C., & Garth, R. Y. (1983). Learning in Groups. (New Directions in Teaching and Learning, No. 14). San Francisco: Jossey-Bass.

    Bruffee, K. A. (1984). Background and history to collaborative learning in American colleges. College English, 46 (7), 635-652.

    Harasim, L. Ed. (1990). Online education: Perspectives on a new medium. New York: Prager/Greenwood.

    Harasim, L., Hiltz, S. R., Teles, L., & Turoff, M. (1994). Learning Networks: A Field Guide. Cambridge MA: MIT Press (in press), 1995.

    Hiltz, S. R. (1986). The virtual classroom: Using computer-mediated communication for university teaching. Journal of Communication, 36 (2), 95-104.

    Hiltz, S.R. (1993). Correlates of Learning in a Virtual Classroom. Int. J. Man-Machine Studies, 39, 71-98.

    Hiltz, S. R. (1994). The Virtual Classroom: Learning Without Limits via Computer Networks. Norwood NJ: Ablex Publishing Corp., Human-computer Interaction Series.

    Hiltz, S.R. and Turoff, M. Virtual Classroom plus Video: Technology for Educational Excellence. Proceedings, Educational Multimedia and Hypermedia 1994, 26- 37.

    Hsu, E. (1991). Management games for management education: A case study. Unpublished doctoral dissertation. Graduate School of Business, Rutgers University, Newark NJ.

    Johnson, David W. (January, 1981). Student-student interaction: The neglected variable in education. Educational Research (pp. 5-10).

    Johnson, D. W., & Johnson, R. T. (1975). Learning together and alone: Cooperation, competition, and individualization. Englewood Cliffs, NJ: Prentice Hall.

    Paulsen, M.F., & Rekkedal, T. (1990). The Electronic College: Selected Articles from the EKKO Project. Bekkestua, Norway: NKI Forlaget.

    Mason, R., & Kaye, A. (1989). Mindweave: Communication, computers and distance education. Oxford: Pergamon Press.

    Turoff, Murray. Designing a Virtual Classroom. Proceedings, ICCAI '95, Taiwan, March 1995.

    Wells, R. A. (1990). Computer-mediated communications for distance learning and training. Boise, ID: Boise State University.

    Whipple, W. R. (1987). Collaborative learning: Recognizing it when we see it. Bulletin of the American Association for Higher Education, 40, (2), 3-7.


    Virtual Classroom is a trade mark of the New Jersey Institute of Technology.