by John Carpinelli

Copyright 1998, John D. Carpinelli, All Rights Reserved

This report summarizes my experiences teaching CoE 353, Advanced Computer Architecture, in Distance Learning mode using Asynchronous Learning Networks (ALNs), during the Spring 1998 semester. During the same semester, I also taught a section of this course via face-to-face (FTF) classroom instruction. This report also presents a statistical analysis comparing the performance of the two sections. Analysis of homework and test scores indicates that students in the ALN section learned the course material as well as students in the FTF section. This analysis produced similar results for all students, only for students who completed the course, and only for regularly matriculating students.

This past semester, I taught a course via Distance Learning/Asynchronous Learning Networks for the first time. The course, CoE 353 - Advanced Computer Architecture, is one I had taught on many previous occasions, so I felt comfortable with the material. However, the mode of delivery was completely new to me and presented several challenges. This report describes my experiences with this course and the new mode of delivery.

During the same semester, I taught a face-to-face section of the same course. Given the opportunity to do so, I completed a comparative analysis of the two sections. The statistical and qualitative results of this analysis are also given in this report.

This course is a required course in the B.S. in Computer Engineering curriculum. Typically it is taken during the second semester of the junior year. The course has the following prerequisite topics, which are covered in previous courses in the curriculum.

For the FTF section, the course met twice a week for 1.5 hour sessions, comprising a total of 42 hours of classroom time for the semester. Students in the ALN section did not have set meeting times. Instead, they viewed videotapes of the lecture material and participated in class discussions on-line. The lectures were taped to an empty classroom during the 1997 summer and fall semesters and comprised approximately 17 hours of time. The material presented on the tapes was the same as that covered in the FTF class. However, due to the lack of classroom questions and some streamlining of the presentation, the tapes covered much less time. Slides were prepared for use during the taping sessions and, along with speaker notes, were made available for students in the ALN section to download via the World Wide Web. This option was not provided to students in the FTF section.

The textbook for this course is Computer Systems Architecture, by M. Morris Mano. The course covers the following topics. The number of weeks in the nominal semester allotted to each topic is given in parentheses.

There were five homework assignments in this course; the last was optional. The homework grades, combined with a class participation grade, accounted for 20% of the total grade. Two tests given during the semester each counted for 25% of the total grade and a comprehensive final exam comprised the remaining 30% of the grade.

The tests were given to the FTF section of the class during a regular class period. Students in the ALN section had an option as to how to take the tests. If they were available, they could take the tests during the class period along with the FTF section. Approximately half the students in the ALN section did this. A second option was for the students to take the test on campus during the evening with a proctor; this was meant for students who worked full-time jobs but were close enough to come to the Newark campus. Two students elected this option. The final option was for students to take the test off site with a proctor of their choosing, within reason. A supervisor or colleague often served as a proctor, though one student had his local librarian serve as the proctor. A couple of students chose this option.

To supplement interaction in the ALN section, NJIT's Virtual Classroomâ /Electronic Information Exchange System (VC/EIES) was used. Originally developed at NJIT, it is one of the oldest ALN systems in use today. It includes discussion groups, online assignment capabilities and an online gradebook. For this course, only the discussion group feature was used. Students in remote courses tend to lose the interaction with instructors and with other students inherent to the traditional FTF class. This feature of VC/EIES was used to minimize this loss of interaction. As a side benefit, the discussion group served as a log of previous discussions in the class which was readily available to all students enrolled in the course. This is one advantage that was not available to students in the FTF section of the course.

This section presents a statistical comparison of the FTF and ALN sections of the course. It first analyzes the performance on homework assignments, then on tests and the final exam, and finally for the entire course.

There were five homework assignments, equally weighted, for this course. The assignments included some problems from the textbook and some created by the instructor. The last assignment was optional; students who did not submit this assignment had it prorated in determining their homework average. Along with class participation, this constituted 20% of the course grade. Students in each section were given the same homework assignments.

Table 1 shows the average grade for each of the mandatory homework assignments. The column labeled ALL assigns a grade of zero to students who did not submit the assignment; the SUBMITTED column only includes students who submitted the assignment. For each case, the total number of students is given in parentheses next to the data.

As can be seen in this table, students in the FTF section achieved a higher average grade on every assignment except the fourth. However, when including only those students who submitted the assignment, the ALN section performed slightly better than the FTF section. Interestingly, though not statistically significant, the section which performed better on each assignment when all students were considered actually performed worse when only submitted assignments were considered.

Table 2 presents the same analysis, but only includes students who completed the course, i.e. did not withdraw from the course. As can be seen in the table, students in the FTF section who completed the course generally performed better than students in the ALN section who completed the course. However, among students who submitted the assignment, students in the ALN section again performed slightly better than their FTF counterparts.

Four students in the FTF section and one student in the ALN section took the course to transfer to their home university; they registered as non-matriculating students. All five of these students completed the course. Table 3 summarizes the performance of each section on the homework assignments, excluding these students, i.e. including only matriculating NJIT students. The results are similar to those of the previous cases.

From the data given, it is clear that students in the ALN section were more likely to not submit a homework assignment. However, students in the ALN section who submitted homework assignments performed comparably to their peers in the FTF section.

In addition to the homework assignments and class participation, students in this course took two tests, each counting 25% toward the final grade, and a comprehensive final exam which constitutes 30% of the final grade. Students in both sections took the same tests and final exam. Cronbach's alpha was calculated for test #1 as .80, for test #2 as .55 and for the final exam as .70.

Tables 4, 5 and 6 summarize the performance of students on the tests and the final exam for all students, students who completed the course, and matriculated students who completed the course. This data excludes students who missed an exam and took a different makeup exam. As shown in the table, students in the ALN section outperformed students in the FTF section on the two tests under all three conditions. However, students in the FTF section scored better on the final exam than students in the ALN section. Note that all students who took the second test, and of course the final exam, completed the course.

Table 7 summarizes overall course averages for all students and for all matriculating students. Overall, students in the ALN section performed better than students in the FTF section. However, if only matriculated students are considered, the two sections performed virtually the same. The FTF students significantly outperformed ALN students on the homework assignments (69.8% vs. 59%) and slightly outperformed them on the final exam (59.7% vs. 57.5%). Students in the ALN section outperformed students in the FTF section on test #1 (52.2% vs. 50.1%) and on test #2 (65.3% vs. 57.8%)

The final course grades for all students, and for all matriculating students, are shown in Table 8. In each section, approximately half of the students achieved a final grade above C. The non-matriculating students decreased this ratio for each section. Of note is that students in the FTF section had a larger percentage of final grades of both A's and F's.

The qualitative differences between the two sections are more difficult to assess than the quantitative performance of the students. For this section of the report, I rely primarily on my own judgment and perceptions.

The ALN section of this course offered several advantages over the FTF section. First and foremost is the flexibility of delivery of the material. Several of the students in the ALN section hold full time jobs and pursue their studies on a part-time basis. For them, ALN was the only way that they could take this course during the 1998 spring semester. This flexibility was also an advantage for on-campus students who, due to conflicts in their schedules, otherwise would not have been able to take the course during that semester.

A second advantage is the lack of necessity to be local to NJIT's Newark campus. Two students, one in upper Sussex County and one in Burlington County, were able to take the course via ALN delivery. If they had to come to the Newark campus, they could never have taken this course.

The FTF mode of delivery still offers several advantages over ALN delivery. Paradoxically, the first advantage is the lack of flexibility. Although flexibility is the foremost advantage for ALN students, many students need the structure of set meeting times in order to successfully complete the course. The withdrawal rate for students in the ALN section of this course was 38%, as opposed to 16% for the FTF section. My perception is that this is primarily due to students who failed to keep up with the course material.

A second advantage of FTF mode is the classroom interaction. ALNs help mitigate the lack of student-student and student-teacher interaction, but they do not completely make up for it. There seem to be some dynamics inherent to the classroom setting which aid in learning.

The first thing that strikes me about teaching a course via Distance Learning/ALN delivery is the large amount of time required, both before the course was offered and while it ran. Although I could have taped my lectures while delivering them in a FTF class, I chose to tape them to an empty classroom because I felt it would provide better control. Also, having spoken to colleagues who taped lectures while teaching, I felt that doing so would shortchange students being taught FTF while the lectures were being taped. During the course, I usually logged in to check on my class discussion group twice a day. Here, the flexibility of when I checked in was an advantage over the set meeting times of FTF instruction.

One thing that I have found to be very important is to get students active in the online discussion group at the beginning of the semester. I started by having the students post a brief introduction of themselves to the rest of the class. From there, class participation dropped off, so I included required online discussion questions starting around the middle of the semester. The next time I teach this course, I plan to introduce discussion questions every week to keep the students active in the discussion groups.

This report has presented the results of a comparative analysis between two sections of a junior-level computer architecture course. By assigning the same homework problems and giving the same tests and final exam to each section, a basis for comparison has been established. As shown by the course data, performance is comparable between students in the two sections; ALN has been shown to be a viable method for students to learn this material.

I enjoyed teaching a class using this mode of delivery. It was different than anything I had done since joining academia. Although it was much more work than a regular class, I would and will teach using this mode again.

This course was developed for ALN delivery with funding from the Alfred P. Sloan Foundation. Virtual Classroomâ is a registered trademark of the New Jersey Institute of Technology. The author thanks Michael Bieber of NJIT's CIS Department for providing a copy of his welcome letter for my use in preparing the welcome letter for this course. The welcome letter presented in Appendix II largely draws on Prof. Bieber's letter in format and content.

The week of March 17 is spring break week.

Homeworks are equally weighted and will be a combination of problems from the text and outside problems. Homework assignments and due dates will be announced in the class discussion group.

Quizzes are closed book and closed notes. One 8 1/2" x 11" sheet of notes may be used for each quiz; three sheets may be used for the final exam.

Welcome to ACCESS NJIT's course CoE 353, Advanced Computer Architecture. This course will be taught via a combination of the "Virtual Classroom" on EIES and video. This letter briefly describes the course components, how they will be used and what you must do to prepare for the course.

First of all, you must read through the material described in this letter now. Do not wait until the week before classes start. Some tasks, such as ordering tapes, have a lead time associated with them. If you wait too long before performing those tasks, it will cause delays which will make it more difficult to complete the work in this course. Completing these tasks now will make things easier for you in this course.

The description for this course can be found in the NJIT Undergraduate Catalog online at http://www.njit.edu/Catalog/CoE353.

This course has three primary prerequisites. First, each student must have passed a course in digital logic design (EE 251, CIS 251 or equivalent). Second, all students must have passed a course in computer organization, such as CoE 252. (Although titled Computer Organization, CIS 251 does not satisfy this requirement since it primarily covers digital logic design.) Finally, each student must have an understanding of the internal organization and assembly language of at least one microprocessor. The processor type is not critical; this course requires familiarity with the associated concepts, not the details for any specific microprocessor.

This course includes about 16 hours of videotaped lectures. This is far less than the time needed to cover the same material in a traditional classroom because it was taped in a studio with no class present. As such, the tapes are fairly dense in material and may require more than one viewing. You should order your tapes now so that they arrive before the beginning of the semester. To order tapes, contact HAVE, Inc. at 1-800-997-2972.

The textbook for this course is Computer Systems Architecture, 3rd edition, by M. Morris Mano. You should obtain this book before the beginning of the semester. You may purchase this book from the NJIT Bookstore, either in person or via mail order by calling 1-800-472-0080.

The note set for this course is available online. These notes are packaged in eight ZIP files, one for each module, and can be viewed using a standard web browser. Visit NJIT's Extended Learning Delivery web page at http://www.njit.edu/DL for information on how to download these files. Once downloaded, unzip them onto your local hard drive for viewing and/or printing.

The Virtual Classroom system on EIES will be used for class discussions. If you do not have an account on EIES, get one now. The web page at http://www.njit.edu/DL contains information on how to obtain an account. You must also have an alternate means of electronic communication, just in case EIES ever experiences a problem. You may obtain an account on NJIT's megahertz system or some other Internet-accessible system. See the web page listed earlier in this paragraph for information on obtaining an account on megahertz.

I strongly recommend that you become conversant with the VC system before the semester begins. If you have not used this system in the past, I suggest that you attend one of the VC/EIES orientation sessions. See http://www.njit.edu/DL for a schedule of these sessions.

I expect to make use of this system extensively throughout the semester for class discussions. You are expected to log in at least three times per week (on different days). When you log in is up to you.

There are several ways to contact me directly. First preference is via the course discussion group. Questions regarding material in this course should always be posted here for all to see, just as everyone would be able to hear a question asked in a traditional classroom. If you want to contact me privately (for example, with a question regarding how I graded one of your homework assignments), feel free to send me email at carpinelli@njit.edu. You may also call me at 973-596-3536 or stop by my office (ECEC 301) during office hours. My office hours are listed on the course syllabus; changes will be noted on my web page, http://megahertz.njit.edu/~carpinel.

This course consists of eight modules, each of which corresponds to 1-2 weeks of material in a traditionally taught course. The syllabus for this course may be accessed via the Extended Learning Services web page at http://www.njit.edu/DL.

This course has five homework assignments, two quizzes and a final exam. You may submit your homework solutions in person or via fax at 973-596-5680. (This is a departmental fax machine, so make sure you include a cover sheet or routing tag on the first page.) Since most of the assignments will require block or circuit diagrams, it is unlikely that you will be able to submit solutions via email. Homework assignments will be posted to the discussion group at least one week before the due date.

The two quizzes and final exam will be given at NJIT's Newark and Mt. Laurel campuses. The dates and times for the two quizzes are given in the course syllabus. The date and time of the final exam is set by the Registrar's office and will be posted in early April, but will definitely be scheduled during the period of May 7-14. If you cannot take either quiz at the Newark campus at its scheduled time, you must notify me before the end of the first week of class. You must notify me if you intend to take the quizzes at the Mt. Laurel campus by the end of the first week of class so that we may make the necessary arrangements.

The following checklist is given as a guide for your pre-course planning. You do not have to submit this checklist to me or the Extended Learning Services office.

Once I receive your initial email message, I'll add you to the class discussion group. At that point, you should access the discussion group and perform any actions listed in my messages to the class. (The first action item will be for you to post a message to the discussion group introducing yourself to the rest of the class.)