Distinguished Professor of Computer and Information Science
Department of Computer and Information Science
New Jersey Institute of Technology
Newark NJ, 07102, USA
The material in this paper was utilized for an Invited Keynote Presentation
at the UNESCO / OPEN UNIVERSITY International Colloquium, April 27-29:
Virtual Learning Environments and the Role of the Teacher, Open University,
Milton Keynes. It also forms the basis of a planned talk at the Third International
ALN (Asynchronous Learning Networks) meeting in NY City, October, 1997
There are forces at work that are going to reshape the practice of distance
learning and higher education in the United States. Technology only enters
as an opportunity to channel these forces in very different directions.
The channeling process is really that of administrative and management
practices and policies that govern the utilization of educational technology
and methods. While there are desirable futures possible it is becoming
evident that many current practices and related economic forces can result
in a future that is quite analogous to the "darkside" of the force.
The views expressed in this paper are solely those of the author and
do not necessarily reflect the official views of any organization with
whom the author may have an affiliation.
Before taking up the objective of this paper I wish to first make it clear
that I, and the others with whom I have worked with in this field, have
long been an advocate of utilizing the computer for facilitating the communication
process among humans and in groups (Turoff, 1970, 1972). Our view of this
technology has always been as a force for positive change and improvement
in the field of group communications and for learning as an important component
(Hiltz & Turoff, 1978, 1993).
Since we began the operation of the first version of EIES (Electronic Information
Exchange System) in 1976 at NJIT, we used Computer Mediated Communications
to augment our regular classes by allowing a stored class discussion to
continue in an asynchronous manner throughout the semester. It did not
take us long to arrive at the conclusion that this greatly facilitated
the ability of the instructor and the students to engage in continuous
and in-depth discussions about the semantic and pragmatic content of the
material. The potential for both education and training at all levels became
obvious (Turoff, 1982, 1986; Turoff & Hiltz, 1977, 1980). This quickly
led us to a number of hypotheses on the potential benefits and to seeking
funds to examine the consequences of this technology on the educational
Our Annenberg/CPB project in the 1980's was perhaps the only major quasi
controlled experiment that compared the use of asynchronous conferencing
to deliver courses with matched standard face-to-face courses across a
wide range of subject disciplines (Hiltz, 1994). These comparative matched
field trials were carried out with regular on campus students and not with
distance students. Therefore we were dealing with students who were comparing
this new experience to regular face-to-face classes and not to correspondence
courses. They showed the that the Virtual Classroom alone was just as effective
for learning as the regular face to face course and also had a number of
significant auxiliary benefits. In at least one subject area the students
did statistically significantly better in grades. Currently we have added
both video tapes of lectures to accompany the use of Virtual Classroom
and the availability of WEB based materials to the two undergraduate degree
programs (bachelor degrees in Computer Science and in Information Systems)
that we offer in a distance mode. Now at NJIT we are adding a growing number
of graduate courses and undergraduate bridge courses for Master degree
In our personal view, as educators, the ability of the computer to structure
the communication procedures and protocols around the educational application
and the nature of the learners was leading us to a quantum improvement
in the educational process. Improving the ability to hold direct communications
with a much greater number of the class members and to encourage the equality
of student discussion as well as the introduction of collaborative learning
methods provides results that could not objectively be measured in the
traditional exam and grading approaches.
In the case of the distance student one significant benefit of the technology
was the ability to allow us to merge our on campus students with our distance
students into one asynchronous conferencing based class. The distance student
received videos of our face to face lectures and everything else was now
equal for both the distance and the on campus student. The distance and
the on campus students were both part of the same class discussion, did
the same assignments in the same time frames, and could work on the same
Distance students and on campus students were integrated in a single
Virtual Classroom and treated as one class.
The vision that many of us have had for this technology is clear in the
literature (Hiltz, 1992,: Harrisam, et. al. 1995). What has happened, however,
is the classic problem of technology transfer. Now that the use of asynchronous
communications for education and training has been popularized into the
rapidly growing metaphor of Asynchronous Learning Networks (ALN), many
of us have become more than a little upset at some of the misconceptions
and misapplications beginning to occur with respect to applying computer
networking to higher education.
Historically it is no comfort that this is the process that often occurs
when new ideas based upon visions become popularized and introduced with
a limited understanding of the their roots (Turoff, 1989). Furthermore
the administrative practices, management, and policies currently associated
with distance education (in Higher Education) are encouraging exactly the
wrong approach to utilize the technology to improve learning and increase
the quality of the educational delivery process. It is quite clear historically
in the field of Management Information Systems that new information and
computer system technologies can be misdirected if the underlying human
organizations and customs (i.e., culture) are not adapted to make the best
use of the technology. History now seems to be repeating itself in the
area of higher education.
This paper addresses the underlying administrative, management, and policy
issues that affect the introduction and utilization of ALN and emphasizes
how they can potentially bring about surprising negative impacts on the
resulting educational delivery process and the institutions involved. The
material here is based upon observations in the U.S. and the degree of
generalization to other countries is unknown. However, distance education
is already crossing international boundaries and we will soon be entering
an international marketplace in distance education. Since the U.S. is today
more of a competitive marketplace (Turoff, 1995) in higher education than
most other countries, what is happening in the U.S. today may be a forecast
of what might occur worldwide in the future.
The marvels of technology present us with many alternative options for
the future of learning. However, it is not the technology but a vast combination
of social, economic, and political factors that will influence the way
we will actually employ this technology in the future. We are beginning
to understand that the introduction of Information systems into organizations
that have human problems often worsen those problems. The use of the technology
alone as a cure of human, managerial, and organizational ills in education
can also have disastrous results. This paper looks seriously at how institutional
policies and practices can influence outcomes of employing new technological
approaches to learning.
In the U.S., college tuition can range from approximately $3,000 to $30,000
for an academic year. Recently the concept of the average student being
right out of high school has been overturned and the majority of college
students have been, or are, in the work place. They are older, may have
a family, and are possibly part time students. They do not get much financial
aid and student loan programs are not anywhere as generous as previously.
The result is that the college student is becoming more of a mature consumer
who is serious about finding the college program that will produce the
best possible outcome for what he or she can afford to invest.
Many of the distance students we are attracting are not in remote areas
but they are people with job and family commitments that force them to
realize that the two to three hours of local travel to attend a course
is far to precious to consider face-to-face classes if the remote course
will be just as effective. Many distance students being attracted to the
ALN programs are the more valuable employees who never considered that
they could take the time needed for regular courses. At the graduate level
students in technical fields seek companies for jobs that will pay for
their courses on a part time basis. Clearly many of the current ALN programs
have been designed to "skim the cream" of the crop that these students
Higher education in the U.S. does not have a very effective lobby and many
state supported institutions are facing, at best, semi-static budgets that
are in effect cuts that cause increased tuition. The true costs at most
state institutions would result in a tuition $15,000 or more if the student
had to pay full costs. Outstanding private institutions cost about $25,000
but are subsidized, in addition, by endowments.
In a recent paper (Turoff, 1997) it was shown that one could start a pure
electronic virtual university for about 2000 students with a tuition of
around $15,000, where every single faculty member was paid a flat $150,000
a year and class size would be in the 25-30 student range. The initial
funding to get off the ground would be fifteen million dollars. This would
also allow each accepted student to take some initial courses free of charge.
This cost is less than a new building on most college campus in the United
States. Clearly there are no colleges where all faculty get that salary
but if it did, it would certainly attract a faculty of top educators. Today
most institutions would rather spend that chunk of funds on a new building.
That is part of the problem we are going to be discussing in terms of management
objectives in academic environments.
At many institutions distance education is a separate administrative operation
from the academic departments. Furthermore, the administration of distance
education and the responsibility for the delivery of the courses is often
in the hands of the distance education administration and not under the
faculty in the department that normally has the academic governance authority
over the face-to-face courses and their content. A significant percentage
of the courses in distance education, at a growing number of institutions,
are taught by adjuncts and non faculty staff. Since the cost of such courses
is usually lower, the administration comes to view such programs as a sources
of a "profit" in the cash flow. The distance learning operation often works
on a fixed fee and the remainder of the tuition paid by the student is
returned to the general budget funds.
Summary: Properties of the Force
The World Wide Web and the Internet make ALN's (anytime, any where)
delivery of courses by computer networks a viable alternative for millions
As a result of costs, college students are becoming intelligent consumers.
Costs of tuition are continuing to rise and no end is in sight.
Public funds for higher education are decreasing on a relative basis.
The typical college student is older, working, may have a family and
will likely be a part time student.
The typical student will consider distance courses a viable alternative
and in some ways more effective for their efficient mastery of the material.
No institution of higher education will have a geographical monopoly
they can count on.
Institutions in the US are viewing Distance Education as a magical
source of money.
Institutions are viewing distance education as a source for new students
who are both talented and well to do.
Administrators are after technology that will automate the delivery
of material with a minimum of academic person power.
Institutions offering similar programs will be in very direct competition
even though they may be thousands of miles apart.
Many students will have to rely on support from the companies they
work for and the companies will want programs that improve the short term
benefits of the student to the company.
Commercialization of the field of higher education is probably the most
visible of the consequences of the force. There are companies in the U.S.
that allow employees they fund for a higher education to sign up only for
courses that they feel are consistent with their objectives for the employee.
Most surveys of industry views and academic views of what is important
subject matter, even in the technical fields, show significant differences
of views. This difference is often associated with short and long term
views of what is important. The common goal conflict is between learning
fundamentals (e.g., the theoretical constructs for any computer language)
as a basis for life long learning ability and training for specific current
skills (e.g., a course in Java) desired in today's marketplace.
There is also a view by some universities administrators that if we just
put our courses on the WEB with some programs to give exercises and some
adjunct faculty, teaching staff, or student labor to run a hundred students
per section, then thousands of students will flock to these "automated"
courses and all the financial problems will go away. The U.S. has invested
sizable capital into trying to establish local colleges so everyone had
easy geographical access. These will probably be the first institutions
to be hit hard by distance programs from major universities and from commercial
sources of training. Current trends seem to indicate that there is going
to be a tremendous long term shakeout of marginal institutions that cannot
deliver quality education or meaningful full degree programs. Today you
can buy videos of "lectures of Harvard Professors" on the open market.
We are entering a period of what may be intense competition and commercialism
for higher education that will spread beyond national boundaries. The commercial
training programs to aid students to prepare for the GREs, GMATs, and MCATs
are indications of the future. Does the student perform on these examinations
of college abilities because of their college education or because of the
training programs for them?
There are a number of commercial companies that advertise that they will
create a virtual university/college on the WEB for any institution. They
will come to your college and in five days train the professors how to
transfer their courses to the WEB, how to teach on the WEB, and they will
manage the whole operation for you. It does not matter to them what subject
you are talking about, they have a single approach that is suppose
to work for any discipline and any degree program. They quote numbers such
as, it will take a professor only thirty hours to convert his or her existing
course to the WEB. To those administrators that think that faculty are
nothing but a problem, this is music to their ears! It recalls for those
of us in Information Systems the echo of the efficiency expert in the factory
and the process engineer in the office. What it suggests is the assembly
lining of higher education. It is the same "automation" approach that has
gotten countless companies in trouble. There is no awareness that the best
way to teach by using the technology may be completely different than what
is normally done in the physical face-to-face environment. There is no
recognition of the need to rethink the education process and the associated
The growing number of these academic consultants and WEB based real-estate
developers for the virtual campuses are evidence of the commercialization
process that seems to be overtaking higher education.
Commercialization can bring about the homogenization of the educational
A secondary, but no less worrisome force, is that many companies are backing
off from their sizable investment in internal training programs are turning
back to colleges and universities to take up the slack. They find the idea
of distance education attractive to reach their nationally and internationally
spread workforce. However, they speak in terms of such concepts as "just
in time training" like learning is analogous to an inventory system. They
confuse learning and training and are pushing institutions to offer small
chunks of education such as certificate programs that are also degree accredited.
While there is a role for colleges and universities to deliver training,
the traditional objective of higher education is to deliver a product to
the "student" that is the "mastery" of the subject and allows him or her
to advance their career even if that means giving the student the ability
to get qualify for a new job in a different organization. We attempt to
provide the student with the fundamental knowledge needed to continue learning
on their own and do not emphasize a particular current technology or skill.
Concepts of "just in time training" and training in current technologies,
in limited packages, is an educational mission that serves the company
and not the student.
With industry picking up an increasing share of the costs of tuition for
students they are beginning to influence the objectives of the educational
mission and it may not always be in a direction beneficial to the interests
of the student. This may be a more long term force of commercialization
than the prior one but it is probably a more dangerous one.
The educational objectives of industry and those of universities
are not necessarily consistent.
The Erosion of Tenure
While there are movements to try to do away with tenure in the educational
system in the U.S., it is not the more obvious lobby efforts that are worrisome.
What is of concern is the subtle administrative practices that can, over
a long period of time, significantly reduce tenure as an important component
of higher education and eliminate the safeguard it represents for the educational
The more insidious approach is the hiring of full time "qualified" staff
to do teaching and the necessary use of adjuncts. Young untenured faculty
are hired in great number and they are encouraged to focus on research
and seeking funds for research. They are alleviated from teaching many
undergraduate courses and these are taken over by full time staff, regular
adjuncts, and even "visiting" professors. While the untenured faculty think
this is fine, it is fundamental change in higher education objectives by
decreasing the importance of faculty performance in education at the undergraduate
level. What is not realized is that some of these untenured faculty are
being groomed for those staff positions when it becomes clear they will
not succeed to get tenure. What they are judged on is their ability to
bring in research funding and generate publications. Of course the long
term implication of this is that the tenure decision is no longer viewed
as important or serious. If they don't succeed they can become staff. They
will end up as contract research appointments or full time teaching staff.
The condition to get tenure will become so stiff that only a negligible
number will get through that filter. A few decades of this and the tenured
faculty will be a true minority of those teaching and gradually fade away.
What we then have is a growing number of educators who have no independence
of view and are totally beholding to the administration for their jobs
and their livelihood. This in the long run forces the system of higher
education to conform to the short term pressures of the marketplace and
growing commercialization. There was a time in this country when only two
universities were left that had outstanding physics education and research
programs in optics and all others had given it up as an unproductive investment
of resources. And then someone invented a working laser! With more and
more of the tuition costs being paid by industry the voice of industry
as to what should or should not be taught will become a strong determinant,
if tenure disappears, of what is taught.
The real reason for tenure is not to just protect minority views
of educators but to protect the educational process and associated academic
decisions on what is taught from the influences of short term market forces,
technology change, economic pressures and associated commercial interests.
We begin to see today degree program in some technical fields formulated
around advertised jargon in the marketplace rather than meaningful bodies
of scientific and technical knowledge. It is like a degree in sociology
would become a degree in the behavior of the X-generation because advertisement
firms wish to hire people who can graduate and be prepared to immediately
design adds for that market group. Industry is sometimes more interested
in the student being certified in a particular software system then in
obtaining an understanding of the fundamentals and theoretical principles
that underlie all such systems.
Over the past few decades many institutions in the U.S. have gone from
situations where the majority of the salaries at the institution were faculty
salaries to a situation today where a majority of the salaries are in administration
and non tenure lines for research and teaching staff, sometimes referred
to as research faculty, adjunct faculty, and/or fellows.
Faculty and Adjunct Compensation and Considerations
New Jersey Institute of Technology (a state funded University) numbers
are used here to illustrate some important quantitative results. The quantities
quoted in this section are the results of taking the NJIT budget report
to the state and doing some reverse engineering on the data given to infer
some useful information not directly quoted in the document. University
budget documents are notorious for not reporting the variables that could
tell you something about performance. It may be that the process of competition
will change that in the long run. Not all aspects of the marketplace are
negative for higher education.
Dividing the instructional salary ($26,262,000) by 300 faculty members
we have the average faculty salary of $87,540 that includes approximately
25% overhead and a normal teaching load of 6 courses a year and two units
for research and administrative activities. Therefore, the faculty cost
for a course is around $10,942 (1/8 of the $87,540).
Ph.D. students all get involved in teaching a course, grading material,
and/or running problem sections. Their compensation including tuition is
about $15,000, for which they may be involved in teaching two courses or
aiding in four by grading or conducting problem sessions. It is fair to
consider this an average of three courses. So the average cost when a Ph.D.
student teaches a course is $5,000. However, it is usually considered that
learning to teach is part of a Ph.D. student's training and usually he
or she would be mentored by the faculty member in charge of the course
and be provided with the faculty member's notes and materials. Furthermore,
in the VC environment (use of a conference system) the faculty member can
actually "listen in" on what is happening and dynamically guide the student
or step in when needed by reading the evolving conference.
Finally, we have the adjunct who is hired from the outside to conduct a
course and the standard fee for this is approximately $2,500. In principle
the academic department is in charge of insuring the quality of the people
doing this and it is certainly true there are a number of excellent industrial
people and ex-students willing to do this for the experience and pleasure
rather than the pay. However, there are still times when individuals are
brought in who do inadequate jobs and fail to really accomplish the objectives
of the course.
The key is that the administrative level is the wrong place to insure quality;
that task has to be done by a faculty member who must have some degree
of responsibility for the particular course and its content. In general,
the pay scales are too low to insure that all adjuncts perform at the level
of quality they should, and there is no incentive for faculty to mentor
these adjuncts in the same manner that they would doctoral students.
With all the emphasis on using adjuncts to save money one has to wonder
why administrators have not discovered the concept of the "adjunct administrator."
This could a retired executive or downsized manager hired on a part time
basis to analyze and make specific management decisions. All the arguments
made for use of adjunct faculty should apply equally well to saving the
costs of management overhead through the use of adjunct administrators.
Faculty $11,000 per course
Ph.D. Student $5,000 per course
Adjunct $2,500 per course.
What we have now is a situation where the administrative incentive is to
hire adjuncts rather than to finance Ph.D. students. Given a climate of
declining funding this sizable difference in economic incentive becomes
a force of its own. This may be even truer in the typical distance operation
where somehow these are students who are considered not quite as important
as the on campus students.
It would be a far more reliable administrative incentive to increase adjunct
salaries to be comparable to doctoral students and to pay a faculty member
a royalty to provide course materials and the mentoring of the adjunct.
Furthermore, the use of conferencing technology would allow faculty to
directly monitor the course being taught by the adjunct and provide dynamic
The use of conferencing like technology for class discussions means there
is a complete record of all the class discussion and the assignments that
have been done online. Such a transcript is far more informative in determining
the lecturer's ability to educate than the usual end of course survey.
In the future we would hope such transcripts become the evidence of teaching
performance both on the part of faculty and adjuncts for advancement and
retention. Furthermore, just as accreditation reviews sample course exams
and assignments, they should have samples of course transcripts to evaluate.
Though it is not reported we can further infer in the NJIT case that the
amount of teaching by adjuncts and non faculty staff of credit courses
is about 30% and growing. In some departments it is over 40%. If institutions
can provide credit courses utilizing instructors who are paid a fraction
of regular faculty pay then the very serious question will be raised by
those paying for the education (public or private institutions) as to why
those higher salaries have to be paid at all. If it is further true that
instructors can be eliminated or handle 100 or 1000 students in a course
because of the development of the "magic multimedia intelligent trainer,"
have another new force at work.
Associated with the salary differences is the issue of ownership of the
course materials and related videotapes and software. This is a major labor-management
issue at those institutions that have not fully recognized faculty ownership
and established the conditions under which such materials may or may not
be used by adjuncts. Currently many institutions provide faculty rights
to course materials for the purpose of ownership of textbooks but they
have introduced far more limited rights with respect to software and videotapes.
The technology is moving quickly to the point where a book is going to
become software (including programs, video, and sound material) and that
is going to lead to much conflict since all these nice distinctions completely
break down. The Collaborative Hypertext concept would lead to dynamic knowledge
bases developed collaboratively by the students and the instructor (Turoff
& Hiltz, 1995) and updated with each new offering of the course. It
would completely replace, in the long run, the text book and each new course
offering would provide a CD-ROM of the current state of the knowledge base.
The sizable difference between adjunct pay and faculty pay is exactly what
leads to associated labor - management problems and some mistrust of administrative
intentions on the part of the faculty. It is an unhealthy long term motivation
factor. All this might sound like a great economic opportunity to administrators.
However, now that students will be able to select a single degree program
from hundreds of offering Universities they will also be able to buy and
will pay for impartial assessments of the quality of these alternatives.
Independent evaluations of not only programs but specific instructors and
courses will also become commercial commodities on the WEB (Turoff, 1995).
After all a year of college is like buying a car and one can now purchase
evaluations of cars on the WEB.
The use of adjuncts is actually an important slack resource that needs
to be employed to allow for swings in load requirements that can occur
because of short term research commitments. Furthermore there is justification
for some number of full time teaching staff to handle the training missions
of the University. However, adjuncts and teaching staff should be under
the direct supervision of the faculty in charge of a course and they should
be reviewed by the appropriate faculty committees to be qualified for the
courses they teach. In many situations such as separate distance learning
programs this is not the case. In some institutions there is no uniform
policy on the supervision of adjuncts or full time teaching staff and it
is resolved on a department by department basis.
Adjuncts and full time teaching staff should be treated in the same
manner (pay, oversight, mentoring, etc.) as Ph.D. students.
Performance Throughput Rates
It has been a major surprise to many people at NJIT that about 80% of those
enrolling in our distance credit courses based upon the conferencing technology
are our regular on campus students. The first reaction of some administrators
was to suggest that we should forbid on campus students to enroll in distance
courses. Clearly the students recognize that courses taught this way can
be as good as face-to-face courses. A careful examination of the throughput
rates at our institution gives us another important reason why this is
The following table indicates the head count and the FTE (Full Time Equivalent)
numbers for the academic year of 1996 and summer. The third column computes
the resulting average credits taken. Full time undergraduates take only
30.6 average credits. However we allocate 149 FTE's from the summer to
make the average full time credits 32 (second line).
For the part time students we see the average credits are 10 for the academic
year. However, if we allocate the remainder of the summer FTEs to the part
time students we get 13.4 average credits. For these average credits we
can then calculate the number of years it takes a student to graduate.
We note that for the part time students at NJIT this can be anywhere from
9.6 to 12.8 years for an undergraduate degree.
These estimates are very optimistic because there are no data on the number
of students who withdraw or who fail a course. We would need data on the
number of repeated courses to obtain a better estimate and that would increase
the length of time. In Computer Science my impression is that that this
could easily be 20% or better of enrollments. If this figure were 20%,
then if the average length of time to complete a degree were five years
this one factor would add an additional year.
For graduate students we note that there are more FTE's than there are
full time students. This is because students will try to take 15 credits
a semester to finish the typical Masters in one year. So we have readjusted
the graduate credits to 30 which is the requirement for the typical master's
degree. This is the second entry in the FTE cell. Finally we allocate the
summer FTEs to make the Full Time students truly full time and then allocate
the remainder to the part time which brings us to 610 FTE's at 30 credits
For the part time students we find the average number of credits taken
is between 7.7 and 8.4. These numbers are optimistic when one factors in
that there are things like seminar courses that are for credit but do not
count toward a degree. In addition, we are not including any credits given
for teaching or taking makeup courses such as English (for foreign students)
or bridge courses to make up undergraduate deficiencies. Also we do not
have data to factor out the Ph.D. students who are taking research and
thesis credits. Based upon the above a part time graduate student will
take 3.6 to 3.9 years to complete the same Masters degree that a full time
student may complete in one year.
The final column recalculates the years to degree under two conditions.
One is that the introduction of remote learning allows the student to take
one more three credit course than he or she is now able to take during
the whole academic year and summer. The second condition is that the student
is able to take two additional courses during this twelve month period.
The assumption is that that the time saved by the student in not having
to commute to campus and the flexibility of participating at the times
of their choice allows this increase in efficiency. Certainly the working
student cannot take advantage of summer programs as they meet four out
of five days a week for six weeks.
9.8 / 8.0
7.8 / 6.6
2.8 / 2.2
2.8 / 2.2
2.6 / 2.1
The results are rather astounding:
The addition of one course during a year is a two to three years
saving for the undergraduate part time undergraduate and one year saving
for the part time graduate student.
For the two course conditions the possible saving is 3 to almost
5 years for the undergraduate and 1.5 years for the graduate student.
Nine to thirteen years to get an undergraduate degree is a very long
time and the fact that distance education offerings can knock three
to five years off of that is a significant benefit that the part time student
intuitively recognizes. I have had a number of on campus students tell
me that they would never have been able to complete their degree without
the distance offerings. Since they have started their degree they have
married, had children and gone through a number of job changes. This is
why our distance learning courses are so popular with our regular on campus
students. They recognize immediately that they solve conflicts in course
offerings and closed sections. If a student is unable to take a course
in a required sequence, he or she can lose six months in just solving the
prerequisite problem. Even full time students face the problem of closed
courses and conflicts in course offerings. With current budget pressures
fewer sections of a course are a natural consequence.
Distance offerings of courses benefit the regular on campus student
whether part time or full time.
Now we come to an example of a management paradox. Currently the office
of Distance Learning at NJIT receives one fee for each true remote student
and a much lesser fee (like 1/3) for each on campus student who takes a
distance learning course. The on campus student taking a distance course
needs the same exact services from the distance learning office as the
true remote student. They have an unsolved problem in providing the resources
to handle this flood of students they are not supposed to have. Of course,
the first administrative knee jerk reaction is: "We should forbid on campus
students from taking distance courses!" Fortunately we have not done that
at NJIT but it was suggested at one point quite seriously.
This is a holdover from the perception that these courses are only for
remote students. Just as a majority of today's college students
do not fit the model of a fresh out of high school entrants to college,
so in the future one could predict that:
A majority of the course work at universities and colleges will
be done remotely and the distinction between distance and on campus student
Essentially the choice in the future is whether the regular student wants
to come to a face-to-face lecture with material projected from a computer
or whether he or she wants to see the video and audio clips prepared by
the lecturer through the WEB or on videotape. The number of in class lecture
hours for a course will go down as the real discussion moves online.
The area that needs considerable rethinking is the budget allocation process
and the associated role of distance learning. In the mode in which many
of us teach, which is the merging of our on campus face to face students
and our remote students into a single virtual class, the distinctions that
currently exist make no sense whatsoever.
If the distance learning operation at an institution is done right
it will gradually put it self out of business as an operation separated
from the rest of the institutional organization.
The view of the distance learning office is usually that their objective
is the service of the remote student. We are entering a world where students
need the flexibility of determining when they are remote and when they
are not. A clear distinction will be hard to maintain and gradually become
There are many aspects where the distance learning program has significant
impact on the regular program. For example, the presence of videos of class
lectures in the library has been a significant aid to the regular student
who has missed one or more lectures because of sickness or work commitments.
It has also proved useful for those with language problems who need to
listen more than once to a lecture. However, there is no library mission
or budget to provide the tapes of courses for access by regular students.
As a result the tape collection availability is a far more limited in hours
than the normal hours for the library. The tape library comes out of the
distance learning office budget and not out of the library.
All organizational units at a college or university have to integrate
distance learning objectives and functions into their fundamental student
services and student functions.
We can look at many similar items in the budget and management objective
structure that are holdovers from the past and being applied to shape the
future. We suspect the same pattern exists at many institutions that are
growing distance learning programs without carefully revamping budgeting
and management incentives. The real root of the problem is:
The fallacy of assuming that the institution is dealing with two
different student populations (on campus and distance) when in fact one
is going to become impossible to differentiate from the other.
One might say that many institutions are approaching distance learning
by walking into the future backwards.
The consequences of this are many budgeting idiocies such as where the
distance student is paying for the costs of classrooms they never use and
the opportunity to use the income from distance students to improve the
distance program is non-existent. Once there is explicit clarification
of the goals of distance education then it is clear the budgeting process
needs to be designed to encourage the accomplishment of those goals. It
is often the lack of consistency between objectives and budget practices
that is at the heart of the difficulties of innovation in organizations.
Early in the evolution of work in Computer Mediated Communications and
the Virtual Classroom for learning it became clear that the objectives
we sought were:
Communications and Group Oriented Distance Learning
Equality of discussion among all members
Use of collaborative learning approaches
Student feedback that allowed us to determine how well we were doing.
Making learning about a topic interesting and challenging on both
an individual and group basis.
The instructor as expert and process facilitator
Complete rethinking of courses for obtaining quality improvements
made possible by the technology.
When we talk about software to support this area we talk about communication
protocols and human roles built into the software to aid the instructor
in promoting equal participation and facilitate the communication process.
We see an instructor as having a mental model of complex problem solving
in that domain and trying to convey to the students the structure of that
model and how to adapt their thinking process to learn and integrate that
model. Whether it be engineering design, computer programming, literary
analysis, historical understandings of events, or whatever, it is still
that fundamental endeavor. When we talk about advanced software in this
area we talk about Collaborative Semantic Hypertext templates that the
instructor can design to allow an intense discussion to be come self organizing
(Turoff & Hiltz, 1995). We also have considered and seen the possibilities
of incorporating learning techniques such as asynchronous gaming (Hsu,
et. al., 1992; Worrell, et. al., 1995).
We discovered very early that these objectives did not come without a price
and that price is "information overload" (Hiltz & Turoff, 1985). Many
of the features we have evolved were designed to allow active discussion
groups without overloading the individual members of the group. Given the
current state of the technology, which still lacks self organizing discussion
structures that the instructor can tailor to the problem domain, we see
current limitations on collaborative classes still in the 20-40 member
range. With dual instructors in a single course maybe 40-60 is possible
for some types of courses.
The effort in this type of learning environment is linearly related
to the number of students.
This type of approach bears no relation to CAI (Computer Assisted Instruction)
and the idea of software systems to teach skills and to train on an automated
basis. However, we see many instances where the interpretation of the use
of communications is to have a faculty member present taped lectures and
have a graduate student or staff person conduct question - answer conferences
with no attempt at collaboration and no real involvement of the actual
faculty member to facilitate and guide the students. When it is done that
way it can lead, obviously, to lower cost course delivery efforts with
large course sizes.
The Darkside of Distance Learning
Maximum number of students
Course multimedia notes and assignments for use on the WEB.
Multimedia material includes video and audio clips to replace lecture.
Animation to replace derivation type instruction.
Teaching assistant (student, staff) to answer questions by individual
Adjunct faculty or staff to back up what TA can't handle.
Third level in hierarchy might actually be person running course
and appearing on video clips but only to handle unusual problems.
CAI and other software to allow the student to conduct skill building
No group but individual students repeating an automated version
of the correspondence course.
with one TA for every 50 students we can have unlimited number of
students in a single offering.
Students can start at any time and end at any time and a course
becomes continuous operation.
There is really nothing wrong with any of the components to the Darkside
when they are used appropriately to develop skills and in effect offer
training to the student. There are software systems that can check for
good coding so that an instructor does not have to spend his or her time
reading and checking every program written by a student. But they will
not teach a student the pragmatics of how to decompose a particular application
for a program design except for essentially trivial examples. The advantage
of automation for skills is that it can free the faculty to teach the stuff
that cannot be automated.
Once a body of subject material is so well understood can be delivered
by the Darkside it will probably be offered quite cheaply as a commercial
software package presented as a product in training and/or self learning.
At this point the material is probably no longer worthy of what students
in the future will expect of a university education. This future may take
twenty or thirty years but it will come.
The future of the University may be the return to the early roots of an
educator and a group of students engaged in an explorative dialogue on
a specific problem domain with the students using the power of dynamic
(multimedia) Hypertext to evolve their associated skills.
If it can be automated is it really a university level education?!
Accreditation of Distance Learning
The Middle States Association of Colleges and Schools (which is one of
the U.S. regional accreditation agencies) recently issued, via the Commission
on Higher Education, a policy statement on Distance Learning (2/97). This
is a very significant document that if taken seriously and if enforced
will go a long way to improving the current situation with distance learning
in the U.S. Until now accreditation reviews have paid little attention
to what was going on in this area.
One suspects that, until recently, the common administration view of distance
education was that it was a source of revenue. If one could offer distance
courses at higher costs to the students, provide less in administrative
costs, and also utilize adjuncts to provide most of the courses, the results
would be a significant profit generating operation. By organizing it as
a separate administration unit the usual department and faculty quality
controls could be avoided or minimized.
The Middle States document appears to recognize fully that the "use of
information and distributed technologies has also allowed institutions
to develop innovative instructional methodologies that enhance and complement
learning in the traditional classroom environment." However they are still
one step away from recognizing that those same technologies can completely
remove the distinction between distance learning and normal classroom learning.
What they do recognize is:
"Educational programs conducted off campus, or special programs offered
on-campus, must meet standards comparable to those of all other institutional
offerings (Characteristics of Excellence in Higher Education, p. 14)."
What is significant is the interpretation of the above guideline in this
new document and we will only select out some items specific to this discussion:
The institution's programs holding specialized accreditation meet
the same requirements when offered electronically.
Clearly this means there should be no distinction on inputs or outputs
associated with either program and that integration is one desirable approach
to accomplishing this.
Qualified faculty provide appropriate oversight of the program electronically
This is specific that is needed as general policy through out higher education.
This is defied in any situation where administrators choose adjuncts and
where there is no responsibility of a faculty member to oversee or mentor
a course taught by an adjunct. This should apply to regular courses as
The program provides faculty support services specifically related
to teaching via an electronic system .
The program provides training for faculty who teach via the use of the
In these times of tightening budgets the new common administrative view
is that faculty should start buying their own equipment. However, this
does not solve the need for specialized peripheral equipment to handle
graphics, drawings, digitization, and multimedia (video and audio recordings)
that needs to be provided in a multimedia lab environment where faculty
can process their own materials. A true multimedia workstation with adequate
peripheral equipment is still in the seven to ten thousand dollar range.
The program provides for appropriate real-time or delayed interaction
between faculty and students and among students.
It is a pleasure to see a realization that electronic mail is no longer
considered sufficient and that some sort of group communication processes
will evidently be required. At least that is how I would like to interpret
Enrolled students have reasonable and adequate access to the range of student
services to support their learning.
Many institutions have significant auxiliary services such as tutoring,
counseling, librarians, etc. that are not yet provided to the distance
students. For example, in a recent case of potential cheating the Dean
of Students offices was at a loss as how to hold a formal hearing (as required)
when a distance student was involved.
Promotion and Tenure
Policies for faculty evaluation include appropriate consideration
of teaching and scholarly activities related to electronically offered
Here is the crux of a trend that will change the nature of college education.
Adjuncts and staff are not going to receive the resources and time to engage
in scholarly activities. Their knowledge of any field that is in evolving
is going to deteriorate over time and so is their ability to teach timely
material or undertake curriculum development activities.
The institution evaluates the program's educational effectiveness,
including assessments of student learning outcomes, student retention,
and student and faculty satisfaction.
Students have access to such program evaluation data.
The institution provides for assessment and document of student achievement
in each course and at completion of the program.
We have been fortunate to have had funds from both Annenberg/CPB and Sloan
to conduct evaluation efforts consistent with the above, but it is not
clear this will be done in any meaningful way in most Distance Education
Offices. To date they have not had to concern themselves with accreditation
Certainly the evaluation data on individual courses, programs, and faculty
is NOT available to students. In the Virtual Classroom area they can find
it in the nature of research reports, publications and books but they are
not likely to know where to look. At some universities the evaluation of
courses is done by students and published in the student newspapers. Ultimately
there will be commercial offerings of course by course evaluations by outside
organizations via the WEB.
Once again this is the issue of academic oversight or mentoring of any
offerings by adjuncts. If accreditation agencies are going to take their
guidelines seriously they need to start requiring review of the resumes
of all the adjuncts being used in an academic program and not merely focus
on easy measures such as the percentage of use. There is nothing wrong
with using a higher percentage if the adjuncts have appropriate backgrounds
and they are mentored appropriately by regular faculty who get proper credit
for doing this function. Formal faculty review of adjuncts and other non
faculty teachers is not currently a uniform institutional practice.
Given that we are entering marketplace atmosphere in higher education and
that the student is in the situation of spending $3000 to $30,000 every
year on a college education, students are becoming intelligent consumers.
For the marketplace to work effectively they need to obtain the information
to make informed choices of what course and program offerings they should
purchase. If the institutions themselves do not supply this information
than third parties will emerge to offer this material to students for a
fee just as one can buy reports on and inspections of new and used cars.
Let us hope that, as a result, colleges do not take on the aspect of a
car dealership and faculty are not replaced by salespersons.
Related to the necessity for consumer information will be the need for
explicit transfer policies that provide the student up front assessment
of their remaining degree requirements. In principle accredit programs
should become the national standard for automatic transfer credit.
The Nature of Learning
Unfortunately some items that appeared in the draft policy statement in
September of 1996 did not appear in the final version of the report.
Every effort should be made to choose technology that supports and
enhances the goals and objectives of the program. If the program requires
collaborative group learning activities, the technology should support
the same activities in distance learning.
That earlier document went further and pointed out that a decent computer
conferencing system is required if collaborative learning is an explicit
objective of the program. It is this area that truly leads to the realization
that an institution cannot teach a subject one way to its regular students
and choose to offer a completely different alternative to the distance
Our evaluation work has shown that the use of collaborative learning approaches
really require courses of not much more than 30 students per section. Otherwise
the amount of communication generated by the instructor, the students,
and the work teams becomes unmanageable and leads to information overload
(Hiltz & Turoff, 1985). Administrators feel that the WEB will allow
them to put out software to support learning that will allow a hundred
students per section with no problem. One can not be sure why the above
item was dropped but it can be suspected that there was a protest since
almost any college course now contains some degree of class discussion
and this would run counter to the idea of large distance sections.
There is often a complete misconception between faculty and administrators
when it comes to understanding the nature of a college education. What
we as faculty believe we are doing is facilitating learning, while a lot
of what is being offered through software is actually training. What we
are doing is attempting to transfer our mental models of a complex academic
domain to the student and the only way we can determine if we are being
successful is with a high degree of feedback from the students that tells
us the extent of that transfer and dynamically allows us to aid the individual
student who has reached the wrong model construct.
Perhaps in the long run there is a very pragmatic observation here.
If a topic can be taught by software and there is no need for human
communication, then it is no longer worthy of a university course, but
is what we have commonly come to view as training and the acquisition of
The accreditation problem would be better served if the accreditation agencies
realized that the root of the problem is that they should be accrediting
the educators as well as the programs. As more and more courses come on
to the WEB it is going to become difficult for consumers to realize what
they are getting for their money. There are already courses in creative
writing, poetry and other such subjects offered by private individuals
over the WEB. There are a growing number of technical courses being offered
by companies and covering, in theory, the same skills offered in university
based courses. Most of these are cheaper than their university counterparts.
Once can also expect nationwide tutoring services by companies and individuals
quite similar to what has happened with the entrance exam coaching operations.
Related Administrative Practices
Two other interesting items that probably also generate some controversy
from administrators and which did not appear in the final document were:
. . policies addressing teaching load, class size, time needed for
course conversion/development, and the sharing of instructional responsibilities
should be reviewed.
. . one of the consistent responsibilities of faculty continues to be
the development and oversight of an institution's academic programs.
Trying to shoehorn the current policies into the new technology is a common
approach that does not work and discourages beneficial use of the technology.
Those, like NJIT, having had grants that covered some the associated conversion
and development costs, had been at an advantage many other institutions
will not have. But the need for improved workload policies still remains
an issue that will end up as part of the labor-management negotiation process.
There is a confusion at many institutions between faculty governance and
union authority in a number of these areas. As a result administrations
may have more power than they should have to dictate in these areas. Also
as a result of financial cutbacks there has been a trend for administrations
to ignore or sidestep faculty oversight.
Warped Views on Distance Education
I was pleased to the see the following dropped but that does not disguise
the fact that most faculty and administrators really start with a hidden
premise in their mind that distance learning is an inferior form of education.
Perhaps one of the major issues facing both providers and recipients
of electronically offered distance learning programs is achieving parity
with traditional on-campus courses.
Starting with such a view becomes a self-fulfilling prophecy as one will
then design a system that is actually inferior. As the reader has gathered
our view is that distance learning can be superior if done correctly. By
incorporating the Virtual Classroom and collaborative learning in traditional
courses, those traditional courses are far better than any normal traditional
course. By eliminating the distinction between distance and regular courses
by merging the students into one section we have no problem in achieving
parity and in achieving a higher quality level of education.
Faculty Developed Materials
This too was dropped and probably because a number of institutions are
actively trying to reduce faculty rights to the materials they developed
for these courses.
Faculty members generally own their notes and teaching products,
which they are paid to develop and deliver. Traditionally, they control
the further or extended use of those materials - both when and how they
It is not clear to faculty at many institutions what this means for the
tapes of lectures, the multimedia software documents, and the software
programs that they develop for teaching. As a faculty member who does his
most productive work from his home, I have to buy my own computer. I have
no problem in putting my own copyright on what I have developed with my
own resources, and to be frank, most course development is done on an unpaid
overtime basis (if one were to count hours). However, the uncertainties
today about this issue probably hold many faculty back from making a sizable
effort in course development.
Furthermore, there are different ownership standards implied for software.
Multimedia versions of course materials are really a mixture of writing
and software so that the distinction between course notes and software
has disappeared. The draft document also said about this and related issues:
"it is incumbent upon both faculty and administration to engage in a dialogue
on these issues and to develop policies that do not undermine the teaching/learning
The Organization of Distance Learning
The final item dropped was perhaps the most important but considering the
separation that already exists at some institutions it no doubt raised
some strong objections:
First and foremost, an institution's distance learning program must
be an integral part of the institution's mission and not an ancillary consideration.
This sums up quite nicely what we have been trying to get across throughout
this examination of distance learning. Given the dropping of this and other
items it becomes clear we still have much in the way of future controversy
to deal with in this area..
Final Observations and Conclusions
Students must be able to get departmental advisement, deal with the registrar,
the library, the bursar, the tutoring center, the student dean and all
the other normal on campus functions. Many of these offices do not provide
WEB based access to these services. If they did, a great many regular students
would also utilize this approach since it is far more efficient for the
student than the phone or having to come to campus to handle a problem.
This problem also results from the concept that the distance students are
separate from the on campus students and that the distance learning office
should be the only interface to the distance student. This view is really
inadequate. For most services the distinction made between Distance Learning
and on campus students should disappear.
Recognizing the importance of remote technology for regular courses
and the integration of distance students into regular courses.
There is little doubt at this point that the technologies being employed
for the distance students are or could be a boon to the improvement of
the quality of education for the on campus students. Using conferencing
based systems for regular face to face classes makes a significant improvement
in the learning process if the instructor has the right training and attitudinal
skills to employ the technology. Furthermore, it is only possible to get
the synergy of a classroom atmosphere in the conferencing technology with
a critical mass of active students, such as 15 or more.
Integrating distance students into regular classes another is also a way
to be able to support small numbers of distance students who are interested
in specialized electives without large student audiences. The technology
may very well make more diverse course offerings economically feasible.
A related corollary is that:
The distinction between distance students and regular students will
disappear in that local students will select when they wish to take a course
via electronic technology or via the physical classroom.
Another option that is also feasible is that some courses can get along
with fewer hours of lecture during the week given the sizable on line discussion
and assignment activities in the conferencing environment. We have a number
of courses that chosen to go to two hours of lecture rather than three.
In effect the third hour used to be discussion and that is now done online.
With the merger of the two, the time normally utilized for in class discussion
can be reallocated to online discussions, to actually reduce the hours
Large discrepancies in the pay to teach a course conveys the wrong pubic
image about the quality of education. There should be economic indifference
by administrators at any institution in the choice of using adjuncts or
Ph.D. students. The same teaching standards and faculty oversight and control
needs to apply to all offerings of courses no matter who teaches it.
Timing is an important component of asynchronous learning offerings. Most
of our summer offerings stretch over the whole summer rather than conform
to the two summer sessions for face to face offerings.
Three equal semesters a year would be an ideal environment for distance
Ultimately a three semester system for Institutions of higher learning
would be an extremely desirable situation for distance courses. It would
also carry considerable appeal for faculty and students who could choose
to take their "summers" during any one of the three semesters. In the new
environment where a majority of students are working full time and/or have
families, the concept of three equal semesters a year has many desirable
qualities. On the other hand the four quarter system probably leads to
a too compressed time scale for asynchronous communication oriented courses.
This latter point is an intuitive conjecture that could bear closer examination.
Those now doing remote classes in the summer usually choose to do it over
the two separate summer sessions that NJIT now offers.
Most institutions need improved tracking and measurement systems to truly
track performance type measures such as throughput times for degrees and
more comparative information about student performance correlated with
types of instructional delivery and programs. It is amazing how little
data is available in this era of institutions gloating over how highly
computerized and modernized they have become.
We are entering an era of a national and worldwide marketplace in
higher education programs and courses.
The networking technology for distance education is opening up all universities
and colleges to competition. Students can now cross geographical barriers
and take courses in any state, or for that matter, any country. The current
growing cost of higher education and the trend to cutbacks of public support
in the U.S. is forcing many students to put costs first in seeking an education
and to act as consumers in a market economy.
Is a community college course just as good as a university course costing
two to ten times more? Are two accredited programs the same when one costs
a lot more than the other? Can I find an employer who will fund my education
on a part time basis? If it is part time, is the distance course just as
good as the on campus course, and shouldn't it save me considerable time
in travel and expense?
Students will become intelligent consumers and will seek information
by which to evaluate their perspective purchases of courses and programs.
Universities need to publish better pragmatic data on things like throughput
rates or related job performance of their graduates. If faculty and the
institutions they represent do not take the initiative to provide evaluative
data for the benefit of the perspective students than others will.
Stephen Erhman (1994) in what I feel is one of the more significant policy
articles in this field stated the following three challenges for higher
Accessibility, especially how to reach and educate the full range of
adults who deserve a chance at an education, despite their location, schedules,
cultural differences, or physical disabilities;
Quality, especially learning for the 21st century: how to improve the
life chances of each of their adult learners, as individuals and as members
of economic, cultural, and political communities; and
Dealing with the first two challenges when money and political support
are in short supply.
If we are going to meet those challenges it is going to take fundamental
changes in higher education.
Distance Learning Policy Statement (Approved February 1997) by the Commission
on Higher Education, Middle States Association of Colleges and Schools,
3624 Market Street, Philadelphia, Pa. 19104.
Ehrman, Stephen C., RESPONDING TO THE TRIPLE CHALLENGE: Facing Post-Secondary
Education: Accessibility, Quality, Costs,; A Report Prepared for the Organization
for Economic Cooperation and Development, Center for Educational Research
and Innovation, Washington, D. C., 1996
Harasim, Linda, Hiltz, Roxanne, Teles, Lucio, & Turoff, Murray, Learning
Networks: A field guide to teaching and learning online, MIT Press, 1995.
Hiltz, S. R., The Virtual Classroom: Learning Without Limits via Computer
Networks, Norwood NJ: Ablex Publishing Corp., 1994
Hiltz, S. R. and Turoff, Murray, (1985), Structuring Computer-Mediated
Communication Systems to Avoid Information Overload. Communications of
the ACM, 28, 7 (July): 680-689. Reprinted by DATAPRO for Corporate Distribution.
Hiltz, S. R. & Turoff, Murray, The Network Nation, Revised Edition,
MIT Press, 1993 (early edition: Addison Wesley, 1978).
Hsu, Enrico Y. P., Hiltz, S. R., and Turoff, Murray (1992). Computer-Mediated
Conferencing System as Applied to a Business Curriculum: A Research Update.
In V. S. Jacob and H. Pirkul, eds., The Impact of Information Technology
on Business Schools: Research, Teaching and Administration, Proceedings
of the 20th Annual North American Conference of the International Business
School Computer Users Group, pp. 214- 227.
Turoff, Murray, (1970) Delphi Conferencing: Computer Based Conferencing
with Anonymity, Journal of Technological Forecasting and Social Change
Turoff, Murray, (1972), Party-Line and Discussion: Computerized Conferencing
Systems, Proceedings of the First International Conference on Computers
and Communications, October, IEEE.
Turoff, Murray, (1982), On the Design of an Electronic University, in Telecommunications
and Higher Education published by NJIT, April.
Turoff, Murray, (1982), The EIES Educational Experience, in Telecommunications
and Higher Education published by NJIT, April.
Turoff, Murray, (1989), The Anatomy of a Computer Application Innovations:
Computer Mediated Communications (CMC), Journal of Technological Forecasting
and Social Change, Volume 36, 107-122.
Turoff, Murray, (1990), Computer Mediated Communication Requirements for
Group Support, Journal of Organizational Computing, Volume 1, Number 1.
Turoff, Murray, (1995), A Marketplace Approach to the Information Highway,
Boardwatch Magazine, April. Updated version in NJIT Magazine, Fall, 1996.
Turoff, Murray, (1997), Costs for the Development of a Virtual University,
JALN Journal of Asynchronous Learning Networks, Volume 1, Issue 1, March
1997, url: http://www.aln.org
Turoff, Murray and Hiltz, S. R., (1977, 1980), Meeting Through Your Computer,
Spectrum, May 1977, 58-64. Reprinted in William C. House, Electronic Communications
Systems, Petricelli Books, 1980.
Turoff, Murray and Hiltz, S. R., (1986), Remote Learning: Technologies
and Opportunities. Proceedings, World Conference on Continuing Engineering
Turoff, Murray, & Starr Roxanne Hiltz, (1995) Software Design and the
Future of the Virtual Classroom, Journal of Information Technology for
Teacher Education, Volume 4, Number 2, 1995, 197-215.
Turoff, Murray, Rao, Usha, & Hiltz, S. R., (1991) Collaborative Hypertext
in Computer Mediated Communications, Proceeding of the Hawaii International
Conference on Systems Science, Volume 4, January, IEEE Computer Society,
Worrell, W., Hiltz, S. R., Turoff, M. and Fjermestad, J. (1995) An experiment
in collaborative learning using a game and a computer-mediated conference
in accounting games. Proceedings of the 28th Annual Hawaii International
Conference on System Sciences, Vol. IV, pp. 63-71. Los Alamitos, CA: IEEE
Computer Society Press, 1995.