An Overview of research activities in Computer Mediated Communications from 1976 to 1991 conducted by the Computerized Conferencing and Communications Center at NJIT by Murray Turoff and Starr Roxanne Hiltz

Table of Contents

OVERVIEW

Facilities to support group work do not have to be physical; they can be constructed in software, or "virtual" structures, allowing participants to "meet" and work together within a Computer-Mediated Communication System (CMCS). These systems use computers and telecommunications networks to compose, store, deliver, regulate, and process communication among the group, and between the computer and the group. Among the types of systems that come under this heading are electronic mail, computerized conferencing, and bulletin-board systems. The most common form of CMCS is "electronic mail," or message systems that deliver discrete text communications from a sender to one or more recipients via computer networks. Conferencing systems order and maintain a permanent data base of discussions and activities related to a task, and are structured to support group work. Conferences have "memberships," and different members may have different roles, supported by software.

Our work on group support systems has focused on computer conferencing. Three main application areas have been of primary importance: medium to large sized scientific and professional communities (Hiltz 1984; Hiltz and Turoff, 1990); Group Decision Support (Turoff and Hiltz, 1982; Turoff, 1991); and educational delivery (Hiltz, 1986, 1992).

The following are key aspects of our work on the design and evaluation of CMCS:

The themes of the importance of structure and the advantages of providing the opportunity for asynchronous communication are expanded upon in the following sections.

GO TO START


STRUCTURES TO SUPPORT GROUP WORK

Colleagues who worked with us for several years, Peter and Trudy Johnson-Lenz (1982) created and defined the term GROUPWARE as:

Intentional GROUP processes and procedures to achieve specific purposes plus softWARE tools designed to support and facilitate the group's work.

A primary emphasis at NJIT has been the design and evaluation of a variety of such "groupware" structures to match the requirements of groups with different tasks, sizes, and social characteristics. Examples of aspects of structure include:

Many CMCS have a single structure or set of tools, which may fit some groups and tasks, but not others. Structures should be "tailorable," so that different groups and conferences using the same system can each employ a set of tools and procedures which best fits its task. This is the design challenge for the current generation of CMC systems.

GO TO START


ASYNCHRONOUS COMMUNICATION

Much of the current work in the area of Groupware and GDSS is carried out under the chronic fallacious assumption that has always plagued the development of Information Systems: the presumption of automation. It is assumed that the best way to do something is the way it was originally done manually. While that may be the easiest thing to sell, it has consistently been demonstrated that this is among the worst ways to design a system in order to gain the benefits that computerization can offer.

The most misunderstood concept in CMC systems is the view that an asynchronous (or non-simultaneous) communication process is a problem, because it is not the sequential process that people use in the face-to-face mode. The approach of "how do we make CMC feel to the user like face-to-face processes?" is incorrect. The real issue is how do we use the opportunity of asynchronous communications to create a group process that is actually better than face-to-face group communications?

The primary advantage of using CMC to support group processes is not that people can engage in the process whenever it is convenient for them. It lies in the very fundamental asynchronous nature of the communication medium. Many of the current design philosophies attempt to maintain the sequential nature of the process that groups go through in face-to-face settings, and assume this is the right way to go. On the contrary, the potential for real improvement in group processes lies in the fact that individuals can concentrate on the part of the problem they can contribute to at a given time, regardless of where the other individuals are in the process.

The rigid specification of a group process may conflict with the ways in which specific individuals can best contribute to the task. The resulting opportunity for asynchronous approaches to group problem solving is to free the individual to deal with the problem in ways consistent with his or her cognitive style. The resulting design challenge is to provide the communication structures to allow for synchronization of the group process, and the organization of the material for the benefit of the group.

GO TO START


RESEARCH FACILITIES AND SUPPORT

The physical facilities to support the research consists of an adequate number and size of computers to support development work as well as experiments and field trials. Over the years, this has involved a number of minicomputers; currently, we have systems running on Hewlett Packard mini-computers, several SUN workstations, and an IBM AIX work station. The systems themselves are "laboratories without walls," which can be accessed from anyplace in the world, using a personal computer, a modem, and a telephone. We also have a small Collaborative Systems Laboratory, with seven work stations, for training experimental subjects.

The Computerized Conferencing and Communications Center at the New Jersey Institute of Technology was established in 1974, based on the following premises about the environment necessary to advance the state of the art in using computers to facilitate group communication:

Based on the above premises, the Center established the Electronic Information Exchange System (EIES 1) utility in 1976, to create an environment for both evolving the technology and studying its impact. EIES 1 was made available to groups from industry, government, non-profits, and academia. EIES 1 averaged about 2,000 users a year from its inception until its current phase-out in 1991. Many existing commercial systems (e.g. Participate) evolved from prototype special communication structures first implemented and field tested under instrumented conditions in the EIES 1 environment. Many of the features and facilities in other commercial systems can be traced back to prototypes in the EIES 1 environment.

Over 25 special sub-systems were developed on EIES over the years from 1976 to 1987. Each represented a special structure and set of tools for specific types of group tasks. Among those worthy of note are:

1. TERMS: The Joint Electron Device Engineering Council (JEDEC) used EIES for selected aspects of its work of promoting hardware and software standardization in microcomputer/large scale integration products. JEDEC's standardization activities are conducted by a series of committees, which ordinarily communicate only through quarterly face-to-face meetings, with phone and/or mail in between. A great deal of JEDEC's work in reaching standards that will obtain the required unanimous approval of its members involves first reaching agreement on a set of terms and definitions that apply to a given standardization topic. In addition to free-form discussion in conferences, a structured decision aid was programmed to support the process of developing and reaching agreement on such a standard set of terms and definitions (Johnson-Lenz, Johnson-Lenz, and Hessman, 1980). Called "TERMS," this system allowed any member of a committee to:

2. RESOURCES: This is a data base subsystem which allowed for the mixing of qualitative and quantitative information in specified formats for both creation and retrieval. The unique feature of RESOURCES is that it was designed to handle the problem of a group of individuals gathering and validating entries to a data base. Therefore, all entries are identified by who is contributing them, and it is possible to associate comments with any entry in the data base.

3. REPORTS: This is a group oriented authoring system that provided the ability to establish outlines of a report and signify different roles for individuals at every major point in the outline. Therefore, one could assign authorship, editing, contributing and organizing privileges to different individuals at different places in the planned report. This system followed many ideas in Englebart's work (1973, 1976) in collaborative composition and added very explicit role structures for editing, organizing, and producing the resulting documents.

4. TOPICS: This system (Johnson-Lenz, 1981) was designed to handle unpredictable information exchange where a large group of people (100 or more) were involved in trying to exchange information. Every member of the network was allowed to send three line inquiries to every other member. Each recipient of an inquiry could select whether to track future responses to the question. Responses could be supplied by any member, but had to be limited to one page. There was a human indexer who had the software supported power to keep all the keys consistent. A human editor had the job of collecting the responses, eliminating duplications and developing summary briefs. The inquiries, responses and briefs went into a data base structure for later retrieval.

5. SURVEY: This was a complete system for doing surveys, and allowed those taking the survey to get summaries of the results. In one specialized evaluation application, we put up a standardized psychological test and provided the user an assessment of their personality type.

6. ZBB and GDSS: One of the most important systems built as part of EIES was an asynchronous version of Zero Based Budgeting (Bahgat 1986). ZBB has largely been a failure in the face-to-face mode, because of the huge amount of communication requirements it normally adds to the already top-heavy budgeting process. The CMC was utilized quite successfully by about 30 people to allocate one and a half million dollars of capital budget funding without a single face-to-face meeting of the group. This research demonstrated that an approach to budget planning that had difficulties being used in the face-to-face mode could be better utilized in the CMC asynchronous mode.

7. DECISION SUPPORT TOOLS AND PROCESSES: In one experiment, a special structure was created which led the participants through 57 steps from introducing themselves to one another and receiving their task, through agreement on a solution. Among the special tools was a rank-ordering routine to help the group understand its preference structure, and "real-time" alerts whenever a group member changed their vote. The various decision support structures experimented with on EIES 1 were generally employed in a "same time/different places" mode.

Another important and unique aspect of EIES 1 was that it included extensive monitoring facilities for evaluation research. In addition to recording the number and length of each type of communication written and read by each member, it included the ability to generate a "who-to-whom" matrix of message traffic for each group, and a conference participation analysis which gave the number and percentage of comments for each member for a requested time period.

The key observation about all these specialized structures on EIES is that they went through considerable design evolution. Both user participation in design conference discussions and feedback from the users of the prototypes were used to tailor the structures to the needs of the users. Another important finding was that it was possible to use the technology as a tool to better understand individual and group problem solving processes (Hiltz, Turoff and Johnson 1982).

GO TO START


EIES 2

Approximately five years ago, the Center decided it had learned most of what could be learned from the single central computer environment, which was the foundation of the internal EIES 1 design. The decision was made to begin the design and implementation of a new generation of the technology that would be suitable for a distributed network environment, and also incorporate what we had learned about the functionality necessary to support the group communication process.

Following our original philosophy, it was also decided to create this new system as a product, so that it could provide field inputs from a wide variety of groups and applications, as well as allowing usage by large populations. There was also the consideration that an outstanding product in this area would provide a degree of financial stability for the research program. EIES 2 operates within the UNIX environment and has been leased or distributed to a number of sites.

An additional important factor was the necessity for this new system to serve as our research platform for continued evolution of the technology.

The combination of these factors led to a number of key design decisions that makes EIES 2 a system that has a foundation that will allow continued evolution and the incorporation of additional functionality.

The basic facilities and types of communication objects included and planned in EIES 2 are:

EIES2 STRUCTURE

OBJECT EXPLANATION
Members
Personal Index
Roles
The users of the system
To track items
Privilege Collections
Groups
Membership List
Group Index
Group Conference
Group Messages
Super members of the system
Status of all members
Shared by all members
Owned by group
Sent and received to group
Conferences
Membership List
Conference Index
Comments
Replies
Attachments
Activities
Topic communication space
Status of all members
Comment keys
Entries in a conference
To higher level comment
Passive file attached to comment
Executable programs (e.g. voting)
Directory
Interest Index
Topic Index
Of members, groups & conferences
Member interests & message sending key
Group and conference topics>
Messages
Notifications
Forms
Lists
Index Entries
Reference Keys
Filters
Labels
Private, Group, and Public
Transaction notices
Structured data collection
Labelled collections of items

Pointers to items
Screening terms
Substitutions for commands or strings

Many of these objects should be clear to anyone who has had some experience with CMC technology (Turoff, 1989; Turoff, et. el., 1989). However, ACTIVITIES are a concept that is very new, not explicitly present in other current CMC systems.

ACTIVITIES ON EIES 2

One of the necessities for a CMC system to service group oriented objectives is the integration of other computer resources within the CMC environment. This means the following types of available capabilities:

There are many approaches to integration with the underlying technology. We utilize the metaphor of an "activity" that can be attached to any communication item, such as a comment or message. This activity, when triggered or done, will execute a program or procedure on the host computer or the network of computers. Unlike EIES 1, where each special structure for computer support had its own commands and interface, EVERY executable program made available in EIES 2 is an activity; and the basic activity menu (Create, Do, Modify, View) applies to all of them. Some examples of Activities that are available or under development on EIES 2 include:

1. List Gathering Activity: An example of a necessary activity is the ability to collaboratively collect a list of structured items and to treat this collection of items as one type of list. One list is the table of contents of a document, such as the document the group is trying to create. Another type of list is a set of tasks to be done. In addition, there could be lists of issues to address, terms to define, alternative criteria for a decision, possible solutions to a problem, etc.

The person creating an activity has to have control over when certain actions are allowed. For example, if people are contributing to the list, the owner of the activity should be able to do things such as "close" it to further contributions, if it has sufficient contributions. The owner should be able to open, at the right time, certain other actions that can take place on an existing list:

There are many types of activities possible, both of a very general and very specific nature. Whatever the nature of a specific activity, it is important that this facility provide the tracking of status of activities for both individuals and the group. The leaders or facilitators of a group need to know who in the group has or has not completed a given activity. Also, the system should have facilities for sending automatic reminder notifications to anyone who has not done a particular activity.

The concept of activities, within a CMC system, is open-ended and represents one of the primary mechanisms whereby future extensions will be made to EIES 2 and TEIES. What is crucial is that the incorporation of a wide range of tailored facilities to support an application be provided via a common interface metaphor, with the same command functions and object definitions applying to the whole set. This is the only way that all the members of the group can quickly acquire and learn the tools as they are carrying out the application.

2. Gradebook Activity: This is essentially a spreadsheet with privileges, created for the Virtual Classroom [TM] project. The rows are student-members of the system, the columns are grades. The instructor has the privilege of defining the grades and their weights, entering and modifying grades (numbers in the spreadsheet), and seeing anybody's average at any time. Students have the privilege of seeing their own grades and weighted average, and the total for the entire class. When new grades are entered in the Gradebook, notification is automatically sent to the students.

This is a particular collaborative spreadsheet structure for a specific purpose; slightly different spreadsheet activities could be made available for tasks such as budgeting or planning. Allowing different individuals to make different estimates of the same budget items is one such example

3. Question/Response Activity: This is similar in function to the "nominal group technique." Each member of the conference must respond to the question, before being able to view the responses of others. This assures independent thinking on the issue, before the participants are influenced by the views of others. It also enforces equal participation.

GO TO START


FINANCIAL SUPPORT

The Computerized Conferencing and Communications Center has applied for and received over 20 grants and contracts over the years. Major support has been received from the National Science Foundation, the Annenberg/CPB project, the New Jersey Commission on Science and Technology, the New Jersey Department of Higher Education, and IBM. In addition, substantial awards of funds or equipment have been made by Apple Computer, Hewlett Packard, and the Department of Defense. Currently, support for research on Distributed Group Support Systems is being provided by the National Science Foundation. The Center continues to seek support for research and development from public agencies, private foundations, and industry.

Funds for operational expenses such as equipment maintenance, communications charges, and support personnel are also generated by membership and use charges for the versions of the systems operated by NJIT, and by software leases and maintenance. New Jersey Institute of Technology has frequently provided support for operating expenses at times when revenues have been insufficient to cover expenses, and in the form of matching funds for grants and contracts.

GO TO START


COMPLETED RESEARCH: User Communities

One line of research has been studies of user communities on EIES and other CMCS. The objective has been to describe the nature of social interaction on these systems, and the interaction of software features, group size, and task.

As part of that effort all the prior work in this area was summarized in a resulting book (Kerr and Hiltz, 1982) which gathered the results of all the efforts in the design and evaluation of CMC systems to that point in time.

The funding for the initial development of EIES came from the National Science Foundation in 1976, for the support of scientific research communities. On-line Communities (Hiltz, 1984) describes the long-term impacts of the system on research and development communities that ranged in size from 10 to almost 200.

This was followed by a four-system, longitudinal study of "determinants of acceptance of computer-mediated communication systems." The study documented how characteristics of the users, the system, and the task or application interact to influence whether or not invited users will become "dropouts," (Hiltz, 1989); the extent to which CMCS is "productivity enhancing" (Hiltz, 1988); and user satisfaction (Hiltz and Johnson, 1990).

GO TO START


COMPLETED RESEARCH: Synchronous Decision Experiments

Our initial studies of the use of computerized conferencing as a group decision support system used groups which participated at the same time, but from different locations. This is because it is not possible to get true "control" for asynchronous groups, and we wished to discover how communications process and decision outcome varied between face-to-face groups and computer-mediated groups, when everything else was held constant. (Turoff and Hiltz, 1982). The three experiments followed a logical progression, as follows.

The first experiment compared the process and outcome of face-to-face versus unstructured computerized conferences. By "unstructured," we mean that the participants were not provided with any decision support tools to aid either their group process or their ability to analyze the data generated during the process. The subjects were college students, so that the experiment could be conducted within laboratory facilities (one-way mirrors, extensive recording equipment, etc.). Basically, we found "no difference" in quality of decision between the two modes, but less likelihood that the CMC groups could reach complete consensus on a decision. We also observed substantial differences in communication process, as measured by Bales Interaction Process Analysis (Hiltz, Johnson, Aronovitch, and Turoff, 1981; Hiltz, Johnson and Turoff, 1982, 1986).

The second experiment in this series moved out into the field for one-day problem-solving exercises. The subjects were managers and staff in business, government, and academic bureaucracies. It replicated all of the procedures for one of the conditions in the first experiment, but also introduced two structures to support decision making in the CMCS environment, in a factorial design. The first structure was the designation by the group of a formal leader, with explicit rights and responsibilities for facilitating the group. The second was "statistical feedback:" the aggregation and tabular display of voting results on the group decision problem. We observed a strong interaction between these two structures. Either the human leader or the statistical feedback, alone, improved consensus, but together they did not. Statistical feedback in the absence of a human leader was detrimental to a group decision which exemplified "collective intelligence." (Hiltz, Turoff and Johnson, 1982, 1991).

The third experiment used groups of executives who were attending a training course, as employees of an organization with a strong corporate culture. It compared face-to-face groups, computerized conferencing groups for which the participants names were identified, and conferencing groups for which pen names were used. The choice dilemma tasks used could show "risky" or "conservative" shifts in the willingness of the participants to take a risk in order to attain a desirable objective. Contrary to some previous experiments by others employing students as subjects, the use of pen names (a form of anonymity) did not result in any great amount of "flaming" or disinhibition among the corporate users. Pen named conferences showed consistent, but statistically insignificant tendencies toward less disagreement about the final group choice, more participation, and greater equality of participation. The final group choices were significantly more conservative in the pen name computer conferences, within the conservative corporate subculture (Hiltz, Turoff and Johnson, 1985, 1989).

GO TO START


EDUCATIONAL APPLICATIONS: THE VIRTUAL CLASSROOM

One type of work that can be done more effectively in teams than alone is learning. A long-term project of the Computerized Conferencing Center has been the design and evaluation of software and appropriate pedagogical procedures to facilitate the delivery of college and high-school level courses on-line. A "Virtual Classroom" [TM] is a teaching and learning environment located within a computer-mediated communication system. Rather than being built of bricks and boards, it consists of a set of group communication and work "spaces" and facilities which are constructed in software.

Prototypes of Virtual Classroom software features were first created and evaluated within EIES 1, and were then incorporated into EIES 2, mainly as "activities." Some of these communication structures resemble facilities or procedures used in traditional classrooms. Others support forms of interaction which would be difficult or impossible in the "face-to-face" environment (Figure 1). All are accessed not by travelling to a university, but by typing and reading from a personal computer which connects by telephone to a mini or a mainframe computer operating the Virtual Classroom software. Participation is "asynchronous," that is, the Virtual Classroom participants dial in at any time around the clock, and from any location in the world accessible by a reliable telephone system.

The objectives of a Virtual Classroom are:

Collaborative learning is defined as a learning process that emphasizes group or cooperative efforts among faculty and students. It stresses active participation and interaction on the part of both students and instructors. Knowledge is gained through an active dialogue in which those participants share their ideas and information.

The initial evaluation objectives were to describe the nature of the educational experiences and outcomes in this delivery mode; to compare them to the traditional (physical) classroom; and to determine those conditions associated with good or poor outcomes. In order to explore these questions, it was necessary to observe a variety of courses, students, and implementation environments. The primary research design rested upon matched, but "non-equivalent" sections of the same course taught on-line and in the traditional classroom. Though the same teacher, text and other printed materials, and midterm and final exams were used, the classes are "non-equivalent" because the students were able to self-select delivery mode. We also looked at some "mixed modes" courses, and at some distance education courses which did not have a matched traditional section.

All together, we collected data from a total of 132 students in completely on-line courses, 96 in mixed mode courses, and 89 in traditional or "control" courses. Most of the data used in the study were collected with a pre and post-course questionnaire. However, we also have behavioral data (including grades and SAT scores, when appropriate or available, and amount and type of on-line activity), and qualitative observations and interviews. Detailed descriptions of the software, methodology and findings can be found in the forthcoming book on this project (Hiltz, 1992).

The results of the field trials we have conducted between 1985 and 1991 are generally positive, in terms of supporting the conclusion that the Virtual Classroom mode of delivery can increase access to and the effectiveness of college-level education. The following is a summary of some of the major findings comparing the Virtual Classroom (VC) to the traditional classroom (TC).

Though the "average" results supported the above conclusions, there was a great deal of variation, particularly among courses. Generally, whether or not the above outcomes occur is dependent more on variations among courses than on variations among modes of delivery. The totally on-line upper level courses at NJIT, the courses offered to remote students, and the mixed mode courses were most likely to result in student perceptions of the Virtual Classroom being "better" in any of these senses.

It was hypothesized that those students who experience "group learning" in the Virtual Classroom are most likely to judge the outcomes of on-line courses to be superior to the outcomes of traditional courses. This was supported by both correlational analysis of survey data and qualitative data from individual interviews. Those students who experienced high levels of communication with other students and with their professor (who participated in a "group learning" approach to their coursework) were most likely to judge the outcomes of VC courses to be superior to those of traditionally delivered courses.

GO TO START


DISTRIBUTED GROUP SUPPORT SYSTEMS & COLLABORATIVE SYSTEMS

Our current efforts focus on the asynchronous or "distributed" group process, whether this be decision-making or the production of some sort of group product. This project, funded primarily by the National Science Foundation, is an integrated program of theory building, software tool development and assessment, and controlled experiments in an area of collaboration technology which we call "Distributed Group Support Systems." Such a system embeds Group Decision Support System (GDSS) tools and procedures within a computer-mediated communication system.

"Distributed" has several meanings: temporal, spatial and technological. The central interest of the program of study is "asynchronous" groups, in which interaction is distributed in time as well as in space. The group members use the system to work together to reach a decision or complete their cooperative work over a period of time, with each person working at whatever time and place is convenient. Though we may use more than one computer system for comparison purposes, the chief vehicle for the research is EIES 2. Distributed systems raise special problems of social and technical coordination. The iterative program of research will seek to identify and solve some of these problems.

The primary objective of the project is to build a general theory, supported by empirical evidence, which will enable us to understand how characteristics of the communications mode, structuring of the group process, and software tools interact with characteristics of the group and its task to affect the process and outcome of decision making. The interdisciplinary, multi-institutional effort will build upon previous GDSS research in "decision rooms," particularly at Minnesota. It aims to significantly extend theory, software tools, and empirical evidence by focussing on the asynchronous environment and comparing it to other conditions for computer-mediated group support. Among those actively involved in the project are Scott Poole of Minnesota and Ronald Rice of Rutgers University.

Building a Theoretical Framework

The first task of the project is to arrive at a theoretical framework which specifies the main variables to be measured and/or manipulated. This theoretical framework, with its associated empirical measures of the variables, will guide the program of research, and will enable us to compare the results of different experiments conducted within the project, and to contrast them to the results of other experiments which have used some of the same variables, particularly at Minnesota.

Our theoretical model started with an expansion of the model proposed by DeSanctis and Gallupe (1987). This asserts that the types of GDSS tools and procedures which will be beneficial are conditional upon three variables: communications mode, group size, and task type. The task type classification was developed by McGrath (1984). Two modes of communication were present in the Minnesota framework: face-to-face (decision room) and dispersed (two or more linked decision rooms). The framework will be expanded and refined as our empirical results and theoretical understanding progress.

Our expanded theoretical framework first adds a third communications mode, asynchronous. Asynchronous use of GDSS tools and processes is in practice a "different" mode of communication than same-time use. It leads to different communications behavior (such as the tendency towards much longer entries by participants) and to some unique coordination problems (such as what to do about "absent" members who are lagging behind the group). Members meeting at the same time in decision rooms can coordinate their use of the technology in a much more "natural" or "intuitive" way than those whose only channel of communication is via computer. Along with the provision of the tools themselves in a dispersed or distributed environment, there must be facilities for meta-communication about when and how the tools will be used, and/or for automatic sequencing of the order of use of a set of tools, based upon pre-set criteria.

The effect of a particular group support system on group coordination and decision making can be understood on a theoretical basis only by examining how its use tends to change the process of group communication, and through changes in process, leads to differences in outcomes. The "system" is a particular combination of communication medium (same place vs. dispersed, synchronous vs. asynchronous), GDSS tools, and structuring of process via a facilitator or instructions/agenda, plus the hardware which presents the system to the user.

The theoretical framework which will guide the research is shown in Figure 2. We have also developed an initial set of research instruments (questionnaires and planned procedures for measuring process and outcome). These instruments and procedures will be further refined as a result of pilot tests for the first few experiments. Methodologies suitable for the study of decision rooms will need to be modified in order to be appropriate for the study of distributed systems.

The Experimental Program

The actual experiments in the project are designed and carried out by Ph.D. students, under the direction of the Principal Investigator and other professionals. The Ph.D. students are recruited primarily from the joint NJIT/Rutgers program in management of computer systems. Interested students first enroll officially in a seminar, where they develop their ideas for a proposal and complete a first draft of the literature review and plans for research. Subsequently, participating students continue to attend the seminar as they further develop their final proposal, begin software development, and conduct pilot studies and then the experiments and analysis. Their progress and problems are shared with and discussed by the members of the seminar, as well as with their primary dissertation advisor.

Students first select a task type. Note that besides the classification based upon McGrath, we are also interested in variations along the dimensions of difficulty and equivocality. They then propose a set of tools or processes which are hypothesized to improve the performance of distributed groups undertaking that type of task. The tools are programmed in Smalltalk on EIES 2. The students do some of their own coding, with the advice and assistance of James Whitescarver, the EIES 2 project director at NJIT.

In choosing the experimental design as part of the proposal, one of the variables is generally the use or non-use of the special software tools developed. The student then picks one or two other "independent variables" for manipulation. The most usual design will be a 2 x 2 factorial. Those independent variables not manipulated are either controlled (held constant, e.g., group size), or measured to be used as co-variates. One or more pilot studies are undertaken for each experiment, to test the tools, procedures, instructions, and measurement instruments.

Thus far, four students have decided to join the project, and are in the process of writing their proposals. The first experiment, to be conducted by Donna Dufner, will replicate a task (an allocation task called "the foundation") and set of procedures used at Minnesota in several studies. Software coding is almost completed for several tools which replicate some of the functionality in the SAMM system used at Minnesota: the ability of a group to build a common list of options, and then to apply several possible voting scales to the items on that list (yes-no, vote for one, or ratings on a scale from 1 to 100). Besides the availability or non-availability of these tools, the second variable to be manipulated for Dufner's experiment is a sequential vs. "asynchronous" agenda or procedure. The sequential groups will be required to all complete each step in the process provided (define the problem, define criteria for a good solution, generate possible solutions, arrive at agreement on a solution) before going on to the next step. The "asynchronous agenda" groups will be permitted to work on any step of the process at any time, with the instruction that they must complete all steps before their task is "completed." In both conditions, groups will be given one week to complete their task. Two sets of pilot studies have been completed for this experiment (Hiltz et. al., 1991; Dufner and Hiltz, 1991).

Also scheduled to be run shortly as part of this project is an experiment by Ajaz Rana. The task will be peer review (ratings and selection of articles or proposals). A polling activity has been constructed to support this experiment, in which any Likert-type, semantic differential, or nominal scale can be constructed and grouped into an on-line poll to support the group in its efforts to select the "best" article or proposal from a number of options, and to identify the strengths and weaknesses of each of the candidates. Group decisions can be compared to expert opinion as a criterion for the "correctness" of the decision made by the group.

A recently completed thesis dealt with the development of a knowledge structure to support collaborative Hypertext (Rao and Turoff, 1990; Turoff, Rao, and Hiltz, 1991; Rao, 1991). Such a framework is intented to allow groups dealing with a complex body of knowledge to contribute their views and information to the group. The existence of a general structure would avoid the problem of having a different system for every type of application area.

GO TO START


FUTURE RESEARCH DIRECTIONS

The research program is tied to both a Ph.D. in Management offered jointly with Rutgers (with a concentration in Information Systems) and a Ph.D. in Computer Science. As a result, we expect a continued balance in research between the development of the technology and the evaluation of applications. Systematic collection of data from users will be the basis for improvements in both functionality and usability.

With respect to the nature of the applications, the following is a reasonable projection for the next three years because it reflects already funded research projects and thesis work that is currently underway or beginning.

The above are the areas that will lead to increased functionality of the EIES 2 system in the next two to three years. As in the past, what happens after that will be greatly influenced by "demand and supply." The "demands" are the priorities expressed by the EIES 2 user community for new features, and the research questions we are interested in pursuing. The "supply" is the availability of funding (through contracts, grants, or sales revenues for the software) to pay for specific research and enhancements.

GO TO START


Back to homepage of Murray Turoff
Back to homepage of Roxanne Hiltz