New Jersey Institute of Technology
University Heights - Newark, NJ 07102-1982

Working Paper: A condensed version of this paper will appear in HICSS 1999
email: turoff@vc.njit.edu
homepage: http://eies.njit.edu/~turoff/

Ó 1998 Murray Turoff

Keywords:
Computer Mediated Communication, Groupware, Hypertext, Hypermedia, Discourse Structures, Group Communication, Visualization, GDSS, Expert Systems, Organizational Memory.

Abstract:
Using application oriented conceptual maps to categorize the group discussion would be an advancement in the design of CMC systems to allow much larger groups to collaborate productively.  The group meta communication process should allow the group to modify and evolve these conceptual discourse templates.

Introduction
The goal of a collaborative discourse structure is to provide a template for the group discussion so that the majority of the discussion can be captured and categorized.  Such a structure would incorporate functionality to allow a group of experts to thoroughly explore and analyze a problem domain by following a discourse structure they could design, maintain and evolve as the knowledge structure for that particular domain.  This might be viewed as a collaborative expert system where the experts maintain and evolve the system for their benefit and for the benefit of practitioners and future practitioners (students).

At the very beginning we must clarify that our use of the term "discourse structure" does not refer to the transaction analysis view common in such discourse models as "speech act theory" or the syntactic grammar models (Novak & Glowin, 1998).  Rather, the concept of a discourse structure, as we use it, is defined as a template for a discussion structure which allows individuals to classify their contributions to the discussion into meaningful categories that structure their relevance and significance according to the nature of the topic, the objective of the discussion, and the characteristics of the group (Turoff, 1991).  This follows in the tradition of the work of Zwicky (1957, 1969) in morphological analysis and the related work in such areas as Inquiry Systems (Churchman, 1971) and the Delphi Method (Linstone & Turoff, 1975; Turoff & Hiltz, 1995).  Delphi was developed as a paper and pencil communication system to allow large groups of knowledgeable individuals to collaboratively examine a complex problem.  They are executed by small design and facilitation teams and successful Delphi exercises of 200-300 individuals have been demonstrated.

While there is a lot of excitement over a higher capacity Internet we are still faced with the problem that it is not yet possible for large collaborative groups to work together as teams effectively in "serious" projects.  Just about all current use is limited to less than a hundred individuals, or "information overload" sets in (Hiltz & Turoff, 1985).  Examples of group communication systems that did allowed a couple of hundred people to work together on the same project were EMISARI (Hiltz & Turoff, 1978/1993) and TOPICS (Turoff et. al., 1993). EMISARI was used to monitor the Wage Price Freeze in 1971 and TOPICS was used as a nationwide system in the late 70’s and early 80’s to allow state legislative science advisors and professional society advisors to do unpredictable information exchange.

Both systems had very specialized structures for group communications, very specific content type classifications and relationships related to the application domains (crisis management and unpredictable information exchange), specific human roles supported by software, and voting capabilities to be able to expose quickly and efficiently areas of agreement and disagreement.  No one wants two hundred replies to a comment that basically say "I agree" or "I disagree," or long arguments when there is no real need for them.  Both systems allowed the information coming from the collaborators to flow into the appropriate categories based upon the responsibilities assigned to the members or the type of action they were responsible for taking.  A very successful system in industry in the 1980’s that followed the same approach was the EQUAL system in IBM designed by David Morris.  Unfortunately it was never described in the academic literature or used outside of IBM.  For the purposes of this paper the key support areas of multiple human roles and privileges supported by software and voting tools will be implicit as the paper will be concentrating on content structures for discourse.  However, all three methods are needed to allow large groups to work together as teams.

Our concept of collaboration entails a situation where everyone is a potential equal contributor to a discussion transcript that becomes important as a memory for that group.  Ideally such a transcript can evolve to become a knowledge base for the collaborators and those who use the results of the discussions.  We still do not know a great deal about how to provide the software structures and tools that large online communities of knowledge workers will find truly useful in assembling, making sense of, and working with very large collections of ideas and multi-media information.  As pointed out by Nunamaker (1997, p. 368), little research has been done on some key issues such as: "How can teams make sense of the vast quantities of information now available to them on-line as they work?"

Over the last few decades, various computer-mediated communication structures and tools have been created that allow "virtual communities" to work together online, not only communicating about specific activities, but also building a kind of knowledge base.  In "Structuring Computer-Mediated Communication to Avoid Information Overload," Hiltz and Turoff (1985, 1992) described a number of fundamental structures and procedures "beyond email" that can help groups to contribute, organize and share information while working together via computer networks for months to years.  These include segmentation into separate group communication structures we call "conferences" organized by topic, with a branch or threading structure for such computer-organized exchanges, in which all replies to a topic are kept in a linked structure.  Simple text-based structures also include message summarizing, filtering, and indexing procedures, such as key words, simple search engines, voting, scanning and summarization features, automated reminders, simple hyperlinks such as those supported by today’s web browsers, etc.

In our early evaluated field trials of CMC systems with scientific research communities, the ways in which software structured tools enabled groups of researchers and practitioners to enhance their productivity were documented (Hiltz, 1984; Turoff et. al., 1993).  However, typically groups were sized from 10 to 100 active participants.  Active collaboration among larger groups in the 100 to 1000 range is the challenge for future group oriented systems.

The hierarchical classification system for comments (i.e., the content independent meta "comment and reply to any level" structure) breaks down because the classification of comments into what is, in essence, a hierarchical index is impossible to keep consistent when done collaboratively by a large group.  The TOPICS system, for example, employed a professional indexer and editor with software supported privileges to maintain consistency in this area for the group as a whole.  Furthermore with no problem content discourse structure for guidance people end up putting multiple subjects in a single comment.  Anyone using the DOS file structure for a large number of text files should be keenly aware of the problem, even for individual own consistency.

Our recent work has been with a related kind of online intellectual community: collaborative learning networks of "teachers and students" (used in quotes because in collaborative learning, the roles are somewhat interchangeable), and the human and information resources they need to conduct entire degree programs online.  We are currently in the third, web-based version of designing and studying "Virtual Classroom" [TM] structures (Hiltz, 1986, 1994; Harasim, Hiltz, Teles & Turoff, 1995; Turoff and Hiltz, 1995; Hiltz & Wellman, 1997). However, we have come up against the same kind of size limitations with the current generation of tools and structures: they are fine for medium sized groups, but break down when very large groups (e.g., class sizes over 50) try to use them for extended and intensive information gathering, discussion and analysis. The use of CMC for classes tends to result in equal participation forced both by the instructor and the software structures used. Fifty students entering 3 comments a week (a typical minimum) is 150 comments shared every week.  Working project groups (e.g. developing software requirements) also tend to foster equal participation and even higher average contribution rates.  By contrast, news groups and message lists tend to exhibit a logarithmic distribution of participation, where only a relatively small number of participants are continually active.

Basic CMC Discourse Structures
Our premise is that the current state of the art of Computer Mediated Communications is limited mainly by the fairly primitive discourse structures underlying most current asynchronous conference systems (Turoff, 1991; Turoff & Hiltz,1995).  Some of the ideas presented in this paper have been implemented in prototype systems or are currently being developed; however, most of them are still just concepts awaiting the needed resources for development and evaluation.

The discussion or conference part of a CMC system may be represented as a specific Hypertext structure.  Most of them today can be described in the following table of relationships (links) between the objects (nodes) that characterize the system.  It is important to note that objects in CMC systems are both the elements of discourse, and the individuals or members who are objects linked to content objects by privileges associated with their human roles.
 

There is significant other functionality possible with respect to such objects as the different roles and associated software powers a person can have in a given conference. However, the basic discourse structure is usually some combination of: temporal occurrence, comment/reply hierarchies, and key word association of comments.

Conceptual Structures
The core concept we are presenting is the need for a group to evolve their conceptual map or discourse structure as a collaborative undertaking, and to be able to modify it as an integral part of the discussion process.  This resulting non linear template would be used to categorize discussion comments.  Conceptual diagrams can be built collaboratively within a group support environment, giving groups a visual representation of the concept so they can discuss it and agree upon its structure and properties.  Similarly, individuals can author and modify the concept diagram to represent and better understand a concept.  People then will use finished concept diagrams as the (perhaps officially published) guiding structure for a topic, through which they interact with that concept.  They can browse, query and access details and instances of the concept from the previously defined visual representation.

Some prior hypermedia systems do provide semantically-typed constructs, which help users maintain context and orientation, thereby helping them maintain a mental model of the domain and where they currently are in it.  MacWeb uses semantically-typed constructs to support knowledge acquisition, model building and operationalizing conceptual models (Nanard & Nanard, 1995).  SEPIA uses semantically-typed constructs to support face-to-face meetings (Streitz, et al., 1994) and to support on-line collaborative discourse (Thuering, et al., 1995).  RICH (Reusable Intelligent Collaborative Hypermedia) is a collaborative authoring environment that uses semantically-typed constructs both to control the collaboration and to guide the authoring process (Wang & Rada, 1995).  Max uses semantically-typed constructs to help analysts distinguish the various operations and meta-information available for objects in a decision support domain (Bieber, 1995).  However, most earlier efforts are single semantic structures and do not provide for user tailoring of the semantic constructs.  For the most part they do not address the necessity for roles and privileges in collaborative efforts.

There has been significant research in the use of semantic and conceptual structures in Hypertext for both individual decision support and for collaboration. Argumentation and discourse systems such as Aquanet (Marshall, et al.1991), gIBIS (Conklin & Begeman, 1989), SEPIA (Streitz, et al., 1992), Virtual Notebook (Shipman, et al. 1989) and Design Intent (Girgensohn, et al., 1996) employ shared views to allow groups to develop shared understandings through semantic hypertext representations.  In addition to these systems, collaborative hypertext authoring environments allow authors to construct hypertexts for themselves or for others.  These systems include ABC (Smith & Smith, 1991) and RICH (Wang & Rada, 1995), systems for technical documentation (Selber, et al., 1995), KMap, which allows groups to develop concept maps (Gains & Shaw, 1995), and the educational software environments envisioned in CHEF (Collaborative Hypermedia Educational Framework, Rana & Bieber, 1997).  Watters et al. (1998a, b) allows students to build concept maps to explain and foster discussions around topics in a political science class.

Recent work on the usability of complex web sites (Shneiderman, 1997) indicates the need to "structure information objects (e.g., hierarchies and networks)" as one of the main four tasks to increase comprehension ability on the part of the viewer.  In other words, maps of the site can be used as a guide to visualizing and understanding the site.  Behind this recommendation is the cognitive overhead caused by a lack of spatial and temporal context for the web based information (Utting and Yankelovich, 1989).  The same observations were the reason why most Delphi Designs (Linstone & Turoff, 1975) were based upon spatial conceptual layouts of the topic being examined.

We view hypertext and conceptual maps as a way to create and integrate knowledge, not just to retrieve it.  We also feel it is the group, collectively, that must be able to tailor and evolve the conceptual map being used.  The clientele is not any possible user but individuals who are knowledgeable or experts in the application domain.  As part of a CMC Group System, graphical representations have the potential to give people a more rapid understanding of a complex concept than a textual or tabular description (Tufte, 1997).  Graphical overviews and maps can also anchor and orient people within a complex topic.  If an expert group creates and evolves the representation, it should be no problem for them to learn and master complex conceptual maps.  If a non expert group is using a map developed by the experts, then they are learning and no one assumes that this is "easy" for complex problem domains.  In this case a strong role of educator or facilitator has to be explicit in the software.

Furthermore, there appears to be a direct relationship between the complexity of the conceptual maps and the degree of actual expertise in the problem domain by those creating the map (Hopkins, et. al. 1987).  We envision that our concept structure and visualization methods can be the major high-level interface for representing and working within a complex concept, research field or organization.  It can be used by groups to discuss, build consensus about, and then show the resolution of a topic of importance.

Example Discourse Templates
An example that applies to any topic (e.g. a meta discussion structure independent of problem domain) is the process of scientific debate (e.g. Hegelian Inquiry Process, Churchman, 1971) which has also been used in policy studies via the Delphi Method (Linstone & Turoff, 1975) and for software requirement formulation (Conklin & Begeman, 1989; Fjermestad, 1995; Schweiger, 1989).  The first implementation of this structure in a group communication system was the Delphi Policy Conference (Turoff, 1970).

Such a group communication activity can be specified by a semantic hypertext structure as shown in Table 2 and Figure 1. In a given situation an option could be almost anything: actions, goals, criteria, requirements, solutions, decisions, etc. In essence these are structures to organize a constructive debate about a topic and the results sought are collective group insights into such things as alternative desirable resolutions and feasible actions to take.

Table 2: Argumentation Template Relationships (link types in table and nodes define rows and columns)
 

from\to link	Argument	Option
Argument	Opposing	pro or con
Option		Alternative
Voting Scales	Importance Validity	Desirability Feasibility

In a typical argument a member can enter a proposition or alternative which in different applications can be such things as actions, goals, solutions, decisions, etc..  Any other member can enter either a pro or con argument associated with one or more of the proposition nodes, with either pro or con links. Certain arguments might be further linked together by being in opposition to one another.  The monitor of the discussion (in some implementations) can choose if the entries are anonymous, with pen names, with real names, or the choice of the individual writers.  Part of the writer's task of creating an argument is indicating which items should have pro/con or alternative links to the one he or she is creating.  Since others might disagree with some of the link choices, we the need human roles and/or voting tools to resolve such disagreements.  Anyone can vote for the degree of desirability and feasibility of the resolution and relative importance and validity of each argument.  These are the only things that a member of the collaborative group is allowed to do in the part of the discussion governed by this template.
 

One may think of such a limited conceptual map as one of a number of domain independent general meta discussion structures that will also be available as tools to link to the appropriate items in a domain dependent conceptual map.

While the debating/argumentation structure seems rather simple and straightforward, consider a very common planning structure used in many successful corporate planning Delphi exercises. One starts with a trend which could be highly quantitative, such as the amount of a product’s sales over the past five years or the number of terrorist actions yearly in the U.S.  The participants are asked to make a forecast for the trend and to indicate the assumptions they are making about the future that will influence the trend.  They are also asked to express any uncertainties (things they don’t think will occur but which would change their projection if they did).  All these are taken as potential assumptions that the group votes on for degree of validity.

The validity vote is used to distinguish all assumptions into five basic categories: Very Likely, Likely, Uncertain, Unlikely, Very Unlikely.  It is the uncertain ones that are focused upon to distinguish between those that can be controlled by actions the organization can take and those that are not.  The group then proposes actions to influence the controllable assumptions and measures or future observations that will determine the occurrence of the other assumptions (Figure 2).  Note all node types usually have self links (Table 3) that are utilized as well; for example, assumptions fall into similar categories (e.g., economic, technology) which results in "membership links" to categories for organizing a large list of assumptions.
 

Table 3: Forecasting/planning Delphi Relationships
 

from/to link	Trend	Assumption	Actions	Measures
Trend	correlated
Assumptions	influences	membership in similar category
Actions		controllable influence	dependency & arguments template
Measures		uncontrollable observation	observation of outcomes	related to
Voting Scales	Significance	Validity	Desirability	Significance

Note that these action nodes can be linked to argument structure as the option node and we now have a more complex combined structure.  There are many such discourse structures (Linstone & Turoff, 1975) that can combine to become more complex structures and as such represent a potential toolkit for collaborative Hypertext.  We note the debating structure (Figure 1) links quite naturally to the problem of deciding which action to choose.  The total communication process in this Delphi for planning is the combination of the two structures.  Instead of the sequential nature of carrying on this process in a face to face discussion the advantage of using this template in an asynchronous CMC environment is that any member can bring up any idea or thought dealing with any aspect of the problem at any time and participate in a much more parallel form of group interaction.  Such a group process frees individuals from the group temporal regulation of their activities and problem solving process.  The computer can now impose a meaningful organization on the contributions that makes it clear what has transpired since a given member last interacted.

When one gets into a specific technical area the resulting template can become extremely complex. An example, as shown in Figure 3, is one we have evolved for the objects that must be described in a typical interface design (Balasubramanian & Turoff, 1995, 1998).  This has a minimum of 17 nodes and considerably more link relationships even without the added categories possible by linking in the software engineering view of the user requirement specifications.  We have developed this template as a tool for teaching interface design, as a tool for aiding the creative part of the design process, as a collaboration tool for designers, and as a mechanism of communication between users and designers.  The latter would require the incorporation of a screen mock up facility that would allow the users to view the final interface.  As a collaborative template one would also incorporate the argumentation template as an attachment possible to any incident of the specified nodes, for anyone who has been involved in design teams knows that even two designers results in a great many differences of opinion.  This template is regularly used as a learning tool for students and has been successfully tested in feasibility trials with professional designers (Balasubramanian, 1998) as an individual support tool.

Clearly this is a template for use by either experts or by students learning to be experts.  Users would see the mockup interface as a way of allowing them entry into details they might need from this lower level structure of the information.  It has been field tested with interface designers who also must invest some time in learning it and relating it to their current models of what they are doing when they design.

Letters on the links in the above diagrams refer to the semantic classification for links in Table 3 and are explained in the referenced papers.  Our objective is to create a capability for this group to establish a discourse structure or blueprint for the nature of their problem solving process and to actually observe the construction of their discussion in a multi-dimensional visualization.  Nodes and links will be objects that have attributes such as the degree of certainty about the concept or relationship and the degree of agreement on meanings (i.e., uncertainty and ambiguity resolution, Daft & Lengel, 1986).  The addition of these types of attributes, as well as such things as the strength or tentativeness (in the mind of the author) of the node or link will allow the expression of social emotional content.  This is necessary for a successful discussion, even more so when users might be contributing comments on usefulness and relative priorities of suggested functionality.

Analysis Support Tools
There are a host of approaches for providing users aids in understanding an evolving discourse oriented collaborative knowledge base.  Tools to analyze complex non linear structures are found in areas such as structural modeling (Lendaris, 1980; Warfield, 1976; Geoffrion, 1987; Anderberg 1973)

Incorporation of general purpose tools is based upon the extent to which we can utilize a fundamental semantic structure to represent knowledge in any field. One example of such a semantic structure is the hypertext knowledge structure based upon the Gilford model of the intellect (Rao & Turoff, 1990; Turoff, Rao, and Hiltz, 1991; Guilford, 1967). That semantic model is represented in the following table:

Guilford hypothesized that human intellectual ability involved the six product types for cognitions, convergent production and divergent production between products. This can be taken to provide the above general knowledge representation in terms of six node types and twelve link types.  For any particular application it is possible to map the semantics of the application into this fundamental model.  For example, the intellectual process of naming (i.e., categorizing items), which is a "collection" node, can be synonymous with gathering, aggregation, set, heading, conglomeration, class, group, etc. (Turoff, et. al. 1991).  The comparable links (cognition productions) between nodes are "membership" (convergent thinking) and "exclusion" (divergent thinking).  Semantic meanings for hypertext nodes and links in any application domain can be mapped to this subset of the Guilford model of the human intellectual processes.

Visualization

Right now one key missing element in asynchronous CMC systems is the appreciation of the evolution of the discussion that occurs in a face to face meeting.  Tools that allow a visualization of that process and of the resultant knowledge base are extremely important for improving understanding of the group process by the members.  Being able to visualize the history of a discussion is like the difference between understanding a painting by observing the finished product, or by seeing the process that created the painting.  The finished product might be satisfactory to novices but not to experts or aspiring experts.  For large groups and for situations where participants may be inactive for periods, or entering the group after it has already been at work for some time, we have the problem of participants seeing only the current state of the discussion without any clear idea of how it got there.  What were the crucial arguments that caused agreement to occur?  Somehow the sequence of the occurrence of certain arguments led to vote or opinion changes on an option, resulting in a consensus.  If one was not present it would take considerable analysis to discover which were the critical arguments.

The voting process is a logical approach to capturing the resulting group dynamics of the discussion and the type of tool to do this could be the following three dimensional visualization.  Let us imagine something akin to a complex organic molecule that, on screen, can be rotated and "zoomed" to focus on different parts and their relationships.  More importantly, the history of this structure can be played back through time.  We are using the argumentation template for the example and there are two types of nodes or atoms (options and arguments); and three types of links or relationships (pro, con, and opposition).  Any member of the group may add to the collaborative construction using these building modules.

The actual contents of a node can be explicit and/or linked to multimedia material stored elsewhere.  There are four collaborative dimensions associated with the material: Completeness, Validity, Significance and Reliability.  The values of these attributes are determined by the voting of the collaborative experts and normalized on a -1 to +1 scale.  To provide greater understanding of the discussion, these could be used to dynamically reorder the spatial dimensions of this material as viewed by the individuals, as illustrated in Figure 4.  For example, each of the four scales is a finite (-1 to +1) interval scale visualized as the sides of two buildings with a shared wall. One building houses arguments and other building houses options.  The third dimension (a wall) is shared and represents the proportion of the eligible votes that have been cast (0 to 1) along the floor and the shared dimensions of validity and feasibility along rising to the ceiling.  Until a voting threshold (sufficient minimum number of votes) is obtained, new options and arguments lie on the ground in the accompanying yard of construction materials.  Links are represented as rubber bands.  One can organize lists based upon links to get linear relationships and views of the discussion or utilize the links to view only various subgraph constructs of the discussion structure.

Note that in three dimensions we can extend the model to four types of nodes and eight unique voting scales.  The movement of discussion atoms to the origin (1 value) in terms of temporal speed and number of votes can be highly informative about the status of a complex discussion for a very large group.

Progress in scientific fields is highly correlated with the invention of instrumentation to detect prior unseen properties.  We truly lack real-time measuring instruments for group processes and it may be one reason why there have not been major changes from the primitive campfire group discussion to the conference table face to face discussion.  The utility of the technology for truly improving group processes will be dependent upon the development of measuring instruments for the group process and the ability of the group to visualize the progress they are making through the use of the instrument.

This might be viewed as a summary of a content analysis; in the past content analysis of group discussions has been a primary tool for understanding the resulting discussion by scientific investigation after the fact (Hiltz, Johnson, & Turoff, 1986).  The presence of the group process in the computer makes possible exposing various dynamic content measures to the group and/or the facilitators (Hiltz, Turoff, & Johnson, 1982).  This is a case where the computer makes possible what could never be possible in normal face to face communications and as a result may one day prove CMC as a more effective group communication process for both expert groups and project teams.

One can log the history of the positions of these nodes and view via animation the actual historical evolution of the discussion.  In most successful group decision support structures (e.g., GDSS, Delphi, Focus, and Nominal Group exercises), it is usual to expect 30-50% of the initial votes to change as part of the group process.  In the above construct one can imagine that the emergence, and rapid movement to the origin, of a particular hypothesis is the result of a small number of evidence nodes that obtain a high level of agreement.  In an animation of the history of the discourse in the above construct this would become a visual sequence of related movements of nodes.

The number of such possible visualizations is extensive.  This one has been based upon judgment scales that have been commonly used in the Delphi process (Linstone & Turoff, 1975).  Certainly experience with real implementations where groups can negotiate and evolve visualizations will serve to improve our understanding of what works best.  The important point is that this type of functionality must be viewed as a toolkit that different types of groups with different applications can adjust to their needs.

Conclusion

To date CMC systems have been largely restricted to the use of meta oriented discussion structures with no specific content structures except for a few isolated historical examples.  On the other hand paper and pencil Delphi exercises were almost exclusively oriented to content specific discourse structures.  As a result many Delphi exercises have been done with more than a hundred respondents.

CMC systems with tailored content oriented discourse and visualization structures can become the foundation to support large scale "electronic community systems." This has been defined as "a computer system that encodes the knowledge of a community and provides an environment that supports manipulation of that knowledge" (Schatz, 1991, p. 88).  Such systems can also support larger communities of learners on a lifetime basis.  For example, a single large scale system could mutually support a whole academic program instead of more limited systems for a single course.

In summary, based upon our past experience with implementing and studying computer-mediated "group support tools" for relatively small collaborative groups, we now believe that the ability to utilize complex discourse and visualization structures that are tailored to the problem domain can ultimately support problem solving and learning communities of scores to thousands of participants (Turoff & Hiltz, 1998).

It is our view that a collaborative discourse structure tailored to a specific area of knowledge can be designed to service three very different communities at the same time.  These are:

1. Investigators and researchers in the field who can modify the underlying conceptual structure that forms the semantic and pragmatic relationships governing the application domain (derived from the theoretical paradigms for the field).
2. Practitioners and others who have job oriented needs for material in the particular knowledge areas. These individuals would not only make use of the information but also be able to contribute empirical data and observations about actual efforts utilizing the knowledge of the application domain.
3. Learners engaged in collaborative learning processes in order to be able to become practitioners in the field.

Future systems must be viewed as toolkits whereby the community of users and its natural specialized groups can evolve the discourse structures, visualizations, voting processes, and human roles appropriate to its nature and its application.

References

Anderberg, M.,
        Cluster Analysis for Applications, Academic Press, 1973.

Balasubramanaian, V.,
        A Systematic Methodology to support the creative idea generation phase of the user interface design process.   Ph.D. Dissertation, Graduate Program in Management, Newark NJ, Rutgers University, January 1998.

Balasubramanaian, V., Turoff, M., Ullman, D.,
        A Systematic Approach to Support the Idea Generation of the User Interface Design Process, HICSS 98, 1998.

Balasubramanian, V. and Turoff, M. (1995)
        A Systematic Approach to User Interface, Design for Hypertext Systems, HICSS 95, Volume III, 241-250.

Bieber, M.,
        On Integrating Hypermedia into Decision support and Other Systems, Decision Support Systems, 14, 1995, 251-267.

Bieber, M., & S. O. Kimbrough,
        On Generalizing the Concept of Hypertext, MIS Quarterly 16(1), 1992, 77-93.

Bieber, M. and S. O. Kimbrough,
        On the Logic of Generalized Hypertext, Decision Support Systems 11, 1994, 241-257.

Blattenberg, R.C., and S.J. Hoch,
        Database Models and Managerial Intuition: 50% Model + 50% Manager, Management Science, (36:8), August 1990, 887-889.

Conklin, J., & Begeman, M. L., 1989,
        gIBIS: A Tool for All Reasons, Journal of the American Society for Information Science 40, 3, 200-213.

Churchman, C.W.,
        The Design of Inquiry System: Basic Concepts of Systems and Organizations, 1971, Basic Books, New York.

Daft, R., & Lengel, R.,
        Organizational Information Requirements, Media Richness and Structural Design, 1986, Manage. Science. 32(5), May, 554-571.

Fjermestad, J., Hiltz, S. R., Turoff, M., et. al.,
        Group strategic decision making: Asynchronous GSS using structured conflict and consensus approaches, HICSS, Vol. IV, 222-231,  IEEE, 1995.

Gains, B., and M. Shaw,
        Concept Maps as Hypermedia Components, International Journal of Human-Computer Studies, 43, 323-361, 1995.

Geoffrion, A. M.,
        "An Introduction to Structured Modeling," Management Science, (33:5), May 1987, 547-588.

Girgensohn, A., A. Lee and K. Schlueter,
        Experiences in Developing Collaborative Applications Using the World Wide Web 'Shell', Hypertext'96 Proceedings, ACM  Press, 1996, 246-255.

Guilford, J. P.,
        The Nature of Human Intelligence, McGraw-Hill, 1967.

Harasim, L., Hiltz, R., Teles, L., & Turoff, M.,
        Learning Networks: A field guide to teaching and learning online, MIT Press, 1995.

Hiltz, S. R.
        The Virtual Classroom: Learning Without Limits Via Computer Networks. Norwood, NJ, Ablex Publishing Corp., 1994.

Hiltz, S. R. 1986,
        "The Virtual Classroom: Using Computer-Mediated Communication for University Teaching." J. of Communication, 36,2 (Spring): 95-104.

Hiltz, S. R. 1984,
        Online Communities: A Case Study of the Office of the Future. Norwood NJ: ABLEX Publishing Corp., Human-Computer Interaction Series.

Hiltz, S. R., Turoff, M.
        Structuring Computer-Mediated Communication Systems to avoid Information Overload, CACM., 28(7), 682-689, July 1985. Reprinted in D. Marca and G. Bock, eds., Groupware: Software for Computer-Supported Cooperative Work, Washington D C, IEEE Computer Society Press, 1992, pp. 384-393.

Hiltz, S. R. & Turoff, M.,
        The Network Nation, Revised Edition. Cambridge, MA: MIT Press, 1993, original edition 1978.

Hiltz, S. R., Johnson, K., & Turoff,
        M., Experiments in group decision making, I. Communication process and outcome in face-to-face vs. Computerized Conferences. Human Communication Research 13 (2), Winter 1986, 225-253.

Hiltz, S. R., Turoff, M., & Johnson, K.,
        Using a computerized conferencing system as a laboratory tool. SIGSOC Bult. 13 (4), 1982, 5-9.

Hiltz, S. R. and Wellman, B.
        Asynchronous Learning Networks as Virtual Communities. Communications of the ACM, Sept. 1997.

Hopkins, R.H., K. B. Cambell and N. S. Peterson,
        Representations of Perceived Relations Among the Properties and Variables of a Complex System, IEEE Transactions on Systems, Man and Cybernetics, (SMC-17:1), Jan/Feb 1987, 52-60.

Linstone, H. & Turoff, M.
        editors: The Delphi Method: Techniques and Applications, Addison Wesley Advanced Book Program, 1975.

Nanard J., and M. Nanard,
        Adding Macroscopic Semantics to Anchors in Knowledge-based Hypertext, International Journal of Human-Computer Studies, 43, 1995, 363-382.

Novak & Gowin,
        Learning How to Learn, Cambridge University Press, 1984.

Nunamaker, J. F.,
        Future research in group support systems: needs, some questions and possible directions, Int. J. Human Computer Studies, 47, (1997), 357-385.

Ocker, R., Hiltz, S. R.,
        Turoff, M. and Fjermestad, J., "The effects of distributed group support and process structuring on software requirements development teams: Results on creativity and quality," J. Management Information Systems, 12, 3 (Winter 1995-1996), 127-153.

Rana A., and Bieber, M.,
        Collaborative Hypermedia Education Frameworks, Proceedings of Thirtieth Annual Hawaii International Conference on System Science (HICSS) Jan. 1997, II, 610-619.

Rana, A. R., Turoff, M., and Hiltz S. R.,
        Task and Technology Interaction (TTI): A Theory of Technological Support for Group Tasks, Proceedings HICSS, 1997.

Rao, U., & Turoff, M., (1990),
        Hypertext Functionality: A Theoretical Framework, Intern. J. of Human-Computer Interaction, 4(2), 333-358.

Schatz, B.
        Building and electronic community system. J. Management Information systems, 8 (1991), 87-107.

Shneiderman, Ben,
        Designing Information-abundant web sites: issues and recommendations, Int. J. Human-Computer Studies (1997) 47, 5-29.

Schweiger, D. M., Sanberg, W.R. and Rechner, P. L.,
        Experimental Effects of Dialectical Inquiry, Devils Advocacy, and Consensus Approaches to Strategic Decision Making, Academy of Management Journal, 32, 4, 1989, 745-772.

Selber, S., D. McGavin, W. Klein and J. Johnson-Eilola,
        Issues in Hypertext-Supported Collaborative Writing, Chapter 11 in Nonacademic Writing: Social Theory and Technology, A. Dvin and C. Hausen (Eds.), Lawrence Erlbaum Press, 1995, 257-280.

Shipman, F., R. Chaney and T. Gorry,
        Distributed Hypertext for Collaborative Research: The Virtual Notebook System, Hypertext '89 Proceedings, ACM Press, 1989, 129-135.

Smith, J., and F. Smith,
        ABC: A Hypermedia System for Artifact-Based Collaboration, Hypertext'91 Proceedings, ACM Press, 1991, 161-177.

Streitz, J. Geissler, J. Haake and J. Hol,
        DOLPHIN: Integrated Meeting Support Across Live Boards, Local and Remote Desktop Environments, Proceedings of the ACM Conference on Computer-Supported Cooperative Work (CSCW'94), 1994, 345-358.

Thuering, Manfred, Joerg Hannemann and Joerg Haake,
        Hypermedia and Cognition: Designing for Comprehension, Communications of the ACM 38(8), 1995, 57-69.

Tufte, Edward R.
        Visual Explanations: Images and Quantities, Evidence and Narrative, Graphics Press, Cheshire Connecticut, 1997.

Turoff, M.,
        Virtuality, invited paper for special section of CACM, Volume 40, Number 9, September 1997, pp. 38-43.

Turoff, M..
        Computer-Mediated Communication Requirements for Group Support, Journal of Organizational Computing, 1(1), 85-113, 1991.

Turoff, Murray, (1970)
        Delphi Conferencing: Computer Based Conferencing with Anonymity, Journal of Technological Forecasting and Social Change 3(2), 159-204.

Turoff, Murray, (1970)
        The Design of a Policy Delphi, Journal of Technological Forecasting and Social Change, 2(2).

Turoff, M. & Hiltz,
        S. R., Supperconnectivity, CACM, July 1998.

Turoff, M., J. Fjermestad, A. Rana, M. Bieber, R. Hiltz,
        Collaborative Hypermedia in Virtual Reality Systems, Proceedings of the third Americas Conference on Information Systems, August, 1997.

Turoff, M., & S. R. Hiltz, (1995a)
        Software Design and the Future of the Virtual Classroom, Journal of Information Technology for Teacher Education, Volume 4, Number 2, 1995, 197-215.

Turoff, M. and S. R. Hiltz, (1995b),
        Computer Based Delphi Processes, in Michael Adler and Erio Ziglio, editors., Gazing Into the Oracle: The Delphi Method and Its Application to Social Policy and Public Health, London, Kingsley Publishers, pp. 56-88.

Turoff, M., S. R. Hiltz, A. N. F. Bahgat, and A. Rana.
        Distributed Group Support Systems, MIS Quarterly; December 1993, 399-417.

Turoff, M., Rao, Usha, and Hiltz, Starr R.. (1991)
        Collaborative Hypertext in Computer Mediated Communications, HICSS 91, January.

Utting, K. and Yankleovich, N.,
        Context and orientation in hypermedia networks, Transactions on Information Systems, 7, 1, (1989), 58-84.

Vennix, Jac A.M., Jan W. Gubbels, Doeke Post, and Henk J. Poppen,
        A Structured approach to Knowledge Elicitation in Conceptual Model Building, System Dynamics Review, (6:2), Summer 1990, 31-45.

Vennix, Jac A. M.,
        Mental Models and Computer Models, University of Nijmegen Publication, ISBN 90-9003318-1, 1990.

Wang W. and R. Rada, 1995,
        Experiences with Semantic Net Based Hypermedia, International Journal of Human-Computer Studies, 43, 419-439.

Warfield, J. N.,
        Toward interpretation of complex structural models, IEEE Transactions on Systems, Man and Cybernetics, (SMC-4), 1974, 405-417.

Warfield, J. N.,
        Societal Systems: Planning, Policy, and Complexity, John Wiley Publishers, 1976

Watters C., Shepherd, Michael, Alexander, C. and Conley, M.,
        Designing the Digital Agora: Pushing the Web Envelope, 31^st HICSS Proceedings, 1998.

Watters, C., M. Conley and C. Alexander,
        "The Digital Aurora: Using Technology for Learning in the Social Sciences," Communications of the ACM 41(1), 1998, 50-57.

Zwicky, Fritz,
        Morphological Astronomy (Berlin: Springer-Verlag), 1957.

Zwicky, Fritz,
        Discovery, Invention, Research (New York: Macmillan), 1969.

Acknowledgments:

Work on Discourse structures in CMC at NJIT has been supported by the Alfred P. Sloan Foundation, the Center for Multimedia Research at NJIT (through a grant from the New Jersey Commission on Science and Technology), and the National Science Foundation (NSF-IRI-9015236). The opinions expressed in this paper are solely those of the author.