Figure 1 - Pre-Authoring Module
Robinson Vida Noronha1,2
Carlos Villalba3
Alexander Romiszowski3
Clovis Torres Fernandes 1
1Centro Federal de Educação Tecnológica do Paraná, Curitiba, Brazil;
2Instituto Tecnológico de Aeronáutica, São José dos Campos, Brazil;
3Syracuse University, School of Education, Syracuse, NY, USA.
Resumo.
Este trabalho aborda a arquitetura e as ferramentas de um ambiente
inteligente de autoria, que apoia o trabalho de professores-autores no desenvolvimento
de exercicios e materiais instrucionais usando a metodologia de “Structural
Communication” (Comunicação Estrutural). Comunicação Estrutural (CE) é um método
pedagógico, implementado online, capaz de avaliar e desenvolver sofisticadas
habilidades intelectuais de pensamento crítico e complexas estruturas conceituais
dos estudantes. O presente trabalho descreve as bases teóricas da metodologia
de CE e as pesquisas sobre seu uso na educação. Apresenta argumentos para sua
maior utilização e identifica as dificuldades mais frequentes encontradas no
processo de desenvolvimento de exercícios de CE. A segunda parte do trabalho
descreve um conjunto de ferramentas para apoio desse processo de design e desenvolvimento.
O ambiente, baseado em sistemas computadorizados de Inteligência Artificial,
utiliza as técnicas de ATS (Automatic Text Summarization) para ajudar os professores
no processo de autoria e CA (Clusters Algorithms) para ajudar na identificação
das estratégias do estudante e geração de feedback apropriado. O ambiente permite
que diversos autores de exercícios de CE compartilhem sua experiência com outros
professores e especialistas, criando assim um ambiente colaborativo de trabalho
e aprendizagem para os próprios professores.
Abstract.
This paper describes the architecture of an intelligent authoring environment
and tool set that supports the work of teachers-authors who wish to develop
instruction using the pedagogical technique called Structural Communication.
Structural Communication (SC) is a pedagogical technique that can evaluate and
develop a participant’s sophisticated cognitive structures, intellectual skills
and critical thinking skills. The first part of the paper outlines the bases
of the SC methodology, reviews the research on its use in education, argues
for greater use of the method and identifies difficulties faced by many would-be
authors in the development of SC exercises. The second part of the paper describes
current work on the design and development of a tool set to assist in this process.
The computer environment uses the AI techniques of Automatic Text Summarization
to help teachers in the authoring process and Clusters Algorithms to help to
identify the main strategies and generate the appropriate feedback. The environment
permits the authors of SC exercises to share their expertise with other teachers
and domain experts, thus acting as a collaborative design and learning environment
for the teachers.
1. Introduction
The current interest and increased activity in distance education (DE) is largely driven by increased availability and decreasing costs of the new information and communication technology (ICT) infrastructures that make it possible to offer E-Learning on both a national and international scale. The boom in E-learning is growing to the proportions of the boom in E-Commerce that preceded the bursting of that bubble in the year 2000. Many are now questioning whether E-Learning may also be on a boom-to-burst trajectory.
One relevant issue is faculty workload. An extensive and detailed account of two years of E-learning in a university context is presented in a report on a study performed at Syracuse University (Doughty, Spector & Yonai, 2003). This study focuses on the factors that impact the cost-effectiveness and sustainability of online university courses. The results show that whilst students work just a little more time in online versions of a course as compared to the conventional face-to-face versions of the same course, the teacher workload is reported to be more than double across whole semesters and programs. This raises the question of whether online learning, as a regular and mainstream course delivery alternative, is in fact sustainable over the longer term.
The present paper reviews work currently being performed by the authors, in Brazil and the USA, on the development of knowledge-based authoring tools and environments for the preparation of a form of self-instruction exercise known as Structural Communication (SC), which is based on the concept of shared human-machine intelligence in the presentation and conduct of learning activities in complex subject matter domains and with objectives that are more process-related (e.g. critical thinking skills application and development) than product-related (e.g. the mastery of specific content). Before we proceed to the main topic of this paper, we briefly review the background to the research that has preceded the current project.
For over a decade (1985-1996), research was conducted at Syracuse University on the methods, media and tools for online learning that might best promote and support the development of critical thinking skills. One aspect of this agenda was the investigation of technological solutions that might provide students with the essential deep conversational experience without any increase (indeed with a decrease) of involvement of the teacher in the process. Quite simply, the teacher cannot "converse" with thousands (even dozens) of students at one time in an educationally effective manner - and we do not have the resources to employ the thousands of gifted teachers that are necessary in order to replicate small-group discussion activities for all students with adequate frequency. Therefore, we should research how to automate some of the functions of the skilled and gifted group discussion (or “conversation”) facilitator.
Our earlier research focused on ways in which small group problem-solving activities could be in some way "multiplied" through the use of currently available technology. We focused specifically on the Case-Study method and turned to online group learning environments as the way to implement such exercises. Details of the studies, illustrating the evolution of our thinking and the practical methods we employed may be found elsewhere (e.g. Romiszowski, 1990, 1994; Romiszowski & Chang, 1992; Chang, 1994); Romiszowski & Villaba, 1998). It is sufficient to say here that our researches demonstrated that it was possible to achieve similar, or indeed superior, learning results (in terms of critical thinking skills development) from online study and discussion of case material (like for example the Harvard Business Cases), as was typically achieved in small-group class sessions led by skilled and experienced facilitators. Furthermore, the online case discussions were so instrumented as to function effectively with minimal involvement of teachers or facilitators. This instrumentation was based on the adaptation of a methodology named Structural Communication (SC) that was initially proposed in the UK in the 1960’s as an alternative to the then-popular programmed instruction methodologies, but designed for subject matter and learning situations that demanded critical analysis and discussion of alternatives as opposed to giving correct responses to predetermined, well structured problems. A full description and practical examples of the SC methodology may be found elsewhere (e.g.: Hodgson & Dill, 1971; Hodgson, 1972; Egan, 1976; Romiszowski, 1986; Slee & Pusch, 1997). The following section presents a brief overview of the key aspects of SC and the issues involved in developing SC exercises.
2. Structural Communication
2.1 Structure.
Structural Communication (SC) is a pedagogical technique that can provide an objective snapshot of a participant’s sophisticated intellectual skills and cognitive structures. It individualizes learning, controls the process by which the students moves through the lessons, faces him with challenges to construct his own multifaceted responses to complex open-ended problems, analyses these responses and provides complex, multifaceted, feedback on all relevant issues revealed by his answer. In contrast to other instructional techniques, Structural Communication doesn't value the simple reproduction of acquired knowledge on the part of the learner, but rather seeks to develop a deep understanding of the structure as well as the content of a complex knowledge domain.
The Structural Communication methodology involves the development of special units of study of the domain. Each unit of learning should be structured in such a way that the learner spends approximately an hour of study to complete the activities foreseen by the author. However, the work of the student is somewhat analogous to the research of the content and planning of the structure of an essay or term-paper type of response – a task that typically takes many (sometimes many dozens) of hours. Thus, the student, has the opportunity to engage in a much larger number of creative knowledge-construction exercises during the time available for study on a given course. This benefit is additional to some of the other pedagogical benefits identified in the research on SC.
A unit of SC learning usually contains the following sections:
The Syracuse University research, mentioned above, thus added a further dimension to the original 1960’s conception of SC. However, the first five sections described above were maintained much as originally conceived by the inventors of SC, as a series of research studies demonstrated this approach to be as effective and more time-efficient than other approaches to the organization of online collaborative and problem-based study of complex subject matter domains. In summary, our research showed that it is possible to multiply and extend the opportunities to incorporate effective group learning activities without the parallel multiplication and extension of the workload of the available teacher/facilitators.
One may ask why the potential of researched methodologies such as Structural Communication has not been realized in large-scale applications in real-world educational systems, given the very positive results achieved in small-scale R&D projects. We suggest that one possible reason for this lack of practical application of a theoretically “good idea” is that the design and development of SC exercises is seen as a complex and difficult task by most educators who have attempted it. Therefore, a logical next step is to develop intelligent authoring tools that may simplify the human author’s task. The next section describes the general process of design and development of SC exercises, identifies some areas of possible difficulty and discusses the possible approaches to machine support of the process.
2.2 Developing a SC unit.
The task of developing a lesson that applies SC to teach some complex understanding to learners is not easy work. The teacher needs to possess the capacity to foresee all the more probable answer combinations that learners are likely to supply. This requires a deep understanding of the learning problems and misconceptions that learners typically have with the domain in question, in addition to full understanding of the domain itself. The novice teacher generally doesn't have this level of expertise. The author needs to identify which topics of the subject are more important. He must elaborate intellectual challenges that cause reflection, restructuring of previously held concepts and promote interest in further discussion of the issues involved. He must identify the core issues of the domain and represent them through a set of elements that form the response matrix. Having made predictions on what response strategies learners may typically follow, the author writes open-ended, constructive, feedback messages for each of these strategies.
The first and second sections of the Structural Communication exercise (Intention and Presentation) are not all that hard work for the typical author. These sections could be created using traditional ID techniques. The Intention is similar to the statement of Objectives in conventional instructional design and the Presentation is very like the course study materials that may be prepared for any conventional course.
The teacher/author, when creating the Investigation, Response Matrix and Discussion sections, needs to be able to not only interpret new situations in terms of principles, but also simulate and predict results as well as elaborate solutions for complex problems. This technique therefore demands creative thinking on the part of the teacher-author because it values deep and highly structured understanding of the domain in question and not just simple memorization of information or mastery of simple concepts (Egan, 1976; Romiszowski, 2000). Also, Morais (2001) verified that teachers tend to use, in their teaching activities, the pedagogical techniques that were applied to them as students –as the methods of questioning and response formulation used in SC are not generally common in the regular school, there may be a considerable amount of learning and readjustment called for on the part of the novice teacher-author.
The teacher, when developing a unit of study with this technique needs to be able to:
How to find and develop these abilities in teachers is not the purpose of the present work. Although initial training plays a role, these abilities are assumed to continue to grow in the teacher during many years of practice. Lesgold (1984) emphasizes that a person acquires expertise through a repetition process (ie. like a marathon athlete) or through exposure to a great diversity of cases, conditions of situations (ie. like a chess player). Romiszowski (1981) draws a similar distinction, suggesting that the first type of procedure is more appropriate for the acquisition of repetitive or “reproductive” skills, while the second is better for mastering the more creative or “productive” skills. However, there is an overall “best practices” process that has been found to lead most often to satisfactory designs. Such best practices shoud be promoted and supported by the proposed online tools for design of SC units.
The design and development of tools – intelligent or not – for the automation of the instructional design (ID) process has quite an extensive history. The “state of the art”, of about a decade ago, was reviewed by Spector (1993) in an edited book that contained chapters on most of the projects that were under way up to that date, including several attempts to develop expert systems that would implement in an automatic or semi-automatic manner a specific general-purpose ID model. These projects are different in some respects from the work discussed in the present paper.
In the first place, as Spector (1993) mentions (pp 213-214), the earlier approaches were restricted to automating the implementation of existing established ID models (specifically those included in Reigeluth, 1983). All these models focus on the design of conventional, often called “objectivist”, instructional sequences that expect the student to master pre-defined, content-related knowledge and skills. The SC model, since its original invention in the 1960’s has focused on the promotion of a desired “learning process” rather than the mastery of specific “learning products”, so it is classifiable in the category that is now, more recently, referred to as “constructivist” learning models (some of these models are reviewed in Reigeluth , 1999) . In this respect, therefore, the present work is breaking relatively new ground.
Secondly, the previous projects reviewed by Spector (1993) were attempts to create intelligent machines that would be applicable across the broad spectrum of instructional design decisions. Perhaps for this reason of intended generality across many different categories of learning, the practical end-results of most of these projects have not had the hoped-for level of influence on the general practice of instructional design. The present work is restricted to furnishing automated support for the creation of just one very specific form of interactive learning exercise and so, due to this relatively modest goal, may possibly be more successful in its attempt to support the creative application of theory into practice.
Lastly, however, we and many in the field agree with Spector when he points out (p 213): “we do not believe that machines are capable of the truly creative, adaptive and innovative activities required in devising effective instructional designs – however, we do see ways for using intelligent machine techniques to assist in the process of designing and developing instructional materials”. In this respect, the present work, being a search for a shared human-machine intelligence approach to support the expert-author in the development of SC exercises, is in line with the predominant views of the field.
2.3. Machine support of the SC design and development process.
There is not that much available in the way of technology-based support tools for the SC design procedures described above. But there are some existing tools that may be applied to some parts of the authoring process. The literature recommends the employment of Concept Maps to identify the main topics of the subject domain and their interrelationships (Novak, 1998). Existing work carried out over the years by many researchers has contributed both tools and research on their use for purposes similar to the initial analysis stages of the SC design process. Some particularly relevant work is the research agenda being implemented by the Institute for Human and Machine Cognition (IHMC) at the University of West Florida. Much of the work of this Institute and earlier work upon which current research and development projects are based is described in a lengthy “Summary of Literature pertaining to the use of Concept Mapping Techniques for Education and Performance Support” (Coffey et al. 2003). The Institute has also developed a suite of automated concept mapping tools (IHMC Cmap tools) that may be downloaded from the Institute’s website. This suite has been successfully applied by the IHMC itself and by other researchers to supporting educators and subject specialists in the analysis and organization of complex subject matter content for the purposes of course design. These tools may therefore be employed for supporting the initial stages of the SC exercise design process. They may also be helpful in mapping out the pattern of typical misconceptions and difficulties that students encounter in the domain, through the comparison of expert-generated and novice-generated concept maps. We shall not be addressing this aspect of SC design support any further in this paper. But just to place this work in context and to illustrate the capabilities of the IHMC tools, as well as to illustrate in more detail the development process of SC exercises, we present, in an Appendix, a concept map generated with the use of the Cmap tools that interrelates key aspects of the SC exercise design process.
Another potential computer technique that may be used to identify the main topics of the subject is Automatic Text Summarization - ATS. This is a process that identifies and locates the most important information in a given source material and produces a condensed summary of that source designed for a given user group or task. The technique produces or selects a piece of text in a group of documents. This selection is based in some rules defined by the user. The method can use statistical procedures and/or heuristic functions to refine the process.
We now come to the unique, and possibly most difficult for the novice teacher, stage of the design process – how to identify which response strategies learners will follow and how to elaborate the feedback messages that will be associated with each pedagogically significant strategy. Romiszowski (2004) recommends some procedures to the author of SC exercises that focus on the implementation of a case study methodology. These procedures a: i) Define the "problem", ii) Analyze the problem and assemble the data. Select or create a situation, iii) Design the case situation, iv) Develop the case material, v) Evaluate the case material and vi) Develop the SC lesson plan. These recommendations may be quite useful to the teacher-author, but they are not easy to implement. It is from this point on in the SDc exercise design process that we see the value on an Intelligent Authoring Tool Environment.
How to identify student's probable response strategies and to write appropriate feedback messages for each relevant strategy is a problem for the novice teacher because he doesn't have enough experience to identify them and interpret the students’ reasons for selecting them. Two techniques may be used to identify the students’ response strategies and to link them to the appropriate feedback messages, previously written by the author, which are to be shown to the learner. The first of these is a rule based system or expert system. A Rule Based System stores the rules in an "if ... then ... else ..." structure. This technique was used in the past to elaborate medical diagnosis and other expert advisory systems (Reiser et al. 1992; Lesgold et al. 1992). The second technique is Clusters Algorithms. Clusters Algorithms are based on the idea of similarity or proximity. In contrast to expert systems that utilize a set of rules to find a “match” with specific entry data points, the clusters algorithm method places a given entry data point in a class together with similar, though not identical, points. The clusters algorithm therefore creates groups of similar instances. In the SC case, the groups would be composed of students with similar response strategies. We believe that teachers and domain experts could be the source of samples for this second technique. The samples would represent how the teacher or expert in the domain resolved the challenge. The Cluster Algorithm will classify the learner’s answer in relation to the groups thus formed by the author.
3. Authoring Tools and Inteligent Tutor Systems.
In the following paragraphs we present a review of some prior work in the area of authoring tools.
EON (Murray, 1988) makes it possible for the author to build a system's interface using a group of templates. These templates define properties like color or size and can store instructions that respond to user and system-generated events, like a mouse click. All the information is stored in a database. The Eon system works with a fixed pedagogical model. The development metaphor is based on a “flow line” which makes it possible for the author to define in a sequential manner the steps or actions to be executed by the program. However, the fixed pedagogical model may be considered as a limiting factor, as the system produced with this tool will not possess the ability to generate new models that can adapt to the profile of the user.
Blessing (1997) presents, in the DEMONSTR8, a way of programming-by-demonstration, in which the author doesn't need to possess specific knowledge beyond the domain knowledge. This authoring process is fast and easy. The style of development used by Blessing employs the principle of learning by repetition, in which the author demonstrates how to resolve a class of problems and the student repeats the author's steps. This style makes it impossible for the author to define the specific strategies to be employed in a specific domain. The tool does not have the ability to control the user's actions, nor does it allow the definition of pedagogical strategies for a given domain. This pedagogical obstacle compromises the efficiency of the learning system because it can use incorrect strategies for a specific domain.
REDEEM (Major 1997) possesses a fixed group of pre-determined pedagogical strategies, but the author can also create his own instructional strategies. This powerful resource is not included in the EON and DEMONSTR8 tools. The learning environment is based on a catalog metaphor of a set of course templates. It uses a library of templates. These templates may be selected for use in the system. The author can change the values of the properties of the components, which makes it possible to adapt the template to a specific domain.
RUI (Direne, 1997) is an authoring system and intelligent tool that makes it possible to teach visual concepts, applicable to medical images. The author has good resources for the production of instructional materials. The pedagogical methodology used in this environment is guided discovery learning. This methodology permits the student to interact with the environment by means of mouse clicks on the images and the use of natural language.
Systems like REDEEM, EON, and COCA (Major, 1991) have a strong pedagogical focus in detriment of capacity for the definition of solutions and strategies for the solution of complex problems, because these systems do not have the ability to represent solution strategies. DEMONSR8 has as its main characteristic the capacity to define the rules for domain problems. This environment doesn't permit the representation of multiple strategies for the same problem, nor does it store and show feedback to the learner. As one can observe, none of these authoring systems and learning shells are capable of combining an approach of deep discussion on the characteristics of a problem with attention to the differences between different students’ solutions to a given problem.
4. Architecture of the Authoring Tool.
This section of the paper presents a computer architecture that can be used to support the authoring process of a SC unit of study. This computer environment also makes it possible for experienced authors to share their expertise (knowledge acquired in years of practice) with novice teachers or authors. Further information on this learning shell and environment are available elsewhere (Noronha et al., 2004).
One key aspect of the computer environment proposed here is that it could be used to improve the skills of teacher-authors engaged in the construction of SC exercises and study units, because it facilitates the sharing of experience of teaching a given domain across a great diversity of conditions. The Computer Environment described here makes it possible for the teacher to give and to share his strategies for the solution of specific complex problems. No one teacher needs to generate all the relevant feedback strategies that may be required to orient the study of different students. He can share his experience with other authors and learn other forms of resolving the problem in question.
The authoring process is divided in two modules. The first one, shown in figure 1, gets information about the subject domain and the instructional objectives. The second tool, shown in figure 2, makes it possible for the author to create a “learning path” (which corresponds to the Intention and Presentation sections), develop the Response Matrix, define the response strategies and store them as a set of rules, and indicate which instructional messages will be shown to the learner in each case.
4.1 The Pre-Authoring Tool.
The Pre-Authoring Tool’s main purpose is to obtain general information on the subject matter, the learner’s information and the instructional objectives. The Authoring Tool will need this information to produce the Structural Communication lessons.
The pre-authoring task (Figure 1) is composed of:
| ATTRIBUTE |
VALUE |
| IDENTIFY THE STEPS FOR |
CALLIBRATION OF OSCILLOSCOPE |
Figure 1 - Pre-Authoring Module
4.2 The Authoring Tool.
The authoring task is shown in Figure 2. It is composed of:
1. Interface for selection and editing of the Response Matrix Elements. This interface allows the author to indicate which elements stored in KB will be included in the Response Matrix. These elements were extracted from the Presentation section (typically a group of Internet documents). The author can edit these elements or create others and insert them in the "Candidates for Response Matrix elements" database..
2. Interface for Editing the Challenge Questions. The author can edit the challenge questions that will lead the learner to achieve the instructional objectives. Authoring these open-ended questions demands a deep knowledge of the subject on the part of the author.
3. Interface for sharing, selection and editing of the response strategies. The response strategies are groups of elements selected by a learner from the response matrix to respond to a given challenge question Each response strategy will typically include, or omit, certain elements from the matrix that the author considers to be key elements for his evaluation of the response . These key elements form a set of "if.. then.. else..." rules for the selection of appropriate feedback messages for each response strategy ( ie. if included elements number {1, 5,17} or omitted elements number {4, 6, 7} then read comment A ). This rule represents the author's “feedback strategy” for responding to a specific set of possible response strategies to a given challenge question. The feedback comments which will be shown to learners that reply in particular ways, are edited by the author. Also, other domain experts may be invited to register their own set of feedback comments and strategies. The author can access these other feedback strategies to specific student responses to the problem to see to what extent he agrees or disagrees with other domain experts and, possibly, to extend or modify his own strategies and comments. This final set of strategies composes the knowledge base in the Expert System , developed into a student's learning shell. The authoring process interface is similar to the learning shell. The author just selects the required elements by means of checkboxes and then writes the feedback messages. The rules that define the feedback strategies and the respective feedback messages are stored in a knowledge- base. Figure 3 shows a “snapshot” of this record. The first column is just a code to be used by the computer system, the second column is the feedback, written by the author of the exercise, the column number 4 is a code to identify who created this record, the column number 5 is the feedback strategy Rule that will be interpreted by the computer environment and the column number 3 is a field to identify which Challenge Question corresponds this record. The only information that the author typed in this example are the feedback messages that are stored in column number 2. The information stored in other columns was automatically generated by the environment.
The author need not identify all possible student strategies. He will indicate which strategies, in his opinion, are more important to the current challenge question. When a second author or domain expert edits this lesson, he will indicate his (possibly other) set of feedback strategies for responding to specific student response strategies to the problem. The diversity of possible strategies for solution of a challenging problem is the main core of Structural Communication. It may be hard for one author to identify all of the student response strategies worth commenting, but a group of authors composed of both novice and experienced and competent teachers, together with other domain experts, could together quickly generate all the principal feedback strategies that may be required to close the gap in any student's knowledge or interpretation of the domain being studied.
5. Summary, limitations and future work.
This paper presented the architecture of an authoring tool to help authors produce Structural Communication Lessons. This authoring tool makes it possible for the author to record his viewpoints about a given subject domain and his strategies for solution of complex problems in the domain. This record is allied with a deeper discussion of the subject. The author's strategies and author's points of view are shared with other authors. The environment emphasizes the acquisition of deep and structured knowledge of the domain instead of the simple repetition of a few specific concepts and facts.
The author has some resources to assist him during the authoring task: i) a simple algorithm to identify the main sentences, based on the Automatic Text Summarization technique (Luhn, 1958); ii) a www-interface to collect information; iii) some automatic mechanisms to represent the author's feedback strategies and selection rules that will be loaded by the learning shell (Noronha et al. 2004).
This authoring environment makes it possible for the author to employ any pedagogical technique in the Presentation and Intention sections. The Structural Communication main core (Response Matrix, Challenge Questions and Discussion guide and rules) is used to outline the instruction. A Learning Path is composed of a group of Internet documents and information on the order in which they will be presented to the student. The addresses of the Internet Documents are stored in a Knowledge Based System.
As regards constraints, a general issue is that the Internet net is a dynamic entity. It is possible that between the authoring process and the execution of the lesson the document had been removed. The environment doesn't have a satisfactory solution for this failure. The Internet document is the source for the algorithmic process used to identify the “Candidate elements for the Response Matrix”. If the author of this document changes its content, and the system does not detect the change, it is possible that the algorithm used to identify the main sentences does not complete this task.
As regards the authoring process itself, the “candidate elements for the Response Matrix” are just a set of suggestions made by the author. This set may not be complete and neither is it necessarily correct. There is no way to automatically evaluate these elements. The teacher's expertise will continue to indicate the good candidates for inclusion in the Response Matrix. There is no formal methodology to do this. The environment automatically creates and stores the feedback strategy rules in a Knowledge Based System. The author does not need to have high-level computer knowledge to use this environment. Until now, the environment just uses the logical connectives AND, NOT and OR to produce, store, load and interpret the rules defining the feedback strategies. Connectives such as TwoOrMore, AtLeast(# N elements ), NoMoreThan(#N elements) for example, are still being developed.
5. References.
Baeza-Yates, R. A. ,(1992). "String Searching Algorithms". Information Retrieval - Data Structures & Algorithms. William Frakes and Ricardo Yates Editors. Prentice Hall, 219-240.
Blessing, S.B.(1997), “Programming by Demonstration Authoring Tool for Model-Tracing Tutors, International Journal of Artificial Intelligence in Education, volume 8, pp 233-261.
Direne, A.I. (1997), Designing Intelligent Systems for Teaching Visual Concepts. International Journal of Artificial Intelligence in Education, volume 8, pp 44-70.
Egan, K.,(1976). Structural Communication., Belmont, CA: Fearon Publishers.
Lesgold, A. M.,(1984). "Acquiring Expertise". In Anderson, J.R. and Kosslyn, S. M. editors, Tutorials in learning and Memory: Essays in honor of Gordon Bower, W.H. Preeman.
Lesgold, A., Lajoie, S., Bunzo, M., Eggan, G.,(1992) "SHERLOCK: A coached Practice Environment for an Electronic Troubleshooting Job". Computer-Assisted Instruction and Intelligent Tutoring Systems: Shared Goals and Complementary Approaches.
Luhn, H. P.,(1958). "The Automatic Creation of Literature Abstracts". IBM Journal of Research and Development, 159-165.
Major, N. (1997), REDEEM: Exploting Symbiosis Between Psychology and Authoring Environments. International Journal of Artificial Intelligence in Education, volume 8, pp 317-340.
Major, N. (1991), Using COCA to build na intelligent tutoring system in simple algebra. Intelligent Tutoring Media, volume 2, No. 3/4, pp 159-169.
Morais, A. M.,(2001) "Práticas Pedagógicas na formação inicial e prática dos professores". Artgo apresentado no seminário de modelos e práticas de formação inicial de professores, 3 - 4 maio 2001, Faculdade de Psicologia e de Ciências da Educação - Universidade de Lisboa.
Murray, T. (1998), Authoring Knowledge Based Tutors: Tools for Content, Instructional Strategy, Student Model, and Interface Design. Journal of the Learning Sciences, volume 7, número 1, pp 5-64.
Noronha, R.V., Silva F. M., Fernandes C. T. , Omar N., (2004) "Structural Communication - A Learning Environment Prototype". Proceedings of the IASTED International Conference WEB-BASED EDUCATION, February 16-18, 2004, Innsbruck, Austria.
Novak, J. D. (1998) "Learning, Creating, and Using Knowledge: Conceptual Maps as Facilitative Tools in Schools and Corporations", Lawrence Erlbaum Associates, publishers, Mahwah, New Jersey.
Reiser, B. J., Kimerg, D. Y. , Lovett, M. C. Ranney, M., (1992) "Knowledge Representation and Explanation in Gil, an Intelliget Tutor for Programming". Computer-Assisted Instruction and Intelligent Tutoring Systems: Shared Goals and Complementary Approaches.
Romiszowski, A. J. (2004) "Structural Communication: How to Design and develop the Exercise". Brazilian Review of Open and Distance Learning, 2(4). Available online: www.abed.org.br
Romiszowski, A (2000). , “A methodology for Case-Study in Virtual Groups”, Proceedings of the ABED International Conference, 2000. Available online as www.abed.org.br
Srinivasdan, P.,(1992). "Thesaurus Construction". Information Retrieval - Data Structures & Algorithms. William Frakes and Ricardo Yates Editors. Prentice Hall, 161-218.
Appendix.
A “Concept Map” describing the process of Developing Structural Communication Exercises.
This “concept map” was created with the IHMC Cmap Tools developed by the Institute for Human and machine Cognition (IHMC) at the University of West Florida ( http://cmap.ihmc.us/ ). It was developed for this paper on the basis of study of a “job performance aid” originally developed as course material at Syracuse University (Romiszowski, 1990). The concept map could, however, have been developed from the study of other sources and have been used to design the job performance aid. The full text of the job-aid document is available online in the Brazilian Review of Open and Distance Learning (Vol.2/No.4), accessible through the Website of the Brazilian Association for Distance Education – ABED. ( www.abed.org.br ).
