A computer science teacher's experiences with the PC as a teaching tool by developing CAL for vocational training in industry for 6 years.
Author: Kjell Øystein Arisland, Department of informatics, University of Oslo and Computers and Learning A/S, Oslo, Norway.
The subtitle implies that this report is about results acquired from developing CAL/CAI-material for industry. Over a period of 6 years, one naturally has both good and bad experiences, as will be described. In addition, there are some results that are unusual, and should therefore be reported. First, here is an overview of the report's contents:
1. Following this introduction, this report starts with the background story.
2. A short history describing the various projects from between 1988 and 1993.
3. Experiences with industry personnel and their input to the project is discussed.
4. The main theme of the project is a search for efficiency in different forms.
5. Another important theme is a search for producing "The elusive a-ha". Various pedagogical models are described: From simulation through presentation to drill and animation.
6. Some implementation limitations are discussed in "The hardware constraints: Reality and trends, technology and market development"
7. Some tools were developed during the projects, and their technical merits are briefly described.
8. Future plans are sketched
9. Conclusions
The term CAL when used in this report is short for CAL or CAI in which the letters may mean any of the following: C - Computer, A - Assisted or Aided, L - Learning or I - Instruction. Note that "Learning" and "Instruction" denote the same process as seen either from the learner or the instructors point of view.
The author is employed as assistant professor at the Department of informatics (Computer science) at the University of Oslo, working in the microelectronics group with teaching (digital design and computer architecture) and research (software tools for VLSI design mainly). In 1987, the author was approached by a representative of the Federation of Norwegian Process Manufacturers and asked to undertake developing software for educational use in the industry, mainly for vocational training. The work started in the fall of -87, and in 1988, the author formed the company Computers and Learning A/S in co-operation with some colleagues. Since then, about a dozen development projects [CAN-94] have been completed in co-operation with Norwegian and Swedish industry, and a set of tools have been developed. Throughout this work, some presumably valuable insight has been gained, mainly related to what works in real life from combined programmers and educators' point of view.
The scope of this report is limited to summing up and discussing some of the many experiences and observations done during the projects described below. The vantage point of the author is that of having a formal computer science background and several years of practical experience of teaching and of developing CAL-courseware for industry. It is recognized that the general field of CAL demands expertise in many areas, some of which are of a very different nature from e.g. computer science. One such field of study is that of the psychologically based theories of learning, in which e.g. Hergenhahn [HER-82] defines learning thus:
"Learning is a relatively permanent change in behaviour or in behaviour potentiality that results from experience and cannot be attributed to temporary body states such as those induced by illness, fatigue, or drugs."
To adopt such a rigorous definition as the basis for the discussions in this report would be overkill.
Another interesting field of study is the field of media in general, of which a very interesting sub-area is that of the advertising industry, in which many in-depth studies have been made relative to human motivation.
Yet another large field with much research material is that of pedagogy. A general difference between traditional pedagogy and CAL, is that pedagogy generally concerns human interrelation while CAL always has the computer as an intermediary. This does not mean that research results in pedagogy do not apply to CAL, but rather that they may or may not apply.
Neither of these fields have been studied in-depth by the author, and one may ask whether it is possible to combine the scientific practices of these fields in any one report. Certainly, this report does not do so.
The following projects have been completed since 1988 [CAN-94]:
This first project involved simulation of a 75% FeSi metal production process. The goal was quite broad: To produce CAL-material that manufacturing personnel at different levels of experience and expertise could use to improve their understanding of the processes involved. The goal proved to be too broad in scope and was reduced in the development process. The co-operating industry personnel in the project did not agree on the desired scope of the project and not all were happy with the outcome. As first projects go, it was very rich in results and experiences relative to the resources allocated. A total of about 600 hours of work was funded in this project, and the results were a CAL course containing some 20-30 hours worth of presentations including some interactive question and answer sessions, some simple animated simulations and one rather complex process simulator. Another result was the first version of an interpreted authoring language which is now the basis of the CANDLE authoring system. An independent evaluation of the use of the Ovnsim course showed that it was one of the most popular such programs in use in the industry in Norway about 2 years after it was completed. The total number of CAL users surveyed were, however, too small for the results to be statistically interesting, but the result was encouraging anyway.
The second project also started out with a basis in simulation, but this project took a different route when some of the industry contacts controlling the project turned out to have a preference for so-called 'electronic book' presentations enhanced with interactive question and answer sessions. The CANDLE authoring system was during this project strengthened with various features for handling text answers from CAL users and responding with confirmations or corrections. For some 3 years, one person was employed by the contracting industry and worked full time on this project developing some hundreds of hours of CAL presentations with mixed-in question and answer sessions. This work was done in direct co-operation with large companies in the pulp and paper industry, and some of the material was tailored to fit different manufacturers production equipment installations. The results from this project was and is considered generally useful by the industry and is used in the vocational training of personnel at several paper mills.
In co-operation with a pharmaceutical company a project to develop CAL material for teaching basic industrial chemical processes was called 'Syntesim', since the initial focus was on simulating a synthesis process, but the focus gradually shifted towards animation. This project had a strong influence on the development of the CANDLE authoring system, making it heavily graphics-oriented. In addition to focusing on graphic visualization of chemical processes, some simulators were developed for a three-stage water cooling system, and the concept of animated simulation was explored and exploited further as a strong pedagogical tool.
Cement is a continuation of the Semsim project, and both concern the manufacture of cement. Semsim ran in parallel with Syntesim and exploited the same graphics capabilities and experienced the same gradual change of emphasis from simulation at the start to simple animations and animated simple simulations at the end. The project, which was funded with government grants to the vocational education board, was so successful that when Semsim ended, the Norwegian and
Swedish manufacturers of cement, Norcem AS and Cementa AB picked it up and jointly funded a continuation project: Cement. The resulting program is now in use at the cement plants both in Norway and Sweden, and visitors from cement manufacturers in other countries want it translated to their languages.
As an experiment, a set of game-oriented drills of basic arithmetic skills was developed. The experiment showed that the CANDLE authoring system was well suited for this type of applications, and the implementation and the pedagogical principles used were well received by teachers as well as users. The programs are now available on the Internet via World Wide Web in English versions. [SMG-94]
A basic chemistry course, a regulator simulation program, an industrial heating, mixing and storage of food products simulator and a traffic sign drill program are other examples of courseware that has been developed. All these projects have one thing in common: They have all been implemented using the CANDLE authoring system, and most of them have contributed in one way or another to improving the authoring system.
The above projects have shown that the use of CAL in Scandinavian industry is still in its early stages. One effect of this is that the responsibility for exploiting CAL is fragmented among various people in various departments. The personnel and training departments, who probably should handle the issue, generally do not have the resources or the technical expertise to act decisively enough to get much done relating to CAL. The information technology departments usually have enough problems trying to keep up with the general development of hardware and software and are rarely interested in the pedagogical aspects of CAL.
The initiative to exploit CAL, in our experience, usually comes from people in the production departments. These people are both technology- and result oriented, and are highly motivated to increase the level of understanding among their operative personnel. Sometimes, however, their grasp of pedagogy and of information technology can be somewhat limited, and this can have detrimental effects on project planning.
CAL is an application of information technology on which most people seem to have strong opinions. This is an asset because it makes it easier to generate initial interest in projects. It is also a considerable drawback, because it makes it harder for experts on CAL to command the necessary respect for their views and recommendations to make implementations consistent and of high pedagogical and technical quality.
In the above projects we have experienced quite disparate views on what to emphasize in project development, for example whether to use text oriented user communication or graphics oriented animation. Quite contradictory opinions sometimes exists, and since available research on such subjects is hardly conclusive, it is difficult to advocate strongly against the views held by any project champion.
Rather than pursuing specific ideas about pedagogical models for all projects, it has been considered more fruitful to follow up on any strong opinions held by key personnel in industry. In other words, throughout the projects various pedagogical models have been tried, and the emphasis has been on producing high quality courseware within the framework of supporting whatever model the industry personnel favours.
In carrying out the projects, numerous teachers and subject matter experts working with vocational training were consulted. In all projects these people showed considerable interest in learning how the authoring tools are used. In several of the projects, the contracts stipulated that the tools to modify and augment the courses with further material must be available to the schools and industry involved, and this was considered crucial both by the industries' project administrators, and by most of the participating teachers and subject matter experts.
In all of the larger projects 2-3 day seminars (6 altogether) were held to teach teachers and other potential courseware authors how to use the authoring tools. The results of these seminars were surprisingly unequivocal:
Of totally between 50 and 60 people attending seminars, less than 10 ever produced any courseware.
Of those that produced courseware, most did so because they were directly involved in one of the above projects as subject matter experts. Only 2 persons continued from implementing contributions to a course to producing whole courses on their own.
The reason some seminar attenders later gave for not producing courseware was that they simply did not have time. The tools were considered quite accessible for simple use, but required in-depth programming understanding for more advanced authoring of animations and simulations.
In contrast, two programmers in Computers and Learning A/S have both created presentations, simulations, animations and animated simple simulations and implemented them for 90% of the contents of the above mentioned projects (with the exception of Papirsim) on a part-time basis while holding other full-time employment.
On the basis of this, it is concluded that access to authoring tools, both the software itself and actual training in using the tools, seem to have little or no impact on whether potential authors actually become CAL courseware authors.
When the first project described here started, the objectives of the project were relatively loosely defined: "To explore how vocational training in Norwegian metallurgical industry may benefit from Personal Computer Aided Instruction".
As the projects developed, this loose definition was replaced by another loose, but more concrete definition of objectives: "To explore and implement methods for how vocational training and other education may become more pedagogically efficient through the use of CAL"
The term "pedagogical efficiency" has turned out to denote a key concept in our work, and is defined as the relationship between the quantity of the material about which knowledge and/or understanding has been transferred from teacher to learner multiplied by a factor describing the complexity of the material and divided by the total amount of working time (effort) spent by the teacher and by the student.
Pedagogical efficiency = (quantity of material * complexity) / time
There are many aspects of pedagogical efficiency, and motivation, author efficiency, implementation efficiency and learning efficiency will be discussed.
In any use of CAL, the computer and the CAL courseware serves as an intermediary between the author (teacher) and the learner, see figure 1. This separates teaching and learning into two distinct processes that may occur in different time and space contexts. This gives rise to separate and possibly different concepts of efficiency for the two processes.
Author ______ ==> _______ Computer and CAL courseware _______ ==> _______ Learner
______ Teaching process _________________________________ Learning process _______
__________________________________Figure 1_________________________________
Motivation plays an important part in pedagogical efficiency. Teacher motivation is important both to produce high quality instruction material and because the students' motivation is usually affected by the teacher's motivation. The motivation of the teacher in the capacity of a courseware author is believed to have a direct strong influence on the quality of the courseware the teacher produces. Students' motivation is believed to have a direct strong influence on the students' receptiveness and their capacity for remembering and understanding data, concepts and relationships.
Since it is costly to measure motivation, no attempt has been made to do so in the work described here, but a lot of effort has been put into providing motivational effects in the instruction material.
Author efficiency can be defined (simplified) as the relationship between material quantity, presentation complexity and the time (working time, effort) it takes to produce courseware given only a list of knowledge and concepts that the author is required to teach the students.
Author efficiency = (quantity * complexity) / authoring time
It is a well known fact from the software industry that the total code output from human programmers per day may easily vary by a factor of up to 10, and sometimes even more, between reasonably competent personnel, and the same difference in output is known to exist between authors of prose literature. It should therefore come as no surprise that the quantity of production output of CAL-material varies widely from author to author.
A major question related to author efficiency is who should do the authoring? At least three alternatives exists:
1. Programmers learn the subject matter and do the authoring.
2. Subject matter experts learn programming tools and doing the authoring.
3. Teams of programmers and subject matter experts co-operating.
In our experience, alternative 1, programmers learning the subject matter and then doing the authoring is most efficient, even though we have seen some good results from both alternatives 2 and 3. Of course, when programmers learn the subject matter, they often need access to subject matter experts, but the s.m.e.s need not be part of the authoring itself. It is our experience that good programmers with authoring experience easily assimilate the necessary information on any technical subject to produce quality courseware. On the other hand, subject matter experts will only rarely learn the necessary programming skills to produce quality courseware in the same amount of time. Alternative 3, co-operation, only seems to work well when the two parties are especially good at co-operating, and the subject matter experts present the material to the programmer, who suggests and implements ways of presenting the material to the learner.
The tools available to implement courseware can be a major stumbling block on the path toward pedagogical efficiency for the author. If the author desires e.g. animated graphics on a full motion background to demonstrate some concept, and the tools available only support text screens, it is easy for the author to be efficient within the given constraints, because there are very few options. However, if some limited graphics capabilities are present, the author may spend an inordinate amount of time trying to create a reasonable approximation of the desired effects. Difficult trade-offs between providing motivating effects and having author efficiency are likely to exist for a long time, even though the tools keep becoming better. In fact, as tools improve, the authors' ambitions have a tendency to increase, thus reducing author efficiency.
Implementation efficiency can be defined (simplified) as the relationship between material quantity, presentation and concept complexity, and the time (working time, effort) it takes to implement courseware given a detailed author specification.
Implementation efficiency = (quantity * complexity) / impl.time
From this definition, it is clear that implementation efficiency is a subset of author efficiency.
Authoring tools have a strong influence on implementation efficiency. Throughout the described projects, the CANDLE authoring system was continuously augmented and improved, and in some cases, what once took hours to implement was simplified to a few mouse clicks. The importance of such authoring tool power is not only, however, in the direct reduction of implementation time given a fixed specification, but also in that the authors are given the opportunity to test ideas by directly implementing sketches of animations and presentations.
Learning efficiency = (quantity * complexity ) / learning time
Since the effort a learner puts into mastering some material is measured in time, the courseware must be designed to minimize this time period. There are clearly many different ways to do this, as will be discussed later.
In sum, we have that pedagogical efficiency is a combination of 1) author efficiency, which in part depends on implementation efficiency and 2) learning efficiency. Each of these are influenced by the motivation of the computer user, be it an author or a learner.
Average pedagogical efficiency =
(author efficiency + learning efficiency * number of learners) / (1 + number of learners)
Since CAL material is replicated at close to zero cost, it is clear that as the number of students of any one course grows sufficiently large, the author efficiency becomes less and less significant.
Over many years now, computer hardware has become functionally better and better and is becoming commonplace in the homes and schools. In the meantime, CAL matures and the computer will probably in time take over as the preferred vehicle for delivering information. CAL authoring will then become a very competitive business with many of the traits of traditional media businesses today.
Assuming this scenario will become reality, another obvious consequence of the above is that if it is at all possible to increase learning efficiency by producing better CAL material, this should be the focal point of CAL research. The primary purpose of developing authoring tools should therefore be to produce CAL material yielding a higher learning efficiency .
Most of the discussed aspects of pedagogical efficiency are very costly to measure quantitatively. We have not had the resources to do such measurements, and feel that for some of the above aspects, trying to measure the progress would be a waste of time. For some aspects, however, it is possible to do crude estimates on increases in efficiency based on experience. Author efficiency is an example.
The author of this report estimates that his own efficiency in producing courseware has been increased by a factor of between 2 and 5 by developing and using the CANDLE authoring system as opposed to using a standard programming language for courseware authoring. This estimate is certainly subjective and hard to verify, and does not in any way reflect any differences or similarities between the CANDLE authoring system and other systems. However, it is based on six years of part time (roughly 15 to 20 hours per week) experience and does suggest that authoring systems are useful in increasing author efficiency when the authors spend enough time getting to know the tools.
Another example of pedagogical efficiency aspects that can be measured is motivation. The advertising industry are experts at measuring motivation, and have proven countless times that motivation can very efficiently be affected by simple psychological stimuli. As mentioned above, no attempt has been made in this work to actually measure the effects of motivation on pedagogical efficiency, but such effects have nevertheless been assumed to be decisive.
The goal of the authoring process is normally to produce courseware that will result in the highest possible learner efficiency for the target audience. Obviously, the courseware must fit the target audience, but even accounting for this, there are many forms of presentation of subject material and interaction with the learner that may be chosen for any subject. The term "pedagogical model" will be used loosely to denote any such form of presentation and interaction with the user. Throughout the described projects, the usage of pedagogical models varied, and the main ones were:
By a simulation is meant the implementation of a mathematical model which allows the computer to imitate over time the behaviour of a process, organism, piece of machinery or any dynamic, time-dependent, non-stochastic relationship between parts of a whole. Simulations may be simple or complex, they may be run in batch-mode or real-time, and their presentation to the user of inputs and outputs over time may vary from crude tables of numbers to fancy real-time graphics displays. Pedagogically, simulations are obviously advantagous, in that they utilize both the computational and the interactive powers of the computer to achieve safe and inexpensive laboratories for any dynamic process, be it simple or complex, easy to grasp or near impossible to fully comprehend theoretically.
Most of the described projects started out with plans for extensive use of simulations, and some relatively complex simulators were designed and implemented. One example is the Ovnsim simulator of a 75% FeSi production melting furnace, another is a cement mill simulator in the Semsim project. The simulators required a large portion of the total authoring resources in the projects. The contributions of the simulators to learner motivation and learner efficiency appears to have been somewhat lower than initially expected, and certainly lower than some other pedagogical models when the required authoring resources are taken into the balance.
By a presentation is meant one or more sequences of static text and pictorial information which the user may access piece by piece, regardless of the organization of the information, whether it is totally sequential like in a book, tree-structured, graph-structured, hyper-media, or any other static organization.
Presentations are probably a necessary part of most CAL courseware, and certainly have been used extensively in the above projects, mainly for filling in between other pedagogical models. Courseware consisting solely of presentations was not made in any of the projects, as most project leaders wanted something more exotic.
When a presentation is augmented with interactivity in the form of questions to the users followed by feedback to the user on how correct or incorrect the answers are, it is called "questions and answers", or Q&A.
Q&A has been used in several of the projects, and has been well received by users. It is known to have good motivational effects. On the other hand not all users care for it, and it has therefore been used in a voluntary manner, allowing users to proceed past questions without consequences. In the Papirsim project, Q&A became the main pedagogical model for a substantial amount of courseware, most of which was developed over three years by a full-time CAL author. The reason for such extensive use of the Q&A model was simply that it was favored by the Papirsim project leaders.
By a drill is meant some fixed number of tests organized around a central theme, like e.g. a graphic picture of objects, and taken in sequence and where the users score of correct or incorrect answers is kept track of and displayed in a game-like manner. Drills usually also have a learning mode, in which correct answers or information can be displayed e.g. by pointing at objects in a picture. An important feature of drills is that they normally have a mode in which correct answers are indicated whenever the learner makes a mistake.
Drills have been used in several different ways in the described projects, and they have usually been very popular with both project leaders and learners. There is, however, a game-like aspect of drills that seems to have a negative psychological effect on project leaders. The attitude is one of "If it is fun, it can't be really serious, and education is a very serious business." This attitude is the main hindrance against exploiting drills more extensively.
By animation is meant a dynamic drawing in which some concept is demonstrated by means of objects moving relative to each other or changing form over time, either automatically or under the control of the learner. Animation is the computer CAL equivalent of Hollywood cartoons.
After the first two years of experimentation in the projects, animation and combinations of simple simulations with animations emerged as the most pedagogically efficient both in terms of motivation, learner efficiency and author efficiency. Therefore, considerable resources were dedicated to producing animation-based CAL courseware. As before said, we have little real proof of this observation, but believe it to be valid. Certainly, it was valid when it came to selling projects, as it was the animation-based parts of the courseware that caught people's attention and sparked interest. It was also valid for author efficiency, as the animations cost considerably less to produce than simulations, and it was valid for both author and learner motivation, animations are simply fun to produce and watch. As for learner efficiency, only further research can tell more objectively if the observation is valid.
When the first three of the described projects were in the planning stages in 1988, an informal, but reasonably exhaustive study of possible hardware platforms was done. Then, as now, it was quite simple to see that the IBM PC-compatible computers account for close to 90% of the total population of "serious" hardware among users in industry, schools and homes. Two niche markets exists in that some types of businesses, such as music and graphics layout, have a large percentage of Apple's MacIntosh hardware, and in universities and research institutions, the Internet community is committed to UNIX-based workstations from SUN, DEC, SGI, HP and others.
The games-oriented hardware was considered, but rejected mainly because of its lack of standardization and history of instability and incompatibility over time.
Given fixed budgets, it was simple to choose the IBM-PC-compatible hardware as the only platform to develop for. Then came the question of graphics capabilities. Since the only standardized graphics with reasonable resolution in 1988 was the EGA 640x350x16 mode, this was chosen. It was chosen not to implement for Microsoft's Windows, since this did not contribute much in added functionality (and still doesn't), and would require better hardware than most users had available at that time. As a consequence, all project courseware developed run well on any PC hardware from 286 and up with an EGA, VGA or better screen.
For the user interface, it was decided that only a fully graphic point-and-click interface would be acceptable and Borland's BGI graphics library was chosen to implement it, along with the C language and later C++.
Several very "promising" technologies were available in 1988 and others have come along since then. In 1988, socalled "Interactive Video" ("IV") were the current rage, since then "IV" seems to have gone out of style and has been replaced by "Multimedia" and "Virtual Reality". For the described projects, we have steered clear of all non-standard and expensive hardware that does not exist in great numbers among potential learners. This strategy may at times have given business competitors an edge in acquiring contracts, but our users have rarely been denied access to our software because of lack of compatible hardware. Also, all our experiences and results have lasting value, and we have spent relatively little time on implementation details, and so far no time at all on porting software from one platform to another.
Concerning multimedia, graphics animation has been utilized as an important tool, and we do expect sound and video to become important factors in CAL courseware over time. However, from a hardware point of view, both sound and video are quite costly, and standardization is still lacking.
Developing tools is very costly, so unless other existing tools have serious drawbacks, one should never consider tool development. Once tool development has started, it is very hard to accept other tools as better, and the "not invented here"-syndrome may well become a factor clouding the issues.
In 1988, all existing authoring tools that we could access without incurring prohibitive costs were studied. To make a long story short, we found none that satisfied our needs for simulation and reasonable author efficiency. Even an industry-leader package that was purchased at $2.000 was by no means powerful enough for our purposes. The main lacking feature in most systems was that of general simulation support.It was therefore decided to develop an authoring language for simulation. The language became a simple subset of the "C"-language, and an interpreter for this language was produced. Some quite powerful animation support was built into the language interpreter, and simulation and graphics control was tightly integrated. As use of the language increased, the need for a graphics editor that would directly support the language grew, and such an editor was developed. The language and editor first went through a stage of internal versions called CANDLE V0.9, before the language was re-written and a new editor programmed from scratch to become CANDLE V1.0 at the end of 1991 [CAN-92].
The CANDLE authoring system has features that are not common in other authoring systems:
1. There is directly interpreted ASCII code, in a "C"-subset syntax.
2. Each file (extension ".CAL") represents one screen of presentation, simulation, animation etc.
3. Each file contains both graphics (usually) and code.
4. There may be hypermedia links in any file (screen), transferring control to another file (screen), both directly in the graphics, and in the code, and parameters may be passed.
5. The interpreter enforces that simulation variables directly control graphics objects.
6. A library of graphics objects that can be linked to simulation variables. Examples: Decimal numbers, bars, graphs and movable on/off windows containing other objects.
7. User input, both by pointing at areas of the screen and clicking, and keyboard input, is handled through special input objects which may directly affect variables.
8. Bitmap objects with or without transparency.
Note that the CANDLE authoring system has been used for implementing the courseware in absolutely all projects that Computers and Learning A/S has undertaken, ranging from mathematically heavy simulations to games oriented graphics based drills and math training for children in elementary school.
The author believes that animation as a pedagogical model can significantly improve learner efficiency. To support or weaken this assumption, a project of producing courseware for teaching mathematics at the 12-14 year old elementary/junior high school level will be developed in two versions with one relying heavily on animation, and another using more conventional book-like presentation methods. The resulting courseware will then be tested using different groups of learners.
The CANDLE system V1.0 was finished in 1991, and has been used and updated from time to time since then. Plans now exist for a CANDLE V2.0 for Microsoft's Windows, and for The X-Windowing system.
The World Wide Web (WWW) is a very powerful part of the Internet, and has fast become one of the most popular services. Its hypermedia text transfer protocol (http) is quite simple, yet very powerful, and clients like Mosaic from NCSA are useful programs for many purposes. In one area, however, WWW is severely lacking, partly because of the statelessness of http, and partly because of the simplicity of the hypertext markup language (html). This area is in interactivity. The interactivity of WWW can, however, be easily improved by hooking a client application interpreter into the user WWW client, much like graphics viewers, sound file players and video players are hooked into e.g. Mosaic. The author therefore plans to develop a CANDLE V2.0 driver for Windows and X (and eventually the Mac) that can be hooked directly onto WWW-clients to interpret .CAL-files fetched by http.
A number of CAL courseware development projects have been completed over a period of 6 years, and some of the more important experiences are:
Different teachers and other experts have different preferences regarding CAL pedagogical models, and it is difficult and probably counterproductive to try to change their minds.
Teachers and industry experts consider control over courseware source code very important, and will go to great lengths to learn authoring tools, but will only rarely use the tools to produce courseware.
Access to authoring tools, both the software itself and actual training in using the tools, seem to have little or no impact on whether potential authors actually become CAL courseware authors.
Efficiency in the task of producing CAL courseware, dubbed "author efficiency", can be improved significantly through the use of authoring tools.
Programmers appear to be more efficient authors of CAL courseware than subject matter experts.
Learner motivation is probably improved by appropriate use of animation in CAL courseware, and it is believed that learner efficiency is also improved.
Standardized hardware has all the necessary power for CAL courseware.
Developing an authoring system is very costly, but the many powerful features of the CANDLE V1.0 authoring system contributed to our courseware development enough to make it worthwile.
A general authoring system can be equally suitable for developing courseware for vocational training in industry and for games-oriented arithmetic training for elementary school use.
The projects described would not have been possible without the expert programming, systems design and pedagogical intuition of Dr Yngvar Berg, Arne Kinnebergbråten and Knut Tvedten. These three have constituted the core development team for the Candle authoring system together with the author of this paper. Developing courseware in the various projects have mainly been done by Dr Yngvar Berg and the author of this paper, but others have also contributed considerably.
[CAN-94] http://www.oslonett.no/html/adv/Candle/candle.html
[HER-82] B.R. Hergenhahn: An Introduction to Theories of Learning, Prentice-Hall, 1982
[CAN-92] CANDLE V1.0 Reference Manual (in Norwegian), Computers and Learning A/S,-92
[SMG-94] http://www.oslonett.no/html/adv/Candle/PedShare.html