Instructional Design for Web-based Training

Instructional Design for Web-Based Training By Kerri Conrad and TrainingLinks HRD Press Amherst, Massachusetts Copyr...

Author: Conrad K.

190 downloads 2079 Views 3MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Instructional Design for Web-Based Training By Kerri Conrad and TrainingLinks

HRD Press Amherst, Massachusetts

Copyright © 2000 by Kerri A. Conrad and TrainingLink All rights reserved. Any reproduction in any medium of the materials that appear in this book without written permission from HRD Press is a violation of copyright law.

Published by: HRD Press 22 Amherst Road Amherst, MA 01002 1-800-822-2801 (U.S. and Canada) 413-253-3488 413-253-3490 (FAX) www.hrdpress.com

Printed in Canada ISBN 0-87425-542-2 Cover design by Eileen Klockars Editorial services by Joan Hiam Production services by Clark T. Riley

Dedicated to my husband, David Conrad, and to my parents, Tom and Bette Dittmer, for their faith

This page intentionally left blank

Table of Contents Preface: How to Use This Book............................................................................... 1 Chapter One: Assessing the Appropriateness of Web-Based Training.......... 9 Chapter Two: Forming and Managing a WBT Team ....................................... 37 Chapter Three: Audience Analysis ...................................................................... 59 Chapter Four: Objectives and Measurement ..................................................... 83 Chapter Five: Defining Learning Paths ............................................................ 111 Chapter Six: Presentation Principles ................................................................. 137 Chapter Seven: Courseware Testing.................................................................. 167 Chapter Eight: Course Evaluation and Revision Maintenance.................... 185 Chapter Nine: Putting It All Together............................................................... 197

Appendix A: Overview of Instructional Design ............................................. 221 Appendix B: Course Hosting and Training Management Systems............. 231 Appendix C: Creating Graphics for the Web................................................... 245 Glossary................................................................................................................... 263

v


Preface

How to Use This Book Preface topics: ♦ Impetus for this book ♦ Our background ♦ Purpose ♦ Intended audience ♦ Fast-track directory: Migration, outsourcing, tools, hosting ♦ Chapter overviews ♦ Acknowledgments

Impetus for This Book As the Web Project Manager of an expanding training firm, I have guided the design and development of several web-based training (WBT) projects. To match a fast-paced, rapidly increasing workload, we doubled our staff seemingly overnight, and as a result, we were faced with a monumental question: How would our department quickly train these new hires to successfully blend instructional and web design principles according to our established team processes—without compromising deadlines or product quality? A book—written by someone else—seemed a reasonable means of supplementing on-the-job training, but we were unable to find a satisfactory text. Our new team members required a step-by-step, practical synthesis, not a history of online learning, a survey of distance education, or a purely 1

Instructional Design for Web-Based Training

2

technology-based exploration. What seemed to be lacking was a detailed set of tested, process-oriented guidelines that would provide a foundation for more advanced WBT development methods. There’s truth in the cliché that “you need to know the rules before you learn how to break (or adapt) them.” Near the end of this book quest, I suggested to Jay, our web technical manager, that we should write a book. Our Background We’re an instructional design firm that is well-versed in theories of learning and instruction, but we are foremost practitioners. Since the founding of TrainingLinks in 1993, we have successfully designed and developed courses for several delivery systems, including traditional classrooms, interactive television, and self-study workbooks, as well as web-based training. Thus, this book reflects a process we have honed throughout numerous training projects for customers representing diverse interests: ♦ ♦ ♦ ♦ ♦ ♦ ♦

Automotive service and repair Health and safety compliance Human resources Marketing theory and techniques Software applications Technology manufacturing and sales Telecommunications

To deliver the best possible training to corporate learners, we leverage our practical experience, instructional design process, and expertise with each delivery system to select the most effective instructional techniques and technologies, given our customers’ business and training goals. We view our clients as team members who participate in a highly effective, co-authored, phased-review approach. This approach grew out of our relationship with one of the world’s leading computer manufacturers. Because our clients compete globally in “hyper-competitive” markets, we enjoy the challenge and opportunity to constantly stretch and adapt our skills in web-based training development. Our web-based solutions have been evolving since 1997, when we first started delivering web-based training to Fortune 100 clients. Our clients’ success depends on their ability to provide quality, just-in-time training to internationally located channels of distribution, sales, and support. As an Application Service Provider for corporate training and education, TrainingLinks empowers these clients and learners by providing complete outsourcing capability through web-based

How to Use This Book

3

hosting, learning data center management, and off-the-shelf and customized web-based courseware.

Purpose The purpose of this book is quite simple—to provide a step-by-step primer for creating effective WBT from start to finish. Our goal is not to speak “the final word” on web-based training, but instead to share an approach we have found effective. We encourage you to: ♦ ♦

Adapt this approach to your own training context Use it as a frame of reference when consulting more tool-oriented sources ♦ Explore coordination strategies for diverse team members on your WBT project As an introductory guide, this book focuses primarily on asynchronous web delivery founded on proven instructional design principles. We believe this focus provides a firm building block for creating web courses of increasing complexity, from the integration of multimedia to synchronous web course delivery.

Intended Audience In broad terms, Instructional Design for Web-Based Training is for anyone who is interested in creating effective WBT. More specifically, this book is for training and project managers, instructional designers, and course developers who: ♦

Require a focused guide to making important connections between instructional design and web design ♦ Are interested in a complete, tested process of WBT creation ♦ Have minimal experience in web design and development but have access to people with experience in those areas ♦ Need to understand basic development issues when seeking or working with a WBT vendor We also recommend this book for web design vendors who want to better serve their customers’ training needs.

4


Special Note: Because we write from the perspective of a training vendor, our use of some terms might be new to readers from academic backgrounds or corporate in-house training departments. For clarification, we refer to our organization as a vendor that creates courseware for other companies, whom we refer to as clients or customers. Some of our practical concerns include business proposals and time-to-market. You may need to identify equivalent terms and issues from your own context. In addition, many discussions throughout the book use the term “web delivery system.” Instructional designers view a delivery system as encompassing not only a delivery method in broad terms, but any technology and media integrated into that system. Thus, a web delivery system uses web technology and media such as graphics, video, chat, email, etc., to provide web-based training.

Fast-track Directory: Migration, Outsourcing, Tools, Hosting In addition to walking through a process of creating instructionally sound WBT, this book addresses logistical issues many training managers face, often under time pressure: ♦ Migrating existing courseware to the Web ♦ Working with a WBT vendor ♦ Selecting an authoring tool ♦ Developing a plan for web hosting If you need immediate information on these topics, turn to Chapter Nine, “Putting It All Together,” or Appendix B, “Course Hosting and Training Management Systems.” Once this shortcut has fulfilled its purpose, take the time to read previous chapters; making truly informed decisions in these areas often requires some knowledge of how web-based training is designed and developed.

Chapter Overviews Several outstanding references are available on web-related technologies, languages, and skills; this book does not seek to “reinvent the wheel.” Rather, most chapters begin with an overview of a major instructional-design step, then introduce related WBT development considerations via detailed examples. We collected these examples from our own colleagues, whose collective experience ranges from instructional, graphic, and web design to


5

systems administration. When applicable, job aids in the form of checklists and planning activities are included. Chapter One, Assessing the Appropriateness of Web-Based Training, defines different types of WBT and offers criteria for evaluating when WBT is a feasible delivery method for various training needs. This chapter also offers a basic method of estimating the cost of a web delivery system and maximizing training investments. Chapter Two, Forming and Managing a WBT Team, explores the core skills, roles, and responsibilities involved in successful WBT development. This chapter incorporates coordination strategies to reduce time-to-market. It closes by presenting a reference guide that introduces such topics as architecture, user interface, and writing for the web. Chapter Three, Audience Analysis, briefly reviews the general instructional design tasks involved in needs, task, and instructional analyses, and then launches into audience considerations from a web-development standpoint. For example, what experience level does the target audience have with web technologies? What systems and applications will the audience be using? Chapter Four, Objectives and Measurement, outlines criteria for effective objectives and measurement techniques for any delivery system. It then addresses related issues particular to a web delivery system: What types of student performance can a web delivery system reasonably measure? How can media enhance measurement capabilities? How can different learning styles be accommodated? Finally, how frequently should interactivity occur? Chapter Five, Defining Learning Paths, emphasizes important connections among learning units and modules, course architecture, and course navigation. After walking through an instructional approach to defining learning units and modules, this chapter examines how they might be organized and accessed in WBT, based on learner needs and training goals. It also covers basic implications of different architectural options. For instance, should learners be held accountable for all available training content if they are not required to access it in a linear fashion? Chapter Six, Presentation Principles, focuses on instructional concerns for user interface (UI) design. Chapter topics include thematic systems, core usability principles, and writing techniques for effective web presentation. The UI is

6


not a superficial matter, but constitutes the learning environment, and therefore, significantly influences the learning experience. Chapter Seven, Courseware Testing, outlines methods for performing three stages of testing web-based courseware—criteria tests, alpha tests, and beta usability tests. This chapter covers key issues surrounding courseware testing: Which evaluation method will generate the data needed? Who should participate in each review? How can usability testing be performed without a lab? Chapter Eight, Course Evaluation and Revision Maintenance, describes the responsibilities associated with maintaining a web course once it becomes available to students. First, the courseware should be evaluated for its effectiveness; a web delivery system can become a useful tool for gathering efficacy data. Second, this chapter covers tasks the original WBT team must complete before progressing too deeply into the mechanics of the maintenance phase, such as documenting project history. Chapter Nine, Putting It All Together, focuses on migrating existing courseware, identifying a qualified vendor, and selecting the right authoring tool for your needs. Because all of these decisions involve some knowledge of WBT design principles and processes, Chapter Nine synthesizes many of the key points explored throughout this book. Appendices A: Overview of Instructional Design ♦ Appendix A presents both the training development cycle and the instructional design model that form the foundation of this book. B: Course Hosting and Training Management Systems ♦ Appendix B discusses a range of hosting options, from minimal requirements to complete training applications known as training management systems. Topics include basic hosting equipment, the features of a training management system, and guidelines to evaluate a qualified Application Service Provider (ASP) who can meet your specific needs.


7

C: Creating Graphics for the Web ♦ Appendix C highlights techniques for creating a course composed of attractive, fast-loading web pages that learners will find easy to read and navigate. Main topics include how to differentiate between graphic file formats and minimize learner confusion when downloading and installing plug-ins, and what to consider when choosing a font. Glossary: Includes key terms relating to instructional design and web development. Companion CD: Includes many of the project planning and evaluation resources interspersed throughout this book. In its entirety, Instructional Design for Web-Based Training presents a complete model for designing, developing, and maintaining web courseware. As practitioners, we have found this approach highly effective, and we hope you will too.

Acknowledgments Direct Contributors Jay Erikson, Kreg Wallace, and Judi Schade were instrumental in coaxing this book out of the virtual and into the actual. Despite their own workloads, they always provided editorial insight on demand. They also agreed to share their expertise directly with readers by contributing the appendices of this book: Judi Schade — Overview of Instructional Design Jay Erikson — Course Hosting and Training Management Systems Kreg Wallace — Creating Graphics for the Web Many thanks to Stephen Hlawaty and Mary Lehti for developing the glossary, and to Lynette Gonzales for developing the companion CD. Executive Sponsors I owe a special thanks to Rob Mygatt and Dru Matthews, founders of TrainingLinks, Inc., for their commitment to corporate education, instructional technology, and “dynamic stability.” I am also grateful to Walt Fischer, Corporate Director, and to Diane Fischer for their business innovation and professional encouragement.


8

TrainingLinks Staff Thanks to the entire TrainingLinks staff for sharing best-practices on a daily basis. Whether or not they were aware of it at the time, these people in particular helped keep this project in motion: Trish Cullum, Kathy Foster, Lynette Gonzales, Christine Lillpop, Sasha Parker, Aaron Reid, Don Schick, Mary Siebe, Kristin Unwin, and Carole Wollam. Industry Reviewers It was an honor to receive advanced reviews from the following people, whose critiques imparted both professional dedication and practical wisdom. The application of their advice is the author’s responsibility. • Bill Gould, Ph.D., Senior Partner, Techsynergy Corporation Dennis Mitrzyk, Hewlett-Packard Company, Solution Reseller and Training Manager • Chris Cottam, ACDelco, Manager of Program Development Denis Garcia, Technical Training Director of IT, Cahners Business Information Finally, I deeply appreciate the publishing guidance of Lois Hart, Ph.D., founder of Leadership Dynamics. Please direct questions and comments to: Rob Mygatt or Kerri Conrad TrainingLinks 1600 Specht Point Drive, Suite F Fort Collins, Colorado 80525-4311 970-223-6565 www.traininglinks.com [email protected]

Chapter One

Assessing the Appropriateness of Web-Based Training Chapter topics: ♦ Overview ♦ What is web-based training? ♦ Determining the appropriateness of web-based training • Needs analysis • Task analysis • Characteristics of training content • Financial considerations ♦ Budgeting guideposts • Basic expenses • Expense schedules • ROI and business metrics ♦ Maximizing WBT investment gains Chapter resources: ♦ WBT project evaluation forms • Minimal target training requirements • Financial practicality • Optimal training environment characteristics 9


10

Overview This chapter surveys major issues that typically surround a decision to invest in web-based training (WBT). As future chapters take you through our process model in detail, these issues will resurface: ♦ ♦ ♦ ♦ ♦

Targeting learner characteristics and needs The skills and knowledge encompassed by training goals The nature and longevity of training content Financial considerations Strategies for maximizing a training investment

As this list implies, we believe that creating effective WBT is not simply a matter of mixing technology with training content. Rather, creating effective WBT is a matter of using technology to realize instructional goals and benefits. Because the process we cover is based on fundamental instructional principles, it relates to all forms of WBT and provides a firm foundation for adding advanced technology components. To provide a context for the issues listed above, the next section defines WBT more specifically.

What Is Web-Based Training? Web-Based training. We hear this phrase frequently and often see it in print, but the definition is usually assumed. So, how does web-based training differ from the numerous learning-oriented sites on the Internet? Although learning sites vary widely, all of them convey information textually and graphically to increase end users’ understanding of a topic. While surfing the Web, for instance, we find how-to guides for HTML programming, gardening, and RAM installation. Many other sites function much as a textbook does for a student, offering perspectives on established bodies of knowledge, from philosophy to physics. These sites serve their audiences well by allowing people to explore their interests. However, few of these learning sites can be categorized as web-based training. Like any of the sites mentioned, training certainly provides information, but information in itself does not constitute training. A web-based course is a hybrid of instructional design and Internet technology. As such, WBT is more than a collection of how-to web pages or a direct translation of a paper-based workbook into HTML. These distinctions lead us to a fundamental definition:

Assessing the Appropriateness of Web-Based Training

11

Web-Based training is the integration of instructional practices and Internet capabilities to direct a learner toward a specified level of proficiency in a specified competency. As this definition implies, instructional design principles form the crux of WBT. Designers of effective WBT provide instructional value by: ♦ ♦ ♦ ♦

Customizing content for the needs of a defined audience Presenting outcome-based learning objectives Logically sequencing material to reinforce those objectives Basing navigational options on existing and desired skill and knowledge of learners ♦ Designing objective-based, interactive learning activities students must complete to receive some form of evaluation The final WBT product can vary greatly, from book-like “page turners” containing only static graphics and text to highly interactive presentations and audiovisual simulations. Web training can also be delivered asynchronously, in which any required communication occurs independently at self-defined intervals (as with email), or synchronously, with communication occurring at the same time for all participants. We find it useful to envision this spectrum beginning with non-facilitated WBT and concluding with facilitated WBT. Non-facilitated training is self-paced, requiring no peer or instructor interaction. Facilitated training involves either asynchronous or synchronous human interaction: Non-facilitated

Mixed Facilitation

Facilitated

Asynchronous

Asynchronous and Synchronous


Several instructional approaches can be incorporated into a single webbased course or web delivery system. To illustrate this, let’s examine a familiar delivery system—the traditional classroom. Often, a classroom course integrates varied teaching strategies, some occurring inside the classroom, and some taking place between meeting sessions: lecture, homework activities, small group work, student presentations, and so forth. Similarly, the teaching strategies of a web-based course span several options, as Table 1.1 indicates.


12

Table 1.1 Types of Web-Based Training Non-Facilitated WBT (Asynchronous)

Mixed WBT Facilitation (Asynchronous and Synchronous)

Facilitated WBT (Asynchronous and Synchronous)

Because it is asynchronous, instruction occurs at a time defined by the learner. Because it does not involve facilitation by instructors or peers, the training is designed to be completed without assistance or complex feedback.

The training includes both non-facilitated and facilitated characteristics. Sample course with mixed facilitation:

The training is primarily based on interaction between the instructor and course participants. This interaction can be synchronous (communication occurring at the same time for all participants) and/or asynchronous (communication occurring between participants at selfdefined intervals, as with email).

Typical WBT elements are text, graphics, streaming video, and multiple choice activities and assessments. Activities and assessments are evaluated via systemmanaged comparison to preprogrammed, correct answers.

A. Non-facilitated, asynchronous component: Students take a series of prerequisite WBT modules to build their skill at writing effective industry reports. In the final module, students are asked to test their knowledge by creating a report and emailing it to an instructor for evaluation. B. Facilitated, asynchronous components: Students email their work to an instructor, who evaluates the quality of each report and then emails the results back to students. The email includes a meeting time and URL for students to meet for an online class discussion using chat. C. Facilitated, synchronous component: Students and instructor meet online at the specified time for class discussion to reflect on the previous assignment.

Facilitated WBT allows instructors to evaluate and respond to student performance; it can also support peer collaboration. Typical asynchronous, facilitating technologies include email, bulletin boards, and listservs. Typical synchronous, facilitating technologies include whiteboards, video conferencing, and chat.


Just as different types of WBT can be combined, a web delivery system can be mixed with other delivery systems for maximum instructional effectiveness. For instance, the training described in the first column of the table could be used as either a prerequisite or a follow-up to a traditional classroom component. Part C of the second column could also occur in the classroom or via video/phone conferencing. The descriptions in Table 1.1 are fairly representative of common web-based technologies. However, technology-based training must be approached from an instructional standpoint to determine how the technology can most effectively support learners. Without first examining the interplay between instructional design and the web delivery system, a barrier might be inadvertently and easily created between students and the training material. This book explores the critical connections between instructional design and web design, primarily using non-facilitated, asynchronous WBT as a reference point. The first step toward effective web-based training is to assess its appropriateness for a specific training need and audience.

Determining the Appropriateness of Web-Based Training We can be charmed by the Web. Virtually every other television commercial invites viewers to visit web sites; presidential figures make reference to the “information superhighway”; and more people “surf” with a web browser than a surfboard. Web environments are also becoming hot training commodities. In an article titled “Train on the Web,” Marianne Kolbasuk McGee writes: “Gartner Group Inc. predicts that technology-based training, including Web-Based solutions, will represent half of all training by 2002...”1 Perhaps a large part of WBT’s appeal stems from its ability to provide self-paced, just-in-time (JIT) training. Indeed, on the topic of WBT, one multimedia training vendor explains that with traditional training, “individuals usually only retain 20%-25% of what they learn, and that retention rates drop as time goes by. With [online] interactive-learning, students can refresh their memory as necessary; retention rates can be as high as 70%.”2 Based on such statements, it is tempting to assume that web-based training is appropriate for all training needs, but this is not true. When used for training purposes, the Web provides a means to an end, not the end in itself. This delivery system should be considered a viable training solution only when it meets criteria from two important categories:

13


14 ♦ ♦

Training goals Audience characteristics and needs

Given these categories, instructional designers select the delivery system only after substantial research into what the training should accomplish, and for whom. For now, let’s look at an overall training development cycle (Figure 1.1) and where delivery selection fits into it. Although this model is specific to our training firm, it is similar to other industry models with which you may be familiar. According to our model, creation of effective training progresses according to eight major milestones. Figure 1.1 Training Development Cycle

Identify Need Define Need

Revise

Evaluate

Training Training Development Development Cycle Cycle

Deliver

Design

Develop Test

Traditionally, selection of a delivery system should not be made until the end of the design milestone. Design focuses largely on gathering audience and training needs, then identifying which training and delivery techniques are best suited to meet those specific needs. Basing design on the delivery system instead of the audience and learning goals can undermine the ultimate success of a project. List 1.1 below outlines the tasks of our instructional-design process. Future chapters address many of these steps in depth, and Appendix

A offers a complete overview of our method. This outline is provided here to illustrate where selection of the delivery system fits into the overall instructional design process: List 1.1:

Instructional Design Tasks 1. Analyze needs-assessment data. 2. Analyze target audience description. 3. Analyze tasks. 4. Identify skill gaps. 5. Identify prerequisite skills. 6. Prepare measurable course objectives. ♦ Describe cumulative and criterion tests.


15

8. Form modules. 9. Determine module sequencing. 10. Analyze client parameters and constraints for effects on design decisions. 11. Gain client approval before continuing. 12. Form logical learning units within modules. 13. Determine sequencing of logical learning units within modules. 14. Determine placement of practices and tests. 15. Identify appropriate types of practice activities and tests. 16. Identify the primary sources of information during the training (job aid, self-instruction, or instructor). 17. Develop a content outline. 18. Identify techniques for presenting content. 19. Determine appropriate delivery system(s). 20. Determine appropriate delivery media. 21. Document details on course blueprint. Identification of the delivery system is one of the final design tasks. All of the steps preceding selection of the delivery system serve as critical inputs to making an appropriate determination. Most importantly, the delivery system should match the needs of the learners and the training goals. The following pages present general guidelines for determining an appropriate match in both areas. Needs Analysis A delivery system should be agreeable to learners from a variety of perspectives. A thorough needs analysis reveals much about who will be taking the course, from audience demographics and entry-level skills to any major issues surrounding learners’ jobs and the training. For instance, what attitudes do they have toward different training they have already received? What do they expect, and what do they prefer? How often do they currently use computers, and for what purposes? What is the nature of their work environment, and is the necessary equipment available? Do they have any particular concerns about training, such as being taken out of a sales field for too long? Also consider the size of the audience. Generally, WBT isn’t costeffective for a small group. Table 1.2 describes optimal audience characteristics in four main areas:


16

Table 1.2: Audience Characteristics Suitable to WBT Attitude

♦ ♦

Computer skills

♦ ♦

Size

♦

Time

♦

To increase training outcomes, learners should be accepting of the Web. Significant resistance to the delivery system can inhibit skill transfer and learner performance. Identify learners’ attitudes toward past training. Prior negative experiences can affect the reception and success of new training. Learners should have computer and Internet (or intranet) access. Learners should have adequate competency in accessing and navigating a web course with minimal coaching. (Course complexity will dictate the required level of computer and Internet literacy.) For WBT to be cost-effective for both instructors and students, it is best suited for large audiences whose learners are geographically dispersed. While a legitimate question is “how large,” there is no truly definitive answer. The number of students who can take a single web course is virtually unlimited. To determine the number of students necessary to make a course cost-effective, we suggest you divide the estimated cost of creating the course by the total number of anticipated students over the life of the course. For more information, see “Expense Schedules” under “Budgeting Guideposts” later in this chapter. WBT is ideal for students and instructors who cannot afford to lose large blocks of time away from work or who cannot afford associated expenses such as travel and lodging. • Although time is required to complete a web course, a web delivery system is ideal for JIT training and can be completed at a learner’s convenience.

Task Analysis In addition to matching audience characteristics, the delivery system must provide an environment that allows learners to develop the desired skills. A task analysis identifies those skills by answering the following questions:


17

♦

What should the students know or be able to do after completing the course? ♦ What subtasks must students complete to achieve the training goal, both prior to and during the training? ♦ Do these tasks have a primarily cognitive or primarily psychomotor orientation? ♦ What kind of reinforcement is required, and how often? From this information and needs analysis data, instructional designers develop a task analysis. These details are critical to selecting an appropriate delivery system and are summarized in Table 1.3. Table 1.3: Training Skill Characteristics Suitable to WBT Skill nature

Objective level

Feedback

Discussion level

Target skills should be cognitively oriented, rather than have a psychomotor orientation. (However, note that psychomotor skills relating directly to the computer, such as programming and software skills, can be effectively taught, practiced, and tested in a web delivery format.) The objective level should be achievable within a web environment and the specified target design. • For example, a multiple-choice quiz and a “drag and drop” exercise each allow learners to recognize or identify, but not necessarily to discuss. • If discussion is required, consider supporting the WBT with email, chat sessions, electronic bulletin boards, or phone or video conferencing, or consider making the WBT a prerequisite to classroom training. The training should not require immediate, frequent, and detailed instructor feedback on learner performance. • Synchronous WBT is possible, but often is less practical and more expensive than asynchronous delivery. Given the nature of the medium, training requiring only minimal to moderate discussion is better suited for web delivery.

18


Characteristics of Training Content As with audience and task analyses, the nature of the target training content is also an important criterion for selecting a web delivery system. As discussed later in this chapter, the bulk of expenses for web delivery accrue during the development phase, more so than for most delivery systems. Thus, it is unrealistic to make large investments in web course development if the training content differs radically from the guidelines listed in Table 1.4 below. Table 1.4: Optimal Content Characteristics Course life

Revision frequency

Presentation consistency

♦ Course life should generally exceed more than a two- or threetime offering equivalent to a classroom course. WBT provides the advantage of course delivery to an unlimited number of students within a broad timeframe. • A web course will usually be more cost-effective in such circumstances. ♦ WBT is well suited to highly dynamic content. A web delivery system provides the ability to update content quickly and easily. • The Web is ideal for content that changes several times per quarter. ♦ A web delivery system offers distinct advantages when training consistency is important. • WBT allows thousands of learners to access the same presentation at all times, regardless of how dynamic the material is.

To explore optimal content characteristics for WBT, let’s examine a current training topic that is highly relevant to human resources—sexual harassment. Although this training is often delivered via classroom with videotape supplementation, the Web offers a promising alternative. Frequently, these classroom courses are taught by lawyers to ensure that the content reflects current case law and to guarantee as much as possible that every student receives the same presentation. Delivering inconsistent training on sexual harassment can place legal liability on a corporation rather than on an individual, should an employee file a harassment complaint. Given this description, the Web is well suited as a delivery system for sexual harassment training for the following reasons:


19

1. Course revisions that reflect current case law can be made quickly and easily. 2. Because learners are guaranteed to receive the same content, employers can hold them legally accountable for that knowledge. 3. Instructional media such as video can be integrated into the training, if desired. 4. Companies can save money by using attorneys as subject matter experts (SMEs) rather than as instructors. 5. The training topic has a long “life expectancy.” There is little indication that sexual harassment training will soon become obsolete. This anticipated longevity offsets potentially high development costs. If the training requires discussion, meetings can be facilitated by human resources personnel after students complete their web training. In this case, mixing WBT with classroom discussion is less expensive than straight classroom delivery, but equally effective. Financial Considerations Even if the proposed training meets criteria in the areas of needs, tasks, and content, financial planning is key to successful course delivery via the Web. Developing WBT requires a significant financial investment early in the project. Realistic development estimates can save you from leaping into the project head first, only to find your funds exhausted prior to completion. Locating or developing realistic estimates, however, is a challenge with which the training industry is well familiar. Kevin Oakes of Asymetrix Learning Systems, Inc., for example, has encountered an outlandish range of estimated times required to develop one hour of computer-based training (CBT)—anywhere from two to 1200 hours, depending on the source. Given this range, Oakes concludes, “The best way to find out how long it will take or how much it will cost to develop a course is to identify a similar course and attempt to discover these figures.”3 Certainly, discovering such figures can become an arduous process, but some general guides are available. To establish basic time estimate ranges, C. Lee and R. Zemke conducted an extensive study that involved training experts from corporate and governmental sectors. Barbara Seels and Zita Glasgow adapted the findings of Lee and Zemke to produce the model that appears in Table 1.5, which approximates the development time required to yield one final training hour by delivery system.4


20

Table 1.5 Development Estimates by Delivery System for Each Hour of Instruction Delivery System Traditional classroom course Self-paced (programmed instruction) Video Computer-based training

Time Required for One Hour of Completed Instruction 40-100 hours 80-120 hours 200 hours 200 – 400 hours

Making Instructional Design Decisions 2/e by Seels/Glasgow © 1998. Adapted by permission of Prentice-Hall, Inc., Upper Saddle River, NJ.

Based on our own data, we have found that WBT tends to fall within the midto-high range of the numbers presented for computer-based training. Although the numbers in this model are not necessarily definitive, they illustrate a pattern and imply an important point for anyone considering a web delivery system: Because development time increases with the intricacy of the delivery system, so will development expenses. Yet it is equally important to acknowledge that if selected for the right reasons, a web delivery system can be most cost-effective in the long-term. Consider, for instance, that a financial firm with several national branch offices plans to deliver the same knowledge-based course to employees in each office. Each branch shares the same intranet that employees access from their desktop computers. In this situation, WBT can deliver the same quality of training at lower cost, particularly since the existing intranet can be used as the platform. Thus, web-based delivery costs will involve placing the course on the intranet, the amount of time required for employees to complete the course, and any help-desk services provided. If this same course were delivered via the classroom, expenses would accrue in the following areas each time the class were offered: ♦ ♦ ♦ ♦ ♦

Instructor (including preparation time) Guides or workbooks for both instructor and employees Classroom facilities and equipment Travel and lodging Time required for employees to complete the course

The difference in expenses for this example might be illustrated as:


21

Cost

2 3

Classroom

3

WBT

1 = Instructional design 2 = Development 3 = Delivery

1 2

Time

Because instructional design involves the same tasks regardless of the delivery system, its expenses remain relatively the same, with the most significant differences arising during development and delivery. The development of web-based courseware requires skills in programming and graphic design, which classroom courseware does not require. For WBT, delivery expenses tend to remain relatively stable and low because a single hosting platform can provide unlimited use to an unlimited number of learners at the same cost. Thus, the greater the enrollment, the lower the delivery cost per learner. Conversely, high enrollment in classroom courses typically raises expenses in the areas of instructors, facilities, instructional materials such as workbooks, and travel and lodging.

Budgeting Guideposts Because the factors of course development often depend on the individual project, it would be misleading to offer a definitive pricing or expense model for WBT. Rather, the above comparison is simply intended to demonstrate a pattern between delivery systems, and even then, this pattern will not apply to every individual training context. As a result, the following pages offer general considerations for budgeting a WBT project. Those considerations are driven by three general principles regarding project complexity, personnel and equipment, and investment analysis: 1—Project complexity: Most design and development costs flow from the complexity of the target courseware, so it is essential to spend adequate time defining what the final training should look like in terms of content, web elements, and support.

22


2—Personnel and equipment: As project complexity increases, so does the complexity of the project team and other resources, equipment, and supplies. 3—Investment analysis: WBT, like all training, should be viewed as an investment in the target training population. Maximizing that investment requires familiarity with the nature and schedule of costs and how to measure returns. As part of sound investing, you should also be aware of nontraining variables that can greatly affect training success. Basic Expenses Basic expenses for creating and delivering a web course depend heavily on the team’s complexity and experience, and the extent to which existing materials, designs, and equipment can be leveraged during development. Table 1.6 outlines where basic expenses for WBT originate. The questions in the “Considerations” column should help you not only identify expenses specific to your needs, but estimate how “costly” those expenses might be.


23

Table 1.6: Itemization of Expenses for WBT Main Expenses Project team

Nature of Expense One-time

♦ ♦

♦ ♦ Analysis: ♦ Audience ♦ Task ♦ Instructional Content design and development

One-time

♦ ♦

One-time

♦ ♦ ♦

Media development

One-time

♦ ♦ ♦ ♦ ♦

Web programming

One-time

♦ ♦

Considerations How many developers does the project require? Are the developers single- or multi-skilled? Can they fulfill more than one function on the team? Does the schedule demand overtime? How familiar are team members with a web delivery system? Is a list of needed contacts already available, or must it be created? Will travel or long-distance phone conferencing be involved? How much content exists, and how much must be created? How familiar are designers and developers with the topic? Are SMEs available for development and review? Is streaming video or audio required? How many graphics will need to be created? Can some graphics be purchased? Is photography necessary? If part of a virtual campus, can an existing interface be used? How interactive will the course be? Is interactivity based on clientside or server-side programming, or both? If server-side, are resources such as processor speed and memory adequate?


24 Main Expenses Hosting (For more information, see Appendix B.)

IT/help desk support

Nature of Expense Ongoing

♦ ♦

Ongoing

♦ ♦

Instructional support

Ongoing

♦ ♦ ♦

Course maintenance

Ongoing

♦

♦

Considerations Does an infrastructure already exist, or will hosting be outsourced? How much growth is anticipated in the training population over the next three years? Will learners need upgrades, plug-ins, or speakers? Will defined hours of technical support be needed? Will an instructor or SME be needed to provide student feedback? What methods will be used for feedback, and how often? Will feedback be synchronous or asynchronous? How often will the course be revised, and are changes in audience or content expected to affect web elements such as graphics and navigation? What skills are needed to maintain the course?

Expense Schedules In addition to knowing the range of costs that can be associated with a web delivery system, the schedule of costs must be anticipated for a reliable WBT budget. Because web-based training is generally characterized by frontend loading the costs, the bulk of expenses are incurred during the first half of the project and then often diminish over the life of the course if it truly is the appropriate delivery system for your audience and training needs. As such, the financial feasibility of WBT is affected by two main variables: 1. The size of the target learning population (includes projected number of learners over time) ♦ Total expected life of the course


25

Table 1.7 compares expense schedules for three delivery methods: classroom, computer-based training (CBT), and web. Table 1.7: Expense Schedule Cost by Project Phase for Classroom, Computer-Based, and Web Delivery Systems Project Phase Audience analysis Design Development

Classroom Delivery CBT Same Same Low High ♦ Content ♦ Content creation creation ♦ Graphic design ♦ Graphic design ♦ Programming (marginal) ♦ Usability testing

Delivery

High ♦ Instructor ♦ Classroom ♦ Classroom supplies and equipment ♦ Travel/lodging High ♦ Study guide reprinting ♦ Study guide redistribution

Maintenance and updates

Moderate ♦ Instructor support ♦ Production and replication of CD master ♦ Mass distribution High ♦ Creation of new CD master ♦ Reproduction of CD ♦ Redistribution

Web Delivery

High ♦ Content creation ♦ Graphic design ♦ Web programming ♦ Usability testing ♦ Servers/hosting contractor Low ♦ Possible instructor/SME support ♦ Data load onto server for online access Low-Moderate ♦ Instant online publishing ♦ IT support

To compare the monetary commitment WBT requires with other delivery systems, a cost comparison per student is useful. Using your own model or the one provided in Table 1.5, estimate the total cost of the desired delivery system and then divide by the number of assumed learners; this will give you a rough estimate of total cost per student. This number can be viewed as a pricing guide and indicator of how many students are needed to make the training financially feasible. For comparison, make the same computation for other delivery systems you are considering. (Note: according to the expense


26

schedule presented in Table 1.7, over the same amount of time, the price of WBT per student should decrease, whereas the price per student for classroom and CBT delivery will typically stabilize or increase.) ROI and Business Metrics Compared to approximating a WBT project Training ROI budget, anticipating the return on investment (ROI) can be a monumental, elusive task; + savings earned by however, ROI is often the measurement in training solution which managers are most interested. And to - cost of solution compound the elusive nature of ROI, this = training profitability measurement has been defined in several ways. For example, computations of ROI vary, depending on what is being measured — return on assets, stockholders’ equity, profitability, etc. For clarification, we’ll define the profitability of training ROI as the savings earned by a training solution less the cost of that solution. Trish Cullum, one of our project managers with a background in evaluation consulting, cautions that the expense of a properly performed ROI analysis can rival the cost and complexity of the actual target training. Consequently, she recommends that ROI analysis be conducted only on a business’s most critical curriculum, instead of on a course-by-course basis.5 The following four steps should improve the validity of training ROI computations. Before and during implementation of a web course, support your investment by: 1. Selecting appropriate, measurable business metrics or objectives The more specifically these objectives can be worded, the better your chances of being able to meet them and of measuring how well they are met. For example: ♦

Decrease the turnover rate to X percent in Department Y within the next year ♦ Improve time-to-market at X percent for Y product beginning Z quarter ♦ Increase sales of Service X to customer group Y at Z percent over the next two years ♦ Reduce Q manufacturing errors for X product on Y line by quarter Z


27

To create business objectives at this level of specificity, you’ll need to have a good understanding of the business problem the target training must address. 2. Creating a system to measure those objectives Before implementing the training solution, identify an initial reference point that describes current conditions. In other words, take some time to complete a pretraining assessment in terms of the business objectives you have selected. To measure the last objective above, for instance, you’ll need to know how many manufacturing errors occur on that line during a given period and how much those errors cost the line in terms of time, parts, and possibly sales and returns. 3. Creating a system that supports the stated business objectives It’s not enough to set business objectives and to offer training. The business, department, or manager should help trainees see the connection between the business objectives and the training. Without those connections, the impact of the training on business goals can significantly diminish. More appears on this topic later in the chapter. 4. Selecting a reasonable period of time to measure the objectives and ROI To determine an appropriate time frame for measuring ROI, consider: ♦ The magnitude of the business objectives and target training ♦ On-the-job support mechanisms needed to help learners transfer skills from the training to their jobs ♦ How quickly the use of training skills can realistically affect the business objectives A comprehensive ROI analysis typically incorporates data from many parts of a business, such as finance and management staff. Because simply attempting to gather this data can create political tensions, Cullum recommends securing buy-in from everyone involved before initiating an ROI project. “Few business departments are interested in providing data that could potentially expose a weakness in their processes.”6

Maximizing WBT Investment Gains The financial success of WBT, or any training for that matter, does not depend solely on the quality of design, the development, the instructor, or the delivery system. More trainers and organizations are recognizing that a

28


business must support training efforts before, during, and after students complete a course. Because web-based training can be a substantial investment, it is important to consider possible methods of supporting learners of the target courseware. Donald L. Kirkpatrick’s widely accepted guide was originally designed to evaluate training on four levels. (Some adapted models include additional levels.) As described by Nancy Chase in “Raise Your Training ROI,” Kirkpatrick suggests that maximizing training ROI does not simply involve numbers and knowledge, but behaviors.7 According to the scale, training is measured in terms of reactions, learning, behavior, and results: 1. Reactions: Students indicate their satisfaction level with the course using an evaluation form. 2. Learning: Pre- and post-test scores are used to measure the extent to which students learned the material. 3. Behavior: A supervisor or other designated person from the workplace assesses skill transfer between the training situation and actual performance context. 4. Results: This level focuses on the training’s financial results, often using ROI.8 In many ways, level three—behavior—is the cornerstone of training effectiveness and financial returns. Students’ learning of and reactions to training material have little effect in an organization if that organization does not encourage learners to actually apply on the job what they have learned. From this perspective, responsibility does not rest solely with students. Rather, the organization and its managers have much to do with training success in general and skill transfer in particular. Nancy Chase of Quality Magazine offers a plan for heightening skill transfer from training to the job. Such a plan, she explains, hinges on achieving “congruence” among business objectives, training objectives, and job performance: 1. Prepare students by conveying the purpose and advantages of training before it occurs. 2. Avoid disrupting students during the training period with phone calls, potentially volatile email, or an increased workload. 3. After training, form a plan with employees for applying training skills on the job. ♦ Assess job performance in terms of the new training skills.9


29

While a web delivery system cannot in itself accomplish all of these steps, it can facilitate completion of them and therefore potentially raise financial returns for the actual web course and job performance. Much talk is spent on taking advantage of web environments regarding interactive exercises and learner-defined sequencing. These features are certainly important, but focusing on these alone can reflect a slightly shortsighted view of the Web’s advantages. A web delivery system can not only deliver a course, but support its effectiveness for both learners and managers in several other ways, a few of which are outlined below: Before Training: ♦ ♦ ♦ ♦

For Learners Message from president or manager explaining purpose of training Diagram showing how training fits into the company’s strategic business plans Course description showing its relationship with other courses List of courses that correspond to job positions, including promotions

After Training: For Learners ♦ ♦ ♦

♦

10-minute review sessions/exercises Online job aids Templates for self-evaluation regarding skill transfer: In other words, ask learners to assess how well they’re applying the training to their jobs. Additionally, ask them how the course could improve or heighten skill transfer. Chat/bulletin boards through which employees share how they are applying their new skills

♦ ♦

For Managers Chat/bulletin board with course instructors to maintain a dialogue on needs and expectations Suggestions for preparing learners to make connections between course material and job responsibilities: One method is for the manager to prioritize course objectives and share with his or

her employees, explaining the rationale for those priorities.

♦ ♦ ♦ ♦ ♦

For Managers Follow-up and evaluation strategies for a particular web course Template for a plan to apply training on the job immediately Access to other managers for sharing strategies to encourage skill transfer Access to student scores and follow-up training tips for on the job A list of related course work available to employees

If these options are associated with a particular course, in addition to capabilities that track learner performance, WBT does more than simply


30

offer a training session: it also offers learners, managers, and instructors a training management system. Such a system can also shorten development time for future courses, because those courses can leverage or adapt existing interface designs and be hosted on the same system.

Conclusion For the right audience, training goals, and content, a web delivery system can provide instruction that is highly cost-effective and that can significantly raise learning outcomes by providing training in an often learner-directed, JIT format. If WBT does not meet key criteria for the audience and training goals, however, a web-based course can become a detriment to, rather than an investment in, the target learning population. To avoid this risk, instructional designers must choose the best delivery system toward the end of the design phase whenever possible. This sequence allows instructional designers to evaluate the appropriateness of a given delivery system based on: 1. 2. 3. 4.

Audience needs and characteristics Training goals Performance tasks the training must teach Financial feasibility

Despite these criteria, we acknowledge that instructional designers must balance idealism with realism: they must contend with budgets, deadlines, and numerous contingencies. Many instructional designers face the challenge of entering training projects for which the delivery system has been predetermined by a manager or customer. In these cases, the designer should take some responsibility to educate the customer about potential risks, particularly when WBT is involved. This chapter concludes with WBT project evaluation forms for assessing the appropriateness of a web delivery system, given your particular training context. These forms offer one method of examining the viability of a web delivery system, but they cannot expose all of the challenges and hidden costs entailed in creating WBT. In exploring basic issues throughout this chapter, we hope we have given you a basis for analyzing your own training context. The following questions are provided to further your analysis. They can be useful when creating or evaluating a WBT budget or when seeking a vendor. (For information on outsourcing, see Chapter Nine.) Some of these questions can also help a vendor address customer concerns:

Assessing the Appropriateness of Web-Based Training ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦

31

On what assumptions is the budget based? Does the budget allow for any contingencies? Do all budget items clearly support project goals? Does the budget include all phases of the project, or does it only cover production time? Are schedules provided for each phase of the project? Are proposed schedules realistic? Can any existing training content be leveraged? How much is known about the target audience? How experienced are potential team members? Have team members worked together before? How are team members’ rates calculated? Does the team have prior experience with needed development tools? Do any development tools need to be purchased? Will the team need to conduct any research? Does a server infrastructure already exist, or must it be created? Are hosting expenses itemized? Does the budget include long-term plans for end user help-desk services?

Typically, a complete WBT project team will not be finalized until these critical aspects of the project are defined. As you answer the questions posed throughout this chapter, however, you must also begin thinking about who will be needed on your team and the range of skills common to the design and development of WBT. The next chapter focuses on these issues.

WBT Project Evaluation Forms These criteria indicate the appropriateness of an asynchronous web delivery system, given an identified training goal and target audience. Data from these forms are just one indicator of project feasibility and should be supplemented with additional evidence where possible. These criteria are separated into three levels: Minimum Target Training Requirements, Financial Practicality, and Optimal Training Environment Characteristics. While some issues overlap between these categories, they are separated as a means of prioritizing the criteria.


32

Minimal Target Training Requirements Use the “evidence” column to describe actual circumstances of the training need or context relevant to the criterion in the first column. Criteria (for minimal requirements) 1. Minimal need for discussion

Evidence

2. No need for immediate instructor feedback

3. Performance tasks are measurable in asynchronous web environment.

4. Learners have (or will have) adequate computer/Internet literacy and access to appropriate computer system. 5. Adequate budget for target web design and hosting infrastructure

Preliminary assessment or special considerations:


Financial Practicality Use the “evidence” column to describe actual circumstances of the training need or context relevant to the criterion in the first column. Criteria (for financial practicality) 1. Target audience large enough for cost-effectiveness

Evidence

2. Course has long anticipated shelf life

3. Content requires frequent updates

4. Critical for all learners to receive exact same content or presentation


33


34

Optimal Training Environment Characteristics Use the “evidence” column to describe actual circumstances of the training need or context relevant to the criterion in the first column. Criteria (for optimal training environment) 1. JIT (just-in-time) training is needed.

Evidence

2. Learners have positive attitude toward WBT, training in general, or both.

3. Learners and instructors cannot afford time or expense for travel to classroom.

4. Instructors or managers need method to simplify training administration and record keeping.



35

References 1.

McGee, Marianne Kolbasuk. “Train on the Web.” InformationWeek Online. Cited 28 January 1999 from www.informationweek.com; INTERNET.

2.

Integrity Training, Inc. “Interactive Learning: What Is It?” 2. Cited 1 February 1999 from www.integritytraining.com/wbtcbt; INTERNET.

3.

Oakes, Kevin. “The Hardest Question to Answer about CBT,” 1-2. Cited 1 February 1999 from www.asymetrix.com/services; INTERNET.

4.

Seels, Barbara and Zita Glasgow. Making Instructional Design Decisions. 2d ed. (Upper Saddle River, NJ: Prentice Hall, 1998), 240.

5.

Cullum, Trish. Interview by the author 30 September 1999.

6.

Ibid.

7.

Chase, Nancy. “Raise Your Training ROI.” Quality Magazine (September 1997), 7-8. Cited 1 February 1999 from www.qualitymag.com/training; INTERNET.

8.

Kirkpatrick, Donald L. Evaluating Training Programs: Four Levels. (San Francisco: Berrett-Koehler, 1994), 21-25.

9.

Chase, Nancy. “Managers Make Training Stick.” Quality Magazine (April 1998). Cited 1 February 1999 from www.qualitymag.com/training; INTERNET.


Chapter Two

Forming and Managing a WBT Team Chapter topics: ♦ Picture of a WBT team • Selecting team members • Common team members and skills ♦ Project-management strategies ♦ Activity: Team orientation – WBT review session Chapter resources: ♦ WBT review guide ♦ WBT review worksheet

Picture of a WBT Team A WBT team looks quite different from either a pure web-development team or a pure instructional-design team. Because developing training for a web delivery system requires skills from both areas, such a team is an intricate unit composed of widely diverse competencies, needs, and expectations. If the team lacks any core skills, your clients and students will most likely find the WBT lacking, too. This chapter covers the main roles and skills you should become familiar with, whether you plan to hire an in-house 37


38

staff or to outsource your WBT projects. For information specifically on outsourcing, see Chapter Nine, “Putting It All Together.” Selecting Team Members A tight budget is perhaps the most common temptation for limiting a WBT team. Well-meaning organizations can have impractical expectations that inadvertently compromise course development and quality. Such expectations might include setting unrealistic schedules or perhaps relying on programmers to double as instructional designers and vice versa. In these situations, it is important to consider how such changes can affect the team, its outputs, and its schedules. When determining who’s needed on the team, consider: ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦

Scope and complexity of all project deliverables, including possible interactivity, such as instructor feedback or student collaboration Skills needed to produce deliverables Deadlines for all deliverables Required equipment and resources Existing training content that can be leveraged Availability of subject matter experts (SMEs) How the courseware will be loaded and maintained on the server Whether a series of courses will follow the initial course, as in a curriculum

Many of these points should be covered in the client’s request for proposal (RFP). Certainly, all of these points should be addressed in your proposal and be discussed not only with the client project manager, but with his or her team as well. If the client has limited experience developing or purchasing WBT, it’s useful to spend some time—even briefly—to highlight key differences between a team that designs for classroom delivery and one that designs for the Web. For example, although the members of an instructional design team will vary depending on project scope and the needed skills and resources, basic members usually include a project manager, an instructional designer, and a course developer (production person). Given a relatively small training project with a reasonable development schedule, a skilled team of one or two people can produce a quality self-paced workbook or classroom course. With traditional classroom delivery, it is not uncommon for a project manager to function both as instructional designer and production person, especially given the userfriendly desktop publishing and presentation software currently available.

Forming and Managing a WBT Team

39

In contrast, producing that same workbook or course for web delivery requires a broader range of skills. The paragraphs below outline typical members of a WBT team; although we have provided a comprehensive list, an individual project might not require a separate person to fulfill each role. To help those interested in outsourcing a project, we have also indicated how project responsibilities are often delegated between clients and vendors, depending on the available resources and project scope. If you develop training from a nonvendor perspective, you might not be accustomed to client-vendor terms. For clarification, think of the “client” as the person or people who have requested a course to be created, whereas the “vendor” is the person or people responsible for actually creating the course. Thus, the list indicates typical relationships between “those who request the training” and “those who create the training.” Common Team Members and Skills Business Sponsor (Client) This is a high-level sponsor from the client organization who provides corporate and possibly financial support. This person should be in a position to resolve internal client conflicts if they arise and to remove roadblocks to project development. Another important role of a business sponsor is to champion WBT as an alternative to traditional delivery media. As noted in Chapter One, training success can be improved if an organization offers support to learners in transferring skills from training to the job. A business sponsor is key in forecasting this need to managers. Project Manager (Client and Vendor) The client project manager serves as the primary customer contact who is responsible for the project within his or her company. Vendor project managers are usually responsible for overseeing the production of all aspects of the project, from content to graphics and web programming. He or she must also function as the main contact point for the client and coordinate hosting options with members of the information technology (IT) department. As part of guiding the project team and development process, the vendor project manager should also work with the client to define how the course will be accessed, used, maintained, and supported.

40


Subject Matter Expert (SME) (Client) An SME has extensive background knowledge in subject areas from which the training content will be derived. The SME works with the project manager, instructional designer, and course developers to identify audience and training needs. As part of this responsibility, the SME also provides and reviews course content. Additionally, the SME will have contact with graphic artists and web programmers to confirm that the training implementation indeed meets participants’ needs. Subject matter experts might also coordinate with the client’s marketing organization to raise awareness of course availability. Instructional Designer (Vendor) The instructional designer works closely with other team members to make certain that the training is founded on solid course design principles and is smoothly integrated with the capabilities of the web environment. The instructional designer is responsible for developing course objectives, defining and sequencing learning units, and designing activities and evaluation methods, all in accordance with learner needs and training goals. Instructional designers should also participate in evaluating how well presentation techniques and media (user interface, graphics, and video, etc.) contribute to overall training effectiveness. Course Developer (Vendor) This production person has some background in instructional design. He or she works closely with the instructional designer to develop course content according to the instructional designer’s blueprint. Course developers also collaborate with other team members, such as graphic artists, to achieve conceptual alignment between the course design and web design. Editor (Vendor) This person should be experienced with writing and editing web documents. As discussed in Chapter Six, the writing style of a web page can substantially enhance or degrade web usability. A solid grasp of page layout principles is also important, as well as the ability to follow relevant style or documentation guidelines of the client. Graphic Artist (Vendor) The graphic artist should be skilled in interface design and usability principles. He or she will collaborate with instructional designers and course


41

developers to create the user interface, page layout, and graphics that reinforce learning. Additionally, the graphic artist should be familiar with techniques and issues that affect graphics creation for the web, including file formats, color palettes, and bandwidth concerns. Working knowledge of a variety of graphics applications and multimedia tools can broaden the scope and professionalism of training deliverables. Web Programmer (Vendor) The web programmer codes course interactions and activities as specified in the blueprint, which can require some collaboration with the instructional designer and course developers during the blueprinting process. In general, the web programmer is responsible for all programming design and coding. In addition, the web programmer will work with the database administrator to develop the application programming interface (API) for transferring data (such as reinforcement feedback and test scores) to and from the database. If the course will be loaded on the client’s server, the web programmer will have to work with the client’s database administrator and systems administrator to develop the APIs and load the courses. At a minimum, this person should know HTML and JavaScript. Working knowledge of an application server (such as Allaire ColdFusion™) and a database (such as Oracle™) are also assets. Database Administrator (Client or Vendor) The database administrator (DBA) is responsible for designing and maintaining the database. At a minimum, the database tracks users and test scores, although it can track many more dimensions of a course and training program, depending on the intricacy of the project design. The DBA must work with the project manager and instructional designer to determine datacollection requirements, and then create a design to meet those requirements. This person will also work with the web programmer to determine how data is transferred between the course and the database. Systems Administrator (Client or Vendor) The systems administrator is responsible for loading course materials onto web servers and then maintaining these servers. If the client fills this role, the vendor should have some contact with the systems administrator for technical constraints that potentially affect the web delivery system. In addition, this person can be responsible for maintaining the network connection and system security, evaluating server performance, and

42


performing load-balancing tasks. Other administrative tasks include performing backups and maintaining version control. Technical Sponsor/IT Representative (Client) This person provides client infrastructure information, troubleshoots technical difficulties, and resolves issues with the information technology department. This role will not be as challenging if a web infrastructure is already in place. Finance (Client or Vendor) If you are developing an online, on-demand training course for public access, a financial specialist is critical for developing a process of monitoring course/site usage and fee collection. This person will need to work with database administrators who are responsible for maintaining server logs and possibly an e-commerce transaction server. This may take the form of debiting a purchase order account or processing a credit card. Marketing/Sales (Client) These people fulfill two primary functions. First, they can help course and web developers better understand the expectations and interests of a web audience. Second, they should position a course for acceptance, either by an internal, corporate audience or by a more diverse web audience if the training will be available for public access. Support Staff (Client) These people support end users by answering learners’ questions via email or phone. Such a staff might include SMEs or instructors for a set number of hours during the week, according to student needs. Some members of the support staff should also be able to answer sales and technical questions. Additional Team Members (Client or Vendor) Depending on project requirements, additional team members might be needed, such as an evaluation specialist, a photographer, or an e-commerce specialist. Conversely, keep in mind that not all WBT projects will necessarily need all of the team members described, and some team members might be qualified to fulfill more than one role. You might have a course developer also function as the editor, for example, or the project manager perform instructional design tasks.


43

Project-Management Strategies Not surprisingly, managing such a team can be challenging. From the description above, an ideal project manager of WBT will have experience in four major areas: instructional design, management, web development, and web server technology. Familiarity with these areas should prepare the manager to successfully handle what might be his or her most formidable task—getting all team members working together smoothly, toward a unified goal. Diverse team members can easily begin a project with equally divergent, sometimes conflicting interests and expectations. For instance, graphic artists and web programmers might have difficulty understanding the work process of course developers. IT is traditionally concerned with budgets, staff, and network and system performance, all of which are affected by a web-based course. The project manager might also have to coordinate efforts of the business sponsor, management, finance, marketing, and support, as well as gather data from SMEs. In short, a WBT project manager must be able to work in a high-stress environment, have good time-management skills, and be able to communicate with and motivate the entire team. If the project manager’s highest responsibility involves motivating everyone to work toward the same vision, he or she must take a critical first step: Convey that vision to or develop it with the entire team as early as possible. In communicating project vision, Barbara Seels and Zita Glasgow, authors of Making Instructional Design Decisions, advise that project managers are “responsible for a plan that lets each group member know what is expected and when, but each group member needs to be familiar with the total plan as well.”1 From our experience, this approach is critical for a WBT project to progress smoothly, because it is the first step to coordinating skills and tasks. We work on this coordination from the beginning of a project, rather than waiting to encourage it midway. We act on this goal in initial team meetings by: 1. Allowing team members to read any preliminary project documentation prior to the initial project kickoff meeting 2. Communicating the overall project plan 3. Outlining tasks and responsibilities 4. Answering any questions regarding the proposal or design document 5. Discussing workflow and the interdependencies between diverse tasks

44


During this process, we clarify other critical points, such as expectations regarding performance and deliverables, and confirming the availability of team members. Once a WBT team is clear on these issues, it is a good idea to clarify related expectations and responsibilities with members of the client’s team. In a client-vendor relationship, there’s little worse than hearing the phrase, “Weren’t you supposed to do that?” When setting expectations, a project manager should not only indicate standards of product quality, but should ask team members to consider what expectations they have of each other. How do individuals prefer to communicate—via email, weekly status meetings, phone? What are the repercussions of missing a meeting or an internal deadline? How much information needs to be shared regarding different aspects of the project? For example, how much do course developers need to know about the possibilities of different scripting languages or bandwidth restrictions? Given the diverse group, is it useful for everyone to attend all meetings? Opening communication on these preferences and needs early can ensure that the project is launched with everyone “on the same page.” It can also help establish rapport that people can rely on during high-tension phases. Some teams find it useful to document these preferences and needs. Discussion of these issues is important, but with all this talk, how do things get done? Like any project, WBT development must be streamlined and task-oriented to progress at a reasonable pace. Time-to-market is also an issue if the course is designed for profit through public Internet access. One strategy for maintaining communication and cooperation on a large team is to designate “branches,” or smaller groups, according to the three major areas of competency—instructional design, web development (includes programming and graphics), and server-side infrastructure development. A team leader is identified for each branch. These leaders play an essential role in performing early quality assurance, monitoring development processes, and often filtering questions and ideas to other branches and to the project manager. They can also help focus people’s efforts toward established project goals. From our experience, this team structure does not pre-empt open communication between team members, but instead facilitates communication. An outline of the scope and structure of a common vendor team appears below. This particular team created web-based courses for corporate intranet access.


45

Team Configuration for WBT via Corporate Intranet Access Project Manager Instructional Branch (identify team leader) ♦ Instructional designer ♦ Course developers ♦ Editor Web Branch (identify team leader) ♦ Graphic artists ♦ Web programmers Server-Side Infrastructure Branch (identify team leader) ♦ Web server administrator Finance (identify team leader) ♦ Financial specialist ♦ E-commerce specialist This model assumes that the WBT client will supply people to fulfill the following or similar roles as needed: ♦ ♦ ♦ ♦ ♦

Business sponsor Subject matter experts Technical representative Marketing/sales Support staff

Bringing experts in from so many areas to share a single project vision can be daunting. The key is to encourage everyone to share expertise in productive, collaborative ways. The rest of this chapter offers a strategy we have found effective, which we describe in the activity below. A WBT review guide is also included as an end-of-chapter resource, and we recommend that team members read the guide before participating in the activity. The guide can be thought of as a “crash course” in web design from an instructional perspective. We offer it in this chapter as a highly condensed, self-contained reference for the sole purpose of putting all team members on the same page very quickly. Key points from the guide are addressed in more depth throughout the book.

46


Activity: Team Orientation — WBT Review Session Shortly after the initial project meeting, taking one more step toward a shared vision and open communication can bring an entire team up to speed quickly. That step focuses on comparing team members’ impressions of other web-based courses and reviewing a handful of favorite web sites. This is an excellent way to gain insights from multiple perspectives and backgrounds, and should also help generate ideas for what your WBT might eventually look like. In this regard, it’s also useful to invite your client to a similar session. If you are developing training for public Internet access, include some of your competitors’ sites in this review. The purpose is not to leverage their material or design, but to identify what expectations have been established within a particular training market and how your course can match or exceed them. Suggested Materials ♦ A computer with Internet access (More than one computer may be necessary for all team members to see the screen.) ♦ WBT Review Guide (appears at the end of this chapter) ♦ WBT Review Worksheet (appears at the end of this chapter) Suggested Approach 1. Ask team members to read the WBT Review Guide prior to the review session. 2. Visit selected sites and discuss them as a team, taking notes on the WBT Review Worksheet for reference or further discussion. 3. Discuss how the review experience relates to the target courseware the team will develop. Alternative: To save time, ask team members to complete Steps One and Two above individually, and to bring their completed worksheets to a meeting described in Step Three. (Make sure everyone on the team visits the same preselected sites or courses.) During Step Two, team members should answer a variety of questions that cover both training and nontraining issues:

Forming and Managing a WBT Team ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦

47

Do you like this site? Why or why not? Is it visually appealing? Does it download fast enough? Is the navigation intuitive? Does anything in the design distract you from content? Can you always tell where you are within the overall course or site architecture? Can you return to the homepage from any screen? How easily can you tell what the company does or offers? Would you visit this site again? How easily can you gauge what the course is about? Is it clear how long it should take to complete the course? Who is the target audience for the course, and how is that conveyed? What initial skill level is assumed? Is the course designed for training or educational purposes, and how can you tell? Is a course description available? Are the course objectives available and well written? Are there any support systems for course participants, such as phone or email contact with SMEs?

During Step Three, ask follow-up questions: ♦

Given our audience and topic, what kind of learning experience do we want to create? ♦ What web capabilities or technologies can help us create that experience? This type of activity can help a WBT team “gel” quickly and avoid flying blindly into virtual oblivion. In particular, course developers who often design for classroom delivery find this activity essential for rediscovering basic WBT principles. When we asked three of our staff members to reflect on the value of this activity after completing it, they offered several insights: Kathy Foster: “It’s useful to see how other trainers address people’s expectations for what a virtual classroom is or can be. Once you see the boundaries in other web courses, you start to evaluate why those boundaries were created and whether your course can challenge them


48

effectively, without sacrificing audience needs or instructional design goals.” Eve Spooner: “It’s important to look at a variety of web courses when you’re designing your own. The variety highlights how important the interface is, because it so often forms a barrier between students and content. Just being in front of a computer and taking a course that’s poorly designed heightens your commitment to your own target audience.” Heather Jarvis: “When reviewing what’s available on the Internet, it’s relatively easy to notice web and instructional design flaws in WBT. However, equally important are the successful aspects. We tend not to notice them as easily because those aspects make us comfortable in the medium. Effective designs can be easy to miss because we accept them unconsciously.”

Conclusion A WBT team is more than a group that develops web pages, and more than a group of instructional designers. Teams that create web courses must span both realms of expertise, and sometimes more. To keep such a varied collection of people working effectively, the web project manager should not only be familiar with all areas, but should be highly skilled at communicating, coordinating, and facilitating. We have found three main approaches particularly useful for WBT development teams: 1. Guide the team toward a single vision: ♦ Convey the entire project plan. ♦ Clarify roles and responsibilities within the plan. ♦ Allow team members to read any initial documentation pertaining to the project. ♦ Designate a leader for each branch or group working on the project. 2. Set expectations: ♦ Identify evaluation criteria for product quality, delivery schedules, and so forth. ♦ Discuss and agree upon methods of working together.


49

3. Share expertise to support learning and improved product quality: ♦ Launch the project by sharing impressions, as a team, of selected web courses. In terms of instructional design, a project team is often formed after a thorough analysis of training and learner needs. The next chapter focuses on a general approach to analyzing those needs and explores relevant web considerations.

50


WBT Review Guide This guide highlights essential areas of interplay between basic web design choices and instructional considerations. It also includes some observations from our own project teams. Main issues from this guide are expanded on in future chapters. In Web Navigation: Designing the User Experience, Jennifer Fleming offers strategies to reduce user frustration toward sites that focus on shopping, informational learning, and entertainment. End users often surf these sites by choice at their convenience. Given the nature of this surfing, these end users are typically not accountable to anyone for remembering or applying the information they have encountered. Consider, then, the importance of reducing user frustration for people who are taking a web course. These people are probably under time pressure for a variety of reasons. They have meetings, project deadlines, and customers to contact. They might also be taking the course in a training lab with a packed schedule and limited space. WBT demands an intuitive design, from a clear purpose and navigation routes to useful interactivity and appropriate writing style. The following sections highlight basic analysis points in three major categories: 1. Course Architecture 2. User Interface 3. Writing Style Early discussion of these categories should help team members focus their efforts, develop a collaborative attitude, and reduce gaps between their outputs. These issues are briefly explored here to serve as a starting point for the team. Course Architecture Information architecture refers to the broad-scale structure of a virtual environment. It involves several aspects that define the location and availability of information. Thus, the architecture of a course encompasses more than the sequencing of activities and modules, but partially defines how learners will perceive the overall site purpose, organization, and


capabilities your course offers. Architecture is not just a cosmetic sheen applied after all the course content has been gathered and coded into HTML. In Information Architecture for the World Wide Web, Louis Rosenfeld and Peter Morville liken web architecture to physical architecture: If the construction of a house, restaurant, or jail is based on who will be using it and why, should web site construction begin any differently?2 Moreover, why should WBT construction begin any differently? Constructing a training site or course without first considering its main purpose and how users expect information to be organized often leads to a product with limited utility. In other words, architecture should derive from purpose and learner needs, not be imposed on them. Significantly delaying exploration of architectural issues will unnecessarily complicate the integration of all WBT elements later in the project. Such delay could also require team members to grapple with and rethink fundamental design concepts instead of refining a solid prototype or alpha draft. Our teams’ preliminary discussions about course architecture are driven by our knowledge of the target audience, our understanding of course goals, and our analyses of other WBT. From exploring those sites, we narrow and evaluate a seemingly infinite field of architectural possibilities. We discuss approaches to architecture that help us achieve our instructional goals and facilitate the learning process for students. At this phase, we consider several questions regarding audience: 1. How does the client articulate the purpose of taking the course? 2. For what information and skills must learners be accountable after course completion? 3. How familiar are target learners with web navigation? 4. What sorts of sites are the target learners accustomed to browsing? 5. What activities do learners typically perform while browsing? 6. How much self-direction will they expect? 7. At what point can self-direction cause anxiety for this audience? Some of these questions seem self-evident, yet require thorough examination; the answers point our teams toward architectures that are tailored to specific goals and learning populations. Consider, for example, how architecture will differ between a nontraining informational site designed for sophisticated end users and a

51

52


web-based course developed for a target audience with limited computer experience. The first audience will expect a high degree of selfdirection—the opportunity to follow their own interests throughout the site, which might include links to other sites. These end users will be concerned with speed. Detailed navigation instructions can become an irritation. In contrast, the second audience—the one taking WBT—is accountable for its performance in the course, and as such, the architecture must support their student-driven needs. Designers of WBT are responsible for logically sequencing information and emphasizing points that will be tested. They must also provide information to help students perform tasks, including guidance for navigating the course and engaging in multimedia activities. Effective web architecture hinges on site purpose and end user needs, and this is no different when designing a web course. The Yale C/AIM Web Style Guide of web design offers guidance in this area. On the topic of training, writers of the manual, Patrick Lynch and Sarah Horton, explain that “Restricting the links to ‘Next’ and ‘Previous’ paging functions guarantees that everyone sees the same presentation, and allows you to make more accurate predictions of user contact time.”3 Additionally, restricting learners to the same course presentation increases the validity of test results. Depending on audience and purpose, the creators of WBT might want to afford participants a high degree of flexibility, but instructional designers are also responsible for designing the training experience. If we provide too many links (either within or outside the site), students cannot be assured they have received all the information on which they will be tested. If we do not guide the information flow in some fashion, we might inadvertently degrade training quality by allowing an incoherent organization to unfold. In turn, student performance can be compromised if lower-level objectives are not met before proceeding to higher-level objectives. For more information on architecture, see Chapter Five. User Interface (UI) The user interface—what people see on a web site and how they navigate it—provides clues to how the site works on a fundamental level. An intuitive interface for WBT is critical for course success because it’s the filter through which participants view a course and a tool that guides user behavior. As a result, the UI also affects student attitudes and


53

performance. In her discussion of user-centered design in Web Navigation: Designing the User Interface, Jennifer Fleming points out that “The challenge, then, is to make your site as easily learned as possible. This doesn’t mean dumbing down content, just anticipating problems and providing for them beforehand. The first thing you’ll need to do is dumb yourself down: your own in-depth knowledge of your site can make it difficult to understand a visitor’s perspective.”4 In addition to gathering needs analysis data on a specific audience, our teams follow Fleming’s advice by becoming end users during WBT review sessions. By role-playing the participant in other WBT courses, we gain valuable distance from our own development efforts. We then examine our own courses with a more critical eye. Interface design includes numerous aspects. For the purpose of a review session, we seek answers to basic questions, then modify our review scope accordingly: ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦

What are the main methods of site navigation, and what level of instruction is necessary? Is it clear when end user action is required? What menu formats are used? Do the menus satisfactorily indicate existing content? What does the site look like? Is it visually appealing? Do graphics and animation enhance the content? Is text easily readable? How far or often must users scroll to access text? Do orienting features help learners locate their position within the course? Do links lead to sufficiently predictable destinations?

As these questions guide our team exploration of web sites and courses, we often find several impressive user interface designs. However, we also typically notice trends in design flaws: ♦ ♦ ♦ ♦ ♦

Inability to distinguish which graphical items are meant to be clicked Uncertainty of how to move to the next or previous page Distracting color schemes Lengthy download times Too many links on one page

54


♦

Failure to indicate whether the user has previously visited a link, page, or module ♦ Text that is too small and blends with background color ♦ Graphics unrelated to content ♦ Broken links If learners cannot discern how to navigate a site, cannot easily read the content, or cannot access all content, they will feel doomed. As course designers, we certainly want to allay even mild learner anxiety. So, one of our main goals is to remove obstacles between students and the course, no matter how small those obstacles are. To achieve these goals, it is critical for course and web developers to collaborate and share their visions for interface design early. For more information on user interface design, see Chapter Six and Appendix C. Additionally, if the sites you review employ special features, such as Java and multimedia to display content, consider whether those features actually enhance content or just make the site look “cool.” Our teams are enthusiastic about the media capabilities of a web delivery system, but we strive to select and align those capabilities according to audience needs and training goals. While reviewing other courses, our teams have noticed a division between designers who rush to exploit possibilities of the medium and others who shy away from nearly all enhancements; some course features seem gratuitously entertaining, while others seem little more than online books (“page-turners”). As you review existing WBT, use the following questions to evaluate graphics and multimedia features, which you can later ask of your own courses: ♦ ♦ ♦ ♦ ♦ ♦

Do graphics, animation, and interactions clearly support course goals and objectives? Can existing graphics, animations, or interactions be simplified? Are there other ways to achieve a similar effect or to convey the same idea? How much cognitive burden do media elements impose on the learner? Could media elements enhance learning where those elements are absent? What is the purpose of interactivity? Is it a change element needed to keep students engaged? Does it simulate an action described in a course objective?

Forming and Managing a WBT Team ♦ ♦ ♦

55

Are multimedia elements slow to download or process? Do multimedia elements aid the learning process? Does the course require one or multiple plug-ins, and is it a hassle to download or install them?

Writing for the Web Despite the unique technical capabilities a web delivery system offers, writing is the crux of course content. Yet numerous end user studies (many by usability expert Jakob Nielsen of useit.com) indicate that people generally have difficulty reading a text-heavy web page. This presents a dilemma for developers of WBT, because often, participants must read all of the text to complete activities and to pass evaluations. Additionally, confrontation with extensive text on one screen can ignite user frustration. Because of this, writers of WBT should observe best-practices for good web writing. Together, those practices are often referred to as a “newspaper style” of writing, which is characterized by short, concise paragraphs with topic sentences that encapsulate the paragraph’s meaning. Chapter Six addresses web writing principles in depth. For the purpose of an initial review, the following questions are a good starting point for discussion: ♦ ♦ ♦

Is the writing concise and “scannable”? Are pages with extensive text cumbersome to read online? Is it clear whether the content spans several pages and when a new topic begins? ♦ Is each page limited to only one main idea?

56


WBT Review Worksheet WBT Aspect 1. Purpose of course: • Is this clearly a training site or course that identifies its audience? • Are course descriptions and prerequisites available? • Can students access course objectives? • How well does the course support those objectives? 2. Use of interactivity: • Do interactions have instructional value? • Are activities and evaluation methods consistent with course objectives? • Do evaluations actually test for the verbs stated in objectives? 3. Level of self-directed learning: • What is the balance between linearity and hypertextuality? • Are learning units sufficiently modularized to allow for short learning sessions or learner-defined training sequences? 4. Relationship among learner, user interface, and course content: • Based on key features of the user interface, what assumptions seem to be made about the target learners? Is a “role” implied for the learner, as with simulation-based content? • How well does the interface connect with the target learner context (if there is an identifiable target learner population)? • Does the user interface complement themes within the course content?

Reviewer’s Perception

Suggested Improvement


WBT Aspect 5. Clarity of navigational options and paths: • Is it clear which items are meant to be clicked? • Are forward/back options provided, or is use of the browser’s navigation buttons required? • Can learners easily tell where they have been? 6. Ease of learning: • What additional aspects either support or hinder learning?

57

Reviewer’s Perception

Suggested Improvement


58

References 1.


2.

Rosenfeld, Louis and Peter Morville. Information Architecture for the World Wide Web. (Sebastopol, CA: O’Reilly, 1998), 2.

3.

Lynch, Patrick J. and Sarah Horton. Yale C/AIM Web Style Guide. Cited 1 January 1999 from www.info.med.yale.edu/caim/manual; INTERNET.

4.

Jennifer Fleming, Web Navigation: Designing the User Experience. (Sebastopol, CA: O’Reilly, 1998), 34.

Chapter Three

Audience Analysis Chapter topics: ♦ Audience analysis • Analysis overview ♦ Needs analysis • Training need identified • Web considerations for needs analysis ♦ Activity 1: Review WBT samples to gather preferences ♦ Task analysis • Web considerations for task analysis ♦ Activity 2: Review WBT sample to gather computer skills data ♦ Instructional analysis • Web considerations for instructional analysis ♦ Case study: Health and safety compliance training ♦ Technology gap assessment • Considerations for intranet and Internet delivery Chapter resources: ♦ Technology gap reference sheet

Audience Analysis This chapter explores the interplay between traditional audience analysis and preliminary considerations for designing a web delivery system. Because the design, development, and delivery of a web course is an involved undertaking with several interdependent steps, it’s important to begin assessing participants’ web-related needs as early as possible to keep a WBT project on track. If the Web appears to be a likely delivery system at the 59


60

outset, or if it is a predetermined project requirement, audience analysis can be structured to gather several aspects of web requirements on the front end. Those requirements can range from technical capabilities and design preferences to support media, such as email and electronic bulletin boards. Audience analysis is the cornerstone of instructional design. In the case of WBT, it provides an opportunity to make sure learners’ needs are truly met rather than becoming subordinated to technological restrictions. As you read this chapter, however, keep in mind the 21 tasks of instructional design presented in Chapter One. If you have the time and resources to follow those steps, you probably won’t integrate direct WBT questions into your initial analysis efforts. Instead, those questions will be asked at the end of design, when it might be determined that the Web offers the greatest advantages over other delivery systems. If this is the case, revisit the sections in this chapter that relate directly to the web delivery system. These sections integrate practical activities and explore how the results of audience analysis can guide fundamental decisions for designing a web delivery system. Analysis Overview The results of accurate audience analysis provide the foundation for any successful training course, because those results ultimately indicate whether training is needed, who must be trained, what they need to be trained in, and why. Once the identification of a problem or need justifies analysis efforts, the analysis process typically involves asking questions to: ♦ ♦ ♦ ♦

Define the nature of the problem Understand why the problem exists Identify who is affected by the problem Identify what kind of solution can solve the problem

To facilitate gathering data in these areas, many instructional designers view the broad process of audience analysis as consisting of three fairly distinct phases:1 Audience Analysis Needs analysis

Task analysis

Instructional analysis

Audience Analysis

61

Although these components are presented in 1-2-3 fashion, audience analysis usually proves to be an iterative and dynamic process, much like the nature of any research effort. Discoveries in any of the above phases can expose additional implications and data gaps in other analysis phases. The more an instructional designer learns about a target audience from various perspectives, the more the nature of that audience can appear to shift and evolve. The next three sections, Needs Analysis, Task Analysis, and Instructional Analysis, walk through the basics of each analysis phase, then conclude with suggestions for integrating web considerations into your research efforts.

Needs Analysis Needs analysis begins with problem identification and definition. In other words, corporate training must solve a business need or problem. Once that need has been identified, instructional designers work with clients and learners to understand everything they can about that need. Such understanding guides instructional designers’ decisions that lead to proposing the most effective training solution. In some cases, a needs analysis will reveal that the optimal solution is not, in fact, a training solution. The following two scenarios illustrate basic differences between a training need and a nontraining need. Scenario A: Williams Computer Store Problem: During the last two quarters, sales have decreased for the Williams Computer Store. Customers have been complaining of poor service in two vital areas—sales and technical repair. Causes: A preliminary assessment revealed that customers were pleased with the quality of service they received, but that it was difficult to obtain service in the first place. Williams Computer Company, only two years old, has enjoyed much success so far and has begun to experience difficulty matching the demands of a rapidly growing volume of business. Likely Solution: Current employees perform well in their jobs, but simply cannot handle the increased number of customers in a timely fashion. Training would not solve this business problem, because the employees do not lack skills. Rather, the company needs to improve its employee-tocustomer ratio. Depending on the entry-level skills of new employees, however, formal training might eventually become a legitimate need.


62

Scenario B: Next Wave Software Problem: Six months ago, Next Wave Software introduced a middleware management application into the market. This product was expected to generate high sales, but so far has produced disappointing financial results. Causes: Upper management rushed time-to-market on their new product and pushed the sales force to penetrate the market as soon as possible. The sales force was accustomed to selling various authoring tools and development packages, but not management applications. While their basic sales skills were strong, few representatives were able to successfully locate leads and to satisfactorily demonstrate key features of the middleware. Many sales representatives originally believed the middleware still contained bugs, but additional testing confirmed that the product was technically superior. Likely Solution: These sales representatives require a better understanding of customer needs, middleware competitors, and the new product they are trying to sell. Formal training would address all of these issues and allow the representatives to practice applying their existing sales skills to the new customer situation. Training Need Identified As a training need becomes apparent, gathering needs analysis data from learners such as those described in Scenario B (Next Wave Software) involves asking a wealth of questions, both of actual learners from the target audience and their managers. Needs analysis questions typically span five broad audience dimensions: 1. 2. 3. 4. 5.

Demographics Knowledge gap Job environment Performance criteria Possible training environment

Sample questions in each category are offered below: 1: Demographics ♦ What are the basic characteristics of the target audience in terms of age, gender, location, and so forth? ♦ What education level do most target audience members have, and in what specialized areas?

Audience Analysis

63

2: Knowledge Gap ♦ What knowledge and skills do learners currently have? ♦ How do those competencies differ from needed or target competencies? 3: Job Environment ♦ In what situations will the target skills and knowledge be used? ♦ What associated tools are used in the job? Are they adequate? ♦ Are there any job factors not directly associated with the training but which could affect the training, such as morale, or language differences? ♦ Into what specific situations should the target competencies transfer? 4: Performance Criteria ♦ At what level of proficiency must learners be able to demonstrate or apply their new skills and knowledge? ♦ Can this level of proficiency vary according to job situations and factors? 5: Possible Training Environment ♦ What are the training preferences of the client and learners in terms of cost, duration, and delivery system? ♦ What training options will best meet those expectations? ♦ How was the last formal training received by the target audience? ♦ What factors led to perceptions of previous training? Asking these questions will lead you to an understanding of what clients and learners believe is required in the training and what it should accomplish. Such information is key to an effective and realistic proposal for designing and developing the target course. When gathering data in the above areas, be conscious of how your questions are formulated in terms of scope, specificity, and bias. The scope of questions should be broad enough to allow for a variety of responses; questions should also encourage specific answers that provide useful input toward training goals. Regardless of the data-gathering method (phone, personal, or group interviews, surveys, etc.), instructional designers must also be aware of their own training biases during the needs-analysis phase.


64

There is a fine line between clarifying an interviewee’s needs and suggesting what you believe those needs are. Making assumptions about your learners will close off avenues of discovery. In short, avoid leading the interviewee. At the conclusion of needs analysis, instructional designers analyze their data to propose a tentative solution that describes the target training in relatively general terms. Your solution should address any segments of the audience discovered during the needs analysis. Judi Schade, one of our instructional designers who specializes in interactive training for television, explains that “an effective analysis will identify any subgroups in the audience. When these segments exist, delivering the same training to the audience at large will not necessarily lead to successful outcomes for all audience segments.”2 Thus, the training solution should accommodate the primary needs and concerns of both the client and learners of the target audience, including any audience segments. Web Considerations for Needs Analysis Several of the questions traditionally asked during needs analysis also support efforts to design the most appropriate web delivery system for target learners. Some of those questions can be phrased directly in terms of WBT: ♦ ♦ ♦ ♦ ♦ ♦ ♦

How large and geographically dispersed is the target audience? Have target learners taken WBT before? How do they characterize those experiences, and how would they improve them? What would be an ideal learning experience for the target students? What are learners’ attitudes toward WBT? What do they perceive as the primary advantages and disadvantages? How will they access a web course? From home or office desktop? From a laptop during travel? From a training lab provided by the employer? What systems or equipment need to be in place for the web course to be successfully delivered? What technology gap exists between current WBT tools and support and needed tools and support?

As mentioned at the beginning of this chapter, audience analysis is often an iterative process, and this is also true of web-related analysis. As you ask these questions, you will notice that complete answers cannot always be given immediately, but will have to be pieced together throughout all three phases of audience analysis. The nature of this iterative process highlights the importance of asking such questions early.

Audience Analysis

65

Activity 1: Review WBT Samples to Gather Preferences As a formal approach to gathering data on some of the above questions, invite client and learner representatives to review short, preselected samples of WBT you have created or gathered from external sources. This is similar to the team building activity described in Chapter Two. We recommend holding separate sessions with clients and learners, because their observations will likely be driven by different concerns. If possible, the instructional designer should collaborate with other team members when developing specific questions for this activity. Web programmers and graphic artists, for instance, will be seeking different types of input from learners. Additionally, it is useful to include one or two of these team members in this review activity, not only to help capture information, but to ask follow-up questions and to make learner needs more tangible. Review Questions: For learners: ♦ Which features of the WBT do participants find useful, distracting, or confusing? ♦ Do they have any specific requests or suggestions for the web training that will be developed for them? ♦ How does this review session compare with other experiences they have had with WBT? For clients: ♦ How can the web design reflect the culture and values of the target audience? For example, would photographs or drawings be more appropriate? ♦ Should the design integrate any logos or artwork already developed to market any product, concept, or process associated with the training?

Task Analysis The task analysis phase of instructional design fills in the research framework created during needs analysis, because it seeks more specific information. Training tasks and content are defined more concretely in this phase.3 For instance, the brief needs analysis in Scenario B (Next Wave Software) indicated that learners required training in the areas of customer


66

needs, the competition, and product features. A task analysis helps clarify what content should be included for each of those three areas by outlining the steps leading to competency. In many ways, a task analysis supports the purpose of needs analysis—to gather critical information on the target audience to ensure that its particular training needs will be met. Similarly, task analysis follows a problem-solving approach. The following general questions support initial task analysis efforts and will lead to more specific areas for further research: ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦

What job tasks must the target audience complete? What are adequate performance levels for those tasks? Must tasks be completed sequentially? Are some of the tasks primarily cognitive, while others are primarily psychomotor? Do corrective procedures or processes exist if an error is made while completing the task? In what settings are those tasks performed? What tools or resources are required for completing the tasks? What are the main inputs to and outputs of the task?

This information can be gathered in a variety of ways from the client, SMEs, and skilled target audience representatives. But before consulting with these groups, consider the different kinds of information you will need to gather and the research methods capable of yielding those results. Three common methods of task analysis are observation of task performance, interviews, and examination of previous, relevant training materials. All of these approaches reveal different aspects of a job task. Consider, for example, the task information you would need for creating training on the effective use of copy machines, as explored below. Observations typically reveal how something is done. Observing someone making copies can indicate: ♦

Where to place the materials to be copied, either on the glass or in the paper feed ♦ How to place a large document in the paper feed to preserve pagination ♦ Which button allows multiple copies to be made at once ♦ Which buttons create double-sided copies or copies that are sorted, stapled, or hole-punched

Audience Analysis

67

Yet observing task performance will not necessarily reveal the cognitive tasks associated with kinetic actions, or the whys behind someone’s actions. Additionally, observations do not always account for special or unusual circumstances. In terms of our example, observational methods alone will not necessarily generate content in the following areas: ♦ ♦ ♦ ♦ ♦

Strategies for copying oddly sized pages Copying book pages back to front to avoid collating pages by hand How to locate supplies such as ink drums, paper, and staples Typical causes of paper jams and how to fix a jam if it occurs How to handle different error messages

Interviews provide a method of identifying the cognitive steps of a task and how special situations should be addressed. To gather this data, ask the person you observe to explain steps aloud during the task. Debrief the task with interview questions to learn otherwise “hidden” information. Additionally, examine any previous training materials to further your task analysis, because they can help you identify relevant content and clarify where your training efforts can and should exceed previous efforts. Web Considerations for Task Analysis The detailed nature of task analysis data allows instructional designers to evaluate more accurately how a web delivery system can meet training needs for the specific target audience. With this information, you will begin forming a concrete plan for delivering the training with optimal results. This data will be critical at every step of the instructional design process. As a preliminary assessment, however, examine task-analysis data to identify performance aspects of the training and how those aspects indicate delivery system requirements. As mentioned in Chapter One, web environments are not in themselves well suited for content that teaches primarily psychomotor skills or content that is largely discussion-based. Task analysis data should indicate the type and level of interactivity the course requires. Explore whether the web course will need to be supported with other elements to enhance the learning experience and overall integrity of the delivery system. Consider these questions in terms of learners’ needs and performance tasks:


68 ♦ ♦ ♦ ♦ ♦

Should the WBT serve as a prerequisite or follow-up to classroom training? Should the WBT be integrated with phone or video conferencing or other interactive media? Are subject matter experts available to answer questions, either via email or phone during established hours? Can the use of email or electronic bulletin boards enhance learning by encouraging interaction among learners and with an instructor? What types of feedback will provide appropriate reinforcement?

It’s important to communicate these assessment results to your client or training owner, because these results can have far-reaching implications throughout the life of a course in terms of time and financial commitment.

Activity 2: Review WBT Sample to Gather Computer Skills Data Needs and task analysis data should offer preliminary indications as to how complex the WBT design must be to adequately convey course content. Generally speaking, the more complex the design, the greater learners’ computer skills will need to be. Sometimes, it is easy to become focused on “jazzing up” the content with technical features rather than keeping learners’ needs at the forefront of web design. In these cases, the most technologically and graphically superior design can become a barrier to learning. To avoid falling into this trap, always perform a task analysis regarding the skills needed to successfully access and complete a web course. An optimal method of performing this assessment closely mirrors the first activity in this chapter, in which clients and users view WBT samples and offer their opinions regarding desired features of the delivery system. This activity has a more technical orientation than the previous activity, and it can serve as a useful follow-up to gather additional information.

Audience Analysis

69

Suggested Materials A demo of a WBT module you have created or selected from an external source that is similar to the target training in terms of navigation system, menus, typical training interactions, and user instructions for performing interactions Suggested Approach 1. Meet individually with a small pool of learners who are representative of the target audience to assess their skills in accessing the demo and completing it. Explain that your purpose is simply to gather information on design needs, and that you’ll do this by observing the person taking the training module. 2. Ask the person to try completing the module without help, but explain that you will provide help if necessary. Encourage the user to voice his or her thoughts while completing the module. 3. As you observe, note when the person seems confused by different design aspects. Specifically, record information such as movements of the mouse pointer: ♦ Which items the person moves the mouse to ♦ Which items the user tries to click ♦ How long the person positions the mouse over different items on the screen ♦ How long it takes the end user to become accustomed to consistent features of the interface 4. Also record how the person dealt with any confusion. 5. Debrief the activity by asking the learner to comment on his or her experience with the training module. Activity Follow-up Draft a preliminary gap report of learners’ current computer skills and how they differ from the skills expected for successful course completion. There are several options for using this information, such as designing specifically for existing computer skills, improving those skills, or blending both of these options by providing detailed instructions within the WBT itself. This last approach is the least desirable, because extensive user instructions within a


70

course can divert students’ concentration from the actual training material rather than support their learning efforts. These options should be discussed with clients, learners, and your WBT team. Keep in mind, however, that a user’s difficulty with navigation does not necessarily reflect a skill gap on the part of the user. Balance your assessment with recommendations for improving the UI. For more information, see Chapter Six, “Presentation Principles.”

Instructional Analysis Efforts from needs and task analyses culminate in instructional analysis. Instructional analysis helps designers focus training for a specific audience because it reveals two important types of skill and knowledge “gaps”: 1. The skill and knowledge learners have at the beginning of a course compared to the skills and knowledge they should have upon successfully completing the course. 2. The baseline level of skills and knowledge students must have to begin the target training compared to their existing levels of skill and knowledge. The first gap indicates the scope of the target training, whereas the second gap indicates a prerequisite some students might need before taking the target training. Thus, information from the first area helps shape the key training elements of the course to be developed, from objectives and measurement activities to the definition and sequencing of content modules. As these elements become defined, an instructional designer evaluates which delivery system can most effectively convey course content to the target audience. The second gap (indicating prerequisites) is equally important. Learners are more likely to successfully complete the target training if they begin from a common, appropriate starting point. Otherwise, as Seels and Glasgow note, “If there is great variability among the members of the target audience, some students will need more and different instruction than others to reach the same goal.”4 Earlier, we referred to this variability as indicating audience “segments” or subgroups. Web Considerations for Instructional Analysis The observation of Seels and Glasgow regarding potential segments of the learning population is salient for anyone who plans to create web-based training. This delivery system offers great potential to address the varying

Audience Analysis

71

needs of a single audience. Accommodating student variability is related to viewing a web delivery system not just as a vehicle for delivering a single course, but seeing its potential as a training management system. However, when designing to account for different student needs, it is important to make sure that consistency and training goals do not become lost in a multitude of options that serve self-directed learning exclusively. The following strategies can help you locate a balance between structure and flexibility in your web course: Strategies: ♦ Attach a brief description of prerequisites to each course. Prerequisites can also be given for individual modules if students are allowed to access them randomly or during different training sessions. ♦ Provide an overview that illustrates relationships between modules, to help students evaluate which modules they should complete if a sequence is not already fixed. For some types of content, a flowchart can offer adequate illustration. ♦ Build short training modules, ranging anywhere from 15 to 45 minutes. ♦ Link sequentially related modules together. ♦ Conclude each module or course with suggested future training. This can be a static suggestion coded into the course or be dynamically generated based on evaluation results. ♦ Maintain online student profiles accessible by instructors, supervisors, and the learner. Profiles might contain course performance records, which courses or modules the student has completed, and courses yet to be completed. One challenge to building a high degree of self-direction in a training course is making certain that a learner takes the modules required for his or her job. A web delivery system can help solve this problem by offering a pretraining assessment that indicates to the instructor or supervisor which courses or modules a student needs. Instructors can then automatically be notified when the student completes the required lessons.

Case Study: Health and Safety Compliance Training Our Web Training department recently developed a series of courses to help safety managers and employees remain current in their federal compliance training. This target audience is diverse, with learners


72

experienced in many industries, including manufacturing, construction, and medicine. Although each course is based on the needs of specific learning groups, our project team faced the challenge of identifying common needs among safety managers and employees, regardless of their industry. We found that the web delivery system would offer the best training solution for everyone involved. We based this decision on findings related to each major area discussed in the first half of this chapter, as well as the criteria for selecting a web delivery system described in Chapter One. Early findings during audience analysis placed the Web as a legitimate candidate for a delivery system: ♦ ♦ ♦

Learners had computer and Internet access. Most of the learners were accustomed to computer-based training. The majority of content had a cognitive, rather than psychomotor orientation. ♦ Classroom delivery of the targeted health and safety courses is traditionally not discussion-based. ♦ Managers wanted an easier, cheaper way of updating course content according to changes in federal regulations. Additionally, web delivery ensured that all students would receive consistent training, regardless of when or where they completed the course. ♦ Managers needed a more time- and cost-effective delivery system than classroom delivery provided. Because of several variables, they often found themselves instructing a classroom of only two to three students. Learners and training managers were specifically concerned with convenience in the following areas: Learners ♦ Students did not want to wait for a classroom course to be scheduled, preferring to receive training on demand. Additionally, many of the students normally had to make appointments with training managers for computer-based training (CBT) to coincide with the managers’ hours, which were characterized as inconvenient. ♦ Students found traditional recordkeeping inconvenient, for two main reasons. First, many departments that require evidence of compliance training do not have linked databases, forcing students to initiate

Audience Analysis

73

administrative activities with each department to prove they had been trained appropriately. Second, employees who had completed their training with one employer had no valid proof to show a new employer. Thus, they had to repeat training they did not necessarily need. With records tracked in a training management system, learners would be able to have their training records available on a continuous basis for transfer to new employers. Managers ♦ Many health and safety managers wanted a solution that would allow them to spend more time on other job duties, such as performing safety audits, handling accident claims, and tracking changes in safety regulations. ♦ Like students, managers wanted a more effective means of maintaining student-compliance training records. With classroom and CBT delivery, managers had to enter the data from student records. In contrast, a web system can automatically process performance information after the course is over. Such a system could also help different departments verify quickly and accurately whether or not an employee has received adequate training. Given this set of training needs and expectations, our team then began to collect the technical data to guide us in developing a stable, suitable web delivery system. The remainder of this chapter explores a range of important technical issues.

Technology Gap Assessment The data gathered during all three phases of audience analysis provides fundamental material from which instructional designers and course developers create target training. This data is relevant to all team members in some regard, however. At the beginning of a training project, web programmers will benefit from learning the highlights of audience analysis and should supplement that data with their own analysis efforts regarding the capabilities needed to support stated learning goals. Creating the appropriate web design is a time-consuming project in itself; it is a mistake to delay this effort until the target training has already been developed. Even if your customer plans to take responsibility for hosting the web training,

74


arrange for your web programmer to talk with relevant people from the customer’s IT department. To establish needs and expectations for the training in technology terms, consider having the web programmer create a technology gap assessment that: 1. Outlines the customer’s current technology environment 2. Compares the current environment to expected needs 3. Recommends and prioritizes changes to the current environment that will be necessary for successful course delivery and maintenance Certainly, such an assessment can take substantial time to develop, but there are several advantages to documenting technology gaps. This documentation helps define roles and responsibilities between clients and vendors. It can also function as a development plan for your web programmer and the customer’s IT department, if appropriate. Additionally, technology restrictions must be considered when developing WBT. Early awareness of specific restrictions can help the instructional designer plan for different contingencies and evaluate related implications. The nature of learners’ computer systems, sometimes more than budgets and delivery system capabilities, defines technical WBT constraints. Planning for those parameters increases your opportunities to create superior courseware despite technical restrictions. From this aspect alone, weboriented audience analysis must be performed early in the project, but it should also expose resource needs and reduce financial surprises in the long term; the technology gap assessment can indicate needed funding for additional equipment, people, or time. You don’t want to learn mid-project that you have been designing for end users with greater technological resources and computer skills than those of the target audience. Imagine, for instance, the implications of integrating Java applets only to learn that your client prohibits the use of certain Java applets as a security precaution. This course might successfully integrate graphics and programming techniques to enhance WBT, but that success is irrelevant if the WBT does not meet given technology constraints. The following pages highlight key areas the gap assessment should address in some capacity. A Technology Gap Reference Sheet appears at the end of this chapter to assist in completing a technology gap assessment.

Audience Analysis

75

Considerations for Intranet and Internet Delivery Whether a course will be delivered via an intranet or the Internet, a technology gap assessment should address these specific issues: ♦ ♦ ♦ ♦ ♦

Expected level of interactivity Evaluation and feedback methods Typical software applications and tasks frequently performed Available training facilities and equipment WBT support systems, including Help Desk and IT

The next two sections highlight general technology issues for WBT delivery over intranets and the Internet. Intranet Delivery With intranet delivery, you can determine (and possibly upgrade) the network connection speed and browser software of end users’ systems prior to launch. Corporate intranets offer a potential speed advantage, typically transferring data at up to 10 Mbps (Megabits per second) compared to a T-1 connection at 1.5 Mbps. (However, system administrators generally will not compromise network performance if the network is already overloaded.) Also, the ability to standardize and upgrade end users’ browser software will eliminate many of the compatibility issues that plague WBT development for the Internet. Consequently, many corporate intranets offer the possibility of delivering more media-rich content. To facilitate design for your target audience, meet with your client’s Systems Administrator to learn which types of computers and operating systems are used on the corporate network. Also ask what type and version of browser software is installed on these computers; often, companies allow only one type and version of a browser. If you plan to deliver media that requires a specific plug-in, ask if the plug-in has been approved by the company for use. You might also enlist the Systems Administrator to install a required plug-in on any networked computers that do not already have it. It is equally important to examine other aspects of your client’s technical capabilities when exploring different media possibilities for the training. Find out restrictions early, so you do not waste valuable time developing training applications that cannot be used. If you intend to deliver audio, for instance, confirm that end users are equipped not only with an appropriate plug-in,


76

but also with a sound card and speakers. If learners do not have both pieces of hardware, they cannot receive audio transmissions through their computers. Make accommodations for any lack of audio capability by providing written transcripts or any audio segments in your WBT, or remove the audio elements altogether. Internet Delivery Gathering technical data on learners who will access your course over the Internet can be much more challenging than gathering technical data for intranet delivery. Internet users constitute a true technological melting pot potentially spread over a large geographic area. In fact, prior to launch, you may have no way of knowing which types of computers, browsers, or plugins your students have installed, nor will you be able to fully anticipate exactly who will be taking your course or their levels of Internet experience. When delivering web content to the world at large, then, it’s critical to account for a high degree of technical variance in fundamental areas such as browsers, authoring languages, and multimedia plug-ins. Browsers Browsers are the software applications that allow users to access the World Wide Web. While there are several different types of browsers, access to the Web is most often made through Netscape Navigator™ or Microsoft Internet Explorer™. Both browsers support many of the same features, so you can be reasonably assured that applications you write for one browser will be supported by the other. However, some caution is warranted: ♦

Different versions of Navigator and Internet Explorer support different features. Your students will almost certainly employ a wide range of different browser versions, each with somewhat different capabilities. ♦ Even though the latest versions of Navigator and Internet Explorer are distributed for free on the Internet, many users are slow to upgrade. ♦ The layout of page content can be displayed differently across browsers. It is generally good practice to develop web content to meet the standards of browsers one version prior to the latest and greatest. Studies by Jakob Nielson, usability specialist, reveal that Internet users are less eager to upgrade their systems than they used to be. At the Internet’s inception, many

Audience Analysis

77

end users were more interested in web technology than in the Web’s content. In contrast, Nielsen’s research indicates that more recent adopters are not so much interested in new technology as how existing technology can be applied to meet daily, practical needs.5 If you build to the latest specifications, you risk alienating a huge portion of your potential audience. Always test any Internet-bound web pages on both Navigator and Internet Explorer to detect deviations prior to final publication. HTML HTML (Hyper-Text Markup Language) is the core language used to display web pages on the Internet. However, you need to be aware that: ♦

Netscape and Microsoft have each developed their own version of the HTML standard. As a result, some HTML tags that work in Navigator do not work in Internet Explorer and vice versa. ♦ Because HTML is an evolving language, some tags and attributes that work in the latest browser versions do not work on previous versions. Again, it is generally good practice to develop your web content to meet the standards of browsers one version prior to the latest and greatest. JavaScript JavaScript is a web scripting language that allows inclusion of dynamic and interactive content in web pages. (JavaScript is not another term for Java. These are completely separate languages. Specifically, Java is used to create both web and nonweb applications, whereas JavaScript can only be implemented in a web environment.) JavaScript is widely used on the Internet, and most end users will be able to make use of JavaScript content. However, there are some limitations: ♦

Browser versions prior to Navigator 2.0 and Internet Explorer 3.0 do not support JavaScript at all. ♦ Older versions of Navigator and Explorer might not support all of the JavaScript commands that the most current versions do. ♦ Netscape and Microsoft have each developed a separate JavaScript implementation. So, even in their most recent versions, some JavaScript commands that work on Navigator do not work on Internet Explorer and vice versa.


78 ♦

Even in JavaScript-capable browsers, it is possible to disable JavaScript capability. This is done by deselecting the “Allow JavaScript” checkbox or similar feature in the browser’s Preferences menu. Some end users disable JavaScript for various reasons, or they may have inadvertently deselected their JavaScript checkbox without being aware of it.

Limitations aside, JavaScript is a relatively safe feature to use in your web applications. The core commands are available across all JavaScript-capable browsers, and the vast majority of end users’ browsers are JavaScript capable. Plug-ins Plug-ins are generally small software modules that can be downloaded and installed on a computer to extend browser capabilities. If the target design of your web delivery system requires plug-ins, you should keep some key issues in mind to ensure learners on the Internet do not become overwhelmed: ♦ ♦ ♦ ♦ ♦

Upon entering a site or course, many learners will lack required plugins. Some plug-ins require users to restart their browsers after installation. If your course uses such a plug-in, be sure to notify your learners and remind them of your site’s URL prior to download. If you use media that require a plug-in, be sure to provide a link from which users can download the plug-in. Most plug-ins are available in different forms to accommodate different computer platforms, so be sure to provide links to all available forms of the plug-in. Inexperienced users can be intimidated by the task of downloading and installing plug-ins. Be sure to provide a statement about what the plug-in is for and a clear description of how to download and install it.

Because downloading plug-ins might require some extra work on the part of learners, it is generally good practice to use only one or possibly two commonly available plug-ins for a given course. Sites and courses that require a lot of preliminary work from students may send them into despair before they even view the first page of course content.

Conclusion Although the three phases of audience analysis focus on research, each phase provides valuable opportunities to build rapport with your client,

Audience Analysis

79

learners, and team members. Productive relationships are especially important when creating WBT, because you will likely encounter a wide range of opinions on web-based training from a variety of key people, including Systems Administrators, managers, and learners. Here are some typical opinions: ♦ ♦ ♦ ♦ ♦ ♦

Web delivery is too expensive. We don’t have the time or money to support a web course. People don’t really learn on the Web. The Web is more effective for entertainment than education. We don’t have the network bandwidth. We cannot let the Web replace our instructors.

If you encounter these opinions, do not alienate your customers by arguing against them. Your role is not to hard-sell a solution, but to tap people’s ideas for suitable alternatives. Experienced instructional designers build rapport during audience analysis not by imposing their views on learners or making quick assumptions, but by expertly asking questions of all representatives involved in the training project. Christine Foran, Ph.D., performs training needs assessments for many of our customers. She explains her approach: I begin with the mindset that my job is first to learn from the client, then to provide guidance based on what I’ve learned. This approach goes a long way in creating a valuable customer relationship or dynamic by indicating that I am genuinely open to learning the client’s needs to develop the best possible training. Performing a needs and task analysis requires you to take client and target audience needs seriously, not yourself. Let them lead the way to the answers, because they’re the experts. You must listen to them and analyze the data along the way. Simply taking charge tends to shut down inputs and training options. Truly productive audience analysis involves being influenced by all of the data, analyzing it, and presenting it back to the customer so that he or she can see possibilities and their implications.6 Applying this philosophy involves keeping track of criticisms pertaining to a web delivery system. Take interest in people’s concerns and make sure your final courseware exceeds their expectations. Receiving mixed input from target audience members before design begins will increase overall training


80

effectiveness and chances of a positive reception. In talking with representative learners during initial phases of audience analysis, you will discover not only their training needs, but also their attitudes regarding WBT, their likes, dislikes, and concerns; the final courseware is more likely to be a success if it is built to address their attitudes. Additionally, you’ll be building learners’ interest in taking the course. Gaining early learner input on the web delivery system can also clarify responsibilities for the business sponsor, or whoever has requested that you develop the course. As a “champion” of WBT, the sponsor needs to become familiar with learners’ attitudes toward a web delivery system, then address and allay any significant concerns. Audience analysis also opens opportunities for team members to coordinate tasks. We cannot emphasize enough that training and audience needs should drive the design and development of courseware. With WBT, many of the tasks are separate, but cannot be performed in a vacuum, completely apart from other team members’ responsibilities and deliverables. Thus, implementing findings from audience analysis is rarely just a matter of fanning out data to team members. Rather, as the data becomes available, people need to discuss how that data affects their tasks. The project manager and instructional designer must emphasize that the target courseware be developed according to, and ultimately must serve, learners’ needs. Effectively designing and developing a web-based course isn’t a matter of tossing the baton between members, but rather working together to synthesize all project elements. From needs, task, and instructional analysis data, the team should begin to synthesize: ♦ ♦ ♦

Learner characteristics Training tasks Instructional methods for teaching those tasks

The next chapter delves further into the last point by focusing on developing objectives and activities for reinforcement and measurement. Throughout the chapter, we offer guidelines for effectively implementing these instructional elements in a web delivery system.

Audience Analysis

81

Technology Gap Reference Sheet Suggested use for web programmers: 1. Gather as much of the following information as possible on the target learning population and target training. 2. Compare learners’ current technology capabilities with those required to support preliminary definitions of the target training. Identify any remaining needs. 3. If learners’ current web environments are insufficient, develop a plan to address weak technology areas. The plan might focus on improving existing technology, altering design plans for the target courseware, or a combination. (If course design must be altered, confirm that it will still meet student and training needs. A radical alteration might indicate the appropriateness of another delivery system.) System and User Specifications • What level of expertise do students have with computers and the Internet? • What systems are learners currently using? • Which type and version of browser do they have? • How will they connect to the web course? • What are the specifications of the modems students use? • Which plug-ins do students have? Courseware Requirements • What types of instant feedback are needed, and how frequently? • What reporting or record-keeping functions are needed? • What level of interaction must the course provide? • What types of activities are needed to reinforce and measure learning? Client’s Network Constraints • What bandwidth restrictions have been established for the course? • What is the current level of network traffic, and how many end users are expected to access the course concurrently? • Is a hosting environment available? • Are there any special security requirements regarding Java, cookies, or ActiveX? ♦ What are the company’s size restrictions on email files?


82

References 1.


2.

Schade, Judi, interview by the author 2 March 1999.

3.


4.

Ibid., 11.

5.

Nielsen, Jakob. “The Increasing Conservatism of Web Users.” Cited 1 February 1999 from www.useit.com; INTERNET.

6.

Foran, Christine, interview by the author 27 January 1999.

Chapter Four

Objectives and Measurement Chapter topics: ♦ Connections between objectives and measurement ♦ The role of objectives • Writing objectives ♦ Implementing objectives and activities in a web delivery system • Evaluation data • Situated learning • Learning styles • Sample objectives and activities ♦ Feedback methods ♦ Considerations for interactivity • Style of interactions • Frequency of interactions Chapter resources: ♦ Objective and measurement worksheet for web delivery system

Connections between Objectives and Measurement Objectives and measurement are often considered the interdependent cornerstones of performance improvement. An objective states in performance terms what learners should know or be able to do after training. Measurements, such as practice activities and tests, indicate how well students have met the established course objectives. Together, then, these 83

84


components form a critical feedback loop for trainers and students by defining the scope of instruction and providing evidence of student performance within that scope. Given this synergistic relationship, the selected delivery system must accommodate a course’s objectives and measurement methods. If the delivery system fails to make such accommodation, trainers have no sound basis for evaluating student performance. For these reasons, instructional designers typically define objectives and measurement methods long before selecting the delivery system, as our list of instructional design tasks from Chapter One indicates: 1. Analyze needs assessment data. 2. Analyze target audience description. 3. Analyze tasks. 4. Identify skill and knowledge gaps. 5. Identify prerequisite skills and knowledge. 6. Prepare measurable course objectives. 7. Describe cumulative and criterion tests. 8. Form modules. 9. Determine module sequencing. 10. Analyze client parameters and constraints for effects on design decisions. 11. Gain client approval before continuing. 12. Form logical learning units within modules. 13. Determine sequencing of logical learning units within modules. 14. Determine placement of practices and tests. 15. Identify appropriate types of practice activities and tests. 16. Identify the primary sources of information during the training (job aid, self-instruction, or instructor). 17. Develop a content outline. 18. Identify techniques for presenting content. 19. Determine appropriate delivery system(s). 20. Determine appropriate delivery media. 21. Document details on course blueprint.

Objectives and Measurement

85

The connection between objectives and measurement is an interesting and complex topic for designers of web-based training, because a web delivery system can simultaneously offer barrier-breaking training possibilities and impose barrier-forming limitations. Despite this paradox, instructional designers should ensure that the web delivery system does not alter the purpose or quality of objectives and measurements. To help you successfully implement these instructional features in a web delivery system, this chapter walks through a range of considerations regarding objectives and practice activities. Although instructional designers typically have responsibility for creating sound objectives and activities, they should become familiar with several web-related options and seek the input of web programmers regarding the tools and programming needed for successful implementation. During this interaction, instructional designers should verify that their ideas for testing methods are possible on the Web and feasible given learners’ computer systems and levels of Internet literacy. As a starting point, let’s examine how instructional designers develop training objectives and measure student achievement of them.

The Role of Objectives Because objectives spell out in performance terms what learners should be able to do after successfully completing a course, they help instructional designers build a rationale for how a course should unfold, rather than allowing it to meander among topics and contingencies not directly related to the course goal. Objectives also provide students a guide to achieving success in a course. Students’ actions will be more productive if they know what they are supposed to be doing and why; if they cannot discern the purpose of a course, their behavior is more likely to lack purpose, too. In addition to outlining what students will learn, effective course objectives also specify measurable levels of desired proficiency; they are stated in terms specific enough that instructors can establish, as clearly as possible, whether the learner can perform the desired skill or has acquired the desired knowledge. When determining target levels of proficiency, instructional designers often use a guide or taxonomy developed by Benjamin Bloom. Bloom’s taxonomy is composed of “six levels within the cognitive domain, from the simple recall or recognition of facts as the lowest level, through increasingly more complex and abstract levels, to the highest order, which is classified as evaluation.”1 The six levels are knowledge, comprehension, application, analysis, synthesis, and evaluation. Using this taxonomy, an instructional designer might ask: “Do students need to know this information at the level

86


of synthesis, or is it sufficient for them simply to be able to comprehend the information? What is the least they need to know to be successful?” To determine the least a student needs to know for success, instructional designers revisit the course goal and their needs and task analysis data to identify the most critical training concepts and tasks. They then derive objectives from this information. Training beyond the breadth or depth of those needs in a corporate environment can lessen skill transfer by obscuring the points that are expected to transfer directly into job performance. Surpassing the training goal can also result in information overload, giving learners more information than they can retain. If a course is content-heavy, think about using job aids and other reference material to reduce the amount of information learners must memorize. The idea of teaching people the least they need to know to be successful might seem surprising and may be more common to instruction for training purposes rather than educational purposes. In a university setting, for example, training on the least students need to know isn’t always congruent with the exploratory, collaborative, or discovery-oriented nature of some disciplines and courses. Thus, we want to stress that we’re focusing on corporate training in which employee-students are responsible for improving job performance in specific, employer-defined ways within an established timeframe. Companies fund training for business reasons, from raising morale and teamwork to reducing the number of production errors or on-thejob injuries. Have your customer review the course objectives to make certain that they actually address the original business problem. To venture beyond the needed solution can waste precious training time and money. Writing Objectives To arrive at measurable objectives, instructional designers typically begin with the stated course goal and break this general statement down into increasingly behavior-oriented statements.2 To develop measurable objectives, we use Robert F. Mager’s widely accepted model. For the purpose of illustration, this chapter covers the elements of objectives that Mager views as essential, although some instructional design models encourage the use of additional components. According to Mager, measurable objectives have three essential characteristics3 :


87

Objective Components 1. Skill or knowledge to be demonstrated • What skill or knowledge set is the learner expected to acquire and demonstrate? 2. Condition of performance • Under which circumstances should the learner be able to demonstrate this skill or knowledge set? 3. Testing criterion • What level of expertise should the student reach? What constitutes acceptable evidence that the student has reached this level of achievement? Instructional designers debate whether objectives must always incorporate the performance condition and testing criterion when presented to learners. Mager advises that these components be included only when it “seems an easy way to communicate your intent” and does not produce an objective that is difficult for learners to grasp.4 An objective that incorporates all three components is typically worded with the following structure: “Given the appropriate hardware and software, be able to install a backup program on a test server with fewer than five errors, within three hours.” The core of a well-written, measurable objective is the verb; it indicates both what the learner will do and the primary basis for measuring the learner’s performance. In the sample objective, for instance, the verb, “install,” is precise, specifying unambiguously what learners must do to successfully complete the course. In other words, learners will not be allowed to pass the course simply by listing the main steps of installation or by describing how to install backup software; they must actually install it, which is a much more complex task. Thus, selecting precise verbs helps assure that learners receive training based on the stated instructional goal and that measurable levels of achievement are adequately defined in conjunction with any performance conditions and testing criteria. Our sample objective also highlights the interdependencies between objectives and measurement activities in that all practice activities should lead up to the student’s proficiency in installing the program. An objective’s verb not only describes a measurable behavior, but implies a particular type of evidence when the behavior is performed, such as a successfully installed program. For comparison, consider the following verbs: appreciate, understand, categorize, repair. The first two verbs, appreciate and understand, represent vague behaviors, because evidence of

88


appreciation and understanding can be defined in numerous ways depending on the perception of an individual instructor. Categorize and repair are more easily measured, because performing these actions produces concrete evidence: “Are the items correctly categorized according to a specified process, hierarchy, or other system of classification?” “Does the equipment function properly after repair?” Depending on the narrowness of behavioral specification, objectives are often classified into two categories, terminal objectives and enabling objectives. Whereas terminal objectives indicate the key behaviors learners should be able to perform after completing a course, enabling objectives identify the steps leading to the achievement of terminal objectives. Both types of objectives contain the same core components and follow the same basic format described earlier. Table 4.1 illustrates the relationships between a course goal, terminal objectives, and enabling objectives. The statements in the table are not worded as objectives, nor are they exhaustive. They do, however, illustrate a derivative process. For a more detailed explanation of this process, see Making Instructional Design Decisions by Barbara Seels and Zita Glasgow. Table 4.1: Deriving Objectives from the Course Goal Course Goal ⇒ Effectively use the office email system. (the intention or purpose of the course) Terminal Objectives ⇒ 1. Send and retrieve email attachments. (behaviors that provide 2. Describe acceptable use of office evidence that learners have met email. the course goal) Enabling Objectives ⇓ (provide evidence of achievement and outline steps to meeting terminal objectives) For terminal objective #1: For terminal objective #2: 1. When sending an 1. Define the purpose of the corporate attachment, select the email policy. attachment’s correct file type 2. List the ten principles of the corporate and encoding. email policy. 2. When receiving an 3. Distinguish between appropriate and executable attachment, scan inappropriate types of email for viruses. messages.


89

After desired behaviors are broken down into terminal and enabling units, instructional designers then word them as we normally think of objectives. The third enabling objective for terminal objective #2, for example, might be worded as: “Given five sample email messages and the corporate email policy, distinguish between appropriate and inappropriate types of email messages.” The appropriate delivery system for this course will accommodate activities that allow learners to practice using the office email system, to become familiar with applicable company policies, and to describe or discuss their understanding of those policies.

Implementing Objectives and Activities in a Web Delivery System Once instructional designers write measurable objectives and identify appropriate practice activities and tests, they must then use this information to select a congruent delivery system. This selection requires instructional designers to ask some essential questions. For our purposes, we’ve focused these questions on a web delivery system in particular: ♦

Can achievement of the established objectives be measured in a web delivery system? ♦ What kind of evidence will demonstrate that a student has met the objectives? ♦ How can that evidence be gathered and evaluated in a web delivery system? A logical starting point for answering these questions is the list of appropriate training task characteristics from Chapter One. Most objectives and activities are well suited to or achievable in a non-facilitated, asynchronous Web delivery system when: ♦ ♦ ♦

Target skills have a cognitive versus psychomotor orientation Little to no detailed instructor feedback is required Little to no discussion is needed

These criteria can seem overly restrictive, until we recall the broad spectrum of WBT types also described in Chapter One: Non-facilitated

Mixed Facilitation

Facilitated

Asynchronous



90


A single web delivery system can offer a mix of strategies and technologies designers can use to support a variety of training objectives, and if done so for sound instructional reasons, these options help create effective learning experiences. The selection of a web-based delivery system requires the designer to examine whether that system can reasonably accommodate the verbs of each training objective. If students are expected to “write,” “discuss,” or “analyze,” a mixed or facilitated system is needed, because these activities require human interaction that’s either synchronous or asynchronous. The last three sections of this chapter offer sample activities, feedback methods, and principles of interactivity designed for non-facilitated, web-based courses. Before delving into those specifics, however, we want to briefly cover three topics that relate to all practice activities regardless of the delivery system—evaluation data, situated learning, and learning styles. We hope these topics highlight some instructional principles that can be easy to lose sight of in a sea of tools and technology options. Enthusiasm for web technology is important, but we encourage people always to temper their interest in training technology with one question: “Will this help the student learn?” Evaluation Data When designing measurements, plan to capture evaluation data that encompasses results from both criterion tests and practice activities. Although students might not receive a score on the latter, performance data on practice activities can expose problem areas in the training design, thus indicating a plan for course improvement. For more information on evaluating course effectiveness, see Chapter Eight, “Course Evaluation and Revision Maintenance.” A different, but related issue involves certification testing in a web delivery system. Perhaps more than any other kind of assessment, certification data are not simply correlated to training material, but are attributed to an individual student’s performance. While technology allows these exams to be administered via the Web and evaluation data easily gathered and reported, that data is not valid unless the exam was proctored by a designated person: Intentionally or not, a student’s password can be misused. The rest of this chapter explores principles for creating effective, webbased practice activities.


91

Situated Learning Designing measurements to simulate those actually used in the workplace is an important step toward evaluating how training affects onthe-job performance. Regretfully, web-based practice activities can be far removed from the job context in which the new skills and knowledge will be applied, and the greater the gap between the training and application environments, the less effective the training is likely to be: Learners will have to make a greater effort at transferring their skills from one context to another. Designers of WBT can lessen this burden on students by striving to provide learning experiences that have a strong alignment with the learner’s work environment in terms of culture, how the new skills will be used, and circumstances that will require the skills to be adapted. These are all basic concepts from situated learning theory, which seeks to enrich learning by integrating “authentic” elements into the instruction. In “Critical Characteristics of Situated Learning: Implications for the Instructional Design of Multimedia,” Jan Herrington and Ron Oliver explain that “usable knowledge is best gained in learning environments which feature [authentic] characteristics” in several respects, such as offering insights into colleagues’ best practices and an “authentic context that reflects the way the knowledge will be used in real life.”5 Because of the asynchronous, sometimes isolated nature of non-facilitated WBT, instructional designers often struggle with how to create a sense of authenticity as described above. Striving for authenticity can fail in WBT when visual or graphical authenticity is used as a substitute for the contextual authenticity required for situated learning. Merely providing an interface that represents real-world objects associated with a given work setting or skill does not ensure that the course design will encourage learners to apply skills and knowledge in a real-world fashion. (For more detail regarding authenticity in interface design, see Chapter Six.) Unless the job context is closely related to computers, the Internet, or software, we recommend looking for fairly simple ways to suggest a feeling of authenticity in webbased practice activities. The next two examples illustrate a clear distinction between an activity that disregards the work context and an activity that aligns more closely with the actual performance context, even though they both rely on a multiple choice format. Example 1 While non-facilitated WBT is not generally well suited for teaching sales

92


skills requiring human interaction, it’s often an ideal delivery system for teaching sales concepts and product knowledge. To complete the activity depicted in Figure 4.1, learners must read the scenario, then answer a multiple choice question.

Figure 4.1 Although this approach does assess a student’s basic knowledge level, it misses an opportunity to help them connect their knowledge with real-world application. This scenario might provide insight into a typical customer profile, but it’s far removed from a sales context in which representatives rarely have the luxury of reading about customer characteristics in such a concise manner. Nor does this activity incorporate any cultural elements, which factor heavily into sales concepts and skills.


93

Example 2 In contrast, the activity in Figure 4.2 gives learners practice in listening to how customers realistically describe their needs, as well as an opportunity to observe a sales representative’s expertise in responding to them. In this example, learners view a video of this one-to-one conversation. The video periodically pauses, and the learner answers a multiple choice question based on the video segment. The learner has the option to replay the segment or to view a transcript of it before answering the question. After submitting an answer, the learner can resume video play. At the end of the activity, the learner receives instant feedback. The short assessment congratulates learners on the sales concepts they recognized, while also indicating areas of understanding that still need improvement.

Figure 4.2 At a minimum, this activity requires learners to have speakers and a plug-in (such as RealNetwork’s RealPlayer™) that allows them to receive streaming video. Because some learning audiences will not have these

94


technological capabilities, it is critical for the WBT team to have a clear understanding of client network constraints and end user systems. It is equally important for the team to work creatively within those constraints to the benefit of students. The sample activities later in this chapter indicate ways to work with technology limitations without disengaging students. Learning Styles As you design and develop web-based activities, consideration for different learning styles is essential for keeping students engaged with a medium that can often seem impersonal. Our above example of situated learning illustrates how a single activity can appeal to a variety of learners, some who favor one sense (auditory, visual) over another when learning new concepts. Such use of multimedia should also have an instructional basis and not be included simply for its own sake. There would be little value, for example, in providing an audio iteration of the purely text-based scenario and multiple choice question depicted in Figure 4.1 above, unless students had a special need. Providing an audio component for that activity would be analogous to a classroom instructor reading all handouts and presentation slides verbatim to his or her students. While the need to engage a variety of learning styles isn’t a new discovery, designers of WBT are still exploring ways of creating that appeal for end users who do not regularly upgrade their systems and software, for users who might have restricted access because their computers are located behind a firewall, or for users who are limited by bandwidth. If these learners cannot take advantage of multimedia, instructional designers must give added attention to varying styles of practice activities. The following pages present a handful of practice questions and activities that indicate ways to achieve this variation. To reinforce the kinds of objectives that can be met in a non-facilitated web delivery system, a sample objective is provided with each activity.


95

Sample Objectives and Activities Game Element Description: Game elements are often incorporated into WBT activities to increase learners’ sense of progress and motivation. Although this approach can be implemented in several ways, it always creates a competitive dimension which can stimulate user interest. (Game-like approaches to webbased activities also involve the risk of introducing gratuitous elements.) The activity in Figure 4.3 below poses three multiple-choice questions. Each correct answer advances the student’s car in an auto race, while each incorrect answer advances a competing car. In this case, using an auto race appeals to the target learners’ interests and motivations. Course: Introduction to Combustion Engines Objective: Recognize the purpose of each major component of a combustion engine. Figure 4.3

96


Fill in the Blank Description: Like multiple choice, fill-in-the-blank is a common form of assessment in distance-learning courses. Often, there’s a wide range of possible answers that an instructor must evaluate. Students using self-study workbooks can also compare their answers to a workbook key. On the Web, however, accounting for possible answers can be a challenge, because programmers must anticipate numerous possible answers and misspellings. For WBT fill-in-the-blank, we suggest using a drop-down box to avoid such difficulties, as in Figure 4.4. Course: Noise Hazards and Hearing Conservation Objective: Identify basic principles of the physics of sound and hearing Figure 4.4

Drag and Drop Description: Just as game elements should connect with the material, drag and drop designs should be developed from content that lends itself to matching and should in some way relate to the concepts being taught. Figure 4.5 illustrates poor use of the drag-and-drop concept. In contrast to the second


97

example in Figure 4.6, it does not fully utilize the course material or available graphic medium, and thus appeals to a very narrow range of learning styles. Course: Noise Hazards and Hearing Conservation Objective: Distinguish between the major components of the ear Figure 4.5

Figure 4.6

98


This activity effectively uses the drag and drop technique because it visually reinforces key terms. Note: Using drag and drop exercises on the Web requires the use of either a plug-in-based application such as Shockwave, a Java applet, or DHTML (which is only available to 4.0 browsers or greater). Point and Click Description: Conceptually, point and click activities are similar to the drag and drop format, but they require less advanced programming. Consequently, the point and click approach works well for end users who cannot download plug-ins. Figures 4.7 and 4.8 illustrate point and click activities. Course 1: Introduction to Hardware Product Features Objective: Identify the main features of the Z server line Figure 4.7


99

Course 2: Principles of Desktop Publishing Objective: Compare common font types Figure 4.8

Simulation Description: Non-facilitated WBT lends itself well to simulated software applications. Strong candidates include software skills, proficiency in processing forms that will be accessed via computer, etc. (However, many simulations that involve frequent, complex feedback might require facilitated WBT for effective skill practice.) The simulation in Figure 4.9 is based on the principles of dimensional data modeling, a technique used to build data warehouses. A data warehouse is a database that is optimized for analysis and decision support. From this database, end users can access a wealth of historical and up-to-date information according to their specialized needs. To serve these needs, the data must be modeled according to preselected data attributes. The example below allows students to practice a skill basic to data modeling—identifying attributes that allow end users to find the information they need. Figure 4.9 shows the question, and Figure 4.10 illustrates feedback. Course: Techniques of Dimensional Data Modeling Objective: Apply the principles of designing dimension tables

100

Figure 4.9

Figure 4.10



101

Feedback Methods So far, this chapter has focused on the connections among objectives, activities, and effective implementation in a web delivery system. As a follow-up to practice activities, web-based courses must also provide instructive feedback to reinforce the learning process. Jerrold E. Kemp, author of Instructional Design: A Plan for Unit and Course Development, writes that “Motivation for learning can be increased when students are informed of how well they are doing during the course of a lesson (often through self-check exercises, tests, informal discussions, and so forth).”6 Encouragement through feedback on performance is particularly important for web-based learning in which students and instructors are often separated by distance (and time, in the case of asynchronous WBT). Feedback on practice activities should guide learners toward meeting the objectives and minimize their sense of isolation, and be used as an opportunity to enrich the learning process. In a traditional classroom environment, instructors typically offer feedback on performance that explains why certain answers might be viewed as correct or incorrect. These instructors take advantage of feedback as opportunities for further teaching, which is just as important for designers of WBT activities. By its nature alone, facilitated WBT allows instructors opportunities for this adaptive instruction. If the WBT is non-facilitated, however, the designer needs to write out a response for each possible correct and incorrect answer to practice activities in order for feedback to be instructive. Three examples are provided below. Feedback Example 1—Pop-Up Windows As illustrated in Figure 4.11, pop-up windows are a common feedback method for non-facilitated practice activities. The feedback is instantaneous, but to less computer- and Internet-savvy students, these windows can look similar to error messages or “alert boxes” at first glance.

102


Figure 4.11

Feedback Example 2—Route to New Screen An effective alternative to pop-ups is rerouting, which is described in Figure 4.12 and depicted in Figure 4.13. With this method, the student is automatically taken to a new screen after completing a question or activity. This screen should concisely incorporate the original question, the student’s answer, and a brief explanation of why the correct answer is, in fact, correct. From this screen, the student resumes the training session. Figure 4.12


103

Figure 4.13

Feedback Example 3—Review Button Giving learners the option to review key training concepts blends elements from both of the above feedback methods. As indicated in Figure 4.14, the learner receives feedback. If the learner answered incorrectly, he or she is given the option of reviewing the concept before continuing with new material. Although this format allows for more self-directed learning, some users will find the extra yes/no click monotonous.

104


Figure 4.14

Learners who answer correctly might receive a pop-up window indicating their success. If a student answers incorrectly and elects to review the material, he or she will either receive a new page or a pop-up window containing the review information. Those who decide not to review simply continue on the training path.

Considerations for Interactivity As a follow-up to practice activities and feedback, we conclude this chapter with considerations for building interactivity into web-based courseware, a necessity for keeping students engaged in the learning process. Interactivity provides students with learning opportunities that do not necessarily involve immediate measurement of their learning. Interactivity also requires some form of student action that leads to the achievement of an instructional goal. In other words, interactivity should both introduce some dynamic dimension to the learning and have an instructional purpose. The act of turning a page or clicking the “next” button of a web-based course does not qualify as a form of interactivity; page turning is simply a mechanical necessity for moving through a course. The most engaging interactions often involve communication between course participants (as with facilitated components such as email, chat, or shared whiteboards) or the use of


105

multimedia and streaming technology, as with inclusion of video similar to that depicted in Figure 4.2 earlier in this chapter. Striving to incorporate multimedia into web-based courseware often leads designers into a paradox, however, because many end users’ systems and networks cannot support rich media delivered via the Web. But rather than designing interactions from the standpoint of constraint, we recommend first designing for what students need, and then working with other team members to determine the best way of meeting those needs given the technology environment. The figures below illustrate two examples of interactivity. Both samples add a dynamic element to instruction by encouraging students to engage with the material further. They also reinforce key concepts in each course, rather than use them simply for “entertainment” value. The first example (Figure 4.15) emphasizes a basic difference between high- and low-frequency sound; this capability can be authored in Macromedia Director and delivered with Shockwave. The second interaction (Figure 4.16) allows students to view a workstation’s internal hardware from different angles; this capability can be produced with JavaScript. Figure 4.15

106


Figure 4.16

In addition to the dynamic quality and instructional value of a single interaction, designers of WBT must also consider both the style and frequency of interactions included in their courses. Style of Interactions If a course repeatedly employs one style of interactivity, interactions will not achieve one of their important purposes—keeping students engaged. To avoid monotony, John Hartnett, a frequent writer for Inside Technology Training, recommends following the “three strikes” rule: “. . . try not to use any single interaction three times in a row.”7 Frequency of Interactions It’s a mistake to label a course with only one or two interactions as an “interactive course.” We have found that interactive principles for satellitebased television are applicable to WBT: Interactivity should be incorporated a minimum of every three to five minutes, or roughly every three screens, to maintain a learner’s interest in the material. The average of three to five minutes per three screens will vary depending on individual reading tendencies and the complexity of a given page.


107

Conclusion When written for corporate training, effective course objectives derive from and support business goals. They are also critical to ensuring that all other aspects of a course are designed to guide learners toward the desired level of achievement. Because objectives are such a foundational training element, make certain that they: ♦ ♦ ♦

Address the business problem that prompted the need for training Connect to how students’ performance will be measured on the job Focus on the least students need to know to be successful

If your client has preselected a web delivery system that is incongruent with the training objectives: 1. Be creative, possibly modifying objectives if instructional value isn’t lost. 2. Consider media options. 3. Talk to the client, acting as trusted advisor and informing him or her of instructional and business implications. We are not suggesting that a slight incongruity definitively precludes the use of a web delivery system, but that you should be prepared to work closely with your client, web programmers, and possibly IT personnel to ensure that related issues can be properly addressed. Exploration of web technology should focus on the environment students need to adequately engage in practice activities. It should also focus on considerations associated with effective instructional measurement: ♦ ♦ ♦ ♦ ♦

Situated learning Learning styles Varying styles of practice activities Timely, instructive feedback Interactivity

A minor caution is worth noting here, however. Although complex designs and media can accommodate many higher level objectives, be aware that increased complexity of a web delivery system typically translates into increased costs in time, tools, and programming skills.


108

Objective and Measurement Worksheet for Web Delivery System Suggested Use: 1. Instructional designer completes columns one and two, then consults with web programmers and database administrators regarding the ability of target web environment to process desired actions. 2. Capabilities and suitable media are documented to be used as action plans for web programmers and possibly database administrators. Objective

Preferred Assessment Method and Type of WBT Needed (non-facilitated/mixed/facilitated)

Evaluation of Web Capability


109

References 1.

The Education Coalition. “Bloom’s Taxonomy.” Cited 3 September 1999 from www.tecweb.org/eddevel/blooms.html; INTERNET. (TEC attributes this information to Carla Lane’s “The Distance Learning Technology Resource Guide.”)

2.


3.

Mager, Robert F. Preparing Instructional Objectives. 2d ed. (Belmont, CA: Lake Publishing Company, 1984), 21.

4.

Ibid., 21, 62.

5.

Herrington, Jan and Ron Oliver. “Critical Characteristics of Situated Learning: Implications for the Instructional Design of Multimedia.” Australasian Society for Computers in Learning in Tertiary Education (ASCILITE) (1995 conference paper), 3. Cited 1 March 1999 from www.ASCILITE95.unimelb.edu.au/info.html; INTERNET.

6.

Kemp, Jerrold E. Instructional Design: A Plan for Unit and Course Development. 2d ed. (Belmont, CA: Fearon-Pitman Publishers, 1977), 60.

7.

Hartnett, John. 1999. “Interacting with Interactions.” Inside Technology Training, July/August, 40.


Chapter Five

Defining Learning Paths Chapter topics: ♦ Evaluating the benefits of hypertext ♦ Instructional principles of learning units • Defining and sequencing modules • Defining and sequencing logical learning units • Developing a content outline ♦ Designing WBT architecture and navigation • Intuitive design by necessity • Linearity • Modularity • Predicting and guiding learner behavior: Cues and learning tools ♦ Activity: Cues and learning tools ♦ Documenting design decisions Chapter resources: ♦ Quick guide to course persistence: Bookmarks, cookies, links, server-side tracking ♦ Sample blueprint for web-based course ♦ Sample web circuit

Evaluating the Benefits of Hypertext The previous chapter focused on implementing objectives, practice activities, and interactivity in a web delivery system. This chapter addresses 111


112

the next related series of tasks instructional designers typically follow—forming learning units and sequencing them. This series consists of four primary tasks: 1. Form modules and sequence them. 2. Form learning units and sequence them. 3. Determine placement of practice activities and tests; then identify practice activities and tests. 4. Developing a content outline. As you might expect, each of these four tasks includes several subtasks. However, our purpose here is not to teach instructional design by walking through each step separately, but to explore these instructional design tasks collectively from the perspective of organizing and labeling instructional materials for web delivery. Web designers often refer to matters of organization, labeling, and routes of accessing information on the Web as architecture and navigation. This chapter focuses both on the structural aspects of these attributes and on methods of documenting related design decisions. For information devoted to architecture and navigation in terms of user interface, see Chapter Six, “Presentation Principles.” Most delivery systems are constrained to a linear format, even though the media might differ vastly. Training delivered via classroom, video, and even interactive television all progress in linear fashion. But because of its global interlinking capabilities, a web delivery system can appear to defy the laws of physics: content can exist simultaneously on different “planes” and be accessed from numerous entry points. When used for training purposes, the hypertextual nature of a web environment offers so many possibilities that instructional designers can become entangled in them, and so can students if they cannot discern how the instructional designer intended the course to progress. One quick tour of various WBT Internet sites unearths a catalogue of catch-phrases or “selling points” based on the nonlinear nature of the Web. For instance, many sites claim that web-based training encourages selfdirected learning by allowing students to: ♦ ♦ ♦ ♦

Select only the content they need Skip content they don’t need Define their own unique paths through the content Explore related links during training

Defining Learning Paths

113

We certainly don’t dispute the ability of WBT to provide these features. However, they are not necessarily beneficial to all students in all cases. Rather, instructional designers should examine these features with a critical eye and assess which ones the target learners genuinely need. They should also assess possible implications of selecting or rejecting different self-directing techniques. Consequently, this chapter discusses considerations for structuring content and creating instructional paths for an asynchronous, non-facilitated web course.

Instructional Principles of Learning Units Effectively defining and organizing instructional elements is no small task. “Instructional elements” is indeed a broad term encompassing tests, activities, content, units within modules, and entire modules. Part of an instructional designer’s job is to assemble these elements into an arrangement that: ♦ ♦

Emphasizes main points and tasks Conveys relationships among instructional elements, as well as main points and tasks ♦ Appeals to learners’ pretraining knowledge and skills ♦ Reflects as much as possible how the skills and knowledge will be applied post-training ♦ Progresses at a reasonable pace, in a logical order Combined, these organizing principles help instructional designers consciously sculpt training materials into a shape that is sensible to target learners. While the Web may seem to escape some of the linear bounds of time and space, people cannot. That is, students must follow a path through training material. The concept of “path” is important, because it is not simply a means to an end (i.e., a vehicle for completing the training), but can actually influence learners’ perceptions of concepts and tasks, as well as the time required to comprehend the training material. How we define and order learning units helps shape the learning experience, how people interpret information, and how they apply it. Consider, for example, how the organization of a simple recipe for instant stuffing can set off a chain reaction of mishaps:


114

1. 2. 3. 4.

Mix bread crumbs, seasoning, and 2 tbs. butter in bowl. Add 1 cup water. Make sure water is boiling. Microwave on high for five minutes.

If a “chef” completes each task before reading the next one, the recipe’s organization will cause him or her to make important assumptions, and most likely, mistakes. A microwave is mentioned only in the last step. Can we expect the chef to use a microwave-safe bowl beginning with the first step? Step two calls for water, but the chef doesn’t learn until step three that the water should be boiling. At this point, the lukewarm water has been mixed with the other ingredients. Learning that the water should have been boiling, should the chef start over, or take his or her chances that the stuffing will still turn out well? We can only hope a help-desk number was included on the stuffing box! Organizing the tasks and content differently would certainly clarify the relationships between tasks and lead to better outcomes: 1. In microwave, boil 1 cup water in microwave-safe bowl. 2. Add bread crumbs, seasoning, and 2 tbs. butter to water. 3. Microwave on high for five minutes. Although simple, our example illustrates the significant influence the organization of learning units can have on students, as well as some basic guidelines for organizing training materials. The next three sections review those guidelines. Defining and Sequencing Modules To determine and sequence modules, instructional designers rely on data gathered from initial design efforts described in Chapters Three and Four: ♦ ♦ ♦ ♦

Needs analysis Task analysis Summary of skill and knowledge gaps Terminal objectives

As emphasized in the previous chapter, once learning objectives are written, they should drive other design decisions, including the formation and placement of course modules. Objectives provide an easy starting point


115

for distilling modules from a mass of training data. To identify modules, we typically follow an approach of two main steps: 1. Group related terminal objectives. 2. Define modules according to groupings. Depending on the complexity of terminal objectives and their associated enabling objectives, we can designate one or more modules per complex, terminal objective, or one module that encompasses a grouping of less complex, closely related terminal objectives. When sequencing instructional tasks, Barbara Seels and Zita Glasgow explain that instructional designers must keep one question at the fore of their organizational strategy: “What must the learner know in order to perform this task?”1 This question informs the approach to module sequencing that we have adapted from J. H. Harless, recognized expert in performance technology. Although Harless’ model includes several considerations and subtasks, his primary guidelines are summarized below: ♦

Begin the training with an overview and address any minor prerequisite knowledge the students need. (If there’s more than a module’s worth of prerequisite knowledge, those should be fulfilled before taking the course.) ♦ Next, unfold the course according to the sequence of job tasks or difficulty of concepts, unless there is good reason for altering the instructional sequence, such as allowing learners more time to practice complex skills or to work with difficult concepts.2 Defining and Sequencing Logical Learning Units We define a logical learning unit as a segment within a module that focuses on a portion of a single task (i.e., one or more steps of the task). Each unit concludes with a practice activity. The guidelines and considerations for sequencing entire modules also apply to sequencing learning units within modules: 1. Designate the first unit within each module as a module preview. 2. Organize units according to expected job requirements, unless altering the sequence would better facilitate the learning of new material and skills. ♦ Begin with the most difficult task steps when appropriate for enabling additional practice opportunities.3

116


Developing a Content Outline According to the list of twenty-one instructional design tasks that appears throughout this book, developing a content outline is the seventeenth task. At this stage of the instructional design process, then, the designer has already developed a firm, orderly framework consisting of objectives, modules, learning units, and practice or testing activities. In general, this framework should indicate rather clearly an appropriate content organization. However, organizing instructional content requires more than simply “plugging” chunks of information into their associated learning units and modules. Instructional designers, such as William R. Tracey, give further consideration to the types of content required for learners to meet the skill gaps addressed by each course objective: “knowledge, habits, elements of skill, and emotionalized controls.”4 Additionally, organizing content involves examining the utility of different cognitive models, such as a problemcentered or job-performance approach. For more information on applying these models to instructional content, see Tracey’s Designing Training and Development Systems.

Designing WBT Architecture and Navigation The terms “architecture” and “navigation” are not particular to webbased training. That is, all web sites are built upon an architecture, a system of organization. Additionally, all web sites provide end users with methods of navigating the architecture of a web site and accessing the available information. Yet instructional designers should have particular concerns with architecture and navigation when creating a web course, because these attributes cannot be left solely to the responsibility of web designers: architecture and navigation heavily influence learners’ perceptions and training outcomes. As such, we encourage instructional designers to view these attributes of a web delivery system as new lenses through which to view the formation and arrangement of modules, logical learning units, and content segments. Creating truly effective architecture and navigation for WBT requires collaboration among the instructional designer, web programmer, graphic designer, and sometimes even with members of the target learning audience. When sequencing instructional materials for obviously linear delivery systems, an experienced instructional designer sets to work confidently, well versed in the guidelines already listed, or with similar guidelines. But when


117

the same designer sets out to arrange the same information for a web-based course, those guidelines might be inadequate to address the unique possibilities a web delivery system offers. In observing instructional designers who are new to the web delivery system, we have noticed two dichotomous approaches to designing course architecture and navigation. In the first typical approach, the instructional designer relies on his or her successful experiences in organizing instruction for linear systems. He or she consciously or unconsciously formats the course as if for paper, an inherently linear medium. Such courses, which allow students to access modules and units in strictly linear fashion, are often referred to as “page turners” if no interactive media are integrated into the course. In the second typical approach, the instructional designer relies on his or her experiences as a web surfer with the luxury of following links of interest in any order. This instructional designer often wants to offer learners such freedom with the purpose of raising interest levels through self-directed learning. We believe some learners and training goals warrant either approach in its pure form, but rarely. The worst method of designing the architecture and navigational system of a web course is to base it on your own interests and surfing preferences, as described above. Instead, when instructional designers collaborate with web designers on architecture and navigation, they should focus on those design possibilities that learners genuinely need for achieving course objectives. Seek a balance of options and evaluate their ability to contribute to (or to confound) the desired learning experience. The rest of this chapter can help you do that. Intuitive Design by Necessity The keys to effectively sequencing instruction initially have nothing to do with the delivery system, but everything to do with learning and training needs. At some point, however, the instructional designer must consider concrete ways in which the delivery system can support those needs, including the effectiveness of different sequencing strategies given a delivery system. In a classroom course, for example, an instructor guides learners through instructional sequences—modules, learning units, activities. The instructor provides transitions between sections of material, points out important relationships, clarifies cause-effect associations, and monitors when learners are prepared to proceed to new levels of complexity. Instructors also answer questions and alter lesson plans when a unique blend of students would benefit from an instructional sequence different from the one initially planned.

118


In asynchronous WBT, there is no such facilitator learners can immediately appeal to for similar guidance. Thus, a WBT development team must design architecture and navigation that is intuitive to learners. These aspects of a web delivery system can help guide students through a course, and indeed, must guide students. The design of architecture and navigation should not only support training goals, but anticipate learners’ behavior and expectations. Students take WBT to learn the material, not to puzzle out the unique preferences of a web designer. As we walk through sequencing considerations, specifically in terms of WBT architecture and navigation, keep in mind a caution mentioned in Chapter One: Do not allow the delivery system to define the design of your course. Rather, base your course first on student and training requirements. Organize the course according to those needs, then modify it according to the following considerations when appropriate. These considerations fall within three main categories—linearity, modularity, and cues and learning tools. Linearity The decision between linear versus hypertextual architecture is perhaps the most challenging issue for instructional designers when defining and sequencing modules. Two common (and legitimate) questions regarding this point are: 1. Does a strictly linear design negate the inherent value of a web delivery system? 2. Does a web course based solely on learner-defined sequencing have much instructional value? Our answer to both questions: It depends. As we stressed early in this chapter, organization affects perception and learning outcomes. So again, evaluate carefully whether your architecture and navigational system support the needs of your audience, the stated learning goals, and objectives. In many cases, a course need not be strictly linear, nor should it be littered with links here, there, and everywhere. To evaluate the advantages of nonlinear linking possibilities, it is useful to be aware of potential drawbacks. A course rich in nonlinear links creates potential for learners to:

Defining Learning Paths ♦ ♦ ♦ ♦ ♦

119

Inadvertently lose their place in the training and not find their way back Be held accountable for information they missed in some obscure link Take advantage of the ability to skim and skip through material Make poor decisions when evaluating whether they need more practice in particular skills and concepts Become disoriented

Mentioning such concerns can appall some web design gurus and avid surfers who revel in the Web’s inherent multilevel design, its wondrous way of granting users access to the nether reaches of virtual space. Yet all good web designers will tell you that architecture and navigation should be based on the target audience, their goals, and purpose for visiting a particular web site. These differ vastly when comparing entertainment sites to WBT, and finer differences also arise between different WBT offerings. An important concept to remember is the definition of WBT offered in Chapter One. WBT is “the integration of instructional practices and Internet capabilities to direct a learner toward a specified level of proficiency in a specified competency.” If students have the ability to bypass learning units or activities, should they be held accountable on exams for any material they missed? Perhaps not, if the architecture and navigation encouraged them to skip activities and content. We are not arguing to remove all self-directing opportunities from WBT, but for integrating them judiciously. The rest of this section describes four basic principles of balancing linear and nonlinear learning paths in WBT: 1. 2. 3. 4.

Standardize the entrance. Minimize the links. Maintain consistency. Orient the learner.

1. Standardize the Entrance In reference to hypertextual learning sites, Jennifer Fleming encourages designers to ground all end users by routing them through a single entrance point before allowing them to chart their own navigational paths.5 A physical (versus virtual) example quickly illustrates this point. A convenience store might have two entrances, one in front for customers, one in back for delivery personnel. Each entrance opens on distinctly different areas within the same store, and those spaces are set up for different activities. What happens if the customer enters through the delivery door, or the delivery person enters

120


through the front door? They’re going to be somewhat disoriented, perhaps disruptive to other customers or employees, and have to work harder to accomplish their goals. Similarly, the entrance to WBT is crucial in setting the stage for how learners approach the rest of the training. At the entrance, whether you define it as preliminary material before the first module or as the first module itself, consider indicating to students how the course is intended to be navigated. What choices do they have? Should they be informed of any particular advantages or organizational scheme? A brief orientation can partially fulfill a classroom facilitator’s function by optimizing learners for particular types of end user behavior. 2. Minimize the Links Allowing learners to self-select the order of modules or learning units is much different from integrating links within modules. Such links can lead students away from essential material and interfere with concentration. Training should provide students only with the least they need to know to be successful. In this sense, they should be encouraged to stick with the material, not to stray from it unnecessarily. Unless a link is truly warranted, we look for other, similar methods of shifting the locus of control to learners. Some options include providing access to small pop-up windows or mini slideshows that allow some degree of exploration while remaining on the same screen. 3. Maintain Consistency Learners come to rely on patterns surprisingly quickly. In WBT, radically altering the approach to linearity mid-course can distract and confuse. Try to establish a pattern between the balance of linearity and hypertext, and maintain it throughout the training. Each of the following suggestions should conclude with one qualification: “if the tasks and concepts do not build on each other, do not facilitate the learning of other material, or do not have to be completed in sequence” A. Allow modules to be accessed in the order of learner preference. B. Group modules that must be completed in sequence, and then let students access each module grouping according to preference. C. Make the modules available only through linear access, and then allow learning units to be accessed according to learner preference.


121

4. Orient Learners Let them know where they are, where they’ve been, and where they’re going. The interface allows numerous methods of indicating location within virtual space. The next chapter, “Presentation Principles,” covers those methods in detail. Modularity The principles of instructional sequencing described at the beginning of this chapter are all based on instructional concerns, but concerns regarding the actual context of instruction must also be recognized. For instance, when defining and sequencing modules for delivery via interactive satellite television, instructional designers are conscious of how learners have been conditioned to watch television for entertainment purposes and how those tendencies can affect instruction: Learners can easily fall into passive roles. To avoid this response, we recommend limiting broadcasts to two hour sessions. Similar contextual considerations vary for WBT depending on the specific target audience, but there are general guidelines for evaluating the modularity of a web-based course in terms of module length and ease of exit and return. Module Length Human attention to activities that require concentration significantly wanes after two hours, especially in educational situations. Think back to your own schooling. Was it frustrating to start reading a chapter for history and realize two hours had passed before you finished the chapter? Have you attended classes lasting more than two hours and noticed your peers start to yawn and fidget? Micro Mods Now consider the WBT student undertaking two Elliot Masie of The MASIE hours of instruction in one sitting. In fact, John Center indicates another Hartnett of Inside Technology Training writes that advantage of highly “. . . retention rates for all forms of CBT plummet modularized content. As online learning matures, after about 45 minutes.”6 At the office or home, database technology will there might be any number of tempting allow many courses to be distractions to break the tired learner’s generated dynamically, based on a student’s user profile. concentration and lead him or her simply to Smaller content segments close the web browser. enable higher customizing Instructional designers can use several potential per profile.7 strategies to urge their students toward success and away from such frustration. These methods

122


all involve helping learners develop a sense of progress. In course and module previews, for instance, we indicate the scope of material students will encounter. If they can anticipate course and module length, either in terms of time or number of sections, they can more easily develop a sense of accomplishment after completing a learning unit or module. Shortening module length is another primary strategy creators of WBT use to encourage students’ efforts. By shortening modules, we’re not suggesting removing essential content, but potentially increasing the total number of modules. Shorter modules and tightly designed learning units allow students to feel they are progressing through the course at a reasonable pace. Regardless of overall course or module length, we generally try to construct learning units that can be completed in increments of ten to thirty minutes, depending on the nature of the material. These short bursts of learning not only create a sense of progress, but can improve retention rates. Additionally, this architecture enables learners to exit and return to the course at natural stopping points, as discussed below. Ease of Exit and Return In the interest of user needs, it is generally bad practice to force students to complete all modules during one session in order to complete the training. Structuring a course in discrete learning units adds to the ability of WBT to provide JIT training and to allow students to learn at their own pace. The navigational system should also support these advantages by letting learners exit a course without completing it in one sitting, and then permitting them to resume the course in the same place at a later time. We refer to the ability to resume a course where one left off as course persistence. On the technical side, this ability can be created in several ways, such as with bookmarks, cookies, links, and server-side tracking. Each method has distinct advantages and limitations. To help you evaluate these options with your team, a brief guide to course persistence is included at the end of this chapter. Predicting and Guiding Learner Behavior: Cues and Learning Tools As any good architect will tell you, architecture, whether for virtual spaces or physical spaces, is not just a matter of aesthetics and mobility, but of utility. What is the purpose of the space? How does the design of that space support the purpose? These are important questions for WBT, in which learners are accountable for their actions within the web delivery system. The utility of a given architectural design is subjective; that is, the utility of a


123

space—its ability to support a user’s goals—depends on who is using it. Data on learners’ characteristics and needs should help you predict their goaloriented behavior during training, and therefore, help you create a space suitable to those goals. In addition, the team should explore the following questions to identify learning tools and cues that support students’ goals: ♦

What actions, such as accessing a glossary, will learners want to perform during training? ♦ When and where will they perform those actions? ♦ What learning tools are needed to enable those actions? ♦ When, where, and how should the appropriate learning tools be made available? Either as learners or instructional designers, many of us are so accustomed to the landscape and culture of physical classrooms that we can miss the significance of these questions. Yet WBT is young enough that it has few conventions associated with it in comparison to the traditional classroom. Consider how unconsciously we rely on classroom elements to guide our behavior: ♦

The type and arrangement of furniture implies relationships between people in the classroom and the primary types of communication that will be encouraged. • Watch the mood of a classroom change when an instructor asks students to arrange their desks into small groups or a large circle. It’s rare that the mood doesn’t change (either positively or negatively), because students’ original expectations are suddenly challenged by the change in environment. ♦ The syllabus outlines the tools required for the course, how to use them, and when they’re needed. ♦ Syllabi also allow students to evaluate the course’s boundaries at a glance in terms of length, workload, standards of achievement, the instructor’s office hours, and so forth. Classroom learners rely on such cues and tools throughout a course to gain a sense of overall “architecture,” to guide their behavior and performance, and to help them navigate the learning landscape, so to speak. What cues or tools can WBT learners rely on? The time to begin exploring this


124

question is at the beginning of WBT architecture design. The exercise below walks through a process that initiates this exploration. Activity: Cues and Learning Tools To help our teams envision what the ideal learning environment might look like in a web delivery system given a specific target audience, we walk through four main steps, which are explained below. The discussion generated at each step should focus not on the team’s personal surfing preferences, but on features that will clearly encourage target students to learn the material. After preliminary discussions, we hone and test the design until it meets the needs of target learners; we verify the effectiveness of design through early prototyping, reviews with SMEs, and usability tests with representative students. The first step derives from a suggestion Jennifer Fleming offers in Web Navigation: Designing the User Experience: “Begin thinking in scenario.”8 1. Using a variety of data (such as needs and tasks analyses, skill gaps, and terminal objectives), generate scenarios to predict learner behavior in the web delivery system. 2. Identify salient learner characteristics and needs based on scenarios. 3. Evaluate a variety of potential learning tools to meet needs: • Chat • Bulletin board • Instant note taking (via a text editor that allows students to type in key points and to print them after the training) • Glossary of terms • Help • Email: instructor or webmaster • Search: for information or for classmates currently logged in • Portable Document Format (PDF) job aids corresponding to modules or learning units 4. Identify optimal placement of learning tools (depends on how learners need to use the tools). Unnecessary or misplaced tools can distract learners rather than raise their performance, interest, and confidence levels. The example below illustrates the main steps of this exercise, as well as how these steps function together as a process.


125

Scenario: Predicting Learner Behavior Course background: ♦ Course title: “Orientation to Strategic Development” ♦ Target audience: new hires in the Research and Development division of a large computer manufacturer • The company consists of several departments that must coordinate strategic objectives, supply shipments, and production schedules. ♦ Course goal: to provide new hires in Research and Development with a brief introduction to each department, its managers, its strategic objectives, and its involvement with Research and Development 1. Generate scenarios. Kristin was with the company for one week before she signed up to take the web-based course. The second module lists the title of each department, along with its goals and upper-level managers. Kristin completed this module successfully. The third module describes the involvement of Research and Development with key departments, which are referred to by acronyms that were introduced in the first module. Kristin wanted an easy way to refer to those acronym meanings, but did not want to lose her place in module three. Instead, she simply continued to move forward through the course. Although she completed the course with relative ease, she feels that because it was so detailed, she won’t be able to retain all of the information. She received an acceptable score, but would like to have copies of diagrams and references presented throughout the course (organization charts, the key to acronyms, etc.). 2. Identify salient learner needs. Kristin encountered three difficulties during training: unfamiliarity with acronyms, insecurity with navigation, and a desire for post-training reinforcement. These difficulties connect to the same need: easy access to reference-oriented information. Because this course is specifically designed for new company employees, many of Kristin’s peers will likely experience similar frustrations. New employees are often overwhelmed with detail in the first few weeks, from how to complete job tasks to finding their way to the water cooler down the hall. 3. Evaluate a variety of potential learning tools to meet needs. A web delivery system offers several options for meeting Kristin’s learning needs: instant note-taking, glossary of terms, search, PDF job aids, and so

126


forth. Given the nature of the needs described, we would select a glossary of terms and PDF job aids as the most useful tools. We would create the glossary in the form of a pop-up window that has both a search option and an index the learner can scroll through; “search” won’t work for terms learners either misspell or cannot remember. We would not select a note-taking function or a button simply titled “search.” The ability to type notes within the training might add to the pressure of being a new hire and trying to memorize every detail he or she encounters, potentially detracting from the advantage of short bursts of online training. Additionally, a button titled “search” is rather vague because it is context dependent. 4. Identify optimal placement of learning tools. As mentioned, tool placement depends on how learners need to use a given tool. Poorly placed tools can distract learners from actual training content or from achieving their goals in a timely fashion. In this scenario, possible areas of placement are: ♦ On the first screen of the course that lists each module title for overview purposes ♦ On every screen of each module ♦ At the beginning or end of each module ♦ At the very end of the course, after the learner completes all activities Both the glossary and the ability to print PDF files of reference segments relate directly to course content, so we would place these tools within each module, where they would be most useful to learners. However, we would use slightly different placement strategies for each tool: Glossary ♦ Consistently placed on each screen of the course • If learners cannot remember a term or its definition, they are less likely to remember a list of words they want to look up when the glossary option sporadically appears. PDF job aids ♦ Consistently placed at the end of each module quiz, available after students submit their answers ♦ Unless they are genuinely needed and carefully placed, any type of printable material offered during a web-based course has the potential to influence the learning experience in undesirable ways.


127

Obviously, allowing tests and test results to be printed can compromise their validity. Less obvious, however, are the risks of providing even small segments of printable content. If placed too early, some learners might print them as a substitute for actually taking the training. If these job aids are only offered at the end of a course, learners might not remember which materials they want to print. Given the scenario, we would place this option at the end of every module, available only after all module activities are completed. We believe this placement supports the described course goal and the needs of these particular new hires.

Documenting Design Decisions We conclude this chapter with two methods of documenting instructional design decisions for web-based courses—blueprints and web circuits. According to our instructional design model, after the team has sequenced modules and learning units, identified content for those segments, and selected the delivery system and media, they must finally “document details on course blueprint.” Blueprints Instructional designers typically create blueprints for all courses, regardless of the delivery system. Blueprint creation marks a milestone between design and development, giving clients and vendors an opportunity to verify their perceptions of what a course should look like—before spending the effort on actually creating the course. While the format of a blueprint is not as critical as the information it contains and its readability, these documents must be intuitive enough to be evaluated and approved by clients and implemented by other team members. Our blueprints include the following items: ♦ ♦ ♦

Course description Course goal Brief target audience description ♦ Statement of testing methods ♦ List of modules ♦ Terminal and enabling objectives

♦

Content descriptions by module and learning unit ♦ Media ♦ Types of activities, tests, and interactions ♦ Approximate duration for each module and entire course

128


For illustration, a partial blueprint appears at the end of this chapter. This blueprint is for a non-facilitated, asynchronous web-based course. Web Circuits (Site/Navigation Maps for Designers) For our web-based courses, we often include a web circuit with blueprints to help clients and team members visualize learning paths on a global level. A web circuit generally maps the paths users can take through a web course, shows where each major element resides in the path, the routes by which they can be accessed, and how each element connects to other elements. Web circuits can also include finer details, such as tool buttons that should be accessible from selected screens. In some cases, “web circuit” is interchangeable with terms you may already be familiar with—site and navigation maps. However, a web circuit should be used for development purposes only, and should not be placed in a web-based course for students to view. A sample web circuit appears at the end of this chapter.

Conclusion Several excellent references on web architecture and navigation are available, such as Jennifer Fleming’s Web Navigation: Designing the User Experience, or Information Architecture for the World Wide Web by Louis Rosenfeld and Peter Morville. But the notion of accountability casts the design of architecture and navigation in a slightly different light for webbased courseware, in which instructional designers are responsible for helping learners master training materials, and learners are responsible for achieving the course goal and objectives. In many cases, WBT students are also accountable to some “authority,” such as a supervisor or an instructor, for making these achievements. At the beginning of this chapter, we summarized a handful of popular claims supporting the liberal use of hypertext in WBT. While we agree that the possibilities of hypertext are exciting, learners can suffer when instructional designers become seduced by those possibilities. In “Making Sense of an Authoring System,” Mark Fritz implies that this seduction can result in “too much learner control and too little monitoring by those who devised the courseware and have an interest in whether trainees learned what they were supposed to learn. Hypermedia puts nearly all the power in the user’s hands.”9 Certainly, students should have some power in their learning, and to disregard the interlinking capabilities of a web delivery system might be


129

viewed as being as inappropriate as integrating media and technology for their own sake. In general, WBT students are best served when instructional designers seek a balance between hypertextuality and linearity by asking: ♦ ♦ ♦ ♦ ♦

What’s the ideal learning environment for these students? How can we improve the modularity of the course? What value will hot links provide this group of learners? What kind of architecture and navigation will seem intuitive to them? What cues can we include to build a bridge between learners and course content? ♦ Which learning tools will support their efforts and guide them along the learning path? The next chapter builds on the concepts of architecture and navigation by offering techniques for creating an effective user interface for WBT.

130


Quick Guide to Course Persistence: Bookmarks, Cookies, Links, Server-side Tracking This guide introduces four methods of creating course persistence in web-based courseware. As an introduction, it is intended to highlight major distinctions between methods, as well as basic advantages and limitations of each method. Bookmarks Encouraging learners to use the bookmarking function of their web browsers is the easiest way to provide exit and return capability. It is also the least reliable. Because the bookmarking feature is part of the web browser and not the actual WBT interface, it might not even occur to learners that it is a viable option. Additionally, using functions that are not part of the WBT interface might appear to be risky. “If I use a bookmark to return, will the course ‘know’ that I completed most of the activities? Will I have to do those again?” Bookmarks have other limitations, too. If the architecture integrates frames, the bookmark might only return a student to the frame the cursor was in when he or she created the bookmark. Upon returning to the course, the student might receive a scrambled presentation or be cut off from important content and navigation controls. Additionally, bookmarks are local. That is, they are stored only on the computer from which they are created, which creates difficulty if a student begins the course on one computer, but needs to complete it on another. Cookies Cookies provide an easy method of returning students to their place in a course. A cookie is a small piece of text data that is stored in a file, often called “cookies.txt,” on a learner’s computer. It allows the web server to remember important information about the people who visit a site. For example, whenever a learner logs off from the course, a cookie could be set that contains data such as “last_page=page.html.” With this information, the web application could automatically return the learners to the page where they left off as soon as they log on again.


131

However, because they write data to the user’s computer, cookies can create security risks; sensitive data, such as passwords or test scores, should never be stored in cookies. Cookies also impose the necessity of using the same computer to complete a course. Links Links can be made to all the sections of a course. This solution is adequate for many learning systems, especially those that are hypertextually rich and that allow students to create their own paths through the content. However, this approach often requires learners to remember where they left off in the course and to find their own way back. To assist these students, we suggest providing a course map, a table of contents, or an index with links to the beginning of each module or learning unit. (Maps, indexes, and content tables also serve as basic orienting techniques, which are discussed further in the next chapter.) Server-side Tracking Server-side tracking provides the most reliable method of tracking student progress throughout a course. Within a module, certain pages are designated as completion points. As students progress past the completion points, data is recorded in the server-side database. Use of completion points and server-side tracking provide some measure of course persistence. That is, should a user’s network connection fail, they can return to the last visited completion point. Because data is stored on the server and not the client computer, students also have the freedom to begin the training on one computer and finish it on another without loss of data. Because server-side tracking requires a database and server programming, it is generally more difficult to implement than any of the above options.

132


Sample Blueprint for Web-Based Course This is not a complete blueprint. We have included the overview page and the description of one interaction simply for the purpose of illustration.

Course Blueprint for Hearing Conservation

Reviewed and approved on: Date____________ By: ____________________________________________ Signature

Created by TrainingLinks

Course Description This course fulfills basic training requirements of OSHA’s Occupational Noise Exposure Standard (29 CFR 1910.95). In a web-based environment, participants will be able to distinguish key connections between the physics of sound and hearing, recognize the importance and appropriate use of personal protective equipment for hearing, and identify the purpose and method of audiometric testing as required by OSHA.

Course Goal Upon successful completion of this course, participants will gain awareness of occupational noise hazards and become familiar with safety measures to protect their hearing.

Target Audience The audience consists of employees in general industry, especially manufacturing industries. They require an interpretation of the OSHA standard that focuses on their needs and responsibilities. The responsibilities of employers and manufacturers are included only if the OSHA standard requires such inclusion or if the information directly affects employee safety.

Course Testing/Measurement After completing the course modules (including end of module practice activities), participants will complete a course examination. The examination consists of thirty multiple choice questions, some of which are scenario-based.


133

Course Modules Introduction Module 1 – Basics of Hearing and Noise Hazards Module 2 – Personal Protective Equipment Module 3 – Audiometric Testing Module 4 – Course Summary Course Examination Module One: Basics of Hearing and Noise Hazards Module Objective Recognize how noise hazards affect hearing and the ear Enabling Content Media objectives

Compare three common types of noise

Same

Define three noise types: 1. Impulse noise (short bursts, as with typewriters) 2. Narrow-band noise (narrow frequency ranges, as with a circular saw) 3. Wide-based noise (broad frequency range, as with a car engine) Multiple choice reinforcement question comparing two of the three common types of noise—answer choices will require participant to distinguish among all three noise types.

Practice, testing, or interactivity

Three graphics and sound files that correspond with the three common noise types

Interactivity: Participant clicks each graphic to receive a demonstration of the noise type

Standard presentation of text on screen

Participant clicks button to indicate answer

Approx. duratio n or # of screens 2 min.

1 min.

134


Sample Web Circuit

Walk-through of Web Circuit 1. The column on the right indicates functions or tools that should be available to students on every screen of the course. 2. Learners select training modules from the module access screen. Upon completing a module, students are automatically returned to the access screen to select the next module. (This method can be applied to both linear and nonlinear courses.) 3. Students can access the glossary and note taking functions only within modules, but not during the course exam. ♦ To ensure each student completes a course evaluation, the evaluation must be submitted before students receive their exam results. (For more


135

information on course evaluations, see Chapter Eight, “Course Evaluation and Revision Maintenance.”) 5. When students receive their exam results, they are also given the option to review key points of the course or to save notes they have typed during the course. 6. From the end of course screen, students can select three options: A. Return to the beginning to retake the course if they received an unacceptable score. B. Email the instructor with questions or comments (Email capability is reserved until the end of this course, encouraging students to fully engage with the material on their own rather than appealing immediately to the help of an authority.) C. Resources such as related URLs or job aids that can be printed after course completion.

136


References 1. Seels, Barbara and Zita Glasgow. Making Instructional Design Decisions. 2d ed. (Upper Saddle River, NJ: Prentice Hall, 1998), 268. 2. Harless, J. H. “Job Aid for Course Design.” (Newnan, GA: Harless Performance Guild, 1990), 9-11. 3. Harless, J. H. “Job Aid for Module Design.” (Newnan, GA: Harless Performance Guild, 1990). 4. Tracey, William R. Designing Training and Development Systems. 3rd ed. (San Francisco: American Management Association, 1992), 231. 5. Fleming, Jennifer. Web Navigation: Designing the User Experience. (Sebastopol, CA: O’Reilly and Associates, 1998), 184-185. 6. Hartnett, John. 1999. “Interacting with Interactions.” Inside Technology Training, July/August, 40. 7. Masie, Elliott. “Advice for Designer of Online Learning—Think Small.” Technology for Learning (May 1997). Cited 23 September 1999 from www.masie.com/articles/advice.html; INTERNET. 8. Fleming, Jennifer. Web Navigation: Designing the User Experience. (Sebastopol, CA: O’Reilly and Associates, 1998), 8. 9. Fritz, Mark. “Making Sense of Authoring Systems.” Training (Nov. 1993), 2. Cited 1 Sept. 1999 from http://www.trainingsupersite.com/tssmain/archives_search/frame.htm; INTERNET.

Chapter Six

Presentation Principles Chapter topics: ♦ Virtual classroom carpentry • What is the UI? • UI development process ♦ Step 1: Create UI prototype • Thematic systems and patterns • Team brainstorm: case studies ♦ Activity: UI brainstorm • UI prototype considerations ♦ Step 2: Write course content • Storyboard template • Effective web writing ♦ Step 3: Integrate UI elements Chapter resources ♦ Reference sheet: General UI guidelines

Virtual Classroom Carpentry In the last chapter, we described methods of organizing the instructional elements of a web-based course, as well as depicting them on instructional blueprints and web circuits. In relation to our instructional design model, finalizing the blueprint signals the end of design and the beginning of development, and implementation of all decisions made to this point. This 137

138


chapter marks a milestone, then. A WBT team not only designs instruction, but also has the opportunity to craft the “classroom,” or the user interface (UI) of the course. As a result, team members have more control over certain aspects of web-based delivery than a team who hands off the final courseware for classroom delivery. Development of the UI involves unique challenges and responsibilities, some of which were introduced in the previous chapter in terms of creating an “intuitive design.” To an extent, web designers view “intuitive design” as synonymous with “usability.” Usability is a fairly cumbersome word, but it represents a simple concept—the degree to which an identified audience finds a web site “usable” given their needs and goals. Although project managers and instructional designers often do not have formal graphic and web design training, they should cultivate a general understanding of usability issues in respect to target learners and be able to assess WBT courseware from the standpoint of the user interface. Additionally, project managers need to supervise the interaction of course developers, graphic designers, and web programmers, all of whom have different priorities and perspectives. To help you negotiate this territory, this chapter covers a process for creating the UI, including graphic and content development, and a method of integrating graphics and content. To begin, the next section defines “user interface.” What Is the UI? A user interface is a visual display of information and controls. In other words, it is what web users see on screen and the navigational devices they use to access information and to perform other tasks such as to download, purchase, or chat. The UI is also the web programmer’s primary tool for communicating where users are, what they can do, and how they can do it. When a UI poses barriers to the goals of an average web surfer, the surfer has a simple solution: leave. Go to another of several hundred similar sites. When developed for training purposes, however, the UI must facilitate the learning experience and anticipate student needs and questions without calling attention to itself. Students taking WBT will likely have a different mindset than the casual web surfer. They are less likely to leave your site once they have begun taking a course, particularly if the training is a job requirement. Thus, it’s critical for WBT to have an efficient, intuitive interface. If UI construction is analogous to building a physical classroom, it cannot simply be reduced to aesthetic issues such as the color of wallpaper or the size of windows, although aesthetic considerations are part of UI design.

Presentation Principles

139

In contrast to web surfers, how can a WBT student contend with UI barriers? ♦ ♦ ♦

Muddle through the training. Complain to a manager or webmaster. Fill out a negative course evaluation.

While this isn’t a positive list, it is realistic. To eliminate the muddle factor, our WBT teams follow the basic rules of UI design: ♦ ♦ ♦

Be consistent. Keep it simple, and avoid screen clutter. Design for the target learner and no one else.

These rules sound easy, but design teams often struggle with the question of how to implement them. Our purpose is not to write a definitive chapter on user interface design, but rather to explore basic concepts relevant to WBT development. Given this goal, we hope you notice connections between key points of this chapter and those of previous chapters. UI Development Process When developing a user interface for WBT, we follow a process of three main steps. While this chapter covers each step in detail, Figure 6.1 provides an overview of the entire process. Figure 6.1: UI Development Process

Create UI Prototype • Team brainstorm • Develop web prototype • Client review

Write Course Content • Create storyboard template • Write draft(s) • Client review

Integrate UI Elements • Import text into user interface • Add graphics • Team and client review

The process begins with a brainstorm during which team members generate design ideas for the user interface. To make sure our designs meet the needs of course participants, we typically invite members from the client team, such as the project manager, an SME, and business sponsor, to this brainstorm session. (We also suggest inviting one or two target course

140


participants for the same purpose.) Using the web circuit, scenarios for cues and learning tools (see Chapter Five), and information from the brainstorm, graphic designers create an interface prototype. Once the client approves the prototype, course developers write course content according to the blueprint, suggesting graphics that reinforce key training concepts. Next, the course is reviewed for quality, accuracy, and relevance to training and learner needs. After the course is approved by the client and SME, content and graphics are integrated with the interface, and the integrated product undergoes a final, online review with the team and client. For more information on testing techniques, see Chapter Seven “Courseware Testing.”

Step 1: Create UI Prototype UI design is traditionally viewed as the domain of graphic designers and web programmers. Yet all team members have uniquely focused skill sets and visions of the target learners and courseware. The instructional designer is typically most familiar with target learners’ characteristics and needs, and this knowledge must be integrated into the UI if the interface is, indeed, a communication tool. Effectively sharing ideas requires a frame of reference to ensure that discussions of UI design are goal-oriented, not a debate over UI “ownership.” The following section explores the UI as a communicative device and should provide a common ground for preliminary discussions of interface design. Thematic Systems and Patterns Within the realm of communication, a UI is a rather unique method. Effective communication is a system driven by mutually agreeable rules for conveying meaning. Often, those rules are not overtly agreed upon, but accepted through convention. The Internet has some conventions, but relatively few; web surfers must often relearn basic navigational and thematic systems whenever they encounter a new site. Because it is impossible to secure “communication agreements” with every member of a web audience or to rely on long-standing conventions, it’s important to: • Be conscious of the WBT interface as a communication system. • Simulate a communication agreement with learners by basing the UI system on their characteristics and needs. • Appeal to users’ unconscious reliance on inductive reasoning.


141

Humans rely heavily on inductive reasoning to Inductive reasoning: filter background from foreground. To help us focus the act of inferring our attention, we require stable patterns in our general principles from individual, surroundings. In the United States, red lights repeated instances eventually turn green; hot water taps are usually marked with an “H”; and barring tragedy and scandal, presidential terms last four years. We don’t waste time wondering whether red will turn to green this time, if we’ll get hot water from the “C” tap, or when the campaign hoopla will return. The UI of a web-based course should be just as stable. Imagine being a visitor to a new country and knowing only a handful of words from your pocket translation dictionary. You’re unfamiliar with the driving rules, but you need to get things done. The citizens are accustomed to all of the rules, the layout of the streets, what the signs mean, etc. Regardless of the seasons or time of day, they know Waltzberg Street always crosses Meer Place, but you wouldn’t be so sure, even if you were able to find those places (which you cannot). If you spend enough time in this new land, however, you’ll learn the patterns and begin taking them for granted. Two rules for the UI of a web-based course derive from this scenario: ♦ ♦

Use patterns to help learners take the system for granted. Use the “language” of the learners when possible to avoid virtual “culture shock.”

For example, if learners can access a glossary through a button on the UI, you will certainly place it consistently, but how should you label it? What words correspond well both with the training material and with learners’ experiences? “Reference,” “terms,” “resources,” “key words,” “dictionary,” “look up,” “definitions”? Once learned, students should not have to relearn the interface logic of a web-based course, its architecture, navigation, or labeling systems. Nor should they be surprised that module four concludes with a quiz if modules two and three concluded with quizzes. If it is well constructed, it will not take long for learners to infer the interface logic of a web delivery system, and we want them to do this as quickly as possible, so they can focus on achieving the course objectives. The use of thematic systems can help UI designers achieve these goals in WBT. We define a thematic system as a stable assembly of web elements that support a central motif or fundamental organizing concept. A thematic


142

system encompasses the spectrum of UI elements. Any set of elements a team selects should be used in consistent ways throughout a web-based course. These elements should also complement each other and the training material: ♦ ♦ ♦ ♦ ♦ ♦ ♦

Navigation bars and buttons Labeling schemes Graphics and photographs Text layout Writing style Use of metaphors, stories, or scenarios Colors and fonts

A thematic system creates an overall impression for end users. For example, if you want to appeal to a target audience of financial brokers, it would be incongruent and distracting if the graphics combined both professional photography and cartoon drawings to illustrate key points. Which approach to graphics would be most fitting for the material and this target audience? Thematic systems can also be as complex as full-blown scenarios or metaphors worked into course content that students experience as they complete the course, essentially positioning them as participants in the scenario, metaphor, or story as they learn the material. However, poor use of metaphor can be worse than using none at all. Metaphors offer one teaching strategy among many and should not be equated with or overshadow training, but should instead be used to facilitate learning. The more narrow and extensive the metaphor, the more a design team must know about an audience’s perspective, background, characteristics, and the environment into which their new skills will be transferred. By definition, metaphors equate an object or idea with another to suggest a likeness. In writing that “Juliet is the sun,” for example, Shakespeare assumed that his audience would be familiar with the sun’s characteristics and would thus learn something about Juliet by applying those characteristics to her. As a decidedly poetic example, consider a design team working on a safety course intended to inform students of how to protect themselves against bloodborne pathogens. The team might base the UI on the metaphor of a lab where bloodwork is performed, but unless all audience segments have some lab experience, the metaphor might actually alienate some learners. Perhaps in reality, many people would be able to use their imagination to bridge the gap between the lab metaphor and course content, but in some cases, metaphors simply impose cognitive burdens. The UI shouldn’t call attention to itself, but


143

should help students focus on the actual content instead of trying to disentangle it from a creative metaphor. Web environments are not literal environments. And why would you want them to be? Virtual space is governed by its own laws of “physics,” and applying physical laws from the physical world can evoke tedium from learners. In Designing Visual Interfaces, for instance, Kevin Mullet and Darrell Sano explain that “The extremely literal translation of the ‘real’ world . . . virtually ensures that users will find the resulting environments cumbersome and inefficient, and probably just as cluttered . . . .”1 In the book’s foreword, Jakob Nielsen provides an example encompassing both literal translations and metaphor: “I recently saw a system where the concept of a ‘queue’ was represented by an icon of a billiard ball (‘cue ball,’ get it?). Such visual puns may be fun to throw around in a design session but they are often detrimental to the novice user trying to make sense of a new visual environment.”2 To avoid imposing our own metaphor preferences on the UI, we distill thematic systems and patterns from fundamental aspects of course content based on questions that range from foundational to aesthetic issues. Relevant questions include: ♦ ♦ ♦

What is the tone of the material? What word associations can we make with the material? Is the material based on any particular philosophy? Are learners familiar with a related philosophy from the environment in which they will apply their skills? ♦ Is the training connected with any particular symbols? Depending on how they are answered, these questions can direct a team’s approach to vocabulary for navigational buttons and labeling systems, key interface shapes, the logic of a navigation bar, color, font, and general layout. The user interface needs to carry learners through the course, and so must function as a unifying element for all learning units, modules, and audience segments. Because the thematic system of an interface is designed for a specific group of learners and a specific training goal, there is no “formula” our teams use to create such a system. But any of the following might function as a unifying theme based on who learners are, what the course is about, and why they are taking the course:


144 ♦ ♦ ♦ ♦ ♦ ♦

Tasks Job or department titles Process phases Locations Events and calendars Tools

Often, the titles of modules and learning units alone provide the basis of the best thematic system, because that basis is logical and informative, yet simple. The first priority of a thematic system is to create clarity, not to provide entertainment value (although we hope learners find the thematic system engaging). The next section illustrates two teams’ approaches to a thematic system. These examples are followed by finer UI considerations, including technical constraints. Team Brainstorm: Case Studies Although we’ve provided a rather lengthy explanation of what a thematic system is, actually developing such a system is a creative process that can extend into several aspects of UI design. The following studies combine the experiences of several TrainingLinks teams. They are presented as two distinct projects only for the sake of illustrating key considerations in our WBT development process. While the training goals differ vastly between the examples, both teams worked to achieve congruence in the UI design between three elements: the training material, the perspective of the learning audience, and the thematic system. The discussions of both teams were heavily audience-oriented and integrated audience analysis data: What is the learners’ work environment like? What are key words from the training material they can identify with? What would they like to see? What kind of design would they view as credible? What kind of interface would they be most comfortable using? 1: Safe Work Practices ♦ Target learners: • Employees who must follow federal regulations for handling chemicals safely ♦ Selected key words from the team’s initial UI brainstorm session: • Stability • Process • Safety • Responsibility


145

• Protection • Clean environment • Regulation Based on these concepts, the team explored several possibilities concerning both graphic design and earlier decisions they had made regarding architecture and navigation. Their main goal was to reflect the above concepts through the UI, but without literally representing the work environment. Aside from the dangers of literal metaphors described earlier, health and safety trainers can get into liability problems if the instruction reveals any behaviors not aligned with federal regulations, so the team felt that a scenario/metaphor approach to a thematic system would be too suggestive. The team had initially planned a heavily linear navigational system, but disagreements as to whether linearity was really necessary arose during the brainstorm. However, the team ultimately decided that the initial, linear navigation best reflected the philosophy of the training, because it was consistent with the orderly work environment of learners. And while not strictly dependent on each other, concepts presented throughout the course were arranged to help learners see how distinct safety practices fit into an overall safety program. As the team continued its discussion, it arrived at several goals for the interface design, which the graphic designers and web programmers used as input for the UI prototype: ♦ ♦ ♦ ♦

An “illuminated” design, one bright and crisp, but not overly clinical A navigation system sensible to users with limited Internet experience A navigation system that allows users to track their own progress An architecture based on modules that correlate to learners’ main safety responsibilities ♦ Graphical depictions of health and safety dangers designed to inform, rather than shock students Graphic designers and web programmers used this input to create the design depicted in Figure 6.2, which functions as an anchor page throughout the course by: ♦ ♦ ♦ ♦

Providing learners with an instant overview Indicating the learner’s location within the course Distinguishing between completed and noncompleted modules Managing learning and navigational processes intuitively, without need for detailed instructions


146

Learners click on a module title to begin. After completing the module, they are automatically returned to the anchor page to select the next module. From this screen shot, we can see the learner’s progress: The first module has been completed, and the second module, “Basics of Exposure,” is now active and accessible to the learner. Figure 6.2 2: Computer Sales ♦ Target learners: • Sales representatives who complete the training to maintain their knowledge of dynamic computer products ♦ Selected key words from team brainstorm: • Precision • Quality • Streamlined • Professional • Speed • Reliability • Cutting edge • Credibility While brainstorming the “mood” of the UI, this team discussed another key factor that needed to be addressed through the interface. The client wanted to boost learners’ confidence in selling its products by communicating its commitment to technology and quality. When designing for an outside client, it is important that you focus on meeting their needs by meeting the


147

needs of their learners. In other words, the design is ultimately for learners, not primarily for managers or executives. Yet the team recognized that the training would gain credibility with the target audience if learners could sense the manufacturer’s presence and philosophy in the design.

Figure 6.3 The team wanted a sleek design to convey the manufacturer’s image to this audience. They also looked for salient organizational principles to harmonize with the instructional sequencing defined in the blueprint. The interface depicted in Figure 6.3 reflects, according to audience data, how learners wanted to access the information, not by discrete topics such as product features or the competition, but by product families. Additionally,

148


Figure 6.4 the sales audience needed JIT training to accommodate their need for instant information in short bursts that they could access between customer contacts and during business travel. Sales representatives also wanted to access the information in any order to prepare for sales calls, so the team designed the modules to be accessed nonsequentially, whereas the units within modules would be navigated linearly to ensure instructional integrity. As with the UI for the health and safety course described earlier, this interface is primarily self-managing, requiring little effort to navigate and find the information students need. We decided to break out units within modules to function as landmarks for easy exit and return to the course. Upon successful completion,


149

each unit is marked complete. The anchor page for this course appears in Figure 6.3. A final specification involved job aids. By definition, job aids serve a purpose that is distinct from training. In contrast to the placement of job aids described in the last chapter, this course has a more directed purpose beyond introducing learners to new-hire issues. So rather than integrating job aids within modules for this sales course, the team decided to add a job-aid button on the navigation bar. Figure 6.4 presents the access page for job aids. They are easily accessible to preserve their just-in-time value, saving students from sifting through modules to find them. The activity below synthesizes considerations that should be made in the first step of our UI design process.

Activity: UI Brainstorm Suggested Materials ♦ Blueprint ♦ Web circuit ♦ Audience analysis and training data Suggested Participants ♦ Instructional designers ♦ Course developers ♦ Graphic artists ♦ Web programmers ♦ Client representatives, such as the business sponsor or SME ♦ Target audience representative Suggested Approach 1. Ask: “What concepts, philosophy, or image do we want to convey through the user interface?” • Write all responses on whiteboard or flipchart. Avoid evaluating the ideas offered. 2. Ask: “Can we narrow this list down? Which ideas complement our blueprint and web circuit?” • Explore complementary elements for their possibilities. Take time to discuss any conflicts. The sooner fundamental issues (such as navigation) are settled, the better, because it can prevent rework later in the project.

150


3. Ask: “How can we express or summarize the goals of the user interface?” • At this point, define the goals as clearly and concisely as possible, evaluating each goal and gaining buy-in from those who have a high stake or level of expertise in this area. • The meeting should conclude with a clear plan and design specifications. The graphic designer and possibly web programmer develop the interface prototype according to those specifications. UI Prototype Considerations As graphic designers and web programmers develop the interface, they must negotiate among several interests: those of learners, clients, and other team members. Depending on team dynamics, contending with the visions of other team members can become a significant challenge; everyone wants to participate in the creative “fun” of developing the UI. Chapter Two pointed out, however, that an effective WBT team requires a solid mix of skills to be successful. While they benefit from the perspectives of other team members, graphic designers and web programmers are trained to ensure that a wealth of parameters are met regarding visual principles, usability, and technical restrictions. This section provides an overview of basic design issues that project managers and instructional designers should be aware of in order to better evaluate project progress. For more information, see Appendix C, “Creating Graphics for the Web.” Our usability expert, Jonathan Payne, explains his views on crafting a web delivery system: “Web designers are driven by a single concern—drawing users into the information they need. A web-based course should include ‘bells and whistles’ only if they’re effective learning tools. The information fashions the interface and tools, not vice versa.”3 When creating the UI, then, the designer’s number one issue should be ease-of-use derived from balanced elements within a single system. Beauty of the interface should come second, with the following goals as priorities: ♦ ♦ ♦ ♦

Keep it consistent. Keep it simple and avoid screen clutter. Make it easy to learn. Provide buttons and links for all of the necessary functions of the course, even those provided as standard browser options such as “forward” and “back.” ♦ Use and place all navigational devices (such as “Next” and “Back” buttons) consistently, so users don’t have to relearn the logic of each page.


151

For consistency, gray out (rather than removing) any buttons that are unnecessary for a given page. ♦ Icons used for navigation should convey clearly (1) that they are meant to be clicked, and (2) what will happen if they are clicked. ♦ Avoid introducing elements that significantly delay the learning of patterns or that disrupt patterns, such as asking students to load a plug-in after the course has begun. Inconsistent or mixed metaphors are also disruptive. Before actually creating the graphics of the UI, designers must also evaluate technical constraints. In some cases, an entire team might agree on certain design aspects, but those aspects might require modification to ensure technical reliability for target learners. At this point in the project, web designers should have a solid profile of end users’ systems, and the web delivery system should be developed to match those profiles. First, consider the browser version. As discussed in Chapter Three, we advise building for browsers at least one version prior to the latest. DHTML allows great possibilities for animated menu systems, but is only compatible with 4.0 browsers or better. Next, consider monitor size. We generally suggest building interfaces to be viewed at 800x600 pixels, because this size is accommodated by most monitors and provides enough room to display appealing navigation bars and button systems without crowding the screen. Keep in mind, however, that many laptops and older monitors are not capable of rendering interfaces above 640x480 pixels. (You can test your site by changing your monitor settings to 800x600 and 640x480, and then opening up your site to see how it looks at these sizes.) To standardize what viewers see, we often launch a course in a sized window, either 640x480 or 800x600, depending on the systems of target end users. While testing for monitor-related difficulties, also consider color schemes. Garish or clashing colors can distract learners and influence their psychology in undesirable ways. In some cultures, for instance, red is thought to represent aggression, and yellow to cause impatience. Additionally, Lynda Weinman and Bruce Heavin, authors of , explain that a color scheme might look great on your monitor, but look undesirable on another monitor. Often this is due to the varied quality of end users’ graphic cards, some of which might only be able to display 256 predetermined colors, substituting “similar” colors for those requested in the image file. The number of colors used can also affect download times for image files.4

152


As they build the interface, graphic and web designers work with these considerations in mind and test the quality of their design before the client views it. The project manager or team should also review or troubleshoot the prototype before it is offered to the client for approval.

Step 2: Write Course Content

Once a client approves the user interface, we are ready to develop course content. (Often, the content and UI can be developed simultaneously.) We begin by building a text-based, storyboard template that mirrors the webbased interface. Compared to coding an entire course in a web environment, a text template requires only a small time investment up front, and can save much effort in the long term. Because it is representative of the UI, the first text-based draft also functions as a storyboard for the client and SME to review before spending budget dollars on coding. Storyboarding is a critical step prior to integrating all elements into a web delivery system. A storyboard represents the layout of all screen items as intended to be viewed by learners, with each board analogous to a single screen. In Web-Based Training: Using Technology to Design Adult Learning Experiences, Margaret Driscoll summarizes the main advantages of storyboarding a course before producing it in a web delivery system: Storyboarding “provides [a] visual of program flow, enables content to be resequenced, and highlights gaps in content or dead-end paths.”5 Thus, it is much easier and more cost-effective to revise a course saved in a word-processing program than to revise a fully coded course complete with graphics, links, and interactive multimedia. The following sections describe both the storyboard template and selected techniques of effective web writing. Storyboard Template Depending on the team and nature of a project, we often create storyboard templates in Microsoft® Word®. The text areas of these templates are sized to match the amount of text space allotted in the actual web interface, which helps course developers appreciate the textual economy


153

demanded by online reading tendencies. This sizing also gives course developers some sense of final draft layout as they write the first draft. A sample page that uses such a template appears below, along with a brief explanation of how the template was created. As Figure 6.5 indicates, our course developers add text boxes that describe to programmers, graphic designers, and client reviewers any graphics or interactions needed in order to reinforce key concepts. Figure 6.5

Creating a Storyboard Template Note: The team member who creates this template must be sure his or her web browser is set up to use document-specified fonts. This ensures that they are seeing the UI as intended. Information or assistance might be needed from the team member who created the UI. 1. Open a new Microsoft Word template. 2. Change the page size to match the size of the user interface. Adjust margins to reflect the size and placement of the text area. 3. Take a screen capture of the user interface.

154


4. Import the screen capture into the header of the Word template. Change picture formatting to “lock anchor” and “float over text.” Size the picture to the same size as the page size and align with paper edges. 5. Adjust formatting styles for fonts, bullets, headings, and spacing to emulate the behavior of HTML formatting as it appears on the UI. 6. Test how accurately the template represents the UI by cutting and pasting sample text from the interface into the template to see if it looks the same. Make adjustments accordingly. The concept for this template was developed by Chris Paakkonen, who manages many of our web-based projects. He explains that “there are limitations to this template if you don’t plan to have a consistently sized text area in the actual user interface.”6 Regardless of whether you use a template, however, we strongly recommend standardizing the text area of each page to minimize eye strain. Mary Lehti, a TrainingLinks web course developer with an extensive background in reader behavior, suggests that short lines of text are easier to read on-screen. “Because long lines of text require sweeping eye movements, readers are more likely to lose their place and experience eye fatigue. Since either outcome creates hardship for learners, it is better to use short lines of text that span only a percentage of screen or window width.”7 Lehti points out that placing “next” or “forward” buttons at the bottom of a page can also reduce reading fatigue: “Learners begin their eye movements at the paragraph heading and work their way down from left to right until they reach the end of the text. If the navigational device is placed at the top of the screen, the learner will have to make an extra eye movement to return to the top.”8 Effective Web Writing Earlier, we described the storyboard template as supporting the “textual economy demanded by online reading tendencies.” Many studies on web writing offer two important conclusions: 1. The style and layout of writing on a web page greatly affect the page’s usability. 2. Readers read differently on the Web compared to other media. Brien Thompson, one of our experienced project managers, recently confronted these realities. Brien works primarily with paper-based media for classroom delivery and self-paced workbooks. The first time he was faced with writing for a web delivery system, he made this observation:


155

I kept catching myself trying to write for the reader of a self-paced workbook. I can assume with confidence how that person will read it, probably from beginning to end. If they decide to skip around, it’s no problem, because they can just go back to a given page or chapter, even the table of contents. The rules of reading paper-based media are just assumed, but this isn’t true of readers on the Web, who have a harder time gauging their location in the overall space.9 Regardless of the media—newspaper, radio, television, lecture—we are accustomed to text being mediated by one of two factors, and often both: physical attributes or a human facilitator. Indeed, simply picking up a book gives us some idea of its total length. Page numbers consistently inform us of where we are, as do several other conventions. In terms of human facilitators, we usually hear rather than see scripts. These facilitators provide a consistent thread throughout the text by tying together and summarizing main points, adding emphasis when appropriate. On the Web, however, students generally cannot rely on physicality or human facilitators. A typical difficulty is a sense of disorientation: How did I get to this information, and why is it relevant? Who wrote it, and how long is it? A familiar analog to this difficulty is the myth of the “paperless office” that digital communication was supposed to make possible. But despite the advantages of digital communication, many of us continue to print our electronic documents simply for the security of holding those sheets of paper in our hands: printed documents become objects we can physically manipulate. The amount of text on a single page can disorient or simply tire readers, especially if the page is rich in links or requires the user to scroll. Numerous end user studies, most performed by noted “usability guru” Jakob Nielsen, indicate that people generally have difficulty reading a text-heavy computer screen. This presents a dilemma for developers of WBT, because participants must often read all of the text to complete activities and to pass evaluations. Because of this dilemma, writers of WBT should observe best practices for good web writing. The web design community appears to have reached general consensus on these writing guidelines: ♦ ♦ ♦ ♦ ♦ ♦

Write short, concise paragraphs. Begin each paragraph with a sentence that encapsulates its main point. Limit paragraphs to one main idea. Make use of bullets and bold face type to highlight key points. Treat each page as a self-contained unit focused on a single purpose. Provide orienting cues such as titles and headings.

156 ♦ ♦ ♦ ♦


Be specific. Organize points logically. Follow the rules of grammar and mechanics. Avoid “sales speak” and clichés to make your point on the Web. (These things often impede readers’ efforts to find the information they really need.)10

In many ways, there is little difference between this set of guidelines and those offered in numerous academic People tend to scan the writing handbooks and style guides that cover the “inverted on web pages. pyramid” or newspaper style. But while the principles are similar, implementing them in a web environment can be surprisingly challenging. Our course designers have strong writing skills. Some are former editors, journalists, or English instructors who publish their works in journals and newspapers. Yet we’ve found that writing for the Web demands a keen awareness of how people read on-screen: they scan. While we hope students read each word of a web-based course, we know they won’t. We can, however, increase the number of words they read by making text easier to read in the web medium. On-screen reading tendencies must be factored into the writing of web courseware. Effective web writing requires a particularly terse style that appears in an easy-to follow format. The examples in Table 6.1 illustrate how we define this style. The text on the left might receive good marks in a college English class for its analysis and use of detail, but not in a web environment. It demands a high level of concentration from readers. Their eyes will tire quickly in front of the screen, especially if they have several similar pages to read. It’s easy to think that more information is better, but remember the old instructional design question: “What’s the least they need to know to be successful?” Certainly, the answer depends on the type of course in question and the specific training purpose, but the old cliché of “less is more” generally applies. As a result of his usability studies, Jakob Nielsen recommends that a web page should contain approximately 50% less text than we might expect on a paper page. 11 We are not suggesting that course content simply be sliced in half, but only that it should be as concise as possible and be spread over a number of pages, rather than being crammed onto a few pages. If learners have to sort through several dense blocks of text, they will eventually (and rather quickly) lose focus on main points and lack optimal mental resources for performing well on the course exam. Instructional designers cannot control how carefully people read, but they can design for a high degree of “scannability.”


157

Table 6.1 Paper-based Version

Web-Based Version

Unfortunately, toy manufacturers seem to place higher importance on the marketing of toys than on the design and production of toys that have educational value. In Buy Me!, Joanne Oppenheim reflects that before 1985, toy makers used to pay the owners of characters for the right to produce toys resembling that character. Today, toy companies create their own characters for market. When creating a character, companies also create an entire line, which includes a supporting cast of characters; accessories to accompany the characters; and a story that can be translated into cartoons, movies, and comic books. After such a line has been developed, other manufacturers pay toy companies for the right to produce an array of non-toy merchandise: bedding, clothing, toothbrushes, books, party goods, dishes, and lunch boxes. This entire process is called “power marketing.” Power marketing attacks children several times a day, making it almost impossible for concerned consumers to play an active part in deciding what types of toys their children will have, or to create a balance of boys’ and girls’ toys. Power marketing translates into big profits for toy companies; in 1986, the toy industry’s profit was over 12 billion dollars, as opposed to the profit of 1980—7 billion dollars.12

“Power marketing” increases toy sales by billions, without concern for educational value. It involves creating product lines to promote toys. Such lines include: • Cartoons/movies • Comic books Other companies support power marketing by producing associated non-toy items: • Lunch boxes • Clothing • Party goods This saturation can make it difficult for parents to become discriminating toy consumers.12


158

How long does it take to read each writing sample presented in Table 6.1? How long does it take to grasp the main points in each sample? How much is lost in the web version? This last question is particularly important for WBT. It is possible to edit content down to such a skeletal state that it cannot “stand up,” becoming cryptic and more difficult for readers to understand. Such occurrences are rare, however. Writers care about their words and hate to see them go, even if it’s for the best. The team’s editor must certainly have expertise—not just “familiarity”—in the principles of good web writing. A project will also progress more smoothly if course developers have a strong sense of good web writing; if not, the editor can become overwhelmed and create a bottleneck. We teach our teams to approach course content with an outline mentality. That is, we encourage course developers to create a content outline, and only then to develop with details and transitions wherever necessary. This skill takes practice and some “reprogramming” to alter a writer’s traditional orientation to writing for paper-based media. Using storyboard templates like those described earlier facilitates “reprogramming.” Cues for Orientation and Emphasis Although actual templates can differ from team to team, we instruct web writers to format each page as they would expect to see it in the final web environment. This generally helps prevent the need for learners to scroll, which can become another disorienting factor. Sometimes, readers are not even aware that scrolling is required to access additional content. A final but important note on methods of orienting readers: Each page should be viewed as an island. We want it to connect with its surrounding pages, but readers need reassurance of where they are within the grander scheme of a course, particularly if it is nonlinear. It is important to provide cues within the text in the form of headings and subheadings. A three-step process might appear as such: Table 6.2 Page One:

Page Two:

Page Three:

Three Steps to Changing a Light Bulb



Step 1 Eliminate the power source.

Step 2 Unscrew old light bulb.

Step 3 Screw in new light bulb.


159

These headings provide readers with an overview of the information, and then help them locate each step relative to the overall process. We use similar techniques for quizzes, tests, and overall course location by indicating how far the student has progressed through a course and then indicating approximately how much material has yet to be completed. Three techniques appear in Figure 6.6. Figure 6.6

(Figure 6.6 continued on page 160.)

160



161

Not surprisingly, overuse of subheadings and similar cues can become distracting, so evaluate what information readers need for grasping the function of a particular page and its relation to other pages. If that relation is undoubtedly clear or is not process-oriented, subheadings might not be necessary. In short, keep headers in a standard, unobtrusive place so they can be easily referenced or taken for granted. Use of bold can also distract readers if it is misapplied. If a WBT project requires several writers or course developers, we ground them in a “theory” of “bolding.” We do not want to encourage our learners to skim, because they will tend to do so naturally. Rather, we simply want to direct their attention to the main idea. In the comparison below, bolding is misused in the sample on the left, where words and phrases are highlighted as a college student might highlight passages in a textbook. This approach could actually undermine the learning process, because it discourages readers from distilling or processing the key points for themselves. Table 6.3 Not surprisingly, overuse of Not surprisingly, overuse of subheadings and similar cues can subheadings and similar cues can become distracting, so evaluate become distracting, so evaluate what what information readers need for information readers need for grasping grasping the function of a the function of a particular page and particular page and its relation to its relation to other pages. If that other pages. If that relation is relation is undoubtedly clear or is not undoubtedly clear or is not process- process-oriented, subheadings might be unnecessary. oriented, subheadings might be unnecessary. Once the first draft has been written using the storyboard template and has been revised by the team editor, then the project manager, SME, and client can view it as it is intended to be seen by students. Although still paperbased, this review should expose instructional issues, incongruent aspects of the thematic system, and large usability problems. Revisions can be made easily in a word processing program; it’s generally advantageous to begin coding, scripting, and creating graphics in earnest only after the client approves the final draft.

162


Step 3: Integrate UI Elements

At this point in the process (and depending on the tools used), our course developers typically turn their final drafts over to programmers and graphic artists. The programmers code the course and paste text into the code. (Course developers can also cut and paste into existing code, but we don’t recommend this approach if the project is on a tight schedule or if the course developer lacks basic HTML experience.) Coding and development of graphics can occur simultaneously, and in some cases, the coder might also be responsible for creating the graphics. We recommend coding the first two modules of a course, then running preliminary tests to assess usability issues that might affect subsequent modules. (The next chapter focuses exclusively on different purposes and methods of testing web-based courseware.) Before placing the course online for the client team to review, the entire vendor team should review its own work, evaluating not only discreet elements, but how those elements work together as a unified presentation. All links should be tested, and the course should be viewed on systems that are representative of those the target audience will use.

Conclusion To the team, the interface is only one component of the process and product of WBT. Team members have seen each piece developed in relative isolation and have their own preferences for how those pieces should be combined. To students, however, the user interface is the training; they depend on it for guidance, content, reinforcement, and general support of the learning process. When students feel comfortable and productive in a webbased course, the interface is transparent, posing no barriers between themselves and the training. Conversely, students who are frustrated by an ineffective interface will often be unable to distinguish among the UI, the instructional design, and the content. Because UI design is so critical to effective web-based courseware, we strongly recommend inviting representative target learners, as well as the


163

client and SME, to initial design meetings. Once an interface has been agreed upon, ensure ample opportunities for quality checks during course development by following a phased production process: create the UI, write the course in storyboard fashion, then create graphics and code the course. The following reference sheet highlights many of the UI principles interspersed throughout this chapter.

164


Reference Sheet: General UI Guidelines Issues of graphic design and navigation: ♦ Keep it simple. ♦ Be consistent. ♦ Design for the target learner and no one else. ♦ Be conscious of the WBT interface as a communication system. ♦ Simulate a communication agreement with learners by basing the UI system on their characteristics and needs. ♦ Appeal to learners’ unconscious reliance on inductive reasoning. ♦ Make it easy to learn. ♦ Provide buttons and links for all of the necessary functions of the course, including “forward” and “back” buttons. ♦ Use and place all navigational devices (such as “Next” and “Back” buttons) consistently, so users don’t have to relearn the logic of each page. For consistency, gray out (rather than removing) any buttons that are unnecessary for a given page. ♦ Icons used for navigation should convey clearly that (1) they are meant to be clicked, and (2) what will happen if clicked. ♦ Avoid introducing elements that significantly delay the learning of patterns or that disrupt patterns. Classic examples: adding numerous access and survey forms to the beginning of a course, requiring learners to download several plug-ins, making them surf the Web to find the plugins, or asking them to load a plug-in after the course has begun. Inconsistent or mixed metaphors are also disruptive. Issues of writing: ♦ Write concise paragraphs. ♦ Begin each paragraph with a sentence that encapsulates its main point. ♦ Limit paragraphs to one main idea. ♦ Make use of bullets and bold face type to highlight key points. ♦ Treat each page as a self-contained unit with a single purpose. ♦ Provide orienting cues, such as titles and headings. ♦ Be specific. ♦ Organize points logically. ♦ Follow the rules of grammar and mechanics.


165

References 1.

Mullet, Kevin and Darrell Sano. Designing Visual Interfaces: Communication Oriented Techniques. (Englewood Cliffs, NJ: SunSoft Press/Prentice Hall, 1995), 36.

2.

Ibid., ix.

3.

Payne, Jonathan, interview by the author 10 April 1999.

4.

Weinman, Lynda and Bruce Heavin. Coloring Web Graphics: The Definitive Resource for Color on the Web. (Indianapolis, IN: New Riders, 1996), 5.

5.

Driscoll, Margaret. Web-Based Training: Using Technology to Design Adult Learning Experiences. (San Francisco: Jossey-Bass Pfeiffer, 1998), 201.

6.

Paakkonen, Chris, interview by the author 5 September 1999.

7.

Lehti, Mary, interview by the author 29 August 1999.

8.

Ibid.

9.

Thompson, Brien, interview by the author 14 April 1999.

10. Nielsen, Jakob. “How Users Read on the Web,” 1, 3. Cited 13 March 1999 from http://www.useit.com/alertbox/9710a.html; INTERNET. 11. Ibid. 12. Oppenheim, Joanne. Buy Me! Buy Me! The Bank Street Guide to Choosing Toys for Children. (New York: Pantheon, 1987), 7-8, 12.


Chapter Seven

Courseware Testing Chapter topics: ♦ Reviewing before release ♦ Pretesting considerations • Budget • Purpose ♦ Criteria tests • Criteria test methodology ♦ Alpha tests ♦ Beta usability tests • Beta usability test methodology Chapter resources: ♦ Criteria test standards

Reviewing before Release The last chapter concluded with a prototype review in which two course modules were coded, tested by the team, and approved by the client. After the team makes revisions based on client feedback and then codes the entire course, the training is almost ready for delivery. To be effective, however, WBT, like all forms of training, must undergo pilot reviews before it is considered final and made available to learners on a regular basis. We refer to this phase in our training development cycle as “Test,” which in our model equates with formative evaluation. As the term implies, formative evaluation is conducted while courseware is still in a formative stage, in order to validate it effectiveness. 167

168


During this phase, the team works with clients and students to confirm that the course addresses the client’s original business problem, fulfills the client’s expectations and training goals, and serves learners’ needs. This chapter focuses on three stages of testing web-based courseware—criteria testing, alpha testing, and beta usability testing. During a criteria test, a person skilled in WBT design evaluates the course based on a predetermined set of standards. Following the criteria test, an alpha test gives clients an opportunity to review an entire course online and to request any final revisions. Beta usability testing is the final stage of testing web-based courseware. During beta usability testing, team members observe target learners while these participants take the training; team members note difficulties encountered by the test participants and revise the course accordingly. As we describe techniques for each method, we also introduce common challenges and solutions. Although methods might vary slightly depending on the delivery system and reason for testing, all courseware testing focuses primarily on questions specific to training: ♦ ♦ ♦ ♦ ♦ ♦ ♦

Do the objectives meet learner and training goals? Do they address the identified skill gaps? Does course content adequately cover the training objectives? Have any unrelated content pieces been included in the course? Can learners relate the training to prior experience as well as to their job tasks? Do practice activities truly help learners meet the objectives? Do the activities appeal to different learning styles? Is the sequencing of modules and learning units effective? How well does the course prepare learners for the course exam? When facilitation is involved, how should instructors prepare to teach the course effectively? Are any specific techniques or materials required?

These questions are critical in reviewing WBT, even the last one regarding instructors. If a web-based course incorporates interaction through email, chat, phone conferencing, or bulletin boards, instructors or SMEs will require some form of training to effectively manage those activities. Supervisors also need some preparation for reinforcing the training on the job. But because students tend to view the user interface as the training, the testing of web-based courseware must also incorporate techniques common to general web design. It can be tempting to skimp on these techniques. Near

Courseware Testing

169

the end of a project, it’s not uncommon for the budget to be tighter than expected. Deadlines are looming. There’s also a misconception that testing any form of computer-based training must take place in a usability lab equipped with observation booths, video cameras, two-way mirrors, microphones, and so forth. Usability labs are an excellent way to test WBT, but meaningful testing can still be accomplished outside of a lab environment. This chapter is designed for readers who do not enjoy the luxury of such a lab. The next section covers basic considerations that should be made before initiating formal WBT courseware testing.

Pretesting Considerations Budget The cost of a WBT review depends on the method used, the people involved, and the extent of data analysis, including the work required to revise the courseware. In the long term, course review is usually well worth the price, but the financial value of courseware testing is not easy to quantify, because it is a preventative measure. (Releasing mistake-ridden WBT programs can discourage students and negatively affect the “bottom line” when skills are not successfully transferred into the workplace.) To maximize an investment in formal testing, the team should test their work early and often throughout development, detecting and correcting major errors before formal testing begins. Ideally, criteria and alpha and beta tests should primarily expose areas that require only fine-tuning. As an instructional design and development vendor, we realize that budgets are not often negotiable once a project is underway, so it is important to build in time and money for adequate courseware review in the initial proposal or business case. Admittedly, “adequate” is a relative term that must be tempered with realism. The cost of courseware review depends on several variables; indeed, these expenses vary among our own WBT projects. Unique factors such as these must be considered: ♦ ♦ ♦

Complexity and length of the target courseware The number of testing sessions needed, based on total course length The number and type of test participants, such as the team, learner representatives, SMEs, client representatives, and in the case of a training management system, instructors or managers from the learner’s workplace

170 ♦ ♦ ♦


The method of testing and associated facilities and materials The amount of data generated during the review Time needed to analyze review data and revise (Before revising a course, all possible changes must be evaluated for their congruence with the course’s instructional design.)

For illustration, we’ll walk through an example with multiple variables. In this particular example provided, courseware testing comprised a large percentage of the total budget—almost 20 percent. This expense includes the following items for the beta usability test: Participants ♦ Five learners from the target audience ♦ Three team members to conduct the test (team members skilled in instructional and web design) Method • Usability tests in which team members observed target learners taking the course, noting points of confusion and areas of difficulty within the training, followed by debrief (The team member-to-test participant ratio was 1:1. Five total testing sessions were conducted, with each team member observing one participant per session; two of the team members conducted two separate sessions apiece.) Facilities and Equipment ♦ Target learners’ work site ♦ Hosting fee ♦ Paper and pens to record observations during usability test Data Analysis and Implementation ♦ Team members summarized the results, distinguishing between critical and noncritical issues. ♦ Changes marked “critical” were implemented first, while noncritical changes were implemented based on priority and available resources. Additional Expenses ♦ Time out of the field and estimated loss of sales ♦ Travel, lodging, and pay for team members

Courseware Testing

171

Although total costs for review can vary widely, costs undoubtedly rise as more people participate in the review process. As such, the project manager should be able to justify a need for those involved by clearly defining their roles. If the skills and number of people involved are well balanced with the complexity and length of the course, review costs should be offset by product quality and learners’ improved job performance, and as a result, return on investment. To maximize the value of courseware testing, it is also important to set standards before the review, hence the earlier distinction between “critical results” and “noncritical results.” When prioritizing possible revisions, the team should keep three questions in their minds: “Does this problem affect a student’s effort to learn? Will it create a poor opinion of the course in the student’s mind, and therefore generate resistance to learning? Is the revision congruent with the training goal and client expectations?” Purpose When examining the budget allocated for courseware testing, we suggest using the purpose of the review as a solid starting point to ensure that testing captures the desired data. Making a determination based solely on budget constraints can lead a team to settle for an inappropriate method by default. Our overall process of selecting a testing strategy is based on the following questions: ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦

What is our purpose in testing right now? Why is it necessary? What kind of data is needed? What types of data do we expect? Who can generate the type of data we need? How can we capture the data we need? For a given method, how many participants are needed, and what background, experience, or user group should they represent? What is their availability? For what time frame will the participants be needed? Will the participants require any assistance from or interaction with team members? Who will be responsible for compiling review data? How will we distinguish between critical and noncritical review results? What is the best review method based on the above answers?

172


After identifying the most appropriate review method, the team should examine its budget and consider modifications accordingly, either to the budget or to the preferred testing strategy. The rest of this chapter walks through different techniques, considerations, and suggested participants for the three stages of WBT courseware testing.

Criteria Tests Criteria testing is the first phase of formal WBT courseware testing. During criteria tests, a person skilled in WBT design assesses usability based on a predetermined set of standards. We refer to this person as an “evaluator.” In other words, the evaluator “grades” the courseware based on a set of instructional and web design criteria, identifying problem points and areas for improvement. This testing is an important form of vendor-driven quality control that is performed before a client or learner views an entire course online. Because it is performed by the vendor organization, the vendor must select a qualified person to conduct testing, someone skilled in both web and instructional design. We strongly recommend identifying a candidate outside of the project team; this person should be able to examine the training with a fresh set of eyes and without preconceived notions of how the course is supposed to look or be navigated. Our teams typically select evaluators from inside our office who are members of other WBT project teams. Our standard criteria list covers eight broad areas: 1. Course introduction 2. Activities and interactivity 3. Indicators of progress within the course in terms of location, material completed, and modularity 4. Writing 5. Learning tools (glossary, PDFs, etc.) 6. Visual appeal of interface 7. Method of concluding the course 8. Technology-related issues These areas and the standards they contain can be modified on a courseby-course basis as needed. The complete list of criteria appears at the end of this chapter. (Many teams find it a useful guide not only for testing a coded course, but for generating design and development standards early in a project.) Because these criteria encompass several issues, we generally enlist

Courseware Testing

173

three to four people to evaluate a given course, which provides the project team with richer, more reliable data. The rest of this section walks through a method of setting up, conducting, and concluding a criteria test. Criteria Test Methodology Test Setup Before testing begins, the project manager should hold a meeting with the evaluators to distribute and discuss the testing criteria. Several points should be established: ♦ ♦

How evaluators will access the course System modifications to test the course on systems representative of the target audience ♦ How evaluators will record their responses (Do they prefer an assistant to take notes? Do they prefer to write their own comments or to use another method?) ♦ A brief background on the client or target audience (Familiarity with characteristics of the target audience allows evaluators to assess the congruence between learner needs and actual courseware design.) Conducting the Criteria Test In “How to Conduct a Heuristic Evaluation,” Jakob Nielsen suggests that tests of this nature must be conducted with the evaluators in isolation. This arrangement helps them maintain objectivity, and therefore generate more reliable comments.1 One challenge we have encountered with criteria testing is finding an effective method of gathering evaluator comments into a wellorganized index. As a solution, we suggest using a comment function that can be integrated into a course for testing purposes (and only for tests that are not intended to simulate the actual training experience). This function appears to evaluators as a button on the navigation bar. Whenever evaluators encounter a problem or point of confusion in the course, they can click the comment button and type their observations. They can also cut and paste course content into the window and make text edits as appropriate. The results are instantly recorded in a database and can be sorted and grouped for analysis in a number of ways: by error type, by reviewer, by course module, or by date or course version. This feature is illustrated in Figure 7.1.

174


Figure 7.1

The comment feature does have certain limitations, however. We encourage evaluators to be as specific as possible, but if they do not type out their responses with adequate detail, solid data can be lost. As a result, we set up this feature to indicate the evaluator’s name on each print-out, which allows us to seek further clarification if needed. Depending on the number of comments a reviewer makes, this test can increase total anticipated course time by 50 to 75 percent. Concluding the Test As Jakob Nielsen suggests, we follow each criteria test with a debrief that involves the project team and all of the evaluators. The meeting should focus on overall impressions and possible solutions to common usability problems identified during the test.2 After the debrief, the project team synthesizes and prioritizes possible revisions. The team then revises the course in preparation for alpha testing.

Alpha Tests Prior to the alpha test, the client has reviewed the course several times, both on paper copy and as a partially coded prototype. An alpha test is an opportunity for the vendor project manager to walk through a fully coded

Courseware Testing

175

course with the client team, ideally including the client project manager, business sponsor, SMEs, and a knowledgeable target audience member. Including these participants should ensure that all course components are synthesized to fit the client’s training goals, vision, and work culture. As a prelude to beta usability testing, an alpha test also allows the client to request any final changes before a small sample of the learning population takes the course. The methodology of an alpha test can be negotiated with a client, although we prefer to meet with the client in person for the actual test. During the alpha, encourage a dialogue focusing on the client’s questions, concerns, and perceptions. Once the team makes final revisions based on this dialogue, offer the client one more opportunity to review the final version prior to beta usability testing.

Beta Usability Tests When all revisions resulting from the alpha are complete, a web-based course then enters the final phase of the testing process—the beta usability test. A beta test should offer a strong indication of how effective the course will be on the target audience, and how well the course will address gaps in skills and knowledge. These pilots often include anywhere from five to twenty student participants, but this number partially depends on the nature of the training, the delivery system, and the size of the total target audience. A course that relies heavily on peer activities and discussion often cannot be tested adequately with fewer than ten to fifteen students. In contrast, testing a self-paced course with five to ten participants is generally sufficient to generate a wide range of comments, as well as to identify patterns in the test data. Beta usability testing hinges on observing members from the intended audience actually taking the WBT on end user systems that are representative of those the target learning population will use. (Considerations include connection speed, monitor size, operating system, browser type, etc.) Usability testing can involve high-tech labs, but a lab is not a necessity. The methodology used in these labs depends on lab capabilities and attending usability engineers, so we won’t be covering lab techniques here. Instead, we want to focus on how to conduct a usability test outside of the lab environment. A bare-bones approach can be highly effective. Ideally, an instructional designer and web programmer should administer the test, with both

176


simultaneously observing one target audience member while he or she takes the web course. The instructional designer and programmer note when and where the learner has difficulty with the instruction or the interface, as well as how the learner contends with those difficulties. While the learner should not rely on the test administrators for a course walk-through, the administrators should offer brief guidance when asked or when the learner is unable to proceed. If project constraints allow for only one test administrator, we strongly recommend that the instructional designer receive some training in usability issues and then conduct the beta. With prior training, the team’s instructional designer will be able to detect inadequacies in course design and basic usability. The following section describes how to set up, conduct, and conclude a beta usability test. Beta Usability Test Methodology Test Setup: ♦ Administer the test at a site that is free of distractions and that provides appropriate computer access. (Betas are typically performed in environments that simulate the actual training environment, such as a classroom. However, holding a WBT beta in some work environments will distract participants and other employees who are not test participants. Additionally, the beta should primarily indicate how effectively the training addresses target audience skill gaps; depending on the nature of interruptions in the environment, this data can be hard to gather when outside variables enter a beta test.) ♦ When possible, maintain a two-to-one ratio between the administrators (instructional and web designer) and learner for every test performed. ♦ Explain the specific purpose of the test and which components will be observed (instructional and content soundness, usability issues such as navigation, etc.). ♦ Emphasize that the participant is not being tested, but that the course is. ♦ Tell the participant that there is no time pressure or requirement for completing the test. ♦ Make sure that the participant knows exactly what tasks will be included throughout the test (typing, clicking, downloading, etc.). ♦ If the test is to be professionally conducted, consider asking participants to sign a statement acknowledging consent.

Courseware Testing

177

Conducting the Test ♦ Help the participant access the course, if necessary. ♦ Encourage participants to think aloud, allowing administrators to write down pertinent, spontaneous comments. ♦ If possible, videotape the participant. A visual record provides excellent data if there is ever user failure on a system. ♦ Always respect the participants; do not ever make them feel that they are slow or not performing well. ♦ Don’t talk unless prompted. ♦ Record the nature of any difficulties with navigation, course content, and instructional design. Note where these problems arise, how participants deal with them, and the outcome. ♦ Note movements of the mouse pointer: • Which items the person moves the mouse to • Which items they try to click • How long the mouse is positioned over different screen items • How long it takes the learner to become familiar with consistent features of the interface, such as navigation ♦ Look for behaviors that suggest difficulties with course content and instructional design: • How long does it take participants to interpret instructions for activities or interactivity? • Are there any patterns in the types of questions participants answer incorrectly? • Which topics do participants attempt to review? Under what circumstances do they initiate a review? What is the nature of the review (i.e., rereading paragraphs, repeatedly consulting a glossary for the same terms, paging back to previous screens, etc.)? • What is the approximate time it takes for the participant to complete each learning unit and module? • Does the participant skip any paragraphs or activities? Concluding the Test ♦ Debrief usability testing with interviews. While general questions might yield valuable data, a structured approach will generate the breadth and depth of data needed. Prepare any interview questions before administering a usability test. The list of standards for criteria


178

testing can be used as a basis. (The complete list appears at the end of this chapter.) ♦ Synthesize the results of each test, and then discuss with the team which revisions are priorities and how best to implement them.

Conclusion Once an entire web-based course is coded, there are several ways of testing it. Skipping or skimming the test phase of the training development cycle can ultimately undermine all components of corporate training—from students and productivity to ROI. We have found that three stages of WBT courseware testing maximize this significant training investment in the long term. As an internal control measure, criteria testing allows a team to detect and correct obvious instructional and interface problems before clients and learners have a chance to notice them. Next, an alpha test provides an opportunity to assess whether a course meets the client’s vision, whereas the beta test involves online delivery to a small sample of the target learning population. Each test relies on a distinct group of perceptions and skills, and thus, each test is necessary for refining web-based training. Because different testing strategies yield different types of data, it’s important to consider in advance the type of input needed to improve a course’s effectiveness for target learners. For example, some content issues will arise during usability testing, but other methods provide more directed content assessment. We suggest performing all three tests (criteria, alpha, beta) to generate a wide range of results and to validate those results. A WBT team also needs to plan beforehand how to manage the data, how to distribute it to appropriate team members, and how to coordinate tasks between course developers, graphic artists, and programmers. WBT consists of several elements that must balance within an instructionally sound framework. Sometimes, changing one element will alter the effectiveness of other elements. Revising one paragraph, for instance, might also necessitate changing a graphic or the graphic’s caption. Different testing methods can also influence users’ perceptions, and therefore results. This is unavoidable, but still something a team must be aware of when interpreting results. Once all beta revisions have been made to the course, it should be ready for delivery to the target audience. For smooth delivery, the project team or client should have a course maintenance plan, including a method of assessing course effectiveness on a continuing basis. The next chapter explores key components of a WBT maintenance plan.

Courseware Testing

179

Criteria Test Standards Description These standards can be used as a guide both for the development and testing of web-based courseware. Avoid limiting your answers to yes/no responses. Comment on the effectiveness of applicable elements within each category, as well as overall category effectiveness. Category 1: Course Introduction Plug-ins ♦ Are learners informed of any needed plug-ins before they can access course content? ♦ Are clear instructions offered for how to download the plug-ins? Entrance ♦ Is the entrance standard for all learners? ♦

Navigation overview

♦

Instructional overview

♦

Do all students receive the same course overview before they can take their own paths through the content?

Is there an indication of how the course is or can be navigated? ♦ Is there a clear explanation for how learners can leave and return to the course?

♦ ♦

♦ ♦

Does the overview indicate a course goal or course value to students? (For example, does it fulfill a prerequisite for another course?) Are terminal objectives listed, and are they measurable over the Web? Are students informed of how their performance will be evaluated and who will receive the results, such as a supervisor? Are all supporting course components explained, such as the need to email assignments to an instructor? Does the introduction include approximate course length?

180


Category 2: Activity/Interactivity Instructions ♦ Are activities and interactions introduced with clear and instructions? structure ♦ Is the purpose or value indicated for each activity or interaction? ♦ Is the information needed to complete each activity readily available, saving participants from backtracking to find related materials such as tables and charts? Frequency ♦ Does an interaction or activity occur at least every three and screens or five to seven minutes? variation ♦ Do the types of activities and interactivity vary at least every third interaction? Objectives ♦ Does each activity and interaction clearly connect with course objectives? Feedback ♦ Are student efforts rewarded with instructive feedback that indicates: How the student answered? The correct answer? Why a selected answer was incorrect?

Category 3: Progress Location ♦ Are strategies used to keep students informed of where they are and what they should be doing? ♦ Are students offered a method of checking the amount of course material they have completed? Modularity ♦ Is an approximate time or length presented at the beginning of each module? ♦ Do modules include an effective number of learning units? ♦ Are learning units delivered in small segments?

Courseware Testing

181

Category 4: Writing Style ♦ Is the writing concise? ♦ Does each page focus on one purpose? ♦ Does each paragraph begin with a controlling idea? Formatting ♦ Is scrolling either absent or minimal? ♦ Are text columns narrow enough to save students from making sweeping eye movements? Level of ♦ Are all sentences necessary and relevant to the course detail objectives?

Category 5: Learning Tools (glossary, PDFs, etc.) Placement ♦ Are the tools placed consistently? ♦ Are the tools available when needed? Utility ♦ Do the tools function as expected? ♦ Do they support learning needs in light of the associated course content? ♦ Do any tools seem to be missing? Labeling ♦ Are all tools labeled to clearly indicate their intended use? ♦ Is the labeling style consistent throughout the course, from a writing perspective? (For example, is it possible for all labels to use either verbs or nouns? Are they all at the same general level of diction?)

Category 6: Visual Appeal of Interface Instruction ♦ Do all graphics add instructional value and do they all support the associated text? ♦ Do any graphics require labels or captions? Do existing captions relate well to both course content and the graphic? Layout and ♦ Are there any instances of screen clutter? continuity ♦ Do all screen elements work together to convey an overall idea, impression, or theme? ♦ Do all graphics share a similar stylistic flavor?

182


Category 7: Course Conclusion Course ♦ Does the evaluation take place before the final exam evaluation results are returned to the student, reasonably assuring that the student will complete the evaluation? ♦ Does all the information that students need to complete the evaluation appear on the same page as the evaluation, saving them from clicking between pages? ♦ Are students assured of confidentiality or anonymity? ♦ Are general fields already completed for the learner, such as the course title, number, and date? Reinforcement ♦ Should the course conclude with supporting materials material students can print or save? ♦ Does the course summarize any next steps regarding the application of new skills and knowledge in the workplace?

Category 8: Technology Multimedia ♦ If the course includes multimedia, is it used effectively, either to reinforce training concepts or to stimulate interest in learning? Errors ♦ Does the course contain any scripting errors or broken links? ♦ If errors are present, have students been given clear, immediate instructions so they can work through the problem? Processing ♦ Were any course components noticeably slow to speed process or download, such as graphics or feedback?

Courseware Testing

References 1.

Nielsen, Jakob. “How to Conduct a Heuristic Evaluation,” 2. Cited 26 April 1999 from www.useit.com/papers/heuristic/heuristic_evaluation.html; INTERNET.

2.

Ibid., 4.

183


Chapter Eight

Course Evaluation and Revision Maintenance Chapter topics: ♦ Evaluation planning ♦ Implementing Kirkpatrick’s model online • Level 1: Reactions • Level 2: Learning • Level 3: Behavior ♦ Maintaining course effectiveness • Revision maintenance team • Documenting revision history • Implementing revisions

Evaluation Planning Like all forms of training, the continued effectiveness of web-based courseware must be maintained through periodic evaluation and revision. The most critical dimensions of evaluation should be established prior to course development and delivery by defining: ♦ ♦ ♦ ♦

The aspects of the course to be evaluated Why those aspects in particular will be evaluated How the appropriate data will be gathered How that data will be used or acted upon 185

186


The methods of evaluating the effectiveness of WBT differ little from those used to evaluate other forms of courseware, although the web infrastructure can be designed to automate numerous evaluation procedures. Many off-the-shelf tools are also available for test and survey development, data analysis, and report generation. However, this automation is only effective when it is based upon solid planning and is viewed as one component of an entire evaluative process. This chapter covers basic considerations for using Internet technology in summative evaluation, which takes place after a course has passed through alpha and beta testing and has been delivered to the target learning population. In Designing Effective Instruction, Jerrold E. Kemp, Gary R. Morrison, and Steven M. Ross walk through several purposes of summative evaluation, which can serve both administrative and training effectiveness goals. Administrative goals include analyzing training data, such as enrollment rates, to allocate resources for training facilities and staffing.1 Tracking such measures in a web environment is common; even low-end server logs automatically track the activity on a given web site, including the number of “visitors,” the frequency of visits, and the length of each visitor’s stay. In contrast to administrative measures, this chapter focuses on effectiveness measures pertaining to the quality of instruction and its connection with student performance. Ideally, an evaluation specialist should serve as a resource for the WBT team, collaborating with the client, instructional designer, and web programmer to define which effectiveness dimensions will be measured and to determine the best tools for conducting each desired form of evaluation. In reality, however, few project teams enjoy the luxury and benefit of working with an evaluation specialist, and so must undertake evaluation with limited resources. To assist these teams, the following sections introduce basic selected techniques for evaluating the effectiveness of WBT, and then address the logistics of implementing revisions in a web environment.

Implementing Kirkpatrick’s Model Online References to Donald L. Kirkpatrick’s four-level model are interspersed throughout this book, and we will use this model to provide a context for making connections among technology, instruction, Level 1: Reaction evaluation, and performance. Because of its complex Level 2: Learning nature, however, level four is beyond the scope of this Level 3: Behavior discussion. Although arriving at level four might be Level 4: Results viewed in the training industry as analogous to reaching Maslow’s pinnacle of “self-actualization,” Kirkpatrick

Course Evaluation and Revision Maintenance

187

proposes that data gathered at previous levels can be equally vital: A lack of data from levels one through three makes it impossible to identify whether a performance problem stems from the training itself or from workplace dynamics and support.2 Throughout this exploration, the intent of each level is briefly reiterated; we also suggest methods for automating related evaluation techniques via the Web, pointing out considerations along the way. Level 1: Reactions Level one evaluation gauges students’ satisfaction with a course. While student satisfaction is not necessarily a measure of training effectiveness, Kirkpatrick explains the value of this information: “Positive reaction may not ensure learning, but negative reaction almost certainly reduces the possibility of its occurring.”3 Internet technology provides an easy method of gathering reaction data through a fairly common means—course surveys. A WBT survey might pose questions about a course in the following areas: ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦

Appropriate level of detail The organization of course materials Whether lessons are presented in an interesting way The relevance of lessons to the student’s job tasks The clarity and utility of instructions for navigating and completing the course The level of learner control provided (pace, navigation, etc.) and whether that control is appropriate for the course material and student The quality or effectiveness of the user interface (graphics, layout, etc.) Availability of or need for resources within the course, such as a help button or a link to references Ease of exit and return Whether or not the student has engaged in web-based learning prior to the course • To help maintenance team members interpret survey results, it is helpful for them to have some knowledge of a particular learner’s previous experience in web environments. We are not suggesting that a lack of experience invalidates a student’s opinion, but that this information can be used to refine interface usability given a specific target audience.

188


While we will not cover the science of writing effective surveys, it is critical to make distinctions between questions that seek information on the instructional design of a course versus its delivery system. Failing to make this distinction can degrade survey validity. Kemp, Morrison, and Ross explain the crux of validity: “Although validity is typically associated with knowledge tests, it has the same importance for all types of evaluation measures. The key idea is that the test assesses what it is supposed to measure.”4 Because the UI affects the learning process, we believe there is value in gathering student reactions to the interface. Be aware, however, that comments pertaining to the UI do not necessarily expose problems with instructional design. Conversely, negative student comments relating to instructional design might be influenced by an ineffective UI, because it is the lens through which they view the training. Techniques and Considerations The following sections offer techniques for developing and presenting online course evaluations in terms of placement, length, and usability. Placement The reliability of survey results depends on several factors, including the number of people who complete the survey. After completing any kind of course, many students groan when surveys are distributed, whether they enjoyed the course or not. This tendency can cause some WBT students simply to exit the course when the survey appears. To ensure a high response rate, we suggest administering the evaluation after the final exam or activity, but before the exam results are reported to the student. A statement following the exam might read: “Congratulations on completing this course. Please fill out a course evaluation while your final score is computed.” Upon submitting the completed survey, learners are given access to their final scores. Length Many web surfers and students alike tend to find lengthy online surveys rather tedious and discouraging. Keep online evaluation forms succinct, posing questions that will gather the most critical data for your needs. To gather a wide variety of data, some web sites present different evaluation forms to different users, which allows each form to be shortened. This technique should be used with caution when trying to assess student reaction to a course, however. If, for example, one version of the survey omits questions pertaining to the UI, it becomes difficult (if not impossible) to


189

analyze the correlation between an individual student’s comfort level with the interface and his or her achievement. The reliability of survey results can thus be compromised if all learners do not receive the same survey. Usability The design of online course surveys should be held to the same usability standards as any other materials that appear in the WBT. Lisa Schmeiser, author of The Complete Website Upgrade and Maintenance Guide, offers an important usability guideline: “Provide all the information the users need to fill out the form on the same page as the form.”5 It is also important to assure students that their survey responses will remain anonymous. Level 2: Learning This level focuses on assessing student performance during the course based on the stated course objectives; measures of this nature are often referred to as “criterion tests.” Certainly, a web delivery system should be programmed to capture this data. However, capturing only the results of criterion-based tests limits an instructional designer’s ability to improve course effectiveness. Internet technology can help document students’ learning and collect other data that indicates how the course can better facilitate learning. If the system is designed to gather data on non-graded reinforcement activities, an instructional designer can identify needed revisions to better prepare students for achievement on criterion tests. That is, low performance on practice activities can expose inaccuracies in the content provided and myriad other design features, including clarity of the content, organization, and the activities themselves. As well as performance on activities and tests, data on how learners use the learning tools provided in a course can shed light on course effectiveness and student performance. For example: ♦

Are there any patterns in usage of the HELP function? Do several learners seek help in the same place? Which help topics do they click? ♦ If the course provides a REVIEW button, how many students use this resource, and how does their review of the material correlate with exam scores and the achievement of related learning objectives? ♦ Does the course include a glossary of terms? If so, which words do students most frequently look up? Do some students look up the same


190

term repeatedly throughout the course? If so, this could indicate areas of course content that might need to be refined. Assess data on glossary usage with caution, however. Look for unusual patterns. Implementing data tracking of this sophistication requires advanced data mining and programming skills. Level 3: Behavior Level three evaluation generally occurs 60 to 90 days after target learners have completed their training. Evaluation specialists gather data to examine the extent to which employees transfer what they learned in class to how they perform their jobs. After level three data has been collected, it is also typically compared with level two data to assess course effectiveness and retention rates. In An Overview of On-Line Learning, Saul Carliner highlights how digital communication can help automate level three evaluation: “If learners have email capability, the system can automatically send them follow-up surveys, tests, and other materials at a predetermined time.”6 If the course objectives pertain to software-related skills, Carliner points out that these skills can be tracked via software.7 However, Trish Cullum, one of our project managers with a background in evaluation consulting, cautions that follow-up tests and surveys are not always clear indicators of job performance. “It is critical to develop instruments that simulate how skills and knowledge are used and measured on the job. This often requires a designated person to observe the learner at work, assessing the congruence between behaviors and objectives. Most surveys yield results from level one—reaction.”8 Follow-up, web-based modules can gather level three data if those modules achieve “simulation” as Cullum describes. Additionally, it is useful for supervisors to take part in observing actual job performance, but often, Cullum explains, supervisors require support in implementing the evaluation process; emailing observation guidelines to supervisors can be the first step.

Maintaining Course Effectiveness Because asynchronous, non-facilitated WBT is delivered continuously—that is, available at students’ convenience—there are no clear boundaries for when course delivery begins and ends. Consequently, it is important to establish a reasonable maintenance or evaluation schedule by which evaluation results are analyzed and relevant revisions made. In the first


191

month of delivering a web-based course, we recommend close monitoring of server logs and help-desk calls or email to the webmaster, as well as calls to an instructor or SME, if one has been designated as a student contact. At the end of this month, all evaluation results should be analyzed and used to guide revisions, and a regular schedule followed thereafter. Rushing to update a course based on the latest technology or rushing to test new UI designs are generally not good reasons to revise WBT. Such motivations typically have no basis in learner needs and training goals. Although it can be tempting to rationalize new technology as having an instructional purpose if it enables richer interactivity or multimedia, a genuine need must still be established. Consideration must also be given as to whether students’ systems will actually be upgraded to support the advanced technology. In “The Increasing Conservatism of Web Users,” Jakob Nielsen notes (rather sorrowfully) that web users are slow to upgrade because “The user base has moved beyond the net-nerds who are interested in the Internet for its own sake.” 9 Revision Maintenance Team Although a revision-maintenance team might be smaller than an original WBT design and development team, revisions can require that many of the original core roles and tasks be fulfilled. We strongly recommend that an instructional designer either guide or be consulted for all changes made to the course, because that designer must assess whether proposed revisions match business and training goals, audience needs, and the existing instructional design. A web programmer is also required to implement coding changes and to update file directories. Depending on the extent of revisions, the following team members might also be needed: an SME, an evaluation consultant, a graphic artist, an editor, a course developer, a systems administrator, and a project manager. Documenting Revision History Regardless of who is ultimately designated to revise the course, it is critical that the original team document the project’s revision history. Revisions can occur months apart, and during that time, team members might move on and off the project. These factors can make it difficult to monitor revisions; therefore, careful documentation is key to maintaining project continuity. If the original WBT team is not dedicated to updating and maintaining a course, they should establish a sensible starting place for the next team by organizing all directories and files, and then documenting that organization, clearly indicating source files for the current version of the


192

course. Each time the course is revised, all of the files should be placed in a folder and assigned a new revision number. For project continuity and backup purposes, all previous revisions should be preserved. A focused effort should be made to document files and course status, even if some original team members will participate in maintenance. Clear documentation is the key to continuity. In a hand-off situation, a new team or project manager should receive all HTML files, the original text and graphics files, the blueprint, web circuit, and the data associated with needs, task, and instructional analyses. If graphics need to be modified, for example, their quality will be better preserved if the originals are available, versus graphics that have been modified and saved several times over. Conventions should be used for all file names within and between each directory for content, graphics, and code. Files should also be organized using a consistent structure. The web programmer can further assist course hand-off by using standard formatting techniques. This can prevent a new web programmer or webmaster from calling eight months after hand-off and rightfully complaining about messy coding that takes extra time to decipher and work with. Here are some basic guidelines for clear coding: ♦ ♦

Standardize the tags in either lower case or capital letters. Use indentations to give HTML files a landscape with some identifiable features; this makes it easier to follow and to locate where coding changes should be made when updating a course. ♦ Add comments to the code. If a piece of code is unusual or complex, the programmer should give a brief explanation within the code. Additionally, a version number and date should be included in newly revised files. ♦ Document a coding or scripting style sheet to include as a read-me file. These are important techniques. If someone else needs to update the course, he or she should have an easier time getting started. The courseware should also be accompanied by some documentation on the project history. Tracking changes to a web delivery system can be a monumental task; it’s easy to become lost in the interlinking branches of web architecture, even if a course is fairly linear. Just as the code should be standardized, so should the methods of documenting revisions and maintenance. There are several ways to document the maintenance history of a web delivery system. Lisa Schmeiser suggests several possibilities, some of


193

which we have mentioned, such as commenting code. Code comments offer a micro-history; a macro-history is equally important and should provide an overview of significant changes made over time to the files in each directory. This macro-history can be placed in each directory as a read-me file. Schmeiser suggests placing read-me files in all relevant directories, or “fullblown formatted files in a central repository.”10 Regardless of which method works best for your particular project and team, Schmeiser explains that “the only two factors that are mandatory are that all of your developers know where to find the documentation and all of your developers note all relevant information when they make changes.”11 Schmeiser’s good advice raises two important questions for the maintenance of a web delivery system: 1. Who will need to access the maintenance history? 2. In which aspects of the history will they be interested? Web programmers, for instance, will need information that an SME will not need, and vice versa. We suggest creating a file for each group of people assigned to maintain content, programming, server administration, and graphics. Table 8.1 illustrates the kind of information different people on a maintenance team might need, aside from the actual courseware. When transferring maintenance responsibilities to a new team, it is important to walk through the files of each directory, explaining the necessity of maintaining a project history and how to use it.

194


Table 8.1 Project History Directories and Suggested Files Content Directory Programming Server Directory Directory ♦ History of ♦ Standards of file ♦ Description of revision dates structure server-side and significant ♦ Code style guide functionality and changes ♦ Description of the types of data between navigational routes to be transferred revisions or architecture between client and ♦ List of course ♦ Directions for server, such as test objectives integrating the scores and ♦ Summary of course with the web bookmarks major revisions site or training ♦ Zip files of the by module and management system revised and coded revision date ♦ File naming course for disaster ♦ Course exam, conventions recovery with each ♦ Description of question coding methods for mapped to the UI number of an objective ♦ Course evaluation results, categorized by course revision ♦ Content style guide ♦ Text files of course content ♦ Storyboards

Graphics Directory ♦ Source files ♦ Files saved in formats for web presentation (GIFs, JPGs, and PNGs) ♦ Description of coding methods for UI

Implementing Revisions Significant revisions affecting either instructional design or the user interface should be approved by the client and an SME, and possibly undergo beta usability testing with a target audience sample. After a newly revised course is re-approved for continued delivery, it is then reloaded onto a web server or stored in a database. If the maintenance team is not part of the hosting organization, the team should learn the method used to deliver the course to prepare files appropriately. Segments of a single course can be simultaneously hosted on both platforms, depending on the kinds of


195

interactions the course contains. Hosting from a database provides greater tracking capability.

Conclusion The appropriateness of a given evaluation method depends on which dimension of course effectiveness will be measured, and this decision should be based on business as well as training goals. The Web is simply a tool for assisting in these efforts and cannot be relied upon as an evaluation method in itself. It should be viewed in a larger context that includes business managers and trainees’ supervisors who are critical in encouraging skill transfer and in supporting the entire evaluation process. The continued effectiveness of a course is contingent upon, but does not simply rest with how evaluation methods are defined and implemented. A conscientious revision plan must follow evaluation efforts to ensure that long-lived web-based courses serve learners equally, no matter when they are required to or elect to access your course. This quality assurance must be underpinned by a revision maintenance team that keeps files well-organized and preserves the project revision history.


196

References 1.

Kemp, Jerrold E. et al. Designing Effective Instruction. 2d ed. (Upper Saddle River, NJ: Merrill/Prentice Hall, 1998), 268-269.

2.

Kirkpatrick, Donald L. Evaluating Training Programs: Four Levels. (San Francisco: Berrett-Koehler, 1994), 24.

3.

Ibid., 22.

4.

Kemp, Jerrold E. et al. Designing Effective Instruction. 2d ed. (Upper Saddle River, NJ: Merrill/Prentice Hall, 1998), 169.

5.

Schmeiser, Lisa. The Complete Website Upgrade and Maintenance Guide. (San Francisco: SYBEX, 1999), 421.

6.

Carliner, Saul. An Overview of On-Line Learning. (Amherst MA: HRD Press, 1999), 97.

7.

Ibid.

8.

Cullum, Trish. Interview by the author 29 September 1999.

9.

Nielsen, Jakob. “The Increasing Conservatism of Web Users,” 2. Cited from www.useit.com; INTERNET.

10. Schmeiser, Lisa. The Complete Website Upgrade and Maintenance Guide. (San Francisco: SYBEX, 1999), 205. 11. Ibid.

Chapter Nine

Putting It All Together Chapter topics: ♦ WBT fast-track ♦ Selecting a vendor • Questions to ask vendors ♦ Course migration • Step 1: Define migration purpose • Step 2: Assess overall training context • Step 3: Analyze course objectives and measurement techniques • Step 4: Perform technology gap assessment • Step 5: Develop and approve project proposal • Step 6: Solicit sponsorship • Step 7: Create new blueprint • Step 8: Develop user interface and content • Step 9: Test courseware • Step 10: Deliver and maintain course ♦ WBT tools • Tool classifications • Authoring tools: User-friendliness, flexibility, and skill Chapter resources: ♦ Selecting a WBT project manager or instructional designer ♦ WBT vendor rating sheet for assessing vendor qualifications ♦ WBT design and development summary ♦ Guide to selecting an authoring system 197

198


WBT Fast-track In “Web-Based Training: Market Trends, Risks and Opportunities,” Brandon Hall writes that “Industry analysts place the 2000 Web-Based training industry revenue between $1 billion and $2.8 billion.”1 While the rapidly expanding web-based training market offers exciting possibilities for learners and the training industry, it equally requires training organizations to carefully evaluate options before investing in the development of webbased courseware. As the interest in WBT grows, consultants will claim they can deliver WBT solutions quickly and cheaply. More mandates will be issued to migrate existing courses to the Web. And more tool vendors will convince buyers that their tools are as easy to use as a word processor (well, almost). We hope the instructional foundation of this book provides a framework for making sound decisions in all three areas. To reinforce this goal, the guidelines and resources in this chapter synthesize many of the hallmarks of effective WBT, specifically in terms of vendors, migration, and tools. The following sections are designed as quick references.

Selecting a Vendor Currently, the WBT market is filled with vendors who promise more than they can deliver. When selecting a vendor to handle your WBT project, begin first by examining the vendor’s experience both in web and instructional design. Many of our customers have tried selecting web developers to create WBT. In these cases, effective courseware is delivered only when the web designers contract with a third-party vendor team of instructional designers and course developers. If WBT is not produced from a foundation of instructional design techniques and processes, the result is typically a beautiful, well-coded course that does not meet the client’s business needs and learning goals. Additionally, whether you are new to web-based training or have extensive experience, we recommend starting small with any new vendor by contracting for one course or a pilot project. This approach gives the vendor a chance to learn your culture and environment, and gives you a chance to evaluate the vendor’s work without risking too many resources. The following list of questions is designed to help you evaluate outsourcing options; it can also be useful when hiring team members for an in-house staff. Reasonable answers are provided for each question.

Putting It All Together

199

Questions to Ask Vendors ♦ For demonstrations not delivered via the Web: Why have you chosen to use a CD for this demonstration, instead of the Web? • Request a web-based demonstration; demonstrations delivered via CD-ROM tend to perform much faster than if delivered from a remote server. ♦ Which plug-ins do your courses typically require? Are these plug-ins readily available to most WBT students? • A preferred answer might be: “The plug-in depends on the kind of interaction required to help students achieve desired learning outcomes.” Additionally, uncommon plug-ins might not be supported in some corporate environments. ♦ If the vendor has previously developed CBT: What are the key differences in developing courseware for each delivery system? • The concise writing and self-study development skills required to produce effective CBT are easily transferable to WBT development. However, many of the branched learning techniques and activities used for CBT must be carefully evaluated before they are used in WBT. Replication of these activities can require the use of a plug-in or might not be possible in a web browser. ♦ Can you develop for international audiences? • A vendor with international experience will begin a discussion of the factors that must be considered, such as connectivity issues and translation considerations. Connectivity speed and reliability differ greatly between continents. ♦ For which platforms have you developed WBT? • All platforms handle content differently and have different constraints. Most plug-ins, for example, are not supported by UNIX. The vendor’s familiarity with such issues can affect development efficiency. ♦ What audience factors determine receptivity to WBT? • The discussion resulting from this question should help you determine the extent to which the vendor takes the audience into consideration. ♦ For what browsers do you develop? • A good vendor will develop to the specification of end users. ♦ What is your design and development process? • A desired answer will include references to an instructional design process that allows the training to be tested with target learners before


200

it is released. While this answer should also include web design issues, emphasis should be placed on instructional value. In addition to these questions, two related resources appear at the end of this chapter: “Selecting a WBT Project Manager or Instructional Designer” and “WBT Vendor Rating Sheet for Assessing Vendor Qualifications.”

Course Migration Many vendors are contracted to migrate existing courses to the Web. Before initiating a migration project or deciding to outsource, first define the parameters of migration: Moving a course to the Web is not simply a matter of converting a text file to HTML, inserting graphics, and then loading the files onto a web server. It is just as important to follow an instructional design method when migrating a course as it is when starting from scratch. Because a design team has the advantage of working from existing content rather than generating purely original material, the production cycle might be slightly shortened; however, many migration projects still require significant budget and time investments. For successful migration to WBT, most of the instructional design tasks covered throughout this book must still be performed, including review of needs assessment and task analysis data, blueprinting, and courseware testing before release. There’s also good reason to budget for sponsorship activities to position a migrated course for acceptance. This section introduces migration issues via a process-oriented approach composed of ten main steps: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.

Define migration purpose. Assess overall training context. Analyze course objectives and measurement techniques. Perform technology gap assessment. Develop and approve project proposal. Solicit sponsorship. Create new blueprint. Develop user interface and content. Test courseware. Deliver and maintain course.

The rest of this section addresses key considerations for each step.


201

Step 1: Define Migration Purpose Will the migration result in training that provides learners with feedback on their performance? Will they be held accountable for their new skills and knowledge in specific ways on the job? ♦ ♦

If yes, skip to step two. If no, continue reading this step.

If the migration does not involve criterion-based evaluation of student performance, the migration might be geared toward other purposes, such as developing an online reference center. Such a project can require more web design than instructional design, ensuring that site architecture, navigation, and search capabilities serve end users’ goals. At the same time, we strongly recommend that an instructional designer participate to: ♦

Work with learners and managers to identify which documents to provide online ♦ Establish how the documents will likely be used and to define the best methods of accessing them (For example, should some documents be provided as PDFs?) ♦ Possibly revise selected documents as job aids to correspond directly with end users’ job tasks ♦ Participate in the web design to provide insight gained in previous tasks If the migration will instead result in outcome-based learner performance, the following steps should be considered: Step 2: Assess Overall Training Context Before mandating or agreeing to migrate existing courseware into a web delivery system, consider the appropriateness of the Web for the overall training context in terms of the ultimate training goal, audience characteristics, and the nature of the content. ♦ Training Goal • What were the original expectations for the overall training outcome? The Web provides an efficient method of teaching cognitive skills but can significantly limit learners’ acquisition of psychomotor and discussion-based skills. The latter competencies might require building advanced components into the WBT, such as net conferencing or chat, which both require a knowledgeable

202


facilitator and additional technology. Achieving the training goal might also require integrating web-based modules with other delivery methods, including classroom sessions or on-the-job training. ♦ Target Audience Characteristics • Asynchronous WBT is well suited for large, geographically dispersed learners who require JIT training. In contrast, synchronous WBT generally has a lower cap on student enrollment at a given time, allowing opportunities for equal participation. Also consider target learners’ level of Internet literacy and the availability of adequate end user systems. A severe gap between existing and required Internet skills might call for training to familiarize learners with web environments. A severe gap between current and needed technology can render a migration project cost prohibitive. ♦ Nature of Course Content • The Web is ideal for dynamic content requiring frequent, instant updates that are easy to perform. Additionally, asynchronous WBT ensures that training participants receive the same training presentation worldwide. As part of content assessment, consider how the original course materials are formatted. Other forms of technology-based training, such as CBT and interactive television, generally contain highly modularized text content that facilitates migration to the Web. In contrast, classroom materials generally require a great deal of revision in terms of appropriate modularity as well as writing style. In either case, assume that the migration process will involve significant media creation. Because of technology differences, media elements from other forms of technology-based training might not be delivered via the Web at comparable quality. Step 3: Analyze Course Objectives and Measurement Techniques First, revisit original needs and task analyses to confirm that the given course objectives are accurate and still support business goals. Update course objectives if necessary and then determine whether learners can achieve these objectives in a web delivery system; revision of objectives might be necessary for successful adaptation to WBT. Multiple choice questions are commonly used to assess student performance in WBT. These questions generally provide evidence of a learner’s ability to identify, recognize, compare, etc.


203

However, multiple choice questions typically cannot assess one’s ability to synthesize, analyze, or evaluate. As a result, skill-based objectives and those focused on advanced cognition require more complex measurement techniques, feedback, and technology: Some objectives are best met in other delivery systems. While analyzing course objectives, define also the type and level of performance tracking the course should provide. Performance data will eventually be used to evaluate and improve course effectiveness, and to examine return on the training investment. Step 4: Perform Technology Gap Assessment The analysis from step three provides a basis for defining the technology needs for both server-side tracking and smooth course delivery to client-side desktops. Gain the assistance of an IT representative to examine weaknesses in the existing technology and to define needed upgrades. If those upgrades are not possible, course objectives might require modification. Significant modifications can compromise the integrity of a course and its ability to satisfy business goals. In this case, the value of migration should be carefully reconsidered. Step 5: Develop and Approve Project Proposal The findings from previous steps should clearly indicate the scope, magnitude, and effectiveness of the migration. If the migration is feasible, a proposal should be written and approved prior to design and development activities. This proposal should address such issues as: ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦

Project milestones and deliverables: design, develop, test, deliver, evaluate, revise The magnitude of instructional redesign and the amount of leverageable course material Approximate schedule and launch date Needed tools and equipment encompassing development and delivery Proposed data collection methods and uses Roles and responsibilities: project manager, instructional designer, graphic artist, web programmer, SMEs, migration sponsors (see Step 6) Outsourcing needs Course hosting plan Revision maintenance plan

204


Step 6: Solicit Sponsorship Enlist someone associated with the target audience (such as a manager) to help position the course for positive reception. Because WBT is often a significant training investment, effort should be made to generate interest in the course before it’s taken, which can be a notable challenge with a migrated course. Some learners will look forward to the change, but be prepared for resistance. If the initial course was ineffective for any reason (poor choice of delivery system, for example), many will assume that the migrated course will be equally ineffective. Conversely, if students found the original training enjoyable, they may question the migration. Classroom courses offered only once or twice a year can encourage learners to take part in peer culture and team-building, so consider ways to integrate those values in the WBT or to offer them through some other means. Learners often place a high premium on instructor-led training. The sponsor should also enlist managers to support trainees’ learning efforts. In a contribution to “Online Learning News,” Eric Parks, a WBT consultant, explains the failure behind so many WBT projects: “We forget why classroom training is so popular . . . . It forces the trainee to set aside time and focus . . . .”2 Parks stresses the need to align job performance criteria with training objectives, an observation that’s been echoed in the training industry for years, regardless of the delivery system. A migration sponsor can facilitate this alignment. Step 7: Create a New Blueprint The blueprint should outline all of the design criteria for course development. When possible, the design should address student concerns uncovered in the previous step. For a summary of instructional design guidelines, see “WBT Design and Development Summary” at the end of this chapter and Appendix A, “Overview of Instructional Design.” Step 8: Develop User Interface and Content Although web programmers and graphic artists are typically responsible for creating the UI, the instructional designer must collaborate with these team members, ensuring that the interface reflects training goals and the audience’s needs and culture. Course content must also be designed for effective web presentation. For a summary of development guidelines, see “WBT Design and Development Summary” at the end of this chapter. Chapter Six, “Presentation Principles,” offers a more detailed exploration.


205

Before significant time is invested in coding the course, the client and SMEs must review the first paper-based draft. After the necessary cycles of revision, the course is approved and coding begins. Before conducting formative courseware testing, the team should review the quality of its work, test the course on systems representative of the target audience, and revise accordingly. Step 9: Test Courseware Different techniques of courseware testing generate different results. Before formative evaluation, define the type of data needed, identify test participants, and form a plan for evaluating the results. As discussed in Chapter Seven, we recommend three phases of courseware testing: 1. Criteria testing in which a person skilled in both instructional and web design evaluates the online course based on a set of design standards, or criteria. This test should occur before the client views the entire course online. Any revisions should be made prior to the alpha test. 2. Alpha testing is an opportunity for the project manager to meet with the client, SMEs, and a target learner. During this test, the participants walk through the final courseware to ensure that it matches the defined training goal, audience needs, and client expectations. After making requested changes, the course is ready for beta usability testing. 3. Beta usability testing is conducted with a sample of the target audience. During these tests, an instructional designer and web programmer observe each participant individually, noting any difficulties the participant encounters with the instructional design and user interface. Step 10: Deliver and Maintain Course Once a course is made available to target learners, it must be maintained on two levels: technology and course effectiveness. Ensure that technical support is allocated for server maintenance and student help-desk services. Additionally, the course must be assessed periodically for learner satisfaction, performance, and indications of problems with both course content and instructional design.

206


WBT Tools As with selecting a vendor or deciding to migrate a course, the purchase of a tool that is used to produce and deliver WBT should not be approached casually. Selection of an authoring tool is a commitment that should harmonize with your work process, skills, customer demands, and targetaudience needs. Any Internet search engine will return nearly overwhelming results on searches for authoring tools. Rather than focusing on individual products, this section provides a framework for evaluating and selecting the best tools for your needs—a determination only you can make. The utility of a tool depends as much on the user’s skills and goals as on the tool’s features. For product reviews, user tips, and market projections, we recommend that you visit The MASIE Center (www.masie.com) or the TrainingSuperSite at www.trainingsupersite.com, or take advantage of Brandon Hall’s ongoing research at www.brandon-hall.com. Tool Classifications Many tools offer mixed capabilities, making them rather difficult to categorize. For our purposes, we distinguish among multimedia, authoring, and infrastructure tools. Multimedia Tools These are used primarily for what Ron Ashmun, a multimedia productions manager, calls “asset creation,” referring to the multimedia elements designed to enhance course content, such as graphics, animation, and audio or video clips. 3 Such tools include Macromedia’s Director™. Most of these tools were originally optimized for creating CBT, although most recent multimedia tools are specifically designed or have been modified for web delivery.

HEADS UP! Will authoring a course in this tool require your students to download a plugin? If so, is it a widely used plugin they are likely to already have or can easily install? Is the typical end user system equipped to convey plugin content as intended?

Authoring Tools Developers use authoring tools to produce course materials for web delivery. These applications range from code-based tools that require the user to have programming skills to WYSIWYG tools that generate code according to options that a user selects. WYSIWYG, pronounced “whizzy wig,” stands for “what you see is what you get.”


207

Code-based tools encompass text and HTML HEADS UP! editors. Two common text editors are Microsoft Notepad and Goyvaerts’ EditPad. (Note: If you On the topic of WYSIWYG use a text editor, always save the file in text-only editors, Kreg Wallace, a web programmer, poses this format. The underlying formatting of word question: “If something goes processing programs like Microsoft Word and wrong with the code — and Corel WordPerfect can mingle with your HTML it always will — will you 4 and lead to unpredictable results.) In contrast to know how to fix it?” text editors, HTML editors offer additional features to increase coding efficiency and productivity. Allaire’s HomeSite, for example, is a code-based HTML editor. Unlike code-based tools, WYSIWYG tools, such as Microsoft Front Page, are optimized for non-programmer end users. Several vendors offer WYSIWYG applications that are specifically designed for efficient training development. Although they have wide market appeal, it can be a challenge to customize the materials produced in WYSIWYG applications. They often generate code that is either inaccessible or that requires considerable coding skill to modify. (We cover more on this topic later.) While some of these tools are browser-independent (producing code that is functional across multiple browsers), some generate browser-dependent code, or code that is supported by only a single vendor. Before authoring a course with browser-dependent tools, confirm that all of your learners have the appropriate browser and that they do not plan to switch in the immediate future. Infrastructure Tools Infrastructure tools function primarily as a means of hosting web-based courseware, although many packages now provide a wide range of features, including authoring capability, electronic payment options, and training management functions such as course scheduling and record tracking. A basic hosting infrastructure consists of a server and an Internet connection. The tools can be purchased either as products, such as application servers, which are actual software tools, or they can be purchased as services. Vendors that offer complete infrastructure service packages encompassing courseware production, hosting, and training management features are known as Application Service Providers (ASPs). Appendix B, “Course Hosting and Training Management Systems,” explores a range of hosting methods, features, and associated considerations.


208

Authoring Tools: User-Friendliness, Flexibility, and Skill Despite the dynamic nature of the tool market, tool manufacturers have always been faced with one primary challenge: balancing ease of use with flexibility. On the surface, user-friendly tools require fewer coding skills, but these tools have limited capability for customizing training for a target audience. A fully or even partially customized product requires coding skill. In “Under Development: Purchase Like a Pro,” Elaine Appleton and Tom McLaren offer this advice: “When you look for ease of use, assess the skills of your team. Don’t expect to get ease of use and flexibility in one tool, no matter what the vendors promise.”5 While in some cases they can reduce production time, WYSIWYG authoring tools cannot anticipate all of the innovative designs a developer can muster. (Perhaps we should be thankful that we’re not so predictable or easily replaced.) Furthermore, a tool cannot replace instructional design skills or develop a course for you; and therefore, it should not be elevated to defining the parameters of your course content, preferred measurement techniques, or level of interactivity. Rather, the instructional design of a course should indicate the most appropriate tools for your needs. The following questions can help you make this assessment. An authoring system guide also appears at the end of this chapter. ♦ ♦ ♦ ♦ ♦

♦ ♦ ♦

Has your client stipulated any graphic design or interface specifications requiring a specific tool? Can the templates provided in this authoring tool be sufficiently customized? For what level of Internet literacy and end user system are you designing? Will target end users and systems be able to handle the products created in this tool? Do you expect the authoring tool to replace any of your current tools? Which features are critical? Is the authoring tool compatible with other tools you plan to continue using? Is a conversion process required to import existing materials? What’s the extent of your experience using authoring tools and scripting languages? How much of a learning curve can you realistically anticipate, and does your development schedule allow for this curve? What levels of support and training does the manufacturer offer? What kinds of interactions, hypertext linking, and evaluation methods must be included in your course, and does the tool match those needs? What kinds of feedback are needed to reinforce the learning process?

Putting It All Together ♦ ♦ ♦ ♦ ♦

209

What types of student data do you need to capture? What tracking capabilities are needed? Who should have access to them? With which aspects of WBT creation do you require assistance: instructional design, media development, online delivery, database tracking, results analysis, etc.? What are the hardware capabilities of the computer on which you plan to load the authoring software? Is an upgrade required? What capabilities must learners’ systems have for quality playback? What range of flexibility must the tool provide in the long term?

This chapter concludes with the following resources: ♦ ♦ ♦ ♦

Selecting a WBT Project Manager or Instructional Designer WBT Vendor Rating Sheet for Assessing Vendor Qualifications WBT Design and Development Summary Guide to Selecting an Authoring System

Selecting a WBT Project Manager or Instructional Designer These criteria serve as a useful starting point when advertising an open position, creating a job description, or interviewing a potential candidate to lead your WBT team. A qualified WBT project manager or instructional designer: 1. Demonstrates instructional design expertise ♦ Has an instructional design background and demonstrates the ability to apply instructional principles and processes • Views the main role of web technology as serving training goals, rather than being used for its own sake (A technologically superior product will not generate learning outcomes if the instructional design is ineffective.) 2. Recognizes critical connections between web and instructional design ♦ Recognizes the interplay between instructional goals and the constraints of a web delivery system • Recognizes the need to craft instructional materials specifically for a web delivery system • Recognizes the implications of technology constraints for instructional value and seeks appropriate alternatives

210


3. Has the ability to develop innovative solutions to work with constraints ♦ Acknowledges the ability of a web delivery system to support training effectiveness through supplementary materials such as online job aids, ten-minute review modules, performance tracking, and so forth • Explores the ability of Internet technology to provide innovative, instructionally sound training solutions for continued reinforcement and learning • Regularly engages in research on or associated with WBT 4. Demonstrates familiarity with the technical foundations of WBT ♦ Appreciates the breadth of skills needed on a WBT team • Demonstrates the ability to coordinate disparate activities for the team • Communicates effectively with colleagues such as programmers and IT staff


211

WBT Vendor Rating Sheet Vendor Name and contact information: ___________________________________________________________ Instructions: 1. For each criterion, place a check mark in the appropriate column, “Does Not Meet” or “Meets or Exceeds.” (If exceeds, make two check marks.) 2. Add the total number of check marks for each column. Note that a vendor’s initial proposal might not contain all evidence of criteria. Information can also derive from conversations with the vendor and any WBT demonstrations they provide to substantiate their qualifications. 3. Carry the total number of check marks from each table to the space provided at the end of this tool under “Rating Summary.” 4. To compare qualification ratings among vendors, complete a separate WBT Vendor Rating Sheet for each vendor.

Criterion: Commitment to client’s business needs The vendor demonstrates commitment to business goals: 1. Describes primary outcomes of the training and illustrates how those outcomes serve business goals 2. Assists the client in clarifying training goals, if necessary 3. Indicates why the proposed web delivery system is a more cost-effective delivery method than other possibilities (Evidence should not focus solely on dollars, but can refer to instructional benefits such as higher retention rates, which eventually translate into financial benefits) 4. Serves as an advisor on the costeffectiveness of WBT regarding audience size, projected frequency of content updates, projected shelf-life of the course, and need for consistent presentation

Does not Meet

Meets (1 check mark) Exceeds (2 marks)

TOTALS: _______________________________ Request further information: YES/NO

212


Criterion: Commitment to learners’ needs The vendor demonstrates commitment to learners’ needs by:

Does not Meet


1. Indicates the ability to tailor the training according to key audience characteristics, such as education level 2. Addresses or seeks to learn the audience’s existing attitude toward web-based training 3. Considers learners’ level of computer and Internet literacy when proposing or designing the web course 4. Indicates methods of transforming data on audience characteristics into an intuitive web design that does not distract learners from course material 5. Discusses the ability of a web delivery system to support ongoing learning efforts through supplementary materials such as online job aids TOTALS: _______________________________ Request further information: YES/NO

Special Considerations:


213

Criterion: Instructional design expertise The vendor demonstrates expertise in instructional design:

Does not Meet


1. Derives all aspects of course design from audience and training needs 2. Develops outcome-based learning objectives that are achievable in a web delivery system 3. Selects instructional techniques that help learners bridge gap between current and target skills 4. Demonstrates the instructional value of all proposed media to be integrated into the web delivery system, such as chat sessions, video, and animations 5. Designs objective-based learning activities students must complete to receive evaluation for a specified level of competency 6. Recognizes when additional media are needed to support learning, such as email or a classroom demonstration to practice skills not achievable in a web system 7. Proposes methods of evaluating courseware effectiveness TOTALS: _______________________________ Request further information: YES/NO

Special Considerations:


214

Criterion: Technological capability The vendor demonstrates technological capability by:

Does not Meet


1. Works with constraints established by IT, including limitations of end-users’ systems (modem speed, plug-ins, etc.) and restrictions of security, bandwidth, etc. 2. Seeks methods of preserving instructional value despite technological constraints 3. Provides tracking and management capabilities 4. Indicates willingness to work with a variety of tools and open, industrysupported standards TOTALS: _______________________________ Request further information: YES/NO

Rating Summary Add the totals from all tables to use as one method for assessing a potential vendor’s qualifications:

All Criteria Table 1: Commitment to client’s business needs Table 2: Commitment to learners’ needs Table 3: Instructional design expertise Table 4: Technological capability

Table Totals Does not Meets or Meet Exceeds ________ ________ ________ ________ ________ ________ ________ ________

FINAL VENDOR RATINGS: ________________ Recommendations:


215

WBT Design and Development Summary 1. Audience Analysis Know your technology constraints and the audience’s level of Internet literacy. These limit the possible complexity of activities. 2. Task Analysis Assess whether the target skills and knowledge can be measured effectively in a web delivery system. Examine how available technologies can be used to support learner achievement of all course objectives, negotiating between course goals and technology for learners’ benefit. ♦ Skills such as “explain” or “discuss” require either oral or written responses, thus indicating a possible need for synchronous, facilitating technology such as net conferencing or asynchronous, facilitating technology, such as email. ♦ Depending on the context, skills such as “arrange” require learners to manipulate items on-screen and necessitate scripting languages beyond the capabilities of HTML. 3. Measurement and Reinforcement Each activity should simulate as much as possible how the target skills and knowledge will be applied and measured on the job, with feedback on activities used as a further instructional opportunity. Additionally, make sure that activities appeal to a variety of learning styles and that interactions occur at least every five to seven minutes or on three screens. 4. Modularity and Learning Paths When creating modules, consider: ♦ Length: highly modularized units can help learners feel the training is progressing at a nice pace and can provide a JIT advantage. ♦ Access: must modules be accessed in sequence? Is self-directed learning important for this audience? 5. User Interface Use your knowledge of the audience and actual job context to guide course architecture and navigation:

216 ♦


What primary organizational or thematic systems does the learner encounter at work regarding the training tasks? What will appeal to the learner’s perspective: job titles, selected concepts, tools, processes, different types of buildings, key diagrams? For example, imagine defining navigation based on a beauty shop metaphor vs. a courthouse. Where would you expect information to be located, and how would you access it? (Keep in mind that such specific metaphors require greater knowledge of the target audience.) ♦ Write course content according to the principles of the “inverted pyramid” or newspaper style, which is characterized by short, concise paragraphs and topic sentences that clearly encapsulate the key point of each paragraph. Additionally, limit each screen to one focus or purpose.


217

Guide to Selecting an Authoring System This guide originally appeared in An Overview of On-Line Learning by Saul Carliner, with contributions by Gloria Gery. Reproduced with permission of HRD Press, Inc. © 1999 (43-45). What does the authoring system do—and is it what you want it to do?

An authoring tool is supposed to simplify development of the program. But it can only do so if the authoring system can create the kind of program that you want. Most authoring systems let you easily (or relatively easily) present information, incorporate graphics and video and audio clips, ask questions, and provide appropriate feedback. But exactly what types of media does the authoring system let you include—and with what restrictions? For example, some authoring systems let you include video clips, but the clips always begin running when the screen first appears. You cannot program the video sequence to start after a certain period of time. Similarly, authoring systems let you quiz learners during a course, but what kinds of questions can you use? Are these the types of questions that you want to ask? How widely is The more widely used the system, the more likely that you will find the authoring assistance with questions and the more likely that the software will be system used? supported in the future. The most commonly used software tools are ToolBook™, Authorware™, Quest™, and IconAuthor™. How easily Each course developer has a particular working style. Some like to can you prepare material exclusively in the authoring system. Others like to work import first in a productivity tool. Only after subject matter experts and others information have reviewed the material do such course developers prefer to work in an into the authoring system. The ease with which these developers can move draft authoring materials into the authoring system can affect their productivity. system? In an ideal situation, the authoring system directly links to word processors and other productivity tools or can read files prepared in those productivity tools without special conversion procedures. How easily Complex products require training to use, and authoring systems are can you learn complex products. For example, they let you perform complex tasks like the authoring asking questions and then linking learners to appropriate material based system? on their responses. But how much training is needed? You should be able to master the basics of an authoring system—preparing basic presentations and multiple choice and true/false questions—in a short period of time so you can feel an early surge of success. More complex tasks, such as creating branches, asking open questions (questions that have two or more words as answers), and simulations require more extensive training, but should be easily mastered.

218


How easily can you learn the In addition, how readily available is formal training? authoring system? (continued) Find out whether training is available in your community and how frequently it is scheduled. Some thoughts to consider: Some software publishers certify the people who provide training in their authoring systems. Also, some universities offer continuing education courses on some authoring systems. To be easy to use and learn, does the authoring system sacrifice “power”? That is, does the authoring system only let you create a basic learning experience because the more complex elements might be hard to create? What does the final product Authoring systems are designed to support online look like? learning. Although they let you include graphics, animation, and video and audio clips, the quality of these materials and your ability to control their appearance on the screen are usually not as good as might be expected in presentation software, such as Director. In exchange for the lower quality and decrease in control, you can easily include questions and other types of interaction, and easily link learners to appropriate material based on their questions. Although you can include interaction in a presentation program, it requires specialized programming skills that are not usually needed in authoring systems. Reproduced with permission of HRD Press, Inc. © 1999.


219

References 1. Hall, Brandon. “Web-Based Training: Market Trends, Risks and Opportunities.” (Sunnyvale, CA: Multimedia and Internet Training Newsletter, 1997), 9. 2. Parks, Eric. “When Learners Don’t Finish.” Online Learning News. Vol. 2, No. 26 (21 September 1999). Lakewood Publications listserv. 3. Ashmun, Ron, quoted in Bob Filipczak. “On the Trail of Better Multimedia.” Training (November 1995), 2. Cited 16 July 1999 from Lakewood Publications at www.trainingsupersite.com; INTERNET. 4. Wallace, Kreg. Interview by the author 17 September 1999. 5. Appleton, Elaine and Tom McLaren. “Under Development: Purchase Like a Pro.” Inside Technology Training (March 1999), 1. Cited 16 July 1999 from Lakewood Publications at www.trainingsupersite.com; INTERNET. 6. Carliner, Saul. An Overview of On-Line Learning. (Amherst, MA: HRD Press, 1999), 43-45.


Appendix A

An Overview of Instructional Design Contributed by Judi Schade Appendix topics: ♦ Overview ♦ Training development cycle ♦ Instructional design model

Overview When an organization seeks training to improve its performance, it must take a systematic approach to the design, development, delivery, and support of that training. While elements of our training development cycle are interspersed throughout Instructional Design for Web-Based Training, this appendix presents a complete overview of our approach to instructional design and development. We have relied on this approach since the founding of TrainingLinks in 1993. The instructional design phase of the training development cycle evolved through years of practice by our own instructional designers, who have benefited greatly from models developed by J. H. Harless, Barbara Seels, Zita Glasgow, Jerrold E. Kemp, William R. Tracey, and Robert F. Mager.

221


222

Training Development Cycle Identify Need Define Need

Revise

Evaluate

Training Training Development Development Cycle Cycle

Design

Each stage of the training development cycle involves tasks that are vital to the success of subsequent stages and to ultimate training effectiveness. After a description of each stage, this appendix lists and describes the tasks we follow during stage three, “Design.”

Stage 1: Identify Need Training is needed when a target audience lacks Test skill and knowledge. However, it is fairly common for an organization to think it needs training, when in actuality, the performance deficiency has a different cause. Employee performance can be affected negatively by organizational issues such as shortcomings in the work environment, ineffective incentive or management systems, or improper placement of personnel. For example, when service representatives are rude to customers, their negative behavior might be symptomatic of their frustration with a serious lack of internal support or tools, not a lack of customer service skills. Genuine training needs are often identified in one of the following ways: Deliver

Develop

♦ ♦

Managers recognize an apparent performance deficiency. A new product, service, government regulation, policy, process, or procedure is about to be introduced, requiring employees to have new skills and knowledge. ♦ Existing courseware no longer meets the requirements of the target audience. ♦ Members of the target audience initiate a request because they would like to expand their skills and knowledge. Identification of a training need leads to an analysis of the need in terms of business goals, financial justification, performance deficiencies, and desired competencies. Stage 2: Define Need After a training need has been identified, that need must be thoroughly defined to focus the training appropriately given organizational goals, current skills, and desired skills. The following questions can begin to clarify the

Appendix A: Overview of Instructional Design

223

business need and justify an investment in training design, development, and delivery: ♦

What are the organization’s basic business goals and organizational objectives? ♦ How does the organization expect the training to contribute to its goals? ♦ What factors contribute to the apparent need for training? ♦ What circumstances prompted the request for training? Answers to these questions are incorporated into a needs assessment, an investigative process focusing on the causes of performance problems. A comprehensive needs assessment identifies an audience’s existing strengths and weaknesses in skill and knowledge. Additionally, defining desired skill and knowledge levels requires documenting the desired competencies for employees in specific jobs. Competency details are best obtained through task analyses with skilled performers, or employees who already exhibit the desired skills and knowledge. Other common needs assessment methods include capability studies, observations, work output analysis, and surveys. Stage 3: Design The design stage begins with an examination of needs assessment and task analysis data. From this information, we identify the areas of skill and knowledge gaps in which target learners require training. During this process, prerequisite training needs are also established. Based on these determinations, we then design the following instructional elements to bridge target learners’ knowledge and skill gaps: ♦ ♦ ♦ ♦

Course objectives Appropriate practice activities and tests Organization of content Instructional techniques

We then select delivery systems that accommodate the above instructional elements. A delivery system should also suit learning styles of the target audience and encourage the effective transfer of learning to the job. (For more information, see the section titled “Instructional Design Model” located in this appendix.)


224

The design stage concludes with documenting all design decisions on a course blueprint that will be used to guide course development. Stage 4: Develop During development, course developers write training content according to the blueprint. After the client and SMEs review first-draft materials, we create a second draft that incorporates requested revisions. We recommend a review and revision of the second draft before conducting any formative evaluation of the courseware. Stage 5: Test Formative testing is best accomplished in two stages—alpha and beta testing. ♦

After second draft revisions are made, an alpha test involves an organized walk through all course materials by content experts, developers, instructors, and skilled performers. The purpose of the alpha test is to ensure that course objectives, activities, tests, content, and materials have been effectively designed and developed to meet overall training goals and audience needs. ♦ After alpha test revisions are made, the course is actually delivered to a small selection of target audience members in a beta test. Depending on learning activities, at least five target learners should participate; observers should include course designers, developers, and content experts. Stage 6: Deliver A course delivery plan depends on the selected delivery system. For nonfacilitated, web-based delivery, target learners can access training at their convenience once the courseware is fully tested and its availability announced. Stage 7: Evaluate At this stage, the methods designed to measure students’ achievement of course objectives and their satisfaction with a course become tools for assessing training effectiveness. Data on student achievement and satisfaction indicate adjustments that might be made to a course to improve its efficacy. For example, data indicating patterns of unusual challenges with specific exercises and test questions also indicate the potential need for design and


225

content revisions. Ideally, evaluation data should derive from results acquired during course activities, as well as from student effectiveness studies conducted thirty days or more after completion of training. Stage 8: Revise “Revise” is the final stage of our training development cycle. Analysis of evaluation data should guide plans for revising the courseware. Immediate revisions based on effective analysis can lead to greater returns on the training investment. The next section provides more detail on the design stage of our training development cycle.

Instructional Design Model Instructional design ensures that a course is developed according to specification. Just as an architect designs a structure on paper before breaking ground, an instructional designer writes a course framework before developing it. Disregarding the role of instructional design can lead to training that does not adequately address the audience and its identified skill and knowledge gaps, in turn causing the training investment to collapse. Instructional design culminates in a course blueprint that outlines how a course should be developed given the outputs of several design tasks. Our instructional design model consists of twenty-one tasks, of which the first five are considered predesign. The rest of this appendix briefly describes each task. 1. Analyze needs assessment data. 2. Analyze target audience description. 3. Analyze tasks. 4. Identify skill and knowledge gaps. 5. Identify prerequisite skills and knowledge. 6. Prepare measurable course objectives. 7. Describe cumulative and criterion tests. 8. Form modules. 9. Determine module sequencing. 10. Analyze client criteria and constraints for effects on design decisions. 11. Gain client approval before continuing. 12. Form logical learning units within modules. ♦ Determine sequencing of logical learning units within modules.

226


14. Determine placement of practices and tests. 15. Identify appropriate types of practice activities and tests. 16. Identify the primary sources of information during the training (job aid, self-instruction, or instructor). 17. Develop a content outline. 18. Identify techniques for presenting content. 19. Determine appropriate delivery system(s). 20. Determine appropriate delivery media. 21. Document details on course blueprint. Tasks 1 through 5: Predesign These tasks include a comprehensive examination of needs assessment and task analysis data, followed by in-depth instructional analysis to determine skill and knowledge gaps and prerequisites. 1. Analyze needs assessment data. An effective needs assessment yields vital information about the target audience’s existing skill, knowledge, attitudes, expectations, and other key factors of significance to instructional designers. 2. Analyze target audience description. Along with results of the needs assessment, a target audience description further establishes the target audience’s existing skill, knowledge, attitudes, and expectations, and other key factors. 3. Analyze tasks. A task analysis identifies the correct methods, techniques, and cognitive processes for proficiently performing the required tasks. Task analysis also establishes the knowledge, tools, and interactions necessary for successful performance of those tasks. Meaningful task analysis data is best gathered through interviews and observations with skilled performers. 4. Identify skill and knowledge gaps. These gaps are defined as the difference between an audience’s existing skill and knowledge and the required skill and knowledge for proficient performance. Instructional designers identify these gaps by comparing data from needs and audience analyses (current proficiency) to task analysis data (target proficiency).


227

5. Identify prerequisite skills and knowledge. Difficulties for both the instructional designer and the target audience can occur when there is an attempt to design a single course for a diverse audience. To resolve these difficulties, designers identify the diversity of an audience’s existing skill and knowledge levels, then determine whether some target learners require prerequisite training. A requirement for prerequisite skill and knowledge helps to ensure that all participants begin the training with a common level of skills and knowledge. Tasks 6 through 21: Design These tasks include all of the critical design activities that culminate in the course blueprint described in task 21. 6. Prepare measurable course objectives. Course objectives should be designed to bridge the skill and knowledge gaps that the training must address. Complete objectives include the skill or knowledge to be demonstrated, the conditions under which it will be demonstrated, and the required level of achievement. The following example includes all three components: ♦

Given a videotaped scenario depicting a salesperson making a sales presentation, knowledge of the components of an effective sales presentation, and an evaluation checklist (conditions), participants will appraise correct and erroneous behaviors exhibited by the salesperson in the scenario (behavior) according to the prescribed criteria for effective performance (standard).

Terminal objectives describe what target learners must know or be able to do as a result of the training; enabling objectives describe the skills or knowledge that lead to the achievement of terminal objectives. 7. Describe cumulative and criterion tests. Cumulative and criterion tests are intended to measure cumulative achievement or overall performance. Terminal course objectives become the basis for describing cumulative and criterion tests. The description includes a vision of the process for administering the test, the test content, the criteria for determining effective performance, and the means by which performance data will be captured and evaluated.

228


8. Form modules. A course is usually comprised of several modules, each covering a single topic, task, or terminal objective. Learning is reinforced and measured at the end of a module according to the module objective and related practice activity or test. 9. Determine module sequencing. Module sequencing usually follows the typical behavioral sequence of the tasks to be trained unless presenting some concepts and behaviors out of sequence better facilitates learning, such as by providing additional practice opportunities to target learners. 10. Analyze client criteria and constraints for effect on design decisions. It is important to verify any specific client criteria and constraints that might affect previous or subsequent design decisions. For example, the client might have already decided on the delivery system. Stipulations like this can influence the effectiveness of course objectives and criterion tests defined in previous design tasks, requiring them to be rewritten and potentially compromising their instructional value. 11. Receive client approval before continuing. Based on findings from task ten (above), the instructional designer and client should discuss any requested modifications that appear incongruent with the training goals and target audience. The designer and client should work together to identify suitable solutions, based on the decisions made in previous design tasks. (It is also at this point that the client first sees the objectives, tests, and modules.) 12. Form logical learning units within modules. Within each module, learning is “chunked” so that concepts and behaviors can be reinforced with practice activities before additional concepts and behaviors are taught. Each module contains one or more units that chunk and reinforce learning at strategic points. 13. Determine sequencing of logical learning units within modules. The sequencing of learning units within modules usually follows the typical behavioral sequence of the task steps being trained unless learning is better facilitated by presenting some concepts and behaviors out of sequence.


229

14. Determine placement of practices and tests. Practices and interim tests are usually placed at the end of logical learning units and modules. In addition to the practice and measurement opportunities placed at the end of individual learning units, several learning units might be practiced and measured together. 15. Identify appropriate types of practice activities and tests. Practice activities and tests are derived from course objectives to ensure that participants are required to perform the desired behaviors according to appropriate criteria and standards. 16. Identify the primary sources of information during the training (job aid, selfinstruction, or instructor). Course objectives specify the key types of knowledge and behaviors that trainees must master, as well as the conditions and standards that determine successful achievement. Depending upon these objectives, some or all of the training might be accomplished through job aids that are self-explanatory or that require only minimal training support for subsequent use on the job. Some or all portions of the training might be effective through self-study, while other portions might require instructor-led delivery in order for course objectives to be effectively met and measured. 17. Organize informational content. Once all previous decisions are made, the designer can now create a detailed course outline that contains all of the informational content, practice activities, and tests. Course developers use this outline to create course content and materials. 18. Identify techniques for presenting content. Primary delivery techniques include lecture, group discussion, and small team exercises. The designer analyzes the course outline to determine the best methods to encourage learner achievement. 19. Determine appropriate delivery system. Delivery systems include traditional classroom, paper-based self-study often accompanied by videotape, technology-based self-study such as WBT, and interactive television. Once the designer determines course objectives, activities and tests, and techniques for presenting content, the choice of an appropriate delivery system(s) becomes apparent. The delivery system

230


should be selected on a module-by-module basis, resulting in a multi-faceted delivery of some courses. 20. Determine appropriate delivery media. Once the content, instructional techniques, and delivery systems are determined, the designer can make appropriate decisions regarding audiovisual support. Depending on the delivery system, delivery media might include computer-based slides with or without animation, video or audio clips, flip charts, or paper handouts. 21. Document details on course blueprint. The course blueprint enables the designer to present a clear vision of the course to the client. In turn, the client can provide detailed feedback on course content and activities before developers begin to create the first draft of course materials. Developers then use the blueprint as a detailed roadmap for developing course materials.

Appendix B

Course Hosting and TrainingManagement Systems Contributed by Jay Erikson

Appendix topics: ♦ Course development to delivery ♦ Basic course hosting • Architecture • Drawbacks ♦ Training-management systems • Purchase a training-management application • Purchase training-management services • Build a custom training-management application • Project requirement considerations • Architecture

Course Development to Delivery Once a course has been tested and final changes have been made, the course is ready for delivery, or hosting. Hosting is the process of making a course available to end users over a network (either a local area network for intranet-based training or the Internet for ubiquitous WBT). Hosting 231

232


encompasses a spectrum of features and complexity, from simply making the courses available online to complete training-management systems. Basic course hosting is relatively easy and inexpensive; it simply requires a course to be loaded onto a web server. The first WBT courses were often hosted in these basic environments. But as online learning becomes more prevalent, hosting options are increasing in features and complexity. Hosting options now include the ability to manage training administration and track several dimensions of student performance. These bundled capabilities, which can be purchased as a service or product, are called “training management systems.” This appendix begins with a description of basic course hosting and concludes with an overview of training-management systems. Use this information to evaluate hosting options and to develop a hosting plan that meets your requirements.

Basic Course Hosting Basic course hosting is the process of making content available on a web server for downloading and display in a web browser. Generally, content is in the form of HTML and graphic file formats; however, through the use of plug-ins you can display other types of content such as streaming video in RealNetworks RealPlayer™ (.ram) format and animations in Macromedia Shockwave™ and Flash™ (.swf) formats. Basic course hosting is the easiest, quickest, and cheapest method for hosting courses, and requires only limited architecture and system administration knowledge to implement. Most WBT courses can be hosted in this environment. The next section provides an overview of a basic web hosting architecture, then concludes with some of the drawbacks of this type of course hosting. Architecture The architecture for a basic web hosting environment can be created with two simple components: a web server and a network connection. A server is a dedicated computer that usually has a special operating system for serverrelated tasks, such as sharing files or printing documents. To build a web server, you simply install a web server software application on a server. The type of web server software application you use depends on the server’s operating system.

Appendix B: Course Hosting and Training Management Systems

233

Linux is one of the most prominent operating systems for web servers. As a free, UNIX-like operating system, Linux has advantages such as stability and performance, not to mention price. However, it is more difficult to install and configure than some other operating systems, such as Microsoft Windows NT™. There are many web server software applications for both Linux and Windows NT. The most popular web server software is Apache Web Server from the Apache Software Foundation. Apache is free, downloadable from www.apache.org, and available for both platforms. Other web server software applications include Microsoft Internet Information Server (IIS)™ and Netscape Enterprise Server™. The above software applications represent only a few of the web server applications available. The other component for basic course hosting is a constant network connection. Unlike the modems most users have at home that require you to dial a number to obtain access to the Internet, direct connections maintain a constant, unbroken connection to the Internet. This allows the web server to respond when the user makes a request for a document in the form of a URL. For example, if you make a request for http://www.cnn.com/index.html in your web browser, the web server will respond by displaying the index.html file that is in the root directory of the web server. The web server, however, can only respond if it is connected to a network or the Internet. There are many types of direct connections, including ISDN (integrated services digital network), T1, T3, DSL (digital subscriber line), and cable modems. T1 and T3 are the most common method of connectivity for web servers. DSL and cable modems are the newest types of network connections and may not be available in all areas. ISDN, T1, and DSL operate over existing copper telephone lines, but they all require special hardware. In addition, DSL is limited to certain distances from the telephone central office. Cable modems use existing coaxial cable, which is a shared medium. As a result, users on the same cable share bandwidth, which can deteriorate performance and lead to security vulnerabilities. These network connectivity options range in price, complexity, connection speeds, and security. In general, however, DSL and cable modems are the easiest and cheapest connectivity options; however, T3 offers the greatest connection speed. Your knowledge of network connectivity and systems administration, as well as the project budget and schedule, will influence your decision as to whether to build a web server environment or outsource to an Internet Service Provider (ISP). Building a web server environment is not difficult, but it does take time. You also need to plan time to perform administrative tasks,


234

such as tape backups. A typical web server might have the following components: ♦ ♦ ♦ ♦ ♦

200 MHz CPU 128 MB RAM (memory) 1 GB hard drive (storage) An operating system, such as Linux A web server software application, such as Apache

The hardware for a typical web server costs approximately $1000, and the software can range from free to several hundred dollars. In general, building your own web server environment for basic course hosting is relatively inexpensive, especially if you intend to host multiple courses. If you are uncomfortable with installing and configuring hardware and software, you may decide outsourcing basic course hosting to an ISP is a better solution. ISPs originally offered dialup Internet access for a monthly fee; however, with the proliferation of content providers and customers wanting to host personal web pages, ISPs have increased their content hosting options. ISPs allow you to store content files on their servers, in addition to providing the network connectivity. Outsourcing can be a relatively inexpensive option, as well, with prices starting around $20/month. In addition to being an inexpensive solution, outsourcing allows you to begin hosting your courses quickly and easily. There are some precautions you should take when seeking an ISP for basic course hosting. First, obtain an agreement that they will not use or access your files without your prior consent. The content of your WBT might be confidential, or at the least, proprietary; you don’t want someone accessing your intellectual property without authorization. Next, determine which services you will receive under a basic plan, and which services will cost you an additional fee. In general, you will have access to a specified amount of storage space (for example 10 MB), and a limited throughput rate, such as 5 GB/month. Throughput is defined as the amount of data that is transferred over a network for a given period of time. You will be charged additional fees if you need more space, or exceed the established throughput rate. For most WBT courses, however, this is more than enough storage space and throughput. You will probably have to pay additional fees for streaming video services, access to an application server such as Allaire ColdFusion™ or PHP™, application under the GNU General Public License, or access to a database such as Oracle8.i™, Microsoft SQL Server™, or T.c.X MySQL™.


235

Regardless of whether you build your own hosting environment or outsource to an ISP, you will have additional course hosting considerations. If you have a lot of users or high availability requirements, you might need a faster system, more memory, or redundant components. Availability is defined as the server functioning properly and providing access to users during specified hours of operation. One project might require the server to be available from Monday through Friday 8:00 to 5:00, while another might need to support multiple shifts in several time zones. If your project has high availability requirements, you can decide to have several web servers, in case one fails, or have multiple servers in different geographical regions. Redundant components such as multiple hard drives, system fans, or power supplies will generally allow a server to continue functioning even if a single component fails. Drawbacks Although basic course hosting is an easy, cost-effective method for delivering courses online, valuable data on student performance and course effectiveness can be lost because basic course hosting does not provide user tracking or a systematic method for collecting user feedback. These metrics are important for evaluating the effectiveness of the training. To perform this type of analysis, a system must capture data on learner progress and course effectiveness. As described in Chapter One, Donald L. Kirkpatrick defines a four-part process for evaluating training1: ♦

Reaction: Students indicate their satisfaction level with the course using an evaluation form. ♦ Learning: Pre- and post-test scores are used to measure the extent to which students learned the material. ♦ Behavior: A supervisor from the workplace assesses skill transfer between the training situation and actual performance context. ♦ Results: This level focuses on the training’s financial results, often using ROI. The first two parts, reaction and learning, can be measured by student responses to quizzes, activities, and course evaluation forms. At some point following the training, a supervisor can assess the other two parts of the process, behavior and results, from user performance on the job. This information can be recorded in a database with student responses and allow


236

you to evaluate the effectiveness of a course. This type of data capture and analysis is one benefit of a training management system.

Training Management Systems In contrast to simple course hosting, which provides no tracking mechanisms, training management systems are being developed to track enduser performance and facilitate training administration. User tracking is one way to measure ROI. For example, in the Telecom industry, correlating user scores with reduced “repeats” after training is one form of ROI analysis. “Repeats” occur when a technician must return to the site twice for the same problem. This might arise for several reasons, such as if the technician can’t fix the problem during the first visit, or if the technician doesn’t know how to fix the problem. By eliminating one percent of the repeats, companies can save several million dollars annually. Training management systems, also known as course management or learning management systems, vary in complexity and features. Common features to facilitate user tracking include course registration, test tracking, and performance reporting. The market for training management systems is still in the “early adopter” phase and thus most systems are custom applications for internal training projects. There are, however, many commercial products available that support a wide variety of requirements. When evaluating training management systems, make sure that the system has features that support the business needs (see the list of features later in this appendix). There are three options for obtaining training management: ♦ ♦ ♦

Purchase a training-management application Purchase training-management services Build a custom training-management application

Purchase a Training-Management Application A training management application provides a quick and easy solution for hosting and tracking courseware. Some of the more popular trainingmanagement applications are Asymetrix Librarian™, LearnLinc™ LearnLinc4™, and MindQ Telemachus™. These applications range in price and features.


237

There are some constraints with training-management applications, however. In general, you must provide the hardware for the application, in addition to performing installation and configuration tasks. On an ongoing basis, you will need to provide system administration services, including maintenance, troubleshooting, and tape backups. Purchase Training Management Services In contrast to purchasing a training-management application, you can purchase a training-management service. When you purchase a trainingmanagement service, a training company or ASP provides and maintains the training-management application, servers, and network connection. You access the application through a customized web site. By outsourcing for training-management services, you will be able to incorporate trainingmanagement administration and tracking quickly and easily. In addition, you will not have to provide systems administration maintenance or support, thereby reducing your delivery costs. Training-management services are a relatively new iteration in training management. There are a few precautions you should take to protect an investment in a training management service. First, it is important to evaluate the quality and performance of the service, and to make sure the features you need are currently available in the application. Next, perform some form of due diligence analysis of the training company or ASP. Investigate the following: ♦

Corporate management: Does the company have a solid management team? Does it understand application development, the training industry, and the future of training management systems? Does it have a plan for future growth and development? ♦ Technical capabilities: What type of technical management does the company have? Does it have experience in application development and the ISP marketplace? Does it have solid system and network design? Does it have some form of redundancy? Does it have a direct connection to a major ISP network backbone? ♦ Financial status: Is the company financially stable? Are there any flags that lead you to question the future viability of the company? Is there any reason to question whether your content might become compromised in the event of insolvency?

238


♦

Company longevity/track record: How long has the company been in business? Does it have customer success stories or a qualified reputation? In addition to performing due diligence analysis, there are a few other steps to protecting your investment. At a minimum, obtain a service-level agreement that specifies application availability in terms of a percentage of the specified time of operation. For example, the contract can specify the application has to be online from 8:00 AM EST to 8:00 PM PST with 95 percent availability. In addition to service level agreements, be aware of additional costs that are not stipulated in the contract. Also, you might want to include a provision in the contract that places the source code for the training management application in escrow during the life of the contract. Although supporting software developed by others is a tricky and arduous process, you want some protection for your investment in case the company dissolves during the contract. Build a Custom Training-Management Application A third option for obtaining some form of training management is to build a custom application. Building a custom application has many benefits, including assurance that it meets your requirements completely, retains control over source code, and has the knowledge to troubleshoot an application designed and built internally. Building a complex trainingmanagement application is a full-fledged software development project, however, that requires expert analysis and design, as well as solid programming and testing. The development cycle can last from three months to more than twelve months, depending on the scope of the project and the number and experience of the developers. Critically evaluate the advantages and disadvantages of this approach before undertaking custom application development. Project Requirement Considerations The requirements of your project are another factor in deciding whether to purchase an application or service, or build a custom application. Start by creating a list of features that are necessary for your project. Then evaluate whether these features are best provided by an application or service, or whether it is better to build a system that meets those requirements. The following is a list of common features for training management systems:


239

User Administration ♦ Log-on verification ♦ New-user signup ♦ Multiple profiles (for example, student, administrator, instructor) that provide functions according to profile specifications Course Administration ♦ Functions, including data integration and tracking, to manage training for all delivery systems (web-based, instructor-led, interactive television, selfpaced, and web conferencing) ♦ Course selection ♦ Course scheduling ♦ Training curriculum and career paths ♦ Collaboration (email, bulletin board systems, chat, shared whiteboards, group collaboration and assignments) ♦ Integration with other business tools (for example Enterprise Resource Planning (ERP) applications or Electronic Performance Support Systems (EPSS)) ♦ Integration with data sources ♦ Context-sensitive help systems ♦ User feedback forms for course evaluations ♦ Manager feedback forms for long-term analysis of student performance Content Development and Administration ♦ Online content creation ♦ Dynamically generated courses based on user profile ♦ Database-driven content to support multiple devices (for example, Palm Pilot™, palmtops, field laptops) ♦ Test-creation tool ♦ Bug-tracking tool ♦ Training snippets: 5- to 15-minute training segments Evaluation ♦ Testing (grading, data aggregation, user responses to test questions) ♦ Reporting (such as performance by user, course, profile, and region) ♦ Preassessment to determine prerequisites or guide dynamic course generation

240


The previous list represents a range of currently available features, although most training management systems have a subset of this list. Most systems have user verification and signup, course selection, some form of reporting, and the ability to track test scores. More advanced systems will include other options from the list. For example, integration of data from other data sources might be a requirement for customers with a large user base. Manually entering user data, such as name, address, email address, manager(s), division, or prior training records for 60,000 users would be cost prohibitive. In addition, manually entering data would introduce the possibility of data entry errors. Dynamic course generation is also becoming an important feature for some companies who have diverse target users. Dynamic course generation is a process for creating courses automatically from a database based on a user profile. For example, a Telecom supervisor might take an ethics course that is delivered to a palmtop device, with specific examples geared for supervisors, whereas a technician might take a similar ethics course, with different examples, that is designed for a field laptop. Finally, training management systems often vary in the way they record user test scores. Since test scores are a key indicator of user performance and course effectiveness, tracking and recording scores is a critical feature of training management systems. Due to the complexity of tracking user test scores, this feature is often implemented in a variety of ways. For example, some systems merely record a percentage of correct responses: The management system would record that Joe received a grade of 78 percent for the Hazard Communication course exam. Other systems aggregate the data for all users for each question. For example, a system might capture the following data: Question 1 – How many home runs did Mark McGwire hit during the 1998-1999 Major League baseball season? Possible Correct Answer Responses Answers A. 50 4 B. 61 16 C. 65 10 D. 70 * 50


241

Most instructional designers prefer to track each user’s response to every question. For example, the system would track that Joe answered D on Question 1 of Course 14: Baseball History. This data will allow you to perform in-depth analysis on the course materials, such as how many baseball fans in the St. Louis area answered the question correctly. Each system will have its strengths and weaknesses, based on the user community or specifications for which it was designed. Careful analysis of your requirements and the available training management systems will help determine which is the best solution for you. The next section provides a brief overview of the architecture of training management systems. This is provided to build awareness of the components of a training management system. Architecture The architecture for training management systems varies as widely as the features. Most management systems are configured as three-tier web-based applications. Three-tier applications divide tasks among three separate processing units: ♦ ♦

Presentation design/logic layer: UI and control logic Application logic layer: Programming logic that controls the functionality ♦ Database design/logic layer: Backend database that supports the application

Presentation Layer

Application Layer

Database Layer

Many products are available for each component of the architecture. Components vary in price, functionality, and complexity. The presentation logic is the interface for the application. It is presented by the web server and

242


displayed in a web browser. The presentation logic is generally developed with HTML and JavaScript and might contain DHTML or multimedia features. The application logic is the programming that controls the functionality of the courses and management application. The application layer is generally the interface between the presentation layer and the database and would typically reside on the web server. Many types of application servers are available, including Microsoft IIS, Allaire ColdFusion, PHP, Bluestone Sapphire/Web™, IBM WebSphere™, SilverStream Application Server™, and Apple WebObjects™. Microsoft IIS and Allaire ColdFusion typically provide the quickest out-of-the-box development time. The third component for training management systems is the database. The database tracks all information about users and courses. Common databases include Oracle 8i and Microsoft SQL Server. Other components of a training management system might include caching servers or geographically dispersed intermediary servers for providing faster access, and e-commerce servers, as well as a variety of tools for monitoring performance, controlling server loads, or facilitating collaboration. Few standards have been established because this market is still in its infancy. The market will experience some consolidation as it moves from the early adopter phase to market maturity, so it is important to choose systems that are based on industry standards and widely supported tools.


References 1. Kirkpatrick, Donald L. Evaluating Training Programs: Four Levels. (San Francisco: Berrett-Koehler, 1994), 24

243


Appendix C

Creating Graphics for the Web Contributed by Kreg Wallace

Appendix topics: ♦ Overview ♦ Graphics file formats: GIF, JPG, and PNG ♦ Considerations for download speed • Interlacing • Reducing file sizes • WIDTH and HEIGHT attributes • Image reuse and file naming • Relative pathnames ♦ Typeface considerations for the Web • Standard browser fonts • Serif and sans-serif fonts • Image-based text ♦ Usability issues • Consistent navigation • Scrolling • Plug-in use and instructions • Exit buttons • Font and background colors • Help systems

Overview In just five minutes of web surfing, an end user can encounter both visual elegance and aggravation. While few would equate WBT with artwork, creating effective graphics for web delivery is an art that requires an 245

246


understanding of how a web page behaves as a canvas. This appendix provides an introductory guide for those interested in experimenting with that canvas. We have selected topics with WBT in mind, focusing on techniques that lead to quick-loading, easy-to-read web pages. We have also included some common strategies for improving a course’s usability.

Graphics File Formats When selecting a format for image files that will be presented over the Web, graphic designers use GIF, JPG, or PNG. Selecting an appropriate format for a particular image is often more a matter of practice than adherence to hard and fast rules. It’s common for a designer to save an image in each of the formats, adjusting various settings to find the best compromise between file size and image quality. You should always keep a high-quality original of the image (such as a PSD if using Adobe Photoshop™), and then save copies of this original as GIF, JPG, or PNG. Resaving the same file from one web graphics format to another can distort an image, as can repeated saving and sizing within the same format. The next sections cover each of these formats in more detail, offering points of comparison between the kinds of images to which these formats are suited and their ability to display transparent or animated graphics. GIF Pronounced either “gif” or “jif,” GIF stands for “graphics interchange format,” which was developed by CompuServe™ specifically for generating the small image files needed on the Web. GIF’s compression scheme works well for illustrations such as line drawings and image-based text, and for images that contain large, solid-color fields. GIF is also great for basic transparency effects and simple animation. GIF Transparency One of the limitations of graphics files used on the Web is that the images they contain must be rectangular. Often, however, you will want to present irregularly shaped images, so that the image appears set within its own space, unencumbered by a rectangular edge. The ability to render parts of an image transparent helps a graphic artist to control the edge shape of their images and to provide some interesting special effects. GIF format is capable of assigning binary transparency to specified colors; that is, individual colors within a GIF file can be displayed as completely opaque or completely

Appendix C: Creating Graphics for the Web

247

transparent. Because it uses this “all or nothing” approach to transparency, GIF format is somewhat limiting. GIF Animation GIF is currently the only format discussed here that is capable of producing animated files. GIF animation works by saving multiple images into a single file, which are then displayed one at a time in succession, much like a slide-show. Unfortunately, because animated GIFs contain multiple images, they can easily grow to bandwidth-busting proportions. To minimize this problem, keep GIF animation simple, using as few distinct images as possible to maintain the smoothness of the animation. Animated GIFs can be made to loop continuously or simply to play through a specified number of times and then pause on a specific image. As users often find the continuous motion of looping animated GIFs distracting or even annoying, use caution when including animated GIFs in your WBT. JPG JPG or JPEG is pronounced “jay-peg.” This acronym derives from “Joint Photographic Experts Group,” which is the committee that wrote the original JPG format. JPG uses a compression algorithm that works well with fullcolor, photograph-quality images or those containing continuous tones, such as blends, fades, etc. JPG will often render these images at significantly smaller file sizes and with greater fidelity to the original than GIF. However, JPG format is generally not suitable for line art or similar illustrations (at which GIF excels). JPG does not handle sharp edges or solid color areas very well and will interpolate “artifacts,” or seemingly random pixels of color, that mar or diminish image quality. Additionally, JPG format does not support transparency, so if you wish to make irregularly shaped images for the Web, you’ll need to use either GIF or PNG formats. PNG PNG (pronounced “ping”) stands for “portable network graphics.” Thomas Boutell developed PNG specifically as a replacement for the GIF file format. PNG offers capabilities spanning the feature list of both GIF and JPG. Although current versions of Netscape Navigator™ and Internet Explorer™ support PNG, neither browser yet correctly implements PNG’s full suite of features. Because of its flexible handling of color bit-depth, PNG can produce images that match or exceed the color quality of comparable GIFs or JPGs.


248

(Bit-depth refers to the number of colors that can be used in an image, where 8-bit translates to 256 colors.) Be aware, however, that PNG file sizes will often be slightly larger than their GIF or JPG counterparts. Remember to experiment with multiple image formats to find your best compromise between image quality and file size. The table below compares bit-depth capabilities between GIF, JPG, and PNG formats. File Format GIF JPG PNG

Bit-depth Capability only 8-bit depth or lower 24-bit depth only 8-bit, 24-bit, or 32-bit depths

PNG Transparency PNG has the most sophisticated transparency support of any web graphics format. Whereas GIF offers only one level of transparency, PNG offers 254 levels of “partial transparency.”1 In other words, PNG colors can be set to appear on a 254-level continuum, from completely opaque to completely transparent. With this much flexibility, assigning PNG transparency can be slightly more involved than with GIF. However, when employed in a web page, partial transparency allows designers to create the appearance of layering in an image, with the web page’s background color or pattern still visible through the PNG image. This capability has several uses, such as subtle drop-shadows or glow effects. PNG Animation A MNG (or multiple-image network graphics) specification for producing animated PNG files is currently being developed as an addition to the PNG format. At this time, major browsers do not support MNG.

Considerations for Download Speed This section covers basic techniques to minimize the time needed for a web page to download. It also includes strategies for creating the illusion of speed to hold an end user’s interest. Throughout the discussion, we continue to make distinctions among GIF, JPG, and PNG.


249

Interlacing You may have noticed that on some web pages, images appear to slowly come into focus as the page is loading. This is interlacing, which is a method of gradually displaying visual elements as they are received over a network.2 Interlacing is a technique to stave off end-user boredom and provide partial information about an image while waiting for a web page to finish downloading. Shawn P. Wallace, author of Programming Web Graphics with Perl and GNU Software, refers to this technique as producing a “perceptual gain in download speed” 3 or creating the perception of a page loading faster than it actually does. GIF interlacing is usually referred to simply as “interlacing”; JPG interlacing is often referred to as “progressive”; and PNG interlacing may be referred to as “Adam7” (the specific interlacing algorithm used by PNG and named for the algorithm’s creator, Adam Costello). Most graphics applications allow you to turn interlacing on or off, when saving images as GIF, JPG, or PNG. Depending on the file type, interlacing usually does not produce an increase in actual download speed. Both interlaced GIF and PNG files tend to be slightly larger than their corresponding noninterlaced files. However, PNG offers a greater illusion of speed than GIF. In Designing Web Graphics, Lynda Weinman writes that “GIF interlacing gives a preview of the image after 1/8th of the image data has been recognized, whereas PNG gives a preview after only 1/64th of the image has loaded.”4 In contrast to GIF and PNG, Wallace notes that interlaced JPG files tend to be slightly smaller than noninterlaced JPGs.5 Reducing File Sizes The length of time an image takes to download is dependent on the size of the image file: smaller files correspond to shorter download times. An effective way of reducing the size of GIF image files is to decrease the number of colors in your images. GIF and other 8-bit image formats store color information in a A GIF file depicted in different palette called a color look-up table (CLUT). stages of the interlacing process


250

Reducing the number of colors used in your GIFs will reduce the size of the CLUT attached to the GIF image, resulting in smaller file sizes. Attempting to reduce the number of colors used in a JPG, however, will have no effect on file size, because JPG is a 24-bit image format and does not make use of a CLUT. JPG files use an adjustable quality setting to control file size. However, lowering the quality setting not only reduces the size of the saved JPG file, but negatively affects the quality of the resulting image. As with GIF and JPG, options for reducing the size of PNG image files are based on the bit-depth of the specified image. An 8-bit PNG file makes use of a CLUT, while PNG files greater than 8-bit allow adjustments to quality settings. Greyscale and monochrome images offer another method of increasing the download speed of web graphics. Greyscale images look similar to blackand-white photographs and are composed of varying shades of grey, white, and black. While similar to greyscale, monochrome images use a specified color in place of black. Greyscale and monochrome images tend to offer significant savings in file size over their full-color counterparts.

Full Color JPG (21KB)

Greyscale JPG (10KB)

Monochrome JPG (10KB)

Monochrome JPG with color highlight (10KB)


251

Using greyscale images with judiciously chosen spots of color can effectively emphasize a particular area of an image, a technique that can be useful for WBT. Although the resulting image is technically neither greyscale nor monochrome, the file savings can still be significant. WIDTH and HEIGHT Attributes You can also decrease the time it takes for a browser to display images by specifying the WIDTH and HEIGHT attributes of an image in the HTML image, or , tag. This technique does not affect download time, but instead reduces the time needed for a browser to render a web page as it downloads. If you do not specify the WIDTH and HEIGHT attributes, the end user’s browser must determine the width and height of each image, which takes more processing time. When WIDTH and HEIGHT attributes are specified, the browser can begin laying out the page immediately, because it already knows how much screen area to allot each image. For illustration, the following line of HTML specifies both WIDTH and HEIGHT:

These attributes should be used with care, however. If you specify an incorrect number of pixels for the WIDTH and HEIGHT of an image, browsers will not correct your HTML, but will instead distort the image to match your specifications. In addition, the WIDTH and HEIGHT attributes should never be used as a way of making an image appear smaller on the page: The file size of the image will remain the same as the original, larger version. Instead, make the dimensions of the image itself smaller, and set the WIDTH and HEIGHT attributes to the correct dimensions. Image Reuse and File Naming Reusing the same image on multiple pages of a web site is a great way to make better use of available bandwidth. Once downloaded, images are cached or stored on the end user’s local computer, so that when an image is referenced again, it can simply be loaded locally, without having to download again from the web server. However, web browsers reference cached images based on the image’s pathname, or uniform resource locator (URL), rather than by the image itself. So if you include multiple copies of the same image on your site using different file names, each file will still have to be downloaded separately. Similarly, if you include multiple copies of the same file that have the same

252


name but are located in different directories, each file will also have to be downloaded separately. Best practice is to include only one copy of the image on your site and reference it from multiple pages with relative pathnames to the same file. The next section covers relative pathnames. Relative Pathnames Image files are included in web pages with the use of a pathname (also known as a URL). A pathname directs a browser to the location of a file on a web server. There are two types of pathnames available for use on the Web: absolute pathnames and relative pathnames. As in the following example, absolute pathnames begin with “http://” and a domain name:

This prefix alerts the browser to search for a requested file within an explicit directory structure on a particular web server (specified by the domain name). While this approach works, it presents some significant limitations. Consider the case of WBT vendors who outsource hosting to an ISP: Initial development and testing of the web course takes place on a server at the vendor location. However, before loading the course on a server belonging to the ISP, all of the absolute pathnames must be changed to indicate their new server and directory locations. This introduces significant opportunities for mistakes, resulting in broken links and inoperable web pages. To reduce the potential for broken links and increase the portability of your websites, we recommend using relative pathnames for all images and local hyperlinks (links to documents on the same server). Relative pathnames do not require developers to indicate a server name or complete directory structure in order to locate a file. Instead, they make use of a dot notation to indicate a file’s relative position to other files in a directory structure. This allows web developers to move their web sites from one server to another without change. An example of an tag that makes use of a relative pathname might be:


253

Typeface Considerations for the Web We’ve given you some simple techniques for speeding up the time it takes for your WBT to process and download. The next section covers basic considerations for making a web page easy to read once it reaches a student. Standard Browser Fonts Although HTML allows designers to specify text fonts, a designer’s control is relatively fleeting unless he or she selects one of the standard browser fonts: Arial, Times New Roman, and Courier New. When users access web pages, their browsers must interpret the designer’s specifications according to their own capabilities, which can cause a page to appear much unlike the designer intended—resulting in a hyperspace font disaster. Let’s say, for example, that a designer formats the text of a web page in Garamond font. The HTML tag for accomplishing this might be:

The FACE attribute in this tag specifies the name of the font to be used; SIZE specifies how large the font should be displayed; COLOR indicates the color in which the text should be displayed. (Colors in HTML are often indicated using a hexadecimal (or six digit) notation. See Lynda Weinman’s Designing Web Graphics (www.lynda.com) for a handy RGB-to-hexadecimal conversion chart and a list of colors that can be referred to simply by name, such as “blue” or “red.”) This tag requests the Garamond font, which is a serif font that is almost surely not installed on every one of the end users’ computers. If the user has Garamond installed, the text will be properly displayed. If Garamond is not installed, the browser will simply display its default font instead. Unintended font substitutions can undesirably alter the entire page layout due to the different leading and kerning standards of various fonts. The best practice, of course, is to select one of the three standard browser fonts. But if a designer is deeply committed to another font (Garamond, let’s say), the following tag will anticipate browsers that don’t have that exact font installed:

This tag prompts a browser first to check whether Garamond is installed. If it is not, the browser will continue searching for each font in the list until it finds one that is installed. Only after it has exhausted all of the provided


254

alternatives will the browser resort to its default replacement font (The “serif” indication requests the browser to use any available serif font). Using such font tags gives designers greater control over the look of their web pages across the varied range of end-user systems. Note: The tag has been deprecated in HTML 4.0, which means that although the tag still works, it might not be supported in future versions of web browsers. Instead, the makers of the HTML specification, The World Wide Web Consortium (W3C), suggest using cascading style sheets (CSS). However, only fourth-generation browsers or greater are capable of handling CSS. Use your own judgment on which to use, but in our opinion, it is likely that the tag will still be supported for years to come. Serif and Sans-Serif Fonts Serif and sans-serif are the most widely used font types. Serif fonts, such as Times New Roman, are characterized by many curves and tails at the edges of the letters. Sans-serif fonts, such as Arial, are characterized by straight lines and sharp angles.

Serif

Sans-serif

Serif font

Sans-serif font

Graphic designers are often instructed that serif fonts are easier to read for large chunks of text because the glyphs and tails of the letters guide a reader’s eye over the words. This is true in the case of printed documents, because desktop publishing applications use vectors to assign shape to the curves in serif fonts, which are then rendered smoothly with high-resolution printers. Some of the design rules developed with print media in mind do not necessarily apply on the Web, however. On the Web, vectors and high resolutions are generally not possible. (Currently, vectors on the Web can only be displayed with plug-in-based applications, and resolution on the Web is always 72 pixels-per-inch. In contrast, common printer resolution is 600 or 1200 dpi, or dots-per-inch.) Because HTML-generated text is aliased, sansserif fonts, with their boxy, straight-edge style, generally present HTMLgenerated text with greater clarity than serif fonts. (Aliasing is covered next.)


255

Image-based Text As an alternative to font tags, text can also be displayed on the Web as graphic images. Using images to present text ensures that the intended font is preserved, regardless of browser font capabilities. Additionally, using images for text allows designers to customize the look of text with special effects such as drop-shadows, glows, etc. This method has significant drawbacks, however: • Image files take longer to download than simple HTML text, so reserve image-based text for occasional items, such as titles and logos. • Creating text images requires more time, effort, and graphics experience than creating HTML text. • Any revisions required after a course is coded are easier to make in HTML text. When creating image-based text, designers have the option of making the image aliased or anti-aliased. Aliasing is a property of rasterized, or bitmapped, image files characterized by jagged edges. Aliased image files are displayed by placing square pixels of color within the grid of a monitor. (Anyone who has played Space Invaders will no doubt recognize a similarity with the aliased examples below.)

Aliased Text

Anti-Aliased Text

Detail of Aliased Text

Detail of Anti-Aliased Text

256


Anti-aliasing is a technique used by graphics applications to minimize the jagged look of rasterized graphics. While anti-aliased images are still composed of squares of color placed within a grid, the edges of anti-aliased text images will have pixels of a color that is a blend between the text color and background color. In this manner, anti-aliased images trick the eye, making edge transitions appear more smooth (and more like the edges of vector images). Our recommendation is to always anti-alias your image-based text unless the text in the image is particularly small (say, 12 pt. or lower). With small font sizes, anti-aliased text tends to appear blurry, while aliased text maintains a crispness that is easier to read. When including image-based text in a web page, be sure to also include the ALT attribute in your tag. The ALT attribute supplies nongraphical text while graphics download, helping users to quickly grasp the purpose and content of a page. The following succession of graphics illustrates the utility of the ALT attribute. Suppose a designer creating a web page includes a title graphic containing the words “Welcome to WBT!” He or she includes the HTML tag, , in order to display the following image:

While waiting for the image to download, the tag’s ALT attribute allows a browser to offer a glimpse of the graphic’s content:

If the ALT attribute had not been included, the end user would simply have seen an icon indicating that the image had not finished downloading:

Including the ALT attribute when using image-based text is simply an easy way to provide users with the information they need as quickly as possible.


257

Usability Issues This section of the appendix covers some basic usability tips that should help students of WBT focus on training content rather than the interface itself. Consistent Navigation If you want to be sure your WBT is easy to use and navigate, pay some attention to the location of navigation buttons on the screen. Whether you keep these navigation buttons on the top, bottom, or sides of the screen, be sure to keep their location consistent across all pages of the training. This will help you make certain that users will not have to learn your navigation system more than once. The navigation buttons should always be visible regardless of scrolling and should stay in the exact same screen space. This allows students to simply place their cursor over the “next” button and proceed through the course without having to move the mouse except when participating in activities. Reducing unnecessary mouse movements keeps users tuned into WBT content, rather than the UI. In this sense, the UI can be thought of as “transparent.” Techniques for maintaining the placement of navigation buttons include placing all navigation buttons in a separate frame, or eliminating scrolling from your WBT so that these buttons are always visible. Scrolling Scrolling is of major concern to developers of WBT. Our best advice is to keep scrolling to a minimum, or to eliminate the need for scrolling altogether. WBT courses that require a lot of scrolling can pose unforeseen difficulties for students. A common pitfall is that users might overlook the presence of a scroll bar and inadvertently skip content that exists below the visible confines of their screen. In this event, students will not have accessed all information pertinent to course quizzes. Furthermore, keeping content segments in small, easily digested chunks makes information easier for students to learn. By chopping your training into these smaller chunks, you can eliminate the need for scrolling in most instances. Plug-in Use and Instructions The use of plug-ins can allow the delivery of highly interactive, mediarich content in WBT. However, unless students are web-savvy, they are likely to experience some confusion about using plug-ins. Always let students know

258


of any required plug-ins before the training begins. There is little more frustrating for a WBT student than to progress halfway through a course, only to discover that the rest of the training requires some plug-in they do not have. Explain plug-in requirements at the beginning of the course and provide instructions for download and installation. You will also need to provide a link to a site at which the plug-in can be found. A sample of a plugin-directions page appears below.

Try to limit your plug-in usage to only one per WBT course, as the extra time and effort required to download and install several plug-ins can discourage some students from participating in the training. Exit Buttons It is common practice to provide an exit button in WBT interfaces. If your course includes such a button, make sure to incorporate ways to keep students informed of their navigation status to avoid unintentional exits. It is best to provide an exit dialogue box, alerting students that “exit” has


259

been clicked and requesting confirmation that exiting the course is indeed what they wish to do. A simple confirmation dialogue can be easily accommodated with a JavaScript confirm box, as illustrated below.

The confirm box will accommodate any inadvertent exit button clicks and reassure students that they are exiting the course in a safe and prescribed manner. Font and Background Colors Using color combinations that make text difficult to read will unnecessarily distract students and diminish the effectiveness of your webbased course. Be sure to use font and background colors judiciously, selecting colors with strong contrasts to ensure readability. Typically, your best combination will be the standard black text on a white or off-white background. Even though white text on a black background presents the same contrast, we have found that a light text on a darker background is more difficult to read.

260


The use of softer contrasts and more vivid colors might appeal to the sensibilities of various users and provide a break from the monotony of black on white. However, if you choose to use colors other than black and white for your text and background, consider whether the combination is easy to look at and read (on screen) for an hour or more. Help Systems The Web is still new as a training delivery system, and often students will require some help with basic browser issues and the use and installation of plug-ins. Even web-savvy students sometimes require help understanding the particulars of WBT interface or management system. Implementing a help system is an important way to accommodate the basic needs of all users and reduce calls to the help desk or frenzied emails to your webmaster. Best practice for WBT is to provide at least a basic help page or set of instructions explaining: • How to use a browser • How to download and install any required plug-ins • How to navigate your particular UI After including these basic items in your help system, try to identify potentially confusing areas of your site and anticipate student concerns by providing a brief set of instructions on the page itself. These pages should also include a link to the specified topic in your help page or help system. Doing so provides a simulation of the context-sensitive help functionality found in popular desktop applications and can significantly lessen the time students spend searching through a help system for a particular topic.


261

References 1.

Roelofs, Greg. “A Basic Introduction to PNG Features.” Cited 19 September 1999 from http://www.cdrom.com/pub/png/pngintro.html; INTERNET.

2.

Wallace, Shawn P. Programming Web Graphics with Perl and GNU Software. (Sebastopol, CA: O’Reilly & Associates, 1999), 10.

3.

Ibid.

4.

Weinman, Lynda. Designing Web Graphics. 2. (Indianapolis, IN: New Riders Publishing, 1997), 61.

5.

Wallace, Shawn P. Programming Web Graphics with Perl and GNU Software. (Sebastopol, CA: O’Reilly & Associates, 1999), 10.


Glossary Aliasing: An attribute of rasterized image files characterized by sharp, jagged edges. Alpha test: A type of formative evaluation during which subject matter experts, developers, instructors, and skilled target audience members walk through a course to determine its effectiveness based on the target audience and established training goal. Anti-aliasing: An optional attribute of rasterized image files and technique intended to reduce the jagged edge effect of aliased images. Anti-aliased images are characterized by the appearance of smooth edges produced by assigning some pixels a color that is a blend between the colors of their adjoining pixels. Applet: A small program written in the Java programming language, designed to run within a web browser. Application server: One component of a three-tier application that provides the programming logic that allows interaction between a user interface and a database. Architecture: The underlying structure of a web site. Architecture refers to the organization and availability of web-site content. Asynchronous WBT: Instruction delivered via a network at a time defined by the student. If the instruction is facilitated by interaction with peers and an instructor, such interaction takes place at self-defined intervals, as with email. Audience analysis: A three-pronged approach to determining how the needs and nature of a learning audience should be addressed effectively during a training course; the three prongs of audience analysis are needs analysis, task analysis, and instructional analysis. Bandwidth: The amount of data, usually measured in bits or bytes, transmitted through a communications line in a specified amount of time and usually measured in seconds. If the communication line were a tunnel and 263

264


the data was water, then bandwidth would measure the amount of water capable of flowing through the tunnel in a given period of time. Beta usability test: A review of the usability and navigation issues of a webbased training course. Beta usability testing follows both criteria and alpha testing, and is the last stage of testing that a web-based training course undergoes before delivery to the learning audience. Beta usability testing involves delivering a course to at least five members of the target audience, while including course designers, developers, and content experts as observers. Bias (in audience analysis): The inadvertent tendency to impose one’s views or assumptions on a learner or learning situation, and to allow those assumptions to guide course design and development rather than relying on genuine learner characteristics and needs. Bitmap image: See “Rastor image.” Blueprint: A document that reflects key decisions made throughout the instructional design process, such as a target audience description, course goal, objectives, evaluation methods, and so forth. Blueprints are used as guides for effective course development. Bookmark: A link to a web site or web page saved on a user’s hard disk and accessible through a menu in a web browser. Bookmarks make recall and retrieval of a web site more efficient on future visits. Also used to mark a page in WBT, so users can return to a specified section of course content at a later time. Brainstorm: An idea-generating session in which participants are encouraged to freely associate on a given concept or problem, and in which judgment of individual ideas is withheld. Broken link: An inoperable reference address or URL on a hypertext document which, when clicked, fails to take the user to another document either on the same or some other server. Browser: A software application that interprets and displays files formatted in HTML. Common browsers include Netscape Navigator™ and Microsoft Internet Explorer™. Bulletin board: An asynchronous Internet forum for the exchange of like information and files among members of a particular interest group.

Glossary

265

Business sponsor: A representative from the organization that has requested training (client) who provides corporate and possibly financial support for a training project. This person also removes roadblocks to project development by acting as liaison between the purchasing organization and the vendor. Cable modem: A type of direct connection to the Internet. Cable modems use existing coaxial cable, which is a shared medium. As a result, users on the same cable share bandwidth, which can deteriorate service and lead to security breaches. CBT: See “Computer-based training.” CGI: See “Common gateway interface.” Change element: A training strategy used to keep the presentation of webbased training from becoming monotonous. Change elements are integrated throughout a training course and can include practice activities, the use of graphics to complement text, and the use of animated graphics or audio to demonstrate key points. Chat: A form of synchronous communication between users on the Internet. Instead of talking, people use their keyboards to type out conversation, which is transmitted over the Internet. Client: A program on one computer that makes a request of a program on another computer, known as the server. Examples: when a web browser requests a page from a web server, or when a user on a networked system depends on the server to run a program or print a file. Client-side programmer: See “Web programmer.” CLUT: See “Color look-up table.” Coder: See “Web programmer.” Color look-up table (CLUT): A palette used by 8-bit graphic files to indicate colors to be displayed in the image. The number of colors used in the CLUT has a direct effect on the file size of the associated image.

266


Comment function: A web form provided for recording reviewer feedback during a criteria test. Comments can be incorporated into a database for later retrieval and analysis. Common gateway interface: An interface program used by web servers to pass data or requests from web users to other programs on the server for processing. After those programs process the data or requests, the server passes the information back to the user’s web browser. Examples: keyword searches, information retrieved from a database, and forms filled out and processed online. Competency: Ability to perform a specified task, as for a job. Computer-based training: Instruction delivered to end users via floppy disks, the Internet, laser discs, or CD-ROM. Cookie: A small piece of information contained in a text file and typically used to profile a user’s preferences at a web site. The use of cookies allows web developers to create programs that recognize users who have visited their site before. Cookies are generated by a web server and stored on a user’s hard disk. Cost comparison per student: A method of estimating the cost of instruction on a student-by-student basis, calculated by dividing the number of expected learners into the total cost of course design, development, and delivery. This number can be viewed as a pricing guide and indicator of how many students are needed to make the training financially feasible. Course developer: A person who produces course content according to specifications in the instructional designer’s blueprint. Courseware: The software implementation of computer-based training that delivers an organized experience of instruction and activities. Criteria test: An assessment of web-based training content and usability performed by an expert skilled in both web and instructional design. Criteria testing is performed by the vendor organization and is completed before the client views the entire course online.

Glossary

267

Database: A collection of stored information organized and designed for immediate access and cross-referencing. A software application used to store such information. Debrief: A discussion and reflection of the outcome of any process or event. Deliverables: The products that result from the work process; that which the client purchases. Delivery system: The method used to present training to a learning population; examples include WBT, classroom, and satellite television. A single delivery system typically integrates various technologies and media, and can be used with other delivery systems. Thus, a web delivery system uses web technology and media such as graphics, video, chat, email, etc., to provide web-based training. Desktop publishing software: Applications used to create a wide range of documents, including books, magazines, brochures, and flyers. A powerful tool that allows one person to write, edit, design, format, typeset, and print a complete document. Development phase: The portion of a training project in which the actual training materials are produced. In web-based training, the development phase includes the production of course content, graphics, and the user interface. DHTML: Dynamic HTML. A combination of tags and options added to the HTML 4.0 specification that allows for greater control of animation and interactivity. Support for DHTML is only available in Netscape Navigator 4.0+™ and Microsoft Internet Explorer 4.0+™. Directory: An electronic folder or similar feature of a computer file system. Directories can contain one or more files, and are used for hierarchically organizing documents into meaningful groups. DNS: See “Domain name system.”

268


Domain name system (DNS): A system by which the Internet and other networks translate domain names into Internet protocol (IP) addresses to locate computers on the network. Download: To receive a file from another location over a network. Examples: a file transferred to a computer over the Internet or a file copied from one computer to another over a local network. Drag and drop: An activity performed by placing a mouse’s cursor on top of an object on the computer screen, holding down the mouse button, and moving the cursor (and object) to another location on the screen. DSL: Digital subscriber line. A type of direct connection to the Internet, which operates over existing telephone lines. DSL requires special hardware and is limited to certain distances from the telephone central office. Dynamic HTML: See “DHTML.” E-commerce: Conducting business transactions online, involving the exchange of money for goods or services over the Internet. Electronic bulletin board: See “Bulletin board.” Enabling objective: A statement that describes in measurable terms what a student must know or be able to do to achieve an associated terminal objective; like terminal objectives, enabling objectives usually consist of three components: (1) skill or knowledge to be demonstrated, (2) conditions under which the demonstration will occur, and (3) a testing criterion specifying a level of proficiency for the demonstration. End user: The final consumer of computer-related products. Facilitated WBT: Instruction delivered via a network that incorporates either synchronous or asynchronous interaction among peers and an instructor. Firewall: A software application used to maintain the security of a web server or local area network (LAN). A firewall is typically implemented to prevent outsiders from gaining access to a company’s intellectual capital in the form of computer files.

Glossary

269

Formative evaluation: An assessment of a course’s effectiveness performed while a course is still in a development stage. Gap assessment: An examination of the distance between an existing condition and a desired condition. GIF: Graphics interchange format. An 8-bit graphics file format developed by Compuserve™ to display bitmap images on the Web. GIF supports simple animation and transparency effects. Graphic artist: A person who designs images to accompany text, often with the aid of a software program. A member of a web-based training team responsible for developing the user interface and illustrating key training concepts. Hardware: The physical components of a computer system: the CPU, electronic circuitry, cables, connectors, and peripheral devices. Help desk: A location or telephone number to which students of web-based training can turn for help with technical issues related to using the training course. Heuristic evaluation: An assessment of a course based on a list of user interface requirements. Experts evaluate the degree to which the interface design of a web-based course complies with usability principles. High availability: The property of some computer systems to run with little or no downtime for long intervals of time. High-availability implementation techniques can be hardware-based, such as the use of redundant data drives, or software-based, requiring the use of a robust operating system (see “UNIX/Linux”). Homepage: The first page a user sees when accessing a web site by its domain name. A homepage typically serves as a directory for the rest of the site. Hosting: A process that enables a web site to be accessed over the Internet or a network. Requirements for hosting a web site include a server, web-server software, and a dedicated connection to the Internet. Facilities for web-site hosting can be developed internally or outsourced to an Internet service provider.

270


HTML: Hypertext markup language. A markup language used to specify placement of text and graphics on a web page. Hyperlink: An electronic connection between hypertext documents on the World Wide Web. The link can also connect to a different point within the same web page. Hypertextuality: The capacity of documents to be linked to other documents. Nonlinear movement by a web user between objects such as text or graphics on the World Wide Web. Inductive reasoning: The act of inferring general principles from individual, repeated instances. Information technology (IT): All aspects of creating, storing, managing, or processing electronic information. The term is used broadly and encompasses all forms of technology related to the computer or telephone industries. Infrastructure: The basic components of any system or organization. For example, in information technologies, infrastructure refers to the hardware required to transmit information between clients and servers. Instructional analysis: The third step of a three-pronged approach to audience analysis. One goal of instructional analysis is to determine the prerequisite skills and knowledge necessary for learners to successfully complete the training. Instructional design: A systematic process (including analysis and design) based upon learning and instructional theory for designing instructions. Instructional designer: A person trained in the principles of instructional design who can apply those principles to a variety of learning situations and materials. Interactivity: The sensory dialogue between a user and a computer program. Interactivity occurs when a user inputs data or submits commands (clicking a mouse button or typing on a keyboard), which in turn elicits output (displaying pop-up windows, change in images, text, sounds, or printouts) from the computer program.

Glossary

271

Interlacing: A graphics file display technique used on the Web that allows portions of an image to be revealed gradually as the file downloads. The order of pixels displayed is determined by the particular interlacing algorithm of the file format; either GIF, JPG, or PNG. Internet: The world-wide computer network used to transmit such electronic data as email and web pages. Computers on the Internet are linked to each other via telecommunications lines, modems, and the TCP/IP protocol, allowing each computer to transmit data to any other point on the network. Internet literacy: The measurable competency and level of experience someone has with using the Internet. Internet service provider (ISP): A company that provides access to the Internet for a fee. Most ISPs provide a means for users to gain dial-up access to the Internet and provide web-site hosting services. Intranet: A company’s in-house network (or LAN) used to access HTML or other web-formatted files. Intranets, as opposed to the Internet, allow access only to a controlled group of users and are not available to the world at large. Inverted pyramid writing style: A method of content organization where the conclusion is stated first, and all of the supporting evidence is presented in order of the greatest importance to the least. This is an inversion of the traditional pyramid, where the conclusion is stated only after evidence and arguments have been presented in order of the least important to the greatest. ISDN: Integrated services digital network. A type of direct connection to the Internet, which operates over existing telephone lines. ISP: See “Internet service provider.” IT: See “Information technology.” Java: An object-oriented, cross-platform programming language developed by Sun Microsystems. Java can be used to create stand-alone applications or be implemented in conjunction with a web browser or server to enhance the functionality of a web site. Java programs displayed in web browsers are known as applets.

272


JavaScript: A scripting language developed by Netscape that allows inclusion of dynamic and interactive content in web pages. JavaScript applications are implemented within a web browser. JIT training: Just-in-time training is delivered to employees just before they will need to apply the new skill on the job. JPG: Joint Photographic Experts Group. A 24-bit graphics file format developed by the Joint Photographic Experts Group for displaying photographic quality or continuous tone images on the Web. JScript: The implementation of JavaScript used by Microsoft Internet Explorer™. Kinetic: Energy produced by motion. A physical action. Kirkpatrick’s scale: A four-level method devised by Donald Kirkpatrick for evaluating training effectiveness that incorporates (1) reaction (learners’ reactions to a course), (2) learning (attitudes changed, or skills and knowledge learned), (3) transfer (training’s impact on changes in job behavior), and (4) business results (impact on organizational goals). Knowledge gap: The measurable distance between learners’ current and desired knowledge levels. Learning representative: A member of the target audience for a training course. Learning units: A segment of information or a task within a module that must be learned before a student can proceed to the next task. Linearity: A sequential progression of objects or events. Linear web-based training requires students to access content and activities in a predefined sequence. Link: See “Hyperlink.” Listserv: An automated mail server that distributes email to all subscribers on a mailing list.

Glossary

273

Markup language: A computer language, such as HTML, used to format page layout, text, and graphics placement in a document. MNG: Multiple-image network graphics. An extension to the portable network graphics (PNG) graphics file format for displaying animated images on the Web. Modem: A computer device that allows dial-up access to the Internet via an internet service provider. Modems make use of existing telephone lines or cable connections. Modularity: A property of instructional design in which broad content topics are “chunked” into smaller, logical learning units. Module: A singular component within a training course. Like chapters in a book, a training course is usually comprised of several modules, each covering a single topic, task, or set of objectives. Learning is reinforced and measured at the end of a module, according to the module objectives and related practice activities or tests. Multimedia: A combination of presentation media, such as text, audio, graphics, and/or streaming video, which appeal to a variety of senses and learning styles. Navigation: The movement of a user between the components of a web site or software application; the means or tools used to facilitate such movement. Needs analysis: The drawing of conclusions and solutions for possible training needs; a systematic process that precedes instructional design. Needs analysis involves gathering and synthesizing data from and about a target audience for whom training might be a potential solution, as well as from work environment conditions, tools, policies, processes, and procedures that impact job performance. Needs assessment: See “Needs analysis.” Non-facilitated WBT: Instruction delivered via a network that does not involve interaction with peers or an instructor; non-facilitated WBT is necessarily asynchronous, because it does not require course participants to be online at the same time.

274


Objective: See “Enabling objective” or “Terminal objective.” Online, on-demand courses for public Internet access: Web-Based training available to the public, typically requiring payment in the form of an online credit card transaction. Operating system (O/S): The initial startup and master control program that runs a computer’s hardware and peripherals and manages all the other programs on a computer. Common operating systems include Windows™, Macintosh™, UNIX™, Linux™, and DOS™. O/S: See “Operating system.” Outsource: Hiring of a company or person outside of the vendor organization to perform a training design, development, delivery or maintenance task, or service. For example, many web-based training vendors outsource the hosting of their courses to internet service providers. Page turner: A web-based course that does not incorporate multimedia or enhanced interactivity in its design. Users interact with page turners on a very basic level: reading text, scrolling up and down, or clicking forward or backward buttons. Pathname: A text string used to indicate the location of a file within a directory structure or network. Also known as a URL, pathnames can be characterized as either absolute or relative. Pixel: The smallest display unit in a graphic image. Grouped in the thousands (or millions), pixels (or picture elements) make up the graphic display on a computer screen. Plug-in: A small software application, or module, which can be installed on a client computer to increase the functionality of a web browser. Plug-ins are typically used to allow interaction with specialized types of computer files, as with Macromedia Shockwave™. PNG: Portable network graphics. A graphics file format developed by Thomas Boutell as a replacement to GIF. PNG is distinguished by its sophisticated handling of transparency and image bit-depths.

Glossary

275

Practice activities: Change elements used to reinforce learning principles and introduce interactive elements into a training course. Proficiency: The level of expertise with which a task is performed. Programmer: See “Web programmer.” Project manager: The primary customer contact who is responsible for a project within his or her company. Prototype: A modifiable, small-scale model of a product that has not yet been released into the marketplace. The findings from prototype testing and review are figured into the final product design. Rastor image: A type of image file that is displayed by mapping specific colors to individual pixels within the grid of a computer monitor. An alternative to vector images. Also known as a bitmap image. Read-me file: A text file distributed in some software packages that usually contains concise installation instructions or software revision notes that might not be included in the printed documentation. Read-me files are usually named “readme.txt” or something similar. Retention rate: The percentage of information presented in a training course that a student is able to recall or make use of once training has been completed. Return on investment (ROI): A means of computing the financial gain of an investment; the savings earned by a training solution less the cost of that solution. Request for proposal (RFP): A document issued by a client to a vendor requesting a description of specified services, schedule, and pricing information required to complete a proposed project. The RFP typically includes a description of project scope and specifications. RFP: See “Request for proposal.” ROI: See “Return on investment.”

276


Sans-serif font: A kind of font that is characterized by straight lines and sharp angles. For example, “Arial” is a sans-serif font. Screen capture: A duplication of all or a portion of a computer’s screen display as an image file. Screen captures are made by “capturing” the display of a computer screen. Screenshot: See “Screen capture.” Scripting language: A computer programming language, such as JavaScript or Perl, which is not compiled and which relies on another software application for execution. For example, JavaScript can only be executed within a web browser and Perl scripts can only be executed by the Perl interpreter. Self-directed learning: Instruction that encourages or requires students to define the pace, content, and structure of a course in which they are participating. Serif font: A kind of font whose characters have curves and tails at their edges. For example, “Times New Roman” is a serif font. Server: A computer dedicated to performing specific tasks, such as sharing files or printing documents and which typically requires a special operating system for performing server related tasks. Examples of server operating systems include Linux and Microsoft Windows NT™. Server-side programmer: See “Web programmer.” Skill gap: The measurable distance between learners’ current and desired skill levels. Skill transfer: The application of skills and knowledge gained in training to the actual performance context, such as a job. SME: See “Subject matter expert.”

Glossary

277

Streaming video: A video file displayed over a computer network with the use of streaming technology. A video file that can begin display before all components of the file have completed downloading. To view streaming video on the Web, you will usually need a plug-in such as Real Player™ from Real Networks. Storyboard: A screen-by-screen diagram that illustrates the text, graphics, and interactivity that will appear in a WBT course. This is the blueprint from which web programmers and graphics artists will work. Style sheet: A file that contains the master page layout instructions for web, word processing, and desktop publishing documents. Style sheets store format settings, such as fonts, margins, line spacing, headers, footers, and tabs. Subject matter expert (SME): A person with extensive expertise in a given subject who evaluates the accuracy of course content according to his or her expertise. The SMEs also work in conjunction with project managers, instructional designers, and course developers to identify audience and training needs. Summative evaluation: An assessment of course effectiveness that takes place after a course has passed through alpha and beta testing and has been delivered to the target audience. Synchronous WBT: Instruction delivered via a network that requires learners and an instructor to be online at the same time to participate in learning interactions. Systems administrator: A person charged with the responsibility of maintaining server or network security. T1: An implementation of the T-carrier system introduced by the Bell System, T1 is a type of direct connection to the Internet, which operates over existing telephone lines, achieving data transmission rates of 1.5 Mbs (or Megabytes per second). T3: An implementation of the T-carrier system introduced by the Bell System, T3 is a type of direct connection to the Internet, which operates over existing telephone lines, achieving data transmission rates of 44.7 Mbs (or Megabytes per second).

278


Target audience: The intended group of learners for whom a training course is designed. Target learners: See “Target audience.” Task analysis: A systematic process of uncovering how a task is performed effectively; a task analysis involves a flow-charted description of each task, along with the steps, knowledge, decisions, tools, and human interactions required to perform each task. Taxonomy: A science or system of classifying related topics or ideas. For example, Bloom’s taxonomy classifies verbs and strategies for constructing training objectives. Technical sponsor: A key client contact for vendors of web-based training. This person provides client infrastructure information, troubleshoots technical difficulties, and resolves issues with IT. Terminal objective: A statement that describes in measurable terms what a student will know or be able to do upon successfully completing a course. Terminal objectives usually consist of three components: (1) skill or knowledge to be demonstrated, (2) conditions under which the demonstration will occur, and (3) a testing criterion specifying a level of proficiency for the demonstration. Thematic system: A stable assembly of web elements that support a central motif or fundamental organizing concept. Three-tier application: A type of distributed software application consisting of three major parts, each located at a different place on a computer network. The three parts consist of a user interface, programming logic, and a database. Time to market: The time it takes to develop a product from conception to its introduction to the marketplace. Training goal: The stated objective of an instructional course or system. Training management system: A comprehensive web-based training application that allows organizations to distribute, track, and maintain their training programs—regardless of the delivery systems used to administer the training.

Glossary

279

Training problem: The impetus for seeking training or requesting that training be developed; usually relates to a business need. UI: See “User interface.” UNIX/Linux: UNIX and Linux are 32-bit multithreaded operating systems that share a similar architecture, used typically for performing server functions that require high availability and robust processing capabilities, or for high-end workstations that require massive processing power. UNIX and Linux come in a variety of “flavors” or distributions. Because of its high availability and low cost (free or almost free), Linux is the web server operating system of choice for many Internet service providers. Update history: A record of changes made to a web site or training course, which are organized by the dates when the changes were made. Usually stored in a read-me file. URL: Uniform resource locator. See “Pathname.” Usability test: A procedure that involves careful observation and recording of how the target end users interact with a Web site. Usability tests reveal the degree to which an identified audience finds a Web site “usable” given its needs and goals. User-centered design: Training or user interface design that is developed with the needs of the end-user in mind, rather than the preferences of the designer(s). User interface: A visual display for communicating and interacting with the computer. The user interface is the virtual “classroom” for WBT: what the web user sees and navigates to access information and to perform tasks. VBScript™: A scripting language developed by Microsoft that allows inclusion of dynamic and interactive content in web pages. VBScript applications can only be implemented within the Microsoft Internet Explorer™ web browser.

280


Vector image: A graphic image produced by mapping a mathematical algorithm to screen coordinates. Vector images are characterized by crisp, smoothly curving edges and are an alternative to rasterized, or bitmapped, images. Vector images can currently be displayed on the Web only with the use of a plug-in, such as Macromedia Director™. Vendor: A company that sells its services and products to another company. Virtual campus: A conceptual arena on the Internet where users can interact with each other to learn, chat, check course offerings and services, or be trained. W3C: See “World Wide Web Consortium.” WBT: Web-Based training. The integration of instructional practices and Internet capabilities to direct a learner toward a specified level of proficiency in a specified competency. Web circuit: A schematic diagram that maps the paths users can take through a web course, showing where each major element resides in the path, the routes by which they can be accessed, and how each element connects to other elements. Web master: An individual who maintains a web site and whose responsibilities might include managing a web server, maintaining network security, designing the web site, writing HTML files, monitoring user traffic, or responding to email. Web programmer: A person who uses scripting and programming languages to create interactivity and functionality within a web environment. Web programmers typically implement designs from graphic artists and develop web applications and database-driven web sites. Web-server administrator: A person who is designated to load data onto a web server. World Wide Web: A portion of the Internet, that encompasses web sites and web pages (excluding email, newsgroups, chat, etc.). Creation of the World Wide Web is generally attributed to Tim Berners-Lee. World Wide Web Consortium (W3C): A body of experts that reviews and approves protocols and standards for use on the World Wide Web.

Instructional Design for Web-based Training

Instructional Design for Action Learning

Instructional Engineering in Networked Environments (Instructional Technology and Training Series)

Multimedia-based Instructional Design: Computer-based Training, Web-Based Training, Distance Broadcast

Multimedia-based Instructional Design: Computer-Based Training; Web-Based Training; Distance Broadcast Training; Performance-Based Solutions, Second Edition

Instructional Design: Concepts, Methodologies, Tools and Applications

Instructional Methods for Public Safety

Training Design Basics

Cognitive Load Factors in Instructional Design for Advanced Learners

The ICU Book - Webbased

Learning to Solve Problems: An Instructional Design Guide (Tech Training Series)

Handbook of Improving Performance in the Workplace, Instructional Design and Training Delivery (Volume 1)

Strength Training for Sport

Dog Training For Dummies

Triathlon Training For Dummies

Dog Training For Dummies

Weight Training For Dummies

Spanish for Reading: A Self-Instructional Course

Planning and Design for High-Tech Web-Based Training

Weight Training For Dummies

Strength Training for Women

Adopting Information Technologies for Instructional Environments

Elementary Training for Musicians

Elementary training for musicians

Training For Dummies

Periodization Training for Sports

Elementary Training for Musicians

A Designer's Log: Case Studies in Instructional Design

Web-Based instructional Learning

Innovations in Instructional Technology

Curriculum, Plans, and Processes in Instructional Design: International Perspectives

Instructional Design for Web-based Training

Instructional Design for Action Learning

Instructional Engineering in Networked Environments (Instructional Technology and Training Series)

Multimedia-based Instructional Design: Computer-based Training, Web-Based Training, Distance Broadcast

Multimedia-based Instructional Design: Computer-Based Training; Web-Based Training; Distance Broadcast Training; Performance-Based Solutions, Second Edition

Instructional Design: Concepts, Methodologies, Tools and Applications

Instructional Methods for Public Safety

Training Design Basics

Cognitive Load Factors in Instructional Design for Advanced Learners

The ICU Book - Webbased

Learning to Solve Problems: An Instructional Design Guide (Tech Training Series)

Handbook of Improving Performance in the Workplace, Instructional Design and Training Delivery (Volume 1)

Strength Training for Sport

Dog Training For Dummies

Triathlon Training For Dummies

Dog Training For Dummies

Weight Training For Dummies

Spanish for Reading: A Self-Instructional Course

Planning and Design for High-Tech Web-Based Training

Weight Training For Dummies

Strength Training for Women

Adopting Information Technologies for Instructional Environments

Elementary Training for Musicians

Elementary training for musicians

Training For Dummies

Periodization Training for Sports

Elementary Training for Musicians

A Designer's Log: Case Studies in Instructional Design

Web-Based instructional Learning

Innovations in Instructional Technology

Curriculum, Plans, and Processes in Instructional Design: International Perspectives

Recommend Documents