Higher Education Institutions and Learning Management Systems: Adoption and Standardization

Higher Education Institutions and Learning Management Systems: Adoption and Standardization Rosalina Babo Instituto Sup...

Author: Rosalina Babo | Ana Azevedo

239 downloads 1983 Views 14MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Higher Education Institutions and Learning Management Systems: Adoption and Standardization Rosalina Babo Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Azevedo Instituto Superior de Contabilidade e Administração do Porto, Portugal

Senior Editorial Director: Director of Book Publications: Editorial Director: Acquisitions Editor: Development Editor: Production Editor: Typesetter: Print Coordinator: Cover Design:

Kristin Klinger Julia Mosemann Lindsay Johnston Erika Carter Myla Harty Sean Woznicki Jennifer Romanchak Jamie Snavely Nick Newcomer

Published in the United States of America by Information Science Reference (an imprint of IGI Global) 701 E. Chocolate Avenue Hershey PA 17033 Tel: 717-533-8845 Fax: 717-533-8661 E-mail: [email protected] Web site: http://www.igi-global.com Copyright © 2012 by IGI Global. All rights reserved. No part of this publication may be reproduced, stored or distributed in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher. Product or company names used in this set are for identification purposes only. Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI Global of the trademark or registered trademark.

Library of Congress Cataloging-in-Publication Data

Higher education institutions and learning management systems: adoption and standardization / Rosalina Babo and Ana Azevedo, editors. p. cm. Includes bibliographical references and index. Summary: “This book provides insights concerning the use of learning management systems in higher education institutions, to increase understanding of LMS adoption and usage”--Provided by publisher. ISBN 978-1-60960-884-2 (hardcover) -- ISBN 978-1-60960-885-9 (ebook) -- ISBN 978-1-60960-886-6 (print & perpetual access) 1. Internet in higher education--Case studies. 2. Education, Higher--Computer-assisted instruction-Case studies. 3. Web-based instruction--Design. I. Babo, Rosalina, 1967II. Azevedo, Ana, 1977LB2395.7.H55 2012 378.1’7344678--dc23 2011022151

British Cataloguing in Publication Data A Cataloguing in Publication record for this book is available from the British Library. All work contributed to this book is new, previously-unpublished material. The views expressed in this book are those of the authors, but not necessarily of the publisher.

Editorial Advisory Board António Andrade, Universidade Católica Portuguesa, Portugal Rosa Maria Bottino, Istituto Tecnologie Didattiche, Italy Dumitru Dan Burdescu, University of Craiova, Romania Adriana Schiopoiu Burlea, University of Craiova, Romania Luís Borges Gouveia, University Fernando Pessoa, Portugal Paulo Coelho Oliveira, Instituto Superior de Engenharia do Porto, Portugal Demetrios G Sampson, University of Piraeus, Greece Steve Wheeler, University of Plymouth, UK

List of Reviewers Kamla Ali Al-Busaidi, Sultan Qaboos University, Sultanate of Oman António Andrade, Universidade Católica Portuguesa, Portugal Judy van Biljon, University of South Africa, South Africa Rosa Maria Bottino, Istituto Tecnologie Didattiche, Italy Dumitru Dan Burdescu, University of Craiova, Romania Adriana Burlea, University of Craiova, Romania Nicholas Caporusso, Institute for Advanced Studies, Italy Lai Yiu Chi, The Hong Kong Institute of Education, Hong Kong Thomas Connolly, University of the West of Scotland, UK Dorota Dżega, West Pomeranian Business School, Poland José Manuel Mesa Fernández, University of Oviedo, Spain Robert W. Folden, Texas A&M University-Commerce, USA Jose Albors Garrigos, Universidad Polytecnica de Valencia, Spain Luís Borges Gouveia, University Fernando Pessoa, Portugal Malinka Ivanova, College of Energetics and Electronics, Bulgaria Arpan Jani, University of Wisconsi, USA Alexandros Karakos, Democritus University of Thrace, Greece Ray Kekwaletswe, Tshwane University of Technology, South Africa Arne Wolf Koesling, Leibniz Universität Hannover, Germany Marc Krüger, Leibniz Universität Hannover, Germany Atik Kulakli, Beykent University, Turkey

Jack Lee, The Chinese University of Hong Kong, Hong Kong Carla Lopes, Faculdade de Engenharia da Universidade do Porto, Portugal Lourdes Moreno, Universidad Carlos III de Madrid, Spain Muhammad Abdul Mugeet, Aga Khan University, Pakistan Lino Oliveira, Escola Superior de Estudos Industriais e de Gestão, Portugal Paulo Oliveira, Instituto Superior de Engenharia do Porto, Portugal Wieslaw Pietruszkiewicz, SDART Ltd, UK Mário Pinto, Escola Superior de Estudos Industriais e de Gestão, Portugal Ricardo Queirós, Escola Superior de Estudos Industriais e de Gestão, Portugal Marina Ribaudo, University of Genova, Italy Sandra Ribeiro, Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Cláudia Rodrigues, Escola Superior de Estudos Industriais e de Gestão, Portugal Lorenzo Salas-Morera, University of Córdoba, Spain Demetrios G. Sampson, University of Piraeus, Greece Anthony Scime, State University of New York, USA Errikos Ventouras, Technological Educational Institution of Athens, Greece Steve Wheeler, University of Plymouth, UK

Table of Contents

Foreword............................................................................................................................................... xv Preface.................................................................................................................................................. xix Acknowledgment...............................................................................................................................xxiii Section 1 Generalities and Perspectives Chapter 1 General Perspective in Learning Management Systems.......................................................................... 1 Robert W. Folden, Texas A&M University-Commerce, USA Chapter 2 Knowledge Sharing in a Learning Management System Environment Using Social Awareness......... 28 Ray M. Kekwaletswe, Tshwane University of Technology, South Africa Chapter 3 Learning 2.0: Using Web 2.0 Technologies for Learning in an Engineering Course............................ 50 Thomas Connolly, University of the West of Scotland, UK Carole Gould, University of the West of Scotland, UK Gavin Baxter, University of the West of Scotland, UK Tom Hainey, University of the West of Scotland, UK Section 2 Implementing and Evaluating Chapter 4 Evaluations of Online Learning Activities Based on LMS Logs........................................................... 75 Paul Lam, The Chinese University of Hong Kong, Hong Kong Judy Lo, The Chinese University of Hong Kong, Hong Kong Jack Lee, The Chinese University of Hong Kong, Hong Kong Carmel McNaught, The Chinese University of Hong Kong Hong Kong

Chapter 5 ANGEL Mining..................................................................................................................................... 94 Tyler Swanger, Yahoo! & The College at Brockport, State University of New York, USA Kaitlyn Whitlock, Yahoo!, USA Anthony Scime, The College at Brockport, State University of New York, USA Brendan P. Post, The College at Brockport, State University of New York, USA Chapter 6 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems.........116 Kamla Ali Al-Busaidi, Sultan Qaboos University, Oman Hafedh Al-Shihi, Sultan Qaboos University, Oman Section 3 Trends and Challenges Chapter 7 A Comparative Study on LMS Interoperability................................................................................... 142 José Paulo Leal, CRACS/INESC-Porto & DCC/FCUP, University of Porto, Portugal Ricardo Queirós, CRACS/INESC-Porto & DI/ESEIG/IPP, Porto, Portugal Chapter 8 Mobile Learning Management Systems in Higher Education............................................................. 162 Demetrios G. Sampson, University of Piraeus & Centre for Research and Technology Hellas, Greece Panagiotis Zervas, University of Piraeus & Centre for Research and Technology Hellas, Greece Chapter 9 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires................. 178 Dimos Triantis, Technological Educational Institution of Athens, Greece Errikos Ventouras, Technological Educational Institution of Athens, Greece Chapter 10 Disability Standards and Guidelines for Learning Management Systems: Evaluating Accessibility..........199 Lourdes Moreno, Universidad Carlos III de Madrid, Spain Ana Iglesias, Universidad Carlos III de Madrid, Spain Rocío Calvo, Universidad Carlos III de Madrid, Spain Sandra Delgado, Universidad Carlos III de Madrid, Spain Luis Zaragoza, News Service, Radio Nacional de España, Spain Chapter 11 The Technological Advancement of LMS Systems and E-Content Software..................................... 219 Dorota Dżega, West Pomeranian Business School, Poland Wiesław Pietruszkiewicz, SDART Ltd, UK

Section 4 Case Studies Chapter 12 Differences in Internet and LMS Usage: A Case Study in Higher Education..................................... 247 Rosalina Babo, Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Cláudia Rodrigues, NID-RH, ESEIG, Portugal Carla Teixeira Lopes, Faculdade de Engenharia da Universidade do Porto, Portugal Paulo Coelho de Oliveira, ISEP, Portugal Ricardo Queirós, KMILT, ESEIG, Portugal Mário Pinto, KMILT, ESEIG, Portugal Chapter 13 LMS Adoption at the University of Genova: Ten Years After............................................................. 271 Maura Cerioli, University of Genova, Italy Marina Ribaudo, University of Genova, Italy Marina Rui, University of Genova, Italy Chapter 14 Effective Use of E-Learning for Improving Students’ Skills............................................................... 292 Lorenzo Salas-Morera, University of Córdoba, Escuela Politécnica Superior, Spain Antonio J. Cubero-Atienza, University of Córdoba, Escuela Politécnica Superior, Spain María Dolores Redel-Macías, University of Córdoba, Escuela Politécnica Superior, Spain Antonio Arauzo-Azofra, University of Córdoba, Escuela Politécnica Superior, Spain Laura García-Hernández, University of Córdoba, Escuela Politécnica Superior, Spain Chapter 15 Strategies of LMS Implementation at German Universities................................................................ 315 Carola Kruse, Technische Universität Braunschweig, Germany Thanh-Thu Phan Tan, Technische Universität Braunschweig, Germany Arne Koesling, Leibniz Universität Hannover, Germany Marc Krüger, Leibniz Universität Hannover, Germany Compilation of References................................................................................................................ 335 About the Contributors..................................................................................................................... 360 Index.................................................................................................................................................... 370

Detailed Table of Contents

Foreword............................................................................................................................................... xv Preface.................................................................................................................................................. xix Acknowledgment...............................................................................................................................xxiii Section 1 Generalities and Perspectives Chapter 1 General Perspective in Learning Management Systems.......................................................................... 1 Robert W. Folden, Texas A&M University-Commerce, USA In order to properly understand learning management systems, it is necessary to both understand where they came from historically and the theoretical foundations upon which they are built. This understanding will allow for an effective comprehension of the elements that need to be involved in the development of these specialized management information systems that target the delivery of quality instruction at a distance. This chapter will attempt to lay that foundation. It will not cover every detail, but should provide the reader with enough background to be able to view these systems from the proper perspective. Chapter 2 Knowledge Sharing in a Learning Management System Environment Using Social Awareness......... 28 Ray M. Kekwaletswe, Tshwane University of Technology, South Africa The premise for this chapter is that learning and knowledge sharing is a human-to-human process that happen independent of space and time. One of the essential facets of learning is the social interaction in which personalized knowledge support is an outcome of learners sharing experiences. To this point, this chapter does not directly address a specific learning management system (LMS) platform but addresses forms of communication that can be encountered as tools of LMS platforms. The chapter argues that LMS ought to be able to facilitate the social interaction among learners not confined to particular places. Learners, because of their mobility, perform tasks in three varied locations or contexts: formal contexts, semi-formal contexts, and informal contexts. In this chapter, learners use social awareness to determine the appropriateness of an LMS tool to engage in a knowledge activity, as they traverse the

varied contexts. Thus, the chapter posits that a ubiquitous personalized support and on-demand sharing of knowledge could be realized if a learning management system is designed and adopted cognizant of learners’ social awareness. Chapter 3 Learning 2.0: Using Web 2.0 Technologies for Learning in an Engineering Course............................ 50 Thomas Connolly, University of the West of Scotland, UK Carole Gould, University of the West of Scotland, UK Gavin Baxter, University of the West of Scotland, UK Tom Hainey, University of the West of Scotland, UK Technology, and in particular the Web, have had a significant impact in all aspects of society including education and training with institutions investing heavily in technologies such as Learning Management Systems (LMS), ePortfolios and more recently, Web2.0 technologies, such as blogs, wikis and forums. The advantages that these technologies provide have meant that online learning, or eLearning, is now supplementing and, in some cases, replacing traditional (face-to-face) approaches to teaching and learning. However, there is less evidence of the uptake of these technologies within vocational training. The aims of this chapter is to give greater insight into the potential use of educational technologies within vocational training, demonstrate that eLearning can be well suited to the hands-on nature of vocational training, stimulate further research into this area and lay foundations for a model to aid successful implementation. This chapter discusses the implementation of eLearning within a vocational training course for the engineering industry and provides early empirical evidence from the use of Web2.0 technologies provided by the chosen LMS. Section 2 Implementing and Evaluating Chapter 4 Evaluations of Online Learning Activities Based on LMS Logs........................................................... 75 Paul Lam, The Chinese University of Hong Kong, Hong Kong Judy Lo, The Chinese University of Hong Kong, Hong Kong Jack Lee, The Chinese University of Hong Kong, Hong Kong Carmel McNaught, The Chinese University of Hong Kong Hong Kong Effective record-keeping, and extraction and interpretation of activity logs recorded in learning management systems (LMS), can reveal valuable information to facilitate eLearning design, development and support. In universities with centralized web-based teaching and learning systems, monitoring the logs can be accomplished because most LMS have inbuilt mechanisms to track and record a certain amount of information about online activities. Starting in 2006, we began to examine the logs of eLearning activities in LMS maintained centrally in our University (The Chinese University of Hong Kong) in order to provide a relatively easy method for the evaluation of the richness of eLearning resources and interactions. In this chapter, we: 1) explain how the system works; 2) use empirical evidence recorded from 2007 to 2010 to show how the data can be analyzed; and 3) discuss how the more detailed understanding of online activities have informed decisions in our University.

Chapter 5 ANGEL Mining..................................................................................................................................... 94 Tyler Swanger, Yahoo! & The College at Brockport, State University of New York, USA Kaitlyn Whitlock, Yahoo!, USA Anthony Scime, The College at Brockport, State University of New York, USA Brendan P. Post, The College at Brockport, State University of New York, USA This chapter data mines the usage patterns of the ANGEL Learning Management System (LMS) at a comprehensive college. The data includes counts of all the features ANGEL offers its users for the Fall and Spring semesters of the academic years beginning in 2007 and 2008. Data mining techniques are applied to evaluate which LMS features are used most commonly and most effectively by instructors and students. Classification produces a decision tree which predicts the courses that will use the ANGEL system based on course specific attributes. The dataset undergoes association mining to discover the usage of one feature’s effect on the usage of another set of features. Finally, clustering the data identifies messages and files as the features most commonly used. These results can be used by this institution, as well as similar institutions, for decision making concerning feature selection and overall usefulness of LMS design, selection and implementation. Chapter 6 Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems.........116 Kamla Ali Al-Busaidi, Sultan Qaboos University, Oman Hafedh Al-Shihi, Sultan Qaboos University, Oman Learning management systems (LMS) enable educational institutions to manage their educational resources, support their distance education, and supplement their traditional way of teaching. Although LMS survive via instructors’ and students’ use, the adoption of LMS is initiated by instructors’ acceptance and use. Consequently, this study examined the impacts of instructors’ individual characteristics, LMS’ characteristics, and organization’s characteristics on instructors’ acceptance and use of LMS as a supplementary tool and, consequently, on their continuous use intention and their pure use intention for distance education. The findings indicated that, first, instructors’ supplementary use of LMS is determined by perceived usefulness, training, management support, perceived ease of use, information quality, and computer anxiety. Second, instructors’ perceived usefulness of LMS is determined by system quality, perceived ease of use, and incentives policy. Third, instructors’ perceived ease of use is determined by computer anxiety, technology experience, training, system quality, and service quality. Furthermore, instructors’ continuous supplementary use intention is determined by their current supplementary use, perceived usefulness, and perceived ease of use, while instructors’ pure use intention is determined only by their perceived usefulness of LMS.

Section 3 Trends and Challenges Chapter 7 A Comparative Study on LMS Interoperability................................................................................... 142 José Paulo Leal, CRACS/INESC-Porto & DCC/FCUP, University of Porto, Portugal Ricardo Queirós, CRACS/INESC-Porto & DI/ESEIG/IPP, Porto, Portugal A Learning Management System (LMS) plays an important role in any eLearning environment. Still, the LMS cannot afford to be isolated from other systems in an educational institution. Thus, the potential for interoperability is an important, although frequently overlooked, aspect of an LMS system. In this chapter we make a comparative study of the interoperability features of the most relevant LMS in use nowadays. We start by defining a comparison framework, with systems that are representative of the LMS universe, and interoperability facets that are representative of the type integration with other broad classes of eLearning systems. For each interoperability facet we categorize and identify the most representative remote systems, we present a comprehensive survey of existing standards and we illustrate with concrete integration scenarios. Finally, we draw some conclusions on the status of interoperability in LMS based on our study. Chapter 8 Mobile Learning Management Systems in Higher Education............................................................. 162 Demetrios G. Sampson, University of Piraeus & Centre for Research and Technology Hellas, Greece Panagiotis Zervas, University of Piraeus & Centre for Research and Technology Hellas, Greece Learning Management Systems (LMS) are widely used in Higher Education offering important benefits to students, tutors, administrators and the educational organizations. On the other hand, the widespread ownership of mobile devices has lead to educational initiatives that investigate their potential as the means to change the way that students interact with their tutors, their classmates, the learning material, the administration services and the environment of their educational institute. This mainly aims to support the continuation of these interactions not only outside the classroom, but also beyond desktop restrictions, towards to a truly constant and instant access from anywhere. As a result, the development of mobile LMS (mLMS) is important for the deployment of feasible mobile-supported educational services in Higher Education. In this book chapter, we address the issue of designing mLMS for Higher Education by studying and applying the W3C Mobile Web Best Practices 1.0 to a widely used existing LMS, namely, the Moodle. Chapter 9 Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires................. 178 Dimos Triantis, Technological Educational Institution of Athens, Greece Errikos Ventouras, Technological Educational Institution of Athens, Greece The present chapter deals with the variants of grading schemes that are applied in current Multiple-Choice Questions (MCQs) tests. MCQs are ideally suited for electronic examinations, which, as assessment items, are typically developed in the framework of Learning Content Management Systems (LCMSs) and handled, in the cycle of educational and training activities, by Learning Management Systems

(LMS). Special focus is placed in novel grading methodologies, that enable to surpass the limitations and drawbacks of the most commonly used grading schemes for MCQs in electronic examinations. The paired MCQs grading method, according to which a set of pairs of MCQs is composed, is presented. The MCQs in each pair are similar concerning the same topic, but this similarity is not evident for an examinee that does not possess adequate knowledge on the topic addressed in the questions of the pair. The adoption of the paired MCQs grading method might expand the use of electronic examinations, provided that the new method proves its equivalence to traditional methods that might be considered as standard, such as constructed response (CR) tests. Research efforts to that direction are presented. Chapter 10 Disability Standards and Guidelines for Learning Management Systems: Evaluating Accessibility..........199 Lourdes Moreno, Universidad Carlos III de Madrid, Spain Ana Iglesias, Universidad Carlos III de Madrid, Spain Rocío Calvo, Universidad Carlos III de Madrid, Spain Sandra Delgado, Universidad Carlos III de Madrid, Spain Luis Zaragoza, News Service, Radio Nacional de España, Spain Currently, the great majority of institutions of higher education use Learning Content Management Systems (LCMSs) and Learning Management Systems (LMS) as pedagogical tools. In order to make these systems accessible to all students, it is important to take into account not only educational standards, but also standards of accessibility. It is essential to have with procedures and well-established method for evaluating these tools, so in this paper we propose a method for evaluating the accessibility of LCMSs and LMS based on a consideration of particular accessibility standards and other technological and human aspects. The method application is for all LMS, in order to illustrate the effectiveness of the evaluation method, we present a case study over the widely-used LMS Moodle . In the case study, the accessibility of Moodle is evaluated thoroughly from the point of view of visually-impaired persons. The results obtained from the case study demonstrate that this LMS is partially accessible. The evaluation shows that the tool provides poor support to the authors of accessible educational contents. Chapter 11 The Technological Advancement of LMS Systems and E-Content Software..................................... 219 Dorota Dżega, West Pomeranian Business School, Poland Wiesław Pietruszkiewicz, SDART Ltd, UK This chapter will present the practical aspects of Learning Management Systems adoption by describing this process from the perspective of evolution, observed for LMS and e-content software at West Pomeranian Business School. The chapter will address issues and found solutions relating to LMS deployment and evolution, noticed during the management of e-learning studies. In its first part, chapter will explain the requirements for different types of studies and how they influenced the shape of LMS systems. In the following sections, the chapter will analyze different technologies and software used in the e-learning process. This analysis will also describe how efficiently use the functionality of e-learning software in relation to the users’ requirements. The last part of chapter will present SPE - SDART Presentation Engine, being an innovative e-learning presentation engine, developed in form of Rich Internet Application, to overcome the limitations observed for the previously used presentation engines.

Section 4 Case Studies Chapter 12 Differences in Internet and LMS Usage: A Case Study in Higher Education..................................... 247 Rosalina Babo, Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Cláudia Rodrigues, NID-RH, ESEIG, Portugal Carla Teixeira Lopes, Faculdade de Engenharia da Universidade do Porto, Portugal Paulo Coelho de Oliveira, ISEP, Portugal Ricardo Queirós, KMILT, ESEIG, Portugal Mário Pinto, KMILT, ESEIG, Portugal The Internet plays an important role in higher education institutions where Learning Management Systems (LMS) occupies a main role in the eLearning realm. In this chapter we aim to characterize the Internet and LMS usage patterns and their role in the largest Portuguese Polytechnic Institute. The usage patterns were analyzed in two components: characterization of Internet usage and the role of Internet and LMS in Education. Using a quantitative approach, the data analysis describes the differences between gender, age and scientific fields. The carried qualitative analysis allows a better understanding of students’ both motivations, opinions and suggestions of improvement. The outcome of this work is the presentation of the Portuguese students’ profile regarding Internet and LMS usage patterns. We expect that these results can be used to select the most suitable digital pedagogical processes and tools to be adopted regarding the learning process and most adequate LMS’s policies. Chapter 13 LMS Adoption at the University of Genova: Ten Years After............................................................. 271 Maura Cerioli, University of Genova, Italy Marina Ribaudo, University of Genova, Italy Marina Rui, University of Genova, Italy The last two decades have seen the spread of LMS among schools, universities, and companies to augment the traditional teaching process with ICT and network technologies. This chapter presents the process leading to the adoption of a Moodle based LMS at the University of Genova in the last decade. By analyzing the data collected from the LMS logs and from questionnaires proposed both to students and teachers, we found out that the needs of the stakeholders are largely limited to resource sharing and organizational support, satisfactorily provided by the current service. Further improvements could be achieved by the introduction of a policy encouraging or forcing the teachers to use the provided LMS. A project on instructional design and, as a case study, the evolution of some of the courses involved in it are also presented. Though the redesign of such courses has improved their results, the impact on the overall organization of the degree program has been negative. We infer that this is due to the excessive freedom the students enjoy in taking their exams in Italy.

Chapter 14 Effective Use of E-Learning for Improving Students’ Skills............................................................... 292 Lorenzo Salas-Morera, University of Córdoba, Escuela Politécnica Superior, Spain Antonio J. Cubero-Atienza, University of Córdoba, Escuela Politécnica Superior, Spain María Dolores Redel-Macías, University of Córdoba, Escuela Politécnica Superior, Spain Antonio Arauzo-Azofra, University of Córdoba, Escuela Politécnica Superior, Spain Laura García-Hernández, University of Córdoba, Escuela Politécnica Superior, Spain The educational system promoted by the European Higher Education Area advocates the introduction of new teaching methodologies in order to improve students’ skills as well as their knowledge in the subject areas they are studying. In response to this, new teaching strategies were implemented in Industrial Engineering and Software Engineering degree courses. The main goal of the project was to improve students’ skills in areas including problem-solving, information management, group working and the acquisition of writing and speaking skills, by means of e-learning tools. In addition to implementing the new strategies, a set of assessments including surveys, forum activity analyses and group tutorial evaluations were also carried out. The combined use of these techniques proved a very useful way of improving the students’ general skills and knowledge, especially in terms of design methods and organisation and planning ability and in general academic performance. Chapter 15 Strategies of LMS Implementation at German Universities................................................................ 315 Carola Kruse, Technische Universität Braunschweig, Germany Thanh-Thu Phan Tan, Technische Universität Braunschweig, Germany Arne Koesling, Leibniz Universität Hannover, Germany Marc Krüger, Leibniz Universität Hannover, Germany In Germany, a learning management system (LMS) has become an everyday online tool for the academic staff and students at almost every university. Implementing an LMS, however, can be very different depending on the university. We introduce some general aspects on the strategies at German universities on how to implement an LMS. These aspects are mainly influenced by two main approaches, the top-down and bottom-up approach, which determine the decisions and actions on different levels at the university. In order to show how the strategies are carried out, we are presenting three case studies from universities based in the German federal state of Lower Saxony. We are going to reveal that both approaches play a part in each strategy, however differently weighted. It becomes clear that networking and collaboration plays a crucial role, not only concerning the technical development of the LMS software but also in organisational and educational terms. Compilation of References................................................................................................................ 335 About the Contributors..................................................................................................................... 360 Index.................................................................................................................................................... 370

xv

Foreword

Learning Management Systems (LMS) are now ubiquitous in institutions of higher education. This has occurred very rapidly with adoption being widespread but with little standardization. LMS’s were first used to support delivery with some communication between teachers and learners, but use has now been extended to support learning activities in innovative and diverse ways. They are also being used to increase student engagement and to track student progress – a vastly different approach to the early years of pushing resources to students. Adoption of LMS’s started with experimentation by a few with small systems. Familiarization facilitated wider adoption until eventually the “institution-wide” adoption of large commercial systems became common. Recent developments have extended the range of options from a few large commercial systems to a wider selection of open source, adaptable and specialized systems. In the intervening years from early adoption to now, higher education institutions have also gained valuable expertise in selecting, implementing, using and evaluating technologies to support learning and teaching some of which has been gathered in the chapters of this book. The authors have focused on a number of areas including: implementation strategies; use of learning management systems and other eLearning technologies; technical developments; evaluation; adoption and acceptance; and supporting skills. Chapter 1 takes a broad historical focus, reflecting on the background of eLearning from the very early days of teaching machines and computer assisted instruction through to correspondence courses and video conferencing. The role of information and communication technologies in supporting higher education processes, including teaching and learning, is explored leading to the overarching concept of learning management systems and how these support different modes of learning. The major conclusion reached is that pedagogy should drive the development and use of an LMS. Chapter 2 argues that learning and knowledge is facilitated by social interactions, implying that communication should be a key component of any eLearning system. The author argues that an LMS should facilitate social interactions at a number of levels independent of temporal or geographic constraints or the context that the knowledge activity takes place in. The communication should be facilitated by whatever means are available independent of specific LMS characteristics. The use of Web 2.0 technologies to support learning in a vocational setting is explored in chapter 3. The authors posit that despite the uptake of eLearning technologies in higher education generally, there is less evidence of uptake in the vocational sector. They aim to answer questions about whether technology can supplement the hands-on approach of vocational training, in particular the use of web 2.0 technologies such as wikis and forums. They present a case study the outcomes of which suggest that although there is potential for educational technologies to offer great benefits for vocational training, there is still much work to be completed in this area.

xvi

Chapter 4 investigates the use of LMS usage logs to facilitate teaching and learning. This is not an uncommon activity in commercial settings and, in an eLearning situation, can reveal valuable information about how the LMS is being used. The authors explore how the data can be analyzed to better inform their understanding of the online activities and, as a result, inform and improve the eLearning strategies that support institutional, faculty and departmental use of the LMS. In a similar vein, chapter 5 explores usage patterns in a specific LMS, ANGEL. Here data mining is used to explore LMS feature use. Using machine learning techniques it is possible to predict: the courses will use the ANGEL system most effectively based on course specific attributes; the interaction between features or sets of features that impact on usage; and those features that are most commonly used. This leads to a set of results that can be used by the institution to inform future decision making regarding feature selection within an LMS, the design, selection and implementation of an LMS, as well as identifying areas requiring additional training. Chapter 6 further explores usage, investigating critical factors that influence the acceptance and use of an LMS by instructors. The author suggests that although LMS survival is determined by instructor and student use, adoption is initiated by the instructor’s acceptance and subsequent use. Through a comparison of instructors’ acceptance of technology, LMS characteristics and organizational characteristics for the acceptance of eLearning, a model of overall acceptance and use is developed in a distance education setting. The study also provides insights into what additional support is needed in situations where computer use and Internet literacy is not high. In chapter 7 the issue of LMS interoperability with other systems in educational institutions is explored. The authors point out that although the LMS is directed at supporting learning it cannot be isolated from other systems in the institution. Two systems have been selected for the comparison representing a significant market share. A number of facets were selected for the comparison, using currently accepted standards, including: system communication with operational environment; learning content management; and academic management. The overall conclusion is that LMS interoperability leaves a lot to be desired. Standards relating to communication and content are relatively well developed but significant work still needs to be done in the area of interoperability of academic management. The advent of mobile technologies and the impact on course management systems in higher education is explored in chapter 8. The authors address the issue of designing such a system in the context of the Moodle LMS, specifically they investigate the application of W3C Mobile Web Best Practices 1.0. A framework is designed for a server-based mobile version of Moodle that follows the W3C guidelines. The enhancing of electronic examinations is explored in chapter 9. The authors explore an extension to multiple-choice questionnaires which allow for novel grading methodologies to be employed. They suggest that simple positive scoring rules and mixed-scoring rules introduce bias for a number of reasons. However the paired scoring method introduced here overcomes some of the shortcomings of these other methods. The authors do emphasis that the initial workload associated with constructing the question bank for multiple-choice questions is high, but overall it is concluded that the enhancements suggested here add to the value of examination tools within an LMS. With the widespread adoption and use of LMS’s in higher education, the authors of chapter 10 highlight the need for these systems to be accessible to all students. An evaluation framework is developed to evaluate the accessibility of LMS’s based on particular accessibility standards as well as other technological and human criteria. The framework is tested using the widely used LMS, Moodle, using the perspective of visual-impairment. A number of findings are reported, not least of which is that Moodle does not meet accessibly standards fully.

xvii

In chapter 11, the authors describe the legal requirements of using eLearning so support distance learning in higher education in Poland. They explore the implications of the regulations and how this has shaped LMS use. They compare the educational processes required of blended- and e-learning pathways and how different technologies can support aspects of those pathways. Extensions to the LMS that are required to support learning needs are discussed. The authors also point out that there are implications for the development of materials (e-content) which increase the cost of production on a number of levels. Finally the authors present an eLearning presentation engine that overcomes some of the difficulties associated with producing e-content. A comparison of Internet and LMS use is undertaken in chapter 12. The authors propose that more attention should be given to users’ Internet skills when developing LMS strategies since most LMS’s are web-based and most students are already adept at navigating the Internet before starting to use the LMS. The results support this proposition but also indicate that consideration should be given to using collaborative platforms such as web 2.0 technologies in place of email since many students are more familiar with using wikis and blogs for example. These outcomes can inform the selection and implementation of suitable digital pedagogical processes and tools that meet both teacher and student needs as well as informing the development of eLearning policies. Chapter 13 presents the process used by one higher education institution to determine an appropriate institution-wide LMS. The institution had a history of pockets of use and innovation to base the selection on. A subsequent evaluation of the adopted system has shown that use of the system is limited to resource sharing and organizational support, the most commonly used activities in the previous collection of systems. The authors suggest that the introduction of an eLearning policy could be instrumental in enhancing and extending the use of the LMS more fully. However they have identified that any extensions of use are beyond the current capacity of the system and support services, a catch-22 situation given the imperative of supporting the modern student digitally. A case study of using eLearning strategies to improve students’ generic skills is presented in chapter 14. Problem-solving, information management, group work and communication skills are the focus of this study. A combined strategy incorporating discussion boards, group tutoring, collaborative learning and peer assessment were implemented together with a number of assessment regimes including surveys, online activity analysis and group evaluations. These have resulted in improved student performance as well as improved perceptions of the accessibility of teachers, even though this is online. The study found that teacher participation is a key factor in motivating students to engage with learning activities as well as to lead discussions. The concluding chapter, chapter 15, explores different strategies that can, and have, been used when implementing an LMS. Two main influences are whether a top-down or a bottom-up approach is adopted. These determine the different levels at which key decisions are made. The authors present three case studies which demonstrate the different approaches. All three use a blend of the two approaches but in different mixes. All institutions eventually implemented the same system despite the different foci. Regardless of strategy it seems the key to success includes good networking and communication throughout the implementation process. Jo Coldwell Deakin University, Australia

xviii

Jo Coldwell joined Deakin in 1997 where she is currently Associate Head (Teaching and Learning) in the School of Information Technology. Before joining Deakin she gained a wealth of experience in both academia (in Australia) and industry (in both Australia and the UK). Jo was eLearning Manager for the Faculty of Science and Technology for a number of years during which Deakin University undertook the first institution-wide implementation of a major learning management system (WebCT Vista). During this time she was intimately involved in the deployment at University, Faculty and teaching levels. Since 2000 she has taught extensively online and her research interests lie in a number of areas associated with engaging tertiary teachers and learners in and with technology.

xix

Preface

E-Learning plays a significant role in education, and its importance increases day by day. Learning environments can take a myriad of distinct forms. Learning Management Systems (LMS) emerge as an important platform to support effective learning environments. According to Wang and Chen (2009), “an LMS employs a range of information and communication technologies to offer an online platform over the Internet, where a whole course can be planned, facilitated and managed by both the teacher and the learner”. In their work it is presented the main functions of some of the LMS nowadays available for educational purposes such as: learning material management, discussion forums, group emailing, audio conferencing, video conferencing, text chat, and whiteboard and synchronous document sharing. For Watson and Watson (2007), the term LMS is used to describe different educational computer applications. LMS is the framework that holds all sides of the learning process, including skills gap analysis. It is responsible to deliver and manage the infrastructural content, to identify and assess individual and organizational learning or training goals, to follow the process in order to reach those goals, and to collect and present data for supervising the learning process of an organization as a whole. With the rising of Web 2.0 and Web 3.0, learning environments are also overflowing Learning Management Systems and Institutions’ boundaries. Learning Management Systems are used all over Higher Education Institutions (HEI) and the need to know and understand its adoption and usage arises. On the one hand, there are different institutional cultures and characteristics and, on the other hand, there are several distinct LMS tools. Considering this it is expected to find out distinct experiences in the adoption and usage of LMS. The richness of each of the experiences can help the worldwide community to better understand how LMS are being used. The most used LMS according to a survey (Babo & Azevedo, 2009) answered by 51 universities from 19 different countries in 5 continents, were Moodle (Moodle, 2009), Blackboard / WebCT (Blackboard, 2009), and Sakay (Sakay Project, 2009). In that study several other LMS were referred such as ItsLearning, Desire2Learn, Claroline, METU Online, Chisimba, High Learn, Formare, Learning Space, First Class, Dokeos, eCollege, Class Fronter, KEWL. The results can be seen as an evolution. In the past years, the proprietary platforms were the most used but currently there is an increase of open source free platforms usage (Bradley et al., 2007). Consequently, there are not many studies regarding the usage level of such tools, concerning students, teachers, tools functionalities, usability, and the entire technological environment. Generally, both proprietary and open source free LMS provide several functionalities, such as, electronic distribution of course syllabi, grades and teachers feedback to students, ability to post hyperlinks to websites, forum for the exchange of ideas, wikis which allows students to swap ideas and information on projects, chat rooms for real time discussion, facilitating emailing and

xx

messaging among the participants (teacher/students, students/students), facilities for students to submit work assignments electronically, the means to administer quizzes and texts online (Janossy, 2008). It is frequent to observe that despite LMS on HEI is offered and usage stimulated, only a few of those functionalities are adopted, either by teachers, or by students. LMS are a powerful technology that has not achieved its full potential yet. As far as we know, understanding the actual aspects of LMS usage in HEI is an issue that is not sufficiently explored on research. Consequently, this is an interesting aspect to be explored and studied. The primary objective of this book is to provide insights concerning Learning Management Systems on Higher Education Institutions. The book aims to increase understanding of LMS adoption and usage providing relevant academic work, empirical research findings and an overview of LMS usage on Higher Education Institutions all over the world. The target audience of this book is composed of Education government members, Higher Education Managers, researchers, academicians, practitioners and graduate students in every field of study. LMS are not limited to a specific academic area being a trend and a new learning approach in any scientific field.

BOOK STRUCTURE This book includes fifteen chapters divided into four sections, namely, LMS Generalities and Perspectives, Implementing and Evaluating, Trends and Challenges, and Case Studies. It counted with the collaboration of researchers from 40 different universities and companies from 35 countries. Despite of the overall quality of the received proposals it was not possible to include all of them. As editors and after serious consideration during the review process, supported by our reviewers’ team and by the Editorial Advisory Board, we chose the best chapters in order to achieve the proposed goals of this book and those chapters which better fits the main focus of the book. In the first section Generalities and Perspectives Robert Folden very well understood the need of a general view of all the aspects related with the Learning Management System issue. One of the problems with much of the Scientific Literature is the assumption that all readers have the same background of the writers. This author very well assumes that is rarely true presenting the readers with the necessary foundations to go further in the field to be studied in depth. This is the chapter that any editor would appreciate to open with a book. Chapter 2 lies upon the premise of the asynchrony of the learning and knowledge sharing process which is a “human-to-human process that happen independent of time and space”. A reflection is presented towards “social awareness to determine the appropriateness of a LMS tool” considering asynchrony and ubiquity through all the process.Uncommonly a social concerning contributes from a technological university author. Both chapters are an excellent treatise of LMS in education. In the line of the two previous chapters, Chapter 3 provide us with an insight of the Distance Education evolution and the evaluation of the use of web 2.0 technologies offered by a chosen LMS in an educational context. Furthermore an interesting case study was developed by University of the west of Scotland and hereby presented. The use of e-learning in vocational courses is well explored in this chapter. Adoption and evaluation are two important and related issues regarding LMS that are presented in section 2. LMS store users’ logs in specific internal databases. These logs contain an immeasurable

xxi

wealth that can and should be used to evaluate LMS usage, in order to help Higher Education decision makers to take better decisions regarding LMS policies. Paul LAM, Judy LO, Jack LEE, and Carmel NAUGHT present, in chapter 4, an interesting study developed in the Chinese University of Hong Kong. The study took place during three years, between 2007 and 2009. The authors define three levels of analysis for the e-Learning activities, namely Popularity, Nature, and Engagement. Using these levels, it was possible to become aware of the different types of LMS usage and to define strategies in order to align its usage with the policies of the institution for e-Learning. Also using users’ logs analysis, but with a different even still very interesting perspective, Tyler Swanger, Kaitlyn Whitlock, Anthony Scime, and Brendan Post study, developed in The College at Brockport, State University of New York, is presented in chapter 5. The study was developed during the academic years of 2007 and 2008. Three data mining tasks, namely classification, clustering, and association, are implemented in order to extract useful knowledge and to obtain meaningful insights on LMS evaluation. This chapter refers to a new and interesting data mining application. In chapter 6, Kamla Ali Al-Busaidi, and Hafedh Al-Shihi present a model that intend to explain instructors’ acceptance of LMS. Despite that LMS usage depends on both students and instructs, it is up to the instructor to start the process and thus it is fundamental to understand which factors affect instructors’ acceptance and consequent use of LMS. The presented model is a valuable contribution in this direction. LMS usage in higher education is gaining momentum each day. As a consequence, new trends and challenges arise. In section 3, these issues are explored. José Paulo Leal & Ricardo Queirós explore, in chapter 7, some issues concerning LMS interoperability. In order to analyze and compare some of LMS interoperability features, a framework was developed and tested using Moodle and Blackboard. The framework defines two facets for LMS interoperability, exploring the main related issues in a stimulating, methodical, and efficient manner. Chapter 8 focuses on the use of mobile devices to access LMS supported courses. Demetrios Sampson & Panagios Zervas present a device developed in order to allow the deployment of LMS courses through online devices. This is advantageously achieved by means of the implementation and validation of a mobile version of Moodle that conforms to guidelines proposed by the World Wide Web Consortium. In chapter 9, Dimos Triantis & Errikos Ventouras contribute with the presentation of an interesting grading scheme applied in multiple-choice questionnaires. The presented grading scheme intends to prevent students from guessing the correct answers and thus develop a fair grading system. The proposed methodology is compared with more traditional ones in a real situation, bringing good insights to this issue. In chapter 10, the important theme of LMS accessibility is introduced. Lourdes Moreno, Ana Iglesias, Rocio Calvo, Nuno Delgado, & Luis Zaragoza, evaluated Moodle LMS, studying in detail accessibility concerns regarding visually impaired users. A new method was developed that leads to a better perceptiveness, which can conduct to the definition of better policies and practices. Dorota Dzega & Wieslaw Pietruszkiewicz presents, in chapter 11, a new technological solution designed to address some specific necessities of a higher education institution. Those necessities are also general necessities of a vast majority of higher education institutions. The referred solution is presented as an innovative extension of Moodle LMS, and consists of a new layer in the eLearning platform, offering additional advantages. The case Study section presents four case studies among Portuguese, Spanish, Italian and German universities. While the first three chapters present deep case studies whose main concern is the better understanding how LMS are being used, the German case study focuses in adoption strategies for LMS.

xxii

Chapter 12 characterizes the students Internet Usage and their LMS usage patterns in a Portuguese University. Chapters 13 and 14 case studies present long-lasting experiments and observation made in Spanish and in an Italian Universities. Chapter 13 presents the effort made by Genova University over the last 10 years in order to adopt “ICT educational support”. Next chapter relates the adoption of an elearning system which involved new teaching strategies and take into consideration student’s workload. The last chapter discusses the taken approaches – Top-Down or Bottom-up- of three German universities regarding the LMS implementation process. Not all the universities have had the same approach. Nevertheless the overall goals were achieved.

REFERENCES Babo, R., & Azevedo, Ana (2009). Learning Management Systems usage on Higher Education Institutions. In Proceedings of 13th IBIMA Conference - Knowledge Management and Innovation in Advancing Economies: Analyses & Solutions (pp. 883-889). Blackboard (2009). Blackboard. Retrieved July 30, 2009 from http://www.blackboard.com/ Bradley, M., Carter, J., Fitzsimons, D., Graham, J., Hurlbut, N., Marshall, D., et al. (2007). Learning Management System Evaluation Report. Executive Summary, Humboldt University. Janossy, J. (2008). Proposed Model Evaluating C/LMS Faculty Usage in Higher Education Institutions. Paper presented at MBAA Conference, Chicago, IL. Moodle (2009). moodle.org - Open-source Community-based Tools for Learning. Retrieved July 30, 2009 from http://moodle.org/ Sakay Project. (2009). Sakai Project Home. Retrieved July 30, 2009 from http://sakaiproject.org/portal Wang, Y., & Chen, N. S. (2009). Criteria for Evaluating Synchronous Learning Management Systema: Arguments from the Distance Language Classroom. Computer Assisted Language Learning, 22(1), 1–18. doi:10.1080/09588220802613773 Watson, W. R., & Watson, S. L. (2007). An argument for clarity: What are learning management systems, what are they not, and what should they become? TechTrends, 51(2), 28–34. doi:10.1007/s11528-0070023-y Rosalina Babo Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Azevedo Instituto Superior de Contabilidade e Administração do Porto, Portugal

xxiii

Acknowledgment

We have learned a great deal from those who lead us along our academic career and gratefully acknowledge our debt to them, especially Professor João Álvaro Carvalho. Our students have contributed in a fundamental way to our work with their editorial assistance, namely: Daniela Gonçalves, Joana Oliveira and Maria Duarte. To all the members of the Editorial Advisory Board, to the reviewers and to the authors, whose names are published in this book and who have assisted us one way or another, we feel very much grateful. We would like to express our thanks to the Publishing team at IGI Global for their expert support and guidance. This book is dedicated to our respective families. Rosalina Babo Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Azevedo Instituto Superior de Contabilidade e Administração do Porto, Portugal

Section 1

Generalities and Perspectives

1

Chapter 1

General Perspective in Learning Management Systems Robert W. Folden Texas A&M University-Commerce, USA

ABSTRACT In order to properly understand learning management systems, it is necessary to both understand where they came from historically and the theoretical foundations upon which they are built. This understanding will allow for an effective comprehension of the elements that need to be involved in the development of these specialized management information systems that target the delivery of quality instruction at a distance. This chapter will attempt to lay that foundation. It will not cover every detail, but should provide the reader with enough background to be able to view these systems from the proper perspective.

INTRODUCTION Before one begins an extensive study of any topic, it is to their advantage to view the topic in a general fashion. It is important to look at the historical development of the content, as well as, some the theoretical underpinnings of the subject. This helps to develop a healthy perspective for viewing the information that will be studied in depth. It can also lead one to consider those areas of greatest interest for future research. Beginning DOI: 10.4018/978-1-60960-884-2.ch001

with a birds-eye view, you will not initially see the fine detail of any individual topic or aspect of the material of interest, but you will gain an understanding of the directions that impacts have come from and being able to better understand where the subject is likely to go.

eLEARNING ROOTS In understanding any system, it is important to understand its roots (see Figure 1). As we look at the past we can understand the present and pos-

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

General Perspective in Learning Management Systems

sibly project the future (Rose, 2004). Without this knowledge, we may never understand the present and not be able to speculate effectively on where we are going (Rose, 2004). To actually understand the foundation of learning management systems, you must begin with a totally different domain of knowledge; that of psychology; most notably, educational psychology (Holmberg, 2005). One must also look at the developments that have occurred technologically (Ozkan, S., Koseler, R., & Baykal, N., 2009) (Wagner, N., Hassanein, K., Head, M., 2008). Where we are today is an outgrowth of where we have been and it is necessary to understand that path if we are to formulate a good sense of where we are going. There are multiple generations that we have come through (Taylor, 2001).

Programmed Learning/Teaching Machines At the beginning of the twentieth century, a group of psychologists were concerned with conditioning as an explanation of behavioral adaptation. They were generally referred to as ‘behaviorists’. They believed that all behavior (learning) could be explained by the concept of conditioning. ‘Learning’, as they saw it, could be accomplished by controlling the use of stimuli and rewards, both positive and negative. An outcome of this process was the development of programmed learning tools (Rose, 2004). Initially these tools were in the form of booklets that allowed the controller to manage the stimulus by applying the appropriate reward and thus produced the desired behavior/learning. These booklets were later moved to a mechanical device or teaching machine (Rose, 2004) (Keegan, 2002). The material was presented in very small steps that were referred to as a frame. The student was then presented with a blank to fill in after which they were provided with the correct answer. In their original conception they were not officially graded or ranked, but the student worked through the material. Based upon student performance, the

2

same material would have been repeated or new material presented. This form of instruction was very linear in nature and mastery was the end goal (Baggaley, 2008). Others provided some form of grading or required mastery before the student was allowed to progress. See http://www.greenchameleon.com/gc/blog_detail/weve_been_imagined/ for pictures showing students using a selection of these machines.

Computer Assisted Instruction When computers came on the scene in the 1950’s (Watson, W. & Watson, S., 2007), they became the teaching machines and the process was referred to as computer assisted instruction (CAI) (Rose, 2004). The foundation of this learning was individualized instruction (Rose, 2004). The theory was that individual students needed to learn at his pace and in his way. This system was also referred to as an Integrated Learning System (ILS) (Rogers, L. & Newton, L., 2001) (Underwood, 1997).The instructor ensured the proper design of materials so logical organization allowed the student to move through the material in an appropriate manner. This process required that each student had access to a computer for the appropriate amount of time for learning to occur. Originally students used ‘dumb terminals’ attached to mainframe computers. Each student could access his or her own files with the results stored in a centralized database. In the 1980’s and 1990’s intelligent computers in the form of PC’s assumed this role (Keegan, 2002) (Eteokleous-Grigoriou, 2009). These computers were eventually connected through local area networks and had the software stored on a centralized server. Students worked in a networked environment with the instructor moving about the classroom to help the students over the difficult portions and to keep the students on task.


Figure 1. eLearning roots

Computer Managed Instruction One of the early goals of computer assisted instruction was to free the instructor from the instructional process and enable her to become a coach for the students (Rose, 2004). Therefore, course systems were designed to allow the student to progress through a set of materials according to a plan established by the instructor (Brush, T., Armstrong, J., Barbrow, D., & Ulintz, L., 1999) (Underwood, 1997). To track the student’s progress and provide for a record keeping process, computer managed instruction (CMI) became available (Rose, 2004) (Underwood, 1997). This generally included

more than one course and allowed the students to move forward in the curriculum according to her individual time frame. Many of the CMI systems provided remediation modules that allowed the students to repeat instruction in concepts that they had not sufficiently mastered on previous attempts through the material (Underwood, 1997). Student scores, number of attempts, and other pertinent information were stored in a system database for the instructor or course administrator to view (Gilman, D., Emhuff, J., Bender, P., Gower, A., & Miller, K., 1991) (Brush, T., Armstrong, J., Barbrow, D., & Ulintz, L., 1999) (Underwood, 1997).

3


Correspondence Course Learning (CCL) Beginning in the mid-nineteenth century, students were able to study in remote locations using correspondence courses (Gaspay, A., Dardan, S., & Legorreta, L., 2008) (Homberg, 2005). These text-based programs allowed the students to receive written information and a series of questions or projects that tested or reinforced the desired learning. In most cases the students worked independently (Taylor, 2001). Instructor involvement came in the form of written communication to students after they had submitted their work for grading (Holmberg, 2005). In the 1970’s and 1980’s, these courses were enhanced with audio or video recordings to provide the student with some content that was not easily communicated by words alone. In most cases the students and instructors were not in the same location, making this a form of distance learning (Keegan, 2002). This was an attempt to enable students who could not afford, for one reason or another, to travel to the instructor to receive an education. Again, this was basically individualized instruction, but it was not as individualized as the Programmed Learning, CAI, or CMI.

Instructional Television Fixed Service (ITFS) Instructional Television Fixed Service was developed beginning in the 1950s and greatly expanded in the 1970’s and 1980’s as a course delivery platform (Saba, 2000). The students participated in classes that were often remote from the instructor. There was usually an instructor present with one group of students while the others were at one or more remote sites. In many cases this was two-way video and audio, but was mostly one-way video with two-way audio. The audio in those cases was done over an audio bridge. This allowed the students to see the instructor and interact in much the same was as they would in a

4

contained classroom environment. This approach used broadcast television to send the video and the instructor audio (Gaspay, A., Dardan, S., & Legorreta, L., 2008). Because of its use of the television broadcast medium, it tended to be quite expensive. In the 1980’s and 1990’s this was enhanced and somewhat replaced by the sending of videotapes to the students to complete their coursework (Taylor, 2001). It was a blend between correspondence courses and ITFS. While this helped to bring the cost down, it did not provide what either the instructor or the student preferred and so did not do well in the long run.

Video Conferencing (VC) This approach gave way in the 1980’s and 1990’s to video conferencing systems (Taylor, 2001). These conferencing systems provided two-way video and audio links between multiple locations. In most cases this was compressed video and an audio bridge (Baggaley, 2008). While the students could see and hear, the quality often left much to be desired. Over time, the systems improved and use increased. They were hampered by the inability of systems to communicate with other manufacturers’ systems. Theoretically, they all complied with a uniform set of standards that should have made communication possible, but each manufacturer included proprietary algorithms that made it virtually impossible. During the latter part of the decade, there was more cross-collaboration, but it was never an effective method of allowing remote students to participate in a classroom.

CIRCLE OF FOCUS Classroom Based Learning Programmed learning, CAI, and CMI focused on improving classroom teaching. They were basically built upon the model of behaviorism which emphasized individualized instruction


with instructor interaction. The focus was upon improving the learning of students in a contained environment with synchronous learning. This can be seen as allowing technology to find a place in the classroom. While there was talk about replacing the instructor, it was mostly a changing of the role of the instructor. Instead of the instructor being the direct dispenser of knowledge, the instructor became the facilitator/coach in the learning environment.

Distance Based Learning Correspondence course learning, video conferencing, and ITFS focused on distance education (Guri-Rosenblit, 2005). They were attempting to solve the problem of meeting the needs of students who could not or would not participate in a selfcontained classroom (Wagner, N., Hassanein, K., Head, M., 2008) (Wagner, N., Hassanein, K., Head, M., 2008). With students distributed over great distances (Guri-Rosenblit, 2005), there was little or no interaction with one another. Their only source of interaction was with the instructor or the instructor-assigned mentor/proctor. While ITFS and video conferencing theoretically allowed for some interaction, it rarely occurred because everything generally came through the instructor and that tended to stifle any student to student interaction. In some cases, distance-based learning was mated with CAI to form a sort of a hybrid approach. The students would complete computer simulations or drill and practice but also participate in video conferences to receive input from their instructor. Methodically, distance education has adapted by using the ever increasing improvements in the information technology to meet the needs of the students and improve the interaction between and among students and faculty (Baggaley, 2008).

INFORMATION COMMUNICATION TECHNOLOGY (ICT) While all of this technology development within education was going on, information communication technology was developing rapidly (Garrison, 2000). The development of the mainframe computer allowed companies and schools to automate business knowledge acquisition, storage, and processing. It led to many improvements in the functioning of the business side of the educational enterprise. Computers began to invade every aspect of business and then began to spread throughout the lives of individuals. As a result, it is nearly impossible to find a major device that does not possess at least a rudimentary computer. Also, individuals often depend on computers to allow them to access information and share information in an open and free fashion (Hamuy, E. & Galaz, M., 2010). Most individuals find it so difficult to carry on a normal life without computer interaction that we are in the process of making wireless internet a ubiquitous factor in our lives (Gaspay, A., Dardan, S., & Legorreta, L., 2008).

Networking During the 1980’s and 1990’s personal computers became networked with one another so that they could share information across platforms. They were able to present information in a graphical manner. This provided for a more enriched environment for the delivery of instruction (Rose, 2004) (Gilman, D., Emhuff, J., Bender, P., Gower, A., & Miller, K., 1991). The systems could blend audio and video in such a manner as to appear seamless. They also allowed for greater interactivity with the students (Gilman, D., Emhuff, J., Bender, P., Gower, A., & Miller, K., 1991). Because they were networked, students could now work on any available computer to continue the work that they had begun previously. Instructors could view individual student work unobtrusively, providing feedback as necessary. Students could participate

5


in elaborate collaborative environments to develop their individual learning. As the computer continued to improve in capability, the range of activities that the students could participate in continued to expand.

Internetworking In the mid-1990’s we saw the development of internetworking with the introduction of the World Wide Web (WWW) (Keegan, 2002). This allowed networks to communicate with other networks far removed from the original source in a reliable fashion (Rose, 2004). This whole development coincided with the development of the graphical user interface (GUI) and the use of media rich environments for the delivery of information. Technology was developing at a very rapid pace, with processors able to handle larger workloads at much faster rates than ever before. Storage space was cheaper and cheaper to provide (Gaspay, A., Dardan, S., & Legorreta, L., 2008). Sound and video delivery systems could provide greater enrichment for less expense. These developments allowed the designers to enrich their information delivery, ignoring most of the limitations of the past (Taylor, 2001). Course material could be asynchronously delivered to many individuals in an individualized manner or synchronously delivered to large masses at one time or in some other combination (Rose, 2004) (Wang, Y. & Chen, N., 2009). As the Internet became ubiquitous, the barriers to delivery of information fell away, allowing for ubiquitous access to the information (education) provided (Taylor, 2001). Students began using computers more and more in their everyday lives. They became ever more comfortable with computer delivery of information in text, video, and audio. This familiarity made it easy to move educational delivery to this platform (Keegan, 2002). Students were able to use the Internet to seek out and obtain information on their own with or without the guidance of an instructor/mentor

6

(Rose, 2004). The technology could fade into the background and allow the course material to move to the forefront, regardless of the overall purpose of the information delivery. The nature of the information delivered was more a matter of perspective than one of substance.

E-Learning/Virtual Learning This ability to move education materials electronically around the world in both synchronous and asynchronous manners created the ability to develop a classroom irrespective of time, space, and proximity (Miller, T. & hutchens, S., 2009). Even with VC and ITFS, you needed to be in fairly specific locations to participate in class activities (Guri-Rosenblit, 2005). While Programmed Learning, CAI, and CMI could be done remotely, they usually were done with the students in one location connected to the same network and using the same hardware system (Baggaley, 2008) (Guri-Rosenblit, 2005). Today, all of those limitations seem irrelevant (Dillenbourg, 2008). Operating on the platform of ICT, educational institutions are delivering all levels of education and training over the Internet (Hamuy, E. & Galaz, M., 2010) (Ozkan, S., Koseler, R., & Baykal, N., 2009). Corporations seek to reduce their training expenses by providing training over the Internet rather than sending their employees to far off places with the related expenses. Even conferences have moved to the Internet to provide similar types of experiences at the attendees’ desktops rather than requiring them to attend the conference at some exotic location. These are also recorded and made available in an asynchronous manner for those who could not or would not attend when the event was going on. ICT is more than just a delivery mechanism; it has become a shaper of what we do (Rose, 2004). It reaches into every area of our lives and has touched nearly every individual on the face of the earth (Guri-Rosenblit, 2005). As the devices that access and share the information have become


ever smaller, they have transformed more and more of our lives (Taylor, 2001). They have become the medium for developing communities far and wide. Individuals can find out even the most mundane or inconsequential pieces of information in a split second (Garrison, 2000). They can also share the most intimate details of their lives with the world just as quickly. These tools also allow instructors and students to share with one another in near-real-time, no matter where they are located. Information is not limited to text or even text and still images. It is possible to share full motion video and audio in very high quality over very great distances. These videos can be of real events or simulated reality. This has opened the way to virtual reality to function in an educational environment. This pervasiveness of ICT is forcing academic institutions at all levels to move their education offerings to an electronic platform to some extent (Ozkan, S., Koseler, R., & Baykal, N., 2009) (Eteokleous-Grigoriou, 2009). It does not require, at this point, that everything be done in an electronic medium, but it does mean that some of it must (Taylor, 2001). This push is coming from multiple sources; from almost every stakeholder group that is involved in the educational process. That pervasiveness also means that instructors have a deeper well of knowledge about their students than ever before (Eteokleous-Grigoriou, 2009). While electronic media allow a degree of anonymity, it also opens up a greater breadth of information about each participant to all of the other participants in the educational process (Dillenbourg, 2008).

INFORMATION COMMUNICATION TECHNOLOGY AND LEARNING MANAGEMENT SYSTEMS For this discussion, we will focus on academic institutions and the systems that are involved there. While they are much the same as those of

other businesses, this focus will enable a clearer correlation to learning management systems in the context that we are viewing them. We need to remember that nearly all of these ICT systems existed in paper form before they appeared in electronic form. Most of what was done in the early systems mirrored what had been done in paper forms. The basic purpose of moving these systems to digital format was to automate the redundant activities in order for the users to focus on those aspects that could not be automated, thereby creating performance improvement (Rentroia-Bonito, A., Martins, A., Guerreiro, T., & Jorge, J., 2008) (Jones, 2004). As the systems have developed, they have increasingly moved into areas that were not possible to do or were easily done in print form (Eteokleous-Grigoriou, 2009) (Cradler, 2008) (see Figure 2).

Finance Information Systems (FIS) The first ICT system to be developed was the finance information system. This system allowed enterprises to track income and expenses (Paulsen, 2002) and use the information to make informed decisions. It is sometimes referred to as an accounting information system (AIS). It has been defined as: A system “ . . . that processes financial transactions to provide (1) internal reporting to managers for use in planning and controlling current and future operations and for non-routine decision making; (2) external reporting to outside parties such as to stockholders, creditors, and government agencies.” (Answers.com, accessed on 08-18-2010) Information was generally input using batch files and outputs were done in much the same way. As technology improved, it was possible to do input during real time and to schedule major outputs during the off hours so as not to degrade the performance of the system. These systems generally are comprised of three major components; the database and its front end, the control system, and the reporting system. The database stores all of the essential information and

7


Figure 2. Information Communication Technology (ICT)

provides the input screens (front end) that allow the users to input the information. That information may be added to the database in real time or stored in a temporary ledger to be added at a more efficient time. Generally, if it is stored in a temporary ledger, it is added to the database at the close of business and the information will be available for reports after that time. The databases also allow for locking of the records so that they cannot be changed in any fashion without creating a record of the change (a part of the control system). The control system allows for the ability to develop appropriate controls on the system and users to ensure the integrity of the information contained in the system. The reporting system allows for the processed information to be displayed in an appropriate manner for the decision makers and other interested parties.

8

Faculty Information System (FIS) Faculty information systems are basically a human resource information system (HRIS) that allows an institution to maintain information on nearly everything that needs to be tracked or analyzed about faculty (current and former) and those who are applying for positions. It is able to be maintained by professionals in human resources and by the faculty themselves in order to maintain the most accurate and up to date information. Not only does it maintain information on the classes taught by each faculty member, but their academic credentials and relevant experience as well. It provides a place to maintain a record of their scholarly activity and their service involvement. These systems also allow for the publication of documents necessary for the faculty to perform their duties in an efficient and effective manner.


Initially these systems did not communicate with any other systems, but more and more these systems are integrated with all of the information systems of an institution so that information only needs to be input one time.

Library Information System (LIS) The initial library information system focused on the card catalogue and the record of where an individual item resided. It was basically a large inventory tracking database for library holdings. It served to automate what a librarian was responsible to do in order to maintain control of the assets of the library. As more and more assets became electronic in nature, it also began to be a repository of those items and became then a digital library system. This system now makes it possible for patrons to access the resources of the library without having to be physically present. It also enables libraries to share their resources with one another in a seamless fashion; enabling their patrons to have access to a much broader set of materials than would be possible otherwise. While LISs served to just automate library processes in the beginning, they have now entered into the process of transforming the way that the library delivers information and the way that patrons access information. With LISs, libraries have moved from being islands of information to becoming portals to the information stored in the network of libraries worldwide.

Student Information System (SIS) Student information systems provide the ability to enter student information into a database that will provide an electronic grade book, student course schedules, and other student-related data needs for a school, college, or university (Paulsen, 2002). These systems grew out of the need to automate the various record keeping responsibilities of an academic institution. They have grown to the point of being able to provide advanced informa-

tion to the decision makers on par with enterprise resource planning systems or the HRIS systems. It is rare for an institution of any size to not have some form of an SIS system. While many of these were developed in-house initially, most are commercially available today. In the K-12 market, these systems are being outsourced to a hosted environment allowing for many other enhancements to be included in the SIS system. Most of the commercially available systems today provide the ability to integrate with all of the other ICT systems functioning in an academic institution.

Learning Management System (LMS) In a simple way learning management systems involve the application of ICT to the process of education, but it is a bit more complicated than that (Dillenbourg, 2008). An LMS provides the structure for the delivery and management of instructional content, while providing assessment of individual and organizational learning goals, tracking of progress toward those goals, and providing the data necessary for the management of the learning process of the institution as a whole (Watson, W. & Watson, S., 2007) (Kim, S. & Leet, M., 2008) (Rogers, 2001) (Perry, 2009). In today’s educational environment ICT is often involved in all types of learning systems (see Figure 3). The term was first used to refer to the management system of the PLATO K-12 learning system (Watson, W. & Watson, S., 2007). Today, there are basically four broad system designs used, as seen in the accompanying diagram. These designs are determined by the pedagogical focus of the education enterprise.

Traditional Learning Historically, this form of learning has been the staple for hundreds of years, from the very founding of higher education institutions. Students meet together in real time and in a specified location with the instructor present. This instructor cen-

9


Figure 3. Learning system models

tered approach is the predominant modality of instruction in higher education. ICT can be used to enhance the educational experience with the use of technology delivering the instruction and maintaining student records. The instructor is the primary determinant of what use will be made of ICT in the delivery of instruction and to some extent to its use in the completion of assignments required for the instruction. Most information will be supplied by an instructor directly or in preprinted materials, but may be supplemented with various forms of ICT in the classroom. The students may utilize ICT outside of the classroom to produce class assignments, submit class assignments, or locate information for class, but the primary delivery of instruction is done in a face to face mode. This instruction is delivered synchronously.

10

Distance Learning Distance education dates from the mid nineteenth century. Students and instructors are separated by time, location, or both in this model of instruction (Gaspay, A., Dardan, S., & Legorreta, L., 2008). The materials may be delivered to remote locations via print or ICT, but this form of instruction does not preclude the use of remote classrooms. This form of instruction may be done synchronously or asynchronously. The emphasis of this mode of instruction is the separation of the student from the instructor and not the delivery mechanism (Gaspay, A., Dardan, S., & Legorreta, L., 2008). It does not matter that the separation is physical or temporal, just that there is some separation between the two participant groups. This mode of instruction does not focus on the delivery mechanism or the technological involvement. The


focus may be the individual or a group (Keegan, 2002). Early forms of this modality focused on individual instruction (correspondence courses, etc.), with technology enabling group instruction beginning in the 1980s. This form of instruction was not highly valued until the advent of open universities in the 1970s. Most distance education in the United States is group focused, while most distance education in Europe is individual focused (Keegan, 2002). Regardless of the focus, most materials are of the pre-prepared variety. Instructors may utilize materials that were prepared by another or even by a group of others. The teaching materials may be produced up to ten years before the student interacts with them in a learning environment. Furthermore, the institution that created the materials may not be the institution that is awarding credit for the course completion. There exist clearing houses for courses that can be combined to create distance learning programs of study.

Blended Learning In the blended learning mode, parts of the instruction are delivered in a traditional format while other parts of it are delivered using ICT (Gaeta, M., Orciuoli, F., & Titrovato, P., 2009). It is this blending of the delivery modality that attempts to use the strengths of both formats to enhance the educational experience (Dillenbourg, 2008). ICT is more than just an adjunct to the process, but the students and instructor are required to utilize ICT in order for learning to occur properly. This form requires that parts of it are synchronous and parts are asynchronous. One form of blended learning is called an integrated learning system (ILS) in which networked computers or terminals with a management system monitors and records student performance results and distributes learning modules based upon those results (Dillenbourg, 2008) (Brush, T., Armstrong, J., Barbrow, D., & Ulintz, L., 1999). These are truly digital systems and can be used over a dis-

tance; they are most generally used as an adjunct to traditional learning. The primary function of these systems is to remediate performance deficits in basic skills. This form of blended learning is much more like the programmed learning roots from which all of this grew. These systems contain a management system that controls the flow of data between the other components, curriculum content that provides the tutorial, practice and assessment modules, and the student record system that maintain registration and performance information on every student enrolled in the system (Rogers, 2001). These are not usually integrated with the ICT systems of the institution. Their structure mirrors the structure of learning management systems without the broad connectivity found in those systems. Some examples of ILS are CentraOne, IntraLearn, Lyceum, and Silicon Chalk, Odyssey, and Plato (the same company that originated the name learning management system).

E-Learning E-Learning had its beginnings in late 1994 or early 1995 and really soared after 1996 when the first modern content management system was developed. In E-Learning all of the instruction is online using ICT to deliver the content. By 1998, it was viewed a mature field of distance education. Michigan Department of Education defines this as “A combination of structured, sustained, integrated, online experiences accessed via a telecommunications network.” This may involve synchronous and asynchronous delivery of instruction. It blends aspects from all of the other models into a model that is not just ICT enhanced, but one in which it would not occur without the use of ICT. In today’s implementation of E-Learning, ICT is transformative and not just supportive of the process of education (Gaspay, A., Dardan, S., & Legorreta, L., 2008). In some circles, E-Learning is used to refer to all learning that occurs through the mediation of ICT, but I use the term more formally to refer to structured/

11


purposeful learning that utilized ICT for its development, delivery, and management in such a way that the instruction is transformed by the medium in which it occurs. It is much similar to the distinction between learning that occurs through independent effort of the student and that which is accomplished through the collaborative effort of instructor and student. Learning happens in both situations, but in the latter, it is substantially different than it would be otherwise because of the nature of the delivery medium. While a learning management system could be utilized in some fashion in each of these models, it is in E-Learning that it is an absolute necessity. ICT in all of these models could be referred to as learning support systems. The problem is that there are no universally accepted definitions for these learning systems that will serve to distinguish them from other ICT systems used in the educational process. For this reason, the terms are used haphazardly to refer to various systems or parts of the system as if they were the same.

COMPONENTS OF A LEARNING MANAGEMENT SYSTEM In a nutshell, LMS is the software that automates the administration and delivery of learning (Watson, W. & Watson, S., 2007) (Baturay, 2008) (Rogers, L. & Newton, L., 2001). It is the overarching ICT that provides all of the functions necessary to provide learning in a digital format at a distance (Wang, Y. & Chen, N., 2009). This does not preclude the ability for it to also serve in a blended learning environment, but it must be able to provide that learning where there is spatial, temporal separation or both. The system may have any number of components, but we will look at the major components that one should expect to find in a modern learning management system. It is important to remember that the technology is generally learning theory independent and can be utilized to function in most any theoretical

12

system (Jones, 2004). However, one must consider theoretical orientation when considering the appropriateness of any learning management system (Wang, Y. & Chen, N., 2009). While this seems pretty straight forward, there is much confusion as to what is meant by the various systems and components of systems. Some authors state that CMSs are LMS that are used by academic institutions while LMS are used in industry (Hamuy, E. & Galaz, M., 2010) (Daniels, 2009). Course management systems are also confused with content management systems, which both use the same acronym, CMS (Daniels, 2009). Course management systems and virtual learning environments (VLE) are basically the same thing with terms differing by the region in which they occur (Daniels, 2009). Course management system is the term used in North America and VLE is the term used in Europe (Daniels, 2009). Content management systems are generally systems used in industry to publish information to various audiences to support their work processes. A related term is learning content management system. It refers to the system that delivers the specific portions of learning content to the learner during the use of the LMS. We will not be referring to content management systems apart from learning and so CMS will always refer to course management systems in this chapter.

Course Management System (CMS) The CMS provides the linkage with all of the other systems involved in the learning endeavor. It is truly the brains of the whole system. This system allows the appropriate individuals to add or remove courses, to sequence courses within a curriculum, add students to the course, assign instructors to individual courses or sections of courses, to monitor other processes of the system, etc. (Watson, W. & Watson, S., 2007) (Daniels, 2009). This is probably the most frequent system confused with LMS. Many individuals believe that is all an LMS does and so what is the difference. To do its


job, this system must interface with the FIS and SIS systems. It is also often linked with the LIS system. These linkages are often rudimentary and not well established. This lack of strong linkage can cause considerable problems to the end users and the institution alike. When it is functioning well, it is like a good traffic officer seeing that everyone and everything gets to where it needs to be in the most efficient manner.

Learning Content Management System (LCMS) The primary purpose of the LCMS is to develop, store, organize, and distribute multimedia content to support the delivery of E-Learning (Watson, W. & Watson, S., 2007). This over arching system has many subsystems that allow it to perform its duties. Most of these occur in the background and the users do not see them or even need to. Course content authoring is one of these background activities. While the instructor uses this system to design the course and upload the appropriate content, they really don’t need to understand what is being done to make it happen. Once the material is stored, usually in the form of learning objects (Watson, W. & Watson, S., 2007), it needs to be accessed at the appropriate time to support student learning (Watson, W. & Watson, S., 2007). These processes need to function effectively as purposed or they will negatively impact the quality of the learning delivered.

Collaborative Learning System (CLS) The CLS is the system that allows the use of the newer aspect of Web delivered content (Web 2.0). This system provides the tools for email communication, discussion groups, newsgroups, instant messaging, blogs, bookmarking, notice board, search tools, etc. Not all LMS have these capabilities, but these and many more are managed by the CLS. This is the part of the LMS that changes the most rapidly. Many times the capabilities in this

system are accomplished by third party apps that are added to the system piece meal. This process can create some very unusual difficulties if the system is not managed effectively. This is also the area where mobile connections are usually made and managed. It is within the CLS that many instructors will develop the social learning that should be part of any learning program that seeks to meet the needs of a diverse student population. However, just using collaborative tools without the requisite understanding and application of social learning theory will not produce the desired results. The importance of this area is supported by belief by some educational practitioners that only 20% of learning occurs based upon formal instruction and that the other 80% is produced by informal instruction that occurs when students interact in social environments involving each other and the instructor.

Assessment Management System All forms of assessment are managed from within assessment management system. This is also where the grade book is located. Assessments could be exams, homework, projects, etc. They could involve student submissions, as well as student participatory activities. The purpose of this subsystem is to ensure that proper delivery and recording of results for all assessment items is maintained. The system should provide the instructor with the tools necessary to assess student performance on the appropriate measures. It should allow the instructor the ability to design these measures with some degree of flexibility. The assessment management system should enable the instructor to provide the student with adequate and timely feedback on their performance so that the student will benefit from the learning and be able to improve the desired outcomes of the learning experience (Watson, W. & Watson, S., 2007).

13


LEARNING MANAGEMENT SYSTEM STAKEHOLDERS There are basically four stakeholder groups that are directly involved in LMS (see Figure 4), but there is a fifth that is peripherally/externally involved. The four primary/internal stakeholders are students, faculty, administrators, and IT staff (Wagner, N., Hassanein, K., Head, M., 2008). The peripherally involved stakeholder group is that group who will hire the students once they complete the education using the system or are involved in accrediting the education institution to be able to offer a degree or credential. Each of these groups holds slightly differing sets of wants and needs, but they also share some commonalities.

Students Students are the ultimate consumers of the LMS output. Generally speaking they are either undergraduate or graduate students of the educational institution. They may be enrolled in one or more courses delivered by the LMS. While there are a number of motivations for their using an LMS, some of the primary ones are access to an education, convenience, and, for a few, because they prefer this mode of learning (Wagner, N., Hassanein, K., Head, M., 2008). Some will be using an LMS because there is no other way to complete the program of study that they desire. In a sense, then, this relates to access. Modern students have grown up with an increasing involvement with technology in their day to day lives. Few, if any, know of a time when they did not have access to the internet to further their contact with one another and with the information that is available from a broader perspective, especially in the western world. Most students today have an extensive background with ICT and use it on a daily basis. This technological background of these students will impact what they expect from any delivery system (Wagner, N., Hassanein, K., Head, M., 2008). It will also play a major role in how they

14

are able to utilize the system to improve their educational outcomes and to meet the goals that they set for themselves in the process, as well as the goals that may be set by the faculty and administrators.

Faculty Instructors, more than anyone else, determine the nature of the E-Learning experience (Wang, W. & Wang, C., 2009). Depending on their theoretical foundation, they play a larger or smaller role determining the educational experience of the students. Generally, they are the arbitrators of whether or not the students will participate in group activities, independent study, or synchronous events, to just name a few ways in which they control the E-Learning experience. If their theoretical persuasion is behaviorist, they may design instruction in such a way as to move the student in a structured manner to the desired outcome, but if they are constructivists, they will allow the students to determine how they will achieve the end result. These are two of the many facets of their role in the instructional process. Ultimately, E-Learning creates many changes for the instructor (Wagner, N., Hassanein, K., Head, M., 2008). Their role is transformed regardless of their theoretical persuasion, because the technology and distance of the students will force that change somewhat. Delivery of information, motivation of students, interaction with students, assessment, etc., will all differ because of the medium of delivery. Instructors will require a differing level of technological sophistication depending upon the level of technical support available, but all will need some level of technological knowledge to be able to function in this environment (Wang, W. & Wang, C., 2009). Instructors in the past spent a great amount of their time in knowledge creation, either in the form of original research or by consuming that research (Jones, 2004). With E-Learning, they will be spending more time in the development


Figure 4. LMS stakeholders

and delivery of learning materials than they have historically done (Wang, W. & Wang, C., 2009). Some studies have indicated that they may spend twice as much time in this process, even with the aid of support staff. This shift in time commitment will result in a shift in the role and expectation of instructors in the future.

IT Staff IT staff are the individuals who have traditionally only been involved in supporting the business side of the institution. If they were involved in educational delivery, it was to support systems that were used in traditional classrooms. They now are involved in systems that must be kept

operational 24/7 (Jones, 2004). These systems are mission critical, but are used by personnel who often times do not understand the intricacies of their function. While the end users will have some technical expertise, they will not generally be technologists, but will have some strong expectations from the systems that they use. This will increase the pressure on the IT staff to provide the services desired. IT staff will have to better understand the educational theories within which the system will function in order for the system to perform according to the needs and expectations of the users (students and faculty). It will no longer be sufficient for IT staff to just understand technology, but they will have to also be cognizant of

15


the environment in which the technology will be implemented (Jones, 2004). It will also force the staff to include the end user in the design process from the very beginning and throughout the lifecycle of the product. These will be different roles for the typical IT staff.

Administrators Administrators in traditional institutions are primarily seeking to create easier access to their institution and remove geographic barriers to student participation (Wang, W. & Wang, C., 2009). In this manner, they are increasing their student population and expanding their academic offerings to a wider market. They are also trying to remain responsive to the market trends and the desire of their targeted market (Jones, 2004). They are attempting to expand the institutional system while maintain fiscal responsibility. In the process of expanding their delivery modalities, they want to maintain the quality of their course offerings so as to not destroy their brand image. This often requires that they need to pay attention to the broader issues of the Environment (the outer ring of our diagram). While the other stakeholders focus on a few of the other stakeholders, this group must pay attention to and balance the needs and concerns of all of the stakeholders. They also must focus on the future needs of the institution by establishing the strategic plans for technological innovation.

Environment The environment is the hodge-podge of others that have an interest in the outcomes of the eLearning endeavor (Jones, 2004). They encompass the various accreditation bodies that must certify the quality of the instruction offered. They are primarily concerned that minimum standards are met that all courses are comparable regardless of the modality of offering. While there are a number of accrediting bodies that focus on distance

16

learning/E-Learning, all accrediting bodies must certify some E-Learning offerings through institutional members. Another group that plays a major role in this process is the potential employer of the students in E-Learning programs. They want to ensure that students have received an adequate education irrespective of manner in which it was delivered. They want the education to provide content that is relevant to the work environment and that meets minimum quality standards. They also may become involved by supporting their current employees as they pursue further education to the benefit of the employer. In this manner they are also concerned with student access. The funders of education are an often overlooked stakeholder group. These are the entities that provide various aspects of the funding for educational institutions. They may be public funders in the form of government agencies or they could be private funders who provide grant funding or give to foundations that support the educational programs of an institution. They may be the parents who are funding their child’s education. Members of this group are primarily concerned that the funds will be utilized to provide a quality education to the students and ensure a respected outcome from their investment. All of these various stakeholders interact with one another seeking to have each other’s needs faced and resolved in the E-Learning effort. None of them stand isolated from the others and individually do not have a preeminent role to play in the eLearning offering. If one is ignored or their needs and desires are not met, it creates the potential for the eLearning effort to fail.

MODELS OF LEARNING MANAGEMENT SYSTEMS The various LMS models differ according to the selection of LMS elements (see Figure 5) and how they are applied to the system and therefore


to the users of that system. While there are many elements that could be utilized to discriminate one LMS from another, we will focus on three. These are the primary elements that influence most decisions concerning the system of choice. The accompanying diagram identifies the three broad elements, but one must bear in mind that most systems are not just one or the other, but are in fact somewhere on a continuum between the extremes within each element.

Source In-House Initially all LMS were developed in-house and were dependent upon the support of the institutional programmers. Most of these systems possessed limited functionality and they were difficult to modify or improve. A few of those systems, like the PLATO LMS, were later transferred to businesses that have either used them in proprietary products or marketed them as proprietary systems. Most in-house developed systems, therefore, are no longer in existence, although there are some who are advocating a return to the in-house developed systems as a way to provide quality at a more affordable cost.

Proprietary Proprietary systems comprise the largest block of systems in operation today. While they all claim to provide the functionality needed by the institution, they are by no means identical. They differ on some very important factors and it would behoove the using institution to choose carefully. While they do differ on the basis of functionality, most provide a basic set of functions along the lines discussed in course management systems. They may or may-not interface with academic administrative systems or an online library system, for instant. They may include some Web 2.0 functionality, but it may be limited in utility. There is constant change within the industry, with companies coming and going by either leaving the industry or

being absorbed by one of the competitors. Some examples of these would be: Blackboard (which recently acquired WebCT Manager and Angel), Desire2Learn, ECollege, & .LRN. There are sites where one can compare these systems in order to decide which one is right for you. With these systems, you can either have a hosted environment, where some provider supplies the hardware and software to make the system work or you, or you can host it yourself on your own servers. There are many pros and cons to each approach and should be considered carefully.

Free/Open Source These systems are supported by a network of programmers around the globe who provide updates to the systems on an almost constant basis (Pan, G. & Bonk, C., 2007). Many of these systems are free to own, but are also often available through a supplier who will provide for the installation and support of the system. They can also be available as a hosted option. They provide a great deal of flexibility to the decision making process. They have traditionally been a niche market, but with Moodle gaining in popularity, they are gaining a larger market share. The basic premise of these systems is what is commonly known as Linus’s Law: Given enough eyeballs, all bugs are shallow (Pan, G. & Bonk, C., 2007). They assume a selfcorrecting mechanism in the peer review process that open-software engenders. This further pushes the costs of these products lower and encourages users to expand their functionality. The development of these products promotes common standards that establish those that are most beneficial to the market. This process also encourages the development of expertise in the broader market arena by encouraging many individuals to be involved in the development of the product.

17


Figure 5. Elements of a learning management system

Time Time refers to whether the student and instructor are involved concurrently or separated by time, in other words, synchronous or asynchronous (Wagner, N., Hassanein, K., Head, M., 2008). Most LMS are designed to be operated in an asynchronous mode. In this mode, an instructor prepares the learning content and interacts with the students mostly on a one to one basis. The interaction does not occur in real (concurrent) time, but with time lapses. This creates the possibility of instructor-student disconnect as one has to wait for a period of time for the other to respond to the inputs. As the system supports a more dispersed student population, possibly spanning the globe, this becomes more of a necessity or least more

18

easily justified. However, there are systems that focus on a synchronous delivery mechanism, such as LaunchForce and LeanLine. These systems put the instructor and students online at the same time. These systems may involve, audio only, audio and video, whiteboard, program share, and other such technologies. In this mode, there is much more the virtual classroom experience with the ability to engender the collaborative involvement of all of the students and instructor in the learning exercises. Collaboration is possible in both synchronous and asynchronous modes, but is more natural in the synchronous environment. Some systems allow for both modes of delivery to be blended together. Such systems as ECollege, Embanet, Jones e-education, and LUVIT eLearning are thought of as total solutions. They


allow delivery in the most effective time frame for the individual course. Many of these systems allow for the content that is delivered synchronously to be recorded and then delivered in an asynchronous fashion. There are tools that often can be utilized with other asynchronous systems to allow for the delivery of portions of the class in a synchronous mode.

Pedagogy Pedagogy is concerned with how a course will be delivered in instructional terms. It will affect the type of interaction that may be involved in the learning process, such as, student to content, student to instructor, and student to student (Hamuy, E. & Galaz, M., 2010) (Wang, Y. & Chen, N., 2009). In the student to content, the student interacts with instructional information, either provided through the course system or accessed by other means, textbook, online, etc. In the student to instructor mode, the student interacts with the content expert or experts to retrieve the needed learning content. In the student to student mode, the students will interact with one another to retrieve information or perform learning activities in a collaborative manner. Pedagogy will also determine the nature of the course structure(see Figure 6); will the course allow the student to move around and choose the order in which material is accessed and work is completed or must they move through in a predetermined orderly sequence (Dabbagh, 2005). Will the student be required to achieve mastery of the material to some predetermined level or will the student construct their own learning with the instructor serving as a mentor/guide? Pedagogy plays a fundamental role in the determination of the nature of the learning experience, whether the learning constructor recognizes the role or not. Pedagogy involves three broad aspects in its structure (Dabbagh, 2005). Each of these aspects plays some role in the impact of pedagogy on the learning process. Pedagogical constructs provides

the foundation out of which the others flow (Dabbagh, 2005). From the constructs will flow the pedagogical strategies that determine the teaching methods that are used in the program. From the strategies one will determine the pedagogical tools that will be need in instructional delivery. It is in this manner that pedagogy will determine the nature of the LMS used in a specific setting.

PEDAGOGICAL CONSTRUCTS These are the basic formulation of the teaching/ learning process (Dabbagh, 2005). They are the models or the components of the models that will guide the development and delivery of instruction (Sabry, K. & Barker, J., 2009). They arise out of one’s understanding of cognition and knowledge. They provide the mechanism for putting theory into practice (Dabbagh, 2005). Some major examples include: behaviorist, cognitivist, and constructivist. While these are not the only models used, most others grew out of them.

Behaviorist Behaviorist pedagogical systems will provide a structured/programmed approach to the teaching/ learning process. This pedagogical system sees the mind as a ‘black box’ that is not knowable and so is not a concern. No attention is paid to the internal processes of learning, in fact, many would say internal processes don’t exist, but this pedagogical system focuses on the measurable outcomes of learning. These outcomes therefore equal learning and rewards are the motivators that support the production of desirable outcomes. Learning is linear and structured. Theorists in this paradigm are Watson, Thorndike, Skinner, and Pavlov.

19


Figure 6. Pedagogical structure

Cognitivist

Constructivist

The cognitivists focus on how the mind processes and uses information to produce learning. They are interested in the mental structures and processes that are necessary to explain human behavior (Dabbagh, 2005). They pay greater attention on the learner’s thoughts, beliefs, attitudes, and values in explaining the learning outcomes (Ardichvili, A. & Yoon, S., 2009). They are more focused on the learners’ differences and seek to accommodate them by varying the instruction material and processes. To them learning is an individualized process. Group activities are relatively unimportant and rarely addressed. They seek to provide organizational structure that allows the learner the ability to move through the material in a highly individualized fashion that ties the new learning to existing informational structures. Theorists in this paradigm are Gagne, Briggs, and Bruner.

Constructivists built off the cognitivistic view that the mind is more than a ‘black box’ responding to stimuli. It instead focused on the processes involved in learning, seeing those processes as internal and, therefore, not visible directly (Homberg, 2005). Constructivists see learning as an active process that works within a context to produce (construct) knowledge (Dabbagh, 2005) rather than just acquiring it. Learners must be actively involved in both acquiring and processing information subjectively (Baggaley, 2008). As learners actively process the information, they will gain a deeper understanding of the content (Ardichvili, A. & Yoon, S., 2009). The learner needs to be encouraged and enabled to search for the knowledge or solve problems on their own rather than provided the content or solutions (Ardichvili, A. & Yoon, S., 2009). All knowledge

20


is seen as social in origin, developed while engaged in activities (projects), mediated by tools (Lytras, M. & Pouloudi, A., 2006). Instructors in this tradition will provide the learners with realworld simulations, collaborative experiences with others, or by providing them with the knowledge and ability to access knowledge at the time that it is needed (Ardichvili, A. & Yoon, S., 2009). Theorists in this paradigm are Dewey, Piaget, Vygotsky, and Bruner. These models/constructs are almost never used solely. They each will often find a place in some part of the E-Learning program. Most practitioners will have a preferred approach to the design and delivery of instruction, but will generally use some of the less preferred methods within their program to meet the needs of the widest number of students or to overcome areas of difficulty in the instructional program.

PEDAGOGICAL STRATEGIES The application of pedagogical constructs to the teaching process leads to three broad pedagogical strategic systems (Dabbagh, 2005). Each system seems to lean toward a specific set of educational methodologies/tools. There is an overarching perspective that guides the choices that are made by these educators in the development of the learning program. This produces the framework for E-Learning (see Figure 7).

Behavioral/Cognitive Strategy The Behavioral/Cognitive framework/strategy promotes facilitated learning. The teacher provides the guidance and direction that is needed for the student to learn. The primary educational methods/tools will be lectures, presentation, textbooks, and teacher directed discussions. The teacher is the source of information that leads to knowledge acquisition in this process. The process is very linear and focuses on measureable

outcomes. Practitioners may or may not pay any attention to individual differences or care much about the intellectual processes that are involved in developing the learning outcomes.

Constructivist Strategy The constructivist framework/strategy focuses on individualized learning. These practitioners see the process of learning as very interactive with the student interacting with the teacher and with the learning materials. The student plays the major role in the learning outcome. Students determine what is and is not important to learn. The teacher moves from the ‘sage on the stage to the guide on the side’ in this process. Learning activities and materials must be somewhat fluid in this process, because it is not possible to know what the student will need before the student needs it. The primary educational methods/tools used will be case studies, self-instructional materials, and questions and answers. The pace of learning will be dictated by the student and their desires.

Social Constructivist Strategy The social constructivist framework/strategy focuses on collaborative learning (Huang, S. & Yang, C., 2009). Learning is seen as a social endeavor and requires the collaboration between and among all participants. The teacher will very often be just one of the learners in this process. The group will determine the nature of the learning. Learning activities will vary with the class involved; the students will bring many of the materials from their own search for answers. The primary educational methods/tools used will be case studies, problem based learning, discussion forums, teamwork, and seminar sessions. The emphasis will be upon the group participation in the learning process with the group playing a major role in the decisions about what will transpire in the process. This is a very interactive strategy requiring consistent feedback from the teacher and fellow students. Learning is

21


Figure 7. Pedagogical frameworks

seen as a process rather than an end. Control rests in both the teacher and students within a context of continuous communication.

PEDAGOGICAL TOOLS Your pedagogical strategy will determine the features, pedagogical tools, that your LMS will need to provide (Dabbagh, 2005) (Sabry, K. & Barker, J., 2009). If your strategy is a behavioral/cognitive framework, you can probably get by with a simple course management system that uses prepackaged course material to deliver instruction in a linear fashion with teacher graded outcomes measures. You will need a simple email communication

22

system to allow the students to ask questions of the teacher and allow the teacher to respond to those questions. You would also benefit from having a document delivery mechanism and a means for the students to submit their work and take exams. Most systems available today provide these features and more. If your pedagogical strategy is in a constructivist framework, you will need the entire aforementioned, but also some means for the student to navigate through the material individually and to set personal learning objectives. They will also need to be able to acquire their own learning materials to provide for their individualized learning plan. There will need to be flexibility in the assessment of learning outcomes to provide


for the individualized nature of those outcomes. You will want to provide for simulations, blogs, message boards, etc. to allow the students to be able to construct their own learning activities and provide mechanisms for easy feedback from the teacher/coach. If your pedagogical strategy is a social constructivist framework, you will need everything already mentioned, but also include mechanisms for multi-way communication so that all participants can communicate with each other in a free and open manner. To be most effective, you will need to provide for both synchronous and asynchronous communication modalities. Students will need to work individually and in groups; as much as possible, the groups should be self selected.

FUTURE RESEACH DIRCTIONS Too much of research focuses on finding something new or defending a current stance on a subject. We need to have researchers who are willing to integrate the findings of others into some sort of a coherent system. This should involve the gathering of information from related disciplines so that we can improve learning management systems by using the knowledge from all appropriate sources. If we become too enmeshed in our limited view of the world, either by just looking at what is being produced in distance learning publications or by looking at only what is being done in our limited geographical sphere, we will not be able to meet the transformations that will occur in the future. We also need to focus on those aspects that will improve the quality of what we do.

CONCLUSION Learning management systems, while a modern phenomenon, have very deep roots in a broad array of disciplines. Each of these disciplines is still functioning today and produces impacts

upon LMS and their use within the educational environment. The nature of that impact is influenced by pedagogy. Pedagogy will drive what the LMS will have and how it will be used in the delivery of E-Learning. In most cases, teachers will not stick with one pedagogical framework, but will use aspects of all of them at some point in the delivery of learning. Students will benefit from this eclectic use of pedagogy. We have only scratched the surface of what is involved in the function of an LMS based on pedagogical decision making. The better the teacher understands the pedagogy underlying the decisions being made, the more effective the LMS will be in meeting those needs. Many LMS implementations fail because there was not a clear understanding of the pedagogical needs of the teachers using the system (Dillenbourg, 2008). This chapter has only lightly touched on the topics presented. It is not intended to be an exhaustive treatment of any of the subjects. It is hoped the readers will continue to pursue those topics of interest and to be aware of those of lesser interest. In this manner, we should all be able to understand the changes that have occurred and be prepared for those that will come.

REFERENCES Ardichvili, A., & Yoon, S. (2009). Designing integrative knowledge management systems: Theoretical considerations and practical applications. Advances in Developing Human Resources, 11(3), 307–319. doi:10.1177/1523422309337593 Baggaley, J. (2008). Where did didtance education go wrong? Distance Education, 29(1), 39–51. doi:10.1080/01587910802004837 Baturay, M. (2008). Analysis of a learning management system model. E-Journal of New World Sciences Academy, 3(3), 464–477.

23


Brush, T., Armstrong, J., Barbrow, D., & Ulintz, L. (1999). Design and delivery of integrated learning systems: Their impact on student achievement and attitudes. Journal of Educational Computing Research, 21(4), 475–486. Cradler, J. (2008). Research on e-learning. Learning and Leading with Technology, 3(5), 54–57. Dabbagh, N. (2005). Pedagogical models for e-learning: A theory-based design framework. International Journal of Technology in Teaching and Learning, 1(1), 25–44. Daniels, P. (2009). Course management systems and implications for practice. International Journal of Emerging Technologies & Society, 7(2), 97–108.

Gilman, D., Emhuff, J., Bender, P., Gower, A., & Miller, K. (1991). A comprehensive study of the effects of an integrated learning system. Indiana State University. Guri-Rosenblit. (2005). ‘Distance education’ and ‘e-learning’: Not the same thing. Higher Education, 49, 467–493. doi:10.1007/s10734004-0040-0 Hamuy, E., & Galaz, M. (2010). Information versus communication in course management system participation. Computers & Education, 54, 169–177. doi:10.1016/j.compedu.2009.08.001 Holmberg, B. (1995). Theory and practice of distance education, 2nd edition. London, UK, and New York, NY: Routledge.

Dillenbourg, P. (2008). Integrating technologies into educational ecosystems. Distance Education, 29(2), 127–140. doi:10.1080/01587910802154939

Holmberg, B., & Ortner, G. E. (1991). Research into distance education. Frankfurt am Main, Germany: Peter Lang.

Eteokleous-Grigoriou, N. (2009). Instilling a new learning, work and communication culture through systemically integrated technology in education. Systems Research and Behavioral Science, 26, 707–716. doi:10.1002/sres.983

Homberg, B. (2005). The Evolution, Principles and Practices of Distance Education (Vol. 11). Hagen, Germany: FernUniversität, ZIFF.

Gaeta, M., Orciuoli, F., & Titrovato, P. (2009). Advanced ontology management sytem for personalised e-learning. Knowledge-Based Systems, 22, 292–301. doi:10.1016/j.knosys.2009.01.006 Garrison, R. (2000). Theoretical challenges for distance education in the 21st century: A shift from structural to transactional issues. International Review of Research in Open and Distance Learning, 1(1), 1–17. Gaspay, A., Dardan, S., & Legorreta, L. (2008). Distance learning through the lens of learning models: New outlets for innovation. Review of Business Research, 8(4), 53–62.

24

Huang, S., & Yang, C. (2009). Designing a semantic bliki system to support different types of knowledge and adaptive learning. Computers & Education, 53, 701–712. doi:10.1016/j. compedu.2009.04.011 Jones, D. (2004). The conceptualisation of elearning: Lessons an implications. Studies in Learning, Evaluation, Innovation and Development, 1(1), 47–55. Keegan, D. (2002, November). The future of learning: From eLearning to mLearning. ZIFF Papiere 119. Hagen, Germany: FernUniversitat, Ziff. Kim, S., & Leet, M. (2008). Validation of an evaluation model for learning management systems. Journal of Computer Assisted Learning, 24, 284–294. doi:10.1111/j.1365-2729.2007.00260.x


Lytras, M., & Pouloudi, A. (2006). Towards the development of a novel taxonomy of knowledge management systems from a learning perspective: An integrated approach to learning and knowledge infrastructures. Journal of Knowledge Management, 10 6), 64-80. Miller, T., & Hutchens, S. (2009). 21st century teaching technology: Best practicess and effectiveness in teaching psychology. International Journal of Instructional Media, 36 3), 255-262. Ozkan, S., Koseler, R., & Baykal, N. (2009). Evaluating learning management systems: Adoption of hexagonal e-learning assessment model in higher education. Transforming Government: People, Process and Policy, 3(2), 11–130. Pan, G., & Bonk, C. (2007). The emergence of open-source software in North America. International Review of Research in Open and Distance Learning, 8(3), 1–17. Paulsen, M. (2002). Online education sytems in Scandinavian and Australian universities_A comparative study. International Review of Research in Open and Distance Learning, 3(2), 1–14.

Sabry, K., & Barker, J. (2009). Dynamic Interactive Learning Systems. Innovations in Education and Teaching International, 46(2), 185–197. doi:10.1080/14703290902843836 Taylor, J. (2001). 5th generation distance education. 20th ICDE World Conference on Open Learning and Distance Education (pp. 1-11). Dusseldorf, Germany. Underwood, J. (1997). Integrated learning systems: Where does the management take place? Education and Information Technologies, 2, 275–286. doi:10.1023/A:1018677616969 Wagner, N., Hassanein, K., & Head, M. (2008). Who is responsible for e-learning success in higher education? A stakeholder’s analysis. Journal of Educational Technology & Society, 11(3), 26–36. Wang, W., & Wang, C. (2009). An empirical study of instructor adoption of Web-based learning systems. Computers & Education, 53, 761–774. doi:10.1016/j.compedu.2009.02.021

Perry, B. (2009, June). Customized content at your fingertips. Training & Development, 29–31.

Wang, Y., & Chen, N. (2009). Criteria for evaluation synchonous learning management systems: Arguments from the distance language classroom. Computer Assisted Language Learning, 22(1), 1–18. doi:10.1080/09588220802613773

Rentroia-Bonito, A., Martins, A., Guerreiro, T., & Jorge, J. (2008). Evaluating learning support systems usability an empiracle approach. Communication & Cognition, 41(1 & 2), 143–158.

Watson, W., & Watson, S. (2007). What are learning management systems, what are they not, and what should they become? TechTrends, 51(2), 28–34. doi:10.1007/s11528-007-0023-y

Rogers, L., & Newton, L. (2001). Integrated Learning Systems -- An ‘open’ approach. International Journal of Science Education, 23(4), 405–422.

ADDITIONAL READING

Rose, E. (2004). “Is there a class with this content?” WebCT and the limits of individualization. The Journal of Educational Thought, 38(1), 43–65. Saba, F. (2000). Research in distance education: A status report. International Review of Research in Open and Distance Learning, 1(1), 1–9.

Abel, R. humes, L., Mattson, L., McKell, M., Riley, K., & Smythe, C. (2007). Achieving Learning Impact 2007. August 2007. Retrieved from Http://www.imsglobal.org/learningimpact2007/ li2007report.cfm. Bates, A. (1995). Technology, Open Learning and Distance Education. London, UK: Routledge.

25


Castells, M. (2000). The Information Age: Economy, Society and Culture: Vol. I. The rise of the Network Society. Oxford, UK: Blackwell. Cooper, P. (1993). Paradigm shifts in designed instruction: From behaviorism to cognitivism to constructivism. Educational Technology, (May): 12–19. Cristea, C. (2010). Education and media in the postmodern pedagogy. Petroleum – Gas University of Ploiesti Bulletin. Education Sciences Series, 62(1A), 102–106. Elias, T. (2010). Universal instructional design principles for Moodle. International Review of Research in Open and Distance Learning, 11(2), 110–124. Freishtat, R., & Sandlin, J. (2010).. . Educational Studies, 46, 503–523. Garrison, D. R. (1990). An analysis and evaluation of audio teleconferencing to facilitate education at a distance. American Journal of Distance Education, 4(3), 13–24. doi:10.1080/08923649009526713 Green, N., Edwards, H., Wolodko, B., Stewart, C., Brooks, M., & Littledyde, R. (2010). Reconceptualising higher education pedagogy in online learning. Distance Education, 31(3), 257–273. do i:10.1080/01587919.2010.513951 Gregor, S., & Benbasat, I. (1999). Explanations from intelligent systems: Theoretical foundations and implications for practice. Management Information Systems Quarterly, 23(4), 497–530. doi:10.2307/249487 Hinostroza, J., & Mellar, H. (2001). Pedagogy embedded in educational software design: Report of a case study. Computers & Education, 37(1), 27–40. doi:10.1016/S0360-1315(01)00032-X

26

Hulsmann, T. (2004). Guest editorial – Low cost distance education strategies: The use of appropriate information and communication technologies. International Review of Research in Open and Distance Learning, 5(1), 1–14. Keegan, D. (Ed.). (1993). Theoretical principlesof distance education. London, UK, and New York. NY: Routledge. Ley, T., Kump, B., & Albert, D. (2010). A methodology for eliciting, modellling, and evaluating expert knowledge for an adaptive work-integrated learning system. International Journal of HumanComputer Studies, 68, 185–208. doi:10.1016/j. ijhcs.2009.12.001 Lockwood, F. (1995). Open and distance learning today. London, UK, and New York. NY: Routledge. Mackenzie, O., & Christensen, E. L. (Eds.). (1971). The changing world of correspondence study. University Park, PA: Pennsylvania University Press. Moore, M. G. (2003). Network systems: The emerging organizational paradigm. [Editorial]. American Journal of Distance Education, 17(1), 1–5. doi:10.1207/S15389286AJDE1701_1 Moore, M. G., & Clarke, G. C. (Eds.). (1989). Readings in principles of distance education (pp. 29–37). University Park, PA: The American Center for the Study of Distance Education. Moore, M. G., & Kearsley, G. (2005). Distance education: A systems view. Belmont, CA: Wadsworth Publications. Ozkan, S., & Koseler, R. (2009). Multi-dimensional students’ evaluation of e-learning systems in the higher education context: An empirical investigation. Computers & Education, 53, 1285–1296. doi:10.1016/j.compedu.2009.06.011


Paulsen, M. (Ed.). (2003). Online education and learning management systems: Global e-learning in a Scandinavian perspective. Oslo, Norway: NKI Forlaget. Sellar, S. (2009). The responsible uncertainty of pedagogy. Discourse: Studies in the Cultural Politics of Education, 30(3), 347–460. doi:10.1080/01596300903037077 Sewart, D., Keegan, D., & Holmberg, B. (Eds.). (1983). Distance education: International perspectives. London, UK: Croom Helm (Routledge). Skinner, B. F. (1968). The technology of teaching. New York, NY: Appleton-Century-Crofts. Watson, D. M. (2001). Pedagogy before technology: Re-thinking the relationship between ICT and teaching. Education and Information Technologies, 6(4), 251–266. doi:10.1023/A:1012976702296 Zawacki-Richter, O., Backer, E., & Vogt, S. (2009). Review of distance education research (2000 to 2008): Analysis of research areas, methods, and authorship patterns. International Review of Research in Open and Distance Learning, 10 6), 21-50.

KEY TERMS AND DEFINITIONS Behaviorism: The theory or doctrine that human or animal psychology can be accurately studied only through the examination and analysis of objectively observable and quantifiable behavioral events, in contrast with subjective mental states. Blended Learning: The use of both classroom teaching and on-line learning in education.

Cognitivism: A theoretical approach in understanding the mind using quantitative, positivist and scientific methods that describes mental functions as information processing models. Constructivism: A theory of knowledge (epistemology) that argues that humans generate knowledge and meaning from an interaction between their experiences and their ideas. E-Learning: Learning that is done at a distance using information communication technologies for delivery. Information Communication Technology: Consists of all technical means used to handle information and aid communication, including computer and network hardware as well as necessary software. Instructional Television Fixed Service: A band of twenty (20) microwave channels available to be licensed by the U.S. Federal Communications Commission (FCC) to local credit granting educational institutions. Pedagogy: Refers to strategies of instruction, or a style of instruction. Programmed Learning: A learning methodology or technique first proposed by the behaviorist B. F. Skinner in 1958. It has three elements: (1) it delivers information in small bites, (2) it is self-paced by the learner, and (3) it provides immediate feedback, both positive and negative, to the learner. Video Conferencing: A set of interactive telecommunication technologies which allow two or more locations to interact via two-way video and audio transmissions simultaneously.

27

28

Chapter 2

Knowledge Sharing in a Learning Management System Environment Using Social Awareness Ray M. Kekwaletswe Tshwane University of Technology, South Africa

ABSTRACT The premise for this chapter is that learning and knowledge sharing is a human-to-human process that happen independent of space and time. One of the essential facets of learning is the social interaction in which personalized knowledge support is an outcome of learners sharing experiences. To this point, this chapter does not directly address a specific learning management system (LMS) platform but addresses forms of communication that can be encountered as tools of LMS platforms. The chapter argues that LMS ought to be able to facilitate the social interaction among learners not confined to particular places. Learners, because of their mobility, perform tasks in three varied locations or contexts: formal contexts, semi-formal contexts, and informal contexts. In this chapter, learners use social awareness to determine the appropriateness of an LMS tool to engage in a knowledge activity, as they traverse the varied contexts. Thus, the chapter posits that a ubiquitous personalized support and on-demand sharing of knowledge could be realized if a learning management system is designed and adopted cognizant of learners’ social awareness.

INTRODUCTION This chapter does not directly address a specific learning management system (LMS) platform but addresses forms of communication that can be encountered as tools of LMS platforms, as

learners share knowledge. The chapter argues that to be able to design LMS that ought to enable social interaction among learners not confined to particular places, we must first understand how learners interact and the tools they use. In this chapter, learners use social awareness to determine

DOI: 10.4018/978-1-60960-884-2.ch002


Knowledge Sharing in a Learning Management System Environment Using Social Awareness

the appropriateness of an LMS tool to engage in a knowledge activity, as they traverse the varied contexts. Knowledge is not a fixed commodity, but a function of our interactions with external resources including tools, media, and other humans (Ryder & Wilson, 1997). This suggests that human knowledge transforms as people socially interact with others and the surrounding environment. Consequently, the chapter is premised on the notion that knowledge is created and transferred through the dynamic interactions among individuals and between individuals and their environments (Nonaka, 1994). Thus, knowledge sharing is social and sensitive to context. It is inferred from the notion of knowledge creation, sharing or transfer that knowledge can be perceived to transform when context and social presence awareness interact. In this regard, context and presence awareness influences the interaction and the problems that could be solved and how they are solved. In this chapter, social awareness is synonymous with context and social presence awareness. Although a great number of studies (e.g., Shariq, 1999; Polanyi, 1966; 1958) have shown that knowledge creation and transfer is essentially a human-to-human process or an outcome of social interaction (Nonaka, 1994), the relationships or roles of context and social presence awareness as catalysts for knowledge sharing and transformation in a learning environment has not been explored. This chapter aims to contribute to that effect. This chapter is about exploring and understanding how, through a learning management system environment, a learner uses social awareness to leverage personalized knowledge sharing. It reveals the actual nature of ubiquitous learning through social interaction where awareness of context and social presence is argued to be the underlying process of the activity. The chapter is on how varied forms of communication for knowledge sharing in an LMS learning environment are an outcome of social interaction coordinated by social awareness. Social awareness is synonymous

with awareness of context and social presence. Context is understood as the situation in which a learner or a group of learners find themselves. Accordingly, context is defined as any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application (Dey and Abowd, 1999). Social presence is re-defined and understood to be the mediated presence of another learner who could provide personalized on-demand knowledge support for a learning problem as the learner traverses varied learning contexts. Learners in contact universities come from varied social backgrounds, with diverse languages and cultures. To this point, one of the prevailing educational challenges is that of providing personalized academic support to under-prepared learners (Jaffer et al, 2006). Awareness of the social environment and social resources is, therefore, fundamental to the provision of personalized academic support to a learner. Learning is made ease when a learner has consistent awareness of context and presence of social resources (Kekwaletswe, 2009; 2007). Ubiquitous personalized knowledge support refers to the provision of context sensitive and anywhere, anytime help as learners traverse varied locations. Learners use awareness of context and social presence as a means to access ubiquitous learning support, interpret and adjust their knowledge – sharing what they know with others through social interaction. The chapter, thus, focuses on the peer-to-peer interaction and the learning environment. The social interaction whose outcome is transformed knowledge and provision of support is location and time independent. Since sharing learning experiences is a ubiquitous phenomenon, learners continuously use awareness of the environment and awareness of available social resources they can draw upon to facilitate knowledge consultation (Kekwaletswe, 2009: 2007). This chapter is about the advancement of the human-centric approach to knowledge creation

29


and sharing through enhanced person-to-person interaction – where context and social presence awareness is of vital significance to how learners create, use, and share knowledge. The chapter uses the contextual inquiry research method to understand how a learner goes about sharing knowledge, and the tools they use. The practical contribution of the chapter is the understanding of learning management systems environments and how learners use social awareness to model their actions for the provision of personalized academic support. The rest of the chapter is as follows: firstly, the background to the research problem is given; secondly, the theoretical foundations for the chapter are articulated followed by the research methodology; lastly, the chapter discusses, with examples of empirical evidence, how social awareness is used to leverage knowledge sharing in the learning environment. The chapter is then concluded.

BACKGROUND TO THE RESEARCH PROBLEM The practical relevance of the chapter is towards enriching personalized on-demand academic support through social awareness. To this point, this chapter focuses on LMS communication tools appropriated by awareness of learners’ context. The ideal social awareness for knowledge sharing is one that is sensitive to the background of a learner (social context includes culture and language), arranging these aspects to provide immediacy on the available social network, independent of the learner’s location and task at hand (Kekwaletswe and Ng’ambi, 2006). The focal point of the chapter is therefore how social awareness is used to support personalized social interaction for a South African university learner as s/he traverses varied learning contexts. The objective of this chapter is not to confirm previously established premises and theories but to find out, through engagement with learners, how they, as they traverse varied learning

30

locations, interpret and use social awareness for interaction whose outcome is knowledge sharing. I argue that the effectiveness of a knowledge sharing within a learning management system is fundamentally affected by the social interaction and the context in which a problem-driven learning activity takes place. In the chapter, learning through a learning management system environment is defined as any sort of learning and knowledge sharing that happens due to social awareness when the learner is not at a fixed, predetermined location – in varied learning contexts. An LMS learning environment conveys ubiquitous learning that is not confined to specific locations, and is time independent. Context aware ubiquitous learning is meant to support learning by identifying a learner’s surrounding contextual environment and social presence to provide a rounded and seamless learning experiences. In a contact university, there tend to be disproportionate access to available social resources between when learners are attending scheduled or formal classes and when they are away from scheduled classes (Kekwaletswe and Ng’ambi, 2006) especially as they move away from campus locations. The challenge for universities is that instructors and tutors are not always available to provide learners with ubiquitous support as learners move away from formal locations or contexts. The alternative for these learners is to consult with a knowledgeable peer who, for the most part, shares a background. There are three types of contexts within which a learner is mobile and for which a learner needs a learning management system and social awareness as a medium to transform knowledge. The following three learning contexts have been discussed by Kekwaletswe and Ng’ambi (2006).

Formal Learning Contexts These contexts represent formal structures – such as scheduled lectures and laboratory sessions – in which a learner’s behaviour and action is shaped according to the university class timetable. Inter-


action in these spaces is usually one-way from instructor to learner using English as the official language of instruction, although learners often interact with each other using their own diverse languages. In the formal learning context, the instructor delivers a lecture and a learner either takes notes or is given a handout. Even though learners are invited to ask questions, there is little time to assimilate the material and meaningfully engage with the learning materials (Ng’ambi, 2004). Learning and knowledge seeking action is, therefore, mostly passive. In this context, social presence is usually availed through wired PCs in the computer labs and the face-to-face presence of tutors, instructors and peers.

Semi-Formal Learning Contexts These contexts represent informal spaces on campus used by learners, usually while waiting for the next lecture to start or after it finishes. They include the library, cafeteria, mingling areas and walk-in laboratories. As learners begin to reflect on the previous lecture and skim through the learning materials, questions begin to arise for which clarifications are required (Ng’ambi and Hardman, 2004). The challenge a learner is faced with is how to find an instructor or tutor who is available for immediate or on-demand consultation. Most instructors schedule consultations and are often unavailable for ad hoc interactions. The dilemma for most learners is that the consultation periods are limited and not always suitable (Ng’ambi, 2004). Consequently, the learner’s alternative is to find the nearest socially present knowledgeable peer or class-mate who can provide social support (Kekwaletswe and Ng’ambi, 2006). In the semi-formal learning context social presence is still availed through wired PCs in the computer labs and face-to-face presence of peers. Since the environment is on campus, where most learners meet and come for formal classes, a learner is still very much aware of the available social network.

Informal Learning Contexts Although it is difficult to be explicit on the characteristics of an informal learning context, these contexts include working during after-hours or weekends at university residences or private homes. In these environments, a learner usually uses his or her mother tongue to consult with peers (Kekwaletswe, 2006) or may write down questions to ask the instructor when they do get into the formal context (Ng’ambi, 2004). There are three things that must be known to provide ubiquitous personalized learning and knowledge support to a mobile learner in an informal learning environment: a) Knowledge about the location of a learner so as to help identify the potential knowledgeable peer; b) the preferred language of a learner in which he or she is likely to be conversant, and c) the awareness of a peer’s social presence and contexts – including location and situation (Kekwaletswe, 2006).

The Research Problem One of the prevailing educational challenges in the new South Africa is that of providing personalized academic support to under-prepared learners (Jaffer et al., 2006). In a contact university (as opposed to distance learning university), where students attend formal lectures and scheduled laboratory sessions, there tends to be inconsistency in social presence or access to available social networks for academic support between when learners are on campus and off campus (Kekwaletswe, 2006). Based on empirical evidence, I argue that in a learning environment that has a learner population with diverse social backgrounds and languages, social awareness of peers with a shared background is fundamental in the provision of personalized academic and social support. I distinguish between a learner being on campus in a formal context and being on campus or off campus in informal contexts.

31


Since knowledge sharing and learning tasks are not confined to particular locations but are carried across different learning contexts, I argue that learning and social resources available to learners ought to move with a learner. I use the term social resources pragmatically to mean knowledgeable peers, tutors, instructors and the awareness of context and presence (Kekwaletswe, 2006; Kekwaletswe and Ng’ambi, 2006). In this regard, the problem is that of ensuring that the quality of resources available to learners remains consistent for supporting a knowledge transforming task or action regardless of time and location of a learner (Kekwaletswe, 2006). In other words, learners should have a consistent social awareness of presence and context. The argument is that when learners solve and engage in a learning task – problem based learning – they bring prior knowledge and experience to the social interaction situation, where the knowledge transformation outcome is influenced by social awareness. Thus, learning management systems ought to leverage this. But first, we need to understand how LMS communication tools are and could be used. In conventional interaction with the learning activity, learners have an impoverished mechanism for providing social awareness of the available social network. Consequently, personalized social support is denied when learners do not have the opportunity to access a consistent social network (ibid.). In order to understand social presence and context awareness and their role in learning system environments, we must understand both what the context is and how social awareness can be used to facilitate the ubiquitous learning. An understanding of the context and a sense of social presence enables the learner to model behavior along the expectations or the shared understanding of a social learning community.

32

THEORETICAL FOUNDATIONS This section lays out the epistemological and ontological foundations of the inquiry and reviews the existing literature that informs the chapter. The inquiry is about learning management system environments that support learning and sharing of knowledge, with context and social presence awareness as catalysts for learning actions. The theory of knowledge cannot be divorced from the situations under which knowledge and learning experiences take place and this is reviewed next, followed by a specific focus on learning environments. Social presence and context for interaction in such environments also receive special attention in this section.

Knowledge To understand knowledge and how it is shared in a learning environment as an outcome of social interaction, I unpack the logical development of how knowledge is created, retained, and used. The construction of knowledge requires processing of data into information, new information is then created which is then communicated or transferred outside of the human brain. With that premise, I briefly define data as raw facts that can be shaped and formed to create information. Thus information is data that is given a meaning within a context. Therefore, only data and information can be captured, transferred or stored outside the brain. Knowledge is created from the processing of information, and during this processing, new knowledge can be acquired or created for future use, when more or new information is acquired and processed (Van Beveren, 2002). Knowledge is transformed into information within the brain to be communicated externally through language or demonstration (ibid.. 2002). Language may include different forms of communication such as text, verbal and body gestures. What then constitutes knowledge?


There are probably as many definitions and explanations of knowledge as there are theories of knowledge from diverse disciplinary perspectives. Without joining the philosophical debate of what exactly knowledge is, it is encouraging to notice that literature has adopted a pluralistic epistemology, acknowledging that there are many types or forms of human knowledge. The following are only some of the definitions of knowledge. •

•

•

•

Knowledge is an individual’s stock of information, skills, experience, beliefs and memories (Alexander et al., 1991). Knowledge is the stock of conceptual tools and categories used by humans to create, collect and share information (Laudon and Laudon, 1995) Goffman (1978) explains knowledge by asserting that the “making of knowledge is a way of enacting reality, giving existence to things and events, and organizing the world.” Nonaka (1994) deems knowledge to be “justified true beliefs”. The theory of knowledge creation sees knowledge as a dynamic human process of justifying personal beliefs as part of an aspiration for the “truth”.

Generally, scholars of knowledge observe knowledge in two ways: “know how” and “knowthat”. The former is created ‘here and now’ in a specific, practical context and conveyed through analogies and metaphors; the latter is contained in manuals and procedures and oriented towards a context-free theory (Patriotta, 2003).

Knowledge as a Multifaceted Phenomenon The evolution of institutional knowledge has been informed by a wide spectrum of theoretical traditions. Knowledge is a multifaceted phenomenon which has been debated in a variety of disciplinary contexts – from philosophy and sociology, to

social psychology and cognitive science; from economics to management and organization analysis. The breadth and depth of the subject would not allow me to trace a lineage of existing knowledge theories. Nevertheless, cognitive theories have looked at knowledge as a representational phenomenon. Winograd and Flores (1986) point out that “a cognitive being ‘gathers information’ about things and builds up a ‘mental model’ which will be in some respects correct (a faithful representation of reality) and in other respects incorrect. Knowledge is a storehouse of representations, which can be called upon for use in reasoning and which can be translated into language while thinking is a process of manipulating representations” (op. cit., p73). In cognition-action theory, the foundational hypothesis is that action always possesses a cognitive basis which is reflected in the representational activities of the mind. The duality of cognition and action underlies the conceptualization of knowing as a computational activity (Patriotta, 2003). Since human behaviour is always oriented towards a goal, action is a form of problem-solving, where the actor’s problem is to find a path from some initial state to a desired goal state, given certain situations along the way. Accordingly, there is a need for researchers to understand how learners acquire new problem representations for dealing with new problems. A general theme uniting many situated approaches to cognition is a change in the way the personenvironment relationship is envisaged. Rather than a person ‘being’ in an environment, the activities of person and environment are parts of a mutually constructed whole. The inside-outside relationship between person and environment is replaced by a part-whole relationship (Bredo, 1994). Learning is centered around problem-solving and is intricately related to the context; ‘context’ here means understanding (a) the problem’s conceptual structure as well as (b) the purpose of the activity and (c) the social milieu in which it is embedded (Scribner, 1987).

33


The above briefly highlighted the multifaceted views on learning and knowledge and how it is created. The highlight is what informs my own definition of knowledge sharing. Having given an overview of what knowledge is and how it may be created, the following section will highlight the role of social awareness in transformational learning. In order to understand social presence and context awareness and the degree of their role in a learning management system environment, we must understand both what the context is and how the social awareness can be used to facilitate the ubiquitous learning.

Social Presence Theory In this section, the concept and Theory of Social Presence is discussed. As I examined the social presence studies and literature, it became apparent that most, if not all, of the studies on the concept employ a positivist approach. Whilst this chapter employed an interpretive approach, it is essential that I highlight the extent to which the theory and concept has been explored in previous research – which in turn informed the redefinition of the concept as used in this chapter. Short, Williams and Christie (1976) asserted that different communication media express varying degrees of social presence based on their ability to transmit nonverbal and vocal information. Thus, they initially introduced the Social Presence Theory as “technical social presence,” defining it as the capacity of the medium itself to present the “salience of the other person in interpersonal interaction” (p65). Two concepts associated with social presence are “intimacy” (Argyle and Dean, 1965) and the concept of “immediacy” (Wiener and Mehrabian, 1968). Intimacy depends on factors such as physical distance, eye contact, facial expression and personal topics of conversation. Immediacy is a measure of the psychological distance which a communicator puts between himself and the object of his communication.

34

This notion of Short et al. (1976) was, however, questioned by communication researchers (Gunawardena & Zittle, 1997; Byam,1995; Walther, 1994) who showed that perceived social presence in mediated interactions varies among participants in the same mediated conversations. That is, many of their research participants perceived mediated discourse as more personal than traditional classroom discussion. The claim by Short et al. (1976) that the quality of the communication media determines its social presence or richness was also disputed by Ngwenyama and Lee (1997) who showed that the communication richness of a media is dependent on who uses the media and how they use it. Gunawardena and Zittle (1997), for example, defined social presence as “the degree to which a person is perceived as ‘real’ in mediated communication” (op. cit., p8). They, like Ngwenyama and Lee, also argued that social presence was as much a matter of individual perceptions as an objective quality of the medium. In a survey to measure students’ perceptions of the social presence of others in a computer conference, Gunawardena and Zittle found that perceived social presence predicted more than half of the variance in students’ satisfaction with the conference. Their results also indicated that students who felt a higher sense of social presence enhanced their computer communication using emoticons to express missing nonverbal cues in textual form. Rourke, Anderson, Garrison and Archer (2001) regard social presence as one of the three fundamental “presences” that support learning, the other two being cognitive presence and teaching presence. Thus, they define social presence as “the ability of learners to project themselves socially and affectively into a community of inquiry” (p50). Rourke et al. identified three categories of social presence indicators – affective experience, cohesive experiences, and interactive experiences – and explored their use in online discussion. Affective experience contain personal expressions of emotion, feelings, beliefs, and values; Cohesive


experience are communication behaviours that build and sustain a sense of group commitment, such as greetings and salutations and group or personal reference; Interactive experience are behaviours that provide evidence that others are attending, such as agreement/disagreement, approval and referencing previous messages. There is, evidently, a lack of social presence research in learning management systems environment. Most studies of social presence have focused on the nature of online discussion and accordingly conceptualized social presence as a single construct with an emphasis on perceptions of the presence of peers (e.g., Swan, 2003 & 2002). As noted by Richardson and Swan (2003), there is some indication that instructor social presence may be equally important. Social presence of instructors has been considered in explorations of “teaching presence” (Shea et al., 2003; Anderson et al., 2001). Researchers have demonstrated both that students perceive the presence of others (Picciano, 2002; Gunawardena & Zittle, 1997; Gunawardena, 1995) and that they socially present themselves (Swan, 2003 & 2002; Rourke et al., 2001) in online course discussions. Nevertheless, there is lack of studies – positivist or interpretivist – looking at learners perceiving the presence of others and socially presenting themselves in a learning environment or context. This chapter addresses this shortcoming and it does so by following an interpretive tradition, diverting from the positivist tradition. In the chapter learners use awareness of a social presence for purposes of social interaction whose outcome is knowledge sharing. In this chapter, social presence is defined and understood to be the mediated presence of another learner who could provide personalized on-demand social support for a learning problem as the learner traverses varied learning contexts. Context and context awareness are also fundamental concepts in a learning environment where a learner is not fixed to particular locations. The concepts of context and context awareness are discussed in the next section, in view of the fact that they form the second phenomenon of the chapter.

Context Awareness Context and context-awareness are fundamental concepts in a learning environment where a learner is not fixed to particular locations. The following discussion of context and context awareness studies and literature is intended to show how the concepts are relevant to learning and ubiquitous social interaction. Lonsdale et al. (2003) describe context as a set of changing relationships that may be shaped by the history of those relationships. The figure below gives their hierarchical description of context as a dynamic process with historic dependencies. In Figure 1, a snapshot of a particular point in the ongoing context process can be captured in a context state. A context state contains all of the elements currently present within the ongoing context process that are relevant to a particular learning focus, such as the learner’s current project, or a learning activity. A context substate is the set of those elements from the context state that are directly relevant to the current learning and application focus, that is to say, those things that are useful and usable for the current learning system. Context features are the individual elements found within a context sub-state. Each feature is atomic and refers to one specific item of information about the learner or his/her setting. In implementing context awareness within their architecture, Lonsdale et al. (2003) derive a context sub-state and use the context features contained within it to determine what content might be appropriate for a learner. Context related to the human environment is structured into three categories (Schmidt et al., 1999): •

•

Information on the user; knowledge of habits, emotional state, bio-physiological situations. The user’s social environment; co-location of others, social interaction, group dynamics.

35


Figure 1. Context hierarchy (Lonsdale et al., 2003)

•

The user’s tasks; spontaneous activity, engaged tasks, general goals.

Context related to the physical environment is also structured into three categories: • •

•

Location; absolute position, relative position, co-location. Infrastructure; surrounding resources for computation, communication, task performance. Physical situations; noise, light, pressure (Schmidt et al., 1999).

An entity needs contextual information to choose between alternative strategies in order to reach its objectives, or to do useful work. Context gives hints about what is or what is not achievable (Rakotonirainy et al., 2000). If the learner’s initial knowledge sharing or transfer objectives are not reachable then they are changed to suit the current context, i.e., the objectives change or actions are taken according to the context to maintain an objective. Abstract context awareness then

36

means being aware of what you can know about your context while concrete context awareness means being aware of what you do know about your context (op. cit.).

RESEARCH METHODOLOGY Empirical Inquiry in Formal Learning Contexts Interactions of learners in formal learning contexts at the University of Cape Town – as a contact university – are usually passive (Ng’ambi, 2004). That is, learners who are mobile hardly need to actively interact with each other in a formal lecture. The assumption is that it is mostly outside the formal contexts that university learners, notably at UCT, begin to interpret and engage effectively with learning materials and therefore need social presence and context awareness about available peers for social support. Since awareness of context and social presence is not much of a need for learners in formal environments, the


empirical inquiry does not focus on formal learning contexts. The empirical evidence suggests that social interaction of learners, where they begin to simplify meanings and begin to apply what they have learned in class, happens mostly outside of the formal learning context. The argument is that South African learners, who traverse varied locations, need more personalized academic support through social awareness as they move away from formal learning contexts. In view of this, the empirical study and evidence does not include learning management system environments in formal learning contexts but focuses on semiformal and informal learning contexts. The empirical evidence was gathered at the University of Cape Town campus and residences using contextual inquiry methodology. Contextual inquiry is a field research framework that depends on conversations with users in the context of their work (Holtzblatt and Jones 1994). It is based on ethnography, where the researcher goes into the research participant’s own environment. It is an explicit step for understanding who the user really is and how the work progresses from day

to day (Beyer and Holtzblatt, 1998). Essentially, contextual inquiry consisted of observing learners’ actions and talking with learners in their learning environment while they were engaged in authentic learning tasks. The focus of the contextual interviews and textual interactions was on the learners’ mundane activities, notably problem-driven social interactions related to learning. In Figure 2, learners use social awareness to interact with others who are not in the same location using Web-based mediated communication to send and read emails or chat through instant messaging. Learners interacted with learning resources, posing or responding to questions via a Web-based learning management system. In the figure, learners also use mobile devices such as Portable Digital Assistants (PDAs) to interact via mobile instant messaging, and mobile phones that could be carried around to interact verbally or send short text messages (SMSs). The study began with sixty learners completing a qualitative questionnaire meant to understand communication tools, forms of interaction, social presence and context awareness. The initial sixty

Figure 2. Representation of the research framework

37


learners were randomly identified as they traversed the university campus and in libraries and laboratories. There were no specific criteria for selection other than that the mobile learners had to represent or give a true reflection of typical learners at the University of Cape Town. Twenty of the sixty were asked to take part in the Web-based LMS and PDA pilot based on their willingness and commitment to be available for the duration of the study. Since the study was on understanding how university learners communicate, the tools they use and how they exhibit awareness, it was not necessary to determine who should participate or not participate in the study, other than that they should be registered university learners. The WiFi-enabled PDAs were to facilitate the “anywhere anytime” online learning interactions and the opportunistic mobile instant messaging. Eight of the twenty learners were interviewed and observed as they interacted with others in authentic LMS learning environments. Although this study was not on a particular LMS platform, participants also used Vula, a university SAKAI-based online learning environment. The textual interactions were in the form of chats and instant messages. The contextual verbal interviews were recorded using a digital recorder and transcribed. The storylines were in the form of thick descriptions including all the environmental details that could be observed and documented. Pictures capturing the actions and environment situations were constantly taken. The focus of the inquiry was on how learners interact, the tools they use and how awareness is used to support personalized learning. In the next section, the empirical evidence is discussed in terms of how learners interact, the appropriateness of the tools they use and how social awareness manifest in a learning environment. It is worth noting that the aim of the chapter was not to study a specific LMS platform but to study how knowledge is shared through awareness of context and social presence – awareness determines learning actions and the appropriateness of LMS tools. 38

KNOWLEDGE SHARING IN A LEARNING MANAGEMENT SYSTEM ENVIRONMENT USING SOCIAL AWARENESS Socio-cultural theories of mediated learning suggest that what is learned will emerge from the relationship between human action, and the social, cultural, institutional and historical contexts in which action occurs. This makes it essential for us to understand these contexts and activities before we begin to investigate issues of learning (Sutherland et al., 2000). In view of what Sutherland et al. (2000) suggests, it is first important for us to understand how social awareness manifest in a learning environment if we were to design appropriate learning management systems that could leverage knowledge sharing and learning actions. This section discusses how social awareness manifest amongst learners sharing learning experiences. In earlier sections, I noted that one of the prevailing challenges in the new South African higher education is that of providing personalized support to under-prepared learners – who by the very nature of the South African population come from diverse backgrounds and cultures. In most African cultures, it is uncommon for younger people to interact with or question their elders. This phenomenon tends to apply in an educational environment where learners are not comfortable questioning the instructors. The alternative for these learners is then to consult and interact with their close friends and knowledgeable peers to provide academic support. For this reason the inquiry focused on social presence of peers and not “teaching presence”. Consequently, this chapter is about interactions amongst learners (peers) for purposes of knowledge sharing, as they traverse varied learning locations. However, it is worth noting that in South Africa, availability and access to wireless and wired computers, remains a challenge. These resources become even more inadequate as you move away from campus. The


study was therefore guided by the three localized learning contexts discussed in section two, which apply to a typical South African contact university. The framework depicts social awareness and presence in three different learning contexts of the mobile learning environment. Social awareness is a mental concept where a peer and a learner become aware of the social network that follows them as they move across the varied learning contexts In Figure 3, a learner is consciously aware of available knowledgeable peers should s/he encounter a learning problem for which s/he needs to consult. By the same token, a peer is consciously aware of the presence of other learners should s/he not be able to address a problem encountered by a learner. That is, a learner and a knowledgeable peer have a consistent social awareness of a social network (social resources) regardless of their location and context. Even though learners interact via Web-based environments, in this study, the social awareness and availability of social resources in

remote locations was mostly mediated by mobile technologies, e.g., mobile phones and PDAs. Knowledge sharing involves two actions: transmission (sending or presenting knowledge to a potential learner) and absorption by the audience (Davenport and Prusak, 1998). If knowledge is not absorbed, it has not been shared. In other words, merely making knowledge available is not sharing. In this regard, social interaction achieved and enhanced by awareness of context and presence is necessary. The research investigated how learners use social awareness – social presence and context awareness – to enable a knowledge sharing interaction. It sought to understand ubiquitous learning where learners support others as they traverse the three learning contexts. Communication or interaction that is mediated by technology is generally grouped into two categories: asynchronous and synchronous. Asynchronous communication occurs between learners independent of time and location. This kind of communication does not need the send-

Figure 3. A framework for mobile learner-to-learner environment (Kekwaletswe, 2006)

39


ing and the receiving learners to be “available” concurrently. Examples include leaving a phone voicemail, posting to or reading a discussion board, and sending and receiving email (although this could also be considered synchronous). On the other hand, synchronous mediated interaction is considered a “real-time” experience between two or more learners. Examples of tools that facilitate synchronous communication include telephones, audio-video conferencing software, instant messaging, virtual chat, virtual classrooms, and whiteboards. Asynchronous and synchronous mediatedcommunication can be used in individual or group learning situations, as well as traditional or online learning environments. Although a significant body of research validates the notion that learning is a social act, learners may still acquire knowledge in a mediated interaction. There are several forms of technology mediated communication, such as electronic mail, web-based consultation environments, instant messaging on wired networks and short message service (SMS) on mobile phones. Although they support mobility of a learner and may afford presence awareness, they do not do so in real-time. In the study, there was a need to select a mediating technology that supports mobility of a learner as well as presence awareness of available peers in real-time. Thus, mobile instant messenger (IM) Jabber client for Pocket PCs (PDAs) called iMov messenger (www.jabber.org) was selected. The mobile instant messaging on PDAs supports an immediate on demand formal or informal expressive interaction. Such real-time interaction could be one-to-one or several concurrent dyadic conversations. Mobile IM provides mobile learners with a real-time interactive space to share learning experiences – exchanging textual messages that do not require detailed email-like messages or face-to-face interactions. Dey and Abowd (1999) suggest that contextaware applications look at the who’s, where’s, when’s and what’s (that is, what the user is doing) of entities and use this information to determine

40

why the situation is occurring. In this chapter, the context-aware learning interaction environment supports (who) a learner as s/he engages with the (what) learning materials, (where) whether in semi formal or informal learning contexts (when) anytime through the use of context awareness and social presence mechanisms. In this chapter, social awareness, and not a learning management system, does actually determine why a learning situation is occurring.

Social Awareness in a Learning Environment In semi and informal contexts, learners engaged in a learning activity are able to use implicit and explicit context awareness to increase or decrease the interaction whose outcome is learners “helping” or “supporting” each other. The notion of social interaction presupposes an existence of two or more learners speaking or acting. And also, embedded in this notion is the assumption that a learning community is socially present without which interaction – “help” or “support” – is impossible. The following tables give examples of how learners use implicit and explicit context and social presence awareness to increase or decrease social interaction whose outcome is personalized learning. To help the reader understand how awareness manifests, the examples are structured as follows: what the researcher wanted to know, the learner’s experience and a brief discussion. The experiences are grouped in a table form per research theme and query. Showing the empirical evidence this way was to allow the reader to see where the low level research questions (elements of awareness) that support the main question are addressed. Table 1 shows how learners are conscious or aware of the situation and the context in which they are interacting and sharing experiences with others, e.g., how fast they would need the response, the content of interaction, etc. It also highlights how presence and availability help the decision


regarding how to communicate. The different mediating tools offer different opportunities and accessibility. The preceding tables showed just some of the examples of how awareness of context and social presence manifest in a learning environment. In a learning environment, the central activity is knowledge where peers provide personalized social support through social interaction. Learners

acquire knowledge and experience through social interaction, drawing upon available social resources for the solution of practical learning problems. The empirical evidence showed that social awareness of the social resources, the learning task, and the environment does influence the action a learner undertakes – hence the appropriateness of an LMS communication tool.

Table 1. How does a learner share knowledge and learning experiences with others? Researcher’s Intention

Different Learners’ Response / experiences

Researcher’s Comments

How does a learner usually communicate with peers, besides face-to-face meetings, when faced with a learning problem and/or needing help?

…through SMS, …I call or email, …I use my mobile phone, …we do online chat, …I facebook sometimes …mobile phone chat (viamiXit). NB: MiXit is a mobile phone service that allows friends to chat (send/receive text) instantly and in real-time.

Here the learner’s intention is to seek help or share knowledge about a learning problem. He or she picks up an appropriate tool that best mediates the thought.

What is a learner’s reason for communicating using a particular tool?

…Depends on how desperate I am, if desperate I phone, if not I SMS. ...Depends on what I want to say …It depends on the situation at the time …Depends on the seriousness of the situation …Availability of funds …Economic limitations and time available …It depends on where I am at the time that I need to contact them.

The experiences show evidence of the learners’ situation as awareness of context. Location, cost, need and reason to interact are taken as context awareness which influences how learners consult each other, e.g., calling will give immediate response.

How does a learner decide which method of interaction to use?

…With email, it.s efficient because I can phrase the issues better and they can review and reply with specific and thorough answers. …It depends on the issues I need to discuss or message I need to get across. ...How they will respond. …With SMS and calls, the response is faster and I can get help I need in time. …It is the fastest way to get feedback from them. …Whichever will reach the fastest in terms of being seen by the person, and the immediate need. …It.s quick and direct (Mobile phone calls). …They are always with them (peers always carry their mobile phones) ...I know they will always have their cell ...Depends on if they are available, and what will be quicker at that moment. ..If they are available (via mobile phones)

Learners use content of the interaction as context for which a mediating tool is chosen. Learners decide on the method of interaction with peers based on how quickly and fast they need the response. Where a short message is enough for the interaction, they text each other. Issues of time are noted as an influencing factor in picking a mediating tool, noting that most interactions are done on-demand to solve a learning problem instantly. Learners use social presence awareness as an influencing factor for choosing a specific mediating tool.

41


Table 2. How does location as context awareness influence the knowledge sharing social interaction? Researcher’s Intention

Different Learners’ Response / Experiences


From which locations is a learner willing to contact his/her peers for a learning task or problem?

…Home, library, hospital …University, basically anywhere on campus ...Only within university premises …Everywhere …Malls and shops …In the lab

Learning tasks and problems are not confined to fixed locations, thus learners could share knowledge about a problem from any location.

If a learner wanted to interact with a peer, how would s/he consider a peer’s present location?

.If they are in a lecture or the library and I have a small issue then I won’t bother them. In essence, I would just take into consideration my need to contact them and their availability in terms of what they’re doing and where they are. .If they were on campus or at the library or maybe at home, I’d feel free to contact them since they will probably be more flexible and relaxed to respond.

Learners use awareness of location to determine if the peer’s location favors a knowledge sharing social interaction.

How does a learner’s knowledge of where a peer is (their location) affect the knowledge sharing social interaction?

..Makes it easier because you would know who is closest to you for help ..It would be great to know if they are nearby, but if they’re not anywhere where I can reach them, then I’ll have to call regardless of where they are. …The closer he or she is, the better, because it will be easier for me to go to her when push comes to shove. …If I knew they were in a club or something, I wouldn’t bother them with school work.

The experiences show that awareness of a peer’s location would determine if it is plausible or would make sense to consult a peer.

Awareness of context and social presence is used as a tool that enables personalized sharing of experiences. An aspect of social awareness includes the location factor and context, which manifests as valuable to how the interaction is influenced. Culture and social background manifest as playing a fundamental role in learning contexts where learners often would rather seek help or support from peers who share a background. A common background allows better understanding and better communication and thus eases up learning and knowledge interactions. Awareness of context also manifests in how an interaction is influenced by a learner’s emotional and physical state, including behaviors and actions of others. Environmental situations also manifest as playing a role in a learner’s knowledge sharing decisions. Social presence of peers manifest as

42

fundamental with the role it plays in the sharing decisions. That is, awareness of a social presence enables an opportunistic learning and knowledge interaction and can also motivate a learner to engage with a learning task.

CONCLUSION This chapter did not directly address a specific learning management system (LMS) platform but addressed awareness and forms of communication that can be encountered as tools of LMS platforms. The chapter argued that to be able to design LMS that would efficiently enable social interaction among learners not confined to particular places, then there is a need to first understand how learners interact and the tools they use. In this chapter,


Table 3. How does a peer’s activity and emotional state as context awareness influence knowledge sharing social interaction? Researcher’s Intention



How would knowledge of what a peer is doing (current activities) affect a learner’s interaction decision?

…Issues like a person sleeping or eating may make me wait a while. ..I need to know if they got my full attention ..If sleeping, I will leave them be, if studying will interact with them …Would respect that they are really busy and seek help elsewhere

A peer’s activity influences the decision about whether to consult with that learner or consult with someone else.

How do actions and behaviors of others affect a learner’s decision to learn or share experiences?

..I am motivated by seeing others work and tend to slack when others aren’t working …When others are discussing work, it increases chances of grasping concepts. ...It doesn’t affect me much, because we all have different learning capabilities. . ..I take seriously any feedback or advice from friends who have done courses I’m doing. ..If I am studying and people are noisy, I will not be able to concentrate. If they are serious, I too will be.

Presence of others, that is, seeing what they are engaged with influences the decision of a learner to get involved and do the same. For some, what others are doing does not influence their decision to work

To what extent do emotional states of a peer affect a learner’s actions?

…Very little, unless they’re my friends and I am concerned, or if it’s group work and they’re slacking due to moods. …You know what you can ask and can’t, their moods may limit your questions. ...Lets you know how to approach them

Learners could be sensitive to the mental state of a peer, thus altering the way they consult for help or sharing experiences.

How does the awareness of a peer’s current emotional situation (e.g., s/he is stressed, happy, sad, cheerful, etc.) help in interacting with them?

…Very helpful, you don’t want to interact and learn with a sad guy, so they ought to be happy, cheerful. If anything they can be stressed. …It’s useful in the sense that I can know whether to bother them or not. It makes little difference in group work unless their situation is very serious. .It makes me understand their behavior and the way in which they react to my questions or experience. …It makes me understand them better and assist in determining what things I can or cannot say. …Will try to approach them correctly, depending on what kind of mood they are in?

The experiences of learners suggest that emotional states and activities of peers do influence a social interaction. The social awareness determines how learners approach their peers for a learning purpose. The emotional states could be regarded as signs and rules, thereby altering the context in which a knowledge sharing activity happens. Although these are unwritten signs and rules, learners know not to disturb or pester a peer who is not in the best of emotion and spirit to help with a learning task.

learners used social awareness to determine the appropriateness of an LMS tool to engage in a knowledge activity, as they traverse the varied contexts. Mobile phones, SMS, PDAs, email, instant messaging were in this chapter seen as tools and forms of LMS communication, without having to study a specific LMS platform. In order to understand learning management systems

and the degree of their role in learning, we must understand both what the communication tools are and how the social awareness can be used to facilitate the ubiquitous learning. An understanding of the context and a sense of social presence enables the learner to model behavior along the expectations or the shared understanding of a social learning community.

43


Table 4. How do environmental situations as awareness of context influence knowledge sharing actions? Researcher’s Intention



To what extent is a learner aware of his/her surroundings, during a learning activity, and how does that influence his/her decision to interact with peers?

..Temperature, I don’t like studying alone when it’s too cold so I normally chat a lot when it’s cold. ...The time of the day determines when I need to chat or where I need to work. ...Noise, too much movements, then I can’t concentrate on my work. …When I talk I can tell who is an understanding person from a lost one. …When I am with people I am not so aware of my surroundings in the sense that I am when I am alone, however, it is when its silent and cold that I tend to SMS/call people with my problems. ...If I see a lot of other students studying and discussing with their friends, they keep me motivated and I ask them questions.

The environmental context and situations do influence how learners act and how a learning action happens. The temperature during the day or ambience or noise levels may help a learner resort to a different knowledge action. For example, a learner saying that she prefers to do more social interaction than studying in isolation when the temperatures drop. A learner consults with peers, taking advantage of their social presence and surroundings.

Table 5. What is the role of Social Presence Awareness in the knowledge sharing environment? Researcher’s Intention



How is it important for a learner to know a peer’s availability before he/she decides to interact with them?

Very important For me, it is quite important

Awareness of social presence or a peer’s availability is essential for knowledge interaction decisions

How does awareness of presence influence a learner’s knowledge action?

..It helps me determine who I can get assistance from ..Puts me at ease since I know my friends could help when I get stuck …I know I’m going to get help ..If they are available and willing to help, I will contact them. …I have a choice to pick any depending on who is more knowledgeable on that particular topic. …I know that when my academic questions arise I’ll be answered with certainty ...Then I could reach them whenever I need them ..It makes learning very easy, we could exchange ideas and information anytime and I will know that I am always able to ask for help

Social presence of knowledgeable peers is important for interaction whose purpose is sharing knowledge experiences. Learners are consciously at ease when they know their peers are available to help with a learning task, at anytime. Knowing a peer is socially present (awareness of social presence) gives a learner a sense of having a personalized academic support that “follows” them regardless of a learning problem and location.

How is a learner able to keep the sense of presence during a technology-mediated interaction, particularly when using textbased instant messaging.

..Make the messages as short as possible ..Instant or quick responses …I don’t want to wait for a response to my question, which is why I use IM.

Short, quick and instant responses grab a learner’s attention in a text-based social interaction.

How is a learner able to read emotions of peers during IM interaction and how can s/ he sense if the peer is interested in talking to them or not?

. ..Words, descriptive uninhibited words ... I know when they have something else in their minds . ..One word answers would indicate that he isn’t keen to talk. . ..The level of response and language used (enthusiasm)

Even though they are not in the same location, learners are able to exhibit social awareness levels during a mediated social interaction

44


The role of context awareness, and how it is experienced, is significant to ubiquitous social interaction whose outcome is personalized knowledge sharing. The context awareness in the learning management system environment manifests in three categories: 1) Mediating tools or applications as context of the sharing action learners use context awareness to decide on the appropriateness of the mediating tool. For example, if a learner is faced with a time-driven learning task she cannot resolve in isolation then she uses a PDA or a mobile phone to call or text message a peer as opposed to writing and sending an email to a peer. In this situation the learner is aware of the need to get instantaneous response, aware of how best to get the instant support, aware of how best to get social presence, as well as aware of the best mediating tool to exploit. 2) Location of a learner as context of the activity - learners need awareness of where they are themselves and where their peers are located, which is significant in determining the plausibility of a social interaction. 3) Learner’s situation –refers to context awareness dealing with culture and language, the learner’s current activity as well as environmental states. This chapter contributed towards an informed insight on how learners share their knowledge in a learning environment – focusing on LMS communication tools and their appropriateness. The chapter has shown that awareness of context and social presence (synonymous with social awareness) is a useful and significant characteristic for sharing learning experiences and the consequent learning support that occur independent of location and time. When learners socially interact through LMS mediation, they use context awareness to identify an appropriate tool and peer, in order to increase or decrease the knowledge sharing experience. Thus, in a learning environment, the social awareness is important since it influences the learning and knowledge decisions and actions, as the learning contexts change.

REFERENCES Alexander, P. A., Schallert, D. L., & Hare, V. C. (1991). Coming to terms: how researchers in learning and literacy talk about knowledge. Review of Educational Research, 61(3), 315–343. Anderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing teaching presence in a computer conferencing context. Journal of Asynchronous Learning Networks, 5(2). Argyle, M., & Dean, J. (1965). Eye-contact, distance and affiliation. Sociometry, 28, 289–304. doi:10.2307/2786027 Beyer, H., & Holtzblatt, K. (1998). Contextual design, defining vustomer-centred systems. MA, USA: Morgan Kaufmann Publishers, Inc. Bredo, E. (1994). Cognitivism, situated cognition and Deweyian pragmatism. Philosophy of Education. Retrieved June, 2010, Online at http:// www.ed.uiuc.edu/EPS/PES-Yearbook/94_docs/ BREDO.HTM] Byam, N. (1995). The emergence of community in computer-mediated communication. In Jones, S. G. (Ed.), Cybersociety. Newbury Park, CA: Sage. Davenport, T. H., & Prusak, L. (1998). Working knowledge: How organizations manage what they know. Boston, MA: Harvard Business Review Press. Dey, A. K., & Abowd, G. D. (1999). Towards a better understanding of context and context-awareness. GVU Technical Report GIT-GVU-99-22. College of Computing, Georgia Institute of Technology, Atlanta, Georgia. Goffman, E. (1978). Behavior in public places. Notes on the social organization of gatherings (4th ed.). New York, NY: Free Press.

45


Gunawardena, C. (1995). Social presence theory and implications for interaction and collaborative learning in computer conferences. International Journal of Educational Telecommunications, 1(2/3), 147–166. Gunawardena, C., & Zittle, F. (1997). Social presence as a predictor of satisfaction within a computer mediated conferencing environment. American Journal of Distance Education, 11(3), 8–26. doi:10.1080/08923649709526970 Holtzblatt, K., & Jones, S. (1994). Contextual inquiry: A participatory design technique for system design. In Schuler, D., & Namioka, A. (Eds.), Participatory design: Principles and practice. Englewood Cliffs, NJ: Prentice-Hall. Jaffer, S., Ng’ambi, D., & Czerniewicz, L. (2006). The role of ICTs in higher education in South Africa: One strategy for addressing teaching and learning challenges. Proceedings of Emerge Online Conference: Learning Landscapes in Southern Africa, 10-21 July. Retrieved from http:// emerge2006.net. Kekwaletswe, R. M. (2006). Social presence and context awareness for knowledge transformation in an m-learning environment. In Proceedings of Emerge Online Conference: Learning Landscapes in Southern Africa (10-21 July). Retrieved from http://emerge2006.net. Kekwaletswe, R. M. (2007). Knowledge transformation in a mobile learning environment: An inquiry of context and social presence awareness. PhD thesis. South Africa: University of Cape Town. Kekwaletswe, R. M. (2009). Conceptualizing ubiquitous learning through context-aware wired and wireless Web services. In Proceedings of the IADIS Mobile Learning conference, Barcelona, Spain (25 – 28 February).

46

Kekwaletswe, R. M., & Ng’ambi, D. (2006). Ubiquitous social presence: Context-awareness in a mobile learning environment. In Proceedings of the IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing, Taichung, Taiwan (90-95). Taichung, Taiwan. Laudon, K. C., & Laudon, J. P. (1995). Information systems: A problem-solving approach (3rd ed.). Orlando, FL: Dryden Press. Lonsdale, P., Baber, C., Sharples, M., & Arvanitis, T. N. (2003). A context awareness architecture for facilitating mobile learning. In Proceedings of MLEARN 2003 (LSDA), London. Ng’ambi, D. (2004). Towards a knowledge sharing framework based on student questions: The case for a dynamic FAQ environment. PhD thesis. South Africa: University of Cape Town. Ng’ambi, D., & Hardman, J. C. (2004). Towards a knowledge-sharing scaffolding environment based on learners’ questions. British Journal of Educational Technology, 35(2), 187–196. doi:10.1111/j.0007-1013.2004.00380.x Ngwenyama, O. K., & Lee, A. (1997). Communication richness in electronic mail: Critical social theory and the contextuality of meaning. Management Information Systems Quarterly, 21(2), 145–167. doi:10.2307/249417 Nonaka, I. (1994). A dynamic theory of organizational knowledge creation. Organization Science, 5(1), 14–37. doi:10.1287/orsc.5.1.14 Patriotta, G. (2003). Organizational knowledge in the making: How firms create, use, and institutionalize knowledge. Oxford, UK: Oxford University Press. Picciano, A. G. (2002). Beyond student perceptions: Issues of interaction, presence and performance in an online course. Journal of Asynchronous Learning Networks, 6(1).


Polanyi, M. (1958). Personal Knowledge. Chicago, IL: The University of Chicago press. Polanyi, M. (1966). The tacit dimension. London, UK: Routledge & Kegan Paul. Rakotonirainy, A., Loke, S. W., & Fitzpatrick, G. (2000). Context awareness for the mobile environment. Retrieved Oct 11, 2009 from ftp:// ftp.cc.gatech.edu/pub/gvu/tr/2000/00-18r.ps.Z. Richardson, J. C., & Swan, K. (2003). Examining social presence in online courses in relation to students’ perceived learning and satisfaction. Journal of Asynchronous Learning Networks, 7(1), 68–88. Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Assessing social presence in asynchronous text-based computer conferencing. Journal of Distance Education, 14(2). Ryder, M., & Wilson, B. (1997). From center to periphery: Shifting agency in a complex technical learning environments. Paper presented at the Meeting of the American Educational Research Association, Chicago, IL. Schmidt, A., Beigl, M., & Gellersen, H. W. (1999). There is more to context than location. Computers & Graphics, 23, 893–901. doi:10.1016/S00978493(99)00120-X Scribner, S. (1987). Thinking in action: Some characteristics of practical thought. In Sternberg, R., & Wagner, R. (Eds.), Practical intelligence: Nature and origins of competence in everyday world. Cambridge, UK: Cambridge University Press. Shariq, Z. S. (1999). How does knowledge transform as it is transferred? Speculations on the possibility of a cognitive theory of knowledgescapes. Journal of Knowledge Management, 3(4), 243–251. doi:10.1108/13673279910303998

Shea, P. J., Pickett, A. M., & Pelz, W. E. (2003). A follow-up investigation of “teaching presence” in the SUNY Learning Network. Journal of Asynchronous Learning Networks, 7(2), 61–80. Short, J., Williams, E., & Christie, B. (1976). The social psychology of telecommunications. Toronto, Canada: Wiley. Sutherland, R., Facer, K., Furlong, R., & Furlong, J. (2000). A new environment for education? The computer in the home. Computers & Education, 34(3-4), 195–212. doi:10.1016/ S0360-1315(99)00045-7 Swan, K. (2002). Building communities in online courses: The importance of interaction. Education Communication and Information, 2(1), 23–49. doi:10.1080/1463631022000005016 Swan, K. (2003). Developing social presence in online discussions. In Naidu, S. (Ed.), Learning and teaching with technology: Principles and practices (pp. 147–164). London, UK: Kogan Page. Van Beveren, J. (2002). A model of knowledge acquisition that refocuses knowledge management. Journal of Knowledge Management, 6(1), 18–22. doi:10.1108/13673270210417655 Walther, J. (1994). Interpersonal effects in computer mediated interaction. Communication Research, 21(4), 460–487. doi:10.1177/009365094021004002 Wiener, M., & Mehrabian, A. (1968). Language within language: Immediacy, a channel in verbal communication. New York, NY: AppletoncenturyCroft. Winograd, T., & Flores, F. (1986). Understanding computers and cognition: A new foundation for design. Norwood, NJ: Ablex.

47


ADDITIONAL READING Bush, M. D., & Mott, J. D. (2009). The transformation of learning with technology: Learner-centricity, content and tool malleability, and network effects. Educational Technology, 49(2), 3–20. Delta Initiative (2009). The state of learning management in higher education systems, report for the California State University System. Dourish, P., & Bellotti, V. (1992). Awareness and coordination in shared workspaces. In Proceedings of ACM CSCW Conference (pp. 107-114). Dourish, P., & Bly, S. (1992). Portholes: Supporting awareness in a distributed work group. Proceedings of the CHI ‘92 Conference on Human Factors in Computing Systems (pp. 541-547). Engeström, Y. (1987). Learning by expanding: An activity theoretical approach to developmental research. Helsinki, Sweden: Orienta-Konsultit. Engeström, Y. (1999). Activity theory and individual and social transformation. In Engeström, Y., Miettinen, R., & Punamaki, R. (Eds.), Perspectives on activity theory. Cambridge, UK: Cambridge University Press. Gardner, C. (2009). A personal cyberinfrastructure. EDUCAUSE Review, 44(5), 58–59. Gobbin, R. (1998). Adoption or rejection: Information systems and their cultural fitness. In H. Hasan, E. Gould, and P. Hyland (Eds.), Information systems and activity theory: Tools in context (pp. 109-124). University of Wollongong Press, Wollongong, N.S.W. Gould, S. J. (1987). Time’s arrow: Time’s cycle: Myth and metaphor in the discovery of geographical time. Cambridge, MA: Harvard University Press.

Gutwin, C., Greenberg, S., & Roseman, M. (1996). Workspace awareness in real-time distributed groupware: Framework, widgets and evaluation. In Proceedings of HCI on People and Computers XI (pp. 281-298). Jonassen, D. H., & Rohrer-Murphy, L. (1999). Activity theory as a framework for designing constructivist learning environments. Educational Technology Research and Development, 47(1), 62–79. doi:10.1007/BF02299477 Lane, L. M. (2009). Insidious Pedagogy: How course management systems impact pedagogy. First Monday, 14(10). Mlitwa, N., & Van Belle, J. P. (2010). A proposed interpretivist framework to research the adoption of learning management systems in universities. Communications of the IBIMA (Vol. 2010). Retrieved from http://www.ibimapublishing.com/ journals/CIBIMA/cibima.html. Morgan, G. (2003). Faculty use of course management systems (Vol. 2). ECAR Research Bulletin. Sclater, N. (2008). Web 2.0, personal learning environments, and the future of learning management systems,” ECAR Research Bulletin (vol. 2008, no. 13). Suchman, L. (1987). Plans and situated actions: The problem of human–machine communication. Cambridge, UK: Cambridge University Press. Taylor, J. (2006). Evaluating mobile learning: What are appropriate methods for evaluating learning in mobile environments? In M. Sharples (ed.), Big issues in mobile learning: Report of a workshop by the Kaleidoscope Network of Excellence Mobile Learning Initiative: University of Nottingham, UK. Retrieved November 29, 2009, from http://telearn.noe-kaleidoscope.org/ warehouse/Sharples-2006.pdf. Tollmar, K., Sandor, O., & Shömer, A. (1996). Supporting social awareness at work, design and experience. In [New York, NY: ACM Press]. Proceedings of CSCW, 96, 298–307.

48


KEY TERMS AND DEFINITIONS Awareness: An understanding of the activities of others, which provides a context for your own activity. Context: Understood as the situation in which a learner or a group of learners find themselves. Contextual Inquiry: A field research framework that depends on conversations with users in the context of their work. Knowledge Sharing: Involves two actions; transmission (sending or presenting knowledge to a potential learner) and absorption by the au-

dience. If knowledge is not absorbed, it has not been shared. Social Awareness: Is synonymous with context and social presence awareness. Social Presence: Defined and understood to be the mediated presence of another learner who could provide personalized on-demand social support for a learning problem as the learner traverses varied learning contexts. Ubiquitous Personalized Support: Refers to the provision of context sensitive and anywhere, anytime support as a learner traverses varied locations.

49

50

Chapter 3

Learning 2.0:

Using Web 2.0 Technologies for Learning in an Engineering Course Thomas Connolly University of the West of Scotland, UK Carole Gould University of the West of Scotland, UK Gavin Baxter University of the West of Scotland, UK Tom Hainey University of the West of Scotland, UK

ABSTRACT Technology, and in particular the Web, have had a significant impact in all aspects of society including education and training with institutions investing heavily in technologies such as Learning Management Systems (LMS), ePortfolios and more recently, Web 2.0 technologies, such as blogs, wikis and forums. The advantages that these technologies provide have meant that online learning, or eLearning, is now supplementing and, in some cases, replacing traditional (face-to-face) approaches to teaching and learning. However, there is less evidence of the uptake of these technologies within vocational training. The aims of this chapter is to give greater insight into the potential use of educational technologies within vocational training, demonstrate that eLearning can be well suited to the hands-on nature of vocational training, stimulate further research into this area and lay foundations for a model to aid successful implementation. This chapter discusses the implementation of eLearning within a vocational training course for the engineering industry and provides early empirical evidence from the use of Web 2.0 technologies provided by the chosen LMS. DOI: 10.4018/978-1-60960-884-2.ch003


Learning 2.0

INTRODUCTION There has been considerable research into the perceived benefits of eLearning and Learning Management Systems (LMS) within education and it is clear that LMS now play a pivotal role in the delivery of eLearning within many educational institutions. The research literature cites many advantages of eLearning, particularly the convenience and flexibility offered by the (asynchronous) ‘anytime, anywhere, anyplace’ education. However, much less research has been carried out into the use of educational technologies and tools within vocational training environments. The purpose of this chapter is to discuss the impact on learning with the introduction of a LMS into a vocational engineering course. This chapter discusses the pilot implementation of Web 2.0 tools within an LMS and aims to answer a number of questions: (i) Can technology supplement the hands-on nature of vocational training? (ii) Can the use of wikis and forums aid vocational training? (iii) Can the pilot be considered a success? Much of the research in this area has been mainly anecdotal and has not considered the different nature of vocational training with most of the research focusing on the traditional educational environment. This chapter utilises both qualitative and quantitative surveys on the views of trainees and instructors and aims to identify the areas within the training programme where the LMS could be utilised further to aid learning. It also considers the areas where the use of the LMS has not been as successful as anticipated and the reasons for this. The next section of this chapter discusses the literature on LMS, the use of Web 2.0 technologies within education, and ePortfolios. The subsequent sections introduce the research rationale, the case study and an empirical analysis of the pilot implementation. The chapter concludes

with a discussion of the findings and provides some recommendations for the implementation of eLearning within vocational training.

PREVIOUS RESEARCH eLearning can be defined as “… any use of Web and Internet technologies to create learning experiences” (Horton, 2003, pp. 13). eLearning is essentially an evolved form of distance education, which Connolly and Stansfield (2007a) describe through a six-generation model, as depicted in Figure 1. The first generation (the ‘correspondence model’) was provided mostly through paper-based instruction, characterized by the mass production of educational materials. The difficulty with correspondence education has been the infrequent and inefficient form of communication between the instructor and the learners. Further, it was difficult to arrange for peer interaction in correspondence based distance education. The second generation (the ‘multimedia model’) was provided through integrated multimedia such as delivering courses via television or introducing material like audio and video tapes, computer-based learning (CBL) in addition to printed material. The third generation was provided through two-way communications media such as audio/video-conferencing and broadcast technology. The fourth generation of distance education (the first generation of eLearning) is defined as mainly passive use of the Internet, consisting primarily of conversion of course material to an online format, low-fidelity streamed audio/video, and basic mentoring using email. However, the educational philosophy still belongs to the pre-Internet era. The fifth generation of distance education (the second generation of eLearning) uses more advanced technologies consisting of high-bandwidth access, rich streaming media, online assessment (eAssessment) and LMS that provide access to course material, communication facilities, and learner services. The sixth generation of distance education (the third

51

Learning 2.0

generation of eLearning) is a more collaborative learning environment based much more on the constructivist epistemology, promoting reflective practice through tools like ePortfolios, Web 2.0 technologies such as blogs and wikis, online communities, and using interactive technologies such as online visualizations, games, and simulations.

We are also now starting to see the development of mLearning (mobile learning) through devices like PDAs (Personal Digital Assistants), mobile phones and smartphones, and tablet devices. For second and third generation eLearning, LMS support new approaches for people to learn and assist with the delivery but also with the way

Figure 1. Models of distance education (adapted from Connolly & Stansfield, 2007a)

52

Learning 2.0

in which information is presented leading to acquisition of new knowledge (Holmes & Gardner, 2006). eLearning and the use of LMS are now an integral part of most educational institutions with educational technologies witnessing exceptional levels of growth in recent years. To support the growth of LMS, many schools, colleges and universities have invested heavily in up-to-date technology. Also referred to as Virtual Learning Environments (VLEs), ‘learning platforms’, ‘distributed learning systems’, ‘course management systems’ and ‘instructional management systems’, LMS combine a range of course/subject management and pedagogical tools to provide a means of designing, building and delivering online learning. LMS are scalable systems that can be

used to support an institution’s entire set of teaching and learning courses. Although LMS are used extensively within educational institutions globally, their use is relatively low within vocational training companies. One reason for this may be that vocational trainers do not appreciate that some aspects of vocational training may lend itself well to online delivery. There are many different LMS products available, some at considerable cost and others available as open source. Regardless of which product is chosen, most LMS contain similar functionalities as shown in Table 1. As with most things, eLearning has advantages and disadvantages. The research literature cites many advantages of eLearning, particularly the convenience and flexibility offered by the

Table 1. Comparison of LMS functionality Features/Tools

LMS Blackboard ProSites

Moodle

Learnwise

Frog

E-portfolio

N

Y

Y

Y

File up-load

Y

Y

Y

Y

Notice/bulletin board

Y

Y

Y

Y

Course outlines

Y

Y

Y

Y

Assignments

Y

Y

Y

Y

Assessments

Y

Y

Y

Y

Multi-media resources

Y

Y

Optional extra

Y

Evidence gathering

Y

Y

Y

Y

Calendar

Y

Y

Y

Y

Administration tools

Y

Y

Y

Synchronous collaboration tools (video conferencing)

Y

Y

Optional extra

N

Forum/discussion board

Y

Y

Y

Y

Email

Y (Internal)

Y (Internal)

Y (Internal)

Y (Internal)

External links

Y

Y

Y

Y

Student home page

Y

Y

Y

Y

Real-time chat

N

Y

Y

Y

Quiz design

Y

Y

Y

Y

Costs

£6,655 per annum (200 users/licences) £3,152 per annum (100 additional users/ licences

Open source Additional costs if hosting required

Ranges from £3,300 per annum for 1,000 users to £27,500 per annum for 50,000 users

Bespoke system - £26,000 Yearly support - £4,500 Standard package - £22,500 Yearly support - £4,500

53

Learning 2.0

(asynchronous) ‘anytime, anywhere, anyplace’ education (McDonald, 2002), which gives learners time for research, internal reflection, and ‘collective thinking’ (Garrison, 1997). Moreover, the text-based nature of eLearning normally requires written communication from the learner, which along with reflection, encourage higher level learning such as analysis, synthesis, and evaluation, and encourage clearer and more precise thinking (Jonassen, 1996). In addition, eLearning courses also have the capability to present multiple representations of a concept, which allows learners to store and retrieve information more effectively (Kozma, 1987). It is also argued that increased social distance provides a number of distinct advantages to online conferences (synchronous or asynchronous). In written communications anonymity of characteristics such as gender, race, age, or social status can be preserved, which can reduce the feeling of discrimination and provide equality of social interaction among participants. In turn, this can permit the expression of emotion and promote discussion that normally would be inhibited (Gunawardena, 1993). eLearning is not without its disadvantages; for example (Connolly & Stansfield, 2007b): • •

• • •

•

54

Costs may initially exceed more traditional methods; More responsibility is placed on the learner who has to be self-disciplined and motivated; Some learners lack access to a PC/Internet or have difficulty with the technology; Increased workload for both students and faculty; Non-involvement in the virtual community may lead to feelings of loneliness, low self-esteem, isolation, and low motivation to learn, which in turn can lead to low achievement and dropout; Dropout rates tend to be higher in eLearning courses than in traditional face-to-face courses, often 10 to 20 percentage points higher.

Perhaps one of the most damaging criticisms is that some eLearning simply replicates the social organization of traditional education and training and that the potential benefits of eLearning - of personalized and accessible learning experiences - are missed. Taking this on board, this is one of the reasons this research is of particular importance. There is a high chance that the traditional nature of the organisation under investigation may simply deploy an LMS but not utilize it to its full potential.

Learner Expectations and Web 2.0 Web 2.0 is a term often associated with “… the social use of the Web which allow[s] people to collaborate, to get actively involved in creating content, to generate knowledge and to share information online” (Grosseck, 2009, pp. 478). Web 2.0 applications, such as blogs and wikis, are being introduced into LMS such as Blackboard and Moodle, providing students with increased flexibility in terms of how they communicate with fellow students and gain feedback from peers. In industry the use of Web 2.0 applications are being gradually introduced as organisations have begun to realise their potential for the purposes of learning and information sharing. The way in which education and training institutions and industrial organisations facilitate learning and information sharing is being determined by the expectations and prior learning experiences of the individuals within them. Within most learning or working environments today, there is a sense of expectancy that innovative learning and communication channels should already be in place to accommodate the diverse ways in which individuals learn. The aspects of institutional and organisational competitiveness that are closely related to the concept of the ‘knowledge economy’ means that institutions promoting learning via technological means now have an interest in developing and running learning initiatives with the minimum of effort (DeRouin, Fritzsche, & Salas, 2005). These particular factors are having an impact

Learning 2.0

upon the format and delivery of eLearning initiatives within educational and industrial contexts. Web 2.0 technologies are attractive within these contexts as they allow greater student independence and autonomy, increased collaboration and an increase in pedagogic efficiency (Franklin & Harmelem, 2007, pp. 1). For this reason, the use of Web 2.0 technologies could be an invaluable asset to those institutions delivering ‘on-the-job’ training, especially where an element of group work is undertaken. It also facilitates knowledge sharing and promotes learning in an industrial context, as well as taking account of pedagogical issues. This was important to our study given that one part of the vocational training to be delivered was a group-based project.

Social Software and Learning 2.0 It could be argued that the concept of eLearning is being enhanced by the rapid development of ‘social software’. McKelvie, Dotsika, and Patrick (2007) state that “social software is a community driven technology which facilitates interaction and collaboration and depends largely on social convention.” Though social software can be used on an individual basis it is predominately concerned with the notions of open and collective communication, dialogue and the ability to liaise with individuals collectively. Social software is having an effect upon eLearning delivery within education and industry as it has altered the way in which learning is taught and learnt. The use of social software allows the learner to generate knowledge and share their learning experiences on a collective level as well as allowing users to openly reflect upon what they have learnt. eLearning distinguishes itself from social software as it is predominately associated with electronic instruction and is better suited for education and training purposes. Web 2.0 is transforming the way in which people learn as the learning is predominately social and self-directed in nature whereas eLearning is normally associated with

individual learning. The use of social software and Web 2.0 technologies have given rise to the term ‘Learning 2.0’, sometimes referred to as eLearning 2.0, which broadly summarizes all opportunities arising from the use of social media for learning, education or training.

The Pedagogy of Learning 2.0 The interactive and collaborative nature of social software makes it highly suited towards sustaining and facilitating what are known as communities of practice (CoPs) or “groups of people informally bound together by shared expertise and passion for a joint enterprise” (Wenger & Snyder, 2000, pp. 139). In conjunction with the concept of CoPs, the learning theory of social constructivism appears to complement and accommodate the principles surrounding the use and learning benefits associated with Learning 2.0. The constructivist view of learning adopts the stance that learners do not learn individually from one another and stresses the relevance regarding the socio-cultural context of learning. Predominately, social constructivism contends that knowledge is formulated through the processes of social interaction and collaborative learning. Though the concept of social software can, in theory, support a wide range of learning approaches, it is inherently applicable towards the learning theories of social constructivism and CoPs. It has been generally regarded that one of the salient aspects of any CoP is its ability to construct and store collective knowledge in what has been referred to as a ‘shared repertoire of communal resources’ (Wenger, 2000). Additionally, CoPs are most usually distinctly defined by the concepts of collective understanding, mutual engagement and shared repertoire (Wenger, 2000). It could be argued that for this reason, the use of Learning 2.0 could be an invaluable asset to organisations. The examples given below demonstrate the growth of those technologies and tools that facilitate knowledge sharing in the wider community.

55

Learning 2.0

ePortfolios As well as the Learning 2.0 technologies, many educational institutions and organisations are utilising ePortfolios, the electronic version of the paper portfolio. As with LMS, the use of ePortfolios is also growing in popularity, particularly within vocational training. A portfolio can be defined as “a collection of documents relating to a learner’s progress, development, and achievements”; an ePortfolio simply indicates that some or all of the evidence is collected in digital form (Beetham, 2009). EDUCAUSE (2005) defines an ePortfolio as “a valuable online tool that learners, faculty, and institutions can use to collect, store, update, and share information. E-Portfolios allow students to reflect on their learning, communication with instructors, document credentials, and provide potential employers with examples of their work”. ePortfolios are well suited to education, vocational training and working environments as they capture the concept of lifelong learning and support individuals as they progress through school, college, Higher Education, training and employment (Richardson & Ward, 2005). Learners gather learning evidence and define these evidences through a self-reflection process. They attribute their competences to learning products or outcomes and reflect on how they acquired those competences. From a pedagogical perspective this process helps learners to better understand how they learn and helps them to become self-directed learners (Berlanga et al., 2008). ePortfolios can be classified into various types – assessment, presentation, learning, personal development and shared ownership (Curyer et al., 2007) but in reality most are a combination. If a standard approach was adopted for ePortfolios, institutions and organisations could share and exchange ePortfolio data, which could lead to the streamlining of the processes connected to prior learning, with student transitions through courses, and with training that involves either sequential or parallel movement through multiple institutions

56

and companies (Curyer et al., 2007). This could also help to fulfil the concept of an ePortfolio being utilised throughout lifelong learning. Richardson and Ward (2005) carried out an in-depth study of 12 different ePortfolios and found, amongst other things, that no two systems were identical or offered the same range of functions. However, it should be considered that an ePortfolio tends to be chosen on a ‘fit-for-purpose’ basis and often vendors may customise their ePortfolios to suit a particular customer, as is the experience of the authors. Most ePortfolios are driven by the learner; that is, the learner is responsible for the maintenance of the ePortfolio and decides who has access to its contents but in some environments, as in the study under investigation, this may not be desirable, as some aspects of vocational training need to be driven by the instructor, not the learner. Internal examiners and external verifiers often need access to trainee assessment material and that assessment material cannot be amended once verified. In our study, the instructors required a measure of customisation of the ePortfolio, which included allowing multiple assessors and the instructors themselves having overall control of the ePortfolio.

ePortfolio Use in Higher/ Further Education According to JISC (2008) ePortfolios represent the latest in a line of technology-based innovations that are becoming an integral part of the learning landscape in Higher and Further Education. Student ePortfolios developed out of faculty-assigned, print-based student portfolios as far back as the mid-80s. They were typically found in art-related disciplines or those that consisted of substantial written components, such as English studies, and gained greater importance in education during the mid-90s. Student ePortfolios are commonly found in college programmes where teachers use them to provide evidence of competence. This includes

Learning 2.0

communications, maths, business, nursing and engineering to record students’ learning experiences and skill set (Lorenzo & Itellson, 2005). The Open University in the UK has been using ePortfolios as an assessment tool in online courses for many years (Mason, Pegler, & Weller, 2004) and the University of Washington developed an ePortfolio in 2001 to allow students to record their entire educational learning experiences in an organised and integrated manner. A survey carried out by Strivens (2007) found that of those who participated in the survey, 20 institutions (54% of those with an ePortfolio) commented that the ePortfolio was available to all students across the institution. The increase in the use of ePortfolios in Higher Education is further supported by Beetham (2009) who states that work is currently being carried out to integrate LMS, student record systems and ePortfolio tools to provide formative feedback. It appears that educational institutions have embraced the concept of ePortfolios and appreciate their value.

ePortfolio Use in Vocational Training Portfolios (paper-based) have been utilised for many years in vocational training; for example, Austria has been using portfolios in teacher training for the past 12 years and covers topics such as supervision and professional upgrade in vocational education and is regarded as a working portfolio, as examination for teachers is impractical (Dorninger & Schrack, 2007). ePortfolios have been endorsed by some of the major vocational examiners such as City & Guilds in the UK. City & Guilds (2009) undertook a survey of 95 colleges and training providers and found that, although cost savings were significant, the main advantage was the reduction in time taken for candidates to complete their qualification when using an ePortfolio. However, although feedback was positive, only 16% of those surveyed used ePortfolios. Some centres expressed fear over technical glitches, which could result in lost work,

and some resistance to change was also noted. The literature demonstrates that ePortfolios offer a valid way to assess and ensure completion of an individual’s training, and while usage is not pervasive, uptake is growing.

METHODOLOGY The aim of this study was to determine whether eLearning could be used successfully within a vocational training environment for engineering students. More specifically, the objectives of the project were (i) to select suitable LMS and ePortfolio systems, (ii) develop some of the provision in an online format, and (iii) evaluate the success of this provision. This study started in late 2008 and the initial part of the study was completed in May 2010. Empirical data was obtained from a cohort of students going through their Induction programme, which was the first course developed in an online format, and from their final group project before completing their training. In addition, the instructors were surveyed in order to ascertain their views of the LMS and the use of the wiki and forum for the group project. The instructors’ survey also included questions relating to the use of the LMS as a learning tool. For this study, the research philosophy of interpretivism was adopted. Interpretivism is an epistemology that advocates that the researcher must understand the differences between humans in their social roles as actors. This philosophy places emphasis on conducting research among people as opposed to objects, such as cars and houses. Interpretivism stems from the intellectual tradition of phenomenology, which is concerned with the ways in which humans make sense of the world around them (Saunders, Lewis & Thornhill, 2007). According to Whisker (2001), phenomenology encourages both quantitative and qualitative methods. However, it should be considered that quantitative research is more often associated with positivism. McBride and Schostak (1994) state:

57

Learning 2.0

“Where quantitative forms of research, employing questionnaires and sampling procedures attempt to eradicate the individual, the particular and the subjective, qualitative research gives special attention to the subjective side of life...qualitative researchers are more likely to ask how it feels... they focus upon the social construction of such things...”. As such, this study is therefore adopting a mixed methodology with an Action Research and within case study approach as the authors are actively involved in the project. Gerring (2007, pp. 20) defines case study as “the intensive study of a single case where the purpose of that studyat least in part- to shed light on a larger class of cases”. It is hoped that at the termination of the project it will be possible to develop a framework that will aid practitioners in the successful implementation of an LMS in vocational training, as well as promote a rich area for future research. The next section will outline the company under investigation, the type of training delivered and how the chosen technologies were selected.

Case Study: Vocational Training This project came about when University of the West of Scotland entered into a two-year partnership programme with a local training organisation, East Kilbride & District Engineering Group Training Association (EKGTA), to explore how much of a Modern Apprenticeship programme could be delivered online. EKGTA is an employer-led training provider for the engineering industry with charitable status. Established in 1966 and recognised as one of the premier training groups throughout Scotland, it aims to serve the needs of the employer, whilst ensuring candidates have the opportunity to develop the knowledge and skills necessary in employment. The Association specialises in training Modern Apprentices at craft and technician levels, and in basic engineering skills training to national standards. EKGTA provides training in other disciplines, such as, Health & Safety, Computer Based Training, Professional

58

Development but apprentice engineering training is EKGTA’s core business and for this reason the main focus is directed towards that training. A Modern Apprenticeship Programme may involve: • • • • •

A period of training in an approved training centre (off the job). Completion of a Level 2 Vocational Qualification. Attainment of core skills to intermediate one level (minimum). Completion of a National Certificate (day release at FE College). Completion of a Level 3 Vocational Qualification in company.

Level 2 competence involves the application of knowledge and skills in a significant range of varied work activities, performed in a variety of contexts. Some of the activities are complex or non-routine, and there is some individual responsibility and autonomy. Collaboration with others may often be a requirement On the other hand, Level 3 competence involves the application of knowledge and skills in a broad range of varied work activities performed in a wide variety of contexts, most of which are complex and nonroutine. There is considerable responsibility and autonomy, and control or guidance of others is often required. There are six instructors who deliver the practical element of the qualification with the academic element delivered onsite by a local Further Education college. Once the trainees return to their company, advisors visit trainees on average every 12 weeks and oversee the continuation of the training. Management appreciate the advancements in technology that could potentially support and enhance the training programmes offered by the Association, however, as the project has progressed, it has become obvious that this sentiment is not echoed by all staff. Before the project, technology within EKGTA was used primarily for the record-

Learning 2.0

ing of completed assignments (ePortfolio) and the storing of lecture material, and was therefore used far below its potential. In the early stages of the project the pilot of the LMS was to be rolled out within the electrical area of the workshop, but on further discussion it was decided that this would not be an inclusive approach. The decision was taken to implement the LMS throughout the entire organisation in two stages: the Induction programme and the group project, thus including all instructors.

Selecting the LMS In discussion with EKGTA, a number of features were identified that the chosen LMS had to support. Existing students were also consulted. Two informal group forums were set up with ten participants in each. This helped to identify those features that students believed would enhance learning. Almost all students agreed that 24x7 access to learning materials would be advantageous, more use of multimedia, such as video demonstrations and interactive quizzes, with immediate feedback to reinforce learning. To identify a suitable LMS, it was first necessary to carry out desk research on a number of different LMS to identify the functionalities that each one supported and to match these against the company’s and the students’ criteria. A short list of potential systems was drawn up that included Blackboard ProSites, Learnwise, Frog and Moodle, Moodle being the only open source option. The outcome of the research demonstrated that there was very little difference in the functionalities of each, as shown in Table 1. The ability to automatically map student evidence of their work to individual components of assessment was regarded by the instructors as the most important function that needed to be supported by the ePortfolio component as this was their most time intensive activity, however, after much research it became apparent that this function was not readily available with any of the

shortlisted LMS systems and it would be necessary to select a separate ePortfolio system to the LMS, although it was hoped the two could be integrated together in some way. After extensive analysis and consultation with management and staff, Moodle was chosen as the LMS as it was highly modular and provided the same features as the commercial systems but at no cost to the company. The other LMS investigated were more suited to educational institutions with large numbers of students. For this project, the maximum student uptake in one year would be limited to 120 and so the need for a large commercial product could not be justified. In addition, Moodle has a large and growing community with almost 40 million users and over 50,000 sites worldwide at the time of writing (Moodle, 2010). The next stage of the project was to identify a suitable ePortfolio system.

Selecting the ePortfolio Again, a short list of potential ePortfolio systems was drawn up based on instructors’ requirements, which included Learning Assistant, One File, Pebble Pad and Mahara, which is open source and an add-on module for Moodle. EKGTA previously used two ePortfolios, one at Level 2 and one at Level 3, which were not integrated in any way. The ePortfolio at Level 2 gives on-campus students access to all lecture material, standard assessment documentation, and in some instances multimedia. At Level 3 advisors use the Modern Apprenticeship online ePortfolio system. At this level, students are off-campus, therefore regular remote communication between advisors and students needs to be undertaken. The Modern Apprenticeship online ePortfolio did not fulfil some of the criteria required by the instructors and was no longer being fully supported and so was excluded from further consideration. After in-house evaluation, it was concluded that Learning Assistant best suited the requirements. Learning Assistant is marketed as an ePortfolio and eAssessment solution for training

59

Learning 2.0

centres that deliver vocational qualifications. It has been designed specifically to meet the needs of the vocational training environment. Learning Assistant, as well as being fully integrated from Level 1 through to Level 4, is a fully supported system with regular updates to meet the changing needs of the students and any changes to the engineering qualification. In addition, the Learning Assistant vendor could customise the system to support multiple assessors and facilitate the automatic mapping of student evidence of their work to individual components of assessment.

Identifying Priority Features After researching a variety of LMS and ePortfolio systems to identify their generic features and the system(s) that is more suited to the vocational training environment, it was then necessary to survey the six workshop instructors and one training advisor to identify those features that would be regarded as a priority for implementation. This gave an indication of where the end-users see technology enhancing learning and to help in the delivery of training. Table 2 provides details of this survey. The lower the number assigned, the higher importance assigned. The data demonstrates that the top five priority features are: •

• • • •

ability to automatically map student evidence of their work to individual components of assessment; a suitable ePortfolio system; evidence gathering; uploading of assignments; a file upload capability.

These features were subsequently prioritised for implementation.

Implementation In the early stages of the project the pilot of Moodle was to be rolled out within the electrical area of the workshop, but on further discussion 60

it was decided that this would not be an inclusive approach. The decision was taken to implement Moodle throughout the entire workshop, thus adopting an inclusive approach. All trainees go through a week long induction programme - a general induction and a workshop induction. A course was developed and populated with two units: general induction and workshop induction. All lecture material and presentations were uploaded to Moodle and self-assessment quizzes designed to reinforce learning. Learning Assistant was customised by the vendor and installed for students to use and a single sign-on was developed between Moodle and Learning Assistant so students only had to log in once to access both systems. Figure 2 shows an example page in the Moodle system and Figure 3shows the reporting system within the ePortfolio.

FINDINGS AND DISCUSSION In this section, we discuss the findings from the first year of using the LMS for training the engineering students. We start with an evaluation of the results from the Induction programme followed by the group project.

Induction Programme Results A five-day Induction programme was run for a cohort of students in Autumn 2009. Demographic information was collected at the start of Induction and an evaluation questionnaire was distributed at the end.

Demographic Questionnaire 26 participants completed the demographic questionnaire. 25 (96%) of the respondents were male and 1 (4%) of respondents were female. The mean age of participants was 17 (SD = 0.748) with a range from 16 to 19. 17 (65%) of participants started an apprenticeship at EKGTA when leaving school, 9 (35%) of participants did not. The

Learning 2.0

Table 2. Survey results for priority features ePortfolio

Student home page

Course outlines

Assignments

Map evidence to assessment

Multimedia

1

6

4

7

2

2

4

13

16

3

2

14

1

13

8

6

2

10

6

16

4

5

3

1

5

12

13

8

1

7

1

12

3

5

2

4

4

10

5

1

2

8

22

82

53

35

14

46

File up-load

Evidence gathering

Notice board

External links

Quiz design

Forum

3

3

5

8

14

15

7

1

12

11

15

8

9

3

7

12

11

14

2

7

12

8

13

14

4

2

14

11

16

9

6

11

14

7

8

13

7

3

9

12

13

11

38

30

73

69

90

84

Real time chat

Calendar

Email

Admin tools

16

2

6

3

9

10

5

6

15

16

4

5

15

11

10

9

15

10

3

6

15

9

16

10

16

14

15

6

101

72

59

45

majority of the participants (18, 69%) left school in 2009, 6 (23%) left school in 2008 and 2 (8%) left in 2007. The majority of the participants (23, 88%) indicated that they had no other full-time employment in the past while 3 (12%) did. 1 participant indicated that they worked in the hospitality sector. 19 (73%) of participants indicated that they had not attended college prior to attending the training course, whereas 6 (23%) of participants indicated that they had attended college first. The courses

that were undertaken were primarily in the areas of construction (joinery, roofing, brick-laying, etc) and creative industries (art, fashion design, media, etc). These courses were undertaken at intermediate, NC and HNC level. Table 3 shows the standard grade qualifications and Table 4 the intermediate/higher qualifications achieved by the participants.

61

Learning 2.0

Figure 2. Screenshot of Moodle page

Table 3. Standard grade qualifications achieved

Table 4. Intermediate or higher qualifications achieved

Standard Grade Qualification

Number with qualification

Mathematics

21 (81%)

Higher Grade Qualification

Number with qualification

English

24 (92%)

Mathematics

20 (77%)

Physics

17 (65%)

English

15 (58%)

Craft and Design

13 (50%)

Physics

11 (42%)

French

16 (61.5%)

Craft and Design

5 (19%)

Biology

6 (23%)

French

3 (12%)

Geography

13 (50%)

Biology

1 (4%)

Chemistry

10 (38%)

Geography

5 (19%)

Modern Studies

13 (50%)

History

5 (19%)

German

2 (8%)

None

3 (12%)

Spanish

1 (4%)

Modern Studies

2 (8%)

20 (77%)

Others

14 (54%)

Other

Evaluation Questionnaire Students were asked to complete a questionnaire at the end of the Induction programme. 41 participants completed the evaluation questionnaire. Participants were asked to rate their level of interest in the training on a scale of 1 to 4 (1 being a low

62

level and 4 being a high level) for each section of the Induction course, namely Day 1 General Introduction, Workplace Environment, Tools & Maintenance, Business Improvement Techniques, ICT, COSHH & Hand Care, Fire Prevention, Risk Assessment, Electrical Safety, Measurements and Materials. The overall results were generally posi-

Learning 2.0

Figure 3. Screenshot of ePortfolio page

tive indicating that the training was sufficiently interesting for the participants. The area of the training that had the highest level of interest was Electrical Safety (Mean = 3.41, SD = 0.59). The areas of training that received the lowest levels of interest were Business Improvement Techniques (Mean = 2.93, SD = 0.65) and the General Introduction on the first day (Mean = 2.93, SD = 0.80). Participants were also asked to rate whether they found the training sufficiently challenging (1 = lowest, 4 = highest). Electrical Safety was rated as the most challenging (Mean = 3.29, SD = 0.60) while the General Introduction on the first day was rated as the least challenging (Mean = 2.75, SD = 0.71). The overall approachability of the instructors was rated highly by the participants in all areas. The average rating for approachability across all areas was Mean = 3.46, SD = 0.86 with a range of 3.32 to 3.59 out of 4. Participants were asked to rate how much they believed that their knowledge improved in each

of the areas. The area rated as having the highest level of knowledge improvement was COSHH (Control of Substances Hazardous to Health) & Hand Care (Mean = 3.51, SD = 0.60). The area rated as having the least level of knowledge improvement was the General Introduction (Mean = 3.17, SD = 0.68). Participants were also asked to rate how appropriate they considered the duration of the activities to be. Overall the results were not particularly positive. The most appropriate rated duration was for Electrical Safety (Mean = 3.15, SD = 0.76) and the least appropriate rated duration was for the General Introduction (Mean 2.75, SD = 0.74). Participant ratings for the helpfulness of teaching aids were generally positive. The highest rating for teaching aids was in the area of Electrical Safety (Mean = 3.49, SD = 0.51) and the lowest rating was in the Helpful Work Place Environment teaching area (Mean = 3.22, SD = 0.52). Participants were also asked to rate if Moodle helped in each of the teaching areas. Table 5 shows the

63

Learning 2.0

rankings of perceived help Moodle provided in each of the teaching categories. Overall, the results of the evaluation questionnaire were generally positive. Electrical Safety consistently achieved the highest ratings for level of interest, challenge, appropriate duration and perceived help provided by Moodle. The General Introduction on the first day received the highest amount of criticism primarily because of its duration as it was considered by participants to be too long and lacked participation and interaction. Participants provided some of the following qualitative comments regarding the duration of the General Introduction: Day 1 was not that interesting because there was not a lot of participation - it was all listening. I think the 3 day introduction as a whole could be delivered as a participation lesson. The first day was not interesting enough. We were just thrown information and didn’t have enough time to absorb the information. Duration - induction was too long and it could involve the trainees more to make it more exciting.

Participants also provide qualitative points for action and improvement. These primarily included: more interaction, more participation and the reduction of the duration of the General Introduction.

GROUP PROJECT The group project was designed to develop and promote team-working skills that are applicable to real-life working in the engineering industry and is the last assessment the students take before going out into industry. The overall objective of the project is for team members to build a truck within a nominal budget. Each member of the team is assigned a role, for example, project manager, chairman, secretary or quality control. To trial Moodle in the group project, it was decided to introduce a wiki for formal recording of the project, a forum to facilitate communication when not all members of the team were present and an assignment folder to allow candidates to upload all documentation for assessment. Two tests were designed: a pre-test to help identify current knowledge and a post-test to help identify new knowledge and students’ views of the wiki and forum. A total of 43 candidates participated in the

Table 5. Ratings of help provided by Moodle in each teaching category Teaching Area

64

Rank

Mean

Standard Deviation

Risk Assessment

1st

3.56

0.50

Electrical Safety

1st

3.56

0.55

Materials

1st

3.56

0.55

Measurements

2nd

3.51

0.60

Work Place Environment

3rd

3.49

0.51

Tools and Maintenance

3rd

3.49

0.55

Fire Prevention

3rd

3.49

0.55

COSHH and Hand Care

4th

3.46

0.50

Business Improvement Techniques

5th

3.44

0.59

ICT

6th

3.41

0.59

Day 1General Introduction

7th

3.33

0.58

Learning 2.0

group project. The candidates were divided into 6 teams with 7-8 students per team. The findings are discussed below.

If someone comes up with an idea then they can post it at anytime even when team members are not there.

Pre-Project Test

You can have access when you are off and record progress.

33 participants completed the pre-test questionnaire. All of the respondents were male. The mean age of participants was 17.18 (SD = 0.73) with a range from 16 to 19.8. The majority of participants (30, 91%) indicated that they had been involved in group work before, while 3 (9%) had not. The majority (27, 82%) of the participants who had participated in group work before indicated that they took part in the group work at school, 10 (30%) at college and 3 (9%) at work. None of the participants had ever used a wiki before. The majority of the participants (24, 73%) described a wiki as an ‘an online collaboration tool with built in tracking’, 7 (21%) described a wiki as ‘a social networking tool’ and 2 (6%) described it as ‘an online instant messaging tool’. The majority of participants (21, 64%) indicated that they had used an online forum before while 12 (35%) had not. 18 (55%) participants had used an online forum for social use, 5 (15%) had used an online forum for a school project, 2 (6%) had used one for a college project and as part of a work project. 12 (36%) of participants believed that an online forum was ‘a social networking tool’, 14 (42%) believed that it was ‘an online chat room’ and 4 (12%) believed that it was ‘an online collaboration tool with built in tracking’. 23 (70%) of participants believed that the use of a wiki 24x7 would encourage teamwork, while 10 (30%) believed this not to be the case. Those who indicated that a wiki would encourage team work gave some of the following reasons: Allows you to keep every member of the team up to date.

Those who indicated that a wiki would not encourage teamwork gave some of the following reasons: Better to discuss the project in person so that you are able to give them a clearer picture of what is done. The whole team would have to use it to benefit and they were not. 22 (67%) of participants believed that the use of a forum would help open up communication between members while 11 (33%) believed this not be the case. Those indicating that the forum would open up communications gave the some of the following reasons: If one team member is not in one day he/she can then see what his/her team has done that day. It can get everyone involved. 17 (52%) of the participants were not aware of any project management tools, while others listed Excel, Word, PowerPoint, efacts, wiki, graphs, flip charts, CAD, Gantt charts and Moodle as project management tools. The majority (22, 67%) indicated that they had not used any project management tools in the past while 11 (33%) had. The majority of participants (21, 64%) indicated that they thought a Project Route Map to be ‘a tool that helps to identify all areas of the project that need to be considered’, 11 (33%) believed it to be ‘a tool that is used to help plan the project and then consulted on a regular basis’ and 1 (3%) believed it to be ‘a tool that is only used to plan

65

Learning 2.0

the project but then filed away’. Participants were asked to list the project resources that they were aware of, 12 (36%) indicated that they were not aware of any, 3 (9%) indicated that they were aware of Moodle as a project resource; 4 (12%) of participants also mentioned the Internet and books as a project management resource. 18 (55%) of participants believed a budget to be ‘a set amount of money that can be spent on the project from start to finish’, 14 (42%) of participants believed that a budget was ‘an amount of money that should cover all cost of the project from start to finish’ and 1 (3%) believed that is was ‘the amount of money that is set for the first stage of the project but is ‘topped up’ as the project progresses’. 14 (42%) of participants identified the information pyramid as something that ‘allows the presenter to identify between the different types of information ensuring that the most important information is not omitted’, 10 (30%) as something that ‘helps the presenter to divide the information into equal parts’ and 9 (27%) as something that ‘helps the presenter to ensure that the correct information is conveyed during a presentation’.

Post-Project Test 26 participants completed the post-test. Participants were asked how much they agreed with the following statement: ‘everyone participated in the group project’. To get an overall rating of the perception of overall participation, strongly agree was recorded as 5, agree as 4, neither agree nor disagree as 3, disagree as 2 and strongly disagree as 1. The mean rating for perceived participation was 2.69 (SD = 1.19) with a range from 1 to 5. This indicates that the level of perceived participation was not particularly high with the majority of participants either disagreeing 11 (42%) or strongly disagreeing 4 (15%) that everyone was participating in the group activity. 13 (50%) of participants described a wiki as ‘an online collaboration tool with built in tracking’, 8 (31%) described it as ‘a social networking

66

tool’, 2 (8%) described it as ‘an online chat room’ and ‘an online instant messaging tool’. 10 (38%) identified a forum as ‘an online collaboration tool with built in tracking’, 7 (27%) identified it as ‘a social networking tool’, and ‘an online chat room’. 2 (8%) described it as ‘an instant messaging tool’. The majority of participants 15 (58%) said that they did not think that a wiki promoted teamwork, while 10 (38%) believed that it would promote teamwork. Those who indicated that the wiki would promote teamwork gave some of the following reasons: People could catch up on work they may have missed. Helped each other with the difficulties. Those who indicated that the wiki would not promote teamwork gave some of the following reasons: Took too long to complete and was repetitive. We didn’t use it much. Participants were asked how much they agreed with the following statement ‘if the wiki was a compulsory element of the group project all team members would have used it’. To produce a mean score of the expected participation level, strongly agree was recoded as 5, agree as 4, neither agree nor disagree as 3, disagree as 2 and strongly disagree as 1. The mean rating was 2.76 (SD = 1.03) with a range of 1 to 4. This means that approximately 50% of participants would use the wiki if it were a compulsory part of the group project. 13 (50%) of participants believed that the forum helped with communication and 13 (50%) did not. Those who believed that the forum helped with communication gave some of the following reasons: All people can communicate if not present.

Learning 2.0

People could see things they missed. Those who believed that the forum did not help with communication gave some of the following reasons: We didn’t use it much. Easier to talk. Participants were asked to rate their level of agreement of this statement: ‘If the forum was a compulsory element of the project, all team members would have used it more for communicating with those team members who are not present on a full-time basis’. To produce a mean score of the expected participation level, strongly agree was recoded as 5, agree as 4, neither agree nor disagree as 3, disagree as 2 and strongly disagree as 1. The mean score rating of expected participation was 3.23 (SD = 0.95) with a range from 1 to 5. This indicates that the forum was more popular than the wiki in terms of expected participation. Participants were asked to list all of the project management tools that they were now aware of, 6 (23%) of participants mentioned a wiki, 8 (31%) answered ‘none’, 2 (8%) mentioned Gantt charts and 1 (4%) mentioned Moodle. 13 (50%) of participants now identified a Project Route Map as: ‘a tool to help identify areas of the project that need to be considered’, 2 (8%) identified it as ‘a tool used to plan the project and is then filed away’ and 11 (42%) identified it as ‘a tool that is used to identify areas of the project that need to be considered and consulted on a regular basis’. Participants were asked to list all of the project resources that they were familiar with now that the project was complete: 6 (23%) of participants mentioned a ‘wiki’, 11 (42%) said ‘none’ and a small number mentioned talks, PowerPoint, Gantt charts and Moodle. The majority of participants (14, 54%) identified a budget as ‘an amount of money that can be spent on the project to cover all project costs from start to finish’, 11 (42%)

identified it as ‘a set amount of money that can be spent on the project from start to finish’, and 1 participant (4%) identified it as ‘the amount of money that is allocated to material for the project’. 6 (23%) of the participants identified the information pyramid as ‘a tool that helps the presenter to divide the information into equal parts’, 9 (35%) as ‘a tool that helps the presenter to ensure that the correct information is presented during the presentation’ and 11 (42%) as ‘a tool that helps the presenter identify between the different types of information ensuring that the most important information is not omitted’.

Comparison of Pre and Post-Tests 21 participants completed both the pre- and the post-tests. A Wilcoxon Signs ranked test indicated that the increase in knowledge was not significant with regards to wikis (Z = -.890, p = 0.03) and forums (Z = -.816, p = 0.414). However, when combined together, a Wilcoxon Signs ranked test indicated that there was a significant decrease in forum and wiki knowledge (Z = -1.807, p = 0.04). The mean in the pre-test was 1.14 out of 2 (SD = 0.65) and the mean in the post-test was 0.81 out of 2 (SD = 0.68). This was primarily due to a significant reduction in wiki knowledge after the training (Z = -.890, p = 0.03). This is possibly due to the fact that the instructors did not promote the wiki as it was not compulsory and instead suggested that the students used the forum to maintain the project documentation. A Wilcoxon Signs ranked test indicated that the increase in knowledge was not significant with regards to project route maps (Z = -1.000, p = 0.317), the information pyramid (Z = -.378, p = 0.705) and budgets (Z = -1.265, p = 0.206). However, when combined together there was a significant increase in the project management knowledge between the pre-test and post-test (Z = -1.615, p = 0.05). The mean in the pre-test was 1.10 out of 3 (SD = 0.94) and the mean in the posttest was 1.42 out of 3 (SD = 0.87). A significant

67

Learning 2.0

correlation was detected between undertaking group work previously and the knowledge score for budget in the pre-test (r = -.414; n = 21; p = 0.03). There was also a significant correlation between undertaking group work previously and the knowledge score for forums in the post-test (r = -.459, p = 0036). A significant correlation between using project management tools previously and knowledge scores about wikis in the post-test (r = -.539, p = 0.01) was also detected. In addition, there was a significant correlation between using project management tools previously and knowledge about Project Route Maps in the post-test (r = -.408, p = 0.03). While some of these correlations were expected, some of the others were not and these will be tested again with subsequent cohorts going through the project module.

I am keen to start uploading my lectures and restructure my lessons to incorporate Moodle All courses could be delivered online but investment in teaching material, software and manpower would have to be made to progress the project Reinforcement of the tutor lead sessions and the ability to access the information from anywhere in the centre It gives those trainees who are self motivated the opportunity to keep pace or catch Groups were able to share information with team members and use it as a note pad for ideas

It is limited to providing a storage facility for course material with the odd quiz to reinforce training

There was a definite divide between the instructors. Some could identify the support that Moodle and the Web 2.0 tools could bring to the training programme, whist others did not. Part of this could be attributed to the traditional nature of engineering; however, the instructors who were new to the company were more open to new ideas. The students who were working under the supervision of those instructors who promoted the use of the technologies were far more receptive and willing to use Moodle and the Web 2.0 tools. However, the students who were supervised by those instructors who did not encourage the use of the technologies were far more resistant to their use. This demonstrates that there must be enthusiasm and guidance in the use of technology to ensure its successful uptake. It should be noted that these issues were brought to the attention of management by the authors, but management felt they could not force instructors to use the technologies.

The wiki only helped to confuse matters in this short space of time

CONCLUSION

Some of the positive comments made by instructors were as follows:

This chapter set out to discuss the impact on learning with the introduction of a LMS into a

INSTRUCTORS RESULTS Instructors were surveyed to obtain feedback on their opinion of the Web 2.0 tools used during the group project, to gauge their acceptance of Moodle, and to identify areas where they believed the LMS could be utilised further. The course is a practical course and would not lend itself well to online delivery It was long winded and the trainees had to build it themselves, the wiki would be ideal for large academic projects that spans a university course or is used in a notational sense

68

Learning 2.0

vocational training course and to answer (i) can technology supplement the hands-on nature of vocational training, (ii) can the use of LMS and Web 2.0 tools (specifically wikis and forums) aid training, and (iii) can the pilot be considered a success? Much of the work carried out in this sector is hands-on and on-the-job training but it also consists of a substantial knowledge element that would lend itself well to the use of eLearning. From the literature review provided within this chapter, it may be argued that the use of educational technologies for practically-based courses (e.g. vocational training) should be a relatively straight forward process. Early eLearning was used in a practical capacity; however, the case study demonstrates that the introduction of learning technologies into a vocational training course was received with mixed reviews. This could largely be accredited to the traditional nature of engineering training but what was obvious from the data collected and by the authors’ experience in working with the organisation, much of the opposition to the technology use was due to the lack of flexibility and understanding from the instructors. From the analysis it can be seen that the technology was received with a mixed response. The online induction course was well received by the students (and was also accepted by the instructors). Unfortunately, the use of wikis and forums for the project module, while well received by the students was not fully adopted by the instructors, possibly as they were not confident of their knowledge and use of the technologies. As a result, they made the use of the wiki optional and instead asked the students to use the Moodle forum to maintain their project documentation. This caused the students to become confused as to the distinction between forums and wikis, resulting in a decrease in learning. The authors conclude that the research demonstrates that educational technologies could potentially aid the vocational training of apprentices and that this area could offer a rich source of future research. However, it is early days to design a framework that will aid practitioners in

the successful implementation of a LMS into a private vocational training organisation but as the empirical data grows this should become possible. At this stage the authors suggest, tentatively, that eLearning can support and in some instances replace some elements of a vocational programme. The authors acknowledge that the practical element would be very difficult to replace with technology but it could be supplemented through other tools such as multimedia and even games technology. Educational technologies offer the opportunity to open up learning in the vocational training sector, rather than restricting it to the traditional 9-5 scenario. The authors would recommend that those educators who deliver traditional vocational training courses be fully consulted during the implementation process as well as receive the necessary training needed to highlight the potential benefits that technology could bring to the overall course being delivered. If buy-in from students can be obtained and instructors can learn to appreciate the value that could be gained from educational technology then its use could diffuse throughout the vocational training sector.

ACKNOWLEDGMENT This project received financial support from the Knowledge Transfer Partnerships programme (KTP). KTP aims to help businesses to improve their competitiveness and productivity through the better use of knowledge, technology and skills that reside within the UK Knowledge Base. KTP is funded by the Technology Strategy Board.

REFERENCES Beetham, H. (2009). E-portfolios in post-16 learning in the UK: Developments, issues and opportunities. Retrieved October 2, 2009, from http://www.jisc.ac.uk/uploaded_documents/eportfolio_ped.doc

69

Learning 2.0

Berlanga, A. J., Sloep, P. B., Brouns, F., BitterRijpkema, M. E., & Koper, R. (2008). Towards a TENCompetence e-portfolio. [iJET]. International Journal of Emerging Technologies in Learning, 3, 24–28. City & Guilds. (2009). How are centres using ePortfolios. Retrieved November 12, 2009, from http://www.cityand guilds.com/44912.html Connolly, T. M. & Stansfield, M. H. (2007a). From e-learning to games-based e-learning. International Journal of Information Technology and Management, 26(2/3/4), 188-208. Connolly, T. M., & Stansfield, M. H. (2007b). From e-learning to online games-based e-learning: Implication and challenges for higher education and training. In Li, F. (Ed.), Social implications and challenges of e-business. Hershey, PA: IdeaGroup Publishing. Curyer, S., Leeson, J., Mason, J., & Williams, A. (2007). Developing e-portfolios for VET: Policy issues and interoperability. Australian flexible learning framework. Retrieved October 16, 2009, from http://e-standards.flexiblelearning.net.au/ docs/vet-eportfolio-report-v1-0.pdf DeRouin, R. E., Fritzsche, B. A., & Salas, E. (2005). E-learning in Organizations. Journal of Management, 31(6), 920–940. doi:10.1177/0149206305279815 Dorninger, C., & Schrack, C. (2007). E-portfolio’s in education-learning tools or means of assessment? In Proceedings of the International Computer-Aided Learning Conference (ICL2007), Villach, Austria. EDUCAUSE. (2005).Retrieved on February 11th, 2011, from E-Portfolios Web page, URL: http://www.educause.edu/ELI/Archives/EPortfolios/5524

70

Franklin, T., & Harmelen, M. (2007). Web 2.0 for content for learning and teaching in higher education. Joint Information Systems Committee (JISC) report. Retrieved August 8, 2010, from http://ie-repository.jisc.ac.uk/148/1/web2content-learning-and-teaching.pdf Garrison, R. (1997). Computer conferencing: The post-industrial age of distance education. Open Learning, 12(2), 3–11. doi:10.1080/0268051970120202 Gerring, J. (2007). Case study research: Principles and practices. Cambridge, UK: Cambridge University Press. Grosseck, G. (2009). To use or not to use Web 2.0 in higher education? Procedia Social and Behavioral Sciences, 1, 478–482. doi:10.1016/j. sbspro.2009.01.087 Gunawardena, C. N. (1993). The Social context of online education. In Proceedings of the Distance Education Conference, Portland, Oregon. Holmes, B., & Gardner, J. (2006). E-learning concepts and practice. London, UK: SAGE Publications. Horton, W. (2003). E-learning tools and technologies: A consumer’s guide for trainers, teachers, educators, and instructional designers. Indianapolis, IN, USA: Wiley. JISC. (2008). Effective practice with e-portfolios: Supporting 21st century learning. Joint information systems committee. London, UK: JISC. Jonassen, D. H. (1996). Computer-mediated communication: Connecting communities of learners. Computers in the Classroom (158–182). Edgewood Cliffs, NJ: Prentice-Hall, Inc. Kozma, R. (1987). The implications of cognitive psychology for computer-based learning tools. Educational Technology, 27(11), 20–25.

Learning 2.0

Lorenzo, G., & Ittelson, J. (2005). An overview of ePortfolios, Educause learning initiative. Mason, R., Pegler, C., & Weller, M. (2004). Eportfolios: An assessment tool for online courses. British Journal of Educational Technology, 35(6), 717–727. doi:10.1111/j.1467-8535.2004.00429.x McBride, R., & Schostak, J. (1994). What is qualitative research. Retrieved May 7, 2009, from http://www.enquireylearning.net/ELU/Issues/ Research/Res1Ch1.html McDonald, J. (2002). Is “as good as face-to-face” as good as it gets? Journal of Asynchronous Learning Networks, 6(2), 10–23. McKelvie, G., Dotsika, F., & Patrick, K. (2007). Interactive business development, capturing business knowledge and practice: A case study. The Learning Organization, 14(5), 407–422. doi:10.1108/09696470710762637 Moodle. (2010). Moodle Statistics. Retrieved August 25, 2010, from http://moodle.org/stats/ Richardson, H. C., & Ward, R. (2005). Developing and implementing a methodology for reviewing e-portfolio products, The centre for recording achievement (CRA). Retrieved October 10, 2009, from http://www.jisc.ac.uk/uploaded_documents/ epfr.doc Saunders, M., Lewis, P., & Thornhill, A. (2007). Research methods for business students (4th ed.). Pearson Education Ltd. Strivens, J. (2007). A survey of e-pdp and ePortfolio practice in UK higher education. UK: Higher Education Academy. Wenger, E. C. (2000). Communities of practice and social learning systems. Organization, 7(2), 225–246. doi:10.1177/135050840072002 Wenger, E. C., & Snyder, W. M. (2000). Communities of practice: The organizational frontier. Harvard Business Review, 78(1), 139–145.

Whisker, G. (2001). The Postgraduate Research Handbook. Basingstoke, UK: Palgrave MacMillan.

ADDITIONAL READING Agostini, A., De Michelis, G., & Loregian, M. (2009). Using blogs to support participative learning in university courses. Int. J. Web Based Communities, 5(4), 515–527. doi:10.1504/ IJWBC.2009.028087 Baxter, G. J., Connolly, T. M., & Stansfield, M. (2009). The use of blogs as organisational learning tools within project-based environments. Int. J. Collaborative Enterprise, 1(2), 131–146. doi:10.1504/IJCENT.2009.029285 Berge, Z. L., & Muilenburg, L. Y. (2005). Student Barriers to Online Learning: A factor analytic study. Distance Education, 26(1), 29–48. doi:10.1080/01587910500081269 Cartelli, A., Maillet, K., Stansfield, M. H., Connolly, T. M., Jimoyiannis, A., Magalhaes, H., & Toland, J. (2008). Identifying and promoting best practice in virtual campuses and e-searning, ED-MEDIA 2008 Conference - World Conference on Educational Multimedia, Hypermedia & Telecommunications, Vienna, Austria (30 June-4 July 2008) (pp. 897-902). Chapman, C., Ramondt, L., & Smiley, G. (2005). Strong community, deep learning: Exploring the link. Innovations in Education and Teaching International, 42(3), 217–230. doi:10.1080/01587910500167910 Cole, M. (2009). Using Wiki technology to support student engagement: Lessons from the trenches. Computers & Education, 52(1), 141–146. doi:10.1016/j.compedu.2008.07.003

71

Learning 2.0

Connolly, T. M., & Stansfield, M. H. (2007). Developing constructivist learning environments to enhance learning. In N. Buzzetto-More (Ed.), Principles of effective online teaching: A handbook for experienced teachers developing e-learning, 19-38.

Lorenzo, G., & Ittelson, J. (2005). An overview of e-portfolios. D.Oblinger (Ed.), Educause learning initiative paper (pp. 1-28). Retrieved October 18, 2010, from: http://www.sorteoudla.org.mx/ promueve/ciedd/CR/tecnologia/AnOverviewofEPortfolios.pdf

Gerrard, C., Connolly, T. M., & Stansfield, M. (2006). The role of staff development to enhance the integration of e-learning within the HE curriculum, European Conference on e-Learning (ECEL) 2006, 11-13 September 2006, University of Winchester, UK.

Paulsen, M. F. (2003). Experiences with learning management systems in 113 European institutions. Journal of Educational Technology & Society, 6(4), 134–148.

Grippa, F., & Secundo, G. (2009). Web 2.0 projectbased learning in higher education: some preliminary evidence. Int. J. Web Based Communities, 5(4), 543–561. doi:10.1504/IJWBC.2009.028089 Hall, H., & Davison, B. (2007). Social software as support in hybrid learning environments: the value of the blog as a tool for reflective learning and peer support’. Library & Information Science Research, 29(2), 163–187. doi:10.1016/j. lisr.2007.04.007 Illeris, K. (2002). The Three Dimensions of Learning: Contemporary learning theory in the tension field between the cognitive, the emotional and the social. Copenhagen: Roskilde University Press. Kim, S. W., & Lee, M. G. (2007). Validation of an evaluation model for learning management systems, Journal of Computer Assisted Learning, Blackwell Publishing Ltd. Larusson, J. A., & Alterman, R. (2009). Wikis to support the “collaborative” part of collaborative learning. Computer-Supported Collaborative Learning., 4(4), 371–402. doi:10.1007/s11412009-9076-6 Lawless, N., & Allan, J. (2004). Understanding and reducing stress in collaborative e-learning. Electronic Journal of e-Learning, 2(2).

72

Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York, NY: Basic Books. Schön, D. A. (1987). Educating the reflective practitioner: Towards a new design for teaching in the professions. San Fransisco, CA: Jossey-Bass Inc. Shin, J., & Bickel, B. (2008). Communities of practice: Creating learning environments for educators (Kimble, C., & Hildreth, P., Eds.). Information Age Publishing. Stansfield, M. H., & Connolly, T. M. (Eds.). (2009). Institutional transformation through best practices in virtual campus development: Advancing e-learning policies organizations. Hershey, PA: IGI Global Publishing. doi:10.4018/978-160566-358-6 Stansfield, M. H., & Connolly, T. M. (2009). An exploration into key issues relating to the adoption of good practices in e-learning and virtual campuses. In Mayes, T., Morrison, D., Mellar, H., Bullen, P., & Oliver, M. (Eds.), Transforming higher education through technology enhanced learning. Higher Education Academy. Wenger, E., McDermott, R., & Snyder, W. M. (2002). Cultivating communities of practice. Boston, MA: Harvard Business School Press.

Learning 2.0

Williams, J. B., & Jacobs, J. (2004). Exploring the use of blogs as learning spaces in the higher education sector. Australasian Journal of Educational Technology, 20(2), 232–247.

KEY TERMS AND DEFINITIONS Communities of Practice (CoPs): Groups of people informally bound together by shared expertise and passion for a joint enterprise (Wenger & Snyder, 2000, pp. 139). E-Portfolios: A valuable online tool that learners, faculty, and institutions can use to collect, store, update, and share information. E-Portfolios allow students to reflect on their learning, communication with instructors, document credentials, and provide potential employers with examples of their work (EDUCAUSE, 2005). eLearning: Any use of Web and Internet technologies to create learning experiences (Horton, 2003, pp. 13).

LMS: The components in which learners and tutors participate in online interactions of various kinds, including online learning (Becta, 2003) Portfolio: A collection of documents relating to a learner’s progress, development, and achievements (Beetham, 2009). Social Software: Social software is a community driven technology which facilitates interaction and collaboration and depends largely on social convention (McKelvie et al., 2007). Web 2.0: The social use of the Web which allow[s] people to collaborate, to get actively involved in creating content, to generate knowledge and to share information online (Grosseck, 2009, pp. 478). Wiki: A website (or other hypertext document collection) that allows users to add content, as on an Internet forum, but also allows anyone to edit the content (G. Avram, 2006)

73

Section 2

Implementing and Evaluating

75

Chapter 4

Evaluations of Online Learning Activities Based on LMS Logs Paul Lam The Chinese University of Hong Kong, Hong Kong Judy Lo The Chinese University of Hong Kong, Hong Kong Jack Lee The Chinese University of Hong Kong, Hong Kong Carmel McNaught The Chinese University of Hong Kong Hong Kong

ABSTRACT Effective record-keeping, and extraction and interpretation of activity logs recorded in learning management systems (LMS), can reveal valuable information to facilitate eLearning design, development and support. In universities with centralized Web-based teaching and learning systems, monitoring the logs can be accomplished because most LMS have inbuilt mechanisms to track and record a certain amount of information about online activities. Starting in 2006, we began to examine the logs of eLearning activities in LMS maintained centrally in our University (The Chinese University of Hong Kong) in order to provide a relatively easy method for the evaluation of the richness of eLearning resources and interactions. In this chapter, we: 1) explain how the system works; 2) use empirical evidence recorded from 2007 to 2010 to show how the data can be analyzed; and 3) discuss how the more detailed understanding of online activities have informed decisions in our University.

INTRODUCTION Learning management system (LMS) is a broad term that is used for a wide range of systems that organize and provide access to online learning services for students, teachers and administrators. These services usually include access control,

provision of learning content, communication tools, and organizations of user groups (Paulsen, 2002). Jovanovic et al. (2007) defined an LMS as “a software environment that enables interactive web-based teaching and supports administration of distance courses, allowing instructors to distribute information to students, producing course mate-

DOI: 10.4018/978-1-60960-884-2.ch004


Evaluations of Online Learning Activities Based on LMS Logs

rial, preparing assignments and tests, engaging in discussions, and managing courses and distance classes” (p. 46). In 2005, 95% of all higher-education institutions in the UK were using an LMS (Browne, Jenkins, & Walker, 2006). At The Chinese University of Hong Kong (CUHK), two LMS (WebCT and Moodle) are centrally supported. Indeed, the majority of the University’s eLearning activities are supported by these central services; apart from one faculty (Engineering), there are relatively few non-centrally-hosted course websites. Effective record-keeping, and extraction and interpretation of eLearning logs, can reveal valuable information on how these LMS are used to facilitate teaching and learning. As noted by Sen, Dacin and Pattichis (2006), the use of logs for tracking user activities is quite common in commercial settings where customer habits and trends are traced and monitored. Reading user logs also applies in educational settings – for example, the study by Black, Dawson and Priem (2008) on how to obtain measures of ‘community’ in online courses. In universities with centralized web-based teaching and learning systems, the logs can be monitored through inbuilt mechanisms to track and record a certain amount of information about online activities. Colace and De Santo (2003) commented that monitoring an LMS can enable detailed and useful information on the LMS’s utilization and efficacy. This information can include trend data if the logs have been collected regularly over time. Such information can provide the basis for various decisions related to the implementation and promotion of eLearning. However, these inbuilt web-log tracking systems do not normally provide institution-level data. The weblogs are reported in an interface designed for individual teachers to get a summary of activities recorded in individual courses (Mazza & Milani, 2005) rather than for analyses of online learning activities at higher levels (e.g. department, faculty or institution). Retrieval of

76

data in WebCT (version 3.8) is even more challenging. As it does not employ a database structure, these records or logs of activities can be extracted using the provided display logs functions which are very limited in functionality. The locations where the record information is stored are not clear because of the lack of a database structure, and so time and effort are needed for: 1) testing through trial and error for the allocation of the desired information; 2) checking whether the data are accurate; and 3) developing software to extract the information for all courses in the University. After retrieval of information, additional work is needed to standardize and automate the integration, interpretation and reporting processes of the log data so that we have a common ground to compare and contrast eLearning uses over time. Romero, Ventura and Garcia (2008) discussed using data-mining techniques to explore the raw log data of servers in order to understand various aspects of learner activities. However, such strategies, though allowing great flexibility in the topics of study, are technically complex. As noted by Black, Dawson and Priem (2008) “server logs are plagued with a low signal-to-noise ratio: simply preparing the data for modeling can consume 80% to 95% of a project’s resources” (p. 67). A system that is more powerful than the LMS inbuilt activity log systems, and can regularly retrieve and interpret the logs into a number of fixed representations for year-by-year comparison and contrast, seems to be what we need. Zhang et al. (2007) reported a similar system called Moodog which monitor students’ activities live on the Moodle LMS. Our system looks at the issue more from an institutional point of view. We may not need to monitor student activity every moment but retrieve and analyze the logs once every semester. Also, because of our particular context, our system reads and integrates as much as possible of the logs from both of the LMS we host – WebCT and Moodle. Earlier development and the framework of the system were reported in Lam et al. (2006).


The work at that time was more focused on the data-retrieval stage. The present paper extends our discussion to the automation of the data interpretation and reporting processes. As noted in Lam, Keing, McNaught, and Cheng (2006), system logs data can provide information on: 1) popularity; 2) the nature of the functions/ strategies in use; and 3) engagement of teachers and students. 1. The notion of popularity is a very simple yes/no specification for each course in the University whether any eLearning activities are recorded in our logs or not. 2. The nature of the eLearning activities recorded for each web-enabled course refers to uses of forums, assignment-submission service, course-content delivery, online quizzes or surveys, grade-book facility, etc. 3. Engagement reflects how involved teachers and/or students are in these activities. This is the level among the three that convey the finest amount of detail about a site. After the recognition that there is a course website (popularity), more information can reveal the actual features and activities having occurred on the site (nature). After learning about the nature of the website, further information can gauge how active teachers and students are on site (engagement). The data report the actual activities recorded and, to a certain extent, they fit the requirements of the naturalistic research paradigm (Alexander & Hedberg, 1994; Grasha, 1990) that collection of evaluation data is usefully done in non-intrusive ways (Lam, McNaught & Cheng, 2008) and should, where possible, be situated in authentic educational settings (Froehlieh, 1994). We do not claim that weblog data represent a comprehensive solution to all evaluation needs. More comprehensive evaluation studies would consider evidence, both quantitative and qualitative in nature, from a variety of sources. The

present paper serves only to illustrate the advantages as well as limitations to using weblogs as a data source. Apart from relatively easy access to the data, the extraction of system logs is also completely non-intrusive to both teachers and students. Furthermore, repeated measures can be taken for a long period of time so that comparison of usage over time is possible. The largely automatic extraction of the logs (once the software for logs extraction is devised), and the standardized methods in the follow-up analysis, reporting, and interpretation of the data, enable an institution to build a mechanism which is suitable for administration on a regular basis (e.g. annually). A comparable system of logs extraction and analysis can then be used in subsequent monitoring. One limitation of this approach, however, is that it monitors only uses of the web that utilize the central system. Also, it has a bias on quantity rather than quality as logs are numbers rather than a full picture of the educational quality of the content on course websites. Also, not all online activities and teachers’/ students’ engagement in these activities can be effectively recorded by the logs. For example, the availability of course outlines on course websites is an online activity that is of interest to universities. However, ‘online course outline’ is not an activity separately recorded by the logs of WebCT or Moodle. It is impossible to identify course outlines unless the researchers go into individual websites and read all the documents. The picture portrayed by the logs is only a partial representation of the total online learning activities and the engagement in these activities. Even if the logs reveal certain information on popularity of, nature of and engagement with activities in websites, care has still to be taken to understand the exact meanings of these logs based on the characteristics of the platforms and how logs are kept in them. Very often, minor adjustments have to be made or there are decisions to make

77


concerning the cut-off points beyond which the records are deemed to fall into another category. In this paper, we: 1. Explain how the system works. 2. Use empirical evidence recorded from 2007 to 2010 to show how the data could be employed to achieve various levels of analysis; and 3. Discuss how the better understanding of the online activities have enabled decisions in a number of eLearning support initiatives in our University. We consider that the data have assisted in better understanding and refining our eLearning strategies and supports at both an institutional level and faculty/ department level.

HOW THE SYSTEM WORKS The following measures have been taken to refine and standardize the data so that we identified the right types of logs for the types of activities we targeted. Clear definitions are also necessary, particularly in our case where we have merged weblog readings from two different LMS. The standardization makes sure the readings from the two platforms record the same underlying activities and record them in a compatible manner. Not all websites created on the servers should be considered ‘active’ websites; this means that there are some sites that are developed but not made accessible to students. For example, some of the websites might have been created by the teachers or the teaching assistants for testing purposes (e.g. perhaps to be used in the next term) and there were actually no real student activities on the site. In our weblog study, therefore, we took care to isolate only the so-called ‘active’ websites – websites that had at least one student access during the course period would be included in the data. The similar ‘active’ concept was also applied in the study of the other four eLearning strategies

78

in our study. A website that had none of these active features may mean a site that was used by the teacher for only announcing course information such as examination dates or news of events. •

• •

•

Whether or not a particular course used ‘forum’ for ‘active discussion’, for example, was judged not only from the existence of a forum on the site, but rather on whether there was at least one student access to the forum as well. ‘Active online quizzes’ meant quizzes that had at least one student attempt. ‘Active assignment submission’ meant that at least one student submitted work to the platform. ‘Active content’ meant the website contained at least one document (could be PowerPoint, Word or PDF documents, or any other multimedia files) for download, and there was at least one student download recorded in the logs.

The use of eLearning strategies is more meaningful in courses that have considerable class sizes. We have introduced a mechanism to filter cases based on class size. In our web logs study, we only studied classes with a class size equal to or larger than 10. Lastly, it is worth noting that the unit of analysis in our study was normally a course. In most cases, it was relatively easy to decide whether a certain course had used eLearning strategies or not. However, some decisions had to be made in many other cases. Sometimes one course was run in a number of sessions (at different times and even by different teachers). Our standpoint was to consider a course eLearning-enabled IF at least one of the sessions had an active website on any one of the LMS. Figure 1 shows the report interface of the weblog system. The interface is comprised of several parts such as searching filters (Boxes 1 and 2), result window (Boxes 3a and 3b) and graph


Figure 1. Inquiry layout of weblog system

window (Box 4). Once a set of filters is submitted into the search engine, results and a graph will be shown in the result and graph windows. Also, in order to provide the flexibility to enable multiple levels of analysis, different parameters can be set for the filters. For example, we can ask for a report on a certain year/ term or a report that contrasts activities in multiple years or terms. Reports can be on institution, faculty, department, or course levels. We can call for records of undergraduate courses, postgraduate courses, or both. The unit of analysis is normally a course but there is an option to drill down into the use of LMS by the individual sessions of a course (taught by the same or different teachers). Also, the system can report actual numbers (e.g. how many active course websites and how many courses we have in the University) or report percentages (e.g. the percentage of courses that have an active website). These parameters are available for the administrators in the Box 1 area in Figure 1.

In Box 2, administrators can specify the exact year or term to report on, as well as the exact faculty, course type and even the course code the report should be limited to if desired. 2009 on the system means the 2009–2010 academic year which is from September 2009 to May 2010 in our context. In addition, there is a choice about class size, i.e. the minimum student number in a class before it can be included (the default is 10). Boxes 3a and 3b in Figure 1 show the weblog results of the whole University in the year 2009. Both the ‘popularity’ (Box 3a) and the ‘nature of activities’ (Box 3b) types of information are shown. Table 1 further explains the sub-categories in these two areas. Box 4 contains graphical representations of the various numerical data reported in Box 3. They are usually sufficient for normal reporting purposes. However, the data and graphs can be exported as an Excel file for further processing and analysis if needed.

79


Table 1. Definitions of the sub-categories in the report Notion

Popularity

Nature of activities

Sub-categories

Definition/ Description

Course Total

Total number of courses

None Total

Courses without using any LMS

WebCT Total

Used WebCT

Moodle Total

Used Moodle

Multi Total

Used both WebCT and Moodle

Active website (AW)

At least one student access to LMS and view website page during the course period

Active content (AC)

At least one document on the active website for access and at least one student download recorded in the logs

Active discussion (AD)

At least one student posts a topic or a thread

Active assignment (AA)

At least one student submitted work to assignment drop-box

Active quiz (AQ)

At least one student attempted quiz on active website

Figure 2 illustrates how the weblog system represents teachers’ and/or students’ ‘engagement’ in various eLearning activities (called level 3 analysis in the system: Box 5). In this example, the data show the amount of activity recorded in using the various functions over all our active WebCT and Moodle websites in the year 2009.

Box 6 shows the tabs that lead to analyses of various types of activities. In the result-grid region now showing (Box 7), we can tell that among all the courses in the year, 128 of them had a website in which student access over the year was between 1 to 100, 731 of them had 101 to 1000 visits, 858 had 1001 to 10,000 visits and 167 had over 10,000 visits in the year. 1586 courses did not have a

Figure 2. Demonstration of result window reporting engagement data

80


website in any of our two platforms or their websites did not record any student visits at all. Similarly, data and graphs concerning engagement in the various activities can be exported as an Excel file for further processing and analysis.

VARIOUS LEVELS OF ANALYSIS The flexibility of the system allows us to conduct various levels of analysis easily. Six main types are distinguished in our framework as shown in Figure 3. The framework is an extended version of the one reported in Lam et al. (2006) as a result of enhanced analytical power through recent developments. In brief, the system provides a general institution-wide overview of eLearning activities (Types A, B and C analysis in Figure 3); it also

has the capability to differentiate faculty- or department-level eLearning practices (Types D, E and F analysis in Figure 3). Type A refers to how popular LMS are across the whole University. Type B refers to the nature of the features used in the LMS. Type C refers to how students and/ or teachers in general are engaged in the various activities. Type D refers to how popular LMS are used in a certain faculty/ department or even a certain course. Type E refers to the analysis of how various eLearning functions are used in the different faculties/ departments or courses. Lastly, Type F analysis distinguishes the engagement in activities by students and/or teachers in various faculties/ departments. Below, we use empirical evidence recorded from the years 2007 to 2009 to illustrate how the data could be employed to achieve these various levels of analyses. We looked at all undergraduate

Figure 3. Model of the monitoring mechanism through system logs

81


and postgraduate courses that had class sizes of 10 or above at our University. These illustrations are not meant to be a comprehensive discussion of the eLearning activities in our University. Only a few key features are discussed. We also only report up to the faculty level, rather than drilling into what happened in individual departments or even courses. The actual names of the faculties are also hidden as the interest of the present paper is not on detailed disciplinary differences. The data represent a few snapshots of some of the analyses the system allows us to do that, as will be discussed later, reveal useful information about the actual use and usage of the LMS in our University.

Popularity of LMS Across the Whole University Figure 4 shows the overall websites built on the WebCT and the Moodle platforms across three years.

The overall impression is that an increasing number of courses began to have a web presence in our LMS. The courses that had a website in one or both of our LMS grew steadily from around 42% (1432 active websites) in 2007, to 46% (1555 active websites) in 2008 and then to over 54% (1891 active websites) in 2009. More websites were built on the WebCT platform because Moodle was not introduced at CUHK until 2007. Nevertheless, the increase in Moodle use over the years is impressive.

Comparison of Two Functions What are the common functions used on these sites? Data show that the main use of these websites was for content delivery. The more interactive functions such as quizzes and discussion were used less. We illustrate this large contrast by highlighting the use of ‘active content’ and ‘active quiz’ in three years in Figure 5. Over 90% of the active websites contained some sorts of content (various types of files such

Figure 4. Usage of an LMS in CUHK courses in the academic years 2007–2009

82


Figure 5. LMS functionality used in the academic years 2007–2009

as doc, PowerPoint or PDF) which have been accessed by students in the course of study. In contrast, only a small percentage of courses had employed online quizzes as an active strategy. The percentage of quiz-containing websites actually decreased quite significantly over the years. In fact, low use was also observed for other interactive functions such as ‘active discussion’ and ‘active assignment’. On average, active-online discussions were found only in about 21.5% (2007), 20.9% (2008) and 14.8% (2009) of the total websites respectively. Also, 27.6% of the websites used the active-assignment feature in 2007. It gradually decreased to 24.0% in 2008 and then to 14.6% in 2009.

Engagement in Four Areas Figure 6 shows students’ visits to the websites and their engagement in some of the website functions in 2009. The data first of all confirmed that LMS are actively used by students. The ‘active website’ analysis showed that more than 15,000 students (over 90% of the total student population) visited one or more of the websites. Many of the students paid quite frequent visits (more than 30% of the students accessed the LMS more than 20 times

during the year). A few even visited the websites over 1000 times. However, the data also showed that students did not visit the websites for the interactive functions. Only 1995 students (recalling that over 15,000 students ever visited the LMS that year) had written anything on any of the online forums (many of these 1995 students wrote only one or two postings). Similarly, relatively few students used the assignment submission function and the quizzes. Many of the students who used the quizzes, however, had completed multiple online quizzes. 4742 students used quizzes and about one-third of them attempted 11–20 quizzes on the LMS. One-tenth even made 20 times or more attempts.

Use of LMS in Four Faculties Different faculties used LMS quite differently. ELearning experience was not the same across disciplines. Figure 7 shows the use of the two LMS in four of our eight faculties. First of all, we observed that an increasing number of courses had a course website and this was true in all of the faculties. This finding is in line with the general picture portrayed on Figure 4. However, the interest in using LMS

83


Figure 6. Frequency of usage of function within the LMS (WebCT) in 2009

varied quite a lot from one faculty to another (e.g. over 90% of courses in Faculty B versus around 50% to 60% in the other three faculties). It is also interesting to note that students were likely to be told to use two LMS because some of the teachers used WebCT while some used Moodle even if they were in the same faculty. This was particularly true in Faculty A where the preferences of WebCT and Moodle by teachers were roughly half and half. Using two different LMS and getting used to the navigation and controls of two systems might have imposed considerable confusion and unnecessary workload on students.

Use of Three Functions in Four Faculties Similar to the University-wide phenomenon as portrayed on Figure 5, using LMS for content delivery was also a common practice across the

84

four faculties inspected (Figure 8). Most of the websites (around 90% or more) in these faculties had active content that had been accessed by students in the respective years. This content-delivery focus was similar among the faculties. What makes faculties different from each other is their use of the other functions of LMS. In Faculty B, we found the use of online discussion a popular strategy (nearly half of the courses used it) in 2007 but the enthusiasm of using this function then dropped significantly (dropped to only about 5% in 2009). The interest of teachers in this faculty in the use of online quizzes also dropped to zero in the period. On the contrary, the use of online discussion in Faculty A and Faculty C courses has been relatively stable. Comparatively, teachers in Faculty D never seemed to have any great interest in online communication.


Figure 7. Use of LMS in different faculties (A–D)

Faculty C impressed us by their interest in using the quiz function. Percentage of courses having quizzes in this faculty has been well above the University mean all the years. Based on personal communications the authors have had with teachers in these faculties, there are many factors involved in explaining these observed differences. Faculty B, for example, was newly established in 2007 and then had significant expansion in teacher and student numbers in the three-year period. The faculty boosted the use of eLearning strategies in the early years but the focus was quickly shifted to other matters as more and more new teaching staff members were recruited and teaching loads became heavy. Faculty C is a professional discipline which requires students to remember and understand significant numbers of terms and information. Online quizzes seem to be a useful tool to facilitate factual learning so that classroom time can be used for activities that require higher-order thinking. Faculty A focuses on social and humanities subjects where discussion may be a more suitable online activity than quizzes. Topics in Faculty D centre on established laws and formulae and there appears to a perception that online discussions are not needed. The use of the online discussion function in the websites of this faculty has been decreasing over the years.

Engagement in Two Functions in Two Faculties Figure 9 selectively looks at two online activities in two of the faculties. It shows that, as expected, students’ engagement in content-oriented uses was similar but their involvement in other online activities could differ from one faculty to another, basically reflecting the uses of these functions as indicated in Figure 9. More than half of the websites in both Faculties A and B had 16 or more content files that were accessed by students in 2009. Websites were content-heavy and accessing of the content by students was popular in both faculties. Their use of the quiz function was, however, quite different. Websites in Faculty C obviously had a higher concentration of quizzes in them: over 10% of the websites contained 6 or more quizzes that have been actively used by students in the year. In general, thus, there is a ‘cultural’ difference in eLearning use in different faculties; perhaps not so much in using the web for content but much more when using more interactive functions.

85


Figure 8. Use of LMS functions in four faculties

REFINING OUR eLEARNING STRATEGIES Monitoring the LMS, or weblogs in our case, can achieve the objective of evaluating students’ online learning activities. By analyzing these data,

valuable guidelines in terms of eLearning design can be developed. Effective record-keeping, and extraction and interpretation of eLearning logs can reveal valuable information on standards of design and development, and program delivery.

Figure 9. Websites and students’ engagement in activities in two faculties

86


Using LMS for More than Content Delivery There are both positive and negative trends in our data concerning adoption of eLearning at CUHK. The overall impression seems to be that more courses have begun to have a web presence in our LMS. However, when we looked at these course websites closely, we did not find many functions were being used actively. Most of the websites had content for students to download. There might also be course information and announcements, but the other discussion, assignment and quiz functions were not popular (35% or below). Moreover, many of these functions did not seem to be sustainable in the sense that we observed a decrease in use of the more interactive functions such as online discussion and online quizzes over the years. Many scholars have also commented on this phenomenon that teachers used LMS mainly for delivery of content; our findings thus are more generally applicable outside our own context. Folajimi (2009) remarked that “computers aren’t fulfilling their potential to effect significant changes in education, are under-utilized, and are not being implemented in very effective or creative ways” (p. 1617). Ginsberg and McCormick (1998) also remarked that computers have only been used mostly to complement traditional teaching but are not designed as tools for effective learning activities on their own. One major drawback of existing LMS is that they are content-centric. Many teachers simply move all their teaching materials to the system. The materials are presented uniformly to all learners regardless of their background, learning styles and preferences. Modern trends in education are for learner-centric design where learners are facilitated to actively engage in the learning process to construct their personal knowledge. Teachers play the role of ‘facilitator’ who guides the learning process instead of being the sole information provider (Ong & Hawryszkiewycz, 2003). The

findings thus suggest that institutional eLearning support should not merely focus on having a web presence in courses or using the Web for courseware delivery. Attention also needs to be on the diffusion and sustained use of interactive online learning activities. Sustainability of these strategies seems to impose particular challenges in our context as the data show not only underuse of the related functions but also a general decrease of use for some of them. The question still remains about whether teachers are reluctant to interact with students on the Web in general, or whether this is only a limitation imposed by LMS. It can be an LMS problem; at CUHK, we are beginning to see a trend that teachers are interested in Web 2.0 strategies and social-media software and services. Web 2.0 or social software tools such as blogs, wikis, podcasts and media-file-sharing systems such as YouTube are supplementing or even “supplanting” (Gray, Chang & Kennedy, 2010, p. 33) the basic Web 1.0 strategies such as emails and forums in education. Users world-wide produce text and other media files for sharing and these materials can be easily adopted as teaching and learning resources elsewhere. The communications that are facilitated in social software also have the potential to extend communications to a scope far beyond the boundary of any single university. For example, Dillon, Wong and Tearle (2007) described internationalization of teaching and learning defying physical boundaries in terms of teachers and students in various places of the world teaching and learning together. We also know of a number of instances at our University where teachers have begun communicating with students through blogs and Facebook. Pituch and Lee (2006) observed that, although factors such as perceived usefulness influence LMS use, the strongest influence on student use might be system characteristics. Do these socialmedia services possess certain characteristics that make them better choices than the LMS, espe-

87


cially when implementing interactive activities? Is the LMS losing ground to these popular Web 2.0 technologies? Integration between the ‘Web 1.0’ LMS and Web 2.0 technologies seems to be needed (Boulakfouf & Zampunieris, 2008). However, teachers may simply refuse online interactions for teaching and learning in general. Levin and Arafeh (2002) suggested that a gap exists between students and teachers in the manner they use technology for teaching and learning purposes. Students use the internet for daily academic tasks, e.g. the internet as a virtual textbook and reference library, or virtual tutor, etc. However, most teachers are slower to adopt the more complex and interactive features of the Web (Morgan, 2003). If this is the case, our support should focus on motivating teachers to rethink teaching rather than just to promote functionalities of the LMS only. In a traditional, largely face-to-face university such as CUHK, blended learning (e.g. Garrison & Vaughan, 2007) offers possibilities that allow teachers to retain their traditional roles while gradually increasing the suite of online tools and strategies they use. In our context eLearning and LMS are at present supplementary tools to assist in the process of teaching and learning. What we want to achieve is that eLearning becomes more integrated into course design and thus complementary, rather than merely supplementary, to classroom teaching. Our support should not be limited to teachers only. Students may use technology in their everyday lives, but they may not be able to use the technology wisely for learning purposes. Kirkwood and Price (2005) pointed out that students may have ability with different computer applications, but only a few can transform and apply their digital competence to learning processes. For instance, students being “familiar with the use of email does not imply expertise in rigorous online debate and discussion” (p. 271). Support for meaningful eLearning thus requires professional development and support for both teachers and students.

88

The fact that students at CUHK use a number of LMS throughout their studies has led the University to plan to adopt a single LMS in the future. At the time of writing the paper, an evaluation of different LMS is underway; the imminent demise of WebCT is a facilitating factor. LMS logs have been important information in this evaluation process. For example, we understand that the future LMS should be very strong in course management and content delivery as these features are used by most teachers. However, we will also examine LMS for user-friendly interactive and communicative functions, and for convenient integration with existing Web 2.0 tools so that student-oriented eLearning strategies might be better used in the future.

Disciplinary Differences: Differentiating Support Strategies The LMS logs showed differences in the use of eLearning strategies between the faculties. This piece of information is crucial to refinement of our approaches to supporting eLearning in individual faculties. For example, we will focus on exploring the decline of interest of using certain eLearning strategies in places where the strategies do not seem to be sustained (e.g. Faculty B in our study). The support in that particular context will focus on known challenges rather than simply explaining and promoting eLearning in a traditional fashion. Another key factor is the perceived intrinsic return on effort in terms of better teaching practice and students’ learning, as well as the expected extrinsic benefit during performance appraisal. Our support to teachers in this circumstance includes both assisting them in lowering the effort in using technology, as well as showing them the usefulness of the strategies in fulfilling one or more of their needs. However, Thomas et al. (2009) interviewed eight teachers at CUHK and noted how these teachers considered innovative e-learning projects to be lonely and time-consuming. Being self-motivated and intellectually committed was a key factor that


sustained these teachers and allowed them to stay engaged. Our University is currently reviewing its strategies for assessing teaching quality and innovation in teaching will gain more prominence. This extrinsic reward should support a move to more active uses of eLearning. In disciplines where teachers seem to have a growing interest in using certain eLearning strategies, we can strengthen our support for these strategies and then also disseminate these teaching ideas to other teachers in related fields who are newer to eLearning strategies. Another structural condition that supports diffusion is the support provided by the department or by the University – in terms of monetary development grants, technical services, and consultative teaching and learning support in terms of eLearning pedagogical design. This support, though somewhat limited, still has significant impact on workload. The contents of this chapter will be presented at the University committee that considers policy for eLearning at CUHK. We know that differences in practice are also at the department level. We have devised an ‘eLearning Liaison Person’ (eLLP) strategy to better inform us the needs of individual departments. ‘eLLPs’ are representatives of individual departments who act as liaisons to inform central services of departmental needs and also act as conduits for conveying information about central services and events. To conclude, the LMS logs are useful in assisting us to continually adjust our eLearning support strategies. Some of the influences include: •

•

•

informing us the need to further investigate teachers’ attitudes and habits of using interactive e-functions; leading us to focus on meaningful uses of the functions rather than merely on explaining the functions of LMS in our promotion and support; informing us the important considerations when planning our future LMS;

•

•

informing us students’ online learning experiences which started a process to revisit support and services needed, including the discussion of a single platform; and assisting us in formulating faculty/department- specific support; eLLPs provide another source of information on disciplinespecific eLearning needs and habits.

FUTURE RESEARCH DIRECTIONS As mentioned above, LMS logs provide convenient data about web uses but are limited to showing the quantity rather than quality of these uses. Evaluating the LMS weblogs cannot provide information on the mode of online participation, interaction patterns and group dynamics. The interaction between users is of most importance when evaluating the function and effectiveness of an LMS (Gunawardena, Carabajal, & Lowe, 2001). The data highlight areas where further exploration seem to be warranted and we need other evaluation strategies (e.g. surveys and focus groups) to confirm these trends and patterns and look for the reasons behind them. One important area that is certainly worth exploring further is the observed decline in the use of communicative functions in the LMS. Is there a genuine decline of e-communication for learning or there are other software and services (such as Web2.0 technologies) that are taking teachers’ and students’ attention away from similar functions in the LMS?

CONCLUSION Effective record-keeping, and extraction and interpretation of eLearning logs can reveal valuable information on eLearning use. In universities with centralized web-based teaching and learning systems, monitoring the logs can be accomplished because most eLearning platforms have inbuilt mechanisms to track and record a certain amount of information about online activities. We reported

89


our strategy in building a system to automatically retrieve and then interpret the logs of eLearning activities in our two centralized LMS. The method has become a convenient and repeatable method for the evaluation of the richness of eLearning resources and interactions over time. In so doing, however, we have to acknowledge that the weblog analyses have limitations and should be interpreted with evidence about eLearning uses collected by other means and from other sources. We explained how the system works, and we used empirical evidence recorded from the academic years 2007 to 2009 to show how the data could be employed to achieve various levels of analyses. Weblogs represent a comparatively easy, automatic, and non-intrusive method to provide relatively quick and accurate data. We found that the logs have informed us the pattern of use of various online activities as well as the change in these activities over time. The new understanding has facilitated decisions in a number of eLearning support initiatives in our University, including the evaluation of new LMS systems for the future, and an increase in sensitivity towards faculty and departmental needs.

REFERENCES Alexander, S., & Hedberg, J. (1994). Evaluating technology-based learning: Which model? In Beattie, K., McNaught, C., & Wills, S. (Eds.), Multimedia in higher education: Designing for change in teaching and learning (pp. 233–244). Amsterdam, Netherlands: Elsevier. Black, E. W., Dawson, K., & Priem, J. (2008). Data for free: Using LMS activity logs to measure community in online courses. The Internet and Higher Education, 11, 65–70. doi:10.1016/j. iheduc.2008.03.002

90

Boulakfouf, S., & Zampunieris, D. (2008). Integrating Web 2.0 features into a learning management system. Conference on e-Learning 2008. Retrieved December 29, 2010, from http://publications.uni.lu/record/1695/files/DZ_Pages%20 from%20ecel08-cd1.pdf Browne, T., Jenkins, M., & Walker, R. (2006). A longitudinal perspective regarding the use of VLEs by higher education institutions in the United Kingdom. Interactive Learning Environments, 14(2), 177–192. doi:10.1080/10494820600852795 Colace, F., & De Santo, M. (2003). Evaluating online learning platforms: A case study. Proceedings of the 36th Hawaii International Conference on System Sciences. Retrieved December 29, 2010, from http://www.hicss.hawaii.edu/HICSS36/ HICSSpapers/ETWBE03.pdf Dillon, P., Wang, R., & Tearle, P. (2007). Cultural disconnection in virtual education. Pedagogy, Culture & Society, 15, 153–174. doi:10.1080/14681360701403565 Folajimi, Y. (2009). Transforming distant pedagogical learning to Web based collaborative system: An Intelligent Tutoring Systems Architecture. In T. Bastiaens et al. (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2009 (pp. 1618–1622). Chesapeake, VA: AACE. Froehlich, T. J. (1994). Relevance reconsidered Towards an agenda for the 21st century: Introduction to special topic issue on relevance research. Journal of the American Society for Information Science American Society for Information Science, 45(3), 124–134. doi:10.1002/(SICI)10974571(199404)45:33.0.CO;2-8 Garrison, D. R., & Vaughan, N. D. (2007). Blended learning in higher education: Framework, principles, and guidelines. San Francisco, CA: Jossey-Bass.


Ginsberg, R., & McCormick, V. (1998). Computer use in effective schools. Journal of Staff Development, 19(1), 22–25. Grasha, T. (1990). The naturalistic approach to learning styles. College Teaching, 38(3), 106–113. Gray, K., Chang, S., & Kennedy, G. (2010). Use of social Web technologies by international and domestic undergraduate students: Implications for learning and teaching in Australian universities. Technology, Pedagogy and Education, 19(1), 31–46. doi:10.1080/14759390903579208 Gunawardena, C., Carabajal, K., & Lowe, C. A. (2001). Critical analysis of models and methods used to evaluate online learning networks. Education Resources Information Center (ERIC). Retrieved December 29, 2010, from http://eric. ed.gov/PDFS/ED456159.pdf Jovanovic, J., Devedzic, V., Gasevic, D., Hatala, M., Eap, T., Richards, G., & Brooks, C. (2007). Using Semantic Web technologies to analyze learning content. Internet computing, 11(5), 45–53. Kirkwood, A., & Price, L. (2005). Learners and learning in the 21st century: What do we know about students’ attitudes and experiences of ICT that will help us design courses? Studies in Higher Education, 30(3), 257–274. doi:10.1080/03075070500095689 Lam, P., Keing, C., McNaught, C., & Cheng, K. F. (2006). Monitoring e-learning environments through analyzing Web logs of institution-wide e-learning platforms. In L. Markauskaite, P. Goodyear, & P. Reimann (Eds.), Who’s learning? Whose technology? (pp. 429–440). Proceedings of the 23rd Annual Australian Society for Computers in Learning in Tertiary Education 2006 conference, Sydney, Australia. Retrieved December 29, 2010, from http://www.ascilite.org.au/conferences/sydney06/proceeding/pdf_papers/p62.pdf

Lam, P., McNaught, C., & Cheng, K. F. (2008). Pragmatic meta-analytic studies: learning the lessons from naturalistic evaluations of multiple cases. ALT-J, 16(2), 61–79. doi:10.1080/09687760802315879 Levin, D., & Arafeh, S. (2002). The digital disconnect: The widening gap between Internet savvy students and their schools. Pew Internet & American Life Project. Retrieved December 29, 2010, from http://www.pewinternet.org/~/ media//Files/Reports/2002/PIP_Schools_Internet_Report.pdf.pdf Mazza, R., & Milani, C. (2005). Exploring usage analysis in learning systems: Gaining insights from visualizations. 12th International Conference on Artificial Intelligence in Education (AIED 2005). The Netherlands: Amsterdam. Retrieved December 29, 2010, from http://linux3.dti.supsi. ch/~mazza/Web_area/Pubblicazioni/AIED05/ aied-ws2005.pdf Morgan, G. (2003). Faculty use of course learning systems. Educause center for applied research reports. Retrieved December 29, 2010, from http://net.educause.edu/ir/library/pdf/ecar_so/ers/ ers0302/ekf0302.pdf Ong, S. S., & Hawryszkiewycz, I. (2003). Towards personalised and collaborative learning management systems. Third IEEE International Conference on Advanced Learning Technologies (ICALT’03). Retrieved December 29, 2010, from http://www.computer.org/portal/web/csdl/ doi/10.1109/ICALT.2003.1215113 Paulsen, M. F. (2002). Online education systems: Discussion and definition of terms. Retrieved December 29, 2010, from http://www.porto.ucp. pt/open/curso/modulos/doc/Definition%20of%20 Terms.pdf

91


Pituch, K. A., & Lee, Y. K. (2006). The influence of system characteristics on eLearning use. Computers & Education, 47(2), 222–244. doi:10.1016/j. compedu.2004.10.007 Romero, C., Ventura, S., & Garcia, E. (2008). Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51, 368–384. doi:10.1016/j.compedu.2007.05.016 Sen, A., Dacin, P. A., & Pattichis, C. (2006). Current trends in Web data analysis. Communications of the ACM, 49(11), 85–91. doi:10.1145/1167838.1167842 Thomas, K., Lam, P., & Ho, A. (2009). Knowledge diffusion in eLearning: Learner attributes and capabilities in an organization. In G. Siemens & C. Fulford, (Eds.), ED-MEDIA 2009 (pp.493–497). Proceedings of the 21st Annual World Conference on Educational Multimedia, Hypermedia and Telecommunications, Honolulu, Hawaii. (pp. 22–26). Chesapeake, VA: Association for the Advancement of Computers in Education. Zhang, H., Almeroth, K., Knight, A., Bulger, M., & Mayer, R. (2007). Moodog: Tracking students’ online learning activities. In C. Montgomerie & J. Seale (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2007 (pp. 4415–4422). Chesapeake, VA: AACE.

ADDITIONAL READING Alavi, M., & Leidner, D. E. (2001). Review: Knowledge management and knowledge management systems: conceptual foundations and research issues. Management Information Systems Quarterly, 25(1), 107–136. doi:10.2307/3250961

92

Becker, R., & Jokivirta, L. (2007). Online learning in universities: Selected data from the 2006 Observatory survey–November 2007. Observatory on borderless higher education (OBHE) online. Retrieved December 29, 2010, from http://www. obhe.ac.uk/documents/view_details?id=15 Coates, H., James, R., & Baldwin, G. (2005). A critical examination of the effects of learning management systems on university teaching and learning. Tertiary Education and Management, 11(1), 19–36. doi:10.1080/13583883.2005.996 7137 Collis, B., & van der Wende, M. (2002). Models of technology and change in higher education: An international comparative survey on the current and future use of ICT in higher education. Center for Higher Education Policy Studies (CHEPS). Retrieved December 29, 2010, from http://doc. utwente.nl/44610/1/ictrapport.pdf O’ Reilly, T. (2005). What is web 2.0? Retrieved December 29, 2010, from http://www.ttivanguard. com/ttivanguard_cfmfiles/pdf/dc05/dc05session4003.pdf Piccoli, G., Ahmad, R., & Ives, B. (2001). Webbased virtual learning environments: A research framework and a preliminary assessment of effectiveness in basic IT skills training. Management Information Systems Quarterly, 25(4), 401–426. doi:10.2307/3250989 Selim, H. M. (2007). Critical success factors for eLearning acceptance: Confirmatory factor models. Computers & Education, 49(2), 396–413. doi:10.1016/j.compedu.2005.09.004 Wang, W. (2007). Features of future learning management system. In T. Bastiaens & S. Carliner (Eds.), Proceedings of World Conference on ELearning in Corporate, Government, Healthcare, and Higher Education (pp. 1332–1335). Chesapeake, VA: AACE.


Zhang, D., Zhao, J. L., Zhou, L., & Nunamaker, J. F. (2004). Can e-learning replace classroom learning? Communications of the ACM, 47(5), 75–79. doi:10.1145/986213.986216

KEY TERMS AND DEFINITIONS Active Content: Content on the website that has been accessed (meant downloading in our study) by students in the specified period of time. Active Discussion: Online forums that have been accessed (meant posting in our study) by students in the specified period of time. Active Quizzes: Online quizzes that have been accessed (meant attempting the quizzes in our study) by students in the specified period of time.

Active Websites: Websites that have recorded student access (meant viewing the front page of website in our study) in the specified period of time. Engagement: LMS logs revealing the amount of activities (e.g. frequency of use) recorded for the various functions. LMS Logs Retrieval and Reading System: An automatic system to retrieve suitable logs from an LMS server, interpret the logs and then represent them in ways that facilitate our understanding of the types and level of activities happened on the LMS. Nature of Activities: LMS logs revealing the common functions being used on the course websites. Popularity: Using LMS logs to find out how many courses opened an active website on one or more of the LMS in our study.

93

94

Chapter 5

ANGEL Mining Tyler Swanger Yahoo! & The College at Brockport, State University of New York, USA Kaitlyn Whitlock Yahoo!, USA Anthony Scime The College at Brockport, State University of New York, USA Brendan P. Post The College at Brockport, State University of New York, USA

ABSTRACT This chapter data mines the usage patterns of the ANGEL Learning Management System (LMS) at a comprehensive college. The data includes counts of all the features ANGEL offers its users for the Fall and Spring semesters of the academic years beginning in 2007 and 2008. Data mining techniques are applied to evaluate which LMS features are used most commonly and most effectively by instructors and students. Classification produces a decision tree which predicts the courses that will use the ANGEL system based on course specific attributes. The dataset undergoes association mining to discover the usage of one feature’s effect on the usage of another set of features. Finally, clustering the data identifies messages and files as the features most commonly used. These results can be used by this institution, as well as similar institutions, for decision making concerning feature selection and overall usefulness of LMS design, selection and implementation.

INTRODUCTION A Learning Management System (LMS) is a course independent framework that provides, delivers, and manages instructional content, identifies and assesses learning, and tracks and records progress towards those goals. It may also provide course

registration and administration, as well as skills gap analysis, tracking, and reporting (Sclater, 2008; Watson & Watson, 2007; Paulsen, 2003). A Learning Management System generally has the same interface and features for all courses at a given school. Typically they have discussion forums, calendars, quiz capability, group work and chat spaces, and gradebooks, and perhaps

DOI: 10.4018/978-1-60960-884-2.ch005


ANGEL Mining

some customization capability for the instructor or individual student (Feldstein & Masson, 2006). Learning Management Systems undergo evaluation for a number of reasons. Evaluations of LMS data may be conducted to assist instructors in understanding how to enhance student learning (García, Romero, Ventura, & Calders, 2007, Romero, Ventura, & García, 2008). Schools may evaluate existing LMS options to select a particular initial choice (Sclater 2008), or when a decision is being considered on whether to replace or not the current LMS; replacement may be due to the current system no longer being supported (Sturgess & Nouwens, 2004). An investigative analysis of the usage of A New Global Environment for Learning (ANGEL), learning management system was conducted at a comprehensive 4-year college. This implementation of ANGEL has been used by the college’s faculty, staff, and students for eight years. The ANGEL Learning Management System “enables efficient and effective development, delivery and management of courses, course content and learning outcomes. Engaging communication and collaboration capabilities, enhance instruction to deliver leading edge teaching and learning” (ANGEL Learning, 2008). This investigation was prompted by the merger of the ANGEL company with a competitor, Blackboard, Inc; and Blackboard’s decision to stop support of the college’s version of ANGEL in 2012 (Blackboard, 2010). These changes present the college with the opportunity and need to assess the usage of the system. These events have forced the college to examine the need for a LMS, and if so what features are necessary in the event a new product needs to be purchased. Given these circumstances, data mining techniques were applied to evaluate which LMS features are used most commonly and most effectively by instructors and students. Data mining techniques are applicable to situations where large amounts of data exist, and the data may contain

internal relationships and patterns that characterize the data set as a whole. In the case of this study, the usage of ANGEL by students and instructors is described by the ANGEL data. This study clearly demonstrated that data mining techniques can be applied to find unknown patterns, interesting patterns, confirm assumptions, and consider statistical results for making decisions on the future of ANGEL or another LMS at a college. The data mining methods of classification, association, and clustering were applied to analyze the data. Classification produces a decision tree which predicts which courses will use the LMS system in the future, based on course specific attributes such as course type, discipline, and the number of students enrolled. Association mining discovers the usage of one feature’s effect on the usage of all other sets of features. Grouping features with similar values is the process of clustering. From the results of these analyses, metrics are formed to indicate usage of features in the LMS.

BACKGROUND There is a similarity between LMS and Knowledge Management Systems (KMS); both provide a repository for knowledge which is valuable for the user. In a KMS the knowledge is kept and used by an organization’s employees. In a LMS the purpose is to disseminate knowledge from instructors to students and to share knowledge in a way to enhance student learning (Haldane, 1998). There are also similarities between Learning Management Systems and distance learning. Distance learning uses LMS like software to provide students with learning materials and activities while tracking student activity (Falvo & Johnson, 2007). Whether the system is an LMS, a KMS, or a distance learning system the organization needs to select and implement the system best suited to their needs. Studies have been done to compare different Learning Management Systems (LMS)

95

ANGEL Mining

prior to installation or to evaluate effectiveness (Beatty & Ulasewicz, 2006). There have been many approaches to system evaluation and selection. Traditional systems requirements analysis has been used to evaluate and select LMS. Functionality desired by faculty has been achieved through requirements analysis followed by system selection and modification (Sclater, 2008) and functionality has been achieved through the use of design patterns (Avgeriou et al., 2003). Studies have been done to determine frameworks for evaluating LMS. Kim and Lee (2008) validated a model for evaluating Learning Management Systems (LMS) used in e-learning fields using exploratory factor analysis. Factor I was instruction management, screen design, and technology and factor II covered interaction and evaluation. Roqueta (2008) used Moore’s (1993) transactional distance theory, the diffusion of innovations theory (Rogers, 2003), and Malikowski’s (2007) model for evaluation of learning systems to conclude that there is a difference between learning management systems and course management systems, and that LMS are preferable in most cases. A commercial tool for evaluation has been developed by 3Waynet Inc. The LMS Evaluation Tool is spreadsheet like and designed to assist in selecting a LMS. With this LMS Evaluation Tool users specify criteria for evaluation, which are considerations that are important to the institution adopting the LMS, these criteria are cost of ownership, maintainability and ease of maintenance, usability, ease of use, user documentation, user adoption/ vendor profile, openness, standards compliancy, integration capacity, learning object metadata integration, reliability and effectiveness, scalability, security, hardware and software considerations, and multilingual support. Each criterion is expanded with relevant questions to be answered. The candidate LMS undergoing evaluation are also rated on their features. The features considered

96

are administration, security, access, integration with other systems, course design, development and integration, course monitoring, assessment design, online collaboration and communications, and productivity tools. These features each have a number of sub-features that are given weights to measure their relative importance. The user rates each candidate LMS on each criterion and feature. The evaluation tool computes an overall score for each LMS in each criterion (Commonwealth of Learning, 2004). Additionally, the independent research company Brandon Hall Research, has completed a study of over 90 LMS with custom comparisons across more than 200 features. The report of this study is available on the World Wide Web in an interactive format allowing users to specify their requirements and complete side-by-side comparisons of selected LMS (Chapman, 2010). Studies have shown that most evaluations of LMS look toward the technology of the system. Specifically, evaluations are done on communication, education management, and file management features and how effectively these features are implemented in the technology. Hall (2003) provides a list of technically oriented considerations in evaluating learning management systems. Lewis, et al. (2005) evaluated nine LMS by considering and comparing the features and capabilities of the systems in terms of content development, group work and participation, the calendar, communication capability, study tools available to students, handling of audio and video, monitoring of student participation and progress, and usability in terms of navigation, interface design and administration. Evaluations often done on the technology itself are conducted by experts and not participants. Feldstein and Masson (2006) outline features that teaching professionals, as well as technical staff, should consider when selecting an LMS. Specifically they suggest looking for the ability to customize the LMS to meet the pedagogical methods of individual instructors, which may vary

ANGEL Mining

by discipline, instructor, and learning styles of individual students. Sturgess and Nouwens (2004) report on a different approach where participation in the evaluation includes an understanding of the organizational culture and sub-cultures. Specifically they identified academics, information technology staff, multimedia development specialists and managers/administrators, as participant groups in selecting a new learning management system. Evaluation of student use of LMS found that the LMS needs to be integrated with other educational activities and other aspects of everyday life. This is more important than the interface design. Students’ idea of what an LMS should provide differs from the impression of the technical staff and other stakeholders who select and implement the system. While students effectively use an LMS remotely this is only undertaken when there is a practical advantage to the student, not just for some educational benefit (Alsop & Tompsett, 2002). Student learning style and its impact on learning management systems is investigated by Graf and Kinshuk (2006). Bayesian Networks have been used to discover user preferences to allow adaptation by the LMS to those preferences (Kritikou, Demestichas, Adamopoulou, Demestichas, Theologou, & Paradia, 2008). Instructors can use data mining techniques such as classification, association and clustering on student LMS data to improve instruction. Clustering techniques can find groups of similar students so that the instructor can direct instruction to the group’s particular needs classification will identify the characteristics of the students in each group. Association mining may discover relationships between these characteristics and student attributes (Romero, Ventura, & García, 2008). There are a number of data mining techniques and associated algorithms. A common technique shared with statistics is classification analysis. A decision tree model is constructed which finds a path to a predetermined class attribute for each data record. A classification decision tree contains

branches that can be converted to rules unique to the data set. Machine learning research produced a number of classification models, the best known of which is the C4.5 algorithm (Quinlan, 1993), which uses information gain to define the model’s structure. A product of machine learning research, association mining is used to find patterns of data where sets of attribute-value pairs occur frequently in the data set. With association mining what class attribute should be in the results is not predetermined. Apriori (Agrawal, Imieliński, & Swami, 1993) is the predominate association mining algorithm. It is an algorithm that produces many rules, and special techniques are needed to reduce the rule set to those that are interesting. The final technique used in this chapter is clustering. Clustering represents each attribute as a dimension and shows where data records occur in this multidimensional problem space. The most popular clustering algorithm is k-means (MacQueen, 1967). Again, analysis of the clusters needs special techniques. In an attempt to increase the robustness and reliability of rules, often a combination of data mining methodologies are applied. Deshpande and Karypis (2002) and Padmanabhan and Tuzhilin (2000) improved classification rules by first using association mining. Li, Han, and Pei (2001) classified records using CMAR (Classification Based on Multiple Class-Association Rules), which identifies frequent patterns and associations between records and attributes. Jaroszewicz and Simovici (2004) employed user background knowledge and a Bayesian Network to determine the interestingness of sets of attributes prior to association mining. Fu and Wang (2005) reduced data dimensionality using a separability-correlation measure that ranks the importance of attributes to improve classification and the usefulness of rules. Scime and Murray (2007) and Murray, Riley, and Scime (2007) used expert knowledge to reduce data dimensionality while iteratively creating classification models. Rajasethupathy, Scime,

97

ANGEL Mining

Rajasethupathy, and Murray (2009) improved the usefulness of rules by identifying “persistent rules,” which are those rules identified by both classification and association mining.

DATA MINING ANGEL DATA Data mining is a process of inductively analyzing data to find interesting patterns and previously unknown relationships in the data. Data mining is used not only to predict the outcome of a future event but also to provide knowledge about the structure and interrelationships among the data. The data mining process identifies relationships that are expressed as classification rules, association rules, and clusters of records. Data mining algorithms are used to create models that describe existing data and relationships within the data. These methodologies create rules used to analyze new data and predict future outcomes. The data set used in this analysis contains information across two years. Data is recorded in the ANGEL data base for every class offered during Fall, Spring, and Summer for every feature. This data consisted of class specific data, e.g. semester, department, course number, and student count, and feature data, e.g. feature (files, messages, folders etc.) and frequency of feature usage. The data set was processed using classification, association, and clustering algorithms as implemented by the Waikato Environment for Knowledge Analysis (WEKA, Witten and Frank, 2005). The intent of this study is not to compare ANGEL to other LMS but rather to assess the usage of the ANGEL features by students and instructors. Basing this study on recent historical data provides a good indication of the expected ANGEL use in the near term future. The college’s course offering and student and faculty profiles are not expected to change in the next few years. Classification mining determines the influence of combinations of attributes on a specific goal. In the case of this ANGEL data, multiple classification

98

models determine the usage of each feature based on student count, department, level, and semester. Association mining finds patterns in the data that reoccur frequently. This is an effective method for finding features that co-occur. Association rule mining was preformed to find the relationships between the features of ANGEL. This could lead to acceptance or rejection of sets of features. Clustering finds natural clusters occurring in data. These multi-dimensional clusters divide the data into groups; sufficiently large groups indicate the most important features in the data. In the ANGEL data a large cluster will indicate the most commonly used features. The WEKA data mining tool is used to apply the data mining techniques, and Microsoft’s Excel was used to prepare the data.

Preprocessing Data often contains attributes and records that are not pertinent to the analysis being conducted. In the case of the ANGEL data set, it consisted of records from all classes offered by the college for the Fall and Spring semesters from Fall of 2007 to Spring of 2009. For each of these classes the feature usage was collected. This original data set contained 9430 records, one for each class section, and 20 attributes (features). From this data set records for Thesis Continuation Credit (TCC) classes were removed, because TCC classes are an extension of a previous thesis course but were not identified by department. The information gained from these sections would have been inconclusive since the original class is not identified. The General Education Program (GEP) class records with sections of 0 or 99 were also removed. These sections are used to place all incoming transfer students in the computer skills exam, passing the exam is a graduation requirement, but an actual class does not exist. Five attributes were removed from the data set. The attribute identifying the section number was not relevant to the scope of our analysis. Determining the difference between section 1

ANGEL Mining

and section 2 is trivial. The attributes indicating the use of wikis, games, blogs, and assessments features were removed due to their extremely low usage. For example, assessments, which posted the highest usage of the four, was only used by 64 classes or .68% of total classes. After removal of the records and attributes the data set was reduced to 9312 department-course level- semester records with 15 attributes (Table 1). Some attributes had their values adjusted. The course number was changed to a nominal Level number, for example 436 became L400, Semester was changed to a nominal value, for example 200709 became S200709 for the Fall semester 2007. Other attributes had values for the number of times the feature was used in the course. For all empty attribute values a zero (0) was inserted, because the absence of a value meant the feature was used zero times for that class. These adjustments created the base data set. The data mining algorithms required slight variations from this base data set. For classification and association mining a nominal data set was needed. The numerical feature data was

discreteized to nominal data using six bins with equal depth binning. This method creates six bins for each attribute where the number of records in each bin is kept close to equal. Nominal data for student count used the college’s Common Data Set from 2008-2009. This document binned student count by the ranges 2-9 (very low), 10-19 (low), 20-29(medium low), 30-39 (medium), 40-49 (medium high), 50-99 (high), 100+ (very high). Classes of a single student are not specified in the Common Data Set, therefore they were assigned a value of ‘single’. Preprocessing resulted in a data set characterized by department-course level-semester records with binned counts of feature usage. Beyond this data preparation, preprocessing done on the data set is specific to the data mining method.

Classification Using Decision Trees The goal of classification of the ANGEL data is to examine the influence of semester, department, level, and student count on each feature. This requires classifying the data with respect to each

Table 1. Attributes Name

Description

Department

The academic department of the course

Drop Box

Number of drop boxes used to electronically collect assignments

Files

Count of files available for download

Folders

Number of folders used to organize content

Form

Number of surveys to be completed by students

Forum

Number of online discussions

Grade Book

Number of grade books (usually one)

Links

Number of URLs to resources outside the LMS

Level

The level of the course (e.g. 100, 200, 300, etc.)

Messages

Number of emails sent from within the course

Pages

Number of content pages available for students to view

Quizzes

Number of LMS administered tests

Semester

The semester in which the section was taught

Student Count

Number of students in the section

TurnItIn

Number of assignments collected and evaluated for originality

99

ANGEL Mining

feature individually. From the data set a feature specific data set was created consisting of semester, department, level, student count and one feature. These data sets were processed individually to create feature specific decision trees. Decision tree analysis creates a tree structure, where each internal node denotes a test on an attribute, each branch represents an outcome of the test, and every leaf represents a class or class distribution (Bagui, 2006). Decision tree analysis allows for easy conversion to classification rules. A decision tree starts as a single node. If all records in that node are the same it becomes a leaf. If they are not all the same, a selection algorithm uses entropy, a measure of the inconsistency of the data, to decide how to divide the records. An attribute is chosen that best divides the records into further purer nodes. The attribute chosen is labeled as the decision attribute. Branches are then created for the known values of the decision attribute, and the records are divided accordingly into the child nodes. This process is performed Figure 1. Generic decision tree

100

recursively on all child nodes (Bagui, 2006). For example, if given a data set consisting of both boys and girls and their favorite sport, using entropy the algorithm selects gender as the first decision attribute producing two child nodes. The data set records are divided at this node. Each subset undergoes the same algorithmic process at the individual gender nodes to further divide the records. This process continues until only pure nodes occur, creating leaves. An example of a completed classification decision tree is presented in Figure 1. This generic decision tree has four nodes or points at which decisions are made. A record would be classified depending on the values of its attributes into one of the leaf nodes, where the class attribute has a specific value. In this example the possible class attribute values are w, x, y, and z. A record with the attribute values (Attr1 = a then Attr2 = 4) would be classified as the class attribute with the value x. This record is classified by following the Attr1- Attr2 edge of the tree and then the

ANGEL Mining

Attr2-ClassAttr=x edge, reaching a ClassAttr=x leaf. Other records may also reach a leaf with ClassAttr=x (e.g., Attr1 = c then Attr3 = g then Attr4 = s), but via a different path. After the decision tree is constructed, each branch of the decision tree is converted into a rule. The tree presented in Figure 1 can be converted into the following Rules: • • • • • •

Rule 1: IF Attr1 = a AND Attr2 ≤ 5 THEN ClassAttr = x Rule 2: IF Attr1 = a AND Attr2 > 5 THEN ClassAttr = y Rule 3: IF Attr1 = b THEN ClassAttr = z Rule 4: IF Attr1 = c AND Attr3 = g AND Attr4 = s THEN ClassAttr = x Rule 5: IF Attr1 = c AND Attr3 = g AND Attr4 = t THEN ClassAttr = w Rule 6: IF Attr3 = h THEN ClassAttr = y

The rules provide insight into how the class attribute’s value is, in fact, dependent on the other attributes. A complete decision tree provides for all possible combinations of the attributes and their allowable values reaching a single, allowable class attribute value. The decision tree algorithm C4.5 builds the decision tree from a training data set using information entropy (Quinlan, 1993). The goal of the algorithm is to classify all the training set records according to the goal or class attribute. The algorithm first determines the attributes available in the first node. The normalized information gain with respect to the class attribute is calculated for each attribute to determine how the records should be split into subsets. The attribute with the highest normalized information gain is selected as the decision attribute. The records are then broken into branches on the values of the decision attribute, creating branches of the tree ending in child nodes. This process is performed recursively (Quinlan, 1993). The class attribute and its values constitute the leaf nodes of the decision tree.

Information gain is the change in information entropy from the current state of the set of records to the proposed state of the set of records. Entropy is a measure of the randomness of the distribution of records in a subset of records with respect to the class attribute. The attribute with the greatest information gain is selected as the node for dividing the data in the decision tree at each node. The information gain with the overall highest gain with respect to the dependent attribute is the root of the tree. The confidence that a record is correctly classified provides the accuracy of splitting the record set. Confidence is calculated by dividing the number of records correctly classified by the total number of records that are classified at the node. Confidence is used to prune the decision tree, ensuring a tree with a reasonable number of leaf nodes and acceptable accuracy. The level of pruning is controlled by setting a minimum confidence level. If a node falls below the minimum confidence that branch is pruned; that is, the records are rolled up to the parent node until the minimum confidence is met This allows for only rules that meet the required accuracy to remain on the final tree. The data set was divided into training and test sets, where the training set contained two-thirds of the records, randomly selected. The remaining records are in the test set. The decision tree was constructed using the training set and validated with the test set. To determine the appropriate confidence level multiple trees were constructed using individual features as the class attribute. Evaluation of multiple decision trees is common in classification problems (Osei-Bryson, 2004). After several trials of varying confidence, 55% confidence was selected. This choice was determined after trials focusing on minimizing the percent incorrectly classified, while maintaining quality leaf nodes. However, the direct relationship of these features needed to be taken into account. As the number of leaf nodes decreases, the num-

101

ANGEL Mining

ber of incorrectly classified records decreases, as well as the quality of rules formed from the nodes decreasing. The goal of classification of ANGEL data is to examine the influence of semester, department, level, and student count on a feature. To accomplish this task, our fully nominal data set was broken into eleven data sets, which in turn were divided into training and test sets. Each data set consisted of semester, department, level, student count and one feature, each having a different feature. For example, semester, department, level, student count, and file. Dividing the data set into these smaller data sets allowed each feature to be a class attribute. Records with zero values for the class (feature) attribute were removed from the respective feature data set. The C4.5 algorithm was executed on each data set with a confidence level of 55%. The decision tree is converted into IF-THEN rules. Each rule represents one branch of the decision tree from the root node to a leaf node. The intermediate nodes provide additional information in the form of branching on the branches. There is one rule for each of the leaf nodes of the tree. The results of the classifications followed a pattern. For the top five features (messages, files, folders, grade book, and links), the attribute with the highest information gain was student count followed by department. From this several rules were formed that concluded that as class size increased the usage of the feature increased. Figure 2 provides a sample of the rules found. For the little used features (drop box, form, forum, page, quiz and turn it in) the attribute with the highest information gain was first department followed by level. From this classification a definite conclusion pertaining to all classes cannot be made. Several departments have lower level classes with heavier users, while other departments have higher level classes with the heavier users (See Figure 3 for rule examples). From classification it is found that features with a less traditional online use such as, admin-

102

istering a test or having students turn in a paper, have usage that is directed more by the department and level than the class size. This suggests users who are more willing to experiment with the new technology create the demand. Those users then have an effect on their colleague’s use of that feature. Whereas, with the more commonly used online features, such as e-mail messages or file sharing, usage is directed by need. This suggests that the more students in a class, the more likely a professor is to post files rather than print the files to hand out in class. These findings support Nichols’(2003) hypothesis that the movement to eLearning, including the use of LMS, is an evolutionary, not revolutionary process. As success is seen in one course where an LMS is used the LMS will be adopted in other, similar courses.

Association Rule Mining The goal of association mining the ANGEL data was to evaluate each feature’s affect on the usage of the other features. This required modification to the original data set. Non-feature attributes were removed so that patterns of feature usage would show the use of one feature with another in the association rules. Association rule mining, unlike classification rule mining, does not need a class attribute in order to find meaningful results. Association mining allows for rules to be generated for any combination of attributes within the data set. Association Rules are not intended to be used together (Witten and Frank, 2005). Each rule generated in association mining implies a different regularity within the data set, and each could result in a very different conclusion. With the ANGEL data, relationships are found between the different features used for a class on campus. To construct association rules itemsets found in the data set are used. An itemset is a collection of attribute-value pairs (items) that occurs in the data set. Each itemset can be converted into a

ANGEL Mining

Figure 2. Sample classification rules of top features

number of rules, where each item in an itemset implies and is implied by every other item and combination of items in the itemset. This results in a very large number of rules. Since association rule mining results in a very high number of rules, restrictions need to be applied to specify whether a rule is significant and valid. The metrics of support and confidence are used to determine the significance of a rule. The support of an itemset is the count or percent of records that contain all the items in the itemset. A support threshold is set which must be met or exceeded for a itemset to be considered a frequent itemset. In the example above the support count threshold was set to two.

Rules that come from the same itemset all have the same support value. A rule contains a premise (IF-part) and consequent (THEN-part) and states that when the premise is true the consequent will be true. The confidence that this rule is correct is also calculated. The confidence is a conditional probability that a record containing the premise will also contain the consequent. It is calculated by dividing the support for the rule by the support just for the itemset that is the same as the premise. Only those rules meeting the user-specified confidence threshold are kept. Each rule is an observation of the data’s behavior. The algorithm chosen for this investigation is the Apriori algorithm. The Apriori algorithm uses itemsets to generate rules, and support based

103

ANGEL Mining

Figure 3. Sample classification rules for forums

pruning to help control the growth of rules. It also uses the Apriori Principle. This principle states that if an itemset is frequent, then all of its subsets must also be frequent (Agrawal, Imieliński, & Swami, 1993). The Apriori algorithm first considers the single itemsets, those with one only item and counts how frequently the item appears in the data. The itemsets that do not meet the minimum support threshold are discarded from the possible one itemsets. The Apriori Principle ensures that all supersets of the one itemsets that are infrequent, are also infrequent. A list of two itemsets is generated from the list of one itemsets. The frequency of the two itemsets is determined and those not meeting the minimum support threshold are removed. Three itemsets are created from the two itemsets and the process continues until no more frequent itemsets can be found. Once all of the frequent itemsets are generated, the rules are generated. If a rule does not meet the minimum confidence

104

level, it gets discarded as well as any rule that is a subset of that rule. The goal of association mining the ANGEL data was to evaluate a feature’s affect on the usage of one or more other features. To create the data set necessary to achieve this goal several attributes (semester, department, level and student count) were removed from the original data set. These attributes were removed to generate only rules pertaining to the use of one feature’s influence on another. For example, the use of folders indicates the use of files. The minimum value for support was set to 0.7, the value for confidence was set to 0.9, and the number of rules was set at 3000. Support and confidence were set at such high values to ensure presentation of rules with certainty. The number of rules was set so as to be exhaustive and find all possible rules within the bounds of support and confidence. The algorithm generated 2,638 rules.

ANGEL Mining

The useful rules can be defined by applying a template (Klemettinen et al. 1994). Only those rules where the predicate consisted of a single feature were kept. The rules then had the appearance of, if attribute X is used, then these other attributes are used. These rules show how using one feature of the ANGEL system influences usage of other features. However, only rules where the feature count was zero were found. Nevertheless, this still is an interesting, and meaningful result. The rules now have the form, if you do not use this feature, then you will not use these other features. In order to reduce the number of rules further, if there existed a rule X where the consequence was a subset of the consequence of rule Y, then X was removed (Rajasethupathy, Scime, Rajasethupathy, & Murray, 2009). After the filtering of rules, 37 rules remained. (Appendix A). Inspecting the rules from association rule mining leads to a few general conclusions. Given the level of confidence and support in this study, the use, or non-use, of email messages or files uploaded to ANGEL have no affect on the other features. There is no correlation between those two features and any of the others. One might assume that if a particular class on ANGEL uploads a large number of files that this high frequency of usage could be seen with other features, however this investigation did not find that to be the case. Another conclusion is that if a class does not use the drop box feature, then they will not use any feature, or any combination of features. When rules were being purged, there existed only one rule with drop box in the predicate. This rule can be used as a metric for ANGEL usage. (Note: it is not logically correct to say that if they do use the drop box feature then they will use other features.) At the theoretical level, García, Romero, Ventura, and Calders (2007) determined that in association mining critical factors in determining usage are the number of messages, emails, documents, and Web pages on the course site. However, the findings of this study indicate that

the number of messages and emails do not influence the feature usage. In association mining student Moodle activity in a course, Romero, Ventura, and García (2008) found that students that do not send messages (or email) do not read them either, which then leads to course failure. While there is no relationship between using the message components of ANGEL clearly use of a LMS features, while not ensuring a passing grade, does mitigate the likelihood of failure.

Cluster Rule Mining The goal in clustering the ANGEL data set was to provide an indication of the most used features in larger classes and therefore by the most students. Additionally, indications of infrequent use of some features, while identifying where these features are used, is found in smaller clusters. In conventional terms, clustering is when you group similar objects together. In data mining, this definition still holds. Clustering in data mining is taking similar records and grouping them together into clusters based on a measurable distance between them. Once these clusters are created, similarities can be determined within the data set that may not have been previously apparent. In clustering each attribute is a dimension in the problem space. Each record is placed as a point in the problem space based on the values of the record’s attributes. The process of clustering is based on comparing the records, calculating the distance between them, and then grouping like records together. The collection of clusters (known as a clustering) has clusters consisting of records that are very similar to records inside the cluster, but dissimilar to those within other clusters. The goal of clustering is to indentify common feature groupings found in ANGEL, as well as, the count of students associated with them. The algorithm used for clustering for this investigation is k-means. Since clustering is graphical the data must be fully numeric. Therefore, the

105

ANGEL Mining

pre-discretized data set was used. This data set contained numerical data for feature usages and student count. The nominal attributes (semester, department, and level) were removed. The k-means algorithm requires an input for the number of clusters (k). The algorithm randomly selects k points to be the initial cluster centroids. The records are formed into these k clusters based on to which centroid a given record is closest, closeness being determined by the distance measure. The centroid of each cluster is recalculated using the records in the cluster. The distance from each record to each centroid is recalculated and records may change clusters. This process continues until the difference between the current centroid and the previous centroid is zero or near zero for each cluster. The value of k determines the number of clusters, varying k results in different clusterings or grouping of the records. To determine the optimal clustering the sum of squared errors (SSE) are compared to the number of clusters. As the number of clusters increases the SSE should decrease, when the SSE begins to remain nearly constant the smallest number of clusters in typically chosen. This process was performed on the ANGEL data set. Fifteen clusterings were computed, increasing k by two from two to thirty. For each analysis of k the SSE was recorded and graphed (Figure 4). From the graph, 23 clusters were chosen for cluster rule mining. This cluster size was chosen as it is the point where the graph levels out. The area of the graph where leveling occurs is considered to be optimal for cluster size because once the graph begins to level off the measure of SSE from one k to the following k+2 is less meaningful. Therefore, the analysis is producing sub optimal clusters, or with less meaningful information than their predecessor. For example, in our data set using twenty-six clusters would not produce better results. It would be more likely to divide good clusters.

106

The goal in clustering the data set was to find natural feature groupings in the data set and the class size associated with that grouping. In the 23 cluster clustering, five clusters were formed that contained 72% of the records, and revealed files and messages as the only features used by those clusters. Furthermore, clustering revealed seven clusters totaling 3.7% of records, that used nine to ten of eleven features. Lastly, the largest cluster holding 24% of records contained an average student count of 1.78 and average of 0 to 0.45 for feature use. That cluster formed the rule that if a class has two or less students they will not use ANGEL. The results received by clustering show that the most commonly used features in ANGEL are files and messages. Most classes do not use the more advanced features in ANGEL, yet there is a small subset of instructors that heavily use advanced features. Finally, classes with two or less students do not use any feature in ANGEL; this may imply that it is not worth the instructor’s time to create content for just a few students. Romero, Ventura, and García’s (2008) study of types of Moodle students found three clusters defining types of students: very active, active, and non-active. Very active students are characterized as having sent more than one message, read about three messages, passed a large number of quizzes, and spent time on Moodle doing assigned work. Non-active students did no assignments, read few messages, completed very few quizzes, and did not spend time on Moodle. Active student’s behavior was between the other two groups. While messages are one of the most commonly used features in ANGEL, it is up to the individual student to define themselves as very active, active, or non-active.

FUTURE RESEARCH DIRECTIONS Today, the use of the computer has become common for statistical analysis of data. Software packages are easy to use, inexpensive, and fast. But

ANGEL Mining

Figure 4. Number of clusters vs. SSE

today’s vast stores of data with immense data sets make comprehensive analysis all but impossible using conventional techniques. A solution is data mining. Classification mining finds the rules that can predict future behavior. Association mining discovers patterns in the data. Clustering provides insight into the data. Currently data mining requires some specialized skill – the ability to understand the processes, the data, and expertise in the domain. In the case of Learning Management Systems, how the system operates and instruction techniques may be necessary to understand the results. This combination of skills is difficult to find in a single person. In the future data mining tools will become easier to use, as statistical and spreadsheet tools are today. Then domain experts will be able to apply data mining to their data without the need for a data mining expert. The rules found and conclusions drawn can then be used with more confidence when making decisions within the domain. Data mining is commonly conducted against transactional data, but data have gone beyond simple numeric and character flat file data. Future LMS data will come in many forms: image, video, audio, streaming video, and combinations

of data types. Data mining research is being conducted to find interesting patterns in data sets of all these data types. Beyond the LMS data, other data sources may supplement the LMS data. Data from corporate and government data warehouses, transactional databases, and the World Wide Web (Scime, 2008) may be added to enhance the LMS. The data mining of an LMS of the future will be multidimensional, accessing all these data sources and data types to find the optimal use of a school’s LMS. In addition to the new forms of data that will need to be mined in future iterations of the LMS, the very nature of the LMS is likely to see significant changes in approaching years. As colleges and universities become more attune to the needs of both regional (Middle States) and program accreditation (NCATE, AACSB, ABET) the usage of the LMS for purely learning or course management activities is likely to transform to put greater emphasis on assessment and accreditation activities. Where the use of specific system features has not been traditionally mandated, it is likely that there could be a shift to the mandated use of features such as rubrics and assessment tools.

107

ANGEL Mining

With this potential shift comes the opportunity for data mining to expand beyond evaluating just the usage of the LMS but to also provide deep insights into the performance of students against institutional and program objectives. Correlating the use of LMS features with student performance could provide the opportunity to enhance the use of the LMS to provide stronger educational outcomes. This in turn, could provide guidance in improving college and university programs to produce stronger and more competitive classes of graduates. Finding more interesting and supportive rules for a domain using multiple methods places constraints on the mining process. This is a form of constraint-based data mining. Constraint-based data mining uses constraints to guide the process. Constraints that can be used can specify the data mining algorithm. Constraints can be placed on the type of knowledge that is to be found or the data to be mined. Dimension-level constraints research is needed to determine what level of a summary, or the reverse, detail is needed in the data before the algorithms are applied (Hsu, 2002). Research in data mining will continue to find new methods to determine interestingness. Research is needed to determine what values of a particular attribute are considered to be especially interesting in the data and in the resulting rule set (Hsu, 2002). Currently there are 21 different statistically based objective measures for determining interestingness (Tan, Steinbach & Kumar, 2006). A leading area of research is to find new, increasingly effective measures. With regard to subjective measures of interestingness, research in domains that is both quantitative and qualitative can lead to new methods for determining interestingness. Further data mining research will find new methods to support existing knowledge and perhaps find new knowledge in domains where it has not yet been applied (Scime, Murray & Hunter, 2010).

108

CONCLUSION In this investigation, statistical methods were used to determine whether or not features were being used in the ANGEL system. Data mining was able to generate results, and from these results conclusions were made from the three different data mining techniques; classification, association, and clustering. A few general conclusions can be made, the majority of classes at this college do not use the more advanced features of ANGEL. The only parts of the system being used by a large number of classes are the messaging and file sharing features. Another conclusion is that there exists a relationship between need and use. If there are a large number of students, then using ANGEL to send messages or share files alleviates the instructor from a certain amount of additional effort. For example, instead of printing off enough copies for the class, the file is shared on ANGEL. Whereas, classes with a low student count do not use this feature as frequently. Association suggests that there is not a relationship between the use of one feature to the use of another feature. However, association does conclude the non-use of features implies the non-use of other features. Furthermore, during pre-processing, it was found that there are features that are not being used at all (wikis, blogs, assessments, and games). Nevertheless, it has been found that there are a small number of instructors who are heavy ANGEL users. These users often utilize many of the features available. These results can be used by this institution, as well as similar institutions, for decision making concerning feature selection and overall usefulness of LMS design, selection and implementation or to identify feature areas needing additional training in their use.

ANGEL Mining

REFERENCES Agrawal, R., Imieliński, T., & Swami, A. (1993). Mining association rules between sets of items in large databases, Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data, Washington, DC. (pp.207-216). Alsop, G., & Tompsett, C. (2002). Grounded theory as an approach to studying students’ uses of learning management systems. ALT-J Research in Learning Technology, 10(2), 63–76. doi:10.1080/0968776020100207 Avgeriou, P., Papasalouros, A., Retalis, S., & Skordalakis, M. (2003). Towards a pattern language for learning management systems. Journal of Educational Technology & Society, 6(2), 11–24. Bagui, S. (2006). An approach to mining crime patterns. International Journal of Data Warehousing and Mining, 2(1), 50–80. doi:10.4018/ jdwm.2006010103 Beatty, B., & Ulasewicz, C. (2006). Faculty perspectives on moving from blackboard to the moodle learning management system. TechTrends, 50(4), 36–45. doi:10.1007/s11528-006-0036-y Blackboard (2010). Blackboardlearn+ ANGEL edition. Retrieved on April 19, 2010, from http:// www.blackboard.com/Teaching-Learning/LearnResources/ANGEL-Edition.aspx Chapman, B. (2010). LMS KnowledgeBase 2010: In-depth profiles of 90+ learning management systems, with custom comparison across 200+ features. Brandon Hall Research, Retrieved on November 9, 2010, from http://www.brandon-hall. com/publications/lmskb/ lmskb.shtml Commonwealth of Learning (2004) LMS Evaluation Tool User Guide, 1.2, 3waynet Inc. and the Commonwealth of Learning.

Deshpande, M., & Karypis, G. (2002). Using conjunction of attribute values for classification. Proceedings of the 11th International Conference on Information and Knowledge Management, McLean, VA (pp. 356-364). Falvo, D. A., & Johnson, B. F. (2007). The use of learning management systems in the United States. TechTrends, 15(2), 40–45. Feldstein, M., & Masson, P. (2006). Unbolting the chairs: Making learning management systems more flexible. E-learn magazine. Retrieved from http://www.elearnmag.org/subpage.cfm?section= tutorials&article=22-1 on April 19, 2010. Fu, X., & Wang, L. (2005). Data dimensionality reduction with application to improving classification performance and explaining concepts of data sets. International Journal of Business Intelligence and Data mining, 1(1), 65-87. García, E., Romero, C., Ventura, S., & Calders, T. (2007). Drawbacks and solutions of applying Association Rule mining in learning management systems. Proceedings of the International Workshop on Applying Data mining in e-Learning, Crete, Greece (pp. 13-22). Graf, S., & Kinshuk (2006). Considering learning styles in learning management systems: Investigating the behavior of students in an online course. Proceedings of the 1st IEEE International Workshop on Semantic Media Adaptation and Personalization, Athens, Greece (pp. 25-30). Haldane, A. (1998). The convergence of e-learning and knowledge management: Fusion or confusion? People development in a knowledge economy, 2. Learning Futures Ltd., Retrieved on April 26, 2010 from http://www.learningfutures.co.uk/LFdocs/ MKelleher_06-01-03_15-29-27.pdf. Hall, J. (2003). Assessing learning management systems. Chief learning officer. Retrieved on April 19, 2010, from http://www.clomedia.com/ features/2003/January/91/index.php.

109

ANGEL Mining

Hsu, J. (2002). Data mining trends and developments: The key data mining technologies and applications for the 21st century, Proceedings of the19th Annual Conference for Information Systems Education (ISECON 2002), San Antonio, TX (Art 224b). Jaroszewicz, S., & Simovici, D. A. (2004). Interestingness of frequent itemsets using Bayesian networks as background knowledge. Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data mining (pp. 178-186). Seattle, WA. Kim, S. W., & Lee, M. G. (2008). Validation of an evaluation model for learning management systems. Journal of Computer Assisted Learning, 24(4), 284–294. doi:10.1111/j.13652729.2007.00260.x Klemettinen, M., Mannila, H., Ronkainen, P., Toivonen, H., & Verkamo, A. I. (1994). Finding interesting rules from large sets of discovered association rules. Proceedings of the 3rd International Conference on Information and Knowledge Management (CIKM’94) Gaithersburg, Maryland, USA (pp. 401-408). Kritikou, Y., Demestichas, P., Adamopoulou, E., Demestichas, K., Theologou, M., & Paradia, M. (2008). User profile modeling in the context of Web-based learning management systems. Journal of Network and Computer Applications, 31(4), 603–627. doi:10.1016/j.jnca.2007.11.006 ANGEL Learning (2008). The ANGEL learning management suite. Lewis, B. A., MacEntee, V. M., DeLaCruz, S., Englander, C., Jeffrey, T., & Takach, E. …Jason W. (2005). Learning management systems comparison. Proceedings of the 2005 Informing Science and IT Education Joint Conference, Flagstaff, Arizona.

110

Li, W., Han, J., & Pei, J. (2001). CMAR: Accurate and efficient classification based on multiple class-Association rules. Proceedings of the 2001 IEEE International Conference on Data mining (pp. 369-376). MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability (pp. 281-297). Malikowski, S. R., Thompson, M. E., & Theis, J. G. (2007). A model for research into course management systems: Bridging technology and learning theory. Journal of Educational Computing Research, 36(2), 149–173. doi:10.2190/10021T50-27G2-H3V7 Moore, M. G. (1993). Theory of transactional distance. In Keegan, D. (Ed.), Theoretical principles of distance education (pp. 33–38). New York, NY: Routledge. Murray, G. R., Riley, C., & Scime, A. (2007, May). A new age solution for an age-old problem: Mining data for likely voters. Paper presented at the 62nd Annual Conference of the American Association of Public Opinion Research, Anaheim, CA. Nichols, M. (2003). A theory for elearning. Journal of Educational Technology & Society, 6(2), 1–10. Osei-Bryson, K.-M. (2004). Evaluation of decision trees: A multicriteria approach. Computers & Operations Research, 31(11), 1933–1945. doi:10.1016/S0305-0548(03)00156-4 Padmanabhan, B., & Tuzhilin, A. (2000). Small is beautiful: Discovering the minimal set of unexpected patterns. Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data mining, Boston, MA (pp. 54-63).

ANGEL Mining

Paulsen, M. F. (2003). Online education and learning management systems. Bekkestua, Norway: NKI.

Sturgess, P., & Nouwens, F. (2004). “Evaluation of online learning management systems. Turkish Online Journal of Distance Education, 5(3).

Quinlan, J. R. (1993). C4.5: Programs for machine learning. San Francisco, CA: Morgan Kaufmann.

Tan, P.-N., Steinbach, M., & Kumar, V. (2006). Introduction to data mining. Boston, MA: Addison Wesley.

Rajasethupathy, K., Scime, A., Rajasethupathy, K. S., & Murray, G. R. (2009). Finding “persistent Rules”: Combining association and classification results’, Expert Systems With Applications, 36(3P2), 6019-6024. Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York, NY: Free Press. Romero, C., Ventura, S., & García, E. (2008). Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51(1), 368–384. doi:10.1016/j. compedu.2007.05.016 Roqueta, M. (2008). Learning management systems: A focus on the learner. Distance Learning, 5(4), 59–66. Scime, A. (2008). Web page extension of data warehouses. In Wang, J. (Ed.), Encyclopedia of Data Warehousing and Mining (2nd ed., pp. 1211–1215). Hershey, PA: Idea Group Reference. Scime, A. & Murray, G. R. (2007). Vote prediction by iterative domain knowledge and attribute elimination. International Journal of Business Intelligence and Data mining, 2(2), 160-176. Scime, A., Murray, G. R., & Hunter, L. Y. (2010). Testing terrorism theory with data mining. International Journal of Data Analysis Techniques and Strategies, 2(2), 122–139. doi:10.1504/ IJDATS.2010.032453 Sclater, N. (2008). Large-scale open source e-learning systems at open university UK. Research Bulletin EDUCAUSE Center for Applied Research, 2008(12), 1-13.

Watson, W. R., & Watson, S. L. (2007). An argument for clarity: What are learning management systems, what are they not, and what should they become? TechTrends, 51(2), 28–34. doi:10.1007/ s11528-007-0023-y Witten, I. H., & Frank, E. (2005). Data mining: Practical machine learning tools and techniques (2nd ed.). San Francisco, CA: Morgan Kaufman.

ADDITIONAL READING Anand, S. S., Bell, D. A., & Hughes, J. G. (1995). The role of domain knowledge in data mining. Proceedings of the 4th International Conference on Information and Knowledge Management, Baltimore, MD (pp. 37-43). Andoh-Baidoo, F. K., & Kweku-Muata, O.-B. (2007). Exploring the characteristics of internet security breaches that impact the market value of breached firms. Expert Systems with Applications, 32(3), 703–725. doi:10.1016/j.eswa.2006.01.020 Ankerst, M., Martin, E., & Kriegel, H.-P. (2000). Towards an effective cooperation of the user and the computer for classification. Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data mining (pp. 179-188). Boston, MA. Bremer, D., & Bryant, R. (2005). A comparison of two learning management systems: Moodle vs blackboard. Proceedings of the 18th Annual Conference of the National Advisory Committee on Computing Qualifications (pp. 135-139). Tauranga, NZ.

111

ANGEL Mining

Brusilovsky, P. (2003). A distributed architecture for adaptive and intelligent learning managament systems. Proceedings of the AIED 2003 Workshop Towards Intelligent Learning Management Systems (pp. 5-13). Sydney, Australia.

Li, J., Tang, J., Li, Y., & Luo, Q. (2009). RiMOM: A dynamic multistrategy ontology alignment framework. IEEE Transactions on Knowledge and Data Engineering, 21(8), 1218–1232. doi:10.1109/ TKDE.2008.202

Carliner, S. (2005). Course management systems versus learning management systems. Learning Circuits, Retrieved from http://www.astd.org/ LC/2005/1105 _carliner.htm on April 19, 2010.

Machado, M., & Tao, E. (2007). Blackboard vs. Moodle: Comparing user experience of learning management systems. Proceeding of the 37th ASEE/IEEE Frontiers in Education Conference, (pp. S4J-7 - S4J-12). Milwaukee, WI.

Coates, H., James, R., & Baldwin, G. (2005). A critical examination of the effects of learning management systems on university teaching and learning. Tertiary Education and Management, 11(1), 19–36. doi:10.1080/13583883.2005.996 7137 Giarratano, J. C., & Riley, G. D. (2004). Expert systems: Principles and programming (4th ed.). New York: Course Technology. Han, J., & Kamber, M. (2001). Data mining: Concepts and techniques. Boston, MA: Morgan Kaufmann. Hofmann, M., & Tierney, B. (2003). The involvement of human resources in large scale data mining projects. Proceedings of the 1st International Symposium on Information and Communication Technologies (pp.103-109). Dublin, Ireland. Kass, G. (1980). An exploratory technique for investigating large quantities of categorical data. Applied Statistics, 29, 119–127. doi:10.2307/2986296 Kumar, D. A., & Ravi, V. (2008). Predicting credit card customer churn in banks using data mining. International Journal of Data Analysis Techniques and Strategies, 1(1), 4–28. doi:10.1504/ IJDATS.2008.020020 Lane, T., & Brodley, C. E. (2003). An empirical study of two approaches to sequence learning for anomaly detection. Machine Learning, 51(1), 73–107. doi:10.1023/A:1021830128811

112

Magidson, J. (1988). Improved statistical techniques for response modeling. Journal of Direct Marketing, 2(4), 6–18. doi:10.1002/ dir.4000020404 Magidson, J. (1994). The CHAID approach to segmentation modeling: Chi-squared automatic interaction detection. In Bagozzi, R. P. (Ed.), Advanced Methods of Marketing Research. Cambridge, MA: Basil Blackwell. Memon, N., & Qureshi, A. R. (2005). Investigative data mining and its application in counterterrorism. Proceedings of the 5th WSEAS International Conference on Applied Informatics and Communications (pp. 397-403). Malta. Paulsen, M. F. (2003). Experiences with learning management systems in 113 european institutions. Journal of Educational Technology & Society, 6(4), 134–148. Phelps, C., & Michea, Y. (2003). Learning management systems’ evaluation focuses on technology not learning. AMIA Annual Symposium Proceedings, (pp. 969). Ramu, K., & Ravi, V. (2008). Privacy preservation in data mining using hybrid perturbation methods: An application to bankruptcy prediction in banks. International Journal of Data Analysis Techniques and Strategies, 1(4), 313–331. doi:10.1504/IJDATS.2009.027509

ANGEL Mining

Robinson, S. M. (2008). Moodle vs. WebCT: How the design of features affect the efficiency of e-learning by influencing interaction and collaboration. Retrieved from http://web.njit.edu/~sr89/ MoodlevsWebct.doc on April 19, 2010. Roiger, R., & Geatz, M. W. (2003). Data mining: A tutorial-based primer. New York, NY: Addison Wesley. Sclater, N. (2008). Web 2.0, personal learning environments, and the future of LMS. Research Bulletin EDUCAUSE Center for Applied Research, 2008(13), 1-13. Su, X., Khoshgoftaar, T. M., & Greiner, R. (2009). Making an accurate classifier ensemble by voting on classifications from imputed learning sets. International Journal of Information and Decision Sciences, 1(3), 301–322. doi:10.1504/ IJIDS.2009.027657 Tozicka, J., Rovatsos, M., & Pechoucek, M. (2007). A framework for agent-based distributed machine learning and data mining. Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems (art 96). Honolulu, Hawaii. Turban, E., McLean, E., & Wetherbe, J. (2004). Information Technology for Management (3rd ed.). New York, NY: Wiley. Vatsavai, R. R., & Bhaduri, B. (2007). A hybrid classification scheme for mining multisource geospatial data, Proceedings of the 7th IEEE International Conference on Data mining Workshops (ICDMW 2007) (pp. 673-678). Omaha, NE. Wagstaff, K., Cardie, C., Rogers, S., & Schrödl, S. (2001). Constrained k-means clustering with background knowledge. Proceedings of the 18th International Conference on Machine Learning (pp.577-584). Williamstown, MA.

Wang, G., Zhang, C., & Huang, L. (2008). ‘A study of classification algorithm for data mining based on hybrid intelligent systems. Proceedings of the 9th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing (pp. 371-375). Phuket Thailand. Wang, Y., Zhang, Y., Xia, J., & Wang, Z. (2008). Segmenting the mature travel market by motivation. International Journal of Data Analysis Techniques and Strategies, 1(2), 193–209. doi:10.1504/ IJDATS.2008.021118 Webb, G. I., & Zheng, Z. (2004). Multistrategy ensemble learning: Reducing error by combining ensemble learning techniques. IEEE Transactions on Knowledge and Data Engineering, 16(8), 980–991. doi:10.1109/TKDE.2004.29

KEY TERMS AND DEFINITIONS Association Mining: A data mining method used to find patterns of data that show conditions where sets of attribute-value pairs occur frequently in the data set. Association Rule: A rule found by association mining. Classification Mining: A data mining method used to find models of data for categorizing instances; typically used for predicting future events from historical data. Classification Rule: A rule found by classification mining. Clustering: A data mining method used to group similar records together based on a measurable distance between the records. Clustering Rule: A rule found by the analysis of clusters. Data Mining: A collection of processes that inductively analyze data to assess known relationships as well as to find interesting patterns and unknown relationships.

113

ANGEL Mining

APPENDIX A: ASSOCIATION MINING RULES

countFolder=(-inf-0.5] 6649 ==> countForums=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 6545 conf:(0.98) countFolder=(-inf-0.5] 6649 ==> countQuiz=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 6502 conf:(0.98) countFolder=(-inf-0.5] 6649 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] 6496 conf:(0.98) countFolder=(-inf-0.5] 6649 ==> countForm=(-inf-0.5] countPage=(-inf-0.5] 6493 conf:(0.98) countFolder=(-inf-0.5] 6649 ==> countForums=(-inf-0.5] countPage=(-inf-0.5] 6463 conf:(0.97) countFolder=(-inf-0.5] 6649 ==> countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 6463 conf:(0.97) countDropBoxes=(-inf-0.5] 8366 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7573 conf:(0.91) countForm=(-inf-0.5] 9018 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countTurnItIn=(-inf-0.5] 8306 conf:(0.92) countForm=(-inf-0.5] 9018 ==> countQuiz=(-inf-0.5] countPage=(-inf-0.5] 8184 conf:(0.91) countForm=(-inf-0.5] 9018 ==> countDropBoxes=(-inf-0.5] countTurnItIn=(inf-0.5] 8161 conf:(0.9) countForm=(-inf-0.5] 9018 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] 8137 conf:(0.9) countForm=(-inf-0.5] 9018 ==> countForums=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 8130 conf:(0.9) countForums=(-inf-0.5] 8797 ==> countDropBoxes=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 8032 conf:(0.91) countForums=(-inf-0.5] 8797 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countForm=(-inf-0.5] 7960 conf:(0.9) countForums=(-inf-0.5] 8797 ==> countQuiz=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7938 conf:(0.9) countGradeBookAssg=(-inf-0.5] 7426 ==> countDropBoxes=(-inf-0.5] countQuiz=(inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 6713 conf:(0.9) countGradeBookAssg=(-inf-0.5] 7426 ==> countForums=(-inf-0.5] countLink=(inf-0.5] countTurnItIn=(-inf-0.5] 6704 conf:(0.9) countGradeBookAssg=(-inf-0.5] 7426 ==> countForums=(-inf-0.5] countForm=(inf-0.5] countLink=(-inf-0.5] 6694 conf:(0.9) countGradeBookAssg=(-inf-0.5] 7426 ==> countQuiz=(-inf-0.5] countForm=(inf-0.5] countLink=(-inf-0.5] countTurnItIn=(-inf-0.5] 6684 conf:(0.9)

114

ANGEL Mining

countLink=(-inf-0.5] 7955 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7312 conf:(0.92) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] 7206 conf:(0.91) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 7198 conf:(0.9) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7197 conf:(0.9) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countPage=(-inf-0.5] 7170 conf:(0.9) countLink=(-inf-0.5] 7955 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7164 conf:(0.9) countPage=(-inf-0.5] 8517 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 7938 conf:(0.93) countPage=(-inf-0.5] 8517 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countForm=(-inf-0.5] 7744 conf:(0.91) countPage=(-inf-0.5] 8517 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 7714 conf:(0.91) countPage=(-inf-0.5] 8517 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countForums=(-inf-0.5] 7682 conf:(0.9) countPage=(-inf-0.5] 8517 ==> countDropBoxes=(-inf-0.5] countQuiz=(-inf-0.5] countTurnItIn=(-inf-0.5] 7668 conf:(0.9) countQuiz=(-inf-0.5] 8767 ==> countDropBoxes=(-inf-0.5] countForm=(-inf-0.5] countTurnItIn=(-inf-0.5] 7963 conf:(0.91) countQuiz=(-inf-0.5] 8767 ==> countDropBoxes=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] 7960 conf:(0.91) countQuiz=(-inf-0.5] 8767 ==> countForums=(-inf-0.5] countForm=(-inf-0.5] countPage=(-inf-0.5] countTurnItIn=(-inf-0.5] 7938 conf:(0.91) countTurnItIn=(-inf-0.5] 9044 ==> countQuiz=(-inf-0.5] countForums=(-inf-0.5] countForm=(-inf-0.5] 8306 conf:(0.92) countTurnItIn=(-inf-0.5] 9044 ==> countForm=(-inf-0.5] countPage=(-inf-0.5] 8278 conf:(0.92) countTurnItIn=(-inf-0.5] 9044 ==> countForums=(-inf-0.5] countPage=(-inf-0.5] 8202 conf:(0.91) countTurnItIn=(-inf-0.5] 9044 ==> countDropBoxes=(-inf-0.5] countForm=(inf-0.5] 8161 conf:(0.9)

115

116

Chapter 6

Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems Kamla Ali Al-Busaidi Sultan Qaboos University, Oman Hafedh Al-Shihi Sultan Qaboos University, Oman

ABSTRACT Learning management systems (LMS) enable educational institutions to manage their educational resources, support their distance education, and supplement their traditional way of teaching. Although LMS survive via instructors’ and students’ use, the adoption of LMS is initiated by instructors’ acceptance and use. Consequently, this study examined the impacts of instructors’ individual characteristics, LMS’ characteristics, and organization’s characteristics on instructors’ acceptance and use of LMS as a supplementary tool and, consequently, on their continuous use intention and their pure use intention for distance education. The findings indicated that, first, instructors’supplementary use of LMS is determined by perceived usefulness, training, management support, perceived ease of use, information quality, and computer anxiety. Second, instructors’ perceived usefulness of LMS is determined by system quality, perceived ease of use, and incentives policy. Third, instructors’ perceived ease of use is determined by computer anxiety, technology experience, training, system quality, and service quality. Furthermore, instructors’ continuous supplementary use intention is determined by their current supplementary use, perceived usefulness, and perceived ease of use, while instructors’ pure use intention is determined only by their perceived usefulness of LMS. DOI: 10.4018/978-1-60960-884-2.ch006


Critical Factors Influencing Instructors’ Acceptance and Use of Learning Management Systems

INTRODUCTION Information and communication technologies (ICT) and the Internet have become major enablers of growth in business. The geographical outreach of the Internet and the wide global adoption of Web 2.0 technologies provide educational institutions with unprecedented opportunities to enhance their offerings. These technologies have transformed students’ perception of information and what they think about Web content and how to use it (Burnett & Marshall, 2003). Schools and universities are forced to investigate new means to revamp the educational process utilizing these technologies. Learning Management Systems (LMS) and e-learning have become the hype lately among stakeholders in education and training. The elearning market was worth more than US $18 billion worldwide in 2004 (Saady, 2005). Several leading universities around the world have adopted LMS for teachers and students to enhance the educational process (Hawkins & Rudy 2007; Browne, Jenkins & Walker, 2006; National Center for Educational Statistics, 2003). About 95 percent of participating institutions in the UK have adopted LMS (Browne et al., 2006); likewise, more than 90 percent of all participating universities and colleges in the US are adopting LMS (Hawkins & Rudy, 2007). Users’ acceptance and actual use of information systems is critical to its success. Likewise for a learning management system, its success to a great extent depends on users’ acceptance and use. Evaluating individual users’ acceptance and use of the e-learning systems is a “basic marketing element” (Kelly & Bauer, 2004). Although a learning management system survives through instructor and student use, the adoption of LMS is initiated by instructors’ acceptance and use. Even when LMS are well in place, instructors may not fully utilize all the features. For example, a survey of more than 800 instructors at over 35 institutions using Blackboard learning management system found that very few teachers use LMS tools for assessing

students or promoting community (Woods, Baker & Hopper., 2004). In addition, research indicates that fear of technology and lack of time may limit instructors’ adoption of LMS (Yueh & Hsu, 2008). Instructors should embrace and prepare for LMS use before preparing students for online learning (Chan, 2008). Even when designing LMS applications, teachers’ needs and capabilities should thoroughly be investigated (Yueh & Hsu, 2008). Therefore, teachers’ perspective within the context of LMS is crucial and should be studied carefully to ensure comprehensive uptake of LMS. Thus, the objective of this study was to investigate the critical factors influencing instructors’ acceptance and use of LMS, which may be influenced by technical and non-technical issues such as the instructors’ personal characteristics and the organization’s characteristics. It is important to analyze non-technical factors that promote the adoption and diffusion of LMS initiatives (Albirini, 2006; ElTartoussi, 2009). Consequently, this study specifically aimed to examine the impact of instructor’s individual characteristics (computer anxiety, technology experience and self efficacy), LMS characteristics (system quality, information quality and service quality), and organizational characteristics (management support, training and incentives) on the instructors’ acceptance (perceived ease of use and perceived usefulness) and use of LMS as a supplementary tool. The study also assessed the impact of the instructors’ acceptance and use of LMS on their intention for continuous supplementary use of LMS and intention for pure use of LMS for distance education. Many organizations start their LMS adoption as a supplementary tool to traditional teaching, hoping that this supplementary adoption eventually will promote the pure use of LMS for distance education. The following sections discuss the background literature, research framework and methodology, analysis and results, and the conclusion.

117


BACKGROUND Learning Management Systems A learning management system is “software that automates the administration of training” (World Bank, 2010). It is the use of a Web-based communication, collaboration, learning, knowledge transfer, and training to add value to learners and businesses (Kelly & Bauer, 2004). Specifically, a learning management system is an Internet application that aims to support education and training activities (Cavus & Momani, 2009) and provides a platform to support e-learning activities (Yueh & Hsu, 2008). Course Management Systems (CMS) and Learning Content Management Systems (LCMS) are sometimes used to indicate LMS (Yueh & Hsu, 2008); other related terms are Computer-assisted Learning (CAL), Computer-based Learning (CBL), and Online Learning (Chan, 2008). It should be noted, however, that LMS applications are not unique to educational institutions; even public and private organizations use such systems for training purposes. Reiser and Dempsey (2002) stated that in large US corporations, 20 percent of all training is delivered via LMS.

LMS Applications and Benefits to Instructors Many LMS applications are available. The most popular LMS used at colleges and universities in the US is Blackboard followed by WebCT, which was acquired by Blackboard, Inc. in 2006 (Falvo & Johnson, 2007). WebCT was used by millions of students from more than 2,500 universities in more than 80 countries (Chan, 2008). Other LMS solutions are Moodle, ATutor, Learn.com, Joomla, and Krawler. Blackboard has course management features that support integration with student databases (Kneght & Reid 2009; Blackboard, 2009). WebCT supports electronic communications such as email, bulletin boards, and chat rooms (Chan,

118

2008). Moodle learning management system, on the other hand, is sometimes preferred over the previous popular LMS packages (see, for example, Beatty and Ulasewicz, 2006). It is scalable open-source software used mainly by North American and European universities that supports group forum participation and features interactive tools between students and instructors (Beatty & Ulasewicz, 2006). LMS applications provide several features to instructors, especially to those who see the real benefits of the Internet in the teaching process (Burniske & Monke, 2001). Yueh and Hsu (2008) described course management tools, group chat and discussion, assignment submission, and course assessment as the primary tools in LMS. In addition, LMS help teachers provide students with educational materials and track their participation and assessments (Falvo & Johnson, 2007). Yildirim, Temur, Kocaman and Goktas (2004) described more technically sophisticated LMS features such as maintaining office hours online, creating student groups, and assigning online projects to groups. Ceraulo (2005) mentioned ePortfolios as a key feature in some LMS applications, which enable instructors to maintain student submissions throughout the course (i.e., tests, assignments, projects). LMS solutions tend also to increase interest in learning and teaching among students and teachers, respectively (Mahdizadeh, Biemans & Mulder, 2008). Aczel, Peake and Hardy (2008), and Naidu (2006) stated that LMS enhance teaching process efficiency and result in cost-savings.

Learning Management Systems Acceptance and Use User Technology Acceptance and Use LMS have been adopted by academic institutions to support their distance education and/or supplement their traditional way of teaching (Rainer, Turban & Potter 2007). User acceptance and use


of information systems is critical to their success. The same is true for a learning management system: its success, to a great extent, depends on users’ acceptance and use. The assessment of technology success has been conducted by utilizing various acceptance and use dimensions. Several studies assessed the determinants of technology acceptance (Bailey & Pearson, 1983; Davis, 1989; DeLone & McLean, 2003; Doll & Torkzadeh, 1988; Venkatesh & Davis, 2000). For instance, Davis (1989) assessed technology acceptance by perceived usefulness and intention to use; Venkatesh and Davis (2000) assessed technology acceptance by perceived usefulness, intention to use, and usage behavior. Alternatively, DeLone and McLean (2003) assessed technology success by user satisfaction and usage. In the LMS context, researchers also assessed instructors’ acceptance and use in various ways. Liaw, Huang and Chen (2007) assessed LMS acceptance by learners’ and instructors’ behavioral intention to use e-learning, which is influenced by perceived usefulness, perceived self-efficacy, and perceived enjoyment. Ball and Levy (2008) assessed LMS acceptance by instructors’ intention to use. Teo (2009) assessed LMS acceptance by teachers’ perceived usefulness and perceived ease of use. None of these studies, however, investigated the direct impact of instructors’ characteristics, LMS’ characteristics, and/or an organization’s characteristics on actual system use. Perceived usefulness, perceived ease of use, users’ satisfaction, and intention to use are important measures for technology acceptance and may eventually correlate with actual use behavior. Some researchers have found direct effects between such external factors and technology use (Igbaria, Guimaraes & Davis, 1995). Nevertheless, these acceptance measures do not explain all the variance of actual usage behavior. In addition, measuring attitudes and their link to actual usage behavior is extremely difficult; therefore, many researchers may choose

to stay with actual use behavior (DeLone & McLean, 2003). Thus, it is important to evaluate the direct impact of these critical determinants on actual system use. System use is and will continue to be an important indication of IS success (DeLone & McLean, 2003). In addition, the actual benefits of any technology may be realized only from actual technology use. Thus, technology use is the main aim of organizations to promote LMS and realize some of its expected benefits. Perceived usefulness and ease of use may be important factors for continuous intention to use. Consequently, As indicated earlier this study aimed to examine the direct impact of instructors’ characteristics, LMS’ characteristics, and organization’s characteristics on instructors’ actual usage behavior, instructors’ perceived ease of use of LMS, and instructors’ perceived usefulness of LMS. Furthermore, the study examined the impacts of these acceptance (perceived ease of use, perceived usefulness), and actual use measures on continuous intention for supplementary use and intention for pure use of the LMS for distance education. Perceived usefulness and perceived ease of use may be important to ensure continuous use of the system and future intention to adopt it for distance education.

Instructor Characteristics The acceptance and use of LMS may, to a great extent, be determined by the characteristics of its users. Several dimensions of users’ characteristics have been proposed and investigated as determinants of technology acceptance. Some of these are users’ experience of technology, users’ self-efficacy, and users’ computer anxiety. Users’ experience of technology has been highly utilized to determine users’ technology acceptance (Venkatesh & Davis, 2000; Thompson, Compeau, Deborah & Higgins, 2006). Individual technology experience is the individual’s exposure to the technology as well as the skills and abilities that

119


s/he gains through using a technology (Thompson et al., 2006); users’ self-efficacy pays a major role in users’ acceptance and use of technology. Self-efficacy is defined as “people’s judgments of their capabilities to organize and execute courses of action required to attain designated types of performances” (Bandura, 1977, p.391). Thus, computer self-efficacy means individual’s selfassessment of their ability to apply computer skills to accomplish their tasks (Compeau & Higgins, 1995). Furthermore, a user’s computer anxiety is considered an important factor for technology acceptance and use. Computer anxiety is defined as “the fear or apprehension felt by individuals when they used computers, or when they considered the possibility of computer utilization” (Simonson, Maurer, Montag-Torardi & Whitaker, 1987, p. 238). Fear of computers may negatively impact technology acceptance and use. An individual’s computer anxiety, self efficacy and experience with technology may share some correlations, but they are not exactly similar individual traits (Ball & Levy, 2008). Computer anxiety is not simply a negative, short-term attitude toward computers that can be reduced by increasing technology experience; it is the users’ individual fear associated with computer use with or without experience (Ball & Levy, 2008). Likewise, individuals’ self efficacy is an individual trait that some individuals might have with or without technology experience. Some individuals are capable to accept and use LMS without a prior experience with technology because they have high level of self efficacy. In the context of e-learning, few studies have investigated the impact of instructors’ dimensions on LMS acceptance. Ball and Levy (2008) investigated the impact of self-efficacy, computer anxiety, and technology experience on instructors’ intention to use emerging learning experience in a small private university in the US and found that self-efficacy was the only major determinant of instructors’ intention. Teo (2009) found that computer self-efficacy directly impacts pre-service

120

teachers’ perceived usefulness, perceived ease of use, and behavioral intention in Singapore. Liaw et al. (2007) found that perceived self-efficacy determines instructors’ behavioral intention to use e-learning in Taiwan. Albirini (2006) investigated the perception of school teachers of the use of ICT in education in Syria, and the results highlighted the importance of teachers’ vision of technology, their experiences with it, and the cultural conditions on their attitudes toward technology. Mahdizadeh and his colleagues (2008) found that teachers’ previous experience with e-learning environments and ease of use explain teachers’ perception of the usefulness of e-learning environments and their actual use of these environments.

LMS Characteristics The characteristics of LMS may have a great impact on the instructor’s acceptance and use of LMS. Characteristics of any information system, including LMS, may be related to system, information, and service support quality as classified by DeLone and McLean (2003). E-learning systems’ quality was found to be significant on the instructors’ perceived usefulness, perceived enjoyment, and perceived self-efficacy, which consequently affect their intention to use the system in the classroom (Liaw et al., 2007). System quality, which refers to the characteristics of a system, is a key in users’ acceptance and use of any technology, including LMS. Researchers, such as Bailey and Pearson (1983), DeLone and McLean (2003), and Seddon (1997) have highlighted the impact of system quality on technology acceptance and introduced several ways to measure system quality. Information quality, which refers to the perceived output produced by the system, is also an important factor in instructors’ acceptance and use of LMS. Information quality refers to the accuracy, relevance, timeliness, sufficiency, completeness, understandability, format, and accessibility of the information (Bailey


& Pearson, 1983; Seddon, 1997). In addition, service quality may be a factor on the instructors’ acceptance and use of LMS; it refers to the quality of support services provided to the system’s endusers. DeLone and McLean (2003) indicated that ignoring service quality endangers IS effectiveness measurements. Common measurements of service quality are tangibles, reliability, responsiveness, assurance, and empathy (Parasuraman, Zeithaml, & Berry, 1988; Kettinger & Lee, 1994). In the e-learning context, few studies have examined the general quality of technology or specific dimension. For instance, from instructors’ and learners’ perspective, Liaw et al. (2007) investigated the impact of e-learning systems’ general quality on perceived usefulness, perceived enjoyment, and perceived self-efficacy, which consequently affect their intention to use the system in the classroom, and found it significant. Albirini (2006) indicates that instructors’ vision of technology impacts their attitudes toward the use of ICT in education. Two significant studies on the impact of technology on users’ acceptance of LMS are Pituch and Lee’s (2006) and Roca, Chiu and Martinez’s (2006), but they are from the learners’ perspective. Roca et al. (2006) investigated learners’ perceived system quality from three dimensions (system quality, information quality, and service quality). They found that learners’ perceived system quality factors (system quality, information quality, and service quality) directly affect their e-learning satisfaction and intention to use and indirectly their perceived usefulness. Pituch and Lee (2006) examined the impact of system quality from three dimensions: the system’s functionality, interactivity, and response. As indicated, limited studies provide a detailed examination of the influence of the three dimensions (system quality, information quality, service quality) of LMS on instructors’ acceptance. This study integrates these three dimensions of LMS on the instructors’ acceptance.

Organization Characteristics An organization’s characteristics play a major role in the behaviors of its employees, including the acceptance and use of any technology such as LMS. Corporate culture plays a key role in the success of any project. Schein defines culture as “the way we do things around here” (1985, p. 12). Cultural values shape an organization’s norms and practices, which consequently influence employees’ behaviors such as LMS utilization. Some of an organization’s characteristics that might be relevant to the utilization of LMS are management support, incentives, and training. Organizational support, represented by senior managers’ support, is also important for instructors to accept and use LMS in their teaching. Management’s support of end-users significantly improves computer usage (Igbaria, 1990). In the e-learning context, senior management support and the alignment of e-learning with the department and university curriculum are important for its adoption (Sumner & Hostetler, 1999). Motivators are also an important factor for instructors’ acceptance to integrate the technology in teaching. Motivators or incentives for instructors can be enforced by having the use of the technology as a factor in a nomination for teaching award, promotion, and tenure (Sumner & Hostetler, 1999). Finally, training end-users is important, and can be in form of workshops, online tutorials, courses, and seminars. There is a lack of empirical studies that capture the influence of organization factors on the acceptance and use of LMS generally. One of these is a qualitative study by Sumner and Hostetler (1999), who categorize the organizational factors that may influence the use of technology in teaching in terms of motivators/demotivators, training, technology alignment, and organizational and technical support. In addition, Teo (2009) found that facilitating conditions, measured by technical support, training, and administrative support, indirectly affect teachers’ acceptance of technology in education.

121


INSTRUCTORS’ LMS ACCEPTANCE AND USAGE FRAMEWORK Framework Development This study aimed to examine the impact of instructor’s individual characteristics, LMS’ characteristics, and an organization’s characteristics on instructors’ acceptance ad usage of LMS as a supplementary tool and, consequently, on continuous use and pure use intention for distance learning. As indicated, few studies have examined this integrated investigation of instructors’ LMS acceptance and usage. This study assessed the individual characteristics based on instructors’ computer anxiety and technology experience, LMS characteristics based on system, information, and service quality; and organizational characteristics based on management support, incentives and training. The study assessed the impact of these factors on the instructors’ acceptance (perceived ease of use and perceived usefulness) Figure 1. Instructors LMS acceptance and use model

122

and usage of LMS and, consequently, continuous supplementary use and future pure use intention. Figure 1 illustrates this study model.

Instructor Individual Characteristics Hypotheses Computer Anxiety Hypotheses Computer anxiety is “the fear or apprehension felt by individuals when they used computers, or when they considered the possibility of computer utilization” (Simonson, et al., 1987, p. 238). Computer anxiety is an important factor for the acceptance of the technology (Ball & Levy, 2008; Piccoli, Ahmad & Ives, 2001; Raaij & Schepers, 2008; Sun, Tsai, Finger, Chen & Yeh, 2008). Fear of computers may negatively affect the acceptance of LMS and the user’s perceived satisfaction (Piccoli et al., 2001). Empirical evidence of the impact of computer anxiety was mixed. Ball and Levy (2008) did not detect a significant link between


computer anxiety and instructors’ intention to use the e-learning; however, Sun et al.(2008) found that computer anxiety significantly impacts the learners’ perceived satisfaction of e-learning, and Raaij and Schepers (2008) found the computer anxiety impacts the learner’s perceived ease of use of e-learning. Therefore we hypothesized that: Hypothesis 1a: Instructors’ computer anxiety is negatively associated with perceived ease of use of LMS. Hypothesis 1b: Instructors’ computer anxiety is negatively associated with perceived usefulness of LMS. Hypothesis 1c: Instructors’ computer anxiety is negatively associated with use of LMS.

Technology Experience Hypotheses Users’ experience with the technology (EUT) also plays a major role in the acceptance of technology (Venkatesh & Davis, 2000; Thompson et al., 2006). An individual’s EUT is his/her exposure to the technology as well as the skills and abilities that are gained through using a technology (Thompson et al., 2006). Therefore, EUT may impact instructors’ acceptance of LMS for their classes. Although empirical quantitative research, such as that of Ball and Levy (2008), found no significant impact of EUT on instructors’ intention to use LMS, researchers Sumner and Hostetler (1999) indicate that current level of computer skills and extent of use of computing skills in teaching are important for instructors’ acceptance of ICT in education. Likewise, Wan, Fang and Neufeld (2007) highlight the importance of technology experience on the learning processes and, consequently, learning outcomes. Mahdizadeh et al. (2008) reveal that instructors’ prior experience with e-learning may explain their perception of the usefulness of e-learning environments and their actual use. Therefore we hypothesized:

Hypothesis 2a: The instructor’s experience with the use of technology is positively associated with perceived ease of use LMS. Hypothesis 2b: The instructor’s experience with the use of technology is positively associated with perceived usefulness of LMS. Hypothesis 2c: The instructor’s experience with the use of technology is positively associated with use of LMS.

Self-Efficacy Hypotheses Self-efficacy is “people’s judgments of their capabilities to organize and execute courses of action required to attain designated types of performances” (Bandura, 1977, p.391). Thus, computer self-efficacy means individuals’ selfassessment of their ability to apply computer skills to accomplish their tasks (Compeau & Higgins, 1995). Several empirical studies found significant effects of computer self-efficacy on the perceived usefulness on information systems (Venkatesh & Davis, 2000; Chau, 2001). The more ability the instructor has to apply a computer application, the most likely s/he perceives it easy to use and useful, and will eventually use it. In the context of e-learning systems, Ball and Levy (2008) found a significant effect of self-efficacy on instructors’ intention to use. In addition, computer self-efficacy was found to be significant on learners’ perceived ease of use (Lee, 2006; Pituch & Lee, 2006; Roca et al., 2006) and learners’ perceived satisfaction (Sun et al., 2008). Therefore, we hypothesized: Hypothesis 3a: An instructor’s computer selfefficacy is positively associated with perceived ease of use of LMS. Hypothesis 3b: An instructor’s computer selfefficacy is positively associated with perceived usefulness of LMS. Hypothesis 3c: An instructor’s computer selfefficacy is positively associated with use of LMS.

123


LMS Characteristics Hypotheses System Quality Hypotheses System quality is essential for the acceptance of any technology, including LMS. Researchers, such as Bailey and Pearson (1983), DeLone and McLean (1992), and Seddon (1997) highlight the impact of system quality on technology acceptance and have introduced several ways to measure it. Instructors’ acceptance of LMS may be determined to a great extent by system quality. The more functionalities, interactivity, and response of LMS, the better is its acceptance and utilization (Pituch & Lee, 2006). Quantitative empirical studies found a significant impact of system characteristics on e-learning acceptance: reliability (Wan et al., 2007; Webster & Hackley, 1997); accessibility (Wan et al., 2007); and system functionality, interactivity, and response (Pituch & Lee, 2006). Albirini (2006) indicates that instructors’ vision of technology impacts their attitudes toward the use of ICT in education. Therefore, we hypothesized that: Hypothesis 4a: LMS system quality is positively associated with the instructor’s perceived ease of use LMS. Hypothesis 4b: LMS system quality is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 4c: LMS system quality is positively associated with the instructor’s use of LMS.

Information Quality Hypotheses Information quality is also important for instructors’ acceptance of LMS, and refers to the perceived output produced by the system. Information quality with great accuracy, relevance, timeliness, sufficiency, completeness, understandability, format, and accessibility are important for the acceptance of an information technology (Bailey & Pearson, 1983; Seddon, 1997). There is a lack

124

of research on the impact of information quality on instructors’ acceptance of LMS. Some research was conducted from the learners’ perspective. Roca et al. (2006) measured information quality of LMS by indicators related to relevance, timeliness, sufficiency, accuracy, clarity, and format, and proved that information quality was directly significant for learners’ satisfaction and indirectly for perceived usefulness. Likewise, Lee (2006) found content quality was significant for learners’ perceived usefulness. In the Middle East, Al-Busaidi (2009), in an exploratory study in Oman, indicated that information quality (sufficiency, accuracy, relevance, timeliness, and understandability) was highlighted as a determinant of learners’ LMS use. Consequently, we hypothesize that: Hypothesis 5a: LMS information quality is positively associated with the instructor’s perceived ease of use of LMS. Hypothesis 5b: LMS information quality is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 5c: LMS information quality is positively associated with the instructor’s use of LMS.

Service Quality Hypotheses Service quality refers to the quality of support services provided to the system’s end-users. Instructors’ acceptance of LMS may, to some extent, be related to the quality of the support services. Common measurements of service quality are tangibles, reliability, responsiveness, assurance, and empathy (Parasuraman et al., 1988; Kettinger & Lee, 1994). Few studies have investigated the impact of service quality on LMS acceptance. For instance, Roca et al. (2006) assessed service quality by indicators related to responsiveness, reliability, and empathy, and confirmed its direct significance on learners’ satisfaction and indirect significance of perceived usefulness in the e-learning context. Thus, we hypothesized that:


Hypothesis 6a: LMS service quality is positively associated with the instructor’s perceived ease of use of LMS. Hypothesis 6b: LMS service quality is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 6c: LMS service quality is positively associated with the instructor’s use of LMS.

Hypothesis 7a: Management support is positively associated with the instructor’s perceived ease of use of LMS. Hypothesis 7b: Management support is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 7c: Management support is positively associated with the instructor’s use of LMS.

Organization Characteristics Hypotheses

Incentives Hypotheses

Management Support Hypotheses Management support is a key factor for the acceptance of any organizational initiative. Senior managers’ open approval and endorsement of LMS adoption promote instructors’ adoption and acceptance of LMS. Managers may support an LMS by encouraging instructors to adopt it and identify a clear vision of the objective of the LMS and how it is aligned with the university vision. Little research has investigated the impact of management support on instructors’ acceptance of LMS. However, in the e-learning context, senior managers should clearly identify the goal of LMS for the university curriculum (Sumner & Hostetler, 1999). This managers’ support assures instructors that using LMS is part of the organization’s culture and is useful and encourages them to adopt and use the system. Managers are recognized as a high authority (Ali, 1990); thus, instructors’ adoption and acceptance of LMS may be associated with the endorsement of their senior managers. Management support of end-users significantly improves computer usage (Igbaria, 1990). Facilitating conditions, including administrative support, indirectly affect teachers’ acceptance of technology in education (Teo, 2009). Consequently, we hypothesized that:

Motivators, in terms of incentives, are important factors for instructors’ acceptance to integrate LMS in their teaching. Incentives can be “nontrivial” monetary and non-monetary incentives. E-learning research lacks the assessment of incentives on LMS acceptance. Motivators or incentives for instructors can be enforced by using the technology as a factor in nomination for a teaching award, promotion, and tenure (Sumner & Hostetler, 1999). These incentives’ policies push instructors to adopt and utilize LMS for their teaching. Therefore, we hypothesized that: Hypothesis 8a: An incentive policy is positively associated with the instructor’s perceived ease of use of LMS. Hypothesis 8b: An incentive policy is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 8c: An incentive policy is positively associated with the instructor’s use of LMS.

Training Hypotheses Providing end-users with training is important, as training improves instructors’ adoption of LMS and enhances the perceived ease of use of LMS, illustrates its potential usefulness, and encourages its use in teaching. Limited research has investigated the impact of training on instructors’ acceptance of LMS, which can be in the form of workshops, online tutorials, courses, and seminars (Sumner &

125


Hostetler, 1999). Facilitating conditions, including training, indirectly affect teachers’ acceptance of technology in education (Teo, 2009). Thus, we hypothesized: Hypothesis 9a: Training is positively associated with the instructor’s perceived ease of use of LMS. Hypothesis 9b: Training is positively associated with the instructor’s perceived usefulness of LMS. Hypothesis 9c: Training is positively associated with the instructor’s use of LMS.

Usage and Future Intention Hypotheses Perception and Usage Hypotheses A technology’s perceived ease of use and perceived usefulness are found to be a significant determinant of the intention to use the technology (Venkatesh & Davis, 2000). The higher the instructors’ perceived ease of use and perceived usefulness of LMS, the higher the actual use. Perceived ease of use is also a significant determinant of users’ perceived usefulness of a technology (Venkatesh & Davis, 2000). Pituch and Lee (2006) found learners’ perceived ease of use of LMS to significantly affect perceived usefulness. Therefore, we hypothesized: Hypothesis 10a: Instructors’ perceived ease of use of LMS is positively associated with their use of LMS. Hypothesis 10b: Instructors’ perceived usefulness of LMS is positively associated with their use of LMS. Hypothesis 10c: Instructors’ perceived ease of use of LMS is positively associated with their perceived usefulness of LMS.

126

Continuous Supplementary Use Intention Hypotheses The intention to use the technology is significantly determined by users’ perceived ease of use and perceived usefulness (Venkatesh & Davis, 2000). The higher the instructors’ perceived ease of use of LMS, perceived usefulness of LMS, and actual use, the more likely it is that they will continue to use it. Continuous intention to e-learning use is determined by perceived usefulness and satisfaction (Hayashi, Chen, Ryan, Wu, 2004). Thus, we hypothesized: Hypothesis 11a: The instructors’ perceived ease of use of LMS is positively associated with their continuous supplementary use intention. Hypothesis 11b: The instructors’ perceived usefulness of LMS is positively associated with their continuous supplementary use intention. Hypothesis 11c: The instructors’ supplementary use of LMS is positively associated with their continuous supplementary use intention.

Future Pure Use Intention Hypotheses Many organizations begin their LMS adoption as a supplementary tool to traditional teaching, hoping that this supplementary adoption will eventually promote the pure use of LMS for distance education. Perceived ease of use, perceived usefulness, and actual use may have an important impact on continuous intention for supplementary use and intention for pure use of the LMS for education. When instructors believe that LMS is easy, useful, and can be utilized for supplementary purposes, they are more likely to adopt it purely for distance education. Technology perceived ease of use and perceived usefulness are found to be significant determinants of the intention to use the technology (Venkatesh & Davis, 2000). Perceived ease of use, perceived usefulness, and supplementary


use are significant determinants of learners’ use of e-learning for distance education (Pituch & Lee, 2006). Thus we hypothesized: Hypothesis 12a: The instructors’ perceived ease of use of LMS is positively associated with their pure use intention. Hypothesis 12b: The instructors’ perceived usefulness of LMS is positively associated with their pure use intention. Hypothesis 12c: The instructors’ supplementary use of LMS is positively associated with their pure use intention.

METHODOLOGY Participants’ Profile This study included 82 instructors from Sultan Qaboos University (SQU), the first and only public university in Oman. Since its launch, SQU has gone through many technology developments: it adopted WebCT and later switched to the open-source Moodle application. Instructors can voluntarily adopt LMS to supplement their traditional classes. The instructors were from different colleges in the university and with different demographics. About 62 percent were male and 38 percent were female. About 5 percent were assistant lecturers, 27 percent were lecturers, 50 percent were assistant professors, 13 percent were associate professors, and 5 percent were full professors. The instructors’ age varied from 20s to above 50s: about 8 percent were in their 20s, 26 percent were in their 30s, 16 percent in their 40s, and 32 percent were 50 or over. Almost 44 percent had less than six years of work experience, 30 percent had less than 11 years, 16 percent had less than 16 years, 7 percent had less than 21 years, and 2 percent had more than 20 years. Most indicated that their computer skills were above average. Almost 71 percent have above average computer skills; 23

percent, about average; and only 6 percent were below average. The majority, about 59 percent, has used the LMS for classes for three years or more; 30 percent have used it for one to two years; and 11 percent have used it for less than one year. The majority of the instructors, about 55 percent, have experience with only Moodle LMS; 9 percent have experience with only WebCT LMS; 31 percent have experience with both WebCT and Moodle; and 5 percent have experience with Blackboard.

Research Questionnaire The questionnaire was distributed to SQU instructors. An invitation email was sent to instructors to complete the study questionnaire either online or on an attached MS Word document. A reminder was sent two weeks after the initial invitation. Most of the instructors filled the questionnaire online (about 95 percent of them).Only five percent of instructors completed the questionnaire as a hard copy. The questionnaire included the constructs to be measured for quantitative analysis, along with demographic questions (e.g., gender, age, degree, LMS usage experience, work experience, and job title). Construct measurements items were phrased according to a five–point Likert scale (1= strongly disagree; 2=disagree; 3=Neutral; 4= agree and 5=strongly agree). To statistically evaluate the study framework, 28 indicators were used. Tables 1 and 2 show the total indicators used for each construct. The LMS characteristic constructs (system quality, information quality, and service quality) were adopted and modified from Roca et al. (2006) and Pituch and Lee (2006); the individual characteristics constructs (computer anxiety, self-efficacy, and technology experience) were adopted from Ball and Levy (2008); while the organizational characteristics’ constructs (management support, incentives, and training) were self-developed, based on Sumner and Hostetler (1999). The LMS acceptance construct (perceived ease of use and perceived

127


Table 1. Independent constructs measures and loadings Construct Measures

Loading

Computer Anxiety 1. I believe that working with computers is very difficult.

0.8913

2. Computers make me feel uncomfortable.

0.9560

3. I get a sinking feeling when I think of trying to use a computer.

0.8698

Technology Experience 1. I feel confident using the e-learning system

0.8465

2. I feel confident downloading/uploading necessary materials from the Internet.

0.8756

3. I feel confident using the chatting and discussion forums.

0.6275

Self Efficacy 1. I could use the e-learning system if I had never used a system like it before.

-0.7948

2. I could use the e-learning system if I had only the system manuals for reference.

0.0894

3. I could use the e-learning system if I had seen someone else using it before trying it myself.

0.8652

System Quality 1. The system offers flexibility in teaching as to time and place.

0.7694

2. The system offers multimedia (audio, video, and text) types of course content.

0.7962

3. The response time of the system is reasonable.

0.6025

4. The system enables interactive communication between instructor and students.

0.7737

Information Quality 1. The information provided by the system is relevant for my job.

0.8434

2. The information in the system is very good.

0.8919

3. The information from the e-learning system is up-to-date.

0.8407

4. The information provided by the system is complete.

0.8494

Service Quality 1. The system support services give me prompt service.

0.8425

2. The system support services have convenient operating hours.

0.8410

3. The system support services are reliable.

0.8761

4. The system support services are easy to communicate with.

0.8904

Management Support 1. Senior administrators strongly support the use of e-learning system.

0.9396

2. I get support by department chair or dean on my use of e-learning system.

0.8506

3. My mangers highlight the importance of e-learning system on my curriculum.

0.9188

Incentives 1. The use of e-learning is a factor in the nomination for teaching award.

0.9529

2. The use of e-learning system is a factor in determining promotion.

0.9558

3. The use of e-learning system is a factor in annual elevation of teaching.

0.9611

Training

128

1. I receive training workshops on how to use e-learning tools.

0.7277

2. I receive on-line manuals on how to use e-learning tools.

0.8759

3. I receive seminars on the use of e-learning tools.

0.8378


usefulness) were adopted and modified from Venkatesh and Davis (2000), and supplementary use, continuous supplementary use, and future pure use were adopted and modified according to Pituch and Lee (2006).

DATA ANALYSIS & RESULTS PLS Analysis Methodology Data was analyzed by PLS-Graph 3.0 software. PLS (partial least square) is a variance-based structural equation model (SEM) technique that allows path analysis of models with latent variables

(Chin, 1998; Chin, 2001). The PLS approach is a variance-based SEM that assists researchers in obtaining determinate values of latent variables for predictive purposes. The PLS does that by minimizing the variance of all dependent variables rather than using the model to explain the co-variation of all indicators (Chin, 1998; Chin and Newsted, 1999). Thus, the model paths are estimated based on the ability to minimize the residual variances of the dependent variables. The PLS algorithm uses an iterative process for the estimation of weights and latent variables scores. The process almost converges to a stable set of weight estimates. The evaluation of the model is based on (1) the assessment of the model mea-

Table 2. Dependant constructs measures and loadings Construct Measures

Loading

Perceived Ease of Use 1. Using e-learning tools is easy to me.

0.8896

2. E-learning tools are clear and understandable to me.

0.9280

3. I find it easy to get the e-learning system to do what I want it to do.

0.7843

Perceived Usefulness 1. Using e-learning system enables me to accomplish tasks more quickly.

0.8581

2. Using e-learning system improves my performance.

0.9251

3. Using e-learning system increases my productivity.

0.9255

4. Using e-learning system enhances the effectiveness on the job.

0.8958

5. Using e-learning system gives me greater control over my work.

0.8831

Supplementary Use 1. I use the e-learning system as many occasions as possible to supplement my teaching.

0.9165

2. I use the e-learning system on regular basis to supplement my teaching.

0.9047

3. I frequently use the e-learning system to supplement my teaching.

0.9051

4. I use the e-learning system to share/seek course information.

0.7357

5. I use the e-learning system to communicate with students

0.8259

Continuous supplementary Use Intention (CUI) 1. I will frequently use e-learning system to do a teaching task.

0.8871

2. I will use e-learning system on regular basis to supplement my classes in the future.

0.8523

3. I will always try to use the e-learning system to do a teaching task whenever it has a useful feature.

0.8915

Pure Use Intention (PUI) 1. I plan to teach purely online courses for distance learners.

0.9249

2. I will use e-learning system to teach purely online courses.

0.9627

3. I plan to teach purely online courses in as many occasions as possible.

0.9430

129


surements by assessing their validity, reliability, and discriminant validity, (2) the analysis of the paths of the structural model (Chin, 1998). Table 1 and Table 2 show the independent and dependant constructs’ measures and loading respectively.

Constructs Validity and Reliability The reliability and the validity are two criteria used by researchers to evaluate the applicability of their measurements to their investigated model. Reliability refers to the consistency of the measures (indicators) of a specific latent variable; whereas, validity refers to how well the concept is defined by the measures (Hair, Anderson, Tatham & Black, 1998; Crano & Brewer, 2002). With PLS, the reliability of the measurements was evaluated by internal consistency reliability, and the validity was measured by the average variance extracted (AVE), which refers to the amount of variance a latent variable captures from its indicators. AVE was developed by Fornell and Larcker (1981) to assess construct validity. The recommended level for internal consistency reliability is at least 0.70, and is at least 0.50 for AVE (Chin, 1998). Tables 1 and

2 show the model constructs’ measurements and loading. Table 3 shows that the study constructs’ reliability and AVE are above the recommended levels for all the constructs except self-efficacy. Therefore, the self-efficacy construct was dropped from the model evaluation. To achieve the discriminant validity of the constructs, Fornell and Larcker (1981) suggest that the square root of AVE of each construct should exceed the correlations shared between the constructs and other constructs in the model. The discriminant validity is used to ensure the differences among constructs (Barclay, Higgins & Thompson, 1995; Chin, 1998). Table 4 shows that the model constructs satisfy that rule, as the square root of the AVE (on the diagonal) is greater than the correlations with other constructs. Thus, all the model constructs have a satisfactory discriminant validity construct.

Model Evaluation and Paths Analysis With PLS, R-square values are used to evaluate the predictive relevance of a structural model for the dependent latent variables, and the path

Table 3. Constructs reliability and validity Construct

130

Total Items

Reliability

AVE

Computer Anxiety (CA)

3

0.932

0.822

Technology Experience (TE)

3

0.831

0.626

Self Efficacy (SE)

3

0.504

0.267

System Quality (SQ)

4

0.827

0.547

Information Quality (IQ)

4

0.917

0.734

Service Quality (SvQ)

4

0.921

0.744

Management Support (MS)

3

0.930

0.817

Incentives (IN)

3

0.970

0.915

Training (TR)

3

0.856

0.666

Perceived Ease of Use (PEU)

3

0.902

0.756

Perceived Usefulness (PU)

5

0.954

0.806

Supplementary Use (USE)

5

0.934

0.740

Continuous Supplementary Use Intention (CUI)

3

0.909

0.769

Pure Use Intention (PUI)

3

0.961

0.890


coefficients are used to assess the effects of the independent variables(Chin, 1998). The significance of the model paths was checked based on their t-values. Table 5 shows the R2 values of the endogenous dependent constructs. The model explains 58 percent of variance in instructors’ usage of LMS; 52.2 percent of variance in instructors’ perceived usefulness of LMS; and 42.4 percent of variance in instructors’ perceived ease of use of LMS. In addition, instructors’ usage of LMS, their perceived usefulness of LMS, and their perceived ease of use of LMS explains 55.3 percent of variance in their continuous supplementary use intention, but only 22.9 percent of their pure use intention of LMS. Table 5 also shows the paths’ coefficients

analysis between the exogenous independent constructs (instructors’ characteristics, LMS’s characteristics, and organization’s characteristics) and the endogenous dependent construct (LMS supplementary use, LMS perceived ease of use, LMS perceived usefulness), and, consequently, future intention (continuous supplementary use intention and pure use intention). The statistical significant of the paths’ coefficients was measured by T-values with at least 95 percent confidence level. The analysis showed that the instructor’s characteristics, the LMS’s characteristics and the organization’s characteristics to some extent have impact on the instructor’s acceptance and use of LMS; see Table 5. First, instructors’ computer

Table 4. Construct’ correlations and discriminant validity Construct

CA

TE

SQ

IQ

SvQ

MS

IN

TR

PEU

PU

USE

CUI

Computer Anxiety (CA)

0.907

Technology Experience (TE)

-0.124

0.791

System Quality (SQ)

-0.072

0.202

0.740

Information Quality (IQ)

-0.069

0.188

0.675

0.857

Service Quality (SvQ)

-0.024

0.076

0.407

0.689

0.863

Management Support(MS)

0.175

0.013

0.257

0.207

0.206

0.904

Incentives (IN)

0.224

-0.196

0.144

0.133

0.140

0.447

0.957

Training (TR)

0.024

-0.001

0.235

0.343

0.347

0.220

0.283

0.816

Perceived Ease of Use (PEU)

-0.362

0.313

0.418

0.436

0.377

0.182

0.042

0.314

0.869

Perceived Usefulness (PU)

-0.175

0.230

0.614

0.381

0.176

0.223

0.175

0.204

0.546

0.898

Supplementary Use (USE)

-0.178

0.168

0.469

0.370

0.146

0.379

0.223

0.436

0.531

0.614

0.960

Continuous supplementary Use Intention (CUI)

-0.328

0.347

0.507

0.353

0.194

0.195

0.167

0.358

0.583

0.647

0.645

0.877

Pure Use Intention (PUI)

0.005

0.091

0.147

0.058

-0.024

0.058

0.094

0.148

0.324

0.467

0.351

0.431

PUI

0.943

131


Table 5. Model Evaluation & Paths Analysis Construct

CA

TE

SQ

IQ

SvQ

PEU (0.424)

-0.340 (h1a)

0.212 (h2a)

0.173 (h4a)

C

0.068 (h5a)

0.150 (h6a)

0.106 (h7a)

0.006 (h8a)

0.182 (h9a)

(0.522) (PU)

-0.033 (h1b)

0.029 (h2b)

0.508G (h4b)

0.056 (h5b)

0.008 (h6b)

0.014 (h7b)

0.104B (h8b)

0.002 (h9b)

0.300G (h10c)

USE (0.580)

-0.088A (h1c)

0.003 (h2c)

0.051 (h4c)

0.115 B (h5c)

0.012 (h6c)

0.203G (h7c)

0.001 (h8c)

0.238G (h9c)

0.139E (h10a)

0.324G (h10b)

CUI (0.553)

0.238F (h11a)

0.320 F (h11b)

0.322G (h11c)

PUI (0.229)

0.077 (h12a)

0.376G (h12b)

0.080 (h12c)

G

E

C

MS

IN

TR

PEU

PU

USE

D

A. Significant at 95% |B. Significant at97.5% |C. Significant at 99% |D. Significant at99.5% E. Significant at 99.75% | F. Significant at 99.9% | G: Significant at99.95%

anxiety negatively impacts their perceived ease of use (Beta -β = -0.340, p= 0.0005, hypothesis h1a) and supplementary use of LMS (β = -0.088, p= 0.05; h1c). Second, instructors’ experience with the technology only significantly impacts their perceived ease of use (β = 0.212, p = 0.0025; h2a). Third, the study was unable to assess the impact of instructors’ self efficacy on their use and acceptance of LMS because of the low reliability and validity of the construct (self-efficacy) measures; thus, hypotheses H3a, H3b, H3c were not tested. Fourth, system’s quality significantly impacts instructors’ perceived ease of use (β = 0.173, p = 0.01; h4a); and perceived usefulness (β = 0.508, p= 0.0005; h4b) of LMS. Fifth, information quality significantly impacts instructor’s actual use of LMS (β = 0.149, p= 0.025; h5c). Sixth, service quality significantly impacts only instructors’ perceived ease of use of LMS (β = 0.150, p = 0.01; h6a). Seventh, management support significantly impacts instructor’s actual use of LMS (β = 0.203, p=0.0005; h7c). Eight, incentives policy significantly impacts instructors’ perceived usefulness of LMS (β = 0.104, p= 0.025; h8b). Ninth, training significantly impacts the instructors’ perceived ease of use (β = 0.182, p = 0.005; h9a) and actual use of LMS (β = 0.238, p= 0.0005; h9c). Tenth, instructors’ perceived ease of use (β = 0.139, p= 0.0025; h10a) and perceived

132

usefulness (β of 0.324, p-value= 0.0005; h10b) of LMS positively associated with their actual use of LMS, and instructors’ perceived ease of use of LMS positively associated with their perceived usefulness of LMS (β = 0.300, p= 0.0005; ; h10c). Eleventh, instructors’ perceived ease of use of LMS (β = 0.238, p = 0.001; h11a), perceived usefulness (β = 0.320, p = 0.001; h11b), and current actual use (β = 0.322, p = 0.0005; h11c) are positively associated with their continuous supplementary use intention. Twelfth, instructors’ perceived usefulness of LMS is only positively associated with their pure use intention of LMS for distance education (β = 0.376, p = 0.0005; h12b). Thus, hypotheses h1b, h2b, h2c, h4c, h5a, h5b, h6b, h6c, h7a, h7b, h8a, h8c, h9b, h12a and h12c were not significant as indicated in Table 5.

DISCUSSION & CONCLUSION Discussion of Findings and Implications LMS include several tools that provide academic and training institutions an efficient and effective means to support distance education and supplement their traditional teaching. Moreover, LMS enable these institutions to capture their


educational materials and preserve them for future reuse. This study examined the impact of instructors’ characteristics (computer anxiety and technology experience); LMS’ characteristics (system quality, information quality, and service quality); and an organization’s characteristics (management support, incentives, and training) on the acceptance of e-learning (supplementary use, perceived usefulness, and perceived ease of use) and, consequently, their future intention of supplementary use and pure use of LMS for distance education in an academic institution. The results showed that instructors’ individual characteristics, LMS’ characteristics, and an organization’s characteristics have various impacts on instructors’ use of LMS, their perceived usefulness of LMS, and their perceived ease of use. First, regarding the instructors’ individual characteristics, the study found that instructors’ computer anxiety impacts their perceived ease of use of LMS and their actual use; whereas instructors’ technology experience impacts their perceived ease of use of LMS. Second, regarding the LMS’s characteristics, the study found system quality impacts instructors’ perceived ease of use and perceived usefulness of LMS, information quality impacts instructors’ actual use of LMS, whereas, service quality impacts instructors’ perceived ease of use of LMS. Third, regarding the organization’s characteristics, the study found that management support impacts instructors’ actual use of LMS, incentives policy impacts the instructors’ perceived usefulness of LMS, whereas training impacts instructors’ perceived ease of use of LMS. In other words, instructors’ perceived ease of use of LMS is determined by instructors’ characteristics (computer anxiety and technology experience) and an organization’s characteristics of training, and LMS’ system and service quality. Instructors’ perceived usefulness of LMS is impacted by system quality, its perceived ease of use, and incentives policy; thus, none of the instructors’ individual characteristics has a direct significant impact on their perceived usefulness

of LMS. Instructors’ use of LMS is impacted by a combination of several factors: their perception of LMS (perceived usefulness of LMS and perceived ease of use), organizational characteristics (training and management support), LMS characteristics (information quality), and instructors’ individual characteristic (computer anxiety). Other factors impact system use indirectly through perceived usefulness and perceived ease of use. Furthermore, the results indicated that instructors’ continuous intention of LMS supplementary use is determined by their current use, perceived usefulness, and perceived ease of use, while future intention of LMS pure use for distance education is determined only by their perceived usefulness of LMS. LMS is promising for developing countries, as they provide tools to efficiently build human resources. This study offered significant findings for researchers and practitioners. First, it comprehensively examined the critical factors influencing instructors’ use of LMS. The study demonstrates that the use of LMS is determined by users’ individual characteristics, LMS characteristics, and an organization’s characteristics. Thus, users, technology, and an organization’s characteristics have various and significant impacts on instructors’ acceptance of LMS. Few studies have examined instructors’ acceptance of LMS; the literature also lacks the investigation of organizational characteristics on LMS acceptance. Second, the study shows that investigating these direct impacts on LMS use is worthwhile. Third, the study shows that although instructors currently use LMS, the perceived ease of use and perceived usefulness of LMS impact their continuous intention to use LMS for supplementary purposes; only their perceived usefulness of LMS impacts their intention to use LMS purely for distance education. Few studies have examined the link between instructors’ use of LMS as a supplementary and a pure tool for distance education. The study also provided useful insights for practitioners (instructors and academic institutions). Organizations, especially in the Middle

133


East where computer and Internet literacy is not as high as in developed countries, should provide training to improve users’ technology experience and lessen their computer anxiety. In addition, organizations should adopt high-quality LMS (in terms of system quality, information quality, and service quality) to promote their adoption and use by instructors. In addition, management support and incentives are important to improve instructors’ use of LMS and their perceived usefulness, respectively. The findings suggest that instructors’ computer anxiety plays a significant negative role on their perceived ease of use and actual use of LMS, therefore, providing a good training and service quality are necessary to improve instructor’s perceived ease of use of LMS. Also, organizations should ensure that management support is in place to encourage instructor’s use of LMS. Furthermore, since instructors’ intention to pure e-learning depends on their perceived usefulness of LMS, ensuring high system quality and attractive incentives policies, as they are determinants of perceived usefulness, are vital for pursuing a pure e-learning for distance education.

Limitations and Future Research This study has few limitations. First, the sample was from one academic institution in Oman and one learning management system; more research can be conducted in several organizations using different LMS in different countries to improve the generalization of the findings. Second, the study assessed LMS usage from instructors’ perspective; further research may assess learners’ and organizations’ acceptance. Third, this study was unable to assess the impact of self-efficacy; new measurements might be developed to improve its reliability and validity across different countries. Moreover, future research could also examine in detail the benefits of LMS for instructors and the critical factors influencing organizations’ deployment of LMS.

134

ACKNOWLEDGMENT This study was granted by Sultan Qaboos University in Oman

REFERENCES Aczel, J. C., Peake, S. R., & Hardy, P. (2008). Designing capacity-building in e-learning expertise: Challenges and Strategies. Computers & Education, 50, 499–510. doi:10.1016/j. compedu.2007.07.005 Al-Busaidi, K. A. (2009). Evaluating the use of e-learning system as a supplementary tool at an academic institution. International Journal of Internet Education, 4, 153. Albirini, A. (2006). Teachers’ attitudes toward information and communication technologies: The case of Syrian EFL teachers. Computers & Education, 47, 373–398. doi:10.1016/j.compedu.2004.10.013 Ali, A. J. (1990). Management theory in a transitional society: The Arab’s experience. International Studies of Management and Organization, 20(3), 7–35. Bailey, J., & Pearson, S. (1983). Development of a tool for measuring and analyzing computer user satisfaction. Management Science, 29(5), 530–545. doi:10.1287/mnsc.29.5.530 Ball, D., & Levy, Y. (2008). Emerging educational technology: Assessing the factors that influence instructors’ acceptance in information systems and other classrooms. Journal of Information Systems Education, 19(4), 431–443. Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioural change. Psychological Review, 84(2), 191–215. doi:10.1037/0033295X.84.2.191


Barclay, D., Higgins, C., & Thompson, R. (1995). The partial least squares (PLS) approach to casual modelling: Personal computer adoption and use as an illustration (with commentaries). Technology Studies, 2(2), 285–309.

Chin, W. (1998). The partial least square approach to structural equation modelling. In Marcoulides, G. A. (Ed.), Modern Methods for Business Research (pp. 295–336). London, UK: Lawrence Erlbaum Associates.

Beatty, B., & Ulasewicz, C. (2006). Online teaching and learning transition: Faculty perspectives on moving from blackboard to the Moodle learning management system. TechTrends, 50(4), 36–45. doi:10.1007/s11528-006-0036-y

Chin, W. (2001). PLS Graph user’s guide, version 3.0. Houston, TX: Bauer College of Business, University of Houston.

Blackboard. (2009). Engaging learners for engaging learning. Retrieved on September 7th, 2009, from http://www.blackboard.com/TeachingLearning/Learn-Platform.aspx. Browne, T., Jenkins, M., & Walker, R. (2006). A longitudinal perspective regarding the use of VLEs by higher education institutions in the United Kingdom. Interactive Learning Environments, 14(2), 177–192. doi:10.1080/10494820600852795 Burnett, R., & Marshall, D. P. (2003). Web theory: An introduction. New York, NY: Routledge Press.

Chin, W., & Newsted, P. R. (1999). Structural equation modeling analysis with small samples using partial least squares. In Hoyle, R. H. (Ed.), Statistical strategies for small sample research (pp. 307–341). Thousand Oaks, CA: Sage Publications. Compeau, D., & Higgins, C. (1995). Computer self-efficacy: Development of a measure and initial test. Management Information Systems Quarterly, 19, 189–211. doi:10.2307/249688 Crano, W., & Brewer, M. (2002). Principles and methods of social research. Mahwah, NJ: Lawrence Erlbaum Associates Publishers.

Burniske, R. W., & Monke, L. (2001). Breaking down the digital walls. Albany, NY: State University of New York Press.

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. Management Information Systems Quarterly, 13, 319–339. doi:10.2307/249008

Cavus, N., & Momani, A. M. (2009). Computer aided evaluation of learning management systems. Procedia Social and Behavioral Sciences, 1, 426–430. doi:10.1016/j.sbspro.2009.01.076

DeLone, W., & McLean, E. (2003). The Delone and Mclean model of Information Systems success: A 10-year update. Journal of Management Information Systems, 19(4), 9–30.

Ceraulo, S. (2005). Benefits of upgrading to an LMS. Distance Education Report, 9, 6–7.

DeLone, W. H., & McLean, E. R. (1992). Information Systems success: The quest for the dependent variable. Information Systems Research, 3, 60–95. doi:10.1287/isre.3.1.60

Chan, C. C., Tsui, M-s, Chan, M. Y. C., & Hong, J. H. (2008). A virtual learning environment for part-time MASW students: An evaluation of the WebCT. Journal of Teaching in Social Work, 28, 87–100. doi:10.1080/08841230802179027 Chau, Y., Patrick, K., & Hu, J. (2001). Information Technology acceptance by individual professionals: A model comparison approach. Decision Sciences, 32(4), 699–718. doi:10.1111/j.1540-5915.2001.tb00978.x

Doll & Torkzadeh, G. (1988). The measurement of end user computing satisfaction, IS Quarterly, 12(2), 259-274. El Tartoussi, I. (2009). Networked readiness in the United Arab Emirates, The 2nd Annual Forum on E-learning Excellence in the Middle East 2009, Dubai. UAE.

135


Falvo, D. A., & Johnson, B. F. (2007). The use of learning management systems in the United States. TechTrends, 51(2), 40–45. doi:10.1007/ s11528-007-0025-9

Kneght, M., & Reid, K. (2009). Modularizing Information Literacy training via the Blackboard eCommunity. Journal of Library Administration, 49, 1–9. doi:10.1080/01930820802310502

Fornell, C., & Larcker, D. (1981). Evaluating structural equation models with unobservable variables and measurement error. JMR, Journal of Marketing Research, 18, 39–50. doi:10.2307/3151312

Lee, Y. (2006). An empirical investigation into factors influencing the adoption of an e-learning system. Online Information Review, 30(5), 517– 541. doi:10.1108/14684520610706406

Hair, J., Anderson, R., Tatham, R., & Black, W. (1998). Multivariate data anlaysis. Upper Saddle River, NJ: Prentice Hall.

Liaw, S., Huang, H., & Chen, G. (2007). Surveying instructor and learner attitudes toward elearning. Computers & Education, 49, 1066–1080. doi:10.1016/j.compedu.2006.01.001

Hawkins, B. L., & Rudy, J. A. (2007). Educause Core Data Service. Fiscal Year 2006 Summary Report. CO, USA: Educause. Hayashi, A., Chen, C., Ryan, T., & Wu, J. (2004). The role of social presence and moderating role of computer self efficacy in predicting the continuance usage of e-learning systems. Journal of Information Systems Education, 15(2), 139–154. Igbaria, M. (1990). End-user computing effectiveness: A structural equation model. Omega, 18(6), 637. doi:10.1016/0305-0483(90)90055-E Igbaria, M., Guimaraes, T., & Davis, G. B. (1995). Testing the determinants of microcomputer usage via a structural equation model. Journal of Management Information Systems, 11(4), 87–114. Kelly, T., & Bauer, D. (2004). Managing intellectual capital-via e-learning-at Cisco. In Holsapple, C. (Ed.), Handbook on Knowledge Management 2: Knowledge directions (pp. 511–532). Berlin, Germany: Springer. Kettinger, W. J., & Lee, C. C. (1994). Perceived service quality and user satisfaction with the Information Services function. Decision Sciences, 25(5/6), 737–765. doi:10.1111/j.1540-5915.1994. tb01868.x

136

Mahdizadeh, H., Biemans, H., & Mulder, M. (2008). Determining factors of the use of e-learning environments by university teachers. Computers & Education, 51, 142–154. doi:10.1016/j. compedu.2007.04.004 Naidu, S. (2006). E-learning a guidebook of principles, procedures and practices, 2nd revised edition, Commonwealth Educational Media Center for Asia (CEMCA), New Delhi, India. National Centre for Educational Statistics. (2003). Distance education at degree-granting postsecondary institutions: 2000-2001 U. Washington, DC: S. Department of Education. Parasuraman, A., Zeithaml, V. A., & Berry, L. (1988). SERVQUAL: A multiple-item scale for measuring customer perceptions of service quality. Journal of Retailing, 64(1), 12–40. Paulsen, M. F. (2003). Experiences with learning management systems in 113 European institutions. Journal of Educational Technology & Society, 6(4), 134–148. Piccoli, G., Ahmad, R., & Ives, B. (2001). Webbased virtual learning environments: A research framework and a preliminary assessment of effectiveness in basic IT skill training. Management Information Systems Quarterly, 25(4), 401–426. doi:10.2307/3250989


Pituch, K., & Lee, Y. (2006). The influence of system characteristics on e-learning use. Computers & Education, 47, 222–244. doi:10.1016/j. compedu.2004.10.007 Raaij, E., & Schepers, J. (2008). The acceptance and use of a virtual learning environment in China. Computers & Education, 50, 838–852. doi:10.1016/j.compedu.2006.09.001 Rainer, R. K., Turban, E., & Potter, R. E. (2007). Introduction to Information Systems: Supporting and transforming business. NJ: Wiley. Reiser, R. A., & Dempsey, J. V. (Eds.). (2002). Trends and issues in instructional design and technology. Upper Saddle River, NJ: Pearson Education. Roca, J., Chiu, C., & Martinex, F. (2006). Understanding e-learning continuous intention: An extension of the technology acceptance model. International Journal of Human-Computer Studies, 64, 683–696. doi:10.1016/j.ijhcs.2006.01.003 Saady, A. (2005). E-learning curve, ITP Technology, Retrieved from www.ITP.net. Schein, E. (1985). Organizational culture and leadership. San Francisco, CA: Jossey-Bass. Seddon, P. B. (1997). A respecification and extension of the DeLone and McLean model of IS success. Information Systems Research, 8(3), 240–253. doi:10.1287/isre.8.3.240 Simonson, M., Maurer, R., Montag-Torardi, M., & Whitaker, M. (1987). Development of a standardized test of computer literacy and a computer anxiety index. Journal of Educational Computing Research, 3(2), 231–247. doi:10.2190/7CHY5CM0-4D00-6JCG Sumner, M., & Hostetler, D. (1999). Factors influencing the adoption of technology in teaching. Journal of Computer Information Systems, 40(1), 81.

Sun, P., Tsai, R., Finger, G., Chen, Y., & Yeh, D. (2008). What drives a successful e-learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers & Education, 50, 1183–1202. doi:10.1016/j. compedu.2006.11.007 Teo, T. (2009). Modelling technology acceptance in education: A study of pre-service teachers. Computers & Education, 52, 302–312. doi:10.1016/j. compedu.2008.08.006 Thompson, R., & Compeau, R., Deborah, & Higgins, C. (2006). Intentions to use Information Technologies: An integrative model. Journal of Organizational and End User Computing, 18(3), 25–47. doi:10.4018/joeuc.2006070102 Venkatesh, V., & Davis, F. (2000). A theoretical extension of the technology acceptance model: Four longitudinal. Management Science, 46(2), 186–204. doi:10.1287/mnsc.46.2.186.11926 Wan, Z., Fang, Y., & Neufeld, H. (2007). The role of Information Technology in technologymediated learning: A review of the past for the future. Journal of Information Systems Education, 18(2), 183–192. Webster, J., & Hackley, P. (1997). Teaching effectiveness in technology-mediated distance learning. Academy of Management Journal, 40(6), 1282. doi:10.2307/257034 Woods, R., Baker, J., & Hopper, D. (2004). Hybrid structure: Faculty use and perception of Webbased courseware as a supplement to face-to-face instruction. The Internet and Higher Education, 7(4), 281–297. doi:10.1016/j.iheduc.2004.09.002 World Bank. (2010). Learning management system. World Bank Institute. Retrieved on April 7th, 2010, from http://web.worldbank.org

137


Yildirim, S., Temur, N., Kocaman, A., & Goktas, Y. (2004). What makes a good LMS: An analytical approach to assessment of LMS. In Proceedings of the 5th International Conference on Information Technology Based Higher Education and Training, Istanbul, Turkey (pp. 125–130). Yueh, H.-P., & Hsu, S. (2008). Designing a learning management system to support instruction. Communications of the ACM, 51(4), 59–63. doi:10.1145/1330311.1330324

ADDITIONAL READING Aggarwal, A., & Bento, R. (2000). Web-based education. In Aggarwal, A. (Ed.), Web-based learning and teaching technologies: Opportunities and challenges (pp. 1–16). London, UK: Idea Group Publishing. Al-Busaidi, K., & Al-Shihi, H. (2010). Instructors’ acceptance of learning management systems: A theoretical framework. Communications of IBIMA [Open Source Journal], Vol. 2010, Article ID 862128, 10 pages. Al-Rawajfih, K., Fong, S., & Idros, S. (2010). Effects of principals’ support on teachers’ selfefficacy in integrating e-learning in the Jordanian Discovery Schools. Modern Applied Science, 4(9), 147–151. Aydin, C. C., & Tirkes, G. (2010). Open source learning management systems in e-learning and Moodle. In Proceedings of Education Engineering (EDUCON), 2010 (pp. 593–600). IEEE. Bailey, C. J., & Card, K. A. (2009). Effective pedagogical practices for online teaching: Perception of experienced instructors. The Internet and Higher Education, 12(3–4), 152–155. doi:10.1016/j. iheduc.2009.08.002

138

Chan, C. C., Tsui, M-s, Chan, M. Y. C., & Hong, J. H. (2008). A virtual learning environment for part-time MASW students: An evaluation of the WebCT. Journal of Teaching in Social Work, 28, 87–100. doi:10.1080/08841230802179027 Garrison, R., & Vaughan, H. (2008). Blended learning in higher education: Framework, principles and guidelines. San Francisco, CA: Jossey-Bass. Ge, X., Lubin, I. A., & Zhang, K. (2010). An investigation of faculty’s perceptions and experiences when transitioning to a new learning management system. Knowledge Management & E-Learning: An International Journal, 2(4), 43–62. Graf, S., Kinshuk, & Liu, T.-C. (2009). Supporting teachers in identifying students’ learning styles in learning management systems: An automatic student modelling approach. Journal of Educational Technology & Society, 12(4), 3–14. Hu, P. J., Chau, P. Y., Clark, T. H., & Ma, W. W. (2003). Examining technology acceptance by school teachers: A longitudinal Study. Information & Management, 41, 227–241. doi:10.1016/ S0378-7206(03)00050-8 Kim, S. W., & Leet, M. G. (2008). Validation of an evaluation model for learning management systems. Journal of Computer Assisted Learning, 24, 284–294. doi:10.1111/j.1365-2729.2007.00260.x Kluever, R. C., Lam, T. C. M., & Hoffman, E. R. (1994). The computer attitude scale: Assessing changes in teachers’ attitudes toward computers. Journal of Educational Computing Research, 11(3), 251–256. doi:10.2190/484T-CPGXEUHG-QW8P Kritikou, Y., Demestichas, P., Adamopoulou, E., Demestichas, K., Theologou, M., & Paradia, M. (2008). User profile modeling in the context of Web-based learning management systems. Journal of Network and Computer Applications, 31, 603–627. doi:10.1016/j.jnca.2007.11.006


Laurillard, D. (2002). Rethinking university teaching: A conversational framework for the effective use of learning technologies (2nd ed.). New York, NY: Routledge/Falmer. doi:10.4324/9780203304846

Russell, M., Bebell, D., O’Dwyer, L., & O’Connor, K. (2003). Examining teacher technology use: Implications for preservice and inservice teacher preparation. Journal of Teacher Education, 54(5), 297–310. doi:10.1177/0022487103255985

Lonn, S., & Teasley, S. D. (2009). Saving time in innovating practice: Investigating perceptions and uses of learning management systems. Computers & Education, 53, 686–694. doi:10.1016/j. compedu.2009.04.008

Selim, H. M. (2007). Critical success factors for e-learning acceptance: Confirmatory factor models. Computers & Education, 49, 396–413. doi:10.1016/j.compedu.2005.09.004

Martin, K., Quigley, M. A., & Rogers, S. (2010). Implementing a learning management system globally: An innovative change management approach. IBM Systems Journal, 44(1), 125–145. doi:10.1147/sj.441.0125 McGill, T., Klobas, J., & Renzi, S. (2011). LMS use and instructor performance: The role of task-technology fit. International Journal on ELearning, 10(1), 43–62. Moore, M. G. (1989). Three types of interaction. American Journal of Distance Education, 3(2), 1–6. doi:10.1080/08923648909526659 Nagar, S. (2010). Study of learning management systems and its effects on distance education. International Journal of Educational Administration, 2(2), 323–327. Ngai, E., Poon, J., & Chan, Y. (2007). Empirical examination of the adoption of WebCT using TAM. Computers & Education, 48(2), 250–267. doi:10.1016/j.compedu.2004.11.007 Ocak, M. (2011). Why are faculty members not teaching blended courses? Insights from faculty members. Computers & Education, 56, 689–699. doi:10.1016/j.compedu.2010.10.011 Ramayah, T., Ahmad, N., & Lo, M. (2010). The role of quality factors in intention to continue using an e-learning system in Malaysia. Procedia - Social and Behavioral Sciences, 2(2) 5422-5426.

Teo, T. (2011). Modeling the determinants of pre-service teachers’ perceived usefulness of e-learning. Campus-Wide Information Systems, 28(2). doi:10.1108/10650741111117824 Toral, S. L., Barrero, F. J., Torres, M. R., Gallardo, S., & Lillo, A. J. L. (2005). Implementation of a Web-based educational tool for digital signal processing teaching using the technological acceptance model. IEEE Transactions on Education, 48(4), 632–641. doi:10.1109/TE.2005.853074 Tucker, A. (2010). Effective practices in e-learning: An online instructor and learner perspective. In J. Sanchez & K. Zhang (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2010 (p. 1251). Volery, T., & Lord, D. (2000). Critical success factors in online education. International Journal of Educational Management, 14(5), 216–223. doi:10.1108/09513540010344731 Waheed, M., & Jam, A. (2010). Teacher’s intention to accept online education: Extended TAM Model. Interdisciplinary Journal of Contemporary Research in Business., 2(5), 330–344. Wang, S.-C. (2010). University instructor perceptions of the benefits of technology use in e-learning. In Proceedings of Computer and Electrical Engineering, (ICCEE ’09). 2nd International Conference (pp. 580-585).

139


Wang, W., & Wang, C. (2009). An empirical study of instructor adoption of Web-based learning systems. Computers & Education, 53(3), 761–774. doi:10.1016/j.compedu.2009.02.021 Zhu, C., Valcke, M., & Schellens, T. (2009). A crosscultural study of online collaborative learning. Multicultural Education & Technology Journal, 3(1), 33–46. doi:10.1108/17504970910951138

KEY TERMS AND DEFINITIONS Learning Management System: A system that provides a platform to support e-learning activities such as communication, collaboration, learning and information/ knowledge transfer.

140

Supplementary LMS Use: The use of learning management system tools to support the traditional face-to-face meetings. Pure LMS Use: The use of learning management system for pure e-learning environment. LMS Success Factors: Factors that determines the success of LMS deployment LMS’s Characteristics: Factors that are related to the system quality, information quality and service support quality of LMS. Instructor’s Characteristics: Factors that are related to the instructor’s individual characteristics such as self efficacy, computer anxiety, technology experience etc. Organization’s Characteristics: Factors that are related to the organization such as management support, training programs and incentives policies.

Section 3

Trends and Challenges

142

Chapter 7

A Comparative Study on LMS Interoperability José Paulo Leal CRACS/INESC-Porto & DCC/FCUP, University of Porto, Portugal Ricardo Queirós CRACS/INESC-Porto & DI/ESEIG/IPP, Porto, Portugal

ABSTRACT A Learning Management System (LMS) plays an important role in any eLearning environment. Still, the LMS cannot afford to be isolated from other systems in an educational institution. Thus, the potential for interoperability is an important, although frequently overlooked, aspect of an LMS system. In this chapter we make a comparative study of the interoperability features of the most relevant LMS in use nowadays. We start by defining a comparison framework, with systems that are representative of the LMS universe, and interoperability facets that are representative of the type integration with other broad classes of eLearning systems. For each interoperability facet we categorize and identify the most representative remote systems, we present a comprehensive survey of existing standards and we illustrate with concrete integration scenarios. Finally, we draw some conclusions on the status of interoperability in LMS based on our study.

INTRODUCTION Interoperability is the ability of different computer systems, applications or services to communicate, share and exchange data, information and knowledge in a precise, effective and consistent way (Martínez & Navarra, 2007). In the eLearning DOI: 10.4018/978-1-60960-884-2.ch007

field this topic is extremely important since there is the need for all systems that typically compose an eLearning environment to communicate and share data consistently. The LMS plays a central role in any eLearning architecture. Choosing an LMS can be a challenging task for an organization. Several studies have been conducted to analyse and evaluate these types of systems from pedagogical and in-


A Comparative Study on LMS Interoperability

stitutional perspectives (Pantel, 1997; Britain & Liber 1998). However, we are not aware of any study to evaluate the interoperability of LMS with other systems typically found in an educational institution. A major issue in LMS interoperability is the eLearning standardization. The concept of course, student, educational resource, summary or grade must be formally described in order to be shared among all the systems in an educational institution. For instance, the difficulty to reuse of a course in schools with LMS from different vendors (or even from the same vendor) is an apt example of the problems found currently in the majority of the LMS. These interoperability issues affect the flexibility of the teaching-learning process and lead to a decrease of end user satisfaction and learning success. In this chapter we make a comparative study of the LMS support for interoperability. This study is part of an effort to select an LMS on which to base the development of eLearning systems integrating heterogeneous components. We chose two LMS vendors - Moodle and Blackboard - since combined they a have a significant share of the LMS market and they follow different approaches to LMS development, namely open source and commercial. We analyse the interoperability features in these LMS split in two facets reflecting the broad classes of systems of a typical LMS operational environment. These broad classes are Learning Content Management Systems and Academic Management Systems. This chapter starts by tracing the evolution of LMS. We proceed with the selection of the systems representative of the LMS universe and of a methodology for comparing them based in interoperability facets. The following two sections analyse separately the learning management content facet and the academic management facet. For each facet we categorize and identify the most representative system, the existing standards and the interoperability issues regarding the communication with the LMS. In the final section we draw conclusions on the results of this study.

LMS EVOLUTION The evolution of eLearning in the last decades has staggering, from the early monolithic systems developed for specific learning domains to new systems featuring reusable tools that can be effectively used virtually in any eLearning course. These types of systems evolved from Content Management Systems (CMS). The CMS was introduced in the mid-1990s mostly by the online publishing industry. This type of system can be defined as a data repository that also includes tools for authoring, aggregating and sequencing content. The main goal of these tools is to simplify the creation and administration of online content (Nichani, 2009). CMS are focused on content with the main purpose to store information and provide access to it. CMS content is organized in small self-contained pieces of information to improve reusability at the content component level. These content components when used in the learning domain are called “learning objects” (LO) and the systems that manage them are called Learning Content Management Systems (LCMS). Nowadays, an LMS plays a central role in any eLearning architecture and can be defined as software application for the administration, documentation, tracking, reporting of training programs, classroom and online events, and training content (Ellis, 2009). Typically it is used by two types of users’ groups: learners and teachers. The learners can use the LMS to plan their learning experience and to collaborate with their colleagues; the teachers can deliver educational content and track, analyze and report the learner evolution within an organization. There are open source systems, such as Moodle, Sakai, .LRN or Dokeos, and commercial systems such as WebCT/ Blackboard or Desire2Learn. They all feature general tools for delivering content and for recreating a learning context. From a course/discipline perspective they provide tools for handling assignments, managing chat rooms and forums, evaluating multiple-choice tests and quizzes, among others. From a learners’ manage143


ment perspective they provide tools for keeping grade books, managing groups of students, and browsing logs. Ashford-Rowe and Malfroy (2009) organize these tools in four groups, namely: •

•

• •

Content - Unit/Course online, Lecture and Tutorial notes, Media (i.e. lectopia, podcast, videocasts), links to scholarly information (readings), links to content resources (i.e. websites), interactive resources (.swf .fla .flv and other file types); Communication - Chat, Announcements, Discussion Board, Email, Blogs and Forums; Collaboration - Wikis, Virtual Classroom and Voice-based communication; Assessment - Quizzes, Reflective learning journals, Portfolios, Grades, Surveys, Practice activities and past exams.

Recently the eLearning community started valuing more the interchange of course content and learners’ information, which led to the definition of standards for eLearning content sharing and interoperability. Standards can be viewed as “documented agreements containing technical specifications or other precise criteria to be used consistently as guidelines to ensure that materials and services are fit for their purpose” (Nichani, 2009). In the eLearning context, standards are generally developed for the purposes of ensuring interoperability and reusability in systems and of the content and meta-data they manage. In this context, several organizations (e.g. IMS GLC, IEEE, ISO/IEC, ADL) are developing specifications and standards (e.g. IMS CP, IMS CC, IMS DRI, LOM, SCORM) in the last years (Dagger & O’Connor & Lawless & Walsh & Wade, 2007). These specifications are closely related with the learning object concept as context independent, transportable and reusable pieces of instruction that are digitally managed and delivered (Rehak & Mason, 2003). There are other definitions for Learning Objects (LO). Rehak & Mason (2003) define a learning object as: “a digitized entity 144

which can be used, reused or referenced during technology supported learning”. As every kind of software, LMS continue to evolve to meet market demands. In relation to interoperability the main trend for the next LMS generation is service-oriented architectures (SOA) (Dagger & O’Connor & Lawless & Walsh & Wade, 2007). In these architectures LMS expose their functions as services and consume services from their operational environments, improving their interoperability with other eLearning systems. In fact, the last few years brought us several initiatives (Smythe, 2003; Wilson & Blinco & Rehak, 2004) to adapt SOA to eLearning. These initiatives, commonly named eLearning frameworks, have the same goal: to provide flexible learning environments for learners worldwide. Usually they provide a set of open interfaces to numerous reusable services organized in genres or layers that can be combined in service usage models (Queirós & Leal, 2010). Other trends result from new market demands such as Web 2.0, Talent Management, Mobile Learning, “Software as a Service” and Open Source Software. With the recent appearance of Web 2.0 tools and the popularity of social networking tools like Facebook and Twitter, there has been a great demand to use similar tools in the LMS to enhance the communication among teachers and students. Talent Management software systems are an extension of traditional human resource management systems. Some researches (Bersin & Howard & O’Leonard & Mallon, 2009) shows that in 2009 more than 70% of large companies have an LMS already and almost 1/3 of these companies are considering replacing or upgrading these systems with integrated talent management systems (Levensaler & Laurano, 2009). With more students working at distance, there has been also a strong demand to make eLearning applications accessible through mobile devices (e.g. Smartphones, PDA) know as Mobile Learning or m-learning. Using LMS Software as a Service (SaaS) schools can relieve the financial burden of maintaining their LMS by outsourcing the host-


ing service. Commercial LMS (e.g. Blackboard, WebCT) have dominated the education market in previous years, but as costs increase, schools and companies are now looking for other options such as open-source solutions (e.g. Moodle, Sakai) that are financially more attractive.

COMPARISON FRAMEWORK The goal of this work is to analyse and compare LMS interoperability features. Given the number of LMS vendors it would be impracticable to study them all. Therefore we selected two LMS that we consider representative of the LMS universe. This selection is based on their prominence in the LMS market and the fact that they cover the open source and commercial development models. Interoperability is in general a complex concept that can be analysed in multiples perspectives and this is surely the case with LMS. To organize our study we identified two broad classes of systems that usually integrate the operational environment of the LMS. Thus, we considered two facets in LMS interoperability, regarding communication and data sharing with these classes.

Learning Management Systems A good number of LMS that were developed in the past fifteen years are still in use and under active development. For the purpose of our study we must concentrate on a few systems that are representative of the LMS universe in terms of their characteristics and market share. A simple categorization of this type of systems is according to their development model. There are fundamentally two: open source systems, such as Moodle, Sakai, .LRN or Dokeos; and commercial systems such as WebCT/Blackboard or Desire2Learn. Figure 1 presents a timeline of the development of several initiatives grouped by their development model.

The Figure 1 shows that the first major LMS adopted a commercial development model but since the beginning of this century there has been a shift towards open source systems. In fact, this shift was already recognized as a trend in LMS development (Davis & Carmean & Wagner, 2009). In spite of the growing popularity of open source, commercial systems are still relevant and they must be included in any representative sample of LMS. In these two categories we decided to select the most popular systems taking as reference the available data on global LMS usage (Davis & Carmean & Wagner, 2009). As part of this study we conducted a survey on eLearning systems usage on Portuguese higher education institutions. We received responses from 20 different institutions and the results for LMS usage are shown in Figure 2. The two most popular LMS in these institutions follow the global trend, which reinforces our choice of the reference systems for our study. We decided to focus our study on Moodle and Blackboard. We chose them since they represent the two main development models used by LMS vendors (open source and commercial); and combined they have a significant share on the LMS market (33.2% on the international market (Davis & Carmean & Wagner, 2009) in 2009 and 80% on our own recent survey). The following paragraphs provide an overview of the selected systems. Moodle (version 1.9.9 - 8th June 2010) is a free and open-source LMS written in PHP and created by Martin Dougiamas. Its name is an acronym for Modular Object-Oriented Dynamic Learning Environment. In early January of 2010, Moodle had a user-base of 46,624 registered sites with 32,464,992 users in 3,161,291 courses in 209 countries and in more than 75 languages (Cole & Foster, 2007). The most common functions of Moodle are the course information and documentation, documents repository, announcements, synchronous and a synchronous communication

145


Figure 1. Timeline of development of major LMS

Figure 2. LMS usage in Portuguese higher education institutions

146


(email, chat room, discussion forum) and assignments. Blackboard (version 9.1 - 1th April 2010) was developed by Blackboard Inc. in 1997 and is an online proprietary virtual learning environment system that is used by over 3700 educational institutions in more than 60 countries. In February 2006, the virtual learning environment called WebCT (Course Tools) was acquired by Blackboard Inc. (Blackboard, 2005) and, as part of the acquisition terms, the Blackboard brand was assumed until now.

Interoperability Facets The interoperability features of a system reflect the operational environment where it is expected to be deployed. The operational environment of an LMS includes different systems and services with which it may have to communicate and exchange data. As depicted in Figure 3 we identified two broad classes of systems that usually integrate the operational environment of an

LMS, each corresponding to a different facet in LMS interoperability. We identified also a layer of infrastructural systems and services that are domain independent but that play an important role in LMS interoperability. For the purpose of this study, the broad classes of systems that we identified as part of the operational environment of an LMS are the following: Learning Content Management Systems (LCMS) are used for the development, management and publishing of digital learning content (e.g. Learning Objects) that the LMS delivers. Examples of these systems are the Learning Object Repositories, e-Portfolio Systems, Authoring Tools, Specialized Evaluators and others. Academic Management Systems (AMS) are used for managing academic data information of an educational institution. Typical features of these systems are the management of courses, classes and students, the enrolment of students in courses, the submission of summaries and grades by teachers, among others.

Figure 3. LMS interoperability facets

147


Apart from these facets the LMS is supported by infrastructure services providing basic functions that are not specific to eLearning, such as directory services for authentication and authorization or printing services. We consider also as part of this infrastructure the web or application server, the database engine and the operative system. In many cases this infrastructural layer is used for implementing ad hoc interoperability solutions. In the following sections the selected systems are analysed and compared regarding these two facets. We categorize and identify the remote systems in each facet, the existent standards and the interoperability issues regarding LMS communication with those systems. There is a huge asymmetry among these facets and this is reflected in the structure of the following sections. Of the two facets, the first has a larger number of systems and mature standards. The systems in second facet are mostly home-grown with few and immature standards to regulate both content (e.g. academic records, course forms, grades, summaries) and communication.

LEARNING CONTENT MANAGEMENT FACET The Learning Content Management facet focuses on the interoperation with systems that provide pedagogical content and services delivered by the LMS. We start by identifying their main types, followed by the existing standards for content and communication, and ending with an example of system integration in this facet.

System Types The content delivered by an LMS can be created, obtained, gathered or evaluated in several types of systems such as Learning Objects Repositories, E-Portfolio systems, Authoring Tools, Specialized

148

Evaluators or Quizzes. In the following sub-subsections some of these system types are detailed.

Learning Object Repositories A repository of learning objects can be defined as a “system that stores electronic objects and meta-data about those objects” (Holden, 2004). The need for this kind of repositories is growing as more educators are eager to use digital educational contents and more of it is available. Learning object repositories can be organized in free (CAREO, POOL, CLOE, EducaNext, ARIADNE) and payed (Sentient Learning, Harvest Road Hive, Learn Exact, Luminas) (JORUM team, 2006). One of the best known repositories of LOs is Merlot (Multimedia Educational Resource for Learning and Online Teaching) which provides pointers to online learning materials and includes a search engine. The Jorum Team made a comprehensive survey (2006) of the existing repositories and noticed that most of these systems do not store actual LOs. They just store meta-data describing LOs, including pointers to their locations on the Web, and sometimes these pointers are dangling. Most of the current repositories are specialized search engines of LOs and with little support for interact with specialized eLearning systems, such as evaluation engines and experimentation environments. These systems require both complete interoperability and specific Metadata. They need service oriented repositories of learning objects, fully compliant with the existing interoperability standards, and supporting new definitions of learning objects for specialized domains. An example of a specialized repository of LO is crimsonHex (Leal & Queirós, 2009), the repository developed as part of the EduJudge project to act as a programming problem repository service to the Evaluation Engine (EE) and the LMS.


ePortfolio Systems

Authoring Tools

An electronic portfolio is a digital collection of student work (artefacts) usually managed in ePortfolio systems and displayed for specific audiences and purposes. The ePortfolios systems usually include (or link) a repository where students organize their artefacts typically for the purpose of assessment. The benefits of an ePortfolio system in an educational institution are shared by students and teachers. Students are able to reflect on their educational experiences and showcase their work in a repository. Teachers are able to evaluate the student progress and provide concrete evidence of the students’ learning. Helen C. Barrett (Barret, 2008) organizes the ePortfolio tools in two categories: individual and institutional. Both are presented in Table 1. In the individual category we can use authoring tools to author portfolios offline (requires web server space to publish online) or web services to create online and publish a presentation portfolio allowing interactivity (Web 2.0). In the institutional category we can use a software-server where an institution installs on their own server to provide space for hosting portfolios or hosted services that an institution adopts (no server required) that host portfolios. In the survey we conducted on Portuguese high education institution no one indicated to be using an ePortfolio system. This fact allows us to conclude that the dissemination of these tools in the educational institutions, at least in Portugal, is still low.

The growing popularity of learning objects lead to the development of specialized editors supporting eLearning metadata. These tools, either open source, freeware or commercial, export the content to SCORM packages and other formats such as IMS CP, IMS CC, HTML, PPT, PDF and Flash. The most important authoring tools can be grouped by their development model such as: • • •

Open Source: eXe, Xerte, ScenariChain Opale and LOMPad; Freeware: Hot Potatoes, MyUdutu, MOS Solo, Reload and Courselab; Commercial: Camtasia, Captivate, QuizCreator, PPT2Flash, PowerQuizPoint.

The majority of the authoring tools support multiple application profiles. RELOAD is arguably the most mature of these projects and is available both as a standalone Java application and as an Eclipse IDE plugin. It supports a broad range of metadata formats but cannot be extended to support specialized formats. The SHAME project (2006) - Standardized Hyper Adaptable metadata Editor – stands out from the rest since it is actually a metadata editing and presentation framework for RDF metadata with support for all kind of metadata based on a previous mapping for the RDF syntax. Some of these tools are specialized in a certain type of multimedia format (e.g. Captivate for video) or activities (e.g. Hot Potatoes for quizzes) and are the best place for the users to create the respective content.

Table 1. ePortfolio tools by categories Individual

Institutional

Authoring tools

Web Services

Software – Server

Hosted Services

Mozilla Composer Dreamweaver Microsoft Office Adobe Acrobat Movie Maker

Google Docs Zoho Writer WikiSpaces

Elgg Mahara OSPI Moofolio, MyStuff (embedded in Moodle)

Digication iWebfolio Epsilen GoogleApps for Education

149


Specialized Evaluators Examples of eLearning systems that provide content can be drawn from different domains. At the heart of a system with automatic evaluation resides an Evaluation Engine (EE). This is an apt example of a specialized eLearning service, performing a specific task and reusable in different scenarios. An EE can supply its services not only to LMS but also to other specialized application services, such as quizzes and contest management systems. Desktop based applications also fit in this approach. This model of combining specialized services can be extended to competitive learning in other domains such as business training, for instance. In this domain teachers use business simulation games to improve the strategic thinking and decision making skills students in particular areas (e.g. finances, logistics, and production). Through these simulations students compete among them, as they would in a real world companies. A business simulation service fulfils a role similar to that of the EE in programming exercises and it also requires a repository containing specialized LO describing simulations. An example of a evaluation engine is the UVA Online Judge EE (Regueras & Verdú & Castro & Pérez & Verdú, 2008), the EE developed as part of the EduJudge project to act as an evaluator of programming problems submitted by students.

Standards In this subsection we introduce several standards related to learning objects. We structured these standards in four groups: packaging, metadata, organization and communication.

Packaging Packaging is crucial to store eLearning material and reuse it in different systems. The most widely used content packaging format is the IMS Content

150

Packaging (IMS CP, 2007). An IMS CP learning object assembles resources and meta-data into a distribution medium, typically an archive in zip format, with its content described in a manifest file in the root level. The manifest file - named imsmanifest.xml -adheres to the IMS CP schema and contains the following sections: • • •

•

Metadata: describes the package as a whole; Organizations:describes the organization of the content within a manifest; Resources:contains references to resources (files) needed for the manifest and metadata describing these resources; Sub-manifests: defines sub packages.

The manifest uses another standard - the IEEE Learning Object Metadata (IEEE LOM, 2002) to describe the learning resources included in the package (c.f. Sub-subsection 4.2.2). Recently, IMS Global Learning Consortium proposed the IMS Common Cartridge (IMS CC, 2010) that adds support for several standards (e.g. IEEE LOM, IMS CP, IMS QTI, IMS Authorization Web Service) and its main goal is to shape the future regarding the organization and distribution of digital learning content.

Metadata The content of LO packages is described by metadata. Its purpose is to support the interoperability and reusability of learning objects. As mentioned previously, the IMS CP manifest contains four sections and is precisely metadata that provides an overall description of the package. Metadata can be used to describe file features in the Resource section. In the manifest the Metadata element is used at two levels: package (overall description of the package) and resource (description of the resource and contained files). In both cases metadata information usually follows the IEEE LOM schema. The IEEE LOM is a data model used to


describe a learning object. The model is organized in several categories that cover general data, such as title and description, technical data such as object sizes, types and durations, educational characteristics and intellectual property rights, among many others. These categories are very comprehensive and cover many facets of a LO. However, LOM was designed for general LO and does not to meet the requirements of specialized domains. For instance, there is no way to assert the role of specific resources. Fortunately, IMS CP was designed to be straightforward to extend through the creation of application profiles. The term Application Profile generally refers to “the adaptation, constraint, and/ or augmentation of a metadata scheme to suit the needs of a particular community”. A well know eLearning application profile is the Sharable Content Object Reference Model (SCORM, 2009) that extends IMS CP with more sophisticated sequencing and Contents-to-LMS communication. The IMS GLC is also responsible for another application profile, the Question & Test Interoperability (QTI) specification. QTI describes a data model for questions and test data and, since version 2.0, extends the LOM with its own metadata vocabulary. QTI was designed for questions with a set of pre-defined answers, such as multiple choice, multiple response, fill-in-the-blanks and short text questions. There are other metadata specifications, such as, the Dublin Core metadata, which provides a simpler and a more loosely-defined set of elements useful for sharing metadata across heterogeneous systems. At the present, the Dublin Education Working Group is extending the Dublin Core for the specific needs of the education community.

Organization Learning objects can be organized in items and an organization defines a path through those items. The IMS CP specification includes a manifest section called Organizations. This section can be

used to design pedagogical activities and articulate the sequencing of instructions. By default, it uses a tree-based organization of learning items pointing to the resources (assets) included in the package. However, other standards could be accommodated in this section, such as IMS Simple Sequencing (IMS SS) and IMS Learning Design (IMS LD). These specifications aims to provide to the teachers mechanisms for coordination of the educational instructions based on students’ profile making the instruction more dynamic and flexible. The IMS LD specification is a meta-language for describing pedagogical models and educational goals. Several IMS LD-aware tools are available as players (e.g. CopperCore, .LRN) and authoring/ export tools (e.g. Reload, LAMS). The IMS SS is a specification used to describe paths through a collection of learning activities. The specification declares the order in which learning activities are to be presented to a learner and the conditions under which a resource is delivered during an eLearning instruction. Despite all these specifications, the design of more complex adaptive behaviour is still hard to achieve.

Communication The standardization of the learning content it is not enough to ensure interoperability, which is a major user concern with the existing systems. The definition of common protocols and interfaces for the communication among systems is also an issue that the major eLearning interoperability initiatives (e.g. NSDL, POOL, OKI, EduSource, IMS) try to address. As an illustration we present the communication guidelines defined by IMS, arguably the most developed ones in this category. The IMS Learning Tools Interoperability (IMS LTI) provides a uniform standards-based extension point in LMS allowing remote tools and content to be integrated into LMS. The main goal of the LTI is to standardize the process for building links between learning tools and the LMS. The LTI has 3 key concepts (Gilbert, 2009): the Tool Provider,

151


the Tool Consumer and the Tool Profile. The Tool Provider is a learning application that runs in a container separate from the LMS. Publishes one or more tools through the Tool Profiles. The Tool Profile is an XML descriptor that describes how a tool integrates with a tool consumer. It is composed by information about the tool metadata, vendor information, resource and event handlers and menu links. The Tool Consumer publishes a Tool Consumer Profile (XML descriptor of the Tool Consumer’s supported LTI functionality that is read by the Tool Provider during deployment), provides a Tool Consumer Runtime and exposes the LTI services. The IMS Digital Repositories Interoperability (IMS DRI) specification deals with the communication with a specific eLearning system: the repository. Within eLearning, repositories are used to store, manage and share LO. One of such efforts was the IMS Digital Repositories (IMS DRI). The IMS DRI specification was created by the IMS Global Learning Consortium (IMS GLC) and provides a functional architecture and reference model for repository interoperability. The IMS DRI provides recommendations for common repository functions, namely the submission, search and download of LOs. It recommends the use of web services to expose the repository functions based on the Simple Object Access Protocol (SOAP) protocol, defined by W3C. Due to their growing popularity other web service interface flavours, such as Representational State Transfer (REST) (Fielding, 2000), should be considered, since they are not excluded from the recommendation. This will improve interoperability with systems that adhere to a more informal style of development.

Integration In the majority of the cases an LMS integrates an organization infrastructure in conjunction with other systems. In the following sub-subsections we present the interoperability features of the refer-

152

ence LMS with the learning content management systems usually found in educational institutions. An integration example between a LMS and one of these systems is also presented.

State of Art The integration with eLearning content management systems can be implemented on the LMS data or business layer. In the former the integration uses the import / export features of both system and relies on the support of common formats. In the later the integration relies on the existence of compatible web services in both systems. Data integration is the simplest and most popular form of integration in content management. For instance, the RELOAD authoring tool can be used to create learning objects in SCORM format and Blackboard supports and imports SCORM packages. Table 2 lists some of the most important eLearning content standards and specifications defined in the last years by educational organizations. For each standard we present the LMS support status. The studied LMS support almost all the LO package standards with exception of the recent IMS CC that is only partially supported. In relation to the design and sequencing of learning activities standards are not yet supported by these LMS, probably due to their complexity. Data integration assumes an important role in the LMS interoperation with system types that do not require a tight integration, as is the case with authoring tools. For instance, the Hot Potatoes system enables the creation of quizzes - interactive multiple-choice, short-answer, jumbled-sentence, crossword, matching/ordering and gap-fill exercises - in HTML format. Moodle includes an activity that imports the quiz (HTML file) previously generated in the Hot Potatoes system. It should be noted that although Moodle supports the QTI format for quizzes described previously, Hot Potatoes cannot export in this format.


Table 2. Reference LMS support of eLearning content standards Moodle

Blackboard

IMS CP

yes

yes

SCORM

yes

yes

IMS CC

partial

partial

IMS QTI

yes

yes

IMS LD

no

no

IMS SS

no

no

It is possible also to integrate an eLearning tool with an LMS on the business layer. For instance, the IMS Learning Tools Interoperability (IMS LTI) provides a uniform standards-based extension point in LMS allowing remote tools to be integrated into LMS. Although this specification is still not explored by the major LMS vendors, obtaining the certified support for IMS LTI is already a major milestone in their development plan. Another integration approach is through Application Programming Interface (API). The LMS include APIs to allow developers to extend their predefined features through the creation of plugins. Table 3 enumerates the approaches used by the selected LMS to address the interoperability issues regarding the integration with the system types referred in subsection 4.1. Moodle version 2.0 (due in September 2010) includes several APIs to enable the development of plugins by third parties to access repositories and portfolios (c.f. the following sub-sub-section). Blackboard uses the Building Blocks technology to cover the integration issues with other systems. A Building Block is simply a web application that runs on the Blackboard application server. This

technology allows third parties to develop modules using the Building Blocks API. For instance, the company Verbena Consulting LLC created a building block that provides a search user interface that allows searching in the MERLOT repository and returns matching results along with the metadata for each learning object.

Example of Integration In this subsection we illustrate the use of the communication APIs in Moodle, arguably the most popular LMS nowadays. Concretely we present the new file APIs of Moodle 2.0 and how it was used for implementing a plug-in for crimsonHex repositories (Leal & Queirós, 2009). The beta version of Moodle 2.0 includes support for different types of repositories. Two APIs are already available to enable the development of plug-ins by third parties systems, including: • •

Repository API for browsing and retrieving files from external repositories; Portfolio API for exporting Moodle content to external repositories.

Table 3. Integration APIs in reference LMS Moodle

Blackboard

Repositories

Repository API

Building Blocks API

E-Portfolios

Portfolio API

Building Blocks API

Evaluators

OPAQUE ws

no

153


We chose the Repository API since it is the most stable of the two. It is organized in two parts: Administration, for administrators to configure their repositories, and; File picker, for teachers to interact with the available repositories. Figure 4 presents the file picker GUI of the crimsonHex plug-in. On the left panel are listed the available repositories as defined by the administrator. Two crimsonHex repository instances are marked with label 1. Label 2 marks the default listing of the selected repository. Pressing the “Preview” link marked with 3 presents a preview of the respective LO. Pressing the “Search” link pops-up a simple search form, marked as 4. For federated search in all available crimsonHex repositories is used the text box marked as 5. The development of this plug-in was straightforward.

Figure 4. crimsonHex plugin interface

154

In terms of programming effort we spent half a day to produce approximately 100 new lines of code. For Moodle each repository is just a hierarchy of nodes. This allows Moodle to construct a standard browse interface. The repository server must provide: • •

a URI to download each node (e.g. a LO); a list of nodes (e.g. LO and collections) under a given node (e.g. collection).

In addition to these requirements, a repository can optionally support authentication, provide additional metadata for each node (mime type, size, related files, etc.), describe a search facility or even provide copyright and usage rules. Each feature of the plug-in is implemented by a method in a PHP class. A typical method includes:


a repository invocation (SOAP or REST), the parsing of its response (using a PHP function to parse the XML data), a selection of the pertinent data (using XPath) and an iteration over the new results (for instance, populating an array with the relevant data).

ACADEMIC MANAGEMENT FACET In this section we analyse the Academic Management facet. The main system type on this facet is the Academic Management System (AMS). An AMS aggregates all the information regarding administrative, financial, technical or scientific processes usual in educational institutions. Examples of these processes are the enrolment of students in courses, the management of grades or the payment of fees. This interoperability facet is not as mature as the one analysed on the previous section and there are still few standards available. This fact burdens the integration of academic management systems with LMS that must resort to ad hoc solutions based on the infrastructural layer.

Integration Unlike in content management, there is a sole type of system in this facet - the AMS - and apparently with very few vendors. We were not able to find in the literature any study on AMS usage. For this reason our analysis is based on the use of AMSs by Portuguese higher education institutions as reflected in the survey we conducted for this study. As mentioned before, the questionnaire inquired on vendors of different types of eLearning systems in use at each institution. We received responses from 20 different institutions and the results for AMS are presented in Figure 5. This data shows that no system is clearly preferred by Portuguese educational institutions. The choices are divided by the systems SIGA, SIGARRA, SOPHIA and Web on Campus. It

should be noted that most of these evolved from home grown systems and are in use in different schools from the same university or polytechnic institute. In some cases spin-offs were created to develop and commercialize these systems but the size of these companies cannot be compared with those developing other types of systems related to eLearning, such as LMS.

Standards An AMS manages different kinds of information. The concept of course, student, summary or grade should be described formally in order to be shared among all the systems included in a educational institution. As far as we know, there are few standards that formalize these content types and how they are communicated to allow the AMS to share data with other systems. Two know-standards are the IMS Learner Information Services (IMS LIS) and the IMS Learner Information (IMS LIP). The IMS Learner Information Services (IMS LIS) is the definition of how systems manage the exchange of information that describes people, groups, memberships, courses and outcomes within the context of learning. The IMS LIS, like its predecessor (IMS Enterprise specification), is focused on the connection between an LMS and an AMS. The IMS Learner Information (IMS LIP) specification addresses the interoperability of internet-based Learner Information systems with LMS. It describes mainly the characteristics of a learner. The learner information is a collection of information about a learner (individual or group learners) or a producer of learning content (creators, providers or vendors). The former is partial supported by Moodle and its implementation in Blackboard is in development. The latter has no support in either LMS.

155


Figure 5. AMS usage

Integration There is an obvious gain in integrating AMS and LMS: avoiding the duplication of processes. For instance, course management is required in both systems and with a tight integration it can be performed in just one of them. Several processes can be performed in only one side and reflected in the other such as: course management, enrolment of students, grades management, summaries management, exams schedule, absences management. In general, educational institutions use ad hoc solutions to implement this type of integration. The most common strategies are: • • •

156

Portals: aggregating content from multiple sources with a common presentation layer; Database Replication: different applications but sharing content; Features Share: presentation independent but sharing some features (e.g. authentication).

Integration usually includes at least one web application, and these are typically designed based on the well known three-tier architectural pattern. There is a potential for integration in any the three classical tiers: presentation, logic and data. The portal strategy integrates at the presentation tier, providing and unified web interface to a number of independent subsystems, including eLearning systems. The major advantage of this strategy is the fact that it gives users a sense of unity, sometimes at the cost of compromising consistency. Feature sharing is integration at the logic tier and is becoming increasingly popular as more systems expose their functionality using web services. Moreover, there are a number of infrastructural services, using or not web services, which can be exploited by eLearning systems. User authentication based in directory services, such as LDAP, is an apt example of this type of integration. Finally, integration may occur at the data tier, and partial database replication is arguably most common example. For instance a LMS may im-


port data on students, courses and student enrolment in courses from administrative systems to avoid the burden of entering this data manually. These integration models are usually combined. For instance, a portal that provides and unified presentation may also adhere to a single sign-on mechanism shared with other services.

CONCLUSION This chapter presents a comparative study on LMS interoperability. Given size of this category we focused on a couple of representative systems Moodle and Blackboard - since combined they represent a significant market share and cover both the commercial and open source development models. We proposed a framework for analysing LMS interoperability by distinguishing two different facets in the way theses systems communicate with their operational environment: learning content management and academic management. We characterized the types of systems that communicate with the LMS trough each facet. Standards are the corner stone of interoperability. Thus we made a comprehensive presentation of the existing standards. We completed the analysis with illustrations of system integration for each facet. The main conclusion of this study is that there is still a long road ahead in LMS interoperability. In general it is not straightforward to connect an LMS to another system. A lot of work has already been done in defining standards but many of them are supported neither by the LMS nor by the system that surround them. The content management facet is much more developed then the academic management facet. Content formats, especially those of learning objects, are already mature and widely supported by the analysed systems. The notable exception is the recent Content Cartridge of IMS that is not yet supported, as is not its companion specification - the Learning Tools Interoperability - that is still being implemented

in Moodle and Blackboard. This specification promises to be a major step towards content interoperability among eLearning systems. Meanwhile, to integrate LMS with content management systems we must resort to system specific APIs. An example of using Moodle 2.0 Repository API was presented to illustrate this type of integration. On the academic management facet there are no AMS system standing out from the crowd and most of those in use, at least in Portuguese higher education institutions, are home grown systems. Standards in this facet are few and immature and not widely supported by existing AMS systems. As a consequence, the integration of LMS and AMS relies on infrastructure services. We presented a set of integration strategies that are commonly used for implementing these ad hoc integrations. This study is part of an effort to select an LMS on which to base the development of eLearning systems integrating heterogeneous components. Unfortunately, from that viewpoint we cannot conclude on the superiority of any of the analysed systems.

REFERENCES Ashford-Rowe, K., & Malfroy, J. (2009). Elearning benchmark report: Learning management system (LMS) usage. Barret, H. C. (2008). Categories of ePortfolio tools, technical report. Retrieved on December 13, 2010, from http://electronicportfolios.com/ categories.html. Bersin, J., Howard, C., O’Leonard, K., & Mallon, D. (2009). Learning management systems. Bersin & Associates. Blackboard.com. (2005). Blackboard & WebCT Announce Agreement to Merge. Retrieved on December 13, 2010, from http://investor.blackboard.com/phoenix.zhtml?c=177018&p=irolnewsArticle&ID=767025.

157


Britain, S., & Liber, O. (1998). A framework for pedagogical evaluation of virtual learning environments. Retrieved on December 13, 2010, from http://www.leeds.ac.uk/educol/documents/00001237.htm. Cole, J., & Foster, H. (2007). Using Moodle Teaching with the popular open source course management system. Sebastopol, CA: O’Reilly Community Press. Common Cartridge Specification, I. M. S. (2010). Version 1.0 Final Specification. IMS Global Learning Consortium Inc. Retrieved November 12, 2010, from http://www.imsglobal.org/cc/ index.html. Dagger, D., O’Connor, A., Lawless, S., Walsh, E., & Wade, V. (2007). Service oriented e-learning platforms: From monolithic systems to flexible services. IEEE Internet Computing Special Issue on Distance Learning. Davis, B., Carmean, C., & Wagner, E. D. (2009). The evolution of the LMS: From management to learning - Deep analysis of trends shaping the future of e-learning. Sage Road Solutions. LLC. Ellis, R. K. (2009). Field guide to learning management systems, ASTD learning circuits. Fielding, R. (2000). Architectural styles and the design of network-based software architectures. Phd dissertation. Gilbert, T. (2009). Leveraging Sakai and IMS LTI to standardize integrations. In 10th Sakai Conference Pearson Education. Holden, C. (2004). What we mean when we say “repositories”, user expectations of repository systems, Academic ADL Co-Lab. Retrieved on December 13, 2010, from http://www.hewlett.org/ NR/rdonlyres/158FC043-A56F-43C6-ABA7EB9A62656FCB/0/RepoSurvey2004-1.pdf.

158

IEEE LOM. (2002). IEEE Standard for Learning Object Metadata IEEE 1484.12.1-2002. Retrieved on November 12, 2010, from http://www.ieeeltsc. org/standards/1484-12-1-2002/. IMS CP. (2007). IMS Content Packaging v1.2 Final specification. Retrieved on November 12, 2010, from http://www.imsglobal.org/content/ packaging/. JORUM team. (2006). E-learning repository systems research watch. Retrieved on December 13, 2010, from http://www.jorum.ac.uk/docs/pdf/ Repository_Watch_final_05012006.pdf. Leal, J. P., & Queirós, R. (2009). CrimsonHex: A service oriented repository of specialised learning objects. In ICEIS 2009, 11th International Conference on Enterprise Information Systems. Levensaler, L., & Laurano, M. (2009). Talent management systems 2010. Bersin & Associates. Martínez, J. Á., & Navarra, P. L. (2007). Content interoperability on e-learning platforms: Standardization, digital libraries, and knowledge management, Revista da Universidad y Sociedad del. Conocimiento. Nichani, M. (2009). LCMS = LMS + CMS [RLOs] – How does this affect the learner? The instructional designer? Retrieved on December 13, 2010, from http://www.elearningpost.com / articles/archives/ lcms_LMS_cms_rlos. Pantel, C. (1997). A framework for comparing Web-based learning environments. Master’s thesis, School of Computing Science, Simon Fraser University, Canada. Queirós, R., & Leal, J. P. (2010). E-learning frameworks: A survey. In INTED2010 Proceedings (pp. 1345-1354).


Regueras, L. M., Verdú, E., Castro, J. P., Pérez, M. A., & Verdú, M. J. (2008). Design of a distributed and asynchronous system for remote evaluation of students’submissions in competitive e-learning. In International Conference on Engineering Education (ICEE ‘08). Rehak, D., & Mason, R. (2003). Engaging with the learning object economy . In Littlejohn, A. (Ed.), Reusing online resources: A sustainable approach to e-learning (pp. 22–30). London, UK: Kogan Page. SCORM. (2004). 4th Ed. Specification. Retrieved on November 12, 2010, from http://www.adlnet. gov/Pages/Default.aspx. SHAME. (2006). SHAME - Standardized Hyper Adaptable Metadata Editor. Retrieved on December 13, 2010, from http://kmr.nada.kth.se/shame. Smythe, C. (2003). IMS abstract framework - A review. IMS Global Learning Consortium, Inc. Wilson, S., Blinco, K., & Rehak, D. (2004). An e-learning framework - Paper prepared on behalf of DEST (Australia), JISC-CETIS (UK), and Industry Canada.

ADDITIONAL READING Aguirre, S., Salvachúa, J., Fumero, A., & Tapiador, A. (2006). Joint degrees in e-learning systems: A Web services approach. Collaborative computing: Networking, applications and worksharing. Al-Smadi & Gutl. (2010). SOA-based architecture for a generic and flexible e-assessment system. In EDUCON’10. Alario, C., & Wilson, S. (2010). Comparison of the main alternatives to the integration of external tools in different platforms, ICERI2010 Proceedings (pp. 3466-3476).

Alario-Hoyos, C., Asensio-Pérez, J. I., BoteLorenzo, M. L., Gómez-Sánchez, E., Vega-Gorgojo, G., & Ruiz-Calleja, A. (2010). Integration of external tools in virtual learning environments: Main design issues and alternatives. Proceedings of the 10th International Conference on Advanced Learning Technologies, ICALT, Sousse, Tunisia (pages 384-388). Apostolopoulos, T. K., & Kefala, A. (2003). An e-learning service management architecture. In Proceedings of the 3rd IEEE International Conference on Advanced Learning Technologies, Athens, Greece (pp. 140-144). Aroyo, L., Dolog, P., Houben, G., Kravcik, M., Naeve, A., & Wild, F. (2006). Interoperability . In Personalized adaptive learning. Educational Tecnhnology & Society. Barret, H. (2010). Electronic portfolios in STEM - What is an electronic portfolio. Retrieved from http://www.scribd.com/doc/40206175/E-Portfolio-Definition Bohl, O., Scheuhase, J., Sengler, R., & Winand, U. (2002). The shareable content object reference model (SCORM)-a critical review, Proceedings of the International Conference on Computers in Education (pages 950-951). Bryden, A. (n. d.). Open and global standards for achieving an inclusive information society. Casella, G., Costagliola, G., Ferrucci, F., Polese, G., & Scanniello, G. (2007). A SCORM thin client architecture for e-learning systems based on Web services. [Hershey, PA: IDEA Group Publishing.]. International Journal of Distance Education Technologies, 5(1), 13–30. doi:10.4018/ jdet.2007010103 Donello, J. (2002). Theory & practice: Learning content management systems.

159


Earl, T. (2005). Service-oriented architecture Concepts, technology and design. Upper Saddle River, NJ: Prentice Hall. Eckerson, W. (1995). Server architecture: Achieving scalability, performance and efficiency in client server applications. Open Information Systems, 19(1). Friesen, N. (2005). Interoperability and learning objects: An overview of e-learning standardization. Interdisciplinary Journal of Knowledge and Learning Objects. Girardi, R. (2004). Framework para coordenação e mediação de Web services modelados comolearning objects para ambientes de aprendizado na Web. Hall, B. (2003). Learning management systems and learning content management systems demystified. Harasim, L. (2006). History of e-learning: Shift happened, The International Handbook of Virtual Learning Environments. Springer. Harman, K., & Koohang, A. (2007). Learning objects: Standards, metadata, repositories, and LCMS. Informing Science Institute, Edição de Informing Science. Hatala, M., Richards, G., Eap, T., & Willms, J. (n. d.). The Interoperability of learning object repositories and services: Standards, implementations and lessons learned. Proceedings of the 13th international World Wide. Holden, C. (2004). What we mean when we say “repositories”, User expectations of repository systems. In: Academic ADL Co-Lab. Malita, L. (2009). E-portfolios in an educational and ocupational context. Procedia - Social and Behavioral Sciences, 1(1), 2312-2316. doi:10.1016/j. sbspro.2009.01.406.

160

Nichani, M. (2009). LCMS = LMS + CMS [RLOs] – How does this affect the learner? The instructional designer? Retrieved from http:// www.elearningpsot.com/articles/archives/lcms_ LMS_cms_rlos Rehak, D. R., & Mason, R. (2003). Keeping the learning in learning objects . In Littlejohn, A. (Ed.), Reusing online resources: A sustainable approach to e-learning (pp. 22–30). London, UK: Kogan Page. Tastle, J., White, A. & Shackleton, P. (2005), E-learning in higher education: The challenge, effort, and return of investment, International Journal on ELearning. Williams, J., & Goldberg, M. (2005). The evolution of e-learning. [Global.]. Universitas, 21.

KEY TERMS AND DEFINITIONS API: Application Programming Interface is a set of rules that a program must follow to access the services provided by other program that implements that API. eLearning: A new form of learning based on technology. A model of teaching and learning based on the online environment, leveraging the capabilities of the Internet for communication and content distribution ePortfolio: A web collection of electronic evidence packaged and managed by a user. These evidences are stored in an ePortfolio tool with a twofold set of features: students organize its achievements and teachers use them to assess the work and evolution of students. Interoperability: Ability of systems to interoperate in a uniform way. The interoperability among systems is usually achieved using standards and specification on content and communication. LO (Learning Objects): Units of instructional content that can be used, and most of all reused, on web based eLearning systems. Usually it’s a


set of files including a XML manifest wrapped with resources and packaged in a ZIP file. LMS: System used mainly by teachers and students to manage the teaching/learning process. Nowadays these type of systems occupy a central

role in the eLearning realm coordinating and integrating a set of features by third-party systems. Metadata: Data about data. Metadata will allow systems to index, search and retrieve files based on semantic data.

161

162

Chapter 8

Mobile Learning Management Systems in Higher Education Demetrios G. Sampson University of Piraeus & Centre for Research and Technology Hellas, Greece Panagiotis Zervas University of Piraeus & Centre for Research and Technology Hellas, Greece

ABSTRACT Learning Management Systems (LMS) are widely used in Higher Education offering important benefits to students, tutors, administrators and the educational organizations. On the other hand, the widespread ownership of mobile devices has lead to educational initiatives that investigate their potential as the means to change the way that students interact with their tutors, their classmates, the learning material, the administration services and the environment of their educational institute. This mainly aims to support the continuation of these interactions not only outside the classroom, but also beyond desktop restrictions, towards to a truly constant and instant access from anywhere. As a result, the development of mobile LMS (mLMS) is important for the deployment of feasible mobile-supported educational services in Higher Education. In this book chapter, we address the issue of designing mLMS for Higher Education by studying and applying the W3C Mobile Web Best Practices 1.0 to a widely used existing LMS, namely, the Moodle.

INTRODUCTION During the last years, several studies have been reported that Technology-Supported Education can be effectively used in Higher Education (HE) for enhancing and enriching the traditional ways DOI: 10.4018/978-1-60960-884-2.ch008

of teaching and learning, offering meaningful learning experiences that bare the potential to address the shortcomings of traditional classroombased learning (Catherall, 2004; Tham & Werner, 2005; Bonk & Graham, 2006; Mayes et al, 2009). Technology-Supported Higher Education can offer a number of services to all relevant actors in HE (namely, students, tutors, administrators and the


Mobile Learning Management Systems in Higher Education

educational organization) such as: (a) transparent access to courses’ materials and activities, as well as, participation to synchronous and asynchronous group-based course activities, (b) on-line submission, marking and feedback on students’ assignments, (c) effective students’ course activities tracking, (d) efficient student enrollment and monitoring, and (e) on-line delivery of courses offered by an educational organization, reaching students outside campus. These services can enhance the opportunity of learning to anyone, anywhere and at anytime without place and time restrictions and promote wider participation, removing the traditional barriers to Higher Education studies (Laurillard, 2005). Learning Management Systems (LMS) are among the most widely used applications in Technology-Supported Education, providing a convenient way to organize and deliver educational and training e-services in formal educational settings (Weller, 2007; Cole & Foster, 2007). LMS are now considered as mainstream applications for the organization, management and delivery of on-line courses in Higher Education, since they enable efficient planning, implementation, administration, tracking and reporting of educational and training activities (Kim & Bonk, 2006). LMS provide to students facilities for enrolling to on-line courses, accessing lecture notes and supportive course material, communicating with their classmates and their tutors through online discussions, participating to on-line assessments, as well as, monitoring their progress and grades. Moreover, LMS enable tutors to organize their courses’ syllabus and teaching material, gather, grade and provide feedback to students’ assignments, track students’ progress and participation to their courses and communicate with their students for answering their questions. Furthermore, LMS provide to the administrators efficient ways for assigning tutors to the courses and administrating enrolled students. Finally, LMS are offering to the Higher Educational Institutions an effective way for on-line delivery of their courses towards

reaching a larger and globally dispersed audience beyond traditional campuses. On the other hand, the widespread ownership of mobile devices and the growth of mobile communications industry has offer the potential for the provision of new services including, internet access without place and device constraints, interpersonal and group communication without place and time restrictions and sharing of digital content in any format (text, image, audio and video) (Naismith et al., 2005). These new services offered by mobile devices can be used for educational purposes aiming to enhance traditional classroom-based and/or desktop-based web-facilitated educational experiences. This has lead to educational initiatives that investigate the potential of mobile devices as the technological means to change the way that students interact with their tutors, their classmates, the learning material, the administration services and the environment of their educational institute. This is important towards supporting the continuation of these interactions not only outside the classroom, but also beyond desktop restrictions towards to a truly constant and instant access from anywhere. (Kukulska-Hulme & Traxler, 2005). Within this context, providing access to LMS via mobile devices bares the potential to enhance Technology-Supported Higher Education and achieve additional benefits for students, tutors, administrators and the educational organization. More precisely, mobile LMS (mLMS) can provide to students new opportunities for accessing courses’ materials and for communicating with their classmates and their tutors beyond desktop restrictions. Tutors can use the mobile devices, so as to instantly and continuously monitor issues related with their courses (i.e. timely submission of students’ assignments, questions that have submitted from the students etc) while they are on the move. System administrators can carry out basic system support tasks even without access to their desktops. Finally, the Higher Educational

163


Institutions are able to make their on-line courses truly accessible to their students. As a result, the issue of developing mobile LMS is important for the deployment of feasible mobile supported educational services in Higher Education. In this book chapter, we target addressing this issue by studying and applying the Mobile Web Best Practices 1.0 (proposed within the World Wide Web Consortium) (Rabin & McCathieNevile, 2006) to a widely used existing LMS, namely, the Moodle, so as to be accessible via mobile devices. The chapter is structured as follows: Following this introduction, the background section discusses Learning Management Systems and their benefits in Higher Education, the mobile devices and their anticipated advantages in the educational process, the international guidelines for developing web applications and/or content for mobile devices, as well as related works and studies in the field of mLMS development. The next section presents the main design considerations for the development of the proposed mobile Moodle system, demonstrates how the W3C Mobile Web Best Practices have been employed in the development of mobile Moodle by giving examples of our implementation and presents the validation process followed, so as to ensure conformance of the developed mobile Moodle with the current W3C Mobile Web Best Practices 1.0. Finally, we discuss future and emerging trends in the field of mobile LMS development, as well as our concluding remarks.

BACKGROUND Learning Management Systems Learning Management Systems (LMS), also referred to as Course Management Systems (CMS) or Virtual Learning Environments (VLE), are web-based applications (that is, they run on a server and are accessed by using a web browser) designed to support planning, implementation, or-

164

ganization, delivery and administration of on-line courses (Weller, 2007). Over the past years, LMS have attracted the attention of Higher Education community, since their available features can simulate the features of classroom-based teaching (Watson & Watson, 2007). Nowadays, LMS are considered as mainstream applications for most HE Institutions (Catherall, 2004; Tham & Werner, 2005; Weller, 2007) and they can offer important benefits to all relevant actors in Higher Education (namely, students, tutors, administrators and the educational organization), which are presented below (Bouhnik & Marcus, 2006; Daniels, 2009): •

•

Students are able to enroll to on-line courses and have access to lecture notes and courses’ supportive material at anytime and from anywhere. They are also able to study specific components of an on-line course in a more flexible way beyond the restrictions imposed by the traditional classroom. Additionally, students are able to continue interaction and communication with their classmates and tutors beyond campus physical presence, through on-line course activities. Finally, LMS provide students with opportunities for participating in on-line assessments and receiving instant feedback for their performance, as well as opportunities for monitoring their individual progress and building their own e-portfolios of academic achievements. Tutors are able through the LMS to organize and manage their courses’ syllabus, upload and share courses’ materials with their students, as well as, gather and grade more efficiently students’ assignments. Moreover, LMS offer effective ways for tutors to communicate and interact with their students and to provide feedback to them, as well as to monitor their students’ academic progress and access their grades, so as to provide additional guidance and


Table 1. Key features of LMS to the key actors in HE Features

Students

Upload and share course materials

Tutors √

Access and download lecture notes and course supportive material

√

Participation to online synchronous and/or asynchronous activities

√

Create online assessments

Administrators

Educational Organization

√

√ √

Participation to on-line quizzes/surveys

√

Access to online grade book

√

√

Gather and review assignments

√

Create and manage course syllabus

√

√ √

Students and Academic Personnel management and administration

√

√

Courses management and administration

√

√

√

√

√

√

Students’ progress tracking

√

Summative students’ statistics

•

•

support to students that are facing difficulties with their courses. Administrators can manage more efficiently the administrative tasks related with the on-line course organization, such as monitoring students’ enrollment, managing students’ records, as well as tutors allocation to the available on-line courses. Educational organizations have concrete benefits from LMS by extending the audience of their courses and by increasing the flexibility of studying and teaching at these institutions removing the barriers of physical presence at campus and/or classrooms. Furthermore, LMS can offer efficient management of HE institutions Human Resources (academic and administrative) and support the implementation of quality management plans.

Table 1 summarizes key features of LMS to the key actors (namely, students, tutors, administrators), as well as to the educational organization (Weller, 2007; Cole & Foster, 2007).

√

There are many different LMS available, which could be divided in two main categories: commercial and open source. Commercial LMS include: Blackboard/WebCT (http://www.blackboard.com), Desire2Learn (http://www. desire2learn.com/) and Lotus (http://www.lotus. com). On the other hand, open source LMS include: Moodle (http://moodle.org), ATutor (http://www. atutor.ca/) OLAT (http://www.olat.org) and Sakai (http://www.sakaiproject.org/). For the purpose of our work, we selected Moodle (http://moodle.org) as the LMS to be customized in a manner that it can be effectively accessed by mobile devices following the guidelines of W3C Mobile Web Best Practices 1.0. Moodle is an open source environment that allows the development of new functionalities and/or customization of the existing ones and it is offered at no cost. It provides compatibility with current learning technology specifications such as SCORM (http://www.adlnet.gov/scorm/) and IMS CP (http://www.imsglobal.org/content/packaging/). Moodle has a large and active community of users (including educational organizations that

165


use it as their LMS, tutors that deploy it for their individual courses, as well as, web developers who are continually modifying and expanding its source code). This offers a solid setting of large scale deployment and an environment for community-based further development.

Mobile Devices Mobile devices, also referred to as handheld devices or handheld computers, are pocket-sized computing devices with capabilities for internet connectivity (either via a wireless network or a mobile communication network), typically having a display screen with touch input and/or a mini keyboard (Naismith et al., 2005). Based on this definition and for the purpose of our work, examples of mobile devices are: the personal digital assistants (PDAs), the smart phones and the mobile phones. During the last years, mobile devices have been used for educational purposes and there are many studies reporting the benefits that they are offering to the educational process, which could be summarized below (Sharples, 2006; Pettit & Kukulska-Hulme, 2007; Corbeil & Valdes-Corbeil, 2007; Faux et al, 2006): •

•

•

166

Mobile devices can facilitate students to continue learning activities, initiated inside the traditional classroom, outside the classroom through their constant interaction and communication with their classmates and/or their tutors without place, time and device restrictions. Mobile devices can enable tutors to interact with their students while on the move, to organize, update and monitor their courses, their course activities and their teaching material from anywhere and at anytime. Furthermore, mobile devices can offer important facilities in including contextual and situated learning activities in traditional on-line course.

Designing Web Applications and/or Content for Mobile Devices Typically, mobile devices have a number of technical restrictions (such as limited memory, low screen resolution, insufficient processor power, low network bandwidth and limited input possibilities), which affect both the web content and the web applications that they can handle. Furthermore, there are not available yet globally agreed standard ways for the design considerations of mobile web applications. To this end, the World Wide Web Consortium (W3C) has proposed an initial set of guidelines for Mobile Web Best Practices 1.0 that can be taken into consideration when developing web content and/or web application for mobile devices (Rabin & McCathieNevile, 2006). The main objective of these guidelines is to improve the user experience when web content and applications are accessed from different mobile devices. The W3C Mobile Web Best Practices 1.0 guidelines are grouped under the following main sections (Rabin & McCathieNevile, 2006): •

•

•

•

Overall Behavior: This section includes general guidelines related to the delivery of web content and/or web application to be accessed via mobile devices. Navigation and Links: This section includes guidelines related with the definition of the structure and the navigation model of a web page or a web site so as to match the technical limitations of mobile devices (such as display size, input mechanisms, etc). Page Layout and Content: This section includes guidelines related with the user’s perception of the delivered content and provides recommendations for the language used in its text and the spatial relationship between its constituent components. Page Definition: This section includes guidelines related with technical aspects on how elements (such as tables, images


•

and video) should be presented in mobile web content. User Input: This section includes guidelines related with user input taking into consideration mobile devices restrictions, such as lack of pointing devices and standard keyboards for text entry, as well as, mobile devices alternative input mechanisms, such as through touch screens.

Related Work: Designing Mobile Learning Management Systems The review of related works in the field of Mobile Learning Management Systems design reveals that there are two main approaches. The first approach is device-dependent and involves the design (in accordance to the particular targeted mobile device technical characteristics) and the development of stand alone applications that can be installed to a specific mobile device, so as to provide access to the course data of the original LMS. The main commercial LMS (used by a number of Higher Education Institutions worldwide) are following this approach and provide stand-alone applications of the original LMS, Typical examples of such mobile LMS are: Blackboard Mobile (http://www. blackboard.com/Mobile/), Litmos Mobile (http:// www.litmos.com/mobile-learning/) and Meridian Mobile LMS (http://www.meridianksi.com/products/mobile_lms/). Moreover, this approach has been also adopted from some development groups that are working with open source LMS (mainly Moodle). Typical examples are stand-alone applications of Moodle for specific mobile devices such as Apple’s iPhones or smartphones that run the Google Android operating system. There are also cases of Higher Education Institutions that they developed and use stand-alone applications of Moodle, such as the Polytechnic University of Catalonia (Forment & Guerrero, 2008) and University of Belgrade (Minović et al., 2008). Although the aforementioned design approach can achieve optimized performance for a specific

device taking advantage of its particular technical characteristics, its main limitation is that a different version of the stand-alone application should be developed for every different type of mobile device. This can be cost-effective only for certain cases where a mobile device has a significant market penetration. The second approach, which was adopted also in our work, includes the customization of the original LMS, so as to enable access via mobile devices without device-specific restrictions. An important design consideration of this approach is that the customization of the mobile LMS should conform to the W3C Mobile Web Best Practices, so as to ensure that mLMS can be accessed by any type of mobile device. An initial work based on this approach has been proposed by Houser & Thornton (2005), who customized limited Moodle features (such as polls, quizzes, wikis and forums), in order to provide access to these features through mobile phones. Another work following this design approach has been proposed by Cheung, Stewart and McGreal (2006), who investigated the customization of Moodle for use via mobile devices within the Athabasca University in Canada. The limitation of these efforts is that they do not claim conformity of the developed mLMS with W3C Mobile Web Best Practices and, moreover, that the customized Moodle features are limited. In our work, we propose the design and the implementation of a server-based mobile version of Moodle with selected features suitable for Higher Education that follows the W3C guidelines for Mobile Web Best Practices 1.0.

THE MOBILE MOODLE Design Considerations In order for a Learning Management System to be designed to support access via mobile devices, we should take into account the following design considerations (Keegan, 2005; Sampson et al, 2008):

167


•

•

•

•

Students, tutors and administrators should be able to access the LMS via any type of mobile device. Students should be able to enroll in and attend courses via their mobile device, check for new course material, upload assignments, send questions to their tutors and communicate with their classmates and tutors through discussion forums. Tutors should be able to conduct basic course management tasks via their mobile device such as: monitor their students’ progress, identify and download newly uploaded students’ assignments, answer students’ questions and communicate with their students through discussion forums. Administrators should be able to carry out basic administrative tasks such as: add new users to the mLMS and allow enrollment of new students to available courses.

According to the aforementioned considerations we decided about the features that the mobile version of Moodle should support. Every course in Moodle consists of resources, activities, and blocks (Cole & Foster, 2007). Resources provide the appropriate tools for adding content such as web pages and links to web sites while activities provide the appropriate tools to add course activities such as forums, quizzes, and assignments. Blocks are some additional features for course support such as a list of all participants or a calendar. Table 2 presents the main features of the proposed Mobile Moodle The implementation of our Mobile Moodle consists of two sub-systems, the PC version and the Mobile version, which communicate through a common database in order for users to have access to the same course material; by identifying the device used to access it (PC or mobile device). As presented in Figure 1, when a user accesses Moodle through a laptop/desktop PC then the information is presented to him/her ordinarily. When a user accesses Moodle through a mobile

168

device then the same information is presented to him/her in an appropriate customized manner. The source files of Moodle that have been customized in order to create the mobile version of the system are the following: •

•

•

Files for defining content format: that is, the CSS (Cascading Style Sheets) files that contain all the appropriate declarations for content format (font size, font color etc.). Files for defining content presentation: that is, the basic source files (PHP or HTML files) that define content presentation (i.e. information displayed in an array form). Files for calling features functions: Mainly PHP files that contain calls of Moodle features functions. For example in this type of files the feature of calendar is called. If we want to remove the calendar in the mobile version of Moodle, then we should delete the corresponding part of code that calls the calendar feature. It is important to note that we can remove also the calendar feature from the administrative part of Moodle, but this would have impact on the PC version of the system because, as mentioned before, the two versions share the same database. Thus, any changes in one version would simultaneously change the other version, too.

System Implementation In order to customize Moodle to support access via mobile devices we applied a set of guidelines selected from Mobile Web Best Practices 1.0, which was discussed in the background section. Table 3 presents the selected guidelines, as well as their description on how they can be implemented. An implementation example of Guideline 1 is that a user can type the URL “ask4reasearch.info/ moodle” instead of having to type “http://www. ask4reasearch.info/moodle/pda” which is the full path for accessing the mobile Moodle version.


Figure 1. The adopted mobile Moodle approach

Figure 2(a) represents an implementation example of Guidelines 2 and 4. The basic navigation of the website should be placed on the top of the page and preferably on a single line. Moodle by default uses an appropriate menu at the top of each page conformant with Guideline 2, so no further adaptation has been made in relation to this guideline. Guideline 4 that indicates the use of clear language is of particular importance for mobile delivery to assist users determine whether information is of interest to them. Thus, the information presented in the system is given in clear and simple language (i.e. a link to lecture material) while headings are used to group similar items in order for a user to distinguish information presented in a page (i.e. an area that presents the Course Activities) (implementing Guideline 4).

Figure 2(b) represents an implementation example of Guidelines 5 and 7. Scrolling is limited to only one direction (vertical) (implementing Guideline 5). The fonts and colors used (defined in CSS files of the theme adopted) provide sufficient contrast of text colors and background color so that information is visible and readable from users (implementing Guideline 7). Figure 3(a) represents an implementation example of Guidelines 6, 10 and 13. The user should not have to scroll significantly to find the primary content of the page. Therefore the most important information (i.e. a forum for posting questions related to the course), should precede information that is not so important (i.e. list of courses that a user is enrolled) (implementing Guideline 6). Informing the browser in advance

169


Table 2. Key features of proposed server-based Mobile Moodle

Blocks

Activities

Resources

Features

Description

Insert a label

Insertion of textual description with the possibility of image input.

Compose a text page

Creation of a simple page of text.

Compose a web page

Formatting options for a web page including more that text

Link to a file or a website

Creation of hyperlinks to files uploaded to Moodle web site or to web sites outside Moodle.

Display a Directory

Organizing files already uploaded to the system in directories.

Choice

A simple poll (multiple choice question).

Forum

Threaded discussion board.

Glossary

Dictionary of terms.

Quiz

A web-based quiz with a variety of question types, such as multiple choice, true/false, short answer and matching.

Assignments

Advanced uploading of files: Learners can upload to the system a specific number of files (the maximum number of files that can be uploaded is defined by the tutor). Upload a single file: Same functionality with “Advanced uploading of files” only that a single file is allowed to be uploaded. Online text: Learners answer to a question placed by the tutor in simple text format. Offline activity: The tutor defines the assignment without the possibility for file uploading.

Administration

Contains all the administration tools of a course such as students’ registration, grades, course backup etc.

Courses

A list of all courses in which a student has enrolled.

Activities

A list of hyperlinks with all the activities added to the course (i.e. forum, assignment).

Search Forums

Search based on keywords in all the existing forums in the course.

Calendar

A calendar for displaying various course or site events.

Participants

Link to a list of all participants in the current course.

about the size of an image avoids the re-flow of a page when the browser is handling it. Thus, in Mobile Moodle the explicit definition of the images’ size is given (i.e. height=”16px” and width=”16px”) (implementing Guideline 10). Given the input limitations of mobile devices, the interface of the system must minimize user input as far as possible. Thus, in current implementation of mobile Moodle limited typing is required (i.e. typing a keyword to find and access a certain thread in a forum) while it is possible for users to select items using the device’s navigation keys (i.e. select Questions Forum) (implementing Guideline 13). Figure 3(b) represents an implementation example of Guidelines 3, 8 and 9. It is important to provide concise and descriptive link text to help users decide whether to follow a link

170

or not (i.e. a link leading to course material). Also it would be helpful to identify the implications of following a link by denoting the size of the targeted file (i.e. 250KB) and/or its format (i.e. PDF) (implementing Guideline 3). It is also helpful to use features of the markup language (i.e. HTML) to indicate logical document structure in a page. Moreover, tables do not work well on limited size screens and by putting for example, navigational links into them may result for the user to scroll both horizontally and vertically to see possible navigational choices. A simple and easy to use alternative to tabular representation, which at the same time can indicate clear document structure, is the use of ordered and/or unordered HTML lists. For example two headings each one followed


Table 3. The W3C Mobile Web Best Practices guidelines implemented in Mobile Moodle

User Input

Page Definition

Page Layout and content

Navigation and Links

Guidelines

Description

1. URIs of Site Entry Points

Keep the URIs of site entry points short.

2. Navigation Bar

Provide only minimal navigation at the top of the page.

3. Link Target Identification

Clearly identify the target of each link and note the target file’s format unless you know the device supports it.

4. Page Content

Ensure that content is suitable for use in a mobile context, use clear and simple language and limit content to what the user has requested.

5. Scrolling

Limit scrolling to one direction, unless secondary scrolling cannot be avoided.

6. Navigation Bars etc. (Extraneous material)

Ensure that material that is central to the meaning of the page precedes material that is not.

7. Color

Ensure that information conveyed with color is also available without color and ensure that foreground and background color combinations provide sufficient contrast.

8. Structural Elements

Use features of the markup language to indicate logical document structure.

9. Tables

Do not use tables unless the device is able to support them. Do not use nested tables. Do not use tables for layout and where possible, use an alternative to tabular presentation.

10. Image Size

Specify the size of images in markup and resize images at the server.

11. Measures

Do not use pixel measures and do not use absolute units in markup language attribute values and style sheet property values.

12. Style Sheets

Use style sheets to control layout and presentation, unless the device is known not to support them, organize documents so that if necessary they may be read without style sheets and keep style sheets small.

13. Input

Keep the number of keystrokes to a minimum, avoid free text entry where possible and provide pre-selected default values where possible.

Figure 2. Implementation example of W3C mobile Web best practices guidelines (i)

171


by a number of links represented as an un-ordered list (implementing Guidelines 8 and 9). Finally, an implementation example of Guidelines 11 and 12 is given below. Style information may be contained in an externally linked style sheet if the device supports use of style sheets. Current implementation demands that the device used to access mobile Moodle supports usage of style sheets (implementing Guideline 12). Moreover, avoiding pixel and absolute measures to define the layout allows the browser to adapt the content presented to fit in the display. An exception to this rule is the situation in which an image has been specifically sized for a particular display as mentioned in Guideline 10 (implementing Guideline 11). The example below defines the layout of the navigation bar used in mobile Moodle where the width of the bar is given in percentage format (i.e. width: 100%) and all the other style elements are relatively sized (i.e. height: 25em).

.navbar { margin-right: 1em; width: 100%; padding: 0em; height: 25em; border-width: 1em; border-style: solid; }

Validation Process In order to validate the conformance of the developed mobile Moodle with the W3C Mobile Web Best Practices 1.0, we used an automated validation tool provided by W3C, namely, the mobileOK Checker (http://validator.w3.org/mobile/). This tool performs various tests on a web page to determine its level of mobile-friendliness by conforming it to the W3C Mobile Web Best Practices 1.0. When a web page passes all tests then it is characterized as mobileOK page and it is considered suitable to be accessed by any mobile device. In our case, all different web pages of the

Figure 3. Implementation example of W3C Mobile Web Best Practices guidelines (ii)

172


developed mobile Moodle were validated with the mobileOK Checker and for all them the tool returned with mobile conformance higher than 98%. For those pages that the tool did not return with full mobile conformance, we corrected the source files that correspond to these pages based on the recommendations provided by the tool. This process was repeated, so as to achieve full conformance to W3C Mobile Web Best Practices 1.0 for all web pages of mobile Moodle.

FUTURE RESEARCH DIRECTIONS In our work, we propose the design and the implementation of a server-based mobile version of Moodle with extended features suitable for Higher Education that follows the W3C guidelines for Mobile Web Best Practices 1.0. This is important for the deployment of feasible mobile-supported educational services in Higher Education. However, it is only an initial step for the future evolution of mLMS. An emerging trend in developing mobile LMS is to exploit current mobile devices enhanced features like position, acceleration sensors or cameras, towards detecting users’ intentions and gathering of contextual information, aiming to provide him/her with personalized situated context-aware educational services on-demand. An initial study towards the development of a mobile LMS, which gathers contextual information and reasoning on them, has been recently proposed by Lehsten et al (2010). It enables personalized mobile learning experience with location-sensitive lecture streaming, campus navigation, and ubiquitous features of the whole university computing infrastructure (Lehsten et al, 2010). Another trend in developing mobile LMS aims to overcome identified limitations of LMS, such as (Zhang et al, 2004; Liu, 2004):

•

•

LMS are used as a ‘one-size-fits-all’ service without taking into consideration individual student learning styles, knowledge level, goals and interests. In such situation, all learners receive same activities and material, regardless of their varying pre-existing knowledge and experience. LMS are promoted as ways to manage students, rather than to promote rich, interactive experiences.

Within this framework, introducing adaptive techniques to mobile LMS, so as to provide students with educational material and activities that fits their learning styles, knowledge level, goals and interests, is attracting increased research attention. An initial study towards the development of an Adaptive Mobile LMS has been proposed by Park et al (2008) and introduces the application of an artificial intelligence technique to a mobile educational device in order to provide a Learning Management System that is adaptive to students’ learning styles (Park et al, 2008).

CONCLUSION Technology-Supported Education exploits the potential of web technologies for enhancing and enriching the traditional ways of teaching and learning, offering meaningful learning experiences that bare the potential to address the shortcomings of traditional classroom-based learning in which tutors and students must physically participate in the learning process. LMS are widely used web-based applications in Technology-Supported Education, providing a convenient way to organize and deliver educational and training e-services in formal education settings such as Higher Education. On the other hand, the widespread ownership of mobile devices and the potential benefits that they can provide to the learning process has attracted the attention of both researchers and prac-

173


titioners in Technology-Supported Education. This has lead to mobile devices-supported educational initiatives that investigate the potential to change the way that students interact with their tutors, their classmates, the learning material, the administration services and the environment of their educational institute, supporting the continuation of these interactions not only outside the classroom, but also beyond desktop restrictions towards to a truly instant access from anywhere. As a result, the development of mobile LMS (mLMS) is important for the deployment of feasible mobile-supported educational services in Higher Education. Within this context, in this book chapter, we propose the design and the implementation of a server-based mobile version of Moodle with selected features suitable for Higher Education that follows the W3C guidelines for Mobile Web Best Practices 1.0.

REFERENCES Bonk, C. J., & Graham, C. R. (2006). The handbook of blended learning. San Francisco, CA: Pfeiffer. Bouhnik, D., & Marcus, T. (2006). Interaction in distance-learning courses. Journal of the American Society for Information Science and Technology, 57(3), 299–305. doi:10.1002/asi.20277 Catherall, P. (2004). Delivering e-learning information services in higher education. Oxford, UK: Chandos Publishing. Cheung, B., Stewart, B., & McGreal, R. (2006) Going mobile with MOODLE: First steps. IADIS International Conference on Mobile Learning 2006, Dublin, Ireland. Cole, J., & Foster, H. (2007). Using Moodle: Teaching with the popular open source course management system, (2nd Edition). Sebastopol, CA: O’ Reilly Community Press

174

Corbeil, J. R., & Valdes-Corbeil, M. E. (2007). Are you ready for mobile learning? EDUCAUSE Quarterly, 30(2). Daniels, P. (2009). Course management systems and implications for practice. International Journal of Emerging Technologies & Society, 7(2), 97–108. Faux, F., McFarlane, A. E., Roche, N., & Facer, K. (2006). Handhelds: Learning with handheld technologies. Bristol, UK: Futurelab. Forment, M., & Guerrero, J. C. (2008, April 1113). Moodlbile: Extending Moodle to the mobile on/offline Scenario. Proceedings of the IADIS International Conference on Mobile Learning, Algarve, Portugal. Houser, C., & Thornton, P. (2005). Poodle: A course-management system for mobile phones. In Proc. of the 4th IEEE International Workshop on Wireless and Mobile Technologies in Education (WMTE ’05). IEEE Computer Society. Keegan, D. (2005). Mobile learning: The next generation of learning. Distance Education International. Kim, K., & Bonk, C. J. (2006). The future of online teaching and learning in higher education: The survey says. EDUCAUSE Quarterly, 29(4). Kukulska-Hulme, & A., Traxler, J. (2005). Mobile learning – A handbook for educators and trainers. UK: Taylor & Francis. Laurillard, D. (2005). E-learning in higher education. In Ashwin, P. (Ed.), Changing higher education: The development of learning and teaching. London, UK: RoutledgeFalmer.


Lehsten, P., Zender, R., Lucke, U., & Tavangarian, D. (2010, March 29-April 7). A service-oriented approach towards context-aware mobile learning management systems. 8th IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops) (pp 268-273).

Sampson, D., Papapaulou, P., Zervas, P., & Papanikou, C. (2008). Supporting higher education teaching through wireless and mobile devices: The case study of mobile Moodle. In Proc. of the AACE E-Learn 2008-World Conference on ELearning in Corporate, Government, Healthcare, and Higher Education.

Liu, Y. (2004). Faculty development and CMS. In Proceedings of Society for Information Technology in Teacher Education International Conference, 2004, 2409–2412.

Sharples, M. (Ed.). (2006). Big issues in mobile learning. Report of a workshop by the Kaleidoscope Network of Excellence Mobile Learning Initiative. Nottingham, UK: University of Nottingham.

Mayes, J. T., Morrison, D., Bullen, P., Mellar, H., & Oliver, M. (Eds.). (2009). Transforming higher education through technology-enhanced learning. York, UK: Higher Education Academy. Minović, M., Štavljanin, V., Milovanović, M., & Starčević, D. (2008). Usability issues of elearning systems: Case-study for Moodle learning management system. In Meersman, R., Tari, Z., & Herrero, P. (Eds.), OTM 2008 Workshops, Lecture Notes in Computer Science 5333 (pp. 561–570). Heidelberg, Germany: Springer.

Tham, C. M., & Werner, J. M. (2005). Designing and evaluating e-learning in higher education: A review and recommendations. Journal of Leadership & Organizational Studies, 11(2), 15–26. doi:10.1177/107179190501100203 Watson, W. R., & Watson, S. L. (2007). An argument for clarity: What are learning management systems. What are they not, and what should they become? TechTrends, 51(2), 28–34. doi:10.1007/ s11528-007-0023-y

Naismith, L., Lonsdale, P., Vavoula, G., & Sharples, M. (2005). Literature review in mobile technologies and learning. NESTA Futurelab Series.

Weller, M. (2007). Virtual learning environments: Using, choosing and developing your VLE. UK: Routledge.

Park, H., Baek, Y. K., & Gibson, D. (2008). Design of an adaptive mobile learning management system. User interface design and evaluation for mobile technology, I, 286-301.

Zhang, D., Zhao, J., Zhou, L., & Numamaker, J. (2004). Can e-learning replace classroom learning? Communications of the ACM, 47(5), 75–78. doi:10.1145/986213.986216

Pettit, J., & Kukulska-Hulme, A. (2007). Going with the grain: Mobile devices in practice. Australasia Journal of Educational Technology, 23(1), 17–33. Rabin, J., & McCathieNevile, C. (Eds.). (2006). Mobile Web best practices 1.0. (W3C Proposed Recommendation). Retrieved on August 25, 2010, from http://www.w3.org/TR/mobile-bp/

ADDITIONAL READING Amin, A. H. M., Mahmud, A. K., Abidin, A. I. Z., Rahman, M. A., Iskandar, B. S., & Ridzuan, P. D. (2006). M-learning management tool development in campus-wide environment. Issues in Informing Science and Information Technology, 3, 423–434.

175


Andronico, A., Carbonaro, A., Colazzo, L., & Molinari, A. (2004). Personalisation services for learning management systems in mobile settings. International Journal of Continuing Engineering Education and Lifelong Learning, 14(4-5), 353–369. doi:10.1504/IJCEELL.2004.005726

Dalsgaard, C. (2006). Social software: E-learning beyond learning management systems. European Journal of Open Distance and E-Learning. Retrieved August 31, 2010, from http://www.eurodl. org/materials/contrib/2006/Christian_Dalsgaard. htm

Armatas, C., Holt, D., & Rice, M. (2005, December 4-7). Balancing the possibilities for mobile technologies in higher education. In Balance, fidelity, mobility: Maintaining the momentum? Proceedings ASCILITE 2005, Brisbane.

Davide, T., & Roberto, B. (2007). A platform to support anytime, anywhere, just-for-me mlearning. Paper presented at the Seventh IEEE International Conference on Advanced Learning Technologies (ICALT 2007), Nigata, Japan.

Boticario, J.G., & Santos, O.C. (2007). An open IMS-based user modelling approach for developing adaptive learning management systems. Journal of Interactive Media in Education.

Driscoll, M., & Carliner, S. (2005). Advanced Web-based training strategies: Unlocking instructionally sound online learning. San Francisco, CA: Pfeiffer.

Chao, I. T. (2008). Moving to Moodle: Reflections two years later, EQ, EDUCAUSE Quarterly, 31(3), 46–52.

Evans, C. (2008). The effectiveness of m-learning in the form of podcast revision lectures in higher education. Computers & Education, 50(2), 491– 498. doi:10.1016/j.compedu.2007.09.016

Chen, C.-M., & Hsu, S.-H. (2008). Personalized intelligent mobile learning system for supporting effective english learning. Journal of Educational Technology & Society, 11(3), 153–180. Coates, H., James, R., & Baldwin, G. (2005). A critical examination of the effects of learning management systems on university teaching and learning. Tertiary Education and Management, 11(1), 19–36. doi:10.1080/13583883.2005.996 7137 Corlett, D., Sharples, M., Bull, S., & Chan, T. (2005). Evaluation of a mobile learning organizer for university students. Journal of Computer Assisted Learning, 21(3), 162–170. doi:10.1111/ j.1365-2729.2005.00124.x Cow, R., Santos, I. M., LeBaron, J., McFadden, T. A., & Osborne, F. C. (2010). Switching gears: Moving from e-learning to m-learning. MERLOT Journal of Online Learning and Teaching, 6(1), 268–278.

176

Goh, T., & Kinshuk. (2006). Getting ready for mobile learning – Adaptation perspective. Journal of Educational Multimedia and Hypermedia, 15(2), 175–198. Herrington, A., & Herrington, J. (Eds.). (2006). Authentic learning environments in higher education. Hershey, PA: Information Science Publishing. Järvelä, S., Näykki, P., Laru, J., & Loukkanen, T. (2007). Structuring and regulating collaborative learning in higher education with wireless networks and mobile tools. Journal of Educational Technology & Society, 10(4), 71–79. Kadyte, V. (2004). Learning can happen anywhere: A mobile system for language learning. In Attewell, J., & Savill-Smith, C. (Eds.), Learning with mobile devices (pp. 73–78). London, UK: Learning and Skills Development Agency. Kim, S. H., Mims, C., & Holmes, K. P. (2006). An introduction to current trends and benefits of mobile wireless technology use in higher education. AACE Journal, 14(1), 77–100.


Kinshuk, Kommers, P., & Sampson, D. (2004). Adaptivity in Web and mobile learning services [Editorial]. International Journal of Continuing Engineering Education and Lifelong Learning, 14(4/5), 313–317. Liaw, S. (2008). Investigating students’ perceived satisfaction, behavioral intention, and effectiveness of e-learning: A case study of the Blackboard system. Computers & Education, 51, 864–873. doi:10.1016/j.compedu.2007.09.005 Parsons, D., Ryu, H., & Cranshaw, M. (2007). A design requirements framework for mobile learning environments. Journal of Computers, 2(4), 1–8. doi:10.4304/jcp.2.4.1-8 Rekkedal, R., & Dye, A. (2007). Mobile distance learning with PDAs: Development and testing of pedagogical and system solutions supporting mobile distance learners. International Review of Research in Open and Distance Learning, 8(2), 1–26. Sampson, D., Gotze, K., & Zervas, P. (2007). Delivering IMS learning design activities via mobile devices. In Proc. of the 7th IEEE International Conference on Advanced Learning Technologies (pp. 765-769), IEEE Computer Society. Shen, R., Wang, M., Gao, W., Novak, D., & Lin, T. (2009). Mobile learning in a large blended computer science classroom: System function, pedagogies, and their impact on learning. IEEE Transactions on Education, 52(4), 538–546. doi:10.1109/TE.2008.930794 Vovides, Y., Sanchez-Alonso, S., Mitropoulou, V., & Nickmans, G. (2007). The use of elearning course management systems to support learning strategies and to improve self regulated learning. Educational Research Review, 2(1), 64–74. doi:10.1016/j.edurev.2007.02.004

Wang, H. (2008). Implement and deploy mobile learning in open universities - The many promises and challenges ahead. In C. Bonk et al. (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2008 (pp. 1340-1343). Chesapeake, VA: AACE. Zurita, G., & Nussbaum, M. (2004). A constructivist mobile learning environment supported by a wireless handheld network. Journal of Computer Assisted Learning, 20(4), 235–243. doi:10.1111/ j.1365-2729.2004.00089.x

KEY TERMS AND DEFINITIONS Learning Management System (LMS): Also referred to as Course Management Systems (CMS) or Virtual Learning Environments (VLE), are web-based applications (that is, they run on a server and they are accessed by using a web browser) designed to support planning, implementation, organization, delivery and administration of web-based courses Mobile Device: Also referred to as handheld device or handheld computer, it is a pocket-sized computing device with capabilities for internet connectivity (either via a wireless network or a mobile communication network), typically having a display screen with touch input and/or a miniature keyboard. Examples of mobile devices include: tablet PCs, personal digital assistants (PDAs), smart phones and mobile phones. Mobile Devices Supported Education: The support of traditional ways of teaching and learning through the use of mobile devices Mobile Learning Management System (mLMS): A Learning Management system that can be accessed from students, tutors and administrators via mobile devices.

177

178

Chapter 9

Enhancing Electronic Examinations through Advanced MultipleChoice Questionnaires Dimos Triantis Technological Educational Institution of Athens, Greece Errikos Ventouras Technological Educational Institution of Athens, Greece

ABSTRACT The present chapter deals with the variants of grading schemes that are applied in current Multiple-Choice Questions (MCQs) tests. MCQs are ideally suited for electronic examinations, which, as assessment items, are typically developed in the framework of Learning Content Management Systems (LCMSs) and handled, in the cycle of educational and training activities, by Learning Management Systems (LMS). Special focus is placed in novel grading methodologies, that enable to surpass the limitations and drawbacks of the most commonly used grading schemes for MCQs in electronic examinations. The paired MCQs grading method, according to which a set of pairs of MCQs is composed, is presented. The MCQs in each pair are similar concerning the same topic, but this similarity is not evident for an examinee that does not possess adequate knowledge on the topic addressed in the questions of the pair. The adoption of the paired MCQs grading method might expand the use of electronic examinations, provided that the new method proves its equivalence to traditional methods that might be considered as standard, such as constructed response (CR) tests. Research efforts to that direction are presented.

INTRODUCTION Learning Management Systems (LMS) ideally support the whole cycle of activities related to the interactions between instructors and students, as well as their interactions with administrative

staff and learning material (Chu, Young, Ngai, Cun, Pearl, & Macario, 2010; Ellis, 2009a). This “all encompassing” characteristic provides an extensive set of features that are available to the users of a LMS. Educational institutions and corporations can use features such as certification of

DOI: 10.4018/978-1-60960-884-2.ch009


Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires

participants, compliance tracking, reporting and statistical processing of educational activities, management approval, authoring of material and content management, as well as assessment ans testing. In the present chapter focus will be placed on the assessment and testing features of a LMS and, more specifically, on variants of electronic examination methods. An electronic examination is an assessment item delivered by the LMS, while the development, management and publishing of the items that the electronic examination consists of are usually tasks of a Learning Content Management System (LCMS), a software application that is either incorporated into a LMS or closely interacts with it (Feldstein, 2004). The use of electronic examinations methods, in the context of LMS, as computer-based learning and evaluation items, created in the framework of a LCMS and managed by a LMS, should be considered for enriching and strengthening the existing assessment schemes, while at the same time constructing innovative assessment techniques (Ganguli et al., 2006; Triantis, Anastasiadis, Tsiakas, & Stergiopoulos, 2007). As stated above, LMS are not directly involved in producing the contents provided to the users of the systems. Nevertheless, the whole “superstructure” that a LMS provides to an educational organization that seeks to benefit from an automated system for managing educational processes, is based on a set of foundations, among which are the assessment and testing tools incorporated into the LMS. The American Society for Training and Development (ASTD) conducted two surveys, in 2009 (189 participants) and 2010 (342 participants), on users of LMS. When asked to rank the most valuable features of LMS, in both years, among 15 features, assessment and testing ranked 1st with 59.3% of participants ranking it most valuable feature in 2009 and 54.5% in 2010 (Ellis, 2009b; Ellis, 2010). In 2009 the feature that ranked 2nd was content management, with 48.4%, and in 2010 reporting, with 51.5%. Electronic examination modules are an important part of

assessment and testing tools used by LMS. In the light of the above, research concerning innovative assessment techniques, incorporated into electronic examination modules, might be an efficient way to further the adoption of LMS. Electronic examinations, as part of computeraided testing systems, enhance student assessment procedures in a variety of ways. For example, the time allocated to scoring the answers of the students and the administrative burden related to the registration of the examination grades might be drastically reduced or even eliminated. Selfexamination material can be provided, which can be used on a distance-based learning framework by the students (Tsiakas, Stergiopoulos, Nafpaktitis, Triantis, & Stavrakas, 2007). The automated extraction of statistical indicators of the students’ performance can also be straightforwardly incorporated to the software implementing and managing the electronic examination (Mattheos et al., 2008; Van der Linden & Glas, 2000). One of the basic features of a LMS is that it enables to keep track of the performance of students across the various types of training activities that the students are engaged in. The existence and management of quantitative indicators, in the framework of a LMS, might enable the detection by the teacher of topics whose assimilation is not satisfactory by the students. In turn, the knowledge that specific material has been deficiently learned by the examinees might focus remedial action in subsequent teaching periods of the examined courses (Buchanan, 2002; Bull, Conole, Davis, White, & Sclater, 2002). Such actions might be integrated in a LMS, by modifying the study material provided to the students, in both printed and electronic form, using a LCMS, and by consecutively keeping track of the modifications to the students performance indicators induced by the changes that have been introduced. Despite of those benefits, the extent of the usefulness of electronic examinations is still under investigation, since a number of matters have to be taken into consideration for each specific application, such as the familiarization of

179


the students with information technology (IT) and the burden imposed on the teacher for producing both the examination forms (questionnaires) and the related electronic study material. Additionally, the possible overcomplexity of the educational software that both examinees and teachers should use must be taken into account and reduced as much as possible, so as not to hamper the familiarization of users with the electronic systems (Buchanan, 2002; DeBord, Aruguete, & Muhlig, 2004; Mattheos et al., 2008; Thomas, 2003). To overcome these problems, the LCMS must be easy to use for the content developer and built in a way to produce user-friendly interfaces for the various electronic examination tools. Multiple-Choice Questions (MCQs) tests are inherently suited to and extensively used in electronic examination systems, due to the facility to extract the final score of the examinee in a fully automatic mode (Mattheos et al., 2008; Van der Linden & Glas, 2000). Nevertheless, drawbacks of the MCQs examination method, mainly related to the influence of guessing in choosing an answer, hamper the extension of their use (Scharf & Baldwin, 2007). In order not to be constrained by the problems related to the most commonly used MCQs grading schemes, examinations based on the more traditional method of constructed response questions (CRQs), where, for example, questions are answered by developing a set of subjects in the form of short essays, are still largely used and often preferred to MCQs. The preference to CRQs, which are not well suited for electronic examinations, since the grading process cannot be usually automated, relies on the fact that in CRQs the examiner has the possibility to check more fully the knowledge of the student and especially the way that he/she developed the subject under question. In order to alleviate those problems of MCQs tests, a specific kind of MCQs examination method has been proposed, which uses sets of pairs of MCQs, referred as “paired” MCQs method (Ventouras, Triantis, Tsiakas, & Stergiopoulos, 2010). MCQs in each pair concern the same topic, but this similarity should not be 180

evident for a student who does not possess adequate knowledge on the topic addressed in the questions of the pair. The questions’ answers are graded in pairs, providing a bonus (penalty) if both (only one) answers of the pair are correctly answered. The objective of the research presented in this chapter is to advance the investigation comparing the paired MCQs grading scheme to the traditional constructed response (CR) test, taken as a standard, as well as their statistical relationship to the most simple grading scheme of MCQs, i.e., the positive scoring rule, according to which students gain points for correct answers and have no losses for wrong answers and omissions. The variation of the parameters controlling the bonus/penalty mechanism of the scoring rule is investigated. In this way, indications could be provided that the examination method based on paired MCQs might constitute a reliable tool for the evaluation of the performance of the students, enhancing electronic examination use.

BACKGROUND Electronic examinations can significantly augment the speed of testing large number of students, if the scoring of the test can be done automatically through an electronic examination platform, without need of judgment application by a human scorer. Tests being comprised of statements that have to be classified as true or false and MCQs tests are such examination methods (Bush, 2006; Freeman & Lewis, 1998; Scharf & Baldwin, 2007). MCQs usually belong to the “single-best answer” scheme, that requires the examinee to indicate the most appropriate single response, from a set of responses/answers that are proposed to the examinee, per question (McCoubrie & McKnight, 2008). A much less frequently used variation consists in using the “elimination procedure” proposed by Coombs et al. (1956), in which examinees are asked to eliminate as many incorrect answers as they can.


The benefits of MCQs tests have been extensively analyzed (Bush, 2006; McCoubrie & McKnight, 2008; Race, 2005; Scharf & Baldwin, 2007). Apart from the obvious time-reduction benefits for the examiner, in testing large number of students, and the fact that the examinee’s score is not dependent on his/her writing speed skill, a broad range of student knowledge acquisition for the field being examined might be tested in a short period of time, provided that a sufficiently large question bank is correctly used in constructing the MCQs test. The question bank should be constructed in a way that all the teaching units/ chapters/modules comprised in the course that will be tested are covered by a number of MCQs, that MCQs of varying difficulty are created per teaching unit and that the number of questions in the data bank is large enough so that repetition of questions in consecutive tests will be avoided as much as possible. Furthermore, ensuring correctness of the use of the data bank implies that for a specific examination, questions will be selected from the data bank in a way to cover all teaching units, in a way proportionate to the relative significance of each unit in the course, with questions of varying difficulty existing in the test for each unit. In electronic examinations using MCQs, there is the possibility to provide as feedback the correct answer to the student, immediately after he/she answered each MCQ item. The provision of immediate on-screen feedback (Scharf & Baldwin, 2007; Race, 2005) might enable examinees to reflect on the procedure they used to reach a wrong answer and therefore avoid consecutive missteps, provided there is adequate time to reflect on the questions. In the more common case where the examinee is required to submit his/her answers to all the questions of the test, without intermediate feedback, the automated nature of the marking makes possible the provision of prompt feedback to the examinee, after the termination of the examination, concerning both the overall test grade and information for correct and incorrect answers that were given by the examinee per

multiple-choice (MC) item. It has been shown that, in addition to the obvious ability of MCQs to test simple memory skills, such as power of information retention and retrieval, MCQs tests can investigate complex abilities and understanding (Scharf & Baldwin, 2007; Manning & Dix, 2008; Curzon, 1997). In the framework of research concerning parameters used as markers of the reliability and validity of examination methods (Anastakis, Cohen, & Reznick, 1991; Bennett, Rock, & Wang, 1991; Bridgeman, 1991; Evans, Ingersoll, & Smith, 1966; McLean, Dauphinee, & Rothman, 1988; Wainer, Wang, & Thissen, 1994; Wass, Wakeford, Neighbour, & Van der Vleuten, 2003), there exist substantial indications that MC scores provide higher reliability and are as valid as scores extracted from examinations based on the CRQs method (Wainer & Thissen, 1993; Lukhele, Thissen, & Wainer, 1994). It could have been expected that those indications would help in promoting the use of MCQs tests in most educational settings where CRQs are still used, especially taking into account the drawbacks of the CRQs examination method concerning the fact that subjects that might be examined cannot cover a significant amount of the material taught during the courses and, more importantly in an electronic examination setting, the inherent inability of introducing automated grading in essay-like responses to questions. Nevertheless, CRQs tests are still widely used, and this might not only be due to their main advantage, namely that they provide to the examiner the possibility to check the path taken by the student to develop the subject under question and reach the requested answers, but also due to the variety of drawbacks that plague MCQs. One category of drawbacks of MCQs tests concerns the requirements imposed on the person who will construct the question bank for the MCQs test. The MCQs examination method relies heavily on the quality of the question bank. As mentioned above, this concerns the need of a large question bank, so as to cover all the teaching

181


units/chapters/modules of the course/material to be examined, in a way proportionate to the extent of each unit and with the existence of questions of varying difficulty per unit. The time burden to produce such an extended bank might often result in poorly written or overlapping questions (Bush, 2006). If higher-order thinking skills are to be consistently checked, the time requirements for constructing the MC items might increase the effort and expertise needed. For example, in order to test the aptitude to solve complex problems and exercises, it has been proposed to fragmentize the initial problem into questions, so that the reasoning path that a student would take could be checked (Stergiopoulos, Tsiakas, Triantis, & Kaitsa, 2006a). Obviously, it is a challenge to construct a variety of such “linked” questions, per teaching unit. The other category of drawbacks of MCQs tests concerns issues related to the scoring rules that are used for grading the MC items of the test. The simplest grading scheme uses the “positive-grades-only” scoring rule, according to which students collect positive points for correct answers, while no losses are incurred for wrong answers or omissions. The influence of guessing in choosing an answer results in collecting some partial scores in the final score, by answering questions by chance, without possessing knowledge of the questioned material. For remedying this problem a variety of marking schemes for written MC tests have been introduced. In this perspective, “mixed-scoring” rules are used, i.e., rules in which students gain points for correct answers and lose points for incorrect answers and, in a stricter variant of the scoring rule, also for omissions (Scharf & Baldwin, 2007). These rules penalize guessing and seem to drastically reduce the gaining of scores through guesswork. On the other hand, the use of such rules have been shown to dissuade examinees from answering a question for which they might possess an intermediate or low level of knowledge, concerning especially those examinees who anticipated a high grade in

182

the examination. More generally, the behavior of examinees in MCQs tests using mixed-scoring rules might be influenced by factors that affect decision making under uncertainty. In turn, this might produce variance in the test scores that is related to the expectations of the examinees and not to the knowledge that is tested (Bereby-Meyer, Meyer, & Flascher, 2002; Bereby-Meyer, Meyer, & Budescu, 2003). Other scoring rules that have been used include the order-of-preference scheme (Bush, 1999; Wood, 1991). According to this, the MCQs used belong to the “single-best-answer” scheme, i.e., there is only one correct answer per question and examinees have to assign an order of preference to every answer for each question. The score per MC item is dictated by the preference they assign to the correct answer. Another MCQs examination variant requires the examinee to associate a level of confidence with each selected answer, so that their degree of certainty in answering is monitored (Pressley, Ghatala, Woloshyn, & Pirie, 1990; McKenzie, Wixted, Noelle, & Gyurjyan, 2001). In this kind of scoring method a wrong answer that is associated with a high degree of certainty is penalized much more than if the wrong answer was related to lack of confidence. Still another variant are the so-called “liberal” tests (Bush, 2006), which enable the choice of more than one answer per question, if the examinee is uncertain which is the correct answer. It might be assumed that the examiner might be able to test the level of knowledge of the examinee since the examinee has the possibility to select between a lot of alternative responses (Bush, 1999). Such alternative MCQs examination variants present to the person who will construct the tests the problem of formulating a suitable grading scheme, since the variability in selections that an examinee might choose when answering a question is much greater than in simple “single-best-answer” schemes. Furthermore, the use of these testing methods might distract the examinee from focusing on answering correctly the questions, when his/her confidence to the answer given is requested, or when having


to decide to select alternative answering tactics in “liberal” tests (Bush, 2006). In a previous work, an alternative MCQs examination method has been presented (Ventouras et al., 2010). According to this method, the MC items given in the test constituted pairs, referred as “paired” MCQs. MCQs in each pair concerned the same topic, but this similarity was not evident for a student who did not possess adequate knowledge on the topic addressed in the questions of the pair. The questions’ answers were graded in pairs, providing a bonus (penalty) if both (only one) answers of the pair were correctly answered. If both answers were wrongly answered no marking was collected by the examinee. The use of this scoring rule aimed at penalizing guesswork, without inducing the “hampering” effects produced by the negative marking part of the commonly used mixed-scoring schemes. In that study a paired MCQs test was given to 63 examinees, in the framework of a computer-based learning system. The same examinees took also a CRQs examination. Students were examined in a PC laboratory room, using an electronic examination platform (details of the platform are given in the following “Methodology of the Examination” section). The results of the paired MCQs examination, when using the pair-wise scoring rule, were statistically indistinguishable with the grades produced by the CRQs examination method, when made to the same sample of students, οn the same topics and with the same levels of difficult. Both the results of the paired MCQs examination, when using the pair-wise scoring rule, and the CRQs examination results differed significantly from those obtained by scoring the same MCQs using a positive-grades-only scoring rule that ignored the pairing of MCQs (referred to as the PSRMCQs examination method). Specifically, with score range (0-100), the mean of the scores of the examinees for the PSR-MCQs method was 58.25 (Standard Deviation, S.D.=18.65), while the mean for the paired MCQs and the CRQs examination methods were 45.95 (S.D.=20.28)

and 46.24 (S.D.=20.37), respectively. Another interesting finding of that study was that, when the CRQs method was compared to the PSR-MCQs method, in an another examination where no paired MCQs were used, the statistically significant difference between the scores of the CRQs and the PSR-MCQs method was greatly reduced when the set of students comprised only those students who got a score greater than 70. Specifically, the difference of the mean scores of the two methods was reduced from 12.42 (p=0.001) for the whole set of students to 5.37 (p=0.047) for the reduced students set. It seems that the higher-grade effect of the PSR-MCQs method was reduced for that set of students. A possible reason for that might be that students who got a higher score than others, were, on average, better prepared and qualified for the examination, and therefore did not apply, or greatly reduced, guessing in selecting the answers to the MCQs. Consequently, there are indications that the methodologies used for alleviating the positive grade bias induced by the positive-grades-only scoring rule, are more important for reducing the amount of students, who by guesswork could take a “pass” note (in the case of that study 50), although their level of knowledge acquisition, recall and/or reasoning capabilities was not sufficient for taking the examination with success.

COMPARISON OF CONSTRUCTEDRESPONSES QUESTIONS AND PAIRED MULTIPLECHOICE QUESTIONS Methodology of the Examination In order to expand the investigation of the comparison of the paired MCQs examination method to the CRQs examination method (taken as a gold standard) and the PSR-MCQs method, mainly concerning the effects of the parameters controlling the bonus/penalty mechanism of the scoring rule

183


of the paired MCQs method, in the examination period of February 2010, a course was selected, in order to compare the three examination methods. The examination of the course was implemented according to the following procedure.

The Examined Course and the Sample of Students The course that was selected for comparing the scores of the three examination methods was an introductory electronics course entitled “Electronic Physics”, which belongs to the group of core background courses and is taught at the Electronics Department of the School of Technological Applications of the Technological Educational Institution (T.E.I.) of Athens. The course contains four teaching units. 30 students participated in a CRQs examination and a MCQs examination. Their answers to the MCQs examination were scored according to the two different scoring variants mentioned in the Background section, i.e., the pair-wise scoring rule and the positive-gradesonly scoring rule. All students had previously been given information about the examination procedure, both in its CRQs and MCQs form and were familiarized with the electronic examination platform used in the MCQs examination, so that they would not be distracted from the presentation format and the requirements for the use of the electronic system. Additionally, they had available related study material, in both printed and electronic polymorphic form (Stergiopoulos et al., 2006a).

Formulation of the Questions and Examination Procedure The CRQs examination was taken by the students in its traditional paper-and-pencil form. The MCQs examination took place in a PC laboratory room, through the “e-examination” application developed at the Technological Educational Institution of Athens (T.E.I.-A.). The development

184

of that application is incorporated in a wider effort to further disseminate the use of PCs in the educational process through the creation of an institution-wide LMS (Triantis et al., 2007; Tsiakas et al., 2007; Stergiopoulos et al., 2006a; Stergiopoulos et al., 2006b). The LMS uses the eClass platform, developed and provided by the Greek Academic Network (GUnet Asynchronous eLearning Group, 2010), based on the Claroline open-source Course Management System (Claroline, 2010). “e-examination” is mainly a managing and editing tool that enables the teacher to create electronic assessment tests that can be used either for examining students or for self-assessment by the students. The electronic tests can be displayed by a web browser, ensuring their portability across PC platforms. The user interface consists of a series of questions, which can be either of the MC or the CR type (Stergiopoulos et al., 2006a; Triantis, Stavrakas, Tsiakas, Stergiopoulos, & Ninos, 2004). For the CRQs examination, a set of three test subjects was created. Two subjects contained 3 CRQs and one subject contained 2 CRQs. Therefore, the examinee had to answer 8 CRQs. The CRQs were short essay subject development questions, of either theoretical or exercise form. The distribution of CRQs was designed so as to cover each teaching unit proportionally. For the MCQs examination, a database was constructed containing a large number (N=400) of MCQs, covering the whole range of the subject material of the course. By using “e-examination”, a first set of MCQs {qa1, qa2, …, qak} (k=40) was randomly selected from the database, taking care to cover each teaching unit proportionally. A weight wai was assigned to each question i=1,…,k, depending on its level of difficulty. With those questions as reference, a second set of new MCQs {qb1, qb2, …, qbk} (k=40) was composed, with each question qbi possessing a similarity to question qai (i=1,…,k), constituting a pair of MCQs, according to the following rationale: a) both questions referred to the same topic and b) the knowledge of the


correct answer for question qai, from a student who had proceeded to a systematic study and was cognizant of the topic, implied the knowledge of the correct answer for qbi and vice versa. The distribution of the weights was as follows: 31 questions had weight 1, 31 questions had weight 1.5, 16 questions had weight 2 and 2 questions had weight 3. For most pairs, the questions of the pair had the same weight, i.e., it was wai=wbi. In some cases though, the weight of the two paired questions differed slightly (i.e., by 0.5). This was done because, for some topics, it was difficult to create a pair of questions that both respected the two requirements (a) and (b) stated above and were also absolutely equal in their level of difficulty. The presentation of the 2k=80 questions that the students had to answer in the PC screen was designed so that the questions were given with a random sequence, taking care that each question qbi was presented after a lapse of at least 10 questions after the presentation of question qai. Questions were automatically given through the “e-examination” software system. In the first phase of the examination, students were given the CRQs. In the second phase of the examination, the MCQs were given to students, in the PC laboratory room. After the end of the pre-determined MCQs examination duration time, a results’ page was produced for each student. The results’ page included all questions with the correct answer and the indication of whether it was correctly or wrongly answered, as well as the final score. One copy was given to the student and one to the examiner, for processing the scores. Care was taken so that the MCQs were, overall, of equivalent level of difficulty with the questions posed to the examinees in the CRQs examination. This enabled the comparison of the scores that would be achieved after the students would have given their answers in the two forms of examinations, i.e., the one using MCQs and the CRQs.

Scoring Methodology For the MCQs, 2 scores m1 and m2 were computed. Score m1 was computed as follows: for each MCQs pair i=1,…,40, the “paired” partial score pi was computed as pi = (qai wai + qbi wbi ) (1 + kbonus )

(1.a),

if both qai and qbi were correct, in which case qai=qbi=1, or pi = (qai wai + qbi wbi ) (1 − k penalty )

(1.b),

if either qai or qbi was correct, in which case it was either qai=1 and qbi=0, or qai=0 and qbi=1, or pi = 0

(1.c),

if both qai and qbi were incorrect, in which case qai=qbi=0. Parameters kbonus and kpenalty were variables that controlled the bonus and penalty mechanism, respectively, of the scoring rule. It should be noted that in the study of Ventouras et al. (2010) kbonus and kpenalty had fixed values of 0.25 and 0.5 respectively and it was wai=wbi for every i. The total score m1, with maximum value equal to 10, was then computed as: 40

m1 =

∑p i =1

40

∑ (1 + k i =1

bonus

i

⋅ 10

(2)

)(wai + wbi )

Therefore, to produce score m1, a bonus is given to the student if he/she answered correctly both questions of the MCQs pair (qai, qbi) and a penalty if he/she answered correctly only one question of the pair. Therefore, through this scoring algorithm, the final score extracted corresponds to the paired MCQs examination method.

185


Statistical Evaluation Rationale

Score m2, with maximum value equal to 10, was computed according to 40

m2 =

∑ (q

ai

i =1

wai + qbi wbi )

40

∑ (w i =1

ai

⋅ 10

(3)

+ wbi )

where qai = [

1 if the question was correctly answered 0

if the questiion was wronlgy answered

(4)

and qbi = [

1 if the question was correctly answered 0 if the questiion was wronlgy answered

(5) This method of scoring ignores any similarities existing between the questions of the pair and does not impose a penalty to the student by negative marking for incorrect answers. Therefore, through this scoring algorithm, the final score extracted corresponds to the PSR-MCQs examination method. Finally, for the CRQs examination category, each of the eight questions j=1,…,8, was graded, with minimum question grade gj being 0 and maximum grade varying from 0.4 to 2.5, according to the importance of the respective question. The sum of the maximum grades was 10. The overall CRQs examination score, m3, was extracted as the sum of the question grades: 8

m3 = ∑ gj

(6)

j =1

In all examination categories 5.0/10.0 was the minimum score required for passing the examination.

186

The comparison of the paired MCQs examination method to the CRQs examination method and the PSR-MCQs examination method has as an objective to provide indications for accepting the paired MCQs examination method as an alternative for CRQs examinations, according to the rationale stated in the Background section. Such an indication might be provided, if the scores obtained through the paired MCQs, using the pair-wise scoring method, are statistically indistinguishable from the scores obtained from the CRQs examination method. Therefore, the hypothesis H0 to be tested in comparing the paired MCQs, the PSR-MCQs and the CRQs examination methods could be stated as: “The mean of the distribution of scores m1, obtained using the paired MCQs examination method, the mean of the distribution of scores m2, obtained using the PSR-MCQs examination method, and the mean of the distribution of scores m3, obtained using the CRQs examination method, are equal”. If hypothesis H0 is rejected, i.e., the overall differences between the three means are significant, then post-hoc pair wise comparisons, with adjustment for multiple comparisons, should be used, in order to check the three “secondary” hypotheses, namely H0(paired MCQs to PSR-MCQs) (i.e., “The mean of the distribution of scores m1, obtained using the paired MCQs examination method is equal to the mean of the distribution of scores m2, obtained using the PSR-MCQs examination method”), H0(paired MCQs to CRQs) (i.e. “The mean of the distribution of scores m1, obtained using the paired MCQs examination method is equal to the mean of the distribution of scores m3, obtained using the CRQs examination method”), and H0(PSR-MCQs to CRQs) (i.e., “The mean of the distribution of scores m2, obtained using the PSR-MCQs examination method is equal to the mean of the distribution of scores m3 obtained using the CRQs examination method”).


Scoring Results and Statistical Evaluation In Table 1 the descriptive statistical values related to the three examination methods used in the examination are presented, i.e., the mean values ( m1, m 2, m 3 ) and standard deviation (S.D.) of the scores m1, m2 and m3 taken from the three methods for the whole set of students examined, and for two sub-sets of students, per examination method: for those students who passed the examination (score >= 5.0) and for those who failed the examination. Furthermore, the number and the percentage of students belonging to each sub-set is given. A value of kbonus=kpenalty=kcommon=1/3 was chosen. The Kolmogorov-Smirnov goodness-of-fit test showed that the distributions of the scores m1, m2 and m3 were each consistent with a normal distribution. The same held for the two sub-sets of students. For the whole set of students who took the examination, repeated-measures ANOVA with one within-subjects factor (method of examination, three levels) indicated that the within-subject effect was significant (F1.266,36.725=273.975, pPopup 1 text

<popup name=”popup2”>Popup 2 text

The third group uses screens like one below: <slide type=”I” title=”Test screen”> A question to answer

An example of screen from the last group is below: <slide type=”T” title=”Dictionary”>

<element name=”element1”>Explanation

bullet point 2,

of element 1

bullet point 3 with italics style.

And later bolded text

1. Author Title, Publisher ...

For the second group of screens an example was provided below:

240

<element name=”element2”>Explanation of element 2

...

As it can be seen screens have an easy to understand structure and are very informative (a typical characteristic of tagged languages). Additionally, the content may include a subset of HTML formatting tags being the most popular and important ones e.g. italics, bold, anchors, breaks or fonts. Using even a simple text editor

The Technological Advancement of LMS Systems and E-Content Software

with a syntax coloring the person editing course may understand parts of the presentation and fill slides templates. The textual course representation is approx. ten times smaller than the size of bitmap converted text. Considering the future extensions, the XMLbased representation of courses is also safer solution as this is an universal form that might be authored by various tools as well as processed by different programming languages. Even in situation when SPE will be rewritten using a new coming technologies (e.g. HTML5) it will be possible to use old XML files or update XML courses to a new versions schema via conversion scripts. This problem should be remembered by each organization deploying a new software as the data usually live longer than the software that was used to create it. Moreover, XML isn’t a proprietary format and there are several technologies extending it e.g. search language XPath or transformation language XSLT.

SPE Further Development Considering the features presented in the last section it is possible to say that SPE-based e-learning courses, incorporating different technologies, are synergic examples of a modern web application featuring the important aspect relating to the quality of web services i.e. flexibility, easy modification, universality or not limiting the future usage of generated data. However, in the future SDART Presentation Engine will be further developed by the additional features, giving SPE even more functionality. It is planned to add new screens, especially more interactive test slides allowing users to self evaluate learned knowledge. The embedded multimedia materials might be currently included in SPE as SWF files, but it is planned to introduce new tags that will be used to represent audio and video files. As SPE could be used for various e-learning tasks i.e. in the higher education, corporate education or government educational projects, SPE will be

extended by various interactive clips enlarging its interactive capabilities and engaging users more with the courses. At the moment it is possible to adopt SPE to a new visual style via replacement of background images and this feature is planned to be further extended and introduced in the configuration part of XML file. Hence, SPE will be even more easy to adapt to a new style of curses. Another important feature of new generation e-learning courses is intelligence. Therefore, SPE will use Artificial Intelligence methods to analyze the progress of learning and recommend the important materials for user e.g. materials that could be forgotten be user from the last time he/ she was learning or materials relevant to semantic search not only matching given keywords. AI will be also used to analyze users’ behavior during the usage of presentations and to improve courses by emphasizing the important parts or restructuring the courses. The experiments over adaptive elearning process were performed by (Liu, &Yang, 2005) proposing APeLS system, by (Rossi, 2009) proposing e-learning joined with Web 3.0 or using SCO approach as it was presented by (Brkovic, Milosevic, &Krneta 2006). The last possible area of further development are increased features for handicapped people. They will include a voice synthesizer and increased visual accessibility e.g. via switchable high contrast. The analysis of e-learning environment from the perspective of students with sight disabilities was done by (Drigas, Koukianakis, &Papagerasimou 2006).

CONCLUSION The technical advancement and changes in the life style increase the popularity and significance of methods and techniques of distance education. As e-learning becomes more popular also the demands and requirements relating to it are changing. The users expect courses to be tailored to their needs

241


and IT services supporting their process of learning. Therefore LMS software must be adjusted to fulfill users’ expectation. These features relate to users’ perspective, but e-learning is also very demanding activity form the business perspective. For the universities the production process of e-learning materials becomes more and more complex, time and resources consuming task, therefore it is essential to use software and technologies offering the high quality for produced materials, keeping the production process to be simple and not expensive. This chapter presented the process of evolution for LMS and e-content software used by West Pomeranian Business School. At the beginning chapter explained the requirements for different types of studies and how they influenced the shape of LMS systems. Later, the chapter analyzed different technologies and software that were used in the e-learning process. In its last part the chapter presented SPE e-content software being an innovative e-learning presentation engine developed in form of Rich Internet Application by SDART Ltd The content of each section included observations and obstacles encountered at different stages of LMS deployment and evolution. The material presented herein does not constitute the final step of LMS evolution as we started the investigation over the further areas of LMS extensions. It is planned to develop a new version of presentation engine SPE e.g. to incorporate artificial intelligence features to make the e-learning content self-adjustable. The AI features are also planned to be used in the module supporting the management of LMS, as the complexity of this process i.e. number of users, observable activities and their character, make this process hard to perform without a supporting software. This module will preprocesses the data, search for the usage patterns and deliver easily understandable comments. In our opinion the further development of Learning Management Systems should and will be done in two major directions – by using intelligent

242

mechanisms, supporting users in the process of learning, being a move towards Web 3.0 applied in e-learning; and by using technologies making e-learning even more available and convenient, moving it from the desktop computers or laptops to mobile devices. Both of these areas are investigation by ZPSB and SDART Ltd.

REFERENCES Brkovic, M., Milosevic, D., & Krneta, R. (2006). SCos for adaptive e-learning. 28th International Conference on Information Technology Interfaces (pp. 29-34). Cavtat/Dubrovnik. Drigas, A., Koukianakis, L., & Papagerasimou, Y. (2006). An e-learning environment for nontraditional students with sight disabilities. 36th Annual Frontiers in Education Conference, San Diego, CA (pp. 23-27). Dżega, D., & Pietruszkiewicz, W. (2010). The support of e-learning platform management by the extraction of activity features and clustering based observation of users. In S. Konstantopoulos, S. Perantonis, V. Karkaletsis, C. Spyropoulos, & G. Vouros (Eds.), Artificial Intelligence: Theories, Model and Applications-SETN 2010 (pp. 315320). LNAI 6040. Heidelberg, Berlin, Germany: Springer-Verlag. Horton, W. (2006). E-learning by design. San Francisco, CA: John Wiley & Sons, Inc. Liu, H., & Yang, M. (2005). QoL guaranteed adaptation and personalization in e-learning systems. IEEE Transactions on Education, 48(4), 676–687. doi:10.1109/TE.2005.858398 Pietruszkiewicz, W., & Dżega, D. (2009). The practical aspects of rich Internet application development and quality factors: RIA–based decision support system. Quality assessment in Web 2009 (Vol. 561). CEUR.


Pietruszkiewicz, W., & Dżega, D. (2010). An application of data mining in the management of e-learning platform . In Korczak, J. (Ed.), Business informatics. Data mining and business intelligence (pp. 60–70). Wrocław, PA: Publishing House of Wrocław University of Economics. Rossi, P. G. (2009). Learning environment with elements of artificial intelligence. Journal of Elearning and Knowledge Society - English Version, 5(1), 193-199. Swedish National Agency for Higher Education. (2008). E-learning quality: Aspects and criteria for evaluation of e-learning in higher education. Report 2008:11 R, PA: Swedish National Agency for Higher Education, Stockholm. Retrieved on July 30, 2010, from http://www.hsv.se/downloa d/18.8f0e4c9119e2b4a60c800028057/0811R.pdf

ADDITIONAL READING Allen, M. W. (2007). Designing successful elearning, Michael Allen’s online learning library: Forget what you know about instructional design and do something interesting. San Francisco, CA: John Willey & Sons, Inc. Berggren, A., Burgos, D., & Fontana, J. M. (2005). Practical and pedagogical issues for teacher adoption of IMS learning design standards in Moodle LMS. Journal of Interactive Media in Education, 2005(02). Bersin, J. (2004). The blended learning book: Best practices, proven methodologies, and lessons learned. San Francisco, CA: John Willey & Sons, Inc. Black, E. W., Dawson, K., & Priem, J. (2008). Data for free: Using LMS activity logs to measure community in online courses. The Internet and Higher Education, 11(2), 65–70. doi:10.1016/j. iheduc.2008.03.002

Boticario, J. G., & Santos, O. C. (2007). An open IMS-based user modelling approach for developing adaptive learning management systems. Journal of Interactive Media in Education, 2007 (2). Retrieved on August 12, 2010, from http:// citeseerx.ist.psu.edu/viewdoc/download?doi=10 .1.1.101.3799&rep=rep1&type=pdf Carliner, S., & Shank, P. (2008). The e-learning handbook: A comprehensive guide to online learning. San Francisco, CA: John Willey & Sons, Inc. Cavus, N. (2010). The evaluation of learning management systems using an artificial intelligence fuzzy logic algorithm. Advances in Engineering Software, 41(2), 248–254. doi:10.1016/j.advengsoft.2009.07.009 Clark, R. C., & Mayer, R. E. (2008). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. San Francisco, CA: John Willey & Sons, Inc. Cole, J. (2008). Using Moodle: Teaching with the popular open source course management system. Sebastopol, CA: O’Reilly Media, Inc. Ćukušić, M., Alfirević, N., & Granić, A. (2010). E-learning process management and the e-learning performance: Results of a European empirical study. Computers & Education, 55(2), 554–565. doi:10.1016/j.compedu.2010.02.017 Del Puerto Paule Ruiz, M., Jesús Fernández Díaz, M., & Ortín Sole, F. (2008). Adaptation in current e-learning systems. Computer Standards & Interfaces, 30(1-2), 62–70. doi:10.1016/j. csi.2007.07.006 Gaeta, M., Orciuoli, F., & Ritrovato, P. (2009). Advanced ontology management system for personalised e-Learning. Knowledge-Based Systems, 22(4), 292–301. doi:10.1016/j.knosys.2009.01.006

243


Hannon, J., & D’Netto, B. (2007). Cultural diversity online: student engagement with learning technologies. International Journal of Educational Management, 21(5), 418–432. doi:10.1108/09513540710760192

Romero, C., Ventura, S., & García, E. (2008). Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51(1), 368–384. doi:10.1016/j. compedu.2007.05.016

Horton, W., & Horton, K. (2003). E-learning Tools and Technologies: A consumer’s guide for trainers, teachers, educators, and instructional designers. Indianapolis: William Horton Consulting, Inc.

Rosenberg, M. J. (2001). E-Learning: Strategies for Delivering Knowledge in the Digital Age. New York, NY: McGraw-Hill.

Kritikou, Y., Demestichas, P., & Adamopoulou, E. (2008). User Profile Modeling in the context of web-based learning management systems. Journal of Network and Computer Applications, 31(4), 603–627. doi:10.1016/j.jnca.2007.11.006 Lewis, B. A., MacEntee, V. M., DeLaCruz, S., et al. (2005). Learning Management Systems Comparison. Proceedings of the 2005 Informing Science and IT Education Joint Conference. Retrieved August 02, 2010, from http://www.ultimedia. co.uk/upload/E-Learning%20and%20LMS%20 Comparisons/Learning%20Management%20 Systems%20Comparison%20P03f55Lewis.pdf Oztekin, A., Kong, Z. J., & Uysal, O. (2010). UseLearn: A novel checklist and usability evaluation method for eLearning systems by criticality metric analysis. International Journal of Industrial Ergonomics, 40(4), 455–469. doi:10.1016/j. ergon.2010.04.001 Rice, W. (2007). Moodle Teaching Techniques: Creative Ways to Use Moodle for Constructing Online Learning Solutions. Birmingham: Packt Publishing. Rice, W. (2008). Moodle 1.9 E-Learning Course Development: A complete guide to successful learning using Moodle. Birmingham: Packt Publishing.

244

Shen, R., Han, P., Yang, F., & Yang, Q. (2003). Data mining and case-based reasoning for distance learning . Journal of Distance Education Technologies, 3(1). Tang, T.Y., & McCall,a G. (2002), Student modelling for a web-based learning environment: A data mining approach, [in:] Eighteenth National Conference on Artificial Intelligence, American Association for Artificial Intelligence, Menlo Park. Watson, W. R., & Watson, S. L. (2007). An Argument for Clarity: What are Learning Management Systems, What are They Not, and What Should They Become? TechTrends, 51(2), 28–34. doi:10.1007/s11528-007-0023-y Zajac, M. (2009). Using learning styles to personalize online learning. CampusWide Information Systems, 26(3), 256–265. doi:10.1108/10650740910967410

KEY TERMS AND DEFINITIONS Blended Learning: A mixed form of learning where traditional learning is used together with e-learning form. E-Content: A set of materials containing course information and exercises. LMS: Software system used to manage the process of learning.


RIA: Web-based application with advanced interactivity and rich visual style. SCORM: A common standard of data storage used to represent universal courses in e-learning platforms.

SPE: SDART Presentation Engine being a software used to present e-content. XML: A tagged language used to represent data structures with logical meaning of its elements.

245

Section 4

Case Studies

247

Chapter 12

Differences in Internet and LMS Usage:

A Case Study in Higher Education Rosalina Babo Instituto Superior de Contabilidade e Administração do Porto, Portugal Ana Cláudia Rodrigues NID-RH, ESEIG, Portugal Carla Teixeira Lopes Faculdade de Engenharia da Universidade do Porto, Portugal Paulo Coelho de Oliveira ISEP, Portugal Ricardo Queirós KMILT, ESEIG, Portugal Mário Pinto KMILT, ESEIG, Portugal

ABSTRACT The Internet plays an important role in higher education institutions where Learning Management Systems (LMS) occupies a main role in the eLearning realm. In this chapter we aim to characterize the Internet and LMS usage patterns and their role in the largest Portuguese Polytechnic Institute. The usage patterns were analyzed in two components: characterization of Internet usage and the role of Internet and LMS in education. Using a quantitative approach, the data analysis describes the differences between gender, age and scientific fields. The carried qualitative analysis allows a better understanding of students’ both motivations, opinions and suggestions of improvement. The outcome of this work is the presentation of the Portuguese students’ profile regarding Internet and LMS usage patterns. We expect that these results can be used to select the most suitable digital pedagogical processes and tools to be adopted regarding the learning process and most adequate LMS’s policies. DOI: 10.4018/978-1-60960-884-2.ch012


Differences in Internet and LMS Usage

INTRODUCTION In recent years, Learning Management Systems (LMS) has been assumed as the cornerstone of the learning environments wide spread in the majority of the academic institutions. The central role occupied by the LMS in a typical architecture of a learning environment forces us to study its structure and its use by all users, whether they are teachers or students, in the teaching/learning process. In order to improve the LMS usage as a tool to support teaching and learning in an effective way, and since LMS are delivered by Internet, it is important to know the Internet students’ usage habits, concerning the Internet and LMS usage. This information will allow us to adopt the most appropriate learning and pedagogical strategies, according the students’ profile, characteristics and preferences. It is also relevant to know whether these habits of Internet and LMS usage are in line with the following studies presented in this section. This study aims to characterize higher education students’ of Polytechnic of Porto regarding their behavior on the Internet and eLearning platform, and it seeks to investigate how students use the information and communication technologies in their learning activities. Based on these, we propose suggestions to improve LMS usage what will be of interest to the definition of organizational policies. The motivation for this work came from the heterogeneity of students’ profiles found in the Polytechnic of Porto and the need to enhance the teachers’ pedagogical strategy based on the students’ profile. The study is focused on gender, age and scientific field. These criteria were chosen based on several studies regarding LMS and Internet and due to the variety of Polytechnic of Porto’s students characteristics. Several studies across the world have been combining the characterization and analysis of students’ Internet usage patterns and perceptions of how technologies could be used in learning at

248

university: Australia (Kennedy, Dalgarno, Sturt, Bennett, Gray, Waycott, Judd, Bishop, Maton, Krause & Chang, 2009), United Kingdom (Conole, Laat, Dillon & Darby, 2006), USA (Walker and Jorn, 2007). Since these LMS platforms present opportunities previously unavailable for academic institutions to apply their financial, logistical, and instructional resources (Uzunboylu, Ozdamli & Ozcinar, 2006), knowing the students perceptions will allow, not only to enhance Polytechnic of Porto teachers’ perceptions of what has the most impact on their students, but also, to define policies regarding eLearning platforms as a support to traditional education delivery and understand needs to be improved to undertake on distance and blended learning courses offers. The remainder of this paper is organized as follows. Section 2 traces the evolution of the eLearning systems, highlighting the LMS. In the following section we begin by describing the research methodology used for the study such as data collection and population and sample. Then, we present the data analysis and discussion. The last section focuses on the main contributions of this work, more precisely, the results of this study and a perspective of future research.

STATE OF THE ART Learning Management Systems Electronic Learning (eLearning) can be defined as the delivery of educational content via any electronic media, including the Internet, satellite broadcast, audio/video tape, interactive TV, CD-Rom and others (Tastle, White, & Shackleton, 2005). Despite some efforts to improve remote education (Harasim, 2006), the genesis of eLearning can be traced with the development of network communication in the late 1960s, more precisely, with the invention of email and computer conferencing. These innovations con-


tribute to the collaboration between teachers and students and initiate a new education paradigm shift (Williams & Goldberg, 2005). During the 1980s and 1990s, there was a significant growth in the number of students studying part-time and also in non-traditional learners, such as, typical 18 to 24 years old students seeking the university demand and women’s returning to the workforce after child rearing (Williams & Goldberg, 2005). The growth in lifelong learning has made the educational institutions to seek for flexible education delivery to satisfy these non-traditional students. By the end of the century, this delivery has been accentuated by the emergence of new forms of distance delivery based on information and communication technologies advances, such as, the Internet. In their first generation, eLearning systems were developed for a specific learning domain and had a monolithic architecture. Gradually, these systems evolved and became domain-independent, featuring reusable tools that can be effectively used virtually in any eLearning course. The systems that reach this level of maturity usually follow a component-oriented architecture in order to facilitate tool integration (Leal & Queirós, 2010). Different kinds of component based eLearning systems target specific aspects of eLearning, such as student or course management. One such case is the LMS that aims to simplify the management of learning within an organization (Harman & Koohang, 2007).This type of system allows students to plan their learning and collaboration activities with colleagues, while teachers may associate educational content and monitor, analyze and report progress of their students. Most LMS’s are structured around courses rather than courses’ content thus, they only support reusability at the course level, where many learners can enroll on a single course. LMS also don’t support the creation of instructional content. This “issue” implies the use of third part content creation tools. Despite these issues, the LMS are the cornerstone of the learning environments wide spread in the

majority of the academic institutions. Typically, they have two types of users’ groups: learners and teachers. Learners can use the LMS to plan their learning experience and to collaborate with their colleagues; teachers can deliver educational content and track, analyze and report the learner evolution within an organization.

LMS in Higher Education Higher education institutions have been challenged by the emergence of new information and communication technologies (ICT), in the last few years. Among various technologies, eLearning and LMS have been widely adopted at higher education, motivating researchers and teachers to study its influence in the learning process (e.g. Misko, Choi, & Lee, 2004; Gras, 2005). In fact, Mayadas (2007) states that in 2006 there were 3,5 million students enrolled in at least one online course in United States (US); this was about 20% of all university students in the US. Since then until now, online learning is been growing 20% a year. Today, New York University enrolls almost 130.000 online students in LMS platforms (NYU, 2010). A vast number of LMS that provide integrated services exists nowadays, like for instance, WebCT, Blackboard, Centra, Learning Space and Moodle (Botturi, Cantoni, & Tardini, 2006; Babo & Azevedo, 2009). Within these eLearning systems, Moodle assumes an important role (Babo & Azevedo, 2009). It is an open source solution, offers a set of scalable tools and services that support communication and collaboration across students and teachers and there are a lot of components that we can add to the platform (Georgouli, Skalkidis, & Gerreiro, 2008). In early January of 2010, Moodle had a userbase of 46,624 registered sites with 32,464,992 users in 3,161,291 courses in 209 countries and in more than 75 languages (Moodle, 2010). According to some authors (Monsakul, 2007; Moodle, 2010), the following are the most common func-

249


tions of LMS’s that we can find in Moodle: i) course information and documentation; ii) document repository; iii) announcements; iv) discussion board; v) external links; vi) synchronous and asynchronous communication (email, chat room, discussion forum); vii) assignments; ix) users profiles. There is also some plug-ins that we can add to Moodle, in order to install other functionalities like, for instance, hot potatoes and Sharable Content Object Reference Model (SCORM) features. According to Landsberger (2004), LMS’s relevant features in higher education are about: i) courseorganizing functions, such as electronic and multimedia contents, evaluation and assessment tools; ii) dynamic tools to facilitate the learning process, such as synchronous and asynchronous discussion groups; iii) student’s environments to enhance collaboration and cooperation between them. With its various functionalities, LMS’s serve different learner’s characteristics, different learning styles and outcomes (Landsberger, 2004; Monsakul, 2007). To understand all these changes and to see how students are dealing with them, several studies were made to characterize Internet usage patterns and LMS implementations in higher education: e.g. testing for a variety of strategies in an eLearning platform (Oliveira, Oliveira, Souza, & Costa, 2006); other findings point the need to LMS support solving problems and critical thinking functionalities (Andronico, Carbonaro, Colazzo, Molinari, Ronchetti & Trifonova, 2004; Monsakul, 2007); still others conclude that teaching and learning pedagogy should be modified to the new student’s typical profile, which includes the capacity of learning anytime, anywhere, across multi-platforms and with multi-tasking (Kubora, Terashima, Nakahashi, & Morioka, 2008). Botturi, Cantoni, &Tardini (2006) conducted a survey at University of Lugano in order to evaluate students’ usage and satisfaction with eLearning platform, Moodle. The survey indicates that 94,3% of the respondents is satisfied with the platform usage, but only communication forums

250

and online submission have been widely used by students and teachers. On the other hand, only 44% of the respondent noticed positives changes in their learning process after the adoption of Moodle platform. Several studies point to different Internet usage habits, depending on the criteria discussed: male students predominate in the use of Internet; female students prefer contents based on science learning environment while male tend to prefer distracting contents; the Internet usage increases by more 10 percentage points in each age group between 2006 and 2009; the number of messages sent and received is clearly greater in men than women (Kubora et al., 2008). Introducing gender as a criterion in studies in the ICT fields consistently show a male bias towards Internet use (Kubora et al., 2008; Madell & Muncer, 2004; Nachmias, Mioduser, & Shemla, 2001; Sink, Stob & Taniguchi, 2008). And, especially in an education setting and, in order to get equity among graduates facing the labor market, gender is an issue that can’t be dismissed (Madell & Muncer, 2004). Age is another criterion that is very much considered in ICT studies. In recent years there has been widespread interest in studying young people’s relation with ICT. Tapscott (1999) introduced, in the late 90’s, the term Net generation to characterize a whole new population born after 1980 that grew in an environment where technological artifacts and digital culture were a part of their everyday life. The computer, Internet, cellular phone, digital camera, digital games and social networks are, among others, examples of how present is digital technology in this generation’s world, in such a way that these are not viewed as new technologies but as tools that have always been there (Prensky, 2001). This generation tends to work and learn, unlike traditional approach, developing some particular characteristics: multitasking, visual learning, hypertext, short messages, and compact information, among others (Kennedy et al., 2009; Simões, 2009). Burlea (2008) found in her study, that the older students


(from 46-64 years old) found more difficulties of individually using the facilities offered by the information system. For example, Kennedy et al. (2009) found slight differences between generations of students and teaching staff in their study, as they said a little unexpectedly. As Polytechnic of Porto is known for having a large percentage of students who are (or has been in any moment of their lives) professionals (as opposed to students that are continuing their studies after high school without going through the labor market), and these students are older, it was found to be interesting to study the behavior and perceptions of different generation students, in the same learning context, towards Internet and LMS platform supporting face-to-face classes. In the study promoted by The Australian Learning and Teaching Council (Kennedy et al. 2009) they considered “a key aspect of the Investigation stage” (p.17) to analyze the discipline in which they were studying: students were asked to nominate the course and subject in which they were responding to the questionnaire and these were used to classify students into the discipline categories, according to the Australian Standard Classification of Education Codes. Their assumption was to consider how the students’ experiences with technology might differ as a function of the field they were studying. They did not found relevant statistical differences between discipline areas, this is, they reported around the same level of technology use in all five discipline areas (Arts, Sciences or others). However, clear differences between the three universities students’ engaged in the study (Kennedy et al. 2009). In Polytechnic of Porto there are also very different study areas and, there is the common sense perception and, to some extent, empirical evidence, that students that engaged in Informatics courses tend to use LMS more than others, since their teachers do it (Rodrigues, Pinto, Queirós, 2010). Rodrigues et al. (2010) studied Moodle sophistication usage, using Janossy & Hover’s (2008) model in one of Polytechnic of Porto schools (ESEIG1) by teach-

ers and, systematically found that Informatics department teachers used Moodle more and in a more complex manner than teachers from other scientific fields.

Research Methodology To characterize Internet and LMS usage patterns and their role in Higher Education, a descriptive case study with students from Polytechnic of Porto was conducted. In Polytechnic of Porto, all courses are delivered in presential classes and students must enroll in a minimum of classes (except for the working students that have a special authorization). LMS (Moodle) is used as a support to presential classes and, it is not mandatory. The students’ usage patterns were analyzed in two components: Internet usage and LMS in Education. Internet usage patterns is characterized by type (e.g.: location, frequency, type of use, motivation) and by the underneath communication tools (e.g.: social networks, chat, email in frequency, activities, benefits). The role of LMS in education deals with items such as: types of functionalities, their frequency of use and their perceived importance.

Data Collection Data was collected through an online questionnaire distributed to students of five Polytechnic of Porto’s schools: ESEIG2, ESTSP3, ESTGF4, ISCAP5, and ISEP6. This instrument was chosen because it allows the collection of a large number of answers at a reasonable cost. The questionnaire had twenty-seven questions, five of which were open. In the first week of June of 2009 a pilot test was carried out with 15 students from various programs and academic years of ESEIG. This test validated the questionnaire’s objectivity, understanding and also the web form’s accessibility. In the second week of June 2009 began the dissemination of the questionnaire, in the

251


various schools of Polytechnic of Porto. The announcement was the same in all schools and was done by: announcements on the school website, announcements on the school learning management system, email messages sent to students and teachers (so they would ask their students to answer the questionnaire) and requests to teachers, who used computers in classrooms to free class time so that students could complete the questionnaire. The questionnaire answers were collected anonymously, from June 5th until the end of July. In July, the schools’ academic services were contacted in order to gather descriptive data about the population.

Population and Sample Similar to what is done in other studies (e.g. Walker & Jorn, 2007), the case study methodology was adopted for this study. Students from five Polytechnic of Porto’s schools compose the universe of this study. The study included students from technological specialization programs and also from undergraduate and graduate programs. From the 1416 obtained answers, 1397 were considered valid (11% of Polytechnic of Porto’s population).

Figure 1. Sample distribution across schools

252

In Figure 1, it is visible the respondents distribution according to the school where they study. According to the Ministry of Science, Technology and Higher Education (DGES, 2010), Portugal has ten study fields: Science (SCI); Health (HEA); Technology (TEC); Agriculture and Natural Resources; Architecture, Arts and Design (AAD); Education; Law and Social sciences (LSS); Economics (ECO); Humanities (HUM); Physical Education, Sports and Show’s Arts. In Figure 2 it is possible to see the distribution of the population and the sample by scientific field. As observed the majority of the students (population and sample) attend courses in the field of Technology (TEC-51%), Economics (ECO-23%), Heath (HEA-13%) and Law and Social Sciences (LSS-9%) and so, the analysis by scientific fields will be based on these main fields. Most of the students are undergraduate (94.6%), 5.2% are graduate students and students from technological specialization programs are 0.2% of this sample. Concerning gender, 51% of students are female and 49% male. As expected, in the population there are more students born in 1980 and after (86%) than born until 1980 (14%)7. However, the percentage of older students (over 29 years old) is large because


Figure 2. Sample distribution across scientific fields

Polytechnic of Porto’s schools also host working students, which are older than ordinary students. The proportion of working students in the population is 25%, 32% of which answered the survey. The student’s sample regarding open questions analysis was defined considering all students who answered all the open questions (n=121). The sample consisted of 65 female students (53,7%) and 56 male students (46,3%). Considering age groups, there are 102 students (84,3%) born in or after 1980 and 19 (15,7%) born until 1980. Finally considering the scientific fields there are 26 students (21,5%) from Health, 52 students (42,1%) from Technology, 13 students (10,7%) from Law and Social Sciences, 27 students (22,3%) from Economics and 3 students (1,7%) that didn’t answered what was the scientific field that they belonged.

DATA ANALYSIS The data analysis was carried out on two major components: characterization of Internet usage, and the role of Internet and LMS in education. In both cases, we have started by a descriptive analysis of the sample and, then, we studied

which differences were statistically significant through hypothesis tests. Our analysis considered three students features: age, gender and scientific field. In the first two, defined only by two groups, we have used the proportion test using the Chisquare distribution in nominal variables and the Mann-Whitney test in ordinal variables. In the scientific field feature, we have decided to select only the four most representative fields, namely Health, Technology, Law and Social Sciences, and Economics. To study the differences between scientific fields, we have used the Chi-square test on nominal variables. On ordinal variables, we have applied the Kruskal-Wallis test to detect differences between the 4 fields and, when differences were found, we have applied the Mann-Whitney test between pairs of scientific fields adjusting the significance level to the number of tests performed. The open questions analysis was done using QSR NVivo 7. To analyze the significant differences found in NVivo7, we applied the proportion hypothesis test in age and gender, and the Chisquare test in the scientific field.

253


Internet Usage Due to the nature of data this section was divided into two sub-sections: Type of Internet usage and Communication tools usage.

Type of Internet usage The majority of students access the Internet at home (91%) and at school (80%). Only 24% say they have mobile Internet access. Most students access the Internet several times a day (65%), 32% connect daily and the remaining 3% connect monthly, weekly and even more rarely. On average, 51% of students are connected to the Internet between 1 and 3 hours per day (Rodrigues et al., 2010). We found that male students stay connected for longer periods of time (p=0.013) (Babo, Lopes, Rodrigues, Pinto & Oliveira 2010a). Students born in 1980 and afterwards stay connected for longer periods of time (w=103902, p=0.000). An explanation for this factor could be the professional duties of the older students, which prevent them to a more lasting usage (Babo, Lopes, Rodrigues, Pinto, Queirós & Oliveira, 2010b). Analyzing by scientific field we found that students enrolled in Technology degrees stay connected for longer periods of time than Health (w=64028, p=0.019) and Economics (w=143266, p=0.000) students. On average, 48% of the time spent on the Internet is for personal leisure (the mode is 50%). The most used Internet tools are email (95%), search engines (92%), instant messaging (58%) and social networks (52%). Students’ gender is related to the usage of some Internet tools. Male students statistically use more instant messaging (p=0.007), forums (p=0.009), games (p=0.000) and wikis (p=0.000) and female students, statistically, use more social networks (p=0.000) and search engines (p=0.039) (Babo et al., 2010b). Students’ generation is also related with the use of Internet tools. Younger students use more email (p=0.000), search engines (p=0.003),

254

forums (p=0.000), instant messaging (p=0.000), social networks (p=0.000), wikis (p=0.000), games (p=0.000). Only blogs are used similarly by male and female in both students’ generations (Babo et al., 2010a). Scientific field is also statistically related with the usage of email (χ2=52.12, p=0.000), search engines (χ2=33.67, p=0.000), forums (χ2=9.13, p=0.000), blogs (χ2=9.13, p=0.028), instant messaging (χ2=38.61, p=0.000), social networks (χ2=48.41, p=0.000), wikis (χ2=54.70, p=0.000) and games (χ2=14.77, p=0.002). A descriptive analysis shows that health students tend to use more search engines, instant messaging and blogs than students from other scientific fields. Technology students use more email, forums, games and wikis and students from Law and Social Sciences use more social networks than students from other scientific fields. According to Table 1, presented below, the main reasons for Internet usage are research concerning class works/study (94%), accessing documents in Moodle or another LMS (77%) and email exchange (77%) (Rodrigues et al., 2010). An overview of Internet usage’s main reasons by gender (Babo et al., 2010b) shows that female students statistically use the Internet more than male students to research concerning class work/study, email, connect with friends and access friends’ social networks. Male students use theZ Internet more than female students to read newspapers, magazines and portals, download music or films, play games, use forums (post or read) and shopping. In general, it is noticeable that female students use the Internet to study, and to socialize with others. While male students are more engaged in leisure activities and enrolled in activities by themselves that do not depend on other people (Babo et al., 2010b). With the two independent samples t-test, it was also verified that youngest students use more Internet based tools (p=0.000) than students born before 1980 (Babo et al., 2010a).


Table 1. Gender, age and scientific field differences regarding Internet usage’s main reasons Gender

Internet Usage’s Main Reasons

Total

Researching concerning class works/ study

94%

Accessing document in Moodle or another LMS

77%

Email

77%

Connecting with friends

72%

Reading newspapers, magazines and portals

52%

Downloading music or films

50%

Searching info (other than school issues)

47%

Sharing information (documents, music, films, etc.)

28%

Accessing friends’ social networks

24%

Gaming

19%

Using forums (post or read)

17%

Shopping

12%

Using Blogs (post or visit others)

9%

Searching new friends/aquaintances

6%

Age

F

M

96%

90%

Older (Born1980)

HEA

TEC

LSS

ECO

97%

92%

97%

93%

p≅0.000

p≅0.008 85%

73%

65%

80%

p≅0.001 82%

73%

87%

p≅0.000 78%

66%

p≅0.000 43%

62%

p≅0.000 44%

57%

70%

83%

72%

63%

84%

70%

35%

56%

p≅0.000

p≅0.000

76%

64%

58%

49%

p≅0.000 55%

52%

57 %

p≅0.000 56%

80%

p≅0.000 49%

17%

80% p≅0.001

p≅0.000 68%

72%

p≅0.000

p≅0.000 8%

82%

43 %

40 %

p≅0.000 42%

p≅0.000 25%

3% p≅0.000 30%

18%

14%

24%

6%

17%

14%

22%

p≅0.000 7%

27%

p≅0.000

27%

14%

18%

10%

11%

p≅0.000 9%

2%

39%

p≅0.019 15%

2%

p≅0.000

9%

17%

p≅0.000 17%

1%

p≅0.000 7%

27%

p≅0.000

p≅0.000 31%

21%

2%

p≅0.000

p≅0.000

4%

16%

8%

12%

p≅0.000

10%

p=0.028

(The absence of p-values means there are no significant differences between the groups.)

Data overview shows that both generations of students use the Internet for different reasons (Babo et al., 2010a). Younger students use it for contacting with friends, download music and movies, share information (documents, music, movies, etc.), visit friend’s webpage in social

networks, play games, and participate in forums. Older students use the Internet mainly for visiting the eLearning platform to look for new documents, email, read the news in newspapers, magazines and portals, to get information about various themes not connected with their studies,

255


and to shop. Non working students, namely, the younger students at IPP, do not shop (shop in a lesser extent) because they cannot afford it (Kwak, Fox & Zinkhan, 2002). In general this descriptive analysis shows that, on one hand, younger students are more enrolled in Web 2.0’s activities like socialization and sharing. On the other hand, students born before 1980 engage in activities typical of the Web 1.0 using the Internet as a mean of getting information, more than to interact with others, to share information, namely personal information (Babo et al., 2010a; Kennedy, 2009). Through Table 1 we can see that almost all reasons for Internet use are statistically associated with the scientific field. The exceptions are: searching for information not related to school subjects, sharing information, using blogs and searching for new friends. The trend shows that health students use the Internet more than other students to: research concerning class work/study, email and connect with friends. Technology students use more the Internet to download music and movies, play games, use forums and shopping. Law and Social Sciences students use more the Internet to: research concerning class work/study, visit the eLearning platform to look for new documents, read the news in newspapers, magazines and portals and accessing friends’ social networks.

Communication Tools Usage In general, 74% of Polytechnic of Porto students have a profile in a social network. Of these, 91% have a profile in HI58, followed by Facebook (23%). About 33% of the students use social networks to make new friends. Nevertheless, there is a statistically significant (p=0.002) difference between gender: 78% of the female students have a profile in a social network, while only 70% of the male students have such a profile (Babo et al., 2010b). Regarding generation there are also differences: 77% of the younger students have a profile in a social network. The importance

256

of social networking is minor among the older generation: only 54% have a profile in a social network. Finally, the possession of a profile in a social network is also related with the scientific field of the student (χ2=18.67, df=3, p=0.000). More than 74% of the students from Health, Law and Social Sciences and Economics degrees have a profile in a social network while 69% of Technology students’ have such a profile. Table 2 shows the most engaged activities in social networks by the students: viewing photos (47%), knowing what my friends are doing/what is happening (36%) and commenting (35%) are the most reported activities in social networks. An overview of the data (Babo et al., 2010b) and Table 2 above, shows that female students are, with statistical evidence, the ones most engaged in activities developed in social networks: view photos, try to find out what their contacts are doing, make comments, talk with other people and share information (photos, videos, music). Male students admit more than female students that they meet new people, searching for new friends and acquaintances. Either, male students are meeting other male students or, female students are less frank about their online activity, another explanation could be, female students are not actively trying to meet new people, thus do not acknowledge this activity. Among younger students, it is observed a more frequent use of social networks. In social networks, the younger students are statistically different from the older in what regards the activities: see photos (p≅0.000), find what their contacts are doing (p≅0.000), do comments (p≅0.000), talk with other people (p=0.003) and, share information (p≅0.000): photos, films, music. Older students statistically update their CV (p≅0.000) more than the younger generation and, search for interesting information other than school subjects (p≅0.000). Through Table 2 we can see that almost all the activities in social networks use are statistically associated with the scientific field. The exceptions are: search for information not related to school


Table 2.Gender, age and scientific field differences regarding most engaged activities in social networks Gender

Activities most developed in Social Networks

Total

Viewing photos

47%

Knowing what my friends are doing/what is happening

36%

Commenting

35%

Age

F

M

Older (Born1980)

HEA

TEC

LSS

ECO

54%

39%

22%

51%

60%

42%

55%

45%

p≅0.000 41%

31%

26%

77%

56%

p≅0.000 44%

33%

Chatting with other people

31%

Sharing information (documents, photos, videos, music)

24%

Meeting new people (search new friends/ aquaintences)

15%

Informing my friends what I am doing

15%

Searching for information about companines

9%

Updating CV

8%

Setting personal meeting with network contacs

3%

Sending messages

0%

p≅0.000

p≅0.000 44%

32%

p≅0.000

p≅0.000

44%

35%

p≅0.000 39%

16%

49%

28%

p≅0.000 18%

Finding interesting things(search information – other than school issues)

Scientific Fields

43%

34%

p≅0.000 8%

p≅0.000 35%

27%

29%

28%

p≅0.000 30%

19%

19%

39%

p≅0.004 26%

14%

p≅0.000 10%

34%

34%

20%

p≅0.000

p≅0.000

22%

p≅0.000 32%

24%

33%

10%

17%

p≅0.008

8%

19%

p≅0.004 32%

28% p≅0.000

17%

7% p≅0.000

5%

8%

12%

10%

p≅0.040 34%

27% p≅0.033

The absence of p-values means there are no significant differences between the groups

subjects, search for information about companies, setting personal meetings and sending messages. The trend shows that health students use more the social networks to: view photos, try to find out what their contacts are doing, make comments and share information. The Law and Social Sciences students use the social networks to: try to find out what their contacts are doing and update their CV. Finally, Economics students use the social networks to talk with other people and try to meet new friends.

In order to connect synchronously, 98% of the students use MSN while Skype and Google Talk have small shares (15% and 11%, respectively). However, male students are more diverse in their tools’ choice (Google Talk, p=0.001 and Skype, p=0.021). Females are much more loyal to MSN hardly using any other synchronous tool (p=0.028). Data shows (Babo et al., 2010b) that 28% of the students use instant messaging several times a day and 34% use it at least daily. Male students are more frequent users (p=0.004).

257


Instant messaging is used by 28% of the students several times a day and 34% use it at least daily. In what refers to differences between generations, younger students are more frequent users (p=0.000). Email is used more than instant messaging. According to data (Babo et al., 2010b), 52% of students use email several times a day and 39% use it at least once a day. Even though there are not statistical gender differences regarding whether students use email, when it comes to the frequency of email usage male students use email more often (p≅0.000): 59% of male students use it several times a day against 44% of the female students. In terms of scientific field we found that, regarding to the use of email, there are some statistical differences between the several fields considered. Health students use it less than Technology students (w=5718, p=0.000) and less than Law and Social Sciences students (w=13623, p=0.001). Economics students use it less than Technology students (w=116093, p=0.000) and less than Law and Social Sciences students (w=27952, p=0.001). From the data collected we can state that the majority of students use the email several times a day with special importance to Technology (58,3%) and Law and Social Sciences (57,3%) students. Unlike the observed with the results of Internet’s session time, social networks and instant messaging usage, older students do a larger use of email (71% use it several times a day against 48% of the youngest ones; p≅0.000). This might be happening due to the fact that older students usually are also workers and can have email connected all day allowing them to check it several times a day.

Role of Internet and LMS in Education In this section we will analyze the use of the Internet and LMS and their role in education. A descriptive analysis of the sample will be made and significant statistical differences in gender, age

258

group and scientific field will be reported. Table 3 analysis the statistical differences encountered by gender and age group in: a) the students’ access LMS platform; b) Students’ evaluation of usability of the LMS platform and c) Students’ evaluation of the importance of the LMS platforms in their learning process. Analyzing Table 3, by gender, male students visit more often the LMS platform than female students (65% of male students visit the LMS platform in a daily basis against 57% of female students), a difference that is statistically significant (p=0.006). It is important to notice there are no statistical differences between the two age groups regarding the frequency of visits to LMS platforms. When it comes to usability it is important to say that no gender or age statistically significant differences were found related to the difficulty in the use of LMS platforms. Analyzing the importance of LMS platforms in students’ learning process it is important to refer that more than 77% of the students consider LMS platforms to have an important role in their learning process. If we associate this result with the fact that most students access LMS platforms, at least once a day, in order to help students’ learning teachers should improve and promote the use of these platforms. In terms of age group we found that older students feel that this kind of platforms are more important to their learning process than younger students, a difference that is statistically significant (p=0.008). This result is consistent with the fact that most of the older students are employed, attend fewer classes and need the LMS platforms to be updated, inquire teachers and colleagues and access documents on the contents covered in classes. No differences were found in terms of gender. The same analysis was made concerning scientific fields (Table 4). Through Table 4 we can see/observe that the major differences encountered are between Health students and students from other scientific fields.


Table 3. Statistical differences by gender and age group regarding students’ access to LMS platform, students’ evaluation of LMS usability’s platform, and the importance of those platforms to students’ learning process Access

< p-0.006

Usability

NDF

Importance

NDF

Born >1980 Born P=0.008

(NDF equals Non Differences Found. The left operands of the signs are the row header. The right operands are the column header.)

In terms of access to LMS platforms Health students have the lowest access to these platforms what is consistent with the fact that Health students have more difficulties when using LMS platforms (only 63% of them say that the use of the LMS platform is easy against more that 78% of the other scientific fields). This last result is statistically significant since the Wilcoxon test shows Health students find it harder to use than Technology students (w=51506, p=0.000), than Law and

Social Sciences students (w=11299, p=0.000) and Economics students (w= 30598, p=0.000). Finally, and in concordance with the previous results, it is important to say that only 59% of Health students say the LMS platform is important to the learning process against more than 79% for students from other scientific fields. This result is statistically significant since the Wilcoxon test shows Health students find it less important than Technology students (w=47211, p=0.000), Law

Table 4. Statistical differences by scientific field in students’ access to LMS platforms and evaluation concerning the usability of LMS platforms and the importance of those platforms learning process HEA Health

Technology

Law & Social Sciences

TEC

LSS

ECO

Access

< w=43761 p=0.000

< w=11348 p=0.000

< w=26457 p=0.000

Usability

< w=51506 p=0.000

< w=11299 p=0.000

< w=30598 p=0.000

Importance

< w=47211 p=0.000

< w=9300 p=0.000

< w=26772 p=0.000

Access

NDF

NDF

Usability

NDF

NDF

Importance

< w=35959 p=0.004

NDF

Access

NDF

Usability

NDF

Importance

> w=26923 P=0.005

259


and Social Sciences students (w=9300, p=0.000) and Economics students (w=26772, p=0.000). According to the students in this study, the tools most used by teachers for educational purposes are: email (88%), Search Engines (36%) and Forums (23%), as illustrated in Table 5. Analyzing the students’ response we can find significant differences in their perceptions about the tools used by their teachers. Only the use of search engines by teachers is not associated with the scientific field. The proportion of students stating that the teachers use the email is larger in health students. This can explain why health students are not so supporters of LMS platforms as students from other scientific fields. As stated by students, search engines are the second most used tool by their teachers. Considering gender, female students, have the perception that their teachers use search engines more than male students (40% against 33%, p=0.008). Tools such as forums (25% against 21%, χ2=3.2482, df=1, p=0.036), blogs (9% against 6%, p=0.032) and Wikis (11% against 7%, p=0.008) are, in male students’ opinion, more used by their teachers.

The trend shows that forums’ are more used by Technology students’ teachers and wikis and blogs are more used by Law and Social Sciences field teachers. These results may be related to the tools that students themselves say they use more frequently: email, search engines and Forums (Babo et al., 2010b). No statistical differences were found in age groups. Regarding the open question where students were asked about what can be done to facilitate or enhance their learning regarding LMS platforms, five dimensions emerged from the qualitative analysis: a) to be used by all teachers (27%); b) update the content available (25%); c) to use a wide variety of functionalities (16%); d) to be used to increase the interaction between teachers and students (9%); e) make easier their access and use (9%) (Table 6). Describing results by gender we can state that female students find that the two most important actions to be performed in the LMS platform to facilitate their learning process are its use by all teachers (40%) and content’s update (27%). For male students the two most important actions are

Table 5. Internet’s tools used by teachers for school purposes Gender

Internet tools used by teachers

Total

Email

88%

Search engines

36%

forums

23%

wikis

9%

F

M

89%

87%

86%

HEA

88%

95%

LSS

ECO

89%

80%

83%

33%

34%

37%

43%

36%

37%

32%

21%

23%

7%

28%

20%

26%

χ2=5.71,df=1 P=0.008 25%

χ =3.25,df=1 p=0.036 2

7%

11%

χ2=45.30,df=3,p=0.000 7%

9%

1%

χ2=5.90,df=1 P=0.008 6%

9%

13%

18%

4%

χ2=57.09,df=1,p=0.000 11%

7%

χ =3.43,df=1 p=0.031 2

(The absence of p-values means there are no significant differences between the groups.)

260

TEC

χ =27.85, df=3, p=0.000

21%

8%

Scientific Fields

Younger Born >1980)

2

40%

blogs

Age Older (Born

Higher Education Institutions and Learning Management Systems: Adoption and Standardization

Learning, Curriculum and Employability in Higher Education

Online Learning and Teaching in Higher Education

Culture and Language Learning in Higher Education

Culture and Language Learning in Higher Education

ADHD and Higher Education

Entrepreneurship and higher education

Handbook of Experiential Learning and Management Education

Handbook of Experiential Learning and Management Education

Hybrid Learning and Education

Computer-Supported Collaborative Learning in Higher Education

Learning to Lead in Higher Education

Understanding learning and teaching: the experience in higher education

Computer-Supported Collaborative Learning in Higher Education

Higher Learning

Learning Together: Peer Tutoring in Higher Education

Learning to Teach in Higher Education

Authentic Learning Environments In Higher Education

Assessment Learning and Employability (Society for Research Into Higher Education)

Applied E-Learning and E-Teaching in Higher Education

e-Learning and Social Networking Handbook: Resources for Higher Education

Combining Service and Learning in Higher Education, Summary Edition

Learning and Teaching across Cultures in Higher Education