Learning in the Synergy of Multiple Disciplines: 4th European Conference on Technology Enhanced Learning, EC-TEL 2009 Nice, France, September ... Programming and Software Engineering)

Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen

Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen University of Dortmund, Germany Madhu Sudan Microsoft Research, Cambridge, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max-Planck Institute of Computer Science, Saarbruecken, Germany

5794

Ulrike Cress Vania Dimitrova Marcus Specht (Eds.)

Learning in the Synergy of Multiple Disciplines 4th European Conference on Technology Enhanced Learning, EC-TEL 2009 Nice, France, September 29–October 2, 2009 Proceedings

13

Volume Editors Ulrike Cress Knowledge Media Research Center (KMRC) Konrad-Adenauer-Str. 40, 72072 Tübingen, Germany E-mail: [email protected] Vania Dimitrova University of Leeds School of Computing Knowledge Representation and Reasoning Research Group E.C. Stoner Building, Leeds LS2 9JT, UK E-mail: [email protected] Marcus Specht Open University of the Netherlands Centre for Learning Sciences and Technologies (CELSTEC) Valkenburgerweg 177, 6419 AT Heerlen, The Netherlands E-mail: [email protected]

Library of Congress Control Number: 2009934787 CR Subject Classification (1998): I.2.6, K.3.2, H.5.3, J.1, J.5, K.4 LNCS Sublibrary: SL 2 – Programming and Software Engineering ISSN ISBN-10 ISBN-13

0302-9743 3-642-04635-5 Springer Berlin Heidelberg New York 978-3-642-04635-3 Springer Berlin Heidelberg New York

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. springer.com © Springer-Verlag Berlin Heidelberg 2009 Printed in Germany Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper SPIN: 12766000 06/3180 543210

Preface

This conference on technology enhanced learning is the fourth event in a series that started in 2006. It was held from September 29th to October 2nd, 2009 in Nice (France). The EC-TEL conference series provides a forum for presenting and promoting high-quality research in the area of technology enhanced learning. The EC-TEL conference was originally launched by the European network of excellence ProLearn and attracted many people from both the ProLearn and Kaleidoscope networks of excellence. In 2009, a new European network, STELLAR, was launched, which continues the work and success of the former networks and takes a broader multi-disciplinary perspective. A key issue is making the research communities aware of the different projects and activities within Europe and beyond. The aim is to build an integrated research arena in which groups with different backgrounds can build on each other and where the synergy between multiple research approaches and disciplines is fostered. The face of learning is changing substantially. As a result, the topic of technology enhanced learning has to take a broader interdisciplinary perspective. Formal learning is surrounded by a variety of opportunities for informal learning, classroom learning is complemented by workplace learning, and even the frontiers between teaching and learning are disappearing. People are learning collaboratively, they engage in knowledge communities and change from knowledge recipients to knowledge producers. These developments are driven by new technologies: large scale knowledge repositories provide learners with content and support them in an individualized and adaptive way; semantic technologies provide contextualized and task-specific information; the Web 2.0 enables people to participate actively in knowledge communication and knowledge construction, mobile and ubiquitous computing technologies enable the integration of informal and formal learning support. These new tools and technical means call for psychological and educational models of learning, which will have to take into account the vast diversity of situations in which learning takes place today, as well as the specific needs of individuals, tutors and organisations. The papers submitted to this conference reflect this broad range of topics. A total of 25% of all submissions used the keyword “user-adaptive systems and personalisation”, which has been a typical topic of advanced learning environments for many years. The keywords “learning communities and communities of practice” and “collaborative knowledge building” were used by 23% of the submissions. These topics indicate a new perspective on learning and a drift from formal to more informal and natural learning. This tendency is also evident in the strong presence of the keywords “informal learning”, “learner motivation and engagement”, “problem and project-based learning”, “distance learning”, “knowledge management and organisational learning”, and “instruction design”.

VI

Preface

One fifth of the submissions exploited the newly emerging technological directions of “semantic web and Web 2.0”. The EC-TEL 2009 was truly international and highly competitive. Overall, 136 paper submissions and 22 poster submissions from 469 authors in 43 countries were received. The majority of submissions came from European countries (29 countries), but authors also came from 8 Asian and 4 American countries, as well as one African country. One submission was received from Australia. Program Committee members, coming from 19 countries, represented a broad spectrum of disciplines connected to technology enhanced learning. A rigorous review process was conducted where each submission was reviewed by at least three reviewers. Out of all submissions, 35 were accepted as full papers (22%), 17 as short papers and a further 35 as posters. In the proceedings, the full papers are allowed up to 15 pages and the short papers and posters up to 6 pages. The conference programme included three keynote speakers who gave an idea of the wide range of technology enhanced learning. Short abstracts of the keynote talks are included in the proceedings. The contributions presented in this volume show the colourfulness of research in technology enhanced learning. They describe technical innovations, demonstrate creative educational settings, invent exciting research questions and show successful implementations. We are confident that this spectrum of research will promote creativity and synergy. A conference of this size would not have been possible without the invaluable help of the organising committee: the workshop chairs Nikol Rummel and Peter Dolog, the doctoral consortium chairs Frank Fischer and Stefanie Lindstaedt, the demonstration chairs Alexandra Cristea and Nikos Karacapilidis, and the industrial session chair Volker Zimmermann. Special thanks go to the head of the local organizing team Katherine Maillet, as well as the publicity chairs Marcela Morales and Mohamed Amine Chatti. The EC-TEL 2009 conference promises to be a stimulating research event, presenting state-of-the-art projects and shaping the future of technology enhanced research in Europe and beyond. September 2009

Ulrike Cress Vania Dimitrova Marcus Specht

Conference Organisation

General Chair Marcus Specht

Centre for Learning Sciences and Technology, OUNL, NL

Programme Chairs Ulrike Cress Vania Dimitrova

Knowledge Media Research Center, Germany University of Leeds, UK

Local Organisation Chair Katherine Maillet

Institut T´el´ecom, Telecom & Management SudParis, France

Doctoral Consortium Chairs Frank Fischer Stefanie Lindstaedt

LMU University of Munich, Germany Know Center, Austria

Workshop Chairs Nikol Rummel Peter Dolog

University of Freiburg, Germany Aalborg University, Denmark

Demonstration Chairs Alexandra Cristea Nikos Karacapilidis

University of Warwick, UK University of Patras, Greece

Industrial Session Chair Volker Zimmermann

IMC, Germany

Publicity Chairs Marcela Morales

Institut T´el´ecom, Telecom & Management SudParis, France Mohamed Amine Chatti RWTH Aachen University, Germany

VIII

Organisation

Programme Committee Heidrun Allert , Austria Katrin Allmendinger, Germany Inmaculada Arnedillo-Sanchez, Ireland Nicolas Balacheff, France Maria Bielikova, Slovakia Zuzana Bizonova , Slovakia Bert Bredeweg, The Netherlands Peter Brusilovsky, USA Daniel Burgos, Spain Manuel Caeiro, France Lorenzo Cantoni, Switzerland Alexandra Cristea, UK Valentin Cristea, Romania Paul de Bra, The Netherlands Carlos Delgado Kloos, Spain Elisabeth Delozanne, France Pierre Dillenbroug, Switzerland Yannis Dimitriadis, Spain Peter Dolog, Denmark Benedict du Boulay, UK Eric Duval, Belgium Dieter Euler, Switzerland Christine Ferraris, France Adina Magda Florea, Romania Dragan Gasevic, Canada Andreas Gegenfurtner, Finland Denis Gillet, Switzerland Monique Grandbastien, France Jorg Haake, Germany Paivi Hakkinen, Finland Peng Han, Germany Andreas Harrer, Germany Christoph Held, Germany Marek Hatala, Canada Eelco Herder, Germany Knut Hinkelmann, Switzerland Ulrich Hoppe, Germany Patrick Jermann, Switzerland Nikos Karacapilidis, Greece Michael D. Kickmeier-Rust, Austria Barbara Kieslinger, Austria David Kirsh, USA Ralf Klamma, Germany Tomaz Klobucar, Slovenia

Rob Koper, The Netherlands Nicole Kraemer, Germany Milos Kravcik, The Netherlands Effie Law, Switzerland Lydia Lau, UK Martin Lea, UK Stefanie Lindstaedt, Austria Andreas Lingnau, Germany Chee-Kit Looi, Singapore Rose Luckin, UK George Magoulas, UK Katherine Maillet, France Alejandra Mart´ınez, Spain Vittorio Midoro, Italy Tanja Mitrovic, New Zealand Riichiro Mizoguchi, Japan Paola Monachesi, The Netherlands Wolfgang Nejdl, Germany Roger Nkambou, Canada Lucia Pannese, Italy Jan Pawlowski, Finland Juan Quemada, Spain Christoph Richter, Austria Uwe Riss, Germany Nikol Rummel, Germany Maggi Savin-Baden, UK Tammy Schellens, Belgium Daniel Schneider, Switzerland Judith Schoonenboom, The Netherlands Peter Scott, UK Evgenia Sendova, Bulgaria Mike Sharples, UK Kiril Simov, Bulgaria Peter Sloep, The Netherlands Pierre Tchounikine, France Stefan Trausan-Matu, Romania Julita Vassileva, Canada Vincent Wade, Ireland Armin Weinberger, The Netherlands Katrin Wodzicki, Germany Martin Wolpers, Belgium Volker Zimmermann, Germany

Organisation

Additional Reviewers Stamatina Anastopoulou Benjamin Huynh Kim Bang Michal Barla Scott Bateman Elizabeth Brown Roman Brun Wenli Chen Manuela Delfino Hendrik Drachsler Mar´ıa Blanca Ib´ an ˜ ez Espiga Raquel M. Crespo Garc´ıa George Gkotsis Israel Gutierrez Zoe Handley Yusuke Hayashi I-Han Hsiao Eva Hudlicka Raija H¨ am¨al¨ ainen Nikos Karousos Sebastian Kelle Tom Kirkham Styliani Kleanthous Kouji Kozaki Barbara Kump

Danielle H. Lee Vignollet Laurence Derick Leony Sarah Lewthwaite Tobias Ley David Maroto Sze Ho David Moh Vlad Posea Francesca Pozzi Andreas S. Rath Traian Rebedea Riad Saba Olga C. Santos Hans-Christian Schmitz Stefano Tardini Jozef Tvarozek Manolis Tzagarakis Elizabeth Uruchurtu Luis de la Fuente Valent´ın Dominique Verpoorten Juan Quemada Vives Michael Yudelson Sam Zeini Sabrina Ziebarth

IX

Table of Contents

Keynotes Making Sense of Sensemaking in the Digital World . . . . . . . . . . . . . . . . . . . Peter Pirolli

1

Towards an Interdisciplinary Design Science of Learning . . . . . . . . . . . . . . Mike Sharples

3

Use and Acquisition of Externalized Knowledge . . . . . . . . . . . . . . . . . . . . . . Friedrich W. Hesse

5

Adaptation and Personalisation LAG 2.0: Refining a Reusable Adaptation Language and Improving on Its Authoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alexandra I. Cristea, David Smits, Jon Bevan, and Maurice Hendrix

7

The Conceptual and Architectural Design of a System Supporting Exploratory Learning of Mathematics Generalisation . . . . . . . . . . . . . . . . . Darren Pearce and Alexandra Poulovassilis

22

Experience Structuring Factors Affecting Learning in Family Visits to Museums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Marek Hatala, Karen Tanenbaum, Ron Wakkary, Kevin Muise, Bardia Mohabbati, Greg Corness, Jim Budd, and Tom Loughin

37

Personalisation of Learning in Virtual Learning Environments . . . . . . . . . Dominique Verpoorten, Christian Glahn, Milos Kravcik, Stefaan Ternier, and Marcus Specht

52

A New Framework for Dynamic Adaptations and Actions . . . . . . . . . . . . . Carsten Ullrich, Tianxiang Lu, and Erica Melis

67

Getting to Know Your User – Unobtrusive User Model Maintenance within Work-Integrated Learning Environments . . . . . . . . . . . . . . . . . . . . . . Stefanie N. Lindstaedt, G¨ unter Beham, Barbara Kump, and Tobias Ley

73

Adaptive Navigation Support for Parameterized Questions in Object-Oriented Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I-Han Hsiao, Sergey Sosnovsky, and Peter Brusilovsky

88

Automated Educational Course Metadata Generation Based on Semantics Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ˇ Mari´ an Simko and M´ aria Bielikov´ a

99

XII

Table of Contents

Searching for “People Like Me” in a Lifelong Learning System . . . . . . . . . Nicolas Van Labeke, George D. Magoulas, and Alexandra Poulovassilis

106

Interoperability, Semantic Web, Web 2.0 Metadata in Architecture Education - First Evaluation Results of the MACE System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Martin Wolpers, Martin Memmel, and Alberto Giretti Phantom Tasks and Invisible Rubric: The Challenges of Remixing Learning Objects in the Wild . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . David E. Millard, Yvonne Howard, Patrick McSweeney, Miguel Arrebola, Kate Borthwick, and Stavroula Varella Can Educators Develop Ontologies Using Ontology Extraction Tools: An End-User Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Marek Hatala, Dragan Gaˇsevi´c, Melody Siadaty, Jelena Jovanovi´c, and Carlo Torniai Sharing Distributed Resources in LearnWeb2.0 . . . . . . . . . . . . . . . . . . . . . . Fabian Abel, Ivana Marenzi, Wolfgang Nejdl, and Sergej Zerr SWeMoF: A Semantic Framework to Discover Patterns in Learning Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Marco Kalz, Niels Beekman, Anton Karsten, Diederik Oudshoorn, Peter Van Rosmalen, Jan Van Bruggen, and Rob Koper

112

127

140

154

160

Data Mining and Social Networks Social Network Analysis of 45,000 Schools: A Case Study of Technology Enhanced Learning in Europe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ruth Breuer, Ralf Klamma, Yiwei Cao, and Riina Vuorikari

166

Analysis of Weblog-Based Facilitation of a Fully Online Cross-Cultural Collaborative Learning Course . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Anh Vu Nguyen-Ngoc and Effie Lai-Chong Law

181

Sharing Corpora and Tools to Improve Interaction Analysis . . . . . . . . . . . Christophe Reffay and Marie-Laure Betbeder

196

Collaboration and Social Knowledge Construction Distributed Awareness for Class Orchestration . . . . . . . . . . . . . . . . . . . . . . . Hamed S. Alavi, Pierre Dillenbourg, and Frederic Kaplan

211

Table of Contents

XIII

Remote Hands-On Experience: Distributed Collaboration with Augmented Reality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Matthias Krauß, Kai Riege, Marcus Winter, and Lyn Pemberton

226

A Comparison of Paper-Based and Online Annotations in the Workplace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ricardo Kawase, Eelco Herder, and Wolfgang Nejdl

240

Learning by Foraging: The Impact of Social Tags on Knowledge Acquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Christoph Held and Ulrike Cress

254

Assessing Collaboration Quality in Synchronous CSCL Problem-Solving Activities: Adaptation and Empirical Evaluation of a Rating Scheme . . . Georgios Kahrimanis, Anne Meier, Irene-Angelica Chounta, Eleni Voyiatzaki, Hans Spada, Nikol Rummel, and Nikolaos Avouris

267

Learning Communities and Communities of Practice Facilitate On-Line Teacher Know-How Transfer Using Knowledge Capitalization and Case Based Reasoning . . . . . . . . . . . . . . . . . . . . . . . . . . . Celine Quenu-Joiron and Thierry Condamines Edushare, a Step beyond Learning Platforms . . . . . . . . . . . . . . . . . . . . . . . . Romain Sauvain and Nicolas Szilas

273 283

Design in Use of Services and Scenarios to Support Learning in Communities of Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bernadette Charlier and Amaury Daele

298

Creating an Innovative Palette of Services for Communities of Practice with Participatory Design: Outcomes of the European Project PALETTE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Liliane Esnault, Amaury Daele, Romain Zeiliger, and Bernadette Charlier

304

Learning Contexts NetLearn: Social Network Analysis and Visualizations for Learning . . . . . Mohamed Amine Chatti, Matthias Jarke, Theresia Devi Indriasari, and Marcus Specht

310

Bridging Formal and Informal Learning – A Case Study on Students’ Perceptions of the Use of Social Networking Tools . . . . . . . . . . . . . . . . . . . . Margarida Lucas and Ant´ onio Moreira

325

How to Get Proper Profiles? A Psychological Perspective on Social Networking Sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Katrin Wodzicki, Eva Schw¨ ammlein, and Ulrike Cress

338

XIV

Table of Contents

Collaborative Learning in Virtual Classroom Scenarios . . . . . . . . . . . . . . . . Katrin Allmendinger, Fabian Kempf, and Karin Hamann

344

Review of Learning in Online Networks and Communities . . . . . . . . . . . . . Kirsti Ala-Mutka, Yves Punie, and Anusca Ferrari

350

Self-profiling of Competences for the Digital Media Industry: An Exploratory Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Svenja Schr¨ oder, Sabrina Ziebarth, Nils Malzahn, and H. Ulrich Hoppe

365

PPdesigner: An Editor for Pedagogical Procedures . . . . . . . . . . . . . . . . . . . Christian Martel, Laurence Vignollet, Christine Ferraris, Emmanuelle Villiot-Leclercq, and Salim Ouari

379

Ontology Enrichment with Social Tags for eLearning . . . . . . . . . . . . . . . . . Paola Monachesi, Thomas Markus, and Eelco Mossel

385

Problem and Project-Based Learning, Inquiry Learning How Much Assistance Is Helpful to Students in Discovery Learning? . . . . Alexander Borek, Bruce M. McLaren, Michael Karabinos, and David Yaron A Fruitful Meeting of a Pedagogical Method and a Collaborative Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B´en´edicte Talon, Dominique Leclet, Gr´egory Bourguin, and Arnaud Lewandowski

391

405

A Model of Retrospective Reflection in Project Based Learning Utilizing Historical Data in Collaborative Tools . . . . . . . . . . . . . . . . . . . . . . Birgit R. Krogstie

418

Fortress or Demi-Paradise? Implementing and Evaluating Problem-Based Learning in an Immersive World . . . . . . . . . . . . . . . . . . . . . Maggi Savin-Baden

433

Project-Based Collaborative Learning Environment with Context-Aware Educational Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zoran Jeremi´c, Jelena Jovanovi´c, Dragan Gaˇsevi´c, and Marek Hatala

441

Learning Design Constructing and Evaluating a Description Template for Teaching Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Michael Derntl, Susanne Neumann, and Petra Oberhuemer

447

Table of Contents

XV

Model and Tool to Clarify Intentions and Strategies in Learning Scenarios Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Val´erie Emin, Jean-Philippe Pernin, and Viviane Gu´eraud

462

Users in the Driver’s Seat: A New Approach to Classifying Teaching Methods in a University Repository . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Susanne Neumann, Petra Oberhuemer, and Rob Koper

477

Motivation, Engagement, Learning Games Generating Educational Interactive Stories in Computer Role-Playing Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Marko Div´eky and M´ aria Bielikov´ a CAMera for PLE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hans-Christian Schmitz, Maren Scheffel, Martin Friedrich, Marco Jahn, Katja Niemann, and Martin Wolpers Implementation and Evaluation of a Tool for Setting Goals in Self-regulated Learning with Web Resources . . . . . . . . . . . . . . . . . . . . . . . . . Philipp Scholl, Bastian F. Benz, Doreen B¨ ohnstedt, Christoph Rensing, Bernhard Schmitz, and Ralf Steinmetz The Impact of Prompting in Technology-Enhanced Learning as Moderated by Students’ Motivation and Metacognitive Skills . . . . . . . . . . Pantelis M. Papadopoulos, Stavros N. Demetriadis, and Ioannis G. Stamelos Creating a Natural Environment for Synergy of Disciplines . . . . . . . . . . . . Evgenia Sendova, Pavel Boytchev, Eliza Stefanova, Nikolina Nikolova, and Eugenia Kovatcheva

492 507

521

535

549

Human Factors and Evaluation Informing the Design of Intelligent Support for ELE by Communication Capacity Tapering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Manolis Mavrikis and Sergio Gutierrez-Santos

556

Automatic Analysis Assistant for Studies of Computer-Supported Human Interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Christophe Courtin and St´ephane Talbot

572

Real Walking in Virtual Learning Environments: Beyond the Advantage of Naturalness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Matthias Heintz

584

Guiding Learners in Learning Management Systems through Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Olga C. Santos and Jesus G. Boticario

596

XVI

Table of Contents

Supervising Distant Simulation-Based Practical Work: Environment and Experimentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Viviane Gu´eraud, Anne Lejeune, Jean-Michel Adam, Michel Dubois, and Nadine Mandran

602

Posters Designing Failure to Encourage Success: Productive Failure in a Multi-user Virtual Environment to Solve Complex Problems . . . . . . . . . . . Shannon Kennedy-Clark Revisions of the Split-Attention Effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Athanasios Mazarakis Grid Service-Based Benchmarking Tool for Computer Architecture Courses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Carlos Alario-Hoyos, Eduardo G´ omez-S´ anchez, Miguel L. Bote-Lorenzo, Guillermo Vega-Gorgojo, and Juan I. Asensio-P´erez Supporting Virtual Reality in an Adaptive Web-Based Learning Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Olga De Troyer, Frederic Kleinermann, Bram Pellens, and Ahmed Ewais A Model to Manage Learner’s Motivation: A Use-Case for an Academic Schooling Intelligent Assistant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tri Duc Tran, Christophe Marsala, Bernadette Bouchon-Meunier, and Georges-Marie Putois Supporting the Learning Dimension of Knowledge Work . . . . . . . . . . . . . . Stefanie N. Lindstaedt, Mario Aehnelt, and Robert de Hoog User-Adaptive Recommendation Techniques in Repositories of Learning Objects: Combining Long-Term and Short-Term Learning Goals . . . . . . . Almudena Ruiz-Iniesta, Guillermo Jim´enez-D´ıaz, and Mercedes G´ omez-Albarr´ an

609 615

621

627

633

639

645

Great Is the Enemy of Good: Is Perfecting Specific Courses Harmful to Global Curricula Performances? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Maura Cerioli and Marina Ribaudo

651

Evolution of Professional Ethics Courses from Web Supported Learning towards E-Learning 2.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Katerina Zdravkova, Mirjana Ivanovi´c, and Zoran Putnik

657

Towards an Ontology for Supporting Communities of Practice of E-Learning “CoPEs”: A Conceptual Model . . . . . . . . . . . . . . . . . . . . . . . . . . Lamia Berkani and Azeddine Chikh

664

Table of Contents

Using Collaborative Techniques in Virtual Learning Communities . . . . . . Francesca Pozzi Capturing Individual and Institutional Change: Exploring Horizontal versus Vertical Transitions in Technology-Rich Environments . . . . . . . . . . Andreas Gegenfurtner, Markus Nivala, Roger S¨ alj¨ o, and Erno Lehtinen A Platform Based on Semantic Web and Web2.0 as Organizational Learning Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adeline Leblanc and Marie-H´el`ene Abel Erroneous Examples: A Preliminary Investigation into Learning Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimitra Tsovaltzi, Erica Melis, Bruce M. McLaren, Michael Dietrich, Georgi Goguadze, and Ann-Kristin Meyer Towards a Theory of Socio-technical Interactions . . . . . . . . . . . . . . . . . . . . . Ravi K. Vatrapu Knowledge Maturing in the Semantic MediaWiki: A Design Study in Career Guidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nicolas Weber, Karin Schoefegger, Jenny Bimrose, Tobias Ley, Stefanie Lindstaedt, Alan Brown, and Sally-Anne Barnes Internet Self-efficacy and Behavior in Integrating the Internet into Instruction: A Study of Vocational High School Teachers in Taiwan . . . . Hsiu-Ling Chen Computer-Supported WebQuests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Furio Belgiorno, Delfina Malandrino, Ilaria Manno, Giuseppina Palmieri, and Vittorio Scarano A 3D History Class: A New Perspective for the Use of Computer Based Technology in History Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Claudio Tosatto and Marco Gribaudo Language-Driven, Technology-Enhanced Instructional Systems Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Iv´ an Mart´ınez-Ortiz, Jos´e-Luis Sierra, and Baltasar Fern´ andez-Manj´ on

XVII

670

676

682

688

694

700

706 712

719

725

The Influence of Coalition Formation on Idea Selection in Dispersed Teams: A Game Theoretic Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rory L.L. Sie, Marlies Bitter-Rijpkema, and Peter B. Sloep

732

How to Support the Specification of Observation Needs by Instructional Designers: A Learning-Scenario-Centered Approach . . . . . . . . . . . . . . . . . . Boubekeur Zendagui

738

XVIII

Table of Contents

Using Third Party Services to Adapt Learning Material: A Case Study with Google Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Luis de la Fuente Valent´ın, Abelardo Pardo, and Carlos Delgado Kloos

744

Virtual Worlds for Organization Learning and Communities of Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C. Candace Chou

751

A Methodology and Framework for the Semi-automatic Assembly of Learning Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Katrien Verbert, David Wiley, and Erik Duval

757

Search and Composition of Learning Objects in a Visual Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Amel Bouzeghoub, Marie Buffat, Alda Lopes Gan¸carski, Claire Lecocq, Abir Benjemaa, Mouna Selmi, and Katherine Maillet A Framework to Author Educational Interactions for Geographical Web Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Nhan Luong, Thierry Nodenot, Philippe Lopist´eguy, and Christophe Marquesuza` a Temporal Online Interactions Using Social Network Analysis . . . . . . . . . . ´ Alvaro Figueira Context-Aware Combination of Adapted User Profiles for Interchange of Knowledge between Peers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sergio Gutierrez-Santos, Mario Mu˜ noz-Organero, Abelardo Pardo, and Carlos Delgado Kloos ReMashed – Recommendations for Mash-Up Personal Learning Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hendrik Drachsler, Dries Pecceu, Tanja Arts, Edwin Hutten, Lloyd Rutledge, Peter van Rosmalen, Hans Hummel, and Rob Koper Hanse 1380 - A Learning Game for the German Maritime Museum . . . . . Walter Jenner and Leonardo Moura de Ara´ ujo

763

769

776

782

788

794

A Linguistic Intelligent System for Technology Enhanced Learning in Vocational Training – The ILLU Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . Christoph R¨ osener

800

e3 -Portfolio – Supporting and Assessing Project-Based Learning in Higher Education via E-Portfolios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Philip Meyer, Thomas Sporer, and Johannes Metscher

806

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

811

Making Sense of Sensemaking in the Digital World Peter Pirolli Palo Alto Research Center 3333 Coyote Hill Road, Palo Alto, CA, USA [email protected]

In this keynote presentation I discuss some of the exciting phenomena and challenges that are emerging as the digital universe evolves to become a more social medium that supports more complex information-seeking and learning activities. This discussion emerges from attempts to extend previous work on Information Foraging Theory [1] to address these new trends in online information-seeking and sensemaking. Information Foraging Theory is a theory of human-information interaction that aims to explain and predict how people will best shape themselves to their information environments, and how information environments can best be shaped to people. The theory has mainly focused on information seeking by the solitary user, but as the Internet and Web have evolved, so too must the theory, and so I will discuss recent studies of sensemaking and the social production, sharing, and use of information in areas such as wikis, social tagging, social network sites, and social search. The opportunity (and challenges) are enormous for developing a scientific foundation to support online groups and communities that are engaged in creating, organizing, and sharing the knowledge produced through social sensemaking. Sensemaking is a natural kind of human activity in which large amounts of information about a situation or topic are collected and deliberated upon to form an understanding that becomes the basis for problem solving and action. It goes beyond simply finding information. It is also involved in learning about new domains, solving ill-structured problems, acquiring situation awareness, and participating in social exchanges of knowledge. Sensemaking involves collecting, organizing and creating representations of complex information sets, all centered on the formation and support of mental models involved in understanding a problem that needs to be solved. Examples of such problems include understanding a health problem to make a medical decision, understanding the weather to make a forecast, intelligence analysis to identify strategic threats, and the collaborative collection and understanding of an emergency by first responders. Seminal papers on this topic emerged quasi-independently in the fields of human-computer interaction [2], organizational science [3], and macrocognition [4]. Making sense of challenging domains of knowledge using the Internet has become a ubiquitous activity in the digital era. For those who have access, the Internet has become the primary resource for learning about science, technology, health and medicine, and current events [5]. As the information environment has become richer, it has become a place to explore and learn over longer periods of time. The Internet and the Web have also become much more social [6] with a variety of technologies to exploit or enhance social information foraging. The Web, blogs, email, Internet groups, U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 1 – 2, 2009. © Springer-Verlag Berlin Heidelberg 2009

2

P. Pirolli

collaborative tagging, wikis, recommender systems, and other technologies are all aimed at supporting cooperative information sharing and their success implies their effectiveness. The utility of such systems typically depends on having large user bases and higher rates of contribution by individuals. With respect to sensemaking, the utility of such sites additionally depends on such factors as how readily people can judge the credibility of the sources and authors of user-generated content, how knowledge produced by one individual transfers to another, and how well specific tools support content learning. In this presentation, I will discuss research addressing some of these needs. I will also discuss some social phenomena that arise from many interacting users including: the effects of diversity and social brokerage, the standing-onthe-shoulders-of-giants effect, the effects of social interference, and the role of user interface interaction costs. Given the increased ease with which it is possible to study social networks and information flow in the electronic world, it is likely that there will be more studies of the effects of technologies on social structure and social capital, hence a need for a suitable theoretical framework. The efflorescence of online social interaction and collective action raises fundamental questions about the conditions and interaction architectures that shape the social and cognitive machinery of people. We need a theoretical framework that is rich and encompassing enough to provide practical guidance on how to design online communities across the space of possible purposes and activities. The framework must be rich and complex enough to produce integrated models that support (a) decomposition of macroscale phenomena down to microscale mechanisms that are (b) relevant to the understanding and design of online communities that evolve over months to years and encompass large numbers of people and (c) predict accurately the effects and tradeoffs of design decisions made at levels ranging from moment-by-moment user interaction to long-term social dynamics. Whether models are developed in agent-based simulations, dynamical systems, or some other approach, there is great opportunity to integrate a new unified theoretical framework.

References 1. 2.

3. 4. 5.

6.

Pirolli, P.: Information foraging theory: A theory of adaptive interaction with information. Oxford University Press, New York (2007) Russell, D.M., Stefik, M.J., Pirolli, P., Card, S.K.: The cost structure of sensemaking. In: INTERCHI 1993 Conference on Human Factors in Computing Systems. Association for Computing Machinery, Amsterdam (1993) Weick, K.: Sensemaking in organizations. Sage, Thousand Oaks (1995) Klein, G., Moon, B., Hoffman, R.R.: Making sense of sensemaking 2: A macrocognitive model. IEEE Intelligent Systems 21(5), 88–92 (2006) Horrigan, J.: The Internet as a resource for news and information about science. Pew Internet & American Life project (November 20, 2006), http://www.pewinternet.org/Reports/2006/ The-Internet-as-a-Resource-for-News-and-Information-aboutScience.aspx (cited June 27, 2009) Lenhart, A.: Adults and social network websites. Pew Internet & American Life Project (January 14, 2009), http://www.pewinternet.org/Reports/2009/Adultsand-Social-Network-Websites.aspx (cited June 27, 2009)

Towards an Interdisciplinary Design Science of Learning Mike Sharples University of Nottingham, Jubilee Campus, Wollaton Road Nottingham NG8 1BB, UK [email protected]

In a world of increasing complexity, confronting global environmental and social challenges, there is an urgent need to enable people of all ages to learn about themselves, their society and their environment. Yet, there is a surprising lack of attention to what this involves. The study of human learning does not form a major part of teacher education programmes and is disappearing from university Psychology courses. It is as if human learning is just too diffuse and difficult a topic to be studied and taught. A central problem with the study of learning is that it is inherently interdisciplinary. Learning as the process of effecting permanent changes to the brain is an aspect of neuroscience; as the acquisition of skills and knowledge, learning forms part of cognitive psychology; as an activity of social and cultural development, it falls under social sciences; as a process of systemic adaptation to societal changes it could be part of history, business or economics. All of these disciplines are essentially descriptive, in that they attempt to understand people and their world. To enable people to learn more effectively also involves the disciplines of design and engineering. Such complexity has traditionally been simplified, so that researchers can understand or influence one aspect, such as change in behaviour, cognitive development, or the design of teaching machines. The time has now come to put all the pieces together, to form a composite picture of how we learn as individuals, groups and societies, and how to create the conditions for more effective learning, across contexts, throughout a lifetime. If this seems like a daunting task, then much of the groundwork has been or is being done. In addition to studying facets of learning, we need to develop new methods to integrate this knowledge and to harness it for the benefit of learners and society. The suggestion is to extend educational psychology and learning science research towards a design science of large complex systems. Such an enterprise needs be international, to build on expertise across many research centres. It should be crosscultural, respecting and celebrating the diversity of settings and approaches to learning. It needs to be design-based if it is to not only describe how learning is currently achieved, but also to develop new methods for enabling and supporting productive learning. It must embrace multiple technologies, including digital media, traditional media and human knowledge, not just as resources for learning, but as integral parts of a complex learning system. It needs to be multi-level and multi-method, seeking to integrate the neural, cognitive, social and cultural aspects of learning. Methods for design and evaluation of human-technology systems, such as socio-cognitive U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 3–4, 2009. © Springer-Verlag Berlin Heidelberg 2009

4

M. Sharples

engineering [1], can provide a basis of complex systems design, and these need to be complemented with design-oriented theories of technology-enabled learning. Some immediate consequences of such an agenda are that this cannot be done be one researcher, or one lab, alone. Just as the Human Genome project required a cooperation of many research labs, a long timescale, a shared infrastructure and ethical framework, and a common set of tools, so the development of an Interdisciplinary Design Science of Learning needs a shared effort to integrate facilities for the co-design of technology-enabled learning and cross-cultural studies of learning effectiveness. Such studies are already underway. For example, the Group Scribbles technology developed at SRI (http://groupscribbles.sri.com/) and the Eduinnnova method from Pontificia Universidad Católica de Chile (http://www.eduinnova.com/english/) are being developed and tested across multiple sites in a worldwide collaboration. The Kaleidoscope and ProLearn networks have already made substantial advances towards forming a cross-national infrastructure and shared understanding for research in technology-enabled learning. The STELLAR network is ideally placed to take on this challenge.

Reference 1.

Sharples, M., Jeffery, N., du Boulay, J.B.H., Teather, D., Teather, B., du Boulay, G.H.: Socio-cognitive engineering: a methodology for the design of human-centred technology. European Journal of Operational Research 136(2), 310–323 (2002)

Use and Acquisition of Externalized Knowledge Friedrich W. Hesse Knowledge Media Research Center at the University of Tuebingen Konrad-Adenauer-Str. 40, 72072 Tuebingen [email protected]

Knowledge acquisition is no longer mainly restricted to classical institutions and formal learning (as in schools and universities) but is also connected to informal learning settings at home in leisure time or at the workplace. Thus, the interplay between formal and informal learning is developing in a new way, mainly in connection with the development of Web 2.0 and the appearance of “social software”. Within these new social software environments different developments are especially interesting, as they offer new ways of learning, knowledge building and use of knowledge. A very special feature has to do with the possibility of externalizing knowledge. Even more, social software (e.g. bookmarking) makes not only externalized knowledge available, but together with the externalized knowledge of other people, resources can be created which are most meaningful for oneself. For cognitive psychologists and learning researchers, social software offers an interesting new demand for further study. Since the beginning of learning research, one can observe some paradigmatic changes which have had a strong impact on which learning processes have been investigated. In the very beginning, research was interested in the process of learning with regard to the manipulation of observable and measurable behavior, for example in learning by heart by Ebbinghaus [1], in classical conditioning by Pavlov [2] or in the highly influential operant conditioning by Skinner [3]. Around 1960 there was a paradigm shift from “learning” to “knowledge”, the so called “cognitive turn” (Neisser [4]). From then on researchers were interested in investigating the internal mental processes, like organization, acquisition, storing and retrieval of knowledge (e.g. Baddeley [5]). This led to a new type of theory and new results. A more recent paradigm shift moved interests from “knowledge” to “externalized knowledge”. Wegner [6] introduced the theory of transactional memory, where people don’t have to know everything themselves, but can use the knowledge of other people. Connected to some of the ideas of Wegner, a lot of developments and activities around Web 2.0 in the years from 2000 on allowed researchers to follow his perspective, especially in connection with features like having quick and easy storage of and access to “(externalized) knowledge”. However, from a research perspective we only understand partly the nature and mechanism of these activities. They are mostly related to tools which are associated with terms like “Social Networks” and “Social Software Tools”. Using a wider scope, such tools can be categorized into at least three groups: those which are primarily concerned with the social exchange between people U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 5–6, 2009. © Springer-Verlag Berlin Heidelberg 2009

6

F.W. Hesse

(like Facebook), those which also address a knowledge exchange (like bookmarking systems) and those which are mainly interested in constructing shared knowledge bases (like Wikipedia). When we take a closer look at the category “knowledge exchange” and especially at the bookmarking systems, we will discover in detail the potential of this social software tool in taking over processes which normally have to be carried out by ourselves, so that there is a new division of labor between the human cognitive system and the social software tools. How is this possible? The processes behind the bookmarking are mainly based on tags which allow all users to assign keywords individually to information or resources (e.g. picture, website, videos). These tags can help to structure, classify and filter individual collections of information and resources. These resources can – at the same time – be saved and filled in different categories. Thus information storage and retrieval is becoming very easy. But there is still the question, what is “social” in social tagging? On the one hand, the individual tags for respective resources are available for all users. On the other hand, all tags can be created by all other users and then aggregated. For a concrete resource this leads to a common description/classification in a bottom-up process, reflecting important connotations and concepts of the resource. In addition, frequently used tags of a resource are weighted more strongly. The whole tagging system additionally makes creating related tags possible. Such tags allow discovering comparable (related) terms and links. Related tags can be used as links in navigations and further search processes. By means of related tags, people with a similar interest or specific expertise can also be identified. Thus related tags have a very special potential for knowledge building, because to a certain degree semantic interpretations are carried out by these tools. Such developments, of course, do enrich our possibilities in the use of knowledge and even making shared knowledge available. But they also lead us to question to which extent these processes are understood by us and whether we are able to control such automatically carried out processes.

References 1. 2. 3. 4. 5. 6.

Ebbinghaus, H.: Über das Gedächtnis. Untersuchungen zur experimentellen Psychologie. Duncker & Humblot, Leipzig (1885) Pavlov, I.P.: Die Arbeit der Verdauungsdrüsen. St. Petersburg (1897) Skinner, B.F.: A discrimination without previous conditioning. Proceedings of the National Academy of Sciences of the United States of America 20, 532–536 (1934) Neisser, U.: Cognitive psychology. Prentice-Hall, Englewood Cliffs (1967) Baddeley, A., Hitch, G.J.: Working memory. In: Bower, G.A. (ed.) Recent advances in learning and motivation, vol. 8, pp. 47–90. Academic Press, New York (1974) Wegner, D.: Transactive memory: contemporary analysis of the group mind. In: Mullen, B., Goethals, G. (eds.) Theories of Group Behavior, pp. 185–208. Springer, New York (1987)

LAG 2.0: Refining a Reusable Adaptation Language and Improving on Its Authoring Alexandra I. Cristea1, David Smits2, Jon Bevan1, and Maurice Hendrix1 1

Department of Computer Science, University of Warwick, Coventry, CV4 7AL, United Kingdom {A.I.Cristea,J.D.Bevan}@warwick.ac.uk 2 Faculty of Mathematics and Computer Science, Eindhoven University of Technology PB 513, 5600MB, Eindhoven, The Netherlands [email protected]

Abstract. Reusable adaptation specifications for adaptive behaviour has come to the forefront of adaptive research recently, with EU projects such as GRAPPLE1, and PhD research efforts on designing an adaptation language for learning style specification [1]. However, this was not the case five years ago, when an adaptation language for adaptive hypermedia (LAG) was first proposed. This paper describes the general lessons learnt during the last five years in designing, implementing and using an adaptation language, as well as the changes that the language has undergone in order to better fulfil its goal of combining a high level of semantics with simplicity, portability as well as being flexible. Besides discussing these changes based on some sample strategies, this paper also presents a novel authoring environment for the programming-savvy adaptation author, that applies feedback accumulated during various evaluation sessions with the previous set of tools, and its first evaluation with programming experts. Keywords: Adaptive Hypermedia, Adaptation Language, LAG, LAOS.

1 Introduction Adaptation and personalization are considered to be both useful and desirable, and came to the fore with user modelling [2] and adaptive hypermedia [3] research. However, adaptive environments are notoriously difficult to author [4] for. Amongst all the components in adaptive environments, about which much has been modelled and written [5][6][7][8][9][10][1], the most difficult part is the specification (authoring) of the adaptive behaviour [8][12] [13][14][1]. Hence, reusability is desirable especially for the adaptive behaviour specification, in the sense of ‘write once, use many’ [12]. Since 2003-2004 a few adaptation languages have been proposed; LAG [15] is, as far as we know, the first such language, followed by LAG-XLS [16] that caters for Learning Styles. Ideally, a single common accepted standard would be best, similarly to content descriptions in the educational 1

http://www.grapple-project.org/

U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 7–21, 2009. © Springer-Verlag Berlin Heidelberg 2009

8

A.I. Cristea et al.

domain (e.g., LOM2, SCORM3). In the GRAPPLE project, such an endeavour is being targeted. However, this is beyond the scope of the current paper. Making a language a common standard and reusable is only the first step, the next is to allow different levels of access to the creation process. This targets the different types of authors that will be able to use the language (e.g., computer savvy or not). One such version of different access levels is given by the LAG framework [17]: adaptation strategy – accessible for all authors, via laymen-level descriptions, adaptation language –accessible mainly to computer savvy authors, adaptation assembly language – only accessible to ‘hard core’ computer savvy authors). Another dimension is brought about by using visualization (e.g. the Graph Author developed for AHA! [18] is using visualisation in order to support authors) and handling support. In previous versions of the LAG language implementation, handling support was envisioned as not allowing an author to insert any wrong constructs [13][17]. In the GRAPPLE project, additionally to this restriction, the ultimate language to be used by the non programming-savvy author will be purely graphical [20]. Whilst this will hide most of the difficulty for the high-level author, it will also need to reduce the flexibility to some degree. When major changes are needed, or when system-system interaction is required, underlying programming languages will support this. Currently, we consider supporting multiple adaptation language output as a desirable feature, besides developing new languages targeted at specific levels of access (transformed into wrapping levels). Thus, this paper discusses general lessons learnt during the last five years in proposing, designing, implementing and using an adaptation language; the changes that the language has undergone in order to better fulfil its role as an adaptation language. Finally, this paper discusses the LAG language [8] as it currently stands, in view of the new extensions that have been performed which aim for it to better fulfil its goal of combining a high level of semantics for authors, with simplicity and portability as well as flexibility. Besides discussing these changes based on some sample strategies, this paper also presents an XML equivalent of LAG, which is to be used instead of the current language for portability between systems, as well as a novel authoring environment for the programmer or programming-savvy adaptation author, that applies feedback accumulated during various evaluation sessions with the previous set of tools. The outcomes of the tool, the adaptation strategies, can be used by any author [17]. This environment’s first evaluation is also presented. The remainder of this paper is organized as follows. Section 2 introduces the new elements in the LAG adaptation language via scenarios for adaptation. It also discusses alternative representations for the LAG language. Section 3 introduces the PEAL environment for authoring, by comparing it with the previous LAG language authoring environment, as well as with alternative solutions. The section concludes with a discussion of this environment and its first evaluation. Section 4 presents related research. In section 5, we draw general conclusions and pointers for further research. 2 3

ltsc.ieee.org/wg12/ www.adlnet.gov/scorm/

LAG 2.0: Refining a Reusable Adaptation Language and Improving on Its Authoring

9

2 The Updated LAG Grammar, via Scenarios for Adaptation 2.1 The LAG Grammar History and Lessons Learnt The LAG language concept was first introduced in [17], together with the LAG framework (hence, the similarity in name between language and framework, although they are distinct entities). As sketched in section 1, the LAG framework distinguishes between adaptation strategy – accessible for all authors, via laymen-level descriptions; adaptation language – accessible mainly to computer savvy authors; an example of such a language is the LAG language, although any adaptation language fits at this level; adaptation assembly language – only accessible to ‘hard core’ computer savvy authors. From the moment it has been proposed, the LAG language was supposed to fill in the ‘missing link’: it had to be and adaptation language, thus at a higher level than what the LAG framework called ‘adaptation assembly language’: it had to be reusable, whereas adaptation assembly languages at the time were not. To give an example, it was possible then to write: (a) IF Concept (‘The Night Watch’) has been visited THEN show Concept (‘Rembrandt’)

However, it was not possible to write: (b) IF Concept (title) has been visited THEN show Concept (author)

Thus, even such simple generalizations were not easily available to authors, who would have to manually connect all concepts, instead of writing reusable rules. Brusilovsky’s taxonomy [3], used for defining the types of adaptation possible, also refers to such an assembly language level4. Take for instance the decision of showing a concept by stretchtext, versus showing it by regular text; or hiding it by removing, or by graying out. These are decisions which may be dependent on the capabilities of the adaptation and rendering engine. A given engine may allow for showing concepts or not, but not for applying strechtext (e.g., the AHA! engine [18]). Using such low-level requirements might make an adaptation strategy impossible to be used by different engines. Moreover, such a low-level requirement may have little to do with the pedagogy involved in teaching a course, for instance. A teacher author might decide that a certain piece of information is necessary for a student or not, but may leave the rest to the engine. Thus, another condition for a language to be an adaptation language was that it had to be able to be converted to lower level assembly language, as per Brusilovsky’s taxonomy, but that this exact conversion is to be left to the interpretation and specifics of the given adaptation engine (hence, the similarity with a programming language which is compiled into assembly language in order to run on a certain system). For the example above, any structure (b) as above, applied to a certain domain, could become something similar to (a). However, an adaptation language may not necessarily have IF-THEN constructs, as they themselves are relatively low level. 4

Although the taxonomy can be used for writing reusable rules, it still only specifies low level actions performed on (usually specific) concepts.

10

A.I. Cristea et al.

Still, for compatibility with the engines of the time, the initial LAG language allowed for IF-THEN constructs, corresponding to assembly language constructs. Supplementary, however, it also defined higher level constructs, specific to the adaptation functionality, which are part of the adaptation language level within the LAG framework. Beside this important distinction, and essentially offering an instantiation of adaptation language ideas, it also defined what such a language should have: it should make use of the application domain (adaptive hypermedia) by 1) allowing it to be simpler and with fewer constructs than a regular programming language or a logic-based language (thus lowering the authoring threshold); Thus, elements were included in LAG only when considered necessary;and by 2) using constructs specific for the adaptive hypermedia domain, or assumptions that can be safely made in that domain. For instance, at the time it was safe to assume that most adaptive hypermedia applications have an underlying tree-like structure (as they were mainly designed for the educational domain, and, to some extent, inherited the organization into chapters-subchapters from educational books). This meant that, although, hypermedia, in theory, are graphs with any connections desired, in practice they were (and many of them still are) trees with given hierarchies. Hence, the GENERALIZE and SPECIALIZE constructs were born, the first to visit concepts higher in the hierarchy, thus of higher generality, and the latter, to visit concepts lower in the hierarchy, thus more specialized. It was then not a working language, just a proposal, which in the following year has been developed [13] towards introducing, first of all, a tool for supporting this grammar. There, we also introduced the concept of adaptation procedures, i.e. code snippets that can be reused by other authors, similarly to how procedures or function calls are used in other programming languages – with the significant, simplifying distinction however that no parameter exchange would take place, and that the extra code would be pasted in its entirely in the place of the ‘adaptation procedure’ call. This made these procedures simpler than regular programming languages, as per requirement 1 above. Moreover, these snippets could be used not only by their initial designer, but also by other authors, effectively creating a tool for customized language extension. This allowed for a higher level of reuse of adaptation languages, whilst keeping the processing simpler than in regular programming. The combination of requirements 1 & 2 above generated the following list of minimal constructs that should be present in high-level adaptation language: (a) constructs allowing domain structure and composition related adaptation: As said above, the domain structure and composition can be used to determine the adaptation process. i. hierarchy-based adaptation: if a hierarchy is present, and concepts are grouped as concepts-subconcepts (such as concept ‘The Night Watch’ is a subconcept of ‘Most Famous Paintings’), this hierarchy can be used to determine the order of appearance (such as the concept ‘Most Famous Paintings’ and its information should be shown before the concept ‘Night Watch’). ii. other relations –based adaptation: the most commonly used relation in domain models in adaptive hypermedia is that of concept-subconcept, as above. However, other relations might be possible, especially in systems

LAG 2.0: Refining a Reusable Adaptation Language and Improving on Its Authoring

11

importing RDF5 structures, for instance. Adaptation languages should be able to use these relations in the adaptation process. iii. domain-concept type-based adaptation: frequently, domain concepts have types (or other attributes). These also can be used in the adaptation process, thus should be accessible via the adaptation language. (b) constructs allowing goal-related adaptation: Adaptive hypermedia goals can be related to the pedagogy involved, if an educational application is envisioned, or to a business goal, in an e-commerce application, for instance. These goals can determine how domain concepts can be used. A simple way of adding such information is via labels and weights overlayed over the domain concepts they refer to. For example, the concept ‘The Night Watch’ can be labelled ‘visual’ to be used in a strategy involving visual versus verbal presentation, could be labelled ‘advanced’ if it is to be used in a drawing and painting class, or ‘beginner’ if it is to be used in a class on famous paintings and painters. Thus, whilst this information is added to concepts in the domain model, it is independent to the domain. This type of independence between domain and goal (or pedagogic) model was proposed as a basis for adaptation language construction [8] and has found recognition as one of the design concepts in the GRAPPLE project. i. label –based adaptation: see above. ii. weight–based adaptation: an alternative to label-based adaptation, numeric values can be used to label concepts with respect to the goal. This alternative is not used very frequently currently. (c)

structure of the adaptation program: i. Constructs defining the ‘adaptation loop’: Unlike regular procedural programs, the concept-driven adaptation in adaptive hypermedia happens in a loop. Users can visit the same concept several times. It may be that similar, or evolving behaviour is needed at successive visits. Thus, similar to the collection of rules in expert systems, the programming constructs in adaptation languages can be triggered repeatedly, and in different orders. An adaptation language should allow for an ‘adaptation loop’, that defines the continuous interaction between user and system, and for mechanisms to ensure that the correct constructs are triggered at the correct time. ii. Constructs allowing for an ‘entry point’: As adaptive hypermedia content is often based on the Web, it suffers from the same draw-back as regular Web hypermedia: first time users may visit the site. Thus, an adaptation language needs to be able to define what these users will be seeing. This is different from the ‘adaptation loop’ above, where users already have some history of recorded behaviour in the system6. The most important difference between the ‘entry point’ and the ‘adaptation loop’ is that the ‘entry point’ is a one-off event. Constructs will be executed here only once.

High level language thus means here a language created from an authoring perspective: an author is concerned about how the content, as well as the goal description for the particular application, can be used to model adaptation. The actual particulars 5 6

http://www.w3.org/RDF/ It is possible for this history of recorded behavior to be imported from a different application. In this case, direct entry into the ‘adaptation loop’ should be enabled.

12

A.I. Cristea et al.

of how the adaptation engine searches, retrieves, and renders each of these actions is of lesser relevance to the author, and could potentially add to the authoring complexity7. As will be shown in the following, the LAG language allows for all these constructs envisioned to be present in an adaptation language. A good update on the fundamental elements of the current basic LAG language is provided in [8]. There, handling of overlay variables, as well as independent variables is shown, for the different static representation layers supported: domain layer, goal and constraints layer (for representation of the goal of the application, such as pedagogical goal for educational presentations, and business goal for commercial applications), presentation layer, and user layer. These, together with the adaptation layer, that hosts the adaptation language, correspond to the layers as defined by the LAOS authoring framework for adaptive hypermedia [8]. Also there, the use of generic variables (for any domain or other static map) versus specific variables (for a given domain map or other static map) is described. Further extensions comprised authoring extensions for collaboration [13], for meta-level reuse [21], in the sense of being able to describe meta-strategies triggering strategies [16], thus allowing reuse of strategies in an automatic manner. In the remainder of this section, we illustrate with the help of scenarios8 other recent developments of the basic language, grouped around the different types of adaptation which the language allows. 2.2 Hierarchy-Based Constructs for Adaptation As previously discussed, domain models in adaptive hypermedia often have a hierarchical structure. The generalize-specialize constructs initially proposed in LAG have been replaced with simpler ones, that can be used as attributes of the concept, such as parent, child and level, order (as inspired by XPath9, in the spirit of using constructs of accepted standards where possible) The strategy shown below is a depth-first strategy, which shows the concept labelled as ‘start’ first, then the rest of the content in a depth first manner using the child-parent relations. The exact meaning of the constructs is given as comments in the strategy below: initialization( // ‘entry point’: this defines what the user first // sees; PM.next = true // allow for a ‘next’ button in the presentation; // please note that no information is given as to how to render // this ‘next’ button; this is up to the engine PM.ToDo = false // don’t allow for a ‘To Do’ list in // the presentation PM.menu = false // don’t allow for a ‘Menu’ in the presentation while true( // show the first, father concept, labelled ‘start’: if GM.Concept.label == start then ( PM.GM.Concept.show = true ) ) ) implementation ( // ‘adaptation loop’: this defines the continuous // adaptive interaction between user and system //if you visited the parent you should be able to if UM.GM.Concept.parent.access then ( // visit the child GM.Concept.show = true )) 7

This statement is based on previous evaluation experiments and interviews. Available for tryout at: http://prolearn.dcs.warwick.ac.uk/strategies.html 9 www.w3.org/TR/xpath 8

LAG 2.0: Refining a Reusable Adaptation Language and Improving on Its Authoring

13

Similarly, for breadth-first strategy, the level of a certain concept can be used to show all concepts of higher or equal level (the rest of the strategy is removed due to lack of space): // // // if

if the current concept level is lower than the current user level, show the current concept (so only show users concepts up to their current level) GM.Concept.level ”.) A user’s alias-address-pairs are to be assigned unambiguously to this individual user or her ID, respectively. To this end, we adapt the approach of Bird et al. ([38]) for computing the similarity of two alias-address-pairs according to the Levenshtein distance ([39]). If the distance is below a certain threshold, we assume that the two alias-address-pairs belong to the same user. Emailmessages that have been sent to or from different email-addresses can now be assigned to the same person. The email-analyzer evaluates email-related CAM-instances for representing a social network. Every person that occurs as sender or recipient of a message is represented by a node within the network. Two nodes are connected by an edge iff the respective persons are involved in the same message (as sender or recipient). The more email-messages two persons are jointly involved in, the stronger the edge between the respective nodes is. Figure 2 shows the CAMera tool displaying a user’s social network. The network represents connections to those persons with whom the user has exchanged at least ten messages within a selected time interval. The email-analyzer provides the user with an interface for browsing and manipulating the network: by marking a person’s node, a list of email-messages in which the particular person has been involved is generated and displayed together with the keywords of these messages. Furthermore, time intervals can be specified on a time line; thereby the keywords are weighted regarding their frequencies within these intervals and thus displayed larger or smaller. By clicking on a keyword, the list of messages is reduced to the messages that contain the selected keyword. The network itself can be manipulated in three different ways: firstly, by naming keywords one can highlight the nodes and edges that have been established due to messages that contain the keywords. Secondly, a user can specify a time interval and reduce the network according to the messages exchanged within this interval. This makes it possible to follow the dynamics of the network in the course of time. Thirdly, a user can set the minimal number of messages that must have been exchanged so that a person and an edge to this person appear in the network. By standard, a person appears in the network, if she was involved in at least one message. By setting a higher minimal number, sporadic contacts can be filtered out in order to make the network representation more concise. The email-analyzer gives a user an insight into the structure of her social network. It depicts the persons with whom the user has been in contact and the issues of her email-exchange. Therefore, it gives an account on a specific type of communication 7

Adapted Dragontalk (L3S, Leibniz Universität Hannover) is a further development of Dragontalk which was developed at DFKI Kaiserslautern ([37]).

CAMera for PLE

515

Fig. 2. Representation of a social network within the CAMera tool

behaviour and it supports the user in reflecting her communications. According to Viégas et al. ([40]), users are fascinated by the possibility of evaluating the social networks that are entailed in their email-conversations. (Thus, the email-analyzer arouses interest even without serving an immediate purpose.) Since communication is an integral part of collaborative learning (v.s., section 2), we assume that monitoring communication behaviour also contributes to the reflection of collaborative learning processes. 3.3 MACE Zeitgeist The second application we introduce here is a Zeitgeist application that is implemented as part of the MACE system. The MACE system ([32], [41]) sets up a federation of architectural learning repositories: large amounts of architectural contents from distributed sources are integrated and made accessible for architects, architecture students and lecturers. The contents are enriched with various types of metadata, among them Learning Object Metadata (LOM) and CAM. Examples of the user actions that are captured within the system are search actions (with the respective search keywords as related data), downloads of metadata on learning resources and downloads of the learning resources themselves, modifications of metadata, etc.8 8

The MACE system is intertwined with the ALOE system ([42]). ALOE is a web-based social media sharing platform that allows for contributing, sharing and accessing arbitrary types of digital resources. Users can up- and download resources; they can tag, rate, and comment on resources; they can create collections and add arbitrary kinds of metadata; and they can join and initiate groups, among other actions. The ALOE system provides observations of user activities related to the MACE system which are stored in the MACE usage metadata store. (See [43] and [44] for more information about ALOE and the system architecture.)

516

H.-C. Schmitz et al.

The Zeitgeist application is a set of web services that together provide an overview on activities within the MACE system. It gives users the possibility to reconstruct their learning paths by retracing which learning resources they accessed, how they found them and which topics have been of interest to them. This information can be used to make learning paths more explicit and to foster the learners’ reflection on their activities. Figure 3 shows the Zeitgeist dashboard that is used to give an overview on a user’s MACE-related activities: the Usage Summary (top box) shows the user activities for January 2009 when she viewed the metadata of 136, downloaded 84, bookmarked 60 and tagged 34 learning resources. Further details on the objects that have been accessed can be viewed by following the links called “Details”. The Usage History (middle box) shows the activities of the user per week, indicating when she viewed metadata, downloaded resources and tagged and bookmarked them. By simple statistics like these, the user recapitulates when she was looking for learning resources and when she found suitable ones. According to the graph, she constantly viewed resources during the week. Downloading and tagging, however, significantly increased after Thursday. Presumably, she started by searching for relevant data in the beginning of the week. By Thursday, she had found what she was looking for. Therefore, she downloaded these objects and tagged them. The Daily Content History (bottom box) lists the resources the user accessed most recently. According to the example given with Figure 3, the user viewed the metadata of “Villa dall’Ava” at 13:04:08 and downloaded the learning resource “Notre Dame du Haut” at 13:03:47. The respective titles of these data objects are linked to the objects themselves. The Zeitgeist dashboard depicted in Figure 3 is a web-based interface. The Zeitgeist data, however, can also be requested by the local CAMera-tool and thus – although this is not yet implemented – be presented through the actual CAMera-interface. That is, MACE Zeitgeist can become a remote component of CAMera. It

Fig. 3. MACE Zeitgeist dashboard

CAMera for PLE

517

provides an individual user with an overview on her MACE-related activities. It can also cumulate and analyze usage metadata of different MACE users and thus present an overview on all MACE-related activities and on general trends in MACE usage. This gives the individual learner the opportunity to compare her usage with the behaviour of the mass of MACE users. She can follow trends or, at contrary, refrain from trends and find new ways of exploring contents. An additional advantage of collecting metadata from different users is that now users can be compared regarding their usage profiles. A very simple usage profile can be defined as the set of objects that have been accessed in a certain time interval; the similarity of user profiles correlates with the cardinality of their intersection. Therefore, the Zeitgeist component not only provides data for reflecting one’s own learning behaviour but can also determine and point to similar learners which might be good cooperation partners. This is a clear advantage over a locally running component that observes and analyzes only a single user’s browsing behaviour. (With the Adapted Dragontalk plug-in, we are already provided with a respective metadata collector.) The local component can collect CAM-instances about the individual user’s interaction with the MACE system. However, it cannot easily integrate other kinds of metadata that are provided with MACE (LOM, e.g.), nor can it account for activities of other MACE users. The Zeitgeist component provides the learner with an overview on her MACErelated learning paths. It lets her remember how she came to the engagement in her current issues. It helps her to maintain an overview on her activities and the development of her interests. Thus, the Zeitgeist component supports her self-monitoring.

4 Conclusions We have argued that self-regulated learning is especially promising regarding learning outcomes and therefore should be supported. Self-monitoring is an essential part of self-regulated learning; the support of self-regulative learning can consist in the support of self-monitoring. We introduced CAMera as a tool for monitoring and reporting on a learner’s computer-related activities. In particular, we introduced an email-analyzer as an internal CAMera-component and a Zeitgeist application that can become a remote component. The CAMera tool is still under development, and – even further – it is necessarily continuous work in progress. It monitors user interaction with application programs and remote services. As application programs change and new tools and services are being developed, metadata collectors have to be exchanged and added. In addition, new tools most probably require new usage metadata analyses. As a consequence, new analysis components have to be implemented. We therefore designed CAMera as an open system to which new components can be easily added. CAMera can also function without observing a learner’s entire computer usage behaviour; it may just monitor the interaction with a few selected applications. So far, we only informally evaluated CAMera and its components by making them accessible to colleagues. The feedback we received was very good: the colleagues like to play with the components; they are interested in the reports and analyses; they state that they understand their own behaviour better. We still have to prove by a formal evaluation that the usage of the CAMera tool in fact, not only in principle, supports

518

H.-C. Schmitz et al.

self-regulated learning effectively. Optimally, we have to show that the usage of this tool leads to better learning outcomes. To this end, test-beds with large groups of selfregulated learners are needed. We are currently designing such test-beds within the European research project ROLE (Responsive Open Learning Environments, [45]). Evaluations within these test-beds will be performed in the near future.

References 1. Torrano, F., González, M.C.: Self-regulated learning: current and future directions. Electronic Journal of Research in Educational Psychology 2(1), 1–34 (2004), http://www.investigacion-psicopedagogica.org/revista/ articulos/3/english/Art_3_27.pdf (accessed: April 19, 2009) 2. Zimmerman, B.J., Schunk, D.H. (eds.): Self-regulated learning and academic achievement: Theory, research and practice. Springer, New York (1989) 3. Schunk, D.H., Zimmerman, B.J.: Self-regulation of learning and performance: Issues and educational applications. Erlbaum, Hillsdale (1994) 4. Schunk, D.H., Zimmerman, B.J.: Self-regulated learning: From teaching to self-reflective practice. Guilford, New York (1998) 5. Boekaerts, M., Pintrich, P.R., Zeidner, M.: Handbook of self-regulation. Academic Press, San Diego (2000) 6. Zimmerman, B.J., Schunk, D.H. (eds.): Self-regulated learning and academic achievement: Theoretical perspectives. Erlbaum, Hillsdale (2001) 7. Madrell, J.: Literature Review of Self-Regulated Learning (2008), http://designedtoinspire.com/drupal/node/600 (accessed: April 19, 2009) 8. Pintrich, P.R.: The role of goal orientation in self-regulated learning. In: Boekaerts, M., Pintrich, P.R., Zeidner, M. (eds.) Handbook of self-regulation, pp. 451–502. Academic Press, San Diego (2000) 9. Zimmerman, B.J.: Self-regulated learning and academic achievement: An overview. Educational Psychologist 25(1), 3–17 (1990) 10. Butler, D.L., Winne, P.H.: Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research 65(3), 245–281 (1995) 11. Kitsantas, A.: Self-monitoring and attribution influences on self-regulated learning of motoric skills. Paper presented at the annual meeting of the American Educational Research Association (1997) 12. Nota, L., Soresi, S., Zimmerman, B.J.: Self-regulation and academic achievement and resilience: A longitudinal study. International Journal of Educational Research 41(3), 198–215 (2004) 13. Fireman, G., Kose, G., Solomon, M.J.: Self-observation and learning: The effect of watching oneself on problem solving performance. Cognitive Development 18(3), 339–354 (2003) 14. Schunk, D.H.: Self-Monitoring as a motivator during instruction with elementary school students. Paper presented at the annual meeting of the American Educational Research Association (1997) 15. Winne, P.H., Jamieson-Noel, D.: Exploring students’ calibration of self reports about study tactics and achievement. Contemporary Educational Psychology 27(4), 551–572 (2002) 16. Gress, C.L., Fior, M., Hadwin, A.F., Winne, P.H.: Measurement and assessment in computer-supported collaborative learning. Computers in Human Behavior (in press) (Corrected Proof)

CAMera for PLE

519

17. Wilson, S., Liber, O., Beauvoir, P., Milligan, C., Johnson, M., Sharples, P.: Personal Learning Environments: Challenging the dominant design of educational systems. In: Proceedings of the first Joint International Workshop on Professional Learning, Competence Development and Knowledge Management (LOKMOL 2006 and L3NCD 2006), pp. 67–76 (2006) 18. Chatti, M.A.: Requirements of a PLE Framework. Blogspot (2008), http://mohamedaminechatti.blogspot.com/2008/02/ requirements-of-ple-framework.html (accessed: April 19, 2009) 19. Chatti, M.A., Jarke, M., Frosch-Wilke, D.: The future of e-learning: a shift to knowledge networking and social software. International Journal of Knowledge and Learning 3(4/5), 404–420 (2007) 20. w3schools.com – Browser statistics, http://www.w3schools.com/browsers/browsers_stats.asp (accessed: April 19, 2009) 21. Wikipedia – Usage share of web browsers, http://en.wikipedia.org/wiki/Usage_share_of_web_browsers (accessed: April 19, 2009) 22. Fingerprint – Email client statistics, http://fingerprintapp.com/ email-client-stats (accessed: April 19, 2009) 23. Wakoopa, http://wakoopa.com/ (accessed: April 19, 2009) 24. Centre for Learning and Performance Technologies: Top Tools for Learning (2009), http://www.c4lpt.co.uk/recommended/ (accessed: April 19, 2009) 25. Brusilovsky, P.: Adaptive Hypermedia. User Modeling and User-Adapted Interaction 11(1-2), 87–110 (2001) 26. Wolpers, M., Najjar, J., Verbert, K., Duval, E.: Tracking Actual Usage: the Attention Metadata Approach. Educational Technology & Society 10(3), 106–121 (2007) 27. Schmitz, H.C., Kirschenmann, U., Niemann, K., Wolpers, M.: Contextualized Attention Metadata. In: Roda, C. (ed.) Human Attention in Digital Environments. CUP, Cambridge (2009) 28. Verbert, K., Jovanovic, J., Gasevic, D., Duval, E.: Repurposing Learning Object Components. In: Proceedings of OTM 2005 Workshop on Ontologies, Semantics and E-Learning, Agia Napa, Cyprus (2005) 29. Ariadne ALOCOM Tools, http://www.ariadne-eu.org/ index.php?option=com_content&task=view&id=65&Itemid=96 (accessed: April 19, 2009) 30. The Flashmeeting Project, http://flashmeeting.open.ac.uk (accessed: April 19, 2009) 31. eXist Open Source Native XML Database, http://exist.sourceforge.net (accessed: April 19, 2009) 32. MACE – Metadata for Architectural Contents in Europe, http://portal. mace-project.eu (accessed: April 19, 2009) 33. JavaMail API, http://java.sun.com/products/javamail (accessed: April 19, 2009) 34. Yahoo! Developer Network: Term Extraction Documentation, http://developer. yahoo.com/search/content/V1/termExtraction.html (accessed: April 19, 2009) 35. tagthe.net, http://tagthe.net (accessed: April 19, 2009) 36. Adapted Dragontalk, http://www.l3s.de/~chernov/pas/Documentation/ Dragontalk/thunderbird_documentation (accessed: April 19, 2009) 37. Epos – Evolving Personal to Organizational Knowledge Spaces,

520

H.-C. Schmitz et al.

http://www3.dfki.uni-kl.de/epos (accessed: April 19, 2009) 38. Bird, C., Gourley, A., Devanbu, P.T., Gertz, M., Swaminathan, A.: Mining email social networks. In: Proceedings of the International. Workshop on Mining Software Repositories, Shanghai (2006) 39. Navarro, G.: A guided tour to approximate string matching. ACM Computing Surveys 33(1), 31–88 (2001) 40. Viégas, F.B., Golder, S., Donath, J.: Visualizing email content: portraying relationships from conversational histories. In: Proceedings of the SIGCHI conference on Human Factors in computing systems, Montreal, pp. 979–988 (2006) 41. Stefaner, M., Dalla Vecchia, E., Condotta, M., Wolpers, M., Specht, M., Apelt, S., Duval, E.: MACE – Enriching Architectural Learning Objects for Experience Multiplication. In: Duval, E., Klamma, R., Wolpers, M. (eds.) EC-TEL 2007. LNCS, vol. 4753, pp. 322–336. Springer, Heidelberg (2007) 42. ALOE-Project, http://aloe-project.de (accessed: April 19, 2009) 43. Memmel, M., Schirru, R.: Sharing digital resources and metadata for open and flexible knowledge management systems. In: Tochtermann, K., Maurer, H. (eds.) Proceedings of the 7th International Conference on Knowledge Management (I-KNOW), pp. 41–48. Journal of Universal Computer Science (2007) 44. Memmel, M., Schirru, R.: Aloe white paper. Technical report, DFKI GmbH (2008) 45. ROLE – Responsive Open Learning Environments, http://www.role-project.eu (accessed: April 19, 2009)

Implementation and Evaluation of a Tool for Setting Goals in Self-regulated Learning with Web Resources Philipp Scholl1, Bastian F. Benz2, Doreen Böhnstedt1, Christoph Rensing1, Bernhard Schmitz2, and Ralf Steinmetz1 1

Multimedia Communications Lab (KOM), Technische Universität Darmstadt, Merckstr. 25, 64283 Darmstadt, Germany 2 Pädagogische Psychologie, Technische Universität Darmstadt, Alexanderstr. 10, 64283 Darmstadt, Germany {scholl,boehnstedt,rensing,ralf.steinmetz}@KOM.tu-darmstadt.de, {benz,schmitz}@Psychologie.tu-darmstadt.de

Abstract. Learning effectively and efficiently with web resources demands distinct competencies in self-organization and self-motivation. According to the theory of Self-Regulated Learning, learning processes can be facilitated and supported by an effective goal-management. Corresponding to these theoretic principles, a goal-management tool has been implemented in an interdisciplinary project. It allows learners to set goals for internet research and assign relevant web resources to them. An evaluation study is presented that focuses on short-term learning episodes and selected results are shown that reinforce the benefits of our approach. Keywords: Goal-Setting, Learning with Web Resources, Self-Regulated Learning, Evaluation.

1 Introduction The importance of the World Wide Web as a major source of information for knowledge acquisition is growing steadily. With the web browser being the gateway, both specifically designed learning materials (e.g. contained in Web Based Trainings) and web resources that have not been designed with the intention to provide learning materials (e.g. weblog posts, wiki articles or community pages) but contain valuable information are available at a large scale. The paradigm of using these resources as learning materials is also known as Resource-Based Learning. Often used in context of lesson-style teaching, we focus on a rather informal, self-directed way of learning. However, major challenges for learners when learning in a self-directed way consist of stating their information needs, formulating search queries, estimating relevance of found resources, filtering irrelevant resources and keeping track of the state of the research, i.e. monitoring progress. These processes require high learner’s competencies of self-organization and self-motivation, as a deep information research is not trivial. Additionally, challenges arise that are based on peculiarities of the internet’s structure: information is out-dated, incomplete or targeted towards another audience and web resources cannot be retrieved. Thus, even if relevant information is U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 521–534, 2009. © Springer-Verlag Berlin Heidelberg 2009

522

P. Scholl et al.

found, it has only a transient use for learners, as usually it is not archived or persisted appropriately (see [7]). Hence, planning, organizing, setting goals and monitoring the involved processes may ease the difficulties of learners and prevent informational disorientation [10]. In this paper, we present an evaluation study of our goal-management tool that has specifically been designed to address some of these challenges. Section 2 presents a basic overview of the theory of Self-Regulated Learning that adequately describes this self-directed process of learning with web resources. Further, we explicate the term Scaffolds that denotes support of this process. We describe the design and implementation of a tool that enables learners to set learning goals prior to internet research and assign relevant web resources to these goals in section 3. This tool has been implemented into the web browser Firefox1, as web browsers are the gateway to most information on the web. Section 4 revisits the results of a previous study and section 5 presents a new study and evaluation of this tool with selected results. Section 6 concludes with a short summary and further steps.

2 Self-regulated Learning and Scaffolds Self-directed, resource-based learning with web resources is a process that is quite demanding for learners: they have to plan, monitor and reflect on their learning process in order to reduce disorientation and enhance quality of their learning achievements. In the following, we present particularities of this kind of learning and possibilities to support it using Scaffolds. 2.1 Self-regulated Learning It has been shown that supporting learners conducting the tasks mentioned above can improve the learning experience and the outcome [8] (e.g. by providing training or support learners writing a learning diary). For learning scenarios using web resources, i.e. hypertext documents, supporting self-regulated learning has shown to improve learners’ understanding and conceptual knowledge of a topic [1]. Central to the theory of Self-Regulated Learning is the notion that learning is a process that is self-directed and needs regulation on the learner’s side. According to Boekarts [3], three different systems have to be regulated in order to learn selfdirected. The cognitive system is performing task editing strategies, the learner will choose a strategy that he deems to be effective and efficient. For example, a learner who is researching information on the internet has to think about search query words that are likely to lead to success, i.e. relevant result resources. In his motivational system the learner regulates his volitional and motivational state, so that he will for example start a learning episode, overcome procrastination or better cope with obstacles. Finally, in the metacognitive system, the learner sets learning goals, devises plans and strategies for executing the actual learning process, monitors his progress on his actions, re-adjusts them if necessary and reflects on his learning process, eventually leading to forming of strategies to enhance his learning processes. 1

http://www.mozilla.com/en-US/firefox/ [online: 2009/04/16].

Implementation and Evaluation of a Tool for Setting Goals

523

Schmitz and Wiese [8] partition the learning process in three phases: before learning, during learning and after learning. Those phases may be combined with the three systems to be regulated [9]. As we focus on metacognitive processes in this paper, we will subsequently only consider processes that are executed in the metacognitive system. According to the theory of Self-Regulated Learning, learners profit from different metacognitive processes performed in each respective phase (see Table 1): Before learning (pre-actional phase), the learner performs goal-setting and planning, whereas while learning (actional phase), the progress and course of actions are monitored and – if necessary – adapted to possibly changed circumstances. Finally, after having learned (post-actional phase), reflection processes are executed in order to optimize future learning processes. Table 1. Overview of phases and respective metacognitive processes according to [2]

Phase Pre-actional Actional Post-Actional

Metacognitive processes Goal-Setting and planning Monitoring, adapting to changed circumstances (regulating) Reflecting, adapting goals and plans for next learning episode (modifying)

Further, [2] map the processes described above to learning episodes of different granularity. For example, an elementary task like a learner researching information on the internet is a rather fine-granular learning episode. For executing an efficient search process, the learner has to set his desired research goals, plan and monitor his process and finally evaluate, whether his learning goal has been met in the next minutes. However, a learner working on a bigger project (e.g. homework, a paper or a thesis) usually plans his approach, monitors and evaluates his process over several weeks. Still, a project will consist of several smaller, possibly related, learning episodes that are executed in the context of the project. In our evaluation, we focus on a short-term learning episode of 45 minutes. 2.2 Scaffolds Vygotsky [12] introduces the term Scaffolding as a “guidance provided in a learning setting to assist students with attaining levels of understanding impossible for them to achieve without external support”. Thus, scaffolds can be seen as learning aids that help learners to execute qualitative learning processes in order to achieve better learning results. In the long term, scaffolds should be designed to advance competencies, thus learners will not be dependent on the scaffolds. According to Friedrich et al. [5], scaffolds can be implemented both directly and indirectly. Direct scaffolds communicate instructions (so-called prompts) that ask the learner to carry out a certain learning action. For example, setting learning goals before starting to learn is such a direct scaffold. Indirect scaffolds can be implemented by design of a learning environment, so that the learner has the possibility to use certain supporting functionalities if required. For example, providing a goal-setting functionality in a program without a dedicated prompt can be seen as an indirect scaffold.

524

P. Scholl et al.

The theory of Self-Regulated Learning postulates specific processes that contribute towards a high-quality learning process. The concept of scaffolding defines and describes different possibilities to realize learner supports. Combining both approaches, learning processes can be assisted and supported according to the presented theoretical principles.

3 The Goal-Management Tool In this section we will derive the concept of a goal-management tool for internet research from the presented theoretical principles and present the implementation. Learners can enter goals, organize them into goal hierarchies (setting super- and subgoals), move them via drag&drop and attach found resources relevant to the respective goals. Each goal can have an arbitrary number of sub-goals and resources, organizing everything in a tree structure with exactly one super-goal – analogue to the directory structure of a common file system. 3.1 Conceptualization The goal-management tool is based on the partition of the learning process into the three phases before learning, while learning and after learning. We focus on the metacognitive processes of goal-setting, planning, monitoring, regulation and finally reflection and modification of the learning process. The scaffolds that support those processes are implemented indirectly, which means that the learner is not instructed to take direct action, but he may choose to use the functionality if he sees the need to. Before beginning with the internet research, the learner chooses a goal-directed approach and plans his course of actions in the learning process. For example, if a learner has the task to research information for the topic “Classical antiquity”, he may begin to structure his approach with the goals “I need to get a general idea about the ancient Rome” and “I need an overview of the ancient Greece”. Each goal can be further subdivided into specific sub-goals, e.g. the ancient Rome may contain the sub-goals “Roman Republic” and “First Triumvirate”. This way, the learner organizes his research goals into a goal hierarchy. Thus, the tool supports processes of goal-setting and planning. During the learning process the learner may attach found information in web resources to the set goals and rate their relevance for the respective goal. Monitoring the learning process is supported by multiple scaffolds, e.g. setting the progress of finishing a certain goal and displaying the goal hierarchy in combination with the already found web resources. Both stimulate the learner to contemplate where in the learning process he is right now, which goals he has already achieved and what goals are still open. In order not to loose focus on the goal the learner is following right now, it is possible for him to activate one goal at a time. This goal is displayed prominently, giving a reminder not to go astray and antagonizing the well-known “lost-in-hyperspace” phenomenon (experiencing disorientation due to information overload and aimlessly following hyperlinks). Further, all goals and found resources can be displayed as a knowledge network and an overview, displaying all goals and resources. This enables the learner to reflect on already found information and the current course of action. Is the learner aware of his inefficient advance, he may alter his research behaviour according to his current situation – for example by defining new goals, re-structuring his goal hierarchy or focussing

Implementation and Evaluation of a Tool for Setting Goals

525

on other goals that are more promising at the moment. Hence, during the research the processes of monitoring and regulation are supported. After learning, the learner has the choice between different alternatives of visualizing all goals and resources, basically the three visualizations already described: the goal hierarchy, the knowledge network and the complete overview. However, the theory of Self-Regulated Learning differentiates between the monitoring and regulation processes mentioned above and the processes of reflection and modification, as these occur after having finished the research. Here the visualizations enable learners to reflect on the finished learning episode, both from the view of the results and the taken approach. Further, if the learner decides to optimize his approach based on his reflection processes, modification processes are executed. 3.2 Implementation and Data Model Research and learning using web resources mostly takes place in the web browser, as most web resources are represented as HTML mark-up. The browser is a virtual window to the internet, downloading and rendering web resources and displaying them to the learner. Therefore, the tool has been implemented as an add-on to the popular open source web browser Firefox.

Fig. 1. The sidebar displaying the tree of goals and resources. The goal "plebs and peasants" is currently activated. At the bottom the buttons for displaying the knowledge network and the overview are located.

526

P. Scholl et al.

Fig. 2. An exemplary goal hierarchy displayed as a knowledge network. Resources with the same tag "Rome" are marked. A resource's detailed description (snippet) is shown in a tooltip.

Due to portability and extendibility reasons the core functionality has been realized in a Java applet. Data transmission with Firefox and the web resources is performed via an ECMAScript interface that both orchestrates the data flow and forwards user interaction within Firefox or the web resource to the applet. The graphical user interface and data storage has been implemented in Java. Applets as a technology were chosen, as they allow integration in HTML as well as in XUL (the Firefox-specific XML dialect for creating graphical user interfaces). Because we focus on short-term learning episodes, we confine properties of goals to a title, a description (which may serve to outline a course of actions or additional information) and the level of progress (with the stages “not started”, “25%”, “50%”, “75%” and “finished”). This level of progress can be set by the learner to keep an overview of his open and finished goals. Further, goals can be tagged (i.e. attaching freely chosen key words) for organization and display in the knowledge network. For longer learning episodes (which are not covered here), additional, mostly temporal, properties are planned, e.g. planned start, planned duration etc. The web resources are inserted into goals by use of the “import” functionality, similar to the process of bookmarking in Firefox. Similar to goals, resources have a title, a description, a relevance rating and tags. As the information need a learner has is often quite specific, just bookmarking a whole web resource is often not enough. Instead, the possibility to extract only the relevant part of the information is more target-oriented towards the real learning goal. Thus, the selected fragment (called snippet) of an imported web resource is stored in the description; learners can access

Implementation and Evaluation of a Tool for Setting Goals

527

that relevant information later without having to access the original web page. Rating the relevance of a resource or the snippet with the stages “not rated”, “not relevant”, “a little relevant” and “relevant” is possible as well. On starting up the web browser, the goal-management tool is displayed in the sidebar. Its user interface shows an overview of the current goal hierarchy and resources (see Fig. 1). Alternative representations of goals and resources may be used, e.g. a display of the goal hierarchy as a knowledge network (see Fig. 2 and [4]). While browsing web resources, they can be imported into the goal tree at the current selection. Both goals and resources may be edited and reorganized later-on.

4 The Previous Evaluation In 2008, we performed an evaluation focussing on the research questions, what differences learning online using different tools make and how explicit prompts are given in order to initiate goal-setting, planning and reflection processes [11]. We asked 64 participants (mainly psychology bachelor students) to answer a knowledge test about the topic “Classical Antiquity” (that we expected the participants to have little prior knowledge about) both before and after learning using Wikipedia for 45 minutes. We formed four different treatment groups: one group having pen and paper available as a means to persist findings, one group using the built-in bookmarking functionality of Firefox and two groups using our goal-management tool. The latter groups differed in the given instructions, one group just used the tool without any instructions, the other group was directly scaffolded to set goals, monitor their progress and finally reflect on their learning processes. In conclusion, we found that scaffolds do influence learning processes. Still, we encountered several issues with the study design. First, we tried to emulate “realistic” environments for the learners, i.e. forming a control group learning using bookmark functionality and a pen and paper group. Therefore, the groups were not comparable in some ways, and we think that influenced the learning outcomes. For example, the pen and paper group did not have to learn using a new tool and could quickly outline information, setting relations between content that was not possible for the other groups. Additionally, the bookmarks group was missing the possibility to save web resource snippets, thus participants had to bookmark the whole page – which many participants thought to be futile, thus not using this functionality. Eventually, the groups using the goal-management tool were only briefly trained to using it before learning. This means that computer competence and experience in using comparable tools had a strong influence on the way students were able to handle the tool.

5 The Study and Evaluation In our second study, we optimized our study design and chose a somewhat different scope. First, we provided sufficient training using the goal-management tool and altered the evaluation and control groups in some respects in order to make them more comparable.

528

P. Scholl et al.

Additionally, following research questions were of interest: • •

What are the differences between learners that organize their found web resources with folders (the control group) and learners that set goals prior to learning (the treatment groups)? What are the differences between learners that are explicitly instructed to execute metacognitive processes (the control group and the first treatment group getting indirect scaffolds) and learners that are free to use the functionality to support their metacognitive processes (the treatment group prompted by direct scaffolds)? Thus, what are the benefits of providing direct scaffolds?

5.1 Evaluation Design 104 students (mostly students of Psychology (74.5%) and Education (13.2%), more than 90% being in their first to seventh semester and being between 19 and 28 years of age) could be won for participating in our study. Due to the field of study a majority of the participants were women (72.6%) and 88.7% speak German as first language. The participants were randomly allocated to three groups: The Control Group (CG, n=34) was using a stripped-down goal-management tool that didn’t exhibit the goal-setting functionality. “Goals” were coined “Folders” and could not be activated or attributed progress. Still, the CG was able to put resources and snippets thereof in a folder and access the different displays of the collected data. The First Treatment Group (TG1, n=35) used the goal-management tool with the complete functionality but was not given instructions on how to organize their research. Hence, this group realized indirect scaffolds as given in section 2.2. The Second Treatment Group (TG2, n=35) used the same tool with integrated metacognitive prompts aimed to activate and support the metacognitive processes “defining relevant goals”, “keeping the active goal in mind”, “finding relevant pages”, “importing relevant information”, “assigning relevant information to the relevant goal” and “learning relevant information”. For example, before beginning the research (i.e. actional) phase, the learners were instructed to set goals for the research. Further, during search, instructions to reflect on whether the found information was relevant for the currently followed goal were given (see Fig. 3). Five minutes before the end of the evaluation, this group was instructed to reflect on their results. The overall study was performed in two sessions for each participant. The first session was exclusively for training with the respective tool variant and the second was the research task. The first session was always held the day before the research task and gave the participants a possibility to get to know the handling of the respective tool variant they would use on the research task. First, they watched an introduction in the respective tool, showing common tasks and the functionality of the tool. Then, the participants were presented a small research task in a topic they were confident with, where they could apply the functionality of their tool variant. Further, demographic data and data about the participants’ self-conceptions about their computer (estimation of their familiarity in using computers and knowledge about relevant computerand internet-related concepts) and skills of self-regulated web search (i.e. the competencies to plan and structure their learning processes, based on items of a standardized questionnaire according to [13]) were collected.

Implementation and Evaluation of a Tool for Setting Goals

529

Fig. 3. Example of a prompt, requesting the learner to reflect whether the imported web resource is relevant for the current research goal

The second session was designed to be approximately 1.5 hours in length. Participants were given a first achievement test (multiple-choice) about the “Classical Antiquity” – a topic that is well-covered in Wikipedia and, as we knew from the previous study, students do not have a lot of detailed prior knowledge about. An example for such a question is “Which event led to the end of the Roman Kingdom?” After each question the participants were asked to state how certain they were with answering this question (from the extremes “I guessed” to “I know and I am sure” in four steps). There were ten different versions of the test, which differed in the order the questions were provided. Participants were given the hint that they would receive exactly the same test again after the learning episode. Each participant received a feedback on his individual test performance. Ten questions which were either answered incorrectly or with uncertainty were provided for the first five minutes of the learning episode. This enabled competent learners to identify knowledge gaps in the achievement test and to re-formulate these into research goals in order to finally answer them correctly. During the research, participants were given updates about the time left. Eventually, the achievement test was administered to the participants a second time. Finally, the participants were asked to answer some questions about their learning and their experiences during the web search, their emotions according to PANAS [6] (a standardized questionnaire aiming at measuring positive and negative emotions), their usage of the goal-management tool and its functionality. Between all the phases of this second session, data about the current motivation and self-efficacy were collected. Besides the questionnaires, further data was collected: all participants’ actions during research were recorded using screen-capturing software and on client side, the click path – a list of all sequentially opened URLs – was stored including timestamp of access. In each session, psychometric tests were executed. Further, all actions the learners performed in the goal hierarchy were logged so we could reconstruct the process later.

530

P. Scholl et al.

5.2 Results of Evaluation For evaluating this study, we needed a topic for the students to research that they were not familiar with, thus we chose “Classical Antiquity”. In order to estimate their prior knowledge in this topic, the students were asked to state how much they knew about the Roman Antiquity (83% stated they have only "rather marginal" or "little" background, whereas only 2% said to have a "very good" knowledge about this subject) and Greek Antiquity (where only 1% of the participants claimed to have a "very good" knowledge about, in contrast to 86% of the participants stated to have a "rather marginal" or "little" background). Due to the topicality of given tasks, goals were usually set in a topic-oriented way, process-oriented goals (e.g. “I need to get an overview of …”) were rarely set. The results presented below are all based on the log files and the questionnaires. 5.2.1 Selected Group Differences To analyze the differences between all three groups including differences within specific phases of action, we conducted one way ANOVAs (Analysis of Variance between groups, comparing group means with each other) with quantitative log data as the independent variables. Table 2 presents some selected significant results. These show that, as presumed, in the pre-actional phase the three groups differ in terms of numbers of goals/folders created and edited, links followed, as well as number of imported, viewed and edited resources. Further, the number of viewed resources and links followed in the post-actional phase varied between groups. A difference between groups over all phases was encountered for moved goals/folders. These results in general indicate different approaches of web search for learners of different groups. Some learners seem to have searched in a very structured manner by first defining their search goals instead of already browsing and persisting resources. These learners also seem to have reduced distracting activities at the end of the learning phase in order to prepare for the post-test. Table 2. Significant Group Differences based on Participants' Actions

Category Creation of Goal / Folder Editing Goals / Folder Moving Goals Following new Link Import Resource View Resource Editing Resource

Phase of action Pre-actional Pre-actional All Pre-actional Post-actional Pre-actional Post-actional Pre-actional

ANOVA2 F(2, 102)=7.729, p walking-in-place > Flying, in virtual environments. In: SIGGRAPH 1999: Proceedings of the 26th annual conference on Computer graphics and interactive techniques, pp. 359–364. ACM Press/Addison-Wesley Publishing Co., New York (1999) 5. Larssen, A.T., Robertson, T., Edwards, J.: The feel dimension of technology interaction: exploring tangibles through movement and touch. In: TEI 2007: Proceedings of the 1st international conference on Tangible and embedded interaction, pp. 271–278. ACM, New York (2007) 6. Tan, D.S., Pausch, R., Stefanucci, J.K., Proffitt, D.R.: Kinesthetic cues aid spatial memory. In: CHI 2002: CHI 2002 extended abstracts on Human factors in computing systems, pp. 806–807. ACM, New York (2002) 7. Balakrishnan, R., Hinckley, K.: The role of kinesthetic reference frames in two-handed input performance. In: UIST 1999: Proceedings of the 12th annual ACM symposium on User interface software and technology, pp. 171–178. ACM, New York (1999) 8. Ängeslevä, J., O’Modhrain, S., Oakley, I., Hughes, S.: Body mnemonics. In: Physical Interaction (PI03) – Workshop on Real World User Interfaces, a workshop at the Mobile HCI Conference 2003, Udine (2003)

Real Walking in Virtual Learning Environments

595

9. Strachan, S., Murray-Smith, R., O’Modhrain, S.: Bodyspace: inferring body pose for natural control of a music player. In: CHI 2007: CHI 2007 extended abstracts on Human factors in computing systems, pp. 2001–2006. ACM, New York (2007) 10. Mine, M.R., Brooks, F.P., Sequin, C.H.: Moving objects in space: exploiting proprioception in virtual-environment interaction. In: SIGGRAPH 1997: Proceedings of the 24th annual conference on Computer graphics and interactive techniques, pp. 19–26. ACM Press/Addison-Wesley Publishing Co., New York (1997) 11. Tan, D.S., Stefanucci, J.K., Proffitt, D.R., Pausch, R.: The Infocockpit: Providing location and place to aid human memory. In: Workshop on perceptive user interfaces, Orlando, pp. 1–4 (2001) 12. Sivilotti, P.A.G., Pike, S.M.: A collection of kinesthetic learning activities for a course on distributed computing: ACM SIGACT news distributed computing column 26. In: SIGACT News, vol. 38 (2), pp. 56–74. ACM, New York (2007) 13. Ascension Technology Corporation, http://ascension-tech.com/realtime/RTflockofBIRDS.php 14. Limniou, M., Roberts, D., Papadopoulos, N.: Full immersive virtual environment CAVETM in chemistry education. In: Comput. Educ., September 2008, vol. 51(2), pp. 584–593. Elsevier Science Ltd, Oxford (2008) 15. Sowndararajan, A., Wang, R., Bowman, D.A.: Quantifying the benefits of immersion for procedural training. In: Proceedings of the 2008 Workshop on Immersive Projection Technologies/Emerging Display Technologiges, IPT/EDT 2008, Los Angeles, California, August 09 - 10, pp. 1–4. ACM, New York (2008)

Guiding Learners in Learning Management Systems through Recommendations Olga C. Santos and Jesus G. Boticario aDeNu Research Group, Artificial Intelligence Department, UNED, Calle Juan del Rosal, 16, Madrid 28040, Spain {ocsantos,jgb}@dia.uned.es http://adenu.ia.uned.es

Abstract. In order to support inclusive eLearning scenarios in a personalized way, we propose to use recommenddrs systems to guide learners thorugh their interactions in learning management systems. We have identified several issues to be considered when building a knowledge-based recommender system and propose a user-centered methodology to design and evaluate a recommender system that can be integrated via web services with exiting learning management systems to offer adaptive capabilities. We report some results from a formative evaluation carried out with users receiving recommendations in dotLRN open source eLearning platform. Keywords: Recommender systems, User experience, Recommendations, Learning Management Systems.

Accessibility,

1 Introduction Information and Communication Technologies have been considered from the beginning as a way to remove geographical and temporal barriers. Moreover, accessibility barriers can also be eliminated if this technology is properly applied. In this context, a proper usage of technology provides even more opportunities to enhance the learning. Addressing the individual needs in the learning process is difficult to achieve in face to face learning scenarios. However, in the current eLearning scenarios, learners can access a virtual course spaces which provides contents and services (e.g. communication tools) through a web-based interface. These interfaces have evolved from simple web pages to complex systems to facilitate the management of the learning. Learning managements systems (LMS) are broadly used in many institutions and efforts are being done to integrate them with their current technological infrastructure. The interactions done by the users in these systems can be gathered. These data, combined with information obtained explicitly from the users can be used to build user profiles. As a result, these profiles can provide the needed information to describe the needs and expectations of the users with the system. With this information personalized responses that address these individual needs could be offered to the users in LMS. U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 596–601, 2009. © Springer-Verlag Berlin Heidelberg 2009

Guiding Learners in Learning Management Systems through Recommendations

597

This situation presents several challenges to the research community on technology enhanced learning (TEL): • • • •

What are the needs of the users in LMS? How can users be supported in their needs when using LMS? Is there a way to evaluate that users are properly supported in the LMS? Can these services be supported in terms of web services architectures?

Our research follows on the idea of combining design and runtime adaptations. According to this approach, adaptations should be applied along the full life cycle of eLearning making a pervasive use of standards to support users in the process. The idea behind is that adaptation is not an idea that can be plugged in a learning environment, but a process that influences the full life cycle of learning, which consists on four steps where the user (and not the system) is the focus.

2 Learning Scenarios and Recommender Systems The first approaches to support learners with technology were done through the usage of Intelligent Tutoring Systems (ITS) [1]. These systems provide direct customized instruction or feedback to students, without the intervention of human beings, whilst performing a task. They contained a description of the knowledge or behaviors that represent expertise in the subject-matter domain the ITS is teaching which was used to detect the misconceptions and knowledge gaps of the learners as they work in the system to offer them the appropriate support. Conceptually, ITS are domain independent, although in practice most ITS have been designed for very specific domains and the knowledge is wired in the system, and hence, any changes on the domain require a development process. Moreover, ITS do not consider the interactions of the users within more collaborative learning scenarios which are supported by a wide diversity of communication tools. Current eLearning scenarios are supported by LMS. A common feature of these systems is the dispersion on the information available and the variety of communication channels to consider. Some can be structured in terms of learning resources within well-design units of learning, but there may be additional materials provided ad-hoc by the tutor or even by some student. There may be interactions done by users with the communication tools that present relevant information for the users. For instance, a discussion in a forum thread may clarify what learners are expected to do in a particular activity of the course. In the context of this large space of information and communication sources that learners are supposed to deal with, we propose the application of recommender systems to guide learners through inclusive eLearning scenarios. eLearning scenarios share the same objective as recommenders for e-commerce applications but there are particularities that make not possible to directly apply existing solutions from those systems [2, 3]. We have carried several attempts to involve users in the process of building a knowledge-based recommender aimed to plug recommendation strategies in standardbased LMS to extend their functionality with adaptive navigation support. A recommendations model was proposed and a prototype of a knowledge-based recommender

598

O.C. Santos and J.G. Boticario

was implemented and integrated into a well-known open source standard-based LMS called dotLRN. This knowledge-based recommender aims at generating suitable recommendations and reasoning about which elements of the domain meet the current user’s needs and context. The recommender system model describes i) what should be recommended (different recommendation types have been identified and can be offered, which relate to the actions that can be done on the LMS objects, such as send a forum message, work on a particular objective or share some opinion), ii) when a recommendation should be provided (considering the user and course context, the conditions of application and the timeout restrictions), iii) how a recommendation should be displayed (considering accessibility and usability criteria) and iv) why a recommendation has been produced (in terms of what category the recommendation applies to, what technique has been used to generate it, and the source that originated the recommendation). Details on the model are provided elsewhere [4]. A snapshot of the system is included next.

Fig. 1. Recommender System integrated in dotLRN LMS

The figure shows a coursepace in dotLRN where, in addition to communication services such as Forums and a SCORM player for the learning materials, a new portlet has been added that provides two recommendations for the current user.

3 Experiences with Users In order to understand the users’ perception on the recommender, we have followed some formative evaluations. We run a couple of experiences in two summer courses. The first course entitled “Accessibility and disability in the University: a development based on ICT” was organized by UNED in July 2008. The participants in this course were mainly accessibility experts and people with disabilities. The second course entitled “Services of the Web: applications towards the frontiers of knowledge” was organized by the UIMP in August 2008. The participants in this course had experience in using web-based applications for learning and teaching.

Guiding Learners in Learning Management Systems through Recommendations

599

A total of 29 users took part in the experience. Two of them were disabled. Anonymity was assured for the participants, since they were randomly given fictitious logins for the experience. The users were given access for one hour to an instance of dotLRN hosted at our institution where we had integrated -via web services- a dynamic support provided by the recommender system. An accessible SCORM-based course was offered to the users as well as several platform services, such as questionnaires, file storage area, forums, calendar, frequently asked questions, chat room, blog, statistics and recommendations. Taking the proposed model as our design framework, we defined thirteen recommendations to be given to the users in different course situations. Once followed the recommendation by the user, they were no longer displayed to them. When the users entered the system, they were recommended to read the help section on the platform usage and to fill in the learning style questionnaire to be able to adapt the contents to thei learning style. Moreover, they were also recommended to go to the course contents. To promote collaboration, once in the course space they were suggested to present themselves in the course forum. Experts from aDeNu group observed the users as they interacted with the system for one hour. Afterwards, users were given a questionnaire to evaluate their satisfaction. The data log was further analyzed and compared with the results from the observations and the responses to the questionnaire. From the questionnaires, the user satisfaction was evaluated positively. From the observations when running the pilots, no usability nor accessibility problem were detected. 55% of the users were able to finish the tasks required in the given time. Analyzing the logs we found out that when users followed the recommendations, the number of users who carried out tasks increased with respect to the users who did not follow the recommendations. Details on the experience are given in [4].

4 On Going Works In this paper we have presented an approach to guide learners through LMS via recommendations and have commented on a formative evaluation carried out with users. For that experience, we defined several recommendations that could be provided in an LMS. The recommendations for this experience did not had a strong psychoeducational background, but were aimed to tests the user’ attitude towards the recommender system within the LMS. In order to define recommendations following this model to feed the knowledge of the recommender that address the real needs of the learners and tutors in eLearning scenarios, we have been researching the appropriate user-centred design methods that help us to obtain from psycho-educational experts samples of psycho-educational sound recommendations. The methodology proposed is described elswhere [5]. Our current approach is to first understand the eLearning domain and elicit relevant recommendations from their users. On a second step, we will apply aritificial intelligence techniques to automate the process of generating recommendations [6]. Moreover, we are also reseraching on a methodological appraoch to evaluate the recommender. From literature, an approach towards the evaluation of adaptive systems is to decompose the adaptation process and evaluate the system in a “piecewise” manner. In this approach, adaptation is “broken down” into its constituents, and

600

O.C. Santos and J.G. Boticario

each of these constituents is evaluated separately where necessary and feasible. The constituents into which adaptation is decomposed are typically termed “layers” and the resulting approach “layered evaluation” [7]. This approach can be used to evaluate the effectiveness of the system and the advantages of the adaptation it provides [8]. We propose to add an extra layer on top of the existing layered approaches from evaluation. The two more cited in literature propose 2 and 5 layers, respectively. The 2-layer evaluation process defined by [9] consists on 1) evaluation of user modeling and 2) evaluation of the adaptation decision making. The 5-layer evaluation process is defined by [10] and consists on 1) collection of input data, 2) interpretation of the data collected, 3) Modeling of the current state of the “world”, 4) Deciding upon adaptation and 5) Applying adaptation decisions. The later is a refinement of the former. For the eLearning domain, we propose an extra layer to evaluate the design time issues, The purpose of this additional layer is to cover those issues which relate to psycho-educational considerations. From our experince, this is a critical sisssue since adaptive systems in education will only sube succesfull in practice when teachers can easily deal with it. In any case, the aim of these layers is to focus the evaluation to different directions, as identified in [6]: 1) the design of user interface of the tools required, 2) the process to design/generate the recommendations, 3) the process to select the appropriate recommendations, and 4) the analysis of the users’ interactions. This new layer (evaluation of design time issues) includes the evaluation of the design of the user interface of the tools required and the process to design the recommendations. The second layer (evaluation of user modeling) covers the analysis of the users’ interactions and the third layer covers the process to select the appropriate recommendations. Several principles are to be taken into account in the evaluation process: i) accessibility, ii) usability, iii) learnability, and iv) standards compliance. Accessibility issues and usability heuristics are to be focused on learnability and they are to be integrated in the layered-based evaluation. The latter provides different layers reflecting the various stages/aspects of the adaptation, starting from low-level input data acquisition or user monitoring, and going up to high-level assessment of the behavioral complexity of the users. This approach provides a series of advantages over those that focus on the overall user’s performance and satisfaction, such as useful insight into the success or failure of each adaptation stage separately, facilitation of improvements, generalization of evaluation results, and re-use of successful practices.

Acknowledgements The work presented here is framed in the context of the projects carried out by the aDeNu Research group. In particular, the EU4ALL (IST-2006-034478) funded by the European Commission and A2UN@ funded by the Spanish Government.

References 1. Psotka, J., Massey, L.D., Mutter, S.A.: Intelligent tutoring systems: lessons learned. Lawrence Erlbaum Associates, Mahwah (1988) 2. Drachsler, H., Hummel, H.G.K., Koper, R.: Personal recommender systems for learners in lifelong learning: requirements, techniques and model. International Journal of Learning Technology (2007)

Guiding Learners in Learning Management Systems through Recommendations

601

3. Tang, T., McCalla, G.: Smart recommendation for an evolving e-learning system. In: Workshop on Technologies for Electronic Documents for Supporting Learning, Proceedings of 11th International Conference on Artificial Intelligence in Education, Sydney, Australia, July 20–24, pp. 699–710 (2003) 4. Santos, O.C., Boticario, J.G.: Users’ experience with a recommender system in an open source standard-based learning management system. In: Proceedings of the 4th Symposium of the WG HCI&UE of the Austrian Computer Society on Usability & HCI for Education and Work, USAB 2008 (2008b) 5. Santos, O.C., Martin, L., del Campo, E., Saneiro, M., Mazzone, E., Boticario, J.G., Petrie, H.: User-Centered Design Methods for Validating a Recommendations Model to Enrich Learning Management Systems with Adaptive Navigation Support. In: Weibelzahl, S., Masthoff, J., Paramythis, A., van Velsen, L. (eds.) Proceedings of the Sixth Workshop on User-Centred Design and Evaluation of Adaptive Systems, held in conjunction with the International Conference on User Modeling, Adaptation, and Personalization (UMAP 2009), Trento, Italy, June 26, pp. 64–67 (2009) 6. Santos, O.C., Boticario, J.G.: Building a knowledge-based recommender for inclusive eLearning scenarios. In: Proceedings of the International Conference on Artificial Intelligence, AIED 2009 (2009) ( in press) 7. Paramythis, A., Totter, A., Stephanidis, C.: A Modular Approach to the Evaluation of Adaptive User Interfaces. In: Weibelzahl, S., Chin, D., Weber, G. (eds.) Empirical Evaluation of Adaptive Systems. Proceedings of workshop held at the Eighth International Conference on User Modeling in Sonthofen, Germany, July 13, pp. 9–24. Pedagogical University of Freiburg, Freiburg (2001) 8. Karagiannidis, C., Sampson, D.G.: Layered Evaluation of Adaptive Applications and Services. In: Brusilovsky, P., Stock, O., Strapparava, C. (eds.) AH 2000. LNCS, vol. 1892, p. 343. Springer, Heidelberg (2000) 9. Brusilovsky, P., Karagiannidis, C., Sampson, D.: Layered Evaluation of Adaptive Learning Systems. In: International Journal of Continuing Engineering Education and Lifelong Learning, Special issue on Adaptivity in Web and Mobile Learning Services, vol. 14(4/5), pp. 402–421 (2004) 10. Paramythis, A., Weibelzahl, S.: A decomposition model for the layered evaluation of interactive adaptive sysems. In: Ardissono, L., Brna, P., Mitrovic, A. (eds.) User Modeling 2005, pp. 438–442. Springer, Heidelberg (2005)

Supervising Distant Simulation-Based Practical Work: Environment and Experimentation Viviane Guéraud1, Anne Lejeune1, Jean-Michel Adam1, Michel Dubois2, and Nadine Mandran1 1

Laboratoire d’Informatique de Grenoble (LIG), CNRS, Université de Grenoble, France 2 Laboratoire Interuniversitaire de Psychologie, Université de Grenoble, France

Abstract. In this paper we present research targeting distant simulation-based practical work in various scientific domains. For the 6 past years, we continuously tried to improve the FORMID environment tools that we have designed and developed for building, running and observing such learning situations. This paper focuses on FORMID-Observer which is the FORMID tool intended to provide teachers with semantic information about the learners’ progress. We present the analysis of teachers’ observation practices during a recent session involving a secondary school group of learners in a practical work in electricity. Throughout the experiment’s results, we aim at showing how teachers’ diagnosis of learners’ domain-knowledge benefit both from the general principles of FORMID-Authoring tool and from the particular features of FORMID-Observer. Keywords: Supervision, Distant monitoring, Semantic data visualization, Teacher interface, Tutoring system, Virtual learning environment, Simulation, Computer supported learning, Distance Learning.

1 Motivation and Background There is an increasing use of e-learning systems providing teaching material via the Web. What happens in a virtual classroom where learning activities are automatically managed by e-learning environments? Which kind of awareness does a teacher need to understand learners’ progression throughout the learning process? These questions are not new [1, 2], but the related answers vary depending on the learning situation. Collecting the pertinent data to know exactly what happens during a learning session depends on the learning context. Some well used course management systems like WebCT/Blackboard, Moodle or Dokeos [3] usually provide general information about the students’ activity. These data are content independent, they provide the teacher with an idea of the effort done by each student, but don’t really inform about the quality of learning. Several research projects have developed tools that allow teachers to keep track of their learners’ interactions with the environment [4, 5, 6, 7], and of their learners’ communication activity in distant learning [8, 9, 10, 11]. Learning data can be collected by various means such as computer-interaction traces, videos, voice U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 602–608, 2009. © Springer-Verlag Berlin Heidelberg 2009

Supervising Distant Simulation-Based Practical Work

603

recordings, etc. The size of the collected data is also a crucial differential factor of awareness. However, making sense of these data remains the most difficult task. Our work is centered on learning situations based on formalized computer-readable learning scenarios [12]. The FORMID environment [13, 14] that we have designed, developed and continuously improved, allows setting up practical work sessions engaging a group of learners to interact individually with a simulation or a micro-world. Our approach consists in tracking very fine-grained information about learners’ simulation-based activity, and showing synchronously the gathered data to the teacher. A specific interface called FORMID-Observer aims to make teachers aware of learners’ progression throughout a virtual practical work session with an underlying approach very close to [15]. In this paper, we address the question of FORMID-Observer utility. In the context of a typical FORMID-managed practical work in electricity involving secondary school students, we try to know if (1) Teachers can understand what distant learners are doing; (2) Teachers perceive which student or sub-group of students fail or succeed in doing a particular exercise and why it happened, while such observations are not trivial [2].

2 Theoretical Aspects about Monitoring a Learning Session Monitoring learner progress in the e-learning context requires a consideration of the mental representation which a teacher generates for the purposes of diagnosing standards of learner development. The cognitive process involved in the perception of onscreen data is intimately connected with the monitoring of learning. The monitoring activity may therefore be conceived as the cognitive mechanisms determining correlations between data drawn from the system and data based on teachers’ acquired knowledge. The meaning of the situation is therefore not established a priori, but is rather constructed from the lowest level data interacting with a variety of cognitive processes. Monitoring a learning session does not merely consist in generating a representation of available on-screen data, but also involves forming a representation of what the on-screen information signifies in terms of learning [16]. In processing surface information, a teacher gradually forms several representations that vary in terms of duration and richness [17]. Except in exceptional circumstances [18], the surface code has a very brief duration and is quickly superseded by a level of representation reflecting meaning rather than the available surface data [19]. In monitoring a learning session, teachers establish connections between different types of on-screen data according to their acquired knowledge of class supervision [20, 21]. The teacher is therefore engaged in several distinct processes: A coherent and internal initial integration of the data provided and selected by the teacher on the interface helps to define connections between different signifying surface elements. Though important, this initial integration is not sufficient. A second integration appeals to long-term memorized knowledge of developmental factors required for learning in a particular area. The teacher supplements the data available with information drawn from his/her experience, to forming an improved mental representation of learner achievements and difficulties. A third integration occurs between indications drawn from the system and the activated knowledge known as a situational or mental model [22, 23].

604

V. Guéraud et al.

The issues relating to a teacher’s use of FORMID-Observer encompass the following: how is effectively integrated the information shown by FORMID-Observer, and what kinds of situational model are devised by teachers?

3 FORMID-Observer: A Flexible Environment for Teachers The FORMID environment (Fig 1) is composed of three distinct tools: (1) FORMIDAuthor for describing computer-supported practical work sessions, (2) FORMIDLearner for engaging learners into such sessions and storing learning traces in a database, (3) FORMID-Observer, on which this paper focuses, for making teachers aware of the class progression throughout a session, thanks to the database. We are dealing with active learning individual situations in which the learner must solve a problem while interacting with a simulation. In FORMID sessions problems are called exercises. Each exercise is described by a pedagogical scenario structured in steps; each step includes: − the goal to be achieved by each learner on the simulation: i.e. a condition on the final simulation state to be evaluated (as correct or not) and traced each time the learner requests an end-of-step validation; − a set of specific situations on the simulation, each revealing a typical error (or at the opposite a good behaviour/reasoning) to be automatically detected and traced during the step: i.e. a condition on the simulation state which value has to be observed by automatic frequent inspections. use

Interacts with

FORMID Environment

FORMID Author

pilot Scenario

refers Interacts with

pilot SIMULATION

use

interprets

FORMID Learner

(traces)

Structures and Learners makes DB sense

displays use

FORMID Observer

(traces)

Fig. 1. FORMID Environment

Tutors DB

Supervising Distant Simulation-Based Practical Work

605

Functionalities to assist the distant monitoring of learners involved in active learning situations are classified by [24] in perception, support and monitoring activity’s management. Our work focuses here on the teachers’ perception of learners' activities. To favour a good perception, FORMID provides teachers with the following features: − When defining a scenario, the teacher specifies what the execution tool will control during the learner's progression, according to his pedagogical approach, to the targeted learners group and to his knowledge of past teaching practices; these high levels indicators are further displayed by FORMID-Observer; − FORMID-Observer provides the teacher with three levels of perception [13,14], thus he can choose which view he needs to achieve his supervision at a given time, according to his monitoring strategy; − At each level, for each displayed event, detailed information on the related simulation state can be displayed and provide the teacher with additional insights of what is good or bad in the simulation state when the event occurred. − Semantic learning traces resulting from the execution are based on (1) each learner' successive end-of-step validation requests and their value (correct or not); (2) each learner's specific situation being detected during the step and their value (error or good behaviour/reasoning). These elements are automatically registered at run-time by FORMID-Learner and encapsulated with learner’s information (who) and time-stamp. Based on the scenario which structures a session, these learning traces are synchronously displayed in FORMID-Observer.

4 Teachers’ Supervision with FORMID: Analysis and Results We are working since a while with four second degree teachers in physics engaged with the French National Institute of Research in Pedagogy (I.N.R.P.). The experiment we discuss here involved these teachers (2 Males, 2 Females, average age: 52) observing a practical work in electricity intended to learners with the same scholar level than they usually are teaching for. There are the preparation steps of our experiment. (1) Session Design: We helped the teachers to design the scenarios related to the learning session they wanted to set up and observe; The resulting FORMID session was structured into 3 exercises using electric circuits’ simulations and its duration was estimated to 90 minutes. (2) Session Execution: fifteen learners of secondary school, unknown by the teachers acting in the experiment, ran the FORMID session. FORMID-Learner automatically registered traces of their individual learning process as explained in part 3. (3) Session Observation: Firstly, a 20 minutes training course introduced teachers with the FORMIDObserver spirit and features. After that, teachers “replayed” individually the 90 minutes session with FORMID-Observer. They were asked to comment aloud all their actions and thoughts when using FORMID-Observer. There are now the technical means we used for our experiment: (1) Observation tracing: All independent teachers interactions with FORMID-Observer were traced and automatically registered (see Fig. 1.); (2) Verbal comments recording (verbatim): We used an external system to record the verbal comments expressed by teachers

606

V. Guéraud et al.

when interacting with FORMID-Observer. Teachers were advised to be as expressive they could be about their own analysis of learners progress. The question we tried to answer was: “How is effectively integrated the information shown by FORMID-Observer, and what kinds of situational model are devised by teachers?” To access the mental representations of the learners’ actions, behaviours and knowledge, we analysed their verbatim recorded during the experiment. We can observe that FORMID-Observer usage allowed teachers to describe a learner or a group doings (79%) /“Dubois has replaced the lamp, but has forgotten to set the switch off, he didn’t modify the resistance again, so the lamp burnt out again!”/. Other verbatim are commentaries about a learner or a group work and concern the method employed to solve the problem. They show the representations that a teacher has from the learner or a group doings (38%) /“Among those who go forward by guesswork, there are those who have a good intuition: they see how to modify the resistance in order to find the right value, and there are those who do nonsense! It’s easy to see that”/. The teachers expressed also the related domain-knowledge they diagnosed as being mobilized by a learner or a group of learners solving the problem (31%) /“They have a wrong reasoning about the tension: they are thinking in the same way that for the previous circuit. On the other hand they have a good reasoning about the intensity."/. Note that the sum of the percentages is superior to 100 % because some verbatim are related to more than one of the previous categories. Some verbatim are both a description of doings and a comment on the method /“OK, basically I see that they have all already first set the switch on to see how the basic circuit works.”/; others are both a comment and a diagnosis /“This one has worked hard and now he knows how to solve the problem for the other series circuits.”/. They may also belong to the three categories /“Perrin is modifying and directly setting the resistance to 30 ohms. This is an interesting tentative. Missed! He validated too early, but it was because he has understood how it works, he didn’t do any calculation but he has understood. Congratulations Perrin!“/. The verbatim analysis seems to show that FORMID-Observer usage allowed the involved teachers to describe what a learner or the group was doing, which method is employed to solve the problem, and also the related domain-knowledge they diagnosed as being mobilized by the learners. The analysis seems also to indicate that the teachers were processing by steps: (1) description, (2) comment and (3) diagnosis.

5 Conclusion and Perspectives This paper presents an observation tool, called FORMID-Observer which is a part of a complete environment based on Web technologies for authoring [25], running and synchronously or asynchronously observing distant practical work learning sessions. A session consists on a set of exercises where learners try individually to solve problems by interacting with a simulation. Indicators of the learning process depend from the simulation states which are continuously evaluated during a session and are previously chosen at the design stage. The combination of the scenario structuring a session and this kind of indicators make sense of the recorded learning traces. Thus the

Supervising Distant Simulation-Based Practical Work

607

four teachers involved in this experiment can be aware of how learners try to solve the problems and what domain knowledge they are mobilizing. Another study [26], based on observation tracing and on a “a priori” interview shows that their use of the three FORMID-Observer levels was clearly dependant of the supervision strategy they clamed to use without observation tool. Others studies are in progress. Beyond their interesting results, these case studies with four teachers allowed us to elaborate and finalize the appropriate methodology for studying, tracing and analysing the use of FORMID-Observer. A larger experimentation can now be realized for generalizing these results.

References [1] Zinn, C., Scheuer, O.: Getting to know your student in distance-learning contexts. In: Nejdl, W., Tochtermann, K. (eds.) EC-TEL 2006. LNCS, vol. 4227, pp. 437–451. Springer, Heidelberg (2006) [2] Hijón-Neira, R., Velásquez-Iturbide, J.Á.: How to Improve Assessment of Learning and Performance through Interactive Visualization. In: ICALT 2008 proceedings, pp. 472–476 (2008) [3] Goldberg, M.W.: Student participation and progress tracking for Web-Based Courses using WebCT. In: Proc.of the 2nd North American Web Conference, Fredericton, NB, Canada (1996) [4] Mazza, R., Dimitrova, V.: CourseVis: Externalising Student Information to Facilitate Instructors in Distance Learning. In: Proc. AIED 2003, pp. 279–286. IOS Press, Amsterdam (2003) [5] Scheuer, O., Zinn, C.: How did the e-learning session go? The Student Inspector. In: Proceedings of AIED 2007, Marina Del Rey, Ca., USA, pp. 487–494. IOS Press, Amsterdam (2007) [6] Razzaq, L., Heffernan, N., Feng, M., Pardos, Z.: Developing Fine-Grained Transfer Models in the ASSISTment System. Journal of Technology, Instruction, Cognition, and Learning 5(3), 289–304 (2007) [7] Feng, M., Heffernan, N.: Towards Live Informing and Automatic Analyzing of Student Learning: Reporting in ASSISTment System. Journal of Interactive Learning Research 18(2), 207–230 (2007) [8] Bratistis, T., Dimitracopoulou, A.: Data Recording and Usage Interaction Analysis in Asynchronous Discussions: The D.I.A.S System. In: AIED 2005 proceedings, pp. 17–24 (2005) [9] Harrer, A., Ziebarth, S., Giemza, A., Hoppe, U.: A framework to support monitoring and moderation of e-discussions with heterogeneous discussion tools. In: ICALT 2008 proceedings, pp. 41–45 (2008) [10] May, M., George, S., Prevôt, P.: Sutdents’ Tracking Data: an Approach for efficiently Tracking Computer Mediated Communications in Distance Learning. In: ICALT 2008, pp. 783–787 (2008) [11] Donath, J., Karahalios, K., Viégas, F.: Visualizing conversation. Journal of Computer Mediated Communication 4(4) (1999) [12] Adam, J.M., Lejeune, A., Michelet, S., David, J.P., Martel, C.: Setting up on-line learning experiments: the LearningLab platform. In: Proceedings of ICALT 2008, pp. 761–763. IEEE Computer Society, Los Alamitos (2008)

608

V. Guéraud et al.

[13] Guéraud, V., Cagnat, J.M.: Automatic semantic activity monitoring of distance learners guided by pedagogical scenarios. In: Nejdl, W., Tochtermann, K. (eds.) EC-TEL 2006. LNCS, vol. 4227, pp. 476–481. Springer, Heidelberg (2006) [14] Guéraud, V., Lejeune, A., Adam, J.M., Dubois, M., Mandran, N.: Flexible Environment for Supervising Simulation-Based Learning Situations. In: AIED 2009, Brighton (UK) (July 2009) [15] Ben-Naim, D., Marcus, N., Bain, M.: Visualization and Analysis of Student Interactions in an adaptive Exploratory Learning Environment. In: International Workshop on Intelligent Support for Exploratory Environment, EC-TEL 2008, CEUR-WS.org, vol. 381 (2008) [16] Graesser, A.C., Millis, K.K., Zwaan, R.A.: Discourse comprehension. Annual Review of Psychology 48, 163–189 (1997) [17] Noordman, L.G.M., Vonk, W.: Memory-based processing in understanding causal information. Discourse Processes 28, 191–221 (1998) [18] Kintsch, W., Bates, E.: Recognition memory for statements from a classroom lecture. Journal of Experimental Psychology: Human Learning and Memory 3, 150–159 (1977) [19] Sachs, J.S.: Recognition memory for syntactic and semantic aspects of connected discourse. Perception and Psychophysics 2, 437–442 (1967) [20] Frank, S.L., Koppen, M., Noordman, L.G.M., Vonk, W.: Modeling knowledge-based inferences in story comprehension. Cognitive Science 27, 875–910 (2003) [21] Frank, S.L., Koppen, M., Noordman, L.G.M., Vonk, W.: Modeling Multiple Levels of Text Representation. In: Schmalhofer, F., Perfetti, C.A. (eds.) Higher level language processes in the brain: inference and comprehension processes, pp. 133–157. Erlbaum, Mahwah (2005) [22] Bransford, J.D., Barclay, J.R., Francks, J.J.: Sentence memory: a constructive versus interpretive approach. Cognitive Psychology 3, 193–209 (1972) [23] Glenberg, A.M., Meyer, M., Lindem, K.: Mental models contribute to foregrounding during text comprehension. Journal of Memory and Language 26, 69–83 (1987) [24] Després, C.: Synchronous Tutoring in Distance Learning. In: Hoppe, U., Verdejo, F., Kay, J. (eds.) Proc. AIED 2003, pp. 271–278. IOS Press, Amsterdam (2003) [25] Cortés, G., Guéraud, V.: Experimentation of an authoring tool for pedagogical simulations. In: Proceedings ofInternational Conference CALISCE 1998, Göteborg, Sweden, pp. 39–44 (1998) [26] Guéraud, V., Adam, J.-M., Lejeune, A., Dubois, M., Mandran, N.: Teachers need support too: FORMID-Observer, a flexible environment for supervising simulation-based learning situations. In: Workshop ISEE, AIED 2009, Brighton (UK) (July 2009)

Designing Failure to Encourage Success: Productive Failure in a Multi-user Virtual Environment to Solve Complex Problems Shannon Kennedy-Clark Centre for Research on Computer Supported Learning and Cognition, Faculty of Education and Social Work at the University of Sydney, Sydney Australia [email protected] http://coco.edfac.usyd.edu.au/

Abstract. The purpose of this research project is to gain an understanding of the initial stage of a productive failure treatment. The research focuses on how learners solve complex or ill-defined problems in Virtual Singapura, a multi-user virtual environment. The research uses a mixed method approach that employs conversation analysis, questionnaires and pre, mid and post-tests. Complex problems, by their very nature, are difficult for learners to connect with, and this project will focus the initial cycle of a productive failure treatment in order to develop a series of design considerations that teachers can implement in an immersive learning environment to help students develop the strategies necessary to engage with complex problems across domains of knowledge. The project aims to inform theory on productive failure, learner processes and learning in immersive environments. Keywords: Productive failure, education, multi-user virtual environments, complex problems

1 Introduction The ability to solve a diverse range of complex problems is a requirement of many learning and workplace situations. However, learners are often taught how to solve a complex problem through the use of highly structured or scaffolded activities and are not provided with opportunities to engage in processes such as defining problems, creating hypotheses and testing these hypotheses [1]. The restriction created by the use of these structures and a lack of opportunities to engage in ill-defined problem solving activities can impair a learner in developing a complex problem solving toolkit as the scaffolds constrain the learner to a narrow problem solving scope. A robust body of literature indicates that students have difficulties both visualising and solving complex problems in domains of inquiry (e.g. science, physics and history). Accordingly, there have been numerous computer supported learning projects aimed at enabling students to grasp often diffuse and complicated concepts such as weather patterns and astronomy [2-6]. U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 609–614, 2009. © Springer-Verlag Berlin Heidelberg 2009

610

S. Kennedy-Clark

However, many of these interventions have focused on the use of highly scaffolded or structured treatments that guide a learner through a series of activities. What is being proposed in this paper is that a move away from scaffolding in the initial encounter with a problem may result in better learning. Research indicates that making mistakes and failing to arrive at the correct answer can encourage learners to reflect on their learning process and to access domains of knowledge and previous experiences, thus encouraging a deeper level of engagement and critical thinking [7-10]. This paper will present the benefits of using Virtual Singapura (VS), a multi-user virtual environment (MUVE), as a platform for learners to engage with complex or ill-defined problems. The research is part of a larger research project, which is the first of its kind in Australia, will focus on the initial cycle of a Productive Failure (PF) treatment in a MUVE.

2 Productive Failure – Reaching an Impasse PF is a learning strategy that designed to enhance or facilitate the transfer of knowledge from one domain activity to another. Recent research utilising PF in complex environments has resulted in significant findings that support the use of a PF treatment [see work by Kapur and Jacobson, Pathak et. al., Jacobson et al.]. As a simple analogy, PF can be viewed as an hourglass wherein students are able to explore an ill-defined problem domain with no structure in the initial activity, before being exposed to a structured activity and then re-exposed to an un-structured activity (see Fig. 1. below). This presents students with an opportunity to reach an impasse in the activity. Often instructors shy away from allowing students to reach an impasse, however research by VanLehn et al. [10] indicates that allowing students to reach an impasse may encourage students to think more critically about a situation and that reaching an impasse can encourage learning. As Kolodner [9] indicates, we may be novices in one domain, but we can bring a multitude of past experiences into this domain that we can apply to attempt to solve the new problem. Kapur [7, 11] further argues that through using unstructured learning activities a student may develop more flexible and adaptive learning in the long run based on their initial failures. Hence, the underlying premise of PF suggests that the lack of structure in the initial activity is the key to successful problem solving in subsequent structured and unstructured activities.

3 Multi-user Virtual Environments as Learning Tools MUVEs are becoming widely recognised for their benefits as learning environments. A MUVE is a persistent virtual environment that is usually accessed online via a downloadable software platform such as Active Worlds or is located online. There is a developing body of literature around MUVEs and with this comes crystallisation of the key criteria that classify a MUVE as a distinct tool from other forms of online learning activities. The five main criteria are a) an avatar that represents the participant, b) a 3D virtual environment, c) the ability to interact with artefacts and agents,

Designing Failure to Encourage Success

611

d) participants can communicate with other participants and, in some instances, communicate with intelligent agents and e) a ‘real world’ context that is created to provide an authentic experience that a student may not be able to encounter in a classroom environment [12-20]. MUVEs, such as Quest Atlantis [21] and VS [22] provide students with an opportunity to visualise and engage with complex learning systems in a setting that is motivating and engaging. Nevertheless, all games and educational MUVEs have limitations and educators need to be aware of these limitations in order to maximise the benefits of the experience for leaner –in this research project the cycles of feedback and iteration may address some of the pedagogical and design issues that concern VS and other MUVE environments.

4 Virtual Singapura – Solving Complex Problems in a MUVE VS was developed in Singapore as part of a collaborative project between researchers at Singapore Learning Sciences Laboratory (National Institute of Education) and faculty in Computer Engineering and in Art, Design, and Media at Nanyang Technological University. The story or scenario for the VS lends itself to the trial of PF and it presents a rich problem solving environment. VS is set in 19th century Singapore and is based on historical information about the cholera epidemic of 1873 – 74. The students are transported back in time to help the Governor of Singapore, Sir Andrew Clarke, and the citizens of the city to try and solve the problem of what is causing the illnesses. Students are encouraged to develop appropriate scientific inquiry skills such as defining the scope of the problem; identification of research variables; establishing and testing hypothesis and presentation of findings. In order to create an authentic learning experience, 19th century artefacts about Singapore have been included in the environment. These artefacts include historical 3D buildings and agents that represent different ethnic groups in Singapore at the time such as Chinese, Malay, Indian, westerners, and historic period photographs. The PF treatment activities will be adapted to suit to specific learning outcomes of the Australian NSW High School Science Curriculum.

5 Research by Design The research is part of a larger research project that uses a Design Based Research framework (DBR). While DBR is often associated with the learning sciences, a field that is known for its utilisation of technology in education, the focus of a DBR approach is on pedagogy and learning theories rather than on the development of technological tools and artefacts. Whilst technology is often an important feature of the research, the learner is still the focus. DBR is often seen as a series of approaches, rather than a single approach that is aimed at the development of new theories and practises in naturalistic settings [23, 24].

612

S. Kennedy-Clark

6 Participants The participants in this study will be drawn from an Australian High School. The participants will be studying science in years 7 - 9 (12 – 15 years of age) to develop scientific inquiry skills this trial is scheduled for December 2009. Pilot studies will be held in August 2009 with pre-service teachers undertaking a Master Degree at Sydney University.

7 Data Collection A mixed-method approach to data collection will be used. The intervention will have three phases. The first phase of the intervention will expose students to an unstructured activity. The second phase will expose participants to a structured activity. The third phase will expose participants to another unstructured activity. Pre, mid and post-tests will be used. Verbal communication analysis of the initial unstructured activity will be used and the data will be coded on two levels – firstly, for convergence of ideas and secondly, for linguistic features [25, 26]. Screen capture software will be used to ascertain what aspects of the environment the learners are focusing on [25, 27, 28]. A broad analysis of this data can express whether students are claiming, predicting, eliciting, creating and acting and will be coded to see how students collaborate when trying to solve or engage with the problem domain.

8 Final Considerations One final point to reflect on is that the aim of this research is not to produce a definitive theory on PF, but rather to complement and add to the small body of work that is currently available, and to hopefully provide further data to substantiate the rationale underpinning the use of a PF strategy in complex problem solving activities. Current research projects indicate that there is unquestionable potential in the use of MUVEs in learning environments, this research into a PF treatment in VS will add to this growing body of work, and may provide another avenue which instructors can use to enable learners to develop problem solving strategies that move beyond the bounds of a traditional classroom environment.

References 1. Zydney, J.M.: Eighth-Grade Students Defining Complex Problems: The Effectiveness of Scaffolding in a Multimedia Program. Journal of Educational Multimedia and Hypermedia 14(1), 61–90 (2005) 2. Bodemer, D., et al.: Supporting learning with interactive multimedia through active integration of representations. Instructional Science 33, 73–95 (2005) 3. Barnett, M., et al.: Using Virtual Reality Computer Models to Support Student Understanding of Astronomical Concepts. The Journal of Computers in Mathematics and Science Teaching 24(4), 333–356 (2005)

Designing Failure to Encourage Success

613

4. Kim, P., Olarciregui, C.: The effects of a concept map-based information display in an electronic portfolio system on information processing and retention in a fifth-grade science class covering the Earth’s atmosphere. British Journal of Educational Technology 39(4), 700–714 (2008) 5. Lowe, R.: Interrogation of a dynamic visualisation during learning. Journal of Learning and Instruction 14, 257–274 (2004) 6. Puntambekar, S., Goldstein, J.: Effect of Visual Representation of the Conceptual Structure of the Domain on Science Learning and Navigation in a Hypertext Environment. Journal of Educational Multimedia and Hypermedia 16(4), 429–459 (2007) 7. Kapur, M.: Productive Failure. Cognition and Instruction 26(3), 379–424 (2008) 8. Hmelo, C.E., Holton, D.L., Kolodner, J.L.: Designing to Learn about Complex Systems. The Journal of the Learning Sciences 9(3), 247–298 (2000) 9. Kolodner, J.L.: Case-Based Reasoning. In: Sawyer, K. (ed.) The Cambridge Handbook of the Learning Sciences, pp. 225–242. Cambridge University Press, Cambridge (2006) 10. VanLehn, K., et al.: Why Do Only Some Events Cause Learning During Human Tutoring? Cognition and Instruction 21(3), 209–249 (2003) 11. Kapur, M.: Productive Failure. In: International Conference on Learning Science, Bloomington, Indiana (2006) 12. Dickey, M.D.: 3D Virtual Worlds: An Emerging Technology For Traditional And Distance Learning. In: Convergence of Learning and Technology, Ohio Learning Network, Easton (2003) 13. Squire, K.D., et al.: Electromagentism Supercharged! Learning Physics with digital simulation games. In: International Conference of the Learning Sciences, Los Angeles, CA (2004) 14. Dieterle, E., Clarke, J., Pagani, M. (eds.): Multi-User Virtual Environments for Teaching and Learning. Encyclopedia of Multimedia. Idea Group, Inc., Hershey (in press) 15. Ketelhut, D.J., et al.: A multi-user virtual environment for building higher order inquiry skills in science. American Educational Research Association, San Francisco (2006) 16. Nelson, B.: Exploring the Use of Individual, Reflective Guidance In an Educational MultiUser Environment. Journal of Science Education and Technology 16(1), 83–97 (2007) 17. Rieber, L.P.: Seriously Considering Play: Designing interactive learning environments based on the blending of microworlds, simulations, and games. Educational Technology, Research and Development 44(2), 43–58 (1996) 18. Shaffer, D.W., Gee, J.P. (eds.): Epistemic Games as education for innovation. Learning through Digital Technologies. Underwood, J.D.M., Dockrell, J.(eds.): British Journal of Educational Psychology. Leicester, pp. 71–82 (2007) 19. Steinkuehler, C.A.: Massively Multiplayer Online Video Gaming as Participation in a Discourse. Mind, Culture, and Activity 13(1), 38–52 (2006) 20. Taylor, T.L.: Multiple Pleasures: Women and Online Gaming. Convergence: The International Journal of Research into New Technologies 9, 21–46 (2003) 21. Barab, S.A., et al.: Making Learning Fun: Quest Atlantis, A Game Without Guns. Educational Technology, Research and Development 53(1), 86–107 (2005) 22. Jacobson, M.J., et al.: An Intelligent Agent Augmented Multi-User Virtual Environment for Learning Science Inquiry: Preliminary Research Findings. In: 2008 American Educational Association Conference, New York (2008) 23. Barab, S.A., Squire, K.: Design-Based Research: Putting a Stake in the Ground. Journal of the Learning Sciences 13(1), 1–14 (2004) 24. The Design-Based Research Collective, Design-based research: An emerging paradigm for educational inquiry. Educational Researcher 32(1), 5–8 (2003)

614

S. Kennedy-Clark

25. Sawyer, K.: Analyzing Collaborative Discourse. In: Sawyer, K. (ed.) The Cambridge Handbook of the Learning Sciences, pp. 187–204. Cambridge University Press, Cambridge (2006) 26. Kapur, M., Kinzer, C.K.: Productive Failure in CSCL Groups. International Journal of Computer-Supported Learning 4(1), 21–46 (2009) 27. Mazur, J.: Conversation Analysis for Educational Technologists: Theoretical and Methodological issues for Researching the Structures. In: Jonassen, D. (ed.) Processes and Meaning of On-line Talk, in Handbook of Research for Educational Communications and Technology. MacMillian, New York (2004) 28. Mazur, J., Lio, C.: Learner Articulation in an Immersive Visualization Environment. In: Conference on Human Factors in Computing Systems, Vienna, Austria. ACM, New York (2004)

Revisions of the Split-Attention Effect Athanasios Mazarakis Forschungszentrum Informatik, Haid-und-Neu-Str. 10-14, 76131 Karlsruhe, Germany [email protected]

Abstract. For the learning process with multimedia contents the split-attention effect postulates that learning results are better the higher the spatial proximity of text and picture elements is. This article shows that by the use of an artificially generated relationship between texts and pictures which are far away (according to the new principles of grouping by Palmer[1]), it is possible to attain learning results which are at least equal. The negative impact of the spatial distance between text and picture elements can therefore be avoided in a different way. So an online survey has been conducted and the data of 869 subjects have been evaluated regarding to their retention and transfer performance. Keywords: multimedia learning, cognitive load theory, cognitive theory of multimedia learning, split-attention effect.

1 Introduction There are two commonly used and very similar cognitive theories of learning with multimedia contents: the Cognitive Load Theory [2] and the Cognitive Theory of Multimedia Learning [3]. But both approaches have theoretical weaknesses if they try to handle effects which came into being directly from the theories. In this context the split-attention effect will be discussed further.

2 Background of the Used Theories According to the Cognitive Load Theory of Sweller [5] there are three different so called "loads": The intrinsic cognitive load, the extraneous cognitive load and the germane cognitive load. These three loads are added up to the cognitive load. Here the extraneous cognitive load is the load, which originates from an unadjusted design of the instructions, like e. g. additional multimedia elements, which divert the attention of the learner. However the germane cognitive load is responsible for the construction and automation of schemata which Sweller [5] regards to be the ideal solution for the learning with multimedia content. For the construction and automation of schemata it is important to observe the limited capacity of the working memory according to Baddeley [6]. U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 615–620, 2009. © Springer-Verlag Berlin Heidelberg 2009

616

A. Mazarakis

Finally, the intrinsic cognitive load again arises from the natural complexity of the information which has to be learned. On the one hand there are elements which can be learnt independently from others and therefore only cause a low cognitive load. Sweller [7] calls this low-element interactivity material. On the other hand there are elements which correspond strongly to each other, called high-element interactivity. Here a high cognitive load arises due to the fact that the information has to be learned simultaneously in order to achieve a high level of understanding by the learner. Besides the already presented Cognitive Load Theory, the Cognitive Theory of Multimedia Learning by Mayer and Moreno [8] is the second prominent theory of learning within the multimedia field. This theory is very similar to the Cognitive Load Theory and is mentioned at this point in order for completeness. 2.1 The Split-Attention Effect Ayres and Sweller [9] define the split-attention as present, when the learner has to divide his attention between different sources and thereby simultaneously has to mentally combine the contents of these sources, e.g. text and picture on a computer screen. For the now arisen split-attention effect the cognitive load is increased, especially the extraneous cognitive load. The usual solution of the problem according to Ayres et al. [9] is shown on the left side of illustration 1.

Illustration 1. Integrated version (left) and separated version (right) of the material in the experiment 1 of Moreno and Mayer [10]

Illustration 1 clarifies the material of an often replicated experiment for the Cognitive Theory of Multimedia Learning of Moreno and Mayer [10]. The picture shows the integrated display format on the left side. The according text is placed in proximity to the corresponding graphic illustration, which should be useful for learning success. The right side shows the separated version, the descriptive text is in remote distance at the lower edge of the screen. The learning success is obstructed according to Moreno et al. [10]. 2.2 New Principles of Grouping In order to find alternatives to the previous approach of spatial proximity, cognitive psychological expansions are considered. Palmer [1] has created amongst others the factors of common region and element connectedness, which have been confirmed by

Revisions of the Split-Attention Effect

617

Beck and Palmer [11] empirically. Due to restricted space only the grouping factor common region is described in this article. The grouping factor common region implies according to Palmer [12] that – all else being equal - elements are perceived as a group if they are integrated within a connected, similarly coloured or uniformly structured area with the same included contour and color. By “all else being equal” Palmer [12] means that all other features are held constant or being eliminated, the so called "ceteris-paribus-rule". However if this is not the case, an estimation of the result can no longer be made due to the fact that interactions are neither measureable nor controllable.

Illustration 2. Example of Palmer [12] for the factor of the common region

An example for the grouping factor common region is shown in illustration 2. It's clarified that the proximity of the points is no longer important for the perceived grouping, although the points within an ellipse are more distant than the two bordering points in the two bordering ellipses. 2.3 Formulation of the Question and Hypotheses Derived from the work of Mayer [4], as well as of Moreno et al. [10] the following result is presented: Text and a graphic illustration should be grouped as near as possible on the computer screen, due to the fact that otherwise it would result in significant losses of learning performance. In this article it is argued against it, that not only the proximity between the elements “text" and "picture" is important, but also that an artificially created relationship between these elements leads to at least equal learning success for the subjects. The following hypotheses are examined: H1. The linked display format with the new principles of grouping according to Palmer [1] does not lead to less learning- and transfer performance than the integrated display format. H2. Persons with less meteorological previous knowledge benefit significantly more according to Mayer [13] - than persons with a high meteorological previous knowledge and therefore produce more and more creative solutions. H3. The animation without a descriptive text performs as a control condition significantly worse than all other test conditions, the animation is therefore not selfdescriptive.

3 The Study In this part the conducted field study will be introduced, an online-survey which was realized on the Internet, in which the subjects had to solve retention and transfer tasks regarding the meteorological phenomenon "The creation of lightning". The study has been divided into three subsequent phases. In phase one the subjects first had to judge

618

A. Mazarakis

about their own meteorological knowledge. This action is analogous to the proceeding of the Moreno et al. [10] experiment. Subsequently in phase two the subjects have been assigned by random to one of six conditions for the experiment in which a three minute-long animation about the creation of lightning has been displayed. The conditions of the experiment provided a connection between the split-attention effect and the new principles of grouping, respectively surveyed the split-attention effect itself. The conditions of the experiments were different in respect to 2 characteristic features: On the one hand the spatial proximity of text to the corresponding animation and on the other hand the used principle of grouping. By the combination of these factors the following six conditions were created: • • • • • •

The integrated text condition with the text placed in spatial proximity (IT) The integrated text condition with common region (ITCR) The control condition without a descriptive text (CG) The separated text condition with text placed in spatial distance (ST) The separated text condition with common region (STCR) The separated text condition with element connectedness (STEC)

Due to the restrictions of the length of this article only two out of six illustrations are shown: STCR and IT. The English translation of the German text is always: “Warmed moist air rises rapidly”.

Illustration 3. Pictures of the animation in the experiment about the creation of lightning. Hereby it is shown on the left side the integrated condition and on the right side the separated condition with common region in German language (STCR).

In phase three the subjects answered five open questions with time constraint connected to the seen animation. The questions in full detail were: 1.) Please explain how lightning works. 2.) What could you do to decrease the intensity of lightning? 3.) Suppose you see clouds on the sky, but no lightning. Why not? 4.) What does air temperature have to do with lightning? 5.) What causes lightning? Therefore the first question was the retention question, questions 2 to 5 the transfer questions. For every correct answer a point was awarded, false answers were not counted.

4 Results for Retention and Transfer Performance The sample included 869 subjects, with 452 of male gender. The subjects were on average 25 years old (sd=7), 63 % were students.

Revisions of the Split-Attention Effect

619

Table 1. F-values for the comparison of the test conditions for transfer performance. Italic printed values cannot be interpreted in an unequivocal way due to the „ceteris-paribus-rule“.

IT ITCR CG ST STCR STEC

IT -------------------

ITCR .01 ----------------

CG 9.43** 10.80*** -------------

ST .74 .91 5.17* ----------

STCR .32 .42 9.82** .26 -------

STEC .24 .33 6.80** .13 .01 ----

* p < 0.05; ** p < 0.01; *** p < 0.001. It is apparent from table 1 that the transfer performance in the linked text conditions (ITCR, STCR and STEC) is not significantly worse than in the integrated text (IT) condition, the first hypothesis is accepted. The results for the retention performance are the same, although not displayed due to page restrictions. Also the control group without descriptive text performed significantly worse in both conditions. Therefore the third hypothesis is accepted, the animation is not self-descriptive. The analysis of variance for the second hypothesis did not lead to significant results with FR(1,867) = 1.47, p < 0.3 and FT(1,867) = 1.30, p < 0.3 respectively, the null hypothesis was retained with no significant differences for both groups.

5 Discussion and Outlook The aim of this article was to test additional possible solutions for the split-attention effect in an empirical way. The until now used way of spatial proximity for knowledge acquisition and knowledge transfer of multimedia contents was extended by the new principles of grouping of Palmer [1] in cognition psychology, detailed by common region and element connectedness. The first hypothesis regarding the equal value of the linked text conditions and the integrated text condition was supported. An artificial connection of the elements text and picture didn't lead to significantly worse results than a display of these elements in spatial proximity. On the other hand the animation without a descriptive text was not self-descriptive, the third hypothesis was confirmed. The results by Mayer [13], which show that novices especially benefit from the integrated formats, couldn't be verified in the course of this study, the second hypothesis was therefore rejected. This study is measured by its size of the sample probably the largest in the context of research done on the split-attention effect. The number of subjects of the 37 studies in the meta-analysis of Ginns [14] in respect to this effect were mostly in the range of two number digits, sometimes even in the very low three digits number of subjects. The results of the present study as well as the results of related studies of Michas and Berry [15], and of Bodemer et al. [16] generally lead to doubts about the often commonly cited universal validity of the split-attention effect. But it has to be stated that the two mentioned studies didn't have the aim of questioning the effect, but can only be interpreted in that direction by the non discovery of this effect.

620

A. Mazarakis

In conclusion it can be recorded that the split-attention effect cannot be replicated as universally valid and science has to carve out the relevant conditions for the occurrence of this effect in the future. However the new principles of grouping have successfully passed their debut in research about the Cognitive Load Theory due to the acceptance of the first hypothesis and should be investigated more extensively and should be used more often in this context.

References 1. Palmer, S.E.: Vision science: Photons to phenomology. MIT Press, Cambridge (1999) 2. Sweller, J., van Merriënboer, J.J.G., Paas, F.G.W.C.: Cognitive architecture and instructional design. Educational Psychology Review 10, 251–296 (1998) 3. Mayer, R.E.: Cognitive theory of multimedia learning. In: Mayer, R.E. (ed.) The Cambridge Handbook of Multimedia Learning, pp. 31–48. Cambridge University Press, New York (2005) 4. Mayer, R.E.: Multimedia Learning. Cambridge University Press, New York (2001) 5. Sweller, J.: Implications of cognitive load theory for multimedia learning. In: Mayer, R.E. (ed.) The Cambridge Handbook of Multimedia Learning, pp. 19–30. Cambridge University Press, New York (2005) 6. Baddeley, A.D.: Human memory: Theory and practice (Rev. ed.). Psychology Press, Hove (1997) 7. Sweller, J.: Evolution of human cognitive architecture. The Psychology of Learning and Motivation 43, 215–266 (2003) 8. Mayer, R.E., Moreno, R.: Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist 38, 43–52 (2003) 9. Ayres, P., Sweller, J.: The split-attention-principle in multimedia learning. In: Mayer, R.E. (ed.) The Cambridge Handbook of Multimedia Learning, pp. 135–146. Cambridge University Press, New York (2005) 10. Moreno, R., Mayer, R.E.: Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology 91, 358–368 (1999) 11. Beck, D.M., Palmer, S.E.: Top-down influences on perceptual grouping. Journal of Experimental Psychology: Human Perception and Performance 28, 1071–1084 (2002) 12. Palmer, S.E.: Common region: A new principle of perceptual grouping. Cognitive Psychology 24, 436–447 (1992) 13. Mayer, R.E.: Multimedia Learning: Are we asking the right questions? Educational Psychologist 32, 1–19 (1997) 14. Ginns, P.: A meta-analysis of the spatial contiguity and the temporal contiguity effects. Learning and Instruction 16, 511–525 (2006) 15. Michas, I.C., Berry, D.C.: Learning a procedural task: Effectiveness of multimedia presentations. Applied Cognitive Psychology 14, 555–575 (2000) 16. Bodemer, D., Plötzner, R., Feuerlein, I., Spada, H.: The active integration of information during learning with dynamic and interactive visualisations. Learning and Instruction 14, 325–341 (2004)

Grid Service-Based Benchmarking Tool for Computer Architecture Courses Carlos Alario-Hoyos, Eduardo Gómez-Sánchez, Miguel L. Bote-Lorenzo, Guillermo Vega-Gorgojo, and Juan I. Asensio-Pérez School of Telecommunication Engineering, University of Valladolid, Camino Viejo del Cementerio s/n, 47011 Valladolid, Spain {calahoy@gsic,edugom@tel,migbot@tel,guiveg@tel,juaase@tel}.uva.es

Abstract. Benchmarking for educational purposes in the context of computer science can be hindered by the low number and the homogeneity of machines to be assessed, and the inaccuracy of the benchmarks to represent specific workloads. Thus, this paper proposes a benchmarking tool developed within a service-oriented grid in order to allow students to benchmark multiple workloads in machines that may belong to several educational institutions. This tool has been validated in a real educational scenario within a course on Computer Architecture. Keywords: Benchmarking, education, architecture, service-oriented grid.

1

Introduction

As part of their learning process, computer science students should develop skills related to the design and evaluation of computer systems. To achieve these learning objectives, the Association for Computer Machinery and the IEEE Computer Society state in their guidelines for Computing Curricula [1] that educators should challenge students with realistic scenarios, so that they can reflect on measurement techniques, as well as on the impact of computer organization to the performance of computer systems for specific workloads. Thus, benchmarking (i.e. execution of pieces of software to get performance measures for standardized workloads) plays a significant role as a quantitative measurement approach. Indeed, several Computer Architecture courses (e.g. [2] or [3]) make use of benchmarking to illustrate basic principles such as the dependence of performance on the workload or design driven to improve the performance/cost relationship. Though it is easy to include a benchmarking activity that supports the evaluation of one machine (for example the lab main server) with a couple of workloads in a Computer Architecture course, it is far more challenging and complex for educators to propose a scenario in which students must advise a realistic customer to acquire a computer system suitable for its workload, due to several reasons. 

This work has been partially funded by the Spanish Ministry of Science and Innovation (TIN2008-03023) and the Autonomous Government of Castilla y Leon, Spain (VA106A08).

U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 621–626, 2009. c Springer-Verlag Berlin Heidelberg 2009 

622

C. Alario-Hoyos et al.

First of all, the number of different machines available for benchmarking is often reduced in most educational institutions, thus hindering the educational interest of the activity and biasing the conclusions of the students. This is mostly because some machines can be too expensive, but it also happens often that available computer systems are much alike in features because they have been acquired simultaneously to benefit from vendor discounts. Moreover, existing machines are often outdated and might not represent an up-to-date realistic case study. Besides, the benchmarks available for these machines may not represent the intended workload making conclusions reached by the students unreliable. In addition, benchmarking increases security concerns since benchmarks are normally run locally in computer systems, and thus students and educators should remotely connect to these machines. This entails additional problems: the administrator burden is increased to create accounts or allow somehow these connections, besides configuring machines and installing benchmarks; and students and educators should handle a larger number of logins, passwords and commands for the remote connection and the execution of benchmarks in different machines, increasing their cognitive load. It may very well happen that students concentrate too much on the procedure of benchmarking, instead of devoting efforts to plan the experiments or to interpret the results. Many of these limitations could be overcome if there was a way that several academic units (let them be departments, schools or whole universities) could share their machines in a secure environment, so that they could be used for benchmarking in addition to other processes. The pool of machines to be benchmarked would thus be much larger and more diverse, enriching the learning activity. The administrative burden could be somehow shared among the different unit’s administrators. Even more, the complexity of handling logins, passwords or commands could be hidden by a visual front-end. In this context, this paper proposes a Grid Service-Based Benchmarking Tool that makes use of the service-oriented grid [4], that allows institutions to federate and share computational resources in a secure and controlled way. This tool has been designed and developed with a front-end to the grid to facilitate educators the definition of an environment for the assessment of computer systems, and students the execution of benchmarks without having to care for establishing remote connections. The approach of sharing computational resources for learning has somehow been followed in the literature. For example, authors in [5] proposed a web portal to allow students and educators to run simulation tools in distributed computer resources, including some tools that might be useful in Computer Architecture courses such as the DLXView or CacheSim5 simulators. Besides, education can be considered as one relevant field in which grid computing can be applied [6], and so, applications such as the collaborative network simulation environment in [7] or the online grid service-based laboratory in [8] have been developed. However, none of them uses the service-oriented grid technology to share machines as a pool of resources for benchmarking, as proposed in this paper.

Grid Service-Based Benchmarking Tool

2

623

Grid Service-Based Benchmarking Tool

This section first justifies the use of the service-oriented grid to overcome the limitations that have been previously introduced. Then, an overview of the Grid Service-Based Benchmarking Tool is reported to steer towards a prototype. 2.1

Service-Oriented Grid to Overcome the Limitations

A computational grid is a large-scale infrastructure composed by heterogeneous resources that are shared by multiple administrative organizations [9]. The access to these resources may be granted to the authorized members by the grid middleware. A service-oriented grid exposes these virtualized resources as services according to OGSA (Open Grid Service Architecture) [10] and the WSRF (Web Services resource Framework) [11] specifications. They promote the transparent access to the resources through a well-defined service interface. Thus, within a service-oriented grid, different educational institutions, distributed geographically, could announce benchmarks that can be run in their machines for the benefit of their Computer Architecture courses. In addition, the service-oriented grid can decrease the administration burden by splitting the administration tasks. The main reason is that the distributed members in the grid usually have their own administrators. A service-oriented grid also provides the infrastructure in charge of controlling the access to a secure environment, for example through credential management or delegation mechanisms. Finally, the service-oriented grid can abstract low level details about resources, as they are exposed through a well-defined service interface. Therefore, users do not need to know how to communicate with remote machines for benchmarking as it is internally done between services and resources. 2.2

Grid Service-Based Benchmarking Tool Architecture

As any other service-oriented application, the design of the Grid Service-Based Benchmarking Tool implies splitting the functionality into a set of different services to maximize the reusability when building other applications. Each of the identified services can be offered by one or more institutions participating in the grid, running them using their own local resources. The general architecture of the Grid Service-Based Benchmarking Tool, as well as the set of identified services is shown in Figure 1 and is described next. The benchmark service is a front-end that allows any institution offering a set of machine/benchmark pairs. The administrator in the local institution, through the administration client, only needs to provide access to the machines in which benchmarks run, to any authorized user. The integration service allows educators through their educator client to select for their students a subset of machine/benchmark pairs from the ones offered by different institutions and gather them as a collection of benchmarks. The index service supports the register and discovery of resources and services. In this case, the index service is used by educators to find machine/benchmark pairs or by students

624

C. Alario-Hoyos et al. Educational institution 2 Client - Service Service - Service Service - Resource

Administrator Benchmark Service

Administration. Client

Index Service

Index Service

Index Service

Integration Service Credential Repository Service

Credential Repository

Benchmark Service

Educator Client

Administration . Client

Educator

Administrator Benchmark Service

Student

Student

Student Client

Educational institution 1

Educational institution delivering Computer Architecture courses

Educational institution 3

Fig. 1. Grid Service-Based Benchmarking Tool architecture. There must be one index service and at least one benchmark service in the institutions offering machines. There must be also at least one credential repository service and at least one integration service.

through the student client to find collections of benchmarks and subsequently execute them transparently. In addition, the credential repository service enables secure access to the tool through credentials. 2.3

Grid Service-Based Benchmarking Tool Prototype

A prototype of the Grid Service-Based Benchmarking Tool has been developed according to the WSRF standards and supported by the middleware Globus Toolkit 4.0 [12]. This prototype implements the following services: benchmark service, integration service, and index service. The first two have been developed from scratch while the last one belongs to the middleware. The internal communication between the benchmark service and the machines that it abstracts is based on SSH (Secure Shell ). It entails an advantage because the machines do not need to be configured to run a service execution environment, thus there is no need to install the middleware and deploy services on them. Instead, SSH access to the machine needs only to be granted through the frontend service. The three clients (administration client, educator client and student client) have been implemented and distributed with Java Web Start being exposed to the users with a GUI. As an example, Figure 2 represents the GUI screenshoot from the Administration and Student Clients.

Grid Service-Based Benchmarking Tool

625

Fig. 2. Screenshots from the Grid Service-Based Benchmarking Tool prototype. a) Administration Client adding a new machine/benchmark pair (verdejo.lab2.tel.uva.es/Dhrystone); b) Student Client executing the benchmark Dhrystone on verdejo.lab2.tel.uva.es, and obtaining the results.

3

Validation

Computer Architecture is a fourth year course in the Degree of Telecommunication Engineering (University of Valladolid, Spain). One of the tasks of this course consists on the students assessing and comparing the performance of several real machines through benchmarks to determine which is the most suitable for a given customer workload. In this educational scenario some experiences with the Grid Service-Based Tool prototype were carried out, using 36 machine/benchmark pairs from two educational institutions: the Computer Architecture lab and the GSIC (Intelligent & Cooperative Systems Group) research group. A questionnaire was voluntarily answered by 47 students after the experience, to detect general tendencies on the validity of the tool and to make suggestions for its improvement. As a sample result, 95.6% of students agreed or completely agreed with the easiness of use of the tool, supporting their opinions with the reduction of their cognitive load. In addition, more than 90% agree with the usefulness of this tool in the context of this course. Nevertheless, students cannot express any opinion about the underlying architecture and technology, because they interact with a front-end client that allows the execution of benchmarks in machines, no matter where they are located, or how this execution is done. Additionally, administrators expressed some positive opinions when configuring this tool. For example, one remarked that he needed to invest less time than in previous years all along the benchmarking activity. The reason is related to the fact that students did not connect explicitely to the machines and thus, no one changed the password or deleted shared files, saving him the time to restore the original configuration. Besides, educators did not found problems when using the tool and even point out that students needed less assistance than in previous experiences.

626

4

C. Alario-Hoyos et al.

Discussion

The Grid Service-Based Benchmarking Tool has proved to overcome the limitations found in typical educational scenarios in terms of available machines and benchmarks, by sharing distributed and more varied resources between several institutions. Furthermore, the administration burden is also shared among local institutions, and even simplified in the provisioning of access to authorized users. Additionally, the tool removes low-level details for students and educators through an execution front-end, reducing their cognitive load and letting them focus on benchmarking plans and results instead of the procedure. Nevertheless, some improvements can be done in the design of this tool. For example, an analysis service can be considered to facilitate the interpretation of the results with statistics for the same benchmark and different loads in the machine to be assessed. Besides, a visualization service may compare graphically these statistics in terms of response time or throughput. Additionally, the architecture could be complemented with an execution service that would be used by administrators to install and deploy benchmarks from a benchmark source repository with several benchmarks compiled for different architectures.

References 1. ACM/IEEE: Computing Curricula 2007: Guidelines for Associate-Degree Transfer Curriculum in Computer Engineering (2007), http://www.acmtyc.org/re-ports/ TYC_CEreport2007Final.pdf (last visited May 2009) 2. Martínez-Monés, A., et al.: Multiple Case Studies to Enhance Project-Based Learning in a Computer Architecture Course. IEEE Transactions on Education 48(3), 482–489 (2005) 3. Figueiredo, R.J., et al.: Network-Computer for Computer Architecture Education: a Progress Report. In: 2001 ASEE. Albulquerque, New Mexico (June 2001) 4. Foster, I., Kesselman, C.: The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann Publishers, San Francisco (1998) 5. Kapadia, N.H., et al.: PUNCH: Web Portal for Running Tools. IEEE Micro 20(3), 38–47 (2000) 6. Fox, G.: Education and the enterprise with the grid. In: Berman, F., Fox, G.C., Hey, A.J.G. (eds.) Grid Computing: Making the Global Infrastructure a Reality, pp. 963–976. John Wiley and Sons, Chichester (2003) 7. Bote-Lorenzo, M.L., et al.: A Grid Service-Based Collaborative Network Simulation Environment for Computer Networks Education. In: ICALT 2007, Niigata, Japan (July 2007) 8. Bagnasco, A., et al.: Computational grids and online laboratories. In: ELeGI Conference. Workshops in Computing, BCS Napoles, Italy (2005) 9. Bote-lorenzo, M.L., et al.: Grid Characteristics and Uses: A Grid Definition. In: Fernández Rivera, F., Bubak, M., Gómez Tato, A., Doallo, R. (eds.) Across Grids 2003. LNCS, vol. 2970, pp. 291–298. Springer, Heidelberg (2004) 10. Foster, I., et al.: The Open Grid Services Architecture, Version 1.0. Technical report, Gridforum (2005) 11. Committee Draft 02: Web Service Resource Framework (WSRF) - Primer v1.2. Technical report, Oasis (2006) 12. The Globus Alliance, http://www.globus.org (Last visited May 2009)

Supporting Virtual Reality in an Adaptive Web-Based Learning Environment Olga De Troyer, Frederic Kleinermann, Bram Pellens, and Ahmed Ewais Vrije Universiteit Brussel, WISE Research Group, Pleinlaan 2, 1050 Brussel, Belgium {Olga.DeTroyer,Frederic.Kleinermann,Bram.Pellens, Ahmed.Ewais}@vub.ac.be

Abstract. Virtual Reality (VR) is gaining in popularity and its added value for learning is being recognized. However, its richness in representation and manipulation possibilities may also become one of its weaknesses, as some learners may be overwhelmed and be easily lost in a virtual world. Therefore, being able to dynamically adapt the virtual world to the personal preferences, knowledge, skills and competences, learning goals and the personal or social context of the learning becomes important. In this paper, we describe how an adaptive Web-based learning environment can be extended from a technological point of view to support VR. Keywords: Virtual Reality, E-Learning, Adaptive Learning Environment.

1 Introduction Virtual Reality (VR) provides ways to use 3D visualizations with which the user can interact. For some learning situations and topics, VR may be of great value because the physical counterpart may not be available, too dangerous or too expensive. The most famous example is the flight simulator that pilots safely teaches how to fly. Most of the time, when VR is considered for learning, it is offer as a stand-alone application (e.g., [1], [2], [3]) and there is usually no way to adapt it to personal preferences, prior knowledge, skills and competences, learning goals and the personal or social context of the learner. Augmenting a virtual world with adaptive capabilities could have many advantages [4]. It may be more effective to guide learners through the world according to their background and learning goals, or only show them the objects and behaviors that are relevant for their current knowledge. In this paper, we explain how VR can be supported in the context of an adaptive Web-based learning environment developed in the context of GRAPPLE, an EU FP7 project. GRAPPLE is mainly oriented towards classical learning resources, but the use of other types of learning materials (VR and simulations) is also investigated. Here, we concentrate on how the learning environment is extended to support VR.

2 The GRAPPLE Architecture GRAPPLE aims at providing a Web-based adaptive learning environment. The two main components are the Authoring Tool and the Adaptive Engine (see figure 1). The U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 627–632, 2009. © Springer-Verlag Berlin Heidelberg 2009

628

O. De Troyer et al.

Authoring Tool (Web-based) allows a course author to define a course at a conceptual level. This is done by means of a (graphical) Domain Model (DM) and Conceptual Adaptation Model (CAM) [5]. The DM describes the concepts that should be considered in the course. The CAM expresses at a high-level and by using pre-defined pedagogical relations (such as the prerequisite relation) how the content and structure needs to be adapted at runtime. The authoring tool can also be used (by a more experienced person) to define new pedagogical relations, called CRTs. Defining a CRT also implies defining the adaptive behavior associated with the relation. Different adaptive behaviors may be possible for the same pedagogical relation, e.g., the prerequisite relation (“A is prerequisite for B”) can be associated with an adaptive behavior that hides the dependent concept B as long as A has not been studied, or with an adaptive behavior that forces the learning sequence A than B. Next, the graphical CAM is translated into a format (called GAL – Generic Adaptation Language) that the adaptive engine can handled. During this translation, the adaptive behaviors, associated with the definition of the pedagogical relations, are used to express the desired adaptive behavior for the course.

Fig. 1. GRAPPLE Architecture

The Adaptive Engine does the actual adaptive delivery of the content. Based on the state of the learner’s profile (captured in the User Model) and the GAL specifications, the Adaptive Engine will select the proper learning resources and deliver the required navigation structure and content to a Web browser. The Adaptive Engine also keeps track of the progress of the learner. Internal variables are maintained in order to be able to instruct the User Model service what to update in the learner’s User Model. This allows runtime adaptation. The adaptive engine of GRAPPLE is implemented as a client-server application. All its functionality is located at the server-side.

3 Adaptive VR in the Context of GRAPPLE To support VR, it was necessary to extent the architecture of GRAPPLE. The VR material considered may range from some simple 3D objects to complete 3D virtual environments (VE) in which the user can navigate and interact with the 3D (or 2D) objects in it. Because GRAPPLE is a web-based learning environment, it is using the XML format for the learning resources. For displaying 3D objects inside a browser, X3D [6] can be used, which is XML-based. Therefore, individual 3D resources can be included or excluded in the same way as the regular XML resources. However, to

Supporting VR in an Adaptive Web-Based Learning Environment

629

support adaptive VE’s, extensions are necessary for the authoring tool and for the adaptive engine. These extensions are necessary because the regular GRAPPLE tools consider a resource as a black box. To be able to adapt the VE itself, i.e. adapting the presentation of the objects in the world, enabling and disabling behaviors and interaction, including objects conditionally, and/or providing dedicated navigation possibilities in the virtual world, this approach is no longer suitable. Although some of the adaptations can be seen as extensions of adaptations for text resources, they are very specific for 3D material and the possibilities are much richer. For instance, to visually indicate that an object has not yet been studied, we may want to give it a different color or make it smaller; when a learner is studying a complex object (like a planet of the solar system), the visual appearance of the object could change according to the aspects being studied (size, temperature, geography, …) or become more detailed while more and more knowledge is acquired. It may also be necessary to disable or enable behavior and the interaction for an object according to the progress of the learner. To allow specifying this, a dedicated VR authoring tool is necessary. Furthermore, it is necessary to provide a preview of the VE to the author while he is specifying the adaptations because otherwise he needs to specify adaptations blindly, which may be very difficult. For instance, it is not possible to replace one 3D object in a VE by any other 3D object, as this object may not fit into the VE. Also behaviors are usually strongly connected to the actual 3D object, and it is not always possible to replace a behavior by any other behavior. The VR authoring tool is also implemented as a Web application (see figure 2). It allows specifying VR-specific CRTs and has a component to define CAMs. The VR specific authoring tool does not need a specific DM component; it uses the DM tool of the general authoring tool. Note the availability of a Previewer. The Loading component is responsible for retrieving the necessarily information from the different repositories (using available web services) and for retrieving the VR resources that needs to be previewed. The Saving component is responsible for storing all the information defined by the author, i.e. newly defined CRTs and CAM specifications. The output format is GAL but we also have an independent XML format to be able to connect to other systems. The extension of the adaptive engine towards VR is realized by means of a browser plug-in (see figure 3) responsible for (1) updating the VE if the adaptive engine instructs to do so, (2) monitoring the learner’s behaviour, and (3) sending information about the learner’s behavior to the Adaptive Engine. For the retrieval of VR content by the VR browser plug-in, a server-side plug-in is added, the VR-Manager. The VR browser plug-in has three components namely the Monitor component, the Update component, and the VR player. An existing VR player is used to visualize the VR content. We require that the VR player supports Document Object Model (DOM) [7], JavaScript [8], Scene Authoring Interface (SAI) [9], and X3D [6]. The scene Authoring Interface is used to communicate to the VR player. In this way, it is possible to update at runtime the scene graph without the need to reload it completely. The Update Component is responsible for interpreting the adaptation requests received from the Adaptive Engine and for translating it into a form that can be understood by the VR player; it instructs the VR player to update the scene. The Monitor Component is responsible for keeping track of what happens in the VE and translating this in a form understandable by the Adaptive Engine, which on its turn will inform the User Model service about the progress made by the learner.

630

O. De Troyer et al.

Fig. 2. VR Authoring Tool - Components

Fig. 3. Adaptive VR Delivery

Fig. 4. Studying the Sun

To validate parts the adaptive delivery of the VR material, we have created a prototype and elaborated an example course with it. As the adaptive engine of GRAPPLE was not yet available at that time, the prototype is based on AHA! 3.0 [10], an adaptive learning engine on which the GRAPPLE adaptive engine will be based. As VR player we have used Ajax3D [11] that uses the Vivaty player [12]. It can use Ajax and can be embedded inside Firefox and Internet Explorer. Both the Monitor Component and the Update Component have been prototyped. To test the prototype, an example adaptive course has been developed. The adaptive course is about the solar system. To investigate the issues related to the combination of different types of content, this course contains plain text explaining the solar

Supporting VR in an Adaptive Web-Based Learning Environment

631

system, as well as a VE of the solar system were the sun and different planets are displayed in 3D (see figure 4). The text as well as the VE will adapt according to the learner’s knowledge and progress. E.g., a planet appears when the learner starts to study it; planets that have been studied will stop rotating; and when all planets are studied the whole solar system is available in the VE.

4 Related Work Brusilovsky et al. [13] have integrated some adaptive hypermedia methods (mainly for navigation) into Virtual Environments. The approach of Santos and Osorio [14] is based on agents that help the user by providing him more information about interesting products, and by guiding him to their preferred area. The work of Moreno-Ger et al. [15] on 2D games is not on VR but is interesting as it provide an authoring tool allowing authors to create adaptive courses. Celentano and Pittarello [16] monitor a user’s behavior and compare it with previous patterns of interaction. Whenever the system detects that the user is entering a recurrent pattern, it may perform some activities on behalf of the user. Chittaro and Ranon did quite some related work, first in the context of e-commerce, later on also for e-learning. Some of their work can be found in [17], [18] and [19]. The work in [19] is close to ours, and especially to the prototype that we have developed and which is also using AHA! (see section 3). However, they don’t provide an authoring tool for specifying the adaptive story lines like we do. Furthermore, the main file containing the VE is reloaded at fixed time intervals to keep the adaptation inline with the student’s user model. This is a serious drawback, especially for large VEs, because it will make the system very slow. In our approach, runtime adaptations don’t require reloading the complete VE.

5 Conclusions and Future Work This paper describes an approach to support the adaptive delivery of Virtual Reality learning material inside GRAPPLE, a Web-based adaptive learning environment. The approach is innovative from different aspects. Firstly, it contains a visual authoring tool for specifying the adaptive strategy for a VE. Next, the adaptation of the VE is done at run-time without the need to reload the VE each time, which will provide the necessarily performance required for large VE’s. In addition, VR material and classical (textual and 2D multimedia) can be integrated in a single course and the adaptation can be performed for whatever type of content. The activities performed by the learner in the VE can be monitored and the effect can be directly reflected in the VE. First a prototype has been developed for the adaptive delivery. Currently, the implementation of the VR-authoring tool has been started as well as the implementation of the final VR-plug-in. Several experiments are planned to validate the approach as well as it usability and effectiveness. Acknowledgments. This work is realized in the context of the EU FP7 project GRAPPLE (215434). The design of the overall GRAPPLE architecture has been a collaborative effort of the different partners.

632

O. De Troyer et al.

References 1. Alexiou, A., Bouras, C., Giannaka, E., Kapoulas, V., Nani, M., Tsiatsos, T.: Using VR technology to support e-learning: the 3D virtual radiopharmacy laboratory. In: Distributed Computing Systems Workshops, 2004. Proceedings 24th International Conference, pp. 268–273 (2004) 2. KM Quest, http://www.kmquest.net 3. De Byl, P.: Designing Games-Based Embedded Authentic Learning Experiences. In: Ferdig, R.E. (ed.) Handbook of Research Effective Electronic Gaming in Education. Information Science Reference (2009) 4. Chittaro, L., Ranon, R.: Adaptive Hypermedia Techniques for 3D Educational Virtual Environments. IEEE Intelligent Systems 22(4), 31–37 (2007) 5. Hendrix, M., De Bra, P., Pechenizkiy, M., Smits, D., Cristea, A.: Defining adaptation in a generic multi layer model: CAM: The GRAPPLE Conceptual Adaptation Model. In: Dillenbourg, P., Specht, M. (eds.) EC-TEL 2008. LNCS, vol. 5192, pp. 132–143. Springer, Heidelberg (2008) 6. Brutzman, D., Daly, L.: X3D: Extensible 3D graphics for Web Authors. The Morgan Kaufmann Series in Interactive 3D technology (2008) 7. W3C Document Object Model, http://www.w3.org/DOM/ 8. JavaScript, http://www.javascript.com 9. Scene Authoring Interface Tutorial, http://www.xj3d.org/tutorials/general_sai.html 10. AHA! 3.0, http://aha.win.tue.nl/ 11. Ajax3D, http://www.ajax3d.org/ 12. Vivaty, http://www.vivaty.com/ 13. Brusilovsky, P., Hughes, S., Lewis, M.: Adaptive Navigation Support in 3-D E-Commerce Activities. In: Proceedings of Workshop on Recommendation and Personalization in eCommerce at the 2nd International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH 2002), Malaga, Spain, pp. 132–139 (2002) 14. dos Santos, C.T., Osorio, F.S.: AdapTIVE: An Intelligent Virtual Environment and Its Application in E-Commerce. In: Proceedings of 28th Annual International Computer Software and Applications Conference (COMPSAC 2004), pp. 468-473 (2004) 15. Moreno-Ger, P., Sierra-Rodriguez, J.L., Ferandez-Manjon, B.: Games-based learning in elearning Environments. UPGRADE 12(3), 15–20 (2008) 16. Celentano, A., Pittarello, F.: Observing and Adapting User Behaviour in Navigational 3D interface. In: Proceedings of 7th International Conference on Advanced Visual Interfaces (AVI 2004), pp. 275–282. ACM Press, New York (2004) 17. Chittaro, L., Ranon, R.: Adaptive 3D Web Sites. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) Adaptive Web 2007. LNCS, vol. 4321, pp. 433–462. Springer, Heidelberg (2007) 18. Chittaro, L., Ranon, R.: Adaptive Hypermedia Techniques for 3D Educational Virtual Environments. IEEE Intelligent Systems 22(4), 31–37 (2007) 19. Chittaro, L., Ranon, R.: An Adaptive 3D Virtual Environment for Learning the X3D Language. In: Proceedings of the 2008 International Conference on Intelligent User Interfaces (IUI 2008), pp. 419–420. ACM Press, New York (2008)

A Model to Manage Learner’s Motivation: A Use-Case for an Academic Schooling Intelligent Assistant Tri Duc Tran1,2, Christophe Marsala1, Bernadette Bouchon-Meunier1, and Georges-Marie Putois2 1

UPMC-CNRS-LIP6, 104 avenue du président Kennedy, 75016 Paris, France {Tri-Duc.Tran,Christophe Marsala, Bernadette Bouchon-Meunier}lip6.fr 2 ILOBJECTS, 104 avenue du président Kennedy, 75016 Paris, France {tran,gmputois}ilobjects.com

Abstract. The scope of our research is to build a non pedagogical intelligent assistant, I-CAN (Intelligent Coach and Assistant to New way of learning) that supports students during their academic schooling and prevents them from dropout. The management of learner’s motivation is one of the main features for schooling success. A high level of motivation implies more engagement and positive emotion to overcome difficulties. The problematic of this paper is “How to enhance the student’s motivation during his academic schooling?” We propose a motivation management framework for a personal intelligent assistant on a LMS (Learning Management System). This framework takes in input data from learner’s academic schooling (absenteeism, tardiness, marks, tasks, sanction) to diagnose learner’s state and to enhance the motivation to learn through an embodied conversational agent. Keywords: motivation diagnosis, motivation enhancement, schooling support, personal intelligent, dropout.

1 Introduction The motivation management is a problem widely developed in the ITS (Intelligent Tutoring System) field but few works are focused on the academic schooling motivation support. The purpose of this paper is to analyze the previous researches and to propose a model of motivation management system for a personal assistant to support students during their schooling. Our assistant I-CAN is designed to be used in a LMS (Learning Management System) or ENT (Espace Numérique de Travail), to assist student’s academic schooling. The data to analyze are principally oriented to the academic learning as absenteeism, tardiness, results, tasks, punishments, ... In a multiagent view, our system will collaborate with an ITS in providing the analysis of the student’s academic schooling. The goal of the motivation intelligent system is to maintain or increase the student’s desire to learn and her/his willingness to expand effort in undertaking the activities that lead to learning [5]. U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 633–638, 2009. © Springer-Verlag Berlin Heidelberg 2009

634

T.D. Tran et al.

The research questions raised in this paper, in the academic context, are: • • •

“What are the mechanisms to evaluate the student motivation?” “Which are the intervention strategies to enhance the student’s motivation”, “How to implement theses strategies for a personal schooling intelligent agent?

To determine the mechanisms to evaluate the student motivation, we will analyze the previous researches in learner’s motivation in the first section and propose a model in three layers to assess the learner’s motivation state from the academic schooling data. In the second section, we will analyze the different strategies and interventions to enhance the learner’s motivation and present our approach.

2 Academic Schooling Motivation Diagnosis Most motivation theorists are convinced that motivation is involved in the learning process. A simple definition of motivation can be [8]: • • • •

internal state or condition that activates behavior and gives it direction; desire or will that energizes and directs goal-oriented behavior; influence of needs and desires on the intensity and direction of behavior; arousal, direction, and persistence of behavior.

2.1 Motivational States Computation In a motivational intelligent system, the first challenge is to determine the learner’s motivational state. We can do it with self-report methods, intelligent analysis from sensors to determine the affective state or with the study of data from the interaction between learners and learning content. The table 1 is an extended review of measuring motivation through user-computer/ITS interaction data [7]. It resumes the different research directions in the learner’s motivation diagnosis. We find that: 1) The motivation is not directly evaluated, the diagnosis uses intermediate indicators. 2) The motivation’s diagnosis uses in most of the cases at least the engagement and the confidence indicators. 3) The computation parameters come from the interaction with a learning content. In our case we don’t have data from the interaction with a learning content; our assistant I-CAN has only the data from the academic schooling. So the input data are: – Academic data: assessment results, quarterly reports, tardiness, absenteeism,

suspension, disciplinary, sanction, trouble relationships with adults … – Interaction data: between the student and the intelligent assistant and with the

information system. These data won’t permit us to assess indicators such as the confidence, confusion, attention, independence. We will focuse on the diagnosis of the performance (regularity, progression, result), the efficiency, the engagement, and emotional/ physical conditions.

A Model to Manage Learner’s Motivation

635

Table 1. Previous researches on motivation recognition Indicators Engagement Engagement

Confidence, confusion, effort Control, challenge, independence, fantasy; Confidence, sensory/cognitive/intere st, effort, Attention and confidence

Engagement, energizing, source of motivation (internal or external) Effort, confidence, independence

Confidence, effort

Computation parameters Time and performance to multiple choice question assessments. number of reading pages; average time spent reading; number of taking tests; average time spend on the quiz; total time for a learning sequence Time to perform the task; time to read the paragraph related to this task; decision time to perform a task; number of finished tasks; number of task performed Interaction with the system : mouse movement Performance to a test; speed to answer; give up

Authors [1]

Number of compiling; number of compiling without errors; ratio of working time and class’ average; number of hints in the process of doing task; number of execution; time from start until typing in editor Time on task percentage; average session duration; average pace of activity within sessions; average time between sessions; exam activities percent; game activities percent Challenge seeking : choice of challenge ‘s level; business : number of help request, adding/deleting organism; hopping : switches one view to another Degree of quiz using In the learning content : spent time, Help request, Number of activities In the exercise : quality of problem solving , spent time, hint or solution request, relevant request; Self report, Answer to system question

[13]

[3]

[10]

[4]

[7]

[11]

[9]

2.2 Our Model of Motivation Diagnosis Our model of motivation is made of three steps and we divide the main problematic: “How to diagnose the learner’s motivation?” into sub-problems to solve. Motivation is an aggregation of performance and efficiency. The first computation step consists in pre-processing the input data and summarizing them into four types of indicators: 1) Performance: result, regularity and progression indicators are calculated from the academic results. It shows the student’s global performance. 2) Emotional state: motivation and emotion are closely linked; the emotion can have an impact on the motivation, and vice versa. We can determine it by a self report method or the analysis of student’s interaction with the assistant and/or LMS. 3) Physical state: self report methods can be correlated with absences or tardiness. 4) Engagement/effort: tardiness, absenteeism, suspension, disciplinary sanction, trouble relationships will help us to analyse the student’s engagement.

636

T.D. Tran et al.

Fig. 1. Motivation diagnosis model

Then the efficiency factor is the aggregation of performance, physical state and the engagement indicators. This model can be implemented by fuzzy rules system and association rules to determine indicators and theirs aggregation. The motivational and its intermediate indicators will help us to design strategies of dialogue acts such as positive feedback, encouragement and praise adapted to the student’s schooling state.

3 Motivation Enhancement Once the learner’s motivational state computed, our intelligent assistant I-CAN will use this information to adapt it to this state and to find adequate strategies to maintain or enhance the learner’s motivation. There are three mains motivation strategies: – Motivational design: it increases the effort put into learning tasks. – Learning design: it changes the learning content or selects/recommends appropriate content – Contingency design: it makes the learner confident that effort and performance are closely coupled with consequences. As our system I-CAN doesn’t manage the contents, the strategies of I-CAN are based on motivational design and contingency design. 3.1 Strategies to Manage Motivation The source of motivation can be categorized as either extrinsic or intrinsic (internal to the person) [12]. Intrinsic motivation will only occur if the learner is highly interested in the activity. If a student has an intrinsic motivation to learn, he will feel satisfaction, enjoyment, and interest [2]. Motivational and contingency design can be done with affective dialogue that contains positive feedback and praise and it uses words and phrases that help attribute success to learner’s effort and ability [14]. The interaction with the learner’s and I-CAN is in natural language with an embodied conversational module and it will follow three principles:

A Model to Manage Learner’s Motivation

637

1) To design a discourse motivational model with communication and educational theories. According to the Ginott model [6], the teacher should practice the congruent communication when giving feedback to students. Congruent communication is a way of communicating that increases self-esteem and decreases conflict. The main rule is “Talk to the situation, not to the personality and character”. 2) The global strategy is to enhance the intrinsic learner’s motivation. Students with high intrinsic motivation often outperform students with low intrinsic motivation. They engage more in learning activities and are more likely to complete course [14]. 3) The development of a self-attribution explanation of the success [8], effort and internal and control ability are needed. The motivation module is designed to be included in an embodied conversation agent; the aim of our model is to build a motivational discourse model. 3.2 Our Motivation Enhancement Design Figure 2 shows the design process to build knowledge database to manage the learner’s motivation and the global working of our motivation system. The construction of the dialogue model is based on two main processes: 1) The first database is constructed from the interview of teachers and the analysis of quarterly reports. We obtain a real corpus of motivational dialog acts (positive feedback, encouragement, praise, reassurance). 2) The second database enhances the first one with the communication and educational theories as the Ginott Model [6]. The motivation management module takes in input the data from diagnosis module and matches it with the knowledge database. Motivation management module can be designed with associated rules or with a decision tree.

Fig. 2. Motivation enhancement model

The motivational intelligent system would be able to improve our assistant I-CAN in several ways; the diagnosis can provide more information about the students’ profiles, give more details about their difficulties. The motivation enhancement module would be able to increase the relation with the student; the dialogue will be more “human” and more personalized. The motivation management can play an important role to reduce the academic dropout.

638

T.D. Tran et al.

3 Conclusion In this paper we have proposed an approach to monitor and to adapt to the learner’s motivational during his academic learning. Our assistant I-CAN can collaborate with an ITS, the information exchanged will improve the performance of the both system. One of the most important limitations of our system resides in the other parameters of the learner’s motivation: – the classroom learning, school environment : teacher, peers, learning process/activity – the social environment : peers, friends, parents, family Our future works consist in exploring more models in motivation research and in developing this motivational module. We need also to analyze the impact of the motivation in a discourse model. We will add the motivation management to our assistant I-CAN and test it in a real world application with students and teachers.

References 1. Beck, J.E.: Using response times to model student disengagement. In: ITS2004 Workshop on Social and Emotional Intelligence in Learning Environments. Maceio, Brazil (2004) 2. Blanchard, E., Frasson, C.: An Autonomy-Oriented System Design for Enhancement of Learner’s Motivation in E-learning (2004) 3. Cocea, M., Weibelzahl, S.: Cross-system validation of engagement prediction from log files. In: Duval, E., Klamma, R., Wolpers, M. (eds.) EC-TEL 2007. LNCS, vol. 4753, pp. 14–25. Springer, Heidelberg (2007) 4. De Vicente, A., Pain, H.: Motivation diagnosis in intelligent tutoring systems. In: Goettl, B.P., Halff, H.M., Redfield, C.L., Shute, V.J. (eds.) ITS 1998. LNCS, vol. 1452, pp. 86– 95. Springer, Heidelberg (1998) 5. Du Boulay, B., Rebolledo-Mendez, G., Luckin, R., Martinez-Miron, E.A., Harris, A.: Motivationally intelligent systems: Three questions. In: Second International Conference on Innovations in Learning for the Future (2008) 6. Ginot, H.: Teacher and child. Congruent Communications, Inc., New York (1972) 7. Hershkovitz, A., Nachmias, R.: Developing a Log-based Motivation Measuring Tool. In: 1st International Conference on Educational Data Mining, Montreal, Canada (2008) 8. Huitt, W.: Motivation to learn: An overview. Educational Psychology Interactive (2001) 9. Kim, Y.-S., Cha, H.-J., Cho, Y.-R., Yoon, T.-B., Lee, J.-H.: An Intelligent Tutoring System with Motivation Diagnosis and Planning. In: 15th International Conference on Computers in Education (2007) 10. Qu, L., Johnson, W.L.: Detecting the learner’s motivational states in an interactive learning environment Artificial Intelligence in Education. The Netherlands, Amsterdam (2005) 11. Rebolledo-Mendez, G., Du Bouley, B., Luckin, R.: Motivating the Learner: An Empirical Evaluation. In: International Conference on Intelligent Tutoring Systems, Jonghli, Taiwan, pp. 545–554 (2006) 12. Ryan, R.M., Deci, E.L.: Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist (2000) 13. Zhang, G., Cheng, Z., He, A., Huang, T.: A WWW-based learner’s learning motivation detecting system International Workshop on Research Directions and Challenge Problems in Advanced Information Systems Engineering, Honjo City, Japan (2003) 14. Weibelzahl, S., Kelly, D.: Adaptation to Motivational States in Educational Systems. In: Proceedings of the workshop week Lernen-Wissensentdeckung-Adaptivität (2005)

Supporting the Learning Dimension of Knowledge Work Stefanie N. Lindstaedt1, Mario Aehnelt2, and Robert de Hoog3 1

Know-Center and Knowledge Management Institute, Graz University of Technology, Austria [email protected] 2 Fraunhofer IGD Rostock, Germany [email protected] 3 Faculty of Behavioral Sciences, University of Twente, Enschede, The Netherlands [email protected]

Abstract. We argue that in order to increase knowledge work productivity we have to put more emphasis on supporting this learning dimension of knowledge work. The key distinctions compared to other TEL approaches are (1) taking the tight integration of working and learning seriously, (2) enabling seamless transitions on the continuum of learning practices, and (3) tapping into the resources (material as well as human) of the organization. Within this contribution we develop the concept of work-integrated learning (WIL) and show how it can be implemented. The APOSDLE environment serves as a reference architecture which proves how a variety of tightly integrated support services implement the three key distinctions discussed above. Keywords: workplace learning, professional learning, self-directed learning, collaboration scripts, user profiles, recommendation systems.

1 The Learning Dimension of Knowledge Work We conceptualize learning as a dimension of knowledge work which varies in focus (from focus on work performance to focus on learn performance) and time available for learning. This learning dimension of knowledge work describes a continuum of learning practices. It starts at one side with brief questions and task related informal learning (work processes with learning as a by-product), and extends at the other side to more formalized learning processes (learning processes at or near the workplace). This continuum covers the whole learning practices typology of Eraut and Hirsh [9] and emphasizes that support for learning must enable a knowledge worker to seamlessly switch from one learning practice to another as time and other context factors permit or demand. Research on supporting workplace learning and life long learning so far has focused predominantly on the formal side of this spectrum, specifically on course design applicable for the workplace and blended-learning. U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 639–644, 2009. © Springer-Verlag Berlin Heidelberg 2009

640

S.N. Lindstaedt, M. Aehnelt, R. de Hoog

In contrast, the focus of our work is on the informal side of the spectrum, specifically covering work processes with learning as a by-product and learning activities located within work processes. In order to refer to this type of learning practices we coined the term work-integrated learning (WIL) [11]. By using this term we emphasize that learning at the workplace needs to be truly integrated in current work processes and practices and makes use of existing resources within an organization – knowledge artifacts (e.g. reports, project results) as well as humans (e.g. peers, communities). WIL is relatively brief and unstructured (in terms of learning objectives, learning time, or learning support). The main aim of WIL activities is to enhance task performance. From the learner’s perspective, WIL is spontaneous and/or unintentional. Learning in this case is a by-product of the activities at the workplace. This conceptualization enables a shift from the training perspective of the organization to the learning perspective of the individual. We have shown in [2] that the learning continuum exists for all commonly agreed upon knowledge work types (create, transfer, apply, package). For example, on the one hand knowledge can be informally created within work practices when people learn from each other based on observations. On the other hand more formalized settings at the workplace such as dedicated brainstorming sessions can be employed to create knowledge. That is, in order to support knowledge work we have to provide learning support for all four knowledge work types on a continuum of formality. Therefore we have chosen to present our proposed WIL support functionalities structured along the four knowledge work types.

2 Supporting WIL with APOSDLE This section provides a brief overview of how the APOSDLE1 environment supports the learning dimension of knowledge work. We already have evaluated much of the presented WIL support in previous prototypes within workplace situations of our application partners as well as within controlled lab studies, for example [12]. Future work in the APOSDLE project will mainly focus on a summative evaluation of the entire APOSDLE environment. This summative evaluation will be carried out at the site of four organizations participating in the project and will span a period of three months. 2.1 Supporting Creation and Transfer of Knowledge Sharing Knowledge Artifacts In APOSDLE knowledge resides in knowledge artifacts: documents, parts of documents (referred to as snippets), notes, collaboration reflections, collections, etc. Such artifacts are containers for more or less structured information which relate to individual or collaborative tasks and activities. Knowledge artifacts are created from resources within the organizational memory by (automatically) attaching metadata which define the relationship and semantic meaning of artifacts in relationship to the work domain. They are shared throughout the organization. 1

Developed in the EU funded Integrated Project APOSDLE (www.aposdle.org).

Supporting the Learning Dimension of Knowledge Work

641

A variety of different knowledge artifact types can be created and edited by knowledge workers. For example, knowledge workers can create notes in relation to other knowledge artifacts. They can create collections containing other knowledge artifacts which stay in relationship to each other. As an important outcome of collaborative learning or work, reflections contain not only transcripts of collaborative activities but also individual reflections of knowledge applied for a certain learning context or situation. Knowledge workers are made aware of knowledge artifacts through automatic suggestions (see below). Scripted, Contextualized Collaboration Collaboration is a social interaction in which knowledge workers transfer and construct knowledge while working or learning together. In APOSDLE, the collaboration process is structured into a pre-collaboration, collaboration and postcollaboration phase to allow a clear allocation of preparatory, executive and closing work or learning functions [13]. This structure is made explicit with the Collaboration Wizard, a visual component which guides all collaborating knowledge workers through the collaboration process. It provides collaboration scripts on macro and micro level [7] for each process phase in order to support collaboration as a structured process. These scripts help knowledge workers to use each process phase as efficiently as possible. In the pre-collaboration, a combination of problem formulating, social and fading script is used to collect all required information for coupling knowledge workers in collaborative interactions. In addition, the Collaboration Wizard contextualizes the work environment of collaborating knowledge workers with information required for a common anticipation of collaborative activities in which knowledge needs to be transferred. This contextual information is taken from previous and current activities of knowledge workers: information they searched for, knowledge artifacts which relate to their activities and tasks, snapshots of individual work environments, etc. 2.2 Supporting Application of Knowledge Providing an Overview of Past Experiences Meta-cognitive skills have been identified as an important ingredient of self-directed learning [3] [13]. In particular studies suggest that mirroring the learner’s actions and their results have positive effects on learning. The goal of these interventions is to provide the learner with a (more objective) external perspective on her actions. APOSDLE provides the user with an overview of tasks performed and topics engaged with in the past. The activities are grouped into three categories: seeking knowledge, applying knowledge, and providing knowledge and are displayed within a tree map: • • •

Seeking knowledge: The user seeks information or help about the topic (for example by accessing hints and contacting colleagues about the topic). Applying knowledge: The user applies knowledge about a certain topic (for example, performing a task which requires knowledge about that topic). Providing knowledge: The user provides knowledge about a topic to other people or to the APOSDLE system (for example, sharing a resource, storing a note, creating an annotation).

642

S.N. Lindstaedt, M. Aehnelt, R. de Hoog

Within the tree map the size of a square is related to the frequency with which the user has been engaged with the topic. The larger the square, the more frequent the engagement has been. This overview of activities allows the user to reflect on her past actions, to immediately asses her activity patterns, and to become aware of topics which she might want to advance further in. Suggesting Knowledge Artifacts In order to apply knowledge to a specific work situation, a knowledge worker has to assess the situation and transform the knowledge to fit the situation. Reducing the effort for this learning transfer is believed to improving the likelihood of application of learned knowledge. APOSDLE takes the following approach: an intelligent recommendation algorithm suggests knowledge artifacts to the learner which are similar to the task or topic at hand and which have been retrieved from the organizational memory – thus improving the likelihood of offering highly relevant information which can be directly applied to the work situation with little or no learning transfer required. In doing so, APOSDLE also utilizes the automatically maintained user profile of the learner in order to compute a learning gap. The learning gap expresses the difference between knowledge about topics needed when executing a work task and the knowledge the user possesses about these topics. Based on this learning gap APOSDLE suggests relevant learning goals which the learner could pursue within her current work situations. Suggesting Knowledgeable People Besides suggesting knowledge artifacts to the user, APOSDLE also suggests people in the organization which are knowledgeable in this specific task or topic. The goal is not, to always suggest the most knowledgeable person (e.g. the official topic matter expert). Instead, our algorithm seeks to identify peers which have (recently) executed the task before and which are believed to possess more or equal knowledge to the user in question. The identification of knowledgeable persons is based on the user profile. 2.3 Supporting Acquisition of Knowledge Learning Paths Sometimes, learners wish to learn a substantive part of a relatively unfamiliar learning domain, but this will frequently take more than several hours to complete. In order to successfully realize such learning, learners should carefully plan and manage the entire learning process. For self-directed learners, planning a learning path is often difficult, as most learners can not rely on instructional knowledge and have limited prior knowledge about the learning domain. In APOSDLE, planning is supported with learning paths. A learning path describes how a learning domain can be traversed in an ordered way when learning about the domain. There are many possible paths through a learning domain. Learning paths can be created by the system or by learners. The learning path wizard helps learners to construct and optimize learning paths. The wizard takes existing knowledge of the learner in the user profile into account and checks whether learners lack the prerequisite knowledge for their learning goals. The wizard suggests prerequisite topics and topics that might be relevant for a learning trajectory.

Supporting the Learning Dimension of Knowledge Work

643

Topics in a learning path are automatically ordered in such a way, that the learning paths can be traversed easily. Basic knowledge is addressed first and more advanced knowledge that builds on the basic knowledge is addressed afterwards. Hints Though in general it is expected that workers are motivated to acquire new knowledge in the context of their work, the knowledge acquisition process can be enhanced by providing hints how one could process the information retrieved. In APOSDLE hints are based on two features of learning: learning goals and the possible instructional meaning of retrieved information. According to Gery [10] and Choo [5], people often have specific questions or requests that come to mind when faced with performing new or complex tasks. For instance, questions like: “What must I do? How do I do it? Am I doing it right?”, or requests like: “Show me…”. The information type associated with such a question or request can reasonably be defined. One way of supporting learners is be to identify a set of relevant questions and requests and a set of related information types. This is similar to the approach followed by Anderson and Krathwohl [1] who developed a taxonomy of learning goals which are subsequently used for assessment purposes. In APOSDLE, we opt for a generic categorisation of information types/materials that could be used to specify and limit the type of content that should be presented to learners (with a specific question). There are some categorisations available (based on projects like LOM, Ariadne, and SCORM), but these are rather low in meaning from a learning perspective. A classification schema is used based on the one developed in the IMAT project [8], which classifies fragments extracted from maintenance manuals into categories like: definition, overview, example, assignment, guideline, how-to, summary, etc. For instance, a learner with the question “How do I do it?” will be referred to fragments from retrieved documents that are labeled with categories like “guideline” or “how-to”. Learning hints can be contemplative in nature (without observable output), but can also be aiming at explicit outcomes that can be observed and assessed by others. In the latter case, the hints will contain an activity giving the opportunity to the user to enter information in specific input fields. Every hint consists of two elements: • •

An activity that states what a learner could do to process the information. A rationale that states why it is considered useful to engage in this activity.

The content of the hints is adjusted to the specific material resource type associated with a learning goal (need). This means that hints that accompany an example are (slightly) different from hints related to a guideline or a constraint.

Acknowledgements APOSDLE (www.aposdle.org) is partially funded under grant 027023 in the IST work programme of the European Community. The Know-Center is funded within the Austrian COMET Program - Competence Centers for Excellent Technologies - under the auspices of the Austrian Federal Ministry of Transport, Innovation and Technology, the Austrian Federal Ministry of Economy, Family and Youth and by the State of Styria. COMET is managed by the Austrian Research Promotion Agency FFG.

644

S.N. Lindstaedt, M. Aehnelt, R. de Hoog

References 1. Anderson, L.W., Krathwohl, D.R.: A taxonomy for learning, teaching and assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. Pearson Education, London (2001) 2. APOSDLE Consortium: Integrated APOSDLE Deliverables 2.8 and 3.5 APOSDLE Approach to Self-Directed Work-Integrated Learning (2009) 3. Bannert, M.: Metakognition beim lernen mit Hypermedien: Erfassung, Beschreibung und Vermittlung wirksamer metakognitiver Strategien und Regulationsaktivitäten. Waxmann Verlag (2007) ISBN 3830918720 4. Bonestroo, W., Kump, B., Ley, T., Lindstaedt, S.: Learn@Work: Competency Advancement with Learning Templates. In: Memmel, M., Ras, E., Wolpers, M., VanAssche, F. (eds.) Proceedings of the 3rd Workshop on Learner-Oriented Knowledge Management, pp. 9–16 (2007) 5. Choo, C.W.: The knowing organization. How organizations use information to construct meaning, create knowledge, and make decision. Oxford University Press, New York 6. Davenport, T.O.: Human Capital: What It Is and Why People Invest it. Jossey-Bass, San Francisco (1999) 7. Dillenbourg, P., Hong, F.: The mechanics of CSCL macro scripts. International Journal of Computer-Supported Collaborative Learning H. 3, 5–23 (2008) 8. de Hoog, R., Kabel, S., Barnard, Y., Boy, G., DeLuca, P., Desmoulins, C., Riemersma, J., Verstegen, D.: Re-using technical manuals for instruction: creating instructional material with the tools of the IMAT project. In: Workshop proceedings Integrating technical and training documentation, 6th International Intelligent Tutoring Systems conference (ITS 2002), San Sebástian, Spain, pp. 28–39 (2002) 9. Eraut, M., Hirsh, W.: The Significance of Workplace Learning for Individuals, Groups and Organisations. SKOPE Monograph, vol. 9. Oxford University: Department of Economics (2007) 10. Gery, G.: Electronic Performance Support Systems: how and why to remake the workplace through the strategic application of technology. Cambridge ZIFF Institute (1991) ISBN 0961796812 11. Lindstaedt, S., Ley, T., Scheir, P., Ulbrich, A.: Applying Scruffy Methods to Enable Workintegrated Learning. The European Journal of the Informatics Professional 9(3), 44–50 (2008) 12. Scheir, P., Ghidini, C., Lindstaedt, S.N.: A Network Model Approach to Retrieval in the Semantic Web. In: Sheth, A. (ed.) International Journal on Semantic Web and Information Systems, vol. 4, pp. 56–84. IGI Global Publishers, Hershey (2008) 13. Simons, P.R.J.: Towards a constructivistic theory of self-directed learning. In: Straka, G.A. (ed.) Conceptions of self-directed learning: theoretical and conceptional considerations, pp. 155–169. Waxmann, Münster (2000)

User-Adaptive Recommendation Techniques in Repositories of Learning Objects: Combining Long-Term and Short-Term Learning Goals Almudena Ruiz-Iniesta, Guillermo Jiménez-Díaz, and Mercedes Gómez-Albarrán Facultad de Informática – Universidad Complutense de Madrid c/ Prof. José García Santesmases s/n, 28040 Madrid, Spain [email protected], [email protected], [email protected]

Abstract. In this paper we describe a novel approach that fosters a strong personalized content-based recommendation of LOs. It gives priority to those LOs that are most similar to the student’s short-term learning goals (the concepts that the student wants to learn in the session) and, at the same time, have a high pedagogical utility in the light of the student’s cognitive state (longterm learning goals). The paper includes the definition of a flexible metric that combines the similarity with the query and the pedagogical utility of the LO. Keywords: User-adaptive Learning, Personalization, Recommendation Techniques, Learning Objects.

Content-based

1 Introduction Although recommender systems have been traditionally applied in e-commerce, their use has been recently transferred to the academic field [1, 2]. Particularly, the use of recommendation technologies has a clear application in e-learning: providing support for personalized access to the Learning Objects (LOs) that exist in repositories. Usually, the high number of LOs makes difficult the access to those adapted to the individual knowledge, goals and/or preferences of the students. In this paper, we present an approach that extends and improves our previous work on LO recommendation in web-based repositories. In [3] we described a novel approach for recommending LOs where a reactive single-shot content-based recommender acted as the primary recommender and its decisions were refined by a collaborative one. The recommendation strategy locates a set of relevant LOs after the student has posed a query. Priority is given to those LOs that are most similar to the student query and were assigned high ratings by other students. This previous recommendation strategy has shown one handicap: it provides a weak personalization that only takes into account the student’s short-term goals disclosed in the session in the form of a query. This way, two students that pose the same query within a session will receive the same recommendations, even if their long-term learning goals and domain mastery differ to a great extent. U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 645–650, 2009. © Springer-Verlag Berlin Heidelberg 2009

646

A. Ruiz-Iniesta, G. Jiménez-Díaz, and M. Gómez-Albarrán

In order to overcome this handicap we have explored a model of strong personalization. As we will show, the new content-based recommendation strategy could be tailored to the student’s long-term learning goals without significantly compromising the in-session interest that the recommended LOs can have for the target student. The content-based recommender described in [3] is now enhanced by giving priority to those LOs that are most similar to the student’s query (short-term goals) and, at the same time, have a high pedagogical utility in the light of her profile (long-term goals). The paper is organized as follows. Section 2 sketches the different knowledge sources required, independently of the educational domain. Section 3 details the two phases of the recommender, retrieval and ranking. Last section concludes the paper and outlines future work.

2 Describing the Required Knowledge We agree with Draschler et al. [4] that the learning field imposes specific requirements on the recommendation process. For instance, recommenders would benefit from taking into account the cognitive state of the learner, which changes over time. Successful learning paths and strategies could also provide guiding principles for recommendation. For instance, recommendation could benefit from simple pedagogical rules like ‘go from simple to complex tasks’ o ‘gradually decrease the amount of guidance’. Learning paths could represent routes and sequences designed by the instructors and successfully tried in the classroom, or they could correspond with successful study behaviour of advanced learners. Our recommendation approach follows a two-step process, retrieval and ranking. The retrieval stage looks for LOs that satisfy, in an approximate way, the student’s short-term learning goals represented in a query (in-session learning goals). These LOs should be “ready to be explored” by the student according to her current knowledge and the defined learning paths. Once LOs are retrieved, the ranking stage sorts them according to the quality assigned to each LO. The quality is computed so that priority is given to those LOs that are most similar to the student’ query and, at the same time, have a high pedagogical utility in the light of the student’ cognitive state (long-term learning goals). Our previous weak personalization required domain knowledge in order to compute the similarity between the query and the domain concepts covered by the retrieved LOs. The adopted strong personalization imposes some additional requirements from the knowledge representation point of view. The retrieval stage requires the existence of suitable learning paths over the different domain concepts as well as information about the student cognitive state in the form of persistent profiles. The ranking stage also profits from the student profiles. It also follows remedial instructional strategies as a guideline for facing the student long-term learning goals and filling the student knowledge gaps, which results in an improvement of her mastery and skills. Next we detail the knowledge sources in our recommendation approach.

User-Adaptive Recommendation Techniques in Repositories of LOs

647

2.1 The Domain Ontology We suggest the use of an ontology to index LOs within the repository. Ontologies provide a general indexing scheme that lets include similarity knowledge between the concepts representing the domain topics, which is a crucial knowledge in the similarity-based retrieval and ranking contexts employed by the recommender. The ontology is populated with concepts in the field of study. Concepts are organized in a taxonomy using the typical relation is_a. The ontology should also establish a precedence property among the concepts. We use this precedence to reflect a traditional sequence or order of concepts used when teaching in the corresponding field. The precedence lets establish the learning paths that will be used in the retrieval stage to filter out LOs that exemplify non-reachable concepts given a concrete cognitive state of the student. 2.2 The Learning Objects In our context the recommended items are LOs of educational repositories. The LOs can be developed according to Learning Object Metadata (LOM). We propose to use the next upper-level LOM categories: General, Life cycle, Technical, Educational, Relation and Classification. The General category plays an important role in the retrieval stage. This category contains keywords that describe what domain ontology concept(s) the concrete LO covers. The other categories represent descriptive information that it is not used in the recommendation phases. 2.3 The Student Profile As we noted above, the strong personalization requires persistent profiles of the students. A profile stores information about the student’ history of navigation −the LOs that she has already explored− and the goals achieved in the learning process. Concepts already explored by the student are assigned the competence level attained in each one. This level is considered as a degree of satisfaction, a metric that allows the recommender to know about the student’ knowledge level on a particular concept. The competence level will be an important element in the retrieval stage.

3 Describing the Recommendation Phases The content-based recommendation strategy presented here follows a reactive approach: the student provides an explicit query and the recommender system reacts with a recommendation response. The student poses a query using the concepts existing in the domain ontology. This query represents her in-session learning goals: the concepts she wants to learn in the session. The recommendation response is obtained in a two-step process, retrieval and ranking, which are described next. 3.1 The Retrieval Stage The retrieval stage looks for an initial set of LOs that satisfy, in an approximate way, the student query. The retrieval process tries first to find the LOs indexed by the

648

A. Ruiz-Iniesta, G. Jiménez-Díaz, and M. Gómez-Albarrán

query concepts. If there are no LOs that satisfy this condition, or if we are interested in a more flexible location, LOs indexed by a subset of the (same or similar) concepts specified by the student are retrieved. This initial set of LOs is filtered. The goal of the filtering process is to discard LOs indexed by concepts non-reachable by the target student. We say that an ontology concept is “reachable” by a given student if, according to her current profile and the learning paths defined in the ontology, it fulfils any of the following conditions: • It is a concept already explored by the student, so that it appears in her profile with its corresponding competence level. • It is a concept that the student has not explored yet but she can discover it: if a concept c1 precedes a concept c2 in the ontology, a student can discover c2 if the student competence level for c1 exceeds a given “progress threshold”. If several concepts c1, c2, ..., ck directly precede a concept cx, cx could be discovered if the student competence level in all the directly preceeding concepts exceeds the given “progress threshold”. 3.2 The Ranking Stage The ranking phase sorts the LOs retrieved according to the quality assigned to each LO. Priority is given to those LOs that are most similar to the student’ query and, at the same time, have a high pedagogical utility in the light of the student’ profile. In order to compute the quality of a given LO L for a student S that has posed a query Q we have chosen a quality metric defined as the weighted sum up of two relevancies: the similarity (Sim) between Q and the concepts that L covers, and the pedagogical utility (PU) of L with respect to the student S: (1) The similarity Sim(L,Q) between the concepts gathered in the query Q and the concepts that L covers requires to compute the similarity between two sets of concepts. A simplification consists on comparing the concept that results as the conjunction of the query concepts (Q_conj_concept) and the concept that results as the conjunction of the concepts covered by L (L_conj_concept). Assuming this simplification, we can use any accepted metric for comparing two hierarchical values, for instance, one that we defined and successfully used in the past [5]: (2) where super(Q_conj_concept) represents the set of all the concepts contained in the ontology that are superconcepts of Q_conj_concept and super(L_conj_concept) contains all the concepts within the ontology that are superconcepts of L_conj_ concept. Sim(L,Q) values lie in [0, 1]. In short, this similarity metric computes the relevance due to the in-session goals that L satisfies. The higher the number of query concepts that L lets learn is, the higher the similarity value is. The more similar L concepts and query concepts are, the higher the similarity value is.

User-Adaptive Recommendation Techniques in Repositories of LOs

649

In order to measure the pedagogical utility PU(L, S) that the LO L shows for a given student S, we have adopted an instructional strategy that promotes filling the student’ knowledge gaps by including remedial knowledge [6]. The goal is to assign a high pedagogical utility to L if it covers concepts in which the student has shown a low competence level. This remedial knowledge could give priority to LOs that the student has not explored yet, or can deal equally explored and not explored LOs. We have decided to compute the pedagogical utility as follows: (3) where AM (L, S) is the arithmetic mean of the competence levels that the student S shows in the concepts that L covers, normalized so that AM (L, S) lies in [0, 1]. This way, PU(L, S) also takes values between 0 and 1, both included. In short, (3) computes a low value of PU(L, S) if the student knows well the concepts that L covers. High values of PU(L, S), on the contrary, are obtained if the student has a poor knowledge of a high number of the concepts covered by L. In addition, (3) deals equally explored and not explored LOs. The resulting quality metric defined in (1), together with the pedagogical utility exposed in (3), lets introduce a considerable degree of personalization in the ranking stage. The final influence of the pedagogical utility and, as a consequence, the level of long-term personalization achieved in the definitive list of LOs recommended, could be controlled by means of the value assigned to the weight α used in (1). Once the value of α used in (1) is fixed, the resulting recommender system exhibits a concrete behaviour with respect to the type of personalization it provides. We can obtain a more flexible behaviour if, in a given recommender, α could take different values. For instance, the value of α could depend on the kind of student that uses the recommender. High values of α could be appropriate for students whose profiles exhibit good performance. These students seldom need knowledge reinforcement training and the recommender could focus on their in-session learning goals giving priority to those LOs that are highly correlated with the query. Low values of α, on the contrary, could be appropriate for students with lower performances, such that the recommender fosters filling knowledge gaps without significantly compromising the in-session interest that the recommended LOs can have for these students.

4 Conclusions and Future Work Our approach fosters high levels of personalization in content-based recommendation of LOs. The filtering step in the retrieval stage gives way to a light long-term personalization: when two students pose the same query within a session but their subject masteries differ, the set of retrieved LOs could be different. Introducing the metric (1) in the ranking stage lets the recommender fosters a strong-personalized recommendation. In order to compute the two partial relevancies, Sim and PU, different approaches and metrics can be tried. This way, the ranking model presented here offers a generic framework for personalized recommendation of LOs that can be instantiated and results in diverse recommendation approaches.

650

A. Ruiz-Iniesta, G. Jiménez-Díaz, and M. Gómez-Albarrán

We have carried out experiments in the Computer Programming domain but the approach could be easily tranferred to others learning domains. In order to alleviate the steep-use curve related with posing a query, we plan to complement the reactive approach with a proactive strategy that proposes the student LOs that could be of interest in a learning session, without the need of an explicit query. Preliminary work about the proactive strategy appears in [7]. Nowadays, we use the information about the navigation history recorded in the student profile in order to visually mark the recommended LOs that the student has already explored. A refinement of the quality metric could also take into account this fact in order to penalize these LOs. Acknowledgments. This work has been supported by the Spanish Committee of Education and Science project TIN2006-15202-C03-03 and the UCM project PIMCD2008-136.

References 1. Farzan, R., Brusilovsly, P.: Social navigation support in a course recommender system. In: Wade, V.P., Ashman, H., Smyth, B. (eds.) AH 2006. LNCS, vol. 4018, pp. 91–100. Springer, Heidelberg (2006) 2. O’ Mahony, M., Smyth, B.: A Recommender System for On-line Course Enrolment: An Initial Study. In: ACM Conference on Recommender Systems, pp. 133–136. ACM, Minneapolis (2007) 3. Gómez-Albarrán, M., Jiménez-Díaz, G.: Recommendation and Students’ Authoring in Repositories of Learning Objects: A Case-Based Reasoning Approach. International Journal on Emerging Technologies in Learning 4(Special issue), 35–40 (2009) 4. Draschler, H., Hummel, H., Koper, R.: Recommendations for learners are different: Applying memory-based recommender systems techniques to lifelong learning. In: Workshop on Social Information Retrieval for Technology-Enhanced Learning (2007) 5. González-Calero, P., Díaz-Agudo, B., Gómez-Albarrán, M.: Applying DLs for Retrieval in Case-Based Reasoning. In: International Workshop on Description Logics, pp. 51–55 (1999) 6. Siemer, J., Angelides, M.C.: Towards an Intelligent Tutoring System Architecture that Supports Remedial Tutoring. Artificial Intelligence Review 12(6), 469–511 (1998) 7. Ruiz-Iniesta, A., Jiménez-Díaz, G., Gómez-Albarrán, M.: Recommendation in Repositories of Learning Objects: A Proactive Approach that Exploits Diversity and Navigation-byProposing. In: IEEE International Conference on Advanced Learning Technologies. IEEE Computer Society Publications, California (2009)

Great Is the Enemy of Good: Is Perfecting Specific Courses Harmful to Global Curricula Performances? Maura Cerioli and Marina Ribaudo Computer Science Department, University of Genova {cerioli,ribaudo}@disi.unige.it

Abstract. We describe the lessons learned in a hands-on project on instructional design techniques and e-learning technologies. Our experience showed that, though each course designed within this experiment improved its results, the global results of the students were not completely satisfactory. Indeed, the restructured courses absorbed the attention of the students to the detriment of traditional programs. We argue that this side-effect is due to peculiarities of the Italian university system.

1

Introduction

Content Sharing, Flickr, YouTube, Social Networking, Wikipedia, . . . These are only some of today buzzwords, the words used in the Web 2.0 era, term defined to denote a set of principles and practices characterizing the ”new” web, intended as a social and participation platform in which the content is produced by a multitude of users, e.g., the wisdom of crowds [1]. Also e-learning takes advantage of Web 2.0 potentialities. Nowadays, in fact, the emphasis is posed on Collaborative Learning, a social oriented e-learning strategy in which collaboration plays the major role. The network is no longer a mere tool for content distribution but it is rather a facilitator for the interaction among the participants involved in the educational process [2], and the literature clearly states that collaborative learning is superior to individual learning. This paper describes our experience at the University of Genova, a traditional university where a Moodle-based platform is used to complement the daily faceto-face educational process. Although the numbers of users enrolled to the platform have a steep growing curve, the “quality” of the average exploitation of the available software tools is poor, certainly far from Web 2.0 philosophy. To fill this gap a course on instructional design [3,4] has been offered to faculty members, and this paper reports on such an experience. Our role in the project has been that of students, and we think we were privileged participants: we are computer scientists, so that it was easy for us to gain confidence in the use of the involved technologies, and the participation of teachers from our curriculum has been quite high, so that we can compare the results and see global trends. In Section 2 we describe the context of our university and the project-based formative model on instructional design methodologies we have been involved. U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 651–656, 2009. c Springer-Verlag Berlin Heidelberg 2009 

652

M. Cerioli and M. Ribaudo

Section 3 describes the results for our specific subjects in the Computer Science curriculum. In Section 4 we discuss the impact of the individual course changes on the overall curriculum year and draw a few conclusions.

2

The Context of Our Experience

The University of Genova is a medium-size traditional university offering face-toface courses. Since the beginning of 2005, efforts have been made to improve the quality of teaching by the introduction of ICT support in the educational process and, accordingly, of a Moodle-based Learning Management System (called AulaWeb 1 ) to be offered as a central service at the university level [5]. Though the numbers and the fast growth of the AulaWeb use are greatly encouraging, so far its use has been mostly unsophisticated, with download of material (slides, papers, lecture notes) as prevalent activity. But, it is now well accepted that the best learning occurs when students are actively involved in the learning process, possibly co-constructing pieces of knowledge, and software tools such as wikis, blogs, fora can be used for introducing some form of collaboration within the class. Initially, resource sharing had the lion’s share of the online activities. This phenomenon is well known in literature for the first attempts to introduce web-based technologies in a traditional educational process, so far mostly based on lecturing and information giving. To make the technological leap and actually take advantage of the full power of Web 2.0 support, instructional strategies and teaching styles have to be changed and to speed up this process professors have to be formed with the help of professionals, so that they can readily overcome their fear of technology and change their way of teaching to accommodate the novelty. In the last year our university has been involved in such a project. The action Web Enhanced Learning (WEL) has been launched in Apr. 2008, with the specific objectives of devising and experimenting a model for the transfer of instructional design knowledge and skills to subject-oriented university teachers. It is worth noting that WEL was not centered on the mere transfer of technological knowledge. Indeed, a short analysis showed that the causes behind the limited usage of tools supporting interactive ways of learning were mostly the lack of place for such kind of activities in the classical in-presence teaching/learning process, preferred by the quasi totality of our faculties. The WEL course consisted of a few initial plenary lectures followed by faceto-face individual meetings with instructional design experts. A plenary meeting has been organized in Feb. 2009 to share the experiences of those colleagues who have been online in the first semester. Thanks to the introduction of a team of instructional specialists supporting faculty members, “many professors indicated their instructional strategies and teaching styles had changed.” Comparing the usage data of the intervals April 07-08 and April 08-09, we can observe that the usage of instruments like wikis and glossaries is increasing this year respectively of almost 75% and 140%, which is a percentage of growth much larger than that of the courses (24%) and the resources (41%). Moreover, 1

http://www.aulaweb.unige.it

Great Is the Enemy of Good

653

the numbers suggest that also the approach to the use of the tools is improving. For instance, not only the number of wikis has increased, but that of pages and versions has dramatically grown, about 4 times the percentage of the wikis themselves! Thus, we can infer that not only we have a larger number of wikis, but also that they are used differently, with more online activities going on.

3

Our Experience: Benefits for the Involved Courses

We describe now our personal experiences on three subjects delivered in Sept.Dec. 2008 with the benefit of the WEL course. The first two courses are both mandatory and expected to have the same audience, that is, the students of the third and last year of the first cycle. The third course is an advanced course for the second cycle and it is interesting to compare the different approaches of the students to this and the previous courses. Programming Advanced Techniques (TAP2 ) since its introduction in 2003/04 has being organized around a project, requiring the students to individually realize a component. The course has a traditional organization, with lessons and small activities in the laboratory finalized to the project (which used to go mostly deserted), and was loosely supported by AulaWeb, to distribute material and technically support the students through a forum. In the past, more than half of the about 90 enrolled students never showed up for a lesson, nor completed any activity, including the project. The students justified their lack of participation by the perceived missing connections of the laboratory activities to the project, and their unpalatable individual organization. But, ignoring the laboratory activities, they did not acquire the knowledge needed to work on the project, so that they failed it (and the overall exam). Thus, during the WEL project, the TAP course has been redesigned around a group activity, simulating in the small the work for the project. As this activity was so clearly connected to the project and not individual, 92 students out of 108 participated in it and all the 89 students active in the first semester successfully completed it. The groups for the activity were formed by teachers and strictly structured, with a specific role for each participant (for a discussion on group formation criteria and team working, see e.g., [6]). All the interactions were required to take place on AulaWeb, and indeed 2/3 of the final score were attributed on the basis of the posts on the dedicated fora. Each group had private forum and wiki as workspace, and the students in the role of managers could interact among them and with the teacher in a devoted forum. The students were enthusiast about the activity and spent on it much more time than expected, so that the size (but not the technical difficulties) of the final project had to be reduced to meet the expected average effort for the course. On the other hand, the results of this year were impressive (see Table 1). The number of active students sky rocketed and the students (in percentage) who passed the project within the semester more than tripled w.r.t. the past and were even more than those who passed the project in a whole year in the past. 2

In the following we will use the acronym of the Italian name of the course.

654

M. Cerioli and M. Ribaudo Table 1. Results of IS and TAP restricted to the first semester

06/07 Enrolled Students 86 Active Students 44.19% Passed Exams 6.98% Passed Projects 6.98%

TAP 07/08 85 35.29% 5.88% 8.24%

08/09 108 82.41% 18.52% 34.26%

06/07 86 75.58% 18.6% 32.56%

IS 07/08 114 75.44% 15.79% 31.58%

08/09 97 87.63% 20.62% 72.16%

Software Engineering with Project (IS) introduces the main concepts of software engineering, relying, to let the students experience such concepts, on a group project on realistic software development, taking up about half credits of the course. Until five years ago, the project was traditionally managed: the students worked on an assignment at their own pace with the only constraint of the final deadline. But, probably due to the inherent complexity of this kind of project, most groups were unable to complete it. Thus, in the academic year 2004/05 the project has been totally restructured, with the introduction of phases, with intermediate milestones and more interaction between students and teachers (see e.g., [7] for a detailed description). With the current organization, the results were much improved (see Table 1 for the data of the last two years). Since IS already had a lot of interaction and community construction, the changes introduced accordingly to the WEL project were mostly technical. The structure of the course on AulaWeb was reorganized to make easier to find information; some activities, like for instance common development of exercise solutions, which were formerly loosely supported by an all-purposes forum, got their own devoted wiki and so on. The results were quite disappointing, being comparable with those of the previous years, as shown in Table 1. The active students are about 12% more, the completed projects have doubled, but the percentage of passed written examinations is only slightly better than the past years. Apparently the students focused on the project to the detriment of the preparation for the written examination and in particular all optional activities, mostly finalized to such preparation indeed, were dropped altogether. We think that IS already took its quality leap when the project organization changed and now technical adjustments gives only small improvements. Network’s Applications 2 (AR2) is an optional course, for the students of the second cycle of study, and this edition involved just 13 participants. Lectures varied from Web 2.0 technologies to the theory of Complex Networks, and the choice was that of mixing in-presence lessons with a small online activity for the first part of the course, that related to Web 2.0 technologies. In the first two weeks of Oct. 2008, students, split into 4 groups formed by teachers, have been asked to collaboratively write on a wiki a CookBook of software examples using technologies such as Web Services, REST, Ajax. For each example (individually chosen by each group) the students had to describe the product, the software language and the software libraries selected to develop it, and the overall architecture. All the decisions have been taken by sending posts to a technical forum

Great Is the Enemy of Good

655

associated with the wiki. 2 out of the 13 initial students dropped out since they realized they could not meet the online activity deadlines. Indeed, these 2 students were in their first level of study and decided to anticipate the subject, but experimented an interference with other courses. The other students realized imaginative projects which have been presented to the class in a demo session two weeks after the end of the online phase. Communication has been intensive: 71 posts3 have been sent on the technical forum. The construction of a community and the acquired healthy habits on communication have also positively influenced the traditional part of the course. Indeed, 95 messages have been sent in another technical forum suggesting scientific papers, URLs, software libraries, . . . , and other 75 messages have also been posted in an all-purposes forum. The oral examination consists of a discussion of all their products plus some questions on the theory introduced during the course. 9 (out of 11) students took and passed the exam within the semester, with 28 (over 30) as average mark.

4

Analysis of Global Data: Lessons Learned

The data shown in Section 3 prove that the WEL project was successful: the involved courses experimented an improvement in their results, more or less depending mostly on how much space for improvement there was, students participated in the proposed activities and so forth. However, this is only one side of the coin. Indeed, during the semester we got negative feedback from other courses having the same expected audience of IS and TAP. The student participation was decreasing, with peeks in occasion of deadlines for activities of IS and TAP. For instance, another mandatory course for the same semester in 2007/08 had 80 enrolled students, 27 out of which took the exam within the semester with 22 positive results. But the same course in 2008/09 had 70 enrolled students, 15 out of which took the exam within the semester with 9 positive results. It is important to note that for both IS and TAP we take care not to exceed the expected effort for average students, monitoring their efforts during the semester and adjusting the assignments accordingly. Moreover, we globally plan the deadlines of the activities of the different courses to avoid conflicts. Hence the interferences should be non-existing or very limited, while they are devastating. We think that the main reason behind these failures is a peculiarity of the Italian university system, where the concept of passing from a year to the next does not exist, as students have to eventually collect positive marks in all the courses in their curriculum, but they can do that in an arbitrary number of academic years, disregarding the learning agreement subscribed at the beginning of each year. Indeed, this scenario does not encourage the students to balance their efforts among the courses by the requirement of passing all of them and so they tend to distribute their time on the basis of their fancy for some topics, their enthusiasm for a proposed activity, the teacher’s charisma. . . But, in this way quite often students invest too much effort in an activity or in a specific 3

Recall this is a niche course with 4 groups only.

656

M. Cerioli and M. Ribaudo

course to the disadvantage of others, so that improving some courses damages the others, even if each course is correctly planned and the internal deadlines are globally orchestrated. Actually, we think that the current trend toward a more participative and technologically supported learning model will amplify the problem, introducing a divide between the modern and the traditional courses, till the latter will be completely extinct. The delicate point will be to find a balance among the different competing courses to let students participate in all of them. We argue that at this aim it is mandatory to change the university organization so that students can be forced to take groups of exams in the same year. Moreover, the instructional design approach should be moved up one level and applied to the design of the overall curriculum, so that different courses and their activities could be harmonized. A totally different story is that of the course of AR2. Indeed, in that case we did not experience any kind of interference with the other courses. Both the students and the teachers claim that the activities proposed were not creating problems. We think that the reason behind the difference is the maturity of the students, who have learned their limits and hence are able to sensibly plan their curriculum and distribute their effort.

References 1. Surowiecki, J.: The Wisdom of Crowds. Why the Many Are Smarter Than the Few. Abacus (2005) 2. Trentin, G.: La sostenibilit` a didattico-formatica dell’e-learning. Social networking e apprendimento attivo. Franco Angeli (2008) (in Italian) 3. Dick, W., Carey, L., Carey, J.O.: The Systematic Design of Instruction, 6th edn. Merrill (2004) 4. Leshin, C.B., Pollock, J., Reigeluth, C.M.: Instructional Design Strategies and Tactics. Education Technology Publications, Englewood Cliffs (1992) 5. Ribaudo, M., Rui, M.: AulaWeb, web-based learning as a commodity. The experience of the University of Genova. In: 1st Int. Conf. on Computer Supported Education, Lisbon, Portugal (2009) 6. Oakley, B., Felder, R.M., Brent, R., Elhajj, I.: Turning Student Groups into Effective Teams. Journal of Student Centered Learning 2 (2004) 7. Astesiano, E., Cerioli, M., Reggio, G., Ricca, F.: A phased highly-interactive approach to teaching uml-based software development. In: Staron, M. (ed.) Proc. of Educators’ Symposium at MoDELS 2007. Research Reports in Software Engineering and Management, IT University of G¨ oteborg, pp. 9–18 (2007)

Evolution of Professional Ethics Courses from Web Supported Learning towards E-Learning 2.0 Katerina Zdravkova1, Mirjana Ivanović2, and Zoran Putnik3 1

University Ss Cyril and Methodius, Faculty of Natural Sciences and Mathematics, Institute of Informatics, Skopje 2,3 University of Novi Sad, Faculty of Natural Sciences and Mathematics, Department of Mathematics and Computer Science, Novi Sad [email protected], {mira,putnik}@dmi.uns.ac.rs

Abstract. Skopje and Novi Sad share several joint courses in Professional Ethics at undergraduate level and at postgraduate level. These courses have been delivered to almost 1000 students from 14 different target groups. For seven years, teaching, learning as well as assessment have been steadily growing from traditional Web supported learning, through blended learning, towards Web 2.0. This paper presents all the stages of the courses evolution using several learning management systems, and the effort to enhance teaching, learning and active contribution of all the actors in the educational process. Particular attention is paid to our latest experience using Moodle and its social networking aspects in education. This survey reveals all the activities during the course delivery including student workload, grading system, and teacher’s efforts to maintain the courses. Student encouraging impressions regarding the content delivery and assessment, their personal opinion about the impact of e-learning 2.0 to quality and quantity of acquired new knowledge, and sincere suggestions to persist in the same direction are the greatest assurance that social networks are currently the best way to deliver computer ethics courses. At the same time, it seems that this approach is the most exhausting and the most challenging for the teachers, but at the same time, the best balance between the effort undertaken and the results obtained. Keywords: Web supported learning, Blended learning, Social networking.

1 Introduction Almost 15 years ago, Markus [1] concluded that despite integration of multimedia and first e-mail etiquecy, negative social impact of new technology “may not prove easy to eradicate”. In their survey from 2003 [2], Morahan-Martin and Schumacher claimed that Internet use caused loneliness. And, they were not sole. Negative social impact of information technology was usually predominant. Starting from 1997, when Jay Cros [3] coined the term e-learning, i.e. Internet-enabled learning computer technology, Internet became one of the basic media for teaching and learning content. Traditional face-to-face education lost its role, and many students decided to attend on-line classes. And, some students became even more aliened than previously. U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 657–663, 2009. © Springer-Verlag Berlin Heidelberg 2009

658

K. Zdravkova, M. Ivanović, and Z. Putnik

First attempt to “socialize” Internet users was the site Classmates.com [4]. Launched too early, it couldn’t get high attention, but soon later, social networking sites made a revolution between Internet users. It was high time to switch from Web 1.0 to Web 2.0. Recent research made by Nielsen Online [5] reports that top 10 social networking sites had almost 76 million unique visitors in September 2008. Compared with 33 million visitors in September 2007, the average growth is 167%. In Web 1.0 a few content authors provided content for a wide audience of relatively passive readers. Web 2.0 is already transforming our social lives and is quickly becoming a competitive tool for education [6]. Very important conclusions connected to usage of social networks in education are given by De Weaver [7]. A survey conducted on a large group of students and instructors, revealed that newer forms of activities, like collaborating and sharing information to a community, are less popular though. Social software has the potential not only to enhance particular aspects of teaching and learning, but also to significantly contribute to the creation of new forms of these activities. Bryant [8] summarises potential developments in this area as: ‘The adoption of social software tools, techniques and ideas will be the most important and visible example of the use of emerging technology in education over the next few years’. Another example of related work is reported by Franceschi [9], where a suggestion for improvement of social networks within e-learning systems has been given. The last, but not the least research is done within Comtella project [10, 11, and 12]. It is an impressive example of the implementation of Web 2.0 in blended classes, based on self-developed peer-to-peer file and bookmark sharing system aimed to share papers in several courses, including a professional ethics course.

2 Related Work The emergence of Web 2.0 technologies promotes the growth of service-based applications and greater user-control over content and connection [13]. Recent developments in web-based services and the enhancement of collaborative tools have fuelled the demand for similarly-specified educational software and services. A lot of universities across the world now deploy blogs, ePortfolios and educational social software for use by the academic community. In spite of the widespread support of these learning tools, still there is no adequate number of reports and analyses to appropriately validate the level of their utilization by tutors and students. But there are some publications bringing more or less optimistic results. The main analysis in [14] was based on observing student access and use of educational tools as well as on the anonymous recording of student experiences of using other social software in a noneducational context. More complex view of educational activities is given in [15]. They concluded that usage of social tools allow students to share capabilities and knowledge, bringing the synergetic effect to learning and life as well. Recent paper by Bernsteiner [16] presents the results of an empirical survey in order to highlight the benefits of the Web-based social software tools from the student’s point of view. As motivation is on different levels, the lecturers have to increase it during lessons. Fortunately there are students, who were highly motivated and were creating the content and adding them to the wikis [17].

Evolution of Professional Ethics Courses

659

3 Setting Up the Scene In last decade, many LMS have been developed to support new education trends. Probably one of the most popular, particularly for educators, is Moodle with more than 28 million users, supporting the delivery of more than 2.5 million courses [18]. Starting from academic 2005/06, both institutions presented in this paper switched from static LMS to Moodle (Skopje), or from static usage of Moodle to social networks (Novi Sad), showing that static LMS at both institutions become obsolete. Encouraged by the appreciation of more than 3500 active participants at both institutions, we can claim that the success of social network strategy in e-education utilized in our institutions is evident. One additional note should be made here – while most of the research papers, and experience reports present positive attitudes and opinions about social networks in general [19] and their usage within e-learning [20], there are some negative positions too [21].

4 Evolution of the Courses: From Static Form to the Social Network The development of educational technologies in last decade directs to think at learning as both a personal and collective experience. Cooperative and the collaborative learning promote the use of social tools in order to involve all e-learners in building a common knowledge. 4.1 Stage One: Web Supported Learning First delivery of the course on ethics at undergraduate level started in October 2002 in Skopje. All the contents were prepared by the teacher and by students. Presentations were oral, and they were followed by small discussions during the lecture. The contents were periodically uploaded on a static course site. Mutual communication between students and the teacher was either face-to-face, or by e-mail. Similar course in Novi Sad for the first time was realised in October 2005. While presented to the students through oral lectures and PowerPoint presentations, the whole teaching material was published on a web-site using Moodle. A greater interest in the topics presented within the course and methods of course delivery, was confirmed by higher number of students in each new school year. Moodle was used for static presentation of teaching material, but since Bologna principles required class attendance, it was more of a material repository, than used for some profound purposes. Although discussions, as a valuable social element of learning were announced at least a week in advance, feedback of all the students independently of the generation was rather poor. Discussions were directed by the teacher, usually involving very few participants. Forums were the only elements of social networks used for publication of announcements of important events, and e-mail correspondence of students with lecturers. They were selected as primary way of communication as they were mostly topic (not people) focused.

660

K. Zdravkova, M. Ivanović, and Z. Putnik

4.2 Stage Two: Blended Learning Beginning of the course for first generation of postgraduate students in Novi Sad and in Skopje started in 2006, when Moodle was implemented, initially aiming to augment face-to-face lectures. Initially, Moodle was mostly used as a repository of teaching materials, either as a fixed collection of files, or as an active set of animated e-lessons. Still, the repository was in its essence static. Communication was aimed towards teachers and teachers only. While lecture attendance was part of the obligatory requirements for the new generations of students, e-communication was still an idea worth introducing, as an attempt in perfection of the course. In order to avoid exhausting oral examinations, an e-test with 250 questions was designed in Skopje. Initially weak results soon became impeccable. After a small investigation, it appeared that students who had already finished the e-test copied the questions and their correct answers, and distributed them to those who will have the exam later. This student fake showed that it was high time to change the delivery of the course, to enhance student active participation, and to change the grading scheme. In Novi Sad, similar repository of around 200 questions exists, but it is not used for etesting, but within “regular” classroom tests instead. Since this assumes presence of the assistant, elements of cheating, while existing, were not that flagrant. 4.3 Stage Three: Active Contribution of All the Participants The experience gathered during the usage of Moodle from some other joined courses [22] showed that the inclusion of other elements available within LMS, like forums, chats, or e-mail usage, could create a more dynamic system, system known in contemporary research as a social network. This academic year, almost all the students participated in social network rather freely. Even those who are recognized as shy and silent persons during lectures, find themselves very involved in discussions, arguments, and even quarrels with other colleagues, when it comes to questions important to them. Yet, this does not come as a surprise, since the tendency of introvert students to reveal their opinions within electronic communication, when not faced literally with the rest of colleagues. Another point worth mentioning is the fact that created social networks influenced widening of topics in question. Even though at the beginning points to be discussed were strictly defined, very often discussions diverged to various directions, touching each matter connected to the original one that is interesting for students. As a natural improvement, forums were used to apply well-known technique of role-playing games. Students were given certain roles and were invited to participate in a scenario connected with some ethical and moral issues, discussing and defending opinions represented by their roles. During a fortnight, student teams actively defended their roles, with an average of 9.73 posts per student. Teachers were also involved in the discussions to direct them. At the end, using a supporting forum, student prepared team reports of their groups. This forum had 10.07 posts per student, showing the usefulness of on-line discussions. It took some time for students to start communicating and sharing opinions, but each year, it eventually came to this point. Probably because of shared experience with previous generations, time between those phases has been shortened.

Evolution of Professional Ethics Courses

661

5 Conclusion and Intended Evolution of the Courses Obvious benefit of the steady evolution of our joint courses towards social software was the active involvement of all the students, including those who are usually idle. With new “socialised” approach, students were motivated, stimulated and sometimes provoked to reveal their own ideas. To support their assertions, they dug into different sources to discover other sources in favour of their opinion. Such research stimulated their intellectual capacities, and prepared them for future research. In many occasions, research was not directed to computers ethics only, but also to related areas. Further great benefit of Web 2.0 in our course was the possibility of relaxed, and at the same time, efficient group collaboration. Using forums, students virtually met their colleagues, followed the development of the group project, and presented their findings. Research and group essay preparation progress was clearly evident at every moment, individual contribution was obvious, so nobody could object that his/her contribution was neglected, or the freedom of speech withdrawn. Grading facilities of Moodle were another asset. At every moment, students could exactly know current result, including the information whether their contribution was successful, or not. Any of usual student complaints connected with the grading, such as underestimation of their workload, or overestimation of the others, was impossible since the entire communication and effort are completely overt to all the students and to teachers. And, completely transparent communication was an ideal way to judge personal achievements in relation to the achievements of the others. Frequent discussions on different topics involving all the students and the teachers were the best way to be always in line with the newest events related to the course, including the breaking news. As a result, the awareness of students and teachers for the course increased. At the same time, the repository of teaching materials enlarged. The last and certainly not the least advantage of Moodle was the impossibility to cheat and to fake personal outcomes. Namely, students couldn’t finish the assignments behind schedule and claim that the deadline was not precise, or that they delivered the assignment on time, because all the closing dates were visible, and Moodle kept records of all their activities. Furthermore, even if somebody decided to do the assignments instead of another colleague (which is still common in the region), he/she could not replace the actual student in the forums. Apart of these advantages, e-learning 2.0 brings some problems. First of all, technical prerequisites must be faultless, such as constant availability of the server, impeccable Internet connection, and a permanently high scalability. In the beginning of the academic year, occasional slow response, due to many users competing for the same resource close to final deadline happened. Hopefully, this problem was gradually settled, because students became more professional. Social networking was exhausting both for the students, and for the teachers. Whenever students were not on-line, they could not actively participate. However, unlike students from the survey mentioned earlier on [21], our students were much more enthusiastic with e-learning 2.0. They never complained that fully transparent approach was a problem for them. However, social software in education is a treat to student privacy. We are aware that this is one of the weakest aspects, and it can’t easily be resolved.

662

K. Zdravkova, M. Ivanović, and Z. Putnik

References 1. Markus, M.L.: Finding a happy medium: explaining the negative effects of electronic communication on social life at work. ACM Transactions on Information Systems (TOIS) 12(2), 119–149 (1994) 2. Morahan-Martin, J., Schumacher, P.: Loneliness and social uses of the Internet. Computers in Human Behaviour 19(6), 659–671 (2003) 3. Cross, J.: An informal history of eLearning. On the Horizon (12/3), 103–110 (2004) 4. Boyd, D.M., Ellison, N.B.: Social Network Sites: Definition, History, and Scholarship, http://jcmc.indiana.edu/vol13/issue1/boyd.ellison.html 5. Nielsen online blog “Connecting the dots”, http://blog.nielsen.com/nielsenwire/wp–content/uploads/ 2008/10/press_release24.pdf 6. Franklin, T., Van Harmelen, M.: Web 2.0 for Content for Learning and Teaching in Higher Education, http://www.jisc.ac.uk/publications/ publications/web2andpolicyreport.aspx 7. De Wever, B., Mechant, P., Veevaete, P., Hauttekeete, L.: E-Learning 2.0: social software for educational use. In: Proc. of 9th IEEE International Symp. on Multimedia, pp. 511–516 (2007) 8. Bryant, L.: Emerging trends in social software for education. British Educational Communications and Technology Agency Emerging Technologies for Learning (2007) 9. Franceschi, K., Lee, R., Hinds, D.: Engaging E-Learning in Virtual Worlds: Supporting Group Collaboration. In: Proc. of 41st Hawaii International Conf. on System Sciences (2008) 10. Vassileva, J.: Harnessing P2P Power in the Classroom. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 305–314. Springer, Heidelberg (2004) 11. Webster, A.S., Vassileva, J.: Visualizing Personal Relations in Online Communities. In: Wade, V.P., Ashman, H., Smyth, B. (eds.) AH 2006. LNCS, vol. 4018, pp. 223–233. Springer, Heidelberg (2006) 12. Vassileva, J., Sun, L.: Using Community Visualization to Stimulate Participation in Online Communities. e-Service Journal 6(1), 3–40 (2007) 13. O’Reilly, T.: What Is Web 2.0: Design Patterns and Business Models for the Next Generation of Software (2005) 14. Stepanyan, K., Mather, R., Payne, J.: Awareness of the capabilities and use of social software attributes within and outside the educational context: moving towards collaborative learning with Web 2.0. In: Proceedings of Conference ICL 2007, pp. 1–9 (2007) 15. Itamar, S., Bregman, D., Israel, D., Korman, A.: Do eLearning Technologies Improve the Higher Education Teaching and Learning Experience? In: Fifth International Conference on eLearning for Knowledge-Based Society, pp. 24.1–24.7 (2008) 16. Bernsteiner, R., Ostermann, H., Staudinger, R.: Facilitating E-Learning with Social Software: Attitudes and Usage from the Student’s Point of View. Int. J. of Web-Based Learning and Teaching Technologies 3(3), 16–33 (2008) 17. Drazdilova, P., Martinovic, J., Slaninova, K., Snasel, V.: Analysis of Relations in eLearning. In: Proc. of IEEE/WIC/ACM Int. Conference on Web Intelligence and Intelligent Agent Technology, pp. 373–376 (2008) 18. Moodle statistics, http://www.moodle.org/stats

Evolution of Professional Ethics Courses

663

19. Alexander, B.: Web 2.0: A New Wave of Innovation for Teaching and Learning? EDUCAUSE Review 41(2), 32–44 (2006) 20. Miler, A.: Moodle from a Students Perspective, http://dontbeafraid.edublogs.org/2008/11/05/ modle-from-a-students-perspective 21. Iadecola, G., Piave, N.A.: Social Software in Educational Contexts: Benefits and Limits. In: Fourth International Scientific Conference eLearning and Software for Education 22. Budimac, Z., Putnik, Z., Ivanović, M., Bothe, K., Schuetzler, K.: On the assessment and self-assessment in a students teamwork based course on software engineering. Computer Applications in Engineering Education, early view (2009), doi:10.1002/cae.20249 23. Costa, C., Beham, G., Reinhardt, G., Sillaots, M.: Microblogging in Technology Enhanced Learning: A Use-Case Inspection of PPE Summer School (2008)

Towards an Ontology for Supporting Communities of Practice of E-Learning “CoPEs”: A Conceptual Model Lamia Berkani1 and Azeddine Chikh2 1

National Institute of Computer Science, INI, Algiers, Algeria [email protected] 2 Information Systems Dept., King Saud University, Riyadh, Saudi Arabia [email protected]

Abstract. The Community of Practice of E-learning (CoPE) represents a virtual space for exchanging, sharing, and resolving problems faced by actors in elearning. One of the major concerns of CoPEs is to favor practices of reuse and exchange through the capitalization of techno-pedagogical knowledge and know-how. In this paper, we present a conceptual model of CoPEs. This model constitutes the theoretical platform upon which an ontology dedicated to CoPEs will be built. This ontology aims to annotate the CoPE’s knowledge resources and services, so as to enhance individual and organizational learning within CoPEs. Keywords: E-learning, CoP of e-learning, O’CoPE, ontology concepts.

1 Introduction Today, we are witnessing a fast and significant expansion of the e-learning domain. Companies, schools, universities, and organizations of all sizes are currently using elearning as a tool of training, learning and professional development. The increase in interest of e-learning is seen through the development of large projects launched everywhere in the world and through the proliferation of specifications and standards for e-learning systems too. However, despite the large quantity of knowledge accumulated in this field, the know-how and the feedback from acquired experience are not always capitalized and exchanged in a systematic way between its actors. Furthermore, this field is facing a number of challenges related to: (1) the difference of interpretation of its concepts. For example, software tools (e.g. simulation or translation tools) used in an online course are considered sometimes either as resources or services; (2) the complexity resulting from the multiplicity of its standards (LOM, SCORM, IMS-LD, IMS-LIP, …), and the heterogeneity of its tools such as authoring tools and LMS (Learning Management systems) like Moodle1, Acolad2 or Blackboard3; (3) the diversity of its teaching domains from arts, literature, fundamental and 1

http://moodle.org/ http: //acolad.u-strasbg.fr/ 3 http://www.blackboard.com/ 2

U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 664–669, 2009. © Springer-Verlag Berlin Heidelberg 2009

Towards an Ontology for Supporting CoPEs

665

applied sciences to engineering requiring different educational approaches and techniques. Accordingly, actors involved in e-learning must exchange efficiently both of their problems and experiences. Based on work done on Communities of Practice (CoPs) and the success they made in collaborative learning [1], especially in the domain of teaching [2; 3; 4; 5], we have thought to extend this technology to e-learning as sub-domain of teaching. So, we consider CoPEs (Communities of Practice of E-learning) as a virtual framework for exchanging and sharing techno-pedagogic knowledge and know-how between actors of e-learning. In [6; 7] we have defined a CoPE and the underlying concepts. In the present paper, we try to refine and enrich the previous definitions through a conceptual model. This model constitutes the theoretical grounding upon which an ontology dedicated to CoPEs will be built. This last will offer a uniform vocabulary to explicitly specify all the CoPE’s concepts, and with which the CoPE’s resources and services can be annotated, so as to support the learning processes in the CoPE. In the following, section 2 introduces the background of our research and some related works. Section 3 presents our main contribution related to the conceptual model for CoPEs. The conclusion in section 4 highlights the main results and opens some future perspectives.

2 Background and Related Work A CoPE is a group of professionals in e-learning who gather, collaborate, and organize themselves in order to: (i) share information and experiences related to e-learning development and use ; (ii) collaborate to solve together e-learning problems (e.g. interoperability, adaptativity) and to build techno-pedagogic knowledge and best practices; (iii) learn from each other and develop their skills in instructional engineering; (vi) promote the use of e-learning standards: IMS-LD, SCORM, LOM... We address in this paper the need to model the CoPE’s concepts, in order to enable the formalization of core aspects related to CoPEs, so as to support automatically the exchanges and the communication within the CoPE. These models will constitute the basis from which an ontology dedicated to CoPEs can be built. This last aims at representing a uniform vocabulary to explicitly specify all the CoPE’s concepts and relations, and will help to annotate semantically both of the knowledge resources and services used in a CoPE. Such semantic annotations (for example, on the profile, role and competencies of a CoPE’s member, on the effective uses of a tool by the CoPE, on the collaboration or cooperation mode preferred by the CoPE for a given common activity, on the arguments leading to a decision making or a problem solving…), can then be used by services such as knowledge search services and they can thus support the learning processes in the CoPE. Most of the work that has been done in the teaching domain through online CoPs didn’t take into consideration a formal modelization of the concepts. For example Teacher Bridge [8] proposed a set of online tools to help create a community of teachers using dynamic web pages creation. This work lacks of semantic annotations of knowledge especially the tacit knowledge. So, there is no mean to retrieve it. However, some existent works are attempting to formalize the CoP’s concepts: Dubé et al. identified in [9] twenty-one characteristics for distinguishing and comparing Virtual CoPs (VCoPs). However, they didn’t try to formalize the VCoPs based on common conceptual models.

666

L. Berkani and A. Chikh

On the other hand, Palette project [10] proposed several models useful for describing a CoP [11]: community, actor, learner profile, competency, collaboration, process/activity, and lessons learnt. These models are dedicated to CoPs in general and are built based on analysis of information sources gathered from twelve existing CoPs.

3 Contribution: A Conceptual Model for CoPEs Fig. 1 depicts the most important concepts of the CoPE: “Community”; “Actor” with “Role” and “Profile”; “Activity”; “Competency”; “Knowledge”; “Environment”.

Fig. 1. Main CoPE’s concepts

Community CoPEs can be characterized by three fundamental features [12]: (i) a mutual engagement, indicates how the CoPE functions and binds members together into a social entity; (ii) a joint enterprise, indicates what the CoPE is about, as understood and continuously negotiated by its members and (iii) a shared repository, represents the CoPE memory including a set of resources (Knowledge, learner profiles, outcomes, ...). Community and Practice are other characteristics of CoPE. Community builds relationships that enable collective learning. While the Practice anchors the learning in what people do. Actor and Role Actors of CoPEs are mainly working in the e-learning domain, with different levels of skills and knowledge based on their training and experience. They might be involved as: (1) members; (2) contributors (individuals participating in particular activities or during some specific periods of the CoPE’s life cycle); or (3) Partners (entities supporting the CoPE). For a better management of their work, the actors can organize themselves in groups on the basis of their objectives and concerns. A group may include actors with different roles. We distinguish two main roles: support member and learner member. The former contributes to the continuous and effective function of the CoPE (e.g. coordinator, animator, reporter, manager and administrator). While the later contributes to the realization of the current activities of the CoPE. Each role is described with data that is either already defined by IMS Learning Design specification (IMSLD) or specific to CoPEs (i.e. enriched with CoPE’s concepts in order to increase its expressing power in modeling learning situations in CoPEs). Fig. 2 shows the elements that have been added: “Category”; “Rights”; “Profile”; and “Participation”;

Towards an Ontology for Supporting CoPEs

667

Fig. 2. Conceptual model of Role

Activity We propose to classify the activities carried out within CoPEs into four categories: Analysis activities; Design activities; Implementation activities; and Utilization activities, corresponding to the steps of an e-learning development life cycle.

Fig. 3. Conceptual model of Activity

Each activity is described with data that is either already defined by IMS-LD or specific to CoPEs (i.e. enriched with CoPE’s concepts in order to capture the richness of interactions, which are inherent to collaborative activities and more particularly within CoPEs). Fig. 3 describes the elements that have been added: “Approach”; “Metadata”; “Classification”; “Execution”; “Result”. Environment The environment is composed of resources and services. Resources are classified by activity-type into: Analysis resources, Design resources, Implementation resources, and Utilization resources. We classify the CoPE’s Services as in Palette project [10]

668

L. Berkani and A. Chikh

into three categories: (1) Knowledge Management Services; (2) Mediation Services; and (3) Information Services. To describe the service sub-concept, we have adopted the Group-service structure proposed in [13] by Hernández-Leo et al., doted of necessary information. Moreover we have enriched this structure and proposed some new elements, among them: ƒ ƒ ƒ

“Service category”: indicates KM, Mediation or Information services. “Service mission”: specifies the nature of the required service (e.g. edition, communication, argumentation, help and research aspects). Its category is determined by “Service category”. “Service profile”: indicates the technical characteristics; techniques; information about connection and access.

In addition, we have proposed the relation “Composed by”, which gives the possibility to define new services by composition of some existent ones. Knowledge One of the major concerns of CoPEs is to capitalize techno-pedagogical knowledge, which can be classified into Tacit Knowledge (TK) and Explicit Knowledge (EK) as defined by Nonaka in [14]. To take advantage of the assets in the CoPE, a categorization of knowledge is done based on the four modes of the SECI framework as defined by Nonaka et al. [15]: (1) Experiential knowledge assets can be interpreted as handson experiences and skills acquired through discussion and shared practice; (2) Conceptual knowledge assets represent the EK articulated through symbols and language; (3) Systemic knowledge assets consist of systematized and packaged EK; and (4) Routine knowledge assets consist of the EK that is customized and embedded in the actions and practices.

Fig. 4. Conceptual Knowledge Model

As shown in Fig. 4, we have adopted this classification and adapted it to CoPE’s context as subclasses of the super class Techno-pedagogic knowledge. The subclasses can be illustrated respectively by the following examples: (1) the use of acquired development and/or utilization skills of an e-learning system during the analysis stage; (2) the use of knowledge acquired from e-learning standards and pedagogical ontologies to design an e-learning system; (3) the use of systemic knowledge (e.g. pedagogical resources) to develop an e-learning system; (4) and finally with some feedback, the utilization stage can lead to the definition of best practices considered as lessons learnt.

Towards an Ontology for Supporting CoPEs

669

The Knowledge concept is composed of other sub-concepts: “Description”, “Context”, “Content”, and “Metadata”.

4 Conclusion In this paper, we proposed a conceptual model of CoPEs. A set of models useful for describing such communities have been presented: “Community”, “Actor”, “Role”, “Activity”, “Environment” and “Knowledge”. Based on these models, an ontology dedicated to CoPEs will be built. This last aims at representing a uniform vocabulary to explicitly specify all the CoPE’s concepts, and with which the CoPE’s resources and services can be annotated. In our future research, we will refine and extend this work. Different aspects may be included, among them: “learner profile”, “process” and “communication”.

References 1. Langelier, L., Wenger, E.: Work, Learning and Networked, Québec, CEFRIO (2005) 2. Center for Teaching Excellence (CTE), http://www.sc.edu/cte/cop/ 3. Learning Network for Teachers (Learn-Nett), http://ute2.umh.ac.be/learn-nett/ 4. ePrep, http://www.eprep.org/ 5. Did@cTIC, http://www.unifr.ch/didactic/ 6. Chikh, A., Berkani, L., Sarirete, A.: Modeling the Communities of Practice of E-learning – CoPEs. In: 4th Annual Conference of Learning International Networks Consortium, LINC 2007 (2007) 7. Chikh, A., Berkani, L., Sarirete, A.: Communities of Practice of E-learning “CoPE” – Definition and Concepts. In: IEEE International Workshop on Advanced Information Systems for Enterprises, IWAISE 2008, pp. 31–37 (2008) 8. Rosson, M.B., Dunlap, D.R., Isenhour, P.L., Carrol1, J.M.: Teacher Bridge: Creating a Community of Teacher Developers. In: 40th Annual Hawaii International Conference on System Sciences, HICSS 2007 (2007) 9. Dubé, L., Bourhis, A., Jacob, R.: Towards a typology of virtual communities of practice, Cahiers du GReSI 03-13 (2003) 10. PALETTE: Pedagogically sustained Adaptive Learning through the Exploitation of Tacit and Explicit Knowledge, http://palette.ercim.org/ 11. Vidou, G., Dieng-Kuntz, R., Ghali, A.E., Evangelou, C.E., Giboin, A., Tifous, A., Jacquemart, S.: Towards an Ontology for Knowledge Management in Communities of Practice. In: Reimer, U., Karagiannis, D. (eds.) PAKM 2006. LNCS (LNAI), vol. 4333, pp. 303–314. Springer, Heidelberg (2006) 12. Wenger, E.: Communities of Practice: Learning as a Social System. Systems Thinker (1998) 13. Hernández-Leo, D., Asensio-Pérez, J.I., Dimitriadis, Y.A.: IMS Learning Design Support for the Formalization of Collaborative Learning Patterns. In: 4th International Conference on Advanced Learning Technologies (2004) 14. Nonaka, I.: The knowledge creating company. Harvard Business Review 69, 96–104 (1991) 15. Nonaka, I., Toyama, R., Konno, N.: SECI, Ba and Leadership: a Unified Model Knowledge Creation. Long Range Planning, vol. 33. Elsevier Science Ltd., Amsterdam (2000)

Using Collaborative Techniques in Virtual Learning Communities Francesca Pozzi Institute for Educational Technology – National Council of Research Via De Marini, 6 16149 Genoa, Italy [email protected]

Abstract. The present paper illustrates the experience gained within two “twin” online courses, where three collaborative techniques, namely the Role Play, the Jigsaw and the Discussion, were used for triggering collaboration and interactions among students. The use of the techniques in the two courses is analyzed by looking at the participative, the social, the cognitive and the teaching dimensions and the way these components vary across techniques and across the two courses. Despite the results are certainly affected by factors that could not be set aside in a real context (the individual differences of students, the topics and sequence of activities, etc.), still it is possible to draw some final considerations concerning the strong points and weaknesses of the three techniques in online learning contexts. Keywords: CSCL, collaborative technique, role play, jigsaw, discussion, evaluation.

1 Introduction Computer Supported Collaborative Learning (CSCL) is the research area that focuses on debate-based learning and peer negotiation in online learning environments (The Cognition and Technology Group at Vanderbilt, 1991; Scardamalia & Bereiter, 1994; Dillenbourg, 1999; Kanuka & Anderson, 1999). In these contexts it is quite common to adopt “techniques” or “scripts” with the aim of providing a structure to activities, so as to foster collaboration and exchange (Kanuka & Anderson, 1999; Dillenbourg 2002; Hernández-Leo et al., 2005; Persico & Sarti, 2005; Jaques & Salmon, 2007; Fischer et al., 2009). Techniques and scripts are usually content-independent and serve as scaffolds to activities (which on the other hand are content-dependent). Examples are: Discussion, Peer Review, Role Play, Jigsaw, Case Study, etc. In this paper a study is described, which illustrates the application of a Jigsaw1, a Role 1

During the Jigsaw (Aronson et al., 1978) the content to be addressed is segmented into subitems and each learner is assigned the task to study in detail his/her sub-item. To do so, all the students who should become “experts” of a specific sub-item, join together in the so called “expert group”, with the aim of discussing the main points of their segment and rehearsing a presentation. At the end of this phase, expert groups are loosened and new groups are formed, called “jigsaw groups”. Within his/her new jigsaw group, each learner is asked to report his/her segment to the others, so that at the end all the groups gain a complete overview of the content.

U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 670–675, 2009. © Springer-Verlag Berlin Heidelberg 2009

Using Collaborative Techniques in Virtual Learning Communities

671

Play2 and a Discussion3 within two online courses. Aim of this study is to investigate the above mentioned techniques in real contexts, and appreciate the differences (if any) in the learning processes they are able to trigger in online learning situations.

2 Research Context and Methods The present study is rooted in the context of two twin courses run in 2007 respectively in Liguria and Veneto on the issue “Educational Technology” (hereafter called “TDSSIS Liguria” and “TD-SSIS Veneto”). The courses were devoted to student teachers and the main aim was that of making students familiarize with the most important issues related to the introduction of ICT in schools. The communities of both the courses consisted of post-graduate adults who were diversified as for backgrounds, interests and expectations from the course; the majority of them was at its first experience of online collaborative learning. In the present study we concentrate on one class of TD-SSIS Liguria, composed of 21 students, and one class of TD-SSIS Veneto, consisting of 24 students; the two classes were tutored by the same tutor. The courses shared the same contents and structure, and thus they both envisaged three subsequent online collaborative activities (lasting 3 weeks each). The first activity was based on a Jigsaw; during the second activity students were proposed a Role Play; finally the last activity was based on a Discussion. The CMC system used for carrying out the online activities was in both cases Moodle4 (Persico et al., 2009). In order to investigate the nature of the interactions occurred while performing the proposed online activities, an evaluation framework was used, which had been previously developed and extensively used to assess similar online experiences (Pozzi et al. 2007; Persico et al., 2009). The model considers four dimensions as those characterizing a learning process in CSCL contexts, namely the participative, the cognitive, the social and the teaching dimensions. In the model, each dimension is defined by a set of relevant indicators that can be used to evaluate it; in particular: -

the participative dimension is defined by indicators of: Active Participation (P1), Reactive Participation (P2) and Continuity (P3); the social dimension is defined by indicators of: Affection (S1) and Cohesion (S2); the cognitive dimension is defined by indicators of: Individual Knowledge Building (C1), Group Knowledge Building (C2) and Meta-Reflection (C3); the teaching dimension is defined by indicators of: Organizational matters (T1), Facilitating Discourse (T2) and Direct Instruction (T3) (Persico et al., 2009).

As far as the methods and means that have allowed to gauge these indicators, an analysis of all the messages exchanged by the students during the activities (1164 2

The Role Play is a technique where students are assigned roles, so that during the discussion they cannot express their personal ideas, but they have to argument positions according to the assigned roles (Renner, 1997; Kanuka & Anderson, 1999). 3 The Discussion is a simple technique where students are asked to discuss around a topic, with the aim of collaboratively carry out a task (usually writing a document, solving a problem, etc.). 4 http://www.moodle.org

672

F. Pozzi

messages) was carried out. In particular, the indicators concerning the participative dimension have been gathered directly from the data tracked by Moodle, whereas the analysis of the cognitive, the social and the teaching dimensions is based on a “manual” content analysis5.

3 Results In the following data are synthesized concerning the participative, the social, the cognitive and the teaching dimensions, as they have been developed during the execution of the Jigsaw, the Role Play and the Discussion respectively in TD-SSIS Liguria and in TD-SSIS Veneto. As far as the participative dimension is concerned, Table 1 reports the data of active participation in the two courses. As one may note, in TD-SSIS Liguria during the Discussion the students sent the highest number of messages, while in the Jigsaw and the Role Play the number of sent messages is nearly the same. Besides, the mean messages per student is quite high in all the three activities. In TD-SSIS Veneto the number of messages is overall lower than in Liguria, but, despite this, here again the Discussion resulted to be the most participated technique, followed by the Jigsaw and then the Role Play. Table 1. Active participation in TD-SSIS Liguria and TD-SSIS Veneto6

Jigsaw Role Play Discussion

Tot. sent msgs. 203 209 265

TD-SSIS Liguria Mean SD msgs per student 7,9 3,87 8,68 5,16 11,04 5,98

Range

1-19 3-24 3-26

Tot. sent msgs. 168 137 182

TD-SSIS Veneto Mean SD msgs per student 5,6 3 5,1 2,6 6,2 2,9

Range

2-12 2-9 1-14

Going further, we looked at more qualitative data: Figure 1 and 2 contain data concerning the social, the cognitive and the teaching dimensions obtained by the three techniques in the two courses. In particular, by looking at Figure 1 (TD-SSIS Liguria), it is interesting to note that indicators seem to follow the same path independently on the technique used, namely: S1 of the social dimension (affection) is always quite low and especially the value of S1 in the Jigsaw and that in the Role Play resulted very close; S2 (cohesion) in contrast reached the highest values in all the three activities and again values of the Jigsaw and the Role Play are very similar. As far as 5

The content analysis was carried out by two coders. Each message was split into units of analysis (“units of meaning” – see Henri, 1992), so that each unit could be classified as belonging to a certain indicator. The inter-rater-reliability between the coders was calculated on a sample of 110 messages (Holsti coefficient = 0,81; percentage agreement = 0,83) (Persico et al., 2009). 6 Unfortunately, due to administrative matters, it was not possible to have data concerning the Reactive Participation (P2) and Continuity (P3) in TD-SSIS Veneto; for this reason these data are omitted here for the TD-SSIS Liguria as well.

Using Collaborative Techniques in Virtual Learning Communities

673

the cognitive dimension is concerned, C1 (individual knowledge building) is always lower than group knowledge building (C2), whereas C3 (meta-reflection) is almost absent in all the three techniques. Values of C1 in the three techniques are again quite close and the same applies to values for C2 and C3. Finally, the three indicators of the teaching dimension (T1, T2 and T3) are more or less all at the same level with the only exceptions of T1 in the Discussion and T2 in the Role Play, which both reached higher levels. Indicators of the three techniques (TD-SSIS Liguria 2007) 350 326

300 250 228 205

200

Role Play

179 163

150

Jigsaw Discussion

142 107

100

97

87 75

72 56

71

64 63

50

121

121

117

77 67 52

29 28 6

0 S1

S2

C1

C2

C3

T1

T2

T3

Fig. 1. Social, cognitive and teaching dimensions in TD-SSIS Liguria

Indicators of the three techniques (TD-SSIS Veneto 2007) 180 167 156

160

148

140 121

120

113

100

Role play

102

99 86

80 62 61 56

60

Jigsaw

81 76

69 68

Discussion

60

59

47 36

40 20

44

44

18 11 5

0 S1

S2

C1

C2

C3

T1

T2

T3

Fig. 2. Social, cognitive and teaching dimensions in TD-SSIS Veneto

As already mentioned, TD-SSIS Veneto (Figure 2) registered an overall lower number of messages. Still, again here there is a common trend in all the three activities. In particular, a bias is registered in the social dimension between affection (S1), which is quite low, and cohesion (S2), which is very high (with the only exception of the Role Play, whose S2, even if higher than S1, is sensibly lower than S2 of the other two techniques). In all the three techniques C1 is lower than C2, with the Jigsaw

674

F. Pozzi

developing the highest cognitive dimension, followed by the Discussion and then by the Role Play. Again here meta-reflection (C3) was not particularly developed by none of the proposed techniques. As far as the teaching dimension is concerned, again here differences among T1, T2 and T3 are not so evident across the three techniques, with the exception of T2 during the Discussion and T3 in the Role Play, which were sensibly lower than those of the other two techniques.

4 Discussion and Conclusions First of all, it should be noted that in both the courses, despite some differences in the values assumed by the indicators, these seem to follow the same trend independently of the technique used. In particular, the group cohesion always shows high values, while affection tends to be much lower; at the same time, it seems that individual knowledge building is on average quite low during this kind of activities, while group knowledge construction is usually high (and this is reasonable as we are in a collaborative learning context), whereas meta-reflection indicators are quite scarce in all of the three proposed activities. It should also be noted that individual knowledge building and meta-reflection are latent variables and therefore, as De Wever et al. (2006) pointed out, their low levels might not necessarily mean that they did not take place but that, simply, they were not made explicit in the student messages. Besides, it seems that all the activities have supported adequate levels of teaching dimension with no particular predisposition for one or the other aspect of it. Together with such a general “common trend”, one should also consider that each activity in our study revealed a specific ability as for supporting one or another dimension, namely: the Discussion resulted more participated by both the groups and the one which mostly fostered the social dimension; the Role Play always obtained the lowest levels for C1, as well as for C2 and C3 while it seems to be quite good as for the teaching dimension (especially for the aspects of discourse facilitation which concern taking responsibility of the group learning process); the Jigsaw obtained in both the courses the highest level of group knowledge building. This leads us to think that, if on the one hand there is no activity that - in principle - is better than others, on the other hand, the technique or script used may have a different impact on the different dimensions, i.e. a low structure seems to foster more the social dimension (as people feel more free to express their own impressions and feelings), whereas a higher degree of structuredness seems to have more positive effects on the cognitive dimension. By taking into account these final remarks, it is also worthwhile noting that some of the data in our study may have been even affected by some factors, which were impossible to be set aside in a real context: the order in which the activities were proposed, the topics of the activities themselves and even the individual differences of students (made explicit for example by the different levels of the participative dimension within the two courses) may have (at least partially) affected the results. It would therefore be interesting to carry out further investigations to ascertain whether there are significant changes in the distribution of the indicators when these variables can be set aside.

Using Collaborative Techniques in Virtual Learning Communities

675

References 1. Aronson, E., Blaney, N., Stephin, C., Sikes, J., Snapp, M.: The jigsaw classroom. Sage Publishing Company, Beverly Hills (1978) 2. De Wever, B., Shellens, T., Valcke, M., Van Keer, H.: Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review. Computers and Education 46, 6–28 (2006) 3. Dillenbourg, P. (ed.): Collaborative Learning: Cognitive and Computational Approaches. Pergamon Press (1999) 4. Dillenbourg, P.: Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In: Kirschner, P.A. (ed.) Three worlds of CSCL. Can we support CSCL, pp. 61–91. Open Universiteit Nederland, Heerlen (2002) 5. Fischer, F., Kolla, r.I., Mandl, H., Haak, J.M.: Scripting Computer-Supported Collaborative Learning. Springer, New York (2009) 6. Henri, F.: Computer conferencing and content analysis. In: Kaye, A.R. (ed.) Collaborative Learning Through Computer Conferencing, The Najaden Papers, New York, pp. 115–136. Springer, Heidelberg (1992) 7. Hernández-Leo, D., Asensio-Pérez, J.I., Dimitriadis, Y., Bote-Lorenzo, M.L., Jorrín-Abellán, I.M., Villasclaras-Fernández, E.D.: Reusing IMS-LD Formalized Best Practices in Collaborative Learning Structuring. Advanced Technology for Learning 2(3), 223–232 (2005) 8. Jaques, D., Salmon, G.: Learning in groups: A Handbook for Face-To-Face and Online Environments. Routledge, London (2007) 9. Kanuka, H., Anderson, T.: Using Constructivism in Technology-Mediated Learning: Constructing Order out of the Chaos in the Literature. Radical Pedagogy 1(2) (1999) 10. Persico, D., Pozzi, F.: Evaluation in CSCL: Tracking and analyzing the learning community. In: Szücs, A., Bø, I. (eds.) E-competences for Life, Employment and Innovation, Proceedings of the EDEN 2006 Annual Conference, Vienna, June 14-17, pp. 502–507 (2006) 11. Persico, D., Pozzi, F., Sarti, L.: A model for monitoring and evaluating CSCL. In: Juan, A.A., Daradoumis, T., Xhafa, F., Caballe, S., Faulin, J. (eds.) Monitoring and Assessment in Online Collaborative Environments: Emergent Computational Technologies for Elearning Support. IGI Global (2009) 12. Persico, D., Sarti, L.: Social Structures for Online Learning: a design perspective. In: Chiazzese, G., Allegra, M., Chifari, A., Ottaviano, S. (eds.) Methods and technologies for learning, Proceedings of the International Conference on Methods and Technologies for Learning. WIT Press, Southampton (2005) 13. Pozzi, F., Manca, S., Persico, D., Sarti, L.: A general framework for tracking and analyzing learning processes in CSCL environments. Innovations in Education and Teaching International 44(2), 169–180 (2007) 14. Renner, P.: The art of teaching adults: How to become an exceptional instructor and facilitator. The Training Associates, Vancouver (1997) 15. Scardamalia, M., Bereiter, C.: Computer support for knowledge-building communities. The Journal of the Learning Sciences 3(3), 265–283 (1994) 16. The Cognition and Technology Group at Vanderbilt: Some thoughts about constructivism and instructional design. Educational Technology 31(10), 16–18 (1991)

Capturing Individual and Institutional Change: Exploring Horizontal versus Vertical Transitions in Technology-Rich Environments Andreas Gegenfurtner1, Markus Nivala1, Roger Säljö1,2, and Erno Lehtinen1 1

University of Turku, Centre for Learning Research, Assistentinkatu 7, 20014 Turku, Finland 2 University of Gothenburg, Department of Education, Läroverksgatan 15, 40530 Göteborg, Sweden [email protected], [email protected], [email protected], [email protected]

Abstract. Popular approaches in the learning sciences understand the concept of learning as permanent or semi-permanent changes in how individuals think and act. These changes can be traced very differently, depending on whether the context is stable or dynamic. The purpose of this poster is to introduce a distinction between horizontal and vertical transitions that can be used to describe individual and institutional change in technology-rich environments. We argue that these two types of transitions trace different phenomena: Vertical transitions occur when individuals, technologies, or domains develop in stable and fixed conditions within set boundaries. In contrast, horizontal transitions occur when individuals, technologies, or domains mature in the synergy with other fields. We develop our argument by working through relevant studies in medicine, and close by outlining implications for future research on professional technology enhanced learning. Keywords: technology, change, professional learning, expertise, humanmachine systems.

1 Introduction Popular approaches in the learning sciences understand the concept of learning as permanent or semi-permanent changes in how individuals think and act. The analysis of these changes is however challenging as the face of learning currently undergoes some substantial changes: These changes relate to the technologies for learning and the technologies at work, along with respective learning contexts and pedagogical models. An important aim of technology enhanced learning (TEL) is to understand the mechanisms and functions of the individual, social, and contextual development associated with technological tools. These developments are not always linear bottom-to-top movements; they also involve side steps. Changes in the individual and changes in the context are multi-directional, although this has been rarely addressed in past research. Several authors state that there is a need to learn more about the dialectics between vertical and horizontal transitions in the development of expertise [1,2], and how the institutional context shapes learning with technology [3]. U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 676–681, 2009. © Springer-Verlag Berlin Heidelberg 2009

Capturing Individual and Institutional Change

677

The purpose of this poster is to introduce a distinction between what we term horizontal and vertical transitions that can be used to capture individual and institutional change in technology-rich environments. This distinction is seen as a methodological tool in that it directs our attention to the analytical practice. We argue that research on both vertical and horizontal transitions has merits and makes valuable contributions to advance our understanding of how to analyze change in technology-rich environments; both have their own explanatory power. Nonetheless, research investigating these transitions differs completely in its focus; while studies on vertical transitions employ a specific focus on individuals within one single domain or on a single tool, studies on horizontal transitions employ a broader perspective in that they extend their focus beyond a single domain or technology. A major problem however is that, in past research, both transitions are mixed up easily. We argue that although vertical and horizontal transitions in technology-rich environments go hand in hand, and thus can be reconciled to same extent, they should not be intermingled blindly: From an analytical stance, we argue that studies investigating vertical or horizontal transitions follow very different strategies and aims. To structure our argument on vertical and horizontal transitions, our discussion is organized in two sections. First, we discuss the concept of horizontal and vertical transitions in more detail. How does the individual and the institutional context change in technology-rich environments? How can this change be captured and analyzed? To illustrate an answer to these questions, we have chosen some significant studies in the domain of medical image diagnosis. Medicine is—among others—one example of a dynamic domain owing to its constant technological progress, and thus useful to show how learning and development occur under conditions of change. Second, we discuss implications of the proposed analytical distinction for future work in professional TEL, and how the analysis of horizontal and vertical transitions can add value for researching what it means to learn with and from technology.

2 Horizontal versus Vertical Transitions in Technology-Rich Environments In order to provide a detailed account for what we term horizontal and vertical transitions, we will focus on two levels: the individual and the institutional context in which the individual is embedded in. From an analytical perspective, the individual and the context are separated here for the sake of discussion. Our argument is put forward in the next two paragraphs by discussing individual and institutional change. 2.1 Individual Change Individual change in technology-rich environments is highly associated with technology. The interaction with technological tools and artifacts in different activities at work can trigger individual trajectories and stimulate the development of expertise. We focus here on individual trajectories although we acknowledge that these can be also related to a collective or an organizational level. Here, the individual development on a continuum of expertise can be traced as a vertical and a horizontal transition. Each is described in turn.

678

A. Gegenfurtner et al.

Traditionally, learning and the development of expertise in high-tech domains has been studied vertically by focusing on the development from novice to expert. The focus is on individual skill acquisition along a continuum of competence development. A typical assumption from studies investigating vertical transitions of an individual is that the institutional context, where the individual is working or learning in, is stable. We argue that such a perspective is useful for analyzing individual differences in routine tasks or in situations where rules are set. Examples for studies investigating individual vertical transitions can be found in classical expertise research, in domains that have reached a sufficient state of maturity. For example, in medicine, the reading of X-ray pictures with its roots back in the 19th century has been one of the most extensively studied tasks [4,5]. X-ray images remained rather constant over decades, and they even today afford the analysis of anatomical features based on grey-scale pictorial representations. Studies have mainly focused on individual differences in decision making, perceptual processes, and the representation of knowledge by comparing novices, intermediates, and experts. These comparisons are typically made in relation to a previously established “best practice”, thus treating the context as something relatively stable. Questions that are usually addressed in studies tracing vertical transitions relate to what are the characteristics of expertise on different skill levels? How can the development from novice to expert in a routine task be explained? Studies tracing horizontal transitions of the individual pose different questions, based on a different underlying assumption. Unlike to studies in stable environments, the interest here is in understanding how individuals adapt to non-routine tasks that emerge through contextual changes. To what extent are skills acquired in routine tasks transferable to non-routine tasks? The focus is on the transfer and generalizability of skills. A typical assumption from studies investigating horizontal transitions of an individual is that the institutional context is dynamic. We argue that such a perspective is useful in technology-rich environments, in order to analyze how professionals react to and cope with contextual changes. Examples for studies investigating individual horizontal transitions are surprisingly rare, although cases can be easily found in dynamic technology-rich domains. For example, in nuclear medicine, the technological standard has used to be positron emission tomography (PET). Recently however, PET has been combined with computer tomography (CT), a technology used in radiology. Physicians in nuclear medicine who have been able to analyze positron emission tomography (PET) images can now extend their skills horizontally to analyze also PET/CT images by crossing the boundaries to radiology. This boundary-crossing helps them adapt to changes in the technical domain standard. To summarize, individual change in technology-rich work environments is associated with technology. Depending on the persistence or change in the technology or domain, individuals can learn through vertical and horizontal transitions. We argue that stable conditions afford vertical learning through the mastering of routine tasks. On the other hand, dynamic conditions afford horizontal transitions through the adaptation to non-routine tasks. While we argue that both vertical and horizontal transitions account for different socio-cognitive processes, they of course complement each other. Specifically, radiologists who became experts in diagnosing X-ray images can also become experts in diagnosing PET/CT images; these two vertical movements are connected through a horizontal shift from one technology to another. We should also

Capturing Individual and Institutional Change

679

note that this shift requires a certain amount of willingness and motivation to be done. How do employees in technology-rich environments regulate their motivation? And which goals and motivational profiles support or impede transitory steps? Future research can address these questions along with the mutual complementarities of vertical and horizontal transitions that constitute individual change. 2.2 Institutional Change Institutional change in technology-rich environments is highly associated with technology. Although the institutional context can be traced on many more levels than just on the level of technology, we argue that changes in work practices, policies, communities, division of labor, or the domain as a whole are mainly following from technological changes. We illustrate this argument with two examples from medicine as a technology-rich environment: (1) the case of MRI as a vertical transition and (2) changing technologies in nuclear medicine and radiology as horizontal transitions. First, vertical transitions can be captured by focusing on one specific tool that is used in a particular domain. Questions that are usually addressed in studies tracing vertical transitions relate to how a technique has developed since its introduction, and how respectively what kind of institutional routines have emerged as a response to the development of the technical tool. In medicine, [6] analyzes the vertical transitions magnetic resonance imaging (MRI) has gone through since its development in the 1970s. First, its name changed from zeugmatography and nuclear magnetic resonance (NMR) imaging to the today established name of MRI. Second, MRI data representation changed from a numerical data output to a pictorial data output. In radiology departments, where MRI apparatuses have been installed, this has caused changes in work practices and also challenged the professional identity of radiologists. The implementation of MRI forced radiologists to adapt their work practices and to reconstitute their professional identities: New interpretation skills were required to make meaning of those new representations, and to handle the scanners appropriately. Since MRI makes no use of radiation, it was unclear if these apparatuses should be installed in radiology departments. Other departments raised a claim for the new techniques and with it a claim for the visual authority to analyze these digital pictures [7]. This example in radiology exemplifies how the transformation of imaging tools implies changes in current work practices which in turn demands professionals to renegotiate and re-organize their expertise, both in terms of individual knowledge and of their identity as a well-established discipline. In sum, the analysis of the vertical transition of one technology has the potential to uncover also the trajectories of institutional routines and how they develop over time. The second avenue to capture institutional change is to analyze horizontal transitions. This can be done by focusing on how a technological tool develops through connections to neighboring domains or by focusing on how a domain as a scientific discipline matures over time. Questions that can be addressed in studies tracing horizontal transitions relate to how a domain becomes more interdisciplinary through the introduction of a technology. How do technologies afford synergies and boundarycrossings to other domains? In medicine, horizontal transitions occur frequently through the evolution of imaging technologies. For example, as described above, nuclear medicine has faced the evolution of its technical domain standard from positron

680

A. Gegenfurtner et al.

emission tomography (PET) to a joint PET/CT image. This is seen as a horizontal transition since it involves a side step to a neighboring domain: PET/CT converges radiologic and nuclear medicine routines to produce and interpret medical images; it cuts across any neat boundaries between these two medical sub-specialties; and it creates a new stream from novice to expert in handling a new technical tool, associated with its emerging work practices and policies. Besides PET, another example is the shift from traditional X-ray technique to tomosynthesis, a new technology in which the images represent the anatomy of the lungs; the image is projected threedimensionally. Since tomosynthesis represents an improvement in the technology for diagnosing cancer in comparison to ordinary X-ray, and since the costs and radiation dose are lower than in the case of computer tomography (CT), the benefits for healthcare and patients promise to be considerable. To make full use of this technological advancement, however, it is important to further our understanding of how professionals develop expertise in using it, i.e. the very process in which they reason and make critical distinctions on how significant signs can be identified and how these should be classified. It is also interesting to trace how the introduction of a digital imaging technique in radiology departments ruptures current work practices associated with an analogous imaging technique. To summarize, institutional change in technology-rich environments is highly associated with technology. We argue that horizontal and vertical transitions of the institutional context refer to different phenomena, and they each occur under different conditions. While vertical transitions describe how a certain technology develops within a stable environment with fixed rules and clear boundaries, horizontal transitions describe how technologies and domains develop by crossing these boundaries to other technologies or domains.

3 Closing Remarks The purpose of this poster has been to introduce a distinction between horizontal and vertical transitions that can be used to capture individual and institutional change in technology-rich environments. We have argued that vertical and horizontal transitions account for different phenomena and they occur in different settings, depending on whether the context is stable or dynamic: Vertical transitions occur when individuals, technologies, or domains develop in stable and fixed conditions within set boundaries. In contrast, horizontal transitions occur when individuals, technologies, or domains mature by extending to other fields. Both transitions can and should be analyzed separately to capture micro- and macro-processes of development and change. It will be a goal for future research to identify when and how vertical and horizontal transitions intersect in the generation of learning. In closing, we discuss two implications of the proposed distinction for future work in professional TEL. First, the distinction in vertical / horizontal transitions indicates the multidirectional nature of individual and institutional development. It would be quite erroneous to assume that these developments are one-way streets. With the current speed of change in technology-rich environments, it is likely that almost every professional faces the challenge of adapting to completely new tools during one’s career. Changes can occur even in domains that seemed to be extremely stable for decades. The added

Capturing Individual and Institutional Change

681

value the vertical-horizontal distinction brings is hence associated with advancing our understanding on the multidirectional dialectics between the individual, the technology, and the broader institutional context in which both are enacted [1,2,7]. The second implication for future research relates to the ‘where’, i.e. the learning spaces in which vertical and horizontal transitions can be found. Multidirectional processes of learning occur frequently outside of school settings. [8] highlighted that the TEL community has invested maybe too much attention on technology-enhanced education and learning in formal institutions, and that they, we, need far more knowledge on learning occurring in informal settings. Hence, the workplace as a learning space becomes a central environment in which we can analyze the multi-directionality of individual and institutional development associated with technology. This is not to disregard the relevance of formal contexts; however, learning pathways over time and space, i.e. vertical and horizontal transitions, can also be addressed in realworld situations such as those arising in corporate technology-rich work settings. To conclude, both implications point to the challenge of capturing individual and institutional change which is due to the multi-directionality of both the mechanisms and the functions of individual, social, and contextual development associated with technological tools in professional work contexts.

References 1. Arnseth, H.C., Ludvigsen, S.: Approaching Institutional Contexts: Systemic versus Dialogic Research in CSCL. Int. J. CSCL 1, 167–185 (2006) 2. Sutherland, R., Lindström, B., Lahn, L.C.: Socio-Cultural Perspectives on TechnologyEnhanced Learning and Knowing. In: Balacheff, N., Ludvigsen, S., de Jong, T., Lazonder, A., Barnes, S. (eds.) Technology-Enhanced Learning. Principles and Products, pp. 39–54. Springer, Berlin (2009) 3. Ludvigsen, S.R., Havnes, A., Lahn, L.C.: Workplace Learning across Activity Systems: A Case Study of Sales Engineers. In: Tuomi-Gröhn, T., Engeström, Y. (eds.) Between School and Work: New Perspectives on Transfer and Boundary-Crossing, pp. 291–310. Pergamon, Amsterdam (2003) 4. Lesgold, A., Glaser, R., Rubinson, H., Klopfer, D., Feltovich, P., Wang, Y.: Expertise in a Complex Skill: Diagnosing X-ray Pictures. In: Chi, M.T.H., Glaser, R., Farr, M.J. (eds.) The Nature of Expertise, pp. 311–342. Erlbaum, Hillsdale (1988) 5. Morita, J., Miwa, K., Kitasaka, T., Mori, K., Suenaga, Y., Iwano, S., et al.: Interactions of Perceptual and Conceptual Processing: Expertise in Medical Image Diagnosing. Int. J. Hum-Comput. St. 66, 370–390 (2008) 6. Joyce, K.A.: From Numbers to Pictures: The Development of Magnetic Resonance Imaging and the Visual Turn in Medicine. Science as Culture 15, 1–22 (2006) 7. Burri, R.V., Dumit, J.: Social Studies of Scientific Imaging and Visualizations. In: Hackett, E.J., Amsterdamska, O., Lynch, M., Wajcman, J. (eds.) The Handbook of Science and Technology Studies, pp. 297–317. MIT Press, Cambridge (2008) 8. Pea, R.: Fostering Learning in the Networked World. Keynote presentation at the Third European Conference on Technology Enhanced Learning, Maastricht (2008)

A Platform Based on Semantic Web and Web2.0 as Organizational Learning Support Adeline Leblanc and Marie-H´el`ene Abel HEUDIASYC CNRS UMR 6599, Universit´e de Technologie de Compi`egne BP 20529, 60205 Compi`egne CEDEX, France {adeline.leblanc,marie-helene.abel}@utc.fr

Abstract. The organization’s knowledge and competences capital is increasingly crucial. Thus today, organizations are aware of the necessity to become learning organizations and to maximize organizational learning. Such a learning can be supported by information and communication technologies and more particularly by Web2.0 technologies. Within the approach MEMORAe we are interested in these new learning forms. We consider that they are connected to the knowledge management practices and we developed a learning environment based on the concept of learning organizational memory. This environment is a web platform using semantic annotations and Web2.0 technologies. Keywords: Knowledge management, Competences management, Organizational Learning, Learning Organizational Memory, Semantic Indexing.

1

Introduction

Globalization, information and communication technologies (ICT), are the new criteria of the economic environment. They transformed our way of learning and working. The organization’s knowledge and competences capital is increasingly crucial. The organization survival depends mainly on its capacity : – To access new knowledge; – To diffuse its competences quickly; – To exploit efficaciously and preserve its fields of expertise durably. However a great number of lessons, experience feedbacks are often acquired then lost. Thus today, more than ever, organization are aware of the necessity to become learning organization, i.e. organizations in which work is embedded in the organizational culture that allows and encourages the training at various levels (individual, group and organization) and the transfers of knowledge and competences between these levels. In short, they have to maximize organizational learning. Such a learning can be supported by Web2.0 technologies. Indeed after the ICT arrival, Web2.0 technologies offer new forms of sharing, exchange and learning. U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 682–687, 2009. c Springer-Verlag Berlin Heidelberg 2009 

A Platform Based on Semantic Web and Web2.0

683

Within the approach MEMORAe we are interested in these new learning forms. We consider that they are connected to the knowledge management practices and we developed a learning environment based on the concept of learning organizational memory. This environment is a web platform using semantic annotations and Web2.0 technologies. In this article, we focus on the modeling and the integration of competences in the MEMORAe2.0 project. Thus we present the link between organizational learning and competences. Then we present the approach MEMORAe and our organizational learning memory. Finally we show the E-MEMORAe2.0 web platform which we have developed.

2

Organizational Learning and Competences

In the current economic environment, to learn became the best means, for a company, to be competitive in preserving knowledge and experiments of each collaborator and each team. To become learning, on the one hand, companies must be able to capitalize and transfer the individual/collective experiments and competences created in their core. On the other hand, they must enable their members to develop their individual competences. According to the Commission of the European Communities1 competence is a combination of knowledge (explicit and implicit), abilities and skills influenced by needs, motives, personal goals, values, standards and attitudes. It is marked by effective use of resources, repeated application and accomplishment of tasks within defined conditions. Schmiedinger [1] extends the competence definition to organizations and includes therefore existing tools and materials to a new definition called ‘organizational competencies’ : Organizational competence is the combination of human competence and physical resources respectively actions successfully carried out by individuals using operating resources and work equipment or materials, to contribute to the organizational performance. These definitions show the necessity to define competences and manage resources linked to competences in order to facilitate organizational learning.

3

Approach MEMORAe

Organizational learning represents the organization capacity to increase the efficiency of its collective action. To favor this capacity organizations need to: – Manage their knowledge, competences and resources (facilitate their creation, share and capitalization),

1

Recommendation of the European Parliament and of the Council on Key Competences for Lifelong Learning (on line). (2005). http://www.ec.europa.eu/education/policies/2010/doc/keyrec en.pdf.

684

A. Leblanc and M. Abel

– Favor group work: define group (members and their functions in the group) and the group aims (project, problem, idea, ...), collaboration (group repository), communication(forum, chat,...) and coordination(shared agenda) between groups members. In the framework of the approach MEMORAe, we propose to answer to these needs in associating: – Knowledge engineering and educational engineering – Semantic Web and Web2.0 technologies to model and build a learning collaborative web platform as organizational learning support [2]. We chose to adapt the concept of Organizational Memory. Dieng define such a concept as an ‘explicit, disembodied, persistent representation of knowledge and information in an organization, in order to facilitate its access and reuse by members of the organization, for their tasks’ [3]. Extending this definition, we propose the concept of Learning Organizational Memory for which users’ task is learning.

4

Learning Organizational Memory Modeling

An organizational memory is composed of knowledge, competences and resources linked to these knowledge and competences. Our learning organizational memory modeling is structured by means of ontologies which define knowledge and competences within the organization. We used these ontologies to semantically index capitalized resources. We distinguished two types of ontology: the domain ontology and the application ontology. Each ontologies are composed by two sub-ontologies which represent competences and knowledge. Knowledge ontologies are describe in [4], in this paper we present competences ontologies. 4.1

Competence Domain Ontology

The domain ontology represents specific conceptualizations of a domain. In the framework of our projects: the domain is learning organization. Competence domain sub-ontology allows to model organizational learning competences. Stader and Macintosh proposed an ontology of organizational competences [5]. We adapt this ontology within our context. The figure 1 show a part of our domain sub-ontology centered on organizational learning competences. 4.2

Competence Application Ontology

The application ontology represents knowledge [6] and competences specific to a given application. In the framework of our project we built the ontology for B31.1 course, which is a course of applied mathematics at the University of Picardy (UPJV) in France. Figure 2 illustrates a part of the B31.1 competences ontology. These two sub-ontology are linked by the relation ‘Put into practice’. Thus a competence like ‘Summarize a random variable’ puts into practice knowledge like ‘random variable’, ‘real random variable’, etc.

A Platform Based on Semantic Web and Web2.0

5

685

E-MEMORAe2.0 Web Platform

In order to put into practice our modeling we developed the environment E-MEMORAe2.0 (see figure 3). The user interface proposes: – An access to different repositories (individual, group and organization), specifying the repository visualized and allowing to access to authorized repositories. – Entry points enabling to start the navigation with a given concept. – A short definition of the current notion. – A part of the ontology centered on the current notion. – A list of resources which contents are related to the current concept. – History of navigation. Thus, by means of this interface, users navigate through the ontologies and can explore the memory content. Vertical navigation (see figure 3) allows to explore subsumption relations and to reach related concepts. Horizontal navigation allows to explore proximity relations (other than subsumption) [2]. E-MEMORAe2.0 gives the possibility of learners to have a private space and participate to share spaces according to their rights. All these spaces (repositories) share the same ontologies but store different resources and different entry points. They can be visualized at the same time. Thus figure 3 illustrates the visualization of three spaces: one dedicated to organization members, one dedicated to gp3 members and one to the connected individual. Let us note that by default, user visualizes two repositories: one concerning his private memory and one concerning his organization memory. However he can choose spaces he wants to visualize by selecting them in the memories choice window (in the left top). These choices are registered and will be considered for the next session. Each

Fig. 1. Part of the domain ontology

Fig. 2. Part of the B31.1 competences ontology

686

A. Leblanc and M. Abel

Fig. 3. E-MEMORAe2.0 navigation interface (in French)

spaces enable to access to entry point (entry point vertical tab is selected) or resources indexed by the concept selected in the ontology map (resources vertical tab is selected). Group can work on a problem. User can use entry point to reach the problem concept (see figure 3). From this concept user can use vertical navigation to see problem type. Then he can use horizontal navigation to reach competences required to solve it. In the same way, users can reach knowledge puts into practice by competences. In such a platform, resource transfers can be done following two mainly ways: – Users can visualize different spaces/memories content at the same time. Thus, they can make a drag and drop to transfer a resource or an entry point from a specific repository to another one. – We developed a semantic forum. All the forum contributions are distributed in the resource space among the other resources (see figure 3). Users don’t access to the forum itself but to the repository resource space and then select resources of Forum type to participate to the forum about the selected concept (knowledge or competence) which thus represent the topic [2]. Consequently, users can exchange ideas about specific topics. We plan to develop semantic chats and semantic agendas in the same way.

6

Conclusion

In this paper we presented links we made between knowledge management, e-learning, semantic web, and Web2.0 technologies to build a collaborative environment in the framework of the approach MEMORAe. We focus to the

A Platform Based on Semantic Web and Web2.0

687

modeling and to the integration of competences in the MEMORAe2.0 project. We present the web platform developed E-MEMORAe2.0. It is a memory where it is possible to organize any resources or micro-resources (produced in the forum framework) in different work spaces (individual, group, organization) around shared ontologies (describing knowledge and competences). Thus users can easily transfer resources from one space to another one. All the micro-resources are capitalized and accessed like any resources in the memory (course, web site, exercise, etc.). With our approach we take into account at the same time formal (access, capitalization and sharing of explicit knowledge) and informal (tacit knowledge externalisation and capitalization) training. Our learning organizational memory allows to structure knowledge and competences of learning organization. It facilitates exchanges and interactions between learners. All these interactions are automatically capitalized and semantically indexed. E-MEMORAe2.0 evaluations gave us good results [6]. Learners used their different memories and forums. Currently our environment is used by academics. We have contact with industrials in order to evaluate such an environment to foster learning and innovation in their organization.

References 1. Schmiedinger, B., Valentin, K., Stephan, E.: Competence based business development - organizational competencies as basis for successful companies. Journal of Universal Knowledge Management (1) (2005) 2. Leblanc, A., Abel, M.-H.: E-MEMORAe2.0: an e-learning environment as learners communities support. International Journal of Computer Science and Applications, Special Issue on New Trends on AI Techniques for Educational Technologies 5(1), 108–123 (2008) 3. Dieng, R., Corby, O., Giboin, A., Ribi`ere, M.: Methods and tools for corporate knowledge management. In: Proceedings of the 11th workshop on Knowledge Acquisition, Modeling and Management (KAW 1998), Banff, Canada, pp. 17–23 (1998) 4. Abel, M.-H., Lenne, D., Leblanc, A.: Organizational Learning at University. In: Duval, E., Klamma, R., Wolpers, M. (eds.) EC-TEL 2007. LNCS, vol. 4753, pp. 408–413. Springer, Heidelberg (2007) 5. Stader, J., Macintosh, A.: Capability Modelling and Knowledge Management, Applications and Innovations. In: 19th International Conference of the BCS Specialist Group on KBS and Applied AI, Cambridge, pp. 33–50 (1999) 6. Leblanc, A., Abel, M.-H.: Using Organizational Memory and Forum in an Organizational Learning Context. In: Proceedings of the Second International Conference on Digital Information Management, ICDIM 2007, pp. 266–271 (2007)

Erroneous Examples: A Preliminary Investigation into Learning Benefits Dimitra Tsovaltzi1 , Erica Melis1 , Bruce M. McLaren1 , Michael Dietrich2 Georgi Goguadze2 , and Ann-Kristin Meyer2 1

German Research Center for Artificial Intelligence Stuhlsatzenhausweg 3, D-66123 Saarbr¨ ucken, Germany [email protected] www.activemath.org 2 Universit¨ at des Saarlandes Fachbereich Informatik, D-66123 Saarbr¨ ucken, Germany

Abstract. In this work, we investigate the effect of presenting students with common errors of other students and explore whether such erroneous examples can help students learn without the embarrassment and demotivation of working with one’s own errors. The erroneous examples are presented to students by a technology enhanced learning (TEL) system. We discuss the theoretical background of learning with erroneous examples, describe our TEL setting, and discuss initial, small-scale studies we conducted to explore learning with erroneous examples.

1

Theoretical and Empirical Background

Correctly worked examples have traditionally been used to help students learn mathematics and science problem solving and have proven to be quite effective (1; 2). However, erroneous examples, that is, worked solutions including one or more errors that the student is asked to detect, explain, and/or correct, have rarely been investigated or used as a teaching strategy, particularly not in technology-enhanced learning systems. The question of if – and how – erroneous examples are beneficial to learning is still very much open. Some theoretical and empirical research has explored the effects of erroneous examples in mathematics learning and provides some evidence that studying errors can support learning by providing new problem solving opportunities and motivating reflection and inquiry, e.g. (3; 4; 5). Moreover, the highly-publicised TIMSS studies (6) showed that math students in Asian countries – where curricula often include the careful analysis and discussion of incorrect solutions – outperform their counterparts in most of the western world. One study explored self-explaining correct and incorrect examples (7; 8). Siegler et al found that when students self-explained both correct and incorrect examples they learned more in comparison to self-explaining correct examples only. Grosse and Renkl 

This research was supported by the German DFG project ALoE (ME1136/7). The authors are solely responsible for its content.

U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 688–693, 2009. c Springer-Verlag Berlin Heidelberg 2009 

Erroneous Examples: A Preliminary Investigation into Learning Benefits

689

also showed some learning benefit of erroneous examples but only for learners with strong prior knowledge and for far transfer learning (9). We plan to take the earlier studies further by investigating erroneous examples used in the context of TEL. In contrast to other studies, we are interested in the correlations between students’ benefit from erroneous examples and the situational and learner characteristics, with an eye toward eventually adapting erroneous examples instruction. To this end, we use the adaptive learning platform ActiveMath (10), a web-based learning environment for mathematics. In contrast to the Grosse and Renkl work, we are investigating erroneous examples with help. Our primary rationale for including help in the empirical studies is that students are not accustomed to working with and learning from erroneous examples and, hence, they need assistance and support in doing so. We hypothesise that learning from the ’errors of others’ can help students enhance their cognitive competencies as well as their meta-cognition and learning orientation. We propose two primary reasons for this. First, a student can best learn error detection and correction by reviewing and studying errors, something that is impossible to do with correct examples – and difficult to do with unsupported problem solving. Second, reviewing erroneous examples appears to be more supportive of a learning orientation rather than a performance orientation. Furthermore, we hypothesise that students will benefit from erroneous examples when encountered at the right time and in the right way. Rewarding a student for error detection may lead to marking of errors in memory such that they will be avoided in subsequent retrieval. Moreover, a student is less likely to exhibit the feared ’conditioned response’ of behaviourism (i.e., internalising the error and repeating it) when studying the errors of other students, since the student has not made the error him/herself and thus has not necessarily internalised it. A student is also unlikely to be demotivated by studying someone else’s error(s), as may be the case when emphasising errors the student has made him or herself. On the contrary, in an earlier observational study, we noticed positive motivational effects of erroneous examples (11). Another issue that we plan to investigate in our research is what system affordances are prerequisite to integrating the benefits of erroneous examples in a learning system and, more specifically, what extensions are necessary to the existing ActiveMath system to implement such affordances.

2

Erroneous Examples in ActiveMath

Observational Study. To begin investigating our research questions on erroneous examples, we designed and conducted an observational study with 25 German 6-graders. The study included two phases, error detection and error correction. Figure 1 displays both phases of an erroneous example presented to a student. The translation (of the first phase) is: Susanne mixes 3 l of milk and 46 l syrup. Susanne calculates how much milk shake is made by adding 3 and 46 . Her result is a 2 l milk shake. Find the error in Susanne’s calculation. Click on the first erroneous step. The student is asked to spot the erroneous step (Schritt 1 in

690

D. Tsovaltzi et al.

Fig. 1. An erroneous example

Figure 1) and then to correct it (Schritt 5 in Figure 1); feedback varies between conditions. For instance, in Figure 1 the student selects a correct step as incorrect (i.e., Step 1) and is flagged. The feedback (translated) is “Not really. Susanne’s 3rd step is wrong”. After displaying the help message, the system asks the student to explain the error, in Figure 2, “Why is the 3rd step wrong?” with the choices • • • •

because Susanne must translate the integer 3 into a fraction because 3 has to be added to both the numerator and denominator of because the 3 has to be cancelled:  3+ 23 I don’t know.

2 3

The first selection is the correct choice. After completing this phase, the student is prompted to correct the error, as shown at the bottom of Figure 1 (Schrit 5, “Now, correct Susanne’s first wrong step”).

Erroneous Examples: A Preliminary Investigation into Learning Benefits

691

Observations. A key observation was that the 6th grade students frequently did not know how to correct the erroneous step, even when they were able to choose the correct explanation for the error. This may mean that although students know the correct rules for performing operations on fractions and can recognise explanations that refer to these rules, they still have knowledge gaps that surface when asked to correct the error. Ohlsson (12) has described Fig. 2. Choices for Explaining the Error this phenomenon as a dissociation between declarative and practical knowledge. The same phenomenon occurred even with students who could solve exercises, but could not correct the erroneous example of the same type, e.g., addition of fractions with unlike denominators. Our interpretation in this case is that students tend to solve problems following well-practiced solution steps, so their knowledge gaps are not always revealed when solving exercises. We believe these gaps may be detected through the use of erroneous examples. Feedback Design. Based on this observation, we designed feedback for helping students correct the error. There are three types of unsolicited feedback provided: minimal feedback, error-awareness and detection (EAD) feedback, and help. Minimal feedback, consists of flag feedback (green colouring for correct and red for wrong answers) along with a correct/incorrect indication. EAD feedback intends to support the meta-cognitive skills of error detection and awareness. For example, for the task in Figure 1, the English EAD feedback would be ”Susanne’s result cannot be correct because 53 l is even less than the 3 l milk”. In the first phase of the erroneous examples (finding the error), students get EAD feedback, and then multiple choice questions (MCQs) which scaffold them to correcting the error. MCQs are explanations of the error like the ones in figure 2 and are nested (3 to 4 layers). Finally, they get minimal feedback and help messages on their choices, and eventually the correct answer. In the correction phase, error correction feedback is provided, e.g., You forgot to expand the numerators. Technical Experiment Support. To facilitate TEL studies with erroneous examples, we implemented an automated presentation of the study materials for use in a classroom setting. All materials are selected through a specific strategy of ActiveMath’s exercise sequencer, which defines the order in which students from a condition/group receive their material. On top of this, a selection routine was implemented that randomly chooses the order in which the sequences of the intervention appear each time a new user logs onto the system, and starts off where it stopped after a break (necessary for longer TEL experiments). Moreover,

692

D. Tsovaltzi et al.

all materials are online, including pre- and post-questionnaires. These features are important for running controlled studies in classrooms in general. Additionally, the erroneous examples and feedback described above, as well as the GUI that represents the worked examples, exercises, and erroneous examples are implemented as a tutorial strategy in ActiveMath. Pilot Study. Later, we ran a study informed by the initial observational study, to get preliminary indications of learning effects, to test the erroneous example design, and the online presentation of examples by ActiveMath. Ten 8thgraders were randomly assigned to one of two conditions (five per condition), and completed the pilot study in two sessions. The condition No-ErroneousExamples (NOEE) included worked examples and fraction exercises, but no erroneous examples. The condition Erroneous-Examples-With-Help (EEWH) included worked examples, exercises, and erroneous examples with provision of help. The design followed a pretest-familiarisation-intervention-posttest schema, with questionnaires also provided. Each group solved five sequences of three items. The posttest consisted of five exercises and two erroneous examples, including conceptual questions on error detection. Although our sample size was too small for inferential statistics, our descriptive statistics showed that the performance of the NOEE group decreased in the post-test (pre-/post-test difference mean=-13.7, stdv=13.6), whereas the EEWH group’s performance, increased (pre-/post-test difference mean=13.1, stdv=7.7). The EEWH condition reported in a group interview that they were satisfied with the help provided by the system and found it easy to understand. No difference in performance was observed in how the students from the different conditions answered the conceptual questions and solved the erroneous examples. However, with scores of 60% vs. 55%, there was certainly room for improvement in conceptual understanding. A positive outcome of the study was that all students reported that it was enjoyable to work with the system (e.g. ”It was fun until the end!”) despite complaints that the system was not fast enough (due to server problems).

3

Outlook

In upcoming studies we plan to investigate the interplay between the two competencies: finding and explaining an error vs. correcting it. In particular, we would like to test if we can eliminate the observed discrepancy that less-advanced (6th-grade) students could find and explain errors, yet could not correct them. Ohlsson (12) argues that when the competency for finding errors is active, it functions as a self-correction mechanism that, given enough learning opportunities, can lead to a reduction of performance errors. Although reducing ones own errors is arguably different from correcting errors of others, our erroneous examples with additional feedback that specifically targets the correction of performance errors seem to be a good candidate for creating the required learning opportunities.

Erroneous Examples: A Preliminary Investigation into Learning Benefits

693

References [1] McLaren, B.M., Lim, S.J., Koedinger, K.R.: When and how often should worked examples be given to students? new results and a summary of the current state of research. In: Love, B.C., McRae, K., Sloutsky, V.M. (eds.) Proceedings of the 30th Annual Conference of the Cognitive Science Society, Austin, TX, pp. 2176–2181. Cognitive Science Society (2008) [2] Trafton, J., Reiser, B.: The contributin of studying examples and solving problems. In: Proceedings of the Fifteenth Annual Conference of the Cognitive Science Society (1993), http://www.citeseer.nj.nec.com/ [3] Borasi, R.: Capitalizing on errors as “springboards for inquiry”: A teaching experiment. Journal for Research in Mathematics Education 25(2), 166–208 (1994) [4] M¨ uller, A.: Aus eignen und fremden Fehlern lernen. Praxis der Naturwissenschaften 52(1), 18–21 (2003) [5] Oser, F., Hascher, T.: Lernen aus Fehlern - Zur Psychologie des negativen Wissens. Schriftenreihe zum Projekt: Lernen Menschen aus Fehlern? Zur Entwicklung einer Fehlerkultur in der Schule, P¨ adagogisches Institut der Universit¨ at Freiburg, Schweiz (1997) [6] OECD: International report PISA plus (2001) [7] Siegler, R.: Microgenetic studies of self-explanation. In: Granott, N., Parziale, J. (eds.) Microdevelopment, Transition Processes in Development and Learning, pp. 31–58. Cambridge University Press, Cambridge (2002) [8] Siegler, R., Chen, Z.: Differentiation and integration: Guiding principles for analyzing cognitive change. Developmental Science 11, 433–448 (2008) [9] Grosse, C., Renkl, A.: Finding and fixing errors in worked examples: Can this foster learning outcomes? Learning and Instruction 17, 612–634 (2007) [10] Melis, E., Goguadse, G., Homik, M., Libbrecht, P., Ullrich, C., Winterstein, S.: Semantic-aware components and services in ActiveMath. British Journal of Educational Technology. Special Issue: Semantic Web for E-learning 37(3), 405–423 (2006) [11] Melis, E.: Design of erroneous examples for ActiveMath. In: Looi, C.-K., McCalla, G., Bredeweg, B., Breuker, J. (eds.) 12th International Conference on Artificial Intelligence in Education. Supporting Learning Through Intelligent and Socially Informed Technology (AIED 2005), vol. 125, pp. 451–458. IOS Press, Amsterdam (2005) [12] Ohlsson, S.: Learning from performance errors. Psychological Review 103(2), 241–262 (1996)

Towards a Theory of Socio-technical Interactions Ravi K. Vatrapu Center for Applied ICT (CAICT), Copenhagen Business School Howitzvej 60, 2.floor, Frederiksberg, 2000, Denmark [email protected]

Abstract. Technology enhanced learning environments are characterized by socio-technical interactions. Socio-technical interactions involve individuals interacting with (a) technologies, and (b) other individuals. These two critical aspects of socio-technical interactions in technology enhanced learning environments are theoretically conceived as (a) appropriation of socio-technical affordances and (b) structures and functions of technological intersubjectivity. Briefly, socio-technical affordances are action-taking possibilities and meaningmaking opportunities in an actor-environment system with reference to actor competencies and technical capabilities of the socio-technical system. Drawing from ecological psychology, formal definitions of socio-technical affordances and the appropriation of affordances are offered. Technological intersubjectivity (TI) refers to a technology supported interactional social relationship between two or more actors. Drawing from social philosophy, a definition of TI is offered. Implications for technology enhanced learning environments are discussed. Keywords: apperception, perception, and appropriation of affordances, technological intersubjectivity, socio-technical systems, technology enhanced learning, computer supported collaborative learning, comparative informatics.

1 Introduction There are two interrelated aspects of interactions in designing, developing, using, and evaluating technology enhanced learning (TEL) systems: (i) interacting with technologies and (ii) interacting with others such as peers and teachers. These two interactional aspects are mutually interdependent and are termed socio-technical interactions. Despite their critical centrality, socio-technical interactions in technology enhanced learning in general have not received necessary and sufficient theoretical consideration. This paper attempts to address this theoretical lacuna and hopes to jumpstart an empirically informed theoretical discussion on socio-technical interactions. As such, this theoretical project is not merely about Human Computer Interaction (HCI) – i.e., interacting with technology – it is also about technological intersubjectivity (TI) – i.e., interacting with people via technology. U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 694–699, 2009. © Springer-Verlag Berlin Heidelberg 2009

Towards a Theory of Socio-technical Interactions

695

2 Theoretical Framework 2.1 Affordances The notion of affordance was introduced by J. J. Gibson [1]. Gibson was primarily concerned with providing an ecologically grounded explanation to visual perception. The ontological foundations of the notion of affordances are materialist and dynamicist [2]. Turvey [2, p. 180] citing Lombardo [3] identifies “the principle of reciprocity— distinguishable yet mutually supportive realities” as the central insight of Gibson’s ecological psychology of visual perception. This principle of reciprocity is highly relevant to technology supported collaboration as multiple individuals each with a specific subjectivity and identity shape mutually supportive interactional realities. The ecological approach is dynamicist but not dialectical and processual, holding that “everything changes in some respects, but not in all respects” [2, p. 175]. Drawing upon foundational work in ecological psychology on the formal definition of affordances [2, 4], the following definition of socio-technical affordance is provided. Narrative expositions follow the definition. 2.1.1 Definition of Socio-technical Affordance Let Wpqr (e.g., person-sending-email-to-another-person system) = (Tp, Sq, Or) be composed of different things T (e.g., concept-mapping technology); S (e.g., conceptmap node creator) and O (e.g., concept-map node receiving partner). Let p be a property of technology T; q be a property of subject S and r be a property of other O. The relation between p, q and r, p/q/r, defines a higher order property (i.e., a property of the socio-technical system), a. Then a is said to be a socio-technical affordance of Wpqr if and only if (i) Wpqr = (Tp ,Sq, Or) possesses a (ii) Neither T ,S, O, (T, S), (T,O), (S,O) possesses a The formal definition of socio-technical affordance presented above is for the minimal situation of dyadic interaction in technology supported interactional environments. For a social situation involving n distinct social actors, an n-tuple would characterize the system. This formalism can be read as an activity system of subject, object and tools [5]. Relating the definition to Latour’s actor-network theory [6], both actors and “actants” are implicated in the notion of socio-technical affordances. The formal definition of socio-technical affordance captures the two facets of interaction in socio-technical systems: (1) interacting with technology and (2) interacting with other persons (technological intersubjectivity to be discussed later). It is important to realize that affordances are action-taking possibilities and meaning-making opportunities in actual situations in an actor-environment system relative to actor competencies and technology capabilities. Norman’s [7] gulf of execution and gulf of evaluation can be read as gulfs in the perception of action-taking possibilities and meaning-making opportunities respectively. Socio-technical affordances are not things or widgets or features or functionalities. This category conflation has been the source of much confusion in the HCI design community [8]. Socio-technical affordances are the relational properties in particular situations of a specific user-technology system. By virtue of being relational properties with reference to an actor, socio-technical affordances can be termed relative to the actor and/or

696

R.K. Vatrapu

the technology, but relativity is not subjectivity. In that sense, affordances are not subjective properties. Affordances are neither arbitrary properties nor are they socially constructed [9]. Affordances are relational through and through, as they are the informational structure to be perceived in ambient arrays of the actor-environment system. The next section presents a brief discussion of the notion of “appropriation of affordances”. 2.2 Appropriation of Affordances Cognition in the ecological psychology sense has been articulated as the “cooperative appropriation of affordances” [10, p. 135]. After Rogoff and Lave [11], “cognition is something one uses, not something one has”. In my reading of Gibson [1], the notion of affordance simultaneously specifies the two concurrent levels of meaning and action. Affordance is a meaning-making opportunity and simultaneously an action-taking possibility in an actor-environment system in a particular situation. Although the perception of affordances can be accounted on ecological grounds, the perception of events cannot be accounted on strictly ecological ontological grounds [12]. The perception of events has interactional consequences in technology supported collaboration. It is here that Gibson’s rejection of a role for higher order cognitive processes is problematic. Social interactional consequences from an individual’s perception of affordances are influenced by a prospective projection into the future as well as a socio-psychological imagination of the other. Adapting Stoffregen’s discussion of behavior [4, p.125], appropriation is “what happens at the conjunction of complementary affordances and intentions or goals”. Based on Stoffregen’s definition of behavior [4], the following definition is offered for appropriation of affordances. 2.2.1 Definition of Appropriation of Affordances Let Wpqr (e.g., person-sending-email-to-another-person system) = c(a, i) be composed of different affordances, a (e.g., e, the opportunity to compose email, f, the opportunity to forward email, g, the opportunity to solve a science problem); and complementary intentions, i (e.g., h, the intention to send email, j, the intention forward email, k, the intention to solve a science problem), where both affordances and intentions are properties of the socio-technical system. A given appropriation b (e.g., sending email) will occur if and only if (and when) an affordance (e.g., e) and its complementary intention (e.g., h) co-occur at the same point in the space–time continuum, where c is a cultural-cognitive choice function. Unlike orthodox cognitivist views of the representational nature of human cognition that posits “copying” the external world, the cultural cognitive conception of socio-technical affordances and their appropriation views interaction as “coping” with the contingencies of the external world [13]. Interactions in socio-technical environments are a dynamic interplay between ecological information as embodied in artifacts and individual interpretation grounded in cognitive schemas. The essential mediation of all interaction is the central insight of socio-cultural theories of the mind [14]. The conception of interaction as being mutually “accountable” and systematic are the critical insights of ethnomethodology [15] and conversational analysis [16]. Accordingly, the cultural-cognitive choice function c represents the cultural-cognitive mediation of interaction. Interactions in socio-technical systems are conceived as the appropriation of socio-technical affordances. Even if socio-technical affordances are

Towards a Theory of Socio-technical Interactions

697

to be directly perceived, their appropriation is still influenced by the cultural cognition of social actors. This renders the concept of affordance ecologically cognitive. The notion of technological intersubjectivity (TI) is discussed next. TI addresses the second aspect of socio-technical interactions in technology enhanced learning environments: how participants relate to and form impressions and opinions of each other during and after technology supported interactions. 2.3 Technological Intersubjectivity Intersubjectivity is the key presupposition underlying human social interaction [17]. Human beings are not only functional communicators but also hermeneutic actors. Technological intersubjectivity is an emergent resulting from a technology supported self–other social relationship. In technological intersubjectivity, technology mediation can sometimes (but not necessarily always) disappear like in Clarke’s [18] third law of technology. 2.3.1 Definition of Technological Intersubjectivity Technological intersubjectivity (TI) refers to a technology supported interactional relationship between two or more participants. TI emerges from a dynamic interplay between the technological relationship of participants with artifacts and their social relationship with others. Information and Communication Technologies (ICT) and the Internet have changed our social relations with others and objects in fundamental ways that transcend technology mediation. Our psychological perception and phenomenological relation with others is changed fundamentally by the advances in information and communication technologies and social software. Our interactions with others and objects are increasingly informed by the logic of technology, hence technological intersubjectivity. (Note that natural language is the bedrock of TI). Technological intersubjectivity deals with the ICT enabled capabilities to place-shift (i.e., to be physically embodied in one physical space but to be able to virtually embodied in a different place) and the ability to time-shift (i.e., to be able to refer back to earlier interactions or to be able to defer forward interactions).

3 Discussion The rethinking of the productive notion of affordances can help inform the design of TEL systems. The concept of affordance has been much used, misused, and abused in fields of human computer interaction [8] as well as in the learning sciences. In my opinion, most current usages of the term affordance are far removed from its ecological origins and subsequent developments in ecological psychology. In many ways, the concept of affordance had been subjected to “conceptual stretching” by uncritical conflation with “technology features”. By returning the concept of affordance to its ecological roots and following its intellectual trajectory since Gibson’s seminal contribution, this theoretical framework rethinks affordances as socio-technical action taking possibilities and meaning making opportunities in an actor-environment sociotechnical system relative to actor competencies and technology capabilities. This

698

R.K. Vatrapu

allows us TEL researchers and practitioners to critically engage with design and evaluation of learning technologies by concentrating on all four aspects: (a) action taking possibilities, (b) meaning making opportunities provided by intended design or creative appropriation, (c) how these are relative to learner competencies in terms of digital literacy, domain-specific knowledge, motivation, critical thinking competencies, and (d) finally the pedagogically innovative technological capabilities built into the TEL system. The definition and discussion of the concept of appropriation of affordance indicates that learners situated in TEL environments might choose to appropriate culturally relevant (or appropriate) affordances. That is, context-sensitive and situation-bounded embodied actions of individual learners engaged in TEL environments will be influenced by not only the micro-genetic unfolding interactional contingencies but also by the macro-structural cultural concerns and metacognitive functions [19-21]. This allows for a richer conception, instrumentation, and analysis of interactional data from the TEL environments [see 22, for a description of a design framework of usability, sociability, and learnability]. The concept of technological intersubjectivity (TI) goes beyond the traditional HCI notions (such as presence and connected presence) and the humanities’ notions (such as networked individualism, information subject) by bringing together both psychological and phenomenological aspects of technology supported social interactions [23]. This provides for a broader and deeper understanding of the new generation of learners that are increasingly growing up with pervasive and ubiquitous information and communication technologies and other computational devices and gadgets (such as the so-called millennials and digital natives). One of the prime arguments for TEL has been that in a world of constant connectivity and near ubiquity of ICTs, technologies must be leveraged pedagogically. However, as pointed earlier, there hasn’t been theoretical work that sought to bring together these macro-sociological, technological, and pedagogical trends and aspirations together into a theoretically coherent framework that can be empirically evaluated. Hopefully, these efforts will jumpstart an empirically informed theoretical discussion on socio-technical interactions in TEL.

Acknowledgments Special thanks to Dan Suthers, Scott Robertson, Marie Iding, Marc Le Pape, Pat Gilbert, Nathan Dwyer, Richard Medina and anonymous reviewers for constructive feedback on an earlier version of these ideas.

References 1. Gibson, J.J.: The ecological approach to visual perception. Houghton Mifflin, Boston (1979) 2. Turvey, M.T.: Affordances and Prospective Control: An Outline of the Ontology. Ecological Psychology 4, 173–187 (1992) 3. Lombardo, T.J.: The reciprocity of perceiver and environment: The evolution of James. J. Gibson’s ecological psychology. L. Erlbaum Associates, Hillsdale (1987) 4. Stoffregen, T.A.: Affordances as Properties of the Animal-Environment System. Ecological Psychology 15, 115–134 (2003)

Towards a Theory of Socio-technical Interactions

699

5. Kaptelinin, V., Nardi, B.A.: Acting with Technology: Activity Theory and Interaction Design. MIT Press, Cambridge (2006) 6. Latour, B.: Reassembling the Social: An Introduction to Actor-Network-Theory. Oxford University Press, Oxford (2005) 7. Norman, D.: The design of everyday things. Doubleday, New York (1990) 8. Torenvliet, G.: We can’t afford it!: the devaluation of a usability term. Interactions 10, 12– 17 (2003) 9. Hacking, I.: The Social Construction of What? Harvard University Press, Cambridge (1999) 10. Reed, E.S.: Cognition as the Cooperative Appropriation of Affordances. Ecological Psychology 3, 135–158 (1991) 11. Rogoff, B., Lave, J.: Everyday Cognition: Its Development in Social Context. Harvard University Press, Cambridge (1984) 12. Stoffregen, T.A.: Affordances and Events. Ecological Psychology 12, 1–27 (2000) 13. Blumer, H.: Symbolic Interactionism: Perspective and Method. Prentice-Hall, Englewood Cliffs (1969) 14. Wertsch, J.: Vygotsky and the social formation of mind. Harvard University Press, Cambridge (1985) 15. Garfinkel, H.: Studies in Ethnomethodology. Prentice-Hall, Englewood Cliffs (1967) 16. Sacks, H., Schegloff, E.A., Jefferson, G.: A Simplest Systematics for the Organization of Turn-Taking for Conversation. Language 50, 696–735 (1974) 17. Crossley, N.: Intersubjectivity: The Fabric of Social Becoming. Sage, London (1996) 18. Clarke, A.C.: Profiles of the future: an inquiry into the limits of the possible. Harper & Row (1962) 19. Vatrapu, R.: Cultural Considerations in Computer Supported Collaborative Learning. Research and Practice in Technology Enhanced Learning 3, 159–201 (2008) 20. Vatrapu, R.: Technological Intersubjectivity and Appropriation of Affordances in Computer Supported Collaboration. Communication and Information Sciences, PhD. University of Hawaii at Manoa, Honolulu, 538 (2007), http://lilt.ics.hawaii.edu/~vatrapu/docs/ Vatrapu-Dissertation.pdf 21. Vatrapu, R., Suthers, D.: Culture and Computers: A Review of the Concept of Culture and Implications for Intercultural Collaborative Online Learning. In: Ishida, T., Fussell, S.R., Vossen, P.T.J.M. (eds.) IWIC 2007. LNCS, vol. 4568, pp. 260–275. Springer, Heidelberg (2007) 22. Vatrapu, R., Suthers, D., Medina, R.: Usability, Sociability, and Learnability: A CSCL Design Evaluation Framework. In: Proceedings of the 16th International Conference on Computers in Education, ICCE 2008 (2008) (CD-ROM) 23. Vatrapu, R., Suthers, D.: Technological Intersubjectivity in Computer Supported Intercultural Collaboration. In: Proceeding of the 2009 international Workshop on intercultural Collaboration, IWIC 2009, Palo Alto, California, USA, February 20-21, pp. 155–164. ACM, New York (2009)

Knowledge Maturing in the Semantic MediaWiki: A Design Study in Career Guidance Nicolas Weber1,2 , Karin Schoefegger1 , JennyBimrose3, Tobias Ley2 , Stefanie Lindstaedt1,2 , Alan Brown1 , and Sally-Anne Barnes3 1

Knowledge Management Institute, Graz University of Technology 2 Know-Center 3 Institute for Employment Research, University of Warwick [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]

Abstract. The evolutionary process in which knowledge objects are transformed from informal and highly contextualized artefacts into explicitly linked and formalized learning objects, together with the corresponding organisational learning processes, have been termed Knowledge Maturing. Whereas wikis and other tools for collaborative building of knowledge have been suggested as useful tools in this context, they lack several features for supporting the knowledge maturing process in organisational settings. To overcome this, we have developed a prototype based on Semantic MediaWiki which enhances the wiki with various maturing functionalities like maturing indicators or mark-up support. Keywords: Knowledge Maturing, Semantic MediaWiki.

1

Introduction

Resources in an organizational environment change over the time. Since enterprises need to become increasingly agile in order to compete successfully, the adaption of the resource to the users needs and the constantly changing requirements are a crucial factor. Resources like e-mail, web content, documents facilitate the fulfillment of our daily tasks by providing the basis for knowledge intensive work. The improvement and gradual standardization of knowledge artifacts over the time and the accompanying organisational learning processes have been characterized as Knowledge Maturing, see [7]. This paper describes a design study conducted in an ongoing EU funded project called MATURE (http://mature-ip.eu/en/) where the objective is to understand the maturing process and provide maturing support for knowledge workers in a collaborative U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 700–705, 2009. c Springer-Verlag Berlin Heidelberg 2009 

Knowledge Maturing in the Semantic MediaWiki

701

environment. The design study as part of the requirements elicitation and analysis aims at identifying requirements for a future system supporting the maturing process of knowledge objects. Also, the purpose of the design study is to ground the theoretical ideas of knowledge maturing into organisational practice. There are several approaches analyzing the theory of Knowledge Maturing. [6] describes conceptual foundations for systems which support knowledge maturing. For that purpose the three dimensions, content, semantics and processes were taken into account. Indicators for content maturing were examined in [2] by analyzing articles within the online encyclopedia Wikipedia. [1] deals with ontology maturing in folksonomies and [4] covers ontology evolution in Web2.0 environments. [5] describes the lifecycle of task patterns as part of process management.

2

Maturing Services for the Semantic Media Wiki

Wikis are prime examples of tools that allow for a collective construction of knowledge in a community setting. There are certainly good examples of Wikis being used as tools for creating a collective knowledge repository, for teaching and learning purposes, and for organizational knowledge management, see [3]. In the perspective of a knowledge worker, Wikis might be very well suited for enabling the maturing of artefacts, especially because of the ease of editing the content and the policy that everyone can edit anything. Additionally, they make the collective construction process traceable (utilizing the wiki’s history functionality) and allow for discussion processes around artefacts. The career guidance sector is heavily content dependent (Labour Market Information, statistics etc.), thus a Semantic MediaWiki was chosen as a basis for a prototype supporting knowledge maturing in this sector. Several functionalities were developed to enrich the Semantic MediaWiki in terms of searching, collaborating, adding semantic mark-up and visualisation. Each of these services to support knowledge workers will be described in detail in the following. Search Support Service. This service provides a search interface which helps the user to aggregate information related to a certain topic without the need to use multiple search engines. Using different search facilities of various web resources (yahoo search, YouTube, wiki articles, Xing) and Yahoo Omnifind to enable including local information sources, the Search Support Service provides a combined interface that is embedded in the edit-mode of a wiki article. By default, the tags suggested by the system on the basis of the existing text in the article, are used as default search keywords. The wide range of information sources, varing between textual content, pictures, persons, ... stimulates the user’s inspiration and so provokes the evolutionary growth. Collaboration Initiation Service. This service offers the facility to initiate easy collaboration with authors of articles or interested persons via Skype (see fig. 2 (marker 4)) by not having to switch to another tool since it is embedded into the wiki and enables easier use. The user can send messages or web-links

702

N. Weber et al.

Fig. 1. Search Service - Interface

to wiki articles in order to support negotiation of and consolidation of artefacts. Additionally, within the visualisation of the wiki network, every author related to an article in the wiki can be contacted by clicking on the author’s node. Maturing Indicator Services. The objective of analyzing content is to facilitate the assessment of the maturity of a document. This maturity level allows to decide whether the maturity of a certain document should be improved by supporting the user in creating or editing a knowledge artefact. The bottleneck in assessing the maturity of text is the selection of qualified attributes reflecting the maturity of the content. Assuming that the readability and the maturity have a strong correlation, see [2], we tested within the design study two metrics for readability scores where both scores analyse English text samples. Mark-up Recommendation Service. Creating semantic mark-up conveys to the enrichment of wiki content. Additional annotation of articles enables the user to browse through the wiki and facilitates the retrieval of knowledge based on semantic mark-up. In addition, mark-up is used as a basis for recommendation of useful resources and visualisation of emergent content structures. The markup recommendation services strive for two goals. First, lowering the barrier for creating mark-up which replaces the complex Semantic MediaWiki syntax and second, improving the quality of structure by recommendation of meaningful, pre-consolidated mark-up. Depending on the content of an article, the system analyses used words and their frequencies to recommend the most used keywords as tags for the article, see fig. 2 (marker 3).In order to categorize articles, the system suggests already existing categories which corresponds best to the newly created content, see fig. 2 (marker 2). Additionally, the user can add a certain category which seems to be appropriate and can train the service with this category such that the system can suggest this category in future for appropriate and related articles. Visual Semantic Browsing Service. This service provides a visualization for the content of the Semantic Media Wiki. Each node in the graph represents either an article in the Wiki or a registered user. Directed edges represent the relations, for instance an article might have an assigned category, author, tag

Knowledge Maturing in the Semantic MediaWiki

703

Fig. 2. A Semantic Media Wiki Edit-Page with additional feature bar

or linked article. A user might have written one or more articles, or a category might contain one or more sections, articles, tags, etc. Depending on the choice of the maximum shown path-length, the user can define how many levels (and nodes) of the network are shown in the visualisation, as well as the type of the representing graph (e.g. hierarchically, cyclic). By clicking on a node in the graph, the visualization is updated and its connected nodes are shown, which enables the user to browse easily through the content of the wiki within the graph. Additionally, new nodes (users or articles) can be created; articles corresponding to a certain node in the graph can be opened and edited in a new browser window; and users corresponding to nodes can be contacted by using the Collaboration Initiation Service. This service supports the daily work of users by enabling visual browsing through wiki content from article to related articles or users. Thus, it assists by providing an overview of related topics and experts and offers easy negotiation by embedding a collaboration service.

3

Evaluation in a Real World Context

In order to gain insight into and to obtain new ideas about how a system could support the knowledge maturing process, a prototype was developed and evaluated in a real world context of career guidance organizations, whose service is delivered by specially trained Personal Advisers (P.A.s). The implementation of the prototype for this design study was done using rapid prototyping which involves iterative design phases using mock-ups and development phases combined

704

N. Weber et al.

Fig. 3. Visualisation Service

with regular input to generate feedback of the viability of our approach on supporting knowledge maturing in the context of career guidance. Questionnaires, interviews and a workshops with potential users from career organizations, were used to gain this regular feedback and input for further development in the evaluation process. Visual Appearance. Visual adaptation of the system would be necessary depending on individual preferences and learning styles. The easier a user can adapt the system to his/her needs, the more likely is it that the motivation of using a system for every day work grows. Easy Access to Relevant Information. Users might lack time to research information and therefore would need easy access to which articles are relevant for them. To support this, each article could have a summary which is shown when articles are listed as a search result or on the top of a page. Additionally, this summary could be shown within the visualisation of the wiki content when an employer moves the mouse over a node representing this article. Accuracy Control Concerning Time and Content. is necessary to make sure the data is accurate, up-to-date and relevant. Long articles are unlikely to be read and it will be too time consuming to search through for the information a user is looking for.Insted of a moderator, the idea of automatic date flags could be used to remind authors and editors to update a certain knowledge artifact. Awareness for Collaboration. Collaboration in organisations support employers to discuss new ideas and to provide help when questions arise or problems are encountered. The user should be able to see immediately who is online and who is not to be aware of whom to ask for help or discussion.

4

Conclusion

The main purpose of this work was to gain insight into the knowledge maturing process in the real world context of career guidance organizations by developing a tool that supports this process. The potential of the system in this context

Knowledge Maturing in the Semantic MediaWiki

705

was to be explored and it was to be researched how the utility of this system could be further enhanced. To support newly appointed personal advisors of career guidance organisations in a typical working process and the corresponding stages of knowledge maturation, a Semantic Media Wiki was employed which was enriched by several user interfaces that extend the usability of the Wiki in terms of collaboration, content visualisation and easy use of the system. Several maturing indicators and services have been designed that try to bridge the gaps in the maturing process. Furthermore, an evaluation of the prototype in a realworld-context helped to gain a deeper insight on features that are relevant for supporting knowledge maturing in career guidance organisations and the main aspects of their requirements can be easily adopted for a system supporting knowledge maturing of a knowlegde worker in other contexts. Acknowledgement. This work has been partially funded by the European Commission as part of the MATURE IP (grant no. 216346) within the 7th Framework Programme of IST. The Know- Center is funded within the Austrian COMET Program - Competence Centers for Excellent Technologies - under the auspices of the Austrian Ministry of Transport, Innovation and Technology, the Austrian Ministry of Economics and Labor and by the State of Styria.

References 1. Braun, S., Schmidt, A.: People Tagging & Ontology Maturing: Towards Collaborative Competence Management. In: 8th International Conference on the Design of Cooperative Systems (COOP 2008), Carry-le-Rouet, France (2008) 2. Braun, S., Schmidt, A.: Wikis as a Technology Fostering Knowledge Maturing: What we can learn from Wikipedia. In: 7th International Conference on Knowledge Management (IKNOW 2007), Special Track on Integrating Working and Learning in Business (IWL), Austria (2007) 3. Jaksch, B., Kepp, S.J., Womser-Hacker, C.: Integration of a wiki for collaborative knowledge development in an e-learning context for university teaching. In: Holzinger, A. (ed.) USAB 2008. LNCS, vol. 5298, pp. 77–96. Springer, Heidelberg (2008) 4. Juffinger, A., Neidhart, T., Granitzer, M., Kern, R., Scharl, A.: Distributed Web2.0 Crawling for Ontology Evolution. International Journal of Internet Technology and Secure Transactions 5. Ong, E., Grebner, O., Riss, U.: Pattern-Based Task Management: Pattern Lifecycle and Knowledge Management. In: WM 2007 Proceedings of the 4th Conference Professional Knowledge Management. IKMS 2007 Workshop Potsdam, Germany, pp. 357–364 (2007) 6. Schmidt, A., Hinkelmann, K., Ley, T., Lindstaedt, S., Maier, R., Riss, U.: Conceptual Foundations for a Service-oriented Knowledge and Learning Architecture: Supporting Content. In: Process and Ontology Maturing. Springer, Heidelberg (2009) 7. Schmidt, A.: Knowledge Maturing and the Continuity of Context as a Unifying Concept for Knowledge Management and E-Learning. In: Proceedings of I-KNOW 2005, Graz, Austria (2005)

Internet Self-efficacy and Behavior in Integrating the Internet into Instruction: A Study of Vocational High School Teachers in Taiwan Hsiu-Ling Chen Graduate School of Technological and Vocational Education, National Taiwan University of Science and Technology, Taipei 106, Taiwan [email protected]

Abstract. The purpose of the study was to explore the relationship between Internet self-efficacy and behavior in integrating the Internet into instruction. Participants in the study were 449 vocational high school teachers in Taiwan. A validation study was conducted with Internet Self-Efficacy Scale (ISES) and Integrating the Internet into Instruction Behavior Scale (IIIBS). The findings revealed that general and communicative Internet self-efficacy might foster behavior in integrating the Internet into instruction. The teachers’ behavior was classified as five aspects: course preparation, teaching activities, learning guidance, assessment, and product sharing. Furthermore, this study employed structural equation model (SEM) to investigate the causal relations among the variables considered in this study. The SEM analysis revealed that teachers with higher Internet self-efficacy showed more Internet integration in their course preparation. In addition, course preparation is a mediating factor between Internet self-efficacy and other four aspects of behavior in integrating the Internet into instruction. Keywords: Internet self-efficacy, integrating the Internet into instruction, vocational high school teachers.

1 Introduction The Internet is widely used in vocational high schools. The use of Internet technology to learn for educational purposes has been growing rapidly. Through the Internet, students can access useful tools and resources to enhance their learning. Perceiving the power of Internet-based technologies, the Ministry of Education in Taiwan has encouraged the use of information technology in schools. In addition, teachers have been encouraged to attend a series of workshops to prompt their informational literacy and further foster their ability to integrate information technology into their teaching. However, the implementation of technology integration is burdensome. Not every teacher is willing to adopt new approach for their teaching. Thus, study on factors related to teachers’ behavior in integrating the Internet into instruction is crucial for educational policy and intervention. Self-efficacy can be described as a person’s beliefs, expectations and perceived confidence in him/her to successfully perform a task [1][2][3]. Research has showed U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 706–711, 2009. © Springer-Verlag Berlin Heidelberg 2009

Internet Self-efficacy and Behavior in Integrating the Internet into Instruction

707

that self-efficacy affects people’s effort to devote while performing a task, and people’s persistence to deal with difficult situations [4][5]. Studies have further revealed a link between teachers’ self-efficacy and students’ achievement [6][7]. Thus, teachers’ self-efficacy is a valuable issue for educators. Harrison, Rainer, Hochwarter, & Thompson [8] suggested that employees with a high level of computer self-efficacy increased performance with computer-related tasks significantly. The explosive growth of computers and the Internet into the classroom over the last decades has made most teachers access to the Internet easily. Therefore, their Internet selfefficacy, which may affect their behavior in integrating Internet into instruction, would be an important topic to study.

2 Methodology 2.1 Participants This investigation used purposive sampling, and focused on the vocational high school teachers in Taiwan. A total of 449 paper-and-pencil survey questionnaires were gathered. The participants were 449 teachers from a selection of schools in Taiwan. They dispersed in 36 vocational high schools, including 316 female and 133 male teachers. 2.2 Instruments Internet Self-Efficacy Scale (ISES) and Integrating Internet into Instruction Behavior Scale (IIIBS) were utilized to meet the purpose of this study. Internet Self-Efficacy Scale (ISES) is designed to assess teachers’ self-perceived confidence and expectation of using Internet, including general internet self-efficacy and communicative internet self-efficacy. ISES used a six-point Likert scale with 10 items, which were adapted from the items developed by Tsai and Tsai [9] and Peng, Tsai and Wu [10]. The ten items were divided into two factors, the first one assessed teachers’ Internet selfefficacy in general (5 items) and the second one assessed teachers’ efficacy in communication and interaction on the Internet (5 items). The Cronbach coefficient alpha reliability for these two scales were 0.91 and 0.96, and the overall alpha was 0.93, indicating that the internal reliability is adequate [11]. Integrating Internet into Instruction Behavior Scale (IIIBS) is designed to understand teachers’ behavior in integrating Internet into instruction. IIIBS has 18 items. IIIBS scale used a six-point Likert scale with five subscales, including Course preparation (4 items), Teaching activities (3 items), Learning guidance (4 items), Assessment (4 items) and Product sharing (3 Items). The Cronbach coefficient alpha reliability for these five scales were 0.95, 0.86, 0.89, 0.90 and 0.83, and the whole scale was 0.94, indicating that the internal reliability is adequate [11].

3 Results 3.1 Descriptive Data Table 1 shows descriptive data for teachers’ responses on Internet self-efficacy and Integrating the Internet into instruction behavior. These teachers displayed better

708

H.-L. Chen

general Internet self-efficacy than communicative Internet self-efficacy (mean score 5.40 versus 4.32). Moreover, they expressed inconsistent tendency for five aspects of integrating the Internet into instruction behavior (between 4.71 and 2.55 in 1-6 Likert scale). They preferred to integrate Internet into their course preparation more than teaching activities, product sharing, learning guidance and assessment. Table 1. Descriptive data for teachers’ scores on ISES and IIIBS

Scale General Internet Self-efficacy Communicative Internet Self-efficacy Assessment Course Preparation Learning Guidance Teaching Activities Product Sharing

Mean 5.397 4.318 2.553 4.710 3.410 3.578 3.531

S.D. 0.659 1.403 1.166 0.891 1.123 1.132 1.191

3.2 The Correlation between Internet Self-efficacy and Integrating the Internet into Instruction Table 2 displays Pearson correlation analysis between teachers’ scores on ISES and IIIBS. It was found that teachers’ communicative Internet self-efficacy and their scores on each scale of IIIBS were all significantly positively correlated. That is, teachers with higher communicative self-efficacy integrated more Internet resources into all aspects of their instruction. On the other hand, teachers’ general Internet self-efficacy was also significantly related to their behavior in integrating the Internet into instruction, except for “assessment.” Both general and communicative Internet self-efficacy played an important role on teachers’ integrating Internet into instruction. High Internet self-efficacy may promote teachers to integrate the Internet into their teaching. Table 2. The correlation between teachers’ responses on ISES and IIIBS

General Internet Self-efficacy Course Preparation .461*** Teaching Activities .272*** Learning Guidance .184*** Assessment .055 Product Sharing .245*** ***p [6,7], an authoring tool for the production and reengineering of IMS Learning Design (IMS LD) Units of Learning (UoL) developed at Complutense University. U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 725–731, 2009. © Springer-Verlag Berlin Heidelberg 2009

726

I. Martínez-Ortiz, J.-L. Sierra, and B. Fernández-Manjón

2 The Language-Driven ADDIE Model The Language-Driven ADDIE (LD-ADDIE) model is sketched in Fig. 1. This model is based on the revised ADDIE model proposed by the US Department of the Air Force (see [2]). It organizes the concepts and phases of the revised ADDIE model into five different layers. More precisely: Quality Improvement Layer Management

System Layer Linguistic Layer

Linguistic Design

Linguistic Analysis

Production Layer System Design

System Analysis Delivery

Support

Evaluation Layer Evaluation

System Implementation

System Development

Linguistic Development

Linguistic Implementation

Administration

Fig. 1. The LD-ADDIE Model

− The evaluation layer includes activities centered on the continuous evaluation of the different aspects of the instructional system. It corresponds to the evaluation phase in the original ADDIE model. − The production layer encompasses the systematic sequence of phases oriented to the production of the instructional system. It corresponds to the other four ADDIE phases (i.e., analysis, design, development and implementation). − The linguistic layer contains phases for the systematic production of the domainspecific languages and the associated toolsets. Although these phases mirror the phases in the production layer, their purpose is very different: to develop the languages and tools used by instructors for the development of learning systems. − The system layer contains the main functions of the learning system: management, administration, support and delivery.

Language-Driven, Technology-Enhanced Instructional Systems Design

727

− Finally, the quality improvement layer represents the mechanisms needed to carry out continuous quality improvement. LD-ADDIE adds a new layer, the linguistic layer, to explicitly address the technological factor of technology-enhanced instructional systems. The aim of the phases within this layer is to develop languages and tools. Also, they are mainly carried out by developers: − During linguistic analysis, developers analyze the instructional domain addressed by the learning system and the vocabulary and terminology used by instructors. The goal is to determine the main terms and concepts in this domain, as well as the relationships between these concepts. This analysis can be carried out using standard domain analysis techniques, as understood in software and domain engineering [3]. − During linguistic design, developers specify the syntax and constraints of the domain-specific language, as well as its operational semantics. In modern software language engineering practice, the language usually will be equipped with several syntaxes [4]: an abstract syntax, in terms of which the operational semantics is defined, and one or several concrete syntaxes, oriented to facilitate the use of the language by instructors. All these syntaxes will be linked by suitable transformations. Operational semantics, in their turn, will specify how technology-enhanced components can actually be produced from utterances in the language. During this phase developers also conceive the tools associated with the language. Typical tools will be authoring tools based on suitable concrete syntaxes, as well as generators of the technology-enhanced instructional components. − During linguistic development, developers build the toolset supporting the DSL. For this purpose, they can use well-established traditional techniques in the construction of language processors [1]. They can also adopt one of the emerging tendencies in software language engineering, based on model-driven software development concepts and the use of language workbenches [4]. − Finally, during linguistic implementation, the DSL and the associated toolsets are made available for instructors. These tools will be integrated into the final leaning system as part of the support function.

3 The Language-Driven ADDIE Model in Practice with <e-LD> To illustrate the LD-ADDIE model, we use <e-LD>, an experimental and highly adaptable and extensible authoring tool for IMS LD UoL developed at Complutense University [6,7]. The tool supports three main functions: − Importation. Using this function, instructors can load pre-existing IMS LD UoL. The function also produces useful information to understand the structure and behavior of each imported UoL: a hypertextual view (Fig. 2a), and a dependency graph with the representation of the dependencies among the design elements related to learning activity sequencing [8] (Fig. 2b). − Authoring. Using this function, instructors can load pre-existing IMS LD UoL. This function lets instructors edit the description of a UoL. For this purpose, they use the visual notation detailed in [7] (Fig. 2c).

728

I. Martínez-Ortiz, J.-L. Sierra, and B. Fernández-Manjón (a)

(b)

(c)

Fig. 2. (a) Hypertextual view of a UoL’s method; (b) a dependency graph; (c) edition of a method in <e-LD>

− Exportation. This function makes it possible to generate an IMS LD UoL automatically from an <e-LD> description. The core of the function is an automatic translation of flowcharts into rule-based systems [9]. Since <e-LD> considers IMS LD UoL as essential parts of a learning system, it is possible to systematize the design of evaluation instruments in terms of the structure imposed by <e-LD> on such UoL (for example a satisfaction survey on a UoL can mirror the static structure of the UoL, say a method decomposed into several plays, each one integrating several acts, each one integrating several role-parts, etc.). Also, <e-LD> plays a prominent role in the different production phases: − During system analysis, instructors can find it useful to examine pre-existing UoL used in previous levels of instruction to determine the students’ expected knowledge and capabilities, as well as to better determine the nature of the learning process and the more convenient performance exigencies. − During system design, instructors can reuse pre-existing UoL in the instructional domain, importing them into the tool and modifying them in accordance with the target learning task. Also, instructors can use <e-LD> to author formalized plans of instruction for technology-enhanced components that effectively determines the instructional methods and strategies.

Language-Driven, Technology-Enhanced Instructional Systems Design

729

− During system development, <e-LD> provides a catalog to determine the different instructional resources and materials to be developed. − Finally, during system implementation, instructors use <e-LD> to automatically generate standardized versions of the authored UoL encoded in IMS LD. Regarding the linguistic layer, the development of <e-LD> follows the principles of modern software language engineering [4]. Indeed, the root of <e-LD> is a DSL developed using the language workbench provided by the Eclipse Modeling Project. Thus, <e-LD> can be meaningfully conceived as the main product of an incarnation of the LD-ADDIE linguistic layer: − As regards linguistic analysis, <e-LD> represents a cost-effective solution to the otherwise costly domain analysis processes. Indeed, <e-LD> reuses many of the conceptual structures of a pedagogically neutral language (IMS LD) with the hope of increasing the applicability of the solution while still maintaining a reasonable domain-specific nature. − During linguistic design, the abstract syntax of the <e-LD> modeling language is characterized as a metamodel [4] that captures the main terms and concepts required to describe UoL in <e-LD>, as well as the relationships between these concepts, and the additional constraints affecting these elements. On the other hand, the concrete syntax corresponds to the aforementioned visual notation. These two syntaxes are related by an abstract-to-concrete-syntax mapping. Thus, by changing the concrete syntax model and this mapping, it is possible to tailor <e-LD> to the particular idiosyncratic requirements of each particular community of instructors. Finally, the operational semantics in <e-LD> are actually defined by the translation of flowchart-oriented specifications to rule-based ones used in the exportation function and described in [9]. − Linguistic development takes full advantage of the Eclipse Modeling Project. Indeed, the metamodels of <e-LD>'s abstract and concrete syntaxes are supported by EMF (the Eclipse Modeling Framework). Translation to IMS LD (carried out during exportation) is currently done as an ad-hoc model-to-model transformation; however, we are starting to refactor this process using the model-to-model transformation languages provided by the Eclipse Model to Model project. <e-LD> also takes full benefit of GMF (the Graphical Modeling Framework of Eclipse) to facilitate the development of the <e-LD> authoring function. Finally, the <e-LD> importation function is implemented as an XML processing component. We are currently refactoring it using XLOP (XML Language Oriented Processing) [12], an environment for the processing of XML documents with attribute grammars [11] also developed at Complutense University. − Finally, during linguistic implementation, <e-LD> is deployed for the instructors as an Eclipse-based standalone authoring tool. Currently we are also working on integrating it with other IMS LD compliant platforms and tools, particularly IMS LD players. Finally, following the guidelines encouraged by LD-ADDIE, <e-LD> is an integral part of the learning systems’ support function. In addition, it is also subject to continuous improvement. The adoption of principles strongly rooted in software language engineering in its design and development facilitates this continuous improvement.

730

I. Martínez-Ortiz, J.-L. Sierra, and B. Fernández-Manjón

4 Conclusions and Future Work In this paper we have described an extension of the ADDIE model for instructional systems design that highlights the collaboration between instructors and developers during the development of learning systems with significant technology-enhanced components. For this purpose, the extension promotes the production of domainspecific languages and associated toolsets as support for instructors. The resulting model (LD-ADDIE) makes explicit a linguistic layer oriented to the systematic production of language-oriented assets. We have illustrated the model with <e-LD>, an authoring tool for IMS LD UoL. From a linguistic point of view, the development of <e-LD> takes advantage of the language workbenches provided by the Eclipse Modeling Framework. We are currently applying the same principles to other language-driven e-Learning systems: <e-QTI>, a toolset for the authoring and deployment of QTI assessments [5], and <e-Tutor>, a system for the description of Socratic tutorials [13]. Finally, we plan to further experiment with the adaptation of (the concrete syntax of) <e-LD> to different communities of instructors in several instructional domains.

Acknowledgements We wish to thank the projects TIN2005-08788-C04-01, TIN2007-68125-C02-01, Flexo-TSI-020301-2008-19, Santander/UCM PR34/07 – 15865 and CID-II-0511-A, as well as the UCM Research Group 921340.

References 1. Aho, A.V., Lam, M.S., Sethi, R., Ullman, J.D.: Compilers: principles, techniques and tools, 2nd edn. Addison-Wesley, Reading (2006) 2. Allen, C.W.: Overview and Evolution of the ADDIE Training System. Adv. in Dev. Human Res. 8(4), 430–441 (2006) 3. Czarnecki, K.: Generative Programming: Methods, tools and Applications. AddisonWesley, Reading (2000) 4. Kleppe, A.: Software Language Engineering: Creating Domain-Specific Languages Using Metamodels. Addison-Wesley, Reading (2008) 5. Martínez-Ortiz, I., Moreno-Ger, P., Sierra, J.L., Fernández-Manjón, B.: <e-QTI>: a Reusable Assessment Engine. In: Liu, W., Li, Q., Lau, R. (eds.) ICWL 2006. LNCS, vol. 4181, pp. 134–145. Springer, Heidelberg (2006) 6. Martínez-Ortiz, I., Sierra, J.L., Fernández-Valmayor, A., Fernández-Manjón, B.: Language Engineering Techniques for the Development of E-Learning Applications. J. Network Comp. Appl. 32(5), 1092–1105 (2009), 7. Martínez-Ortiz, I., Sierra, J.L., Fernández-Manjón, B.: Authoring and Reengineering of IMS Learning Design Units of Learning. IEEE Trans. on Learning Tech. (March 27, 2009), http://doi.ieeecomputersociety.org/10.1109/TLT.2009.14 8. Martínez-Ortiz, I., Sierra, J.L., Fernández-Manjón, B.: Enhancing IMS LD Units of Learning Comprehension. In: ICIW 2009 (2009)

Language-Driven, Technology-Enhanced Instructional Systems Design

731

9. Martínez-Ortiz, I., Sierra, J.L., Fernández-Manjón, B.: Translating e-learning FlowOriented Activity Sequencing Descriptions into Rule-based Designs. In: ITNG 2009 (2009) 10. Mernik, M., Heering, J., Sloane, A.M.: When and how to Develop Domain-Specific Languages. ACM Comp. Surv. 37(4), 316–344 (2005) 11. Paakki, J.: Attribute Grammar Paradigms – A High-Level Methodology in Language Implementation. ACM Comp. Surv. 27(2), 196–255 (1995) 12. Sarasa, A., Sierra, J.L., Fernández-Valmayor, A.: XML Language-Oriented Processing with XLOP. In: WAMIS 2009 (2009) 13. Sierra, J.L., Fernández-Valmayor, A., Fernández-Manjón, B.: From Documents to Applications Using Markup Languages. IEEE Software 25(2), 68–76 (2008)

The Influence of Coalition Formation on Idea Selection in Dispersed Teams: A Game Theoretic Approach Rory L.L. Sie, Marlies Bitter-Rijpkema, and Peter B. Sloep Open University of The Netherlands, Centre for Learning Sciences and Technologies, Valkenburgerweg 177, 6419 AT Heerlen, The Netherlands {Rory.Sie,Marlies.Bitter,Peter.Sloep}@ou.nl

Abstract. In an open innovation environment, organizational learning takes place by means of dispersed teams which expand their knowledge through collaborative idea generation. Research is often focused on finding ways to extend the set of ideas, while the main problem in our opinion is not the number of ideas that is generated, but a non-optimal set of ideas accepted during idea selection. When selecting ideas, coalitions form and their composition may influence the resulting set of accepted ideas. We expect that computing coalitional strength during idea selection will help in forming the right teams to have a grand coalition, or having a better allocation of accepted ideas, or neutralising factors that adversely influence the decision making process. Based on a literature survey, this paper proposes the application of the Shapley value and the nucleolus to compute coalitional strength in order to enhance the group decision making process during collaborative idea selection. Keywords: idea selection, game theory, coalition formation, dispersed team, open innovation.

1

Introduction

With the increased use of Internet technology, companies are increasingly trying to reduce transactional costs. R&D costs may similarly be reduced by the adoption of Internet technology, as this fosters the communication in dispersed working teams and across collaborating companies. Indeed, with the adoption of these collaboration tools, we are well on the road to open innovation. The expertise relevant for the design of a new product is not always available within the boundaries of one team or firm. Hence the idea of open innovation suggests to create online distributed teams in which people from different companies and disciplines co-operate on the design of a new product. However, utilising a team’s full innovation potential poses some serious problems. Most research thus far has focused on the extension of the set of ideas, and researchers have tried to neutralise potential pitfalls. There are however indicators that dispersed teams do come up with enough ideas, but just do not select the right ideas. Hence, we U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 732–737, 2009. c Springer-Verlag Berlin Heidelberg 2009 

The Influence of Coalition Formation on Idea Selection

733

should take a closer look at enhancing idea selection, rather than looking at ways to extend the set of ideas during idea generation[1]. Focusing on the idea selection stage of the creative process and the corresponding coalition formation may help find ways to optimise the selection process. Besides, people are often more risk averse when having an idea accepted with a chance of having more ideas accepted and more risk-seeking when preventing to lose an idea that had already been accepted. This in turn may lead to the escalation of commitment by participants. Eventually ideas have to be accepted, and as a result of the escalation of commitment, we search for an optimal allocation of accepted ideas among all participants to satisfy each participant, also known as satisficing: this may lead to the adoption of minimally acceptable solutions. These non-optimal solutions may also be caused by coalition formation during both idea generation and idea selection. We will further explain this in this paper. This paper presents a literature review on the problems dispersed teams currently face during the idea generation and selection process. Furthermore, it stresses the use of coalitional strength during the process of idea selection by presenting a game theoretic approach. In section 2, we will describe economic and psychological factors that influence collaboration, followed by a game theoretic approach meant to overcome problems in idea selection. We will draw our conclusions in section 3 based on the literature review in section 2. The theoretical framework sketched in this paper will be part of a PhD study conducted within the EU FP7 funded idSpace project. Future research to be conducted in this context will be described in section 4.

2

Theoretical Background

When looking at the incentives for collaboration, we see that collaboration is a way for people to learn from each other, or to create new things with the combined knowledge that they have. In corporate environments, teams are created to generate innovative solutions or new products. While historically we see that research and development mainly took place inside the firm, we now see a tendency towards an increase of inter-firm alliances to support so-called open innovation[2]. The reasons for alliances between companies involve sharing risks, obtaining access to new markets and technologies[3], reducing product-tomarket times, and pooling complementary skills[4,5]. Research and development departments of these companies tend to use open innovation to introduce new products faster and at a lower cost. This however requires collaboration and the corresponding notions of trust, reciprocity and negotiation, as co-operation is likely to have competitive aspects as well[6]. When firms collaborate through open innovation, we see that they are hindered by a variety of problems. They may experience individual problems regarding decision making, such as emotional involvement, exogenous factors[7], bounded rationality[8] and escalation of commitment[9]. Besides, the collaboration may be subject to group deficiencies, such as social loafing, group think and group polarisation. The latter two influence the formation of coalitions in open

734

R.L.L. Sie, M. Bitter-Rijpkema, and P.B. Sloep

innovation teams. Especially in idea generation and selection, we see that people need additional support for their ideas to have them accepted. Hence, they form coalitions to stand stronger against other people’s coalitions and ideas. They are self-interested, however, as one may support other people’s ideas in return for their support, also known as reciprocity. As a consequence, coalition formation in idea generation leads to a non-optimal set of accepted ideas. For instance, in a collaborative idea generation session, when person A is above person B in the organisation’s hierarchy, person A may be more informed on the company’s strategy and mission statement. Therefore, person B, who actually has a good idea, will be likely to accept person A’s ideas, as he knows person A is more informed. Though, person A may rather be self-interested, and names one of his own moderate ideas that is not so close to the organisation’s strategy. Thus, person A names an idea with a lower utility, but person B is willing to form a coalition under the presumably rational thought that person A is higher informed and acts accordingly. This example shows that good ideas are often generated in collaborative idea generation, but due to individual and group deficiencies, the selection of ideas is disturbed. In order to overcome the problem of a non-optimal set of accepted ideas, we need to study the influence of coalition formation on the allocation of accepted ideas. To compute this, we need to know what the share of each participant is in the coalition. After doing so, we may propose a division of the coalition’s payoff. A considerable amount of research has been conducted on the division of the coalition’s payoff. In formal game theory, there exist mainly two types of approaches to compute the share of each participant in the coalition and thus the division of the coalition’s payoff: the Shapley value[10] and the nucleolus[11]. Both these concepts are central to games in coalitional form, also known as many-person co-operative games. In such games, players may gain profit from their actions and this profit may be transferred to others as a result of forming coalitions. This transferrable utility is expressed in the form of side payments among players. Side payments are a from of sharing profit from mutually beneficial strategies. For instance, consider three companies that decide to co-operate and share their R&D departments. They find out that it is wiser to shut down one R&D department to reduce the costs. The revenue will then be accountable to the two other R&D departments, whereas the third company made the decision to shut down their R&D department to reduce costs, a decision from which all three companies benefit. Therefore, the company that shut down its R&D department will receive a share of the profit made by the other two company’s R&D departments, the so-called side payment. To compute the side payment, we need to compute the value of a coalition with respect to not forming a coalition. The characteristic function v of the game defines the values of the set of coalitions that may be formed by the players. To compute the values of the set of coalitions, we first need to define what eligible coalitions are. For instance, if we have three players, eight different coalitions may be formed. First, we have the empty coalition denoted by , an empty set with no players. Second we have the one-person coalitions {1}, {2} and {3}. The

The Influence of Coalition Formation on Idea Selection

735

two-person coalitions are{1,2}, {1,3} and {2,3}. The grand coalition in which every player participates is called N. The grand coalition is considered to be the coalition that has the highest payoff, thereby satisfying the common statement that the sum of the whole is more than the sum of any of its parts. The Shapley value focuses on the way participants of an n-person co-operative game view the value of forming a coalition. The so-called players of the game weigh the value of co-operation against the value of not co-operating. The value of the game is computed by taking the value of the coalition and subtracting the value of the sub coalitions, divided by the number of participants in the coalition. For instance, if the coalition {1,2} has value 4, coalitions {1} has value 2 and coalitions {2} has value 1, then the value of coalition {1,2} is 4 2 1 = 1. We denote this as constant c{1,2} = v{1,2} v{1} v{2}. Let’s assume that the following values are given: c{1} = 2 c{2} = 1 c{1,2} = 1 c{1,3} = 3 cN = -2 We can now compute the Shapley value for person 1, which is the sum of the constant values of each coalition person 1 participates in divided by the number of participants in the coalition. With the values given above, we compute player 1’s Shapley value φ 1 = c{1} + c{1,2}/2 + c{1,3}/2 cN/3 = 2 + 1/2 + 3/2 2/3 = 3 + 1/3. If we do this for all three players, we have the Shapley value for the coalition N. The Shapley value may then be used to divide the coalition’s payoff. In our example, player 1 receives a 3 + 1/3 share of the coalition’s payoff of for instance 12. Another way of dividing the coalition’s payoff is the nucleolus. The nucleolus is an extension of the Shapley value, that is, we try to find the characteristic function v and the minimal amount of payoff the players would receive if they co-operate. The payoff vector containing the minimum payoffs is called the imputation, which has the form x = (x1, ..., xn). We then ask the participants how dissatisfied they are with the proposed imputation (that is, the worst division of payoffs) and try to minimise the maximum dissatisfaction. The payoff computed by use of the nucleolus may differ from the Shapley value, as we take into account what the players expect to have. For instance, if a bank goes bankrupt, people would like to claim their savings. Player A has 2000 euros in his savings account, player B has 4000 euros in his savings account and player C has 6000 euros in his savings account. However, the bank has only 7200 euros to divide among the players. Player C is sure of receiving 1200 euros, as players A and B receive a total of 6000 euros. Thus v(C) = 1.2. If we do the same for A and B, we find v(A) = 0 and v(B) = 0. Similarly, v(AB) = 1.2, v(AC) = 3.2, v(BC) = 5.2 and v(ABC) = 7.2. After a series of calculations, the nucleolus v is found to be (1,2.1,4.1), while the Shapley value is (1.2,2.2,3.8). The division of payoff would then be respectively (1200,2100,4100) versus (1200,2200,3800). The Shapley value and the nucleolus will thus lead to different payoff distributions. For player B this makes a difference of 300 euros extra money, while player C will receive 300 euros less. If we compare this to the pro rata distribution of (1200,2400,3600), we see that player C, will actually receive 500 euros extra when the nucleolus is used for payoff distribution.

736

R.L.L. Sie, M. Bitter-Rijpkema, and P.B. Sloep

If we translate the example given above to idea selection, it is not always the case that we have an equal distribution of the set of ideas among participants, based on their individual skills in idea generation. If we compare the outcomes for the coalitions and the individual payoff when not co-operating, we may see different distributions of the payoff. For instance, if we base our imputation on the number of ideas generated during individual idea generation, it may be that forming a coalition pays off. We expect that this is the reason why people choose to form coalitions during idea selection.

3

Conclusions

We think that studying coalition formation in open innovation is a sensible approach, which regrettably has been ignored thus far. We need to pay attention to the way coalitions are formed during collaborative idea selection and to what extent this influences the allocation of accepted ideas among the participants. Based on literature, we see that people often run into a number of problems while co-operating, such as escalation of commitment, bounded rationality, group think and group polarisation, which may lead to the formation of coalitions in such a way that a non-optimal set of ideas are accepted during idea selection. It is shown that the nucleolus and the Shapley value may lead to different distributions than the pro rata distribution of ideas. We expect that if we present the participants with the computations of the nucleolus and Shapley value, they may become better aware of the group’s potential, thus forming coalitions that are better suited to optimise the set of accepted ideas. And if such coalitions are not formed, a moderator may try to put different people together during idea selection to have the right coalitions formed. However, forming coalitions may not always be beneficial for all participants, due to the problems we have sketched in this paper. We may thus choose to try to neutralise the factors that benefit some, but are detrimental to others. For instance, if a group is polarised, we may add people that bridge the gap between the groups that represent the poles to prevent a sub optimal idea from being accepted. If we do so, we may deviate from the original game theoretical notions of the Shapley value and the nucleolus, as we include external (social) factors.

4

Future Research

The above overview suggests many avenues for further research on coalition formation in open innovation. These avenues will be investigated in the context of the EU funded idSpace project, which focuses on tools for distributed, collaborative product innovation. The following steps are envisaged. Based on the literature, we will first define a model that describes the formation of coalitions in idea selection. This will be followed by a social simulation that will help us in analysing the resulting set of accepted ideas. After that, we will try to adapt the model in such a way that we will be able to predict the formation of coalitions. The desired result of our final model will be either the optimisation of the

The Influence of Coalition Formation on Idea Selection

737

formation of ’optimal’ coalitions, that the influencing factors of ’sub-optimal’ coalitions will be neutralised, or that the right people will be chosen in advance of idea generation to eventually have a grand coalition during idea selection. These findings will be empirically tested and underpinned in suitable contexts in which open innovation takes place. We will also look into the possibility of extending our results to contexts in which collaboration takes place which is not necessarily focused on (open) innovation. A case in point would be so-called Learning Networks [12], which are online, social networks designed to foster non-formal learning and knowledge exchange. Acknowledgments. This paper provides a theoretical framework that will be part of a PhD study conducted within the idSpace project. The idSpace project is partially supported/co-funded by the European Union under the Information and Communication Technologies (ICT) theme of the 7th Framework Programme for R&D. This document does not represent the opinion of the European Union, and the European Union is not responsible for any use that might be made of its content.

References 1. Barki, H., Pinsonneault, A.: Small group brainstorming and idea quality: Is electronic brainstorming the most effective approach? Small Group Research 32(2), 158 (2001) 2. Chesbrough, H.W.: The era of open innovation. MIT Sloan. Management Review 44(3), 35–41 (2003) 3. Hagedoorn, J.: Inter-firm R&D partnerships: an overview of major trends and patterns since 1960. Research Policy 31(4), 477–492 (2002) 4. Kogut, B.: The stability of joint ventures: Reciprocity and competitive rivalry. The Journal of Industrial Economics, 183–198 (1989) 5. Powell, W.W., Koput, K.W., Smith-Doerr, L.: Interorganizational collaboration and the locus of innovation. Administrative Science Quarterly 41, 1 (1996) 6. Nash, J.F.: Two-Person cooperative games. Econometrica 21(1), 128–140 (1953) 7. Tetlock, P.E.: The impact of accountability on judgment and choice: Toward a social contingency model. Advances in Experimental Social Psychology 25 (1992) 8. Simon, H.A.: Models of bounded rationality. MIT Press, Cambridge (1982) 9. Shubik, M.: The dollar auction game: a paradox in noncooperative behavior and escalation. Journal of Conflict Resolution 15(1), 109–111 (1971) 10. Shapley, L.S.: A value for n-person games. contribution to the theory of games. Annals of Mathematics Studies 2, 28 (1953) 11. Schmeidler, D.: The nucleolus of a characteristic function game. SIAM Journal on Applied Mathematics, 1163–1170 (1969) 12. Sloep, P.: Fostering sociability in learning networks through Ad-Hoc transient communities. In: Purvis, M., Savarimuthu, B.T.R. (eds.) ICCMSN 2008. LNCS (LNAI), vol. 5322, pp. 62–75. Springer, Heidelberg (2009)

How to Support the Specification of Observation Needs by Instructional Designers: A Learning-Scenario-Centered Approach Boubekeur Zendagui Computer science laboratory of the Maine university IUT of Laval/ Dep. Info 52 rue des docteur Calmette et Guérin. 53020, Laval – France [email protected]

Abstract. In this paper, we present the conceptual model we propose to specify observation needs. Because our work takes place in a learning scenario reengineering context, the observation process is prepared while instructional designers define their learning scenarios. Our work aims at helping these designers to specify the informations they want to get by the observation of the learning situation progress in order to improve the underlying learning scenario for future uses. In this paper we show how the observation needs specification can be guided by informations specified in learning scenarios. We will show how we use the Engeström triangle to model the observation context and how, from the context, some observables will be proposed and used by some observation techniques we propose to use, to define the informations to get by the observation process. Keywords: Instructional design, Observation, observation needs, learning scenario, observation context.

1 Introduction The preparation of a distant learning situation is generally done by the design of a learning scenario that contains informations about the learning activities, usually by using an educational modeling language (EML) [1]. This scenario is qualified as a predictive model of the learning situation. Our goal is to help instructional designers to specify, for a given predictive model, what is important to observe when the actors implied in the learning situation perform their activities. The results of the observation process will be used to improve the predictive scenario for future uses: reengineering of the learning scenario. Into the REDiM project [2], we noted that it is difficult for instructional designers to specify their observation need: they have to guess how will be used the learning environment and have to make some assumptions about the learning situation progress; it is necessary to clarify and formalize the description of observation needs in order to guide the development of tools for collecting and analyzing tracks. The lacks of expressiveness from both learning scenario and EML may also add some difficulties to specify observation needs. U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 738–743, 2009. © Springer-Verlag Berlin Heidelberg 2009

How to Support the Specification of Observation Needs by Instructional Designers

739

We want to help instructional designers in specifying the informations they want to have by the observation process. These informations are specified in a model we call the observation needs. These observation needs are defined in the design of a learning scenario step and used to guide the observation process during the learning situation progress. They help to define and develop the observation means, ie. tools to collect and analyze data about the effective progress of the learning situation. We think that thanks to these observation needs, the observation process will produce more helpful informations for designers to improve their learning scenario. To assist instructional designers in their observation needs specification task, we have studied the observation activity and its preparation within both classic face-toface and distance learning situations. We also worked with a teacher that uses the UMTice1 learning environment to give some courses in addition of the ones given in the classroom. From this theoretical study and work with the teacher, we proposed the conceptual model of observation needs presented in the next section.

2 Conceptual Model of Observation Needs We define an observation need as composed of four parts (see Fig.1).

Fig. 1. The observation needs conceptual model

The observation objectives are useful to define the “why” of observation needs. This information allows designers to explicit what they want to do when they will know the results of their observation needs from the concrete observation of the learning situation runtime. This information can also be useful to facilitate the reuse of observation needs for other situations sharing the same objectives. 2.1 The Context Layer The observation context allows to define conditions under which the activity to observe will be done. It consists in selecting one or more pedagogical scenario elements concerned by the observation need and defined to be used by the learning situation actors. Contexts are important and must be well defined since it allows to identify the potential observables. 1

UMTice is the learning platform of the Maine University, it is based on the Moodle learning platform (www.moodle.org/).

740

B. Zendagui

Several works deal with the modeling of context [4] but there not a unique and single definition for this concept. This depends on the context of its use. We use the definition from [5] who focuses its definition of context on the concept of entity. The context is a set of inter-related entities playing roles. An entity can be an object, person, tool, or anything that may influence the activity to study. To guide the definition of the context entities, we use the Engeström triangle [6] from work on the activity theory, because the study of the media use, when an actor is doing a particular activity, is one among the basic principles of this theory [7]. In our research context, these media have an important roles in the progress of a learning situation.

Fig. 2. The Engeström triangle (at left), the representation of the context of an activity (at right)

We consider that the context of any activity can be represented in a single form by using the Engeström triangle [6] (Fig.2 right side) in which each activity is done by a subject and guided by an object. The result is a production. To perform an activity, the subject uses some tools and can interact within a community by respecting some rules. The community members have to share tasks to to achieve the activity objectives [6]. The entities composing the context of an observation need are the concepts of the Engeström triangle (Fig.2 left side). Each activity is performed by one or more actors, and is guided by a particular object. We represent this object by the production which can be of two kinds: a tangible production (eg. production of a report) or intangible production (eg the acquisition of knowledge). Learning activities are done by using tools which can in turn be of two kinds: services/materials to ensure the good functioning of progress of learning activities, and pedagogical tools/materials needed to guide and structure the learning activities. In the context of an activity, actors play roles, have tasks to perform and have to respect some behaviors and functioning rules. This simple representation allows on one hand to facilitate the construction of context by using a limited list of entities types that have a significant impact on learning activities [8] and, on the other hand, to help the instructional designers to ask the right questions about the learning activities progress and the choice of which informations to consider in observing of a given activity [9]. Our works are not based on a particular EML. Each EML proposes a specific vocabulary and a dedicated semantics for specifying learning situations. To guide the definition of the context, the element of EMLs have to be annotated according to the concepts of the Engeström triangle. This allows to give a simple and common

How to Support the Specification of Observation Needs by Instructional Designers

741

semantic for elements of any educational modeling language and to unify the modeling of an observation need context whatever the used EML. The observation need context definition is guided by the learning scenario and the annotated EML. Each element of the observation need context is an element of the learning scenario. 2.2 The Observables Layer The data collected when learners and tutor use the learning environment are specified thanks to the definition of observables. An observable is defined in [2] as a variable that get a value by the observation of the learning situation progress. Because our works takes place within the design phase of learning scenarios, we define an observable as any learning scenario elements for which designers want to get informations after the end of a learning session. The observables are pedagogical scenario elements for which designers want to get informations after the learning situation execution. Concretely, these observables are defined at a scenario level but are conforming to those defined at the EML level. Their specification is done by selecting observables among those that can be automatically proposed according to the context delimitation and the observables identified in the annotated EML. An observable can be any element of the pedagogical scenario. In the process for the observation need specification we propose in [10], there is a step in which an expert analyzes the EML in order to identify the potential observables. This identification is made by adding annotations on the elements of the EML considered, by this expert, as relevant to observe. The result of this step is the same EML enriched with informations about observables. One element of the EML can be used to define various elements of the learning scenario. Therefor all observables defined on one EML element can be used for all elements of the learning scenario conformed to this EML element. The originality of this approach is that an EML is analyzed once in order to define the observables and used in the observation needs specification of all learning scenarios defined thanks to this EML. In an observation need, we define two kinds of observables: the declared observables and the selected observables. The declared observable set is automatically built by using the annotations added to the EML in the observable identification step of our process [10]. These annotations are used to identify the observables of each learning scenario element and which, thereafter, will be proposed to instructional designers. The declared observables are attached to each learning scenario element added to the context. In our mind, this context/observables representation allows instructional designers to form a vision of the learning situation they want to observe and to provide them with all variables whose values can attest the effective progress of the activities they defined in the learning scenario. According to their observation needs, instructional designers can choose to use or not one declared observable. The selected observables set is then a subset of the declared observables. This set contains only the declared observables chosen by instructional designer according to their observation needs.

742

B. Zendagui

2.3 The Informations Layer From our previous work with instructional designers using the UMTice learning platform, we note that a list of pair , where “observed” is the value of the observable obtained by the observation activity, is sometimes not sufficient to give the informations instructional designers are waiting for to understand the effective learning activities progress. The available data contain a lot of informations. Instructional designer can use all these informations or make efforts to select only the relevant one for them. To address this problem, we propose to provide a set of observation techniques instructional designer can use to specify the informations they want to get by the observation process. Each observation technique is a kind of a function whose result is a simple data like a number or a percentage, or a set of data like all messages posted by learners in a forum. By using an observation technique, one knows the nature of it result. Choosing an observation technique between various ones depend on it result nature and the observation needs of instructional designers. We aim, in this layer, to provide instructional designers with a set of observation techniques that allow them to define all informations they want to get and this by using the selected observables. To this end, we propose to use the sign and the category of behavior techniques. These two observation techniques are used in the classical face-to-face learning situation observation [3]. A sign is a particular behavior to observe. For example, a learner begins an activity later then what was planned in the learning scenario. The signs based observation can be used to focus the observation on some specific behaviors. The use of behavior categories to observe the learning situation progress regroups several behaviors in homogeneous sets and analyzes them as a whole to better understand the behavior of the learning situation actors. For example, by defining a category of behavior in which there are the messages exchanged between students within a forum can enable the detection of active and passive learners or learners in difficulty. The analysis of each message alone can be relevant, but the analysis of all messages in a chronological order could provide more informations to instructional designers. In our mind, sign and category of behavior techniques are two examples of the use of observation techniques. Other observation techniques can be used in this layer.

3 Conclusion In this paper we presented the conceptual model of observation need we propose. In our research context, observation needs are defined during the elaboration phase of learning scenarios by instructional designers. Our goal is to guide the specification of observation needs by using the information defined in the learning scenarios. To this aim, we use the Engeström triangle to structure and guide the definition of the context of observation needs. The context definition allows to provide a vision of the learning situation instructional designer want to observe and it allows to propose some observables that can attest the effective progress of the learning situation. Instructional designers can select some observables to define the informations they want to get by the observation process. The originality of our approach relies on the definition of observation needs in relation with informations about learning situations defined in learning scenarios and

How to Support the Specification of Observation Needs by Instructional Designers

743

annotations made on the used EML. Because EMLs can be used to define many learning scenarios, annotations must be defined in a generic way to be used for all learning scenario. We are working currently on defining techniques allowing to contextualize annotations to each learning scenario used to define observation needs.

References 1. Koper, R., Tattersall, C.: Learning Design – a handbook on Modeling and Delivering Networked Education and Training. Springer, Heidelberg (2005) 2. Choquet, C.: Engineering and re-engineering of TEL systems, the REDiM approach. Professor’s degree thesis. Le Maine University, France (2007) (in French) 3. Wragg, E.C.: Introduction to classroom observation, 2nd edn. Routeledge (1999) 4. Strang, T., Linnhoff-Popien, C.: A Context Modeling Survey. Workshop on Advanced Context Modeling. In: Reasoning and Management as part of UbiComp, Nottingham, England (2004) 5. Rey, G.: Méthode pour la modélisation du contexte d’interaction. RSTI – ISI – 11/2006. Adaptation en contexte, 141–166 (2006) 6. Engeström, Y.: Learning by expanding: an activity-theoretical approach to developmental Research. Orienta-Konsultit Oy, Helsinki (1987) 7. Kaptelinin, V., Nardi, B.A.: Activity Theory: Basic Concepts and Application. In: CHI 1997, Los Angeles (1997) 8. Kurti, A., Spikol, D., Milrad, M., Svensson, M., Pettersson, O.: Exploring How Pervasive Computing Can Support Situated Learning. In: Proceedings of the Workshop of the Pervasive Learning 2007: Design Challenges and Requirements, Toronto, Ontario, Canada (2007) 9. Kaenampornpan, M., O’Neill, E.: Modeling context: an Activity Theory approach. In: 2nd European Symposium on Ambient Intelligence, EUSAI, Eindhoven, The Netherlands (2004) 10. Laforcade, P., Zendagui, B., Barré, V.: Specification of observation needs in an instructional design context: A Model-Driven Engineering approach. In: CSEDU 2009, Lisbonne, Portugal, March 23-26 (2009)

Using Third Party Services to Adapt Learning Material: A Case Study with Google Forms Luis de la Fuente Valent´ın , Abelardo Pardo, and Carlos Delgado Kloos Telematics Engineering Department, University Carlos III of Madrid, Av. Universidad 30, Legan´es, Spain {lfuente,abel,cdk}@it.uc3m.es http://gradient.it.uc3m.es

Abstract. Current Learning Management Systems were typically conceived to offer a self-contained “one size fits all” learning environment. Adaptive educational systems have been exhaustively studied and proposed to satisfy the different needs of students, but they have a poor presence in the LMS market due to integration issues. The emerging trend in the web is toward combining very specialised services into highly personalised environments, and LMS are no exception. This paper presents the Generic Service Integration architecture conceived to embed the use of any third party service as a regular resource in a learning experience. A course author includes a description with the required functionality and the appropriate service is searched and instantiated at enactment time. A case study is presented where Google Forms are used to implement assessment in a IMS Learning Design based course, and adapt its content based on the obtained results. Keywords: IMS Learning Design, adaptive educational systems, service integration.

1

Introduction

Learning Management Systems (LMS) in educational institutions have reached a stage of widespread adoption. The variety of commercial and open-source products conform a wide spectrum of possibilities to manage learning experiences. Most of current LMS can be described as a “one size fits all” service, where as much functionality as possible is provided. However, the trend emerging on the web points to open LMS that allow integration with third party services. One factor that pushes this integrating trend is the innumerable amount of services conforming what is called the Web 2.0. Most of participants of a learning experience are likely to use 2.0 services, but still they are forced to use the counterparts offered by the LMS. Some LMS offer email, bookmark collections, picture albums, personal web pages, etc. This tendency suggests educational systems to act 

Corresponding author.

U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 744–750, 2009. c Springer-Verlag Berlin Heidelberg 2009 

Using Third Party Services to Adapt Learning Material

745

as service orchestrators where the functionality is given by third party providers and the pedagogical structure of the course is promoted at the LMS. As analysed in [1] and [2] tools for adaptive learning support, where the objective is to individually satisfy the different needs of multiple users by means of taking into account different student profiles, have a poor presence in the LMS market mainly due to their lack of integration capabilities. As a consequence, the integration of generic services in a educational platform would promote the inclusion of adaptive schemes in learning experiences. Specifications such as IMS Learning Design [3] (henceforth simply IMS LD) conceived to capture the structure and interaction in a learning experiences include a limited but specific formalism to define the interaction with services. However, as pointed out by Martel et al., the activities mediated by services cannot be observed [4]. Thus, the tight and effective integration of a generic service with a simple formalism has been proved extremely complex. The objective of this paper is to answer the following questions: Is it possible to adapt a learning experience based on information originated in a third party service? Can this service be instantiated differently depending on the conditions of the learning scenario? To answer these questions, this paper presents a case study where a Unit of Learning described in IMS LD adapts the activities during the enactment phase through a set of values obtained from the interaction with Google Forms. The integration between IMS LD and this third party service is supported by the Generic Service Integration (GSI) architecture [5]. This paradigm proposes a set of minimum requirements for a service to be integrated in a learning experience. The results of a first pilot experience are also included to show how this new functionality affects a Unit of Learning over its entire life cycle. The rest of this document is organised as follows. Section 2 summarizes related work and proposes the GSI architecture. Section 3 details the use case in the context of the different stages of a learning experience. The paper concludes presenting the results obtained in a pilot experience and a brief discussion of future avenues to explore.

2

Adaptation Using Third Party Services

The term “adaptation” refers to purposely changing one or several aspects of a learning environment to cater to the needs of a student. The effectiveness of adaptation has been a matter of great discussion (see [6] for a thorough examination of some of them or [7] for a discussion in the context of engineering education). As pointed out in [1], tools that facilitate adaptive learning tend to be extremely specialised in an aspect of the learning process at the expense of the integration in a learning management system. In recent years, there has been an effort to provide adaptive tools that are integrated in conventional LMS. The second area relevant to the ideas presented in this paper is the use of third party services. A learning experience may require (or take advantage of) the orchestration of a set of external services. An example in this direction is the

746

L. de la Fuente Valent´ın, A. Pardo, and C. Delgado Kloos

concept of a Personal Learning Environment (PLE) [8]. In [5], a new paradigm is outlined to allow IMS Learning Design engines and services to exchange information and react to each other events. In order to offer versatile service integration while maintaining the usability of a platform, a pragmatical approach is taken. The proposed solution is based on a set of minimum requirements to be fulfilled by both an LMS (the IMS LD engine) and a service. Learning Design [3] is a specification that supports the formal description of activity-centered learning. IMS LD allows multiple pedagogical approaches to be modelled as a Unit of Learning (UoL) [9]. A UoL contains the description of all the activities, instructions on how the participants should interact, and a set of properties and conditions to be deployed in a virtual learning context. Learning Design offers an adequate formalism to achieve adaptation of educational experiences. The case study presented in this paper uses the Generic Service Integration architecture implemented in GRAIL [10], a Learning Design engine integrated in the open source Learning Management System .LRN. 2.1

Generic Service Integration Architecture

The proposed Generic Service Integration architecture provides the semantics to include third party services in IMS LD courses, but can be easily extended to work with any course authoring/delivery framework that supports basic group management, saves the state of the course and reacts depending on this state. In GSI, the integration of a third party service in a learning course has two requirements: the definition of the service usage in the context of the course, and a runtime environment capable of enacting the service functionality. Thus, the proposal consists of two areas related to the course life-cycle: a semantic description to capture service behavior, and an execution model to use the service. The proposed vocabulary has been generically defined. It is assumed that each supported service needs to provide specific meaning for the used verbs. With this approach, the definition of the expected behavior can be written specifically for a concrete service (for example, a blog in Wordpress) but leaving room during enactment to use an alternative that complies with the given requirements. – Groups element: During the authoring phase the course participants are not known, but a description of the grouping policy can be given. Groups are directly mapped into IMS LD roles. – Tool element: Describes the functionality required in the service, expressed in an abstract notation (set-values, open, close, modify-permissions, etc.). Permissions are also very defined at generic level. Tool information also include metadata to easily delimit the type of services suitable to be used and facilitate the searching procedure during run time. – Constraints element: Defines the requirements on the service behavior. While the Tool element defines the required operations, this element contains the detailed description of how and when these actions must be triggered. This description of the service functionality and usage is packaged within the UoL, and will be interpreted when the course is deployed in a compliant

Using Third Party Services to Adapt Learning Material

747

learning management system. Service configuration is then performed once the course participants have been assigned to groups, and before the course is fully available for them. It follows a description of the actions that must take place during course (and therefore, service) deployment. – Service search: Based on the service description included in the UoL, the runtime engine must select the one that best matches the given requirements. This selection can be fully automatic or may require manual intervention. – Service configuration: Once the service is selected, a binding stage is required to connect the community of users in the LMS with the corresponding community of users in the service. This stage depends on, the access requirements imposed by the service. Technologies such as OAuth [11] are conceived precisely to simplify the information exchange in this type of scenarios. – Enactment: Actions during this stage include facilitating service access to the course participants, managing the data exchange between the service and the LMS, and invoking the proper operations in the service.

3

Embedding Third Party Assessment in a Course

Assessment is accepted to be a weakness in the current IMS LD specification, and the problem has been faced in different ways [12] [13]. The approach taken in this case study is the use of the third party service Google Forms to provide assessment facilities. Google Forms allow to easily create web forms whose submissions are stored in a Spreadsheet. Data can be accessed through a public Application Programming Interface, and authentication issues are mediated by the SubAuth protocol, which follows the same principles as OAuth [11]. The Google Forms service has been included via GSI. The inclusion of a service has implications in the whole course life cycle. A simple course has been chosen to illustrate the process in all its stages: authoring, deployment and enactment. All interactions with the third party service are depicted in figure 1. The course used for the study, described in IMS LD terminology, consists of two act: the first is devoted to a profiling questionnaire while the second is composed by a set of suggested readings that are based on the results obtained in the previous test. During these activities, the members of the teaching staff are in charge of tracking the activity and monitor student results. The profiling questionnaire is integrated in the authoring phase by the inclusion of groups, tool and constraints elements described in section 2. If a GSI service is found during course instantiation (as in this case), the third party service must be allocated and configured. The GSI architecture serves as the launcher of software units, called plugins. The engine then selects the service that best matches with requirements, and configures the proper plugin. From this moment onward, any service request will have the selected plugin as mediator. Consequently, output from the service can be parsed and formatted to be adjusted to property data types. GSI service configuration requires a pre-enactment phase in which course participants enter the course without all deployment steps being finished. At

748

L. de la Fuente Valent´ın, A. Pardo, and C. Delgado Kloos

Fig. 1. Interaction among actors during the deployment and enactment phases

this point, participants must grant permissions to access their personal data in the third party service1 . In the presented example, only teachers are requested to do so. Some extra adjustments may take place in this phase: the form’s target where data is submitted can be only obtained after service deployment, and current limitations of the API imposes manual intervention to fill the proper value. When the enactment phase starts, interactions with the third party service come about. First, students insert their responses in the Spreadsheet by using a form. As the plugin acts as a mediator, an identification token is attached to each response. Second, teachers can access at any time to the actual Spreadsheet. Last, the course engine retrieves all the gathered data (which is obtained as an ATOM feed) and make next activity behave depending on the student responses.

4

Results and Conclusions

The case study presented in this work was included as part of a regular postgraduate programme in a higher educational institution. A total of 19 students took part in the experience. It is relevant to remark the successful deployment of the experience, with the robust definition of service life cycle phases as the key factor. Further, IMS LD provided a framework where adaptation was possible. 1

SubAuth tokens can be revoked at any moment by the account owner.

Using Third Party Services to Adapt Learning Material

749

The usage of third party services is potentially much more versatile than the use presented in this article. It would be possible, for example, to use spreadsheet to calculate a formula that takes student results as parameters. This calculated value could be used in the adaptation strategy. The difficulty of IMS LD in the field of data manipulation can be avoided by using a more specialised tool. The results of the case study show the potential of the deployment infrastructure: services from different vendors can be coordinated by IMS LD courses in order to provide adaptation not restricted to facilities build within the LMS. Derived from the experience under study, further developments are suggested to improve the GSI model: it is straightforward that a larger set of plugins needs to be developed. The use of the model is restricted to supported services, so it is desirable to feature a wider set of choices. A rise in the available plugins number will bring up the matter of service search facilities. Keywords, in which the system is based currently, may not be good enough for a larger set of available plugins. Further researches on better search techniques, such as semantic based search, must be accomplished to improve usability of the architecture.

Acknowledgement This work has been partially funded by the Project Learn3 (TIN2008-05163/TSI) from the Plan Nacional I+D+I and the Spanish National Project FLEXO (TSI020301-2008-19, www.ines.org.es/flexo).

References 1. Brusilovsky, P.: Knowledgetree: a distributed architecture for adaptive e-learning. In: Proceedings of the 13th international World Wide Web conference on Alternate track papers & posters, pp. 104–113. ACM, New York (2004) 2. Meccawy, M., Blanchfield, P., Ashman, H., Brailsford, T., Moore, A.: WHURLE 2.0: Adaptive Learning Meets Web 2.0. In: Dillenbourg, P., Specht, M. (eds.) ECTEL 2008. LNCS, vol. 5192, pp. 274–279. Springer, Heidelberg (2008) 3. IMS Learning Design specification (February 2003), http://www.imsglobal.org/learningdesign/ (last visited April 2009) 4. Martel, C., Vignollet, L.: Using the learning design language to model activities supported by services. International Journal of Learning Technology 3(4), 368–387 (2008) 5. de la Fuente Valent´ın, L., Miao, Y., Pardo, A., Delgado Kloos, C.: A supporting architecture for generic service integration in IMS learning design. In: Dillenbourg, P., Specht, M. (eds.) EC-TEL 2008. LNCS, vol. 5192, pp. 467–473. Springer, Heidelberg (2008) 6. Coffield, F., Moseley, D., Hall, E., Ecclestone, K.: Should we be using Learning Styles? What research has to say to practice. Learning and Skills Development Agency (2004) 7. Felder, R.M., Brent, R.: Understanding student differences. Journal of Engineering Education 94(1), 57–72 (2005) 8. Wilson, S., Liber, O., Johnson, M., Beauvoir, P., Sharples, P., Milligan, C.: Personal learning environments: Challenging the dominant design of educational systems. In: Memmel, M., Burgos, D. (eds.) Proceedings of LOKMOL 2006 in conjunction EC-TEL 2006, Crete, Greece, October 2006, pp. 67–76 (2006)

750

L. de la Fuente Valent´ın, A. Pardo, and C. Delgado Kloos

9. Koper, R., Tattersall, C. (eds.): Learning Design. A handbook on Modelling and Delivering Networked Education and Training. Springer, Heidelberg (2005) 10. de la Fuente Valent´ın, L., Pardo, A., Delgado Kloos, C.: Experiences with GRAIL: Learning design support in .LRN. In: TENCompetence Workshop on Current Research in IMS Learning Design and Lifelong Competence Development Infrastructures (2007) 11. OAuth core 1.0, http://oauth.net/core/1.0/ (last visited April 2009) 12. Miao, Y., Sloep, P., Koper, R.: Modeling units of assessment for sharing assessment process information: Towards an assessment process specification. In: Li, F., Zhao, J., Shih, T.K., Lau, R., Li, Q., McLeod, D. (eds.) ICWL 2008. LNCS, vol. 5145, pp. 132–144. Springer, Heidelberg (2008) 13. Dalziel, J.: Implementing learning design: the learning active management system (LAMS). In: Proceedings of the 20th Annual Conference of the Australasian Society fon Computers in Learning, pp. 593–596 (2003)

Virtual Worlds for Organization Learning and Communities of Practice C. Candace Chou School of Education University of St. Thomas 1000 LaSalle Ave., MOH 217, Minneapolis, MN 55403, USA [email protected] Abstract. An increasing number of organizations have established presences in Second Life or virtual worlds for organizational learning. The types of activities range from staff training, annual meetings, to leadership development and commercial transactions. This paper reviews relevant literature on how virtual worlds, especially Second Life, are utilized for organizational learning. Specific emphases will be on the translation of applicable learning theories into the pedagogical design of virtual worlds. Furthermore, the paper explores how organizations establish virtual communities of practice. Finally, examples of virtual worlds that are established for organization learning are examined. Keywords: virtual worlds, virtual communities of practice, organization learning.

1 Introduction Virtual worlds, which refer to a 3D virtual learning environment that supports multiple learners, have been employed by an increasing number of corporations, universities, and education agencies for learning and training [1]. Virtual worlds have a low barrier-to-entry for content creation, can be programmable, and provide an abundant reusable instructional content [2]. In the last few years, a rapidly growing number of business and higher education institutions have established presences in Second Life and other similar virtual worlds. People enter the virtual worlds in Second Life via an avatar to represent themselves. The avatars can walk, talk, and move around the same way that they would move in real world. Most of the current discussions have focused on the pedagogical applications of virtual worlds for learners in higher education. Although some of the theoretical principles can be applied to learners in both education and business, domain-specific examples based on the shared theoretical principles can provide practitioners in organizations a better framework in adopting virtual worlds for training and development. This paper will focus on theoretical frameworks for organization learning in virtual worlds and examples of workplace learning in virtual worlds, especially in Second Life.

2 Literature Review This section will start with a general discussion on the affordances of virtual worlds and the capabilities of virtual world to support learning. Next, the discussion will examine the theoretical principles that provide the guidelines for learning in virtual worlds. U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 751–756, 2009. © Springer-Verlag Berlin Heidelberg 2009

752

C.C. Chou

2.1 Affordances of Virtual Worlds As technology evolves, new technological capabilities can infuse innovative approaches into teaching and learning activities in education and the workplace. In Second Life’s virtual world, learners can utilize many of its features to form learning networks, create new identities, and construct new worlds with flexible building tools. In table 1, Jarmon [3] summarized how these new technological features can afford users to transform their experiences in the 3D virtual world. Table 1. Affordances / Extended Capabilities in 3-D Virtual World of Second Life [3]

Affordance Communication/community Embodied social presence

Extended Capability Voice, chat, SL groups, search 3-D perspective on avatars (oneself & others) Building/engineering/design/sculpting Highly flexible robust tools & training Animation and scripting Motion, behaviors, sensors, lighting, sound Data visualizations & simulations Modeling, infinite scale, micro/macro, role-play, spreadsheet conversion, historical, art Sound & spatial relationships Example: reflexive architecture, avatar orchestra Language immersion Example: 27 language-specific islands Learning communities created by & for Example: Educators Coop Residential users Island International SL collapses geography Low capital expense operations costs Overhead, travel, equipment, training, energy Fundraising Am. Cancer Soc., Katrina Relief, kiva.org Recruitment/administration/management Universities, IBM > 1500 employees in SL Bringing distance & online learning Online-course class photo only in SL together in the 3-D virtual world The integrated functions in virtual worlds have presented new opportunities for learning. We are seeing a convergence of social networking, 3D, multimedia, voice, chat, videos, search in Second Life. The extended capabilities are especially appealing to learners who hope to be able to have more control of their online presence in a highly engaged and connected environment. 2.2 Adult Learning Theory Designing learning opportunities for organizations in virtual worlds requires one to have a good understanding of how adults learn. Malcolm Knowles was one of the first

Virtual Worlds for Organization Learning and Communities of Practice

753

educators to establish principles for adult learning. Knowles [4] identified five characteristics of adult learners. Zielke, Roome, & Krueger [5] matched the characteristics of adult learning with the features of virtual worlds as summarized below: • • • • •

Adults are autonomous and self-directed. Virtual worlds enable independent learning. Adults have accumulated a foundation of life experiences and knowledge. Virtual worlds encourage sharing of life experience with others. Adults are goal-oriented. Virtual worlds allow goal-setting and increase skill levels for use in work or hobby. Adults are relevancy-oriented. Virtual worlds provide the opportunity to check own progress and re-learning if needed. Adults are practical and value knowledge that are useful to work. Virtual worlds offer opportunities for problem-solving and immediate application of materials to be learned.

How can the virtual worlds maximize learning within the framework of adult learning? Zielke, Roome, and Krueger [5] presented a case study on how virtual world can also assist people with disabilities to experience physical activities through their avatars. Activities such as dancing, walking, and running, not possible in real-life, are possible in virtual worlds. The new found capabilities can strongly motivate learners to engage in learning. 2.3 Communities of Practice The concept of communities of practice (CoP) has been identified by many as a means to effective knowledge management in organization learning [6]. The concept has existed in various parts of the world for centuries. However, it did not become an established theory in organization learning until Lave and Wenger [7] theorized it in their seminal: Situated Learning: Legitimate Peripheral Participation. Wenger [8] defined a community of practice as “groups of people who share a concern or a passion for something that they do and learn how to do it better as they interact regularly” along three dimensions: • • •

What it is about – its joint enterprise as understood and continually renegotiated by its members How it functions: - mutual engagement that bind members together into a social entity What capability it has produced – the shared repertoire of communal resources (routines, sensibilities, artifacts, vocabulary, styles, etc.) that members have developed over time.

Can communities of practice be established online and/or in virtual worlds? Research has shown that virtual communities of practice are emerging [9, 10]. Virtual CoP has been a more standard term to describe a network of individuals “who share a domain of interest about which they communicate online” [11]. There is a difference between a virtual learning community and virtual CoP. The former aims at enhancing the knowledge of the participating members via formal education or professional development. The latter enhances the knowledge of community members via informal

754

C.C. Chou

learning. Novice members tend to move from peripheral to center through observation of experts and apprenticeship with experienced members. The literature review section has summed up the most recent development in theoretical frameworks relevant to Virtual Worlds. The Adult Learning principles present the pedagogical principles in designing learning opportunities in virtual worlds. The essence of CoP lends itself well for organizations to use virtual worlds for both formal and informal learning.

3 Case Examples University campuses and business have established locations in virtual worlds. Cross and O’Driscoll [12] observed that corporations are using virtual worlds for the following purposes: • • •

A new level of always-on, real time connectivity for collaboration Empowering both customer and employee groups Making informal viral learning a core mechanism of transformation

Werner [13] suggested that virtual worlds have become an appealing venue for training and development for the following reasons: (1) engagement, (2) low cost relative to real life, and (3) quick and easy to change. Virtual worlds are engaging because learners can immerse themselves in a 3D environment that has a high-fidelity to the real environment and move freely in-world with an identity of their choices. Virtual Worlds have been commonly used for the following types of workplace learning: (1) 3D demonstration, (2) simulation, and (3) virtual meetings. The following sections introduces workplace-related examples. 3.1 Workplace-Related Examples 3.1.1 3D Demonstration Palomar West Hospital in Second Life is a prototype of the hospital that is under construction and due to open in 2011. It was designed to provide a preview of the new facility to hospital staff, future patients, media, and the larger medical community [14]. The site can be accessed through the SLRL: http://slurl.com/secondlife/PalomarWest%20Hospital/33/127/34/ 3.1.2 Simulation Role plays through simulation is a common form of organization learning in virtual worlds. General Electric (GE) has utilized Second Life to provide performance-based training. A role-playing strategy game was employed to elicit a time-critical strategic behavior in response to a forced outage situation [15]. 3.1.3 Virtual Meetings IBM was one of the pioneers in utilizing virtual worlds for organization learning and training. IBM has 50 islands and more than 20,000 employees in virtual worlds. In 2008, IBM held an annual meeting for the 200+ members of the Academy of Technology. The conference venue consisted of breakout rooms, a simulated Green Data

Virtual Worlds for Organization Learning and Communities of Practice

755

Center, a library, and areas for community gathering. IBM estimated that the return of investment (ROI) for the Virtual World Conference was roughly $320,000 [16]. 3.2 Communities of Practice The above-mentioned examples presented more concrete and observable cases of organization learning. However, communities of practice for the purposes of organization learning in virtual worlds are limited. Research on business-related virtual CoP in virtual worlds is still a relatively new area. As more organizations establish presences in virtual worlds, more research data will provide a better understanding of the processes and outcomes. Here is a small sample of professional organizations that serve as the venues for virtual CoPs. • • • •

American Society for Training and Development (ASTD): http://slurl.com/secondlife/ASTD%20Island/113/84/23) International Society for Technology in Education (ISTE) Islands: http://slurl.com/secondlife/ISTE%20Island/93/83/30 New Media Consortium (NMC) Campus: A large consortium of universities, organizations, and museums that supports events, classes, demonstration, and art exhibits. http://slurl.com/secondlife/NMC%20Campus/136/91/23) Gronstedt Group: Weekly “Train for Success” sessions bring training and communication professionals globally to explore the new development in leading corporations, http://slurl.com/secondlife/Wolpertinger/161/82/51

4 Conclusion and Future Trends In this paper, the applications of adult learning theory and communities of practice to organization learning in virtual worlds were reviewed. Examples of workplacerelated learning were introduced. Although virtual worlds have been in existence for decades [17], it was not until the introduction of Second Life to the public in 2003 that establishing communities in virtual worlds became a norm in the academic and the business world. It is not clear how organizations can be more effectively exploring the opportunities offered by virtual worlds for organization learning. More studies on how communities of practice in virtual worlds can contribute to knowledge construction, collaboration, and motivation are needed. What will the future hold for organization learning through virtual CoP in virtual worlds? In addition to the affordances of technology and the usability of virtual worlds, it is also important to cultivate a sense of belonging to encourage information sharing, collaboration, and interaction. The development of virtual worlds will be as exciting as the World Wide Web in the 90s.

References 1. The New Media Consortium. The Horizon Report. 2007 Edition (2007), http://www.nmc.org/horizon/2007/report (retrieved April 18, 2009) 2. Mason, H.: Experiential Education in Second Life. In: Livingstone, D., Kemp, J. (eds.) Proceedings of the Second Life Education Workshop, pp. 14–18 (2007), http://www.simteach.com/slccedu07proceedings.pdf (retrieved April 15, 2009)

756

C.C. Chou

3. Jarmon, L.: Learning in Virtual World Environments: Social Presence, Engagement, & Pedagogy. In: Encyclopedia of Distance and Online Learning. IGI Global (2008) 4. Knowles, M.S.: Andragogy in Action. Applying modern principles of adult education. Jossey Bass, San Francisco (1984) 5. Zielke, M.A., Roome, T.C., Krueger, A.B.: A Composite Adult Learning Model for Virtual World Residents with Disabilities: A Case Study of the Virtual Ability Second Life® Island [Electronic Version]. Journal of Virtual Worlds Research 2 (2009), http://jvwresearch.org/ (retrieved April 17, 2009) 6. Kimble, C., Hildreth, P.: Communities of Practice: Going One Step Too Far? [Electronic Version] (2005), http://ideas.repec.org/p/wpa/wuwpio/0504008.html (retrieved April 15, 2009 ) 7. Lave, J., Wenger, E.: Situated learning: Legitimate peripheral participation. Cambridge University Press, Cambridge (1991) 8. Wenger, E.: Communities of Practice: Learning as a social system [Electronic Version] (1998), http://www.co-i-l.com/coil/knowledge-garden/cop/lss.shtml (retrieved April 16, 2009) 9. Dubé, L., Bourhis, A., Jacob, R.: Towards a Typology of Virtual Communities of Practice [Electronic Version]. Interdisciplinary Journal of Information, Knowledge, and Management 1, 69–93 (2006), http://www.ijikm.org/Volume1/IJIKMv1p069-093Dube.pdf (retrieved April 19, 2009) 10. Kondratova, I.L., Goldfarb, I.: Virtual communities: design for collaboration and knowledge creation. In: Proceedings of the European Conference on Products and Processes Modeling, ECPPM 2004 (2004), http://iit-iti.nrc-cnrc.gc.ca/ iit-publications-iti/docs/NRC-47157.pdf (retrieved April 15, 2009) 11. Gannon-Leary, P., Fontainha, E.: Communities of Practice and virtual learning communities: benefits, barriers and success factors (2007), http://www.elearningeuropa.info/files/media/media13563.pdf (retrieved April 16, 2009) 12. Cross, J., O’Driscoll, T., Trondsen, E.: Another Life: Virtual Worlds as Tools for Learning [Electronic Version]. eLearn Magazine (2008), http://www.elearnmag.org/ subpage.cfm?article=44-1§ion=articles 13. Werner, T.: Using Second Life for workplace learning (March 25, 2009), http:// www.slideshare.net/twerner/ using-second-life-for-workplace-learning032509?type=powerpoint (retrieved April 10, 2009) 14. Hanna, A.: Palomar Medical Center West (2008), http://www.collaborationproject.org/display/case/ Palomar+Medical+Center+West (retrieved April 10, 2009) 15. Werner, T.: Best use of virtual worlds for learning (January 30, 2009), http://www.brandon-hall.com/awards/awards/?p=380 (retrieved April 20, 2009) 16. Linden Lab. How meeting in Second Life transformed IBM’s technology elite into virtual world believers (2009), http://secondlifegrid.net.s3.amazonaws.com/ docs/Second_Life_Case_IBM.pdf (retrieved April 20, 2009) 17. Damer, B.: Meeting in the Ether: A brief history of virtual worlds as a medium for usercreated events [Electronic Version]. Journal of Virtual Worlds Research 1 (2008), http://www.jvwresearch.org/v1n1.html (retrieved April 1, 2009)

A Methodology and Framework for the Semi-automatic Assembly of Learning Objects Katrien Verbert1, David Wiley2, and Erik Duval1 1

Dept. Computerwetenschappen, Katholieke Universiteit Leuven, Celestijnenlaan 200A, B-3001 Leuven, Belgium {Katrien.Verbert,Erik.Duval}@cs.kuleuven.be 2 Instructional Design and Technology Department, Brigham Young University, Provo, UT, USA [email protected]

Abstract. One of the major obstacles in developing high quality content for learning is the substantial development cost and effort. In addition, the return on investment is often low, as developed learning materials are difficult to reuse and adapt to new and different educational contexts. In this paper, we present a semi-automatic content assembly methodology to automate, at least partially, the reuse of existing learning content in high quality and effective learning sequences. In addition, we present a case study that integrates the approach into the LAMS learning design environment. Keywords: learning object reuse, learning object metadata, learning design.

1 Introduction Many existing course documents merge the representation of content and the instructional approach [1]. Such hardwired pedagogy restricts the options for teaching and learning, both in terms of reusability and adaptation of learning sequences. Typically, teachers create their teaching strategies and content from scratch or reuse parts of existing course documents by ad-hoc and time-consuming copy-and-paste actions. In addition, adaptation to individual learning or teaching styles, background, experiences, interests or preferences is generally not possible, unless learning content is specifically designed for personalization purposes [2]. In this paper, we present a semi-automatic content assembly methodology for the generation of learning sequences tailored to different pedagogical approaches, based on the explicit design of these sequences by a teacher. The assembly framework employs a teacher model, an instructional model and a domain model to enable the focused retrieval and aggregation of learning resources into learning sequences. Learning resources are retrieved through the GLOBE network of educational repositories [http://globe-info.org/] and from various community driven websites, such as WikiAnswers.com, ProProfs.com and Wikipedia. The assembly framework is described in the next Section. We present a case study that integrates the approach into the LAMS [3] learning design environment in Section 3. Related work is discussed in Section 4, followed by conclusions and remarks on future work. U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 757–762, 2009. © Springer-Verlag Berlin Heidelberg 2009

758

K. Verbert, D. Wiley, and E. Duval

2 Content Assembly Framework The content assembly framework supports the selection and assembly of existing learning resources. The framework employs the following models to enable the focused retrieval and aggregation of resources: -

-

The instructional model captures the semantics of the pedagogical strategy employed by a learning sequence and is based on [4]. Narrative structures within this model outline the flow of concepts of a particular learning design strategy and are used as templates when assembling learning sequences. An example is an inquiry based learning strategy that sequences activities like "answer questions", "vote on a list", "discuss responses", "read expert view", "discuss expert view" and "personal reflection". The domain model represents the knowledge domain of a course. It includes concepts outlined in the objectives of a course and their interrelationships. The teacher model defines teacher attributes to enable the personalized aggregation of learning resources [5]. The model includes attributes for representing the level of expertise of the teacher, interests and activities, teaching strategy preferences, background, and presentation styles (Fig. 1).

Fig. 1. Semi-automatic content assembly framework

The assembly engine maps instructional, domain and teacher concepts to PLQL queries and federates the queries to SQI-enabled repositories. The approach is exemplified in [6]. PLQL [7] is primarily a query interchange format for repositories. SQI [8] is a query transport standard that is widely used within the technology enhanced

A Methodology and Framework for the Semi-automatic Assembly of Learning Objects

759

learning community. The GLOBE alliance [http://globe-info.org/] builds on the SQI standard to enable worldwide access to learning repositories. Moreover, to enable retrieval of relevant content resources on the Web, several SQI wrappers were built on top of community driven websites that host large amounts of content, such as WikiAnswers.com, ProProfs.com and Wikipedia. The wrappers retrieve both relevant pages and relevant fragments within the pages. The engine typically exploits the structure of pages to identify content fragments that are reusable, such as individual questions and answers of multiple choice questions or animations within HTML pages. Simple screen scraping approaches are employed to retrieve relevant parts of domain specific websites. Depending on the granularity of the narrative concept, single assets or larger compositions are retrieved, such as single questions versus entire surveys.

3 LAMS Case Study We integrated the assembly approach into the LAMS Learning Activity Management System [3] that integrates different environments for authoring, running and monitoring learning designs. The LAMS authoring environment enables authors to sequence different types of learning activities, such as discussion activities and web polls, as illustrated in Fig. 2. In the next step, learning resources can be added to the learning activities. We have extended the LAMS authoring environment to automate, at least partially, this process. An author can create a sequence of activities or reuse an existing learning design. Suppose she wants to teach the concepts of velocity and acceleration in an inquiry based learning strategy that sequences the activities "answer questions" (a1), "vote on a list" (a2), "discuss responses" (a3), "read expert view" (a4), "discuss expert view" (a5) and "personal reflection" (a6). For activities a1 and a4 that have associated learning resources, the assembly engine generates content suggestions based on domain concepts (velocity and acceleration), instructional concepts (answer questions and read expert view) and teacher attributes (in our current prototype: language, familiar measures and weights, and typical student age range). Learning resources are retrieved on-the-fly from learning object repositories and online Web sources and shown in the content suggestions area, as illustrated in Fig. 2. To obtain a first indication of the quality of the generated content suggestions, a small-scale experiment was conducted in April 2009 at Brigham Young University, during a post-doctoral stay of the first author of this paper. Six staff members of the Instructional Design and Technology department and six students in history and social sciences teaching were asked to reuse an inquiry based sequence and to rate the quality of the generated content suggestions. Two dimensions of quality were assessed: relevancy and accuracy. Relevancy measures whether the content suggestions are applicable and helpful for the task at hand. Accuracy is defined as the extent to which the content is correct, reliable and free of error. The mean for both dimensions on a 7 point scale was 6.5833 (0.51493 std dev.). Although these results are only preliminary, they indicate that participants found the generated content highly relevant and accurate.

760

K. Verbert, D. Wiley, and E. Duval

Fig. 2. LAMS plug-in

A Methodology and Framework for the Semi-automatic Assembly of Learning Objects

761

4 Related Work Reuse is considered to be an effective strategy for building high-quality learning sequences [9]. Whereas both basic and applied research have been conducted in the area of decomposing content into reusable components, little research is available on the automated reuse and assembly of content. In contrast, numerous research efforts have been made to support the development of adaptive personalized courses based on content that has been designed specifically for the course at hand [2]. Typically, multiple models are employed to support adaptivity. Dagger et al. [4] identify an instructional model, a learner model, a teacher model and a domain model. The ADAPT project [10] identifies the context of use, content domain, instructional strategy, instructional view, learner model, adaptation model and detection mechanism. The GRAPPLE project [11] identifies a domain model, a user model, a context model, an instruction model and an adaptation model. In this paper, we shifted the focus from the learner to the teacher, as automated assembly of existing learning resources requires quality control by the teacher. Currently, there exist a range of tools to author learning sequences. Reload LD Editor [12], aLFanet LD Editor [13], CopperAuthor [14] and ASK-LDT [15] are examples of form-based editors for authoring learning designs. MOT+ [16], LD Suite [17], LAMS [3] and ACCT [4] are visual editors. Rather than developing yet another learning design environment, we incorporated our assembly strategy in the widely used LAMS authoring environment. LAMS was chosen because it provides a visual user interface that is targeted to be usable by teachers. In contrast, many of the formbased editors require good knowledge of the IMS learning design specification. In addition, LAMS was released as open source software in 2005 and has a large user community, which can potentially provide a solid basis for targeted validation.

5 Conclusion and Future Work In this paper, we have presented a methodology and framework to automate the assembly of learning resources. The framework retrieves learning resources from the Web and GLOBE repositories based on a teacher model that captures teacher characteristics, an instructional model that captures the pedagogical strategy and a domain model. The approach enables the focused retrieval and aggregation of content fragments tailored to different pedagogical approaches, teacher preferences, etc. In addition, we presented a case study that integrates the approach into LAMS. Future work will focus on validating the approach in real-world settings. One of the major motivations for integrating the approach in LAMS is the fact that LAMS is already used on a global scale. By capturing automatically the actual use by students of generated content suggestions, we will retrieve good indications of the quality of the generated learning sequences. Acknowledgements. The research leading to these results has received funding from the European Community Seventh Framework Programme (FP7/2007-2013) under grant agreement no 231396 (ROLE) and grant agreement no 231913 (STELLAR).

762

K. Verbert, D. Wiley, and E. Duval

References 1. Bush, M.D., Mott, J.D.: The Transformation of Learning with Technology. LearnerCentricity, Content and Tool Malleability, and Network Effects. Educational Technology Magazine (March-April 2009) 2. Vercoustre, A., McLean, A.: Reusing Educational Material for Teaching and Learning: Current Approaches and Directions. International Journal on E-Learning 4(1), 57–68 (2005) 3. Dalziel, J.R.: Implementing Learning Design: The Learning Activity Management System (LAMS). In: Crisp, G., Thiele, D., Scholten, I., Barker, S., Baron, J. (eds.) Interact, Integrate, Impact: Proceedings of the 20th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education, December 7-10, Adelaide (2003) 4. Dagger, D., Wade, V., Conlan, O.: Personalisation for All: Making Adaptive Course Composition Easy. IFETS Journal of Educational Technology and Society, Special Issue on Authoring of Adaptable and Adaptive Educational Adaptive Hypermedia (2005) 5. Virvou, M., Moundridou, M.: Adding an Instructor Modelling Component to the Architecture of ITS Authoring Tools. International Journal of Artificial Intelligence in Education 12, 185–211 (2001) 6. Wiley, D.: Learning objects and the new CAI: So what do I do with a learning object (1999), http://opencontent.org/docs/instruct-arch.pdf 7. Ternier, S., Massart, D., Campi, A., Guinea, S., Ceri, S., Duval, E.: Interoperability for Searching Learning Object Repositories: The ProLearn Query Language. D-Lib Magazine 14(1/2) (2008) 8. Simon, B., Massart, D., van Assche, F., Ternier, S., Duval, E., Brantner, S., Olmedilla, D., Miklos, Z.: A Simple Query Interface for Interoperable Learning Repositories. In: Proceedings of the 1st Workshop on Interoperability of Web-based Educational Systems, pp. 11–18 (2005) 9. Schluep, S.: Modularization and structured markup for web-based learning content in an academic environment. Shaker Verlag, Aachen (2005) 10. Garzotto, F., Cristea, A.I.: ADAPT: Major design dimensions for educational adaptive hypermedia. In: Proc. of ED-MEDIA 2004, June 21-26, pp. 1334–1339 (2004) 11. De Bra, P., Pechenizkiy, M., van der Sluijs, K., Smits, D.: GRAPPLE: Integrating Adaptive Learning into Learning Management Systems. In: Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2008, pp. 5183–5188. AACE, Chesapeake (2008) 12. Reload, Project, http://www.reload.ac.uk/ (accessed April 20, 2009) 13. Van Rosmalen, P., Boticario, J.: Using Learning Design to Support Design and Runtime Adaptation. In: Koper, R., Tattersall, C. (eds.) Learning Design. A Handbook on Modelling and Delivering Networked Education and Training, The Netherlands, pp. 291–301. Springer, Heidelberg (2005) 14. Van der Vegt, W.: CopperAuthor. Heerlen: Open University of The Netherlands (2005), http://www.coppercore.org 15. Sampson, D.G., Karampiperis, P., Zervas, P.: ASK-LDT: A Web-based learning scenarios authoring environment based on IMS Learning Design. Advanced Technology for Learning 2(4) (2005) 16. Paquette, G., Lundgren-Cayrol, K., Léonard, M.: The MOT+ Visual Language for Knowledge-Based Instructional Design. In: Botturi, Stubs (eds.) Handbook on Virtual Instructional Design Languages (2008) 17. Elive Learning Design, http://www.elive-ld.de/content/index_eng.html

Search and Composition of Learning Objects in a Visual Environment Amel Bouzeghoub, Marie Buffat, Alda Lopes Gançarski, Claire Lecocq, Abir Benjemaa, Mouna Selmi, and Katherine Maillet Institut TELECOM, TELECOM & Management SudParis, CNRS Samovar 9 Rue Charles Fourier, 91011 Evry Cedex France {Amel.Bouzeghoub,Marie.Buffat,Alda.Gancarski,Claire.Lecocq, Abir.Benjemaa,Mouna.Selmi,Katherine.Maillet}@it-sudparis.eu

Abstract. This paper presents a complete visual environment which supports the search and composition of learning objects (LOs). It focuses on the end user, learner or teacher. Learners search for LOs in order to learn a new concept or to follow a lesson. Teachers search for LOs for direct use during their lessons or in order to reuse and assemble them with others, thus creating their own, novel LO. Nevertheless, the inner complexity of an LO makes searching for and reusing composed LOs a complex task as well. The end user has to be assisted during this task. The core of our environment is built with a navigational and iterative query language, and a composition model. An iterative, navigational, query language is a complex language. The end user cannot express search queries directly in such a textual language. In the same way, the teacher cannot use a complex textual language to compose a new LO. Our environment is a suite of visual interfaces, supporting interaction with the end user while hiding the inner complexity of the system. Last, a validation module validates the consistency of a composed LO and provides for the dynamic annotation of metadata. Keywords: Learning Object Composition, Visual Search, Dynamic Annotation.

1 Introduction Internet facilitates the development of a large number of web-based educational systems. These systems manage pedagogical resources, also called Learning Objects (LO), available on the Web. In [1], several repositories of LOs are cited, such as the ARIADNE knowledge pool [2]. The reuse of existing LOs is a major issue in organizations in which many LOs are created. In order to facilitate the search and reuse of LOs, several standards like LOM [3] and SCORM [4] were created to define sets of metadata to describe existing LOs. These standards are used in web-based educational systems designed for sharing LOs, and have been quickly adopted by the general public. The first feedback on operational systems provided two conclusions: (1) Describing LOs simply by using a set of metadata is insufficient; semantics have to be added to this description in order to enrich search and reuse (composition) processes, adaptation processes and to improve application interoperability; (2) The efficient reuse of LOs requires the definition of rich composition operators, which remains a U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 763–768, 2009. © Springer-Verlag Berlin Heidelberg 2009

764

A. Bouzeghoub et al.

complex topic [5]. In order to palliate these two weaknesses, several models have been proposed which enrich the semantic dimension of standards. For example, SIMBAD [6] includes semantic models of the learning domain, learners and composed LOs. The search for an LO may be done by a learner or by a teacher: learners in order to learn a concept or a lesson; teachers in order either to use an LO directly during their lessons or to reuse it with others, thus creating a new LO. Nevertheless, considering the inner complexity of LOs, the search, reuse and annotation of complex LOs are also complex. Existing tools are inappropriate: they do not provide any support to authors (no clear visualization of the components of an LO, no rich query language, no composition language using existing LOs [7]). To our knowledge, a few works provide some answers to these problems: [8] in which composition is not taken into account and [9], a project which is still under study. We propose thus an iterative approach to search for LOs: the end user browses a set of LOs and within the inner structure of each one, chooses the ones he/she is interested in. The end user composes his/her own LO, as in the case of a teacher. An iterative, navigational, query language is a complex language. It is not conceivable to propose a textual interaction to end users. Moreover, query results are complex: the number of LOs may be too great to be presented as a list, the inner structure of an LO may be complex and recursive. To resolve these problems we propose a sequence of rich and intuitive visual interfaces. This paper is organized as follows: Section 2 describes the SIMBAD model we used to build our system. Section 3 presents the system architecture, describing the different user interfaces, the query engine, the composition validation and the dynamic annotation. Section 4 concludes and proposes perspectives to our work.

2 SIMBAD Model The SIMBAD model includes semantic models of the knowledge domain (domain ontology), of the learner (her/his knowledge, preferences) and of the LO (content, prerequisites, knowledge gained at the end of the learning). An LO may be atomic or complex. A complex LO is built by applying (if necessary, recursively) composition operators on LOs (atomic or complex). The composition of an LO is a graph. This graph can only have one entry (one LO) but may have several exits. We have chosen five operators, three simple operators (SEQ for the sequence, PAR for parallelism, ALT for alternative) and two more complex operators (AGG for aggregation of two LOs and PROJ to define an LO by projection of another). For example, let R10 be a complex LO; its composition graph is defined by: R10 = R1 SEQ (R5 ALT (R2 SEQ (R3 PAR R4))). R1 is atomic; R2 to R5 are complex.

3 System Architecture The system architecture is presented in Fig 1. The following scenario illustrates the utilization of the different components. A teacher searches for an LO in order to compose a new course. She/He specifies her/his research criteria in the query interface (Fig. 1, n°1). This query is sent to the query engine which interacts with the knowledge server and sends back the answers to the result visualisation interface (Fig. 1,

Search and Composition of Learning Objects in a Visual Environment

765

n°2). At this stage, the teacher may explore the structure of the LO (Fig. 1, n°3). The teacher may copy/past each LO or a component of the LO which she/he wants to keep (Fig. 1, n°2 and 3) and it is sent to the composition editor (Fig. 1, n°4). When she/he validates a composition, the validation module checks its validity and annotates it automatically. The result of these annotations is proposed in an annotation interface (Fig. 1, n°5) for final validation and storage in the knowledge server. Interfaces Query Interface

1

2 Results Visualization Interface

Query engine

3 LO Visualization Interface

4

5

Composition editor

Adding LO form

Knowledge Server

Validation Module

Fig. 1. System architecture

3.1 Search and Composition User Interfaces The user interacts with four interfaces for searching and composing. The query interface (n°1) is a form which allows the user to send requests by specifying the results criteria, and this in an iterative way (the user always has the possibility to refine his/her search). The result interface (n°2) presents all the LOs corresponding to the query results in a structured way. These results are not all visible on the screen simultaneously because there would be too much information. The user must then be able to navigate through these results, in an intuitive way. The LO visualization interface (n°3) must make it possible to explore the composition, potentially recursive, of the LO. Finally, the composition interface (n°4), which is offered to authors, is a graphic editor with which they can compose their own LOs by re-using existing LOs. A study of visualization techniques confronted with the needs of our result interface (n°2) shows that the use of 2D is the most appropriate. Textual display, although very easy to use, cannot structure the results; 3D, although it offers intuitive visualization, imposes navigation features which are quickly disturbing (although we usually see in 3D, we move in 2D on the ground). 3D can be justified for displaying a very large amount of information, which is not necessary in our case. Among the 2D visualization tools, Grokker [10] (a search meta-engine) can be easily adapted to our needs. The results are dynamically generated, organized in a treelike classification and the user explores these results by navigating from the highest level. Grokker’s visualization principles apply equally to our results interface (n°2, results organized with the concepts defined in the domain ontology) as to our LO visualization interface (n°3, recursive composition structure). As with Grokker’s tool, only one “step” of the tree can be visualized at a time, and it is interesting for the user to build his/her mental model. In our application, for each query, a subset of the domain ontology is returned.

766

A. Bouzeghoub et al.

This one being voluminous, the user builds his/her mental model of the ontology step by step, with a reasonable cognitive load. The fact that the user perceives the ontology progressively throughout his/her search allows him/her to learn progressively, learning is based on his/her interests. Fig. 2 illustrates the initial visualization of the query results and then the navigation (zoom) on this result. The LO composition interface (n°5) is a free editor. The author can search for LOs or parts of LOs by using search and LO composition interfaces (n°2 and 3). He/she can select them and drop them into his/her composition space. He/she can then define operators between the LO and thus create a new LO which will be added to the system.

Fig. 2. Results windows, initial (left) and after a zoom (right)

3.2 Query Processor The query processor takes queries from the user interface (n°1) and translates them to be sent to the Ontobroker knowledge server. Ontobroker takes queries and commands to add facts to the knowledge base specified in F-Logic. Facts correspond to semantic descriptions of LOs. Queries contain criteria specified by the user to search for LOs. When the user searches for an LO, each input of the user interface form is verified: if an input is filled, the associated criteria are used to generate an F-logic query. For example, query R1 allows searching for LOs having a SIMBAD metadata description. In R1, O is the variable representing an LO; this variable is instantiated and returned. Suppose that, facing the results of R1 processing, the user refines the search criteria filling the input title of the form. Query R2 is then obtained. R1 : "FORALL O,X,N,MN#X:N#SIMBAD]@M"" R2 : "FORALL O,X,N,M N#X:N#SIMBAD]@M" AND EXISTS G1,T N#X[N#hasElement>N#G1:N#General[N#title->T]]@M AND contains(T,\""+title+"\")@M". OntoBroker returns the result of a query to the query processor. Let O1 a LO belonging to the result. O1 may be described in the following way:

Search and Composition of Learning Objects in a Visual Environment

767

[O1, S1, "\"http://www.owlontologies.com/lom.owl#\"",""\"http://www.owlontologies.com/\"#'lom.owl'"] The interesting information of each LO in the result is taken from its SIMBAD description, like the title and subject. This information is sent to the user results visualization interface (n°2) using an XML document. 3.3 Validation and Annotation of Composed LO The composition of an LO is performed by following a composition pattern (generic graph or composition model) or on the fly. The first case is safer because the pattern proposed by an expert or a teacher is normally valid while the second case may entail problems of composition validity. We focused on the second problem of composition validity with a particular interest for free composition. An LO is valid if its composition graph complies with a set of constraints to ensure the coherence from a structural, semantic and pedagogic point of view. The structural validation checks whether the topography of the graph is correct, by controlling, for example, that the graph has only one start node. As a result, using graph patterns implies structural validity. The more complex semantic validation examines the coherency in the sequencing of the LO. It is necessary, for example, to verify that the level of acquisition of the LO increases with the progression in the graph and not the opposite, or a learner having the required prerequisites has access to at least one path of the graph. The semantic validity is never assured, whatever the composition type (pattern-based or free composition). The pedagogic validation is based on the accordance of the composition graph with a known learning theory. This latest type of validation is not implemented yet. The validation phase is followed by a phase of annotation before storing the new LO in the knowledge server. The author must enter the whole set of metadata which describes it. We propose to facilitate this task by generating some metadata automatically. We use a LOM model which handles complex and long categories, and it is difficult to motivate authors to describe their productions with such a model. Hence, the system deduces metadata values of a composed LO from the metadata of its components. For example, in the heading ` Life cycle', “contribution” indicates authors who contributed to the modification of the LO, the date of the contribution and their role (e.g., author, editor). The contribution of a composed LO is the aggregation of the contributions of each atomic LO. The semantic metadata (contents, prerequisite) can also be generated automatically. Indeed, composition operators have well defined semantics which make it possible to automatically generate the semantics of a composed LO from the semantics of its atomic components.

4 Conclusion Today LO reuse is a hot research topic at the representation level, but few studies have been devoted to user friendly interactive interfaces for LO search and composition. In this context, our system is innovative because, through a specific visual environment, it allows end users (learners and teachers) to be able to easily express their

768

A. Bouzeghoub et al.

queries and view the results. Moreover, the systems offer the possibility to an author (1) to create a new LO by compositing several existing ones, (2) to verify its structural, semantic and pedagogical validity, and (3) to annotate it, by automatically generating part of the associated metadata. Our system is a complete tool for managing LO: creation, search, validation, annotation and insertion in the knowledge base. As a next step we plan to test the system with real users in a real context.

References 1. Beck, R.: Learning Objects Collections (2007), http://www.uwm.edu/Dept/CIE/AOP/LO_collections.html 2. Duval, E.: The Ariadne Knowledge Pool System. Communications of ACM 44(5), 72–78 (2001) 3. Learning Technology Standard Committee: IEEE Standard for Learning Object Metadata, IEEE Std 1484.12.1 4. Advanced Distributed Learning Initiative (ADLI). Sharable Content Object Reference Model. The SCORM Content Aggregation Model. V. 1.2. adlnet.org/ (2007) 5. Harris, M.C., Thom, J.A.: Challenges facing the retrieval and the reuse of learning objects. Workshop on learning object repositories as digital libraries: current challenges. In: 10th European Conference on Digital Libraries (ECDL) Workshop (2006) 6. Duitama, F., Defude, B., Bouzeghoub, A., Lecocq, C.: A framework for the generation of adaptative courses based on semantic metadata. Multimédia Tools and Applications 25(3), 377–390 (2005) 7. Lopes Gançarski, A., Bouzeghoub, A., Defude, B., Lecocq, C.: Iterative search of composite learning objects. In: IADIS International Conference WWW/Internet, Vila Real, Portugal (October 2007) 8. Ramzay, J., McAteer, E., Harris, R., Allan, M., Henderson, J.: Flexible, structured support for the reuse of online learning objects. In: Networked Learning conference (2004) 9. Chaudhry, A.S., Khoo, C.S.G.: Issues in developing a repository of learning objets for Lis education in Asia (2006) 10. http://www.grokker.fr (2009)

A Framework to Author Educational Interactions for Geographical Web Applications The Nhan Luong, Thierry Nodenot, Philippe Lopistéguy, and Christophe Marquesuzaà IUT de Bayonne Pays Basque, LIUPPA-DESI, 2 Allée du Parc Montaury 64600 Anglet, France {thenhan.luong,thierry.nodenot,philippe.lopisteguy, christophe.marquesuzaa}@iutbayonne.univ-pau.fr

Abstract. This paper focuses on the production of authoring tools that teachers may use to prototype interactive geographical web applications. We present some computational models and a toolset that we designed to address some needs of teachers trying to make use of particular localized documents called “travel stories”. Our research challenge is to enable teachers to design interaction scenarios for such a domain, avoiding any programmer intervention. In the design process, the teacher typically faces three activities: (a) Identification of candidate documents, (b) Evaluation of the adequacy of the document and (c) Production of the learning application making use of the selected document. In this paper, we mainly focus on the (c) Production activity. We highlight the necessary use of an “agile” approach to shorten as much as possible the delay between the design and the evaluation step of a prototype. To address the technological challenges raised by such an aim, we present WIND framework and we discuss its capabilities while considering some examples of interactive scenarios generated with WIND framework. Keywords: geographic information, interaction design, interaction programming, agile approach, web application, authoring framework.

1 Introduction Educational scenarios are particular design artifacts that take advantage of current enhancements in the domain of “Science of Design” [1]. Research works dedicated to the design of educational scenarios propose new paradigms, concepts, approaches, models and theories that promote stronger bases for the design of TEL environments [2]. These bases are foundations enabling to improve the processes of both coding, evaluating and maintaining such type of application. Recent works focus on: – the definition and role of a pedagogical scenario [3, 4], – the definition of visual instructional languages [5] and executable [6] scenarios, – the definition and the evaluation of methodological principles allowing designers to produce and to re-use such scenarios [7]. U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 769–775, 2009. © Springer-Verlag Berlin Heidelberg 2009

770

T.N. Luong et al.

Nowadays, there is also a strong emphasis on the process of transforming an abstract scenario (that a teacher is able to understand and to design by himself) into an executable scenario (that an execution infrastructure can process). Several works promote model-driven engineering techniques and tools to fully integrate the functions supported by the chosen infrastructure. This approach is interesting not only because of underlying model-driven scientific challenges [8], but also because modeldriven transformations must respect educational constraints specified in the scenario produced by the teacher [9]. Most of identified works are still in progress and it is thus difficult to know if they will soon provide teachers with toolboxes fitted to the design and the implementation of constructivist learning situations. Moreover the complexity of required technologies may put the teacher out of the play (when the educational scenario becomes very detailed, when the scenario is deployed on a target infrastructure…). In most works, the author is a pedagogic engineer. Such designer profiles can be found in e-learning firms but not in most classrooms and teaching institutes: though they are not computer-scientists, teachers must fully handle the design process (from the scenarization step to the deployment step). This paper focuses these particular teachers: we propose a framework that they may use to edit/prototype and to evaluate by themselves an educational scenario. This framework targets the scenarization of interactive resources to be used in inquiry learning activities [10] for a specific applied domain: travel stories. In the second section, we present the background and the objectives of these research works. This leads us to present in the third section WIND framework that facilitates an “agile” production of interactive scenarios. The conclusion section proposes a synthesis about WIND and our future directions of investigation.

2 Background and Objectives The DESI1 group aims to propose software architectures and tools to re-vitalize localized documents that generally rest in the depth of archives, museums and libraries. In particular, travel stories offer very challenging revitalization objectives because tourists and teachers could benefit from e-services developed from such localized documents. Travel stories have intrinsic characteristics that make them good teachingresource candidates. A travel story is a sort of text whose author tells what he discovered while travelling through a territory or a country. On the one hand, the author tries to very precisely present the places he/she visited; on the other hand, he/she tries to tell the events that occurred during his/her trip, he/she reports on his/her activities and explorations. Indeed, the travel story aims at using words to describe the travel reality: the travel story is told day after day, the duration of the trip is often explicitly written in the text in conjunction with the travelled locations. Moreover, travel stories provide an opportunity to ground the design and the operation of systems with text, map and calendar components that require extensive human-machine interaction. Following the experiment available at http://erozate.iutbayonne.univ-pau.fr/ forbes2007/exp/, we proposed three authoring steps to assist as much as possible the process of authoring educational applications making use of travel stories. The first 1

DESI is a French acronym that means Electronic Documents, Semantics and Interaction.

A Framework to Author Educational Interactions

771

teacher’s task consists in selecting in a corpus of documents the travel stories that deal with geographical areas [11, 12]. The second task consists in evaluating the adequacy of the document as regards of teacher’s pedagogical aims [13]. The third teacher’s task consists in producing a highly interactive application based on the semantics of the validated travel story. To this end, the teacher needs an authoring environment that enables him/her to quickly evaluate/correct his/her conceptual choices. This paper focuses on this third task. Following A. Gibbons' works [14, 15], our approach aims at breaking such a design dichotomy/gap for the particular case of educational applications making use of travel stories. Indeed, Gibbons’ instructional design layers are: content, strategy, control, message, representation, media-logic, data-management. Design of each layer can be considered separately from the other layers, providing an important modularization to the design effort. Applied to our application domain (study and design of educational applications making use of geographical information embedded in travel stories), Gibbons’ design theory leads us to design interactions at four levels of abstraction, as suggested in [16]: 1. The most elementary level deals with the data and geographical information embedded/retrieved in a text (cf. the data-management, media-logic and representation layer) that may be associated with goals and activities (input and output parameters), participants, artifacts provided to participants (e.g. map, text, calendar components). 2. The second level (cf. the message layer) focuses on the messages exchanged during an interaction, their structure and the way they are generated from the data and geographical information manipulated by a learner. It also enables to control the execution of the interaction model in accordance with the satisfaction of a certain conditional expression. For example, an interaction may become mandatory depending on previous interactions with an icon representing a particular place mentioned in a travel story. 3. The third level (cf. the control layer) considers the possibility to introduce decisions and commands that enable to change the behavior of interaction scenarios according to the aims of participants having specific rights for the considered educational unit. For example, a learner can decide to mask the calendar component because he/she wants to ignore the travel story’s chronology. 4. The fourth level (cf. the strategy layer) considers the use of events to decide the performance of changes: events can dynamically occur during execution, they can be triggered from the evaluation of time conditions, goals achievement, activity reports. For example, if the learner never interacts with the map component, this means that he/she probably needs some support to take advantage of the map component capabilities. In the next section, we present a framework that addresses interaction design according to these four abstraction levels.

3 A Framework to Facilitate an “Agile” Production of Interactive Scenarios WIND favors empirical design approaches enabling a teacher to easily formalize and evaluate his/her educational ideas by using (as a learner) the automatically generated

772

T.N. Luong et al.

application. Evaluation step is therefore used to check/criticize his/her pedagogical choices. We thus define the concept of an “agile” design tool as a piece of software supporting such an approach. Indeed, we may define an “agile” method as a design approach not only by fully implying the end-user along the whole process but also by rapidly integrating his/her requirements in a technical solution [17]. Final quality of the generated application is ensured thanks to a continuous control all the production process along. As a consequence, each teacher’s pedagogical choice must be fully and automatically traduced into executable code. This constraint implies the use of an applicative model as a design framework. An applicative model is an application generic model that may be instantiated all the design step along; each instance of this model is then automatically traduced into executable code. Our proposed design approach is based on a model-driven approach [18, 19] which is also used for TEL engineering [8]. We distinguish three levels: 1. The generic applicative model describes the core concepts of the application classes. 2. The application model created by the teacher during his/her design step. This model is an instance of the previous generic model. It describes the characteristics of the application desired by the teacher. 3. The code generated from the application model designed by the teacher. The WIND generic applicative model [20] defines the core of the interactive web applications that a teacher will be able to produce. An interaction may be simply defined as a triple <area, event, reaction>. Interaction is the central mechanism which characterizes the applications we wish to develop. The expressive capacity of the interactions may be declined to express not only simple interactions but also more complex ones. This WIND generic applicative model is described in a WIND-XSD schema (available at http://erozate.iutbayonne.univ-pau.fr/Nhan/WINDv2/schema.xsd). Each concept of the model is described as a specific element. This XSD schema helps to instantiate WIND generic applicative model into XML format that describes the interactions of a WIND application model. That is to say it permits to describe a web-based application embedding textual, map and calendar interactive components. Taking advantages of JavaScript, WIND generic applicative model is supported by a WIND-API that implements the different classes as well as their associated methods. WIND-API proposes a homogeneous layer built on lower level APIs, specialized in the handling of text, map and calendar. To avoid any programmer intervention in the teacher activity devoted to the application production, we have developed a JavaScriptCodeGenerator2. The JavaScriptCodeGenerator can parse any WIND-XSD compliant data file (e.g. a WIND application model description) in order to generate JavaScript code for interactions that WIND API can execute. These technologies enable us to shorten the delay between the design and the evaluation step of a prototype. The implementation of WIND interactions may simply be done with four main steps3 : 2

For example, see the XML file at http://erozate.iutbayonne.univ-pau.fr/Nhan/WINDv2/ data.xml and the automatically produced web application that the end-user can exploit: http://erozate.iutbayonne.univ-pau.fr/Nhan/WINDv2/generator.php?file=data.xml 3 A complete example is available at http://erozate.iutbayonne.univ-pau.fr/Nhan/WINDv2/

A Framework to Author Educational Interactions

773

1. Defining the components of the application and their characteristics. 2. Defining reactive areas for each component defined in the previous step. 3. Defining possible reactions for the reactive areas. 4. Defining interactions upon previously defined reactive areas and reactions.

4 Discussion and Future Directions WIND is an operational framework that allows both describing and implementing interactions of web applications mixing texts, maps and calendars. This framework promotes an agile process fitted to designers without computer-science skills: its characteristics make easier the description of interactions. Yet, WIND still needs further developments. We must extend our current framework because WIND does not completely address the design of the four interaction layers presented in section 2. Current version of WIND framework correctly addresses the data-layer: it enables designers to manage sensible parts corresponding to the main concepts (places, dates, movement-verbs, etc.) automatically retrieved in travel stories. As a consequence, interaction design can take advantage of these specific sensible parts. However, we still need to extend WIND framework to manage the same way more complex concepts of an itinerary. Current version of WIND framework enables a designer to specify the messages exchanged with a learner, the semantics of the messages and their appearance. Moreover, conditional messages are easy to describe with WIND functionality, thus satisfying the requirements of the second level/layer. Current version of WIND framework enables a designer to specify who controls an interaction, how it is initiated, what the learner’s degrees of liberty are. WIND provides functionality needed to design mixed-initiative interactions, thus satisfying the requirements of the third level/layer. However, current version of WIND framework fails to completely address the strategic layer because WIND does not provide any functionality to assess cognitive processes. The event-reaction mechanism implemented by WIND provides the required functionality to design reactions on cognitive events, but we do not currently provide designers with any means to detect such cognitive events. Our first experiments have shown that WIND is really helpful to rapidly design and assess inquiry activities making use of the semantics of travel stories. Directions of research discussed above will certainly enhance the instructional design addedvalue of WIND framework. We also need to propose an evaluation protocol to determine to which extend can teachers concretely exploit the current WIND framework and its corresponding authoring tools.

Acknowledgments This research is supported by the French Aquitaine Region (project n°20071104037) and the Pyrénées-Atlantiques Department (“Pyrénées : Itinéraires Educatifs” project).

774

T.N. Luong et al.

References 1. NSF 2007, Science of Design: National Science Foundation 07-505, Program Solicitation (2007), http://www.nsf.gov/publications/ pub_summ.jsp?ods_key=nsf07505 2. Tchounikine, P.: Pour une ingénierie des Environnements Informatiques pour l’Apprentissage Humain. Information Interaction Intelligence 2(1), 59–93 (2002) 3. Pernin, J.-P., Emin, V., Guérayd, V.: ISiS: An Intention-Oriented Model to Help Teachers in Learning Scenarios Design. In: Second European Conference on Technology Enhanced Learning, pp. 338–343 (2008) 4. Dillenbourg, P., Tchounikine, P.: Flexibility in macro-scripts for computer-supported collaborative learning. Journal of Computer Assisted Learning 23(1), 1–13 (2007) 5. Nodenot, T.: Scénarisation pédagogique et modèles conceptuels d’un EIAH: Que peuvent apporter les langages visuels? International Journal of Technologies in Higher Education 4(2), 85–102 (2007) 6. Ferraris, C., Martel, C.: LDL for Collaborative Activities. In: Botturi, L., Stubbs, T. (eds.) Handbook of Visual Languages in Instructional Design; Theories and Practices, pp. 226– 253. IDEA Group, Hershey (2007) 7. Villiot-Leclercq, E.: Modèle de soutien à l’élaboration et à la réutilisation de scénarios pédagogiques. Doctorat en Sciences Cognitives de l’Université Grenoble 1 (2007) 8. Laforcade, P., Nodenot, T., Choquet, C., Caron, P.A.: MDE and MDA applied to the Modeling and Deployment of Technology Enhanced Learning Systems: promises, challenges and issues. Architecture Solutions for E-Learning Systems (2007) 9. Caron, P.-A.: Web Services Plug-in to Implement “Dispositives” on Web 2.0 Applications. In: Duval, E., Klamma, R., Wolpers, M. (eds.) EC-TEL 2007. LNCS, vol. 4753, pp. 457– 462. Springer, Heidelberg (2007) 10. Olson, S., Loucks-Horsley, S.: Inquiry in the National Science Education Standards: a guide for teaching and learning. National Academies Press, Olson (2000) 11. Loustau, P., Nodenot, T., Gaio, M.: Design principles and first educational experiments of π R, a platform to infer geo-referenced itineraries from travel stories. In: International Journal of Interactive Technology and Smart Education, ITSE (2009) 12. Gaio, M., Sallaberry, C., Etcheverry, P., Marquesuzaà, C., Lesbegueries, J.: A Global Process to Access Documents’ Contents from a Geographical Point of View. Journal of Visual Languages and Computing 19, 3–23 (2008) 13. Loustau, P., Nodenot, T., Gaio, M.: Spatial decision support in the pedagogical area: Processing travel stories to discover itineraries hidden beneath the surface. In: 11th AGILE International Conference on Geographic Information Science, pp. 359–378 (2008) 14. Gibbons, A.S.: What and how do designers design? A theory of design structure. Tech. Trends 47(5), 22–27 (2003) 15. Gibbons, A., Stubbs, T.: Using Instructional Design layers to categorize design drawings. In: Botturi, L., Stubbs, T. (eds.) Handbook of Visual Languages in Instructional Design; Theories and Practice. IDEA Group, Hershey (2007) 16. Caeiro-Rodríguez, M., Llamas-Nistal, M., Anido-Rifón, L.: The PoEML Proposal to Model Services in Educational Modeling Languages. In: Dimitriadis, Y.A., Zigurs, I., Gómez-Sánchez, E. (eds.) CRIWG 2006. LNCS, vol. 4154, pp. 187–202. Springer, Heidelberg (2006)

A Framework to Author Educational Interactions

775

17. Vickoff, J.P.: Systèmes d’information et processus agiles. Hermes Science (2003) 18. Seidwitz, E.: What models mean. IEEE Software, 26–32 (2003) 19. Bézivin, J., Blay, M., Bouzeghoub, M., Estublier, J., Favre, J.-M.: Rapport de synthèse de l’Action Spécifique CNRS sur l’Ingénierie Dirigée par les Modèles: Action Spécifique MDA du CNRS (2005) 20. Luong, T.N., Etcheverry, P., Nodenot, T., Marquesuzaà, C.: WIND: an Interaction Lightweight Programming Model for Geographical Web Applications. In: International Opensource Geospatial Research Symposium, OGRS (to appear, 2009)

Temporal Online Interactions Using Social Network Analysis Álvaro Figueira Universidade do Porto, Faculdade de Ciências, DCC - CRACS Rua do Campo Alegre, 1021/1055, 4169-007 Porto, Portugal [email protected]

Abstract. Current Learning Management Systems generically provide online forums for interactions between students and educators. In this article we propose a tool, the iGraph, that can be embedded in Learning Management Systems that feature hierarchical forums. The iGraph is capable of depicting and analyzing online interactions in an easy to understand graph. The positioning algorithm is based on social network analysis statistics, taken from the collected interactions, and is able to smoothly present temporal evolution in order to find communicational patterns and report them to the educator. Keywords: Visualization of online interaction, Web-based learning, Automatic graph drawing, Temporal analysis, Online discussion forums.

1 Introduction Characterizing interactions of a group that usually communicates through an online context is frequently not a simple task. We do recognize that currently written communication has assumed particularities (emoticons, capitalizations, exaggerated punctuations) that were not considered in the past. We propose a tool to help characterizing online interactions depicting them in the form of a graph that in turn is built with the help of “social network analysis” indicators. The proposed graph represents all interactions occurred up to the drawing moment. This characteristic allows building a “history” of interactions, drawing each network state in a singular frame. A slideshow of all available frames can then provide additional insight for the teacher as he may analyze the class according to different key moments and observe its progress over time, such as actors that maintain leadership roles during most of the time, or actors that shift between more or less active positions in different moments.

2 Online Social Network Analysis Social Network Analysis (SNA) consists in the “mapping” and analysis of the relations between people, groups or organizations, through a visual representation and also a mathematical analysis. The visual representation results in a network that includes a set of actors that interact among themselves as well as information flows. U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 776–781, 2009. © Springer-Verlag Berlin Heidelberg 2009

Temporal Online Interactions Using Social Network Analysis

777

The illustration of this network is represented as a Graph, with actors as vertices and interactions as a set of ties, between two or more vertices, represented by lines. We use the event “reply to” a previous posted message as an atomic interaction. The counting process of the answers that were received and sent begins at the first “reply to” in a discussion. The analysis process in SNA generally consists in three perspectives. The first reports to the actors’ positions individually; the second, to group action and, the third to the group or community as a whole. Centrality measures are seen as fundamental attributes of a social network. We assumed Freeman’s [1] procedures to calculate centrality. According to Hanneman & Riddle [2], many Sociologists argue that one of the basic properties of social structures is power. In our study, for a reason of semantic proximity, we will also use this concept intending that it could also be understood as “influence”. Scott [3] also recalls the need to carefully choose indicators and how to apply them keeping in mind the understanding of their properties and if they are adequate and relevant for the study that is being conducted. For the iGraph we used three indicators: Centrality Degree, calculated by summing the vertices that are adjacent to vertex i. Actors who obtain higher results in this indicator may be characterized by being more autonomous and having more influence in the network. Clique identification in a network [1],[2] allows us to locate group of actors where all possible connections are present and expand our comprehension of the group at a global range. Density is one of the mostly used indicators [3],[4]. This indicator reveals, in percentage, the high or low connectivity of a network and is defined as the ratio between the existing and the possible connections. Centralization Index is an indicator for analyzing the network as a whole and is expressed in percentage. It is characterized by the existence of an actor that clearly plays a central role, while being connected to all the vertices in the network.

3 Building a History The motivation for adding history is to understand how the online community evolved along time. Although a graphical representation of the current state of an online community is of much use, that representation lacks the temporal dimension which may hide important aspects of interaction that occurred in past. For example, it is possible that at some time during the interaction, we could find actors that played important roles during the development of the interactions and then their leadership was overcome by two or three other actors, which in present time makes their past importance much reduced. 3.1 Drawing the iGraph There is a vast literature and research area concerning automatic graph drawing [5]. A variety of layout algorithms that are based on graph theoretical foundations have been developed in the last three decades [6]. In 1963 Bill Tutte wrote a paper: How to draw a graph [7] in which he suggested an algorithm that places each vertex into the center of its neighbors. A long way of research has been pursued since then. However, some basic criteria, supported by psychology studies [8][9], still hold: vertices displaying the objects should not overlap each other, nor the lines representing the edges. Moreover, one would want to minimize the edge crossings.

778

Á. Figueira

Our algorithm evolves from a set of basic principles/premises to improve readability and ease of understanding: a) distribute vertices avoiding overlapping; b) information hubs should tend to be placed near the center; c) minimize the crossing of edges and of vertices; d) group cliques; e) dense net tends to spread equally. According to those principles, we built a model of “orbits” in which we place the vertices equally spaced in clockwise manner. The outer orbit is placed near the border of the drawing canvas. In Fig.4 we depict this model.

Fig. 4. Vertex positioning model

The orbit of each vertex is computed according to its centrality degree (the greater the degree, the closer to the center will be the orbit). The centralization index is useful to compute the radius of the closest orbit to the center (a centralization index of 100% means that the smallest orbit will have radius of 0). The net density parameter is useful to set the number of possible orbits (a dense net, will have more orbits). We present the drawing algorithm in Listing 1 where we define k-clique as a clique with k vertices, and an object as either a clique or a vertex. 1. Clique Detection: identify all cliques of size ≥ 3 2. Clique Reduction: pairs of cliques of size n, sharing n-1 vertices are treated as a single n-clique plus the other vertex 3. Let the total number of objects be the number of cliques plus the number of remaining vertices (outside of a clique) 4. Orbit assignment: for each object compute its orbit: 4.1. if object is a vertex compute its normal centrality degree C CD 4.2. if object is a k-clique compute ∑ i as the clique centrality degree, where k

CiCD is the CD of vertex i 5. Layout: dispose objects clockwise, equally spaced 6. Vertex Permutation: for each clique find a permutation (P) of its vertices that minimizes distance to other vertices outside the clique: min{∑ distances( Pi )} i

Listing 1. Vertex positioning algorithm

3.2 Creating Temporal Continuity Our system is based on a series of sequential graphics, each restricted to a temporal frame. Each frame is replaced by the next frame manually, or in automatic mode (in a

Temporal Online Interactions Using Social Network Analysis

779

slideshow). Coherency of graph layout between different frames is ensured by establishing two important premises: the minimum temporal slot for each time frame is the “reply to” interaction and the algorithm for graph drawing must be deterministic to create an illusion of movement, and vertex positioning continuity. Taking the “reply to” relation the following may happen: a) the network density changes; b) the centralization index changes; c) two centralization degrees change; d) a new Clique is formed. In situations a) and c), despite vertices may change their size, the illusion of continuity is preserved. Situation a) creates more orbits and therefore triggers a new assignment of vertices to orbits (preserving continuity). Situation d) may lead to the creation of a neighbor Clique that may continuously increase its size, eventually “absorbing” the other Clique. If this process culminates in the dissolution of the previous Clique, favoring the new one, then a new permutation of vertices has to be found (as in step 7 of the listing). This situation hampers the illusion of continuity between frames and has to be solved by performing a local animation on the involved vertices. To better understand the algorithm we provide an example listed on Table 2: Table 2. Example of interaction between three actors Posts 1 2 3 4 5 6

0 A

1

2

B C A B B

Posts 7 8 9 10 11 12

0

1 A B

2

C C A B

According to interactions listed on Table 2, there are 11 interactions to consider: from post 2 to post 12. In Fig. 6 we show four time frames of the temporal evolution of the iGraph. For the sake of understanding and simplicity we depict frames correspondent of “momentum zero” (T=0), of interactions 5, 8, and 12 (final iGraph).

Fig. 6. Illustration of four time frames for temporal interactions listed in Table 2

From the analysis of Fig. 6 it is clear to understand the benefits of using a temporal evolution in the iGraph: if there would be no temporal evolution of the iGraph, one would look at time frame 12, and conclude that actor C has been away of the discussions and that actor B has a leader role in the forum. However, by observing the evolution of the iGraph it is possible to see that actor B by interaction 5 had a much smaller

780

Á. Figueira

importance and is in fact the “outsider”. Only by interaction 8 all actors have the same number of interactions with each others, and from then on actor B takes the lead.

4 The Interface The iGraph system uses LMS forums to mine for posted messages and presents to the teacher an interface embedded in a web page (as shown in fig 7), which is based on a previous seen proposal [10]. The Centrality Degree is divided into input and output cases: the former is the number of actors that respond to an actor, while the later is the number of actors to which an actor replies. The Centralization Index is also divided into input and output cases, and expressed in percentage. The use of isolated nodes makes the graph include nodes that do not have any link to another node.

Fig. 7. The iGraph interface

Below the graph, it is possible to select any forum that is created in the scope of the current online course and the mode for the iGraph. It is also possible to show cliques of n vertices. In its present version, each actor is assigned a letter which is resolved to his (her) actual name in the box at the lower right.

5 Conclusions We presented a system with an automatic process for characterizing online interactions in discussion forums. The system is capable of depicting current state interactions or a throughout analysis frame-by-frame since the beginning of the forum participations. Trying to obtain illusion of continuity between frames led to the development of a positioning algorithm and a methodology for frame transition. Although we are conscious that it is not currently possible to find the optimal vertex positioning in reasonable time, we believe that our algorithm finds a sub-optimal layout which is not humanly easy to improve, that is aesthetic and easily readable.

Temporal Online Interactions Using Social Network Analysis

781

References [1] Freeman, L.C.: Centrality in Social Networks: Conceptual Clarification. Social Networks 1, 215–239 (1978), http://moreno.ss.uci.edu/27.pdf (accessed: March 2008) [2] Hanneman, R.A., Riddle, M.: Introduction to Social Network Methods. [electronic version] (2005), http://faculty.ucr.edu/~hanneman/ (accessed: March 2008) [3] Scott, J.: Social Network Analysis: a Handbook. Sage, London (1997) [4] Borgatti, S.P., Everett, M.G.: Network Analysis of 2-Mode Data. Social Networks, pp. 243–269 (1997), http://www.analytictech.com/borgatti/papers/ borgatti%20-%20network%20analysis%20of%202-mode%20data.pdf (accessed: March 2008) [5] Jünger, M., Mutzel, P.: Graph Drawing Software. Springer, Heidelberg (2004) [6] Nishizeki, T., Rahman, S.: Planar Graph Drawing. Lecture Notes Series on Computing, vol. 12. World Scientific, Singapore (2004) [7] Tutte, W.T.: How to Draw a Graph. Proceedings of the London Mathematics Society 13, 743–768 (1963) [8] Purchase, H.: Which aesthetic has the greatest effect on human understanding? In: Di Battista, G. (ed.) GD 1997. LNCS, vol. 1353, pp. 248–261. Springer, Heidelberg (1997) [9] Purchase, H., Allder, J.-A., Carrington, D.: User preference of graph layout aesthetics: A UML study. In: Marks, J. (ed.) GD 2000. LNCS, vol. 1984, pp. 5–18. Springer, Heidelberg (2001) [10] Figueira, A., Laranjeiro, J.: Interaction Visualization in Web-Based Learning using iGraphs. In: Proceedings of Hypertext 2007, Manchester, UK (2007)

Context-Aware Combination of Adapted User Profiles for Interchange of Knowledge between Peers Sergio Gutierrez-Santos1, Mario Muñoz-Organero2, Abelardo Pardo2, and Carlos Delgado Kloos2 1

London Knowledge Lab, Birkbeck College, University of London, UK 2 University Carlos III of Madrid, Spain [email protected], {mario,abel,cdk}@it.uc3m.es

Abstract. This paper presents a system that connects students with complementary profiles, so they can interchange knowledge and help each other. The profile of the students is built by a modified intelligent tutoring system. Every time the user profile is updated, a gateway updates the profile stored in the user's personal terminal using a web-service based communication mechanism. The terminals (e.g. mobile phones) are able to find and communicate between themselves using Bluetooth. When they find two complementary user profiles, they help the users getting into contact, thus providing the benefits of social network tools but at short-range and with physical context awareness. Two students are complementary when one knows what the other wants to learn and viceversa, so they can be of mutual help. Keywords: mobile learning, bluetooth, profile matching.

1 Introduction Traditional learning environments are changing significantly. The introduction of pervasive technologies is enhancing the learning process making it more ubiquitous and personalized. However, the anytime-anywhere personalized learning requires also the deployment of an anytime-anywhere personal environment that helps and guides the learning process. This paper defines and provides an implementation of such a ubiquitous personalized tutoring environment by combining a modified intelligent tutoring system with a context aware mobile profile matching service. We describe theoretical aspects as well as implementation issues of such a system. We aim at applying the system in our own university, where students with different profiles can help each other. In other words, the system is expected to work in a traditional learning environment where many students attend lectures and have to study later on their own. Students have the need of more personalized attention, because the very few teachers are not able to adapt their lectures to the specific characteristics of each and every student. The possibility of having a personal tutor would greatly increase the learning of the students. However, the resources are scarce in a traditional education environment and it is not feasible to provide a personal human tutor for each and every student. Another possibility is to build intelligent tutoring systems U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 782–787, 2009. © Springer-Verlag Berlin Heidelberg 2009

Context-Aware Combination of Adapted User Profiles for Interchange of Knowledge

783

(henceforth ITS), that support the learning process of the students by providing feedback on their errors and lacks of knowledge for a specific domain. However, full ITS are very costly to build [4]. There is a third way. A student can ask for help to a peer student that has a deeper knowledge. This “more able peer” is somehow similar to having a personal tutor. However, students are not professional teachers and might not be interested in helping their peers more than occasionally, unless they have something to exchange. The work presented here is based on an economic view of this scenario. In our view, knowledge and expertise in different domains are the scarce resources. Students have a varying amount of knowledge about different subjects. If two students have complementary user profiles (e.g. the first one has a deep knowledge of operating systems, while the second has mastered the computer architecture part of the course) they might be interested in being put in touch to help each other. Therefore, they can interchange what they know and help each other. The use of short-range communication technologies to disseminate information about knowledge and learning needs leads to spontaneous collaboration [1]. Once the system has built a proper profile (i.e. learner model), this is submitted to the personal communication terminal of the student (e.g. mobile phone with Bluetooth capabilities). The terminal operates autonomously from then on, looking for similar devices in the surroundings. Once two such terminals identify themselves, they interchange their user profiles (i.e. learner model). If two profiles are found to be complementary, a message is shown to the students along with additional information. This information aims at facilitating the contact between the two human students (i.e. breaking the ice) and encourages their knowledge interchange. Many systems have tried to benefit from inherent context that exists in short-range technologies such as Bluetooth. The work presented in [5] shows a Bluetooth-based ad hoc e-learning system that connects together students and instructors so that the students can participate in a face-to-face lecture using their personal mobile devices and instructors can receive instant feedback about the students. Although this work uses some of the concepts and technologies presented in this paper, its scope is limited to facilitate student-instructor interactions in a face-to-face class. The work presented here connects the concepts of ITS with context-aware mobile profile-matching applications. Another related work is the one presented in [2] which defines and implement a pervasive communication system from a central learning management system to mobile students based both on SMS and Bluetooth. The idea of synchronizing the status of a central learning management system with the mobile learners is similar to ours. However, we introduce a profile based synchronization mechanism from which peer to peer relationships among students can be established.

2 Architecture The combined central e-learning server-oriented and student’s mobile peer to peer architecture of the system that we have defined is depicted in Figure 1.

784

S. Gutierrez-Santos et al.

Fig. 1. Architecture of the system

The architecture presents two main parts. The first one is the server, that contains an Adaptive Profiler (in our case, a modified intelligent tutoring system) and a synchronization gateway. As a consequence of the interactions between the students and the profiler, the students’ profiles (i.e. user models) stored in a database are populated. These profiles contain the information about the strengths and weaknesses in the learning process of each student. The second part is deployed on the mobile terminals of the students. It contains both the implementation of the synchronization interface used by the gateway to update the student’s profile, and the peer-to-peer profile matching application used to find other students with complementary profiles. It is important to note that the word "server" is used in the figure to express that the Adaptive Profiler and the Gateway are located in a central machine. The server does not actually export any service. The personal terminal, however, does export one synchronization service, shown in Figure 1 with the method setProfile(). In the server part, the two main components are the Adaptive Profiler and the Gateway. The first one is responsible of building the user profile, while the second takes care updating the user profiles to the mobile devices. The user interacts with the Adaptive Profiler through a web browser, either from a desktop computer or from the mobile phone itself. The relational database acts as the indirect communication means between the Profiler and the Gateway. The user profile is stored in a database that is accessible by both the Profiler and the Gateway. The Gateway is responsible of withdrawing the user profile (i.e. learner model) from the database and sending it to the mobile Personal Terminal. This communication is performed using web services. The web service at the mobile Personal Terminal implements a method setProfile() that is called by the Gateway to update the stored user model. The mobile Personal Terminal implements the modules to communicate with the server and with other peers. The module that takes care of the communication with the server implements the setProfile() service. The module that is responsible of the communication with peers looks for other terminals in the surroundings. Once a another terminal is located, communication is established between them in order to interchange the user profiles they store. This communication mechanism is based on Bluetooth providing a contextualized protocol for finding nearby complementary students.

3 Communication Server-Terminal As we have presented in the previous sections, the different interactions between the e-learning users and the modified ITS define the properties of their user profiles.

Context-Aware Combination of Adapted User Profiles for Interchange of Knowledge

785

These profiles are periodically updated by the Adaptive Profiler and need to be synchronized with the context-aware personal user application running on the user’s mobile device. Since mobile devices tend to only implement the consumer part for web-service based communications we have defined and implemented a complete environment for developing and executing web-service based server applications on limited mobile devices. This part of the system is based on a simplification of the J2EE Servlet API on top of which we define a SOAP processing Servlet capable of exporting concurrent web services. One of these web services is the user profile synchronization web service. As described in [3], we have defined and implemented a simplified Servlet API for mobile devices that concentrates on providing the basic functionality required to process HTTP requests. On top of the implementation of this Servlet API we have created the WebServiceServlet which implements the doGet() and doPost() methods to parse the SOAP part of a web service invocation. The main information contained in the web service invocation is the name of the operation to execute and the values of the parameters. The WebServiceServlet parses the XML content of the SOAP message, obtains the name of the operation, creates an array of arguments, instantiates the service class implementing the business logic of the web service and executes the associated method. The result generated is then encapsulated in a SOAP response message and sent back to the client. The UML sequence diagram in the invocation process is shown in Figure 2. We have included the implementation of the synchronization web service in order to show the entire invocation process.
  



    

!"

  

     # $  % &

' & ( &!"

Fig. 2. Synchronization process

The implementation of the synchronization web service contains the business logic for the communication between the Gateway and the mobile device. The class contains two main methods. The setProfile() method implements the synchronization protocol between the server and the mobile device. The call() method is needed to connect the synchronization web service class to the WebServiceServlet described in the previous sub-section in systems that do not provide introspection mechanisms (e.g. MIDP profile in J2ME).

786

S. Gutierrez-Santos et al.

4 Communication between the Terminals After interacting with the server, mobile students have their personal profiles synchronized in their mobile devices. The personal profile describes the strengths and weaknesses of the student. When different students get nearby each other either in class, in laboratories or even at the canteen, they may be interested in meeting other students with complementary profiles. We have implemented a Bluetooth based “communication with peers” module for mobile devices in MIDP. This module detects mobile devices near the student, validates that the discovered devices implement the profile matching service and exchanges the student profiles. If there are any students with appropriate complementary profiles, the module shows their details about them and their profiles in order to facilitate face-to-face interaction. The Bluetooth technology provides both the appropriate distance for the communication (showing details only of students a few meters away) and the appropriate service discovery mechanism to find the surrounding mobile personal terminals. Our implementation uses the DiscoveryAgent of the LocalDevice to continuously find devices near the student (we are only interested in devices that implement the profile matching service).

5 Conclusions and Future Work This paper presents a system that helps students finding other students with complementary profiles. The search is performed in short range, making it context-dependent and specially suited for blended learning scenarios in which students interact in classes, at the library, etc. Using context-aware technologies makes it possible to create a sort of virtual market of knowledge, in which students interchange what they know, but without the high cost of advertise themselves. The paper has presented the architecture of the system. The most important parts are the Adaptive Profiler (a modified ITS that builds the user profile) and the module of Communication with Peers at the personal terminal, that is responsible of locating other terminals and interchanging user profiles. Communication between the terminal and the server is also an important issue, which has made it necessary to create a web service infrastructure on the mobile terminal. The system assumes that students interact with the ITS mostly individually (e.g. from home), but have many opportunities to interact among themselves during the day (e.g. in the labs). We do not know yet the influence of non-technical factors (e.g. personal issues or likings) can influence the validity of our scenario. This demands further investigation.

Acknowledgements The work presented in this paper has been partially funded by the Spanish “Programa Nacional de I+D+I” by means of the project TIN2008-05163/TSI “Learn3: Towards Learning of the Third Kind”.

Context-Aware Combination of Adapted User Profiles for Interchange of Knowledge

787

References [1] Heinemann, A., Mühlhäuser, M.: Spontaneous Collaboration in Mobile Peer-to-Peer Networks. In: Steinmetz, R., Wehrle, K. (eds.) Peer-to-Peer Systems and Applications. LNCS, vol. 3485, pp. 419–433. Springer, Heidelberg (2005) [2] Mitchell, K., Race, N.J.P., Mccaffery, D., Mccaffery, D., Cai, Z.: Unified and Personalized Messaging to Support E-Learning. In: Fourth IEEE International Workshop on Wireless, Mobile and Ubiquitous Technology in Education, pp. 164–168 (2006) [3] Muñoz Organero, M., Delgado Kloos, C.: Web-Enabled Middleware for Mobile Devices. In: International Wireless Applications and Computing 2007 Conference, Lisbon, Portugal, July 6-8 (2007) [4] Murray, T.: Authoring Intelligent Tutoring Systems: An Analysis of the State of the Art. International Journal of Artificial Intelligence in Education 10 (1999) [5] Zhang, Y., Zhang, S., Vuong, S., Malik, K.: Mobile Learning with Bluetooth-Based E-Learning System. In: 2nd International Conference on Mobile Technology, Applications and Systems (2005)

ReMashed – Recommendations for Mash-Up Personal Learning Environments Hendrik Drachsler1, Dries Pecceu2, Tanja Arts2, Edwin Hutten2, Lloyd Rutledge2, Peter van Rosmalen1, Hans Hummel1, and Rob Koper1 Open University of the Netherlands, 1 Centre for Learning Sciences and Technologies & 2 Computer Science Department, PO-Box 2960, 6401 DL Heerlen, The Netherlands {hendrik.drachsler,lloyd.rutledge,peter.vanrosmalen, hans.hummel,rob.koper}@ou.nl, {pecceu,ekh.hutten,tg.arts}@studie.ou.nl

Abstract. The following article presents a Mash-Up Personal Learning Environment called ReMashed that recommends learning resources from emerging information of a Learning Network. In ReMashed learners can specify certain Web2.0 services and combine them in a Mash-Up Personal Learning Environment. Learners can rate information from an emerging amount of Web2.0 information of a Learning Network and train a recommender system for their particular needs. ReMashed therefore has three main objectives: 1. to provide a recommender system for Mash-up Personal Learning Environments to learners, 2. to offer an environment for testing new recommendation approaches and methods for researchers, and 3. to create informal user-generated content data sets that are needed to evaluate new recommendation algorithms for learners in informal Learning Networks. Keywords: recommender system, mash-up, personalisation, personal learning environments, MUPPLE, informal learning, emergence, learning networks.

1 Introduction Nowadays, Internet users take advantage of Personal Environments (PEs) like iGoogle or Netvibes to create a personal view on information they are interested in. The existence of PEs inspired researchers in Technology-Enhanced Learning (TEL) to explore this technology for learning purposes. As a consequence Personal Learning Environments (PLEs) were invented for learners [1, 2]. Because of the combination of various Web2.0 sources in a PLE they are also called Mash-Up Personal Learning Environments (MUPPLEs) [3]. MUPPLEs are a kind of instance of the Learning Network concept [4] and therefore share several characteristics with it. Learning Networks consist of user-generated content by learners who are able to create, comment, tag, rate, share and study learning resources. Due to the large amount of learning resources and learners the Learning Network can show emerging patterns. Learning Networks are from the bottom-up driven because their content is not created by paid domain experts but rather by their U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 788–793, 2009. © Springer-Verlag Berlin Heidelberg 2009

ReMashed – Recommendations for Mash-Up Personal Learning Environments

789

members. These networks explicitly address informal learning because no assessment or accreditation process is connected to them. MUPPLEs also support informal learning as they require no institutional background and no fees. Instead the focus is on the learner independent from institutional needs like student management or assessments. Although, they are most appropriate for informal learning, educational scenarios are imaginable where MUPPLEs become integrated into formal courses as well. MUPPLEs are used to combine different information from the web that is supportive to the individual learner regarding the personal competence development. Most of the time, the sources are free to use and selected by the learner. A common problem for PEs and MUPPLEs is the amount of data that is gathered in a short time frame. The learners can be overwhelmed by the information they receive or they might have problems selecting the most suitable learning resource for their personal competence development. Therefore, we developed a recommender system that offers advice to learners to find suitable learning resources for their individual competence development. The main purpose of recommender systems on the Internet is to pre-select information a user might be interested in. The motivation for a recommender system for MUPPLEs is to improve the ‘educational provision’; to offer a better learning goal attainment and to spend less time to search for suitable learning resources [5]. In the following section we first discuss related work (section two). After that we introduce the ReMashed system (section three) and finally discuss future research (section four).

2 Related Work Nowadays, ‘mashing’ information becomes a widely used activity on the Internet. Various tools (Yahoo Pipes, Dapper, Openkapow etc.) provide the opportunity to combine information from other websites in a new way. Users do not need special programming skills to use the tools in order to combine different Internet sources. The users can make advantage of public APIs of Web2.0 services and standardized XML formats like Jason to mash data in a new way. In TEL several European projects address these bottom-up approaches of creating and sharing knowledge. The TENCompetence project addresses learners in informal Learning Networks [6]. The iCamp project explicitly addresses research around MUPPLEs [3]. They created an easy programmable and flexible environment that allows learners to create their own MUPPLE for certain learning activities. However, these systems face the problem that the emerging behavior of these bottom-up approaches gathers large amounts of data. With the ReMashed system we want to offer navigation support for such emerging bottom-up MUPPLEs to help learners to find the most suitable data for their learning goals. In recommender system research, extensive studies is going on to take advantage of tags for recommendations [7, 8]. Single systems like Delicious or Flickr offer recommendations to their users based on their data and also researchers take advantage of single Web2.0 services to create recommender systems [9]. However, the combination of different Web2.0 services to recommend information based on mashed tag and

790

H. Drachsler et al.

rating data has not been attempted so far and especially not for learners in MUPPLEs. Thus, ReMashed offers a new approach by mashing data of learners from various Web2.0 services to provide pedagogical recommendations.

3 The ReMashed System A prominent example of ReMashed from a different domain is the MovieLens project created by the GroupLens research group. They offer a movie recommender service where people can rate movies and get recommendations for movies. Besides this attractive services GroupLens created a frequently used data set for the development of recommender systems and related research [10]. Likewise ReMashed has three main objectives: 1. to provide a recommender system for MUPPLEs to learners, 2. to offer an environment for testing new recommendation approaches and methods for researchers, and 3. to create informal user-generated-content data sets that are needed to evaluate new recommendation algorithms for learners in informal Learning Networks.

Fig. 1. The user interface of the ReMashed system. On the left side, the mashed information from delicious and blogs are shown. On the right side, the rating based recommendations for the current learner are presented.

In order to test our recommendation approach for MUPPLEs we designed a MashUp that enables learners to integrate their Web2.0 sources (see Fig 1). The system allows the learners to personalise emerging information of a community to their preferences. They can rate information of the Web2.0 sources in order to define which contributions of other members they like and do not like. ReMashed takes the preferences into account to offer tailored recommendation to the learner. ReMashed uses

ReMashed – Recommendations for Mash-Up Personal Learning Environments

791

collaborative filtering [11] to generate recommendations. It works by matching together users with similar opinions about learning resources. Each member of the system has a 'neighborhood' of other like-minded users. Ratings and tags from these neighbors are used to create personalised recommendations for the current learner. The recommender system combines tag and rating based collaborative filtering algorithms in a recommendation strategy. Such a recommendation strategy reacts on certain situations by using the most suitable recommendation technique. The recommendation strategy is triggered by certain pedagogical situations based on the profile of the learner or available learning resources [12]. In the initial state of ReMashed, learners have sign up for the system and have not rated any learning resources. ReMashed identifies the cold-start situation of the recommender system [11] and recommends resources based on tags of the Web2.0 sources of the current learner. It computes the similarity between the tag cloud of the current learner with other learners and learning resources. After the learner started to rate resources above a certain threshold a rating based Slope-One algorithm provides additional recommendations to the learner. ReMashed is an Open Source project based on PHP5, Zend Framework 1.7 with the Dojo Ajax framework, MySQL database, Apache Server and the Duine recommendation engine. ReMashed is following the Model-View-Controller programming concept and is therefore fully object oriented. It consists of five sub-systems (see Fig 2), a user interface, a data collector, a user logger, a recommender system and the Duine prediction engine [13].

Fig. 2. Technical architecture of the ReMashed system

792

H. Drachsler et al.

─ The User Interface is responsible for user interaction, authentication of users, registration of new users and updating of user data. ─ The Data Collector establishes the connection between the Web2.0 services and gathers new data into the ReMashed database via a CRON job that runs every hour. ─ The Logger offers logging methods to the other subsystems. It stores log messages and monitors user actions in the system. ─ The Recommender System composes the recommendations for every user and puts them into the database. It allows implementing new recommendation algorithms in PHP but it also provides a connection to the Duine 4.0 prediction engine based on JAVA that can be used to compute recommendations for the learning resource. ─ The Duine Prediction Engine offers extensive options for configuring various recommender algorithms. It provides a sample of most common recommendation algorithms that can be combined in algorithm strategies, thus it is possible to create new recommendation strategies that follow pedagogical rules. We tested the system in an usability evaluation in a group of 49 users from 8 different countries [14]. The evaluation phase ran for one month and was concluded with an online recall questionnaire. In that timeframe 4961 resources were collected, 420 resources were rated and 813 recommendations were offered. The overall satisfaction with the system was positive. Nevertheless, the participants suggested particular improvements we will take into account for the future development of the system.

4 Conclusions and Future Research This article presented the ReMashed system, an evaluation tool for recommender systems for learners in informal Learning Networks. The article showed the design and implementation of a recommender system for MUPPLEs. The future developments of ReMashed rely on an end-user perspective and on a researcher perspective. Regarding the end-user perspective ReMashed needs to integrate additional Web2.0 features (i.e. integrating social networks like facebook). This may improve the isolation of informal learners towards the organisation of learning communities. Retrieved information from social networks can be used to improve the recommendations and strengthen the communities; for instance, learners that have certain social relationships will likely want to share their learning resources with their community. The type of relationship between learners can affect which kinds of recommendations are given. In addition, ReMashed should provide a widget interface to enable learners to integrate recommendations from ReMashed into their MUPPLEs. Such a widget has to provide the recommendations and the possibility to rate learning resources to further personalise the needs of the learners. From a researcher perspective, ReMashed opens the possibility to provide usergenerated-content data sets of various domains. Comparable to the famous MovieLens data set, a standard for the evaluation and development of recommender system algorithm in TEL can be created. Further, when considering different ReMashed communities in health, education or public affairs, data sets from theses domains can be used to develop solutions for the cold-start problem of recommender system by providing an already rated data set for a particular domain.

ReMashed – Recommendations for Mash-Up Personal Learning Environments

793

Acknowledgement Authors’ efforts were (partly) funded by the European Commission in TENCompetence (IST-2004-02787) (http://www.tencompetence.org).

References 1. Liber, O., Johnson, M.: Personal Learning Environments. Interactive Learning Environments 16, 1–2 (2008) 2. Wild, F., Kalz, M., Palmer, M. (eds.): Mash-Up Personal Learning Environments. CEUR Workshop Proceedings Maastricht, The Netherlands, vol. 388 (2008) 3. Wild, F., Moedritscher, F., Sigurdarson, S.E.: Designing for Change: Mash-Up Personal Learning Environments. eLearning Papers 9 (2008) 4. Koper, R., Tattersall, C.: New directions for lifelong learning using network technologies. British Journal of Educational Technology 35, 689–700 (2004) 5. Drachsler, H., Hummel, H., Koper, R.: Identifying the Goal, User model and Conditions of Recommender Systems for Formal and Informal Learning. Journal of Digital Information 10, 4–24 (2009) 6. Wilson, S., Sharples, P., Griffith, D.: Distributing education services to personal and institutional systems using Widgets. In: Wild, F., Kalz, M., Palmer, M. (eds.) Mash-Up Personal Learning Environments, Proceedings of the 1st MUPPLE workshop. CEUR-Proceedings, Maastricht, The Netherlands,vol. 388 (2008) 7. Shepitsen, A., Gemmell, J., Mobasher, B., Burke, R.: Personalized recommendation in social tagging systems using hierarchical clustering. In: Recommender Systems 2008, pp. 259–266. ACM, New York (2008) 8. Symeonidis, P., Nanopoulos, A., Manolopoulos, Y.: Tag recommendations based on tensor dimensionality reduction. In: Recommender Systems 2008, pp. 43–50. ACM, New York (2008) 9. Garg, N., Weber, I.: Personalized, interactive tag recommendation for flickr. In: Recommender System 2009, pp. 67–74. ACM, New York (2009) 10. Sarwar, B.M., Karypis, G., Konstan, J., Riedl, J.: Recommender systems for large-scale e-commerce: Scalable neighborhood formation using clustering. In: Fifth International Conference on Computer and Information Technology (2002) 11. Herlocker, J.L., Konstan, J.A., Riedl, J.: Explaining collaborative filtering recommendations. In: Proceedings of the 2000 ACM conference on Computer supported cooperative work, pp. 241–250 (2000) 12. Drachsler, H., Hummel, H., Koper, R.: Personal recommender systems for learners in lifelong learning: requirements, techniques and model. International Journal of Learning Technology 3, 404–423 (2008) 13. Van Setten, M.: Supporting people in finding information. Hybrid recommender systems and goal-based structuring. Telematica Instituut Fundamental Research Series No. 016 (TI/FRS/016) (2005) 14. Drachsler, H., Peccau, D., Arts, T., Hutten, E., Rutledge, L., Van Rosmalen, P., Hummel, H., Koper, R.: ReMashed - An Usability Study of a Recommender System for Mash-Ups for Learning. In: 1st Workshop on Mashups for Learning at the International Conference on Interactive Computer Aided Learning, Villach, Austria (submitted)

Hanse 1380 - A Learning Game for the German Maritime Museum Walter Jenner and Leonardo Moura de Araújo HS Bremerhaven, An der Karlstadt 8, 27568 Bremerhaven, Germany [email protected], [email protected]

Abstract. In an one year lasting project at the University of Applied Sciences in Bremerhaven a digital learning game for the German Maritime Museum in Bremerhaven was developed. It is targeted to school pupils in the age between 10 and 14 and should explain the importance of the cog for trading activities between Hanse cities in the 14th century. More detailed learning objectives were defined through a survey with history teachers from Bremen. The historical research was done in cooperation with the museum. Another key-interest was the design and building of an easy-to-use and attractive computer terminal including a special control-interface for the game. The resulting game is evaluated in an user-test with 29 school pupil. It shows that the game makes fun and is easy to understand. Approx. 50% of the pupils achieved all learning objectives.

1

Game–Based Learning in a Museum

One part of the duty of a museum is to provide and transport information to the visitor1 . Traditional museum exhibits show parts and aspects of the topic the museum or the particular exhibition is dealing with. The visitor has a passive role and no possibility to "respond". Interactive exhibits, in contrast, enable the visitor to participate and explore actively the information provided by the museum. The learning effect can increase with interactive exhibits in so far that exhibitions can be more "entertaining" [1] as well as "inspire and provoke exploration ... and to tempt people to look more thoughtfully at traditional museum displays" [2]. Anne Fahy described it like that [3, p. 89]: Interactive Devices have an active and important role to play in the communication process. This is emphasized by research carried out by the British Audio Visual Society which showed that whilst we only remember 10 per cent of what we read, we remember 90 per cent of what we say and do (Bayard-White 1991). 1

See the definition of a museum by ICOM: http://icom.museum/statutes.html#3

U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 794–799, 2009. c Springer-Verlag Berlin Heidelberg 2009 

Hanse 1380 - A Learning Game for the German Maritime Museum

1.1

795

Game–Based Learning

Game–based learning means that learning content is embedded within a game. In the last years a lot of researches have shown that learning through games can have various advantages. Richard van Eck points out one advantage of games [4, p. 4]: Games embody well-established principles and models of learning. For instance, games are effective partly because the learning takes place within a meaningful (to the game) context. What you must learn is directly related to the environment in which you learn and demonstrate it; thus, the learning is not only relevant but applied and practiced within that context. Learning that occurs in meaningful and relevant contexts, then, is more effective than learning that occurs outside of those contexts, as is the case with most formal instruction. Van Eck stresses the advantage that within a game new knowledge is more meaningful as it can be applied directly. The success of a certain action or strategy is usually shown immediately. Another strength of game-based learning is that learning is joyful as it happens while playing. Traditional learning situations, like lectures in school or self-study from books have the negative picture of being boring and pupils have to be "forced" to learn (e.g. to pass exams). The motivation of playing computer games is much higher as playing is seen as pleasure and not as work. Malone and Lepper researched about what can people motivate to learn, and they have found out that many features found in games (like challenge and performance feedback) positively influence motivation for learning [5]. They differentiate between intrinsic and extrinsic motivation, whereas they define intrinsically motivated learning as learning that occurs in a situation in which the most narrowly defined activity from which the learning occurs would be done without any external reward or punishment. [5, p. 229] They state the hypothesis that intrinsically motivated learning will lead to better learning results. 1.2

Putting the Exhibits in Context

Historic exhibits are dead objects, they are no longer in use nowadays. It is hard to imagine, why certain objects were important in times which are completely different to the present. The conserved cog, which is the main attraction of the exhibition about medieval ships in the German Maritime Museum (GMM), is more then 500 years old and destroyed to a large extent. No doubt that it has an enormous historic value, but without the context of how it was used in the past it cannot be fully understood. Within a game the museum visitor can be enabled to experience the past and learn about the context in which the shown exhibits were used.

2 2.1

Restrictions Target Group

As a target group for the game, pupils aged between 10 and 14 years were taken.

796

2.2

W. Jenner and L. Moura de Araújo

Needs for a Terminal Game in a Museum

As the game should be played on a computer terminal within a museum, it must be easy to understand. A quantitative study by Fleck et al. [6] has shown that a typical museum visitor spends 1-2 minutes at a museum object. However, if the visitor is engaged within that time, the time at one exhibit can increase to 10-15 minutes. The same study has shown that labels and instructions for interactive exhibits are usually not read. Interactive exhibits are tried out directly and people just refer to the instructions if they fail. For a learning game in a museum that means that it is necessary to motivate the visitor within 1-2 minutes to play the game. Long instructions should be avoided and in contrast it should be possible to explore the game. To allow exploration of the game, it must be intuitive and easy to use (which also includes the computer terminal). Finally, the overall game time should not be longer than 10-15 minutes. To summarize, these three requirements were defined: – The game should start immediately. – A tutorial should make it possible to explore the game step by step. – Intuitive hardware controls should make the controlling as easy as possible.

3

Results

The final result is a simulation game. The player takes the role of a young captain of a cog, based in Lübeck, who has to sail and trade goods in the Northand Baltic Sea. The game time is limited to 5-10 minutes which correlates to one sailing season within the game. Roughly, the game can be divided into two different parts, one part is a sailing simulation which considers the special way of sailing in the medieval time. The player has to follow landmarks in order to find the next city, he2 can be attacked by pirates, and he depends on wind from the back, as cogs had a yardarm sail which required exactly that. The second part of the game happens when the player has arrived in a city (Fig. 1). He has to show his skills as a trader, by selling and buying goods. In order to show the devoutness of people in medieval times, it is also possible to donate money to the church. As the player donates more money his influence in the city increases, which has a positive effect on his final score. Also, if he donated enough money, the gods might help him when pirates attack. There is also a high score list of the ten best players, which should be a motivating reward. 3.1

Direct Start of the Game and the Tutorial

The game can be started very quickly—instead of presenting long instructions at the beginning, small junks of information are presented step by step. After 2

Although in this report the player (the user, etc.) if referred to in the male form, it is directed at both sexes.

Hanse 1380 - A Learning Game for the German Maritime Museum

797

Fig. 1. Trading part of the game in Lübeck. Important parts of city—as the church— are based on old drawings.

the player successfully finished one step in the tutorial the next step is shown. Therefor the new knowledge is connected to the current situation in the game and thus should be remembered easier. 3.2

Computer Terminal

To control the cog in the sailing simulation the player uses a miniature model of a capstan and a rudder. The design of the controls is connected to the real look of those instruments. Firstly, the mental mapping of the control to its corresponding function should be supported by that. Secondly, due to this similarity to the real instruments, the player also gets an impression how these instruments look like on cogs. Also, the whole terminal design looks like a small cog, which creates a more interesting atmosphere and invites people to use the terminal. Additionally the game uses a touchscreen for user-input. 3.3

User Test

With an unfinished prototype of the game a user test with 29 pupils fitting the target group was conducted. It tested if the pupils are able to understand the game and control the cog, if they like the game (and which parts of it) and if they achieve the learning objectives. Additionally it included questions about general usage of computer games.

798

W. Jenner and L. Moura de Araújo

Attitude Towards Computer Games. Some pupils play computer games daily and all of them play at least multiple times per week. Regarding the preferred genre no clear preference can be found. The games range from "shooting games" (in particular Counter-Strike), strategy games, racing games to simulation games (The Sims). Shooting games are more popular for boys (7 boys and 3 girls stated to play shooting games), whereas The Sims is only played by girls in this test group. The majority of the tested pupils have not played games in museums so far (21 of 29). Usability. In general the usability of the game was good. All of the pupils understood how to control the cog and they rated the difficulty of it with 2,213 . 89,29 % of the tested pupils understood what their task in the game is. 89,29% understood how the current time of the season is indicated. 72,41% understood how the damage of the cog is indicated. 96,55% understood how the wind is indicated. And 85,71% understood the landmarks. On the question how much they like the game and single parts of it (graphics, sound, dialogue, overall) an average of 2,164 was achieved. Learning Objectives. In general not all children achieved the learning objectives, which were requested in the post-interview. 89,66% of the pupils remembered at least one hanse city. The naming of correct products was more difficult, but the trading feature was not fully implemented in the test-version of the game. 44,83% of the pupil could name the correct duration of a trading season, but again the prototype was not finished regarding that aspect, so it is not a surprise to have this result. The century in which this game takes place was not remembered well, just 41% did so. The same percentage of pupil could name the trading alliance, this game is dealing with. As this knowlegde is not needed within the game, it supports the hypothesis that factual knowlegde, which is not applied in the game, is not remembered very well. Summary. A general positive result is that most pupils liked the game. An overall grade of 2,16 is promising. It shows that the game-play functions and that the goal to make a good game in general is reached. In particular the victory condition of the game is communicated well (89,29% understood it), which by supporting the competitive element is an important part of a game [7]. What is also very positive is that the vast majority understood the game itself and the interface very well.

4

Conclusion

Learning objectives need to be integrated strongly within the game. Information which is just provided but not needed to successfully finish the game will not 3 4

On a scale from 1 to 4, where 1 is too easy and 4 is too difficult. An a scale from 1 to 5, where 1 is very good and 5 is very bad.

Hanse 1380 - A Learning Game for the German Maritime Museum

799

be remembered. Roughly two different ways to integrate learning content can be observed. Firstly, content can be transported via rules. For example if the objective is that the player should know how long a trading season is, then the according game rule can stress that the player has to finish a task within one trading season. Another way to integrate a learning objective into a game is via a feature. An example used in this game are pirates. The according learning objective is to show the danger of pirates in the medieval time. It is implemented in a way that on special routes the players cog might be attacked by pirates. To survive the attack of pirates the player then has various possibilities which correlate to the possibilities that seamen had in medieval times. At the same time it got clear that information which is not directly integrated into the game is not remembered. Our tests have shown that not many children could remember the name of the famous trading union ("Hanse") although textual hints refer to it multiple times and also the name of the game itself "Hanse 1380" which is very prominently placed.

References 1. Witcomb, A.: Interactivity: Thinking beyond. In: Macdonald, S. (ed.) A Companion to Museum Studies, pp. 353–361 (2007) 2. Stevenson, J.: Getting to grips. Museums Journal, 30–32 (May 1994) 3. Fahy, A.: New technologies for museum communication. In: Hooper-Greenhill, E. (ed.) Museum, media, message, pp. 82–96. Routledge, London (2002) 4. Eck, R.V.: Digital Game-Based learning: It’s not just the digital natives who are restless. EDUCAUSE Review 41(2) (2006) 5. Malone, T.W., Lepper, M.R.: Making Learning Fun: A Taxonomic Model of Intrinsic Motivations for Learning. In: Conative and Affective Process Analyses. Aptitude, Learning, and Instruction, vol. 3 (1987) 6. Fleck, M., Frid, M., Kindberg, T., O’Brien-Strain, E., Rajani, R., Spasojevic, M.: From informing to remembering: ubiquitous systems in interactive museums. IEEE Pervasive Computing 1(2), 13–21 (2002) 7. Salen, K., Zimmerman, E.: Rules of Play: Game Design Fundamentals. MIT Press, Cambridge (2003)

A Linguistic Intelligent System for Technology Enhanced Learning in Vocational Training – The ILLU Project Christoph Rösener Fachrichtung 4.6 Angewandte Sprachwissenschaft sowie Übersetzen und Dolmetschen, Universität des Saarlandes, Bau A2 2, Postfach 15 11 59, D-66041 Saarbrücken [email protected]

Abstract. In this paper I will describe a linguistic intelligent software system, using methods from computational linguistics, for the automatic evaluation of translations in online training of interpreters and translators. With this system the students gain an online interface offering them proper translation training. The main aim in developing such a system was to create an e-learning unit which allows the students to translate a given text in a special online environment and afterwards receive an automatic evaluation of the entered translation from the system. This is done on a computational linguistics basis using special analyzing software, model solutions and stored classifications of typical translation mistakes. Keywords: Vocational training, Language Learning, Natural Language Processing.

1 Introduction The types of interactive e-learning units used in the vocational training of translators and interpreters are currently limited by the technical possibilities provided by various e-learning systems. On the one hand there are e-learning units where users can obtain an automatic evaluation that is performed by the system. On the other hand, the evaluation of the texts is done by tutors. In the aforementioned case the given data is initially sent to the relevant tutor. After the evaluation of the texts by the tutor the results are sent back to the students or stored in an online rating system. If the elearning unit offers automatic evaluation by the system, the variety of units is very limited. In most cases the units are term and definition questions, multiple choice exercises, cloze units, exercises to reconstruct text or word order etc. But there is one thing all these exercises have in common: it is not possible to automatically evaluate free text. These texts can only be evaluated by a tutor. An automatic evaluation of free texts with regard to the quality of language and translation is not yet available1. 1

Approaches such as NIST [4], BLEU [5], Levenshtein are based on a measurement of character string similarity. Thus they are not really a yardstick for translation quality.

U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 800–805, 2009. © Springer-Verlag Berlin Heidelberg 2009

A Linguistic Intelligent System for Technology Enhanced Learning

801

2 Description In this paper I will describe an intelligent software system, using methods from computational linguistics, which is able to evaluate free text translations record-by-record automatically. In addition the system is able to give qualified feedback for each mistake found automatically. The process-scheme is shown in Figure 1.

Fig. 1. Process-scheme

3 Requirements For the successful implementation of such a system certain requirements were necessary. These included as the core the linguistic resources. Furthermore it was necessary to provide additional resources, including the source texts and possible model translations as well as examples of possible mistakes. A differentiated error code and corresponding feedback texts were also required and material about special translational problems. For an initial automatic evaluation of the posted translation commercial spell and grammar checkers are used. For a more profound analysis the posted text is morphosyntactically and semantically analysed. For this also, depending on the source language, various existing software packets are used. Finally special software for the comparison between the analysed translation posted by the students and the stored model solutions and examples of possible mistakes had to be developed within the project. In the process both model solutions as well as possible mistakes are stored in the system. A consistent, differentiated error code, which describes precisely the various mistake-scenarios, provides the basis for detailed feedback messages to the students. The system was initially intended to focus only on special translational problems of a certain language pair. Therefore it was necessary to provide material for these specific problems together with corresponding examples.

802

C. Rösener

4 Approach For the prototypical system (special translational problems E->D and F->D respectively) as a spell and grammar checker the existing software "Duden Korrektor Plus" of the Duden Verlag Mannheim is used. This software provides spell and grammar checking in consideration of the context. In detail the system offers correction of typing errors, spelling of hyphenated words, upper and lower case, compound or separate spelling, abbreviations, punctuation, mistakes in congruency, typography and regimen. This is done on the basis of the standard Duden dictionaries and books of reference [1]. For the morphosyntactic and semantic analysis of the posted translation the program MPRO is used in the ILLU system. MPRO is a software package for the morphosyntactic and semantic analysis of texts, which was developed by the Institute of Applied Information Science (IAI) in Saarbrücken. The program assigns a bundle of linguistic information to every recognized character string of a text. Normally the basic form (citation form) and part of speech (noun, verb, adjective etc.) are generated. Furthermore MPRO provides information about the inflection (case, number, gender, tense, person) as well as the structure of a word. For so-called "meaningful words" (nouns, adjectives, verbs, adverbs) the program also provides a semantic class. The assigned information is added to each string in form of a feature bundle. For the analysis of a word MPRO uses a dictionary of morphemes. The dictionary for German presently contains about 90,000 entries [2].

5 Comparison Module Due to the morphosyntactic and semantic analysis there are many features available for the comparative operation between the posted translation and the stored model solutions and possible mistakes. At word level the most important are the original string and the basic form, case, number, gender, tense and part of speech. At sentence level there are some more, e.g. word occurrence, word order, marking of phrases or sentences to name but a few. For the comparison operation it was necessary to define distinct parameters on the basis of which the comparison is made. On the one hand the feature bundles which are used for the comparison had to be defined. After that it was essential to define a method to compute a measure of similarity between the posted text and the stored model solutions and possible mistakes. Initial tests led to the implementation of a prototypical comparison module. The program computes whether certain feature bundles between two structures are identical or not. Depending on the various linguistic features this is done using different strategies to find the differences between the structures. Finally the various mistakes, if any, are determined and the result is sent to the next module. A differentiated definition of possible types of mistakes and their classification was one of the basic requirements of the system. Here the complexity of the error code corresponds directly with the quality of the system. The more differentiated the error code, the more powerful the system is. It is however not necessary to redefine everything. In the past many research projects have dealt with typical translational mistakes. Some of the material that was acquired in these projects was used for the prototypical system2. 2

The material acquired e.g. in the MeLLange project [3].

A Linguistic Intelligent System for Technology Enhanced Learning

803

Fig. 2. Detailed process sequence (translation E->D or F->D respectively)

The implementation of rules for the determination of mistakes was very labourintensive at the beginning of the project. But together with the aforementioned comparison operation these rules are responsible for the quality. The more differentiated the rules for a certain translation and the corresponding model solutions and possible mistakes, the more high-quality the system is. Beside rules based on the morphological, syntactic and semantic level (e.g. false verb, false relative pronoun etc.) it is also possible to implement rules which are sentence specific (e.g. changed constituents, word occurrence). If the topic of a certain unit is a particular translational problem, it is also possible to define specific rules for this. So far only rules on the morphological and syntactic level have been implemented. One of the ideas of the project is, that after initially collecting all rules as singular rules per text and translation, perhaps at a later date specific rules can be summarised to more abstract rules. Additionally this might be a chance to gain interesting results for translation studies. After the translational mistakes have been precisely determined by the comparison operation the corresponding feedback messages are sent back to the students. After processing one sentence the messages are given back to the students. Until now there is a fixed set of possible feedback messages implemented. But there is no restriction concerning the form of the feedback messages. It is for example possible to store not only detailed feedback messages for specific translational problems. In the future whole e-learning units and links to special phenomena and further literature can be provided.

6 Examples and Preliminary Results In Figure 3 an example is given to show how the system works. The original text in the source language is shown in the text field "Originalsatz". The text field "Lösung" contains a possible model translation, which is shown to the students on demand. The textfield "Lösungshinweis" contains advice for a possible solution and is also shown to the students on demand. Further down the corresponding model solutions and

804

C. Rösener

Fig. 3. User interface of the prototypical System (tutor interface; translation E->D)

possible mistakes in German are entered into the system. Due to linguistic intelligence this is possible on a phrase basis. This provides more possible combinations and therefore variety for possible translations. The system has the basic strategy of identifying first correct and false solutions. If this process is finished and none of the stored model solutions or possible mistakes correspond with the posted translation, the system gives feedback to check the translation again. At the same time the students can on demand obtain advice about a possible solution as well as a model solution for the current translational problem. And simultaneously the possibility of a separate evaluation of the posted translation by a tutor is given within the frame of parallel translation lessons or via email.

7 Evaluation and Conclusion The advantage of interactive e-learning units for translators and interpreters is, as for all e-learning systems, their availability. It is an additional e-learning possibility, which the students can use independent of time and place. A further advantage is that it also reduces the workload of the lecturers. Within the translation lessons only special translational problems need to be covered. No more time need be spent on spelling and grammar mistakes. These mistakes have already been corrected automatically by the system. Furthermore, interactive e-learning units for translators and interpreters are particularly suitable for the consolidation of special translational problems. Special translational phenomena can be explained by model sentences and texts. With the help of a detailed feedback system additional material can be provided for the students. Here the system can be constructed in a modular way and used in addition to the translation lessons, where attendance for students is obligatory. In implementing

A Linguistic Intelligent System for Technology Enhanced Learning

805

the present prototype the implementation of the rules for the comparison module turned out to be difficult. This requires further analysis. Perhaps the use of certain existing methods, e.g. "fuzzy-match" techniques of TM Systems is a solution to this problem. A further difficulty turned out to be the storing of model solutions and possible mistakes. During the implementation of the current system various interfaces were developed. Finally now both these things are possible with the help of a special tutor interface, which is easy to use and therefore suitable also for lecturers without any programming knowledge. Another disadvantage of the outlined system is that the automatic evaluation of translations is only possible record-by-record. Thus not all possible versions of a translation can be covered. Perhaps this problem can be solved in the near future by techniques used already in the alignment process of Translation Memory systems. However, it has been demonstrated that the development of linguistic intelligent interactive e-learning units used in the vocational training of translators and interpreters is possible. Further tests with the prototype will need to demonstrate whether the students accept such systems. Certainly the potential effects of such a system on the elearning community are obvious: When it is possible to evaluate free text with relation to certain stored model solutions or other requirements, the system represents a powerful software tool which can be used not only in the vocational training of translators and interpreters, but also in other areas, where the possibility of free text input is desirable.

References [1] [2] [3] [4] [5]

Duden Verlag Mannheim. Bücher und Software. Bibliographisches Institut & F. A. Brockhaus AG (2007), http://www.duden.de/produkte/ Maas, H.-D.: Multilinguale Textverarbeitung mit MPRO. In: Lobin, G. (ed.) Europäische Kommnikationskybernetik heute und morgen. KoPäd, München (1998) MeLLange: Multilingual eLearning in LANGuage engineering (2007), http://mellange.eila.jussieu.fr/ NIST: Automatic Evaluation of Machine Translation Quality Using N-gram CoOccurrence Statistics, Automatic Evaluation of MT Quality, NIST (2005) Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: BLEU: A Method for Automatic Evaluation of Machine Translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (2002)

e³-Portfolio – Supporting and Assessing Project-Based Learning in Higher Education via E-Portfolios Philip Meyer, Thomas Sporer, and Johannes Metscher Institute for Media and Educational Technology Universitätsstr. 2, 86135 Augsburg, Germany [email protected]

Abstract. e³-portfolio is a software tool which supports learning and working in student project groups. Besides features for collaboration via social media, the software offers an electronic portfolio system. The e-portfolio helps to integrate informal project-based learning into the formal curriculum of higher education. This paper gives an overview of how the software tool is designed and relates the design to the underlying didactic concept. Keywords: Project-based learning, e-portfolio, e-collaboration, e-assessment.

1 Introduction Practical experiences and key competencies are becoming increasingly important for students in today’s working life. One way to attain those competencies is to take part in self-organized project groups at the periphery of their university. Here students learn to solve problems and become part of a community of practice [1]. At the University of Augsburg students can get such extra-curricular learning activities accredited through the study programme "Problem Solving Competencies" [2]. This study programme builds on the reflection of the student’s experiences via e-portfolios and focuses the assessment on the articulation of the competencies that the students acquire [3]. The organisation of that study programme is facilitated by the software tool outlined below.

2 Description of the Software Tool The technological basis of the software tool is the open-source platform and content management system Drupal (www.drupal.org). The various features of Drupal are utilised to foster collaboration of the users. The tool is structured into three parts: Students organise their project groups in the community area. They create their journals and project reports via the portfolio area. Further it structures the assessment process of the student’s learning achievements and their accreditation in the assessment area [4]. When visiting the website (www.begleitstudium-problemloesekompetenz.de), a welcome page informs the users about the aims of the study programme (e.g. press U. Cress, V. Dimitrova, and M. Specht (Eds.): EC-TEL 2009, LNCS 5794, pp. 806–810, 2009. © Springer-Verlag Berlin Heidelberg 2009

e³-Portfolio – Supporting and Assessing Project-Based Learning in Higher Education

807

releases, reference to the project blog, interviews with participants). The three main areas, however, can only be accessed to their full extend after registration.. In the following sections these areas are described – in their functionality for unregistered and already registered users. 2.1 Community Area For unregistered users the community area gives an overview of the project groups that take part in the study programme (e.g. campus magazine or campus radio). Each project has a public space where they can present themselves, the project ideas (e.g. via video interviews with the project leaders) and descriptions of the activities participants can take on. Project groups can adapt this public area to their "corporate design" to ensure the identity of the project is maintained. News about the project can also be published to inform others about the initiative. After registration the internal community area provides access to all the groups of which the user is a member or owner. Registered users can create new project groups or join existing groups by request.

Fig. 1. Overview of the features in the community area

Additionally the community area features various tools for project and knowledge management. There is a community blog where discussions within the group can take place and by which the group can organize their collaboration by announcing important dates and deadlines. Moreover there is a wiki for each group which offers the functionality to share knowledge between the group members. And there is a document repository which allows to publish meeting protocols and to share files.

808

P. Meyer, T. Sporer, and J. Metscher

2.2 Portfolio Area In its unregistered view the portfolio area shows exemplary profiles from participants of the study programme. In short video interviews participants describe what motivated them to attend the project group and what is special about being part of their project. Aside you can view some personal information about the participants and browse through their learning journals. After registration the participants can write their project diary in form of a blog in the portfolio area. Here students periodically reflect on the experiences they make during their project activities. The reflection process is scaffolded by some guiding questions like “What happened since the last entry in my project diary?” or “What are my thoughts and feelings as to the current situation in the project?”. At the end of each semester students can create a project report. This report summarises the salient events during the participation in the project and presents them in form of a learning history.

Fig. 2. Overview of the features in the portfolio area

The portfolio area also helps the students to keep track of all their diary entries and project reports. Here they can collect all these items and prepare them for submission to the assessment area. 2.3 Assessment Area In its public view the assessment area is rather unspectacular. It shows a description of what this area is supposed to offer, namely a space for registered users to submit project diaries and reports and to get feedback for their learning and working achievements. The registered view of the assessment area thus enables the organisation of all the achievements that have been performed in the context of the study programme and their accreditation in the formal curriculum.

e³-Portfolio – Supporting and Assessing Project-Based Learning in Higher Education

809

Fig. 3. Overview of the features in the assessment area

After the participant has completed all building blocks of the study programme, she can obtain the certificate "Problem Solving Competencies". If the student wants to have the credit points that were gained during the project work accredited in the formal studies, the project report has to be handed in via the assessment area and becomes graded by the coordinator of the co-curricular study programme.

3 Underlying Didactical Concept The platform was designed to support a didactical concept which focuses on the integration of informal learning activities into the formal university curriculum [4]. The three main areas described above therefore differ in the degree of formalisation of the learning setting (see Fig. 4). The community area is very close to the practice of the project group as an informal learning community. Students discuss, collaborate and share their experiences, but this all happens on an informal level with a low degree of formalisation. In the portfolio area the students begin to formalise their experiences by writing them down in a personal diary. But this still happens close to the context of what is actually going on in the project practice and the involvement of theoretical assumptions is marginal. Finally, in the assessment area, the students decide which of the texts and artifacts they created during the project work are worth being submitted to the programme coordinators. The students choose entries, where the reference to the goals of their formal studies is obvious. They also make assumptions in their project report on how their project participation and their formal studies relate to one another. In figure 4 the portfolio-based assessment strategy is summarised: The students collect their working achievements and diary entries in the working portfolio. At the end of the semester they combine these artifacts to a coherent learning history in the story portfolio. Via the test portfolio they finally argue what competencies they acquired in a project report and show how their experiences relate to their formal studies.

810

P. Meyer, T. Sporer, and J. Metscher

Fig. 4. Areas of e³-portfolio and blended assessment strategy

4 Conclusion and Future Work This article described the features of a software tool which is currently being used at the University of Augsburg. The software tool supports the collaboration of student’s project groups and it offers a way to integrate informal learning activities into the formal curriculum of higher education via a blended assessment strategy based on eportfolios. Recently, evaluation studies have shown that students want more interconnectedness between the different areas of the software tool. Especially in regard to the portfolio and the assessment area the current state of implementation lacks the functionality to give feedback on the content provided by the participants. Due to the collaborative nature of the community area there is already a lot of interactive functionality present. However, we are planning to introduce even more features in the community area that can support group collaboration.

References 1.

2.

3.

4.

Dürnberger, H., Sporer, T.: Selbstorganisierte Projektgruppen von Studierenden: Neue Wege bei der Kompetenzentwicklung an Hochschulen. Erscheint im Tagungsband der 14. Europäischen Jahrestagung der Gesellschaft für Medien in der Wissenschaft. Waxmann, Münster (in press) Sporer, T., Reinmann, G., Jenert, T., Hofhues, S.: Begleitstudium Problemlösekompetenz (Version 2.0): Infrastruktur für studentische Projekte an Hochschulen. In: Merkt, M., Mayrberger, K., Schulmeister, R., Sommer, A., Berk, I.v.d. (eds.) Studieren neu erfinden – Hochschule neu denken, pp. 85–94. Waxmann, Münster (2007) Reinmann, G., Sporer, T., Vohle, F.: Bologna und Web 2.0: Wie zusammenbringen, was nicht zusammenpasst? In: Keil, R., Kerres, M., Schulmeister, R. (eds.) eUniversity - Update Bologna. Education Quality Forum. Bd. 3, pp. 263–278. Waxmann, Münster (2007) Sporer, T., Jenert, T., Meyer, P., Metscher, J.: Entwicklung einer Plattform zur Integration informeller Projektaktivitäten in das formale Hochschulcurriculum. In: Seehusen, S., Lucke, U., Fischer, S. (Hrsg.) DeLFI 2008. Die 6. e-Learning Fachtagung Informatik der Gesellschaft für Informatik e.V. Gesellschaft für Informatik, Bonn (2008)

Author Index

Abel, Fabian 154 Abel, Marie-H´el`ene 682 Adam, Jean-Michel 602 Aehnelt, Mario 639 Ala-Mutka, Kirsti 350 Alario-Hoyos, Carlos 621 Alavi, Hamed S. 211 Allmendinger, Katrin 344 Arrebola, Miguel 127 Arts, Tanja 788 Asensio-P´erez, Juan I. 621 Avouris, Nikolaos 267 Barnes, Sally-Anne 700 Beekman, Niels 160 Beham, G¨ unter 73 Belgiorno, Furio 712 Benjemaa, Abir 763 Benz, Bastian F. 521 Berkani, Lamia 664 Betbeder, Marie-Laure 196 Bevan, Jon 7 Bielikov´ a, M´ aria 99, 492 Bimrose, Jenny 700 Bitter-Rijpkema, Marlies 732 B¨ ohnstedt, Doreen 521 Borek, Alexander 391 Borthwick, Kate 127 Bote-Lorenzo, Miguel L. 621 Boticario, Jesus G. 596 Bouchon-Meunier, Bernadette 633 Bourguin, Gr´egory 405 Bouzeghoub, Amel 763 Boytchev, Pavel 549 Breuer, Ruth 166 Brown, Alan 700 Brusilovsky, Peter 88 Budd, Jim 37 Buffat, Marie 763 Cao, Yiwei 166 Cerioli, Maura 651 Charlier, Bernadette 298, 304 Chatti, Mohamed Amine 310 Chen, Hsiu-Ling 706

Chikh, Azeddine 664 Chou, C. Candace 751 Chounta, Irene-Angelica 267 Condamines, Thierry 273 Corness, Greg 37 Courtin, Christophe 572 Cress, Ulrike 254, 338 Cristea, Alexandra I. 7 Daele, Amaury 298, 304 de Hoog, Robert 639 de la Fuente Valent´ın, Luis 744 Delgado Kloos, Carlos 744, 782 Demetriadis, Stavros N. 535 Derntl, Michael 447 De Troyer, Olga 627 Dietrich, Michael 688 Dillenbourg, Pierre 211 Div´eky, Marko 492 Drachsler, Hendrik 788 Dubois, Michel 602 Duval, Erik 757 Emin, Val´erie 462 Esnault, Liliane 304 Ewais, Ahmed 627 Fern´ andez-Manj´ on, Baltasar Ferrari, Anusca 350 Ferraris, Christine 379 ´ Figueira, Alvaro 776 Friedrich, Martin 507

725

Gaˇsevi´c, Dragan 140, 441 Gegenfurtner, Andreas 676 Giretti, Alberto 112 Glahn, Christian 52 Goguadze, Georgi 688 G´ omez-Albarr´ an, Mercedes 645 G´ omez-S´ anchez, Eduardo 621 Gribaudo, Marco 719 Gu´eraud, Viviane 462, 602 Gutierrez-Santos, Sergio 556, 782 Hamann, Karin 344 Hatala, Marek 37, 140, 441

812

Author Index

Heintz, Matthias 584 Held, Christoph 254 Hendrix, Maurice 7 Herder, Eelco 240 Hesse, Friedrich W. 5 Hoppe, H. Ulrich 365 Howard, Yvonne 127 Hsiao, I-Han 88 Hummel, Hans 788 Hutten, Edwin 788 Indriasari, Theresia Devi Ivanovi´c, Mirjana 657

Lu, Tianxiang 67 Lucas, Margarida 325 Luong, The Nhan 769

310

Jahn, Marco 507 Jarke, Matthias 310 Jenner, Walter 794 Jeremi´c, Zoran 441 Jim´enez-D´ıaz, Guillermo 645 Jovanovi´c, Jelena 140, 441 Kahrimanis, Georgios 267 Kalz, Marco 160 Kaplan, Frederic 211 Karabinos, Michael 391 Karsten, Anton 160 Kawase, Ricardo 240 Kempf, Fabian 344 Kennedy-Clark, Shannon 609 Klamma, Ralf 166 Kleinermann, Frederic 627 Koper, Rob 160, 477, 788 Kovatcheva, Eugenia 549 Krauß, Matthias 226 Kravcik, Milos 52 Krogstie, Birgit R. 418 Kump, Barbara 73 Law, Effie Lai-Chong 181 Leblanc, Adeline 682 Leclet, Dominique 405 Lecocq, Claire 763 Lehtinen, Erno 676 Lejeune, Anne 602 Lewandowski, Arnaud 405 Ley, Tobias 73, 700 Lindstaedt, Stefanie N. 73, 639, 700 Lopes Gan¸carski, Alda 763 Lopist´eguy, Philippe 769 Loughin, Tom 37

Magoulas, George D. 106 Maillet, Katherine 763 Malandrino, Delfina 712 Malzahn, Nils 365 Mandran, Nadine 602 Manno, Ilaria 712 Marenzi, Ivana 154 Markus, Thomas 385 Marquesuza` a, Christophe 769 Marsala, Christophe 633 Mart´ınez-Ortiz, Iv´ an 725 Martel, Christian 379 Mavrikis, Manolis 556 Mazarakis, Athanasios 615 McLaren, Bruce M. 391, 688 McSweeney, Patrick 127 Meier, Anne 267 Melis, Erica 67, 688 Memmel, Martin 112 Metscher, Johannes 806 Meyer, Ann-Kristin 688 Meyer, Philip 806 Millard, David E. 127 Mohabbati, Bardia 37 Monachesi, Paola 385 Moreira, Ant´ onio 325 Mossel, Eelco 385 Moura de Ara´ ujo, Leonardo 794 Muise, Kevin 37 Mu˜ noz-Organero, Mario 782 Nejdl, Wolfgang 154, 240 Neumann, Susanne 447, 477 Nguyen-Ngoc, Anh Vu 181 Niemann, Katja 507 Nikolova, Nikolina 549 Nivala, Markus 676 Nodenot, Thierry 769 Oberhuemer, Petra 447, 477 Ouari, Salim 379 Oudshoorn, Diederik 160 Palmieri, Giuseppina 712 Papadopoulos, Pantelis M. 535 Pardo, Abelardo 744, 782 Pearce, Darren 22

Author Index Pecceu, Dries 788 Pellens, Bram 627 Pemberton, Lyn 226 Pernin, Jean-Philippe 462 Pirolli, Peter 1 Poulovassilis, Alexandra 22, 106 Pozzi, Francesca 670 Punie, Yves 350 Putnik, Zoran 657 Putois, Georges-Marie 633 Quenu-Joiron, Celine

273

Reffay, Christophe 196 Rensing, Christoph 521 Ribaudo, Marina 651 Riege, Kai 226 R¨ osener, Christoph 800 Ruiz-Iniesta, Almudena 645 Rummel, Nikol 267 Rutledge, Lloyd 788 S¨ alj¨ o, Roger 676 Santos, Olga C. 596 Sauvain, Romain 283 Savin-Baden, Maggi 433 Scarano, Vittorio 712 Scheffel, Maren 507 Schmitz, Bernhard 521 Schmitz, Hans-Christian 507 Schoefegger, Karin 700 Scholl, Philipp 521 Schr¨ oder, Svenja 365 Schw¨ ammlein, Eva 338 Selmi, Mouna 763 Sendova, Evgenia 549 Sharples, Mike 3 Siadaty, Melody 140 Sie, Rory L.L. 732 Sierra, Jos´e-Luis 725 ˇ Simko, Mari´ an 99 Sloep, Peter B. 732 Smits, David 7 Sosnovsky, Sergey 88 Spada, Hans 267

Specht, Marcus 52, 310 Sporer, Thomas 806 Stamelos, Ioannis G. 535 Stefanova, Eliza 549 Steinmetz, Ralf 521 Szilas, Nicolas 283 Talbot, St´ephane 572 Talon, B´en´edicte 405 Tanenbaum, Karen 37 Ternier, Stefaan 52 Torniai, Carlo 140 Tosatto, Claudio 719 Tran, Tri Duc 633 Tsovaltzi, Dimitra 688 Ullrich, Carsten

67

Van Bruggen, Jan 160 Van Labeke, Nicolas 106 Van Rosmalen, Peter 160, 788 Varella, Stavroula 127 Vatrapu, Ravi K. 694 Vega-Gorgojo, Guillermo 621 Verbert, Katrien 757 Verpoorten, Dominique 52 Vignollet, Laurence 379 Villiot-Leclercq, Emmanuelle 379 Voyiatzaki, Eleni 267 Vuorikari, Riina 166 Wakkary, Ron 37 Weber, Nicolas 700 Wiley, David 757 Winter, Marcus 226 Wodzicki, Katrin 338 Wolpers, Martin 112, 507 Yaron, David

391

Zdravkova, Katerina 657 Zeiliger, Romain 304 Zendagui, Boubekeur 738 Zerr, Sergej 154 Ziebarth, Sabrina 365

813