Technologies for E-Learning and Digital Entertainment: First International Conference, Edutainment 2006, Hangzhou, China, April 16-19, 2006, Proceedings (Lecture Notes in Computer Science)

Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen

Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen University of Dortmund, Germany Madhu Sudan Massachusetts Institute of Technology, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Moshe Y. Vardi Rice University, Houston, TX, USA Gerhard Weikum Max-Planck Institute of Computer Science, Saarbruecken, Germany

3942

Zhigeng Pan Ruth Aylett Holger Diener Xiaogang Jin Stefan Göbel Li Li (Eds.)

Technologies for E-Learning and Digital Entertainment First International Conference, Edutainment 2006 Hangzhou, China, April 16-19, 2006 Proceedings

13

Volume Editors Zhigeng Pan Zhejiang University, Hangzhou, China E-mail: [email protected] Ruth Aylett Heriot-Watt University, Edinburgh, UK E-mail: [email protected] Holger Diener Fraunhofer Institute for Computer Graphics, Rostock, Germany E-mail: [email protected] Xiaogang Jin Zheijinag University, Hangzhou, China E-mail: [email protected] Stefan Göbel ZGDV e.V. - Computer Graphics Center, Darmstadt, Germany E-mail: [email protected] Li Li Hangzhou Dianzi University, Hangzhou, China E-mail: [email protected]

Library of Congress Control Number: 2006923674 CR Subject Classification (1998): K.3.1-2, I.2.1, H.5, H.3, I.3 LNCS Sublibrary: SL 3 – Information Systems and Application, incl. Internet/Web and HCI ISSN ISBN-10 ISBN-13

0302-9743 3-540-33423-8 Springer Berlin Heidelberg New York 978-3-540-33423-1 Springer Berlin Heidelberg New York

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. Springer is a part of Springer Science+Business Media springer.com © Springer-Verlag Berlin Heidelberg 2006 Printed in Germany Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper SPIN: 11736639 06/3142 543210

Preface

Edutainment 2006 is an international conference on research and development on e-learning and digital entertainment. The main purpose of the conference is the discussion, information and opinions exchange on the development and use of such systems. It provides a very interesting opportunity for researchers who want to attend or present communications at these events. The conference includes plenary invited talks, workshops, tutorials, paper presentation tracks and panel discussions. "Edutainment" is a recently coined term that expresses the union between education and entertainment in a television program, game or website. Today, the world of multimedia games and activities is a place where education and entertainment meet. Edutainment has evolved as a prospering research topic banding together formerly disjoined disciplines stemming from education, entertainment as well as computer science. Thus, with this conference, we can bring people from different fields together to discuss techniques for e-learning and digital entertainment as well as about the future of edutainment. This conference developed from the previous Europe–China Workshop on E-learning and Games called Edutainment 2005 (April 28-March 2, 2005), which was originally based on the ELVIS project, an EU–Asia link project (Prof. Ruth Alyett was the project coordinator and Prof. Zhigeng Pan was the project leader). It was also a sub-event for celebrating the 30th Anniversary of EU China Diplomatic Relations. During this workshop, experts from home and abroad were invited to give keynote speeches, and about 90 people attended the workshop. At this year’s Edutainment 2006, an established and still–growing community of researchers gathered together to exchange results and visions. The Edutainment 2006 conference program provided traditional scientific talks, but also workshops, tutorials an a symposium. We received 467 submissions in total from 22 different countries, including China (mainland, Honk Kong, Taiwan), USA, UK, Germany, France, Australia, Canada, Switzerland, Korea, Japan and Malaysia. In total, 173 papers (with 121 regular papers, and 52 short papers) were accepted for this volume. The papers in this volume cover topics including: E-Learning Platforms and Tools, Learning Resource Management, Practice and Experience Sharing, E-Learning Standards, Mobile Learning, Education and Remote Classrooms, Effectiveness of VR for Education, Life–Long Learning, Collaborative Environments, Remote Group Simulations, Collaborative Learning, Virtual Reality in Education, Game Design and Development, Game Engine Development Game AI and Artificial Life, Game Physics, Game Rendering, Virtual Characters/Agents, Online/Mobile Game/Video Game, Storytelling and Game Narrative, Affective Interaction in Games, Digital Museum, Digital Heritage, Animation Techniques, Augmented Reality, and Mixed Reality. The wide range of questions, ideas, concepts and applications discussed in the contributions of this volume reflect the vitality and engagement of the e-learning and

VI

Preface

game communities and their neighboring disciplines. The current research situation in edutainment demands interdisciplinary cooperation and mutual stimulation. This accounts for the fact that some contributions address purely technological questions, whereas others present fundamental philosophical concepts. However, most authors search for a middle way that comprises both new technological and conceptual ideas. With the strong support of Springer, the proceedings of the subsequent Edutainment conferences will also be published in the Lecture Notes in Computer Science (LCNS) series. And we do hope in the future more and more authors will be involved in this conference and contribute more to the edutainment field. February, 2006

Zhigeng Pan Ruth Aylett Holger Diener Xiaogang Jin Stefan Göbel Li Li

Organization

Acknowledgements and Sponsoring Institutions The international conference series on Technologies for E-Learning and Digital Entertainment (called Edutainment) has been initiated by the DEARC (Digital Entertainment and Animation Research Centre in Zhejiang University) and VRMM (Virtual Reality and Multimedia Division) at the State Key Lab of CAD&CG in Zhejiang University. However, Edutainment 2006 was such a big success owing to the financial and practical support of various institutions. Sponsors • Zhejiang University • VR Committee of the China Society of Image and Graphics In cooperation with • Hangzhou National Animation Base, Zhejiang Province, China Organizers • State Key Lab of CAD&CG, Zhejiang University, China • DEARC, School of Computer Science, Zhejiang University, China Co-sponsors • INI-GraphicsNet • International Journal of Virtual Reality (IJVR) • IFIP SG 16 on Entertainment Computing • Nature Science Foundation of China • National Lab on Machine Perception, Peking University, China • Key Lab of VR Technique of MOE, Beihang University, China • Institute of Computer Application, Sun Yat-sen University, China • Hangzhou Dianzi University, China • Nanjing Normal University, China • The Hong Kong Polytechnic University, Hong Kong We would really like to thank all of them for offering the opportunity to organize Edutainment 2006 in a way that provided a diversified scientific and social program. Especially, we would like to thank all members of the International Program Committee and Organizing Committee for their great job in defining conference topics, reviewing the large number of submitted papers, and managing to put all the material together for this great event. In addition, this event also functioned as one of the sub-events for celebrating the 70th birthday of Prof. Shi, in honor of his contribution to the computer graphics community. We also would like to express our sincere thanks to Prof. Encarnação (the Editor-in-Chief of Computers & Graphics), for his strong support in organizing this international conference. Furthermore, Prof. Jose Encarnação arranged to have a

VIII

Organization

short special issue on edutainment in Computers & Graphics in support of this event and in celebration of the 70th birthday of Prof. Jiaoying Shi.

Committee Listings Conference Honorary Chairs Yunhe Pan, Zhejiang University, China Judith Brown, ACM SIGGRAPH, USA

John Staudhammer, IEEE, USA Lawrence Rosenblum, NSF, USA Conference Chairs Jiaoying Shi, Zhejiang University, China José Encarnação, INI-GraphicsNet, Germany Steve Cunningham, NSF, USA

Program Chairs Ruth Aylett, Heriot-Watt University, UK Holger Diener, Fraunhofer IGD, Rostock, Germany Zhigeng Pan, Zhejiang University, China

Workshop Chairs Miguel Encarnação, The IMEDIA Academy, USA Adrian David Cheok, NTU, Singapore

Tutorial Chairs Stefan Goebel, ZGDV, Germany Yiyu Cai, NTU, Singapore

Publicity Chairs Wolfgang Müller-Wittig, CAMT, NTU, Singapore Rynson Lau, City University of Hong Kong, China Martin Goebel, Germany

Financial Chair: Mingmin Zhang, Zhejiang University, China

Publication Chair: Li Li, Hangzhou Dianzi University, China

Organization

Organizing Co-chairs: Xiaogang Jin, Zhejiang University, China Yi Li, Nanjing Normal University, China Jiajun Bu, DEARC, Zhejiang University, China Kin-chuen Hui, The Chinese University of Hong Kong

Program Committee Isabel M. Alexandre, Portugal Ruth Alyett, UK Elizabeth Andre, Germany Steffi Beckhaus, Germany Mark Billinghurst, New Zealand Judy Brown, USA Yiyu Cai, Singapore Jim Chen, USA Dongyi Chen, China Carola Conle, Canada Nuno Correia, Portugal Adrian David Cheok, Singapore Sanjay G. Dhande, India Holger Diener, Germany Stepahne Donikian, France Miguel Encarnação, USA Bernd Froehlich, Germany Lisa Gjedde, Denmark Gernot Goebbels, Germany Stefan Goebel, Germany Martin Goebel, Germany Lynne Hall, UK Kin-chuen Hui, Honk Kong, China Masahiko INAMI, Japan Gangyi Jiang, China Xiaogang Jin, China Woochun Jun, Korea Hirokazu Kato, Japan Lars Kjelldahl, Sweden Christian Knöpfle, Germany Jarmo Laaksolahti, Sweden Rynson Lau, HK, China Jong Weon Lee, Korea Li Li, China Xuelong Li, UK Yi Li, China

Sandy Louchart, UK Lizhuang Ma, China Maic Masuch, Germany Wolfgang Müller-Wittig, Singapore Ryohei Nakatsu, Japan Stéphane Natkin, France Malcolm Padmore, UK Ana Paiva, Portugal Zhigeng Pan, China Edmond Prakash, Singapore Marc Prensky, USA G.W.M.Rauterberg, Netherlands Paul Richard, France Abdennour El Rhalibi, UK Albert “Skip” Rizzo, USA Judy Robertson, UK I-Fan Shen, China Ulrike Spierling, Germany Kurt Squire, USA Peter Stephenson, USA Gerard Subsol, France Mohd Shahrizal Sunar, Malaysia Franco Tecchia, Italy Naoko Tosa, Japan Ming Hsin Tsai, Taiwan, China Frederic Vexo, Switzerland Charlie C. L. Wang, Hong Kong, China Yangsheng Wang, China Yongtian Wang, China Kok-Wai Wong, Australia Enhua Wu, Macao, China Ruigang Yang, USA Gino Yu, HK, China Pao-Ta Yu, Taiwan, China

IX

Table of Contents

Keynote Speeches Abstracts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1

Invited Papers VR-Enabled Bio Edutainment Y.Y. Cai, C.W. Chan, B.F. Lu, C. Indhumathi, Y. Jiang . . . . . . . . . .

4

Learning Abstract Concepts Through Interactive Playing Jim X. Chen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5

Virtual Reality and Mixed Reality for Virtual Learning Environments Zhigeng Pan, Adrian David Cheok, Hongwei Yang, Jiejie Zhu, Jiaoying Shi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

6

E-Learning Platforms and Tools Intelligent Tutorial Planning Based on Extended Knowledge Structure Graph Zhuohua Duan, Yunfei Jiang, Zixing Cai . . . . . . . . . . . . . . . . . . . . . . . .

7

Perceptual Evaluation of Pronunciation Quality for Computer Assisted Language Learning Chao-Lei Li, Jia Liu, Shan-Hong Xia . . . . . . . . . . . . . . . . . . . . . . . . . . .

17

Designing Social Navigation for a Virtual Community of Practice Wen Xu, Karel Kreijns, Jun Hu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

27

Applying the Adaptive Learning Material Producing Strategy to Group Learning Bin-Shyan Jong, Te-Yi Chan, Yu-Lung Wu, Tsong-Wuu Lin . . . . . . .

39

Comparison of Two Learning Models for Collaborative e-Learning Chan Jung Park, Jung Suk Hyun . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

50

On Demand E-Learning with Service Grid Technologies Zongwei Luo, Yulian Fei, Junyuan Liang . . . . . . . . . . . . . . . . . . . . . . . .

60

XII

Table of Contents

Architecture of an Authoring System to Support Interactive Contents Creation for Games/E-Learning Kojzi Miyazaki, Yurika Nagai, Ryohei Nakatsu . . . . . . . . . . . . . . . . . . .

70

Trans-disciplinary Avenues in Education: Computing and Art Selim Balcisoy, Elif Ayiter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

80

Enhanced SCORM Sequencing Rule for e-Learning System Yong-Sang Cho, Dae-Jun Hwang, Tae-Myung Chung, Sung-Ki Choi, Woo-In Bae . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

90

Research on Intelligent Mutual Decision-Making in Virtual Learning Environments Ruwei Yun, Yi Li . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

100

Monitoring the Experiment Process and Diagnosing the Experiment Mistakes Made by Students with Petri Net Modeling Jyh-Cheng Chang, Shao-Chun Li, Maiga Chang, Jia-Sheng Heh . . . . .

108

The Research on a Kind of Knowledge Network for Self-learning Xingwei Hao, Xiangxu Meng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

116

Research on Personalized Knowledge Service System in Community E-Learning Yan-wen Wu, Qi Luo, Yu-jun Liu, Ying Yu, Zhao-hua Zhang, Yan Cao . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

124

Using Instruction Strategy for a Web-Based Intelligent Tutoring System Yongjun Jing, Shaochun Zhong, Xin Li, Jinan Li, Xiaochun Cheng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

132

E-Learning Resource Management Knowledge Laxmi Saxena . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

140

Visual Tools and Examples to Support Active E-Learning and Motivation with Performance Evaluation Mohamed Hamada . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

147

Research and Application of Stereoscopic Video Based e-Learning System Linjuan Pang, Gangyi Jiang, Yu Zhou, Mei Yu . . . . . . . . . . . . . . . . . . .

156

Fusion Education by Humanities and Sciences in the Case of e-Learning and Hands-On Masayuki Hata, Masato Honma, Hitoshi Matsubara, Taku Osanai, Takeshi Osanai . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

161

Table of Contents

XIII

Knowledge Representing and Clustering in e-Learning Chunhua Ju, Xun Wang, Biwei Li . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

166

Programming of Informatized Instructional Design Platform for Physics Yongjiang Zhong, Ju Liu, Shaochun Zhong, Yamei Zhang, Xiaochun Cheng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

171

A New Method for Testing Embedded Software with Scenario Pattern Yu Ren, Tongwu Li . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

178

The Construction of Fuzzy Set and Fuzzy Rule for Mixed Approach in Adaptive Hypermedia Learning System Naomie Salim, Norreen Haron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

183

Knowledge Management System Based on Semantic Web in e-Learning Community Huang Qi, Shouqian Sun . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

188

A Platform for Interactive VR Storytelling Changhoon Park, Michitaka Hirose, Heedong Ko . . . . . . . . . . . . . . . . . .

193

E-Learning System and Graphics Education Reviewing the Potential of Virtual Learning Environments in Schools Malcolm Padmore, Lynne Hall, Bob Hogg, Gareth Paley . . . . . . . . . . .

203

Designing a Virtual Learning Environment to Support the Study of Crime and Its Prevention for Teenagers Lynne Hall, Karen Padmore, Mike Hodge, Giles Oatley . . . . . . . . . . . .

213

Mental Vision: A Computer Graphics Teaching Platform Achille Peternier, Daniel Thalmann, Fr´ed´eric Vexo . . . . . . . . . . . . . . .

223

3-D Graphical Hypermedia Meets Interactive E-Books: A New Paradigm for Experiential Learning Joan Mazur, Ruigang Yang, Mingxuan Sun, Rebecca Kayrouz . . . . . . .

233

e-Calculus at IZTECH ¨ Unal Ufuktepe, G¨ unnur Ufuktepe, Asli Deniz, Veli D¨ undar . . . . . . . . .

243

SKIT: A Computer-Assisted Sketch Instruction Tool Greg Coombe, Brian Salomon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

251

Exploring the Fourth Dimension: The Design of a Multimedia Learning System for Generalization F.L. Lee, J.H.M. Lee, M.K.H. Wong, H.Y. Law, L. Wang . . . . . . . . .

261

XIV

Table of Contents

Curriculum Development for the Gifted Elementary School Students in Computer Yeongwha Lee, Woochun Jun, Byeong Heui Kwak . . . . . . . . . . . . . . . . .

271

Computer-Assisted Teaching in Class Situation: A High-School Math Lab on Vectors Maud Marchal, Peggy Provent, Frederic Ruyer, Pirouz Djoharian, Fabrice Neyret . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

281

Interactive Multimedia System for Distance Learning of Higher Education Yan Liu, Wong Hang Chit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

291

A Multimedia Contents Development and Implementation Model Based on Computer Graphics Courseware Teuk-Seob Song, Yoon-Chul Choy, Soon-Bum Lim . . . . . . . . . . . . . . . .

301

Study on the Long-Range Marine Cruise System of Unmanned Aerial Vehicles Zhixiong Qiu, Peizhen Lan, Xinyu Lin . . . . . . . . . . . . . . . . . . . . . . . . . . .

311

An Extendable System for the Specification and Generation of Interactive Self-tests Manfred Widera, Barbara Messing, Gabriele Kern-Isberner, Malte Isberner, Christoph Beierle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

316

A Wireless LAN-Based Robust and Scalable Virtual Laboratory for E-Learning Yongkai Fan, Jun Lin, Tianze Sun, Wenju Yuan . . . . . . . . . . . . . . . . .

322

Component Based E-Learning System Using Item Analysis Hwa-Young Jeong . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

326

Collaborative GIS Environment for Exploratory Spatial Data Analysis Based on Hybrid P2P Network Yang-Won Lee, Key-Ho Park, Ryosuke Shibasaki . . . . . . . . . . . . . . . . .

330

Storytelling, Intelligent Agents and Affective Interaction Storytelling in Virtual Reality for Training Nicolas Mollet, Bruno Arnaldi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

334

Supporting the Development of Interactive Storytelling Skills in Teenagers Judy Robertson, Judith Good . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

348

Table of Contents

XV

700 Years Later: Using Multimedia to Bring the Stories of Medieval Asian Scrolls to Today’s Students Kevin Travers, Christina M. Finneran . . . . . . . . . . . . . . . . . . . . . . . . . . .

358

Developments in Affect Detection from Text in Open-Ended Improvisational E-Drama Li Zhang, John A. Barnden, Robert J. Hendley, Alan M. Wallington . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

368

Theoretical Framework for Construction of Representation Through Interactive Narrative Baptiste Campion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

380

Edutainment Aspects in Hypermedia Storytelling Wolfgang Heiden . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

389

A Theatre of Ethics and Interaction? Bertolt Brecht and Learning to Behave in First-Person Shooter Environments Dan Pinchbeck . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

399

Using Cognitive Affective Interaction Model to Construct On-Line Game for Creativity Hsien-Sheng Hsiao, Kai-Hsin Wong, Meng-Jie Wang, Kuang-Chao Yu, Kuo-En Chang, Yao-Ting Sung . . . . . . . . . . . . . . . . .

409

Story Pacing in Interactive Storytelling Stefan G¨ obel, Rainer Malkewitz, Felicitas Becker . . . . . . . . . . . . . . . . . .

419

Narrative Structure Analysis of Lecture Video with Hierarchical Hidden Markov Model for E-Learning Yu-Chi Liu, Xi-Dao Luan, Yu-Xiang Xie, Duan-Hui Dai, Ling-Da Wu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

429

4-Dimensional Context Management for Interactive Virtual Storytelling Seulki Kang, Heedong Ko, YoonChul Choy . . . . . . . . . . . . . . . . . . . . . . .

438

Integration of Game and Education PS-DA Model of Game Addiction: Theoretical Hypothesis and Case Analysis Peng Deng, Zhiting Zhu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

444

A Case Study of Game Design for E-Learning Pei-Chi Ho, Szu-Ming Chung, Ming-Hsin Tsai . . . . . . . . . . . . . . . . . . .

453

XVI

Table of Contents

Static and Dynamic Difficulty Level Design for Edutainment Game Using Artificial Neural Networks Kok Wai Wong, Chun Che Fung, Arnold Depickere, Shri Rai . . . . . . .

463

Study of Dance Entertainment Using Robots Kuniya Shinozaki, Yousuke Oda, Satoshi Tsuda, Ryohei Nakatsu, Akitsugu Iwatani . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

473

Smart Ambience Games for Children with Learning Difficulties Horace H.S. Ip, Belton Kwong . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

484

Teaching an Undergraduate AI Course with Games and Simulation Philip Hingston, Barbara Combes, Martin Masek . . . . . . . . . . . . . . . . . .

494

Experimental Approach to Affective Interaction in Games Holger Diener, Karina Oertel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

507

A Framework to Help Designing Innovative Massively Multiplayer Online Games Interactions Anne-Gwenn Bosser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

519

Earth and Planetary System Science Game Engine Falko Kuester, Gloria Brown-Simmons, Christopher Knox, So Yamaoka . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

529

Learning Online: A Comparative Study of a Situated Game-Based Approach and a Traditional Web-Based Approach M.S.Y. Jong, J.J. Shang, F.L. Lee, J.H.M. Lee, H.Y. Law . . . . . . . . .

541

Design and Contents of a 3Dblog System and Its Applications to Edutainment Yoshio Nishio, Takami Yasuda, Shigeki Yokoi . . . . . . . . . . . . . . . . . . . .

552

Design and Implementation of Farmtasia: A Game Designed for the VISOLE Teaching Style E.T.H. Luk, M.K.H. Wong, K.K.F. Cheung, F.L. Lee, J.H.M. Lee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

566

Teaching Agent Systems’ Design Using 3D Interactive Computer Games In-Cheol Kim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

572

Cyranus – An Authoring Tool for Interactive Edutainment Applications Ido A. Iurgel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

577

Research of Virtual Campus Environment Study Using VRML Weihua Hu, Chuying Ke, Guorong Wang . . . . . . . . . . . . . . . . . . . . . . . .

581

Table of Contents

XVII

A Multimodal Fusion Framework for Children’s Storytelling Systems Danli Wang, Jie Zhang, Guozhong Dai . . . . . . . . . . . . . . . . . . . . . . . . . .

585

Design of a Cartoon Game for Traffic Safety Education of Children in China Zhen Liu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

589

Cinematographic Techniques for Edutainment Applications Felicitas Becker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

593

Game Design and Development Efficient Large-Scale Terrain Rendering Method for Real-World Game Simulation Dong-Soo Kang, Yun-Jin Kim, Byeong-Seok Shin . . . . . . . . . . . . . . . . .

597

Federate Migration in Grid-Based Virtual Wargame Collaborative Environment Hai Huang, Wei Wu, Xin Tang, Zhong Zhou . . . . . . . . . . . . . . . . . . . . .

606

Cyclic Reproduction Scheme in Genetic Algorithm for Evolutionary Game Sang-Won Um, Suk-Han Lee, Tae-Yong Kim, Jong-Soo Choi . . . . . . .

616

Model Searching Algorithm Based on Response Order and Access Order in War-Game Simulation Grid Yunxiang Ling, Miao Zhang, Xiaojun Lu, Wenyuan Wang, Songyang Lao . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

627

Creating Emotions by Characters Design for Computer Games F. You, I. Palmer, W. Godfrey, Z.B. Zheng . . . . . . . . . . . . . . . . . . . . . .

638

Efficiently Maintaining Consistency Using Tree-Based P2P Network System in Distributed Network Games Kyung Seob Moon, Vallipuram Muthukkumarasamy, Anne Thuy-Anh Nguyen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

648

Real-Time Neuroevolution to Imitate a Game Player Hyun-woo Ki, Ji-hye Lyu, Kyoung-su Oh . . . . . . . . . . . . . . . . . . . . . . . .

658

Improving Game Processing in Multithreading and Multiprocessor Architecture Abdennour El Rhalibi, Madjid Merabti, Yuanyuan Shen . . . . . . . . . . . .

669

ODECAL, a Flexible Open Source Rag Doll Simulation Engine Pang Lih-Hern, Tan Yee Siang, Wong Chin Foo, Wong Lai Kuan . . .

680

XVIII Table of Contents

Computer Supported Remote Learning and Gaming Using Tele-Face Mouse System Sang Chul Ahn, Jin Hak Kim, Hyoung-Gon Kim . . . . . . . . . . . . . . . . . .

688

Pheromone-Based Pre-processing to Improve Move Generation in Board Games Shenyi Chen, Hui Qian, Miaoliang Zhu . . . . . . . . . . . . . . . . . . . . . . . . . .

692

Design and Implementation of a Game Physics Editor Using XML Byungyoon Lee, Jonghwa Choi, Dongkyoo Shin, Dongil Shin . . . . . . . .

696

A Virtual Billiard Game with Visual, Auditory and Haptic Sensation Yoshihiro Takamura, Norihiro Abe, Kazuaki Tanaka, Hirokazu Taki, Shoujie He . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

700

Chinese Chess Based on Jabber Quanyu Wang, Siyin Liu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

706

Dynamic Terrain LOD with Region Preservation in 3D Game Engine Guojun Chen, Jing Zhang . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

711

Mobile Computing, Network and Collaboration Design and Implementation of a Mobile Class Web Site Using Intelligent User Interface Yeonho Hong, Woochun Jun, Byeong Heui Kwak . . . . . . . . . . . . . . . . .

716

A Wearable Learning and Support System for Manufacture Application Shiji Xiahou, Dongyi Chen, Zhiqi Huang . . . . . . . . . . . . . . . . . . . . . . . . .

726

Analysis and Research About an On-Line Collaborative Learning Teams Based Grids ZhangJian Chen, TongSen Hu, Yan Yu . . . . . . . . . . . . . . . . . . . . . . . . . .

735

Of Collaborative Learning Team: An Approach for Emergent Leadership Roles Identification by Using Social Network Analysis Punnarumol Temdee, Bundit Thipakorn, Booncharoen Sirinaovakul, Heidi Schelhowe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

745

Concepts for Extending Wiki Systems to Supplement Collaborative Learning Silvan Reinhold, Daniel F. Abawi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

755

3D Collaborative Virtual Environments over the Web Leandro Balladares, Rolando Menchaca, Rub´en Peredo . . . . . . . . . . . . .

768

Table of Contents

XIX

Research on Multimedia Transmission of Mobile Learning Based on Wireless Network Zhijun Yang, Dongfeng Zhao . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

778

Interactive Mobile Pet Game JongYeol Yang, TaeKyung Lee, Keechul Jung . . . . . . . . . . . . . . . . . . . . .

785

Multi-stroke Freehand Text Entry Method Using OpenVG and Its Application on Mobile Devices Gaoqi He, Zhigeng Pan, Christophe Quarre, Mingmin Zhang, Huijun Xu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

791

A Streaming Engine for PC-Based 3D Network Games onto Heterogeneous Mobile Platforms Gi Sook Jung, Soon Ki Jung . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

797

Hybrid Triangle-Based Sub-texel Precision Anisotropic Filtering for Mobile Devices Bailin Yang, Xiangcheng Li . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

801

A Combination Scheme to Improve TCP Throughput over Multihops Wireless Mobile Ad Hoc Networks Wei Ren, Hai Jin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

805

Web Virtual Reality Edutainment in Biology and Physics Tomaz Amon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

810

Graphics Modeling and Rendering for Games Survey and Practice of 3D City Modeling Liying Wang, Wei Hua . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

818

Collaborative 3D Modeling of Large-Scale Virtual Geographic Environment Weijun Yang, Jianhua Gong, Chuanwen Hu, Weixing Wang . . . . . . .

829

A 3D Model Feature-Line Extraction Method Using Mesh Sharpening Hao Jing, Bingfeng Zhou . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

840

3D Body Reconstruction from Photos Based on Range Scan Hyewon Seo, Young In Yeo, Kwangyun Wohn . . . . . . . . . . . . . . . . . . . .

849

Mesh Fitting Based 3D Character Modeling Kai-Man Tong, Kin-Chuen Hui, Charlie C.L. Wang . . . . . . . . . . . . . . .

861

XX

Table of Contents

Antialiased Shadow Algorithms for Game Rendering Nailiang Zhao, Yanjun Chen, Zhigeng Pan . . . . . . . . . . . . . . . . . . . . . . .

873

Efficient Out-of-Core Rendering of Complex 3D Scenes Yu Gao, Baosong Deng, Yingmei Wei, Lingda Wu . . . . . . . . . . . . . . . .

883

An Efficient Method for Rendering Detailed Soft Shadow Feng Zhou, Guoping Wang, Heng Wang . . . . . . . . . . . . . . . . . . . . . . . . .

893

Research of Dynamic Terrain in Complex Battlefield Environments Xingquan Cai, Fengxia Li, Haiyan Sun, Shouyi Zhan . . . . . . . . . . . . . .

903

Procedural Terrain Detail Based on Patch-LOD Algorithm Xuexian Pi, Junqiang Song, Liang Zeng, Sikun Li . . . . . . . . . . . . . . . .

913

Texture Synthesis Using Incremental Patches Nikolaos Ersotelos, Feng Dong . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

921

Image Based Active Model Adaptation Method for Face Reconstruction and Sketch Generation Yuehu Liu, Yunfeng Zhu, Yuanqi Su, Zejian Yuan . . . . . . . . . . . . . . . .

928

Space Object Geometry and Behavior Modeling Method Yang Zhou, Caozheng Lan, Qing Xu, Zhihui Gong . . . . . . . . . . . . . . . .

934

Interactive Sketch Generation for Dunhuang Frescoes Jianming Liu, Dongming Lu, Xifan Shi . . . . . . . . . . . . . . . . . . . . . . . . . .

943

Real-Time Walkthrough System of Large-Scale Infrared Dynamic Scene Zhangye Wang, Changbo Wang, Yan Zhou, Qunsheng Peng . . . . . . . .

947

Animation Techniques for Edutainment Dynamic Surface Deformation and Modeling Using Rubber Sweepers Chengjun Li, Wenbing Ge, Guoping Wang . . . . . . . . . . . . . . . . . . . . . . .

951

A Novel Algorithm for Extracting the Boundaries of Two Planar Curves’ Morphologic Summation WenYu Liu, HaiRong Liu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

962

An Efficient Algorithm for Content-Based Human Motion Retrieval Yan Gao, Lizhuang Ma, Junfa Liu, Xiaomao Wu, Zhihua Chen . . . . .

970

A Fast Individual Face Modeling and Facial Animation System Bin Ding, Yangsheng Wang, Jian Yao, Peng Lu . . . . . . . . . . . . . . . . . .

980

Table of Contents

XXI

Digital Buddhist Image Creation by Haptic Deformation Jong-Phil Kim, Jeung-Chel Park, Beom-Chan Lee, Kwan H. Lee, Jeha Ryu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

989

Expression Animation of Interactive Virtual Humans Based on HMM Emotion Model Guojiang Wang, Zhiliang Wang, Xiuyan Meng, Shaodong Teng, Yinggang Xie . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

999

Retrieval of 3D Video Mosaics for Fast 3D Visualization Jaechoon Chon, Eihan Shimizu, Ryosuke Shibasaki . . . . . . . . . . . . . . . . 1008 Style Animations Generated from Dynamic Model Dengming Zhu, Zhaoqi Wang, Shihong Xia . . . . . . . . . . . . . . . . . . . . . . . 1018 Research and Implement of Virtual Human’s Walking Model in Maintenance Simulation Xiaojun Lu, Yan Li, Hangen He, Yunxiang Ling . . . . . . . . . . . . . . . . . . 1027 An Integrated Color Changing Simulation System Based on Colorimetric and Chemical Modeling Xifan Shi, Dongming Lu, Jianming Liu, Yunhe Pan . . . . . . . . . . . . . . . 1037 An Experimental Facial Synthesis System Using Graph Cut and Gradient Domain Fusion Xiaohong Jiang, Fen Dai, Hanqing Jiang . . . . . . . . . . . . . . . . . . . . . . . . 1047 Skin Incision Using Real-Time Cutaway Based on FE Analysis Zhongyu Chen, Guohua Wu, Ronghua Liang, Guofeng Zhang . . . . . . . 1053 Physically-Based Real-Time Music Fountain Simulation Huagen Wan, Yujuan Cao, Xiaoxia Han, Xiaogang Jin . . . . . . . . . . . . 1058 Generating 3D Paper-Cutting Effects Yan Li, Jinhui Yu, Honxin Zhang, Jiaoying Shi . . . . . . . . . . . . . . . . . . 1062 Interactive Simulation of Large-Scale Forests in Virtual Reality Applications Chengfang Song, Qifeng Tan, Long Zhang, Wei Chen, Yi Gong, Yu Guan, Qunsheng Peng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1066 Dynamic Textures Using Wavelet Analysis Jianbing Shen, Xiaogang Jin, Chuan Zhou, Hanli Zhao . . . . . . . . . . . . 1070 Style Conversion of Cartoon Animation Tianzhou Chen, Jinhui Yu, Qunsheng Peng . . . . . . . . . . . . . . . . . . . . . . 1074

XXII

Table of Contents

Application of 4-Point Subdivision to Generate In-Between Frames in Skeletal Animation Fan Zhou, Xiaonan Luo, Hao Huang . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1080 Recursive Curves and Surfaces in Grassmann Space for Computer Modeling and Animation Xiaonan Luo, Shujin Lin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1085 Virtual Five Animals Exercise Yue Qi, Xu-kun Shen, Qin-ping Zhao . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1090 Adaptive Level Set Method for Mesh Evolution Guoxian Zheng, Jieqing Feng, Xiaogang Jin, Qunsheng Peng . . . . . . . 1094 Energy Matting Yu Guan, Xiao Liang, Zi’ang Ding, Yinan Fan, Wei Chen, Qunsheng Peng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1098 Computer Simulation of Multi-phase Coupled Heat and Moisture Transfer in Clothing Assembly with a Phase Change Material in a Cold Environment Shuxiao Wang, Yi Li, Hiromi Tokura, J.Y. Hu, Aihua Mao . . . . . . . . 1103 A Distributed Forest Fire Fighting Simulation System Based on HLA Chongcheng Chen, Liyu Tang, Xiaogang Feng, Kaihui Lin . . . . . . . . . 1107

VR, Augmented Reality and Mixed Reality Experimental Researches on Gaze-Based 3D Interaction to Stereo Image Display Yong-Moo Kwon, Jai Kyung Shul . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1112 System Design for On-Line Distributed Computational Visualization and Steering Qishi Wu, Mengxia Zhu, Nageswara S.V. Rao . . . . . . . . . . . . . . . . . . . . 1121 Multi-user Live Video in a Shared Virtual World for Enhanced Group Collaboration Seon-Min Rhee, Jiyoung Park, Myoung-Hee Kim . . . . . . . . . . . . . . . . . 1131 An Improved 6DOF Electromagnetic Tracking Algorithm with Anisotropic System Parameters Zhigang Yan, Kui Yuan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1141 Image Texts-Based Navigation for Augmented Game Anjin Park, Keechul Jung . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1151

Table of Contents XXIII

A Framework of Collaborative Workspace Based on Multi-stereo Vision and Shared Mixed Reality Shengjin Wang, Yaolin Tan, Xiaoqing Ding . . . . . . . . . . . . . . . . . . . . . . 1161 Digital Canvas: A Projection-Space Interaction Tool Jun Park . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1171 An Augmented Reality Toolkit Based on SEDRIS Myeong Won Lee, Min-Gun Lee, Sung-Gon Kim, Kicheon Hong . . . . 1180 Designing Interactions for Augmented Reality Based Educational Contents Jinseok Seo, Namgyu Kim, Gerard J. Kim . . . . . . . . . . . . . . . . . . . . . . . 1188 The ARTable: An AR-Based Tangible User Interface System Youngmin Park, Woontack Woo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1198 Effects of Guided and Unguided Style Learning on User Attention in a Virtual Environment Jayoung J. Goo, Kyoung S. Park, Moonhoen Lee, Jieun Park, Minsoo Hahn, Hyungil Ahn, Rosalind W. Picard . . . . . . . . . . . . . . . . . . 1208 Using Olfactive Virtual Environments for Learning Organic Molecules Ang`ele Tijou, Emmanuelle Richard, Paul Richard . . . . . . . . . . . . . . . . . 1223 Single Camera Based Optical Tracking System for Personal Entertainment Dongdong Weng, Yue Liu, Yongtian Wang . . . . . . . . . . . . . . . . . . . . . . . 1234 Computer Interaction by Camera Tracking Torsten Engelbrecht, Linqiang Chen, Yigang Wang . . . . . . . . . . . . . . . . 1243 Haptic Interaction System for Co-evolution with Reactive Virtual Human S.Z. Jeong, N. Hashimoto, M. Sato . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1252 Study on Attitude Measurement System for Virtual Surgery Navigation Xiaoming Hu, Yue Liu, Yongtian Wang . . . . . . . . . . . . . . . . . . . . . . . . . 1262 Dynamic User Modeling for Sketch-Based User Interface Zhengxing Sun, Bin Li, Qiang Wang, Guihuan Feng . . . . . . . . . . . . . . 1268 Multi-modal Virtual Environments for Education: From Illusion to Immersion Emmanuelle Richard, Ang`ele Tijou, Paul Richard . . . . . . . . . . . . . . . . . 1274

XXIV Table of Contents

Effects of Filtering Policies on Task Performance in a Desktop CVE System Ling Chen, Wei Liu, Gencai Chen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1280 Computer-Aided Training System of Educational Virtual Dissection Using Visible Korean Human Koo-Joo Kwon, Byeong-Seok Shin, Min Suk Chung . . . . . . . . . . . . . . . . 1284 Poultry.Internet and Internet Pajama: Novel Systems for Remote Haptic Interaction Keng Soon Teh, Shang Ping Lee, Adrian David Cheok . . . . . . . . . . . . . 1288 Haptic Puppetry for Interactive Games Sujeong Kim, Xinyu Zhang, Young J. Kim . . . . . . . . . . . . . . . . . . . . . . . 1292

Digital Heritage and Digital Museum An Image Retrieval System Based on MPEG-7 and XMLDB Query for Digital Museum Hui Xu, Hui Xiang . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1303 Digitization of Culture Heritage Based on Tour into the Picture Yaping Zhang, Yang Zhao, Jiaoying Shi, Dan Xu . . . . . . . . . . . . . . . . . 1312 Virtual Museum Net Pier Augusto Bertacchini, Eleonora Bilotta, Elvira Di Bianco, Gianpiero Di Blasi, Pietro Pantano . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1321 Digital Repair Research on Archeological Relics Zhong Li, Lizhuang Ma, Mingxi Zhao, Zhihong Mao . . . . . . . . . . . . . . . 1331 A Teacher’s Tool in Game-Based Learning System: Study and Implementation Jingguang Liu, Lu Wang . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1340 Experiments Towards 3D Immersive Interaction for Digital Libraries Rodrigo Almeida, Pierre Cubaud, J´erˆ ome Dupire, St´ephane Natkin, Alexandre Topol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1348 The Technology of Stereo Photography and Virtual Reality in Research of Virtual Museum Zili Li, Hui Li, Yuanchu Li . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1358

Table of Contents

XXV

A Blind Watermarking Algorithm for Animation Used in Digital Heritage Li Li, Zhigeng Pan, Shusen Sun . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1366 Content-Based 3D Model Retrieval for Digital Museum Jie Tang, Fuyan Zhang . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1371 Digital Restoration of Historical Heritage by Reconstruction from Uncalibrated Images Dan Hou, Xia Shen, Xiaowei Li, Yue Liu, Yongtian Wang . . . . . . . . . 1377 Building a Multi-purposes Digital Media System Using MPEG-21 and XML Hua-Yi Lin, Yueh-Min Huang, Tzone-I Wang . . . . . . . . . . . . . . . . . . . . 1383 Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1391

Abstracts Digital Stories and Learning Ruth Aylett School of Maths and Computer Science, Heriot-Watt University, Edinburgh, E14 4AS, UK [email protected] Abstract. In this paper we consider the role of story in digital technology enhanced learning and the issues involved in constructing digital narrative learning environments. We will see that story has played an important role in education for a very long time, well pre-dating the use of digital technologies, and can be closely related to a number of theoretical approaches to learning. However story has recently become a much more significant topic for research, not only in education and digital technology, but also in other fields, such as knowledge management, and we examine why it has become such a central concern. The paper then looks at the existing use of story in fielded educational applications, and questions how far it has actually been used as an essential mechanism producing a narrative learning environment – rather than merely as motivational support. The relationship between story and computer games is an important issue here. We argue that the interactivity of digital media raises fundamental problems for the use of story which are still challenging researchers, and look at the various approaches to dealing with these problems, citing a number of example systems. Finally, we discuss the direction in which the technology is moving what it offers now and what it may offer in the near future, and what technology support is currently available for creating digital narrative learning environments.

Images Enhance Learning – Or Do They? Judith R. Brown ACM SIGGRAPH, USA [email protected] Abstract. There have been a number of discussions and studies on the necessity for and impact of images and animations for learning, especially in science and engineering. These discussions took place throughout the world — at SIGGRAPH conferences in the United States, Eurographics conferences in France and Spain, an IFIP conference in South Africa, and a Computer Graphics Education workshop at Zhejiang University in China. This talk will summarize the findings from these activities and challenge the audience to think about whether their use of visuals enhances learning and how to ensure that they do. The activities mentioned above were supported by a National Science Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, p. 1 – 3, 2006. © Springer-Verlag Berlin Heidelberg 2006

2

Abstracts Foundation grant, and a white paper on visual learning for science and engineering summarizes the findings. The white paper includes examples of successful uses of images, animations, and interactions, as well as studies that show both success and failure in the use of visuals for learning. This white paper also gives recommendations for additional research topics, and it will be available in both the ACM and Eurographics Digital Libraries.

Mixed Reality and Human Centered Media for Social and Physical Interactive Computer Entertainment Adrian David Cheok Interaction and Entertainment Research Center, NTU, Singapore Abstract. This talk outlines new facilities within ubiquitous human media spaces supporting embodied interaction between humans and computation both socially and physically with the aim of novel interactive computer entertainment. We believe that the current approach to developing electronic based entertainment environments is somewhat lacking with regard to support for multi-person multimodal interactions. We discuss some different research prototype systems: the Virtual Kyoto Garden, Touchy Internet, Tilt-Pad, and the Human Pacman. The functional capabilities implemented in these systems include spatially-aware 3D navigation, tangible interaction, and ubiquitous human media spaces. Some of its details, benefits, and issues regarding design support are discussed.

Entertainment Park for Children Guozhong Dai Institute of Software, Chinese Academy of Science, China Abstract. This talk outlines the design and development of entertainment part, mainly for children. State of the art in then area is reported first, then we will talk about design experiences, implementation techniques, and demos.

Brand Edutainment and Digital Museums Karin Dietz Knowledge Gesellschaft für Wissenstransfer mbH, Berlin/Germany, and Edutainment International GmbH, Berlin/Germany Abstract. This talk will give the attendees an idea of what brand edutainment is about. The specific point of view will be market and marketing, giving awareness

Abstracts to future challenges and chances to be met by those who manufacture or service brands with their corporations. Brand Edutainment is showing up as a creative application for museum-like environments to walk in. Within the conference context this talk tends to provide an input on how concepts, ideas and performances of brand edutainment might in a marketable sense be linkable with different kinds of future museum concepts, e.g. those integrating digital and virtual technology.

Virtual Reality Technology for Museum Exhibits Michitaka Hirose University of Tokyo Abstract. Museum is now a good experimental field of VR technology. I will talk about several museum exhibits which utilized latest VR technology which include /font>high resolution VR /font> supported by sophisticated image environment such as IPT ( immersive projection technology ), and real world VR supported by mobile technology such as wearable computers. I will also talk about interaction design and scenario generation for these kind of novel exhibits.

Some Remarks on Graphics and Visualization Lawrence J. Rosenblum United States National Science Foundation Abstract. The last decade has seen many exciting advances in graphics and visualization. One goal in managing a science program is to identify new opportunities and then provide the impetus for new work in those areas. In these introductory remarks I will briefly discuss some of the more important advances from the last decade in graphics and visualization as well as new opportunities that are emerging.

CG Technologies for Art, Culture, and Edutainment Matthias Unbescheiden and Jose Encarnacao Fhg-IGD, Germany Abstract. This talk will present the state of the art report on the topic “Computer graphics technologies for art, digital heritage, and edutainment”. Some experimental results in these area done in Fhg-IGD will be shown.

3

VR-Enabled Bio Edutainment Y.Y. Cai, C.W. Chan, B.F. Lu, C. Indhumathi, and Y. Jiang Bio-informatics Research Centre and School of Mechanical & Aerospace Engineering, College of Engineering, Nanyang Technological University, Singapore Tel: +65-67905777; Fax: +65-67911859 [email protected]

Abstract. In this post-genome era, structural and functional biology is playing an increasingly important role in Life Science research and development. Transforming current bio education for future research biologists is therefore a challenging task. Learning structural biology starting from K-12 year or in preuniversity education is an issue of strategic importance. This paper presents our experimental work on bio edutainment for structural biology learning in secondary schools. Through playing, students will have a more intuitive way to understand better bio-molecular structures. Immersive and interactive games may potentially motivate students to explore the wonders of Life Science. The bio edutainment solution developed is powered by the Virtual Reality technology.

Fig. 1. Protein structures (α helices in pink, β strands in blue, loops in yellow and side-chains in stick-and-ball)

Note from the Editors: This paper is an invited paper; only abstract is here, the full paper is published in the special Issue, “Edutainment”, Computers & Graphics, 2006, No. 1 (Guest Editor: Prof. Zhigeng Pan, the program co-chair of this conference). That issue is devoted to Edutainment’2006, also for celebrating the 70th birthday of Prof. Jiaoying Shi, co-chair of this conference. Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, p. 4 – 4, 2006. © Springer-Verlag Berlin Heidelberg 2006

Learning Abstract Concepts Through Interactive Playing Jim X. Chen George Mason University, Fairfax, VA 22030, USA [email protected] Abstract. In this paper, we present strategies and approaches used to implement a synthetic learning environment which includes distributed interactive simulation, computational steering, interactive visualization, and artificial intelligence for learning abstract scientific concepts and entertainment. We use NavierStokes equations as a case study to show our ideas and methods. Students are allowed to interact with one another and engage in realistic collaborative exercises. The resulting technology is a testbed that reflects the benefits of a schoolhouse for training and education, especially for scientific abstract concepts, yet does not have the physical limitation that instructors, students and resources be collocated both spatially and temporally. Our innovative use of computational steering and interactive visualization allows students to visualize and manipulate the physical process, and understand the abstract concept through entertainment.

Fig. 1. Simulation and understanding by manipulation

Note from the Editors: This paper is an invited paper; only abstract is here, the full paper is published in the special Issue, “Edutainment”, Computers & Graphics, 2006, No. 1 (Guest Editor: Prof. Zhigeng Pan, the program co-chair of this conference). That issue is devoted to Edutainment’2006, also for celebrating the 70th birthday of Prof. Jiaoying Shi, co-chair of this conference. Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, p. 5 – 5, 2006. © Springer-Verlag Berlin Heidelberg 2006

Virtual Reality and Mixed Reality for Virtual Learning Environments Zhigeng Pan1, Adrian David Cheok2, Hongwei Yang1, Jiejie Zhu1,*, and Jiaoying Shi1 1

State Key Lab of CAD&CG, Zhejiang University, Hangzhou 310027, China [email protected] 2 Interaction and Entertainment Research Center, NTU, 50 Nanyang Drive, Singapore

Abstract. This paper explores educational uses of virtual learning environment (VLE) concerned with issues of learning, training and entertainment. We analyze the state-of-art research of VLE based on virtual reality and augmented reality. Some examples for the purpose of education and simulation are described. These applications show that VLE can be means of enhancing, motivating and stimulating learners’ understanding of certain events, especially those for which the traditional notion of instructional learning have proven inappropriate or difficult. Furthermore, the users can learn in a quick and happy mode by playing in the virtual environments.

Fig. 1. The user holds the HMD, and point to one of the boards to view the virtual garden and menu

Note from the Editors: This paper is an invited paper; only abstract is here, the full paper is published in the special Issue, “Edutainment”, Computers & Graphics, 2006, No. 1 (Guest Editor: Prof. Zhigeng Pan, the program co-chair of this conference). That issue is devoted to Edutainment’2006, also for celebrating the 70th birthday of Prof. Jiaoying Shi, co-chair of this conference. *

Corresponding Author.

Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, p. 6 – 6, 2006. © Springer-Verlag Berlin Heidelberg 2006

Intelligent Tutorial Planning Based on Extended Knowledge Structure Graph
Zhuohua Duan1,3 , Yunfei Jiang2 , and Zixing Cai3 1

3

Department of Computer, School of Information Engineering, Shaoguan University, 512003, Shaoguan, Guangdong, China [email protected] 2 Institution of Software, SUN YAT-SEN University, 510000, Guangzhou, Guangdong, China College of Information Science and Engineering, Central South University, 410083, Changsha, Hunan, China

Abstract. Intelligent tutorial planning (ITP) is an important component of intelligent tutorial system (ITS). Models of domain knowledge, models of tutorial methods and models of learner are three key elements of ITS. In this paper, the concept of extended knowledge structure graph (EKSG) is presented. An EKSG integrates models of domain knowledge, models of tutorial methods and models of learner organically. Based on the EKSG, algorithms JUDGE and TPLAN are put forward to resolve ITP problems. The algorithm JUDGE calculates the optimal solution graph when there is a solution, and the algorithm TPLAN calculates optimal tutorial plan based on the solution graph. Both algorithms are proved to be correct, the efficiency of them is also discussed.

1

Introduction

Intelligent tutoring system (ITS) typically consists of a set of correlative coursewares, each courseware fulfills teaching task of one or more knowledge points. Planning is a problem-solving method which produces an action sequence (i.e. plan) to achieve the objective [1]. Intelligent tutorial planning (ITP) tailors learning plan for a given student according to domain knowledge structure, teaching methods and cognitive level of the student. It combines a set of coursewares organically, provides individual learn path for learner, and is a key module of ITS. Earlier researches on planning concentrate on physical actions of robots[1]. Peachey et al. firstly use planning techniques to construct intelligent tutorial system [2]. Henceforth, tutorial planning has received extensive attentions in the field of computer-aided instruction (CAI). Based on blackboard architecture, Murray proposes the concept of dynamic instructional planning which has the ability to generate, monitor, and revise instructional plans during the course


This work is partially supported by the National Natural Science Foundation of China Grant #60234030 and Master Excellent Course Project of China and CSU.

Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, pp. 7–16, 2006. c Springer-Verlag Berlin Heidelberg 2006


8

Z. Duan, Y. Jiang, and Z. Cai

of instruction [3]. Katz et al. point out that there are two criteria for effective automatic curriculum planning, i.e. cognitive and motivational: (1) selection of tasks that are at the appropriate difficulty level for the student; i.e., challenging, but not frustrating and (2) enough variety in the chosen tasks to sustain the student’s interest [4]. In [5], the authors present and investigate the application of multi-level planning techniques, together with meta and micro level knowledge architecture model, in building the domain knowledge, planning and navigating the curriculum of an intelligent tutoring system. Elorriaga points out that the ability to adapt the instruction to the student is based on two issues: the learner model and the instructional planning [6]. Martens presents a web- and case-based planning agents in intelligent tutoring [7]. Li et al. propose a layered tutorial planning scheme, including global tutorial planning, tutorial scheme scheduling, and local tutorial planning within scheme [8]. Researchers realized that it is critical for ITS to combine domain knowledge, student model, teaching strategy and their relationship in a systematical way [9,11]. Literature [9] points out that a courseware contains three key components, namely, domain knowledge model, student model and teaching strategy model. JIANG firstly combines the three key components to solve the problem of tutorial planning [10]. He proposes the concept of knowledge structure graph (KSG), and presents an ITP framework and algorithms based on KSG. In a KSG, the relationship between different knowledge points is represented with an AND/OR graph with weighted k-arcs. A k-arc in KSG represents several teaching methods for a knowledge node implicitly. The main drawback of KSG is that teaching methods are difficult to maintain in KSG and it is difficult to represent the relationship between student model and teaching methods. To handle this problem, this paper proposes the concept of extended knowledge structure graph (EKSG). In an EKSG, teaching methods and their relationships with domain knowledge and students are represented explicitly. It will be showed in Section 2 that the three key components of ITS, namely, domain knowledge model, teaching strategy model and learner model, are combined organically in EKSG. Based on EKSG, the intelligent tutorial planning algorithms are proposed. The correctness of the algorithms is proven, and the efficiency of the algorithms is analyzed.

2

Extended Knowledge Structure Graph

A good knowledge representation model for ITS must be able to represent the relationships of domain knowledge, teaching strategies and student models. In this paper, the concept of EKSG is presented to serve as knowledge representation model for ITS for the task of tutorial planning. Definition 1. A directed acyclic graph (DAG) is called as an extended knowledge structure graph (EKSG), if and only if it satisfies the following conditions:

ITP Based on Extended Knowledge Structure Graph

9

Fig. 1. An extended knowledge structure graph

• It contains two types of nodes: knowledge nodes and method nodes; • The next nodes of a knowledge node are method nodes, which represent several optional methods for learning the knowledge node. The next nodes of a method node are knowledge nodes, which represent all the requisite knowledge needed to apply this method; • Several weights are assigned to each method node. Each weight represents the cost needed for a certain type of learner to use this method. EKSG have two important features. Firstly, the teaching methods are represented with nodes. So the information of method nodes is easy to be managed. Secondly, domain knowledge, teaching strategies, and learner models are integrated into an EKSG systematically. An example of EKSG is shown in Fig. 1. Knowledge nodes are denoted with circles, and teaching method nodes are denoted with rectangles. There are 6 knowledge nodes, namely, ‘A’, ‘B’, ‘C’, ‘D’, ‘E’, ‘F’ and ‘G’, and 4 teaching method nodes, namely, ‘M1 ’, ‘M2 ’, ‘M3 ’ and ‘M4 ’. ‘L1 ’ and ‘L2 ’ denote two kinds of learners. There are two methods for learning knowledge ‘A”, namely, ‘M1 ’ and ‘M2 ’. When ‘M1 ’ is applied, the learner must have mastered the knowledge ‘B’, ‘D’, and ‘E’. The costs of teaching methods using by different learners are showed in a two-dimensional table, as shown in Fig. 1. For example, the cost for student ‘L1 ’ to learn the knowledge ‘A’ with method ‘M1 ’ is 3. For the sake of clarity, let bs(M) denote the set of next nodes of ‘M’, g(M) denote the previous node of ‘M’, c(L,M) denote the cost for learner ‘L’ using method ‘M’. For instance, in Fig. 1, bs(M1 )={B,D,E}, g(M1 )=A, c(L1 ,M1 )=3.

3

Tutorial Planning Problems and Their Representation

A tutorial planning problem (TPP) must provide the following information: an EKSG, a learning goal (G), a set of basic knowledge (BS) and a learner model

10

Z. Duan, Y. Jiang, and Z. Cai

(L). Usually, the EKSG has been constructed by domain experts, and all the learners share a global EKSG. Definition 2. A tutorial planning problem (TPP) defined as a triple (G, BS, L), where G denotes the learning goal, BS denotes a set of knowledge which is mastered by the learner L, and L denotes the learner who submits the planning problem. Definition 3. A tutorial planning (TP) of a TPP (G, BS, L), denoted by TP(G,BS,L), is a method sequence satisfying 1) if G∈BS, then TP=( ); 2) if G / then TP=(M1 ,M2 ,. . . ,MN ), where g(MN )=G, bs(Mi ) ⊆
∈BS, ( g(Mj )) ( 1≤i≤N).

BS ∪

1≤j is a pulsing power up.

on what had been learned; 20% was allocated to producing a controller that compiled and ran successfully; and 20% was calculated from the controller’s final placing in the tournament. While the assessment value of the competitive component was small, it was sufficient to rouse students’ competitive spirits and get their creative juices flowing. Postings to the students’ on-line forum reflected their enjoyment and satisfaction in beating the instructor-authored controllers, and finding new ways to improve their solutions. It was interesting to observe the increasing sophistication of the strategies that students developed. Initially, most students were concerned with mastering the syntax of the Java language and the rationale of the toolkit, and were satisfied with controllers that simply responded sensibly to the game situation – trying to get power ups when they were available, shooting at opponents when in range, running away when outmatched. As their facility with Java and the toolkit improved, students began to consider anticipating possible strategies of opponents, and how these might be exploited. This in turn led them to attempt more difficult programming, and to invent more creative uses for fuzzy reasoning.

6

Student Experiences

When developing an interactive tool to assist students in their learning, it is vital to seek their feedback on the learning experience. Student comments were

502

P. Hingston, B. Combes, and M. Masek

sought after the first stage of the assignment, and an online survey was conducted near the end of the unit. Overall, our observation is that the student response is very positive. Some examples of student reflections on the first stage of the assignment illustrate this: Several students reported on the motivational and immersive aspects of the experience, while also acknowledging the deeper learning that occurred. “Other than the obvious benefits of this assignment aiding in improving my personal level and awareness of programming, it proved to be an entertaining experience with a level of interaction I have not previously experienced in the world of programming. After working on the saucer for a few hours it starts to grow on you and a desire to improve/optimize it becomes innate as you grow attached to the little circle of floating pixels on the screen, including a slight feeling of joy as your little fighter rains death upon its enemies. Likewise, I expect the second part of this assignment to also be an interesting experience as I have found this one.” (Student feedback 1). “Way to go Dr Phill!!! This unit has been one of my favourites cause you’ve made it fun and interesting” (Student feedback 2). “The assignment was probably one of the more interesting assignments I’ve done in my 3 years at uni. And even though I had no prior knowledge of java I was still able to complete the assignment to the point where I was satisfied with the work I’d done” (Student feedback 3). Another student commented on the advantages of being able to ‘see’ their program in action and how this improved their understandings of fuzzy logic. “The final code enabled us the see how the saucer created by the team was compiling and when it was executed, the saucers were able to perform the instructions that were given to us by the tutor. It was clearly visible that the saucers were running after each other and as they were the whole idea was to shoot at each other and the one that lost all the energy first lost the competition. From the creation of this program we were able to expand our knowledge on fuzzy expert systems, as well as to find out what fuzzy expert system[sic] can perform.” (Student feedback 4). Another student felt that the opportunities to test, evaluate and improve the original model, was a realistic application of the theory being studied in the unit. “After a several testing of the saucer, it can defect[sic] all the other opponents and get a good value of the energy remaining. After completing this assignment, all team members are able to apply intelligent systems techniques to design and implement a solution to a realistic problem.” (Student feedback 5). Fifty (41%) of the one hundred and twenty students doing the unit completed the online survey. The timing of the survey (last week of semester) may account for the number of students completing the survey, which may have been higher if the survey had been administered earlier. The survey asked students to reflect on their learning using the AI Toolkit. Eighty eight percent of the survey group (44 students) felt that using the AI toolkit provided them with extra support when learning about AI and Java programming. Ninety four percent of the survey group (47 students) reported that being able to see their code in the AI

Teaching an Undergraduate AI Course with Games and Simulation

503

toolkit as it was working helped them to better understand the underlying code. These students also felt that being able to see the animated version of the game together with the AI toolkit helped them to better understand AI principles and concepts (r = 0.414, p < 0.005). Forty two students (84%) felt that using games programming to understand AI principles was an appropriate way to learn about AI. Forty one students (82%) felt the competition and the AI toolkit was a realistic way of learning about intelligent systems. These students also reported that the competition was a positive motivational factor (r = 0.575, p > 0.005); being able to see the code as it was working alongside an animation of the game helped them with their understandings of AI principles and concepts (r = 0.433, p < 0.005) and the underlying Java code (r = 0.471, p < 0.005); and the AI toolkit and the competition provided an interesting and useful way of learning about AI principles (r = 0.754, p < 0.005). Forty two students (84%) felt that participating in the competition encouraged them to improve their programming and helped them to better understand the theory of fuzzy logic and systems. There was almost total agreement from students in the survey group that the AI toolkit and the competition provided an interesting and useful way of learning about AI principles (45 students or 90%). Student feedback reported in the university’s Unit and Teaching Evaluation Instrument (UTEI) also indicated a very positive response to the unit content and the assignments. Of the fifty-five students who responded to the UTEI, 89% were satisfied with the unit, while 93% felt the unit had challenged their thinking. Again, 93% of UTEI respondents felt that the assessments had assisted their learning, while 27% provided additional commentary in the comments section about the effectiveness of the assignments. These results indicate that student engagement with the unit materials and their understandings were significantly enhanced by the use of the AI Toolkit. Students’ feelings of self-efficacy and confidence are important indicators of how they feel about their learning. The fact that many students also took the time to add positive comments about the AI Toolkit in the UTEIs and the online survey at the completion of the unit is another indication that the students not only enjoyed the assessment task, but felt that the AI Toolkit and the task was a relevant and worthwhile learning experience.

7

Future Work

The research described in this paper is part of an ongoing project within the School of Computer and Information Science at Edith Cowan University to integrate games programming into undergraduate pedagogy. The AI unit is now running for the second time using a modified game. As the project progresses, a more in-depth survey will be conducted, while project logbooks will provide additional information about students’ learning experiences. A series of in-depth interviews will also be conducted with students to develop an instrument to ascertain if deep learning has occurred. The interviews will provide a rich source

504

P. Hingston, B. Combes, and M. Masek

of additional data about the use of the learning object (AI Toolkit), the effectiveness of embedded assessment and the type of learning taking place in this immersive learning environment. The in-depth interviews will be conducted by a Masters student and include a research study on the collection of consistent qualitative data using interview techniques. This aspect of the data collection represents a quality assurance measure of the qualitative data. The results and analysis of the second phase of the project should be available by April 2007. However, early observational and anecdotal evidence, results from the pilot survey and the positive feedback from the university UTEIs, indicate that the use of animated competitive simulation games in teaching AI techniques, adds another dimension to the learning experience that motivates students while actively engaging them in the learning and with the learning materials.

8

Conclusion

Integrating a competitive game and simulation exercise to teach AI techniques has provided students in an undergraduate computer science course with opportunities to actively engage with the learning materials and the learning experience. Feedback from students participating in the unit indicates that the simultaneous display of the graphical animation and an AI toolkit that shows the internal workings of their controllers, has helped them to develop mental models of the AI algorithms. The authentic assessment tasks also reflect real world practice and include team work, individual development using the action research cycle and a competitive element. Feedback indicates that the toolkit, the competition and the gaming is providing students with a richer, more relevant and enjoyable learning experience.

Acknowledgements The authors would like to thank Maria Woodhouse who acted as the Research Assistant, designed the deep learning checklist and undertook the data collection and analysis for the second phase of the project. Maria has since been awarded a Master of Information Services based on her work for the Project. This research is supported by an Edith Cowan University Teaching and Learning Grant.

References 1. About Learning, www.aboutlearning.com/ Last accessed 19 December 2005. 2. Atkin, M. S., Westbrook, D. L., and Cohen, P. R. “Capture the Flag: Military Simulation Meets Computer Games”, In Papers from the AAAI 1999 Spring Symposium on Artificial Intelligence and Computer Games, 1-5. Technical Report SS99-02. Menlo Park, Calif.: AAAI Press, 1999. 3. Brand, J.E. “Gameplay Australia 2005”, Gold Coast, Queensland, Bond University, 2005.

Teaching an Undergraduate AI Course with Games and Simulation

505

4. Byrne, R.W. and Whiten, A. (Eds) “Machiavellian Intelligence: Social Expertise and the Evolution of Intellect in Monkeys, Apes and Humans”, Oxford, Clarendon Press, 1988. 5. Campbell, M., Hoane, A.J. and Hsu, F.-h. “DeepBlue”, in Schaeffer, J. and van den Herik, J. (ed.) “Chips Challenging Champions: games, computer and Artificial Intelligence”, pp 3-9, Elsevier, Amsterdam, 2002. 6. Combes, B and Ring, J. “If you help us build it, we will come!” - The role of the Teacher Librarian as an online curriculum facilitator and innovator. Virtual paper presented at the ASLA Online I: Constructing Communities of Learning and Literacy Online Conference, May 2004. 7. Durkin, K. and Aisbett, K. “Computer games and Australians today” Sydney, Office of Film and Literature Classification, 1999. 8. Fogel, D. “Blondie24: Playing at the Edge of AI”, Morgan Kaufmann Publishers, September 2001. 9. Fraser, W.J. “The foundations of continuous assessment: its link to performancebased, authentic, competence-based and outcomes-based assessment”, University of Pretoria: Pretoria. Unpublished article, 1999. 10. Friedman, T. “Making sense of software: computer games and interactive textuality”, in Jones, S. (Ed.), Community in Cyberspace, Sage, Thousand Oaks, CA., 1994. 11. Hauser, M. “Minding the behaviour of deception”, in Whiten, A. and Byrne, R. (eds.) “Machiavellian Intelligence II”, Cambridge University Press, Cambridge, 1997. 12. Jayakanthan, R. “Application of computer games in the field of education”, The Electronic Library, 20 (2) 98 - 102, 2002. 13. Kumar, D. “Pedagogical dimensions of game playing”, ACM Intelligence Magazine, 10 (1), 2000. 14. Lawrence, R. “Teaching Data Structures using Competitive Games”, IEEE Transactions on Education, Volumn 9, No. 3, pp 205-260, IEEE, 2004. 15. Lepper, M. R., and Malone, T. W. “Intrinsic motivation and instructional effectiveness in computer-based education”, In R. E. Snow and M. J. Farr (Eds.), Aptitude, learning and instruction. Volume 3: Cognitive and affective process analysis. Hillsdale, NJ: Erlbaum, 1987. 16. Li, S. “Rock-em, sock-em Robocode!”, http://www-106.ibm.com/developerworks/ java/library/j-robocode/index.html, Last accessed 18th September 2004. 17. Malan, S.P.T. “The ’new paradigm’ of outcomes-based education in perspectives”, Journal of Family Ecology and Consumer Sciences. 28, p 22-28, 2000. 18. McCarthy, B. “Welcome to 4MAT”, www.aboutlearning.com/ Last accessed 13 October 2004. 19. Michael, D. and Chen, S. “Serious games - games that educate, train, and inform” Boston, MA, Thomson Course Technology PTR, 2006. 20. Muller, M. “Computer Go survey”, http://www.cs.ualberta.ca/ mmueller/cgo/ survey, 2001. 21. Negnevitsky, M. “Artificial Intelligence: A Guide to Intelligent Systems”, Pearson, Harlow, 2002. 22. Noda, I., Matsubara, H., Hiraki, K. and Frank, I. “Soccer server: A tool for research on multiagent systems”, Applied Artificial Intelligence, Vol. 12, pp 233-250, 1998. 23. Pillay, H. “An investigation of cognitive processes engaged in by recreational computer game players: Implications for skills of the future”, Journal of Research on Technology in Education, 34 (3) pp 336-351, 2002.

506

P. Hingston, B. Combes, and M. Masek

24. Schaeffer, J. and van den Herik, J. (Eds) “Chips Challenging Champions: games, computer and Artificial Intelligence”, Elsevier, Amsterdam, 2002. 25. Schaeffer, J. and van den Herik, J. “Games, computers and artificial intelligence”, in Schaeffer, J. and van den Herik, J. (Eds) “Chips Challenging Champions: games, computer and Artificial Intelligence”, pp 3-9, Elsevier, Amsterdam, 2002. 26. Shannon, C. “Programming a Computer for Playing Chess.” Phil. Mag. 41, 256275, 1950. 27. Sweetser, P. “Current AI in Games: A Review”, http://www.itee.uq.edu.au/ penny/Game%20AI%20Review.pdf. Last accessed 18 September 2004. 28. Tesauro, G. “Temporal difference learning and TD-gammon”, Journal of the ACM, Vol 38, No. 3, pp 58-68, 1995. 29. Turing, A.M. “Computing machinery and intelligence”, Mind, 59, pp 433-460, 1950. 30. von Neumann, J. and Morgenstern, O “The Theory of Games and Economic Behavior”, Princeton University Press, 1944. 31. Whiten, A. and Byrne, R.W. (Eds) “Machiavellian Intelligence II: Extensions and Evaluations”, Cambridge, Cambridge University Press, 1997.

Experimental Approach to Affective Interaction in Games Holger Diener and Karina Oertel Fraunhofer Institute for Computer Graphics, Joachim-Jungius-Str.11, 18055 Rostock, Germany {holger.diener, karina.oertel}@igd-r.fraunhofer.de

Abstract. Current findings suggest that human-computer interaction following the basics of human-human interaction, in which emotions play a critical role. We performed a set of consecutive experiments in a laboratory environment for identification, recognition, visualization and interactive computing of affective states within a gaming application framework. To start our research and to get a data base we analyzed more than 90 test hours of user tests using rating-scales and physiological measurements. As results we provide (1) a mini-game with extra features for the induction and obtaining of affective states, (2) integrated data mining methods with recognition rates up to 70 percent, (3) different kind of visual representation of recognized emotions, and (4) an architecture for control of an affective game.

1 Introduction Good HCI design means to develop usable, easy to learn and motivating interactive systems. A lot of methods and standards for the implementation of usability do exist, but a lack of adequate software applications still remains. Current studies show the relevance of computer games for user-centered design (e.g. [2]). Since computer games exist more than 30 years, many users of today's applications were playing games when they were young and got used to a certain game appearance. Therefore, they expect applications to appear like games [8] and to be as easy to use as games. Computer games could be a good source to mediate functionality and motivate users and are possibly one of the most successful application domains in the history of interactive systems. Game based applications can gain a new level of quality of use, if they will incorporate the analysis of and adequate reaction to user emotions [7]. This is especially true for edutainment applications which need to stimulate the motivation of users in order to be successful. Recognition of emotions is a key technology for understanding the motivation of users. In this paper we describe an experimental approach which tended to get a better understanding of emotion while playing a computer game and to identify and manipulate emotion inducing features. We firstly introduce approaches to structure and to assess emotion in human-computer interaction and describe the experimental setup for our gaming studies. Second, we elucidate the methods, measurements and concepts. Third, we present and discuss our results. Finally, a conclusion and an outlook on future work complete this paper. Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, pp. 507 – 518, 2006. © Springer-Verlag Berlin Heidelberg 2006

508

H. Diener and K. Oertel

1.1 Concept and Structure of Basic Emotions To describe the structure and to get a classification scheme of emotion two key approaches were often deployed: First a discrete approach, claiming the existence of universal basic emotions (e.g. [3]) and second a dimensional approach (e.g. [10]). Dimensional emotion theories assume dimensions rather than discrete categories to be the basis of emotion.

Fig. 1. Russel's Circumplex Model of Affect (4 quadrants for classification of emotion)

According to a dimensional view of the structure of emotion, all emotions are characterized by their valence and arousal. However, arousal and valence are not claimed to be the only dimensions or to be sufficient to differentiate equally between all emotions, but they have shown to be the two main ones. We will base our proposal on describing emotions in an affective computing environment on a two-dimensional coordinate system according to Russell’s circumplex model of affect (see figure 1), which is characterized by valence (miserable-pleased) and arousal (sleepy-aroused). 1.2 Relation Between Emotions and Performance Arousal is a major aspect of many learning theories and is closely related to other concepts such as anxiety, attention, agitation, stress and motivation. One finding with respect to arousal is the Yerkes-Dodson law, what predicts an inverted U-shaped function between arousal and performance. It dictates that performance increases with arousal, but only to a certain point: when levels of arousal become too high, performance will decrease. It has been proposed, that different tasks may require different levels of arousal. So the optimal level of arousal is lower for more difficult or intellectually tasks and higher for tasks requiring endurance and persistence. 1.3 Assessment of Emotions in Computational Environments Emotions modulates almost all modes of human communication – word choice, tone of voice, facial expression, physical behavior, posture, skin temperature and clamminess,

Experimental Approach to Affective Interaction in Games

509

respiration, muscle tension, and more [7]. But we have to consider that in a computational environment emotions would probably not very strong and possibly only a restricted number of emotions occur. Furthermore measuring techniques have to be non-intrusive, fast, non-interrupting and simply applicable. However, some stable results could be found of variables like heart rate, skin conductance level, and blood pressure that seem to characterize certain basic emotions and require no complex test set-up. For a correlation between physiological variables and the affective dimensions of valence and arousal several studies have provided evidence [e.g. 9, 11]. Despite these findings, the additional obtaining of subjective data still remains essential to could define stable correlations.

Fig. 2. The scales valence (top) and arousal (bottom) of the Self-Assessment Manikin (SAM)

The Self-Assessment-Manikin (SAM) [4] is an instrument which has been widely accepted and often used for obtaining subjective data. It is based on Russel’s circumplex model of affect [10] and consists of pictures of 5 manikins each in different states of a) valence; b) arousal and c) dominance – of which mostly the first two are used (see figure 2).

2 Experimental Setup To investigate emotions that occur during the interaction with a computer game, we chose an application that should be allowed to change easily, to enable the application to induce or manipulation different emotions without attracting the subject’s suspicion. For example, the state of a word processor only changes when the user enters something or clicks a button and is in that case highly determined by the user. In an application that changes its state itself, like a game that requires the user to react on something, the user has less influence on the game, and the experimenter who decides how the application behaves has more influence options. The measurement of the skin temperature and the skin conductance level from the subject’s left hand accounts for a further constraint: The application should allow a single-handed interaction. These criteria are met by the computer game Tetris. Tetris is known for its simple but intriguing gameplay. Its popularity should make sure, that our subjects like the game, and are trying to be successful in playing it. Since the task of the player is to react on different pieces that occur randomly the experimenter has a high level of control. For example, varying the speed or form of the dropping pieces could induce different emotions. Additionally, Tetris can be played with a single hand.

510

H. Diener and K. Oertel

Fig. 3. User interface of EmoTetris; different levels

A new Tetris-game according to our requirements was developed: EmoTetris [6], which differs from available Tetris games in several ways. First, it was developed so that the game could be easily modified, even without any programming knowledge. Then, the original gameplay was extended to offer a broader range of possible manipulations. A method to present online-questionnaires was also included. At last, the program logs the user input and its state and features a method to synchronize the game data with the physiological data collected by another device. Unlike other games, EmoTetris is completely configurable by configuration files. This allows the experimenter to use different graphics, sounds and define the form, speed and behavior of different pieces. As mentioned above, the gameplay was also modified. First, new types of pieces were introduced, like the meteors that drop very fast. Second, special events may occur during the game. For example, an animation may announce that the score of the player has been reduced or that a new level is entered. Third, playing conditions were added. A condition is a specific set of game properties. It contains among others the frequency, in which the various pieces appear, the kind of feedback given or the game speed. Fourth, we developed a method to display online-questionnaires. The questionnaires were used to assess the subject’s mood during the game. The questionnaire was designed so that it could be filled out with a minimum of mouse clicks, to reduce time consumption and thus interference with the game. Fifth, EmoTetris can read a script that contains a timetable for different events like the onset of a new condition or the termination of the game. In order to make EmoTetris a valuable scientific tool, the program has the capability to log the user input and the state (for example: piece position, pile height, playing condition, etc.) in short intervals. Additionally, EmoTetris may synchronize its output data with data from other logging devices using an IP network connection.

3 Recognition and Manipulation of Affective States To make a system reacting to affective states it has to be able to recognize emotion and therefore a clear understanding of how relevant emotions are characterized is

Experimental Approach to Affective Interaction in Games

511

required. So we are describing 4 phases of our experimental approach: firstly the identification of affective states as a preparatory study, secondly the induction of, and thirdly the recognition and representation of affective states, and finally the integration. 3.1 Identification of Affective States To identify and describe affective states according to the two-dimensional approach we developed two test scenarios. The first scenario contained a writing task (tests participants had to write an official letter using MS Word) for an interaction of neutral valence and arousal. The second scenario contained a playing task (test participants could choose and play one minigame of four: Pacman, Magic Balls, Smashing and Solitaire) and was placed in quadrant I of Russel's Circumplex Model (see figure 1) for a positive and arousing interaction. A total of 10 subjects participated (7 males, 3 females). They mostly were students at the University of Rostock with an average age of 25.1 years. All participants had experiences with standard software and WWW and stated to use the computer often or very often. The experiment took place in a laboratory environment using the RealEYES-Setup1. It consisted of 10 sessions, in each of which one person participated at one time. All sessions began with a short introduction of the participant and closed with a questionnaire for individual data. A test observer controlled the task performance from a remote work station. To define the emotions we employed the SAM rating scale (see figure 2). We used a 9ary instead of this 5ary scale to obtain more detailed self-reports. The test participants had to rate arousal at first and valence after that by pressing especially marked keys on the keyboard. For rating an acoustic signal sounded every 90 or 60 seconds or even more often whenever the observer watched abnormalities in face video or physiological data. Physiological data, that means stress level (galvanic skin response) and heart rate (pulse), were recorded simultaneously and registered in log files. They were inspected as mean values of an interval of 10 seconds. These started each 10 seconds before the acoustic signal for self rating and ended with this signal. 3.2 Induction of Affective States To induce and gather different affective states during human-computer interaction the test subjects had to play EmoTetris in different playing conditions and were unaware of the emotion-inducing manipulations. The conditions were introduced to enlarge the variability of the players' emotions and differed mainly in game speed, the feedback given, and the frequency in which the various types of pieces occurred. The data recording took place in a laboratory environment and lasted five weeks. Every workday of this time two participants (males, aged: 28 and 35) were video-observed while playing EmoTetris, approximately 25 minutes semi-daily. The recorded data were entered into our emotion database [1] for analysis with SPSS and data mining techniques. 1

The RealEYES-System was developed for usability studies and allows a synchronous recording and playback of different user and system data; more at http://www.igdr.fraunhofer.de.

512

H. Diener and K. Oertel

The Tetris-principle has been extended in EmoTetris by adding events and a set of new pieces. An event is a sudden change in the game. An event was accompanied by an image or an animation and a specific sound, in order to draw attention to the change and to make the change more plausible. Additionally, four more pieces with special characteristics were included in the game. The pieces were designed to match all the sectors of Russell’s model. Arousal was manipulated by the speed of the pieces in a way, that more arousing pieces were faster. The valence was manipulated by their usefulness. The experimental conditions were developed following two different approaches. First, after setting up a baseline condition, we tried to modify this condition in four ways, so that every section of Russell’s model was matched by one condition. Thus, we tried to induce joy, anger, boredom or contentment. Second, two less theory-based conditions were added that represent common computer interaction experiences. The widespread situation in which an application ceased to react on user input was operationalized by a “loss-of-control”-condition. The loss of control was basically realized by reducing the reliability of the keyboard so that 20% of the keystrokes were not processed. The more positive feeling that accompanies the interaction with new and better software was accounted for in two new-conditions. Each new-condition was either a joy- or a baseline-condition enhanced with more animations and feedback, both will subsequently be handled as a single new-condition. In the game, each condition lasted for approximately six minutes, including the time to fill out questionnaires. The conditions differed in a number of variables (see table 1). Table 1. Test conditions – Overview on major differences in each test condition (abstract) for experiment 2 joy

anger

baseline

game speed keyboard reliability feedback score increase/ reduction preview invalid/ event piece characteristics most frequent piece preview animations and sounds

medium

high

high

low

low

100%

100%

100%

100%

100%

80%

neutral

positive

negative

negative

positive

rarely any

1/1

2/0

0/2

boredom

0/0

contentment

loss-ofcontrol medium

condition

2/0

2/3

1

0

1

1

0

normal

rather useful

rather hindering

rather hindering

no preview

rather useful

---

lava

meteors

parachutes

balloons

1

2

1

1

2

rather hindering meteors, parachutes 0

normal

normal

normal

normal

normal

few

The game speed was set high compared to the baseline condition in the joy- and anger-condition. It was low in the boredom- and contentment-condition. In all other conditions, it was set to baseline. Before the subject played their first round of EmoTetris, a presentation was shown that made the subjects familiar with the game. Then, the subject were connected to the physiological instruments and started to play. Each session were divided in four segments of about 6 minutes that differed in the playing condition. In the first and

Experimental Approach to Affective Interaction in Games

513

third segment, the condition was always baseline. In the second and fourth segment a combination of the conditions baseline, joy, contentment, anger, boredom and loss of control was applied, resulting in 30 necessary sessions. In the first 20 seconds of the second and fourth segment, the player was transported to the third level. The order of the sessions was arranged by giving randomization, but giving higher priority to the sessions including baseline, anger and joy. Thus, conditions including anger, joy or baseline tended to appear early in the experiment. After that, the sessions, in which the data was not properly recorded, were repeated. The last three sessions included the new-condition. Taken together, subject A participated in 36 Sessions (3 sessions repeated) and subject B participated in 39 sessions (6 sessions repeated). Physiological data was collected in three ways. A breast-belt was used to record the heart rate. The skin conductance level and skin temperature were measured on the middle- and ring finger of the left hand. This configuration was chosen to ensure only a small influence of the sensors on the test subject while playing. The data was recorded with a sampling rate of 20Hz. To synchronize the data EmoTetris created timestamps every 50ms and sent them to the logging application via the network interface. Subjective data was assessed via online-questionnaires at fixed intervals of the gameplay and had to be filled out with the mouse. To minimize the interference of the questionnaire with the game, we had to construct very short questionnaires. We assessed a range of emotions with visual analogous scales. This method has been shown to deliver valid results [9]. 3.3 Recognition and Representation of Affective States To recognize emotions based on physiological data we applied data mining and machine learning techniques. The WEKA toolkit was employed to carry out the steps feature extraction, selection and classification of data. For the feature selection the raw data of heart rate, skin conductance and skin temperature was divided into intervals, starting right before the self assessments. Our tests showed that a time interval size of about 100s leads to well distinguishable features. In the next step the features were extracted from every interval using common preprocessing filters. To achieve a robust classification special care was taken to balance the number of assembled learning examples from different affective states. For the following learning phase the classifier "J48" was used (an implementation of the C4.5 algorithm by Quinlan from the data mining toolkit WEKA). One advantage of this classifier is the clear and concise output of the learned concept in form of a decision tree. With the help of the decision tree the work of the classifier is transparent and comprehensible. In addition, the features that are most useful for detection can be derived from the tree (e.g. the measure for the variation of the galvanic skin response was used in high levels in many of the decision trees, thus showing strong relevance). As we were able to classify the day of the test based on physiological data with 85% confidence, further refinements of the classification included the application of a day-matrix and a fisher projection (FP) to level out day dependencies in the physiological data. To give an informative feedback and represent recognized results we provide different visualizations: (1) verbal; (2) star-plot based visualization; (3) comic faces

514

H. Diener and K. Oertel

we have designed and in-house tested for their differentiability and expressiveness (see figure 4). The visualization in the star-plot enabled a process-view of valence and arousal beside the display of recent classification result. The visualization is based on the preprocessed data and is an integrated feature of EmoTetris.

Fig. 4. EmoTetris with integrated online-visualization of affective states; right further comic faces

3.4 Integration The visualizations demonstrate the prioritizing function. To integrate a emotion recognition module into an application we developed a special operative component (integration agent) using preprocessed data. This component aims to guarantee the optimum of arousal (see chapter 1.2) and consists of 3 functions (see figure 5): (1) evaluation function; (2) catalogue of events; (3) inference function. The evaluation function valuates the active affective state and defines the difference between the active state and the optimum state. The inference function calculates the adequate event for the aimed affective state depending on the recognized affective state. The catalogue of events protocols the relevance of single events for intended manipulation of affective states.

Fig. 5. Functioning of the integration module

Experimental Approach to Affective Interaction in Games

515

4 Results and Discussion 4.1 Identification of Affective States To analyze the log files, we extracted the self-ratings and the physiological data2 which go with them and found out: (1) The self-ratings for playing are concentrated in q1 (positive, arousing); (2) The self-ratings for writing are concentrated in the middle of the coordinate system (neutral emotional and physical arousal); and (3) Physiological data3 show a medial value of heart rate and stress level during playing and high values during writing [5]. Indeed the felt arousal in interaction with a computer game found no expression in these. 4.2 Induction of Affective States To evaluate the quality of the test conditions [6] for the induction of affective states we carried out analyses of variance of the data from the questionnaires using SPSS 124. Subject 1. The analysis of variance (ANOVA) is significant for all items (df = 6, p ≤ .001). The responses to the questions on valence correspond to the expectations. The values of the loss-of-control-condition are significantly lower than the values of the other conditions (p ≤ .001), with the exception of anger. The values for arousal vary. The answers to questions on joy are relatively high in the joy-condition and very low in the loss-of-control- and in the anger-condition. These differences are significant (p ≤ .05). Loss of control differs significantly from all other conditions (p ≤ .001), only the contrast to the anger-condition is irrelevant (p = 0.57). Higher values for anger were conversely stated in the loss-of-control and in the anger-condition (not significant). In the other conditions these values were lower. High values were stated (p ≤ .001, only the contrast to the anger-condition is lower, p ≤ .01) on the question on helplessness particularly in the loss-of-control-condition, but also the joy- and the anger-condition tend to result in higher values. The values for anxiety are in general very low, so that they are hard to interpret. Only in the loss-of-control-condition they are relatively high (p ≤ .001). Boredom increases particularly in the loss-of-control- and in the angercondition. These conditions differ from the joy-condition partly significant (p ≤ .001 in contrast to loss-of-control and p = .062 in contrast to anger). The analysis of other variables shows results which are hard to interpret for boredom. The anger- and the loss-of-control-condition seem to induce low results for euphoria. The statements for tenseness, contentment and perceived skill are irregular. The time perception seems to be influenced by the faster joy- and anger-condition, but longer time intervals assessed for these conditions. This could be caused by a higher number of pieces. Subject 2. The analysis of variance is significant for all items, apart from valence, anxiety, tenseness and perceived skill (df = 6, p ≤ .001, p ≤ .016 for joy). Arousal was relatively high assessed by subject 2 in the joy-condition. It is significantly higher 2

3

4

To guarantee a comparability of data, we executed the z-transformation for heart rate and stress level of every participant. Between -1 and +1 are two third of all data for stress level and heart rate. The mean value is 0. We assume a normal distribution for stress level and heart rate. The new-condition was not yet included in our analysis.

516

H. Diener and K. Oertel

than in the baseline- and in the boredom-condition (p ≤ .001). The assessment of joy shows only a difference between the joy- and the anger-condition (p ≤ .05) as well as between the baseline- and the anger-condition (p ≤ .05). Like subject 1 subject 2 particularly feels anger in the loss-of-control-condition (p ≤ .001 in contrast to all other conditions, apart from the anger-condition) and in the anger-condition (p ≤ .05 in contrast to the baseline-, joy- and boredom-condition). In the loss-of-controlcondition the felt helplessness is higher than values of all other conditions (p ≤ .001, only in contrast to anger p ≤ .05). The same results were found for boredom. Values for boredom were in the loss-of-control-condition higher than in the joy- (p ≤ 0.01); baseline-, contentment- and anger-condition (each p ≤ .05). The boredom-condition induces values for boredom which are between the loss-of-control-condition and the other conditions. All other questions show not relevant differences. 4.3 Recognition and Representation of Affective States The recognition rates (see Table 2) are based on comparisons with subjective and possibly unreliable data from the questionnaires; if a test subject were not able to properly estimate the emotional state or chose a wrong value in the rating widget there is no way to get good classification results at this stage of experimental research. Since the final application of the recognition of emotions from physiological data has to work in an on-line scenario, the application of methods like the day-matrix becomes more difficult. A possible solution could be to switch between pre-computed data-sets after recording a few seconds of data. Another requirement of the final application is the detection of a prevalent emotion. To detect a prevalent emotion we are currently experimenting with a factor analysis. The factor analysis could also be useful to eliminate inconsistent self ratings. Table 2. Best results (percent) for offline-recognition

5 Conclusion This paper has suggested that emotions play a critical role in human-computer interaction. Based on recent findings we assumed, that computer games are a good choice for emotion induction experiments and will influence a person’s mood automatially, i.e. without subjects actively trying to get into the prescribed emotional state.

Experimental Approach to Affective Interaction in Games

517

To summarize, our findings from our emotion identification experiments indicate that the physical load during a writing task could be considerable even though it was felt as a relatively relaxing work. This result should be taken seriously and examined in further analyses. Conversely, playing certain computer games causes that users feel emotionally and physically stimulated, but do not imply a high physical arousal. Moreover these computer games are obviously qualified to relax and to reduce physical arousal to an average value. However, this is not true for all computer games, since many of them are designed to create a high physical arousal (e.g. Ego Shooter). Five further conclusions can be made from our research. First, tests which based on our game EmoTetris allow a minimum invasive emotion induction and complex recording applied in a laboratory. Furthermore, the program is of a flexible design which allows extensions of further emotion-inducing components and the integration of other bio-sensor data. Second, EmoTetris is an effective instrument to activate systematically different feelings and affective expressions especially for the humancomputer interaction domain. Third, precise features and logging results we gathered provide a qualified data basis for the emotion recognition. Fourth, compliance values between subjective assessment and induced emotions indicate that our induction was successful. Five, recognition rates of emotions based on physiological data – using data mining methods – are up to 70 percent. To recognize emotions decision trees help to distinguish important from unimportant features but much better classification results may be achieved using the naive Bayes method. A weakness of our approach is that the results we obtained with data of 10 + 2 test participants may not be the same for other subjects. However, the methodology for inducing, gathering and analyzing the data in this paper is not dependent on the subjects; the approach described in this paper is general. The provided results can be considered as indicators of emotion contributing to specify valid pattern of physiological responses to differentiate between emotional states.

References 1. M. Blech, Ch. Peter, R. Stahl, J. Voskamp and B. Urban, “Setting up a multimodal database for multi-study emotion research in HCI,” (accepted for publication), 11th International Conference on Human-Computer Interaction HCI, July 2005, Las Vegas, Nevada, to be published. 2. J. Dyck, D. Pinelle, D., B. Brown, C. Gutwin: “Learning from Games: HCI Design Innovations in Entertainment Software,” In Proc. GI’03, pp. 237–246, 2003. 3. P. Ekman, “An argument for basic emotions,” Cognition and Emotion, 6, pp. 169-200, 1992. 4. P. J. Lang, “Behavioral treatment and bio-behavioral assessment: computer applications,” In J. B. Sidowski, J. H. Johnson and T. A. Williams (Eds.), “Technology in Mental Health Care Delivery Systems,” pp. 119-137, Ablex, Norwood, NJ, 1980. 5. K. Oertel, G. Fischer and H. Diener, “Physiological Response to Games and Non-Games: A Contrastive Study,” 3rd International Conference on Entertainment Computing ICEC, Sept. 2004, Eindhoven, The Netherlands, pp. 402-405. 6. K. Oertel, R. Schultz, M. Blech, O. Herbort and B. Urban, “EmoTetris for Recognition of Affective States,” 11th International Conference on Human-Computer Interaction HCI, July 2005, Las Vegas, Nevada, CD-ROM.

518

H. Diener and K. Oertel

7. R. W. Picard, E. Vyzas, and J. Healy, “Toward machine emotional intelligence: Analysis of affective pychological state,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(10), pp. 1175-1191, 2001. 8. Prensky, M.: Digital Game-Based Learning. New York: McGraw Hill. 2001 9. C. M. van Reekum, T. Johnston, R. Banse, A. Etter, T. Wehrle and K. R. Scherer, “Psychophysiological responses to appraisal dimensions in a computer game,” Cognition and Emotion, 18(5), pp. 663-688, 2004. 10. J. A. Russell, “A Circumplex Model of Affect,” Journal of Personality and Social Psychology, 39, pp. 1161-1178, 1980. 11. P. Zimmermann, S. Guttorrmsen, B. Danuser and P. Gomez, “Affective Computing – A Rationale for Measuring Mood with Mouse and Keyboard,“ International Journal of Occupational Safety and Ergonomics, 9(4), pp. 539-551, 2003.

A Framework to Help Designing Innovative Massively Multiplayer Online Games Interactions
Anne-Gwenn Bosser Kwansei Gakuin University, School of Science and Technology, 2-1 Gakuen Sanda 669-1337, Japan

Abstract. This paper presents FITGap, an object-oriented framework dedicated to designing innovative interactions occurring in a Virtual Environment such as a Massively Multiplayer Online Game (MMOG). This framework has been designed as the underlying software of a future development environment for online games prototyping. FITGap is an open framework based on an extensible basic-blocks library, and uses a deterministic scheduling model to ease the development and tuning of the final application. Its semantics is well-defined to allow future analysis tools.

1

Playing in a Persistent Virtual World

As a consequence of the growth of the Internet, a new genre of game has emerged since a little less than a decade. Since the worldwide success of games like Everquest1 and Lineage2 , players from all around the world enjoy a second social life on Massively Multiplayer Online Games. In order to participate in the virtual world in which the game takes place, the player builds their own avatars, choosing their characteristics among a number of possibilities provided by the game. The game then consists of a persistent virtual world, where hundreds or thousands of people, depending on the size of the game, can play together or against each other in a real-time fashion. The different types of characters are balanced in order to be complementary and players get an advantage in collaborating with each other. 1.1

Current Gaming in Virtual Worlds

The current Massively Multiplayers Online Games (MMOG) follow the traditional game-play set up as a standard since the first commercial success of the genre, which is inspired by the text-based games known since the end of the 70’ under the acronym MUD (Multi-User Dungeons). Fights between players or


1 2

Part of this work has been supported by the Japan Society for the Promotion of Science. http://eqlive.station.sony.com/ http://www.lineage.com/

Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, pp. 519–528, 2006. c Springer-Verlag Berlin Heidelberg 2006


520

A.-G. Bosser

against Non Player Characters (NPC) are turn-based, hit-points are computed using a random factor applied to competence degrees. The targets are locked, and it is not easy for a player to avoid a hit using reaction abilities. Even in the very recent World of Warcraft3 , the way players interact with each other or with the virtual world is still more or less the same than in Everquest. This kind of game-play favours game-designs based on strategy in combats and management of avatars abilities upon interactivity, in order to provide a level of intricacy and subtlety sufficient to retain players’ interest. Indeed, when reaching a high level of competence, and in order to fully enjoy the scenario of the game, a player has to gather a full team of other complementary players, each one being responsible of a special task and thus highly specialised. The first issue is that an efficient team can take a long time to gather at the right place for enjoying the gaming experience. The second major issue is that the way the player builds-up their avatar day after day, to gain abilities and special equipment must be optimal. In order to make sure of always finding team mates, players are thus encouraged to follow the avatar management guidelines which appear regularly even during the open beta-test on dedicated community forums to ensure they will stay useful for other players. Therefore, MMOG as we experience them nowadays are adapted to a hard-core gaming, nearly professional audience. This may also be a factor contributing to the addiction from which some MMOG players suffer : to enjoy all the aspects of these games, you have to play a lot, some people would say too much and this addiction issue starts to be considered very seriously in several countries. 1.2

What Will Be the Next Online Games?

This lack of diversity in current game-plays is not only due to the complex nature of the communications on the internet. As a matter of fact, the stateof-the-art knowledge available to enhance specific interactions is growing with the new interest of the academic research in this domain. But due to the huge risk that companies take when they decide to invest in such a game, they often choose to rely on solutions that have been proven safe and sound in a previous game. However, with a new generation of players used to First Person Shooters (FPS) such as Quake4 accessing to Massively Multiplayer Games, we currently witness some attempts to incorporate more interactivity in game-plays. In this kind of game, the abilities of the players themselves have a great influence: their reflexes and real-time decisions have as much importance as their avatars abilities. Neocron5 is an example of such an attempt of a more reactive game-play. While the game didn’t succeed to reach maturity during its two years of existence due to strong technical issues, it raised a very strong interest in the 3 4 5

http://www.worldofwarcraft.com/ http://www.planetquake.com/ http://www.neocron.com

A Framework to Help Designing Innovative MMOG Interactions

521

online gaming community. As a matter of fact, this kind of game-play can be designed in order to provide more freedom to the player, because the necessary level of intricacy and challenge can be reached not relying on aspects depending solely on gaming time (like a perfect avatar abilities and equipment management), but depending on the players abilities. It also provides more freedom to implement a game scenario that the player can enjoy as soon as he is connected. We think that the evolution of the market will soon show a great interest for such hybrid game-plays, inspired by FPS games or other styles of multiplayer games. However, it is still very difficult to achieve an innovative game-play in a massive, persistent and potentially worldwide context. 1.3

The Importance of Interactions Design

In terms of game-play, the main difference between a typical MMOG and other multiplayer games resides in the way the players interact with each other and with the virtual world. Let us take back the comparison between a typical MMOG and a FPS, and consider the interaction which consists in shooting another player’s avatar: In a FPS game, this interaction is modelled in a realistic way: the player has to aim a moving target (the avatar of another player), which will be considered as hit only if the player has been skilful enough. The impact of the hit can also differ, depending on the part of the body of the avatar that has been hit. The perception of the other players’ avatars moves has thus to be as accurate as possible, or at least synchronised in a way that would be considered as fair for all the players involved in the action. In order to achieve this requirement, the position updates of the targeted avatar have to be transmitted to the player aiming at it as fast as possible. This is generally implemented using a network protocol which is fast to the detriment of reliability, and using a communication model where updates are transmitted as soon as available. In a typical MMOG, avatars positions do not have the same importance. Once the target is selected, through a simple mouse-click, the player decides which attacks he launches according to a strategic analysis, without having to worry about aiming. For each attack, the server decides of the success and amount of hit-points using a random formula taking into account the avatars’ abilities. It is thus not necessary to transmit positions updates as fast as in a FPS game-play. Actually, it can even be a bad technical decision: the region of the virtual world in which the fight takes place can be very crowded, and players’ computers could be saturated by messages they don’t have the time to process. In order to provide innovative game-play, technical choices have to match the requirements of the game-designed interactions. However, these technical choices are very difficult to make, due to the complex nature of a MMOG. In the next section, we describe the technical aspects involved when designing an interaction to show its complexity and to demonstrate that a tool helping in this task would be very useful.

522

2

A.-G. Bosser

Technical Challenges

A MMOG is a persistent application, which has to be accessed ideally 24/7. The level of synchronisation between all the hosts of the application has to be as high as possible, or simulated by intricate techniques to provide a good level of immersion for all the players in the game. Security issues, including cheating, are also a very important concern since a single cheater can ruin the pleasure of thousands of players or destroy the carefully designed economic balance of the world. Of course, scalability is also an important issue, as a massive game should be able to cope with a lot of players interacting with each-other, and sudden peaks of activity in any region of the virtual world. As a distributed application, a MMOG is also a very demanding application in terms of network and hardware resources, which have to be evaluated carefully since their consumption is tightly correlated to deployment costs and since their lack can prevent the game to be playable. Having all this requirements in mind, we conducted in [5] a detailed review about the state-of-the-art techniques currently available to achieve a MMOG, which we will sum-up here: Playability: We studied the quality of the interaction requirements for different genres of games. For instance, depending on game-play and technical choices, [13, 14] give synchronisation time bearable by the players. Distributed Architectures: A lot of authors have been proposing architectures for multiplayer games. Despite the fact that they have troubles addressing cheating issues, some Peer to Peer architecture have been designed together with synchronisation techniques [8]. Server architecture are a more realistic approach, and vary according to the design of the logical server into several tiers with different responsibilities. In [1], a first tier manages client communication using an interest management service, saving bandwidth at computation costs, while a second server-tier manages the game state. In [7], the logical server is divided in mirrored-servers, fully synchronised, and each client connects to a mirror according to the network topology, thus sparing the communication time between the clients and the logical server at synchronisation cost. The system RING [15] has been adapted to the classical architecture following a static division of the virtual world, thus sparing bandwidth by exploiting the fact that players interact with their neighbourhood. Communication Models: The quality of the application synchronisation is tightly correlated to the chosen communication model. Fine-grained techniques, such as distributed objects models like the publisher/subscriber model have also been used in the context of online games [3] despite the security issue they may raise. To help spare bandwidth, and because of the current imperfections of the IPmulticast, some proposals have been made to use application-level multicast like described in [10]. Interest Management techniques [9], exploit the specifics of virtual worlds like players perceptions. The accuracy of the bandwidth savings

A Framework to Help Designing Innovative MMOG Interactions

523

generally grows with the complexity of the computation, and can sometimes have an impact on the latency. Delay and Latency Compensating Techniques: Despite raising a real security issue, delay and latency compensating techniques [2] are often necessary to provide the player with the feeling of a realistic immersion in the game. In particular, a good use of the techniques called Dead-Reckoning, while actually decreasing the level of synchronisation of the application by introducing potentially inaccurate prediction of future states, can provide the player with the feeling that this level is high. The first conclusion of this research is that the technical design of each interaction in the game is tightly correlated to its requirements. We also concluded that whereas the quality of the interactions are tightly correlated to game-play choices, each technical solution optimising the use of one of the desired characteristic of the game (scalability, synchronisation between the hosts of the application, system resources consumption, latency, security and robustness against cheating) has an impact on another one.

3

A Prototyping Tool to Tune In-Game Interactions

The technical design of in-game interactions according to game-play choices is critical for the success of the project: the game will be playable, enjoyable by players, only if the requirements of the interaction are met. As we have seen earlier, there is a wide range of technical solutions to mix and match to achieve such a goal. Thus, we think this design should be done as early as possible, in parallel with the game design itself in order to verify its feasibility, instead of waiting the final stage of the beta-test to witness that the technical design does not meet the expectations. The safest way to achieve an innovative game-design is to handle the whole project in an iterative way, by realising a succession of always more accurate prototypes. As a matter of fact, innovative software industries are currently showing more and more interest for agile developments methods. Thus, our proposal consists in using a prototyping environment to help matching the game design with the technical design. In order to build such a development environment, which should include static and dynamic analysis tools to help checking the technical solutions while the deployment conditions (bandwidth, Internet-like network context, deployment architecture, number of players) are not yet encountered, we designed and developed a framework with a strongly defined semantics [5]. The framework itself has been developed in Java, as the current aim was to demonstrate how to integrate it in the complete development solution and to illustrate its usage. Although we are not currently dealing with performance issues, the accent has been put on a realistic conception. The most complex aspects of the framework have been formalised in an operational semantics, for both local and distributed computations. As we have seen earlier, designing an interaction in a MMOG is a matter of giving a specific solution to a trade-off, and this solution can not be generically

524

A.-G. Bosser

applied to all games, neither to different data within one game. This led us to a low-level model which does not take any technical decision narrowing the range of the possible designed interaction while highlighting the design issues. This low-level model is to be used with an extensible library facilitating the design of commonly used techniques in a more high-level fashion. The semantics of the framework execution is well-defined, has a good level of determinism and uses a simple and user-friendly threading model thus facilitating development, debugging and maintenance tasks while allowing the further development of accurate analysis tools in the future development environment.

4

The FITGap Underlying Model

A Virtual Environment can be basically viewed as a set of data replicated along a distributed architecture, and the purpose of creating such an application is to define how the consistency between each different view/host of the game will be achieved. The model we defined allows the organisation of these data into states, and the definition of replication models, which specify how, when and where each state-update on a local host has to be replicated. 4.1

Distributed Replications Model

A replication model is attached to a state and has three attributes defined by the user as follows, using framework primitives or the extensible building blocks library. The scope is the list of the hosts to which the state has to be sent. For instance, this can be the server for a client state replication model on a classical client-server architecture. The timeliness expresses when the state has to be propagated. For instance, this can be each time the state changes, or each time a specific condition linked to other states of the local host is verified.The communication properties is tightly correlated to the network protocol properties necessary for the replication, and includes traditional properties like reliability, ordering and encryption. A state may have several replication models representing several consistency requirements (for instance, a player move may be replicated in a real-time unreliable fashion to the other players each time the state changes, but periodically and reliably to the server for failure management). The attributes of a replication model are context-dependent, and can be described by using several user-defined states. When a replication is received by a host, a security check is made, to verify that the sending host is authorised to modify the replicated state before proceeding to the state update. A user-defined callback can be attached to each replication model, to specify some post-replication computations upon the local game state. Furthermore, additional behavioural components can be defined and processed by the framework if necessary. Those behavioural components are built

A Framework to Help Designing Innovative MMOG Interactions

525

by coupling a timeliness property, and a callback which is triggered when the timeliness is verified. We will use the term schedulable to refer either to a replication model or a behavioural component resulting from such an association. Both states and replication models are dynamically managed. They can be created and activated at runtime, according to the events occurring in the virtual world. This dynamic behaviour is defined by the user of the framework. When defining a schedulable, the user has to describe two sets of states: the observed states are the states which changes trigger the timeliness attributes to check if the condition is verified, and the necessary states are the states used to describe the attributes of the schedulable, or used in the attached callback computation. A schedulable can be triggered if and only if all its necessary and at least one of its observed states have been created. To facilitate the design of the application, we have also defined a state arithmetic and composition rules which allow the user to follow the intuitive notion of game objects composition. A more detailed description of the state arithmetic and application dynamics can be found in [4]. 4.2

Fair Threading Model

The processing of a callback on one host may be demanding, depending on the responsibilities of the state to which it is attached. Multi-threading is therefore a necessity to enable the user to share resources between callbacks, according to their complexity. Furthermore, the objects of the game are represented by states, and it is far more natural for the game designers to describe their behaviour in a concurrent style, object by object. Traditional preemptive approaches to scheduling have several disadvantages according to our aim which is to simplify the developer’s task. The programming style is complex, due to the necessity to control access to shared data and the difficulty of determining when a thread switch occurs. Furthermore, debugging a traditional multi-threaded program can be a nightmare because of the nonreproducibility implied by most models. Therefore, we have chosen a scheduling model inspired by a simplified version of Fair Thread[6]. This model is cooperative, thus easy to work with for the developer, who controls when each thread relinquishes control to the scheduler by using the cooperate instruction. The model defines a fully reproducible execution of the callbacks and the state-changes they control: the way the scheduler chooses the next thread to release is fully specified according to a strict roundrobin algorithm. Thus, modulo the events coming from the network or other external producers, the semantics of the execution is clearly defined, facilitating the testing and debugging tasks. The model is fair once a division of each callback in sequences of code separated by two cooperate instructions has been defined by the user. As a consequence, the user controls the amount of resources each thread uses, by adequately using the cooperate instruction for dividing each task into several arbitrary demanding sections. The callbacks are linked to the scheduler according to the order in which the replication models they belong to is triggered. When a new callback is linked to the

526

A.-G. Bosser

scheduler while others are still running, its first segment will be processed as soon as the current segments of the running callback have finished their execution. The next issue to consider is introduced by the use of a separate scheduler handling the callback execution process, and is a direct consequence of the two common ways to manage the communications in a distributed application: message passing updating style, and delta updating style. Indeed, some computations are more easily described when a callback execution uses the state values valid when the replication model was triggered (message passing updating style): for instance, controlling the validity of a player move according to other object locations at that moment, or, more generally, when a condition on a state value has changed between the replication model triggering and the actual callback processing. However, some other processes, such as a callback adding a delta to a state, require fresh values (delta updating style). Another example is the need to detect fresh state changes which make the callback execution obsolete such as during a costly path-finding algorithm. To address both situations, the schedulables are processed with state copies, and the user defines the way these copies are synchronised with the game-state. The necessary states are copied into a local memory once the schedulable is triggered. When the process described by the callback requires fresher values, the developer can use a new instruction, flash, to replace the current state values by the corresponding states values in the game-state memory on the local host. According to the semantics we have described, the user has the guarantee that between a flash and a cooperate instruction, a state value can not be changed by another callback execution. As a consequence, if the design does not allow this state to be modified directly through the network (as is the case for purely local state not subject to distant replication) or another external event, the state and the state copy have the same value between a flash and a cooperate instruction.

5 5.1

Comparison with Other Models FitGAP vs. Classical Models

State Replication Models vs Object/Agent Based Models: The model we propose is based on the fine-grained propagation of data arranged into states. In our model, the design is thus guided par the definition of the interactions. As opposed to object oriented or agent-oriented approaches, our model decouples the data from their processing, from behaviours. However, it is possible to design applications according to those paradigms by giving an adequate modelling of the states and their association with schedulables. This de-coupling we provide between data and computations is common to the philosophy of meta-programming models like aspects for instance. Fair Scheduling of Callbacks and Threads: We have sometimes used the vocabulary about multi-threading along this paper. However, and despite the fact that our implementation uses Java Threads, the Fair scheduling of callbacks we presented does not need all the power of modern threading models. One of the

A Framework to Help Designing Innovative MMOG Interactions

527

characteristics of these models is that threads share a memory. Here, the callbacks compute state copies, which are only synchronised with the game-state on-demand and which are not shared by the other computations. 5.2

Related Works

Virtual Environment System Object Model: In [12], the author proposes an object oriented universal architecture for the development of virtual worlds, ranging from FPS session-based games to MMOG. This architecture comprises 4 layers of object-oriented development, built one above each-other. This onion-like architecture has been designed in order to give a modular solution to the problem of the design of virtual worlds. VESLOM and our approach are similar in the way they advocate the use of an extensible basic-blocks library built above a generic execution model (called here the Universal Platform). One of the main difference is that this approach is guided by the architecture compared to our approach which is guided by the interactions. Some services of the Universal Platform are thus defined as replaceable and customisable basic-blocks in our library. OpenPING: In [11], the authors present an enhanced version of the platform developed in the context of the European project PING6 . This enhanced version includes the use of reflection, a meta-programming technique which aims at giving an application the ability to reason about itself, thus adapting to fluctuating execution conditions. The authors show how to use this ability to implement different degraded modes for bandwidth adaptation. This open model seems to be equivalent to our approach in terms of flexibility. The structural de-coupling between the data and the way they are processed, which can change depending on the context, is also a common aspect between OpenPING and our approach.

6

Current Achievements and Perspectives

We have now prototyped the framework using the Java language, including the fair scheduler, and a minimal toolbox of basic-blocks, in order to show that the model can address the described issues leaving the choice of the accurate solution to the user, while defining where are the hooks to integrate it in a future development environment. Some typical interactions are being modelled in several toy-applications, in order to give accurate examples of how to use the framework to create a multi-player game and to demonstrate its generality. As our current aim is mainly to illustrate the use of the model we propose, we are not currently dealing with performance issues although our design choices are aware of it. We are currently working on a more dedicated fair scheduler implementation, as the current one relies on Java Threads and it is not the best choice for efficiency considering our light callbacks model. The most interesting perspective concerning a complete development environment is the possibility of including tools based on our model to perform some 6

http://www.pingproject.org

528

A.-G. Bosser

static analysis of the game prototype. As the approach is as deterministic as possible, we think we will be able to create realistic analysis tools to help matching the game designed interactions with the technical design or to help with the evaluation of the system resources necessary to deploy a MMOG.

References 1. L. Aarhus, K. Holmqvist, and M. Kirkengen. Generalized two-tier relevance filtering of computer game update events. In Proceedings of ACM NetGames, April 2002. 2. Y. W. Bernier. Latency compensating methods in client/server in-game protocol design and optimization. In Proceedings of the Game Developers Conferences, March 2000. 3. A. R. Bharambe, S. Rao, and S. Seshan. Mercury: A scalable publish subscribe system for internet games. In Proceedings of ACM NetGames, April 2002. 4. A.-G. Bosser. Replication model for designing multi-player games interactions. In Proceedings of CGAMES, November 2005. 5. A.-G. Bosser. R´eplications Distribu´ees pour la D´efinition des Interactions de Jeux Massivement Multi-Joueurs. PhD thesis, Universit´e Paris VII, 2005. 6. F. Boussinot. Java Fair Threads. INRIA Research Report 4139, 2001. http://www.inria.fr/rrrt/rr-4139.html. 7. E. Cronin, B. Filstrup, A. R. Kurc, and S. Jamin. An efficient synchronization mechanism for mirrored game architectures. In Proceedings of ACM NetGames, April 2002. 8. C. Diot and L. Gautier. A distributed architecture for multiplayer interactive applications on the internet. In IEEE Networks magazine, vol. 17, n.4, 1999. 9. K. L. Morse. Interest management in large-scale distributed simulations. Technical Report ICS-TR-96-27, Department of Information and Computer Science, University of California, Irvine, 1996. 10. L. E. Moser, Y. Amir, P. M. Melliar-Smith, and D. A. Agarwal. Extended virtual synchrony. In The 14th IEEE International Conference on Distributed Computing Systems (ICDCS), 1994. 11. P. Okanda and G. Blair. Openping: A reflective middleware for the construction of adaptive networked game applications. In Proceedings of ACM NetGames, August 2004. 12. M. Oliveira. Virtual environment system layered object model. In Proceedings of the conference Advances in Computer Entertainment, 2004. 13. L. Pantel and L. C. Wolf. On the impact of delay on real-time multiplayer games. In Proceedings of the Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV), May 2002. 14. N. Sheldon, E. Girard, S. Borg, M. Claypool, and E. Agu. The effect of latency on user performance in Warcraft III. In Proceedings of ACM NetGames, May 2003. 15. M. W. Stefan Fiedler and M. Weber. Network topologies for scalable multi-user virtual environments. In Proceedings of the 1996 Virtual Reality Annual International Symposium (VRAIS 96). IEE Computer Society, 1996.

Earth and Planetary System Science Game Engine Falko Kuester, Gloria Brown-Simmons, Christopher Knox, and So Yamaoka Calit2 Center of GRAVITY, University of California - Irvine, Irvine, CA 92697, USA {fkuester, gbs, knoxc, syamaoka}@uci.edu

Abstract. The widespread use of on-line computer games makes this medium a valuable vehicle for information sharing, while scalability facilitates global collaboration between players in the game space. Game engines generally provide an intuitive interface allowing attention to be shifted to the understanding of scientific elements rather than hiding them between a wealth of menus and other counterintuitive user interfaces. These strengths are applied towards promoting the understanding of planetary systems and climate change. Unconventional interaction and visualization techniques are introduced as a method to experience geophysical environments. Players are provided with dynamic visualization assets, which enable them to discover, interrogate and correlate scientific data in the game space. The spirit of exploration is to give players the impetus to conceptualize how complex Earth and planetary systems work, understand their intrinsic beauty and the impact of humans, while providing a sense of responsibility for those systems.

1

Motivation

The study of climate change, introduced by the Earth system science (ESS) community, has become a focus of major international Earth observing and modeling campaigns [1, 2]. While ESS data indicates that climate change is heavily influenced by human activity [3], the analysis of climate change data is challenging. Model data sets are time consuming to generate, extremely large, cover long periods of time, and are difficult to interactively examine. Other challenging aspects include communicating the results to policy makers and the public in order to support decisions for climate change mitigation and adaptation. ESS data is not easily accessible to the general public untrained in climate research. Thus, the data and associated Earth processes remain opaque to most people and they are usually not aware of how everyday activities such as their use of natural resources impacts the climate. Therefore, we seek to find ways to engage the public in the pursuit of geophysical and human impact knowledge. Climate change information is typically disseminated to the public in popular media through newspapers, magazines, radio, television and, with some added artistic freedom, in motion pictures. Winchester described the global climate change resulting from the massive 1883 Krakatoan volcanic eruption and the Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, pp. 529–540, 2006. c Springer-Verlag Berlin Heidelberg 2006


530

F. Kuester et al.

technological milestones (communications) which made Krakatoa the first global media event [4]. Oral histories of the 1883 event were reported by Sumatrans to CNN in 2004 as ancestral stories of the eruption and resulting ocean born disaster. Popular media has also provided information on global warming and international agreements such as the Kyoto Protocols [5, 6]. Major newspapers often link on-line news articles to sophisticated model based visualization such as those used to illustrate the 26 December 2004 tsunami [7] with related lesson plans [8]. Scientific visualization techniques for depicting realistic geothermal events, “El Nino,” and ozone depletion have been in the public domain since the 1980’s when computer graphic techniques were first used to enhance broadcast weather presentations. In 1994, a NBC subsidiary transformed daily weather reports into “environmental spots.” The Global Learning and Observations to Benefit the Environment (GLOBE ) program produced weekly “global environmental observation spots” on CNN in 1995. More recently, broadcast news agencies integrate source material from Google Earth and other on-line GIS ESS portals. Motion pictures depicting extreme geophysical events are illustrated by such films as Krakatoa (1933); Krakatoa East of Java (Metro-Goldwyn-Mayer, 1969); Twister (Universal and Warner Brothers, 1996); A Perfect Storm (Warner Brothers, 2000); The Day After Tomorrow (20th Century Fox, 2004); Category 6: Day of Destruction and Category 7: The End of the World (2004 and 2005, CBS). Although the special effects are often pushed past the limits of nature to unsuspecting audiences the hail depicted in The Day After Tomorrow was half the size of the large seven inch hail actually recored in a midwestern U. S. storm [9]. Recent analysis [10] of college students indicate that students have widely varied understandings of ESS and some students base both scientific and nonscientific concepts on information gleaned from diverse sources including prior education in combination with information from popular media and, at times, religion. Today’s students are proficient with several types of interfaces that in many cases govern their lives: cell phone, TV remote controllers, joysticks and game consoles. The majority of U. S. students are introduced to video, computer and on-line games prior to entering college and most play the games at home [11]. Oblinger calls this generation the “Millennials [12]” and has found that they prefer active learning. Considering that according to Entertainment Software Association (ESA) surveys, over 50% of Americans play computer and video games, 35% of game players are under 18 [13], and game players are increasingly playing games rather than watching TV [14], this media provides an important and interesting path to disseminate information and stimulate critical thinking. In Canada, Denmark, Japan, Korea, Norway, Sweden, and the U. S., over 50% of the households had access to the Internet by 2003 [15]. Surveys indicate that Korea has the highest percentage of inhabitants with broadband access listed as just under 25% in 2003 [16]. Combined with the trend towards broadband connectivity users increasingly play games on-line. For example, the massive multiplayer on-line game (MMOG) World of Warcraft enlisted 2M+

Earth and Planetary System Science Game Engine

531

U. S. participants, and South Korea’s MMOG Lineage enlisted 4M+ subscribers. In addition, the majority of experts in a recent Pew survey estimated that by 2014 90% of all Americans will go on-line from home [17]. The potential of this home computer/internet and computer/gaming base to engage in geophysical play is enormous, especially considering that “sim” or simulation based games are one of the dominant genres in the game industry. The possibility for on-line simulation games to expand into human environmental impact genres and instill a sense of wonder and excitement towards learning for large collaborating constituencies through immersive game play is a significant motivational factor for our research.

2

Approach

One way of providing game engines with the capabillity to visualize natural phenomena and climate change data is to create tools for integrating accurate geophysical data. ESS observational data, models and simulations can be used as inputs to game assets to achieve this goal. We categorize ESS games into three major methods: those based on observational data; those based on geophysical models; and those based on integrated assessment models. Observational data is captured by satellites as well as Earth based instruments. In contrast, model data is generated and can be forward looking as in a weather forecast or backward looking to simulate historical patterns. An integrated assessment model is a more complex geophysical model as it provides more control over other parameters impacting climate such as human activity and economics. In addition to observational data being critical for the validation of the developed scientific models, it can be used to enhance simulation data and generate photo-realistic scene effects such as cloud patterns and wind movement. Models and their associated climate change scenarios are useful in constructing mechanisms for complex goal oriented games. A prototype MMOG with an open ended discovery scenario is presented to illustrate the viability of ESS data mapping into multiplayer game space and new types of visualization tools, called “visualizers,” for collaborative data interrogation are introduced. We seek to compensate for the inability to directly observe the phenomena which Libarkin [10] suggests is a limiting factor to climate change comprehension. Among our overarching goals is for the participants to develop principles of how geophysical models can be “forced” by simply interacting with them; and hopefully, over time see how they can achieve positive environmental results of their collective behavior in the real world. An added benefit is that the ESS integration game tools also provide scientists with a means to rapidly explore large datasets within a setting that provides a fresh vision. The specifics of the current work focus largely on the first application, however, it should be noted that the tools can be used in alternative interfaces to easily serve the second application. Our basic assumptions are that the players are in a transient environment in which there are persistent ESS simulations, that all players can interact with the

532

F. Kuester et al.

same simulation, and that players do not have a deep seeded knowledge about the system’s dynamics or a pre-conceived idea of how the data is traditionally visually represented. The research focus is on how to best represent data for geophysical systems that are normally not “seen” by players, and how to provide for interaction with the data in a format that allows spatial and dynamic effects to be swiftly studied and conceptualized.

3

Related Work

Aspects related to data representation and visualization interaction have been well researched in traditional scientific visualization and virtual/augmented reality venues, however, they have rarely been addressed for computer games which is a medium that has a much greater variance in time and space from the user’s point of view. Thus, there is very little research in the area of integrating advanced scientific visualization techniques or ESS modeling into game engines. The background research is drawn from interdisciplinary areas including scientific visualization, visual art, virtual and augmented reality, ESS, modeling and simulation, education and MMOGs; as well as engineering practice for emerging new computational and communication technologies. Research has been conducted on using models and simulations in ESS learning environments [18, 19, 20]. The majority of the projects are designed for supplemental K-12 curriculum such as EdGCM, Earth System Simulation and the GLOBE program which use ESS models in various ways. EdGCM [21] supports limited parameter boundary manipulation, model runs of the GISS GCM Model II, and interactive analysis of the resulting simulation/visualization; Earth System Simulation [22] focuses on spatial scale by using a series of models including the PSU/NCAR MM5, the NWP, a numerical model for regional atmosphere [23] and a GCM [24]; and GLOBE [25] supports the selection of a number of NOAA NCEP modeling resources that are specifically related to the GLOBE observation suite then presents the simulations as isosurfaced maps. The public participatory project, Climate Prediction Net [26] supports the downloading of APIs to enable processing of HadCM3, HadSM3 and UM model ensembles on globally distributed PC’s. Many exhibits throughout the U. S. now enlist traditional ways to interpret ESS data such as iso-surfaces and puesdo-colored maps. The American Museum of Natural History, for example, presents pre-computed results from models as animation. The Earth mantel model TERRA, the Parallel Ocean Program (POP) model and the Regional Atmospheric Modeling System (RAMS) [27] are included in the Earth Hall exhibition. The POP model uses 20 non-uniformly spaced depth levels and bathymetry. The resulting visualization includes “hill shading” techniques to depict large surface structures. Particle traces were also used to identify surface circulation patterns. Computer ESS games available to the public include an early very popular single player climate change game titled SimEarth (Maxis) based on the GAIA model [28]. The game was well received by the gaming community, teachers and

Earth and Planetary System Science Game Engine

533

the lay public, but, the computational time delay often frustrated the players. The GAIA and geophysical models underlying the game produced various simulations and forecasts including ocean circulation, atmospheric circulation, and mantle tectonic movement. The technological factors that were detrimental and limiting to SimEarth have, for the most part, been resolved since SimEarth’s premier in 1991. Maxis has produced other “sims” including their most recent multiplayer Spore which has implications for disseminating concepts related to ESS including geological time and climate change. Role playing multiplayer interactive games include NCAR’s Disaster Dynamics [29]; and NitroGenius based on the Integrated Nitrogen Impact Assessment Model on a Regional Scale (INITIATOR) representing processes in the nitrogen chain [30]. Educators have proven the effectiveness of using models and simulations in games as they observed methods for soliciting ones ability to understand events using features that are characteristic of games. Winn states that games can “exploit challenge, curiosity and, to some extent, fantasy in order to heighten presence” increasing the players abilities to “observe and reason about what they observe” in order to understand events [31]. Kleiboer has shown that games also provide “occasions for decision, allowing for the enactment of roles and direct experience with the immediacy of simulations” [32]. These methods have direct relevance to games focused on environmental behavior since human induced climate change is of the “complex curricular nature” Winn identifies as being the type of learning problem especially well suited for immersive computer games. Simulation games provide a sense of ownership and control through the use of interactive environmental simulation. Furthermore, these games are fun and challenging, encouraging long term engagement and a significant commitment to learn the simulation model and is properties. Learning theorists studying the use of these mechanisms believe “their purpose is to bring to the fore for exploration, and combine in novel ways some important underlying mechanisms otherwise unnoticed” [33]. Identification of constructs for geophysical systems which visually trigger cognitive processes is one aspect of our research. Kepes [34], Marr [35] and Zeki [36] provide a continuum of visual theory relevant to the perceptual features of primal vision. Experimentation in perceptual techniques for ESS visualization [37] illustrates these features as used by biological vision systems to capture visual attention and identify what is being seen. These cognitive functions have been observed to be most active during the preattentive state [38] and are well suited for communicating the properties of geophysical systems in transient spaces where reducing the time it takes to appreciate a construct is critical. Perceptual techniques are useful in addressing questions raised by educational theorists who have focused on the problems associated with “verbal overshadowing” demonstrating the need to allow for deeper use of visual skills during learning processes. Perceptual attributes can be seen in the kinetic visualization described in Lum et al. [39, 40]. Kinetics have been shown to exploit motion while effectively funneling representations through to visual understanding [41]. Particle systems are a type of kinetic visualization and have been developed by

534

F. Kuester et al.

Bruckschen et al. [42] and Kuester et al. [43] for interactive immersive environments; and by Sabo [44], and Kruger and Westermann [45] for game engines. Techniques for modeling geophysical phenomena have until recently treated the world as a solid object [46]. Global time stamp based scalable frameworks, an underlying mechanism relevant to ESS data and game engine integration, have been researched by Kim et al. [47]. Research on methods to render natural phenomena using game engines has been described by Fritsch and Kaka [48]; Perbet and Cani [49], Chenney [50] and Shi et al. [51] for wind effects; and Harris [52] and Umenhoffer and Szirmay [53] for imposter clouds.

4

The EPSS-GE

We have developed a prototype for an Earth and planetary system science game engine (EPSS-GE ) [54] to provide a means for very large groups of individuals to collectively interact with accurate geophysical simulations informed by the best data available in addition to streaming inputs from modeling resources. The EPSS-GE is designed to support a range of applications built for client side game engines (CGE). The demonstration EPSS-GE CGE is an open ended discovery game using Intergovernmental Panel on Climate Change (IPCC) models as the simulation engine with interaction provided through a simple interface. The EPSS-GE CGE supports interaction with ESS data in a non-photorealistic abstract form by employing data reduction and aesthetically biased symbolic representations designed for a fast paced transient environment (immersive game scene) and subtle player interaction mechanisms. The goal of our closely monitored reciprocal feedback is to give the players a virtual representation of the dynamics of the geophysical system they are exploring. The use of geophysical data and models in games for scientific visualization has until now been limited by game architecture, the lack of ESS data ingest handlers, and the lack of data visualization techniques appropriate for MMOGs. Thus, the focus of the EPSS-GE work is the reduction, transformation and translation of scientific visualization into CGE technology. The project combines aspects of grid computing, distributed simulation and visualization with scientifically anchored ESS data in order to study real-time computer graphics and multi-modal interaction, in the context of multiplayer gaming frameworks. This also requires a new paradigm for network interaction that fuses CGEs (specifically MMOGs) with complex scientific simulations while dealing with the heterogeneous resources. The latter calls for adaptive technology to automatically scale (reduce in size and complexity) the presented contents to satisfy local processing, networking and interface limitations. At the core of this work are the processing of ESS data by highly responsive systems and the translation of data analysis techniques into new game methodologies. Real-time game engine scene handling techniques by default support perceptually based image recognition. For example, the low resolution of background elements (terrain detail) de-emphasize them so that processing can be devoted to elements that require a higher level-of-detail which in our example are the sci-

Earth and Planetary System Science Game Engine

535

entific visualization constructs. We explored perceptual techniques for scientific visualization and selected kinetic methods as they are especially well suited for the analysis of geophysical systems. Kinetics are supported as particle systems within the EPSS-GE prototype CGE application. The “player” encounters phenomena first by motion, then motion and form, and then color. The normally invisible atmospheric substances are dynamically etched out from space through player interaction. 4.1

Data

The most recent IPCC data for the Fourth Assessment Report (AR4) consists of an ensemble of model results of 100 year time series simulations spanning the years 2000-2100 for a range of scenarios. Data set variables include surface precipitation, surface temperature, air temperature, wind direction, net downward short wave flux in air, net upward longwave flux in air, and cloud area fraction. The variables are calculated on a longitude, latitude, and altitude grid. The models use different surface grid resolutions ranging from 72 x 46 to 256 x 128 with a range of different grid types. Our prototype EPSS-GE uses the IPCC AR4 INM-CM3.0 model from the Institute for Numerical Mathematics in Moscow [55]. INM-CM3.0 has a 5 x 4 degree resolution in longitude and latitude and 21 vertical levels. The second model used in the prototype is the National Center for Atmospheric Research’s (NCAR) Community Climate System Model (CCSM) [56]. The CCSM is the highest resolution model within the IPCC AR4 ensemble and produces data sets that are approximately an order of magnitude larger than those produces by the INM-CM3.0 model. Data Reader. In order to ingest the IPCC model data, we implemented an EPSS-GE Networked Common Data Format (NetCDF) reader. NetCDF’s wide spread adoption by the ESS and space sciences ensures our ability to inquest a variety of data and to compare the results of the ESS models. The EPSSGE NetCDF asset loads the NetCDF data and makes the data available to other game assets (implemented as specific visualization tools) for interactive 3D space exploration. The NetCDF asset is a data-only game asset invisible to players and was built using a publicly available NetCDF library [57]. 4.2

Visualization Assets

A set of game assets for visualization of ESS data, called “visualizers,” is supported within the CGE application enabling the players to interactively explore the environment. Conventional visualization strategies for scientific data, are frequently focused on the extraction of iso-contours or surfaces, identification of gradients, interfaces and characterization of vector fields. To provide more flexibility and a means for the intuitive exploration of complex relationships by experts as well as novice users, a set of particle-based approaches for data interrogation is introduced. Global surface temperature data was selected as one of the core parameters and served as the basis for the development of the three visualizers which can be manipulated in multiplayer mode.

536

F. Kuester et al.

RPE - Responsive Particle Emitter. A RPE is a basic particle visualizer, which draws from a combination of color mapping, velocity mapping, and direct time mapping techniques. The RPE responds to the data in two ways. 1) Velocity is mapped such that particles go up in the hotter areas and go down in the colder areas. 2) The hotter the surface temperature is the more reddish, the colder the more bluish. These effects combine such that the RPE emulates a fire or smoke in a hot area and chilled air in a cold area. The velocity mapping is influenced by the behavior of the gaseous matter in the real world to help a player understand the surface temperature data. This visualizer also responds to the time advancement of the data. DCPE - Directional Constant-Velocity Particle Emitter. A DCPE is a particle stream visualizer traveling upward at a constant velocity and utilizing time-height mapping. The time axis is set to the vertical axis of the 3D game space extending from the first time step to the end of the series. The player can observe the particles assume the color associated with datum as they pass through levels. Each level corresponds to a single (monthly) time step. The data set contains 1,200 time steps corresponding to one hundred years. The vertical time axis for the DCPE was bound to three years, limiting the time scale, in order to present the data at a resolution that was more accommodating of exploration for the temperature data set. The length of displayable years can be extended, extending the life of particles also, which defines how long the particles stay within the game space. DFPE - Directional Focus Particle Emitter. A DFPE visualizer emits particles along the player’s viewing direction. The particles fill a small deltaic region in front of the player. The DFPE is particularly useful in illuminating areas of the data space in collaboration with other players. The time mapping of this asset can be either time advancement mapping or time-height mapping. 4.3

Game Play

Single Player Mode. A player is randomly placed into the geospatially referenced 3D game space containing surface temperature data and a global reference base map as the “game board.” The space also contains the interactive visualizers. The player can freely walk around the game space, and can manipulate the position and function of the visualizer assets. The player can pick a visualizer up and observe its color change as the player walks through the data space. The player can throw the visualizer around data space, for instance, to observe it as it moves about from any distance. The visualizer assets are automatically regenerated at certain locations, providing an infinite supply of assets. Since the visualizers will not disappear from the virtual space while the game is running, the player can propagate visualizers throughout the game space as long as the computational resources are sufficient. Multiplayer Mode. Multiplayer mode, a collaborative exploration of the data space, is one of the most important aspects of our research in the use of MMOGs

Earth and Planetary System Science Game Engine

537

to develop climate change concepts. The Torque Game Engine (TGE) was selected due to its open architecture and networking capability. The number of players is basically bounded by the bandwidth of the network and computational power of the computer. In the multiplayer mode, the player sees other players exploring the same data space and can interact with them. Players can exchange visualizers, chat via a chatting window, and collaboratively explore the data space.

5

Conclusions

EPSS-GE was evaluated on HIPerWall, a massively tiled display utilizing Calit2’s LambdaRail optical network, the OptIPuter, designed to enable scientists generating terabytes and petabytes of data to interactively analyze data from multiple mass storage sites. Adapting traditional game mechanisms to support collaborative geophysical experiences, in addition to incorporating unconventional visualization techniques, provides a vehicle for students to transition from game play to geophysical analysis. The game play can be very engaging as observed during highly focused sessions and meta-game activities using dialogue to develop strategies. Observation of student approaches towards data exploration, indicates that the kinetic environment and the entities within it were extremely important for perception. This technique relayed the inherent dynamic quality and beauty of the geophysical system under study while also exploiting the transient visual characteristics of the fast paced game space and technology of game engines. In this project, motion articulated through kinetics is an important vehicle leading to understanding, and thus learning. In the future, these techniques will be combined with other visualization paradigms for geophysical data that support interactive visual analysis and reasoning.

Acknowledgments This research was supported, in part, by the University of California Irvine, Council on Research Computing and Library Resources (CoRCLR) under award number MI-2005-2006-26, the California Institute for Telecommunications and Information Technology (Calit2) Graduate Student Fellowship program, and by an HP Technology for Teaching Grant under award number 15923. We thank Professor Charles S. Zender from the Department of Earth System Science at UC Irvine for his support and ESS data, as well as Calit2 SURF-IT Fellow Michael Brown for generating the unified ESS data sets. The above support is greatly appreciated.

References 1. Molina, M.J., Rowland, F.S.: Stratospheric sink for chlorofluoromethanes: Chlorine atom catalysed destruction of ozone. Nature 249 (1974) 810–812 2. Houghton, J.: Global Warming: The Complete Briefing. 3rd edn. Cambridge Univeristy Press (2004)

538

F. Kuester et al.

3. Rowland, F.S.: Atmospheric changes caused by human activities: From science to regulation. Ecology Law Quarterly 27 (2001) 1261–1293 4. Winchester, S.: Krakatoa: The Day the World Exploded, August 27, 1883. New York: Harper Collins (2003) 5. Allen, W.L.: Global warming, bulletins from a warmer world. National Geographic 206 (2004) 6. Kolbert, E.: The climate of man. The New Yorker (2005) Part I Apr 25 81.10:56-71, Part II May 2 81.11:64-73, Part III May 9 81.12:52-63. 7. Titov, V.V., Murty, T.: (Asia’s Deadly Waves Simulation) New York Times Interactive Feature http://www.nytimes.com/packages/khtml/2004/12/26/ international/20041227 QUAKE FEATURE.html. 8. Bank Street College of Education: (Asia’s Deadly Waves Lesson Plans) New York Times on the Web Learning Network http://www.nytimes.com/learning/issues_in_depth/20050104.html. 9. Zender, C.S.: The Morning After The Day After Tomorrow. The Orange County Weekly (2004) 10. Libarkin, J.C., Anderson, S.W., Science, J.D., Beilfuss, M., Boone, W.: Qualitative analysis of college students’ ideas about the Earth: Interviews and open-ended questionnaires. Journal of Geoscience Education 53 (2005) 17–26 11. Jones, S.: Let the Games Begin, Gaming Technology and Entertainment Among College Students. Pew Internet and American Life Project, Pew Research Center (2003) 12. Oblinger, D.G.: The next generation of educational engagement. Journal of Interactive Media in Education, Special Issue on the Educational Semantic Web 8 (2004) 1–18 13. ESA: Entertainment software association facts and research, game player data (2005) http://www.theesa.com/facts/gamer_data.php. 14. Lowenstein, D.: State of the industry address (2005) Speech Delivered at the 11th Electronic Entertainment Expo, http://www.theesa.com/archives/ 2005/05/e3 2005 state o 1.php, downloaded Sep 2005. 15. : Information and Communications Technologies, OECD Information Technology Outlook. Organization for Economic Co-Operation and Development (OECD) (2004) 16. : Information and Communications Technologies, OECD Communications Outlook. Organization for Economic Co-Operation and Development (OECD) (2005) 17. Fox, S., Anderson, J.Q., Rainie, L.: The Future of the Internet. Pew Internet and American Life Project, Pew Research Center (2005) 18. Gobert, J.: Harnessing technology to support on-line model building and peer collaboration. In: Proceedings of the Teaching Geoscience With Visualization: Using Images, Animations, and Models Effectively Conference, On the cutting edge (2004) 10–30 19. Jackson, D.F.: Case studies of microcomputer and interactive video simulations in middle school Earth science teaching. Journal of Science Education and Technology 6 (1997) 127–141 20. Jackson, S.L., Hu, J.T., Soloway, E.: Scienceworks modeler: Scaffolding the doing of science. In: Conference Companion CHI ’94, ACM (1994) 249–250 21. Chandler, M.A., Shopsin, M., Richards, S., Sohl, L.E.: The Basic Guide to EdGCM, Draft v.2.3.4. New York: Columbia University (2005) 22. Thompson, O.E., Johnson, D., Kalnay, E., Zhang, D., Cai, M., Suarez, M., Yanuk, D., Schaack, T.: Computationally intensive models in the classroom. Journal of Earth System Science Education 1 (2001) 1–13

Earth and Planetary System Science Game Engine

539

23. Arkin, P.A., Thompson, O.E., Bonner, W.D.: Diurnal variations of the summertime wind and force field at three midwestern locations. Monthly Weather Review 104 (1976) 1012–1022 24. Held, I.M., Suarez, M.J.: A two level primitive equation model designed for climatic sensitivity experiments. Journal of Atmospheric Sciences 35 (1978) 206–229 25. GLOBE: GLOBE visualization data directory (2005) downloaded 16 March 2005 http://viz.globe.gov. 26. Stainforth, D., Kettleborough, J., Martin, A., Simpson, A., Gills, R., Akkas, A., Gualt, R., Collins, M., Gavagham, D., Allen, M.: ClimatePrediction.Net: Design principles for public-resource modeling research. In: Proceedings of the 14th IASTED International Conference, Parallel and Distributed Computing and Systems, IASTED (2002) 32–38 27. McPherson, A., Painter, J., McCormick, P., Ahrens, J., Ragsdale, C.: Visualizations of Earth processes for the American Museum of Natural History. Computer Graphics (1999) 11–15 28. Wilson, J.L.: The SimEarth bible. Berkeley : Osborne McGraw-Hill (1991) 29. McGinnis, S.: (Disaster Dynamics: Serious Games for Disaster Education) National Center for Atmospheric Research (NCAR) http://swiki.ucar.edu/dd/. 30. Erisman, J.W., Hensen, A., de Vries, W., Kros, H., van de Wal, T., de Winter, W., Wien, J.E., van Elswijk, M., Maat, M.: The Nitrogen Decision Support System: NitroGenius, ECN-C-02-012. Petten: Energy Research Center of the Netherlands (ECN) (2002) 31. Winn, W., Windschitl, M., Fruland, R., Lee, Y.: When does immersion in a virtual environment help students construct understanding. In: Proceedings of the International Conference of the Learning Sciences (ICLS ‘02), International Society for the Learning Sciences (ISLS) (2002) 497–503 32. Kleiboer, M.: Simulation methodology for crisis management support. Journal of Contingencies and Crisis Management 5 (1997) 198–206 33. Ackermann, E.: 2 Perspective-taking and object construction: Two keys to learning. In: (Y. Kafai and M. Resnieck Eds.) Constructionism in Practice: Designing, thinking, and learning in a digital world. Mahwah, N.J.: Lawrence Erlbaum, Associates, Inc. (1996) 25–37 34. Kepes, G.: The Language of Vision. Chicago: Paul Theobald (1944) 35. Marr, D.: Vision: a computational investigation into the human representation and processing of visual information. San Francisco: W. H. Freeman (1982) 36. Zeki, S.: Inner Vision: An Exploration of Art and the Brain. Oxford University Press (2000) 37. Healy, C.G., Evans, J.T.: Perception and painting: A search for effective, engaging visualizations. IEEE Computer Graphics and Applications 22 (2002) 10–15 38. Healy, C.G., Booth, K.S., Enns, J.T.: High-speed visual estimation using preattentive processing. ACM Transactions on Computer-Human Interaction 3 (1996) 107–125 39. Lum, E.B., Stompel, A., Ma, K.L.: Kinetic visualization: A technique for illustrating 3D shape and structure. Proceedings of IEEE Visualization ’02 (2002) 435–442 40. Lum, E.B., Stompel, A., Ma, K.L.: Using motion to illustrate static 3D shape kinetic visualization. IEEE Transactions on Visualization and Computer Graphics 9 (2003) 115–126 41. Zeki, S., Lamb, M.: The neurology of kinetic art. Brain 117 (1994) 607–636

540

F. Kuester et al.

42. Bruckschen, R., Kuester, F., Hamann, B., Joy, K.I.: Real-time out-of-core visualization of particle traces. In: Proceedings of the Parallel and Large Scale Data Visualization and Graphics Symposium, IEEE (2001) 45–50 43. Kuester, F., Bruckschen, R., Hamann, B., Joy, K.I.: Visualization of particle traces in virtual environments. In: Proceedings of the Virtual Reality Software and Technology Conference (VRST ’01), ACM SIGCHI and SIGGRAPH (2001) 151–157 44. Sabo, M.: Improving advanced particle system by adding property milestones to particle life cycle. In: CESCG. (2004) 45. Kruger, J., Westermann, R.: GPU simulation and rendering of volumetric effects for computer games and virtual environments. In: Proceedings of EUROGRAPHICS. (2005) 46. Johnston, D.: 3D game engines as a new reality. In: Proceedings of the 4th Annual CM316 Conference on Multimedia Systems, Southampton University, UK (2004) dj301.pdf downloaded Nov 2005 http://mms.ecs.soton.ac.uk/mms2004. 47. Kim, S.J., Kuester, F., Kim, K.H.K.: A global timestamp-based scalable framework for multi-player online games. In: Proceedings of the Fourth International Symposium on Multimedia Software Engineering (MSE ’02), IEEE (2002) 2–10 48. Fritsch, D., Kada, M.: Visualization using game engines. In: Proceedings of the XXth Congress, Commission 5, 35.B5, IAPRS (2004) 627–631 49. Perbet, F., Cani, M.P.: Animating prairies in real-time. In: Proceedings of the Symposium on Interactive 3D Graphics (I3D ‘01), ACM (2001) 103–110 50. Chenney, S.: Flow tiles. In: Proceedings of the Symposium on Computer Animation (SCA ’04), ACM SIGGRAPH/EUROGRAPHICS (2004) 233–242 51. Shi, L., Yu, Y., Wojtan, C., Chenney, S.: Contollable motion synthesis in a gaseous medium. The Visual Computer 21 (2005) 474–487 52. Harris, M.J.: Real-time cloud rendering for games. In: Programming Track, Proceedings of the Game Developers Conference, GDC (2002) 1–5 53. Umenhoffer, T., Szirmay-Kalos, L.: Real-time rendering of cloudy natural phenomena with hierarchical depth imposters. In: Proceedings of EUROGRAPHICS. (2005) 54. UCI Calit2 Center of GRAVITY: Earth and planetary system science web site (2006) http://cg.calit2.uci.edu/projects/epssge. 55. Dianski, N.A., Volodin, E.M.: Simulation of present-day climate with a coupled atmosphere-ocean general circulation model. Izvestiya. Atmospheric and Oceanic Physics (English Translation) 38 (2002) 732–747 56. Collins, W.D., Bitz, C.M., Blackmon, M.L., Bonan, G.B., Bretherton, C.S., Carton, J.A., Chang, P., Doney, S.C., Hack, J.J., Henderson, T.B., Kiehl, J.T., Large, W.G., McKenna, D.S., Santer, B.D., Smith, R.D.: The community climate system model: CCSM3. Journal of Climate Special Issue on CCSM (2005) 57. Unidata: Network Common Data Form (NetCDF) (2005) NetCDF-3.6.0-p1 library http://www.unidata.ucar.edu/software/netcdf/.

Learning Online: A Comparative Study of a Situated Game-Based Approach and a Traditional Web-Based Approach M.S.Y. Jong, J.J. Shang, F.L. Lee, J.H.M. Lee, and H.Y. Law Centre for the Advancement of Information Technology in Education, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong

Abstract. “Online Learning” has been commonly viewed as a mechanism for empowering improved learning outcome, increased flexibility of aligning the individual need of learners, and better quality of educational interaction. In fact, a lot of “digitized” and “ready-to-use” learning and teaching resources are already available online; nevertheless, we must not confuse quantity and quality, as these resources may just continue to perpetuate teacher-centred approaches, rather than student-centred approaches. The present research aimed to compare the educational values, learning effectiveness, students and teachers’ perceptions of a new online educational paradigm - Situated Game-based Learning with Traditional Web-based Learning in secondary education in Hong Kong. A combination of quantitative and qualitative research methods were employed for data collection and analysis. Results showed that, under the present research settings, although no significant difference of students’ learning outcome with respect to these two approaches was found, the participating students and teachers were quite positive towards the educational paradigm of Situated Game-based Learning. This provides vital insights and a basis for further investigating the paradigm’s application and development for learning and teaching.

1

Introduction

Since the wide spread of the Internet and World Wide Web (WWW) in early 90’s, learning online has become possible and prevalent. Plenty of learning and teaching materials have been digitized and made available on the Internet. It is not hard to imagine that if you make a simple Internet search on an educational topic, hundred or even more links of relevant “educational” resources will be automatically displayed on your screen. “Anytime,” “anywhere” and “any pace” are all familiar attitudes inherited to these “ready-to-use” online resources. The question, however, is how many of these resources can really deliver the vision of a learner-centred approach. In fact, Online Learning has been viewed as an effective means for empowering improved learning outcome, increased flexibility of aligning the individual need of learners, and better quality of educational interaction. Nevertheless, we Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, pp. 541–551, 2006. c Springer-Verlag Berlin Heidelberg 2006


542

M.S.Y. Jong et al.

have to be cautious in interpreting the full availability of WWW technology and web-based educational resources as a corresponding increase in ability to enable a paradigm shift to take place [3], that is, from “a largely textbook-based teachercentred approach to a more interactive student-centred approach,” which has been strongly advocated by many education policy makers (e.g., [1]). 1.1

Traditional Web-Based Learning (TWBL)

In the early stage of the WWW history, “hypertext” was widely recognized as a powerful medium for disseminating information and enabling non-sequential reading [2]. In order to prevent online readers getting trapped in the hypertext environment, a clear and effective navigation menu, headings and sub-headings have been strongly recommended for putting in every learning website. Chunking pieces of text into smaller “self-contained” paragraphs with bullets has also been another vital technique for putting educational content online. On top of that, according to the well-known jargon of “a single picture is worth a thousand words” [11], adding pictorial, schematic, symbolic or figural graphics to enrich the content always sounds a crucial act. In fact, a lot of web-based educational resources have been designed in the fashion that goes with these “golden” features (e.g., [7]). Normally, learning with this sort of resources is conducted by navigating between pages of content in a web browser or a learning management system (LMS) (e.g., WebCT or BlackBoard). 1.2

Situated Game-Based Learning (SGBL)

One of the frequently received negative feedback of web-based learning from students is “hard to stay motivated” [14]. Moreover, the traditional fashion of chunking or fragmenting learning materials may eventually end up with creating unrealistic learning context and depriving the rationale behind the knowledge itself. In fact, some educators (e.g., [10]) have also highlighted that most of the existing web-based learning systems are just used as a repository of digitized educational materials, without taking the full advantages of the immense power of WWW and the frontier of web-based learning should be further extended. As suggested by the name, Situated Game-based Learning is a union of computer game-based learning [16] and situated learning [6]. It is a “marriage” of educational materials and computer games, in which both content and context are smartly designed to put learners into an environment that is similar or analogous to where the knowledge of the educational content can be applied in the future. Without chunking or turning the content into a series of “split-screens,” the learning process takes place unintentionally rather than deliberately. In 2001, with situated learning as a framework, Lee et al. [9, 8] developed a web-based educational game, namely “Tong Pak Fu and Chou Heung: the Probabilistic Fantasy” [8] (hereafter referred as TPFCH-PF) for learning concepts from simple to conditional probability. The skeleton of TPFCH-PF is based on a well-known Chinese folklore “Tong Pak Fu and Chou Heung.” Tong Pak Fu was a legendary poet, scholar, painter as

Learning Online: A Comparative Study of a Situated Game-Based Approach

543

well as womanizer in the Ming Dynasty, and his tales have been inspiring numerous Chinese folklores. A very well-known one is his love story with Chou Heung. The story started with one day Tong in a busy market square saw Chou who was a maid working in the Wah Mansion. Although Tong had already got eight wives, he still could not resist the “electrified pulse” induced by Chou. He was deeply attracted. After many unsuccessful flirting and dating attempts, Tong finally won the heart of Chou and they fell in love. Tong wanted Chou to be his 9th wife; however, the master of Chou, Madam Wah required Tong to take and pass an ultimate examination, otherwise she would not allow Chou to marry him. In the final challenge, Tong had to be able to pick Chou from a group of face-masked brides within a limited number of guessing attempts. In fact, this folklore is very famous in the Chinese community and has appeared in popular books and movies. The background and storyboards of TPFCH-PF are composed of five different courtship stages of Tong towards Chou. In the game, a learner is situated in these five courtship stages and plays the role of Tong to court and date his beloved lady Chou. In each stage there is one “large” problem which is composed of several sub-problems, the learner needs to solve the problem independently by understanding the concept and knowledge underpinned to the solution to that problem. Nevertheless, the “Wise Genie” who is one of the characters in the game, will give help and advice to the learner whenever necessary in each stage. Moreover, some handy explanation of the new probability terms emerged in the game are also anchored.

2

Research Aims

The present research aimed to compare students’ perceptions and learning outcome with respect to the two online learning approaches mentioned in Section 1, i.e., Situated Game-based Learning (hereafter referred as SGBL) and Traditional Web-based Learning (hereafter referred as TWBL). On the other hand, according to some empirical studies (e.g., [5, 15]), teachers’ opinions are always significant in influencing the success of an educational innovation because they are the ultimate designers of learning and teaching activities in the educational process. Therefore, in order to draw a clearer picture for this comparative investigation, the teachers’ perceptions of these two approaches should also be studied. The following three research questions directed the whole study: – What are the students’ perceptions of Situated Game-based Learning (SGBL) and Traditional Web-based Learning (TWBL)? – Which learning approach can empower better learning effectiveness? – What are the teachers’ perceptions of these two learning approaches?

3

Research Design

The present comparative study used a combination of quantitative and qualitative research methods, and was conducted in July 2005 in Hong Kong. The analysis was completed in September 2005.

544

3.1

M.S.Y. Jong et al.

Participants

A convenient sample of 158 Secondary 4 (comparably equivalent to K-10) students and 4 teachers from two Hong Kong secondary schools were invited to participate in the study. In this paper, these two schools are indicated as School A and School B. The student participants consisted of 71 males and 87 females, with the mean age of 16.0 (roundup to 1 decimal place) and with heterogeneous academic achievement in the schools. All of them had already learnt some elementary concepts of probability in the previous academic year. The teacher participants were the Math teachers of the student participants from the respective schools; 2 from School A and 2 from School B. 3.2

Learning Materials

TPFCH-PF [8] was adopted as the learning material for the experimental group students to conduct SGBL. On the other hand, another online learning resource in the fashion of TWBL [7] but covering exactly the same amount of learning content as TPFCH-PF was explicitly created for the control group students to learn in this study. 3.3

Procedure

At the beginning of the study, the student participants in each school were randomly assigned into either experimental group or control group, with an approximately equal proportion. The whole research procedure was divided into 3 phases as shown in Figure 1, and was executed in each school individually. Phase 1: Student Pre-test. A paper-based pre-test was designed to assess students’ prior knowledge so as to later compare their subsequent knowledge after the learning experiment with a post-test. The pre-test consisted of 9 relatively straightforward probability calculation problems in multiple-choice format, with

Fig. 1. Research Design

Learning Online: A Comparative Study of a Situated Game-Based Approach

545

1 mark per problem. Both experimental and control group students had been asked to take the same pre-test in approximately 1 week before the learning experiment took place. Phase 2: Learning Experiment. The experiment was held as a 3-hour parallel learning session. The experimental group students were arranged to conduct SGBL with TPFCH-PF [8] with their own workstations in a computer lab, while the control group students were arranged to conduct TWBL with [7] in another computer lab. In each side, there was a teacher participant to facilitate the group to learn and answer their questions whenever necessary. During the experiment, these 2 teachers were required to swap their group of facilitation at least 1 to 2 times. This arrangement aimed to enable them to experience and observe how their students learnt with these 2 approaches, and to foster more fruitful discussion in the interviews conducted in Phase 3. This parallel session took 3 hours including a 15-minute break in the middle of the session. Phase 3: Student Perception Survey, Student Post-test and Teacher Interview. For the student participants, on the same day after the learning experiment, both the experimental and control groups were required to fill in a same paper-based questionnaire for gathering their demographic data and their perceptions of the learning approach that they had experienced respectively. After that, all of them were required to take a same paper-based post-test which consisted of 16 probability calculation problems in multiple-choice format, with 1 mark per problem. Unlike the pre-test, the problems appeared in the post-test required the application of the concepts and knowledge covered in the learning experiment. For the teacher participants, within one week after the experiment, they were all individually interviewed in semi-structured style, which aimed to study their perceptions of these 2 learning approaches in terms of educational values and learning effectiveness.

4

Research Findings

We report the findings of the study with respect to the student and teacher domains. Interestingly, most of the findings from both sides align in general. 4.1

Student Findings

Students’ Perceptions of the Approaches. According to the result of the perception survey, it was found that the SGBL approach was significantly more favored than the TWBL in the students’ perspective. Table 1 shows a set of 5-point scale survey questions (from 5:Strongly Agree, 4:Agree, 3:Neutral, 2:Disagree, to 1:Strongly Disagree) as well as the mean score and p-value with respect to each question answered by the experimental and control group students. Another yes/no question was also asked in the survey to study whether they were keen on the learning material that they had worked on during the experiment. The result showed that the majority of the experimental group students

546

M.S.Y. Jong et al. Table 1. Student Perception Survey Result

Fig. 2. Student’s Perception of the Learning Materials

(81%) liked the educational game of TPFCH-PF, but less than half of the control group students (47.3%) liked the Traditional Web-based Learning material (Figure 2). In addition, an open-ended question was included at the end of the survey which aimed to gather more students’ opinions on these two learning approaches. In the experimental group, most of their ideas focused on suggesting how to improve or enhance the existing game-play interface of TPFCH-PF so as to make the learning more interesting. On the other hand, in the control group, most of their written opinions were negative towards TWBL, like “very boring,” “too many reading,” “incapable of arousing learning interest” and “no interaction,” etc. In fact, this showed an alignment of perspectives between 2 groups. They considered “fun” and “interactivity” are very vital attributes in a learning process. Learning Effectiveness of the Approaches. This analysis was based on the result of students’ pre-test and post-test. The full mark of the pre-test and posttest were 9 and 16 respectively. Table 2 shows the descriptive statistics of the result of the tests.

Learning Online: A Comparative Study of a Situated Game-Based Approach

547

Table 2. The Pre-test and Post-test Results

Table 3. The significance test result of One-Way Analysis of Covariance

According to the significance test result of one-way analysis of covariance (Table 3), no significant difference of the post-test result with respect to these 2 learning approaches was found (p-value = 0.567). There was only a significant relationship between the result of post-test and pre-test; the students, who had done better in the pre-test, also did better in the post-test. Under the present research settings, it was found that although the student participants were keener on the SGBL approach, no evidence supported that it was better than the TWBL approach in terms of learning effectiveness. 4.2

Teacher Findings

As mentioned in Section 2, another vital aim of the present research was to study the teachers’ perceptions of these two learning approaches in accordance with their educational practice as well as their observation and facilitation experience in the experiment. All these findings were drawn by the qualitative analysis of the individual teacher interviews. First of all, the 4 participating teachers were quite positive towards the SGBL approach in terms of motivating students to learn, and believed that it was much more capable than the TWBL approach in arousing students’ learning interest: “Almost all experimental group students were quite engaged and devoted when they were learning with TPFCH-PF. They tried to solve the problems in the game even they were unsure the answers to those problems . . . However, I could not see such phenomenon emerged in the control group.”

548

M.S.Y. Jong et al.

“The most valuable educational value of TPFCH-PF is that students can gain a lot of motivation of learning . . . obviously, the traditional Webbased material is unable to make such thing . . . it is lack of interaction and the students can only learn passively.” “. . . very great! The students were actively exploring the knowledge in the game. It was quite student-centred.” One of the participating teachers from School A also highlighted that: “I saw there were some students, after they had finished the learning game, they tried to use the Internet search engine to search some probability terms, like ‘conditional probability’ . . . I was surprised, they were so ‘proactive’. It is so magic . . . this game is a very good learning motivator.” Moreover, another participating teacher pointed out that the SGBL approach could make learning more authentic: “. . . in fact, it was a very good chance for them to appreciate that math knowledge is not just only for examination, but also for solving some real-life problems.” Nevertheless, some of them argued that the TWBL approach is more flexible and convenient when reviewing or re-studying of the learning materials is needed: “The TWBL material provided more flexible navigation . . . the students could navigate the content very easily, especially when they wanted to review or reinforce the concepts that they just learnt.” “. . . the flow of game is based on situations or scenarios, it is quite inconvenient when a student wants to make an ad-hoc review on specific knowledge points . . .” “The good stuff offered in TWBL is non-linear access of the learning material.” Although they very much believed that the SGBL approach could arouse their students’ learning interest, they showed their uncertainty on whether it could have enabled them to have better learning outcome: “The students seemed very engaged in the game, but I am really not sure how much they could learn from it.” “According to my observation in the experiment, it seemed that there were a portion of students just using a ‘trial and error’ strategy to ‘solve’ the probability problems in the game stage by stage, without gaining the real understanding of knowledge underpinned to the solution to the problems.” On the other hand, they stressed that the SGBL approach should not only involve the students themselves, but also the teacher. Interestingly, all of them suggested a very similar “blended” strategy for enhancing SGBL. The suggested “blended” strategy is composed of 2 stages; the first stage is to let students

Learning Online: A Comparative Study of a Situated Game-Based Approach

549

learn with the game by themselves. It is like a scaffolding process that the students learn and acquire knowledge interestingly in a learner-centred fashion. The second stage is a reinforcement process in which the teacher should correct, strengthen and further extend the knowledge that the students have learnt in the game.

5

Conclusion and Discussion

In the present study, most of the findings from both student and teacher domains aligned. The teachers perceived that Situated Game-based Learning was much better than Traditional Web-based Learning in terms of arousing students’ interest and motivating them to learn. In fact, this was very coherent with the students’ actual thoughts. According to the result of the student perception survey, in comparing with TWBL, SGBL was found as more favored, more interesting, more explanatory, more stimulating, more challenging, and more capable to empower students to learn with confidence and retain the learnt knowledge in their mind. Nevertheless, under the present research settings, no evidence supported that SGBL was more capable to empower the students to have better understanding of the learning materials. It was again very coherent with the teachers’ thoughts; in fact, the participating teachers also showed in the interviews their uncertainty on the learning effectiveness of this approach. They also argued that the SGBL material may not be flexible enough when reviewing or re-studying of the learning content is needed. Yet, they suggested a 2-stage “blended” strategy to enhance the existing SGBL approach. The first stage is to use game-play to scaffold the students to acquire some basic knowledge of the topic in a more interesting way; the second stage is for the teacher to correct, strengthen and further extend the knowledge that the students have learnt in the game. Although there was no significant evidence to prove that SGBL could empower better learning outcome, both student and teacher sides were quite positive towards its educational paradigm. It sounds quite optimistic that this approach can alleviate the frequently received negative feedback of web-based learning of “hard to stay motivated,” and can especially act as an effective stimulating agent for less motivated students. In fact, this aligns with Malone and Lepper’s “intrinsic motivation” [12] theory which strongly advocates that “fantasy,” “challenge,” and “curiosity” are all crucial elements in a learning process. Moreover, the positive perspectives and initiatives from the teacher side are also encouraging. Teachers do things for good reasons only [13]. The teachers’ attitudes towards SGBL vitally determine whether they will take actions to adopt it into their teaching practice, and further explore the possibility of its application in education [4]. Under the present research settings, the experiment was taken as an approximately 3-hour learning session. Within this short period of time, it may be hard to fully examine the learning effectiveness of Situated Game-based Learning. In fact, Lave and Wenger [6] suggested that situated learning is a relatively longterm approach which needs time to immerse learners in a learning environment

550

M.S.Y. Jong et al.

so that they can benefit from the behaviors and activities associated with the context. In order to have a closer look on this approach, further development of SGBL materials with a wider range of intra or inter disciplinary topics for carrying out another relatively longer-time learning study and experimental investigation is crucial. On the other hand, it should also be highlighted that all teacher participants coincidentally suggested a very similar “blended” strategy for improving SGBL, which provides a basis for another more meaningful investigation on further enhancement of this approach.

Acknowledgements This study is supported by the Courseware Development Grant for Web-based Teaching Beyond the Campus, the Chinese University of Hong Kong and the Development Grant, Research Grant Council, Hong Kong.

References 1. Hong Kong Education and Manpower Bureau. IT for learning in a new era: 5-year strategy 98 to 03. EMB, Hong Kong, 1998. 2. R. Horn. Mapping Hypertext. Lexington Institute, 1989. 3. M. Keppell. Response to: Information Technology in Education—Way Forward. HKIEd, Hong Kong, 2004. 4. S. Kerr. Lever and fulcrum: educational technology in teachers’ thought and practice. Teachers College Record, 93:114–136, 1991. 5. S. Kerr. Visions of Sugarplums: the Future of Technology, Education, and the Schools. Technology and the Future of Schooling. University of Chicago Press, 1996. 6. J. Lave and E. Wenger. Situated Learning: Legitimate Peripheral Participation. Cambridge University Press, 1990. 7. F.L. Lee, J.H.M. Lee, and T.S. Lau. Learning probability. (Retrieved Nov 1, 2005) http://www.cse.cuhk.edu.hk/~mhp/note. 8. F.L. Lee, J.H.M. Lee, and T.S. Lau. Tong pak fu & chou heung: the probabilistic fantasy. (Retrieved Nov 1, 2005) http://www.cse.cuhk.edu.hk/~mhp. 9. F.L. Lee, J.H.M. Lee, and T.S. Lau. Fantasy-based Learning on the Web—Tong Pak Fu and Chou Heung: the Probabilistic Fantasy. In Proceedings of the Second IEEE International Conference on Advanced Learning Technologies, pages 493– 495, 2002. 10. J.H.M. Lee and F.L. Lee. Virtual interactive students-oriented learning environment: extending the frontier of web-based learning. In Proceedings of the Scholarship of Teaching and Learning, 2001. 11. W.S. Li. A single picture is worth a thousand words: The effect of images on online learning content. In Proceedings of the First Teaching and Learning Symposium, pages 173–180, 2001. 12. T.W. Malone and M.R. Lepper. Making learning fun: a taxonomy of intrinsic motivations for learning. In R.E. Snow and M.J. Farr, editors, Aptitude, Learning and Instruction: Cognitive and Affective Process Analysis, pages 223–253. Lawrence Erlbaum Associates, 1987.

Learning Online: A Comparative Study of a Situated Game-Based Approach

551

13. L. Miller and J. Olson. Putting the computer in its place: a study of teaching with technology. Journal of Curriculum Studies, 26:121–141, 1994. 14. E.J.S. Morris and C.P. Zuluaga. Educational effectiveness of 100% it courses. In Proceedings of 20th ASCILITE Conference, 2003. 15. L. Olson. Alliance aims for ‘break the mold systems,’ not just schools. Education Week, 12(4):8–10, 1993. 16. M. Prensky. Digital Game-Based Learning. McGraw Hill, 2000.

Design and Contents of a 3Dblog System and Its Applications to Edutainment Yoshio Nishio1 , Takami Yasuda2 , and Shigeki Yokoi2 1

2

Kinjo Gakuin University, 2-1723, Omori, Moriyama-ku, Nagoya 463-8521, Japan [email protected] http://anny.kinjo-u.ac.jp/~ ynishio Graduate School of Information Science, Nagoya University, Furo-cho, Chikusa-ku, Nagoya 464-8610, Japan {yasuda, yokoi}@is.nagoya-u.ac.jp

Abstract. In the field of edutainment, it is very important to produce Web3D works efficiently, and to be able to visualize or interactively experience obscure heritage architectures and historical events. In this article, we describe 3Dblog technology, which we have proposed and developed. In addition, we explain a virtual system that we constructed by applying the proposed 3Dblog technology. In addition to being a useful and enjoyable learning tool, 3Dblog not only enables non-professional users to construct objects easily, it also allows both object constructors and users to easily participate in the construction of a virtual environment collaboratively. This brings forth new possibilities in experiencing virtual heritage . . .

1

Introduction

Internet users are continually demanding ongoing sophistication of homepage contents. Web3D [6] is one system that makes it possible for creators and users to handle three-dimensional data on the Internet comparatively easily, and we consider it very important that Web3D be applied to homepages. However, despite its usefulness and importance, it has been very slow to catch on, since content creators need to undertake some study to use Web3D effectively. We have been teaching VR in our classes for many years and as a result have accumulated a great number of Web3D works [4] created by students, but now we need to make good use of all these Web3D works. To solve these problems, we are proposing and developing 3Dblog technology, which makes it possible for creators and users to handle Web3D data easily. Apart from our own previous research in this field [1], there has been very little additional study, with the known exception of 3Dblog in [2] and [3], that combines Web3D and blogging technology. Although it is certainly possible to display and handle 3D objects on networks by using Web3D only, we can produce a 3D environment more easily with Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, pp. 552–565, 2006. c Springer-Verlag Berlin Heidelberg 2006 

Design and Contents of a 3Dblog System

553

our proposed 3Dblog technology. Moreover, because 3Dblog makes it easy for users to share 3D common data and cooperate with each other, it becomes simple to produce edutainment contents. The result is that not only can 3Dblog participants create contents completely by themselves, but they can also enjoy the learning experience. As an experiment to evaluate 3Dblog, we have produced a virtual-reality system featuring the Pan Pacific Exposition [5] held in Japan in the Showa period. The results revealed that 3Dblog includes a number of good features. The purpose of this study is to propose, develop, and improve 3Dblog technology, with the aim of providing a superior developing environment, making it possible to reuse and create Web3D works effectively for all users whether professional or amateur, creator or spectator . . .

2 2.1

3Dblog Technology A Proposal for 3Dblog Technology

3Dblog is the idea and technology we have proposed, one that was born by applying Web3D to blog technology. Put simply, 3Dblog is proposed as follows. 3Dblog is a system that combines Web3D with the concept and technology of blogs. It enables users to contribute their own Web3D works with a minimum of difficulty, and also modify and develop their works collaboratively and dynamically . . . 2.2

Features of 3Dblog Technology

3Dblog technology has the following features. – It makes it easy to produce, control, and manage homepages that deal with Web3D works. – Many users can produce and use Web3D works easily and dynamically. – The trackback function makes it simple for users to contribute Web3D works and also exchange opinions. – 3Dblog provides simple developing tools for specific, applied fields. (VR education, display of houses, development of 3D characters, 3D block games, and so on.) – As the system has library functions, it is easy for users to reuse past works that they or others have produced. 2.3

Registration of Web3D Components by Creators and Advancement of Use of the Components in a Community

In our computer classes, in which we teach Web3D use, we have created and accumulated a large number of Web3D works and components. Although we once exhibited these works on an HTML-based home page, it was merely for reference and scarcely any students reused them; in fact, very few of the works

554

Y. Nishio, T. Yasuda, and S. Yokoi

have ever been reused. If we do attempt to reuse such data, however, there are many problems to solve. These include the center coordinates, scale values and directions of works, types of lights, background, sensor and timer settings, and the category and hierarchical file structure of these works and components. If no information about the data file structures exists, users of the works must try again to interpret and understand the data files. This is a fatal defect for users trying to reuse these works. Furthermore, we noticed that it takes a long time to make these useful Web3D data available. One advantageous method of solving these problems that hold back the reuse of earlier works is to employ the database of Web3D data. Specifically, users register their Web3D works and components, to which data property information such as the file name, additional information, classification, and key words are added. Users can then search the desired data easily by enlisting powerful search functions such as SQL. In this way, it is possible to reactivate data of previous Web3D works . . . 2.4

Collaborative Development, Accumulation, and Reuse of Web3D Data

Figure 1 shows the concept of our developing environment. First, the user needs to establish fundamental information in the 3Dblog user-registration pages, thereby generating the user’s 3Dblog page immediately. The style of the user’s 3Dblog page results from the registered fundamental information. Users and

Fig. 1. The concept of 3Dblog (The collaborative development, accumulation and reuse of Web3D data)

Design and Contents of a 3Dblog System

555

creators can then use archives of past works and contribute their own as well. The Web3D content on a 3Dblog page is modified or added to collaboratively and dynamically, new relationships will be built between content creators and users . . . 2.5

The 3Dblog System’s Structure

We commenced this research using Zope. While Zope is a reliable system, it has a weak point with respect to graphic user interfaces (GUI); therefore, we switched to Perl language. Though Perl is very convenient and makes it easy to develop a 3Dblog environment, it also has a shortcoming in regard to GUI: it cannot support a mouse. Because it is imperative to manipulate 3D objects in 3Dblog, this weak point is fatal. Currently, we are developing and researching 3Dblog technology on the basis of Java and JSP. Figure 2 represents the creation of a user’s 3Dblog page. At the beginning, it is necessary for user who wants to establish a 3Dblog site to set up fundamental data such as the design, title, and administrator password. The 3Dblog generator then produces a fundamental 3Dblog homepage. Figure 3 represents the block diagram of 3Dblog. First, users will enter the top page of 3Dblog. Although the manager of this 3Dblog site would set the style sheet data (CSS) when he or she generates this 3Dblog site, users can also change the style sheet data easily. In practice, the reference servlet is the main page of this site. The reference servlet dispatches the Reference & Contribution (JSP) program when user wants to refer or contribute some data. In this system we will deal with Web3D data as 3D data. Users can produce 3D works with relevant applications and CAD, which we have prepared. They can then contribute their 3D work to the contribution servlet, after which the contribution servlet dispatches the reference servlet and submits the data to a database program (data control & data search java program). We adopted MySQL as the database handle program, as it features many benefits; for example, it is very convenient to import into our system, and new versions of it are very easy to obtain. Users can also search whatever data they want and reuse old archived data without

Fig. 2. Creation of a user’s 3Dblog page

556

Y. Nishio, T. Yasuda, and S. Yokoi

Fig. 3. Block diagram of 3Dblog

any problems. To search data, there are plenty of search keys such as author name, genre, date, category, and so on. Two primary features of 3Dblog are the trackback function and the RSS function. The trackback function makes it easy to form communities: When we write someone’s news into our own blog page, a link is established from someone else’s blog page to our own. Technically speaking, our 3Dblog sends trackback ping data to another 3Dblog site, that site receives the trackback URL, and then the site adds the link automatically. RSS is also a new concept that enables our system to produce RDF data.

Design and Contents of a 3Dblog System

557

Fig. 4. The 3DCAD for 3Dblog

Figure 4 shows the 3D-CAD we developed for 3Dblog. The kind of Blog tool we are developing is for installation on a desktop PC. In the following, we describe the language and the developing environment used to develop the 3Dblog system. – – – – –

Language: Java, JSP, Web3D Environment: Eclipse 3.1.1 Server software: Apache2, Tomcat OS: Windows XP Pentium 4 (3 GHz) machine

The important point concerning the features of 3Dblog is that anybody can make a 3Dblog homepage and begin to administer it at once. All that is needed is to set up simple, fundamental data. Therefore, it is possible to start a 3Dblog homepage for any project. Figure 5 shows an example of the 3Dblog top pages. To contribute Web3D works, creators must enter their own fundamental information. They then edit and contribute pages, and produce their own Web3D works. Since the system provides a list of Web3D works and a common space for all creators of this project, creators can not only contribute to and edit the components, but also remodel common Web3D work collaboratively. Naturally, IDs and passwords are need to administer these components and common work . . .

558

Y. Nishio, T. Yasuda, and S. Yokoi

Fig. 5. Example of the 3Dblog top pages

Fig. 6. Application for the education of virtual reality

3

Applications for Education of Virtual Reality

In the field of edutainment for students, it is very important to provide applications of 3Dblog. Figure 6 shows applications for the education of virtual reality and intelectual education. These are simple developing tools for specific applied fields. The figure of left side shows the application for virtual reality class in our university. For intellectual education for elementary school students, we made virtual block application (the right side of Fig.6). These are useful and enjoyable learning tools. . . .

Design and Contents of a 3Dblog System

4

559

Application for the Real World

Figure 7 and 8 show an application for housing development, an application that actually exists and was used by one of the authors to purchase his own home.

Fig. 7. Application for housing development

Before buying a home, potential buyers can estimate the layout of properties in a certain area. This real-world application includes elements that many people participate in the production of a real town, row of houses, street, or community. It means that this application gives users a place for edutainment about city planning. This application helps users who want to own a new house in a housing development, whether it exists or not in the real world, to build their own dream home. Such a housing development is brought to life with new and old residents because the aspect will change dynamically every day. . . . 4.1

Virtual System Based on the Pan-Pacific Exposition

The 3Dblog system enables many students to easily participate and collaborate in a project while enjoying the learning experience. 3Dblog differs from VR and CG in some respects; specifically, we can place our own works on a site, participate in the project, produce, link to other works, and experience collaborative work. Our students tried to virtually reproduce the Pan-Pacific Exposition and experienced participatory learning with 3Dblog. The little-known Pan-Pacific Exposition (Han-Taiheiyo Hakurankai) (Fig.9) was held in 1937, but we consider it worthwhile to revive such a historical event. Ten students in our group cooperated to create the virtual system. First of all, we divided a map of the Pan-Pacific Exposition into four parts (Fig.10), and we divided the ten students into five groups. Four groups covered one of each of the four parts, while the remaining group gathered four data items from the other four groups. Each group

560

Y. Nishio, T. Yasuda, and S. Yokoi

Fig. 8. Application for housing development

Fig. 9. A picture of the Pan-Pacific Exposition

Design and Contents of a 3Dblog System

561

Fig. 10. The divided parts (The east, sourth, east and central part)

Fig. 11. The 3D model of each parts (The east, sourth, east and central part)

read the coordinates from the original map and generated three-dimensional models (Fig.11). The final group collected these three-dimensional models and unified them into to one. It was necessary to unify them to standardize the data as preprocessing. Our 3Dblog possesses a composition tool that can handle the components’ position, size and direction. Figure 12 shows the composition tool. Finally, we

562

Y. Nishio, T. Yasuda, and S. Yokoi

Fig. 12. The composition tool

Fig. 13. The virtual Pan-Pacific Expo system

Fig. 14. Top view

Design and Contents of a 3Dblog System

563

Fig. 15. The Expo’s central station

Fig. 16. The landscape of the Peace Tower

added hyperlinks for information. We show the virtual Pan-Pacific Exposition system in Figs. 13-18 . . .

5

Applicable Fields for 3Dblog

Furthermore, we added information to the pavilions as hyperlinks. Figure 19 shows an example of the information. However, we noticed some problems in the user interface.

564

Y. Nishio, T. Yasuda, and S. Yokoi

Fig. 17. The Expo’s main entrance

Fig. 18. The Night view of the Expo information

There are so many applicable fields for 3Dblog. Some of them are listed as follows: – – – – –

Open gallery Historical archives of a town 3D community space 3D diary Classes and seminars ...

Design and Contents of a 3Dblog System

565

Fig. 19. The addition of pavilion

6

Conclusions

To date, we have outlined the technology of 3Dblog technology and experimentally produced a virtual system featuring the Pan-Pacific Exposition of 1937. In the PPE project, students could produce the relevant components and integrate them as a site very smoothly. It was proved that we could enjoy the experience of interacting through this system. The result of our experiment clearly shows that this system is very useful for produce Web3D works collaboratively. In the future, we intend to improve and expand the existing Web applications of 3Dblog, especially for a proper human interface . . .

References 1. Y.Nishio, S.Yokoi: Unification Use of Web3D by use of Distributed Environment with Zope. Information Communication Society, Vol. 1. (2004) 63–64 2. Y.Nishio, S.Yokoi: A Study on Construction Method of 3Dblog Technology, Asia Society of Art Science, Vol. 1. (2005) 35–36 3. Y.Nishio, S.Yokoi: Suggestion and Application of 3Dblog. Information Communication Society, Vol. 4.(2005) 5–6 4. Y.Nishio: Development of Collaboration Tools and Virtual Stores. Studies in Social Sciences, Kinjo Gakuin Daigaku Ronsyu, Vol. 44. (2002) 61–71 5. H.Nakata: The Re-creation of the Han-Taiheiyo Nagoya Heiwa Hakurankai by use of CG. Information Communication Society, Vol. 1.(2004) 65–66 6. T.Hirouchi: Web3D Graphics. Pierson Education Corp.

Design and Implementation of Farmtasia: A Game Designed for the VISOLE Teaching Style E.T.H. Luk, M.K.H. Wong, K.K.F. Cheung, F.L. Lee, and J.H.M. Lee Centre for the Advancement of Information Technology in Education, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong

Abstract. The expeditious growth of the Internet and the related technologies hastily contribute and fill up educational applications. While most of them are intended to facilitate the teaching process and are teacher-oriented, the society has continuously asked for a paradigm shift to a student-centred approach. The VISOLE teaching style is a learning paradigm is made up of student-oriented elements which contemplate to infuse learning with amusement. Farmtasia is an online game designed to implement the VISOLE idea with situated learning. Students are suggested to play this online game in groups, interact and compete with other group members. Eventually they are expected to learn the subject knowledge embedded in the game and develop high-order skills through playing the game. The design and implementation of the game follow two principles. The first one is to make the game as realistic as possible so that students can learn from an authentic environment. The second one is to inject fun and interactive components in the game, so that students are eager to immerse and interact in the multiuser environment.

1

Introduction

VISOLE (Virtual Interactive Student-Oriented Learning Environment) [2], a new Web-based learning paradigm, facilitates learning through the Internet by combining initial scaffolding and a near-real virtual gaming environment. The scaffolding stage aims at equipping students with a high-level overview of the curriculum for providing background knowledge and kick-starting the students’ learning motives. The gaming environment then serves as a platform for students to explore, synthesize, and deepen their knowledge. The VISOLE paradigm is usually done through three stages: scaffolding, online gaming stage, and evaluation. The virtual gaming environment provides an ideal platform for multi-disciplinary learning. For example, the Farmtasia game encompass knowledge points from geography, biology, chemistry, technology and economics. Operating a farm in the game successfully and effectively requires the mastery of all the related knowledge. Scaffolding is conducted by teachers in classes. What the teachers do is just to outline in a high level way about the knowledge that will be involved, and how the knowledge from different disciplines is related. Students will also be Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, pp. 566–571, 2006. c Springer-Verlag Berlin Heidelberg 2006


Design and Implementation of Farmtasia

567

suggested where more detailed knowledge can be found, such as searching them on the Internet. The students are then required to take part in an online game to deepen their knowledge. Farmtasia is the first one that demonstrates this purpose. In this game, students are divided into groups of four, in which each player assumes the role of a farm manager interacting and competing against other group members for resources and revenues in nature and the global economic market. The scaffolding stage provides only sufficient background and high-level knowledge. To tackle the scenarios in the game and to win the game, students need to take note and analyse the observed data and search for relevant knowledge in order to make the right decisions. On one hand, the game help students to synthesize learned knowledge in a near-real environment. On the other, students also pick generic and high order skills. Learning is therefore meaningful but not just memorizing facts. The gaming element also makes learning fun. An important feature of Farmtasia is that all players’ actions and activities in the game are logged. This feature allows teacher to (a) observe and understand students’ progress and (b) to extract interesting scenarios from the game proceedings as case studies for class discussion and reflection purposes. The multiplayer nature of the gaming platform ensures the composition of complex and often unique game scenarios as a result of collective behavior of all players. Class discussions and reflections are valuable activities for deepening students’ understanding of the subject matter and analytic skills. The last step of the learning experience is evaluation, which is based on students’ performance in the game, reflection reports, and class discussions. Information thus gathered is a means to judge students’ abilities and the effectiveness of the system in helping students to learn.

2

Features and Design of Farmtasia

In Farmtasia, players compete for for financial gain and reputation. Farm activities include growing crops, maintaining an orchard, and keeping livestock. A player can invest on these three areas of the farm to make money by growing and selling high quality products in the market. Reputation is the result of good and sustainable environmental practices. With no difference from our real world, hard work does not guarantee rewards, and sagacity may not come along with fortune. Catastrophes from nature (which can also be injected by the teaching panel when they find the game proceedings too “boring”) and disasters caused by other players can ruin one’s achievement in a single day. As a farm owner, the player should prepare for the worst and fights against the nature and other competitors. 2.1

Special Features of Farmtasia

Scientific Models. We aim for reality as far as possible in Farmtasia. Our simulation is based on real data and sophisticated scientific models. The geographical model is in charge of four-seasoned climate, alternating temperature,

568

E.T.H. Luk et al.

rain fall, wind dynamics and humidity. A biological model was also developed to control how the living things evolve in the game. Player can experience how their plants sprout, flourish and languish, and witness how their cattles mature and even mate with each other. Last but not least, the economic model deals with the exchange of toil, goods, and money. Record and Replay. We all manifest ourselves from history, and we are educated from history as well. Farmtasia enables teachers and players to replay the game day by day, in order to see what was happening in the past, reflect, and prepare better for the future. Mini Games. Each mini game in Farmtasia is related to a routine activity in the farm, such as weeding, cropping, and scarecrowing. There are two purposes for mini games. First, they are fun. Second, performance of the players in the mini games contributes also to how well the relevant activities are carried out by the players. Every time a player signs on to Farmtasia, the player will be assigned to play a randomly selected mini game individually. All players’ actions and the game results are recorded in the server for later retrieval. A player can later replay and compare all players’ performances in one go to check how good the player is relative to others. This creates the illusion of real-time competition with other players as well, without requiring all players to be online at exactly the same time. Discussion Forum. A discussion forum provides a venue for players to express their ideas and share experiences. The forum also helps teachers to play the role of facilitators. Teachers can initiate and guide discussions, and even evaluate students’ performance there. A Flexible Online Game. Another special feature of Farmtasia is that it does not require students to be online together. With a JAVA virtual machine enabled browser which is very common nowadays, students can get access to the game anytime and anywhere, even from public computers in a coffee shop. There is no need to concern about geographical separation, synchronization of playing time, downloading and security issues. 2.2

Character Design

Characters are important elements of any game. Good character design would certainly increase students’ interest to play. Figure 1 shows some of the characters used in the game. In Farmtasia, each character is represented by an object, and different forms (i.e., the actions an object is taken) is referred to as a state. A state pattern [3] is used to represent an object’s behaviour at a certain state and this is enabled by using polymorphism. This method makes it possible to have two different objects performing in the same way, or the same object performing differently in different situations.

Design and Implementation of Farmtasia

569

Fig. 1. Characters in Farmtasia

Polymorphism [1] is a method which allows a generic interface to be specified by a super class, leaving its subclasses to implement the method as needed. In Farmtasia, a super class called Character is defined. This superclass provides a framework and pattern for its child classes, such as Worker and Sheep. The inheritance structure between superclasses and subclasses saves a lot of development effort. The superclass describes in general what actions need to be done, and a subclass inherits the structure and gives a concrete implementation of the actions. 2.3

Game Control Design

Simplicity is the guiding principle in the game control design of Farmtasia. We adopt the method used in most real-time strategic war game by allowing players to control more than one characters in a select-and-go manner. To move one or more characters, a player has to select the characters first. This can be done by either clicking on the character, or by dragging a region around the group of characters to be selected. After selection, another mouse clicking can command the characters for specific actions. The mini games’ control conserves the same doctrine. All mini games require only one mouse button or no more than three keyboard buttons, allowing the player to enjoy the games with intuitive and simple control.

3

Implementation

Different measures are instigated to make the Farmtasia implementation more secure, flexible and scalable. 3.1

Three-Tier Network Structure

With consideration to scalability, flexibility and security, we adopt a three-tier network model in Farmtasia as showed in Figure 2. The three-tier network model consists of the following. Client Tier: interacts with the players directly and is responsible for the presentation of the game interface. Server Tier: protects the data from direct access by the clients and is also responsible for player account management and global simulation of the game world.

570

E.T.H. Luk et al.

Fig. 2. Three-tier Network Model

Database Tier: is responsible for data storage. Any players can enter the game at the client side with any Java-enabled Web browser, which is then connected to the server. There is no direct access to the database from the client-side which makes the data more secure. As described earlier, every player’s action and every instance of the game world are both stored. This is done by the server and the data received are stored in the database. At 3am every night, the server iterates the game state by doing a global simulation of what the players have done during the day. In the simulation, events such as pollution, prosecution and worker strikes, will be generated and broadcasted to all the players for their next gaming session. 3.2

Struggling with Speed

The adoption of Java applets in our implementation allows anytime-anywhere access and cross platform compatibility. However, the disadvantage of Java applet is speed. Typically, Java programs are not directly executed by the computer’s hardware; instead, it is done by a special program called Java Virtual Machine (JVM) to interpret the Java program line by line. The JVM consumes both processor time and memory while managing a Java program’s execution and generates a significant amount of overhead. We tackle this speed problem using Frame skipping and Partial Redraw. Frame Skipping. To keep Farmtasia running at full speed on slow computers, the rendering frame rate of our game will be adjusted automatically. The slower the computer, the lower the rendering frame rate. However, the actual frame rate of our game will not be affected during the adjustment. This can make sure our game have a constant game play speed on different computers. Partial Redraw. During the redraw pass, the screen objects with their position or frame changed will be marked as dirty. Only screen region with dirty objects

Design and Implementation of Farmtasia

571

will be updated and refreshed. This partial redraw not only increases efficiency of the interface but also speeds up our game on a slow computer.

4

Conclusion

Farmtasia is an online game designed and implemented to realize the VISOLE teaching style, which encompasses high-level scaffolding, game playing for indepth learning, and then students reflection. An important feature of Farmtasia is that players interact in a near-real virtual world, in which players encounter scenarios from geography, biology, chemistry, technology and economics. The game is competitive in nature, but players also have to learn to trade off sustainable development and mere monetary gain. The game not only help players acquire and synthesize knowledge in the subject matters, but also develop generic and high-order skills. Farmtasia and the VISOLE teaching style are going to be field-tested in the coming months. It will be interesting to study and evaluate how well VISOLE can enhance students’ learning experience and efficiency.

Acknowledgement The work described in this paper was substantially supported by grants from the Research Grants Council of the Hong Kong Special Administrative Region. (Project nos. CUHK4200/02H)

References 1. T. Alexander. Massively Multiplayer Game Development. Charles River Media, 2003. 2. L.Y. Chiu, T.H. Luk, J.H.M. Lee, F.L. Lee, Y. Leung, and K.C. Chau. Virtual interactive student-oriented learning environment (visole)—a new web-based learning paradigm. In Proceedings of the Nineth Global Chinese Conference on Computers in Education (GCCCE’2005), pages 834–837, 2005. 3. E. Gamma, R. Helm, R. Johnson, and J. Vlissides. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, 1995.

Teaching Agent Systems’ Design Using 3D Interactive Computer Games In-Cheol Kim Department of Computer Science, Kyonggi University, Suwon-si, Kyonggi-do, 442-760, South Korea [email protected]

Abstract. In this paper, we introduce UTBot, a virtual agent platform for teaching agent systems’ design. UTBot implements a client for the Unreal Tournament game server and Gamebots system. It provides students with the basic functionality required to start developing their own intelligent virtual agents to play autonomously UT games. UTBot includes a generic agent architecture, CAA (Context-sensitive Agent Architecture), a domain-specific world model, a visualization tool, several basic strategies (represented by internal modes and internal behaviors), and skills (represented by external behaviors). The CAA architecture can support complex long-term behaviors as well as reactive short-term behaviors. It also realizes high context-sensitivity of behaviors. We also discuss our experience using UTBot as a pedagogical tool for teaching agent systems’ design in undergraduate Artificial Intelligence course.

1 Introduction Within the academic setting, pedagogical approaches are needed that provide opportunities for students to perform meaningful experimentation through which they can learn many of guiding principles of agent system development. Interactive computer games are known as one of killer applications for human-level AI [4]. They can provide the environments for research and education on the design of intelligent agent systems. Computer-controlled characters or Non-Player Characters (NPC) in these games are autonomous agents capable of playing without any human intervention. They integrate all the human-level capabilities such as real-time response, interaction with the environment, communication with teammates, planning their activities, learning from experiences, and common sense reasoning. The UTBot was designed to enable a project-based curricular component that facilitates the use of the Unreal Tournament game engine [2] and Gamebots system [1] in undergraduate Artificial Intelligence course. There are several aspects which make it difficult for beginner students to build their own intelligent virtual agent for the UT game and Gamebots system from scratch. 1. There is a large amount of low-level work that needs to be done before starting to develop sophisticated behaviors, for example, parsing the sensor input and creating a world map. Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, pp. 572 – 576, 2006. © Springer-Verlag Berlin Heidelberg 2006

Teaching Agent Systems’ Design Using 3D Interactive Computer Games

573

2. It is not easy for beginners to figure out what agent control architecture is appropriate for dynamic virtual environments. 3. Without any templates and sample behaviors, many undergraduate students can not undertake their project promptly. The UTbot addresses these issues in an effort to provide an effective platform for teaching undergraduate students agent systems’ design.

2 Computer Game Environment Unreal Tournament (UT) is a category of video games known as first-person shooters, where all real time players exist in a 3D virtual world with simulated physics. Every player’s senses are limited by their location, bearings, and occlusion within the virtual world. The Gamebots [1] is a multi-agent system infrastructure derived from Unreal Tournament. The Gamebots allows UT characters to be controlled over client-server network connections by feeding sensory information to client agents and delivering action commands issued from client agents back to the game server. In a dynamic virtual environment built on the Gamebots system and the UT game engine, agents must display human-level capabilities to play successfully, such as planning paths, learning a map of their 3D environment, using resources available to them, coordinating with takes their adversaries into account. Although the Gamebots system is a great platform for students to build their own intelligent virtual agent, a large amount of low-level work and behavioral complexity make it difficult for a beginner to finish this project in a three-month course.

3 Agent Architecture In order to support students’ development of their own agent for the UT and Gamebots environment, we provided them a generic agent architecture called CAA(Context-Sensitive Agent Architecture). Our CAA consists of (1) a world model; (2) an internal model; (3) a behavior library; (4) an interpreter; and (5) a set of sensors and effectors. The world model contains a set of objects representing current beliefs or facts about the world. The world model is defined as an abstract class to be implemented as a domain-specific world model for a certain kind of application. The world model is constantly updated upon the sensor information. On the other hand, the internal model contains a set of objects representing internal modes, or intentions. Each internal mode can be viewed as an implicit goal to be pursued. Depending on the changes of the world model, the internal model may be updated accordingly. Transitions between distinct internal modes can be modeled and designed as a finitestate machine. The behavior library contains a set of pre-defined behavior objects. The behavior class has three sub-classes: external behavior, internal behavior and conversational behavior. While external behaviors change the state of the environment through effectors, internal behaviors change the internal state – namely, the internal mode and parameters- without any change of the environment. Conversational behaviors can be used to communicate with other agents in a certain

574

I.-C. Kim

agent communication language or protocol. Conversational behaviors can be also viewed as a special kind of external behaviors. The behavior class has five main member methods: applicable(), utility(), maintainable(), run(), and failure(). The applicable() method checks if the preconditions of a behavior can be satisfied against the world model and the internal model. The utility() method computes the relative utility of an applicable behavior by considering the current world model and internal model. Whenever multiple behaviors are applicable for a given situation, the highest-utility behavior is automatically selected and executed from them. The maintainable() method continually checks the context of a behavior throughout the execution of the behavior once it starts execution, to make sure that the behavior is still applicable to the intended situation. The run() method is the main body of a behavior. It gets called when the selected behavior starts execution. This method usually generates one or more atomic actions, sets some member variables, and returns. Finally, the failure() method is a procedural specification of what the agent should do when a plan fails. In the CAA, the life cycle of a behavior object consists of seven distinct states: create, waiting, executing, interrupt, fail, resume, and finish. The interpreter controls the execution of the entire CAA system. Whenever there is new or changed information in the world model or internal model, the interpreter determines a set of applicable behaviors by calling the applicable() method of each behavior. From this set of applicable behaviors, it selects the highest-utility behavior by using the utility() methods. By invoking the run() method of the selected behavior, the interpreter starts the execution of the behavior. Once the selected behavior starts execution, the interpreter continually checks the behavior’s context by calling the maintainable() method periodically. If the context of the behavior gets unsatisfied with either the current state of the world model or of the internal model, the interpreter immediately stops the execution of the behavior, and then replaces it with a new behavior appropriate to the changed situation. Sensors periodically perceive the surrounding environment and update the world model. The input to sensors is divided into several sub-classes: visual input, aural input, physical input, and server input. Effectors execute the atomic actions requested by the run() method of the current external behavior and, as a result, affect the environment. Each sensor and effector has its own thread and work concurrently with the interpreter. An intelligent virtual agent based on the CAA can have multiple domain-specific sensors and effectors.

4 UTBot The UTBot is a virtual agent platform which provides students with the basic functionality required to start developing their own intelligent virtual agents to play UT games. The UTBot includes the CAA agent architecture, a domain-specific world model, a visualization tool, several basic strategies (represented by internal modes and internal behaviors), and skills (represented by external behaviors). The UT world model contains both static and dynamic information. Static information does not change during the course of a game. Static information includes, for example, the agent’s name and ID, the team number, the number of team members, the maximum team score, and the address of the game server. In contrast, dynamic information

Teaching Agent Systems’ Design Using 3D Interactive Computer Games

575

continually changes during the game. Dynamic information includes, for example, the agent’s position and direction, the health and skill information, the current weapons and armors, a partial world map, and the discovered domination points. The UT internal model contains an internal mode and the related parameters. There are five distinct internal modes: Explore, Dominate, Collect, Died, and Healed. The internal parameters such as the starting position and the target object may accompany one of the Explore, Dominate, and Collect modes. The UTBot has several basic external behaviors such as Explore, Attack_Point, Defend_Point, Collect_Powerup, Collect_Weapon, Collect_Armor, Chase, Attack, Retreat, and MoveTo. The applicable() and maintainable() methods of each external behavior may contain conditions against the UT world model, the UT internal model, or both. However, most of applicable external behaviors of the UTBot are primarily categorized depending on the internal mode. During the game, therefore, the set of applicable external behaviors is first restricted by the current internal mode of the agent. Although more than one external behavior is applicable at a certain internal mode, the utility values may be different among them. To transit from one internal mode to another, the UTBot has a set of internal behaviors such as ExploreToDominate, DominateToCollect, and CollectToDominate. This set of internal behaviors forms a unique strategy for determining the UTBot’s external behaviors. The UTBot provides its unique visualization tool. With this tool, users can launch and destroy UTBots. Using this tool, users can also easily keep track of both the current state and executing behavior of each UTBot. Therefore this tool can help users to analyze and debug individual behaviors in detail.

5 Lessons The author has taught an undergraduate class in Artificial Intelligence at the Kyonggi University three times during the years 2002, 2004-2005. On average, the class is attended by 40-50 students each semester. Since it is junior class in the Computer Science department, the students are assumed to be competent Java programmers and to have some background knowledge on distributed systems. However, there are no formal prerequisites for the class. The class has used as textbook Artificial Intelligence: A New Synthesis written by Nilsson. This textbook emphasizes so called agentoriented approach to AI and deals with how to build intelligent agent systems. We believe that the difficulty in building intelligent agent systems can only be properly understood by actually building fairly complex systems. As such, our class uses a hands-on approach to teaching intelligent agent systems. Our class has used the UT game engine and Gamebots system as an educational tool since the first time the class was taught. The use of UT and Gamebots has been very successful. The students are made to form teams of four to five students. The teams compete in a UT Domination tournament at the end of the semester. The students are given a month to complete their assignments after some period of practice. The grade for the project is determined by the team’s standing in the tournament and the quality of their final report and presentation. The tournament format has proven to be a great motivator. After the first class we learned that the basic Gamebots client offers so little functionality that students had to spend almost all their time trying to

576

I.-C. Kim

implement basic behaviors such as exploring the unknown environment and constructing a partial world map instead of focusing on the architectural and strategic aspects of agent design. In order to ease this burden we developed the UTBot system. The UTBot provides a generic agent architecture (CAA) and implements a basic player that maintains a world model and executes many useful behaviors such as exploring, attacking and defending a domination point, and collecting power-ups. Our experience using the UTBot as a learning tool has been overwhelmingly positive. Its use allows students to directly experience the problems inherent in building intelligent agent systems. Even though a short time was given for undergraduate students to build an intelligent virtual agent, all of eleven teams have successfully completed their projects and were satisfied with the result. However, we are not entirely satisfied with our current UTBot implementation. It provides neither mechanism for coordinating team activities nor for self-learning from experiences. We are going to extend the UTBot by building up these additional functionalities for next year class.

6 Conclusions We introduced UTBot, a virtual agent platform for teaching agent systems’ design. The UTBot implements a client for the Unreal Tournament game server and Gamebots system. It provides students with the basic functionality required to start developing their own intelligent virtual agents to play autonomously UT games. Through our experience using the UTBot in undergraduate Artificial Intelligence class, we are sure that it is a very effective tool for teaching principles of agent system development.

References 1. Adobbati, R., Marshall, A.N., Scholer, A., Tejada, S., Kaminka, G.A., Schaffer, S., Sollitto, C.: GameBots: A 3D Virtual World Test Bed for Multiagent Research. Proceedings of the 2nd International Workshop on Infrastructure for Agents, MAS, and Scable MAS (2001) 2. Gertmann, J.: Unreal Tournament: Action game of the year. (1999) 3. José M. Vidal, Paul Buhler, and Hrishikesh Goradia: Tools and Lessons from a Multiagent Systems' Class. Italics, 4(3) (2005) 4. Laird, J.E. and van Lent, M.: Human-level AI’s Killer Application: Interactive Games. AAAI Fall Symposium Technical report, North Falmouth, Massachusetts (2000) 80-97

Cyranus – An Authoring Tool for Interactive Edutainment Applications Ido A. Iurgel ZGDV e.V., 64283 Darmstadt, Germany

Abstract. This paper presents the authoring tool Cyranus, which implements novel authoring methods for highly interactive edutainment applications. Both concluded work and ongoing and future issues are presented. Among the concluded work, a framework is described that integrates a hierarchic transition network with non-graph based methods, in particular with a rule-based system. This facilitates the authoring process considerably, and enhances the power to control the logic of an application.

1

Introduction

The creation of the logic of an edutainment application must not become a bottleneck in the production chain. But current authoring methods show clear limits, inasmuch they require unintuitive heavy programming that constraints the creative process and excludes direct authoring by content and education specialists. Established methods that do not require heavy programming - basically directed graphs - limit power of control and are not appropriate in many situations. This paper presents achieved results and ongoing work on solving problems of authoring by enhancing authoring tool principles. The authoring tool that exemplifies these efforts is called Cyranus. It was or is being used within art-e-fact (cf. [3], www.art-e-fact.org), Virtual Human (cf. www.virtual-human.de), Virtual Human with Social Intelligence (a project commissioned by SAP Research Center California for creating virtual assistants and companions), its concepts are being reused and adapted for Inscape (cf. www.inscapers.org), and others.

2

Related Work

Mateas’ and Stern’s recent work represent important advance in interactive storytelling. But the creation of their interactive story is an enormously complex endeavor and requires heavy programming (cf. [1]). No commercial authoring tool offers a satisfactory solution for the problem of enabling easy but powerful visual authoring. For example, the standard tool for multimedia applications, Macromedia Director, offers a time line and enables to employ scripting; but scripting represents a switch out of the visual support into pure programming. Gebhard et al. [2] have combined within their authoring environment, as in Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, pp. 577–580, 2006. c Springer-Verlag Berlin Heidelberg 2006


578

I.A. Iurgel

Cyranus, rules and a directed graph approach, though their strategy does not allow to dismiss the edges, but instead relies on rules for the generation of specific events that are checked at the guard conditions of the transitions. Without entering into details, some principles of Cyranus were already described in [3], where other aspects were on focus.

3

Enabling Directed Graph Control

Directed graphs are still the standard control method for entertainment applications, and they are appropriate and sufficient in many contexts. Cyranus offers a hierarchic transition network for those cases, which is derived from Harel’s State Charts ([6]).

Fig. 1. On the left a screen shot of Cyranus. On the right details. Note that the composite state ”curious” carries an ”R”, and that this state appears again at the upper reference pane. This is a reference state. In this example, it controls the behavior of virtual characters that are waiting for some input of the user. It can be reused at different places. Reference states are drag-and-dropped from the reference pane.

Behaviors that can occur at many places of the interactive application pose a problem for the directed graph approach. For this, Cyranus introduces reference states, which are composite states kept automatically identical; they are in fact only references to an original state kept outside the transition network. Additionally, reference states permit Cyranus to enforce a strict encapsulation policy that forbids transitions to cross the boundaries of composite states. Instead, reference to the state that would lie outside the currently active composite state can be employed.

4

Integrating Rule Based Control

Often, directed graphs for control of interactive applications would become very dense, and the guard conditions very complex. Authoring becomes almost

Cyranus – An Authoring Tool for Interactive Edutainment Applications

579

impossible. Therefore, Cyranus allows the activation of a state without using a transition. The main activation engine is the Jess engine that employs the rule system Jess (cf. herzberg.ca.sandia.gov/jess). In order to retain the encapsulation of the composite states, and in order to allow for the concurrent use of different activation engines, an activation engine is attached to exactly one specific composite state, and the states that can be activated are exactly the direct daughter states of this composite state. Every activation engine has its own data storage, keeps a representation of the meta data of the states, and is informed about every external event of the system. Several activation engines can compose a hierarchic chain, following the hierarchy of the composite states. These engines are called down-to-top, when no guard condition of a transition is passable, until some engine can handle the situation (or the system gives up).

Fig. 2. Employing an activation engine. Most of the circles - terminal states - are not attached to transitions, and they have an arrow on top. This arrow symbolizes that they contain meta data, and can thus be activated by an activation engine. It is possible to combine transitions with the activation engine.

For example, we employ a Jess engine in art-e-fact, for the ”hot spot”scenario. There, the user is pointing to a painting to discover its details, and virtual characters comment and explain the painting, with the help of video recognition. With the activation engine framework, it is quite easy to handle the many possible permutations sensibly. Note the obvious analogy between this example, and the logic of common learning adventure games, and note also that a top level composite state can be regarded as a ”scene” of a dramaturgy, or as a ”lesson” of a learning application. I.e., if the topology of states and transitions, and the guard conditions become too complex, it is possible to employ instead or additionally story and learning models, recurring to an activation engine, to control the sequencing.

580

5

I.A. Iurgel

Future Work

Future work will address the modularity and interactivity of the authoring process. Machine learning technology will be employed, in particular Case-Based Reasoning. A first demonstrator for this technology will be shown at the CeBIT 2006, within the project Virtual Human. The goal is to increase the interactivity of the authoring process itself. The application environment shall be able to suggest behaviors and decisions, relying on formal models, rules and previous cases, and the author, in a dialog with the Cyranus environment, will define further rules, behaviors and cases to improve the performance of the system.

6

Conclusion

Not only the runtime AI processes of interactive edutainment application can possess remarkable complexity; the AI support of the authoring process itself must grow with this increase of complexity. For example, in the authoring mode, a session of the entertainment application should explain to the author its autonomous behavior choices, enabling the author not only to correct this behavior, but also to correct the reasons that the system had to choose it. In spite of possible formalization of models for education, story and personality, future authoring tools will need to support authoring as an essentially artistic and intuitive, rather than engineering process. The author will need to interact more with the running application to develop ideas and correct imperfections, following his expertise and also his intuitions e.g. on education or drama.

References 1. Mateas, M., Stern, A.: Procedural Authorship: A Case-Study Of the Interactive Drama Facade. Digital Arts and Culture (DAC), Copenhagen, November (2005) 2. Gebhard, P., Kipp, M., Klesen, M., Rist, T.: Authoring scenes for adaptive, interactive performances. Proc. of the Second International Joint Conference on Autonomous Agents and Multi-Agent Systems. Melbourne, Australia. ACM Press 3. Iurgel, I.: Narrative Dialogues for Educational Installations. NILE 2004, 10th - 13th August (2004) 4. Iurgel, I., Ziegler, M.: Ask and Answer: An Educational Game where it Pays to Endear your Capricious Virtual Character. IVA 05, International Symposium on Intelligent Virtual Agents, September 12-14, Kos, Greece, (2005) 5. Iurgel, I.: From Another Point of View: Art-E-Fact. TIDSE (2004), 26-35 6. Harel, D.: On visual formalisms. Communications of the ACM. V. 31 I5 pp. 514-530, (1988)

Research of Virtual Campus Environment Study Using VRML Weihua Hu, Chuying Ke, and Guorong Wang Computer Science Department, Hangzhou Dianzi Univeristy, Hangzhou, 310058, P.R. China

Abstract. This paper introduce some technology about using VRML to construct 3D virtual scene and its interactive roaming by virtual campus environment’s researching and practice. Key techniques on building virtual campus, embeded virtual avatar for interaction and browser speed improvement are introduced.

1

Introduction

Virtual Reality (VR) is a new way of optimization between person and computer. It can construct more realistic virtual world in 3D vision, audible, tactile. It can also help users walking through the virtual world and interact with virtual entities designed through a nature feeling. VRMLVirtual Reality Modeling Languageis considered as the second Web language after HTML, which can be used to describe 3D object and its action. Users construct Virtual World by integrating various media styles such as textpicturesound, inserting program codes of Java. We use VRML in many virtual ramble environment which based on Web. By object-oriented description, expressive virtual environment are realized by entities’ configuration, packaging, reuse and hyper link. Virtual campus study environment constructed by VRML is more expressive that a simple 3D scene. It can show campus’ scenery through the net. Users can virtually walk into campus, feeling the nature resemblance of various campus scene. Followed by virtual tutor, users can acquaint himself with each building’s relative position and its function. By invoking video introduction, they can know the characteristic of different introduction. In virtual classrooms, users can learn by interactive materials. Features like these are very important for freshmen. Foreign users can also login the net to explore our virtual campus.

2

Key Techniques of Constructing Virtual Campus

Our goal is to direct space relation to developers to build different models of main buildings, roads in the campus. All the models would be organized as a way to constitute the virtual campus. Browser can make use of the input device control to see the point and angle of users’ view. Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, pp. 581–584, 2006. c Springer-Verlag Berlin Heidelberg 2006 

582

2.1

W. Hu, C. Ke, and G. Wang

Constructing Buildings’ Model

Buildings in virtual campus can be modeled by the real building such as library, administration building, comprehensive building and the second experiment building, etc. Altogether, 8 structure models are built. Models constructing are used by several nodes. Appearance node helps to give a nice out-looking which making use of Box, Cone, Cylinder, Sphere(surface of sphere), ElevationGrid (high distance mesh), Extrusion( pull to stretch the noodles),etc. The external appearance node includes the Material, ImageTexture (the portrait veins), PixeITexture (the pixel veins), MovieTexture (the image veins), TextureTrasnsform (the veins variety). Relative positions of the nodes is implemented by local coordinate system.

Fig. 1. A side view of a building modeled in virtual university campus

2.2

Tree Models

It is quite simple for implementing virtual trees. We can develop the veins diagram and sticking to a tree. The steps are: First, we must adopt from a set of Billboard nodes, and use the shape node as the sub-node of that nodes. The nodes of Billboard contain one special function, that is to make its sub-node face to the audience forever. Next is to specify the value of X, Y direction of the square box. It should defined according to the breadth and high of the tree in the scene, distinguish to 2 and 5 here. The number of Z direction should establish for enough small, make the square-panel looks as without thickness of an elephant. At last, the veins diagram needs the background transparent, this is the key of manufactures the tree. It is handled as follows(1)make use of the digital camera to take photos of the selected tree in campus.(2)Make use of the Photoshop picture handle software to handle the taken picture documents. (3)Establish the background of the picture as background transparent. (4)Adjust the size of the picture, then save for the format of gif. 2.3

Text Display

According to the norm of VRML2.0, the VRML supports all character lists of UTF8. But many browsers do not support Chinese types. The latest browser of Blaxun4.4 has already provided the manifestation of the Chinese characters. The browser that this system adopt also does not support Chinese characters, but

Research of Virtual Campus Environment Study Using VRML

583

Fig. 2. Trees in our virtual campus

system in many places need the marking of the Chinese characters, especially the school name beside the front gate of the school must be meant out with the Chinese characters. Consideration of the same Proceed as the mold setting up with tree, the manifestation of the Chinese characters also adopts the method realization of the picture. Different from tree setting up the model is a node, another different place is the picture of the Chinese characters is to make use of the tool of UleadCool3d born, and save as the transparent background format of gif, other places likes trees mold setting up. 2.4

Browser Speed Improvement

Speed of browser is critical for net-based users. Follows methods are used to improve the speed: (1) Reuse models in the scene. Some places, such as door, window, even a parts of a certain on the sides are all same. Make use of the DEF and USE to the same part so heavy can simplify to describe the document biggest with the mechanism. (2) Set distance which can see object, reducing the browser to exaggerate the quantity of the object once. (3) Makes use of view cut techniques, carrying out the same object with different models to cut over automatically. (4) Combines the enactment that can see the distance, making use of trigger machine technique or a views cut over the technique excellent turn the solid building body of demonstration. 2.5

The Realization of Navigation

To achieve a vivid simulation of virtual view, the system adopts navigate tutors. The realization that navigate automatically is constitute by following methods: navigate the person to set up the model and navigate the route to be specified. The orotund and introductive control, sport control and view control are also designed for assistance. We use Zapa Character Builder 98 to build virtual tutors. Because the born object exists a great deal of redundancy information, and in order to carry out the sport control, we must understand its internal structure.

584

W. Hu, C. Ke, and G. Wang

The virtual tutor stands on the front gate and gives an overview of our virtual campus. Then, she will lead you walking through the campus. Following certain predefined route, she will make simple introduction to the specified view points.

3

Conclusion

3D virtual campus study environments are the application that the virtual realistic technique is in the aspects of educating and studying. This paper shows our implemented virtual campus as an example makes the first step to be practical for many users. 5 implement aspects are introduced. Such techniques are also helpful to other virtual education environment.Future work includes building the virtual teaching of the exhibition virtual student’s community, establishing simulation of class teaching environment, which make more users understand our virtual campus, help them to study in such virtual environment.

References 1. Budzynski, C.J. and Zabora, B.J.: eLearning: The Evolution of Education and Training. (Industry Analysis) Baltimore, MD: Legg Mason, Inc. (2000) 2. Cattagni, A., Westat, E.F.: Internet Access in U.S. Public Schools and Classrooms 1994-2000 (Report No. NCES 2001-071). Washington, DC: National Center for Education Statistics. 3. Gong, J.H., Lin, H.: Virtual Geographic Environment-A geographic Perspective on Online Virtual Reality. Beijing:high Education Press.(2001) 4. Lacy, K.: MSUB Online Program Review 1998-2001. Unpublished manuscript, Montana State University-Billings. 5. ISO/IEC1 4772-1:1997, VRML97 International Standard. The VRML Consortium,http://www.vrml.org 6. Schank, R.C.,:The Virtual University. Psychology and Behavior (2000),3(1):9-16. 7. George, E.H.,: CECA(International Committee of Museum Educators) Conference Jerusalem Israel,(1991),1:15-22. 8. Brooks, M., Brooks J.,: In Search of Understanding: the Case for Constructivist Classrooms, Alexandra, VA: ASCD,(1995),1(3). 9. Steve, B.,: Virtual Reality. The Computer Science and Engineering Handbooks,(1997), 1(4):1354-1374 10. Cronin p.,:Report on applications of Virtual Reality Technology to Education, HRHC, University of Edinburgh.(1997). 11. Yun, R.W., Pan, Z.G., Li, Y.,:An educational virtual environment for studying physics concept in high schools, Advances in Web-based learning (2005):326-331. 12. Hu, W.H., Zhu, J.J., Pan Z.G.:Exploring an agent-driven 3D learning environment for computer graphics education, INTELLIGENT VIRTUAL AGENTS (2003):355-355 13. Pan, Z.G., Zhu, J.J, et al. Interactive Learning of CG in Networked Virtual Environments. Computers and Graphics. (2005),29(2).

A Multimodal Fusion Framework for Children’s Storytelling Systems Danli Wang, Jie Zhang, and Guozhong Dai Institute of Software, The Chinese Academy of Science, Beijing 100080, China {danli, zhangjie, guozhong}@ios.cn

Abstract. Storytelling by a child, as a training activity, influences significantly a child’s linguistic abilities, logic and thought process, imagination, and creativity. There presently exist many software-based storytelling applications. However, most are deemed incompatible or not suitable to Chinese children. Due to the limited vocabulary of pre-school and lower-level grade school children, speech-based and pen-based input models are considered the most effective way of input. But now there is not an effective multimodal mode to solve the problem for children’s storytelling systems. In this paper, we propose a multimodal fusion framework that utilizes pen and speech techniques to incorporate both context information and linguistic attributes of the Chinese language into the design. Based on the proposed framework, we formulated specific methods of integration, and developed a prototype for our proposed system.

1 Introduction The importance and magnitude of difficulty of children’s education are well-known. In light of the rapid advancements in computing and networking technologies, developing innovative approaches to children’s education with such advanced technologies has become a necessary trend. Hence, research in this area has focused on early child development and education, in search of ways to effectively exploit computing technologies to match current state of the art in content and interface development. The objective is to maximize the benefits while minimizing any potential disadvantages, and also to provide an attractive alternative for learning to the children outside of the classroom. During a child’s language learning process, storytelling has significant influence in training the child’s ability of linguistic expression, logic and thought, imagination, and creativity [1]. Modern education has demonstrated the trend in utilizing innovative technologies to aid in child education. However, due to the characteristics of young children: IQ, knowledge, and lack of experience, successful designs must take into consideration the ability and constraints of the children, especially children under the age of 6. Presently there exist several storytelling application software for children [2,3,4], but these are not very suitable, in content and format, to Chinese children. To address these concerns, we intend to exploit and integrate the strengths of the existing software, while combining the unique characteristics of the Chinese culture, to create Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, pp. 585 – 588, 2006. © Springer-Verlag Berlin Heidelberg 2006

586

D. Wang, J. Zhang, and G. Dai

a storytelling environment for the Chinese children, which will enable them to fully explore their imagination and creativity. To design storytelling software suitable for children, one must not only consider the content, one must also consider the appropriate methods of interface. Due to the limited vocabulary of pre-school and lower-level grade school children, speech-based input model is considered the most effective way of input. However, due to the current low rate of speech recognition, it would be more viable to combine speech- and pen-based multimodal user interface for storytelling software. Since the concept of multimodal user interfaces was proposed by Bolt in 1980, there have been many relatively mature methods and application systems[5,6]. Basically, there are two major architectural approaches to the analysis part of multimodal systems: (i) early fusion--fusing modes already at signal or recognition level and (ii) late fusion -- multimodal semantic processing[7].Yet many multimodal applications are based on the assumption that each of the single modes does not present recognition errors [8, 9]. A better solution was proposed by Zhang et al.[10] to provide fusion for multimodal information. However, eye-activated tracking would not be very suitable to children’s storytelling system. Therefore, this paper proposes a context knowledge-based multimodal fusion algorithm to solve the problems encountered in children’s storytelling applications. In this paper, we adopted speech and pen gestures as our primary multimodal interface model, and provided a context-based fusion framework. Based on this algorithm, we designed the multimodal storytelling system. The remainder of this paper is organized as follows: Section 2 introduces our proposed multimodal fusion framework. The realization of our multimodal children’s storytelling system is discussed in Section 3. Summary and conclusion are provided in Section 4.

2 Multimodal Fusion Framework Focusing on the uniqueness of storytelling systems, we chose a multimodal interface approach that uses speech to describe every sentence in the story, and uses pen gestures to point to a target, position, path, etc. Figure 1 illustrates our multimodal fusion framework based on speech, pen-based gestures, context and knowledge. To realize our proposed fusion framework, we first set a time domain range value. During this time range, if only one mode contains information, then the framework provides the result directly. On the other hand, if both modes contain information, then the system follows the framework in Figure 1. The process of our proposed multimodal fusion consists of these steps: first perform recognition and understanding of information from the two modes. Secondly, determine if the information is referring to the same semantics. If so, then directly produce the fused semantics. Otherwise, then fusion is performed using context information and knowledge within the system. Thirdly, the fusion results are used to plan and output tasks, using the appropriate output formats, to the user. The user can also input corrective information given any fusion errors.

A Multimodal Fusion Framework for Children’s Storytelling Systems

sp eec h

rec ogn itio n u nd er stad in g

con tex

co n sisten cy

p en gestu re

rec ogn itio n u nd er stad in g

No

587

knowl edge a ction

f usi on

r es ul t

p lan n in g

r epr es en t at i on

Yes

user f eedback

Fig. 1. Framework of multimodal fusion

3 Multimodal Fusion-Based Storytelling System for Children Based on the multimodal fusion framework discussed in Section 2, we developed a storytelling system for children. The system was supported natural speech and pen as its interface with the computer. The child performs creative activities via constrained natural language and pen motions. The system provided feedback to the user input with 3-D graphical animation to describe the story content, coupled with descriptive text and speech information. The OpenGL library enables the 3-D graphical animation. Figure 2 depicts a system developed from our framework (the story was adapted from a Chinese folk story named “Little Horse Crosses the River”.) The story is “told” by a child using speech to describe scenes of the little horse crossing the river. The child also uses the pen to point/draw the direction, path, and destination of the little horse as the animal crosses the river. The system, based on such input from the child, produces dynamic and animated content of the story, along with textual description. User can also provide information to correct any errors from the system.

Fig. 2. Little horse crosses the river

588

D. Wang, J. Zhang, and G. Dai

4 Conclusion This paper discussed the advantages and current state of the art for multimodal interface technologies. In this work, we proposed a multimodal fusion framework with speech and pen gestures being the interface modes, emphasizing the unique attributes of children’s storytelling systems. Using the proposed context-based multimodal fusion framework, a system was developed. Feasibility of the framework was verified through this system.

Acknowledgements The research is supported by National Grand Fundamental Research Program (Grant No.2002CB312103) and National Natural Science Foundation of China (Grant No.60373056).

References 1. Wright, A. Creating stories with children. England: Oxford University Press,1995 2. Benford, S., Bederson, B., and et al. Designing Storytelling Technologies to Encourage Collaboration Between Young Children. In Proceedings of CHI 2000, 556-563. 3. Montemayor, J., Druin, A., et al. Physical Programming: Designing Tools for Children to Create Physical Interactive Environments CHI 2002, ACM Conference on Human Factors in Computing Systems, CHI Letters, 4(1), 299-306. 4. Bers, M & Cassell, J. Storytelling Systems: Constructing the Innerface of the Interface. Cognitive Technologies Procedings '97, IEEE, pp.98-108. 5. Cohen P. R., Johnston M., and et al. Quickset: Multimodal interaction for distributed applications. In Proceedings of ACM Multimedia, 31- 40, Seattle, WA.1997 6. Oviatt S.L. Multimodal Interfaces, Handbook of Human-Computer Interface, Ed. By J.Jacko & A.Sears, Lawrence Erlbaum: New Jersey, 2002. 7. Pfleger N. Context Based Multimodal Fusion, In Proceedings of ICMI’04, pp 265-272, 2004, State College, Pennsylvania, USA. 8. Neal J.G., Thielman C.Y., Dobes A., Haller S.M., Shapiro S.C. Natural language with integrated deictic and graphic gestures, M.T.Maybury & W. Wahlster(Ed.), Readings In Intelligent User Interfaces (pp.38-51). San Francisco: Morgan Kaufmann Publishers, 1991 9. Campana E., Baldridge J., Dowding J., Hockey B.A., Remington R.W., Stone L.S. Using eye movements to determine referents in a spoken dialogue system. In Proceedings. of workshop on perceptive user interface. Orland, Florida, 2001. 10. Zhang Q, Imamiya A, Go K, Mao X. A. Gaze and Speech Multimodal Interface. In Proceedings of the 24th International Conference on Distributed Computing Systems Workshops (ICDCSW’04), 2004.

Design of a Cartoon Game for Traffic Safety Education of Children in China Zhen Liu Faculty of Information Science and Technology, Ningbo University, Ningbo 315211 [email protected]

Abstract. A method of developing cartoon game on traffic safety education of Chinese children is presented in this paper. Based on the psychological characteristic of children and constructivism learning theory, the design of game scenario, character’s emotion and interaction is introduced. The scene in this cartoon game is based on Gibson’s theory of affordances and all the signs on objects are easily distinguished. A finite state machine is used for expressing the emotion state of a cartoon character, and NPC (non player character) is introduced to arouse the interest of a child user. The cartoon game can run both on local PC and on Internet, a child user can learn safety knowledge by interactive method. The result will offer a new means for traffic safety education of children in china.

1

Introduction

Children’s traffic accidents rise continuously in china. According to statistics, 14400 Chinese pupils died from traffic accidents in 2000 year, and the death number reached to 16000 in 2001 year, traffic safety education of children is a very urgent task in china. Watching case analysis and teaching safety regulations are traditional methods of safety education of children. These methods seldom consider the psychological characteristic of children. Educational effect is rarely visual and vivid, and is hard to arouse the notice of children. The logic thought of children has not grown up completely, children like visual and interesting things by their psychological characteristic [1]. According to the viewpoint of psychologist Jean Piaget, children’s knowledge roots in cognitive constructions for objective world [2]. Construction of learning environment is very important to raise children’s learning interest. Knowledge acquirement is realized not only by teachers, but also by the children’s initiative construction of inner connection among things in learning environment, modern cartoon games offer a technical guarantee for realization of constructivism. A cartoon game can play a very important role in safety education of children. In general, a cartoon game has the expression force of exaggeration and can attract children to learn in interactive manner [3]. The abstract safety regulations Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, pp. 589–592, 2006. c Springer-Verlag Berlin Heidelberg 2006


590

Z. Liu

are shown in a vivid cartoon scene, a child user can act as a character in cartoon game and perceive safety situation. In a word, learning by a cartoon game has the advantage that traditional education methods cannot compare with.

2

Design of Scenario

The cartoon game in this paper includes some scenarios with given safety themes. For example, the theme of “pedestrian should walk across Platform Bridge” is explained as follows: When the leading cartoon character walks near to longitudinal road where there is Platform Bridge, the screen will appear two flickering arrowhead hints for selecting, one is “walking across Platform Bridge”, and the other is “walking across the road”. When a game user selects “walking across Platform Bridge” hint by mouse, the leading character will turn to Platform Bridge and walk along the Platform Bridge to the other side of the street; when a game user selects “walking across the road” hint, the leading character will go on to the middle of the street. A car will comes urgently to him and the brake is too late, the car bumps down the leading character, the animation of traffic accident and a crying cartoon face will appear on screen, the process is shown in Fig. 1- Fig. 3.

Fig. 1. The leading character walks near to Platform Bridge

Fig. 2. A traffic accident under Platform Bridge

Design of a Cartoon Game for Traffic Safety Education of Children in China

591

Fig. 3. The leading character walks across Platform Bridge

In general, people only perceive finite objects in a given scenario by attention mechanism [4], and the scenario should include objects that are relative to the safety theme. According to Gibson’s theory of affordances [4], a clear design of sign (color or shape) on objects is helpful for understanding affordances that indicate the possible use of given objects. The cartoon game should teach children to distinguish signs in traffic environment. For example, a game user can easily distinguish Platform Bridge from other objects by different color and shape.

3

Design of Emotion and Interaction of Characters

Emotion of characters is very important to computer games [4]. Emotion interaction of characters is realized mainly by nonverbal manner. Nonverbal communication includes a variety of signals, body language, gesture, touch, physical distance, facial expression, and nonverbal vocalization. In this paper, the emotion of a cartoon character is expressed in feature picture of facial expression and body pose. Emotion state of cartoon character can be expressed by FSM (finite state machine). That is formalized with three tuple , I is input set that includes the information of scenario and possible selections of game users, E is the emotion set that includes possible emotion state for children. P is the emotion expression set that includes body pose and facial expression of cartoon character. There exist two functions: E × I→E, E × I→P. These two functions can be constructed by productive rules. The cartoon game has some NPC (non player character) that can guide a game user to play. A point constable NPC will tell the meaning of red traffic signal lamp with text and sound. When a user selects green traffic signal lamp, the leading character will walk across to the other side of street safely, and the point constable will tell encouragement information. A NPC builder helps the leading character to detour construction site (see Fig. 4). In order to realize the interaction between the leading character and a NPC, the lead-ing character has “social interaction radius” for different NPC, when the distance from the leading character to a NPC is smaller than “social interaction

592

Z. Liu

Fig. 4. A NPC builder hints possible falling things

radius”, the interaction is active, and a NPC character will say some words to the leading character.

4

Conclusion

The design of cartoon game for traffic safety education of children is presented in this paper. The designers should consider the psychological characteristic of children in order to arouse their interest. The traffic safety scenario of the cartoon game is designed by psychology theory, the shape of leading character and objects are colorful and interesting for Chinese children. The leading character has the ability of emotion expression and interaction with some NPC that can tell safety regulations by text and sound. The demo system is developed by FLASH cartoon tool; the result will offer a new means for safety education of children in china.

Acknowledgements The work described in this paper was co-supported by forepart professional research of ministry of science and technology of the People’s Republic of China (grant no: 2005cca04400), University Research Project of Zhejiang Province Education Department (grant no: 20051731)

References 1. Bernstein, D.A., Stewart, A.C., Roy, E.J., Wickens, C.D.: Psychology (forth edition), New York: Houghton Miffin Company (1997) 360-361 2. Leslie, P.S., Steff, G., Jerry, E.G.: Constructivism in education, Lawrence Erlbaum Associates, Inc, Hillsdale(1995) 3. Rhodes, G.: Macromedia Flash MX 2004 Game Development, CHARLES RIVER MEDIA Released (2004) 4. Liu, Z., Pan, Z.G.: An Emotion Model of 3D Virtual Characters In Intelligent Virtual Environment, In First International conference on affective computing and intelligent interaction, ACII2005, LNCS 3784, Beijing ,China, (2005),629-636

Cinematographic Techniques for Edutainment Applications Felicitas Becker ZGDV Darmstadt e.V., Digital Storytelling Department, 64283 Darmstadt, Germany [email protected] http://www.zgdv.de/distel/

Abstract. In film, cinematographic techniques are used in a concurrent sense, predominantly to achieve creative goals. The use of cinematographic techniques in edutainment applications would especially affect the modeling of virtual characters as well as the authoring and perception of interactive stories.

1 Introduction Some cinematographic techniques, such as camera perspectives and editing, have already been quite successfully adapted for computer graphics and various applications. However, research and development has focused mainly on implement- tation issues, while psychological effects and general consequences have been widely neglected. This paper introduces the possibilities of using cinematic techniques as a generic setting to enhance emotional involvement in edutainment applications. After listing related work, the creative use of cinematographic elements in film is described. Thenceforth, an equally creative use of cinematography in edutainment applications will be discussed in the scope of modeling virtual characters and creating digital, interactive stories.

2 Related Work He et al [1] codified the rules of cinematography to allow an automatic camera placement and editing. De Loor et al. [2] describe a virtual camera, which can be partly controlled by the user’s interactions. Cozic et al. [3] propose predefined viewpoints which are generated from a set of visual constraints. Potmesil et al. [4] were the first to generate depth of field blur with a varying kernel size dependant on the point depth on the pixel. Cook et al. [5] describe the method of distributed raytracing to simulate depth of field. Tomlinson et al. [6] are the predominant scholars who consider emotions as part of their work in this field. They describe an interactive system in which the character’s mood is conveyed by the camera.

3 Cinematography in Film First, the camera has to be set at a certain perspective. In a standard shot the camera is positioned at the characters’ eye-level. Angles which differ from this natural perspective Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, pp. 593 – 596, 2006. © Springer-Verlag Berlin Heidelberg 2006

594

F. Becker

can be used to make certain statements or to guide the viewer emotionally: High angles can be used to position characters as being inferior. With low angles, characters are perceived as being dominant, powerful, or menacing. The camera perspective can also convey a certain point of view (POV), guiding the viewer’s empathy and understanding of a character’s position. The camera’s distance to a subject, i.e. field size, directly affects the viewer’s emotional distance to the character. The viewer’s attention can be focused on a certain detail by showing it in a close-up. In turn, a long shot allows the viewer to take in the surroundings and the context of a scene. Editing describes the process of cutting and splicing shots, creating the film’s final form [7]. While editing is a purely technical term, montage describes a creative idea which leads to a particular form of editing. Since Pudowkin, Eisenstein and Wortow, montage is understood as the creation of a meaning, which cannot be conveyed by the detached shots themselves [8; 9]. In this sense, adjacent shots A and B are combined to produce the new meaning C [7]. A montage always conveys an implicit author, who affects the flow of information and leads the viewer to see things in a certain way. The strong effect of montage has been demonstrated most vividly by the filmmaker Lev Kuleshov [8]. Beyond emotional appeal and the creation of new meaning, montages are used for: parallel action (by cross-cutting between two or more scenes) flashbacks (memories and stories are inserted in the main narrative) flash-forwards (predictions and visions) and match cuts (linking two shots or scenes by visual, aural or metaphorical parallelism). Editing, in terms of technical feasibility, is already discussed in the field of computer graphics. Montage however – i.e. the creative theory and realization of editing since the Soviet Revolution – has been mostly neglected. Focus is considered as a very important variable in the syntax of a film shot [7]. With a shallow focus, characters standing in the foreground are in focus while the background is completely out of focus. Shallow focus suggests psychological introspection, and is therefore commonly employed in genres such as the melodrama, where the actions and thoughts of an individual prevail over everything else. Shallow focus also allows focusing the user’s attention on a certain object or character without having to cut to a close-up thereby cutting out any additional information involved in the scene.

4 Possibilities for Edutainment Applications One of the main aspects to be considered, are the effects cinematography has on the modeling of virtual characters. In this field of research, a discussion is going on, whether virtual character are to be modeled as virtual humans or rather as virtual actors [10]. Acting theories cannot be discussed in length here, but the following headwords should give an impression of how cinematography influences an actor’s work: - facial expressions/gestures are supported by perspectives/field size [7] - the acting space is smaller (e.g. close shot), low-key behavior is visible - editing can be associative, giving an actor’s performance meaning merely by the following or preceding shot [7;8;9] - the camera itself can become an agent [11], a narrator or even an actor

Cinematographic Techniques for Edutainment Applications

595

Thus, virtual character can be endued with a far more subtle set of behavioral skills when using cinematographic techniques. It can be assumed, that virtual characters will be perceived as being more emotional – and thus more emotionally appealing – in connection with cinematography. It is often suggested to use stories as backbones of edutainment applications [e.g. 12]. In connection with the narrative, cinematographic possibilities raise a completely new issue: integrating subplots in edutainment applications. In films the main plot is used to keep up the arc of suspense, while subplots are used to address emotionally evocative topics. In this sense, subplots are vital to make a film emotionally involving [13]. Narratives supporting edutainment applications could be enhanced with subplots in the same way. By using associative and parallel editing, different story lines can be intertwined and the main plot can be effectively combined with subplots, achieving the most emotionally intriguing overall narration possible.

5 Summary and Outlook The emotional expression of virtual characters could be supported and even assumed by cinematography in large parts. The meaning of certain coherences would no longer need to be narrated by a virtual character, they could be told by the camera or a particular kind of editing. The narration can become more varnished and fuller with the use of cinematography. Thus, cinematographic techniques in edutainment applications can increase emotional appeal as well as the overall entertainment value.

References [1] L.-W. He, M. F. Cohen, D. H. Salesin. The Virtual Cinematographer: A Paradigm for Automatic Real-Time Camera Control and Directing. Proceedings of SIGGRAPH ’96. Computer Graphics Proceedings, Annual Conference Series, pages 217-224, August 1996. [2] P. de Loor, P.-A. Favier, J. Tisseau. Programming Autonomous Entities with Purposes and Trends for Virtual Storytelling. International Conference of Virtual Storytelling. pages 40-43, September 2001. [3] L. Cozic, S. B. Davis, H. Jones. The Intruder: Expressive Cinematography in Videogames. Working Paper. Lansdown Centre for Electronic Arts, Middlesex University, 2003. [4] M. Potmesil and I. Chakravarty. A lens aperture camera model for synthetic image generation. In: Computer Proceedings SIGGRAPH 81 Computer Graphics, ACM Press, pages 297-305, 1981. [5] R. L.Cook, T. Porter, and L. Carpenter. Distributed Ray Tracing. In: Conference Proceedings SIGGRAPH 84 Computer Graphics, pages 137-144, 1984. [6] B. Tomlinsons, B. Blumberg, D. Nain. Expressive autonomous cinematography for interactive virtual environments. Proceedings of the 4th International Conference on Autonomous Agents (AGENTS00), pages 317–324, NY. ACM Press. 2000 [7] J. Monaco. How to Read a Film. Oxford University Press. London/New York 2000. [8] S. Eisenstein and . Leda (ed.). Film Form. Harcourt Brace, Jovanovich, 1949 [9] D. Harrah. Aesthetics of the Film: The Pudovkin-Arnheim-Eisenstein Theory. In: The Journal of Aesthetics and Art Criticism. No. 2, December, pages 163-174. 1954

596

F. Becker

[10] A. S. Gordon and M. van Lent. Virtual Humans as Participants vs. Virtual Humans as Actors. 2002 AAAI Spring Symposium of Artificial Intelligence and Interactive Entertainment. Stanford University 2002 [11] J. Funge, X. Tu, and D. Terzopoulos. Cognitive modeling: Knowledge, reasoning and planning for intelligent characters. Proceedings of SIGGRAPH 99, Computer Graphics Proceedings, pages 29–38, Los Angeles, 1999 [12] A. Hoffmann, S. Göbel, O. Schneider. (2005). Storytelling Based Education Applications. Accepted book chapter to E-Learning and Virtual Science Centers, Idea Group Publishing, Hershey, US, 2005 [13] R. McKee. Story: Substance, Structure, Style and The Principles of Screenwriting. Regan Books New York, 1997

Efficient Large-Scale Terrain Rendering Method for Real-World Game Simulation Dong-Soo Kang, Yun-Jin Kim, and Byeong-Seok Shin Inha University, Department of Computer Science and Information Engineering, 253 Yonghyeon-Dong, Nam-Gu, Inchon 402-751, Rep. of Korea [email protected] [email protected] [email protected] Abstract. Terrain modeling and rendering is essential in construction of realistic virtual environment, for interactive computer games application. Advanced high resolution scanning technologies such as LIDAR enable us to obtain dense and accurate irregular terrain dataset. However, unlike the regular sampled dataset such as DEM and DTED, irregular dataset cannot be rendered in real-time due to random distribution of samples. Moreover, because they generally have huge amount of point data, it is very hard to manipulate them in consumer PC. We propose a fast and efficient terrain rendering method using large-scale irregular dataset for computer games application. First, it reconstructs the geometry by converting irregular dataset into regular one, through resampling input data with regular interval considering the spatial distribution of sample points. It can generate animated scene by applying quad-tree based rendering method to converted dataset. Experimental results show that its image quality is not deteriorated in comparison to the result of rendering reconstructed geometric models.

1

Introduction

There are two typical data sources in terrain modeling and rendering. The one is regular sample data such as DEM (Digital Elevation Map) and DTED (Digital Terrain Elevation Data), and the other is irregular sample data scanned by LIDAR (LIght Detection And Ranging). We can achieve real-time rendering of DEM data by using the spatial data structures such as quad-tree [1], [2], [3], [4] and ROAM (Real-time Optimally Adaptive Meshes) [5] since they support efficient view-frustum culling and level-of-detail selection. However, most regular sample data inherently have low resolution (more than 10m per sample). Therefore, it is very hard to represent surface detail of target terrain. In these days, LIDAR scanner mounted on airplane or helicopter can produce high-definition height field data (less than 1m per sample). There are several methods to generate original geometry such as Delaunay triangulation [6], [7], [8], [9]. Even though we can show the surface detail, it takes a long time to reconstruct geometric information from the irregular samples. It is hard to apply the dataset to real-time application even in the case of exploiting highly optimized methods. We propose an efficient terrain rendering method by converting the Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, pp. 597–605, 2006. c Springer-Verlag Berlin Heidelberg 2006


598

D.-S. Kang, Y.-J. Kim, and B.-S. Shin

huge amount of irregular dataset to a set of regular grids. First, we eliminate noises from input dataset since they do not contribute to the final results. We perform interpolation and noise removal of resampled data to generate a set of uniform grids containing height information. Finally, we render a terrain using quad-tree based continuous level-of-detail (CLOD) method. In Sect. 4, we explain our method in detail. In Sect. 3, we show some experimental results concerning preprocessing cost, quality of images and rendering speed. Lastly, we conclude our work.

2

Efficient Rendering Method for Large-Scale Terrain Data

For the real-time rendering in large-scale terrain data, we exploited the quadtree method. However, since quad-tree method can only handle uniform grids, it needs to convert irregular dataset into regular one. The geometry reconstruction process and rendering step are represented in Fig. 1.

Fig. 1. Overall process of our method to render the irregular sampling data using quad-tree based continuous level-of-detail method

2.1

Noise Removal

The dataset from 3D scanner may include noises because of the instrumental inaccuracy. In general, noises have considerably larger or smaller data than average value of the sample data, and they lead into unexpected artifacts. We eliminate the noise using pseudo-grid [10] which divides 3D space into rectilinear grids along with three principle axes (x-, y- and z-axis). It checks the number of points in each pseudo-grid. If it is less than the threshold value ε, it removes them from sample data. Although it selects incorrect data (that is meaningful samples), resulting image is not significantly influenced by them since each grid contains sufficient samples.

Efficient Large-Scale Terrain Rendering Method

599

Fig. 2. An example of a pseudo-grid in z-axis for an irregular dataset. If the threshold value ε is 10, gray points are recognized as noise and removed. f means the number of samples per z-axis.

2.2

Resampling

Next step is to convert irregular samples into uniform grid. We assume that the regular grids are formed along with x- and y-axes of which the size is n × n and each grid cell contains height values of its samples. Then, we have to define the size of grid and range of meaningful sample data using the spatial distribution of sample data in each cell. When we set the size of gird as large value, a cell has a lot of sample data and surface detail cannot be maintained due to exceeded averaging. On the contrary, when the grid size is too small, it generates too much grid and takes a lot of time for resampling. In addition, it might produce a cell that does not contain sample data, which cause holes on rendered image. Ideal case of grid subdivision is that a cell has only one sample. Followings are resampling steps. 1. It computes total number of sample data Pt . √ 2. Let a size of grid be N × N , it sets the value of N as | Pt |. 3. It sets horizontal range of height map Rx as max{xi }-min{xi } where xi is x-coordinates of Pt samples. 4. It sets vertical range of the grid Ry as max{yi }-min{yi } where yi is ycoordinates of Pt samples. 5. Intervals on x-direction and y-direction are Rx /N and Ry /N respectively. After defining the grid size, we calculate the average value of samples in each grid. It represents height information of each cell. 2.3

Interpolation

Due to randomness of sample position in irregular dataset, some grids contain more than two samples and some grids have no sample. If we do not consider those grids, holes appear on the final image. In order to remove the holes, we use the bilinear interpolation method using the distance to nearest neighboring samples. Interpolation is performed only on samples within an area of threshold

600

D.-S. Kang, Y.-J. Kim, and B.-S. Shin

Ti (Ti = max(N ×0.05, 2)) and the remaining grids are set as the minimum height value since the height value of samples far-away for target grid does not affect to it. All of the grid cell fill-up with appropriate height values, we can apply real-time terrain rendering method. 2.4

Quad-Tree Based CLOD Method

Quad-tree based algorithm recursively subdivides two-dimensional space into four quadrants. An algorithm specifically designed for height fields was presented by Lindstrom [1]. It uses a dynamically changing quad-tree and a bottom up strategy to determine whether a node has to be subdivided or should be merged with adjacent nodes. For that purpose it calculates an upper bound on the projected pixel error. Rottger et.al [2] proposed an algorithm that uses a top-down strategy to create a triangulation and exploits geo-morphing. Vertex removal is performed depending on its distance to the view point as well as local surface roughness, which is pre-calculated. Using a top-down approach we only need to visit a fraction of the whole data set in each frame, which allows for high frame rates even with large height fields. In a terrain data represented by a quad-tree, the number of triangles downloaded into graphics pipeline is determined by global image resolution, regardless of the size of height field. So it can control the image quality so as to accommodate hardware capacity. Rendering process in quad-tree based CLOD technique is summarized as follows: (1) calculates error metric recursively for all leaf nodes while traversing

Fig. 3. Global resolution criterion: distance(l) versus size of quad-tree nodes(d)

Fig. 4. Measuring surface roughness by estimating the elevation difference

Efficient Large-Scale Terrain Rendering Method

601

height field data. (2) propagates computed error metric of leaf nodes to their parent nodes. (3) determines whether current node can have child nodes or not by applying accuracy level determination function to each node using a set of error metric calculated before. (4) generates triangle mesh while recursively traversing the quad-tree made from height field. (5) renders the triangle mesh. Determining whether subdivision is required in leaf nodes, we have to consider the distance from viewer to a specific point on a terrain. As shown in Fig. 3, node size (denoted as d) should be enlarged as the distance (denoted as l) increases. We can control the size of node with regard to the distance to satisfy l/d < C. Here, C is the minimal desired resolution of image. The number of triangles is determined by the value of C. When we set a large value, more number of triangles will be produced from quad-tree since it should be subdivided to smaller nodes. With the second criterion we want to increase the resolution for regions of high surface roughness. When dropping one level of the hierarchy, new error is introduced at exactly five points: at the center of the quad-tree node and the four midpoints of its edges. An upper bound to the approximation error can be given by taking the maximum of the absolute values of the elevation difference dhi computed along four edges and two diagonals of the nodes (Fig. 4). The error can now be computed by pre-calculating the maximum of the absolute values of these elevation differences, which is called d2: d2 =

1 max(i=1..6) |dhi |. d

(1)

A revised version of the subdivision criterion (1) which includes the d2-values for handling surface roughness can now be given in terms of a decision variable f : f=

l subdivide if f < 1. d · C · max(c · d2, 1)

(2)

The newly introduced constant c specifies the desired global resolution. It directly influences the number of polygons to be rendered per frame. Table 1. Comparison of geometry reconstruction time according to the number of points total number of point(Pt ) 12,961 17,846 25,179 52,119 127,048 378,064 430,674 845,425 1,626,972 3,408,187

delaunay triangulation(A) (sec) 4.546 14.391 26.922 158.156 923.329 7,286.344 9,579.781 33,513.563 n/a n/a

our method(B) (sec) 0.156 0.313 0.453 0.953 3.109 3.562 3.625 12.453 18.765 39.516

efficiency (A/B) 29 46 59 166 297 2045 2642 2691 n/a n/a

602

3

D.-S. Kang, Y.-J. Kim, and B.-S. Shin

Experimental Results

Our method was implemented on a PC equipped with Pentium IV 3.0GHz CPU and 1GB main memory and NVIDIA Geforce FX5600 graphic hardware. Table 1 shows the geometry reconstruction time for different input datasets, and Fig. 5 shows the rendering results. To compare the quality of rendered images,

Fig. 5. Resulting images of terrain visualization using 34,379 points LIDAR data. (a) shows direct plotting of original samples. Since this method does not have its geometry, it cannot produce shaded images. (b) and (c) are the images rendered to Delaunay triangulation method and (d) and (e) are the images rendered to our quad-tree based CLOD method. (b), (d) images are rendered to wire frame, and (c), (e) images are shaded results. In our experiment, the rendering speed of (a), (d) and (e) achieved in real time (about 45fps), while the rendering speed of Delaunay triangulation method was less than 10fps

Efficient Large-Scale Terrain Rendering Method

Fig. 6. Resulting images of terrain simulation using our method

603

604

D.-S. Kang, Y.-J. Kim, and B.-S. Shin

we also implemented the direct plotting of original samples and the Delaunay triangulation method [6]. Fig. 5(a) shows direct plotting of original samples. Although this method renders the scene fast, the final image is unrealistic since point data does not have geometric information. Fig. 5(b) and Fig. 5(c) depicts the images using the Delaunay triangulation method as a wireframe (b) and a shaded image (c). The method shows realistic results compared with direct plotting method. However, it requires huge amount of computation time as the data sizes is increased. For example, when the number of point is 845,425 it takes over 9 hours to make the geometric model in consumer PC. Fig. 5(d) and Fig. 5(e) illustrates the rendered images using our method. In general, because the quad-tree based CLOD method can render terrains fast and efficiently, our method also can be rendered with the identical frame rates. Although it cannot present fine details on terrain surface in comparison to Delaunay triangulation method, overall shape and important features are preserved. Since our method requires less than 1 minute even when the point number is 3,408,187 our method renders a terrain scene with a little preprocessing time (within 1 minute) compared with the Delaunay triangulation method with moderate image quality. We can apply our method to represent 3D map of computer games see Fig. 6, because realistic 3D simulation require high-resolution terrain models. Average rendering speed is 45fps.

4

Conclusion

Recently, terrain representation is important factor in the filed of games. And the most important issue is to produce high quality images in real time. To render huge amount of large-scale irregular terrain datasets such as a LIDAR data, we propose the geometry reconstruction procedure from irregular datasets to uniform grid datasets. Based on the converted representation datasets, we can render the terrain scene fast and efficiently by applying quad-tree based continuous level-of-detail method. Experimental results show that our method normally produces high-quality images, and requires less preprocessing time compared with the Delaunay triangulation method. It can be used to construct and render the virtual environment in several applications.

Acknowledgements This research was supported by the MIC (Ministry of Information and Communication), Korea, under the ITRC (Information Technology Research Center) support program supervised by the IITA (Institute of Information Technology Assessment).

References 1. Lindstrom, P., Koller, D., Ribarsky, W., Hodges, L., Faust, M., Turner, G.: RealTime Continuous Level of Detail Rendering of Height Fields. Proceedings of ACM SIGGRAPH 96 (1996) 109–118

Efficient Large-Scale Terrain Rendering Method

605

2. Rottger, S., Heidrich, W., Slasallek, P., Seidel, H.: Real-time Generation of Continuous Level of Detail for Height Fields, Proc. of 6th International Conference in Central Europe on Computer Graphics and Visualization (1998) 315–322 3. Ulrich, T.: Continuous LOD Terrain Meshing Using Adaptive Quadtrees, Gamasutra, 2000, http://www.gamasutra.com/features/200000228/ulrich 01.htm 4. Shin, B.S., Choi, E.K.: An Efficient CLOD Method for Large-Scale Terrain Visualization, Lecture Notes in Computer Science, Vol. 3166 (2004) 592–597 5. Duchaineau, M., Wolinsky, M., Segeti, D., Miller, M., Aldrich, C., MineevWeisstein, M.: ROAMing Terrain: Real-time Optimally Adaptive Meshes, Proceedings of ACM SIGGRAPH 97 (1997) 81–88 6. Cohen-Steiner, D., Colin de Verdiere, E., Yvinec, M.: Conforming Delaunay triangulations in 3D, Proceedings of the 18th Annual Symposium on Computational Geometry (2002) 199–208 7. Aggarwal, A., Guibas, L., Saxe, J., Shor, P.: A Linear-time Algorithm for Computing the Voronoi Diagram of a Convex Polygon, Proceedings of the 19th Annual ACM Conference on Theory of Computing (1987) 39–45 8. Attali, D., Boissonnat, J.-D. and Lieuter, A.: Complexity of the Delaunay Triangulation of points on surfaces the smooth case, Proceedings of the 19th Annual Symposium on Computational Geometry (2003) 201–210 9. Akenine-Moller, T., Haines E.: Real-Time Rendering, AKPETERS Ltd., Second Edition (2002) 10. Cho, W.S., Jwa, Y.S., Chang, H.J., Lee S.H.: Pseudo-Grid Based Building Extraction Using Airborne LIDAR Data, Proceedings of ISPRS 2004, Vol. XXXV (2004) 298–301

Federate Migration in Grid-Based Virtual Wargame Collaborative Environment Hai Huang, Wei Wu, Xin Tang, and Zhong Zhou School of Computer Science and Engineering, Beihang University, Beijing 100083, P.R. China [email protected] Abstract. Recently, grid-based virtual wargame collaborative environment has become a hot topic, in which load balance plays an important role. In this paper, we survey existing technologies in federate migration for the purpose of balancing load, and focus on the selection of destination node for a federate to migrate to. Different from traditional methods, the proposed method takes into consideration not only the load of CPU & memory, but also communications among federates when selecting the destination node. Moreover, the procedure of migration is described and a prototype is implemented on the basis of Globus Toolkit 3.0 to validate its functionality and performance.

1

Introduction

Simulation has already been applied in many fields and is becoming the third important approach to know and change the objective world subsequent to theoretical and experimental research. The virtual wargame collaborative environment, in which users interact and share battlefield information, is the concrete application of simulation in military affairs. After going through three stages of SIMNET, DIS, and ALSP, it has evolved into High Level Architecture (HLA), which was adopted as the IEEE standard 1516 in September 2000. In HLA, the simulation applications and underlying supporting software organized to reach specific simulation goal are called federation. Simulation applications in a federation are called federates and they communicate with each other via Run Time Infrastructure (RTI). Based on HLA, large-scale virtual wargame collaborative environment can be developed over the Internet. With wide adoption of HLA standard, some limitations emerge in such aspects as scalability, convenience, robustness, and component reusability. Over the past several years, the concept of Grid, proposed by Ian Foster as secure and coordinate resource sharing and problem solving in dynamic, multi-institutional virtual organizations, has become a hot topic. Some research institutions attempt to adopt grid technology in HLA simulation to solve some problems of current HLA simulation [1, 2, 3, 4]. In previous work, we have established the Aegis simulation grid prototype [5]. The ultimate goal of Aegis is to establish a Simulation On Demand (SOD) environment, allowing users to customize or set up a federation to carry on collaborative simulation through a browser/client at any time and place. Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, pp. 606–615, 2006. c Springer-Verlag Berlin Heidelberg 2006 

Federate Migration in Grid-Based Virtual Wargame

607

A lot of computing resources are involved in large-scale grid-based collaborative simulation in SOD environment and a simulation run may last a long time. The load of nodes can vary significantly due to the uncertainty and unpredictability of federates, and the availability of node resources is not guaranteed because of human factor or malfunction. Therefore, it is necessary to balance the load among distributed nodes so as to improve utilization of resources and make simulation advance as normal when a certain node is overloaded or unavailable. The most widely used methods of balancing load are selecting an appropriate destination node for a newly-joined federate to run on, or migrating a federate at overloaded node to a lightly-loaded node. Since the federate migration also involves the selection of destination node, this paper focuses on balancing load based on federate migration. The rest of the paper is organized as follows. In Sect. 2, related work of federate migration is explained. Sect. 3 describes the migration process of a federate in SOD environment. In Sect. 4, a prototype is implemented in order to validate the functionality of federate migration, and finally main conclusions and future work are presented in Sect. 5.

2

Related Work

The most fundamental method of migrating a federate is to utilize standard interface FederationSave and FederationRestore of HLA. That is, FederationSave is used to save states of all federates in a federation and states of RTI before migration begins. FederationRestore is used to restore the federation execution after the migrated federate resigns from the federation at source node and then rejoins in the federation at destination node. Such method is simple and easy since the standard HLA interface is used. However, migrating a federate causes suspension of all federates in the federation, which incurs more cost. In 2001, J. Luthi, etc. realized Resource Sharing System (RSS) [6] in the Network of Workstations (NOW), in which load balance was achieved by migrating federates among different workstations. In RSS, migration process is controlled by a central Manager. In order to reduce the adaptation effort for the existing HLA federates, a CommFed federate is introduced as a communication agent between HLA federates and Manager. Owners of workstations can control the workstation to join or resign from RSS through a local client. A third party FTP server is used to transfer the state data of migrated federate from source node to destination node. Although RSS realize federate migration at NOW, it is valuable for federate migration in grid-based collaborative simulation. One of the defects of RSS is the introduction of FTP server for transferring state data, which is time-consuming since two ftp connections should be established. K. Zajac, etc. utilized FederationSave and FederationRestore to save the internal data of the whole HLA, and develops a Migration Library (ML) to support the capturing of user’s data [7]. User’s data is transmitted through GridFTP service provided by Globus. In the broad sense, Peer-to-Peer Computing (P2P) is one form of grid computing. M. Eklof, etc. implemented federate migration in P2P environment based

608

H. Huang et al.

on JXTA platform [8]. An HLA Manager is employed to control federate migration in the whole federation. The states of federates are saved into an ASCII file. Since HLA Manager only supports conservative time management strategy, such method requires all federates to be both time-constrained and time-regulating, which results in limited range of application. W. Cai, etc. realized federate migration in grid based on LMS [9]. Every federate is integrated with a LMSHandler, which is responsible for pausing and saving of the federate. The federate states are saved into an intermediate file, which is uploaded to a third party FTP server via FTP service provided by Globus. The destination node downloads the state file from FTP server and restores federate execution. Because the third party FTP server is used to transfer the state file, larger time expenses are inevitable. In order to avoid saving the states of the whole federation and reduce the transfer delay of state file caused by the third party FTP server, W. Cai, etc. proposed a kind of federate migration protocol based on LMS [10]. A message counter is integrated at each federate. Before resigning form federation, the migrated federate compares its counter to counters of federates that have communication with it to judge if any message would be lost. At the same time, the protocol overlaps the simulation execution of the federate to be migrated and the join federation operation performed by the restarting federate at the destination node to shorten the migration duration. The state information is transferred in a P2P way between the source node and destination node. It is transparent for non-migrated federate during migration, without federation wide synchronization or a third party host, such as FTP servers. However, the proposed protocol needs relatively much adaptation effort for existing HLA federates.

3

Federate Migration in SOD

To migrate a federate in SOD, several factors should be considered. For example, which federate should be migrated? Which node the migrated federate should migrate to? How to ensure no message loss during migration? In this section, the procedure of federate migration is presented, and the method of seeking the optimal destination node is investigated. 3.1

Procedure of Federate Migration

The simulation federation with federate migration functionality in SOD environment is shown as Fig. 1. All nodes in the federation bear the same status, that is, every node can be used for storing simulation data resources and running federates, with federates communicating with each other through RTI. In order to provide a uniform view to data resources and computing resources, an Index Service is run at a well-known node. The address and access method of various simulation resources exposed as grid service can be obtained through inquiring Index Service. At the same time, Index Service, supported by Integrated Management Module (IMM) running at every node, is able to reflect the load

Federate Migration in Grid-Based Virtual Wargame

609

status of every node dynamically. Scheduler running at the well-known node, supported by Index Service, is responsible for selecting the most suitable destination node for a federate to migrate to. IMM comprises of six independent subfunction modules: (1) Monitor: reports the load status of local node to Index Service at regular intervals; (2) ResourceLoad: downloads simulation resources necessary for running federate from other nodes; (3) startApp: starts a simulation federate; (4) StateLoad: transfers the saved federate state from source node to destination node; (5) Publisher: provides graphical user interface for simulation developers to upload and publish their simulation resources as grid service; (6) Client: allows local node to join or resign from SOD environment. In addition, there is a special federate called Controller in the federation, which controls the migrated federate at source node to resign federation while communicating with common federates through RTI.

Node Scheduler Index Service

Node IMM

Node IMM

Federate

Controller

...

Node IMM

Node IMM Federate

RTI

Fig. 1. Federation with Federate Migration Functionality in SOD Environment

In SOD environment, user accesses Index Service running at the well-known node through a client (such as the browser) to obtain such information as available nodes and simulation federates, distributes the federates to appropriate nodes to form a federation, and at the same time the IMM at local node monitors load status and reports to Index Service at regular intervals. During federation execution, when a migration is needed a lot of preparations need to be done before the migrated federate save its states into a TXT file (see Fig. 2): (1) When the load of a certain node exceeds its threshold or the owner of the node requires the node to resign from SOD through Client, IMM selects a federate at this node as the migrated federate and requests Scheduler to start federate migration process (step a); (2) Scheduler accesses Index Service to obtain load information of every node in SOD after receiving federate migration request and then selects the optimal destination node for the migrated federate to migrate to (step b, c); (3) Scheduler notifies the destination node to download migrated federate (if necessary), so the destination node needs to access Index Service to obtain the address of migrated federate code (step d, e, f, g);

610

H. Huang et al. Monitor/Client

Scheduler

Index Service

ResourceLoad

startApp

(a) (b) (c)

(d) (e)

(f) (g)

(h)

(i)

Fig. 2. Preparation of Federate Migration Monitor/ Client

Scheduler

Federate @Source

Controller (j)

(k)

Federate @Destination

StateLoad (l)

(m)

(n)

Fig. 3. Procedure of Federate Migration

(4) After the resources necessary for running migrated federate are downloaded, IMM starts migrated federate at the destination node and notifies Scheduler that migrated federate has restarted (step h, i). Now the preparation for migration has been done, and the federate migration can begin (see Fig. 3): (1) When Scheduler knows that the migrated federate has already started at the destination node, it notifies Controller to let the migrated federate at the source node save state information and resign from the federation (step j, k); (2) Connection is established between IMMs at the source and destination nodes, and the state information of migrated federate is transferred (step l, m); (3) The migrated federate at the destination node restores its states from the previously saved state information and continues running. By for now, the federate migration is finished (step n). 3.2

Destination Node Selection

To effectively balance the system load, we should chose the most suitable target node for a federate to migrate to. Most systems only measure the CPU load, memory usage, and some also gather hardware system information about the nodes, like clock frequency, processor type, amount of memory etc., and then takes a decision from that. However, in SOD, the nodes are geographically distributed, and the communication delay among each other bears great impact on the duration of a federation run; on the other hand, network bandwidth is an important resource in wide area network, therefore we should minimize network resource cost when performing a federation run. Consequently, in this paper the CPU load, memory storage, and communications among federates will be considered when selecting the destination node.

Federate Migration in Grid-Based Virtual Wargame

N

611

i

Router

Node

Source Node

Fig. 4. Procedure of Federate Migration

Suppose there are m nodes in SOD environment, the CPU speed and memory capacity of node Ni are Ci and Mi respectively, and the corresponding CPU and memory usage during simulation execution are ci and mi respectively, where i ∈ N , 1 ≤ i ≤ m. Also, suppose the federation comprises of n federates, and let sq,p be the number of object attributes of federate fq that federate fp subscribes, hp,q be the number of hops of the shortest path routing between nodes on which federate fp and federate fq run respectively, where p, q ∈ N , 1 ≤ p, q ≤ n. Without loss of generality, suppose federate fu running on node Ni will be migrated (see Fig. 4), then the destination node must be Nj , where i, j, u ∈ N , j j and Mthreshold are the thresholds 1 ≤ i, j ≤ m, i = j, 1 ≤ u ≤ n. Let Cthreshold of CPU and memory of Nj . Based on above definition, the maximum available CPU and memory capacity on Nj can be expressed as: j − cj /Cj )Cj , (Cthreshold

(1)

j (Mthreshold − mj /Mj )Mj .

(2)

The network resource cost when fu runs on Nj can be approximately computed, as follows: n  v=1,u=v

hv,u sv,u +

n 

hu,v su,v =

v=1,v=u

n 

hu,v (sv,u + su,v ) .

(3)

v=1,u=v

Based on (1), (2) and (3), the preference value for fu selecting Nj as the destination node is defined as: j j pref erence(fu , Nj ) = w1 (Cthreshold − cj /Cj )Cj + w2 (Mthreshold − mj /Mj )Mj n  +w3 /( hu,v (sv,u + su,v ) + ε) , v=1,u=v

where wk (k = 1, 2, 3) is the weight that CPU, memory and communication bears in the preference value, and ε is an infinitesimal positive number which is introduced to avoid dividing by zero.

612

H. Huang et al.

Now, the destination node for fu to migrate to can be derived from: n

decision(fu , SOD) = {Nj | Nj = max pref erence(fu , Nk )} , k=1,k=i

(4)

which means the destination node should be the one that bears maximum available CPU and memory capacity, and at the same time costs minimum network resource after migration.

4

A Prototype Implementation and Test Result Analysis

In order to validate the proposed load balance system for SOD, we have implemented a prototype based on BH RTI 2.1 and Globus Toolkit 3.0 (GT3). Besides the functionality test, performance test is also carried out and its migration time is compared with that of system deploying traditional third party FTP server. 4.1

Prototype Implementation and Functionality Test

In the prototype, the SOD environment comprises of three PCs, named Apple, Tangxin and Grape respectively. Scheduler and Index Service run on Grape, and the federate (helloWorld) code resides on Grape, too. Although Scheduler and Index Service can run on different machines separately, in the implementation they are placed on the same computer in order to shorten the communication time between them. Also, federates’ code can exist on more than one computer in SOD environment. However, only one kind of federate named helloWorld is used in the prototype, and its code is simply uploaded to grape. Different from the DMSO helloWorld, the helloWorld we used is adapted to including methods for capturing its current states to a TXT file before migrating and for restoring from the previously saved state file. At the beginning of a federation run, Apple downloads helloWorld code from Grape via GridFTP of GT3, and then two helloWorld federates are started to form a federation in which federates communicate with each other via BH RTI 2.1. Through a graphical user interface, the information of all existing simulation nodes can be reviewed, including name, IP addresses and load status, where the load is a normalized value of CPU and memory usage. The federate migration is based on a simple scenario - user sends out migration request by specifying the source node Apple using the graphical user interface. After the source node is input, a helloWorld federate at Apple is randomly selected as the migrated federate. Based on (4) described in Subsect. 3.2, Scheduler chooses Tangxin as the destination node between the two candidates: Grape and Tangxin. During federate migration, both the downloading of helloWorld code from Grape by Tangxin and the transferring of saved helloWorld state file from Apple to Tangxin employ the GridFTP service provided by GT3. At the bottom of graphical user interface, the status of migration process is displayed dynamically (see Fig. 5). As can been seen from Fig. 5, the Event Loop Iteration value of helloWorld before migration is 10, and helloWorld restores successfully with Event Loop

Federate Migration in Grid-Based Virtual Wargame

613

Fig. 5. GUI for Federate Migration

Iteration value of 11 at the destination node after migration. Also, Fig. 5 shows that the load on source node and destination node has some decrease (from 2.1999998 to 1.03) and increase (from 1.18 to 2.87), respectively. Above results show that federate on heavy load source node can be migrated to light load destination node and then restore its execution, which significantly contributes to load balance in SOD environment. 4.2

Performance Test

In this test case, federate migration time is considered as the primary factor of performance. In order to validate the outperformance of our prototype, we adapt it to transferring state file through Grape which acts as the third party FTP server in traditional methods. The federate migration time of our prototype and the adapted one is recorded respectively and then compared. As stated in Subsect. 3.1, before the migrated federate saves its states and resigns from the federation at source node, a “copy” of migrated federate has already been started as well as the publishing and subscribing operations have already been performed at the destination node. Therefore, the time cost on downloading federate code from other node and joining the federation will not affect the total execution time of a federation run. So we have only recorded the time from helloWorld starting to save its states at source node to restoring from previously saved state file at the destination node. While migrating, HelloWorld federate is required to save two states named “Population” and “CountryName” respectively. In order to test the migration time in such a case that a federate has more than two states, a lot of redundant states are poured into helloWorld.

614

H. Huang et al.

Fig. 6. Migration Time of helloWorld

The thick line, with its vertical coordinate axis on the right of Fig. 6, illustrates the migration time of helloWorld with different number of federate states in our prototype; while the thin line, with its vertical coordinate axis on the left, illustrates the time when the traditional third party FTP server is deployed. It can be seen that the migration time using a third party FTP sever is about two times longer than that using GridFTP in our prototype and the difference between them increases with the number of federate states. Also, it is obvious that the number of federate states has little impact on the migration time in our method, which can be explained as follows: the establishment of GridFTP connection contributes to most of the migration time.

5

Conclusions and Future Work

The grid-based virtual wargame collaborative simulation has already attracted great concerns in international modeling and simulation community. In this paper, the load balance is studied and a kind of federate migration method, which does not require the federation wide synchronization or a third party host, is proposed. Different from other methods, the proposed method takes into consideration not only the load of CPU & memory, but also communications among federates when selecting the destination node for a federate to migrate to. As a result, the destination node is the one that has max available CPU and memory capacity, and at the same time minimizes the network cost. Experimental results show that it is more efficient than traditional methods employing third party FTP server. However, the implementation is just a preliminary step and much work leaves to be done. For example, as for fine-grained virtual wargame simulation, it is crucial to ensure the migrated federate exactly receives all its subscribed messages. So, the received messages must be handled (if any) when restoring at the destination node.

Federate Migration in Grid-Based Virtual Wargame

615

Acknowledgements We would like to thank all the members of HLA and RTI project teams in VRLab for their helpful discussions. This paper is supported by the National Grand Fundamental Research 973 Program of China under the grant No. 2002CB312105.

References 1. K. Zajac, A. Tirado-Ramos, Z. Zhao, P. Sloot and M. Bubak: Grid Services for HLA-based Distributed Simulation Frameworks. Grid Computing: First European Across Grids Conference, LNCS Vol. 2970 (2004) 147–154 2. Y. Xie, Y.M. Teo, W. Cai, and S.J. Turner: Extending HLAs Interoperability and Reusability to the Grid. 19th ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS 2005), USA, June 2005. 3. J.M. Pullen, R. Brunton, D. Drake, etc.: Using Web Services to Integrate Heterogeneous Simulations in a Grid Environment. 2004 ICCS Workshop on HLA-based Distributed Simulation on the Grid, LNCS Vol. 3038 (2004) 835–847 4. R.P.Z. Brunton, K.L. Morse, D.L. Drake, B. Moller and M. Karlsson. Design Principles for a Grid-Based HLA Federation. Proceedings of the IEEE 2004 European Simulation Interoperability Workshop, Paper 04E-SIW-056, 2004 5. Wu, W., Zhou, Z., Wang, S.F., Zhao, Q.P.: Aegis: a Simulation Grid Oriented to Large-scale Distributed Simulation. The Third International Conference on Grid and Cooperative Computing, LNCS Vol. 1000 (2004) 413–422 6. J. Luthi and S. Grossmann: The Resource Sharing System: Dynamic Federate Mapping for HLA-based Distributed Simulation. Proceedings of 15th Workshop on Parallel and Distributed Simulation (2001) 91–98 7. K. Zajac, M. Bubak, M. Malawski, and P. Sloot: Towards a Grid Management System for HLACbased Interactive Simulations. The Seventh IEEE International Symposium on Distributed Simulation and Real-Time Applications (2003) 4–11 8. M. Eklof, M. Sparf, and F. Moradi: Peer-to-peer-based Resource Management in Support of HLA-based Distributed Simulations. Simulation-Transactions of the Society for Modeling and Simulation International (2004) 181–190 9. W. Cai, S.J. Turner, and H. Zhao: A Load Management System for Running HLABased Distributed Simulations over the Grid. The 6th IEEE International Workshop on Distributed Simulation and Real-Time Applications (2002) 7–14 10. W. Cai, Z. Yuan, M.Y.H. Low, and S.J. Turner: Federate migration in HLA-based simulation. Computational Science - ICCS 2004, LNCS Vol. 3038 (2004) 856–864

Cyclic Reproduction Scheme in Genetic Algorithm for Evolutionary Game Sang-Won Um, Suk-Han Lee, Tae-Yong Kim, and Jong-Soo Choi Department of Image Engineering Graduate School of Advanced Imaging Science, Multi-media, and Film Chung-Ang University, 221 Huksuk-Dong, DongJak-Gu, Seoul, South Korea {sangwon, ichthus, jschoi}@imagelab.cau.ac.kr [email protected]

Abstract. This paper describes how Genetic Algorithm can be used to control the level of difficulty in a game based on a user’s skill. An algorithm is proposed to control the difficulty of a game according to user’s propensity in our previous work [1], [2]. But the searching spaces are very narrow and convergence of chromosomes is also slow because the game progresses step-by-step and chromosomes are evaluated simultaneously while playing the game. Thus, a method is presented to expand the searching spaces and converge quickly on the Genetic Algorithm in the domain of sequential progress.

1 Introduction Most computer game developers are coupling their AI to their animation/simulation systems to generate characters which move with more reality and accuracy. However, it can be argued that game AI must associate itself with not only the game’s appearance but also with the game-play or intelligence. Most activity in the area of game AI is concerned with efficient ways of searching the game tree. However, there has been a mutual lack of interest between game developers and academic AI researchers [3], [4]. Most AI research systems are big hunks of code that require a significant investment of time to understand and use effectively. AI researchers rarely use computer games for their research, apart from classic board and card games such as chess, checkers, and bridge [5]. The game of chess is traditionally referred to as the ‘Drosophila’ of artificial intelligence research; a test bed for the development of ideas in game playing programs. It has received decades of research effort, leading eventually to the creation of Deep Blue [6] that defeated World Champion Garry Kasparov in 1997. Many games have contributed to the development of AI research. For example, John McCarthy [7] has stated that “The computer game Lemmings can serve as new Drosophila for AI research,” and so on. The game AI must be constructed so as to make the game-play more interesting and abundant. Controlling the level of difficulty of a game is one of the most important features to take into account in developing a game. Players want more difficult missions and tasks. At the same time only the limited resources may want a particular level of difficulty using minimum resources. Making a game hard is very easy. Increasing the enemies will do. But if a game is too difficult, it may fail to grab the Z. Pan et al. (Eds.): Edutainment 2006, LNCS 3942, pp. 616 – 626, 2006. © Springer-Verlag Berlin Heidelberg 2006

Cyclic Reproduction Scheme in Genetic Algorithm for Evolutionary Game

617

player’s interest. Generally, the level of difficulty can be adjusted by changing the pace of a game or by increasing the quantity or intensity of ‘enemies’ [3], [4], [8]. This paper focuses on adapting AI research to games. The exclusive AI algorithm is proposed for games. First, however, the features of computer games are discussed in order to explain the distinct characteristics of game AI. Traditionally, computer games have the following features: 1. 2. 3. 4.

Most games update the scene every 1/24 seconds The users proceed on sequential events Playing time is limited and very short from artificial intelligence’s point of view Excitement is a vital part for a game, thus it doesn’t need global optimum

A developer of an AI program for a game must be concerned with the above features. Many developers, however, are not interested in game AI, although they may have the enthusiasm to make scenes of 24 frames per second. Most game AI techniques to date have been focused on the realistic and smart behavior of game units or game appearance [3], [4], [8]. In this paper, a User Adaptive Difficulty Control algorithm (UADC) is proposed that controls the difficulty of a game based on the player’s propensity and proficiency using a Genetic Algorithm [1], [2]. It is appropriate to incorporate a Genetic Algorithm into a game for the following reasons: 1. 2. 3.

There are infinite variations of the optimum according to game players. It is hard to generalize the formula for the level of difficulty and interest of g ames. It is not necessary to search for the global optimum.

The first reason means that GA is suitable for searching an infinite space or undefined domain, and the second means it is not continuous. Several games have been released in which the AI has at least some human difficulty levels, such as Deep Blue. A better, more difficult game is not always more fun to play against. For these reasons, it is proper to apply a Genetic Algorithm into a game. It is a lot easier to make a game AI more difficult to play against than to make a more fun game. Section 2 of this paper introduces the authors’ previous work UADC and section 3 suggests an enhanced UADC called Cyclic Reproduction Scheme (CRS) that is a more optimized game environment. Section 4 demonstrates that CRS is an efficient way of applying GA to the game domain using schema theory. Section 5 describes the experimental method and results. Lastly, section 6 concludes the research and suggests possible future research work.

2 User Adaptive Difficulty Control Algorithm First, the Factors of Difficulty Control (FDC) and Factors of Users’ Adaptation (FUA) for a game have been defined. Generally, an example of FDC is the game speed or the amount of the enemies. The level of FDC influences the level of difficulty of the game; this is different from players’ propensities. An example of FUA is the game score, level of player or playing time. A game is composed with appropriate

618

S.-W. Um et al.

FDC according to the players’ FUA. A different game is then made based on players’ propensities. 2.1 UADC in a Game The representative game genres are analyzed and the possibility that UADC should be adapted to the various genre of game is identified. As shown in Table1, a general example of FDC and FUA for each game genre is defined [1], [2]. Table 1. FDC and FUA for each game genre in User adaptive difficulty control algorithm

FDC is used as chromosome and FUA is used as fitness in the Genetic Algorithm. The demonstration of the algorithm was presented by using Tetris as in previous works [1], [2]. The computer game Tetris is a 1-player game displayed on a screen by a 2-dimensional image. In the case of Tetris, there may be an easy sequence of pieces or hard one for the users according to the sequence of pieces. If four I-shape pieces and a square piece come out, parallel four I-shape pieces are laid down and the rest of the blank area is occupied by square pieces. Then, the player can remove 2 lines in the game-board. This is an example of typical easy patterns. Fig. 2 shows an example of a typical hard pattern.

Fig. 1. Easy sequence of pieces

Fig. 2. Hard sequences of pieces

Usually, the score of a game is an efficient way to present the user’s adaptation. However, the density of blank-blocks, including cleared lines, provided better results and thus is a more useful measure of user’s adaptation in Tetris [1], [2]. Fig. 3 shows using the UADC and its description of the process. For the purpose of a clear understanding, some notations are defined as ‘piece’ and ‘block’. The block means the atom of the game-board, and all entities of the game-board are composed by the block. The piece is composed of four blocks, and it is represented by the numbers 0 to 6 (as shown in Fig. 4). In the case of the tested Tetris, the game-board is composed of 12×23 blocks (as shown in Fig. 3).

Cyclic Reproduction Scheme in Genetic Algorithm for Evolutionary Game

619

2.2 Representation GA is used to control the difficulty of a game that dynamically changes patterns according to the skill of the players. User adaptation can be represented as the density of blank-blocks, including cleared lines, that represents the fitness of a player to the game. The difficulty of the game can be represented as the order of the pieces. The gene is the order of pieces. The chromosome is composed of four pieces and a fitness value as shown Fig. 4. The roulette-wheel selection is used. The offsprings are then produced as a one-point crossover [9], [10]. The offsprings are then mutated using flip mutation from 0 to 6 [10]. As shown Fig. 3, the fitness equation contains some parameters such as B, C, and L, where parameter B means the number of blank-blocks in the game-board, parameter C is the number of cleared lines, and parameter L is the number of existing lines in the game-board. The roulette-wheel selection is employed with a crossover rate of 0.8 and a mutation rate of 0.01. The GA runs while playing the game. The population size is set to 8 chromosomes [11], [12].

Fig. 3. Workspace and definitions of genes and fitness function

Fig. 4. The pieces and encoding chromosome

Fig. 4 shows an example of an encoding chromosome, and real number encoding (the integers from 0 to 6) is used here. The fitness formula below is used to evaluate the fitness of sequential chromosomes: B . (C + L) ×12 ( B : Blank block C : Cleared line L : Last line)

Fitness =

(1)

If the number emerged pieces after starting game is T, and b is the block, then a line is composed of 12 blocks (12b) and 1 piece is composed of 4 blocks (4b). See the Fig. 3 the game workspace. The relationship between a block, piece and line is as follows: 1 Line = 12 blocks, 1 Piece = 4 blocks .

620

S.-W. Um et al.

Cleared lines, last lines, total pieces and blank-blocks have following relationships:

(L + C )× 12 (b ) = T × 4(b ) + B (b ) (L + C )× 12 = 4T + B.

(2)

Last lines and cleared lines are the summation of total pieces and blank blocks. If F is the target of user’s adaptation, the equation (1) is expanded as follows: B , F (4T + B ) = B, 4T + B 4 FT + FB = B, 4 FT = (1 − F )B. F=

(3)

Then, B 4F = , T 1− F

∴

B = C. T

(4)

F is the target of user’s adaptation. F is constant and therefore the proposed algorithm maintains a user state of blank-blocks per T (T means the number of current emerged pieces).

Fig. 5. The graph of B/T follows fitness: B means blank-block, T means the number of pieces

In our previous research, the difficulty of a game was controlled based on the player’s propensity and proficiency using the Genetic Algorithm. As a result, a clear difference could be seen between beginners and experts in this game by observing the number of clear lines [1], [2]. 2.3 The Problem of Using a Genetic Algorithm for a Game The searching time of the proposed algorithm is proportionally increased with emerging piece and game time. However, the searching spaces are very narrow and convergence of chromosomes is slow because the game progresses step by step and chromosomes are evaluated simultaneously with the playing of the game. If a chromosome is evaluated, then 4 T is needed. Therefore, searching space is limited since the playing time is limited. If a population is composed of 8 chromosomes, then it spent 4T in evaluating a chromosome and it spent 32T in evaluating a population. On average, a game ends when it reaches 180T; thus it does not have enough time to find the optimum. However, it is not always true that increased difficulty means more fun. Therefore, the game does not need a global optimum and the algorithm can still show

Cyclic Reproduction Scheme in Genetic Algorithm for Evolutionary Game

621

a reliable result. Nevertheless, it is necessary to expand the searching spaces and converge quickly in order to find a user propensity and make a robust algorithm [1], [2].

3 Cyclic Reproduction Scheme It has been mentioned that it is necessary to expand the searching spaces and converge quickly on population because the game progresses step-by-step and chromosomes are evaluated simultaneously with the playing of the game. When an adaptive model is sought for sequential events, it is helpful to pay attention to cyclic crossover and the sliding-window method. Chromosomes can be made using sliding-window memory in order to evaluate more chromosomes. Consequently, it is possible to expand the searching space and save the searching time.

Fig. 6. Cyclic Reproduction Scheme by Sliding-window

A quick response can be obtained through this change. If N is the number of total pieces that are emerged in a single game, then UADC searches for the N/l number (l means the length of chromosome) of chromosomes. However, the proposed algorithm searches N-l+1 number of chromosomes. Therefore, the searching space becomes wider. Moreover, it can search the problem space more quickly than previous works. The evaluating time of a population is reduced as much as possible by using the similarity of each chromosome.

Fig. 7. Emergence for Evaluating fitness in UADC (left) and CRS (right)

As shown in the left of Fig. 7, the order of chromosomes is made so as to evaluate the four chromosomes and there are 16 pieces that emerge. Evaluating the chromosome in this way requires the chromosome to emerge in four pieces, and so it is very slow to evaluate the chromosomes and search the set of solutions. Thus, an approach of overlapping evaluation by the cyclic crossover method is proposed. As shown in the right of Fig. 7, chromosome {5012} is overlapping “1” and “2” with chromosome {1230}. This reduces evaluation time of the chromosomes by two times compared to the previous work. First, the overlap of the permutation is found at the head or tail of

622

S.-W. Um et al.

each chromosome. Secondly, the chromosomes are formed into a line and are continuously overlapped in the same order of genes. As Fig. 7 shows, searching times are reduced by 3 times of T (emerged piece) compared to the model set out in the left of Fig. 7. The time for the proposed algorithm is proportional to how similar the chromosomes are.

4 Evaluation of Convergence Using Modified Schema In the proposed algorithm, a more mathematically advanced result compared to the previous work is proved using schema theory. The sliding-window causes the similarity of chromosomes in a population to increase. Thus, schema theory has to be modified, because of the increase in the similarity. 4.1 Assumption It is assumed that the schema is the permutation of events (the pattern of integers from 0 to 6 that represent the pieces in Tetris) and the position of the schema is not influential. Since the game plays sequentially, current states are involved with the previous states. Therefore, the position of schema is irrelevant to the emergence of events in the game. Under these assumptions, if a schema consists of the three numbers of “340,” then the five patterns of chromosomes that have same schema are shown in Fig. 8. The symbol ‘*’ is not influential from pattern to pattern, because the emergence of events only influence each other in the game. Therefore, schema do not contain the symbol ‘*’ in the assumption here. We already mentioned that, Fig. 8 shows a formation of genes contained same schema “340,” and so on.

Fig. 8. The Schema for Proposed Algorithm: same schema in proposed algorithm

4.2 Influence of Crossover If a crossover is performed which mates with neighboring chromosomes, then the portion of the population is different from schema theory. O(H) is used as a substitute for δ(H) in the original schema theory, and schema theory is changed by the location of the symbol ‘*’. O(H) is the order of schema and δ(H) is the length of schema. O(H) means the number of defined (except for the ‘*’ symbol) bits and δ(H) means the distance between the far left and the far right defined bits in the chromosome [10], [13]. As an assumption, the schema never contain the symbol ‘*’. Generally, O(H) is shorter than δ(H), and the schema theory of CRS for crossover is shown as follows:

Cyclic Reproduction Scheme in Genetic Algorithm for Evolutionary Game

Pdest = Pc

δ (H ) l −1

,

Psurv = 1 − Pdest ,

623

(5)

~ ( H , k + 1) = m( H , k ) f ( H , k ) ⎡1 − P δ ( H ) ⎤. m c l − 1 ⎥⎦ f ( k ) ⎢⎣

P c is probability of crossover, Pdest is probability of destroying the schema, Psurv is the probability of survival, and m(H,k) means the number of schema H in the generation of k. The formula is modified to consider the cyclic reproduction as follows: B

B

Pdest = Pc

O( H ) O( H ) × l −1 l −1

(6)

2 ⎡ δ (H ) ⎤ ⎛ O( H ) ⎞ ⎤ ⎡ ⎢⎣1 − Pc l − 1 ⎥⎦ >> ⎢1 − Pc ⎜⎝ l − 1 ⎟⎠ ⎥. (∵ O ( H ) < δ ( H )) ⎣⎢ ⎦⎥

(7)

According to the assumption, O(H) is used as a substitute for δ(H) of original schema theory, schema do not have the symbol ‘*’, and O(H) is smaller than δ(H). Because the schema does not contain the symbol ‘*’, O(H) is smaller than δ(H). The probability of destruction is relevant to δ(H) in original schema theory. If the crossover is executed in a point of schema, a schema cannot survive. But neighboring chromosomes have a much higher probability of containing the same genes and, if crossover is executed with neighboring chromosomes, then schema theory is derived as the formula (6) in the proposed algorithm. Since O(H) is smaller than δ(H), the probability of survival is increased. 4.3 Influence of Mutation The schema theory of mutation is set out below [10], [13]: Psurv = (1 − Pm ) o ( H ) ≅ 1 − Pm o( H ).

(8)

If the sliding-window is applied, the proportion of selected chromosomes that includes a schema H is shown with the formula below. E(H) is the probability of mutation containing the schema H, thus E(H) is derived as follows: h

h = l − O( H) + 1 , E(H) =

∑i i =1

h⋅n

, Psurv = 1 − Pm ⋅ E(H) , and (1 − Pm ) o ( H )

k ⎡ i ∑ ⎢ >> ⎢1 − Pm ⋅ i =1 h⋅n ⎢ ⎢⎣

⎤ ⎥ ⎥ ⎥ ⎥⎦

o( H )

(∵ E(H) 1 − Pm E ( H )O( H ).

(9)

The h is the number of chromosomes including a schema H in a population. The probability of including the schema H is different from both schema theories because of the cyclic reproduction schema. For example, at least 5 chromosomes contain schema “340” in the population shown in Fig. 8. The h relates to the length of chromosome ‘l’ and the order of schema H ‘O(H)’ and ‘E(H’) is derived as the summation of integers from 1 to h divided by h and n (the number of the population). Then,

624

S.-W. Um et al.

the probability of destroying the schema H is smaller than the original schema theory because the population increases in the similarity of chromosomes. Therefore, more schemas exist in the population of the proposed algorithm. In the proposed algorithm, the schema equation is modified by the changes of crossover and mutation as follows: k ⎡ i ∑ ⎢ ~ ( H , k + 1) = m( H , k ) f ( H , k ) ⎡1 − P ⎛⎜ O ( H ) ⎞⎟ ⎤ ⎢1 − P ⋅ i =1 m ⎢ ⎥ m c k ⋅n f ( k ) ⎣⎢ ⎝ l − 1 ⎠ ⎦⎥ ⎢ ⎢ ⎣ 2

k ⎡ i 2 ∑ ⎢ O( H ) ⎞ f (H , k ) ⎢1 − Pc ⎛⎜ = m( H , k ) ⎟ − Pm ⋅ i =1 k ⋅n f (k ) ⎢ ⎝ l −1 ⎠ ⎢⎣

⎤ ⎥ ⎥ ⎥ ⎥ ⎦

⎤ ⎥ ⎥. ⎥ ⎥⎦

(10) k ⎞ ⎛ 2 ∑i ⎟ ⎜ ⎜∵ P ⎛⎜ O ( H ) ⎞⎟ P ⋅ i =1