Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen
Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen University of Dortmund, Germany Madhu Sudan Massachusetts Institute of Technology, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max-Planck Institute of Computer Science, Saarbruecken, Germany
5618
Gavriel Salvendy Michael J. Smith (Eds.)
Human Interface and the Management of Information Information and Interaction Symposium on Human Interface 2009 Held as Part of HCI International 2009 San Diego, CA, USA, July 19-24, 2009 Proceedings, Part II
13
Volume Editors Gavriel Salvendy Purdue University, Grissom Hall, Room 263 315 North Grant Street, West Lafayette, IN, 47907-2023, USA E-mail:
[email protected] and Tsinghua University, Department of Industrial Engineering Beijing 10084, P.R. China Michael J. Smith University of Wisconsin Department of Industrial and Systems Engineering 1513 University Avenue, Madison, WI 53706, USA E-mail:
[email protected] Library of Congress Control Number: Applied for CR Subject Classification (1998): H.5, H.3, H.4, K.4.3, D.2 LNCS Sublibrary: SL 3 – Information Systems and Application, incl. Internet/Web and HCI ISSN ISBN-10 ISBN-13
0302-9743 3-642-02558-7 Springer Berlin Heidelberg New York 978-3-642-02558-7 Springer Berlin Heidelberg New York
This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. springer.com © Springer-Verlag Berlin Heidelberg 2009 Printed in Germany Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper SPIN: 12704330 06/3180 543210
Foreword
The 13th International Conference on Human–Computer Interaction, HCI International 2009, was held in San Diego, California, USA, July 19–24, 2009, jointly with the Symposium on Human Interface (Japan) 2009, the 8th International Conference on Engineering Psychology and Cognitive Ergonomics, the 5th International Conference on Universal Access in Human–Computer Interaction, the Third International Conference on Virtual and Mixed Reality, the Third International Conference on Internationalization, Design and Global Development, the Third International Conference on Online Communities and Social Computing, the 5th International Conference on Augmented Cognition, the Second International Conference on Digital Human Modeling, and the First International Conference on Human-Centered Design. A total of 4,348 individuals from academia, research institutes, industry and governmental agencies from 73 countries submitted contributions, and 1,425 papers that were judged to be of high scientific quality were included in the program. These papers address the latest research and development efforts and highlight the human aspects of the design and use of computing systems. The papers accepted for presentation thoroughly cover the entire field of human–computer interaction, addressing major advances in knowledge and effective use of computers in a variety of application areas. This volume, edited by Gavriel Salvendy and Michael J. Smith, contains papers in the thematic area of Human Interface and the Management of Information, addressing the following major topics: • • • • • • •
Interacting with the World Wide Web Intelligent Techniques for Access to Information and Personalization Visual Interfaces, Visualization and Images Mobile Devices and Services eHealth Applications and Services Education, Learning and Entertainment Information Systems in Safety-Critical Domains
The remaining volumes of the HCI International 2009 proceedings are: • • • • •
Volume 1, LNCS 5610, Human–Computer Interaction––New Trends (Part I), edited by Julie A. Jacko Volume 2, LNCS 5611, Human–Computer Interaction––Novel Interaction Methods and Techniques (Part II), edited by Julie A. Jacko Volume 3, LNCS 5612, Human–Computer Interaction––Ambient, Ubiquitous and Intelligent Interaction (Part III), edited by Julie A. Jacko Volume 4, LNCS 5613, Human–Computer Interaction––Interacting in Various Application Domains (Part IV), edited by Julie A. Jacko Volume 5, LNCS 5614, Universal Access in Human–Computer Interaction––Addressing Diversity (Part I), edited by Constantine Stephanidis
VI
Foreword
• • • • • • • • • • •
Volume 6, LNCS 5615, Universal Access in Human–Computer Interaction––Intelligent and Ubiquitous Interaction Environments (Part II), edited by Constantine Stephanidis Volume 7, LNCS 5616, Universal Access in Human–Computer Interaction––Applications and Services (Part III), edited by Constantine Stephanidis Volume 8, LNCS 5617, Human Interface and the Management of Information––Designing Information Environments (Part I), edited by Michael J. Smith and Gavriel Salvendy Volume 10, LNCS 5619, Human-Centered Design, edited by Masaaki Kurosu Volume 11, LNCS 5620, Digital Human Modeling, edited by Vincent G. Duffy Volume 12, LNCS 5621, Online Communities and Social Computing, edited by A. Ant Ozok and Panayiotis Zaphiris Volume 13, LNCS 5622, Virtual and Mixed Reality, edited by Randall Shumaker Volume 14, LNCS 5623, Internationalization, Design and Global Development, edited by Nuray Aykin Volume 15, LNCS 5624, Ergonomics and Health Aspects of Work with Computers, edited by Ben-Tzion Karsh Volume 16, LNAI 5638, The Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience, edited by Dylan Schmorrow, Ivy Estabrooke and Marc Grootjen Volume 17, LNAI 5639, Engineering Psychology and Cognitive Ergonomics, edited by Don Harris
I would like to thank the Program Chairs and the members of the Program Boards of all thematic areas, listed below, for their contribution to the highest scientific quality and the overall success of HCI International 2009.
Ergonomics and Health Aspects of Work with Computers Program Chair: Ben-Tzion Karsh Arne Aarås, Norway Pascale Carayon, USA Barbara G.F. Cohen, USA Wolfgang Friesdorf, Germany John Gosbee, USA Martin Helander, Singapore Ed Israelski, USA Waldemar Karwowski, USA Peter Kern, Germany Danuta Koradecka, Poland Kari Lindström, Finland
Holger Luczak, Germany Aura C. Matias, Philippines Kyung (Ken) Park, Korea Michelle M. Robertson, USA Michelle L. Rogers, USA Steven L. Sauter, USA Dominique L. Scapin, France Naomi Swanson, USA Peter Vink, The Netherlands John Wilson, UK Teresa Zayas-Cabán, USA
Foreword
Human Interface and the Management of Information Program Chair: Michael J. Smith Gunilla Bradley, Sweden Hans-Jörg Bullinger, Germany Alan Chan, Hong Kong Klaus-Peter Fähnrich, Germany Michitaka Hirose, Japan Jhilmil Jain, USA Yasufumi Kume, Japan Mark Lehto, USA Fiona Fui-Hoon Nah, USA Shogo Nishida, Japan Robert Proctor, USA Youngho Rhee, Korea
Anxo Cereijo Roibás, UK Katsunori Shimohara, Japan Dieter Spath, Germany Tsutomu Tabe, Japan Alvaro D. Taveira, USA Kim-Phuong L. Vu, USA Tomio Watanabe, Japan Sakae Yamamoto, Japan Hidekazu Yoshikawa, Japan Li Zheng, P.R. China Bernhard Zimolong, Germany
Human–Computer Interaction Program Chair: Julie A. Jacko Sebastiano Bagnara, Italy Sherry Y. Chen, UK Marvin J. Dainoff, USA Jianming Dong, USA John Eklund, Australia Xiaowen Fang, USA Ayse Gurses, USA Vicki L. Hanson, UK Sheue-Ling Hwang, Taiwan Wonil Hwang, Korea Yong Gu Ji, Korea Steven Landry, USA
Gitte Lindgaard, Canada Chen Ling, USA Yan Liu, USA Chang S. Nam, USA Celestine A. Ntuen, USA Philippe Palanque, France P.L. Patrick Rau, P.R. China Ling Rothrock, USA Guangfeng Song, USA Steffen Staab, Germany Wan Chul Yoon, Korea Wenli Zhu, P.R. China
Engineering Psychology and Cognitive Ergonomics Program Chair: Don Harris Guy A. Boy, USA John Huddlestone, UK Kenji Itoh, Japan Hung-Sying Jing, Taiwan Ron Laughery, USA Wen-Chin Li, Taiwan James T. Luxhøj, USA
Nicolas Marmaras, Greece Sundaram Narayanan, USA Mark A. Neerincx, The Netherlands Jan M. Noyes, UK Kjell Ohlsson, Sweden Axel Schulte, Germany Sarah C. Sharples, UK
VII
VIII
Foreword
Neville A. Stanton, UK Xianghong Sun, P.R. China Andrew Thatcher, South Africa
Matthew J.W. Thomas, Australia Mark Young, UK
Universal Access in Human–Computer Interaction Program Chair: Constantine Stephanidis Julio Abascal, Spain Ray Adams, UK Elisabeth André, Germany Margherita Antona, Greece Chieko Asakawa, Japan Christian Bühler, Germany Noelle Carbonell, France Jerzy Charytonowicz, Poland Pier Luigi Emiliani, Italy Michael Fairhurst, UK Dimitris Grammenos, Greece Andreas Holzinger, Austria Arthur I. Karshmer, USA Simeon Keates, Denmark Georgios Kouroupetroglou, Greece Sri Kurniawan, USA
Patrick M. Langdon, UK Seongil Lee, Korea Zhengjie Liu, P.R. China Klaus Miesenberger, Austria Helen Petrie, UK Michael Pieper, Germany Anthony Savidis, Greece Andrew Sears, USA Christian Stary, Austria Hirotada Ueda, Japan Jean Vanderdonckt, Belgium Gregg C. Vanderheiden, USA Gerhard Weber, Germany Harald Weber, Germany Toshiki Yamaoka, Japan Panayiotis Zaphiris, UK
Virtual and Mixed Reality Program Chair: Randall Shumaker Pat Banerjee, USA Mark Billinghurst, New Zealand Charles E. Hughes, USA David Kaber, USA Hirokazu Kato, Japan Robert S. Kennedy, USA Young J. Kim, Korea Ben Lawson, USA
Gordon M. Mair, UK Miguel A. Otaduy, Switzerland David Pratt, UK Albert “Skip” Rizzo, USA Lawrence Rosenblum, USA Dieter Schmalstieg, Austria Dylan Schmorrow, USA Mark Wiederhold, USA
Internationalization, Design and Global Development Program Chair: Nuray Aykin Michael L. Best, USA Ram Bishu, USA Alan Chan, Hong Kong Andy M. Dearden, UK
Susan M. Dray, USA Vanessa Evers, The Netherlands Paul Fu, USA Emilie Gould, USA
Foreword
Sung H. Han, Korea Veikko Ikonen, Finland Esin Kiris, USA Masaaki Kurosu, Japan Apala Lahiri Chavan, USA James R. Lewis, USA Ann Light, UK James J.W. Lin, USA Rungtai Lin, Taiwan Zhengjie Liu, P.R. China Aaron Marcus, USA Allen E. Milewski, USA
Elizabeth D. Mynatt, USA Oguzhan Ozcan, Turkey Girish Prabhu, India Kerstin Röse, Germany Eunice Ratna Sari, Indonesia Supriya Singh, Australia Christian Sturm, Spain Adi Tedjasaputra, Singapore Kentaro Toyama, India Alvin W. Yeo, Malaysia Chen Zhao, P.R. China Wei Zhou, P.R. China
Online Communities and Social Computing Program Chairs: A. Ant Ozok, Panayiotis Zaphiris Chadia N. Abras, USA Chee Siang Ang, UK Amy Bruckman, USA Peter Day, UK Fiorella De Cindio, Italy Michael Gurstein, Canada Tom Horan, USA Anita Komlodi, USA Piet A.M. Kommers, The Netherlands Jonathan Lazar, USA Stefanie Lindstaedt, Austria
Gabriele Meiselwitz, USA Hideyuki Nakanishi, Japan Anthony F. Norcio, USA Jennifer Preece, USA Elaine M. Raybourn, USA Douglas Schuler, USA Gilson Schwartz, Brazil Sergei Stafeev, Russia Charalambos Vrasidas, Cyprus Cheng-Yen Wang, Taiwan
Augmented Cognition Program Chair: Dylan D. Schmorrow Andy Bellenkes, USA Andrew Belyavin, UK Joseph Cohn, USA Martha E. Crosby, USA Tjerk de Greef, The Netherlands Blair Dickson, UK Traci Downs, USA Julie Drexler, USA Ivy Estabrooke, USA Cali Fidopiastis, USA Chris Forsythe, USA Wai Tat Fu, USA Henry Girolamo, USA
Marc Grootjen, The Netherlands Taro Kanno, Japan Wilhelm E. Kincses, Germany David Kobus, USA Santosh Mathan, USA Rob Matthews, Australia Dennis McBride, USA Robert McCann, USA Jeff Morrison, USA Eric Muth, USA Mark A. Neerincx, The Netherlands Denise Nicholson, USA Glenn Osga, USA
IX
X
Foreword
Dennis Proffitt, USA Leah Reeves, USA Mike Russo, USA Kay Stanney, USA Roy Stripling, USA Mike Swetnam, USA Rob Taylor, UK
Maria L.Thomas, USA Peter-Paul van Maanen, The Netherlands Karl van Orden, USA Roman Vilimek, Germany Glenn Wilson, USA Thorsten Zander, Germany
Digital Human Modeling Program Chair: Vincent G. Duffy Karim Abdel-Malek, USA Thomas J. Armstrong, USA Norm Badler, USA Kathryn Cormican, Ireland Afzal Godil, USA Ravindra Goonetilleke, Hong Kong Anand Gramopadhye, USA Sung H. Han, Korea Lars Hanson, Sweden Pheng Ann Heng, Hong Kong Tianzi Jiang, P.R. China
Kang Li, USA Zhizhong Li, P.R. China Timo J. Määttä, Finland Woojin Park, USA Matthew Parkinson, USA Jim Potvin, Canada Rajesh Subramanian, USA Xuguang Wang, France John F. Wiechel, USA Jingzhou (James) Yang, USA Xiu-gan Yuan, P.R. China
Human Centered Design Program Chair: Masaaki Kurosu Gerhard Fischer, USA Tom Gross, Germany Naotake Hirasawa, Japan Yasuhiro Horibe, Japan Minna Isomursu, Finland Mitsuhiko Karashima, Japan Tadashi Kobayashi, Japan
Kun-Pyo Lee, Korea Loïc Martínez-Normand, Spain Dominique L. Scapin, France Haruhiko Urokohara, Japan Gerrit C. van der Veer, The Netherlands Kazuhiko Yamazaki, Japan
In addition to the members of the Program Boards above, I also wish to thank the following volunteer external reviewers: Gavin Lew from the USA, Daniel Su from the UK, and Ilia Adami, Ioannis Basdekis, Yannis Georgalis, Panagiotis Karampelas, Iosif Klironomos, Alexandros Mourouzis, and Stavroula Ntoa from Greece. This conference could not have been possible without the continuous support and advice of the Conference Scientific Advisor, Gavriel Salvendy, as well as the dedicated work and outstanding efforts of the Communications Chair and Editor of HCI International News, Abbas Moallem.
Foreword
XI
I would also like to thank for their contribution toward the organization of the HCI International 2009 conference the members of the Human–Computer Interaction Laboratory of ICS-FORTH, and in particular Margherita Antona, George Paparoulis, Maria Pitsoulaki, Stavroula Ntoa, and Maria Bouhli. Constantine Stephanidis
HCI International 2011
The 14th International Conference on Human–Computer Interaction, HCI International 2011, will be held jointly with the affiliated conferences in the summer of 2011. It will cover a broad spectrum of themes related to human–computer interaction, including theoretical issues, methods, tools, processes and case studies in HCI design, as well as novel interaction techniques, interfaces and applications. The proceedings will be published by Springer. More information about the topics, as well as the venue and dates of the conference, will be announced through the HCI International Conference series website: http://www.hci-international.org/
General Chair Professor Constantine Stephanidis University of Crete and ICS-FORTH Heraklion, Crete, Greece Email:
[email protected] Table of Contents
Part I: Interacting with the World Wide Web Development of a Coloration Support Tool for Making Web Page Screens User-Friendly for Color Blind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Michiko Anse and Tsutomu Tabe The Persuasive Effects from Web 2.0 Marketing: A Case Study Investigating the Persuasive Effect from an Online Design Competition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Asle Fagerstrøm and Gheorghita Ghinea
3
10
Formalizing Design Guidelines of Legibility on Web Pages . . . . . . . . . . . . . Fong-Ling Fu and Chiu-Hung Su
17
The Assessment of Credibility of e-Government: Users’ Perspective . . . . . Zhao Huang, Laurence Brooks, and Sherry Chen
26
Auto-complete for Improving Reliability on Semantic Web Service Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hanmin Jung, Mi-Kyoung Lee, Won-Kyung Sung, and Beom-Jong You
36
Effects of AJAX Technology on the Usability of Blogs . . . . . . . . . . . . . . . . Sumonta Kasemvilas and Daniel Firpo
45
Usability Evaluation of Dynamic RSVP Interface on Web Page . . . . . . . . Ya-Li Lin and Darcy Lin
55
“Online Legitimacy”: Defining Institutional Symbolisms for the Design of Information Artifact in the Web Mediated Information Environment (W-MIE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Emma Nuraihan Mior Ibrahim and Nor Laila Md Noor Evaluation of Web User Interfaces for the Online Retail of Apparel . . . . . Dominik Rupprecht, Rainer Blum, and Karim Khakzar
65 74
A Coauthoring Method of Keyword Dictionaries for Knowledge Combination on Corporate Discussion Web Sites . . . . . . . . . . . . . . . . . . . . . Shinji Takao, Tadashi Iijima, and Akito Sakurai
84
An Empirical Study the Effects of Language Factors on Web Site Use Intention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hui-Jen Yang and Yun-Long Lay
94
XVI
Table of Contents
Part II: Intelligent Techniques for Access to Information and Personalization Enhancing Document Clustering through Heuristics and Summary-Based Pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sri Harsha Allamraju and Robert Chun Email Reply Prediction: A Machine Learning Approach . . . . . . . . . . . . . . . Taiwo Ayodele, Shikun Zhou, and Rinat Khusainov An End-to-End Proactive TCP Based on Available Bandwidth Estimation with Congestion Level Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sangtae Bae, Doohyung Lee, Chihoon Lee, Jinwook Chung, Jahwan Koo, and Suman Banerjee Smart Privacy Management in Ubiquitous Computing Environments . . . Christian B¨ unnig A Fuzzy Multiple Criteria Decision Making Model for Selecting the Distribution Center Location in China: A Taiwanese Manufacturer’s Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chien-Chang Chou and Pei-Chann Chang A Hierarchical Data Dissemination Protocol Using Probability-Based Clustering for Wireless Sensor Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . Moonseong Kim, Matt W. Mutka, and Hyunseung Choo An OWL-Based Knowledge Model for Combined-Process-and-Location Aware Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gunhee Kim, Manchul Han, Jukyung Park, Hyunchul Park, Sehyung Park, Laehyun Kim, and Sungdo Ha Human-Biometric Sensor Interaction: Impact of Training on Biometric System and User Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Eric P. Kukula and Robert W. Proctor Representing Logical Inference Steps with Digital Circuits . . . . . . . . . . . . . Erika Matsak
105
114
124
131
140
149
159
168
178
An Interactive-Content Technique Based Approach to Generating Personalized Advertisement for Privacy Protection . . . . . . . . . . . . . . . . . . . Wook-Hee Min and Yun-Gyung Cheong
185
Loopo: Integrated Text Miner for FACT-Graph-Based Trend Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ryosuke Saga, Hiroshi Tsuji, and Kuniaki Tabata
192
Table of Contents
Using Graphical Models for an Intelligent Mixed-Initiative Dialog Management System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Stefan Schw¨ arzler, G¨ unther Ruske, Frank Wallhoff, and Gerhard Rigoll Input Text Repairing for Multi-lingual Chat System . . . . . . . . . . . . . . . . . . Kenichi Yoshida and Fumio Hattori
XVII
201
210
Part III: Visual Interfaces, Visualization and Images Interactive Object Segmentation System from a Video Sequence . . . . . . . Guntae Bae, Sooyeong Kwak, and Hyeran Byun COBRA – A Visualization Solution to Monitor and Analyze Consumer Generated Medias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Amit Behal, Julia Grace, Linda Kato, Ying Chen, Shixia Liu, Weijia Cai, and Weihong Qian Visual String of Reformulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Arne Berger, Jens K¨ ursten, and Maximilian Eibl Industrial E-Commerce and Visualization of Products: 3D Rotation versus 2D Metamorphosis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Francisco V. Cipolla Ficarra, Miguel Cipolla Ficarra, and Daniel A. Giulianelli Evaluating the Effectiveness and the Efficiency of a Vector Image Search Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Patrizia Di Marco, Tania Di Mascio, Daniele Frigioni, and Massimo Gastaldi
221
229
239
249
259
Building and Browsing Tropos Models: The AVI Design . . . . . . . . . . . . . . . Tania Di Mascio, Anna Perini, Luca Sabatucci, and Angelo Susi
269
A Multiple-Aspects Visualization Tool for Exploring Social Networks . . . Jie Gao, Kazuo Misue, and Jiro Tanaka
277
Multi-hierarchy Information Visualization Research Based on Three-Dimensional Display of Products System . . . . . . . . . . . . . . . . . . . . . . Zhou Hui and Hou WenJun Efficient Annotation Visualization Using Distinctive Features . . . . . . . . . . Seok Kyoo Kim, Sung Hyun Moon, Jun Park, and Sang Yong Han Content Based Image Retrieval Using Adaptive Inverse Pyramid Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mariofanna Milanova, Roumen Kountchev, Stuart Rubin, Vladimir Todorov, and Roumiana Kountcheva
287 295
304
XVIII
Table of Contents
Event Extraction and Visualization for Obtaining Personal Experiences from Blogs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yoko Nishihara, Keita Sato, and Wataru Sunayama
315
Minato: Integrated Visualization Environment for Embedded Systems Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yosuke Nishino and Eiichi Hayakawa
325
Batik KR Semantic Network: Visualizations of Creative Process and Design Knowledge for the Malaysian Batik Designers’ Community . . . . . Ariza Nordin, Nor Laila Md. Noor, and Ahmad Zainuddin
334
A Tool for Analyzing Categorical Data Visually with Granular Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kousuke Shiraishi, Kazuo Misue, and Jiro Tanaka
342
Part IV: Mobile Devices and Services Understanding Key Attributes in Mobile Service: Kano Model Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Seung Ik Baek, Seung Kuk Paik, and Weon Sang Yoo Discovering User Interface Requirements of Search Results for Mobile Clients by Contextual Inquiry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . David L. Chan, Robert W.P. Luk, Hong Va Leong, and Edward K.S. Ho Evaluation of Pointing Efficiency on Small Screen Touch User Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ryosuke Fujioka, Takayuki Akiba, and Hidehiko Okada An Integrated Approach towards the Homogeneous Provision of Geographically Dispersed Info-Mobility Services to Mobile Users . . . . . . . Dimitrios Giakoumis, Dimitrios Tzovaras, Dionisis Kehagias, Evangelos Bekiaris, and George Hassapis Legible Character Size on Mobile Terminal Screens: Estimation Using Pinch-in/Out on the iPod Touch Panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Satoshi Hasegawa, Masako Omori, Tomoyuki Watanabe, Shohei Matsunuma, and Masaru Miyao
355
365
375
385
395
Location-Based Mixed-Map Application Development for Mobile Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hyo-Haeng Lee, Kil-Ram Ha, and Kwang-Seok Hong
403
A Comparison of Artifact Reduction Methods for Real-Time Analysis of fNIRS Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Takayuki Nozawa and Toshiyuki Kondo
413
Table of Contents
Investigation on Relation between Index of Difficulty in Fitts’ Law and Device Screen Sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hidehiko Okada, Takayuki Akiba, and Ryosuke Fujioka Influence of Vertical Length of Characters on Readability in Mobile Phones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Masako Omori, Satoshi Hasegawa, Tomoyuki Watanabe, Shohei Matsunuma, and Masaru Miyao
XIX
423
430
Intelligent Photo Management System Enhancing Browsing Experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yuki Orii, Takayuki Nozawa, and Toshiyuki Kondo
439
Freeze TCPv2: An Enhancement of Freeze TCP for Efficient Handoff in Heterogeneous Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Minu Park, Jaehyung Lee, Jahwan Koo, and Hyunseung Choo
448
Expanding SNS Features with CE Devices: Space, Profile, Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Youngho Rhee, Hyunjoo Kang, Yeojin Kim, Juyeon Lee, and IlKu Chang Empirical Evaluation of Throwing Method to Move Object for Long Distance in 3D Information Space on Mobile Device . . . . . . . . . . . . . . . . . . Yu Shibuya, Keiichiro Nagatomo, Kazuyoshi Murata, Itaru Kuramoto, and Yoshihiro Tsujino Usefulness of Mobile Information Provision Systems Using Graphic Text -Visibility of Graphic Text on Mobile Phones . . . . . . . . . . . . . . . . . . . Tomoyuki Watanabe, Masako Omori, Satoshi Hasegawa, Shohei Matsunuma, and Masaru Miyao
458
468
476
Part V: eHealth Applications and Services The Importance of Information in the Process of Acquisition and Usage of a Medicine for Patient Safety: A Study of the Brazilian Context . . . . . Patricia Lopes Fujita and Carla Galv˜ ao Spinillo
489
A Proposal of Collection and Analysis System of Near Miss Incident in Nursing Duties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Akihisa Furukawa and Yusaku Okada
497
Effects of Information Displays for Hyperlipidemia . . . . . . . . . . . . . . . . . . . Yang Gong and Jiajie Zhang Clinical Usefulness of Human-Computer Interface for Training Targeted Facial Expression: Application to Patients with Cleft Lip and/or Palate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kyoko Ito, Ai Takami, Shumpei Hanibuchi, Shogo Nishida, Masakazu Yagi, Setsuko Uematsu, Naoko Sigenaga, and Kenji Takada
503
513
XX
Table of Contents
The Evaluation of Pharmaceutical Package Designs for the Elderly People . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Akira Izumiya, Michiko Ohkura, and Fumito Tsuchiya
522
Implications for Developing Information System on Nursing Administration – Case Study on Nurse Scheduling System – . . . . . . . . . . . Mitsuhiko Karashima and Naotake Hirasawa
529
Analysis on Descriptions of Dosage Regimens in Package Inserts of Medicines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Masaomi Kimura, Kazuhiro Okada, Keita Nabeta, Michiko Ohkura, and Fumito Tsuchiya Non-intrusive Human Behavior Monitoring Sensor for Health Care System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Noriyuki Kushiro, Makoto Katsukura, Masanori Nakata, and Yoshiaki Ito Impact of Healthcare Information Technology Systems on Patient Safety . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Byung Cheol Lee and Vincent G. Duffy Patient Standardization Identification as a Healthcare Issue . . . . . . . . . . . Mario Macedo and Pedro Isa´ıas A Proposal of a Method to Extract Active Ingredient Names from Package Inserts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Keita Nabeta, Masaomi Kimura, Michiko Ohkura, and Fumito Tsuchiya Examination of Evaluation Method for Appearance Similarity of PTP Sheets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yoshitaka Ootsuki, Akira Izumiya, Michiko Ohkura, and Fumito Tsuchiya Identifying Latent Similarities among Near-Miss Incident Records Using a Text-Mining Method and a Scenario-Based Approach . . . . . . . . . Tetsuo Sawaragi, Kouichi Ito, Yukio Horiguchi, and Hiroaki Nakanishi Patient Safety: Contributions from a Task Analysis Study on Medicine Usage by Brazilians . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Carla Spinillo, Stephania Padovani, and Cristine Lanzoni Remote Consultation System Using Hierarchically Structured Agents . . . Hiroshi Yajima, Jun Sawamoto, and Kazuo Matsuda
539
549
559 566
576
586
594
604 609
Part VI: Education, Learning and Entertainement How Mobile Interaction Motivates Students in a Class? . . . . . . . . . . . . . . . Akinobu Ando and Kazunari Morimoto
621
Table of Contents
XXI
Sensation Seeking, Self Forgetfulness, and Computer Game Enjoyment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiaowen Fang and Fan Zhao
632
Development of an Annotation-Based Classroom Activities Support Environment Using Digital Appliance, Mobile Device and PC . . . . . . . . . . Yoshiaki Hada and Masanori Shinohara
642
An Empirical Investigation on the Effectiveness of Virtual Learning Environment in Supporting Collaborative Learning: A System Design Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Na Liu, Yingqin Zhong, and John Lim
650
Personalization for Specific Users: Designing Decision Support Systems to Support Stimulating Learning Environments . . . . . . . . . . . . . . . . . . . . . . Laura M˘ aru¸ster, Niels R. Faber, and Rob J. van Haren
660
Construction of Systematic Learning Support System of Business Theory and Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yoshiki Nakamura and Katsuhiro Sakamoto
669
Learning by Design in a Digital World: Students’ Attitudes towards a New Pedagogical Model for Online Academic Learning . . . . . . . . . . . . . . . . Karen Precel, Yoram Eshet-Alkalai, and Yael Alberton
679
Promoting a Central Learning Management System by Encouraging Its Use for Other Purposes Than Teaching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Franz Reichl and Andreas Hruska
689
Framework for Supporting Decision Making in Learning Management System Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yuki Terawaki
699
Statistics-Based Cognitive Human-Robot Interfaces for Board Games – Let’s Play! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Frank Wallhoff, Alexander Bannat, J¨ urgen Gast, Tobias Rehrl, Moritz Dausinger, and Gerhard Rigoll The Design and Development of an Adaptive Web-Based Learning System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chian Wang
708
716
Part VII: Information Systems in Safety-Critical Domains Human-System Interface (HSI) Challenges in Nuclear Power Plant Control Rooms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jo-Ling Chang, Huafei Liao, and Liang Zeng
729
XXII
Table of Contents
The Impact of Automation Assisted Aircraft Separation on Situation Awareness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Arik-Quang V. Dao, Summer L. Brandt, Vernol Battiste, Kim-Phuong L. Vu, Thomas Strybel, and Walter W. Johnson Separation Assurance and Collision Avoidance Concepts for the Next Generation Air Transportation System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . John P. Dwyer and Steven Landry Analysis of Team Communication and Collaboration in En-Route Air Traffic Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kazuo Furuta, Yusuke Soraji, Taro Kanno, Hisae Aoyama, Daisuke Karikawa, and Makoto Takahashi Comparison of Pilot Recovery and Response Times in Two Types of Cockpits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Vishal Hiremath, Robert W. Proctor, Richard O. Fanjoy, Robert G. Feyen, and John P. Young
738
748
758
766
Information Requirements and Sharing for NGATS Function Allocation Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nhut Tan Ho, Patrick Martin, Joseph Bellissimo, and Barry Berson
776
HILAS: Human Interaction in the Lifecycle of Aviation Systems – Collaboration, Innovation and Learning . . . . . . . . . . . . . . . . . . . David Jacobson, Nick McDonald, and Bernard Musyck
786
Redefining Interoperability: Understanding Police Communication Task Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gyu H. Kwon, Tonya L. Smith-Jackson, and Charles W. Bostian
797
Unique Reporting Form: Flight Crew Auditing of Everyday Performance in an Airline Safety Management System . . . . . . . . . . . . . . . . . . . . . . . . . . . . Maria Chiara Leva, Alison Kay, Joan Cahill, Gabriel Losa, Sharon Keating, Diogo Serradas, and Nick McDonald Pilot Confidence with ATC Automation Using Cockpit Situation Display Tools in a Distributed Traffic Management Environment . . . . . . . Sarah V. Ligda, Nancy Johnson, Joel Lachter, and Walter W. Johnson A Study of Auditory Warning Signals for the Design Guidelines of Man-Machine Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mie Nakatani, Daisuke Suzuki, Nobuchika Sakata, and Shogo Nishida Computer-Aided Collaborative Work into War Rooms: A New Approach of Collaboration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jeremy Ringard, Samuel Degrande, St´ephane Louis-dit-Picard, and Christophe Chaillou
806
816
826
835
Table of Contents
Optimizing Online Situation Awareness Probes in Air Traffic Management Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Thomas Z. Strybel, Katsumi Minakata, Jimmy Nguyen, Russell Pierce, and Kim-Phuong L. Vu A Development of Information System for Disaster Victims with Autonomous Wireless Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yuichi Takahashi, Daiji Kobayashi, and Sakae Yamamoto Situation Awareness and Performance of Student versus Experienced Air Traffic Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kim-Phuong L. Vu, Katsumi Minakata, Jimmy Nguyen, Josh Kraut, Hamzah Raza, Vernol Battiste, and Thomas Z. Strybel Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
XXIII
845
855
865
875
Development of a Coloration Support Tool for Making Web Page Screens User-Friendly for Color Blind Michiko Anse and Tsutomu Tabe Aoyama Gakuin University, Department of Industrial and Systems Engineering, 5-10-1 Fuchinobe, Sagamihara-city, Kanagawa, Japan {anse,tabe}@ise.aoyama.ac.jp
Abstract. Websites are providing more and more information because of their convenience. Information is often discriminated by color vision ability between individuals. Some people cannot discriminate information depending on the coloration. In these cases, information cannot be correctly discerned. Therefore, the development of a supporting tool for configuring screens is needed. Keywords: color blind, web page, coloration support tool.
1 Introduction Difference in color vision between individuals is caused by three types of nerve cells (cone cells that absorb different wavelengths). Persons who have normal chromatic vision perceive color with all three types of cone. Color deficient observers perceive colors at different levels depending on the status of impaired cone function. Color deficient observers include protanopes (red cone does not function), deuteranopes (green cone does not function) and tritanopes (blue cone does not function), which, when only two colors are perceived, is called dichromatic vision. Color deficient observers account for 5% and 0.2% of East Asian males and females, respectively. They account for 8% and 4% of Caucasian males and black males respectively. According to the ratio, one out of 20 Japanese is color deficient, which is a considerable number [1, 2]. Information collection is indispensable in daily life. The spread of the Internet is allowing websites to provide more and more information. The use of colors is effective for emphasizing information to be provided because colors increase image discrimination. Therefore, colors are widely used in websites. However, information dispensed by colors may not be passed on correctly because color vision varies between individuals. We should thus use picturesque colors for passing information, and this information should be able to be understood even by color deficient observers.
2 Methods Offering Support to Make Color Web Page Screens That Can Be Discriminated by Color Blind 2.1 Methods to Simulate Color Vision There are two main methods to simulate the vision of color deficient observers. The first method is to look through a filter to simulate imperfect color vision or to use a M.J. Smith and G. Salvendy (Eds.): Human Interface, Part II, HCII 2009, LNCS 5618, pp. 3–9, 2009. © Springer-Verlag Berlin Heidelberg 2009
4
M. Anse and T. Tabe
liquid crystal display monitor with a color vision simulating function. The second method is to convert normal color vision to anomalous color vision via VsCheck software, etc, after reading a file on a personal computer. By these methods, we can see what normal color vision looks like when converted into abnormal color vision, allowing us to realize how color deficient observers see things and find our extensive use of colors confusing. 2.2 Methods for Detecting Confusing Colorations Uding (software) can detect confusing colorations displayed by simulation software and change the coloration [3]. ColorSelector (software) determines whether or not a specified combination of colors can be discriminated by color deficient observer [4]. They modify already existing files or determine whether two chosen colors can be discriminated or not. Therefore, they have a disadvantage in that they can be used only after files have been created or require time-consuming work to investigate whether two colors chosen from many combinations can be discriminated. 2.3 A Method for Supporting Coloring-Decisions When Making Web Page Screens Using a color vision simulator or ColorSelector for determining coloration when creating web page screens, is quite troublesome. Therefore, the author has devised a tool that supports the determination of coloration when making web page screens, and has created and verified the prototype.
3 Functions of the Web Page Screen Coloration Support Tool It should be provided as an add-in function of web page creation software. A user selects an area requiring the support of coloration during the creation of a web page. Then, the user calls up the coloration support tool. The functions of the tool consist of “Select color,” “Diagnose” and “Display.” The user can determine coloration with the consideration of color deficient observers by calling up this tool and reflecting the coloration on the screen. This tool helps a user make colorations so that two colors on a background color can be discriminated by color deficient observers. This tool has two modes, one for characters and the other for illustrations only. 3.1 “Select Color” Function A user can select a background color and two colors on the background color from color palette tabs. Each color palette has 216 colors called web-safe colors that appear as almost the same color on both Windows and Macintosh. A user can specify a color he/she wants to use by clicking a color on the color palette or drawing a color out of the screen by using a dropper. A user selects a combination of colors, for example, “background and character color” whose coloration the user wants to evaluate. The color of any character which is inappropriate for the selected background color is
Development of a Coloration Support Tool for Making Web Page Screens User-Friendly
5
crossed out on the tab. Character colors which are appropriate for the selected background color, yet requiring some caution are checked with a triangular symbol. Color combinations which are deemed to be inappropriate for even one color vision out of the four types are also crossed out. 3.2 “Diagnosis” Function In the “Select color” mode, the color palette displays whether color combinations are appropriate or not. After determining the background color, and colors 1 and 2, visibility by the four vision types is diagnosed. The result is displayed by circle (appropriate), triangle (Due consideration is required though it is not inappropriate) and cross (inappropriate). The value of RGB of persons with normal color vision displayed on the color palette is converted to the value of RGB of the other three types of vision and saved in the table in order to diagnose colorations in the four types of color vision. The RGB value is used in the calculations listed below. In the diagnosis, combinations of colors are determined to be appropriate if they meet certain standards (i.e., differences in brightness, luminance and color). Other combinations are determined to be inappropriate. Color combinations with a color difference of 500 or more and a brightness difference of 96 through 125 are determined that caution be exercised, though they are not inappropriate. In WCAG, when the RGB of two colors is assumed to be (R1, G1, B1) and (R2, G2, B2), if the conditions listed below are met, the two colors are considered to be discriminated from each other concerning brightness and color difference. However, the WCAG standard is too strict and the number of usable colors becomes too small. Thus, in this study, among combinations of colors with a color difference of 500 or more, it was ascertained that the brightness difference can be alleviated to 96 in an experiment, expanding the range of options. In WCAG, Brightness difference, Contrast ratio and Color difference are defined as these expressions (1), (2) and (3) [5]. Brightness difference: ((R1 - R2) * 299 + (G1 - G2) * 587 + (B1 - B2) * 114 ) / 1000 ≥ 126 . Contrast ratio: (L1+0.5) / (L2+0.5) ≥ 5 . Where L1=max(((R1 / 255) ^ 2.2*0.2126 + (G1 / 255) ^ 2.2 * 0.7152 + (B2 / 255) ^ 2.2 * 0.0722), ((R2 / 255) ^ 2.2 * 0.2126 + (G2 / 255) ^ 2.2 * 0.7152 + (B2 / 255) ^ 2.2 * 0.0722) L2=min(((R1 / 255) ^ 2.2 * 0.2126 + (G1 / 255) ^ 2.2 * 0.7152 + (B2 / 255) ^ 2.2 * 0.0722), ((R2 / 255) ^ 2.2 * 0.2126 + (G2 / 255) ^ 2.2 * 0.7152 + (B2 / 255) ^ 2.2 * 0.0722) Color difference: (max (R1, R2) – min (R1, R2)) + (max (G1, G2) – min (G1, G2)) + (max (B1, B2) – min (B1, B2)) ≥ 500 .
(1)
(2)
(3)
6
M. Anse and T. Tabe
Fig .1. shows the methods of the diagnosis to appropriate combinations of colors.
Fig. 1. The flow of the diagnosis to appropriate combinations of colors
3.3 “Display” Function First, colors selected by the color palette are judged whether they are appropriate or not. Second, the vision of color deficient observers is simulated. Third, the coloration is determined to be appropriate or not. Finally, coloration of the selected area is replaced with a new one on the web page screen being created. The area that was saved is colored by the background color. The other two areas are colored by a new coloration. They are put to the original position to make an image with the new coloration.
4 Tools This study has two types of coloration support tool. One is for the area including characters. The other is the area consisting of only images. The number of target areas is limited to three. If there are four or more areas, this tool must be used repeatedly. Each tool has tabs for each area. A tab has a color palette which displays whether the
Development of a Coloration Support Tool for Making Web Page Screens User-Friendly
7
coloration is appropriate or not. A user can select combinations of colors which are deemed to be appropriate from the color palette or select colors from the screen by using the dropper function. A user can understand the vision parameters of selected combinations of colors by the color deficient observers. He/she can understand whether the coloration on the selected areas is appropriate or not by the four vision types. Selected areas can be colored with the coloration. 4.1 Tool for Characters A user selects an area including a character string and background to be colored. Then, the user calls up the coloration support tool. The selected area is displayed on the tool. The coloration support tool has three tabs. The first tab is for the color of the characters. The second tab is for the background color which surrounds the color of the characters. The third tab is for the color which surrounds the background color. A user can specify a color from 216 colors from the color palette on the tab. A user can also use a dropper to select a color from the screen being created. After selecting a color, a user can display the vision seen by a normal color observer, protanope, deuteranope and tritanope by the “Reproduce defective color vision function.” The “Diagnose” function displays the evaluation (i.e., circle, triangle and cross) of the total color combination by the color vision. The “Reflect on screen” function draws the selected color combination on the selected area on the web page screen. Fig. 2. shows the main and sub screens of the character coloration support tool.
Fig. 2. Character coloration support tool
8
M. Anse and T. Tabe
4.2 Graphic Drawing Tool A user selects an area which he/she wants coloration support. Then, the user calls up the graphic drawing coloration support tool. The user selects the background color, and first and second color. Then, visibility of the two specified colors by the four color vision types is displayed on the tool. The originally selected image is displayed in a separate window. The window has three buttons denoting the defective color combinations above them. Pressing the button displays a window with the coloration corresponding to the “visibility” of each color combination. The user selects a tab and a new color to exchange with the first color. Then, selecting the second color tab displays compatibility with the new first color using symbols (no symbol, triangle and cross). The user selects a color other than the colors with the cross symbol. Presses “See all” to open a window with the selected coloration. Presses buttons in the window to check the vision by the color deficient observers. After determining a satisfactory combination, presses “Diagnose” to check the vision by the four color visions. If all color visions have no problem (circle), presses “Reflect on screen” to re-draw the coloration of the original graphic. Fig. 3. shows the main and sub screens of the Graphic coloration support tool.
Fig. 3. Graphic coloration support tool
Development of a Coloration Support Tool for Making Web Page Screens User-Friendly
9
5 Verification of Effectiveness A test was performed to draw graphics which can be discriminated by color deficient observers by using the character coloration tool and graphic drawing coloration tool. Subjects included 20 university students (males: 8, females: 12). Subjects were made to understand color vision deficiency by reading a manual. Then, they were ordered to make two screens by using the tools. After the test, the vision by color deficient observers was displayed by using a color vision simulator to check whether coloration that could be discriminated by color deficient observers was made or not. Two color deficient observers were ordered to view the screens made by all the subjects. It was proven that the coloration had no problem. Therefore, all subjects could make graphics which could be discriminated by color deficient observers. All subjects answered that the tool had supported them. Some subjects answered that the tool was difficult to use. As a result, the functions selected for developing this “web page screen preparation support tool in consideration of color deficient observers” are appropriate.
6 Conclusions We are not really aware that color deficient observers account for approximately 5% of the total population. It is understood that normal color observers can enjoy rich expressions of various color while color deficient observers cannot enjoy them as readily. As W3C proposed, we should be careful not to express information that is dependant solely on colors. We should use easy tools to make coloration in consideration of color deficient observers, especially when creating highly public information websites. For that purpose, we should develop tools to support coloration in combination with existing software. Acknowledgements. The author sincerely appreciates Ms. Saori Azuma and Ms. Kazuko Sato who graduated in 2007 from the Department of Industrial and Systems Engineering, College of Engineering and Science, Aoyama Gakuin University for their considerable contribution to this study.
References 1. 2. 3. 4.
Ikeda, M.: Basic Color Engineering. Asakura shoten, Tokyo (2000) Fukami, K.: Color Discrimination. Kanehara-shuppan, Tokyo (2003) UDing, http://www.toyoink.co.jp/ud/index.html Color Selector, http://jp.fujitsu.com/about/design/ud/assistance/ colorselector/ 5. Techniques For Accessibility Evaluation And Repair Tools W3C Working Draft, April 26 (2000), http://www.w3.org/TR/AERT
The Persuasive Effects from Web 2.0 Marketing: A Case Study Investigating the Persuasive Effect from an Online Design Competition Asle Fagerstrøm1 and Gheorghita Ghinea2 1
The Norwegian School of Information Technology, Schweigaardsgt. 14, 0185 Oslo, Norway
[email protected] 2 School of Information Systems, Computing and Mathematics, Brunel University, Uxbridge UB8 3PH, London, United Kingdom
[email protected] Abstract. This case study investigates the effect from a Web 2.0 campaign, an online design competition, accomplished by a company that produces and marketed feminine care products (sanitary towels). The target segment for the campaign was girls in four Nordic countries, in the age between 14 and 25. The main characteristic for the target segment is that they are not much interested in the product category. Our interpretation is that the online design competition had a persuasive effect towards the target segment. By using the internet in an interactive and social way, companies can achieve brand awareness and create a positive attitude towards a brand in low-involvement segments. Suggestions for further research are given. Keywords: Web 2.0, Interactivity, Persuasion, Involvement, Interactive marketing.
1 Introduction Web 2.0 can be used to do what traditional advertising does: to push information to persuade consumers to buy products or services. For example, a company may implement a blog on its web site and regularly publish information about products and their benefits. However, according to Parise et. al. [1] that kind of approach misses the point of Web 2.0. Instead, companies should use the Web 2.0 tools to get the consumers involved. To investigate to what extent Web 2.0 marketing has the ability to influence the target segments’ intentional effort, we used the elaboration likelihood model (ELM) of persuasion as a guide to data analysis and interpretation. The ELM, developed by Petty et al. [2], is based on the idea that attitudes are central in guiding the consumer’s decisions and other behaviors. While attitudes can result from a number of cues in the consumers setting, persuasion is a primary source. The ELM framework suggests that important variations in the nature of persuasion are a function of the likelihood that receivers will engage in elaboration of (thinking about) information relevant to the persuasive issue [2]. M.J. Smith and G. Salvendy (Eds.): Human Interface, Part II, HCII 2009, LNCS 5618, pp. 10–16, 2009. © Springer-Verlag Berlin Heidelberg 2009
The Persuasive Effects from Web 2.0 Marketing
11
By investigating the persuasive effect from interactive and social campaign companies could better understand why Web 2.0 communication works, and as a result increase their benefits from marketing activities online. This paper is structured as follows: In the first section we give a presentation of present studies on the effect from online interactive and social marketing. Second, there will be a short presentation of the ELM framework. Third, based on a case study we discuss the persuasive effect that the Web 2.0 marketing may have had on the target segment. Finally, our last section contains concluding comments on the use of Web 2.0 activities to influence consumers’ intention to purchase by means of interactive and social marketing. Suggestions for further research are given.
2 Related Work Web 2.0 is a term that is used to describe changing trends in the use of World Wide Web technology and web design. The term was introduced for the first time in 2004 by Dale Dougherty, a web pioneer and O´Reilly VP, at a conference brainstorming session between O´Reilly and MediaLive International. Web 2.0 is, according to Tim O´Reilly, the business revolution in the computer industry caused by the move to the Internet as platform, and an attempt to understand the rules for success on that new platform. The rules for success entail using the Internet in a interactive and social way [3]. Most authors focus on describing the way of using Web 2.0 in interactive online campaigns in general [e.g. 4]. However, little has been done to explain the persuasive effect that Web 2.0 marketing has toward consumers. Hoffman and Novak [5] point out that the Web frees customers from their traditional passive role as receivers of marketing communications, gives them much greater control over the information search and acquisition process, and allows them to become active participants in the marketing process. The unique forms of interactivity, “machine interaction” and “personal interaction” respectively, have contributed to a rapid diffusion of the Web as a commercial medium in the last several years [5]. Interactivity can be conceptualized from various scopes. For example, Ghose and Dou [6] conceptualize interactivity from a marketing perspective, identifying 23 functions of interactivity mainly driven by communication-based conceptualization of interactivity. Another example is Ha and James [7] who conceptualize interactivity from an interpersonal communication perspective focusing on interactivity as communication, either through a medium or without the aid of a medium. The latter conceptualization of interactivity will be used in this paper as basis for discussion and interpretations. Some studies have examined the effect of interactivity on companies’ web sites. Accordingly, Coyle and Thorson [8] did an experiment on the effect of interactivity and vividness on commercial web sites. Results show that perception of telepresence (simulated perception of direct experience) grew stronger as levels of interactivity and levels of vividness in web sites increased. In a study on the attractiveness of a web site, Ghose and Dou [6] found that the greater the degree of interactivity, the more likely it is for the company’s web site to be considered as a top site. In addition, they find that the “customer support” component of interactivity has a significant positive
12
A. Fagerstrøm and G. Ghinea
impact on the likelihood of a company’s web site being included in a list of highquality web sites. Interesting studies have been done on the effect from interactivity. On the other hand, none of these studies investigate the persuasive effect that interactivity has on the consumer’s intention to purchase a brand.
3 The Elaboration Likelihood Model According to O`Keefe [9], persuasion can be defined as: “a successful intentional effort at influencing another’s mental state through communication in a circumstance in which the persuaded has some measure of freedom.” The ELM, developed by Petty et al. [2], is a theory that proposes a global view of how attitudes are formed and changed. The basic idea is that the receivers (e.g. online consumers) will vary in the degree to which they are likely to engage in elaboration (thinking about) of information relevant to the persuasive issue. For example, when information becomes more personally relevant, consumers are willing to engage in extensive issue-relevant thinking. They will pay attention to a specific web site or an online campaign, evaluate thoroughly the information that is presented, and, recall from memory other issues that are relevant to the specific situation. However, sometimes when the information is not personally relevant, consumers will not undertake much issuerelevant thinking and display relatively little elaboration. The degree to which consumers engage in issue-relevant thinking forms a continuum, from cases of extremely high elaboration to cases of little or no elaboration. The ELM suggests that important variations in the nature of persuasion are a function of the likelihood that receivers will engage in elaboration of information relevant to the persuasive issue. Two types of persuasion process can be engaged depending on the degree of elaboration [2]: a central route involving systematic cognitive thinking and a peripheral route involving cognitive shortcuts. The central route to persuasion represents the persuasion processes involved when elaboration is relatively high, and persuasion is achieved through the consumer’s thoughtful examination of issue-relevant considerations. The peripheral route is the persuasion processes involved when elaboration is relatively low. When persuasion is achieved through peripheral routes it usually comes about because the consumer uses some simple decision rules to evaluate the advocated position. For example, the consumer may be guided by whether they like the color or the design of the web site. That is, the consumer may rely on peripheral cues as guide to attitude, rather than to engage in extensive issue-relevant information processing.
4 Case: An Online Design Competition Our research has an inductive approach which indicates that we gather empirical data without having a hypothesis in advance. Because it is difficult to separate the object to be studied from its context, we found that a case study was an appropriate method to use [see 10]. The rationale for this choice is also that the case study is suitable for investigating up-to-date processes or behaviors of others, which happen in their reallife context but are little known [11]. The key to successfully designing a case study is
The Persuasive Effects from Web 2.0 Marketing
13
to have developed beforehand a theoretical proposition to guide data collection and data analysis [10]. A company that had accomplished a Web 2.0 campaign was chosen. The company produces and marketed feminine care products (sanitary towels) in 85 countries worldwide. In Europe the products occur under the brand name Libresse™. Besides demographics (gender and age) the company reported that there are two main characteristics of the target segment for feminine care products: first the consumer often stick to the brand she decides to choose first time, and, second the consumer is not much interested in the product category (sanitary towels). This is a challenging situation for the marketing department of Libresse™. To be able to achieve brand awareness and attitude towards the brand the company accomplished additional research to better understand the target segment. Results from the market research (survey and focus groups) show that the target segment has varying interests. However, fashion design was one of the most dominant interests reported by the respondents. As many as 25 % of the respondents, age 14 to 25, reported that they want to work with fashion and design. The owner of Libresse™ sees this as an opportunity for their 2007 campaign. A two month online design competition was created as the main communication activity in the campaign toward the Nordic segment (Iceland was not included). The other communication channel that was part of the campaign was print, TV and stores. Libresse™ did also create a package design exclusively for the online design competition. The target segment was invited to design a pair of underpants on the Libresse™ web site. The competition was open for each and everyone but the main target segment was girls between 14 and 25 years of age. Girls who were attracted to the invitation were able to use a design tool to choose color and patterns for their underpants (see figure 1). With help of the drawing program, she could create the submission by choosing between templates, complete figures, and by freehand drawing. The Libresse™ brand name was strategically placed on the top left of the web site. The web site also presented the jury and the attractive prizes. When the consumer was satisfied with the underpants design, she could submit it on the Libresse™ web site and join the design competition. However, she could also
Fig. 1. Design tool on Libresse™ web page
14
A. Fagerstrøm and G. Ghinea
invite friends by Facebook™ to vote for her design underpants. If she didn’t want to design a pair of underpants, she could vote for her favorite pattern, and in addition send it as a postcard. Every week the winner in each country was lining up for the final. The winner of the design competition obtains a sum of money and, even more important, her underpants were launched in 180 JC™ stores around the Nordic countries. So, the dream of being a fashion designer could be realized on Libresse™ web site.
5 Campaign Outcomes One goal that Libresse™ had for the campaign was to increase the number of visitors to each country’s website by 25 %. In total (see figure 2), the number of visitors increased from 277 657 to 483 036, in other words, an increase of 205 379 visitors (+ 74 %). The time consumers spent at the web site also increased by 60 % from approximately 12 minutes to 19 minutes.
Fig. 2. Visitors at Libresse™ web site
The response from the target segments was enormous. As much as 90 000 underpants were designed and submitted to the design competition by girls in the Nordic countries (see figure 3). Sweden had the highest participation with 40 500 submitted underpants. Norway was second with 17 100, and then Denmark and Finland with both 16 200 submitted underpants. Impact-wise, the media impact was extremely strong, especially the digital. Within the blog world, the competition was one of the major topics during the summer 2007. In regard to sale, Libresse™ also witnessed an increase in sale in the campaign period.
The Persuasive Effects from Web 2.0 Marketing
15
Fig. 3. Numbers of submissions in Nordic segments
6 Discussion The increase in sale in the campaign period can be explained by means of the ELM. The ELM is based on the idea that under different conditions, receivers will vary in the degree to which they are likely to engage in elaboration of information relevant to the persuasive issue [2]. The basic idea is that consumers are more likely to carefully evaluate the attributes of a product when the purchase is of high relevance to them. Conversely, the likelihood is great that consumers will engage in a very limited information search and attribute evaluation when the product holds little relevance or importance to them. The target segment of Libresse™ reported that they are not much involved in the product category (feminine care products). Their engagement in information search and attribute evaluation will therefore most probably be limited. As presented in the introduction, ELM makes a distinction between two routes to persuasion [2]: a central and a peripheral route. The central route to persuasion represents the persuasion processes involved when elaboration is relatively high, and the peripheral route represents the persuasion processes involved when elaboration is relatively low. According to Petty et al. [2] attitude changes that occur via the peripheral route occur because the attitude issue or object is associated with positive or negative cues. In the Libresse™ design competition campaign, brand awareness and a positive attitude towards the brand are most probably achieved because the brand Libresse™ is associated with fashion design (positive cues). The peripheral route to persuasion can explain the brand awareness, positive attitude towards the brand, and finally, increase in sale for Libresse™ in the campaign period.
7 Conclusion This case study has demonstrated that the peripheral route to persuasion strategy can be realized with the support of Web 2.0 marketing towards low-involvement
16
A. Fagerstrøm and G. Ghinea
consumers. By using the Internet in an interactive and social way, companies can achieve brand awareness, positive attitude towards a brand, and finally, increase in sales in the target segment. This study is not without limitations. One limitation is its lack of empirical data. Its interpretative design has obvious limitations, especially with regards to internal validity (to what extent the interactive design competition is a cause to the increase in sales). In spite of the limitations, the online marketing implication of the ELM is apparent: when planning an online campaign, companies should consider to what extent the target segments are involved in the product category. A follow up study could be to conduct an experiment which would increase validity regarding causal conclusions. Another follow up study could be to investigate the persuasive effect of Web 2.0 marketing towards high-involvement consumers. To what extent can companies use Internet in an interactive and social way toward target segments that are likely to engage in elaboration of information relevant to the persuasive issue? A third follow up study could be to replicate the present study in different contexts and see if it gives the same results.
References 1. Parise, S., Guinan, P.J., Weinberg, B.D.: The Secrets of Marketing in a Web 2.0 World. The Wall Street Journal (2009) 2. Petty, R.E., Cacioppo, J.T., Schumann, D.: Central and Peripheral Routes to Advertising Effectiveness: The Moderating Role of Involvement. Journal of Consumer Research 10(2), 135–146 (1983) 3. O’Reilly, T.: What Is Web 2.0: Design Patterns and Business Models for the Next Generation of Software. Communication & Strategies, First Quarter (1), 17–37 (2007) 4. Moran, M.: Do It Wrong Quickly: How the Web Changes the Old Marketing Rules. IBM Press, Upper Saddle River (2008) 5. Hoffman, D.L., Novak, T.P., Chatterjee, P.: Commercial scenarios for the Web: Opportunities and challenges. Journal of computer-mediated communication 3(3) (1995) 6. Ghose, S., Dou, W.: Interactive functions and their impacts on the appeal of internet presence site. Journal of Advertising research 38, 29–43 (1998) 7. Ha, L., Lincoln, J.E.: Interactivity reexamined: A baseline analysis of early business Web sites. Journal of Broadcasting & Electronic Media 42(4), 456–473 (1998) 8. Coyle, J.R., Thorson, E.: The Effects of progressive levels of interactivity and vividness in web marketing sites. The Journal of advertising 30(3), 65–77 (2001) 9. O’Keefe, D.J.: Persuasion: Theory & Research, 2nd edn. Sage Publications, Inc., London (2002) 10. Yin, R.K.: Case study research: Design and Methods, 3rd edn. Applied social research methods series, vol. 5. Sage publications, Thousand Oaks (2003) 11. Amaratunga, D., et al.: Quantitative and qualitative research in the built environment: application of “mixed” research approach. Work Study 51(1), 17–31 (2002)
Formalizing Design Guidelines of Legibility on Web Pages Fong-Ling Fu and Chiu-Hung Su Department of Management Information Systems, National Cheng-chi University, Taipei 11605, Taiwan
[email protected],
[email protected] Abstract. Screen design of web pages is challenging because web pages contain lot of icons, consisting not only of texts with various fonts but also graphics with different sizes and content. The objectives for screen design of a web page can be either to provide aesthetic beauty, to convey complex information, to improve legibility, or some combination of the above. This study chooses to formalize design guidelines of legibility because information is becoming more and more complicated in web pages and hampering the efficiency of information searching. This study proposes six measurements of screen legibility: screen ratio of navigator to content, font size variety, variety of icon types, color contrast between background and foreground, content density, and number of alignment points. These six factors were then use to measure the legibility effectiveness of the startup page on four different yahoo.com sites. Combined with the results from a survey study, we concluded that all six factors were validated to be attributes with a significant and measurable impact on web site legibility. Keywords: Guidelines of Web pages, Web pages design, Screen layout design, Legibility design, Complexity measurement.
1 Introduction A Web site is like a big house: it should be firm in its basic structure (e.g. secure and stable), should be functionally convenient (e.g. easy to use), and should be a delight to use [6], [16]. Proper interface design helps all the above needs to be satisfied. Screen design is very challenging because a web page contains lot of icons, which consist of not only texts with various forms but also graphics with different sizes and contents [13], [14]. The objectives for screen design of a web page can be either to provide aesthetic beauty, to convey complex information, to improve legibility, or some combination of the above [14]. Among them, previous studies seem to have been more focused on aesthetics. This study chooses to formalize design guidelines of legibility because information is becoming more and more complicated in web pages and hampering the efficiency of information searching. M.J. Smith and G. Salvendy (Eds.): Human Interface, Part II, HCII 2009, LNCS 5618, pp. 17–25, 2009. © Springer-Verlag Berlin Heidelberg 2009
18
F.-L. Fu and C.-H. Su
2 Experimental Design The objectives of a Web site design are influenced by its tasks [18]. There are three different types of Web sites based on the nature of the tasks: pleasure-oriented (hedonic), productivity-oriented (utilitarian), or hybrid [21], [8], [19]. At one end of the continuum, Web sites that provide solutions to problems and are typically visited out of necessity to effectively support utilitarian tasks [19]. Utilitarian tasks would include many e-shopping behaviors such as information searching, analysis, comparison, and evaluation. In such situations, users would like to accomplish their tasks effectively and efficiently. At the other end of the continuum, Web sites that are experiential, entertaining, and gratifying to the senses effectively support more hedonic tasks such as games[19]. Along the continuum between utilitarian and hedonic Web sites, there are also hybrid Web sites with most e-retailers supporting both task types [19] [21]. The more emphasis placed on hedonic importance, the more influence by aesthetic design of interface. On the other hand, the more emphasis placed on utilitarian importance, the more influence by efficiency on information searching [19]. Therefore, to validate the measurements of legibility, this study selected four startup screens of four Yahoo websites from Japan, USA, Taiwan, and Korea as controls to the same task. The home page of any website can be quite large, but when entering the site, the initial content to display is limited to whatever fits into the space of the user’s screen; this is what is referred to as the “startup screen” in our experiment. The startup screen
Fig. 1. Startup screen and structure model of yahoo.com.jp
Fig. 2. Startup screen and structure model of yahoo.com
Formalizing Design Guidelines of Legibility on Web Pages
19
Fig. 3. Startup screen and structure model of yahoo.com.tw
Fig. 4. Startup screen and structure model of yahoo.com.ka
of the Yahoo.com used in four different countries, and each of their structure models are listed below in Fig. 1 to 4. The unit of web page measured is a ‘block’ which is represented as a rectangle in the structure model [4]. To formulize the Design Guidelines of Legibility on Web pages, we based our experiment on the findings and results of previous research into this topic. A summary of the key findings from previous research is provided below: (1) Legibility of screen ratio (LR) involved the ratio of navigator/content with best performance achieved using a 23/77 ratio [20]. In terms of the magnitude or size of the LR, the closer to the perfect ratio the better the results, as shown by the following analysis: LR = 1/
｜W
nav
- 23
｜ ∈ [0,1]
(1)
where Wnav is the width ratio of navigator to total screen. (2) Legibility of font size variety (LS) is based on the Principle of Economy - a careful and discreet use of display elements to get the message across as simply as possible [3]. LS involved the categorization of elements into groups according to actual physical size and variation in those sizes [5]. LS =
1 nsize
∈ [0,1]
where nsize is the number of sizes and n is the total number of objects.
(2)
20
F.-L. Fu and C.-H. Su
(3) Legibility of density (LD) is the extent to which the screen is filled with objects [4].
∑a LD= 1 − 2 n
i
i
a frame
∈ [0,1]
(3)
where ai and aframe are the areas of object i and the frame, respectively; and n is the number of objects on the frame. (4) Legibility of alignment (LA): In order to achieve simplicity on the screen, the smaller the number of alignment points, the better [17], [9]. LA =
( n vap
3 ∈ [0,1] + n hap + n )
(4)
where nvap and nhap are the numbers of vertical and horizontal alignment points, respectively; and n is the number of objects on the frame [3]. (5) Legibility of icon type (LI): Graphic ingredients enabled considerable presentation enhancements, making screens easier to understand and use [2]. Every icon was perceived as an information unit. Icons with same size and font were perceived as one type of icon. Every graph icon is different from others because graph icons contain different graphs and texts with different size, color, or font. LI =
1 ntype
∈ [0,1]
(5)
where ntype is the number of sizes and n is the total number of objects. (6) Legibility of color contrast (LC): The combination of colors that had higher levels of contrast between background and foreground (BW, YB) generally led to better performance than combinations of lower contrast [7], [12], [1]. Color design is better when simple, not exceeding more than 4 per screen [15]. Also, the impact of color on
Fig. 5. I.R.I 116 Palette [23]
Formalizing Design Guidelines of Legibility on Web Pages
21
visualization depends on the area size which the color is used [10], [11]. Therefore, the formula for calculating LC is as follows: LC =
∑ C × A − ∑ C × A ∈ [0,1] 4
n
i
i
i
5
(6)
i
i
where Ci is the degree of contrast between background and foreground and Ai is the % of icon area in the web page. According to I.R.I 116 palette, the contrast of the hues was classified as either identity, similarity, analogous, contrast, blur, contrast or complementary (fig. 5). The researchers set the degree of contrast Ci as: identity is 0, similarity is 0.2, analogous is 0.4, blur 0.6, contrast 0.8 and complementary 1,
3 Results The indicators of legibility, LR, LS, LI, LC, LD, and LA are calculated based on formulae 1 to 6 as mentioned above, and the legibility results for each of the four web pages are shown in the following six tables. Table 1. LR of four web pages
Japen
USA
Taiwan
Korea
% area of navigator
17
15
16
15
% area of Content
77
79
78
79
0.203
0.142
0.167
0.142
LR
Table 2. LS of four web pages
Japen
USA
Taiwan
Korea
n
145
109
146
146
nsize
40
52
55
66
LS
0.025
0.019
0.018
0.015
Taiwan
Korea
Table 3. LI of four pages
Japen
USA
ntype
47
62
69
81
n
145
109
146
146
LI
0.021
0.016
0.014
0.012
22
F.-L. Fu and C.-H. Su Table 4. LC of four pages
Japen
USA
Taiwan
Korea
C1, W1
0.099
0.095
0.104
C2, W2 C3, W3
0.051 0.017
0.025 0.015
0.035 0.027
0.130 0.026
C4, W4
--
0.006
0.007
C5, W5
--
0.009
LC
0.167
0.132
--
0.012 0.010 --
0.173
0.178
Table 5. LD of four pages
Japen
USA
Taiwan
Korea
270353
243785
281157
976 665
976 665
976 665
288123 976
Σaframe
649040
649040
649040
649040
LD
0.167
0.249
0.134
0.112
Σai area width area high
665
Table 6. LA of four web pages
Japen
USA
Taiwan
Korea
nhap
53
61
71
79
nvap
63
69
75
n nhap + nvap + n
145 261
109 239
146 292
61 146 286
LA
0.011
0.013
0.010
0.010
A summary of the legibility values is shown in Table 7. According to the average of all the legibility values, we consider Yahoo Japan to be the best; it got the highest values in LR (ratio of navigator/content), LS (font size variety), and LI (icon type variety). The Yahoo USA Web page came in second; it got highest values in LD (density) and LA (alignment). Yahoo Taiwan is a little lower than the previous two while Yahoo Korea ranks last.
Formalizing Design Guidelines of Legibility on Web Pages
23
Table 7. Summary of legibility of four web pages
indicator
Japen
USA
Taiwan
Korea
LR
0.203
0.142
0.167
0.142
LS
0.025
0.019
0.018
0.015
LI
0.021
0.016
0.014
0.012
LC
0.167
0.132
0.173
0.178
LD
0.167
0.249
0.134
0.112
LA
0.011
0.013
0.010
0.010
average
0.099
0.095
0.086
0.078
rank
1
2
3
4
Research into information systems has proposed that the solution to managing complexity is to include enough variety in the attributes of the system’s basic elements. By manipulating attributes such as position, size, icon type and color, web elements are allowed to be distinguishable yet grouped. The other important factor is how strongly and in how many different ways these groups can be related to the task at hand, such as density and alignment in the study performed by Xing [22]. Based on the data in Table 7, we consider the Yahoo Japan home page to be the best at providing distinctiveness to the web page elements (block) through variety in position of navigator (index), text size, and icon type. Yahoo USA also provided stronger relations to group the blocks, through density and alignment, than the other websites included in this study.
4 Survey A survey study was conducted to test the robustness of the formulae for legibility using model screens. The subjects who judged the legibility of actual screens comprised of 64 undergraduate student volunteers from a university in Taiwan who each took the course “Introduction of software”. The subjects consisted of 26 males and 38 females; the average age was 29.2 years old, with 35 majoring in business, 1 computer science major and 28 others. The average previous experience on web is 7.8 years. The average web usage per week was 20.6 hours. The questionnaire contained pictures of the Web pages of Fig. 1-4 and questions regarding the rating the legibility of pair on figures. For each pair, viewers assigned a numerical value between one and nine, indicating their perception of the Web pages’ difference in legibility. Value of five meant no difference between the pair. Values from one to four represented “very much distinct clear and legible”, “distinct clear and legible”, “more clear”, and “a little clear” the former compared to the latter respectively. Values from six to nine represent the degree of legibility which the latter is better than the former. The means and standard deviations of viewer’s judgment are listed in Table 8.
24
F.-L. Fu and C.-H. Su Table 8. Statistics of the pair comparisons
Japan Korea Taiwan USA
Japan Mean (S.D.) -2.81 (1.7) 3.32 (1.7) 5.64 (1.7)
Korea Mean (S.D.) 6.19 (1.7) -5.98 (1.7) 7.03 (1.6)
Taiwan Mean (S.D.) 5.68 (1.7) 3.02 (1.7) -6.29 (1.6)
USA Mean (S.D.) 3.36 (1.7) 1.97 (1.6) 2.71 (1.6) --
The values of comparisons in Table 8 indicates that Yahoo Korea is the worst, Yahoo Taiwan is worse than Yahoo USA and Yahoo Japan, but better than Yahoo Korea. Yahoo USA and Yahoo Japan are better than the other two and Yahoo Japan is a little better than Yahoo USA. The order of legibility from best to worst is USA Japan > Taiwan > Korea. The results of viewers’ judgments on legibility almost resemble those obtained using the measures proposed by the above formulae, with the exception of Yahoo Japan. The mean score of legibility of Yahoo Japan was 0.099 and that of Yahoo USA was 0.095. We can ignore the difference between the web pages Yahoo USA and Japan because it is very small. Based on the similar results generated, we conclude that the proposed formulae are valid.
≧
5 Conclusions Web site designers are in need of practical guidelines on how to create and measure effective design. This study proposed six measurements of screen legibility: ratio of navigator to content size, font size variety, variety of icon type, color contrast between background and foreground, content density, and number of alignment points. The former four are the factors that help the viewers to distinguish differences in the blocks on a web page, and the last two help the viewers to connect the groups of blocks to a function or task. All of them are useful for decreasing information complexity. Utilizing these six legibility factors to evaluate display pages of four international yahoo.com sites, we calculated the legibility ranking, in order from most legible to least, to be: USA, Japan, Taiwan, Korea. We then conducted a survey with real human users to verify the theoretical results, finding our theoretical projections to be a validate match to recorded human opinion. Yahoo Japan was effective in providing variety in: position of navigator, font size and icon type. Yahoo USA was effective in providing connections between groups using space and alignment. In the end, we can conclude each of the six legibility factors contains a real and significant impact on web page legibility. As legibility is more important for utilitarian and hybrid web sites, and aesthetics is more important to pleasure-oriented web sites, further studies can help advance understanding of the relationship and complementary nature between aesthetics and legibility.
Formalizing Design Guidelines of Legibility on Web Pages
25
References 1. Bedrogi, P.: Chromaticity Contrast in Visual Search on the Multi-colour User Interface. Displays 24, 39–48 (2003) 2. Ch’ng, E., Ngo, D.C.L.: Screen Design: a Dynamic Symmetry Grid Based Approach. Displays 24, 125–135 (2003) 3. Ngo, D.C.L., Teo, L.S., Byrne, J.G.: Formalising Guidelines for the Design of Screen Layouts. Display 21, 3–15 (2000) 4. Dong, Y., Ling, C., Hua, L.: Effect of Glance Duration on Perceived Complexity and Segmentation of User Interfaces. In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4552, pp. 605–614. Springer, Heidelberg (2007) 5. Fu, F.L., Su, C.H.: Measuring the Screen Complexity on Web Pages. In: Smith, M.J., Salvendy, G. (eds.) HCII 2007. LNCS, vol. 4558, pp. 720–729. Springer, Heidelberg (2007) 6. Kim, J., Lee, J., Han, K., Lee, M.: Businesses as Buildings: Metrics for the Architectural Quality of Internet Businesses. Information System Research 13(3), 239–254 (2002) 7. Ling, J., van Schaik, P.: The effect of text and background colour on visual search of Web pages. Displays 23, 223–230 (2002) 8. Massey, A.P., Khatri, V., Montoya-Weiss, M.: Usability of online services: The role of technology readiness and context. Decision Sciences 38(2), 277–308 (2007) 9. Miyoshi, T., Murata, A.: A Method to Evaluate Properness of GUI Design Based on Complexity Indexes of size, Local Density, Aliment, and Grouping. In: 2001 IEEE International Conference on Man and Cybernetics, vol. 1, pp. 221–226 (2001) 10. Parry, M., Eberle, S.D.: Aesthetic measure applied to color harmony. Journal of the Optical Society of America 34(4), 234–242 (1944) 11. Moon, P., Spencer, D.E.: Area in Color Harmony. Journal of the Optical Society of America 34(2), 93–103 (1944) 12. Näsänen, R., Ojanpäa, H.: Effect of Image Contrast and Sharpness on Visual Search for Computer Icons. Display 24, 137–144 (2003) 13. Ngo, D.C.L., Teo, L.S., Byrne, J.G.: Modeling Interface Aesthetics. Information Science 152, 25–46 (2003) 14. Schenjkam, B.O., Jönsson, F.U.: Aesthetics and Preferences of Web Pages. Behaviour & Information Technology 19(5), 367–377 (2000) 15. Shneiderman, B., Plaisant, C.: Designing the User Interface, 4th edn. Addison Wesley, England (2004) 16. Palmer, J.: Web Site Usability, Design, and Performance Metrics. Information Systems Research 13(2), 151–167 (2002) 17. Parush, A., Nadir, R., Shtub, A.: Evaluating the Layout of Graphical User Interface screens: Validation of a Numerical Computerlized mode. International Journal of HumanComputer Interaction (2005) 18. van Schaik, P., Ling, J.: The effects of screen ratio and order on information retrieval in web pages. Displays 24, 187–195 (2003) 19. Valacich, J.S., Parboteeah, D.V., Wells, J.D.: The Online Consumers Hierarchy of Needs. Communications of the ACM 50(9), 84–90 (2007) 20. van Schaik, P., Ling, J.: The effects of screen ratio and order on information retrieval in web pages. Displays 24, 187–195 (2003) 21. Van der Heijden, H.: User Acceptance of Hedonic Information Systems. MIS Quarterly 28(4), 695–704 (2004) 22. Xing, J.: Measures of Information Complexity and the Implications for Automation Design, National Technical Information Service, Springfield, Virginia (2004) 23. I.R. I 116 Palette, http://www.iricolor.com/04_colorinfo/sensetest.html
The Assessment of Credibility of e-Government: Users’ Perspective Zhao Huang, Laurence Brooks, and Sherry Chen School of Information System, Computing and Mathematics, Brunel University, Uxbridge, Middlesex UB8 3PH, UK {zhao.huang,Laurence.brooks,Sherry.chen}@brunel.ac.uk
Abstract. Electronic government is increasing worldwide; however there are still some problems which influence users’ interaction with them. One of these problems is trustworthiness, which appears to be affected by whether e-government websites demonstrate their credibility. This study uses an empirical approach to evaluate credibility of e-government websites, especially at the local level in the UK. The evaluation consists of three steps: free interaction, task-based interaction and questionnaire. The results indicate that the majority of credibility problems are related to “site easy use”, “site looks professional” and “site update”. The value of the study is that it provides guidance for designers to improve the credibility of e-government websites. Keywords: e-government website, credibility, web-based online systems.
1 Introduction With the rapid development of the Internet, users have increasingly been able to interact with Web-based online systems. Among a variety of Web-based online systems, electronic government (e-government) is becoming part of the revolution applied in the public sector. Nowadays, thousands of e-government websites are widely accessible via the Internet. Such a rapid growth arises from the way that e-government websites have the potential to change the working environment of the traditional government to enhance access and delivery of government services [1]. However, e-government is still facing a big challenge to interact with users. Trustworthiness can be seen as the underlying catalyst for e-government adoption [2]. With higher trustworthiness, users can overcome perceptions of risk and uncertainty in the use and acceptance of online systems [3]. Trustworthiness can be affected by whether online systems demonstrate their credibility [4]. In general, credibility refers to reliability, accuracy, authority and quality [5]. Regarding e-government website particularly, credibility can be enhanced by the site look, information quality and readability [4]. This suggests that there is a need to consider credibility when developing e-government websites. By doing so, egovernment websites can be accepted by a wider range of users. As such, credibility evaluation of e-government website becomes crucial in order to develop user-centered e-government. However, existing research has not paid much attention to evaluating credibility. To this end, the paper aims to assess M.J. Smith and G. Salvendy (Eds.): Human Interface, Part II, HCII 2009, LNCS 5618, pp. 26–35, 2009. © Springer-Verlag Berlin Heidelberg 2009
The Assessment of Credibility of e-Government: Users’ Perspective
27
credibility of e-government websites, particularly focusing on the local level in the UK. To carry out the evaluation, an empirical study has been conducted based on Fogg’s credibility guidelines [6]. Accordingly, the paper is presented as follows: Section 2 presents the theoretical background from literature to demonstrate the importance of credibility to e-government website. In section 3, an empirical study is designed to evaluate credibility of e-government websites. This allows the detection of credibility problems which are discussed in section 4. Finally, conclusions are drawn and possibilities for future study are recommended in section 5.
2 Theoretical Background: e-Government Websites Governments worldwide have caught on to the revolution in Internet and Web technologies and made significant attempts to deliver their services to citizens, business and other government agencies via the Internet in the public sector [9]. Generally, e-government is the use of the Internet, as a tool to achieve better government, which enables richer information resources, higher quality services and greater participation [10]. All services of e-government can be produced via information presentation, interaction, transaction and integration [11]. The benefits of e-government can increase transparency; service delivery; civil service performance; policy effectiveness; strengthen citizen trust and achieve big cost savings [10]. Equally, the number of e-government website worldwide has increased from 142 in 1995 to more than 50,000 in 2001 [8]. In the U.S., more than one million users visit federal websites every week [12]. A survey reports that 60% of respondents prefer to choose e-government to deal with their demands [13]. This suggests that there are a large number of users who are using and have been willing to engage in e-government website services during the last decade. However, with the rapid growth of e-government websites, there is still a big challenge for e-government to interact with users. Sillence et al. [14] found that trustworthiness is a key factor in user decisions about website engagement. In other words, the success of e-government websites will largely depend on reliability [15]. Further, a range of studies demonstrate and adopt credibility to explain interaction between users and information system. In these, credibility of the website directly results in users’ satisfaction [7]. The evidence indicates that credibility is an important factor which influences users’ interaction with system. 2.1 Credibility Credibility is a highly complex concept. A simple definition is “believability” [6]. Many studies attempt to identify multiple criteria in credibility assessment. Berlo, et al. [16] extend believability by adding “safety,” “qualification,” “dynamism,” and “sociability”. Robins and Holmes [17] argue that “sociability” is a strong factor relating to credibility. Within web-based systems, Rieh [5] indicates that except for “trustworthiness” and “dynamism”, “authority” can be used instead of “expertise”. However, Toms and Taves [18] argue “reputation” as a comprehensive factor which includes “trustworthiness”. Fogg [19] defines credibility as consisting of trustworthi-ness and expertise. Trustworthiness is the users’ perception in terms of goodness and the unbiased nature of
28
Z. Huang, L. Brooks, and S. Chen
the system [19]. Expertise is seen as the users’ perception regarding knowledge and skill with the system resources [6]. Based on this concept, the major problems of credibility lie within “Aesthetic design”; “Information structure”; “Information focus”; “Company motive”; “Usefulness of information” and “Accuracy of information” [19]. It seems that credibility is a key factor in determining the success of e-government [4]. Users will not fully accept, satisfy and interact with e-government until credibility issues have been addressed in sufficient details in e-government design. Thus, there is a need to evaluate credibility in current e-government in order to develop more user centered e-government.
3 Methodology Design 3.1 Conceptual Framework Having demonstrated the importance of credibility to e-government website, this paper reports an empirical study to evaluate credibility of e-government websites. Firstly, a set of tasks is developed for participants to perform (Section 3.2). Subsequently, a guideline-based questionnaire is designed to capture participants’ perception about credibility (Section 3.3). Then, three local e-government websites in U.K. are selected to be evaluated (Section 3.4). The evaluation procedure composes of three steps: a free interaction; a task based interaction and questionnaire (Section 3.5). Once the evaluation is completed, a score is assigned to each questionnaire question to indicate the seriousness of credibility problems. Finally, the data collected is presented and discussed (Section 4), and these findings can be used to help credibility design in further development of e-government websites (Section 5). 3.2 Task Design The study aims to detect the credibility problems of e-government websites. In the evaluation, the participants are required to perform a set of practical tasks on e-government websites. These tasks are representative activities that users would be expected to perform. Generally, three characteristics are used to categorize the services on e-government websites: information dissemination, products offer and user participation [8]. Information dissemination is the presentation of information. Products offer refers to one-way delivery services, such as forms download; registration; job searching. User participation is regarded as the interactive services which involve users in two-way communication, for example, participating in government decision making. 3.3 The Guidelines Based Questionnaire Design To measure credibility, a questionnaire based on Fogg’s credibility guidelines [6] is used to capture participants’ perception. Three phases are constructed of the questionnaire design. Firstly, there is a need to extend existing guidelines to meet the specific requirements of e-government websites. Secondly, since Fogg’s guidelines are broad principles which can not prescribe a step by step approach to cover specific elements, it is important to develop associated criteria for each guideline. Finally, the
The Assessment of Credibility of e-Government: Users’ Perspective
29
questionnaire consists of a series of questions which are developed based on associated criteria from the extended Fogg’s guidelines. Extension of Guidelines. Fogg’s set of guidelines (see Table 1) is used as a benchmark for assessing credibility [7]. However, as Fogg’s ten guidelines were developed 10 years ago and for general websites, it is important to extend these guidelines in order to fit the specific needs of e-government websites. As e-government serves the public, users’ participation is given more attention [11]. Barnes [20] demonstrates that the service interaction quality is the main factor in users’ satisfaction. Furthermore, Garcia et al. [15] use privacy and transparency as criteria to evaluate e-government Therefore, the existing guidelines are extended by adding three new guidelines: Transparency; Service agility and Privacy (see Table 2). Table 1. Fogg’s credibility guidelines [6] Credibility Guidelines G1. Site looks professional G2. Information accuracy G3. A real organization behind site G4. Highlight the expertise in site G5. Show the trustworthy people G6. Make it easy to contact you G7. Site ease to use and useful G8. Update site’s content often G9. Restraint promotional content G10. Avoid errors of all type
Explanations The site considers layout, typography, consistency The site shows the validation of the materials The site needs to prove it is a legitimate organization The site indicates an expert team and authority services The site shows people who convey trust through the site The site provide clear contact information at any time Users can easily complete their tasks using the site The site is up to date and review its content regularly The site should clearly distinguish sponsored content The site prevents a problem occurring in the first place
Table 2. Extended Guidelines Extended Guidelines G11. Transparency G12. Service agility G13. Privacy
Explanations The site should keep users informed a clear governmental operations The site should provide flexible services to fit in different user paths The site should protect users’ information and secure its services
Development Criteria. Although Fogg’s guidelines are extended, it may encounter difficulties to assess detailed aspects of credibility in questionnaire. Therefore, associated criteria for each guideline need to be developed. The precise criteria for each guideline may provide a step to step process to closely focus on specific aspects so that the specific question in the questionnaire can be exactly designed based on those criteria. Credibility Questionnaire. A questionnaire ensures that all participants are asked the same questions and provide quick responses. Therefore, a questionnaire is created to assess credibility. The participants will be asked to complete a set of questions using five-point Likert scales, which presents their level of agreement to those statements. The main advantage of five-point scales is that an odd number of response formats with a neutral level in the middle may not force the participants to choice a positive or negative option when they really do not have. The other advantages include gathering
30
Z. Huang, L. Brooks, and S. Chen
respondents’ opinion by summing up the participants’ responses to the same latent variable and the options are approximately equal spaces across the continuum of approval [21]. 3.4 e-Government Website Selection The evaluation is conducted using three local e-government websites in the U.K. so that evaluation results can be comparatively analyzed. The reason behind choosing the local level to evaluate is that the local e-government website is the closest level for citizens, frequently used by the general public and an important role for citizens’ participation [22]. In addition, previous studies have found the various problems in local e-government. For example, Yang and Paul [23] detect that the majority of problems in local e-government lie within security and content update issues. The three local e-government websites will be called: London Authority1, London Authority2 and London Authority3. 3.5 Experimental Evaluation Procedure To conduct the experiment, 30 participants are assigned to evaluate three e-government websites (10 participants for each site). Each evaluation follows the same three steps: free interaction; task-based interaction and questionnaire. Firstly, the participants freely look at the e-government website so that a general perception and initial interaction are developed. Subsequently, task-based interaction requires participants to complete a number of practical tasks. User scenario is used as the technique to translate and present the selected tasks to the participants. Having completed all tasks, the participants communicate their perceptions of credibility in the questionnaire.
4 Discussion of the Results To obtain a comprehensive evaluation, both quantitative and qualitative approaches are used to analyze the results. The former uses the numeric results from the closed questions of the questionnaire to identify the overall credibility of each e-government website (Section 4.1), while the latter presents the useful features and problems from open-ended questions to indentify the strengths and weaknesses of credibility (Section 4.2). 4.1 Quantitative Measurement In order to comparatively evaluate the results, a score has been assigned to each question in the questionnaire. This has been calculated by multiplying the number of answers under each weight of five-point scale. Subsequently, the sum of these calculations is added together to obtain the score for each question. Lower scores indicate the most serious credibility problems (see Table 3). By assigning the scores in this way, it is easy to identify the most serious problems in each e-government website. More importantly, it can comparatively highlight the specific categories of credibility that cause the most common credibility problems among the e-government websites evaluated.
The Assessment of Credibility of e-Government: Users’ Perspective
31
Overall, the three e-government websites appear to be clear and fairly straightforward. Different parts of information are properly displayed in the various formats so that users can easily read information presented, which can also be quickly accessed. The main menu is suitably located on the left side of the page and the quick links are always available at the top of each page. It helps users to easily start the tasks and quickly access shortcuts whenever they need. In addition, a clear logo and staff photos add to the credibility of the site. Meanwhile, other reliable government websites are presented and linked with the target e-government website. This helps to improve the reputation of the site. After that, the navigation descriptions are useful to support users identify where they are in the site, with advertisements restrained to help users’ concentrations. However, some serious credibility problems have been found in each egovernment website evaluated (Table 3). In London Authority 1, firstly, information is not consistently presented in the different colors, so users feel a little confused when they try to identify information through the site. Secondly, it dose not offer concise instruments or messages to support user selection of sub-options correctly. Subsequently, although advertisements are restrained, there are still some difficulties in distinguishing ads from content. Finally, the structure of the site is quite unclear, massive number of option makes site looks less credible. The most serious problems in London Authority 2 are (see Table 3): firstly, information may not be presented in consistent colors, so users may spend extra time to identify information. Secondly, any awards earned by the website are not displayed properly. Thirdly, the site may need to improve its usability. Fourthly, the information about latest updates is not clearly indicated, especially for online forms and documents. Lastly, the site lacks a secure message to keep users informed during the data transaction process. The problems in London Authority 3 indicate the similar difficulty of information identification by colors. Moreover, some information lacks a good balance between breadth and depth. During online transactions, the site does not clearly indicate progress which may cause users to lose their patience easily. Lastly, privacy needs to be considered, such as personal information protection and secure message presentation. The quantitative results indicate that the most common credibility problems of the three e-government websites are related to “site ease to use”, “site looks professional” and “site update”. For example, in “site ease to use”, users feel difficulty with the links because the links already visited are not clearly marked. This suggests that more attention needs to be drawn to information access and navigation [8]. There is also a lack of balance in information presentation between breadth and depth. This suggests that designers need to consider different users’ tendencies in searching information. More specifically, if users want an overview of information, more categories with fewer levels can be used. Conversely, if users search for detailed information, fewer categories with more levels are needed. In addition, users expect a clear indication of progress in the process. This suggests that user control over the process is paramount. With regard to “site looks”, users find it hard to identify relevant information. The reasons may be firstly layout, instruction and colors are not used consistently throughout the site. Secondly, some information can not exactly match with its
32
Z. Huang, L. Brooks, and S. Chen Table 3. Crediblity problems
London Authority 1 Guidelines Questions 2 G1 4 G2 6 G4 14 G6 20 21 22 23 G7 24 G8 25 G9 27 G10 30 G12 37 London Authority 2 Guidelines Questions G1 2 15 G5 16 22 23 G7 24 G8 25 G13 39 London Authority 3 Guidelines Questions G1 2 G3 10 16 G5 17 G7 23 G8 25 38 G13 39
Problems Information is not identified in the different colors Page is not labeled to show its relation to others The information is not well organized Instruments displayed by the system are not concise The site does not show detailed contact information The system is not easy to use Navigating the system is not easy It is not clear how far left of the quote process The site is not in a way that makes sense to user The system has not a latest update It is not easy to distinguish ads from content The site is not free from typographical errors The site does not offer agile functions Problems Information is not identified in the different colors It is not easy to find an “about us” page The site does not display any awards it has owned Navigating the system is not easy It is not clear how far left of the quote process The site is not in a way that makes sense to user The system has not a latest update A secure message is not appeared Problems Information is not identified in the different colors The site does display photos of offices or staff The site does not display any awards it has owned No enough information about who is in charge of It is not clear how far left of the quote process The system has not a latest update Confidential areas are not secure A secure message is not appeared
Top five lowest scores 31 36 35 33 35 36 36 35 35 35 34 36 36 Top five lowest scores 26 29 25 26 26 27 25 28 Top five lowest scores 31 31 26 28 31 26 25 29
categories; especially where the pictures are too vague to make sense. This suggests that a higher aesthetic treatment is needed to increase credibility [17]. Regarding “site update”, users find difficulty with identifying updated versions for forms, documents and system. In addition, users worry about privacy and security associated with personal information and safety. The possible solution is widespread adoption of digital certificates and a public key infrastructure [20]. On the other hand, by summing the scores together for each e-government website, an overall assessment can be made of the credibility problem of each e-government website. In details, London Authority 2 has the lowest score with a total score of 1305. London Authority 3 is placed next one with a total score of 1386. The egovernment website with the fewest credibility problems is found to be London Authority 1 which has a total score of 1470.
The Assessment of Credibility of e-Government: Users’ Perspective
33
4.2 Qualitative Assessment During the evaluation, comments about the problems and the good features which are not covered in the questions are recorded. The most frequently cited useful features are regarded as strengths and the most frequently encountered problems are considered as weaknesses (Table 4). Table 4. Qualitative results for London Authority 1,2 amd 3
1G 2G 3G 4G 5G 6G 7G 8G 9G 10G 11G 12G 13G
London Authority1 Strengths Weaknesses Consistent Too many colors options Content is Irrelevant reliable Pictures Democracy Irrelevant offered info. Full info. Too much info. Staff photos Confused options General No detailed contact contact Easily Too many access options None No clear update Few ads None Clear Message is not categories concise Terms, None conditions User’ path Search is limited A sign-in None offered
London Authority2 Strengths Weaknesses Logo, Flashed format images None Unclear pictures Relevant None contents Lots of No user ID info. required Staff No award photos presented Relevant None contact Easily Poor category access None No updated date No ads None FAQs is Site map is not provided helpful None No term, conditions None No secure messages None No password required
London Authority3 Strengths Weaknesses A clear loge Color is not constant Detailed info. Unclear subhead Right URL None None
No services feedback None No sources of news Quick contact None Links work properly None
Search engine is limited No site update
No ads None
None Unclear categories No transparency
None None None
No languages support None
Common Strengths • Site looks professional: generally, the logos are used consistently. In addition, the content designed can properly match the organization and be organized logically. Information can be easily read in the sites, which can also be quickly accessed. • Easy to contact: a fixed contact location on each page. Users can quickly find general contact information whenever need. Most contact information is categorized by different departments and offer various contact methods. • Easy to verify the information accuracy: all e-government websites offer much information. Most information is presented at the right level of detail. Furthermore, the URL can properly match the content presented on each page. In particular, the news on the sites indicates their source and the latest update with a clear date.
34
Z. Huang, L. Brooks, and S. Chen
• Restraint with promotional content: in e-government websites, most advertisements are restrained to help users’ concentrations. Common Weaknesses • Highlight expertise in the organization: some information can not be matched to categories, especially for pictures. Users easily lose patience if they choose irrelevant categories. Moreover, the instruments for options are not concisely explained. • Easy to use: users often get confused where links already used are not clearly marked and the key links are not located in an important place. Additionally, too much information is presented at the home page so that users have to spend more time reading it carefully. In particular, users feel it is difficult to find information because the site search engine can not support advanced search. • Site update: not all information is updated regularly with a clear date, especially for forms and documents. Furthermore, the updated date and version of the website is not indicated at the main page.
5 Conclusion This paper reports an empirical study which assesses credibility in existing egovernment websites, focusing on three local e-government websites in the UK. The evaluation results indicate that the current e-government websites have a much room for improvement of their credibility. The most significant problems are found within the areas of “site easy to use”, “site looks professional” and “site update”. These evaluation results suggest that designers need to pay more attention to credibility in egovernment website design so that e-government websites may be more widely accepted and accessed, ultimately to achieve user centered e-government. Therefore, this study develops an approach to identify credibility problems of e-government websites. The value of this study not only lies within the guidance for designers to improve the credibility of e-governments, but also help designers of other web-based systems to enhance their credibility. However, this study uses a questionnaire-based approach to evaluate credibility, which emphasizes on user perception. In order to obtain a more comprehensive evaluation, both user perception and user performance are recommended. Future studies are needed to assess users performance with the tasks In addition, the results of this study indicate that usability difficulties have an important impact on credibility. Designers who enhance the usability of a web site are likely to enhance the site’s credibility [19]. This suggests that there is a need to conduct future research to evaluate usability of e-government website. The findings of such a study can be used to analyze the relationship between usability and credibility.
References 1. Basu, S.: E-government and developing countries: an overview. International Review of Law Computers & Technology 18(1), 109–132 (2004) 2. Warkentin, M., Gefen, D., Pavlou, P.A., Rose, G.M.: Encouraging citizen adoption of egovernment by building trust. Electronic Markets 12(3), 157–162 (2002)
The Assessment of Credibility of e-Government: Users’ Perspective
35
3. Pavlou, P.A., Gefen, D.: Building effective online marketplaces with institution-based trust. Information Systems Research 15(1), 37–59 (2004) 4. Araujo, M.C.R., Grande, J.I.C.: Performance in e-government: website orientation to the citizens in Spanish Municipalities. In: Proceedings of european conference on egovernment, Trinity College, Dublin (2003) 5. Rieh, S.Y.: Judgment of information quality and cognitive authority in the web. Journal of the American Society for Information Science and Technology 55(8), 743–753 (2002) 6. Fogg, B.J., Tseng, H.: The elements of computer credibility. In: Proceedings of the CHI 1999 conference on human factors and computer system, pp. 80–87 (1999) 7. Grady, L.O.: Future directions for depicting credibility in health care web sites. International Journal of Medical Informatics 75, 58–65 (2006) 8. Kumar, V., Mukerji, B., Butt, I., Persaud, A.: Factors for successful e-government adoption: a conceptual framework. Electronic Journal of E-Government 5(1), 63–76 (2007) 9. Tambouris, S.: European cities platform for online transaction services. In: Proceedings of the European Conference on E-Government (2001) 10. OECD: The E-government imperative. In: OECD E-Government Studies. OECD, Paris (2003) 11. Tapscott, D.: Blueprint to the Digital Economy. McGraw-Hill, New York (1998) 12. Eschenfelder, K.R., Beachboard, J.C., McClure, C.R., Wyman, S.K.: Assessing U.S. federal government websites. Government Information Quarterly 14(2), 173–189 (1997) 13. James, G.: Empowering bureaucrats. MC Technology Marketing Intelligence 20(12), 62– 68 (2000) 14. Sillence, E., Briggs, P., Harris, P., Fishwick, L.: A framework for understanding trust factors in web-based health advice. International Journal Human-Computer Studies 64, 697–713 (2006) 15. Garcia, A.C.B., Maciel, C., Pinto, F.B.: Electronic government: a quality inspection method to evaluate e-government sites. Springer, Heidelberg (2005) 16. Berlo, D.K., Lemert, J.B., Mertz, R.J.: Dimensions for evaluating the acceptability of message sources. The Public Opinion Quarterly 33(4), 563–576 (1969) 17. Robins, D., Holmes, J.: Aesthetics and credibility in web site design. Information Processing & Management 44(1), 386–399 (2007) 18. Toms, E.G., Taves, A.R.: Measuring user perceptions of web site reputation. Information Processing and Management 40, 291–317 (2004) 19. Fogg, B.J.: Credibility and the world wide web. Persuasive Technology, 147–181 (2003) 20. Barnes, S.J., Vidgen, R.: Interactive e-government services: modeling user perceptions with eQual. International Journal of Electronic Government 1(2), 213–228 (2004) 21. Gill, J., Johnson, P.: Research methods for managers. Paul Chapman Publishing Ltd., Boca Raton (1991) 22. Michael, C., John, F.: Capacity building: facilitating citizen participation in local governmence. Australian Journal of Public Administration 64(4), 64–80 (2005) 23. Yang, J.Q., Paul, S.: E-government application at local level: issues and challenges: an empirical study. International Journal of Electronic Government 2(1), 56–76 (2005)
Auto-complete for Improving Reliability on Semantic Web Service Framework Hanmin Jung1, Mi-Kyoung Lee1, Won-Kyung Sung2, and Beom-Jong You1 1
Information Technology Research Lab., KISTI, Korea 2 Dept. of Policy Research, KISTI, Korea {jhm,jerryis,wksung,ybj}@kisti.re.kr
Abstract. This paper presents two methods for enhancing auto-complete which providing search keywords that the user wants. The first is to display only search keywords that can guarantee a successful search result in real time regardless of document’s insertion, deletion, and update. The second is to display search keywords with their entity types such as person, institution, and topic. To accomplish them, we introduce an auto-complete table that stores the entities extracted and indexed from input documents and their document frequency (DF). An auto-complete manager checks whether each entity in the table can guarantee a successful search result or not by considering its DF, and provides proper entities with their types to the user. To verify the effect of the auto-complete, we are designing a comparative experiment. OntoFrame 2007 without the functions will be compared with OntoFrame 2008 with the functions for discovering the effect of our auto-complete on the reliability of Semantic Web services. Keywords: Auto-complete, Semantic Web, Semantic Web Framework, OntoFrame, Reliability, Document Indexing.
1 Introduction Auto-complete is a feature for predicting a word or phrase that the user wants to type in without the user actually typing it completely. Auto-complete on the Web is usually implemented by Ajax (Asynchronous JavaScript and XML) which is one of key technologies classified in Web 2.0 (see fig. 1). They are widely applied over the Web sites including digital libraries, commercial portals, and enterprise applications [1]. It is expected to be applied more and more in the viewpoint of enhancement of the user experience. However, most auto-completes simply display search keywords retrieved from the user’s query logs and system dictionaries without considering the quality of search results. The reliability of such auto-completes would be dropped in case that the user selects a search keyword which can not generate a successful search result. It is not a special case for small/medium enterprise portals because they suffer from relative short of contents compared with commercial portals such as Google, Yahoo, and Amazon. Even worse, different types of search keywords are mixed in their auto-complete lists. It compels the users to look up whole of the list for finding a M.J. Smith and G. Salvendy (Eds.): Human Interface, Part II, HCII 2009, LNCS 5618, pp. 36–44, 2009. © Springer-Verlag Berlin Heidelberg 2009
Auto-complete for Improving Reliability on Semantic Web Service Framework
37
Fig. 1. Classic Web application model and Ajax Web application model1
search keyword they want. These problems can be solved by introducing two methods that display only search keywords that can guarantee a successful search result and provide their entity types such as person, institution, and topic. This research deals with the way how the two methods can be achieved on a Semantic Web service framework.
2 Related Studies Han and Lee deals with costs and benefits of internet search with costumers’ prospective [2]. They reveal the fact that pains during the search are closely related with the user satisfaction, thus it should be designed to minimize the time and effort required. Auto-complete is an important function in that it can reduce the pains of the user by providing a way to select a search keyword without typing it completely. Another study on the usability of auto-complete concludes that the user satisfaction and search efficiency (the time needed to complete a given task) are affected by the function [3]. It also reports many positive comments were received from the users. Even for mobile devices, auto-complete enables the users to finish their tasks with fewer errors [4]. The study has clearly proven its advantages in terms of satisfaction, efficiency, and stability. However, studies on auto-complete enhancement are comparatively insufficient against its importance. A patent created by Lee and Wales proposes auto-complete using multiple dictionaries in ways of lookup and merge [5]. Auto-complete introduced 1
http://www.adaptivepath.com/images/publications/essays/ajax-fig1_small.png
38
H. Jung et al.
by Miki et al. can be classified as an advanced function in that it deals with ontology and data conversion [6]. An application form is generated in ways of referring the ontology data that knowledge managers have constructed manually. When the users input data in the form, the auto-complete function recognizes language types and field types in order to convert the data into appropriate values. However, the study is not closely concerned with ours because we concentrate on only the construction of search keywords in auto-complete list. A study on auto-complete for predicting Chinese characters with partial uses a prefix tree decoder [7]. It also adopts speech recognition for supporting bi-modal Chinese character input. Liu et al. found the bi-modality improves the input speed, but their research scope does not match with us exactly. Bangalore et al., as a significant study on auto-complete for improving reliability, introduces a method for providing search keywords that are able to generate a successful search result in the UMLKSK interface [8]. It uses a flag to mark the success or failure of search results. In case of success, the corresponding search keyword is marked with the flag. This method ensures a successful search result by displaying only the search keywords by considering the flag when the auto-complete list is provided. However, the auto-complete list would be occasionally out-of-date because the marking occurs only when the user enters a search keyword. Even after adding an input document which includes a search keyword that has caused the failure previously, the keyword will not be display in the auto-complete list until the user types manually the search keyword without referring to the list.
Fig. 2. Example of a search result with failure caused by selecting an improper search keyword in auto-complete list (‘Wizwid’ online shopping mall)
Many popular Web sites, even including enterprise portals, provide auto-complete for helping the users to find search keywords in ease. They usually use the popularity of the users’ input regardless of success/failure of search results as shown in fig 2. Because they do not consider incremental data add-up, which would cause mismatch problem between auto-complete and search results, search using auto-complete occasionally fails. As a new technology, Google offers keyword suggestions in real time in the form of auto-complete. However, mismatch in the number of documents occurs as shown in fig. 3. It may be caused by incomplete incremental indexing
Auto-complete for Improving Reliability on Semantic Web Service Framework
39
Fig. 3. Example of mismatch between auto-complete and search results in the number of documents (‘Google’ suggest; 7,710,000 results in auto-complete and 7,300,000 in search results)
management related with auto-complete. The following section explains how we resolve these problems with an auto-complete manager and an auto-complete table.
3 Auto-Complete in OntoFrame OntoFrame, a Semantic Web service framework [9] [10], aims at search and discovery of science and technology information using Semantic Web technologies. It includes a search engine and a reasoning engine. The former searches full-text documents and the latter discovers implicit knowledge by exploiting relations between instances, i.e. entities. OntoFrame gathers legacy data and transforms them into RDF (Resource Description Framework) triples by referring to predefined ontology schema designed for a specific application domain. The reasoning engine expands the triples at idle time, i.e. forward chaining, using user-defined rules, and puts back the results into an RDF triple store. OntoFrame-based service communicates with the two engines through Web Services, SPARQL (Simple Protocol and RDF Query Language) queries, and XML documents. OntoFrame provides entity-centric unified search 2 . Predefined entities are managed in a URI server which is a semantic data management tool with the RDF triple store. The server is referred by a document indexer for acquiring entities and their types from an input document (see fig. 4). Auto-complete is applied to OntoFrame for helping the user to search with convenience and efficiency. To sustain reliability, auto-complete should provide search keywords that guarantee successful search results. However, the previous version of OntoFrame (OntoFrame 2007) provided a simple auto-complete function which just shows a search keyword list matched with the user’s input string. The keywords are pre-extracted and refined topic keywords from test collection. Even the case that a keyword never appears in indexed documents, it can be displayed in auto-complete list, 2
Entity-centric unified search can be defined as a unified search generating a Web page which consists of service components selected dynamically according to the user’s input corresponding with an entity reserved in the system. Different kinds of search result pages would be generated in case of different entity types.
40
H. Jung et al.
Document
Document Indexing OntoFrame Service URI Server
Auto-complete Manager Auto-complete Table
Fig. 4. Auto-complete process
Fig. 5. Example of changes in the auto-complete table according to document’s indexing (P: person, I: institution, T: topic)
which will cause a failed search result. Even worse, the use of all of the topic keywords increased the load of auto-complete. We also found the users’ reliability of the service is degraded when they select improper keywords, i.e. keywords that can not guarantee a successful search result, in the list. Thus, this version of OntoFrame (OntoFrame 2008) introduces an auto-complete manager for guaranteeing successful search results from selecting a search keyword in auto-complete and further increasing usability by displaying entity types additionally. After extracting entities from an input document, the auto-complete manager updates the document frequency (DF) of the entities in an auto-complete table, i.e. DF of the entities would increase by one as shown in fig. 5.
Auto-complete for Improving Reliability on Semantic Web Service Framework
41
When the user enters a search keyword, the auto-complete manager finds the entities matched with the keyword by looking up the auto-complete table. Then, it checks whether DF of each retrieved entity is zero or not. Those entities with more than zero are displayed in the auto-complete list. Since the entities in the list are extracted from indexed documents, search results generated from them would be always successful. This method is different from that of Bangalore et al. because DF of entities is updated instantly according to document’s indexing, and only proper entities are always displayed in the auto-complete list [8]. The following pseudo codes explain how the auto-complete manager deals with the auto-complete table for generating an auto-complete list corresponding with the user’s input. manage_auto_complete(String) { Entity[] = get_matched_entity_list(String); Valid_entity[] = check_doc_frequency(Entity[]); sort_entity_by_type(Valid_entity[]); return Valid_entity[]; } sort_entity_by_type(Entity[]) { find_entity_type(Entity[], Person_entity[], Institution_entity[], Topic_entity[]); sort_entity(Person_entity[]); sort_entity(Institution_entity[]); sort_entity(Topic_entity[]); sorted_entity[] = merge_entity(Person_entity[], Institution_entity[], Topic_entity[]); return Sorted_entity[]; } update_auto_complete_table(String[], Operation) { While (Entity = get_next_string(String[])) { Switch (Opreation) { Case Insert: increase_doc_frequency(Entity); Case Delete: decrease_doc_frequency(Entity); } } } The above functions lookup and manage the auto-complete table. Manage_auto_complete() gets the user’s input string from OntoFrame service. It finds the entities of which name is matched with the input string. Check_doc_frequency() checks each entity whether it has DF of more than zero. Entities with document frequency of zero are excluded. By referring the auto-complete table, sort_entity_by_type() makes a merged entity list sorted by their types, e.g. topic and person. Then manage_auto_complete() returns them to the service. DF of each entity is managed by update_auto_complete_table(). The function increases or decreases DF according to operation types, i.e. insert and delete. In case of document’s update, insert process would follow delete process. Finally, the auto-complete manager returns a proper entity list as shown in the following example and fig. 6.
42
H. Jung et al.
Fig. 6. Example of auto-complete in OntoFrame (It shows topic and person entities matched with “Seman.”)
API: EntityList manage_auto_complete(String keyword) Calling example: manage_auto_complete(“sem”) Result example (see fig. 3): [semantic activation, Topic] [semantic association, Topic] [semantic blocking, Topic] … [Semantha,Ellis, Person] Current number of the entities stored in the service is 923,449 (652,507 for topic and 270,942 for person). We found that 362,319 topic entities (55.53% when compared
Fig. 7. Example of search result generated from topic entity “neural network”
Auto-complete for Improving Reliability on Semantic Web Service Framework
43
Fig. 8. Example of search result generated from person entity “Jinde Cao”
with total topic entities) acquired from indexing of about 200,000 journal papers on information technology and bioinformatics lead to successful search results. Fig. 7 and fig. 8 show examples of successful search results for topic and person entities.
4 Conclusion We introduced an enhanced auto-complete with two functions; displaying only search keywords that can guarantee a successful search result in real time regardless of document’s insertion, deletion, and update and displaying search keywords with their types such as person, institution, and topic. For accomplishing them, an auto-complete manager and an auto-complete table are used. The manager gets the user’s input string and returns a proper entity list by looking up the table. To verify the effect of the auto-complete, we are designing a comparative experiment. The task to be achieved is “Find a representative researcher and an institution for top 5 topics in the auto-complete list corresponding with ‘neural’.” OntoFrame 2007 3 without the functions will be compared with OntoFrame 20084 with the functions. We expect to find the effect of our auto-complete on the reliability of Semantic Web services.
References 1. Beauheim, C., Wymore, F., Nitzberg, M., Zachariah, Z., Jin, H., Skene, J., Ball, C., Sherlock, G.: OntologyWidget – a Reusable, Embeddable Widget for Easily Locating Ontology Terms. J. BMC Bioinformatics 8, 338 (2007) 3 4
http://isrl.kisti.re.kr/wsearch/search/main.jsp http://150.183.113.186:8080/OntoFrame_ISRL/2008_new/main.jsp
44
H. Jung et al.
2. Han, D., Lee, E.: Exploring the Costs and Benefits of Internet Search from the Online Customers’ Perspective: Implications for the Consumer Adoption of the Semantic WebBased Search Engines. J. Business Education Research 11(1) (2007) (in Korean with English Abstract) 3. Kluge, J., Kargl, F., Weber, M.: The Effects of the Ajax Technology on Web Application Usability. In: International Conference on Web Information Systems and Technologies (WEBIST 2007) (2007) 4. Udyaver, S.: Experimental Comparison of Usability of Hybrid Mobile Devices. In: The 20th Computer Science Seminar (2004) 5. Lee, K., Wales, K.: Methods and Systems for Implementing Auto-complete in a Web Page. US 2004/0039988 A1 (US. Patent) (2004) 6. Miki, T., Ogawa, H., Matsuda, N., Miura, H., Taki, H., Hori, S., Abe, N.: Auto Complete Method for Web Application from Based on Term Hierarchy. In: The 20th Annual Conference of the Japanese Society for Artificial Intelligence (2006) (in Japanese with English Abstract) 7. Liu, P., Ma, L., Soong, F.: Prefix Tree Based Auto-Completion for Convenient Bi-modal Chinese Character Input. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2008) (2008) 8. Bangalore, A., Browne, A., Divita, G.: UMLSKS SUGGEST: An Auto-complete Feature for the UMLSKS Interface Using AJAX. In: AMIA 2006 Annual Symposium (2006) 9. Jung, H., Lee, M., Kang, I., Lee, S., Sung, W.: Finding Topic-Centric Identified Experts Based on Full Text Analysis. In: The 2nd International Expert Finder Workshop at ISWC 2007 + ASWC 2007 (2007) 10. Sung, W., Jung, H., Kim, P., Kang, I., Lee, S., Lee, M., Park, D., Hahn, S.: A Semantic Portal for Researchers Using OntoFrame. In: The 6th International Semantic Web Conference and the 2nd Asian Semantic Web Conference (ISWC 2007 + ASWC 2007) (2007)
Effects of AJAX Technology on the Usability of Blogs Sumonta Kasemvilas and Daniel Firpo School of Information Systems and Technology, Claremont Graduate University, 130 E. 9th Street, Claremont, CA 91711, USA {Sumonta.Kasemvilas,Daniel.Firpo}@cgu.edu
Abstract. AJAX can enhance Web applications by updating a part of the Web page instead of the whole page. This change of technology relates to a usability issue. We used WordPress 2.3 to create two versions of blogs: non-AJAX and AJAX. Then we conducted an experiment by giving a task scenario to eight participants. We collected performance data by recording users’ mouse movements during the experiment and collected preference data by providing a questionnaire after the tasks. Finally, we conducted post-experiment interviews to gather participants’ experiences. The quantitative results show that AJAX did not improve users’ performances the first time they used it, while qualitative interviews demonstrate participants’ satisfaction with AJAX blogs. Keywords: AJAX, Blog, Ease of Learning, Efficiency of Use, Error Frequency and Severity, Experiment, Satisfaction, Usability, WordPress.
1 Introduction AJAX (Asynchronous JavaScript and XML) is a set of technologies such as eXtensible Markup Language (XML), XMLHttpRequest, Cascading StyleSheets (CSS), and the Document Object Model (DOM), combined with JavaScript. It is not a single technology, but several technologies that, when used together, enhance the capability of Web applications in innovative ways [1]. Many corporations use AJAX in applications, such as Google Maps, Gmail, iGoogle, Hotmail, and Yahoo Flickr. An advantage of AJAX is that Web browsers do not need to refresh the whole page. They only need to reload a portion of the page, which makes the Web site load faster and increases performance, because users can receive responses from the server faster than in classical Web applications. However, this also leads to a problem with AJAX. Users who are familiar with old-style Web applications may not be able to effectively use Web applications that contain AJAX technology due to a lack of understanding of how AJAX works. The users may not notice what has changed on the screen, or they might be waiting for the results to display, or for a response from the browser. Users might need to get accustomed to new conventions. This change of technology relates to a usability issue [2]. Blogs, a slang form of the term “Web logs,” are a Web 2.0 technology used extensively on the World Wide Web. Blogging has gained quite a bit of popularity in the recent years. “Blog” was named as the “word of the year” in 2004 by MerriamWebster. The large increase in popularity of blogs has led to the rapid growth in the M.J. Smith and G. Salvendy (Eds.): Human Interface, Part II, HCII 2009, LNCS 5618, pp. 45–54, 2009. © Springer-Verlag Berlin Heidelberg 2009
46
S. Kasemvilas and D. Firpo
number of blog users, from individuals using them for personal use, to larger entities such as companies or universities. Blogs play a vital role not only in businesses and organizations, but also in academia. For example, the Brisbane Graduate School of Business at Queensland University of Technology records students’ experiences through the use of the ‘MBA blog’ [3]. Thus, it is in our interest to study human computer interaction of AJAX technology in blogs.
2 AJAX Classical Web applications use HTTP protocol to give requests to and receive responses from the server. This process takes time and refreshes the user’s whole screen, and a sign such as a rotating globe, a spinner, or a progress bar shows the user that the browser is loading. However, now we have moved on to a JavaScript-based paradigm, which calls an AJAX engine to send asynchronous HTTP requests to the server. AJAX supposedly improves the effectiveness of the Web by updating a part of the Web page instead of the whole page. It seamlessly exchanges small quantities of data between the browser client and the server. Because refreshing the Web page is not required when updating, the page becomes more interactive. This results in higher responsiveness, speed, and functionality [2]. AJAX is a client-side technology. In the AJAX Web application model, instead of sending an HTTP request from client to server as before, the user interface of the browser client will send a JavaScript call to the AJAX engine. Then, the AJAX engine sends an asynchronous HTTP request to the Web and/or XML servers. From the server side to the client side, the server sends XML data to the AJAX engine, and the AJAX engine sends HTML and CSS data to the user interface of the client. This also decreases bandwidth usage when compared to classical Web applications. AJAX also gains the advantage of JavaScript, in which developers only need to code once, and their code can work across different platforms. This makes AJAX more adaptable. However, AJAX has several disadvantages. Because it does not require a whole page reload, users cannot push the back button to go back to a prior state. This can cause problems because most users become accustomed to this behavior when they use Web browsers. Moreover, users cannot add bookmarks to keep a certain stage of the page when the page dynamically updates only small portions on the screen [4]. These problems may be solved by adding more complicated programming. Recently, Kluge, Kargl, and Weber [5] reported on the effects of AJAX technology on usability in Web applications such as message boards and auto-completion widgets, while our study focuses specifically on blogs. They studied only user satisfaction and time for completion of tasks. They looked at time for completion of tasks as a parameter of efficiency of use. They concluded that AJAX technology dramatically improves users’ satisfactions and efficiency of use in some scenarios. However, our study not only focuses on efficiency of use and satisfaction, as in Kluge et al’s [5] study, but also on other measures of usability, such as ease of learning and error frequency and severity, which were not measured in their study.
Effects of AJAX Technology on the Usability of Blogs
47
3 Research Question and Hypotheses Development Krug [6] asserts that one of the most important factors for Web usability is when users glimpse the Web page and are capable of easily interacting with it. Users should not have to take a long time to think about how to use the Web page. Oulasvirta and Salovaara [7] assert that it is important for a user interface to be invisible, as an interface must be uncomplicated (requiring no other knowledge besides the common sense of the user to operate) and interactive (ready to respond to the user on a continuous basis). Usability relates to the ability of users to learn and utilize a system or a product, such as a Web site or a computer application, to accomplish their purposes [8]. It includes the degree of satisfaction they perceive from learning and using the system [8]. Measurement of usability involves a group of factors that we need to consider for a user interface [8], [9]. First, ease of learning is measured from the ability of users who have never seen the user interface before to easily accomplish simple tasks without substantial training [8], [9], [10]. This includes effectiveness or the ability of users to successfully achieve tasks. Second, efficiency of use refers to how quickly users who have used the system before can complete their tasks easily and without frustration [8], [9], [10]. Third, memorability refers to how easily users can capably use the system again after not using the system for a period of time [8], [9]. Fourth, error frequency and severity refers to how often users make mistakes when using the system, the severity of these mistakes, and how easily they can recover from these mistakes [8], [9]. Finally, participative satisfaction refers to the extent to which users are satisfied with the system [8], [9], [10]. Currently, few studies have been conducted on the relationship between AJAX technology and Web usability when users utilize blogs. We are interested in finding out if AJAX technology within a blog leads to improvements in Web usability. We will examine whether adding AJAX plugins to a blog has a positive effect on the usability of that blog. To measure if AJAX technology affects the usability of a blog, we categorize these factors into two groups: preference and performance [10]. First, preference is measured from participative satisfaction. Second, performance is measured from ease of learning, error frequency and severity, as well as efficiency of use. We did not measure memorability because the experiment was done in one session. In addition, Kluge et al. [5] concluded that user satisfaction and efficiency of use when using AJAX technology are higher than when not using AJAX. Thus, we hypothesize that: HYPOTHESIS 1: The usability of blogs with AJAX technology is greater than in regular blogs. H1a: The ease of learning in blogs using AJAX technology is greater than in regular blogs. H1b: The error frequency and severity in blogs using AJAX technology is lower than in regular blogs. H1c: The participation satisfaction of users using blogs with AJAX technology is greater than the participation satisfaction of users using regular blogs. HYPOTHESIS 2: The efficiency of use in blogs using AJAX technology is high when users have used AJAX-enhanced blogs before.
48
S. Kasemvilas and D. Firpo
4 Experiment Design 4.1 System Design To test Web usability, we created two versions of a blog that had similar interfaces. The first version did not have AJAX plugins, but the second version did. We used WordPress 2.31 to create blogs in this study. WordPress is a well-known blog application that provides easy installation and usage. Calendar, search box, and comment are basic functions of blogs in WordPress. One can find these functions in most WordPress blogs. This makes it easier to compare the usability of regular blogs and blogs using AJAX technology. In this study, we installed three AJAX plugins. First, AJAX Calendar2 was used to find articles for each day and month. This plugin allows the reader to click a special button to show all articles in a specific month, a service which the regular calendar does not offer. When the user clicks that button, AJAX will retrieve all articles in that particular month and display them on the bottom of the calendar. This plugin makes AJAX blogs faster than their non-AJAX counterpart, since it refreshes only this specific part of the page. Second, LiveSearch3 allowed users to search articles within the blog. This plugin shows the results in a pop-up style menu using AJAX technology (similar to how in Google Suggest, as users type each character, the software displays a set of words that relates to these characters to predict what the user is searching for). Finally, Inline AJAX Comment4 allowed users to add comments and auto-update those comments without a full page reload. With AJAX technology, this plugin provides a much smoother, faster commenting feature in the blog. Users can also click to Show or Hide comments. 4.2 Task Scenario A task scenario was designed for readers because most people use blogs in their daily lives to find information about specific interests and participate with other users. The set of tasks is a basic operation readers perform when they visit a blog. The readers will find interesting articles, read them, search for related information, and add comments. In AJAX version, participants complete a set of tasks by using plugins that contain AJAX technology, while in non-AJAX version, participants use the default plugins that come with the original version of WordPress. To test Web usability, participants needed to accomplish a set of tasks that was listed on the instruction sheet. First, participants needed to login as guests. Second, they needed to use the calendar in the left sidebar to find an article named “Blog#2” in September, and post a comment on that article. Third, before writing any comments, participants needed to use the search box in the left sidebar, input the word “dog,” and find the article related to this word. Then, they had to copy the paragraph from the article to add in a comment box of the blog’s article. They needed to do the same set of tasks in both non-AJAX and AJAX 1
http://codex.wordpress.org/Version_2.3 http://wordpress.org/extend/plugins/ajax-calendar/ 3 http://wordpress.org/extend/plugins/livesearch/ 4 http://www.ditii.com/2006/07/07/wordpress-plugin-inline-ajax-comments/ 2
Effects of AJAX Technology on the Usability of Blogs
49
versions. Then, they used an AJAX version one more time at the end of the experiment so we could measure efficiency of use. 4.3 Method and Procedure To identify usability problems, our experiment was conducted on a small group of users as per Nielsen’s suggestion [9]. Nielsen [11] claimed that “[t]he best results come from testing no more than 5 users and running as many small tests as you can afford.” More test subjects would only result in a marginal increase in the number of problems, errors, and different completion times for tasks found [11]. Moreover, the small number of participants for qualitative usability testing is enough [9]. We recruited eight participants for this study. All users had to complete all three experiments (non-AJAX, AJAX for the first time, and AJAX for the second time) within the same day, back to back to back. To decrease participant bias and avoid an order effect resulting from the participants’ expectations and impressions for the next version of the blog after completing the first one, we used counterbalanced design. Half the participants used the non-AJAX version first, then did the same tasks again with AJAX version, while the other half used the AJAX version before the non-AJAX version. After all eight participants finished the set of tasks with these two versions (AJAX and non-AJAX), we asked them to repeat the set of tasks with the AJAX version to measure efficiency of use. We measured whether they improved performance efficiently after they learned how to use the new kind of blog that incorporated AJAX technology the first time around [12]. At the end of the experiment, participants were asked to complete questionnaires about their background, blog experience, and questions about satisfaction with the blogs. After participants finished answering the questionnaire, we conducted postexperiment interviews to collect qualitative data to substantiate quantitative results. 4.4 Usability Measurement In the usability test, we collected quantitative data of both participants’ performance and participants’ preference (Table 1). We collected two types of data: what really happened when participants used the blog (performance data) and what participants thought when they used it (preference data) [10]. For the performance metric, we used the free trial software All In One Keylogger5 for Windows. This software can visually capture users’ mouse movements and to be reported in visual logs and HTML reports. It records where users click on the screen, how they move the mouse, and when they perform certain actions. We used this information to calculate a performance metric for the usability test. To prevent threats to validity, this software has a “hidden mode” feature. Participants were not aware that the researcher observed their movements when they used the blog. For user preference, we used a questionnaire to measure how satisfied the users were when using the blog. In the questionnaire, ratings on a 1 to 5 Likert scale were used to allow for variation in data. For example, to find out to what extent participants prefer any version, the participants could express their feelings from ‘strongly disagree’ (1) to ‘strongly agree’ (5). 5
http://www.relytec.com/download.htm
50
S. Kasemvilas and D. Firpo Table 1. Usability metric (Adapted from [10])
Performance Usability Metric Total time to completion (Time) Number of steps (Step)
Amount of confusion (Confusion) Pathway analysis and the number of user errors (Error) Preference Usability Metric User satisfaction User comments Preference ratings
Usability Measure Ease of learning: - Can participants complete the task scenario quickly? Ease of learning: - How many steps do participants take in order to successfully complete the tasks? Error frequency and severity: - How many times do participants get confused? Error frequency and severity: - How many errors do participants make when they use the blog? Usability Measure Participative satisfaction: - Do participants get pleasure from using the blog? Participative satisfaction: - Are participants confused when they use the blog? Participative satisfaction: - Do participants prefer the design of the blog?
5 Results 5.1 Data Analysis The participants were eight graduate students aged between 20-40 years old. Three of them were male and the other five were female. All participants were familiar with using the Internet, and half of them had their own blogs. In hypothesis H1a, we were interested in whether the ease of learning in blogs using AJAX technology is greater than in regular blogs. H1a was tested by paired sample t-test on Time and Step parameters of participants using non-AJAX blogs and AJAX blogs for the first time (Table 2). The results indicated that the presence of AJAX increases the time needed to complete tasks for first-time users. At the significant level of .05 (1-tailed), users spent significantly more time completing tasks in AJAX blogs than in regular blogs. However, number of steps participants took to complete tasks in AJAX and nonAJAX blogs were not significantly different. Thus, H1a was disconfirmed. In hypothesis H1b, we were interested in whether or not the error frequency and severity in blogs using AJAX technology was lower than in regular blogs. To test H1b, paired sample t-test was performed on the participants’ Confusion and Error parameters when they used non-AJAX blogs and AJAX blogs for the first time (Table 2). At the significant level of .05 (1-tailed), contrary to our hypothesis, users had more confusion and errors in AJAX blogs than in regular blogs. Thus, H1b was also disconfirmed. In hypothesis H1c, we were interested in whether the participation satisfaction of users when they used blogs with AJAX technology was greater than their participation satisfaction when they used regular blogs. We calculated mean and standard deviation to examine whether the average preference of users when using non-AJAX blogs was different from when they used AJAX blogs. We tested the hypothesis H1c by using paired sample t-test of preference between the two versions
Effects of AJAX Technology on the Usability of Blogs
51
Table 2. Difference of performance scores between AJAX for the first time (AJAX1) and NON-AJAX Usability Dimension (N = 8) Time Step Confusion Error
AJAX1 M SD 455.88 144.21 13.13 4.36 6.25 5.31 4.50 5.10
NON-AJAX M SD 322.13 109.90 14.75 3.92 1.50 2.73 0.50 0.76
t
p
2.01 -0.60 1.91 2.11
.042 .284 .049 .037
Table 3. Difference of preference scores between AJAX and NON-AJAX Usability Dimension (N = 8) Satisfaction
AJAX M SD 3.58 0.97
NON-AJAX M SD 3.13 0.56
t
p
0.89
.201
Table 4. Difference of performance scores between AJAX for the first time (AJAX1) and AJAX for the second time (AJAX2) Usability Dimension (N = 8) Time Step Confusion Error
AJAX1 M SD 455.88 144.21 13.13 4.36 6.25 5.31 4.50 5.10
AJAX2 M SD 176.13 86.63 9.13 1.25 0.13 0.35 0.00 0.00
t
p
5.87 2.50 3.20 2.50
.000 .021 .008 .021
of blogs, non-AJAX and AJAX (Table 3). The results indicated that, at the significant level of .05 (1-tailed), users’ satisfaction with AJAX blogs was not greater than their satisfaction with regular blogs. Thus, H1c was not supported. In hypothesis H2, we were interested in whether the efficiency of use in blogs using AJAX technology is high when users have used AJAX-enhanced blogs before. Thus, H2 was tested by paired sample t-test on participants’ Time, Step, Confusion, and Error parameters when they used AJAX for the first time and when they used it again for the second time (Table 4). At the significant level of .05 (1-tailed), users spent less time, took fewer steps, with less confusions and errors in AJAX blogs the second time around. Thus, H2 was confirmed. From the data, participants using AJAX for the first time took more time to finish the task (Figure 1), felt more confused, and made more errors compared to when they did the same set of tasks without AJAX. Therefore, H1 was disconfirmed. However, in H2 when using the AJAX version for the second time, participants spent less time (Figure 1), used fewer links, were less confused, and made slightly fewer errors than when they performed the set of tasks without AJAX. Thus, the data supports H2. This indicates that learning is a very important step for introducing a new technology.
52
S. Kasemvilas and D. Firpo
Total Time to Completion 800 700
687 637
Time (seconds)
600 500
497
475
454
436
409
400
376
300
370 326
298
AJAX1
339 299
308
309
295
AJAX2
246
215
200
Non-AJAX
164 127
106
100
123
117
119
0 1
2
3
4
5
6
7
8
Participants
Fig. 1. This graph shows the time to completion of each participant when each participant used Non-AJAX blogs, AJAX blogs for the first time, and AJAX blogs for the second time. Three participants out of eight spent less time when they used the AJAX version for the first time as compared with the non-AJAX version. Seven out of eight spent less time on the AJAX version the second time as compared with the non-AJAX version. All participants spent less time when they used the AJAX version for the second time as compared with the first time.
5.2 Post-experiment Interview Although the number of participants was not large, which may cause the power of the statistical test to be low, the results of most statistical analyses reached significant level of alpha .05. However, most of the results surprisingly stand in direct contrast to our hypothesis. Thus, we would like to triangulate the results using both quantitative and qualitative methods. We conducted post-experiment interviews to collect impression of participants when they used two versions of blogs and to see whether their satisfaction matched their performance when they worked with the blogging software. Some sample questions were asked such as: Were you confused when you used the AJAX blog for the first time? Why? If you prefer the AJAX blog to the non-AJAX blog, please give your reasons why. In the study, we had eight participants, and two of them did not like the AJAX version because the interface was more complicated than a non-AJAX version. One participant explained that I prefer non-AJAX version especially if I know exactly what I am searching for. AJAX version confuses my usage. In general, I prefer simplicity rather than sophistication when I search any web browsers. Another participant said: I don’t like it because it is too complicated. It is not necessary. The rest of the participants liked the AJAX version because it provided functions that help users do their jobs faster and more interactively. The following comments point out the potential benefits of AJAX version:
Effects of AJAX Technology on the Usability of Blogs
53
It’s much easier to find what I want using calendar and search box. I like search box the most due to the ease of use. It works like Google Suggest. The interface is much better than non-AJAX version. The AJAX blog has functions that help users do their jobs faster. I love the search box. AJAX blog has more functions and takes less time to refresh. It’s more useful when one knows how it works. AJAX should have increased usability performance and preference, and while the interviews showed that most of the participants liked AJAX, the statistical results did not support our first hypothesis. This may be because some of the AJAX-specific features were confusing for them when compared with regular blogs. Participants expressed that when they used the AJAX blog, they felt confused with issues such as how AJAX Calendar only changes a small portion of the screen. They did not understand how it worked and found it difficult to use when compared with the nonAJAX version. This may be explained with the need for time to get used to new technology. Participants’ performances were low when they used the AJAX blog for the first time, and they confessed that they had not knowingly used AJAX technology before. Thus, they felt unfamiliar with the new interface conventions. However, the participants’ performance when they used AJAX for the second time showed that after the users learned and understood how to use AJAX technology, their performance improved significantly.
6 Discussion and Conclusion This study attempts to measure how AJAX technology affects Web usability when users use a blog. We designed two versions of a blog, which have a similar user interface. However, WordPress provides many different theme styles. Thus, a theme chosen in this study may not have the same result as if we had used other themes. The experiment was controlled in a closed environment to record users’ mouse movements but it still depended on the speed of the Internet and CPU of the computer used during the experiment. This may cause delay times or load times that do not relate to user performance. Kluge et al’s [5] study shows that AJAX technology increases efficiency of use and satisfaction in AJAX Web application in some task scenarios, but our study showed that AJAX technology does not always increase usability. This may point out that learning for new technology is vital, especially when users are familiar with traditional technology. In the United States, the National Institute of Standards and Technology (NIST) has begun to address some of the AJAX technology issues in the need for Web administrators and IT Management to monitor usability and security of Webcode design, development, test, and maintenance [13]. Blogs are a prevalent Web 2.0 tool that has been widely adopted amongst a broad subset of society, including many users who are not particularly tech savvy. To us, it is important to investigate what happens when one applies new technology to a tool so we can learn how to improve the technology and enhance human computer interaction. From our results, we can see
54
S. Kasemvilas and D. Firpo
that although in an easy-to-use tool with widespread adoption, such as blogs, there are many challenges to overcome in applying new technology that users are not familiar with. From this perspective, the results of this paper suggest the problems of using AJAX technology in a blog and may help Web developers use AJAX technology to improve Web usability of a blog.
References 1. Garrett, J.J.: Ajax: A New Approach to Web Applications, http://www.adaptivepath.com/ideas/essays/archives/000385.php 2. Paulson, L.D.: Building Rich Web Applications with Ajax. Computer, 14–17 (2005) 3. Williams, J.B., Jacobs, J.: Exploring the Use of Blogs as Learning Spaces in the Higher Education Sector. Australasian Journal of Educational Technology 20(2), 232–247 (2004) 4. West, J.: Ajax: not just another acronym or is it? Searcher 14, 13–15 (2006) 5. Kluge, J., Kargl, F., Weber, M.: The Effects of the AJAX Technology on Web Application Usability. In: WEBIST 2007 International Conference on Web Information Systems and Technologies, pp. 289–294 (2007) 6. Krug, S.: Don’t Make Me Think: A Common Sense Approach to Web Usability. New Riders Press, New York (2000) 7. Oulasvirta, A., Salovaara, A.: A Cognitive Meta-analysis of Design Approaches to Interruptions in Intelligent Environments. In: Conference on Human Factors in Computing Systems, pp. 1155–1158. ACM, New York (2004) 8. Usability gov., http://www.usability.gov/basics/whatusa.html 9. Jakob Nielsen’s Alertbox: Usability 101: Introduction to Usability, http://www.useit.com/alertbox/20030825.html 10. Usability gov., http://www.usability.gov/basics/measured.html 11. Jakob Nielsen’s Alertbox: Why You Only Need to Test With 5 Users, March 19 (2000), http://www.useit.com/alertbox/20000319.html 12. Nielsen, J.: Usability Engineering. Morgan Kaufmann, San Francisco (1993) 13. National Institute of Standards and Technology, http://csrc.nist.gov/publications/nistpubs/800-28-ver2/ SP800-28v2.pdf
Usability Evaluation of Dynamic RSVP Interface on Web Page Ya-Li Lin and Darcy Lin Department of Statistics, Tunghai University, Taichung, Taiwan 40704
[email protected] Abstract. The usability of rapid serial visual presentation (RSVP) interface was evaluated using subjective preference questionnaire and performance measurement methods. Forty-two students voluntarily participated in this study. The results indicated the shelf interface moving from bottom-left to upper-right along a linear trajectory with moving speed of 20~30 frame per second (FPS) are most preferable. The carousel interface following circular trajectory in clockwise with moving speed of 10~15 FPS are most preferable. “Meets user experience”, “aesthetic and simple design”, “effective to use”, and “easy to learn” all conform to the usability goals. In addition, the results based on performance measurement showed a logistic regression model with RSVP mode and moving speed are fitted very well. There is the highest probability estimation of correct recognition for the carousel interface and moving speed of 30 FPS, however, the shelf interface and 15 FPS has the lowest probability estimation of correct recognition. Keywords: Dynamic Interface, Subjective Preference, Rapid Serial Visual Presentation (RSVP), Usability Evaluation.
1 Introduction Search for stock and weather information, prices for flights and mortgages, and new book issued is common in Internet search. A typical search session consists of: (1) formulating and entering the query, (2) browsing the search results, and (3) viewing selected result page. Our work aims to investigate and provide a user-centered interface for browsing the search results in the process of search session. Once a user has launched a query, the search engine must look in a variety of databases and return a set of relevant results. There are multiple ways to deliver information to the user and multiple ways to let the user use the result. However, there must be a phased implementation of content searches, both from a consumer usability perspective, as well as an advertiser/merchant perspective. Performance measures will be carrying out on the second phase of browsing the search results. Foster (1970) first used Rapid Serial Visual Presentation (RSVP) to mean rapidly displaying words in a sequence in the same visual location. RSVP originated as a tool for studying reading behavior [3], but lately has received more attention as a M.J. Smith and G. Salvendy (Eds.): Human Interface, Part II, HCII 2009, LNCS 5618, pp. 55–64, 2009. © Springer-Verlag Berlin Heidelberg 2009
56
Y.-L. Lin and D. Lin
presentation technique with a promise of optimizing reading efficiency, especially when screen space is limited [9]. The reason for the interest is that the process of reading works a little different when RSVP is used and that it requires much smaller screen space than traditional text presentation [8]. RSVP is a method of displaying information using a limited space in which each piece of information is displayed briefly in sequential order [2, 4, 5, 11]. With the development of dynamic design, fastmoving RSVP interface could emphasize its advantage of showing more image information at a time, but compared with slow-moving RSVP interface, its relative advantage could be less mental workload [8]. Can user experience of search results be improved by using dynamic RSVP interface? The images of shelf RSVP interface shown in the fixation area are used to compare with carousel RSVP interface [2, 11]. We are going to provide usability evaluation of image visualization of dynamic RSVP interface in this study. The objective of this study is to evaluate the usability of RSVP interface using subjective preference questionnaire and performance measurement methods. A prototype of simulated E-bookstore system is designed to collect the subjective preference ratings of predetermined designing factors at the beginning of the study. To evaluate the usability for web users, usability evaluation is used to achieve specified goals with effectiveness, efficiency, learnability, memorability, and user satisfaction [7]. Both RSVP display (carousel and shelf) and moving speed (10, 15, 20, 30, and 40 FPS) were varied in the simulated interface of E-bookstore. We would like to propose the following researcher’s hypotheses: (1) Could the design factors, such as RSVP, moving speed, and moving direction affect the subjective preference rating on dynamic RSVP interface? (2) Could the design factors affect the performance of recognition on dynamic RSVP interface? (3) Are there usability goals conforming to user experience?
2 Design of the Dynamic RSVP Interface The simulated E-bookstore interface would contain contents of web searching result. The searching results will be shown on the dynamic RSVP interface as the experimental Web pages. Two kinds of RSVP interfaces were considered based on their trajectory. The carousel RSVP interface is defined as a series of images display appear successively running from the bottom of page in a clockwise (Carousel I) or counterclockwise (Carousel II) along the circular trajectory. The shelf RSVP interface is defined as the linear trajectory of the images followed the diagonal running from bottom-left to upper-right (Shelf-I), from bottom-right to upper-left (Shelf-II), from upper-right to bottom-left (Shelf-III), and from upper-left to bottom-right (Shelf-IV). Specifications of design factors and their factor levels for subjective preference questionnaire and performance measurement are shown in Table 1. The prototypes of simulated E-bookstore interface used in the preference-based phase were illustrated in Table 1. The layer design and the number of frames per second (FPS) were used to produce the moving effect of image visualization. FPS in Micromedia Flash MX means the moving speed for each image [6]. The exposure times for one image are 20, 13.3, 10, 6.67 and 5 seconds corresponding to 10, 15, 20, 30 and 40 FPS. The task assigned to each participant is to browse the searching results after entering a query.
Usability Evaluation of Dynamic RSVP Interface on Web Page
57
Table 1. The specifications of design parameters and their factor levels for dynamic RSVP interface Design parameter RSVP Trajectory
Factor level Carousel I Circular Clockwise
Carousel II Circular Counterclockwise
Shelf I Shelf II Linear Linear Bottom-left to Bottom-right upper-right to upper-left
Shelf III Linear Upper-right Moving to bottomdirection left Growing to Growing to Constant to Constant to Growing to Image size Shrinking Shrinking Shrinking Shrinking Constant Dynamic Dynamic Dynamic Dynamic Dynamic Image position Bottom-right Bottom-left Position of the From 9-12-3 From 3-12-9 Bottom-left o’clock maximum image o’clock Number of 5 5 5 5 5 images visible Total number 10 10 10 10 10 of images
Shelf IV Linear Upper-left to bottomright Growing to Constant Dynamic Bottomright 5 10
Example
3 Research Methods Before the usability experiment the preparation work included the selection of participants and experimental factors, construction and design of the experimental Web pages and the dynamic RSVP Interface, and design of searching results. 3.1 Subjective Preference Questionnaire The subjective preference questionnaire is a structured field of usability assessment. It is useful in the early stages of user-centered design development. The International Organization for Standardization (ISO) defines usability of a product as “the extent to which the product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” Usability is generally regarded as ensuring that interactive products are easy to learn, effective to use, and enjoyable from user’s perspective [10]. More specifically, usability is broken down into the following goals: effectiveness, efficiency, safety, utility, learnability, and memorability. 3.2 Participants Forty-two undergraduate and graduate students (21 females and 21 males) coming from Tunghai University voluntarily participated in the subjective preference questionnaires. The distribution of age ranged form 23 to 30 years old (mean age of 24.6 years old and standard deviation of 1.72 years old). They all had the experience of surfing the internet and had normal vision or corrected vision reaching at least 20/25 and no color-blindness.
58
Y.-L. Lin and D. Lin
3.3 Experimental Design Dynamic RSVP interface consisted of ten images shown in turn on the Web page of Ebookstore (see Figure 1). Objective function is defined as the correct recognition of targeted image. Design factors are RSVP mode and moving speed. RSVP modes include clockwise and counterclockwise carousel RSVP and four types of shelf RSVP modes (shelf I-IV) (see Table 1). Moving speeds include five levels of 10, 15, 20, 30, and 40 FPS which their exposure time for each frame is 20, 13.3, 10, 6.67, and 5 seconds respectively. Each participant continuously viewed sixteen different searching results imposed on Web pages of E-bookstore and a recognition task was assigned.
(a)
(b)
Fig. 1. Illustration of (a) shelf RSVP and (b) carousel RSVP shown on the simulating E-bookstore interface
3.4 Experimental Procedure After filling in the self-reported background document, the rules of answering subjective preference questionnaire were explained by the experimenter. The varied combinations of RSVP interface on the simulated E-bookstore Webpage were shown one factor at a time. Subjective preference questionnaire was implemented associated with a simulated E-bookstore interface. The favorite display type of RSVP interface on the simulated E-bookstore Webpage was chosen individually by each participant until she/he has finished all the question items of subjective preference questionnaire. In addition, each participant continuously viewed sixteen different searching results presented on Web pages of E-bookstore and a browsing task was assigned. The participants are asked to correctly recognize whether a targeted image has been shown or nor after they finished the browsing task. The questionnaire for user interface satisfaction (QUIS for short) will be implemented after finishing the experiment of performance measurement. 3.5 Apparatus and Materials This study uses a Pentium IV desktop computer (CPU1.62GHZ, 896MB RAM) with Microsoft Internet Explorer 6.0, a 17-inches TFT-LCD monitor (1280×1024 pixels). Micromedia Flash, Dreamweaver and Firework MX 2004 (copyright @Cyberlink Corporation) were used to design the simulated E-bookstore Webpage.
Usability Evaluation of Dynamic RSVP Interface on Web Page
59
3.6 Model Building of Recognition Based on the design of experiment, the objective function of browsing task is collected. Logistic regression model is appropriate for the fitting of probability of correct recognition [1]. Design factors include RSVP mode and moving speed. RSVP modes include carousel and shelf RSVP displays based on the major difference of circular and linear trajectory (Table 1). Moving speeds include four levels of 10~15 (the groups of 10 and 15 FPS being combined due to the consideration of sample sizes), 20, 30, and 40 FPS with the exposure time of 20~13.3, 10, 6.67, and 5 seconds per frame respectively. In addition, gender and college background variables are considered as individual difference. Define Y as the recognition variable of targeted image. The value of Y equals 1 if the participant could correctly recognize the targeted image; otherwise, Y equals 0. Let π be the probability of correct recognition, we have the odds of correct recognition to be [π/(1 - π )] . The logit function log [π/(1 - π )] of π , symbolized by “ logit (π ), ” is the log odds of correct recognition. Whereas π is restricted to the 0-1 range, the logit can be any real number. The proposed model for the fitting of probability of correct recognition initially concludes the main effects of gender, college, RSVP mode, and moving speed as well as the interaction of RSVP mode and moving speed and is shown as follows: π = P( Y = 1 | X ) =
exp (X ′β ) , 1 + exp( X ′β )
(1)
where X denotes the design matrix of two-factor interaction of RSVP mode (RSVP for short) and moving speed (FPS for short), gender (G for short), and college (C for short), that is, X ′ = [1 : G, C , RSVP | FPS] (use “|” for interaction), β is the parameter vector corresponding to X the logistic regression model is expressed as follows:
，
⎛ π ⎞ logit[P(Y = 1 )] = logit(π) = log⎜ ⎟ = X ′β , ⎝1− π⎠
(2)
The nominal-scale explanatory variable with k categories in Equations (1) and (2) is appropriate to be treated using (k-1) indicator variables. For example, design factor RSVP is a categorical variable with two categories using one indicator variable. FPS may be regarded as continuous or regarded as categorical with four categories. If FPS is regarded as nominal-scale explanatory variable, it is going to use three indicator variables to describe their four categories. The frequencies of recognition for the groups of 10 and 15 FPS are combined and renamed as FPS15. We have the value of FPS15 equals 1 if the moving speed is 10 FPS or 15 FPS; otherwise, FPS15 equals 0. Similarly, the value of FPS20 equals 1 if the moving speed is 20 FPS; otherwise, FPS20 equals 0. The value of FPS30 equals 1 if the moving speed is 30 FPS; otherwise, FPS30 equals 0. The value of FPS40 equals 1 if the moving speed is 40 FPS; otherwise, FPS40 equals 0. Let (G, C, RSVP, FPS15, FPS20, FPS30) each take values 0 and 1 to represent the nominal-scalar categories of explanatory variables. The coding values of indicator variables corresponding to the nominal-scale explanatory variables are described as follows:
60
Y.-L. Lin and D. Lin ⎧1, if Female , G=⎨ if Male ⎩0,
⎧1, if major in Art and Design C =⎨ if major in Management ⎩0,
⎧1, if carousel mode , RSVP = ⎨ otherwise ⎩0,
⎧1, if FPS = 20 FPS 20 = ⎨ ⎩0, otherwise
,
⎧1, if FPS = 10 or 15 , FPS15 = ⎨ otherwise ⎩0, ⎧1, if FPS = 30 FPS 30 = ⎨ ⎩0, otherwise
Rewrite Equation (2) as following: ⎛ π ⎞ logit[P(Y = 1)] = logit(π ) = log⎜ ⎟ = β 0 + β1G + β 2C + β 3RSVP + β 4 FPS15 ⎝1− π ⎠ + β 5 FPS20 + β 6 FPS30 + β 7 RSVP × FPS15 + β 8RSVP × FPS20 + β 9 RSVP × FPS30,
(3)
The parameter corresponding to indicator variable RSVP in Equation (3) are β 3 . The value of e β 3 represents the odds ratio defined as the ratio of correct recognition odds between carousel and shelf RSVP interfaces. The values of FPS15 = 1, FPS 20 = 0, FPS30 = 0 in Equation (3) are being substituted for moving speed of 15 FPS. The values of FPS15 = 0, FPS 20 = 1, FPS30 = 0 are being substituted for moving speed of 20 FPS. The values of FPS15 = 0, FPS 20 = 0, FPS30 = 1 are being substituted for moving speed of 30 FPS. The parameters corresponding to indicator variables FPS15 , FPS 20 , and FPS 30 in Equation (3) are β 4 , β 5 and β 6 . The value of e β 4 represents the odds ratio defined as the ratio of correct recognition odds between 15 FPS and 40 FPS. Similarly, the values of e β 5 and e β 6 represents the odds ratios of correct recognition between 20 FPS and 40 FPS as well as between 30 FPS and 40 FPS.
4 Results and Discussion 4.1
Comparison of Subjective Preference
Based on the result of subjective preference questionnaire, the favorite percentage distribution of moving direction for RSVP displays is shown in Figure 2. For shelf RSVP display, 57% of users chose the direction of moving from bottom-left to upperright as their favorite one, however, only 5% of users chose the direction of moving from bottom-right to upper-left (Figure 2(a)). The differences of subjective preference proportions among four moving directions of shelf RSVP display are statistically 2 significant ( χ = 25.62 , P-value
3.2 Pedagogical Aspects of the Textbook and the Video Lecture's Design Following constructivist principles of learning via problem-solving in authentic situations [7], the online textbook contains links to brainwork exercises, performance tasks and links to articles and authentic examples on the internet. The current study examined students' attitudes to the above components. As table 3 shows, items 1-3 assessed the actual use of the brainwork exercises by the students. It was found that the more demanding the tasks, the less students favored them (from high preference for examples (mean=3.94) to medium preference for performance tasks (mean=3.17). Nevertheless, the contribution of these components to students' understanding and motivation was found to be high (mean=4.2). Note that the 'components contribution to learning' measure was calculated as a mean of the scores of items 4-9 (Table 3). These items measure the contribution of knowledge construction, relevance to the learning themes, dynamic learning, understanding and internalization of the learning material, satisfaction and fulfillment from the learning and the level of interest in the texts. These six items were found to have high internal validity (Cronbach Alpha=0.91). 3.3 Video Lectures and Discussion Groups Video lectures, given by the course developer, as well as discussion groups, led by the course instructor, were included in the course's CLE in order to bridge the gaps between the course developer and the course instructor, the course instructor and the students and the students and their peers. More than 90% of the respondents reported that they observed at least one lecture and 87% of the respondents reported on the necessity of the video lectures. In addition, the possibility to listen to the lectures,
684
K. Precel, Y. Eshet-Alkalai, and Y. Alberton
Table 3. Students' attitudes toward the pedagogy of the course’s instruction and the influence of the learning environment design and content on learning processes
Exercises components in the textbook: To which extent did you: 1. perform the exercises in the textbook? 2. stop and think about the questions and issues raised? 3. stop and examine the examples the text refers to? The components' contribution to the learning process: 4. Knowledge construction 5. Was relevant to the learning themes 6. Dynamic leaning 7. The level of interest in the text 8. Gratification from learning 9. Understanding and internalization of the learning material Total 'components' contribution to learning' measure** 10. The online textbook's functional design lead you to refer to, think of, or understand the course's contents. Video lectures 11. The acquaintanceship with the course developer contributed to the learning experience 12. The lectures contributed to learning focalization in each unit 13. Listening to the lectures combined with the presentation and examples contributed to understanding the learning material Discussion groups (DG) 14. The satisfaction from the level of discussions 15. Organizing the discussion groups according to units contributed to focalization of discussions in the DG. 16. Organizing the discussion groups according to units contributed to receiving assistance when needed.
Number of respondents
Mean*
Stdv
48 47 48
3.17 3.40 3.94
0.97 0.90 0.81
47 46 46 48 48 47 448 45
4.15 4.33 4.22 4.23 4.21 4.15 4.21 3.67
0.81 0.70 0.81 0.81 0.82 0.81 0.67 1.13
50
3.78
1.18
50 32
4.00 4.40
1.16 0.80
49 49
3.59 3.96
0.84 0.96
49
3.86
1.10
*The answer's scale was 1 – "not at all" – 5 "Very much" **The measure was calculated as a mean of items 4-9 (internal validity, Cronbach Alpha=0.91)
combined with the presentation and the examples, was found to contribute the most to learning (mean=4.4, table 3). Almost all respondents (98.2%) visited the discussion groups. Most of them (67.9%) reported that they continuously followed the activity in the discussion groups while others visited them occasionally. Of all the respondents, 28.6% reported on active involvement in the discussion groups. The students' satisfaction from the discussions' level was found to be higher than average (mean=3.59). 3.4 Course Textbook Format: Reading from Print versus Digital Displays The optimal format for presenting the course's learning materials – in a printed or digital textbook – was examined in relation to three different learning assignments: reading, tasks' implementation and preparation for the final exam (Table 4). As can be seen from table 4, the overall preference of more than half of the respondents (57.9%) was for combining the printed and the digital textbook. Of all the respondents,
Learning by Design in a Digital World
685
Table 4. Students' preferences regarding the textbook's format (printed, digital or combination) in relation to various learning assignments
How do you usually read the course's textbook? Which book do you usually use to prepare the course's tasks? Which book do you prefer to use prior to the final exam? General preference*
Digital textbook % 10.3 15.8
Printed textbook % 50 57.9
Combination % 39.7 26.3
14
59.7
26.3
5.26
36.84
57.9
*This measure integrates the respondents' preferences of the three learning assignments into one measure in the following way: students who preferred the digital textbook in all assignments – "Digital textbook", students who preferred the printed textbook in all assignments – "Printed textbook", all other preferences – "Combination".
Table 5. Print versus Digital – reasons that influence the respondents' preferences
It is hard to read long texts from the computer's screen I'm used to reading and studying from printed textbooks The digital textbook enables easy access to examples of computerized learning environments and other references The printed textbook can be read everywhere It is easy to navigate in the digital textbook You can't mark or write notes in the digital textbook The digital textbook contains interesting information that can not be found in the printed version In the printed textbook you can find what you want easily The reading in the digital textbook requires time investment
High/large influence % 56.8
Little influence % 35.1
No influence % 8.1
Number of respondents 37
59.5
24.3
16.2
37
62.9
22.9
14.2
35
64.9
24.3
10.8
37
47.1
32.3
20.6
34
55.9
32.3
11.8
34
20.0
34.3
45.7
35
62.2
29.7
8.1
37
32.4
50.0
17.6
34
36.84% preferred the printed textbook solely while few students (5.26%) preferred the digital textbook solely. Results (Table 5) show that for most of the respondents (more than 60%), the most influential factors in choosing the printed textbook were the convenience of the printed book’s accessibility and the ease in finding information. The major reasons for choosing the digital textbook were the fast access to online examples of computer-based learning environments and the easy access to other links, which are embedded in the text.
686
K. Precel, Y. Eshet-Alkalai, and Y. Alberton
Table 6. Students' attitudes toward the design of the digital textbook and the video lectures Number of respondents The digital textbook: 1. Text's design – chosen font 2. Text's design – font's size 3. Text's organization in layers 4. Tasks' integration 5. Links and examples' integration 6. Navigation Total Usability measure** The video lectures: 7. Functional design of the video lecture's interface 8. Time length of the video lectures
Mean *
Stdv
46 46 46 45 45 44 46
3.41 3.41 3.46 3.74 3.64 3.39 3.50
0.62 0.62 0.69 0.63 0.53 0.75 0.50
49 48
3.45 3.25
0.68 0.86
*The answer's scale was 1 – "is not suitable for learning" – 4 "Very suitable for learning" **The measure was calculated as mean of items 1-6
3.5 Usability Aspects of Course Design Students' attitudes toward usability issues in designing the course's online textbook and website were examined in the current study (Table 6). Results indicate the students' high satisfaction from various usability aspects of the course's CLE (i.e. its ease of use and friendliness) and the digital textbook. Results show that the organization of the course's digital contents facilitated navigation and reading (mean=4.28). High scores were given to specific design elements, such as the font's type and size (mean=3.41 for both), text's organization (mean=3.46), the integration of tasks and examples in the course's CLE (mean=3.64) and the ease of navigation through the text and the CLE (mean=3.39). The general usability measure, as calculated from items 1-6 in table 6, was high (mean=3.5). In addition, the navigation in the course's CLE, which offers the students flexibility in reaching the course's contents 'from everywhere', was found to be highly usable and the students used this flexibility wisely and in various ways. For example, half of the respondents reached the course's readings via the "Articles" button in the course's CLE homepage, while 20% of the respondents reached it via links in the digital textbook or the "time-table" area in the course's CLE.
4 Discussion and Conclusions The purpose of the current study was to examine students' perceptions of pedagogical, design and usability issues regarding a fully-online course and its learning environment. Results of the current study make a meaningful contribution to our understanding of students' perceived value of learning and instruction in online environments. Students' high rating of the pedagogical and design course elements illustrates the great importance of 'designing in advance' (contrary to 'designing in retrospect'), which takes into account the problems involved in online learning in present-days academic courses [15]. The students' strong preference for the Blended Learning model, which was found in the current study, is compatible with reports by
Learning by Design in a Digital World
687
most current studies on online learning models. Our findings illustrate the need to adjust the instructional model to the content and the learning objectives [6]. Findings of the current study indicate students' high evaluation of the interactive learning components, such as discussion groups and constructivist tasks. Results of the current study reinforce the widely-reported students' preference for reading academic texts in a print format compared to the digital one [16], mainly because of navigation, availability and ownership reasons. Only few students preferred the digital over the printed textbook because of the accessibility to the online examples. Nevertheless, as indicated in many studies, (e.g. [10]), our knowledge of the nature of digital reading is not yet clear, emphasizing the need for solid research data in order to reach conclusions regarding the preferred format for reading academic texts. The high satisfaction from the usability components of the course's CLE, which was found in the current study, is exceptional compared to the general low satisfaction of LMS sites reported by many studies (e.g. [3, 4]). Extremely high satisfaction (mean=4.7 on a 1-5 scale) of the course's CLE was also found in the general course's instruction surveys that was given to students at the end of each semester. We believe that this high satisfaction is an outcome of the major investment in designing the pedagogical and usability elements of the course 'in advance'. Nevertheless, the usability of some components (i.e., the personal notebook) was evaluated as low. Further research is needed to clarify the reasons for these evaluations. It should be noted that results of the current study have a few limitations: (1) the sample was relatively small (2) participants were M.A. students in an educational technology graduate program, and many of them have higher computer skills than the average student. Thus, the high level of satisfaction found in the research might not represent students from other disciplines and (3) even though the questionnaire utilized in the study was modified from the Open University's standard instruction satisfaction questionnaire, it did not undergo through a large-scale validation process. In futures studies, after validating the questionnaire, special emphasis should be put on testing a larger group size, comparing students' attitudes from various disciplines and of different proficiency levels, and comparing online courses that are based on different pedagogical models. Notwithstanding, results of the current study shed new light on our understanding of the proper design of a blended online academic course: in-advance pedagogical and visual design of online learning. In addition, results indicate the potential of the current model in bridging the gap that is typical of online learning between students and instructors and students and their peers, and in creating meaningful learning by employing "online pedagogical considerations" to course design.
References 1. Andrews, R., Haythornthwaite, C.: The Sage Handbook of E-Learning Research. Sage Publications, L.A (2007) 2. Balcytiene, A.: Exploring individual processes of knowledge construction with hypertext. Instructional Science 27, 303–328 (1999)
688
K. Precel, Y. Eshet-Alkalai, and Y. Alberton
3. Bonk, C.J.: The perfect e-storm: emerging technology, enormous learner demand, enhanced pedagogy, and erased budgets. Part 1: Storms # 1 and #2. The Observatory on Higher Education (2004a), http://www.obhe.ac.uk/products/reports/ publicaccesspdf/Bonk.pdf 4. Bonk, C.J.: The perfect e-storm: emerging technology, enormous learner demand, enhanced pedagogy, and erased budgets. Part 2: Storms # 3 and #4. The Observatory on Higher Education (2004b), http://www.publicationshare.com/part2.pdf 5. Bonk, C.J., Wisher, R.A., Lee, J.: Moderating learner-centered e-learning: Problems and solutions, benefits and implications. In: Roberts, T.S. (ed.) Online collaborative learning: Theory and practice, pp. 54–85. Idea Group Publishing (2003) 6. Bonk, C.J., Graham, C.R. (eds.): Handbook of blended learning: Global Perspectives, local designs. Pfeiffer Publishing, San Francisco (2006) 7. Bransford, J.D., Sherwood, R.D., Hasselbring, T.S., Kinzer, C.K., Williams, S.M.: Anchoredinstruction: Why we need it and how technology can help. In: Nix, C., Spiro, R. (eds.) Cognition, Education and Multimedia: Exploring Ideas in High Technology, pp. 115–141. Lawrence Erlbaum Associates, Hillsdale (1990) 8. Cuban, L., Kirkpatrick, H., Peck, C.: High access and low use of technology in high school classrooms: Explaining an apparent paradox. American Educational Research Journal 38(4), 813–834 (2001) 9. Eshet, Y.: Teaching online: survival skills for the effective teacher. Inroads-The SIGCSE Bulletin 39(2), 16–20 (2007) 10. Eshet, -A.Y., Geri, N.: Does the medium affect the message? The influence of text representation format on critical thinking. To be appear in Human Systems Management 26(4) (2007) 11. Graham, C.R.: Blended Learning Systems: Definition, Current Trends, and Future Directions. In: Bonk, C.J., Graham, C.R. (eds.) Handbook of blended learning: Global Perspectives, local designs. Pfeiffer Publishing, San Francisco (2006) 12. Guri-Rosenblit, S.: Eight paradoxes in the implementation process of e-learning in higher education. Higher Education Policy 18, 5–29 (2005) 13. Lazenby, K.: Technology and educational innovation: A case study of the virtual campus of the University of Pretoria. Doctoral Dissertation. The University of Pretoria: Pretoria, South Africa (2003), http://upetd.up.ac.za/thesis/available/etd03172003-094954 14. Osguthorpe, R.T., Graham, C.R.: Blended learning environments: Definitions and directions. The Quarterly Review of Distance Education 4(3), 227–233 (2003) 15. Shemla, A., Nachmias, R.: How Do Lecturers Integrate the Web in Their Courses? WebSupported Courses at Tel-Aviv University. In: Pearson, E., Bohman, P. (eds.) Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2006, pp. 347–354. AACE, Chesapeake (2006) 16. Spencer, C.: Research on learners preferences for reading from a printed text or from a computer screen. Journal of Distance Education 21(1), 33–50 (2006)
Promoting a Central Learning Management System by Encouraging Its Use for Other Purposes Than Teaching Franz Reichl and Andreas Hruska E-Learning Centre, Vienna University of Technology, Gusshausstr. 28/E015, A-1040 Wien, Austria {Franz.Reichl,Andreas.Hruska}@elearning.tuwien.ac.at
Abstract. Vienna University of Technology’s E-Learning Centre introduced Moodle in 2006 as the university’s central learning management system. Custom interfaces to existing IT infrastructure as well as modules developed and deployed according to user’s needs provide for a seamless digital workflow in the university’s teaching, learning and organisational processes. Encouraging the use of the LMS as a multi-purpose tool to support all kinds of co-operation and communication activities in addition to curricular teaching led to a rapid and significant increase in user numbers and encouraged university teachers to deal with educational questions and to develop innovative learning and teaching solutions, based on the LMS. Keywords: e-learning, learning management systems, Moodle, learning communities, communities of practice, collaborative work, business integration.
1 Background Vienna University of Technology’s administrative software TUWIS has grown over more than 40 years; modules to satisfy urgent needs and demands have been added step-by-step, resulting in a very heterogeneous and expensive to maintain enterprise information technology landscape with complex interfaces. Concerning e-learning, the situation was similar until a few years ago. Entrepreneurial teachers and researchers at Vienna University of Technology had carried out many successful projects and activities beyond self study which were able to demonstrate that immense benefits can be achieved by media supported learning, e.g.: • iChemEdu [1, 2] developed an internet-based laboratory information and management system, iChemLab, an e-book based e-content pool, iChemLecture, and an e-self-assessment tool, iChemExam; an ever growing database application initially containing more than 450 detailed synthetic experimental protocols has been developed by extracting and revising information from some 8,600 student work reports collected over the last years; • MODULOR and virtual campus for architecture [3]: to support face-to-face learning, the faculty for architecture started to implement a virtual campus, consisting of a learning portal, a media database, and groupware and courseware tools; • blended learning in continuing education, with active facilitation [4]. M.J. Smith and G. Salvendy (Eds.): Human Interface, Part II, HCII 2009, LNCS 5618, pp. 689–698, 2009. © Springer-Verlag Berlin Heidelberg 2009
690
F. Reichl and A. Hruska
Many of the successful and sustaining projects have developed their own hardware and software solutions. While some of these systems contain interfaces to the university’s administration system, many of them were implemented as stand-alone solutions with their own registration and authentication modules – thus resulting in redundant and partly inconsistent data. 1.1 Evolving E-Learning and E-Teaching Strategies In 2004, a new University Law released the Austrian universities into autonomy and gave them the opportunity and the obligation to develop their own strategies. Vienna University of Technology founded an E-Learning Centre as an organisational and administrative structure to consolidate earlier e-learning and e-teaching initiatives and to make the experiences and developments sustainable and applicable for a wider group of users. The E-Learning Centre supports teachers and students in all departments and all fields of study and maintains and permanently improves e-education tools. The university’s strategy aims at improving the quality and efficiency of their study offers: the application of new media shall intensify learning processes, improve learner’s perception of complex subjects, and it shall enable particular groups of students to participate in courses who would otherwise be disadvantaged. The university’s medium to long term goal is to support each basic course in initial studies by means of e-learning – not by replacing the face-to-face courses, but by establishing blended learning as the standard in learning and teaching. In parallel, learning provisions are improved with respect to quality management. Establishing quality standards follows state of the art processes as defined by ISO 9000 and ISO 9126 and described by various publications, e.g. [5]. According to Hagner’s characterisation of technology adaptors (distinguishing between entrepreneurs, second wave, reward seekers, and reluctants [6]), Vienna University of Technology aimed at creating a large group of second wave adopters by consolidating entrepreneurs’ initiatives and by building a “learning community of practice” among the university’s teaching professionals. To take the reward seekers on board, the rectorate has initiated an annual E-Learning Award for outstanding achievements in teaching with new media, encouraging teachers to share their achievements with colleagues. To create synergies, three significantly different universities in Vienna (University of Technology, University of Natural Resources and Applied Life Sciences, Academy of Fine Arts) collectively developed, implemented and evaluated their e-learning and e-teaching strategies; they joined forces in the project Delta 3 [7, 8, 9, 10, 11] which was carried out from October 2005 to September 2007 and was co-funded by the Austrian Federal Ministry for Education, Science and Culture during the tendering for e-teaching/e-education strategies at universities and Fachhochschulen [12]. 1.2 Implementing E-Learning Services and Infrastructure A successful blended learning strategy requires comprehensive offers for support, consultation, and qualification of university teachers, students and managers. Since the university followed a holistic approach for implementing e-learning strategies,
Promoting a Central Learning Management System
691
dealing with strategic, organisational, administrative, financial, and legal aspects, didactics and curricular integration, competence and expertise, acceptance and incentives, public relations and marketing, and quality assurance (compare [13]), efficient and effective implementation of such a strategy requires to develop users’ competencies in many such areas. Thus, the E-Learning Centre developed a portfolio of information and qualification offers that can be rapidly adapted to different needs and demands of users at the university. Integrated support structures have been created which offer a wide spectrum of individualised, problem oriented (or rather solution oriented), low threshold services to university teachers, adequate for any need and any amount of time available, ranging from online FAQs and a helpdesk to consultancy, information events and workshops, as well as coaching and guidance [11].
2 Implementation and Application of Moodle as the Central LMS The use of a centrally managed and maintained learning platform significantly increases the efficiency for teachers and students, leads to a more homogeneous presentation of learning provisions, and reduces management and support overhead. The E-Learning Centre of Vienna University of Technology implemented the LMS (Learning Management System) Moodle (Modular Object-Oriented Dynamic Learning Environment) as its central LMS under the university specific “brand name” TUWEL (Technische Universität Wien E-Learning) with specific adaptations towards the university’s corporate design, with interfaces to existing services, and with features and functions in order to support specific teaching and learning processes. Moodle is open source software with currently more than 28 million users on approximately 50,000 registered validated sites in more than 200 countries [14]. The most important extensions to Moodle were features allowing the integration of the LMS TUWEL into the university’s existing administration systems, thus enabling the use of existing data without having to duplicate them. Access to TUWEL is controlled via the university’s centrally managed authentication system so that all staff members and students can use the learning platform immediately, without additional administrative entry barriers. Information provided by TUWIS upon login is used by TUWEL to determine authorisation and rights of users, so that teachers have access to all the courses provided by their department. Upon every login, user data in TUWEL are updated from data in the central administration system. Additional interfaces connect the already existing e-learning tools (e.g. iChemLab, iRecord) to TUWEL and thus also to the administration system. Data concerning authentication, authorisation, enrolment, group management and meta-data on lectures are imported into TUWEL, and grades can be calculated from results of learning processes and exported from TUWEL back into the administration system TUWIS. All these features are implemented in a Moodle block “TUWEL Toolbox” so that they can be accessed easily from within each TUWEL course [15, 16]. On the other side, TUWIS (which contains all the relevant information) has been enhanced with a menu entry to generate and to announce a course in TUWEL, based on available data from TUWIS; this feature also automatically generates links between course entries in TUWEL and in TUWIS [15, 17].
692
F. Reichl and A. Hruska
2.1 Specific Features The standard open-source software package Moodle has been enhanced with special features oriented at educational requirements and based on users’ needs and requirements to support faculty specific learning processes, e.g.: • “checkmark assignments”: in many courses, students have to solve exercises at home and have to declare (by entering checkmarks into a list of exercises) before the start of a face-to-face group meeting which exercises they were able to solve; during the meeting, randomly selected students have to demonstrate their solutions; the process of submitting the “checkmarks” and the grading of the assignment are represented in TUWEL’s assignment type “checkmark assignment”; • advanced “working group” handling: for splitting large numbers of learners into groups, student working groups can be built in TUWEL in addition to the groups defined in TUWIS; teachers define the number of groups and the maximum number of participants per group; students can then form groups with their colleagues, and a learning product jointly developed for an assignment can be uploaded by one of the students; teachers are able to grade such an assignment for the whole group; • “activity reports”: this feature provides teachers with the possibility to anonymously contact groups of students depending on their activity (or non-activity) level with regard to specific assignments or downloads of documents; • advanced display features, e.g.: mathematical notation, rendering of LaTeX notation, molecular rendering of Jmol descriptions as 3D visualisations, syntactic highlighting of programming language code, displaying mind-maps created with the open-source software freemind via a flash plug-in [15, 18]; • advanced scheduling: a powerful scheduler enables teachers to announce a number of time slots to their students, e.g. for oral examinations, for discussing the results of their exercises, or for meetings during office hours; several teachers and tutors of a course may generate time slots, and students are able to see all time slots provided by all relevant teachers and to register for a specific time; • iRecord: a media-based e-portfolio has been developed by the faculty for architecture; a specific TUWEL assignment feature allows the submission of documents from TUWEL assignments directly into the iRecord system [15]. 2.2 Promoting the LMS for E-Learning, E-Teaching – and beyond A low threshold usage of the centrally maintained learning platform was solicited, aiming at first increasing the quantity of e-learning provisions; later on, every course shall use the centrally supported learning management system and apply those of its functions (administration, content-presentation, organisation of learning activities, communication, feedback, assessment) which are relevant for the specific course. We correctly expected that awareness for the quality of (online) learning and teaching would rise at a later stage, after teachers took their first attempts; like other universities, we experienced that the application of an LMS and the availability of support and training have encouraged the university teachers to deal with educational questions and to develop innovative solutions, thus influencing teaching in a positive way [11].
Promoting a Central Learning Management System
693
Fig. 1. TUWEL LMS use by semester (number of teachers includes staff and tutors)
TUWEL went online in March 2006 with less than 50 courses created by teachers of the computer science faculty. The big challenge was – and still is – to promote the benefits of deploying the platform for teaching purposes. In October 2008, TUWEL provides approximately 350 different courses per year (see Fig. 1). The platform is used almost any time and by up to 5,870 different users per day (see Fig. 2). Like at many other universities, the centrally supplied LMS is used by approximately 15% of the teachers. However, many learning activities applying new media have evolved in parallel (e.g. [19, 20]). As expected, the platform was most intensively used by the faculties and departments with the largest number of students (Informatics: 5.277 students, 874 beginners in winter term 2008/09; Architecture: 3,968 students, 842 beginners in winter term 2008/09 [21]). The largest “regular” courses for students, with up to 800 participants, are – as expected – from these two faculties. However, the LMS TUWEL is not only used for courses in regular curricula but also for courses for the university’s staff members, e.g.: • a welcome-event “getTUgether” is offered twice per year and provides new staff members with information about the university’s structure and organisation, services and service units; • specific information and training is offered for security officers and fire wardens; • the university’s library offers introductory courses to students and researchers; • introduction to the LMS’s features is also provided in TUWEL in several formats (e.g. content creation tutorials, TUWEL tutorials).
694
F. Reichl and A. Hruska
Fig. 2. Increase in the number of different TUWEL users per day
Interestingly enough, the university’s management and the central university offices and service departments very soon discovered the capabilities and benefits of such a platform which is easily available and accessible to all university employees and students. They thus use the TUWEL platform regularly for communication, co-operation and administrative purposes, e.g.: • the E-Learning Centre itself uses TUWEL to administer the annual E-Learning Award: general information is provided in “online resources”, the upload of applications is handled by TUWEL’s assignment feature which allows the jury members to download and evaluate the applications; • in the year 2006, the university had to decide whether to build a new campus at the outskirts of Vienna or to expand the capacity of its buildings located downtown; the rector’s office used TUWEL’s “online resources” to present different concepts, invited staff members and students to discuss pros and cons in TUWEL’s forums and deployed Moodle’s “feedback” module to survey university members’ opinions and preferences; • so far, there is no other centrally maintained tool for content management and for computer supported co-operative work at the university; various commissions and planning committees thus use TUWEL’s upload and download features (online resources and forum entries with attachments) to exchange and co-operatively work on documents; • the medical service department organises appointments with staff members for health checks and specific programmes, e.g. vaccination campaigns, courses, and consulting services: staff members are in the role of course participants, the medical staff acts as teachers; during the summer term 2008, more than 600 appointments have been scheduled, deploying TUWEL’s advanced scheduling;
Promoting a Central Learning Management System
695
• deploying Moodle’s “feedback” module, the teachers’ and researchers’ association ran a survey to investigate on job satisfaction, making use of TUWEL’s possibility to guarantee that only staff members were able to fill out the form (and only once per person) and that the replies were completely confidential; • physics teachers co-ordinate contents and learning material of their courses: they use Moodle’s “database” module to collect and retrieve data on the relationship between courses, teachers and learning material and to organise for peer reviewing and co-ordinating their teaching material. We originally expected the departments with the largest number of students to be the ones offering the courses with the largest number of students – but this is not the case. Some non-curricular activities which are implemented in the TUWEL LMS as courses have well over 1,000 registered participants – they are thus larger than all the “regular” courses offered to students (see Fig. 3): The largest (by number of participants) currently offered course is the TUWEL Tutorial introducing teachers to the use of the LMS with 1,362 participants, followed by the “Medical Services” course with 1,214 participants. The largest course in TUWEL so far has been the “Location Debate” in 2006 with 2,925 participants who generated 13,500 “action log entries” for this specific course on the most intensive single day. Especially the “Location Debate” (which started shortly after TUWEL had been launched) involved many of the university’s staff members, and it thus provided more than 1,000 teachers with a positive first contact to the LMS; within only a few days, the number of TUWEL users increased by approximately 1,500, providing the E-Learning Centre with a “jump start” for many of the university’s established teachers. In 2006 and 2007, the E-Learning Centre of Vienna University of Technology provided essential e-teaching training for two persons of (nearly) every department
1 2 3 4 5 6 7 8 9 10 11 12
course Location Debate (Rector's Office) TUWEL Tutorials (E-Learning Centre) Medical Service Department Software Engineering and Project Management Beginning Students' Information and Orientation Algorithms and Data Structures Building Theory Mathematics 1 Social Issues in Computing Urban Development Fundamentals of Computer Science Introduction to Computer Programming
13
Teachers' and Researchers' Association
curriculum non-curricula non-curricula non-curricula Informatics Architecture Informatics Architecture Informatics Informatics Architecture Informatics Mechanical Engineering non-curricula
Fig. 3. Largest TUWEL courses by number of participants
participants 2,925 1,362 1,214 778 772 749 745 707 669 574 554 415 406
696
F. Reichl and A. Hruska
who further on act as multipliers and provide first-level support for the department’s teachers, further increasing LMS user numbers. Since 2007, all new staff members get in touch with TUWEL for the welcome-events “getTUgether”.
3 Outlook In January 2008, Vienna University of Technology started the Enterprise Application Integration project TISS (TU Wien Information Systems and Services, [22, 23]) which will replace the old TUWIS System step by step with a common technological architecture and will enable an interactive application management, supporting the addition of new services. Existing systems will be integrated into a homogeneous, consolidated platform. TISS will provide TUWEL which has grown into an important tool for learning and teaching with interfaces for integration. Features currently offered by TUWIS, TUWEL, and other platforms in parallel will be harmonised further. TUWEL’s features will thus become even easier to use, more effective and efficient. This development will increase university teachers’ motivation to use the LMS to improve their teaching provisions.
4 Conclusions In order to utilise the full potential of a centrally managed and maintained learning management system for increasing efficiency for teachers and students by leading to a more homogeneous presentation of learning provisions, the E-Learning Centre of Vienna University of Technology promoted the benefits of applying the platform for teaching purposes by integrating the LMS into the university’s central administration system, by adapting the LMS to specific needs and demands of its users, and by encouraging the application of the LMS various purposes. The learning management system which had originally been introduced to support teacher-student communication for learning purposes thus has become a multipurpose tool supporting all kinds of other co-operation and communication activities, due to the benefit of having a well introduced interface. The E-Learning Centre actually encourages this originally not intended use of the learning platform to further promote its application for teaching purposes. Teachers who had not come in touch with efficient and effective use of online communication and co-operation before experience the benefits at first hand. This gets them start thinking about introducing ICT supported processes into their course concepts, consequently creating new courses and increasing the number of courses available, but also dealing with questions of educational quality and increasing the quality of the courses.
References 1. Fröhlich, J., Krebs, H., Lohninger, H., Untersteiner, F., Gärtner, P.: iChemEdu - the e Learning concept of the Faculty of Technical Chemistry at the Vienna University of Technology. ZIDline 12 (2005), http://www.zid.tuwien.ac.at/zidline/zl12/ 2. http://www.ichemlab.at/ (March 6, 2009)
Promoting a Central Learning Management System
697
3. http://modulor.tuwien.ac.at/ (Feburary 27, 2009) 4. Reichl, F., Vierlinger, U.E., Obermüller, E.: Active Learner Support for eLearning in Continuing Engineering Education: Theory and Practice. In: CIEC - Conference for Industry and Education Collaboration, Savannah (2005) 5. Ehlers, U.-D., Goertz, L., Hildebrandt, B., Pawlowski, J.M.: Quality in e learning. Use and dissemination of quality approaches in European e learning. A study by the European Quality Observatory. Office for Official Publications of the European Communities, Luxembourg (2005), http://www2.trainingvillage.gr/etv/publication/download/ panorama/5162_en.pdf (Feburary 27, 2009) 6. Hagner, P.R.: Interesting practices and best systems in faculty engagement and support. National Learning Infrastructure Initiative (NLII) White Paper (2001), http:// www.educause.edu/ir/library/pdf/NLI0017.pdf (Feburary 27, 2009) 7. http://www.delta (Feburary 27, 2009) 8. Csanyi, et al.: (AutorInnenkollektiv des Projekts Delta 3): Delta 3. Ein eStrategie–Projekt der Technischen Universität Wien, Universität für Bodenkultur Wien & Akademie der bildenden Künste Wien. In: Seiler Schiedt, E., Kälin, S., Sengstag, C. (eds.) E-Learning Alltagstaugliche Innovation?, pp. 97–107, Waxmann, Münster (2006) 9. Fröhlich, J., Herbst, I.R., Reichl, F.: Delta 3 – ein interuniversitäres Projekt zur Entwicklung und Umsetzung von e Learning-/e Teaching-Strategien an den Partnerinstitutionen. ZIDline 13 (2009), http://www.zid.tuwien.ac.at/zidline/zl13/delta3.html (Feburary 27, 2005) 10. Henkel, B., Herbst, I., Krameritsch, J.: Delta3. Ein Strategieprojekt der Technischen Universität Wien, Universität für Bodelkultur Wien & Akademie der bildenden Künste Wien. 11. Europäische Jahrestagung der Gesellschaft für Medien in der Wissenschaft, Zürich (2006) 11. Reichl, F., Csanyi, G.S., Herbst, I.R., Hruska, A., Obermüller, E., Fröhlich, J., Michalek, C.R., Spiegl, A.: Delta 3 - A Strategic E Education Project Creating Added Value from Complementarity. In: Luca, J., Weippl, E. (eds.) Proceedings of ED-MEDIA 2008 World Conference on Educational Multimedia, Hypermedia and Telecommunications,Association for the Advancement of Computing in Education (AACE), Chesapeake, pp. 465–473 (2008) ISBN 1 880094 65 7 12. http://strategie.nml.at/ (Feburary 27, 2009) 13. Kleinmann, B., Wannemacher, K.: E-Learning an deutschen Hochschulen. Von der Projektentwicklung zur nachhaltigen Implementierung. HIS Hochschul-Informations-System GmbH (2004) 14. http://www.moodle.org (Feburary 27, 2009) 15. Hruska, A., Potocka, K., Reichl, F.: Mit zwei Klicks zum neuen TUWEL-Kurs. ZIDline 18 (2008), http://www.zid.tuwien.ac.at/zidline/zl18/tuwel_kurs/ (Feburary 27, 2008) 16. Hruska, A., Potocka, K., Reichl, F.: E-Learning mit TUWEL - Tendenz stark steigend. ZIDline 16 (2007), http://www.zid.tuwien.ac.at/zidline/zl16/tuwel/ (Feburary 27, 2007) 17. Hruska, A., Potocka, K., Reichl, F.: TUWEL - News WS2008 & Moodle Konferenz, ZIDline 19 (2009), http://www.zid.tuwien.ac.at/zidline/zl19/ (March 6, 2009) 18. http://freemind.sourceforge.net/wiki/index.php/Main_Page (Feburary 27, 2009)
698
F. Reichl and A. Hruska
19. Csanyi, G.S., Jerlich, J., Pohl, M., Reichl, F.: Formal and Informal Technology Enhanced Learning for Initial and Continuing Engineering Education. In: IACEE 11th World Conference on Continuing Engineering Education (WCCEE), Atlanta (2008) 20. Csanyi, G.S.: Informal Learning in the Context of Formal Academic Education - Some Factors of Success. In: Szűcs, A.,, T. (eds.) New Learning Curtures - How do we learn? Where do we learn? EDEN Annual Conference 2008 - New Learning Cultures, Lisbon. EDEN - European Distance and E-Learning Network, Budapest (2008) 21. http://www.tuwien.ac.at/wir_ueber_uns/zahlen_und_fakten/daten/ (Feburary 27, 2009) 22. Kleinert, W., Grechenig, T., Költringer, T., Bernhart, M., Knarek, A., Schönbauer, F.: The Making of TISS: Juni 2008. ZIDline 18 (2008), http://www.zid.tuwien.ac.at/zidline/zl18/ (March 6, 2008) 23. Suppersberger, M., Bachl, S., Staud, P., Knarek, A., Kleinert, W.: TISS – Planen der Straßen und Roden im Dickicht. ZIDline 19 (2009), http://www.zid.tuwien.ac.at/zidline/zl19/ (March 6, 2009)
Framework for Supporting Decision Making in Learning Management System Selection Yuki Terawaki The University of Tokyo Graduate School of Arts and Sciences 3-8-1 Komaba, Meguro-ku, Tokyo 153-8902, Japan
[email protected] Abstract. When introducing computer systems as well as the e-Learning systems such as Learning Management System (LMS), requirements analysis is very important. In recent years, it has been noted that most of the people who introduce e-Learning systems in Japan are non-professionals of information technology. Generally, requirements analysis is well known for being very difficult for the information technology professional, and even more difficult for the non-professional. Therefore, this paper proposes a framework for supporting decision-making in Learning Management Systems selection for the nonprofessional. Using Analytic Hierarchy Process (AHP) and Quality Function Deployment (QFD), this framework recommends a LMS based on the priority of requirements. Keywords: Requirements Engineering, RE, decision making, Analytic Hierarchy Process, AHP, Quality Function Deployment, QFD, Learning Management System, LMS.
1 Introduction In Japan, information systems used for education support, such as e-Learning, have been intergraded into higher education. Unfortunately, the number of people who are qualified to introduce the e-Learning system is low. Furthermore, those people are not always information technology professionals. This creates a unique situation which requires unique attention. When introducing the computer systems, requirements analysis about e-Learning is very important, as well as usual computer systems. Requirements analysis must include identifying stakeholders and acquiring their needs, and modelling these by defined languages. There are a number of inherent difficulties in this process. Generally, requirements analysis is well known for being very difficult for the information technology professional, and even more difficult for the non-professional. Therefore, this paper presents a framework for supporting decision making in Learning Management Systems (LMS) selection for the non-professional. Using Analytic Hierarchy Process (AHP) and Quality Function Deployment (QFD), this framework recommends LMS based on the priority of requirements. The framework is developed as a web-based application, and then the appropriateness is discussed through experimentation. M.J. Smith and G. Salvendy (Eds.): Human Interface, Part II, HCII 2009, LNCS 5618, pp. 699–707, 2009. © Springer-Verlag Berlin Heidelberg 2009
700
Y. Terawaki
This paper is organized as follows. Section2 outlines AHP and QFD, and reviews some related Requirements Engineering (RE) works in order to consider for appropriateness of these techniques. Section3 describes proposed framework. Section4 presents LMS selection support system developed as web-based application nameed JULY. Section5 describes experimentation. Section6 accounts for experimental results. Section7 discusses the availability of this work. Finally, Section8 concludes this research.
2 Supporting Decision Making for Learning Management System In our daily lives we have to make decisions very regularly and choose from a myriad of options. We choose the best candidate in consideration of various factors like knowledge of the chosen object, standard, speculation, and preference, etc. However, it is not easy to say that all factors will be considered when decision making done. Moreover, if knowledge and the standard about the chosen object are clear, it is comparatively easy to choose, but if not, when comparing them, bias will occur. A major technique of the decision-making method to deal with such a problem has the AHP. Furthermore, this research aims to support non-professionals of information technology, and people who have a lack expertise with LMS function and the requirements for LMS. To support these people, this research adds QFD to AHP. QFD helps to decrease language mismatches between customer needs and LMS features. By using QFD, a subjective requirement of non-professionals can be correlated with the feature of LMS function and the professionals’ objective LMS information. As a result, the non-professional is able to understand the functional elements of LMS and can utilize the AHP. The next section outlines AHP and QFD, and the appropriateness of the use of these techniques are discussed by referring to earlier studies of RE. 2.1 AHP: Analytic Hierarchy Process AHP was developed by T.L. Saaty in the 1970's [1]. AHP is a structured technique for helping people deal with complex decisions. AHP is also a theory of measurement through paired comparisons and relies on the judgments of experts to derive priority scales. AHP first decompose user’s decision problem into a hierarchy. The hierarchy is composed of three elements. There is the goal, the criteria and the alternatives, with the goal at the top, the alternatives at the bottom, and the criteria in the middle. Users of the AHP obtain rankings and weighting through paired comparisons of items in each criterion that are converted to normalized rankings using the Eigenvalue method. This means that the relative rankings of alternatives are presented in ratio scale values that total one. 2.2 QFD: Quality Function Deployment In the 1970's QFD was developed by Yoji Akao, Shigeru Mizuno and others. QFD is recognized as a way to expand and implement the view of quality [3]. QFD shows that it is particularly beneficial for interdisciplinary communication, clear understanding of customer and/or user requirements, consent about the solutions found, completion of documentation of all steps taken, a profitable product and satisfied
Framework for Supporting Decision Making in Learning Management System Selection
701
customers. It has been widely applied in many industries worldwide, such as automobile, electronics, food processing, computer hardware and software ever since. QFD gradually decomposes the work forming the quality by the purpose and the means. QFD uses matrices to organize and relate pieces of decomposed data to each other. These matrices are often combined to form a basic tool of QFD. In the RE and Software Engineering (SE) communities, QFD is called Software Quality Function Deployment (SQFD) [2][3]. SQFD focuses on improving the quality of both the software development process and the product. It has been applied to improve software quality in many large organizations. As a result, it's recognized as being able to enhance communication between customers and software developers and testers. It also improves customer satisfaction. 2.3 To Calculate Weight of User's Requirements AHP has received some interest in the RE and SE communities. For instance AHP is used as part of the development of systems based on COTS (Commercial-Off TheShelf)[4][5]. Maiden et al. [5] not only confirmed usefulness of AHP but also debated the issue when they are proposing a template-based method called PORE (Procurement-Oriented Requirements Engineering) to support off-the-shelf systems selection. According to Maiden et al., AHP should be used to weight customer requirements, but not to determine product compliance to these requirements. This research agrees on weighting customer requirements. Moreover, they pointed out the following problems: (i) Assumption for using AHP that all criteria are independent is not satisfied. (ii) As the number of judgments increases, paired comparison, also increases. These problems are discussed at not only RE communities but also communities that consider calculation method of AHP, such as Operations Research. As previously described, AHP normalized the weight sum to be one. For this reason, when there is a subsidiary between criteria, and due to newly added criteria, the original weight will change. However, assumption that all criteria are independent is not always satisfied. Generally, it is difficult to remove a subsidiary completely. Therefore, this research uses the technique proposed by Ichihashi [6]. Ichihashi normalized the weight of criteria’s maximum value to be one. 2.4 Decomposition of Users’ Requirements When users of AHP make paired comparison, they use knowledge about criterion. Therefore, being limited only to the use of AHP cannot support non-professionals of information technology. Hence, it is important to provide requirements using nontechnical language for users of this framework (the target user). Moreover, paired comparison become available by determining these requirements of nontechnical language. Actually, the problem of vocabulary is a critical issue to be treated. According to Finkelstein et al. [4], there is a language mismatch between COTS features description and customers needs, and this mismatch increases the chances of selection failing. Therefore, it is necessary to decompose included requirements in unclear user's needs. Also, it is important to disclose the relationship between the decomposed user’s requirements and the LMS function (in other words, the relationship between nontechnical language and technical language).
702
Y. Terawaki
In RE communities, Goal-Oriented Requirements Engineering (GORE) has been recognized as a leading technique of eliciting or decomposing unclear customer needs. To elicit the requirements for an expected system from stakeholders, GORE uses a hierarchical tree structure. Directly borrowed from problem reduction methods in Artificial Intelligence, AND/OR graphs may be used to capture goal refinements. For instance, KAOS (Knowledge Acquisition in autOmated Specification) developed by Lamsweerede et al. [7] is the leading method of GORE. KAOS not only has features of GORE but also allows formal verification by applying temporal logic to each description of the goal. GORE has several advantages if this research uses it. However, QFD and GORE are similar in some respects. To obtain concrete requirements, GORE gradually decomposes a customer’s requirements from the highlevel goal to sub-goal (low-level goal). QFD decomposes requirements told by an ordinary customer’s voice, and makes a hierarchical list of customer’s requirements. The hierarchical list of customer’s requirements is combined with the hierarchical list created by technical personnel through decomposing data such as the function, operativeness, and flexibility, etc.,. Therefore, depending on the creation of matrices (QFD), representation of the hierarchical tree of GORE can be possible. To obviate mismatch of language, and to make AHP available, this research needs to provide the target user the told requirements by using a user's (or customer’s) voice. Moreover, a requirement by target user's voice demands a translation to the features of LMS function. Therefore, to propose framework, this research uses some ideas from QFD.
3 Concept and Calculation This framework proposes recommended LMS based on priority of target user’s requirements even if the target users are not familiar with functions and requirements of LMS. As for calculating the priority of target user’s requirements, paired comparison by the target user becomes available through providing the told requirements by using a user (or customer) associated with functional elements of LMS. Fig. 1 shows conceptual diagram of the framework.
Fig. 1. Conceptual Diagram
Framework for Supporting Decision Making in Learning Management System Selection
703
Strictly speaking, the typical framework user is a non-professional of information technology or technical personnel (person skilled in information technology and LMS). To provide the requirement statements associated with functional elements for non-professionals of information (target user), this framework requires the knowledge of technical personnel in the first stage. In the first stage, a technical personnel needs to create two data files for this framework: One is a “Requirements- Function Table” and the other is a “LMS- Function Table” (in this research I created the two data files). These data files are created in order to approach requirements of the target user by using QFD. The “Requirements- Function Table” represents correlations between user’s (or customer) requirements and LMS functions. The “LMS- Function Table” represents correlations between LMS names and LMS functions. The “RequirementsFunction Table” and the “LMS- Function Table” creates the categories such as the requirements of BBS, Quiz, and Material using the KJ method. Thus, the decision hierarchy is built. In the next stage, the “Requirements- Function Table” and the “LMS- Function Table” are utilized for providing the recommended LMS. The framework represents a pair of requirement statements at once from the “Requirements- Function Table” for the target user. The target user then makes decision about them: for instance, which is more important. This means that the target user makes paired comparisons by himself. The priority of the target user’s requirements is then calculated. Furthermore, a recommended LMS is calculated by associating the priority of the target user’s requirements with the “LMS- Function Table”. The calculation method is as follows. The priority of the target user’s requirements obtained by using AHP is defined as follows.
W r = [w1
w2
... w 3 ]
T
(1)
Matrix R, the “Requirements- Function Table” is defined as follows: rows are number of requirements ‘m’, columns are number of function ‘n’, and entries are ‘(m, n)’.
R = [rmn ]
(2)
Matrix L, the “LMS- Function Table” is defined as follows: rows are number of LMS ‘m’, columns are number of function ‘n’, and entries are ‘(m, n)’.
L = [lmn ]
(3)
The weight of function is calculated by the following expression.
W f = RT ⋅ W r
(4)
The weight of LMS is calculated by the following expression.
Wl = L ⋅ W f
(5)
In addition, the framework arranges weight of “ W l ” in descending order and then presents it to the target user as recommended LMS.
704
Y. Terawaki
4 Implementation This section presents the development environment and system features for the web based application, JULY. Development environment: The framework was developed as a web-based application called JULY. JULY is organized as follows. Server- side programs are written in the Ruby programming language. Two data files, the “Requirements –function Table” and the “LMS –function Table” are stored on server-side. Client- side programs use Asynchronous JavaScript + XML (Ajax) programming language. The web browser that a target user uses on the client side assumes FireFox and Internet Explorer. System features (Disclosure of requirement statements and recommended LMS): First, a target user chooses a category, then makes paired comparisons with the requirement statements contained within the chosen category. JULY provides only one pair of requirement statements on the screen at a time from the “Requirement Function Table” in random order for the target user. Depending on the context of the requirement statements, the target user needs to change the measurement scale of AHP when the target user makes paired comparison: for instance, the measurement scale needs to change to another convenient verbal, such as “likable”. Thus, when making paired comparisons intuitively, JULY utilizes graphics in order to represent a measurement scale. Finally, JULY provides the result screen including the recommended LMS, the priority of function and the priority of requirements.
5 Experimentation Subjects for the experiment were university staff with the potential to introduce LMS. There were a total of 20 subjects: 3 in their 20’s, 13 in their 30’s, 3 in their 40’s, and one person in their 60’s. 5 were bachelor’s degree holders, 10 were master's degree holders, and 5 were doctorate degree holders. The ratio of men to women was 3:1 and the ratio of Arts to Sciences was 2:3. Methods: Based on the scenario, the experiment was conducted through selecting a pseudo LMS. The scenario is described briefly as follows: a teacher who takes charge of a 300-student class wants to use the LMS to distribute materials with online, and to conduct a online quiz. There are two kinds of scenario: Scenario A has the requirements priority about distributing materials is high, Scenario X has the requirements priority about quiz is high. Depending on the scenario used for the examinee, this framework will calculate different recommended-LMS. Pseudo LMS were created based on the actual product. There are 5 pseudo LMS: one has more functions, and is suitable for the scenario; one has less functions, and is suitable for the scenario; one has more functions, and is moderately suitable for the scenario; one has less functions, and is moderately suitable for the scenario; one is not suitable for the scenario. One examinee executes the selection twice: one using JULY, and the other without using JULY. This experiment keeps the first experimental results secret to examinee.
Framework for Supporting Decision Making in Learning Management System Selection
705
If examinee does not use JULY, the functions forming LMS are given to them on an A3 card. This information is the same as JULY’s data. In addition, if the examinee matches the requirements of a scenario, this experiment assumes that the more multifunctional LMS becomes highly ranked. Collection method of data: There are two types of analysis data: one is the serverside log of JULY, the other is the time taken by examinee until obtaining a recommended LMS. The server-side logs collected the examinee’s action record: They show which category was clicked, and how long need time for a pair of requirement statements, and which measurement scale was clicked. The time taken by examinee was timed with a stopwatch when examinee selects LMS by using JULY, and without using JULY, respectively.
6 Experimental Result Tendency of selected LMS: Fig. 2 shows the number of examinees who obtained the ranking of LMS matching requirements of scenario. When using JULY, 14 examinees had obtained ranking of LMS which matched requirements of scenario. When examinee selects LMS without use of JULY, there were 8 examinee that matched requirements of scenario. Fig. 2 shows that examinees who failed to select LMS suited to the requirements tended to select LMS suited to the requirements by using JULY.
Fig. 2. The number of examinees who obtained the LMS meeting the requirements of scenario
Required time to answer a pair of requirement statements: Fig. 3 shows the vicissitude of time: time the examinees required to answer a pair of requirement statements. Fig. 3 shows that from the first question during the tenth question, examinee took from 15 seconds to 45 seconds per question. Thereafter examinee took from 5 seconds to 8seconds per question. Initially, examinee needs more time to understand JULY’s operation. However, once examinee understood JULY’s operation, examinee can make paired comparison within a relatively short time. The time taken to select LMS: Fig. 4 shows the data grouping the time taken when examinee selects LMS by using JULY, and without using JULY, and then measured T-statistics. Difference in the time of two populations is significant: t0 = 6.114 > t
706
Y. Terawaki
Fig. 3. The vicissitude of time: time the examinee required to answer a pair of requirement statements
Fig. 4. Result of T-statistics
(19, 0.02) = 2.861). In this result, it is clear that time is shortened when using JULY, rather than when LMS is selected by not using JULY.
7 Discussion This research shows the following result. 1. Examinees who failed to select LMS suited to the requirements tended to select LMS suited to the requirements by using JULY. 2. It is clear that time is shortened when using JULY, rather than when LMS is selected by not using JULY. When examinees do not use JULY, examinees cannot select LMS corresponding to requirements of the scenario. However, when examinees use JULY, examinees tend to be able to select LMS corresponding to requirements of the scenario. In addition, necessary time for selecting LMS is reduced by using JULY. This means that comparing all together five pseudo LMS is too difficult. When examinees do not use JULY, requirements are complexly intertwined with related factors such as fondness, speculation and comprehension for LMS. However, if
Framework for Supporting Decision Making in Learning Management System Selection
707
examinees use JULY, there are only two comparative requirement statements at a time. They were able to assess many requirement statements efficiently in a short time. In addition, requirement statements are easy to understand for examinees due to using QFD. This means that the “Requirements- Function” tables were created appropriately.
8 Conclusion There are a lot of existing LMS including commercial LMS or open-sourced LMS. Even if we specialized in accessible LMS when we introduce LMS, it is still difficult to make a decision about necessary function by recognizing all functions of each LMS. It is exceedingly difficult to compare a lot of functions all together, and to implement coherent estimate if knowledge and the standard are not clear. In addition, selecting LMS takes a lot time. To make up for the shortfall in human resources, this research proposed a framework consisting of adding QFD to AHP. This framework allowed recommendation of LMS based on priority of user’s requirements about LMS. Utilizing either AHP or QFD is not enough to satisfy the staffing shortage for introducing the LMS representing the eLearning system as described in section one. In this framework, the multicriteria decision making method AHP is available to non-professionals of information technology themselves, through providing non-technical terms for them, and by using the customer-oriented method QFD. The fusion of these methods was effective, and utilizing this framework will be contributory to solving the staffing shortage for introducing the e-Learning system in Japan.
References 1. Satty, T.L.: How to make a decision -The Analytic Hierarchy Process. European Journal of Operational Research 48 (1990) 2. Liu, X.F.: Software quality function deployment. Potentials IEEE 19(5), 14–16 (2001) 3. Herzwurm, G., et al.: QFD for Customer-Focused Requirements Engineering. In: Proceedings of the 11th IEEE International Conference on Requirements Engineering (2003) 4. Alves, C., Finkelstein, A.: Challenges in COTS Decision-Making- A Goal-Driven Requirements Engineering Perspective. In: Proceedings of the 14th International Conference on Software Engineering and Knowledge Engineering, pp. 789–794 (2002) 5. Naiden, N.A., Ncube, C.: Acquiring COTS Software Selection Requirements. IEEE Software 15(2), 46–56 (1998) 6. Ichihashi, H.: On a normalization of the grade of importance whose maximum value attains 1. In: 15th FUZZY System Symposium, Kobe, pp. 307–312 (1989) (in Japanese) 7. Darimont, R., Lamsweerede, A.v.: Formal Refinement Patterns for Goal-Driven Requirements Elaboration. In: Proceedings FSE4 4th ACM Symposium on the Foundations of Software Engineering, pp. 179–190. San Francisco (1996)
Statistics-Based Cognitive Human-Robot Interfaces for Board Games – Let’s Play! Frank Wallhoff, Alexander Bannat, J¨ urgen Gast, Tobias Rehrl, Moritz Dausinger, and Gerhard Rigoll Department of Electrical Engineering and Information Technology, Institute for Human-Machine Communication Technische Universit¨ at M¨ unchen 80290 Munich, Germany
Abstract. The archetype of many novel research activities is called cognition. Although separate definitions exist to define a technical cognitive system, it is typically characterized by the (mental) process of knowing, including aspects such as awareness, perception, reasoning, and judgment. This especially includes the question of how to deal with previously unknown events. In order to further improve today’s human-machine interfaces, which often suffer from deficient flexibilities, we present a cognitive humanrobot interface using speech and vision. The advancements against regular rule-based approaches will become obvious by its new interaction strategies that will be explained in the use case of a board-game and a robot manipulator. The motivation behind the use of cognition for human-machine interfaces is to learn from and adapt to the user leading to an increased level of comfort. For our approach, it showed proof that it is effective to separate the entire process into three steps: the perception of external events, the cognition including understanding and the execution of an appropriate action.1
1
Introduction
The excellence research cluster Cognition for Technical Systems CoTeSys is working on establishing a more intelligent and useful behavior of technical systems, in particular for unpredictable events in its surroundings. Therefore, cognition [1,2] could be the way to surpass the current constraints (rule-based and static behavior) incorporated in nearly all kinds of today’s operating technical systems. Such a cognitive system will monitor its environment and respond on changes. Furthermore, its skills will improve over time using generic learning algorithms. In general there are many possibilities where technical systems equipped with cognition can cover new areas of applications. The cluster of excellence focuses on ambient living [3] and advanced robotics [4], especially in an industrial context [5]. 1
All authors contributed equally to the work presented in this paper.
M.J. Smith and G. Salvendy (Eds.): Human Interface, Part II, HCII 2009,LNCS 5618, pp.708–715, 2009. c Springer-Verlag Berlin Heidelberg 2009
Statistics-Based Cognitive Human-Robot Interfaces for Board Games
709
However, the before mentioned two scenarios (ambient living and advanced robotics) entail many restrictions and constraints due to the technical state of the art as well as the field of application, therefore, we consider to work on a further field of application focusing on pure cognitive processing (playing games). The rest of this paper is organized as follows: In Section 2, a brief description of the hardware set-up in action is given. In Section 3, we introduce the underlying system architecture for the presented cognitive game application. Afterwards, in Section 4, we have a closer look on the actual interaction between the humanoid player and the cognitive playing counterpart with its actuator in form of an industrial robot manipulator arm. The paper closes with a summary and an outlook over the next planned steps.
2
Hardware Set-Up
The constructed hardware set-up is built by two major components. First, the actuator used to interact in the board game is an industrial robot arm. Second, for the presentation and scene observation a framework is mounted above the game interaction plane. This plane is equipped with the industrial robot arm. Besides, the necessitated computations for the cognitive game playing interactions and manipulation of the pawns are executed in a distributed manner, involving two processing units (standard PC with AMD Phenom 2.2GHz quad-core and four gigabyte RAM). One of these units is exploited to allow remote access towards the robot controller as well as a displaying module. The remaining unit is in charge of the cognitive game processing as well as scene surveillance. 2.1
Cognitive Interaction Unit
The industrial robot manipulator arm currently involved in the cognitive game playing process, is a Mitsubishi robot RV-3SB. This robotic interaction unit has one arm with six degrees of freedom and it can lift objects with a maximum weight up to three kilograms. The effective radius for performing gripping interactions is limited to a total of 0.642 m around the robot’s trunk. Its tool point is equipped with a force-torque-sensor and a tool-change unit. The instantaneous communication between the robotic arm manipulator and the game control unit is implemented via a network interface provided by a middle-ware similar to CORBA. This approach facilitates the interaction between different operating systems as well as different programming languages. Additionally, the outsourcing of the robotic control unit in an own module allows for an easy substitution and integration of other robotic manipulators in the future. A client-server model has been chosen as communication paradigm. The implemented server provides the client with three dimensional access to all reachable points of the robot arm. However, for the game operation we constrained the allowed movements in two parallel x-y-planes, see Figure 1. One plane is used for the translation of the pawns from one game field point to another. The pick-up and place operation is performed in the second movement layer, closer to the actual game plane.
710
F. Wallhoff et al.
Table Projection Unit Camera
z y
Cognitive Interaction Unit
x
Translation Plane
Pick-Up and Place Plane
Fig. 1. Overview of the working x-y-planes
2.2
Projection and Observation Units
A projection unit is attached in the framework mounted above the interaction plane. Via this projection unit the board game field and required interaction buttons are displayed onto the workbench at a total resolution of 1024 × 768 pixels. The obeyed vision sensor consists of a regular webcam with a resolution of 640 × 480 pixels running at 15 frames per second. Furthermore, a microphone with sufficient sound quality for speech recognition is also embedded in this device leading into a handy setup. These sensors allow for the recognition of speech commands, the recognition of movements and actions on the desk as well as the detection of gaming pieces and the number of eyes on the dice.
3
System Architecture
The connection and interaction of the above described sensors and actuator is formed by the well structured system architecture. This architecture has a modular concept, basing on the approach presented in [6,7,8]. The therein presented real-time database is capable of handling huge amounts of data from different input modules, having a different update rate. The desired module interaction is achieved together with the Internet Communication Engine (ICE) middle-ware. We will have a closer look on these participating modules in the following. 3.1
Dialog Manager
The management and adjustment of information exchange between the user and the system is realized by the dialog manager. In this case we use a commercial version of a dialog management system for speech recognition and speech synthesis. According to the current step within the game and perceived events,
Statistics-Based Cognitive Human-Robot Interfaces for Board Games
711
the system can utter information for the user or comment on these events. The speech recognition is primarily required at the start of the game for initializing the interaction with the player. Therefore, simple decisions have to be made, e.g. which color the pawns of the player should have. In these first interaction steps, the necessitated grammar for the speech recognition can be adapted in run-time to enhance the recognition rate. Afterwards, the state machine will transit into the playing mode, where the game logic and the related robotic actions are performed. Therefore, to enable a cognitive playing interaction, the “Cognitive Playing Strategy Module” takes over the control to create reasonable playing moves. 3.2
Cognitive Playing Strategy Module
This module is responsible for determining the next playing moves. At first, the rules of the board game “Mensch a¨rgere dich nicht” have to be incorporated into the knowledge database. This database is composed of a PROLOG-based rules collection, which can be accessed via a C++ wrapper. The database is managed and kept up-to-date with current positions of the pawns to infer the next possible allowed moves. After extracting all allowed possible moves, a Bayesian Network is used to determine the best choice. Since the training of the Bayesian Network is entirely done by data retrieved from previous games, the inference will improve its performance continuously. With this gained result the related gripping and movement actions are conducted by the robot. 3.3
Displaying Module
The content shown by the projection unit is controlled by the displaying module. This module is implemented as a server and offers remotely callable procedures to display images at given coordinates and size onto the workbench. Highlighting of objects and special regions for interaction, e.g. for rolling the dice, can also be done with the incorporated functionalities. Finally, this module can also project Soft-Buttons from above for interaction purposes. The content of these buttons can be freely chosen and their location can be arbitrarily selected. The spatial information about these buttons is written back into the real-time database for the image processing based detection of button activation. 3.4
Dice Recognition Unit
For gaining more natural feelings and moods in playing with the here presented system, a dice recognition unit is also implemented. The dice has to be tossed in a dedicated area on the game surface, highlighted with the table projection unit. In the following, the required steps are described more precisely. 1. First of all, the user has to toss the dice. 2. In the first processing step, the top plane of the dice is recognized. This is performed by applying a thresholding filter operation on the observer image. We are currently using a dark dice with light eyes on a white surface. Therefore, the dark area of the dice can easily be extracted from the filter
712
F. Wallhoff et al.
image. After applying morphological operations to the filter image, only those areas remain in the filter image, that contain the dice. 3. In the next processing step, the before extracted dice area is further analyzed. The inverse filter operation of the above described is applied to the dice region. Additionally, noisy elements in the filter image are removed. At this step, only those surfaces are visible, which are related to the number, that has been tossed. 4. At this stage, the filter image can further be analyzed to detect the actual number of eyes. Due to the chosen setting, only the number of eyes on the dice are visible in the filter image. By applying the connected components algorithm [9], these areas are labeled. By counting the amount of labeled surfaces, one gets the actual tossed value of the dice. 3.5
Robot Server
This module is working on the second computing unit and offers an interface to control the robot. However, it is not only a simple server performing all received orders, but it also provides for a certain degree of secure handling, because it prevents harmful commands resulting in damage of the hardware set-up. In addition, the system can be interrupted at any time by the user and will continue with the last requested action on demand. 3.6
Controlling Unit
Since all participating modules are built as stand-alone units, a central controlling unit is implemented to keep track of all relevant game data. The main task of this unit is to synchronize data streams provided by the real-time database with delays caused by the user and the movement of the robot. Besides the current game status, adjustments to the visual output like scaling or rotation are stored in this module as well. Thereby, the robot server can proceed the actual movement based on the transformation matrix which is used for the visual output.
4
Let’s Play
In the first of the subsequent two sections, we will present a short overview on the game constituting the foundation for the cognitive interaction scenario, where a cognitive computer system takes part in a contest in playing “Mensch ¨argere dich nicht” against a human player, see Figure 2. In the Section 4.2, the required adaptations for the realization of the presented human-machine contest will be delineated in greater details. 4.1
Rules
“Mensch ¨argere dich nicht” is a classical German board game. It exhibits similarities to the Indian game Pachisi, the American game Parcheesi, and the English game Ludo.
Statistics-Based Cognitive Human-Robot Interfaces for Board Games
713
Fig. 2. “Mensch a ¨rgere dich nicht” with a robot
In the classical style a wooden board is used for the game interaction. On this board all pawns are placed. For each player there are four pawns, which are color-coded (e.g. red, green, blue, yellow) to avoid any possibilities of confusion in the playing. The game can be played by 2, 3 or 4 players – one player per board side. At each board side a so-called home area is situated. This home area is the starting place, where the four pawns of each player are placed at the beginning of the game. The objective of the game is the following: Every player tries to bring the four pawns into the so-called “home row”. The starting point for each player is the home area. The player has to pass his pawns clockwise around the game board from the home area into the home row. One player starts to toss a dice. If he tosses a six, he will put one of his four pawns on the “start field”. The player has to toss again, and the number of eyes of the dice determine the fields, which he can pass with his pawn. If another pawn of an opponent player is at the destination field, this opponent pawn has to be moved back to its home area again. If he tosses no six, he can toss the dice twice again to obtain a six. If no six has been tossed, the turn moves over to the next player.
714
4.2
F. Wallhoff et al.
The Robot Way of Playing
Due to the today’s state of the art of robotic gripping and observation techniques, some constraints and restrictions are imposed on the current interaction setup. First, the robot is unable to toss a dice, therefore, a random number generator is delivering the robot’s numbers for the moves in the board game. Second, because of the small size of the pawns and the current vision-based surveillance all moves in the board game (robot’s and player’s) are conducted by the robot. Therefore, a touch screen display is used as input for the human’s moves. The player selects a field of the board game displayed at the touch screen and thus initiates the according movement of the pawn carried out by the robot. Besides, the board game is projected via the table projection unit.
5
Conclusion and Future Work
In this paper we presented a new form of a cognitive interaction scenario. The focus was laid on the cognition required for a human-machine contest in playing the German game “Mensch ¨argere dich nicht”. The implementation of the cognition playing strategy exhibits great potential for becoming a dignified counterpart for a human. However, in the current set-up, there are still many constraints, we would like to surpass in the future. First, the human should be allowed to move the pawns on his own, thus enabling him to cheat as well as to bring the system disadvantages by misplacing the pawns of the robot. For the realization of this vision, it is absolutely mandatory that the vision-based pawn detection and recognition has to be improved tremendously. On the other side, the cognitive system should attempt to betray the player as well from time to time for improving his winning opportunities and making the game more natural.
Acknowledgment This ongoing work is supported by the DFG excellence initiative research cluster Cognition for Technical Systems – CoTeSys, see www.cotesys.org for further information and details. The authors further acknowledge the great support of Matthias G¨ obl for his explanations and granting access to the RTDB repository. Furthermore, we want to thank all our partners within our CoTeSys-Projects for the fruitful discussions and implementation work to make our visions and ideas become reality.
References 1. Vernon, D., Metta, G., Sandini, G.: A Survey of Artificial Cognitive Systems: Implications for the Autonomous Development of Mental Capabilities in Computational Agents. IEEE Transactions on Evolutionary Computation 11(2), 151–180 (2007), http://dx.doi.org/10.1109/TEVC.2006.890274
Statistics-Based Cognitive Human-Robot Interfaces for Board Games
715
2. Anderson, D.M.L.: Embodied cognition: A field guide. Artificial Intelligence 149(1), 91–130 (2003), http://cogprints.org/3949/ 3. Beetz, M., Stulp, F., Radig, B., Bandouch, J., Blodow, N., Dolha, M., Fedrizzi, A., Jain, D., Klank, U., Kresse, I., Maldonado, A., Marton, Z., M¨ osenlechner, L., Ruiz, F., Rusu, R.B., Tenorth, M.: The assistive kitchen — a demonstration scenario for cognitive technical systems. In: IEEE 17th International Symposium on Robot and Human Interactive Communication (RO-MAN), Muenchen, Germany (2008) (invited paper) 4. Lenz, C., Suraj, N., Rickert, M., Knoll, A., R¨ osel, W., Bannat, A., Gast, J., Wallhoff, F.: Joint actions for humans and industrial robots: A hybrid assembly concept. In: Proc. 17th IEEE International Symposium on Robot and Human Interactive Communication (August 2008) 5. Z¨ ah, M.F., Lau, C., Wiesbeck, M., Ostgathe, M., Vogl, W.: Towards the Cognitive Factory. In: Proceedings of the 2nd International Conference on Changeable, Agile, Reconfigurable and Virtual Production (CARV), Toronto, Canada (July 2007) 6. Goebl, M., F¨ arber, G.: A real-time-capable hard- and software architecture for joint image and knowledge processing in cognitive automobiles. In: Intelligent Vehicles Symposium, pp. 737–740 (June 2007) 7. Stiller, C., F¨ arber, G., Kammel, S.: Cooperative cognitive automobiles. In: 2007 IEEE Intelligent Vehicles Symposium, June 2007, pp. 215–220 (2007) 8. Thuy, M., G¨ obl, M., Rattei, F., Althoff, M., Obermeier, F., Hawe, S., Nagel, R., Kraus, S., Wang, C., Hecker, F., Russ, M., Schweitzer, M., Leon, F.P., Diepold, K., Ebersp¨ acher, J., Heißing, B., W¨ unsche, H.-J.: Kognitive automobile - neue konzepte und ideen des sonderforschungsbereiches/tr-28. In: Aktive Sicherheit durch Fahrerassistenz, Garching bei M¨ unchen, April 7-8 (2008) 9. Jankowski, M., Kuska, J.-P.: Connected components labeling - algorithms in mathematica, java, c++ and c-sharp. In: Mitic, P., Jacob, C., Crane, J. (eds.) New Ideas in Symbolic Computation: Proceedings of the 6th Mathematica Symposium, Positive Corporation Ltd. Hampshire, UK (2004)
The Design and Development of an Adaptive Web-Based Learning System Chian Wang Department of Information Management National Changhua University of Education Changhua, Taiwan
[email protected] Abstract. Currently, most web-based learning systems do not differentiate the content materials presented to the various types of learners. Content adaptation is a concept inspired by enabling dynamic presentation generation based on the learner’s preferences, e.g. knowledge level, gender, age, language, or past visits. The goal of content adaption is to take the heterogeneous and changing needs of the learners into account and thus to provide the most appropriate contents and the best learning satisfaction. To handle content adaptation and dynamic presentation generation, an XML-based content description mechanism called ADAM is proposed in this paper. ADAM’s goal is to enhance learning effectiveness through providing the most appropriate materials under changing learners’ requirements and preferences while simplifying the process of presentation composition. Keywords: content description, multimedia, web-based learning, XML.
1 Introduction The rapid development of the Internet does not only create new types of information, but also changes how people access information. At the same time, many kinds of applications are inspired by the openness and the robustness of the Internet. Among them, web-based learning (WBL) systems have been one of the hottest research topics. The explosive growth of the Internet and the increasing amount of multimedia contents have made WBL systems an important information source for many people. The major benefit of these systems is that they allow learners from anywhere to learn with rich multimedia materials at any time. This kind of growth also comes with increasing diversity and heterogeneity in terms of the learners’ capabilities, backgrounds, and preferences [1, 3]. However, the contents of most WBL systems are static and are designed for a single type of learners in mind only. That is, all the learners are provided with the same web pages and hyperlinks. In many cases, these systems are not suitable for the diverse types of learners coming from all over the world. Thus, they can not truly satisfy an individual learner’s needs. Given the huge amount of learners on the Internet, there are considerable interests in systems that are able to satisfy the diverse needs of different types of learners. M.J. Smith and G. Salvendy (Eds.): Human Interface, Part II, HCII 2009, LNCS 5618, pp. 716–725, 2009. © Springer-Verlag Berlin Heidelberg 2009
The Design and Development of an Adaptive Web-Based Learning System
717
Though more and more learning applications provide rich multimedia information, they do not differentiate content materials presented to the various types of learners. For example, learners with lower knowledge level or foreign learners may experience frustration due to improper content presentations or inability to understand certain materials. The multimedia-rich contents can definitely enhance learning results but may not be satisfactory for learners with specific requirements, such as dubbed audio and subtitles. As a result, most WBL systems are targeted at certain types of learners, and others usually experience frustration or feel lost when accessing these systems. The lack of content adaptation to accommodate this kind of variety or heterogeneity raises challenging research topics for enabling more effective learning over the Internet. Content adaptation is a concept inspired by the issue for enabling dynamic presentation generation based on the learner’s preferences, e.g. knowledge level, gender, age, language, or past visits. The actual contents generated for a given learner is thus a combination of his own preferences and the adaptive criteria of the corresponding media objects. The goal of content adaptation is to provide the most suitable and personalized materials to each learner [2, 4]. To handle content adaptation and dynamic presentation generation, a content description mechanism called ADAM is proposed in this paper. Adaptive content generation is a mechanism that can dynamically compose the corresponding media objects into a presentation according to the learner’s preferences to enhance browsing results. The goal of content adaption is to take the heterogeneous and changing needs of the learners into account and thus to provide the most appropriate contents and the best learning satisfaction. Content adaption also has beneficial business implications beyond just providing a better browsing result. One of the main benefits is to increase the web site visit time of the users. This also means that the users are more likely to stay at the site, thus resulting in a greater profit for ecommerce sites. Currently, HTML (Hyper Text Markup Language) has been the presentation platform for most web-based applications. HTML provides a simple and efficient way for content description. However, HTML lacks for some capabilities when used in certain applications. First, HTML can not handle adaptive presentations. Because with an HTML file, the browser can generate only one presentation. If the user wants to make minor changes to the presentation, e.g., changing the subtitles from English to Chinese, modifying the HTML file is required. This makes the development of adaptive learning systems very inefficient because the instructor must compose several versions of HTML files for different types of learners. Second, HTML can not describe the temporal relationships of the media objects in a presentation, e.g., playing an audio file when a specific video segment is over. With these shortcomings, HTML is not suitable for generating dynamic presentations. For an adaptive learning system, a more robust and dynamic content description mechanism is required. That is, the system should be able to dynamically generate presentation contents according to the learners’ requirements and preferences. Also, it is preferred that the instructor can relieve the burden of composing presentations as much as possible. With ADAM, the instructor only needs to compose a base version of the content description file and specify the adaptive criteria in the file. After that, ADAM generates the adaptive presentations by including only the media objects with the conditions evaluated to be true. ADAM’s goal is to enhance learning effectiveness through providing the most
718
C. Wang
appropriate materials under changing learners’ requirements and preferences while simplifying the process of presentation composition. With the popularity of WBL systems and the increasing diversity of learner backgrounds, there is a need to have adaptation of the learning presentations. Typical examples include the materials for learners with different knowledge levels and language preferences. For an instructor, it takes a lot of effort and time to compose suitable presentations for diverse types of learners to achieve adaptive learning. He/she must consider each individual’s knowledge background, preferences, interests, and some other criteria when composing the presentations. ADAM’s main feature is that it requires only one XML-based description file to specify the adaptive structures and to generate the corresponding presentations. That is, ADAM is a general framework for dynamic presentation generation and can handle the adaptation needs of web-based learning. XML is adopted because it (i) has been a W3C (the World Wide Web Consortium) standard, (ii) allows the specification of a document which is independent of its final presentation, and (iii) is platform independent. Though adaptive presentations can certainly enhance learning effectiveness, they also increase the complexity of presentation composition. ADAM can help the instructors to relief the burdens because they only need to edit a single content description file and ADAM will generate the adaptive presentations accordingly. That is, ADAM can help the instructors to compose the courseware that have different renderings to different types of learners. We propose ADAM to specify a presentation scenario through three dimensions: spatial, temporal, and adaptive. In this paper, we also describe the development of a prototype WBL system to demonstrate the feasibility of ADAM. When a learner requests a presentation, ADAM handles the request by parsing the corresponding content description file and the learner’s preferences and then makes adaptation decisions to generate the final presentation.
2 Design of ADAM As suggested by the researchers, adaptive multimedia presentations are among the important factors in keeping students engaged in learning. This section describes the detailed design of ADAM. We start from how the adaptive presentations are generated. 2.1 Generation of Adaptive Presentations To support adaptivity, ADAM aims to adapt the materials presented to a learner according to his/her preferences, knowledge level, and the adaptation criteria specified by the instructor. Conditional media objects are used to accomplish the generation of adaptive presentations. That is, each constituent media object of a presentation is associated with a condition indicating which type of learners should be presented with it. In this way, several variants of the presentations associated with a specific learning subject are prepared. Each variant presents the materials in a different style to satisfy the learner’s needs. Fig. 1 depicts the flow of generating adaptive presentations.
The Design and Development of an Adaptive Web-Based Learning System
719
Fig. 1. Generation of adaptive presentations
In Fig. 1, the learner management module is responsible for collecting a learner’s basic information such as gender, age, language, education, etc. This information can be obtained when the learner registers. Also, when the user logs in for the first time, he/she will be given some tests such that the system can know his/her knowledge level about the subject that he/she is going to learn. In addition, the learner can have some other options to accommodate his/her needs, e.g., the language of subtitles and audio, and video resolution. For the instructor, the system should provide a friendly user interface and some management functions to assist the composition of learning presentations and the specification of adaptive criteria. With the criteria, ADAM can dynamically change and adapt the presentation contents for various types of learners. Customization of the contents gives the learners some more opportunities for different learning styles. After that, an XML-based content description file is generated. When generating a presentation, ADAM takes both the learner’s preferences and the adaptive criteria specified in the description file into account. Only the media objects with the corresponding adaptation rules evaluated true are included in the generated presentations. For example, if the learner prefers Chinese, then the Chinese version of subtitles and audio will be presented. As revealed in Fig. 1, the instructor only needs to compose the content description file once. Also, a single content description file can be used to generate various types of presentations. Each type of presentations is specifically tailored to accommodate a variety of individual differences and requirements. 2.2 Overall System Architecture As described previously, the literatures have revealed the importance of adaptation on students’ learning performances. As a case study, an English learning system based on ADAM was developed to support the course “English Conversations” that is offered to the freshmen in the Department of Information Management at National Changhua University of Education, Taiwan. The main characteristic of the system is
720
C. Wang
adaptive presentation engine (ADAM)
presentation #1
presentation #2
web browser
web browser
learner #1
learner #2
.…. .…. .….
learner model
presentation
authoring
content description script file
presentation #n
web browser learner #n
Fig. 2. System architecture
that the presented contents are adapted to the learner’s preferences and knowledge level. Fig. 2 depicts the system architecture. As depicted in Fig. 2, the system is developed in the form of three modules: the content description script file, the learner model, and the adaptive presentation engine, i.e., ADAM. The content description script file is XML-based and is composed by the instructor to specify how the presentations are generated. As to ADAM, its job is to adapt different aspects during the learning process, e.g., adapting the content according to the learner’s prior knowledge, generating the presentations through the selection and combination of appropriate media objects, and modifying the corresponding hyperlinks, etc. The learner model is a simple data structure that can reflect the characteristics of different learners. Currently, the learner model contains two categories of information. (i) The personal profile includes static data such as account name, password, real name, student ID, gender, birth date, e-mail address, etc. (ii) The knowledge profile identifies the learner’s knowledge level about a specific subject. 2.3 The Content Description Language A multimedia presentation is composed of a set of media objects, e.g., video, audio, text, images, etc. With most application systems, it is preferred that the media objects can be reused to produce different presentations. Thus, a presentation’s specification should be separated from its actual content. In this subsection, we introduce an XMLbased content description language, called Adaptive Multimedia Markup Language (AMML). AMML’s purpose is to facilitate the specification requirements of ADAM. Because AMML is based on XML, it also allows the specification of document structures independent of their final presentations, which is a basic requirement of
The Design and Development of an Adaptive Web-Based Learning System
721
AMML(root) CMML(root)
region
head
body
layout
content
root-layout
time container
id
width
width
height
par start
height
col
end
top
row
dur
left
media object
seq
clipstart
streaming
audio
non-streaming
video
image
text
src
src
src
src
size
size
size
rate
rate
clipend
class
interaction
content
quality
language
switch
interation
switch
switch
switch
id
id
id
id
id
play
degree
size
sys_language
bitrate
usr_language
stop pause resume jump
Fig. 3. The tree structure of AMML’s tags
WBL systems. In order to fully benefit from XML’s flexibility and expandability, AMML’s syntax is formally described as an XML Document Type Definition (DTD) and therefore AMML can take full advantages of all the existing XML tools. With AMML, the description of an adaptive multimedia presentation is organized around three dimensions: spatial, temporal, and adaptive. The tree structure of AMML tags is depicted in Fig. 3. In the remaining part of this section, we describe AMML from each of these three dimensions and explain their usages. As shown in Fig. 3, an AMML description file is consisted of three major blocks. 1. The head block describes a presentation’s properties of the spatial dimension, i.e., the layout of the media objects and how the presentation looks like. 2. The body block specifies a presentation’s properties of the temporal dimension, i.e., the relative timing sequences of the media objects. This allows the organization of the media objects in a presentation over time. 3. The class block describes the adaptive requirements.
722
C. Wang
To simplify the specification of a multimedia presentation, the underlying model of AMML is interval-based. That is, the presentation is composed by a set of media objects that have some temporal relationships and each media object has a corresponding time interval characterized by a “start”, a “duration”, and an “end” attributes. In AMML, the synchronization among media objects is specified by both the composite objects and their temporal relationships. The composition of media objects is used to temporally group interval elements and is described by the “container” tag that is depicted in Fig. 3. Currently, two temporal relationship tags are provided by AMML: (i) The “seq” tag means that the media objects are presented sequentially. (ii) The “par” tag means that the media objects are presented in parallel. As to the adaptive criteria, they are specified in the class block of an AMML document. Currently, four elements are supported: user interaction, content, video quality, and language. 1. The so-called user interactions are the control functions similar to those of VCRs, e.g., pause, fast forward, and rewind. With these user interactions, the learner can control the presentation flow and speed at any time during the presentations. However, in an adaptive WBL system, not all levels of learners should be provided with the same user interactions. In stead, the system should provide a proper set of user interactions to each level of learners. For example, for a higher level of learners, more use interactions can be provided. Because they have more background knowledge and should be able to control what they want to view. On the contrary, for the lowest level of learners, only the basic pause function is provided to restrict their browsing behaviors. In this way, the instructor can have control over the learners’ learning states and behaviors. 2. Adaptation by content means that the system can generate the presentations about the same subject but with various difficulties. For example, when having a math course, higher-level students can have materials about 4-digit addition while lowerlevel students can have materials about 2-digit addition. When a learner selects a course, his/her information in the learner model is used to generate the most appropriate materials. In this way, the instructor can examine each learner’s learning state more easily. 3. Because the learners’ communication environments vary a lot, it is reasonable that the system delivers the media objects with an appropriate quality to the learners. That is, if there are several versions of a video object, the system will choose the one that has the adequate screen resolution and frame rate after detecting the network bandwidth to the learner. If the detection is not successful, a default version of the media object can be provided. 4. With WBL systems, learning becomes without any nationality and geographical limits. Because the learners are coming from all over the world, language becomes a major consideration of WBL systems. To improve efficiency, ADAM can dynamically adapt the language part, e.g., subtitles and speech, of a presentation, instead of generating the whole presentation from scratch.
3 System Development As mentioned previously, we developed a real WBL system to demonstrate the feasibility of ADAM. This section describes its implementation. Our goal is that
The Design and Development of an Adaptive Web-Based Learning System
723
ADAM and AMML can be integrated seamlessly to further improve learning effectiveness. 3.1 System Components Fig. 4 depicts the components of our system. There are three major modules in Fig. 4: the XML parser, the synchronization processor, and the media content processor. CMML script file
Spatial Controller
course database
XML Parser
Synchronization Processer
Media Controller
media request multimedia Database Media units
media buffer
user interface display
Temporal Controller
decoder
rendering Media Content Processer
Fig. 4. System components
The operation flow of the system is as follows. 1. The instructor composes the course’s content description files in AMML to specify the temporal, the spatial, and the adaptive properties of the presentations. These description files are stored in the course server’s database. 2. When the learner wants to view a presentation, its content description file is retrieved from the server and passed to the XML parser for content generation. 3. The XML parser parses the AMML tags in the description file into a parse tree using the document object model (DOM) and keeps the corresponding attributes, including media type, start time, end time, and spatial location, of each media object in a parse table. 4. The synchronization processor contains the temporal controller and the spatial controller. They together allocate each media object in a presentation according to the object’s temporal and spatial properties stored in the parse table. 5. Finally, the media controller of the media content processor retrieves the corresponding media objects from the multimedia database. At the client site, the media objects are first stored in the receiving buffer temporarily. When the media objects are about to be presented, they are passed to the decoder, then the renderer, and finally the user interface display, to be played out under the control of the temporal controller and the spatial controller. 3.2 An Illustrated Presentation As mentioned previously, we developed an ADAM-based WBL system to support the “English Conversation” course. This subsection demonstrates some major results.
724
C. Wang
1
2
a b
c
3
Fig. 5. An illustrated adaptive presentation
When a learner logs in for the first time, he/she is asked to fill in some personal information. After that, a pre-class test is given to determine his/her knowledge level about the course. The purpose of these steps is to gather the attributes of the student model for each learner. ADAM then utilizes the information stored in the student model to support adaptivity. To accomplish this goal, each media object is associated with a condition indicating which type of learners should be presented with it. Also, ADAM supports adaptive navigation that the selection and the color of hyperlinks are adapted to the individual learner by taking into account the information in the student model and the instructional strategy. Fig. 5 shows an illustrated presentation. In Fig. 5, the presentation can be divided into three main areas. They are described as follows. 1. Video area: This area is used to place video and audio objects. The audio is associated with the video such that different versions of speech, e.g., Chinese or English, can be included. 2. Image area: This area is used to place image objects, e.g., JPEG files. ADAM also provides a slide-show feature to image objects. This is, the displayed image is automatically changed as the time goes by. 3. Text area: Text and audio objects can be placed in this area. The text can be the subtitles of the video or some annotations. In addition to the three areas for placing media objects, Fig. 5 also shows three sets of control buttons, i.e. , a, b, and c. These buttons provide VCR-like user interactions. As mentioned previously, the web browser does not provide these user interactions. They are implemented in JavaScript. Also, some of the buttons may be disabled because the functions are not provided to this learner.
4 Conclusion and Future Work In this paper, we proposed ADAM, an adaptive multimedia content description mechanism. The main feature of ADAM is that only a content description file in
The Design and Development of an Adaptive Web-Based Learning System
725
AMML is required to generate various presentations. In order to demonstrate the feasibility of ADAM, an English-learning WBL system is developed. After developing ADAM and the WBL system, two evaluations, i.e., the expert review and the small group evaluation, are conducted to judge the values of our work. According to the results, the experts and the learners have positive attitude towards the perceived efficacy and enjoyment of the system. From the view point of instructors, with ADAM, it is easier to re-use the materials to compose adaptive presentations to accommodate the needs of various types of learners.
References 1. Benson, V., Frumkin, L., Murphy, A.: Designing Multimedia for Differences: e-Lecturer, eTutor, and e-Student Perspectives. In: Proceedings of the Third International Conference on Information Technology and Application, vol. 2, pp. 159–164 (2005) 2. Chen, C.M., Liu, C.Y., Chang, M.H.: Composing a Complex Biological Workflow through Web Services. Personalized curriculum sequencing utilizing modified item response theory for web-based instruction. Expert Systems with Applications 30(2), 378–396 (2006) 3. Gu, Q., Sumner, T.: Support Personalization in Distributed E-learning Systems through Learner Modeling. Information and Communication Technologies 1, 610–615 (2006) 4. Lin, C.B., Young, S.S.C., Chan, T.W., Chen, Y.H.: Teacher-oriented adaptive Web-based environment for supporting practical teaching models: a case study of “school for all”. Computers and Education 44(2), 155–172 (2005)
Human-System Interface (HSI) Challenges in Nuclear Power Plant Control Rooms Jo-Ling Chang1, Huafei Liao1, and Liang Zeng2 1 Bechtel Power Corporation 5275 Westview Drive, Frederick, MD 21703 {jjchang,hliao}@bechtel.com 2 School of Industrial Engineering Purdue University West Lafayette, IN
[email protected] Abstract. This study uses factor analysis to examine 30 errors due to humansystem interface (HSI) in nuclear power plant control rooms. The results are used to validate the factor structure and the Decision-Action Model developed in this paper. Ten U.S. commercial operating nuclear plants, total of 18 units, participated in this study at the time this paper was written. The result is a fivefactor structure: Operations Uncertainties, Design Improvements, Misoperations, Equipment Control, and Human Factors Redesign. The completed DecisionAction Model provides current operating plants with suggested corrective actions for each type of potential HSI errors. Keywords: Human error, human-system interface, control room, human performance, factor analysis, corrective action.
1 Introduction With the increasing demand for clean and reliable energy in recent years, there is much attention on new construction and increasing output for nuclear power plants in the United States. Concerns for nuclear safety have also risen in view of the numerous modifications current operating plants are installing. This study aims to provide additional insights through operator experience on control room HSI. Human errors, when systematically analyzed and evaluated, provide information on the causes and correction methods. Among the human error analysis/evaluation methods, statistical analysis is effective in identifying error categories, occurrence patterns and trends, and revealing the hidden interrelationships between errors and their casual factors. Such knowledge can significantly improve human error prevention and corrective measure evaluation.
2 Background Improving currently operating nuclear power plant control room design is a continuous effort. A review of recent operating experience (OE) from the Institute of Nuclear M.J. Smith and G. Salvendy (Eds.): Human Interface, Part II, HCII 2009, LNCS 5618, pp. 729–737, 2009. © Springer-Verlag Berlin Heidelberg 2009
730
J. Chang, H. Liao, and L. Zeng Table 1. Control Room Human-System Interface Hypothetical Factor Structure
Event Operation Based
Item 1. Operation movements. Operation requires small movements or jerking/unsmooth motion. 2. Simultaneous operation. Operator required to multi task. 3. Control room/simulator discrepancies. Trained actions are not applicable to real scenarios. 4. Operate equipment incorrectly. Due to inattention to details/distractions. 5. Inappropriate compensation. From lack of trust in equipment. 6. Over reliance. From over trusting equipment. 7. Defeated safety features. Manual override of safety feature. 8. Inexperience. From lack of operating hours on equipment. Controller 9. Operate on wrong equipment. Due to similarity. Design 10. Controls too far apart. Need excess movement to operate Based consecutive actions. 11. Controls too close together. Poor design leads to inadvertent operation. 12. Incorrect function allocation – Manual actions designed to be automated. 13. Incorrect function allocation – Automated actions designed to be manual. 14. Equipment allowing failures. Allowing operation outside of design parameters. 15. Work-around’s. Known defects that require operators to take less direct action. 16. Time limit to operation. Operation cannot be completed within the allowed time. 17. No operator intervention allowed. To abort or assume control as necessary. Deficient 18. No alarm noting abnormal conditions and/or failures. Indication 19. Insufficient plant information. Based 20. Boolean indication. Indication without level of severity. 21. Unreliable indication. Indication known to reflect plant condition imperfectly. 22. No feedback. Action is performed with no confirmation. 23. No projection. No indication on anticipated result from action. 24. No trending. No indication on equipment failing over a prolonged time period. Ambiguous 25. Control panel visually crowded. Cannot take in presented Indication information at a glance. Based 26. Color/Sound coordination. Many indications of the same color/sound or all indications having different colors/sounds. 27. Over-indication. A single failure represented by more than one alarm. 28. Non-intuitive control. 29. Display challenges. Display font size/color or inconsistency in acronyms/labeling/terminology. 30. Data searching. Extensive navigation needed to look for known existing data.
Source [4] [5] [6] [7] [8] [5] [9] [10] [11] [12] [13] [14] [14] [15]
[16] [17] [18] [19] [20] [21] [22] [17] [3] [23]
[24] [17]
Human-System Interface (HSI) Challenges in Nuclear Power Plant Control Rooms
731
Power Operation (INPO) database shows that control room HSI design may still need to make improvements to increase safety and reduce cost of operation. Between 1991 and 2008, numerous plant events resulting in either a plant trip/transient or technical specification violation were still being reported to INPO. The cost of these errors could be as high as a million dollars per day for repairs and rework. The United States Nuclear Regulatory Commission (USNRC) provided Human Factors Engineering (HFE) guidelines in NUREG-0700 [1] and identified different areas of HSI in both advanced and conventional control rooms. The HFE guideline provides the following categories: Information Display, User-System Interface, Process Control and Input Devices, Alarms, Analysis and Decision Aids, InterPersonnel Communication, Workplace Design, and Local Control Stations. The OE search conducted by the authors in August 2008 on the Institute of Nuclear Power Operations (INPO) OE database revealed 146 plant events between 12/3/1990 and 4/24/2008. These events were listed under control room operator work group and man-machine interface casual factor. Each plant event was reviewed for contributing factors; many were used in the hypothetical factor structure. The hypothetical factor structure utilized the USNRC HFE categories as a starting point and items were expanded or deleted compared to the actual events found in OE. Categories developed for control centers in nuclear or other industries were also examined. Examples of these categories include Davey discussed factors important for structuring review criteria [2] and Grozdanovic’s research on the control center of the railway traffic in Yugoslavia [3]. This study reviewed literature available on control room HSI and related issues to create the hypothetical factor structure listed in Table 1. This table lists one source for each item. For some of these items, the factors contributing to the errors are discussed in several different sources. For items such as these, a representative reference is selected and listed. Items that do not have a reference source listed are simplified or paraphrased from the literature review and cannot be pinpoint into a single source document.
3 Model Development A review of recent plant events relating to control room HSI showed that the majority of the incidents, approximately 70%, did not lead to any immediate corrective actions. The focus of this study, in addition to developing a factor structure, is to provide corrective action guidelines for current operating plants. As such, a Decision-Action Model was developed (see Table 2) to offer current operating plants suggested corrective actions for each of the items in the purified factor structure, which effectively lists the causes of all OE submitted by operating plants. Future errors may Table 2. Decision-Action Model Group I (no incident) Correct Decision + Correct Action Group III Correct Decision + Incorrect Action
Group II Incorrect Decision + Correct Action Group IV Incorrect Decision + Incorrect Action
732
J. Chang, H. Liao, and L. Zeng
be associated with one or more of these items and the suggested corrective actions may help to effectively eliminate similar errors from reoccurring. Decision, in this model, represents cognitive errors. These errors pertain to knowledge and judgment of the operator. This type of error is similar to the concept of “mistakes” described by Norman, 1981 [25]. It is described as representing defects in the formulation of strategy, generated only during the planning process as the result of inappropriate knowledge of the relations between parts of the plant or between physical quantities. Action, in this model, represents the physical errors, or “slips” in Norman’s words, which are imperfections of attention monitoring or errors that occur while implementing intended plans [25]. This model is influenced by Chen-Wing and Davey [26], which illustrates that Designer, Operations, and Human/Technical Resources’ roles on error reduction. Since the Decision-Action Model focuses on current operating plants, as opposed to designing new constructions, the Designer’s role is eliminated. Suggested corrective actions are also based on the same study. These suggested actions are as follows: • Group I Suggested Corrective Action: N/A • Group II Suggested Corrective Action: Improvement to operations procedure, general guidelines, and pre-job briefing. • Group III Suggested Corrective Action: Additional operator training, peer check, management oversight. • Group IV Suggested Corrective Action: Control room modification with human factor re-evaluation to the extended condition.
4 Research Method 4.1 Procedure A survey was developed based on the hypothetical factor structure shown in Table 1 to examine power plant operators’ opinions on HSI errors. Each survey question consisted of two parts: a 7-point Likert scale ranging from Strongly Disagree (1) to Strongly Agree (7), and an option to select the contribution of the error from Operator Decision, Operator Action, or both. The survey contained 32 questions, which included two paired questions to estimate the internal consistency of participants’ responses. Ten commercially operating nuclear power plants, 18 units total, participated in this ongoing study at the time this paper was written. Several methods, e.g., telephone, email, and/or post, were used to contact the head of the Operations Department at each plant. Participants were directed to either forward the online survey URL to qualified licensed operators or mail back completed hard copies of the surveys. By the time when this paper was written, 138 responses were received, out of which eight were completed through paper surveys, and the remainder were collected online. 4.2 Profile of Participants The first 138 responses were analyzed for this study. A single operating unit is expected to have around 20 licensed operators. The approximate number of operators at each unit was verified through operations training instructors at several plants.
Human-System Interface (HSI) Challenges in Nuclear Power Plant Control Rooms
733
From this information, the study’s current participation rate is around 38%. As indicated above, surveys were distributed by the head of the Operations Department. While some plants sent the surveys to all operations personnel, others selected small groups of individuals to participate in this study. This accounts for the low participation rate of this study. Out of the 138 responses, 14 were discarded due to low internal consistency. 3.8% of remaining 124 responses were from USNRC Region I, 21.5% was from USNRC Region II, 18.5% was from USNRC Region III, and 53.4% was from USNRC Region IV. The remaining responses did not provide enough plant information to identify their regions. These regions are designated by the USNRC to oversee the operation of power producing and non-power producing reactors in the United States [27]. Participants’ ages ranged from 27 to 63 (mean = 47.0, standard deviation = 7.55). 18.1% had 1 to 10 years of operations experience, 20.3% had 11 to 20 years, and 61.7% had over 20 years. 4.3 Analysis Descriptive Statistics. The overall internal consistency as estimated by Cronbach’s alpha coefficient was 0.70, which indicates that the survey has acceptable internal consistency. The general characteristics of the survey results on the items loaded on the purified factor structure (see discussion later in this section) were examined. The mean scores for each of the items were between 3.6 and 6.2 and the standard deviations were between 0.87 and 1.60. The overall mean of all the items in the purified factor structure is 5.1. Items scoring above the overall mean indicate relatively strong preferences of the participants and are in bold font. Item 12, incorrect function allocation – manual actions designed to be automated, has the lowest mean value. This suggests that operators believe that designing manual actions to be automated would reduce, rather than increase, the chances of making an error. Three-way ANOVA shows that there are no significant effects of age, experience, or plant of employment on all 18 loaded items. Factor Analysis. Maximum likelihood factor analysis with varimax rotation was conducted to explore the hidden factor structure determined by the correlations among survey items. Five factors, whose eigenvalues are larger than 1.0, were retained (see Table 3). Factor loadings are presented in Table 3. Items with loading higher than, or close to, 0.50 are considered significant. Items 29, 18, and 15 are very close to 0.50 and therefore considered significant as well. The items loaded on Factor 1 pertain to errors caused by doubts in information presented on the job; thus, Factor 1 is classified as “Operations Uncertainties”. Factor 2 contains items describing existing design with the need for improvements and is labeled “Design Improvements”. Factor 3 includes operation-based errors and is categorized as “Misoperations”. The two items loaded onto Factor 4 are equipment allowing failures and over reliance. As such, Factor 4 is labeled “Equipment Control”. Factor 5 includes problems arising from basic human factor issues and is labeled “Human Factors Redesign”.
-0.01 0.21 0.17 0.24 0.32 0.28 0.56 0.52 0.51 0.50 -0.06 0.24 0.42 0.07 0.11 0.09 0.33 0.17 2.7 5.1
15.5
36.4
0.69 0.66 0.55 0.50 0.49 0.49 0.26 0.09 0.11 0.38 0.13 0.22 0.32 0.16 0.04 0.03 0.11 0.24 20.0 6.9
20.9
20.9
21 22 28 19 29 18 24 16 26 20 4 7 25 14 6 12 11 10
51.0
14.6
0.23 0.18 0.20 0.28 0.28 0.33 0.32 0.09 0.19 -0.06 0.51 0.50 0.49 0.20 0.17 0.01 0.46 0.25 2.4 4.8
Factor 3
63.0
12.0
0.33 0.23 0.03 0.32 0.10 0.12 0.29 0.14 0.04 0.42 0.17 0.26 0.14 0.57 0.51 0.30 -0.04 0.37 1.9 4.0
Factor 4
75.2
12.2
0.09 0.03 0.12 0.10 0.12 0.16 -0.08 0.12 0.15 0.12 0.12 0.03 0.30 0.14 0.21 0.69 0.57 0.54 1.6 4.0
Factor 5 6.0 5.4 5.5 5.8 5.2 5.7 4.9 5.0 5.0 4.9 6.2 5.3 5.1 5.2 4.3 3.6 4.9 4.6
Meanb 1.02 1.20 1.20 0.99 1.29 1.14 1.26 1.11 1.23 1.34 0.87 1.57 1.19 1.40 1.46 1.40 1.37 1.50
SD
4.4
4.8
5.5
5.0
5.6
Factor Mean Action (%) 11 (8.9) 29 (23.6) 47 (38.2) 12 (9.8) 33 (26.8) 44 (35.8) 16 (13.0) 44 (35.8) 32 (26.0) 17 (13.8) 30 (24.4) 23 (18.7) 58 (47.2) 29 (23.6) 15 (12.2) 52 (42.3) 87 (70.7) 90 (73.2)
: p < 0.0001; **: p < 0.01; *: p < 0.05 a : Reference Table 1 for Item Description b : Means above the overall mean (5.1) are in bold type. Factor loadings in bold type are considered to be significant. Five factors explained 75.2% of total variance.
***
Eigenvalve Variance explained by each factor % Variance explained by each factor Cumulative % total variance explained
Factor 2
Factor 1
Item No.a
Table 3. Descriptive Statistics and Rotated Factor Pattern Decision-Action Decision (%) Both 85 (69.1) 27 (22.0) 63 (51.2) 31 (25.2) 40 (32.5) 36 (29.3) 79 (64.2) 32 (26.0) 63 (51.2) 27 (22.0) 54 (43.9) 25 (20.3) 78 (63.4) 29 (23.6) 40 (32.5) 39 (31.7) 57 (46.3) 34 (27.6) 91 (74.0) 15 (12.2) 50 (40.7) 43 (35.0) 53 (43.1) 47 (38.2) 37 (30.1) 28 (22.8) 70 (56.9) 24 (19.5) 90 (73.2) 18 (14.6) 51 (41.5) 20 (16.3) 22 (17.9) 14 (11.4) 21 (17.1) 12 (9.8)
2 1
57.04*** 12.57** 0.56 49.33*** 9.38** 1.02 40.89*** 0.19 7.02** 50.70*** 5.00* 11.84** 4.64* 16.98*** 53.57*** 0.01 38.76*** 42.89***
734 J. Chang, H. Liao, and L. Zeng
Human-System Interface (HSI) Challenges in Nuclear Power Plant Control Rooms
735
Compared to the hypothetical factor structure in Table 1, Factors 1 and 2 in the purified factor structure include a shuffling of items from the original Deficient Indication and Ambiguous Indication categories. Factor 3 maps closely to the items in the Operation Based category and Factor 5 includes only items from the Controller Design Based category. Model Population. Chi-square tests were performed to determine if the responses lean towards Operator Action or Operator Decision. Table 3 shows the number of responses that selected Action, Decision, or both for each loaded item. The Decision heavy items are populated into Group II of the Decision-Action Model, Action heavy items are populated into Group III, and the remaining items are populated into Group IV. The final model is shown in Table 4. Table 4. Populated Decision-Action Model Group I (no incident) Correct Decision + Correct Action Suggested Corrective Action: N/A
Group III Correct Decision + Incorrect Action • Control panel visually crowded • Controls too close together • Controls too far apart Suggested Corrective Action: Additional operator training, peer check, management oversight.
Group II Incorrect Decision + Correct Action • Unreliable indication. • No feedback • Insufficient plant information • Display challenges • No trending • Color/Sound coordination • Boolean indication • Operate equipment incorrectly • Defeated safety features • Equipment allowing failures • Over reliance Suggested Corrective Action: Improvement to operations procedures, general guidelines, and pre-job briefing. Group IV Incorrect Decision + Incorrect Action • Non-intuitive control • No alarm noting abnormal conditions and/or failures • Time limit to operation • Incorrect function allocation – Manual actions designed to be automated Suggested Corrective Action: Control room modification with human factor re-evaluation to the extended condition.
5 Conclusion This study collects nuclear power plant operator opinions on control room HSI errors. The factor means, shown in Table 3, indicate that the highest priority should be
736
J. Chang, H. Liao, and L. Zeng
placed on the category of Operations Uncertainties. The remaining four factors, in the order of importance, are: Misoperations, Design Improvements, Equipment Control, and Human Factor Redesign. The populated Decision-Action Model (Table 4) also provides some insight to control room design. A large number of items are populated in Group II. This may be interpreted as suggesting that improvements in the planning process will have better results in reducing HSI type errors. Operating Experience was revisited to compare this study’s finding to documented plant events. Out of the 106 plant events originally evaluated, 17% fall under Operations Uncertainties, 21% fall under Design Improvements, and 22% fall under Misoperations. No event strictly falls into the category of Equipment Control or Human Factor Redesign. This comparison confirms that this study’s conclusion of category importance ranking is valid.
Acknowledgments The authors would like to acknowledge the support and directions offered by Dr. Gavriel Salvendy and Mr. Karey Kimmel.
References 1. United States Nuclear Regulatory Commission: Human Factors Engineering. In: Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants, NUREG-0800, ch. 18, rev. 2, U.S. NRC (2007) 2. Davey, E.: Criteria for Operator Review of Workplace Changes. In: Canadian Nuclear Society Conference (2000) 3. Grozdanovic, M.: Methodology for Research of Human Factors in Control and Managing Centers of Automated Systems. The Scientific Journal Facta Universitatis: Working and Living Environmental Protection 1(5), 9–22 (2000) 4. Millstone 2: Automatic reactor scram after a steam generator feed pump trip. Plant Event # 336-040315-1, Institute of Nuclear Power Operations, March 15 (2004) 5. Sheridan, T.B., Parasuraman, R.: Human-Automation Interaction. In: Reviews of Human Factors and Ergonomics, vol. 1, pp. 89–129. Human Factors and Ergonomics Society (2005) 6. Monticello: Half scram due to cold water transient during valve operation at Monticello Nuclear Generating Plant. Plant Event # 263-070314-1, Institute of Nuclear Power Operations, March 14 (2007) 7. Darlington 2: Unit 2 turbine leading (normal) mode inadvertently entered. Plant Event # 932-041101-1, Institute of Nuclear Power Operations, Feburary 8 (2005) 8. Gentilly 2: Recirculated service water diesel motor pump 7131-P36 damaged. Plant Event # 851-001126-1, Institute of Nuclear Power Operations, November 26 (2000) 9. Cernavoda 1: Inadvertent draining of in-service fire water tank 7140-TK1 results in start and damage of the diesel engine driven pump. Plant Event # 121-030612-1, Institute of Nuclear Power Operations, June 12 (2003) 10. Cooper 1: Unintended increase in reactor power due to misoperation of reactor recirculation pump speed control. Plant Event # 298-020212-1, Institute of Nuclear Power Operations, Feburary 12 (2002)
Human-System Interface (HSI) Challenges in Nuclear Power Plant Control Rooms
737
11. Trillo 1: During routine tests, start-up of emergency diesel generator GY60 activated by reactor protection system. Plant Event # 715-071128-1, Institute of Nuclear Power Operations, November 28 (2007) 12. Xiaoming, C., Zhiwei, Z., Zuying, G., Wei, W., Nakagawa, T., Matsuo, S.: Assessment of Human-Machine Interface Design for a Chinese Nuclear Power Plant. In: Reliability Engineering and System Safety, vol. 87, pp. 37–44. Elsevier Science S.A, Amsterdam (2005) 13. Nine Mile Point 1: Two control rods scrammed during rod scram timing test. Plant Event # 220-040504-1, Institute of Nuclear Power Operations, May 4 (2004) 14. Hugo, J., Engela, H.: Function Allocation for Industrial Human-System Interfaces. In: 4th International Cyberspace Conference on Ergonomics, International Ergonomics Association 15. Perry 1: Reactor operation in unanalyzed region. Plant Event # 440-060709-1, Institute of Nuclear Power Operations, July 9 (2006) 16. Point Lepreau 1: Containment isolation system button-up during degassing of the degasser condenser. Plant Event # 908-990426-1, Institute of Nuclear Power Operations, April 26 (1999) 17. Naito, N., Itoh, J., Monta, K., Makino, M.: An Intelligent Human-Machine System Based on an Ecological Interface Design. In: Nuclear Engineering and Design, vol. 154, pp. 97– 108. Elsevier Science S.A, Amsterdam (1995) 18. Point Lepreau 1: Primary heat transport (PHT) thermal transient. Plant Event # 908040606-1, Institute of Nuclear Power Operations, June 6 (2004) 19. Gentilly 2: Local radiological alert due to a moderator leak in upgrading plants. Plant Event # 851-020212-1, Institute of Nuclear Power Operations, Feburary 12 (2002) 20. Duane Arnold 1: High temperature in fuel pool because of procedure use problem. Plant Event # 331-000111-1, Institute of Nuclear Power Operations, January 11 (2000) 21. Bugey 3: Two bank rod assemblies blocked in low position. Plant Event # 783-031230-1, Institute of Nuclear Power Operations, December 30 (2003) 22. Burns, C., Vicente, K.J.: A Participant-Observer Study of Ergonomics in Engineering Design: How constraints drive design process. In: Applied Ergonomics, vol. 31, pp. 73–82. Elsevier Science S.A, Amsterdam (2000) 23. Gentilly 2: 3481-TK2 tank draining and 3481-P1 and P2 pump cavitations. Plant Event # 851-050530-1, Institute of Nuclear Power Operations, July 12 (2005) 24. Susquehanna 2: B circulating water pump shutdown instead of B condensate pump. Plant Event # 388-050527-1, Institute of Nuclear Power Operations, May 27 (2005) 25. Norman, D.A.: Categorization of action slips. Psychological Review 88, 1–15 (1981) 26. Chen-Wing, S.L.N., Davey, E.C.: Designing to Avoid Human Error Consequences. In: Workshop on Human Error, Safety, and System Development, Paper Session 5 (1998) 27. United States Nuclear Regulatory Commission, http://www.nrc.gov
The Impact of Automation Assisted Aircraft Separation on Situation Awareness Arik-Quang V. Dao1, Summer L. Brandt1, Vernol Battiste1, Kim-Phuong L. Vu2, Thomas Strybel2, and Walter W. Johnson1 1 NASA Ames Research Center Moffett Field, CA 94035, United States of America {quang.v.dao,summer.l.brandt,vernol.battiste-1, walter.w.johnson}@nasa.gov 2 California State University Long Beach, Dept of Psychology 1250 N Bellflower Blvd, Long Beach, CA 90840, USA {kvu8,tstrybel}@csulb.edu
Abstract. This study compared situation awareness across three flight deck decision aiding modes. Pilots resolved air traffic conflicts using a click and drag software tool. In the automated aiding condition, pilots executed all resolutions generated by the automation. In the interactive condition, automation suggested a maneuver, but pilots had the choice of accepting or modifying the provided resolution. In the manual condition pilots generated resolutions independently. A technique that combines both Situation Global Assessment Technique and Situation Present Awareness Method was used to assess situation awareness. Results showed that situation awareness was better in the Manual and Interactive conditions when compared to the Automated condition. The finding suggests that pilots are able to maintain greater situation awareness when they are actively engaged in the conflict resolution process. Keywords: automation, conflict resolution, situation awareness, cockpit display of traffic information, CDTI, cockpit situation display, CSD.
1 Introduction Without additional tools and automation, increases in air traffic densities during the next few decades will exceed the capabilities of current-day controllers to manage them [1]. However, adding tools and automation to the air traffic management system will require a new distribution of roles and responsibilities across ground control, the flight deck, and automation. In response, the Joint Planning and Development Office (JPDO), in its vision of the Next Generation Air Transportation System (NextGen), has suggested multiple concepts of operation for distributing roles and responsibilities, including the delegation of aircraft separation responsibilities to flight crews, aided by automation, or to automation alone. Automated systems have been successfully used in many areas of transportation to increase operator workload capacity and system safety. However, automation has M.J. Smith and G. Salvendy (Eds.): Human Interface, Part II, HCII 2009, LNCS 5618, pp. 738–747, 2009. © Springer-Verlag Berlin Heidelberg 2009
The Impact of Automation Assisted Aircraft Separation on Situation Awareness
739
been shown to impact operators’ information acquisition, information analysis, decision making, and action [2]. Thus, a major issue in implementing automation in a system where humans remain “in-the-loop” is its impact on operator situation awareness (SA). As a first step towards addressing this issue, researchers from the NASA Ames Flight Deck Display Research Laboratory (FDDRL) and California State University Long Beach conducted a study comparing SA of commercial pilots performing a conflict resolution task across three levels of automation aiding. 1.1 Automated Separation Assurance on the Ground and the Flight Deck One responsibility of an air traffic controller is to detect and resolve upcoming conflicts (losses of legal separation) between aircraft. Since the number of conflicts increases with traffic density, this can lead to an unacceptable increase in controller workload. One solution is to provide conflict detection and/or resolution automation that take some of the work off the controller. In addition, researchers have been investigating the possibility of delegating separation responsibility to appropriately equipped flight decks [3]. On such flight decks, the pilot would be provided with technology that supports conflict detection and resolution. As one of the primary proponents of such automation, Erzberger [4], at the NASA Ames Research Center, has been developing an “auto-resolver”. While his efforts have been aimed mainly at a ground-based implementation, several concepts of operation could be envisioned which would utilize this automation. For example, Homola [5] examined controller acceptance of concepts where this automation would be used 1) to autonomously generate resolutions without involving pilots or controllers (fully automated mode), and 2) to allow air traffic controllers to request conflict resolutions on demand, and then modify them as they wished. In turn then, the general goal of the present study was to examine pilot acceptance of this same automation, and for this paper, to look at its impact on pilot SA. 1.2 The Impact of Automation on Situation Awareness Situation awareness refers to the operator’s understanding of the state of the relevant environment and his or her ability to anticipate future changes and developments in that environment [6]. A widely cited definition of SA specifies three levels of the construct in terms of 1) perception, 2) comprehension, and 3) projection [7]. Factors that impact SA in the context of automation include automation complacency, automation mistrust, workload, and automation transparency. Automation complacency occurs when the human falls out-of-the-loop due to over trust in the system [8]. In the extreme case, the human no longer actively processes information to maintain an awareness of the system state, diminishing his or her ability to recover from automation failure. Automation mistrust occurs when the human perceives the automation to be unreliable and devotes excessive attention to monitoring the automation. SA can also be diminished when workload is very high, resulting in attentional tunneling [2] where all attentional resources are drawn to the primary task, reducing the amount of resources available for perceiving and processing other information in the environment. SA is also impacted by the requirement to evaluate choice alternatives when interacting with decision support automation [9]. In these,
740
A.-Q.V. Dao et al.
operators must often integrate the information provided by the automation with their assessment of the situation. This additional workload could also reduce the resources available for maintaining SA. A system is “transparent” when the underlying reason and information behind the behaviors of automation are understood by the operator [10]. In a fully transparent system an operator may be led to attend to too much system information, with resulting high workload and diminished SA [11]. In the other extreme, a system that does not display information or provide adequate feedback regarding system behavior may reduce workload at the cost of transparency. Thus, the lack of transparency can also lead to diminished SA. A system that supports good SA provides transparency at a manageable workload level. 1.3 SA Probing Techniques SA measures are classified as probing techniques, rating scales, or performancecorrelated measures. Probe techniques have been identified as the most promising of the measures because they are sensitive to the operator and task environment [12] and can provide diagnostic information regarding the cause(s) of poor SA. The Situation Global Assessment Technique (SAGAT) and Situation Present Awareness Method (SPAM) are two commonly used SA probe techniques [7][13]. SAGAT questions are administered by stopping the scenario at random intervals so that SA probes do not interfere with the current tasks. Because the operator does not have access to the display during the simulation pauses, this technique is highly dependent on his or her memory for information in the simulated environment. Unlike SAGAT, SPAM does not index SA solely on the basis of the information the operator can remember. Instead, SPAM probes are administered during the course of the scenario while operators have access to information on their displays. This technique is based on the notion that information can be stored in memory or looked up on the display. If SA information, or the location of the information, is in the operator’s awareness, then response times are expected to be faster than when the information had to be located. One drawback of SPAM is that the questions may interfere with the operator’s primary task [14]. The present study attempted to avoid the problems associated with SPAM and SAGAT probes by using short 3-minute scenarios, where a conflict was presented and resolved prior to answering probe questions. The probe questions were administered after the conflict resolution and while pilots had full access to their displays, so much of the information in the environment did not have to be held in memory. And, because the scenario stopped, there was no additional workload imposed on the operator’s primary tasks while answering the questions. 1.4 The Current Study Commercial pilots viewed scenarios requiring conflict resolutions, and were presented three aiding conditions: automated, interactive, and manual. In the automated condition, pilots were required to evaluate resolutions generated by the automation. They were not allowed to modify this resolution. In the interactive condition, pilots were given the automated resolution, but were free to modify it with a Route Assessment Tool (RAT) prior to executing it. The RAT allowed the pilot to
The Impact of Automation Assisted Aircraft Separation on Situation Awareness
741
graphically stretch and bend the flight plan to generate a preferred conflict free path but did not itself provide a recommended resolution. In the manual condition, pilots resolved the conflict using the RAT alone (no automation). Using the above SA probe technique, the current study assessed pilot SA across the aforementioned three automation levels (manual, automated, and interactive). We predicted that workload in the manual condition would impose a greater demand on the pilot’s attentional resources, reducing SA compared to conditions with automation aiding. We expected that SA in the fully automated condition would be lower than in the interactive condition due to complacency. We reasoned that the SA obtained by actively examining alternatives to the suggested resolutions in the interactive condition would more than offset the loss due to the increase in workload.
2 Method 2.1 Participants Seventeen commercial airline pilots with glass cockpit experience participated in this study and were compensated $25/hr. Data from five of the pilots were excluded due to difficulty in learning to use the display and conflict resolution tools. 2.2 Apparatus All scenarios were presented on a 30” monitor using a Cockpit Situation Display (CSD; see Figure 1), developed by NASA FDDRL.
Fig. 1. CSD with conflict alerting and route proposal
The CSD, a PC-based 3D volumetric display, provided pilots with the location of surrounding aircraft, plus the ability to view the expected 4D trajectories of ownship and all traffic [15]. Embedded within the CSD was logic that detected and highlighted conflicts. Pilots were told that conflict detection was 100% reliable. In addition, the CSD had pulse predictors that emitted synchronous bullets of light that traveled along the displayed flight plans at a speed proportional to the speeds of the associated aircraft. In this way, a prediction of up to 20 minutes into the future was added to the display and provided a graphical way of confirming how close ownship was expected to come to any other aircraft.
742
A.-Q.V. Dao et al.
A version of Erzberger’s auto-resolver was implemented to aid pilots with the conflict resolution task in the automated and interactive conditions [4]. In addition, in the interactive and manual conditions participants were asked to resolve conflicts by modifying their flight plan using the RAT [16]. This tool was linked to conflict detection software allowing the pilots to find conflict-free paths. Proposed resolutions created by the automation or the operator were color coded in gray to distinguish them from the current route (amber if in conflict; magenta in nominal conditions). A pool of candidate questions was generated to capture two important dimensions of SA, time frame and processing category. A question’s time frame refers to whether it queried awareness involving past, present, or future events. A question’s processing category was divided into recall and comprehension. Recall refers to queries where the pilot had to recall the information, or where the information was located on the display. Comprehension refers to queries where the pilot had to process information before a response was made. Together, these dimensions generated six distinct probe types (see Appendix A for sample questions). All questions were checked by a commercial airline pilot for relevancy to the task. The six types of probes were counterbalanced so that each aiding condition received four probes from each category. Presentation order was randomized for each trial. 2.3 Design and Procedure The SA data were collected within a larger simulation that examined pilots’ acceptance of, and preferences for, automated conflict resolutions, and the impact of these automated resolutions on SA [17]. This paper analyzes the SA probes as a function of 3 aiding conditions, 3 question time frames, and 2 question processing categories. Participants completed 3 blocks of 16 trials. Each block presented only one level of aiding. For the automated and interactive conditions, half of the proposed resolutions were vertical and half horizontal. All participants received the automated condition in the first block due to constraints imposed by goals of the larger study [17]. Following the first block, participants were trained on how to use the RAT to create or modify the conflict resolutions. Half of the participants then received the manual condition followed by the interactive condition, while the other half received the opposite order. Each trial lasted about 3 minutes with the entire experiment lasting approximately 4 hours. Participants were provided breaks after each block of trials. During the initial 15 seconds of each trial participants could manipulate the display in any manner, but could not modify the route. The automation proposed resolution was then displayed in the automated and interactive conditions. Pilots could modify the route in the interactive condition. No automation resolution was shown in the manual condition, but the pilots used the RAT to create a resolution. For all aiding conditions, pilots had a 90 second window in which to examine/modify/create a resolution. Pilots terminated this window by executing the resolution. Pilots were encouraged to change their display between 2D and 3D views when analyzing the conflict to fully visualize the possible resolutions, especially those generated by the automation. Transitioning between 2D and 3D views prevented the pilots from making perceptual errors in differentiating vertical from horizontal resolutions. The pilots were also trained to use a number of decluttering features and could zoom in or out of the situation by varying the horizontal and vertical range.
The Impact of Automation Assisted Aircraft Separation on Situation Awareness
743
At the end of each trial, pilots were given three SA questions. It was emphasized that the questions were being used to assess SA. Therefore, participants were NOT to deliberately search for potential information being queried but to focus on the primary task, conflict resolution, during the active trial. The display remained accessible while the probe questions were presented in order to allow pilots to look up information.
3 Results Responses to the SA questions were reviewed for anomalies and missing values, and three comprehension questions were thrown out as unscorable because definitive answers could not be determined from the information available on the display at the end of the trial. The review also uncovered missing data points (4%) due to computer error and incomplete trials. These data were replaced with unweighted means calculated from the responses for each individual participant for the specific condition. No participant was missing all responses for any condition. 3.1 Analysis of Correct Responses to SA Probes The percent of correct responses to the SA questions was analyzed in a 3 (aiding condition: automated, manual, interactive) x 3 (time frame: past, present, future) x 2 (process: recall, comprehension) within-subjects analysis of variance (ANOVA). Means and standard deviations are presented in Table 1. The main effect of aiding condition was not significant, F(2, 22) = 1.37, p = .27. Participants answered a similar number of the questions correctly in the automated (73%), manual (72%) and interactive (69%) conditions. Table 1. Means and standard deviations for percent correct responses and reaction times in seconds to SA questions % Correct RT Aiding Condition Automated 73 (8) 27.17 (9.01)ab Manual 72 (8) 18.20 (5.60)a Interactive 69 (10) 19.62 (6.33)b Time Frame Past 78 (7)a 18.12 (4.40)a Present 78 (9)b 17.38 (4.52)b Future 58 (8)ab 29.49 (8.33)ab Process Recall 80 (7)a 19.07 (5.79)a Comprehension 63 (10)a 24.26 (5.50)a Note. Means in the same column within a condition that share a subscript differ at p -10dB > -7dB square wave: -16dB > -13dB > -10dB − Modulation period: 1Hz > 2Hz > 4Hz − Frequency: 320Hz-640Hz > 640Hz-1280Hz > 1280Hz-2560Hz The signals are combined by following three patterns: one parameter change, two parameters change, three parameters change. − incremented by one parameter: three signals − incremented by two parameters at the same time: three signals − incremented by three parameters at the same time: one signal These patterns are made on both sign wave and square wave, so the total is 14 signals. Each signal is fifteen seconds (the parameters is changed every five seconds). Evaluate the first 5 seconds
Evaluate the next 5 seconds
Evaluate the last 5 seconds
Play The highest urgency
The lowest urgency
Repeat Fig. 2. An example of the display of the second experiment
Procedure A note PC and a headphone are also used for this experiment. The 14 samples are played in random order. Subjects are required to evaluate the degree of urgency by sliding a bar when the signal is played(Figure 2). One signal is composed of three 5-second parts. So
832
M. Nakatani et al.
they have to evaluate three times for one signal. Subjects are sixteen university students and graduate students(13 males and 3 females. Age is from 22 to 25.)
The degree of urgency
Result Average time to evaluate 14 signals was about five minutes per one subject. The slide evaluation score is ranged from 0 to 9. In Figure 3, the evaluation score of the reference value of parameter is center justified for comparison of all signals. When one parameter is changed, the degree of urgency increases slightly according to the incremented parameter. Among them, frequency is an effective parameter to inform the dangerous situation. When two parameters are changed at the same time, the degree of urgency increases more. When three parameters are changed altogether, the degree of urgency considerably increases (Figure 3). 9 8 7 6 5 4 3 2 1 0
The basis value
Three parameter are changed all together. Modulation is changed.
Frequency is changed. Pitch is changed.
First 5 second
Modulation: 1Hz Frequency: 320Hz Pitch: -13dB -16dB
Next 5 second 2Hz 640Hz -10dBd -13dBd
Last 5 second 4Hz 1280Hz -7dB (sign wave ) -10dB (square wave)
Fig. 3. The result when one parameter is changed and when three parameters are changed at a time
4.3 Discussion Previous researches say that the higher the frequency of signal is, the more danger people feel it. But the result of the first experiment suggests that the high frequency isn't always more anxious. Though it is uncertain that the response time is the best way as the evaluation method, the result shows that subjects respond at short times in the particular values of parameters. This suggests that the warning signals have the appropriate parameters. The result of the second experiment suggests the possibility to control the impression of the warning signal. By combining the number and range of the parameters, we will be able to operate the urge to escape. Furthermore the dynamic change of the warning signal will prevent people from getting accustomed to the emergency call and cause a refreshing feeling.
A Study of Auditory Warning Signals for the Design Guidelines
833
This time subjects are only students. We have to examine with subjects in a wide age range and compare with different cultures.
5 Conclusion Today, disaster prevention and mitigation are the important problems. A lot of efforts have been done to minimize damage. To develop the appropriate evacuation calls is one of the tasks, because most people don’t escape despite hearing the call. The auditory property of the warning signals has been studied by many researchers. Most of them use the subjective evaluation and don’t focus the temporal changes of signals. In this paper, we adopt the measurement of response time to evaluate quantitatively and focus on the temporal change of signals. We think the dynamic change of signals will prevent people from getting accustomed to the emergency call and cause a refreshing feeling. The result of two experiments shows the possibility to control the urgency impression by operating the parameters. As future tasks, we have to make experiments with more subjects and more countries. Our final goal is to construct the guideline of interface design for emergency information system.
References 1. Honma, M., Katada, T.: Study on the Relationship between Disaster Advance Information and Resident Evacuation In Tsunami Disaster Prevention. Journal of Disaster Information Studies 6, 61–72 (2008) (in Japanese) 2. Katada, T., Kuwasawa, N., Kanai, M., Kodama, M.: Study of Social technology and Reassurance from Tsunami Disaster. Sociotechnica 2, 191–198 (2004) (in Japanese) 3. Kuwano, S., Namba, S., Shick, A., Höge, H.,, Fastl, H., Fillipou, T., Florentine, T., Moesch, H.: The timbre and annoyance of auditory warning signals in different countries. In: Proceedings of the International Congress on Noise Control Engineering, pp. 3201– 3206 (2000) 4. Edworthy, J., Loxley, S., Dennis, I.: Improving Auditory Warning Design: Relationship between Warning Sound Parameters and Perceived Urgency. Human Factors 33(2), 205– 231 (1991) 5. Stanton, N., Edworthy, J.: Human Factors in Auditory Warnings, Gower Technical, UK (1998) 6. Namba, S., Kuwano, S., Kinoshita, K., Kurakata, K.: Loudness and timbre of broad-band noise mixed with frequency-modulated sounds. Journal of the Acoustical Society of Japan (E) 13(1), 49–58 (1992) 7. Guo, S., Nakazawa, M., Ouchi, Y., Yamasaki, Y.: 1bit Alert Sirens. In: Proceedings of 1bit Fourum 2005, pp. 14–20 (2005) (in Japanese) 8. ISO7731: Ergonomics– Danger signals for public and work areas – Auditory danger signals (2003) 9. ISO7029: Acoustics – Statistical distribution of hearing thresholds as a function of age (2000) 10. Kuwano, S., Namba, S., Shick, A., Höge, H.,, Fastl, H., Fillipou, T., Florentine, T.: Subjective impression of auditory danger signals in different countries. Acoustical Science and Technology 28(5), 360–362 (2007)
834
M. Nakatani et al.
11. Yamauchi, K., Takada, M., Iwamiya, S.: Functional imagery and onomatopoeic representation of auditory signals. Acoustical Science and Technology 59(4), 192–202 (2003) (in Japanese) 12. Namba, S., Kuwano, S., Mizunami, T.: Subjective evaluation of synthesized signals. In: Proc. 3rd Jt. Meet. Between ASA and ASJ, pp. 451–454 (1996) 13. Rogers, W.A., Lamson, N., Rousseau, G.K.: Warning Research: An Integrative Perspective. Human Factors 42, 102–139 (2000)
Computer-Aided Collaborative Work into War Rooms: A New Approach of Collaboration Jeremy Ringard, Samuel Degrande, Stéphane Louis-dit-Picard, and Christophe Chaillou ALCOVE, bâtiment IRCICA Parc scientifique de la Haute-borne, 50 av. Halley 59650, Villeneuve d'Ascq, France {Jeremy.Ringard,Samuel.Degrande,Christophe.Chaillou}@lifl.fr,
[email protected] Abstract. This paper presents the realization of a new software and hardware platform for collocated collaborative work. Our objective is to take the most of the various competences of the teammates. We have created an architecture named MVT (model, view, tool) for supporting collaborative interaction in warroom-like environments. This software distribution offers various interactions modalities, allowing multi-skilled teams to collaborate using different input devices, thanks to multiple visualization and interaction channels. Keywords: collocation, collaboration, CVE, war room, teamwork.
1 Introduction For several years, technologies that could allow efficient real-time collaboration between co-workers separated by a geographic distance have been a major problematic for scientific community. Increasing needs from industry regarding project management brought researchers about working on the development of some tools allowing several teammates to work together on a common project. Several approaches have been issued, through technologies like videoconferencing, document sharing applications, or collaborative virtual environments (CVE). CVEs represent a major subject of research: it consists in bringing several distant users together in a common 2D or 3D environment. These CSCW (computer supported collaborative work) technologies have mainly focused on distant collaboration, which is a situation that occurs while working on coconception activities, but some real collaboration's use cases are still left aside: in a real situation, the co-workers are more likely to be located in the same building. Moreover, when the project requires the intervention of distant users, there is a poor probability for the entire team to be geographically dispersed. This involves group to group collaboration, rather than peer to peer. Consequently, companies are massively adopting others collaboration strategies for teamwork: the “war room” configuration [1][2]. The main advantage of this type of collocated work resides in the permanent and direct contact between the co-workers which allow them to be able to respond immediately to any unexpected issues. M.J. Smith and G. Salvendy (Eds.): Human Interface, Part II, HCII 2009, LNCS 5618, pp. 835–844, 2009. © Springer-Verlag Berlin Heidelberg 2009
836
J. Ringard et al.
Despite the massive adoption of this type of configuration by companies, very few scientific research have focused on software support for collocated work such as war rooms, let-alone regarding interaction on virtual models. In this study, we present a new approach of computer aided collaborative work, through the development of a software that can be assimilated to a traditional CVE, but optimized for collocated multimodal collaboration. Our proposition is a new software and hardware platform that allows a team of users to interact and communicate in the same way than in a war room-like environment. The software platform is especially designed for scenarios like working on virtual models (from architecture to complex mechanic systems). Every user is able to interact differently on the objects thanks to a large range of tools. Before describing precisely our work, we will firstly present the state of art about collaborative software solutions. The next section will describe the concept and the problems addressed by our war room, before detailing the functionality of our prototype. We will conclude this paper by presenting the future works that are planned.
2 State of Art Among all the technologies developed to assist teamwork on a project by synchronous interaction, tools like CVEs are the most complete. This kind of support consist in allowing people to cooperate by virtually bringing them together in a virtual environment (most likely 3D), providing them several tools for interacting on shared virtual objects, and some distant communication means[3][4]. The major benefit brought by CVEs is the possibility to set up purely virtual teams. However, those technologies turn out to be not properly adapted to some use cases: in most of the real situations, the working group is composed at least partly by some collocated people. For these reasons, the working methods have been adapted to this parameter, through the use of war rooms. Jason Leigh has published some complete works regarding this kind of configuration [5]: Working in a war room consists in bringing a whole collaborators team in a closed space, for working on a common project. This project can be a simple brainstorming task, crisis management, or every type of task that requires good reactivity and instant feedbacks between co-workers. Regarding material, the observation of a “traditional” war room is quite interesting, because of the diversity of writing surfaces that are involved in the room [5]. We can cite, among other things, a surface dedicated to the permanent display of some particular information (a corkboard to pin the planning). In the same way, another large scale surface can be used for several teammates to work together on one subtask (a blackboard). Each surface is parented to one kind of subtask. An interesting fact regarding this surfaces diversity is the spatiality of processed data: Leigh et al. [5] highlights the fact that people talking about one element of the project naturally refers to the place where the associated surface is located. Generally, the link between spatial distribution of data and the sequence of collaboration is strong. Mark et al. [1] mention the fact that every user establishes a interdependencies map, that allow them to immediately identify which teammate is
Computer-Aided Collaborative Work into War Rooms
837
the most likely to perform a subtask, depending of everyone's competences. The efficiency of a collocated team have been highlighted in multiple papers [6][2][7]. In our vision of the war room, we can associate this notion of spatial and human distribution of the competences with the multiplicity of writing surfaces: We propose to enrich these surfaces by replacing them with more complex interactive devices. As a consequence, a direct link can be established between the devices and the users’ competences. The major consideration enhancing the war room efficiency is a consequence of the natural human contact: Direct communication, without using third-party technologies such as videoconferencing or virtual avatars, improves comprehension thanks to non-verbal communication items that are not (or not properly) available when working through a virtual environment. Moreover, the spatial proximity of the teammates provides a better accessibility, and a favorable atmosphere for teamwork. Finally, the mutual comprehension between each user is enhanced by informal conversations that could occur between people in the room [8]. Few approaches tackle the notion of collocated collaboration, especially when using multitouch tabletop devices [9][10]. Unfortunately, those researches only offer one single interaction modality, by working exclusively on tabletop devices. Streitz et al. have developed “I-LAND”, a collocated, multi-computer platform for collaboration [11]. However, this solution does not provide multiple heterogeneous displays and tools for interacting on the documents. Some other papers have been presented, describing collaboration between a group and a single distant user, or more generally group to group collaboration [12][13][14]. However, those approaches don't provide any real interaction activity, and rather focus on communication tasks.
3 Computer Aided War Room: The Concept Even if the papers described above bring some interesting facts for setting up collocated collaboration, none provide a generic tool for virtual interaction in a war room-like environment. This is precisely what we are developing, through the realization of a computer support adaptable to several scenarios. Therefore, our proposition consists in a single room equipped with various workstations, and a virtual environment linked to each computer, allowing collaborative interaction. Our interpretation is different from standard CVEs, since the notion of software support is not used in the same way (fig. 2): − According to CVEs, the couple user-computer is fixed, so this couple is considered as one single entity. As a consequence, in a way, the collaboration do not occurs between users, but between computers, The PC to PC link offering a communication channel (videoconferencing, avatars, telepointers,...), and an interaction broadcasting channel. CVEs are not adapted for collocated work: grouping several CVE-connected computers in the same room do not fit our purpose, because most CVE applications provide a WYSIWIS (“What you see is what I see”), relaxed or not. Therefore, the available interactions are the same for each user. This concept does not fit the fact that each user is specialized for one kind of task. Moreover, the fact that each user
838
J. Ringard et al.
is permanently linked to his workstation lower their freedom, as well as the cerebral stimulation provided by collocated teamwork. − Contrary to the configuration discussed above, our approach emphasizes diversity, from visual representations and from interaction. In our proposition, the major collaborative entity is composed of the whole team. The semantic links established between the users and their workstations are breakable and modifiable as required, depending of the needs of each teammate at every instant. The stations count is independent from the physical team. The interaction objects, as well as the virtual tools, can be diffused freely among the workstations (including simultaneous displays) depending of the desired activity distribution.
Fig. 1. Left: Traditional CVE concept. Right: war room application on the right
The development of our war room essentially relies on three elements: Firstly, the heterogeneity of the hardware devices provided to the users, then the interfaces plasticity, and finally the spatial localization of the teammates in the room. 3.1 Hardware Heterogeneity The devices catalog available now allows us to benefit from a large variety of interaction modalities and visualization surfaces. It appears natural to take advantage of this diversity by providing on each workstation the most adapted interaction tools and display methods. When collaborating on a large scale project, multiple different skilled professionals are likely to be involved. As a result the users' needs regarding the project's object are not the same, depending of the human specialties involved. Let us take a simple example: A vehicle conception project. This kind of task leads to the presence of several people: A specialist of electronic, who would be in charge of the electric cables repartition inside the vehicle's body, will be more likely to work with detailed blueprints illustrating the inside of the vehicle. In the same time, another specialist (a designer for instance) working on the same project need to work directly on the shape of the car body. This subtask may be easier to perform by interacting directly on a 3D view.
Computer-Aided Collaborative Work into War Rooms
839
Consequently, even when working on the same project, the collaborators' visions are singularly different. Our choice is to take advantage of the hardware diversity to answer those different needs. A multitouch tabletop display, for instance, will be more likely to be used for 2D interaction. In the same way, a large scale stereoscopic screen coupled with a 6 dof input device will appear perfectly adapted for interaction in a 3D environment. Some mobile terminals like PDAs or smartphones, can be useful too, providing tools adapted to the mobility of these devices. 3.2 Plasticity To extend the use of hardware diversity in the war room, we want to propose a functionality that can be assimilated to the notion of interface plasticity [15]. While working in the war room, each change in interaction modality potentially involve a move from the user, to pass from one computer to another. However, in the case of a short-time interaction, it could be useful to provide to the user the ability to import dynamically the tools and/or the objects necessary for this interaction, avoiding physical move in the room. This proposition involves the necessity to adapt dynamically the interface and the interaction modalities from the second workstation to a new display/input device that was not originally planned. We notice that the two notions of hardware heterogeneity and of spatial distribution of activity are closely linked. 3.3 Collocation This third subsection is focused on a specific goal: To take advantage of collocation and the associated phenomena for enhancing collaboration. We have to make sure that the spatial repartition (of data and users), and software tools are pertinent enough to provide optimum conditions. The common denominator between the multiple questions is the users' behavior during collaboration. For instance, regarding nonverbal communication, the fact that the users are brought together in the same room allow them to naturally benefit from the gesture, the gaze direction, or facial expressions. All these elements have a semantic signification that allows the teammates to better understand each other. This could significantly enhance the collaboration, especially in our war room, where virtual and real environment coexist to constitute a single mixed environment. Deictic gesture, for instance, which is used to show something to the others, not only apply to the 3D scene displayed, but can refers to the physical space, i.e. the room. Moreover, we have seen in the section above that co-workers established several dependences map, including tasks, specialties, and data's localization. This phenomenon may be determining regarding the collaboration's efficiency. Consequently, the spatial distribution of the physical devices could influence the teamwork. In the same way, the distribution of virtual objects across the workstations must keep coherence regarding the dependences map. An evaluation should be performed to understand exactly, through several different configurations, what choices are the most judicious, and what the potential effects of the distribution on teamwork are.
840
J. Ringard et al.
4 War-Room Software Support Proposal 4.1 Software Design We have chosen to start from a existing platform to develop the war room software: SPIN|3D [3]. SPIN|3D is a CVE software platform whose major particularity is the fact that every object in the environment is composed of a structure close to the well known Model-View-Controller structure (MVC), named Model-View-Tool (MVT). (fig. 2). In the MVT architecture, the tool is a component that is used to interact on the virtual object by modifying the data associated to this object. The data describing an interaction object is stored in a dedicated component: the “model”. The third component, the “view”, is a graphical representation of this object, for display. The view is a form of translation of the model's data. A single object can have several views.
Fig. 2. An example of distribution of the application
Contrary to the standard SPIN|3D architecture, the presence of every virtual object on the devices is not always necessary. This is why we have developed a completely free distribution of these components between the workstations, by adding a network communication layer on the links between the Model, the View, and the Tool components. So the three components can be distributed on three different computers acting as a single MVT structure, the distribution being transparent to the platform's high-level layers. In practical terms, this enables the fact to split the tasks and to make them independent from the hardware configuration of the war room. Various scenarios are possible (fig.2), such as: − In a configuration providing one device optimized for 3D interaction, while another station offers a better 3D display, the user can use the two computers simultaneously: the first one with the input device (like a remote control), and the second one for visualization. − If a user is working using a 3D view, he is able to send (or duplicate) his local view to a large scale display workstation, to make the object viewable by the teammates. This distributed architecture enables multiple possibilities that we have illustrated through the realization of a prototype, presented in the next section.
Computer-Aided Collaborative Work into War Rooms
841
4.2 The Prototype To illustrate the various features provided by our architecture, we have developed a prototype, including 3 different computers (fig.3): − A 3D accelerated PC, connected to a large scale projection display. This station is also equipped with a wireless interaction stylus (prototype input device [16]) that allows 3D manipulation, and a ARTrack infrared tracking system. − An interactive tabletop, dedicated to 2D interaction through a tactile interface. − A standard laptop. The scenario proposed is a car review application, allowing users to perform simple interactions on a virtual vehicle. On the tabletop, a 2D interface is available for interactions on blueprints views. Thanks to this interface, the user is able to select a color value and to drag it to the car parts. In the same way, the doors can be manipulated with sliders or buttons, which fit better on a 2D interface. Naturally, every interaction performed on this 2D interface is directly displayed on the 3D views. The others workstations are used to provide direct interaction on the car's 3D model, through different parts manipulation. The 3D view is displayed on one station at a time (laptop or 3D optimized PC). Thus, the users are able to “teleport” the car's 3D view from one workstation to the other, depending of the computer they want to use for visualization and interaction. As those two computers don't provide the same computing power, the level of detail of the 3D mesh used as a view is different, depending on the displaying station. Moreover, to avoid too many moves from the teammates in the room, they are able to teleport their pointer between the computers, allowing them to interact at a distance from the display station, and using their local input device for interaction. Consequently, the coworkers are able to interact simultaneously on the 3D car model, using a common display surface, but using their own input devices, and so different interaction modalities. This demonstrator has been presented during the professional exhibition “Laval Virtual 2008” in France, and has received a very positive feedback from the users. A lot
Fig. 3. The prototype
842
J. Ringard et al.
of visitors have shown a great interest in our activity repartition, especially about the complementarity of each viewing/input device. After a brief presentation of the concept and of the presence of a single distributed object, the use of the installation has not presented any major issue. The ability to modify a 3D object thanks to the 2D interface has been pointed as interesting for the most part of the users. A lot of them have expressed their desire to enable distant collaboration with the room.
5 Future Works 5.1 Mixed Presence While working on large scale projects, bringing every teammate in the same geographic point is not always possible. Therefore, it can be interesting to keep all the advantages provided by CVEs by allowing a third party user (or team) to integrate the collocated team without moving. The major issue raised by this feature lies in an asymmetry, which is involved in two ways: First, human asymmetry, already known as “mixed presence” [17]. Groups involved in distant collaboration are confronted to a problematic communication configuration. Secondly, hardware asymmetry can be problematic. If a distant user wants to join the team, he must be able to get connected to the war room from his personal computer, without suffering the lower interaction possibilities of his devices. These two asymmetries must be evaluated on our prototype to understand how to avoid communication issues between distant co-workers and how to enhance collaboration in such a mixed environment. 5.2 Semantic Data To formalize the technology presented above, we can say that the software side of the war room focuses on multiple “interaction channels”. A channel is defined by a display type (2D, 3D, video...), and its associated interaction modalities. Even if it is possible to factorize several interactions between channels to keep them compatibles with any interface, some other interactions may be only available for one channel, without any equivalence on the others. In a situation presenting a single object displayed on two workstations through different channels, every user must be aware of what the others are doing: To maintain the “social awareness”, the interactions (and ideally, the whole interaction sequence) must be displayed on every station, even if the channel are not compatibles. The solution we are currently working on consists in changing the semantic level of the interactions: In a improved-MVT structure as described above, most of the data stored in the model are geometric or visual data. By raising the level of this data to use real semantic information, the graphical interpretations can adapt to the corresponding display channel. The next step in this work is to create a precise architecture introducing semantic abstraction of the MVT model: this solution requires the introduction of software adapters to convert the semantic data into geometric one. In this way, the structure would approach some known models such as PAC (presentation, abstraction, control),
Computer-Aided Collaborative Work into War Rooms
843
that can be used for semantic abstraction of data. The main difference between improved-MVT and these models lies in the absence of a generic “tool”-like component in PAC.
6 Conclusion The proposition we have presented in this paper allows multiple users to collaborate in a collocated environment, while taking the most of the available hardware diversity. The distribution of MVT components between the workstations of the war room provides to the users a great flexibility, allowing them to be fully independent from their position in space, from the display surfaces, and from the input devices. This independence allows taking into account the teammates' specialties by providing them the most adapted interactions modalities and display methods for the subtask they want to perform. Moreover, this solution offers very good inter-users communication, thanks to the direct contacts allowed by collocation. The spatial repartition of the activity, as well as the possibility to invite any distant user to virtually join the collocated team, raises some questions regarding the ability to collaborate in such an environment. As a consequence, an evaluation has to be performed to answer these questions. In the same way, developing the ability to set up a group to group collaboration using war rooms, taking into account hardware and human heterogeneity constraints, represents a new question. Acknowledgments. We would like to thank Dominique Pavy and Arnaud Bouget from Orange Labs for their participation to realize the prototype. This work is funded by the ANR Part@ge project (06 TLOG 031) and by the IRCICA research federation from CNRS.
References 1. Mark, G.: Extreme collaboration. Commun. ACM 45(6), 89–93 (2002) 2. Teasley, S., Covi, L., Krishnan, M.S., Olson, J.S.: How does radical collocation help a team succeed? In: CSCW 2000: Proceedings of the 2000 ACM conference on Computer supported cooperative work, pp. 339–346. ACM, New York (2000) 3. Picard, S.L.D., Degrande, S., Gransart, C.: A corba based platform as communication support for synchronous collaborative virtual environment. In: M3W: Proceedings of the 2001 international workshop on Multimedia middleware, pp. 56–59. ACM, New York (2001) 4. Margery, D., Arnaldi, B., Chauffaut, A., Donikian, S., Duval, T.: Multi-threaded or modular animation and simulation kernel or kit: a general introduction. In: Virtual Reality International Conference, pp. 101–110 (2002) 5. Leigh, J., Johnson, A., Park, K., Singh, R., Chowdhry, V.: Amplified collaboration environments. In: VizGrid Symposium (2002) 6. Olson, J.S., Covi, L., Rocco, E., Miller, W.J., Allie, P.: A room of your own: what would it take to help remote groups work as well as collocated groups? In: CHI 1998: CHI 98 conference summary on Human factors in computing systems, pp. 279–280. ACM, New York (1998)
844
J. Ringard et al.
7. Teasley, S.D., Covi, L.A., Krishnan, M.S., Olson, J.S.: Rapid software development through team collocation. IEEE Trans. Softw. Eng. 28(7), 671–683 (2002) 8. Bos, N., Olson, J., Nan, N., Shami, N.S., Hoch, S., Johnston, E.: Collocation bindness in partially distributed groups: is there a downside to being collocated? In: CHI 2006: Proceedings of the SIGCHI conference on Human Factors in computing systems, pp. 1313–1321. ACM, New York (2006) 9. Nacenta, M.A., Pinelle, D., Stuckel, D., Gutwin, C.: The effects of interaction technique on coordination in tabletop groupware. In: GI 2007: Proceedings of Graphics Interface 2007, pp. 191–198. ACM, New York (2007) 10. Tang, A., Tory, M., Po, B., Neumann, P., Carpendale, S.: Collaborative coupling over tabletop displays. In: CHI 2006: Proceedings of the SIGCHI conference on Human Factors in computing systems, pp. 1181–1190. ACM, New York (2006) 11. Streitz, N.A., Geißler, J., Holmer, T., Konomi, S., Müller-Tomfelde, C., Reischl, W., Rexroth, P., Seitz, P., Steinmetz, R.: 1999. i-LAND: an interactive landscape for creativity and innovation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: the CHI Is the Limit, Pittsburgh, Pennsylvania, United States, May 15 - 20, 1999. ACM, New York (1999) 12. Bezerianos, A., McEwan, G.: Presence disparity in mixed presence collaboration. In: CHI 2008: CHI 2008 extended abstracts on Human factors in computing systems, pp. 3285– 3290. ACM, New York (2008) 13. Mcewan, G., Rittenbruch, M., Mansfield, T.: Understanding awareness in mixed presence collaboration. In: OZCHI 2007, pp. 171–174. ACM, New York (2007) 14. Mark, G., Abrams, S., Nassif, N.: Group-to-group distance collaboration: examining the ”space between”. In: ECSCW 2003: Proceedings of the eighth conference on European Conference on Computer Supported Cooperative Work, Norwell, MA, USA, pp. 99–118. Kluwer Academic Publishers, Dordrecht (2003) 15. Demeure, A., Calvary, G.: Plasticity of user interfaces: towards an evolution model based on conceptual graphs. In: IHM 2003, pp. 80–87. ACM, New York (2003) 16. Ecole Supérieure d’Informatique, Electronique, Automatique. ESIEA, http://www.esiea.fr 17. Tang, A., Boyle, M., Greenberg, S.: Display and presence disparity in mixed presence groupware. In: AUIC 2004: Proceedings of the fifth conference on Australasian user interface, Darlinghurst, Australia, pp. 73–82. Australian Computer Society, Inc. (2004)
Optimizing Online Situation Awareness Probes in Air Traffic Management Tasks Thomas Z. Strybel, Katsumi Minakata, Jimmy Nguyen, Russell Pierce, and Kim-Phuong L. Vu California State University Long Beach, Center for the Study of Advanced Aeronautics Technologies 1250 N Bellflower Blvd. Long Beach, CA 90840, USA
[email protected],
[email protected],
[email protected],
[email protected],
[email protected] Abstract. We examined the effectiveness of situation awareness probe questions in predicting sector performance and behavior in a human-in-the-loop simulation air traffic management (ATM) simulation with low (50%) and high (75%) traffic densities. Probes were presented online during the performance of the air traffic management task, and the accuracy and response latencies were measured. Hierarchical linear modeling was used to analyze the predictive power of each category type. Response latencies for conflict probe questions predicted performance metrics associated with separation assurance. Keywords: situation awareness measurement, air traffic management, NextGen.
1 Introduction The most impacted operators in Next Generation Airspace Transportation System (NextGen) will be those of pilots and air traffic controllers (ATCs). Pilots operating in NextGen environments may assume expanded responsibility for flight planning and separation. ATCs will be using tools that enable them to safely and effectively share responsibility for separation assurance with aircrews and automation, while at the same time being centrally involved in managing aspects of new air-trafficmanagement (ATM) concepts. Presently, the impacts of these NextGen ATM concepts and technologies are unknown, yet success in meeting NextGen objectives depend on optimized function allocations between pilots, ATCs, and automated tools. Effective function allocation requires measures of operator situation awareness (SA), workload, and performance that can assess the impact of changing task demands. Unfortunately, reliable, valid, and robust measures are presently unavailable [1]. SA can be defined as either the processes used to develop and maintain awareness [2], or the information that determine the state of awareness [3]. A precise definition for the construct is still being debated, and consequently, there are no robust measures of the construct. SA measures usually fall into one of three types: subjective, performance-based, and probe. Probe measures query the operator about awareness of information. Two probe methods are commonly used: Situation Awareness Global M.J. Smith and G. Salvendy (Eds.): Human Interface, Part II, HCII 2009, LNCS 5618, pp. 845–854, 2009. © Springer-Verlag Berlin Heidelberg 2009
846
T.Z. Strybel et al.
Assessment Technique (SAGAT) [2], and Situation Present Assessment Method (SPAM) [3]. With SAGAT, an “offline” probe technique, the simulation is frozen, the operator’s displays are blanked, and the operator is queried about information in the simulation. SPAM is an online technique in which probe questions are administered to operators individually during a scenario [3]. Durso et al. [3] showed that SPAM reaction times predicted novice ATC performance after variance due to individual differences in cognitive skills was removed. SPAM reaction times have been shown to be related to measures of ATC [3, 4] and pilot [5] performance. However, some investigations have found that online probing reduces performance and increases workload [4, 5]. One limitation of online probes is that a standard method for developing probe questions is nonexistent. For offline probes, a Goal Based Task Analysis Technique is recommended, but this technique is time consuming and focuses on information requirements without assessing either priorities or understanding of the task. Online probe questions are usually developed with subject matter experts, but information is needed on what (i.e., information content) to query and how (i.e., question format) to query in order for the technique to be useful in comparing NextGen concepts. In our previous investigations, probe questions addressed SA process (recall and comprehension) and time frame (past, present and future), but the content of information probed was not systematically manipulated [4, 5]. The present study examined the relative effectiveness of questions, based on types of processing, time frame, and information content, for predicting ATC performance variables. These categories were investigated in an ATM simulation in which ATCs managed traffic while responding to online probe questions.
2 Method 2.1 Participants Seven students enrolled in the Aviation Sciences Program at Mount San Antonio College and nine retired air traffic controllers (6 TRACON and 3 ARTCC) participated in the simulation. For more information regarding participant background, see Vu et al. [6]. Each participant ran in six test scenarios with the order of scenario presentation counterbalanced between participants. 2.2 Apparatus The simulation was run using the Multi Aircraft Control System (MACS) developed in the Airspace Operations Lab at NASA Ames Research Center. MACS is a medium fidelity simulation for simulating both ground and air side operations [7]. Each participant’s ATC station was a simulated DSR display of combined sectors ZID 91 and 81. Simulated datalink and conflict probe tools were unavailable for ATC-pilot communications and conflict probing, although a simulated datalink window located outside of the DSR screen was used for online probing. Participant ATCs communicated with pseudopilots located in an adjacent room via VoiceIP software [8]. Six 40-minute scenarios were created, three of which approximated current-day low (50%) and high (75%) traffic densities. An automated ghost controller station
Optimizing Online Situation Awareness Probes in Air Traffic Management Tasks
847
managed all traffic outside the participants’ sectors, and initiated handoffs to the participant ATCs 15nm outside the sector boundaries. ATC participants, when appropriate, initiated AC handoffs to the ghost controller, which were automatically accepted after 30 seconds. 2.3 Procedure Twelve probe questions were developed for each scenario. These were administered at three-minute intervals beginning at four minutes into the scenario. Probes were presented to participants in a datalink window located on the right side of the DSR display at roughly eye level. Participant responses were made with a CH Products Multifunction Panel that allows keys to be arranged in any order. Each key was programmed with a macro consisting of key presses, mouse movements and clicks to send a coded message from the probe display. Probe queries were administered by an experimenter located in an adjacent room. A probe sequence began with a “Ready Question” message sent to the participant’s datalink window accompanied by an audio alert. When the participant had sufficient time to take a question, he/she pressed the Ready button sending an affirmative message back to the experimenter. The experimenter immediately sent the probe question, and the participant responded by selecting one of the six buttons located on the bottom of the response panel. If the Ready response was not acknowledged after two minutes, the query was withdrawn and the next probe was sent one minute later. Queries were developed with subject matter experts who were familiar with the scenarios. The individual questions fit into one of three information processing categories, search/recall, comprehension and subjective assessment, and two time frames, immediate-past/present and future. Examples of questions fitting each combination of processing and time frame are presented in Table 1. Search/recall questions (e.g., Questions 1 and 2 in Table 1) could be answered by retrieving information from memory or finding information on the ATC display. No other processing was required to respond correctly. Comprehension probes (Questions 3 and 4 in Table 1) were used to assess the operator’s understanding of the situation. Correct answers to these queries required the operator to retrieve information from memory or the display and process it. Subjective rating questions (Questions 5 and 6 in Table 1) were questions in which the participant provided an assessment of either the likelihood of an event or severity of a conflict. For each processing category, the probe question was directed at either the immediate past or present state of events, or required projection into the future. In addition to the processing/time frame categorization, the content of probe questions addressed three areas of ATC task knowledge: Sector Status, Commands and Communications, or Conflicts (see Table 1). Questions on sector status requested information regarding current sector state, such as number of aircraft, number departures or distance to a boundary. Command/Communication questions probed ATCs knowledge of the next likely command to be issued, the last command issued, handoffs, and communication errors. Conflict questions probed knowledge of current and future conflicts between an aircraft pair. In addition to information contained in the question, the format of the questions was categorized as Multiple Choice, Yes-NO or rating. Multiple choice questions were answered by selecting one of six alternatives,
848
T.Z. Strybel et al.
Table 1. Examples of probe queries and their classification based on processing (RC: Recall, CMP: Comprehension, SB: Subjective Assessment), time frame (IP: Immediate Past/Present, F: Future), Information Content (SEC: Sector Status, COM: Command/ Communication, CNF: Conflict), and question format (MC: Multiple Choice, OT: other). Processing & Time Frame
Information Content & Question Format
REC CMP SB
Sample Question
1. How many AC are in descent to SDF NOW? 2. Will FDX32 be the next overflight to exit your
SEC
COM
CNF
IP F IP F IP F M OT MC OT MC OT C 9
9 9
9
sector?
3. How many pilot read back errors in the last 5 min.? 4. How many conflicts will ASQ381 have if you
9
9 9
9
take no further action?
5. Rate concern about SWA2898 and AWE989. 6. Rate likelihood you will vector EGF494 for traffic.
9
9 9
9
usually representing a quantity. For example a query “How many aircraft ...” was responded by selecting one of six response buttons labeled 0 thru 4, and 5+. Yes-no questions required answer of agreement/disagreement, and ratings were made on a six item scale corresponding to the six response buttons with the left-most button labeled “very low/very unlikely” and the right-most button “very high/very likely.” Thirtyseven probes were multiple choice format, 17 yes-no and 18 rating. For subsequent data analysis, yes-no and rating questions were combined into an “Other” category. Unfortunately, the number of questions addressing each content area, processing category/time frame, and format combination was not equivalent. Therefore, each category was analyzed separately, and the interpretation of our results is limited to the effects of each probe category. Participants’ responses to probe questions were time stamped and saved in MACS data files. The correct answers for each scenario and participant were obtained by reviewing scenario video and audio recordings and MACs data files. The mean percent correct for probes were determined and averaged based on processing categories, time frame, information content and question format. Response latencies for correct and incorrect answers were also determined as a function of each category. These were analyzed as a function of participant group and traffic density. The results of participant experience are reported elsewhere in this volume [6]. The following ATC performance variables were analyzed:
Optimizing Online Situation Awareness Probes in Air Traffic Management Tasks
849
• Mean Handoff Time: The average time per aircraft between accepting a handoff and handing it to the next sector. • Handoff Time Standard Deviation: Standard deviation of handoff times for each participant and sector. • Mean Sector Time: The average travel time through the sector per AC. • Sector Time Standard Deviation: Standard deviation of sector times in a scenario. • Number LOS. Total number of LOS per scenario. • Average Vertical Distance: The average vertical distance between each aircraft pair. From voice transcripts we obtained measures participant behaviors: • Percentage of altitude, heading and speed changes: Relative number of changes made to aircraft in terms of altitude, heading and speed. • Number of Traffic Advisories: Number of messages that pointed out nearby traffic. • Number of Corrections: Number of times a corrections to an instruction was issued. • Total Number Communications: Number voice messages sent by the ATC participant. We examined the effects of probe categories on accuracy and latency, and the effectiveness of each probe measure in predicting ATM performance behaviors.
3 Results 3.1 Probe Performance The percentage of correct responses and response latencies were analyzed with separate mixed ANOVAs with factors of experience, traffic density, processing, and time frame. A significant interaction of time frame and processing category was obtained for accuracy, F(2,28) = 10.91, p
