Multimedia and E-Content Trends

Peter A. Bruck (Ed.) Multimedia and E-Content Trends

VIEWEG+TEUBNER RESEARCH Smart Media und Applications Research Herausgeber: Prof. Dr. Peter A. Bruck

Peter A. Bruck (Ed.)

Multimedia and E-Content Trends Implications for Academia

VIEWEG+TEUBNER RESEARCH

Bibliographic information published by the Deutsche Nationalbibliothek The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data are available in the Internet at http://dnb.d-nb.de.

1st Edition 2008 All rights reserved © Vieweg+Teubner | GWV Fachverlage GmbH, Wiesbaden 2008 Readers: Christel A. Roß | Anita Wilke Vieweg+Teubner is part of the specialist publishing group Springer Science+Business Media. www.viewegteubner.de No part of this publication may be reproduced, stored in a retrieval system or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the copyright holder. Registered and/or industrial names, trade names, trade descriptions etc. cited in this publication are part of the law for trade-mark protection and may not be used free in any form or by any means even if this is not specifically marked. Cover design: KünkelLopka Medienentwicklung, Heidelberg Printing company: STRAUSS GmbH, Mörlenbach Printed on acid-free paper Printed in Germany ISBN 978-3-8348-0754-0

Preface This book contains a selection of papers and presentations from the 6th EADiM Academic Network Conference held in Graz in November 2007. The Academic Network originates from the assessment process of the EUROPRIX Multimedia Awards. Many enthusiastic teachers and instructors have encouraged their students and graduates to take part in this competition with their final projects or first products in terms of innovation and added value for users. The network was formally established in 2002 by instructors of award-winning interactive e-content projects. Participants in the network share an interest in identifying successful approaches in teaching and in optimizing their teaching methods. The network has started off a European masters program and a number of other cooperation projects. The Academic Network’s annual conference provides a forum for the exchange of latest experiences, it is a meeting place for nominees and winners from the EUROPRIX Multimedia Awards, and it reflects on the development of e-content technology and industry. Academics, instructors, and professors from all parts of Europe present their new approaches to successful and effective teaching. They discuss methods on how teaching models can better meet the requirements of companies and industry at large. Discussions have been raised how academia can come up to the rapid development of the ICT industries and the fast change of platforms for which content is needed. This book provides a good overview of the range of issues addressed and the work done by the academics associated with EUROPRIX. It is also meant to contribute to the development of teaching models and methods to keep track with the development of IT technologies. Graduates from departments for interactive media will benefit from these developments and become the future content creators of these technologies. The approach of using such IT technologies (mobile, interactive storytelling) for teaching and to transport learning content easily and comprehensively to the students is of further interest. In that sense academia uses the available tools to support learning, and in doing so it seeks to promote the content creation for these applications and devices.

VI

Preface

The books includes papers from the following conference panels: • Augmented Realities and Smart Interfaces • Mobile Location Based Applications • The Mobile Content Paradigm • Current Mega Trends for e-Content Development • Teaching Models – Expectations to Academia and How They Could be Addressed. We invite all readers to join the network and the next conferences. All those instructors, teachers, professors, and tutors are invited who can contribute to the aforementioned discussions and wish to join the circle for the exchange of experiences or to seek new instruments for the effective and successful teaching with interactive media. Please check current information at: www.academics.europrix.org. We would like to thank Melissa Lee Price, the past Chair of the Academic Network during the period covered by this book, for her enthusiasm and ideas which she has brought to the network over and over again. We would also like to thank Rodica Mocan and Emin Dogan Aydin, Chair and Vice Chair of the current period, for their active contribution to the network and their willingness to strengthen the community, follow its goals, and prepare the next conference. We are grateful to Jak Boumans and Cai Melakoski, our colleagues in EADiM, for their support and all the work they do to prepare the conferences and to keep EADiM alive and developing. Without them neither the Academic Network nor this book would exist.

Peter A. Bruck and Jana Egger

Content Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

V

Introduction: The Technological and Economic Dynamics of the Multimedia Content Industry Peter A. Bruck . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1

A Augmented Realities and Smart Interfaces TangibleCubes – Implementation of Tangible User Interfaces through the Usage of Microcontroller and Sensor Technology Stephan Setscheny . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5

Investigation on the Relationships among Media Characteristics, Presence, Flow, and Learning Effects in Augmented Reality Based Learning Kye Bokyung . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

21

Visualization of Machine-Aided Measurements of People Counts in Different Infrastructures Christoph Perhab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

39

B Mobile Location Based Applications Pushing Location Based Games Further – How to Gain End User Suitability Andreas Jakl, Christoph Grün, Jens Krösche and Stephan A. Drab . . . . . . . .

59

Mobile Game Based Learning: Designing a Mobile Location Based Game Sandra Schadenbauer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

73

Thessaloniki’s City Guide: a Tourist Site for Handheld Devices Niki Theodorou . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

89

C The Mobile Content Paradigm The Mobile Paradigm for Content Development Chris Bennewith and Richard Vickers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

VIII

Content

The Complexities of Developing Accessible Web Content For Mobile Devices Richard Hancock . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

D Current Mega Trends in e-Content Creation Teen Appeal – Touching the Moving Point Christina Handford . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Happiness and the Family 2.0 Paradigm Rodica Mocan and Stefana Racorean . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 Aesthetics as an Attribute to Usability: a Critique of (Some) Previous Works Raphael Kominis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

E Teaching Models Is it Possible to Conciliate “e-Learning”, “Learning By Doing” and “Cross Cultural” Approaches When Learning New Technologies? Carina Roels and Alain Gourdain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Events as Organisational Stories: an Event Based Approach for Learning Media Production Tomi Numento and Pekka Uotila . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 The Learning Effectiveness of Cross-Discipline Collaboration Within a Media Production BA Project Mik Parsons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

About the Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

Introduction: the Technological and Economic Dynamics of the Multimedia Content Industry Peter A. Bruck, Austria Technology liquefies business models Multimedia content production is no easy business. The technological spiral of innovation is continuously changing the value chains and business models. While this might be true for many sectors of the ICT industry, it is particularly relevant for those who deal with creative contents and innovative applications. Every new wave of innovation in technology has an immediate impact on the value chain by shortening all material production and distribution cycles and stretching the creative and design cycles. The effects can be clearly seen even in the hardware industry when one looks at the previous successes and recent struggles of Dell and the demise of the old, hardware selling IBM and its re-engineering as an IT integrator and service company. It can also be demonstrated for the software industry where the changes in the cycle of software development are fast paced. The digital interactive content industry appears, however, to be the sector even more affected when it comes to consequences of the innovation spiral. Being the top layer of the information and communication technology system, the content industry is the mainly unasked recipient of innovation from the network layer, the hardware layer and the software layer. Innovations for the content layer do not come in stages; rather, they are continuous, multiplying each other in effects and creating a fluid base for producers, creators and designers. The networks are transmitting faster, platforms are converging, software more computational intensive, applications more virtual reality. The ubiquity of multimedia is today already a fact in most European countries and its effects will deepen in the years to come. Nonetheless, there might be some reassurance on the horizon. The convergence of networks and rapid diffusion of high-speed broadband and the market advent of 3G mobile services will shift attention in all ICT sectors from provision of connectivity and the sale of devices towards content and applications that promise new business opportunities, growth and employment. A new era of demand has the potential to give a significant boost to the digital economy. The potential for digital content growth is thus quite high and growth might only just beginning. Demand for content from consumers and intermediaries exploit-

2

Peter A. Bruck

ing the potential of multiple content delivery channels is extending and supplanting infrastructure push as a major driver. Wire line broadband and mobile high speed connections have to be considered disruptive technologies. They are challenging established business models while creating important development opportunities in many sectors. Mobile content and applications will be major drivers of mobile telecommunication services and content industry revenues in developed countries. Games will be a key application sector for wire line and wireless, other e-content areas will follow. The relationships between content originators and final users are changing, intermediaries are being created or replaced, and attitudes to content ownership and acquisition are changing. User generated contents are a true revolution not just in terms of the development of the media but also in terms of the broader organisation of society. The technologies which enable virtually all citizens of a European country to publish in text, image and sound have brought about a new area for political and economic relations. For the content industry and its professionals, these developments pose enormous challenges and require them to sharpen their roles in the social organisation of information provision and knowledge creation. Many aspects of their work have become redundant or are not any longer being paid for due to the advances in user generation of content and the flood of blogs, information sharing sites and social software platforms. However, complete disintermediation has its clear limits in the value add for instance of editorial selection and commentary or the success of proprietary solutions due to their leadership in technology and service. Rather, the experience of the last three years shows that the major brands from media (in Austria: ORF, Standard), telecom industries (in Europe: Vodafone live, T-zones) or IT (Apple music) systematically increase their market share in paid, legal contents while universities, cultural and government institutions dominate the free access market. Industry ground shifts challenge Academia’s teaching canon The above discussed developments add up to significant shifts in the grounding not only of the economic and business side of digital content industry. The changes affect as well all teaching and research in universities and colleges. Some key issues are worth recounting at the beginning of this book: Wiki Movement: Users are challenging established ownership and distribution arrangements, whether through P2P networks or open access/open archive publishing conventions, or through new mass distribution and inter-community trading. Network availability and broadband applications create possibilities for new

Introduction

3

forms of expression by users. See the success of Wikipedia and currently 256 language versions where users are the content creators for entire encyclopaedias. Different sectors react differently: Scientific, technical and medical publishing has gone towards full digitisation and digital delivery while lifestyle magazines are staying largely print. In the games sector a new on-line segment multiplayer has developed where multiplayer involvement points to entirely new intensities and content formats. Intellectual property and copying issues remain crucial. Three modes of pay: Internet content is seen widely as having to be free of charge. Digital media subscription, pay per use/view and access charges remain the key ways for generating revenues. Companies survive if they are able to generate positive revenue feedback cycles when growing numbers of paying users foster the marketing, development, and distribution of online content and services, which in turn might draw more paying users. Content Gap and Economic Issues: The creative ICT applications and digital content industries are challenged to adapt to broadband, both mobile and fixed; to co-operate and change roles among value chain players (in particular between content owners, network operators, Internet service providers, hardware and consumer electronics suppliers); to fight digital piracy and deal with the role of file-sharing. Major concerns are the role of intellectual property in protecting ownership in both products and services, the enforcement of copyright in a digital world, defining and monitoring fair use and the boundaries of legitimate use, and the interaction between competition law and copyright; to create a regime for digital rights management and customer authentication; to put into place efficient payment methods (especially for micro-payments). . Content Gap and SMEs: Operating in the new interactive content industries is highly complex and challenging: legal issues are critical, the definition of software and application products complex and licensing negotiations often more lengthy and complicated due to intricate technical issues and differing legal regimes across platforms and countries. In addition, oligopolistic content markets with a strong role of market leaders, exclusive access to content or networks (network access gatekeepers) make it very difficult if not impossible for SMEs to stay in the market in the longer run and deploy broadband applications and content. Financing Cycles: The climate for private investment in the creative ICTs is acyclical to the technological advance: Three to five years ago money was readily available, but the technology mostly narrowband; today rich media (DVDOffline) and broadband (Online and Mobile) can deliver new contents and innovative services, but many investors have been burnt five to seven years ago. Often, investment in digital content and digital delivery has to be sustained by

4

Peter A. Bruck

margins derived from traditional market activity. Only few successful new ways of generating revenue have emerged. Moore’s Law is working to increase Content Gap: Performance increases and productivity gains also increase functionalities and reduce prices for users. Often, these gains require structural changes in content creation and delivery industries. On the supply side the new generations of ICTs are leading to changes in the market structure of telecommunications, information services and content firms. Essentially, all the players must reinvent themselves. Network operators need to generate revenue to support investment in next-generation networks and replace loss of traditional business (see: Telecoms around Europe have started TV via ADSL in the last years -> Triple Play). For intermediaries, the market churn is very high and there are few winners. Market complexities increase: New sets of business activities and new roles emerge in the creative ICT and content industry: content design and aggregation, marketing of publishing offers, rights acquisition / management, packaging and distributing content, content protection, management of emerging publishing services, design and sale of interactive advertisement spaces, profiling users, integrated billing management, payment management, customer relation management, security/control services, access management. In order to successfully manage multiple roles and the often combined but then again separate activities a critical size of company or organisation is required. They involve a high degree of co-ordination as well as competition along value chains. Politics is simple: In many countries, public policies do not keep up with the changes in technologies and markets. They adjust individual policies and the regulatory environment sufficiently quickly for smaller market players. However, it is often the case that neither speed nor direction have been recognised and measured and that too little economic analysis is available for networked and traditional businesses in content sectors. Key factors: Governments and their agencies have to recognise their role as content creators and model users, of the importance of procurement and the establishment of best practice know-how and guidelines (see: www.EUROPRIX.org; World Summit Award www.wsis-award.org). Governments have to cooperate with industry to speed up the creation of infrastructures for and the public acceptance of micro-payment systems, electronic signatures, and authentication. They have to counteract piracy and assist in the clarification of use rights along content creation. Finally, governments should consider supporting and investing in the creation of content clusters and a digital content funding for all those areas where there is a significant public interest (health, education, cultural identity).

A

Augmented Realities and Smart Interfaces

TangibleCubes - Implementation of Tangible User Interfaces through the Usage of Microcontroller and Sensor Technology Stephan Setscheny, Austria Abstract The interaction between human beings and technology builds a central aspect in human life. The most common form of this human-technology interface is the graphical user interface which is controlled through the mouse and the keyboard. In consequence of continuous miniaturization and the increasing performance of microcontrollers and sensors for the detection of human interactions, developers receive new possibilities for realising innovative interfaces. As far as this movement is concerned, the relevance of computers in the common sense and graphical user interfaces is decreasing. Especially in the area of ubiquitous computing and the interaction through tangible user interfaces a highly impact of this technical evolution can be seen. Apart from this, tangible and experience able interaction offers users the possibility of an interactive and intuitive method for controlling technical objects. The implementation of microcontrollers for control functions and sensors enables the realisation of these experience able interfaces. Besides the theories about tangible user interfaces, the consideration about sensors and the Arduino platform builds a main aspect of this work. As a result, a practical realisation of this theoretical point of view illustrates the possibilities of the integration of sensors and microcontrollers in technical input devices. A theoretical and practical evaluation of various sensors and their qualification as human-technology interfaces is based on this realisation. The result of this evaluation was the TangibleCubes application, which integrates a tangible interaction concept.

1 Introduction The interaction between humans and computers is affected by the mouse and the keyboard as most usual input devices and the graphic user surface as the output medium. Because of the need to open new interaction possibilities to users, the tangible interaction technology gains in importance.

6

Stephan Setscheny

This more intuitive human technique interaction by tangible user interface is made possible by the use of sensors, actuators and microcontrollers, which enable a detection and interpretation of human actions. The areas of tangible user interface, microcontroller and sensor technology raise different questions. In the context of this thesis following questions are treated: •

What differentiates the tangible user interface from conventional graphical interfaces?

•

Which possibilities are offered by the realisation of new interaction interfaces through the microcontroller technology?

•

Which role do sensors play within a realisation of a human technique interaction?

In order to answer these questions the paper gives a short introduction about the human technology interaction by the use of tangible user interfaces and an overview about the Arduino platform, which is the essential element in the developed application. The main content of this paper focuses on the TangibleCubes application. This application has been realised in context of a diploma thesis and shows an example of an intuitive tangible user interface. This interface exists of two physical cubes, which contain several sensors and a central microcontroller for detecting human interactions. These uncommon interactions can be placed by the user through rotating and moving the cube. As the development includes two physical cubes the TangibleCubes application shows an intuitive way of human and/or group computer interaction. The prototype of this application and the implemented interaction functionalities are explained in the main part of this paper.

2 Related Work Researches have shown that in the areas of tangible user interfaces and input devices, which are based on the rotation of cubes, comparable developments can be identified. 2.1 Telekom Austria Cube A design study in 2007 was conducted by the Research Studios Austria in cooperation with the Telekom Austria. The result of this study was the Telekom Austria Cube. The cube is a new variant of a remote control. The usual functions of a classical remote control (On/Off, volume regulation, transmitter choice as well as portal navigation) are implemented through intuitive movements of the cube, such as rotation, movement and vibrating. The aim of this study was to develop an intui-

Tangible Cubes

7

tive control element for most household and entertainment devices and to realise this device in the purpose of Pervasive Computing. 2.2 ToolStone The ToolStone forms a cordless physical input device, developed by June Rekimoto andEduardo Sciammarella. With this input device the user is able to select software tools by changing the physical position of the device. This physical input device was developed in order to extend the reduced function range of the mouse but it should not displace it. The background is that inefficient development is accelerated by the constantly growing complexity of program products and the variety of menus and available tools for development. To get rid of this problem the ToolStone shifts these visual selection and marking tools into the physical area of manipulation.

3 HCI with Tangible User Interfaces Tangible user interface serve to more intuitive and more productive interaction between users and computers. Tangible user faces make this possible by an interaction with physical objects. Further digital information is represented in a physical form, so that the separation between input device and representation is waived. These physical objects and artefacts implements the parts of input and output in one object. Ullmer and Ishii define the expression “tangible user interfaces”:„Tangible interfaces give physical form to digital information, employing physical artefacts both as representations and controls for computational media.” In comparison to this, the graphical interfaces make a fundamental distinction between “input devices,” such as the keyboard and mouse, as controls, and graphical “output devices,” such as monitors and head-mounted displays, as portals for representations facilitating human interaction with computational systems. Tangible interfaces, in the tradition of the abacus, explore the conceptual space opened by the elimination of this distinction.

4 Arduino Platform Arduino is an open-source electronics prototyping platform based on flexible, easy-to-use hardware and software. It is intended for artists, designers, hobbyists, and anyone interested in creating interactive objects or environments. Arduino can sense the environment by receiving input from a variety of sensors and can affect its surroundings by controlling lights, motors and other actuators. The microcontroller on the board is programmed using the Arduino programming language (based on Wiring) and the Arduino development environment (based

8

Stephan Setscheny

on Processing). Arduino projects can be stand-alone or they can communicate with software that is running on a computer (e.g. Flash, Processing,MaxMSP). The practical prototype that was developed is based on this platform. Therefore the Arduino version Arduino BT is integrated in the input devices to fulfill central control and communication tasks. The Arduino BT is an Arduino board with an Atmel168 microcontroller and a built-in bluetooth module that enables a wireless communication. Figure 1 illustrates the Arduino BT board, which is integrated in the input devices of the TangibleCubes prototype.

Figure 1: Arduino BT Board [7]

5 Tangible Cubes Application In the context of this thesis a tangibles user interface, which makes it possible to control a software application through direct interaction with physical cubes, was developed. The aim of this tangible interaction technology was to create an intuitive input device, which gets along without the use of a mouse pointer and

Tangible Cubes

9

which is totally controlled by movements of two physical cubes. Apart from the characteristic of the cubes as tangible interaction devices, these cubes form physical containers for carrying digital files, which are administered on a central server. The motivation for its functionality as data containers is founded in the fact, that music, picture and film files are stored on computer systems, although people duplicate these files on more mobile sources as for instance CDs and DVDs. The example of the TangibleCubes tries to go around this unnecessary duplication of files, by serving them just as carriers of references to these centrally saved files.The defined goal of this practice realisation was the development of a device, which unites both an intuitive, tangible input device and the characteristic as storage medium. In this case the TangibleCubes application should demonstrate the possibilities for new input devices through the integration of microcontrollers and advanced sensor technology. 5.1 System Overview The system contains two physical cubes with integrated control electronics, which are connected to a desktop computer over Bluetooth. The software on the computer makes the interpretation of user interactions possible. Beside this it includes an administrative and a playing mode and enables to store a ID reference of the media files on the cube. The server side file management takes place over a MySQL data base, which administers file information and file references. Figure 2 illustrates the communication ways of the system.

Figure 2: System overview

10

Stephan Setscheny

5.2 Software design For the successful realisation of the TagnibleCubes application it was necessary to program the Atmel168 microcontroller, which is integrated in the Arduino platform, with a hardware near programming language. Besides this, the realisation of an example application for the data exchange and the data replay at the medium computer was necessary. Further more, the connection establishment and data exchange over Bluetooth between the cubes and the software running on the media computer represented a critical component during the development of the application.

5.2.1

Media Computer Software

The software running on the media computer was developed with the Macromedia Flash 8 authoring environment and builds a central part in the functionality of the application. This software adopts the connection establishment to the individual cubes as well as the entire interpretation and the processing of the detailed sensor data which is measured from the cube interaction. Apart from the sensor data interpretation the supply of an administrative interface between the connected cubes, the server-based data storage and the possibility of the medium replay are central tasks of the software.

5.2.2

Microcontroller Software

Programming the software running on the microcontroller and planning the routines forms the second emphasis during the software-technical application development. The microcontroller that is integrated in the cube and the program running on it serves on the one hand as transmission interface to the software of the medium computer, as for example during the transmission of the sensor data, and on the other hand as a direct signal interpretation and control element.

5.3 Hardware Design Apart from the software part the hardware realisation builds another focus in the development process. The challenge was to develop two hardware input devices, which enable a simultaneous Bluetooth connection with the media computer and the possibility of a simultaneously control of the program. This realisation of the practical example contains the draft and the construction of two hardware cubes and the implemented electronics circuits. The technical hard of the electronic

Tangible Cubes

11

was realised through the microcontroller Arduino BT board, which allows a fast development of prototypes.

Figure 3: Design of the cubes

With the technical realisation to enable the interaction and feedback possibilities the followingcomponents with a value of 180.48 Euros were integrated into the cube: • Arduino BT board (microcontroller) • Accelerometer sensor (detecting of inclination and vibration) • Tilting sensor (180 degree interaction) • Light resistance (measurement of luminous intensity) • Engine with force weight (vibration feedback) • 8-bit LCD (display for status feedback) • LED - red, green, blue (signal lights and ambient lighting)

Figure 4: T-module with integrated electric circuit

12

Stephan Setscheny

5.4 Interaction Human computer interaction includes input and output methods. The TangibleCubes application integrates a tangible interaction concept and various kinds of feedback methods. The following chapter gives an overview about the realised interaction options.

5.4.1

Input methods

The application works through innovative tangible input devices. These input devises are formed by two physical cubes, which enable the user to interact with a GUI through different input methods. These methods are explained in this section.

5.4.1.1

Rotation and angle of inclination

The rotation and the angle of inclination along the x and y axis of the cube forms the most important input functionality. In principle a direction-independent inclination recognition is possible by the implemented accelerometer sensor. For the realised application only a recognition of inclination along the axis x and y was integrated, as this recognition for the application control is sufficient. This input by the measurement of the angle of inclination serves as main navigation and control possibility. As the interaction was kept as intuitive as possible, either the x or the y axis as navigation direction and the other direction serves for controlling various elements and their interaction possibilities. As a result of not statically fixed direction on performing a navigation and/or a control function, their suitability depends on the viewed interacting direction. By slopes of the inclination along the y axis the user can change between the different control elements in this application mode. During inclination of the cube along the x axis the active data list can be paged through.

Tangible Cubes

13

Figure 5: Rotation and angle of inclination with detection borders

Beside control elements with the descriptive inclination interaction, the software includes elements that are directly depending on the inclination degree of the cube. The volume bar of the player mode for example, sets the volume on 75 percent in case of a 45 degree inclination and a simultaneous strong pressure on the pressure detection area.

Figure 6: Volume control through the angle of inclination

14

5.4.1.2

Stephan Setscheny

Pressure sensitive button

The integration of a push-button actuator gives the user the possibility by manipulation of the pressure area to navigate through the application and to control elements. The integration of such an additional input mode was necessary for the activation of rotation and angle of inclination functionality of the application. This input mode avoids inadvertently interactions by just carrying or picking up the cube. The interaction distinguishes between two possible states of the pressure area. In order to be able to interact with the software, the user must exert a light pressure on the detecting surface so that the inclination recognition gets accessible. The interaction with some elements of the application needs stronger pressure on the detection area, which is the second state. Elements that require this pressure state are for example the file copy function and the volume bar.

Figure 7: Interaction through pressure activation

5.4.1.3

180 degree rotation

A cube rotation of 180 degrees along its y axis makes it possible to change between data administration mode and player mode.The current adjustment of the cube is measured by a digital tilting sensor. In case of a straight position this sensor returns the value 1 volt, and thus the data processing mode. Otherwise it returns 0 Volt and the software shows the player mode.

Tangible Cubes

15

Figure 8: 180 degree rotation

5.4.1.4

Agitation

The third interaction possibility forms the strong agitation of the cube. The software reacts on this interaction by deleting song references from the represented lists in the data administration mode. Detecting, whether the physical cube is shacked along the y axis by the user, is similar to the inclination recognition, realised by the usage of the accelerometer sensor. The measurement takes place via counting of excursions of the axle sensor along its y axis. This is made possible by the G-forces, which result from acceleration and stopping the cube with the integrated sensor.

Figure 9: Agitation interaction and detection borders

5.4.2

Feedback methods

The TangibleCubes application implements different techniques for giving feedback to user. These techniques use the physical as well as the digital way for the representation of information.

16

5.4.2.1

Stephan Setscheny

Graphical user Interface

The graphic software feedback with the GUI of the medium computer is the only expenditure, which is not directly indicated to the user at the cube. This software forms the most important feedback component of the application in regard to the inclination and rotation input. 5.4.2.2

Vibration

The haptic feedback is realised by the implementation of a vibration motor in the cube. The TangibleCubes application, comparable to the usual vibration alarm that signals calls on mobile phones, uses the feedback of the vibration to announce changes of the connecting status.

5.4.2.3

Light

The light based feedback is divided into two parts. On the one hand there are the signal lights with the colours red and green and on the other hand there is the blue ambient background light. The red signal light which is integrated into the cube strengthens the vibration feedback by a visual feedback component. Beside the red LED a green LED is integrated into the cube, which signals the user the manipulation of the pressure sensor and hence the availability of the inclination interaction.The blue ambient light should help the user to find the cubes in dark areas. This blue LED depends on the measurement of a light sensor, which is integrated into the cube. The integrated microcontroller interprets the measured strength of the ambient light and turns on the blue LED if necessary. 5.4.2.4

LCD-Display

The two-line LCD forms the second, digital representation beside the graphic user interface. The 16x2 sign matrix, which is available at the integrated display, signals the developed connections in the first line of the announcement and in the second line the up-to-date implemented functionality mode of the cube. 5.5 Further fields of usage Besides controlling media files the TangbileCubes application presents some further fields of usage, which are described beneath: 5.5.1

Control of household and entertainment media

Based on the technical extension of a direct Bluetooth connection between two or even more microcontroller a controlling of various Bluetooth household appliances or the control of televisions and other entertainment devices would be

Tangible Cubes

17

conceivable. Further more, the cubes could serve as carriers for configuration attitudes of these devices. 5.5.2

Emotional Computing

An interesting operational area would be the emotional computing conceivable by biotechnical or thermal technical sensor extensions for detecting emotions of the user. 5.5.3

3D interaction

The three-dimensional freedom, which the cube offers to the user, makes a more intuitive control of three-dimensional applications possible. Both, the control of 3D-authoring programs as well as the interaction in 3D-games are possible fields of usage.

6 Usability Tests To examine the usability of the implemented TangibleCubes application and the implemented tangible interaction methods, usability tests were organised. The aim of the tests was to find answers for the following questions: •

Do the integrated interaction possibilities form an intuitive and understandable input device?

•

Does existing knowledge in the area of the information technology affect the way of the interaction?

•

How far does the familiarity with the application and the system influence the users by their efficiency in fulfilling tasks?

•

Which hardware- and software-technical improvements can be made?

6.1 Test method The performed test method was a Co-Discovery test, with groups of two persons. This methodology is similar to the Thinking Aloud test, except that it is performed with more than one person. The test was arranged with three groups, which distinguish from each other by their amount of previous knowledge about information technology and the interaction functionalities of the TangibleCubes application. 6.2 Results The evaluation of the accomplished questionings, the logging and the video recording showed both problems and possibilities for improvement and positive factors of this innovative input technique.

18

6.2.1

Stephan Setscheny

Hardware design

With the concluding questioning both the size of the cube with an evaluation of 2.67 out of 5 points and the weight of the cube with the average of 3.33 points were criticized. From the questioning an optimal size of the physical cube results with a side length of 65mm, whereby these data is naturally depending on the individual hand size. 6.2.2

Interaction functionalities

With an average evaluation of over 4 out of 5 points, the integrated possibilities to control the program were seen positively. In particular the 180 degree rotation for changing between the modes and the vibration function for the deletion of unwanted files did not represent any problems for the tested persons. The configured sensitivity was a critical point in case of the inclination interaction but in general this was seen positively. At the beginning the interaction activation over exercising a light pressure on the pressure area produced little confusion. In the course of the tasks this interaction activation became a natural interaction for the users. The interaction over exercising a strong pressure on the pressure area can be seen as problematic for first time users as well as for those who have experiences in information technology. This prevents first time users from partly fulfil some tasks. Despite this, the task fulfilment was no problem for users with basic knowledge skills in the interaction functionalities. 6.2.3

Feedback methods

The feedback possibilities of the application were evaluated differently. It became evident that physical feedback functionalities like the vibration and the light signals in contrast to the common digital functionalities, as the LCD display and the three-dimensional cube representation of the software, were assessed more positively. This evaluation illustrates the meaning and the important role, which the physical feedback method has in the realisation of input devices.

7 Conclusion The implementation of new tangible input devices through the usage of the microcontroller and sensor technology was the goal defined at the beginning of this thesis. In case of this, the focus was on the possibilities for interfaces between human and technology and the associated bidirectional interaction. In context of this viewpoint the subsection of the tangible user interface was an important point of view which was illustrated with examples of related applications. During the theoretical part the bases of the Arduino platform microcontroller tech-

Tangible Cubes

19

nology and the human technology interaction which were necessary for the realisation of such input devices were described. Based on this theoretical consideration the example application TangibleCubes was realised on basis of the open source platform Arduino. By this application an experienceable input device and storage medium was created, which unites the typical characteristics of tangible user interface and different feedback techniques. Usability tests examined this example application and the integrated interaction possibility on their applicableness as a human technology interface. According to the results of the usability tests, as well as by the theoretical and practical treatment of the subject the conclusion is that the intensified implementation of microcontrollers and sensors enables new possibilities for interaction interfaces. It can be seen that by the advancements in the field of sensor technology new intuitive and interactive interaction methods can be realised. In contrast to these methods common input devices are mostly based on simple buttons and switches. The implemented application of the TangibleCubes illustrates such an innovative interaction interface between humans and technology on basis of a prototype.

References Arduino Homepage. Guide to Arduino;04.06.2007 Arduino Homepage. ArduinoBT: Information about the Arduino BT; http://www.arduino.cc/en/Guide/ArduinoBT; 06.06.2007 pcb Europe Homepage. Arduino BT; http://pcbeurope.net/catalog/product_info.php?cPath=29&products_id=52&osC sid=189c984a2d25cda feb04d741dc9ed1ac; 09.06.200 Rekimoto J.; Sciammarella E.: ToolStone: Effective Use of the Physical Manipulation, Vocabularies of Input Devices. Sony Computer Science Laboratories 2000 Sharlin E.; Watson B.; Kitamura Y.; Kishino F.; Itoh Y.: On tangible user interfaces, humans and spatiality. In: Personal and Ubiquitous Computing, Volume 8, pp. 338-346. 2004 Telekom Austria Cube: Pressekonferenz; http://wai.telekom.at/Content.Node/innovation/pr-cube-ferscha.pdf; 09.05.2007 Ullmer B.; Ishii H.: Emerging frameworks for tangible user interfaces. In: IBM SYSTEMS JOURNAL, Volume 39, pp. 915-931. 2000

Investigation on the Relationships among Media Characteristics, Presence, Flow, and Learning Effects in Augmented Reality Based Learning Kye Bokyung, Republic of Korea

Abstract This study’s goal is to examine which factors of augmented reality (AR), the fruit of future technologies, help to improve learning effects and reveal the relationships of those factors. To that end, we studied preceding researches and selected five factors which can influence learning effects in augmented reality based learning. We discovered the effectiveness structure of media utilization in augmented reality based learning through investigating relations of those factors. The five factors selected were: sensory immersion, navigation, manipulation, presence, and flow. A questionnaire was made based on these research questions and a survey was conducted on 290 fifth graders at two elementary schools. A total of 272 cases were examined for this study (incomplete and untrustworthy questionnaires were excluded) and analyzed using a structural equation model. The results showed that with the exception of navigation, all the factors such as sensory immersion, manipulation, presence, and flow had meaningful influence on satisfaction, knowledge & understanding, and learning effects of application. In particular, manipulation factor was proved to have a direct effect on satisfaction and the application aspect of learning effects, indicating that strengthening manipulation through the tangible interface of augmented reality can be an important factor in learning satisfaction and application fields. In addition, sensory immersion was proved to have a meaningful influence on immersion in learning and learning effects. In terms of learning effects, application of augmented reality media was shown to have more influence on application factors than on knowledge & understanding. Based on such study results, we propose the following tasks

1 Introduction Media has always been an object of interest even before the emergence of advanced types of media, and efforts have been continuously made to link media

22

Kye Bokyung

with education. Thanks to the development of computer technology, the fundamental question of, “Will media improve education capabilities?” has now expanded into a different question: “How will technology change education?”(Banathy, 1991: Reigeluth, 1991). The main area of concern for media researchers, however, still lies in the effectiveness of teaching media(Kim Youngsoo, et al., 2006; Na Ilju, 1994). Despite expectations that media can raise the interest or attention of learners by providing a variety of information, it is difficult to find specific studies examining how media improves learning effects or which features of media are related to learning activities. This study’s goal is to examine which factors of augmented reality (AR), the fruit of future technologies, help to improve learning effects and reveal the relationships of those factors. This study examines the relations of those factors based on the following aspects. First, this study, by separating sensory immersion through audio-visual effects from flow, was to examine whether augmented reality’s sensory immersion effect would lead to actual immersion in learning contents and courses and ultimately, to learning effects. Second, in terms of the cognitive aspect of learning effects, we separated acquisition of knowledge from understanding & application for this study. augmented reality technology, which tends to enhance circumstances and context through the combination of reality and virtual reality, is expected to have an effect not only on acquisition and understanding of simple concepts, but also on knowledge application by expanding the scope of learning (Ryu Jihyeon, et al., 2006). Therefore, this study examined whether augmented reality based learning can be meaningfully utilized not in the acquisition of decontextualized knowledge, but in an actual context of knowledge application.

2 Theoretical Background 2.1 Augmented Reality Augmented reality (AR) is a technology that provides a more advanced sense of absorption and reality for users by seamlessly combining the real world with virtual world in real time (Azuma, 1997). Both virtual reality and augmented reality are based on virtuality. Augmented reality is located in the middle, between reality such as TV monitors, and virtual reality, which allows users to become immersed in a computer-created virtual location. Augmented reality also increases a sense of reality by adding virtual information to the user's real environment. Augmented reality differs from virtual reality technology (which completely replaces a real environment with a virtual reality) in that it maintains the

Augmented Reality Based Learning

23

information of the user's real environment. Figure 1 displays the position of augmented reality on the continuum between real and virtual environments. Since Mark Weiser's published article on visions of “Ubiquitous Computing” in 1991, the world has been rapidly changing its direction toward a new paradigm that makes technology invisible. In order for computers to become ubiquitous and invisible, the union between the physical environment and digital information is essential along with a tangible manipulation method that is superior to the existing graphic user interface (GUI) (Ishii & Ullmer, 1997). Augmented reality is a three-dimensional medium which supports such a tangible interface and enables seamless interaction between people and information. 2.2 Presence Marvin Minsky, a professor of artificial intelligence at MIT, was the first scholar to show an interest in presence theory (1979). The definition of presence varies depending on researchers, but it can be divided into the following two parts with a perceptual concept as its common basis. The first part is a condition in which people do not recognize the existence of media when something is presented through the given media. People who do not recognize the media itself when watching TV or movies are a case in point. The second part is a sense of "being there", in which people feel as though they are together with the media even when they are somewhere else. Presence is diversely classified according to scholars. Heeter (1992) classified presence into these three aspects, positing a subjective personal presence, social presence, and environmental presence. He argued that subjective personal presence was important given that recognizing one’s self is a primary issue not only in the real world but also in virtual reality. He also argued that social presence was a necessary next stage, enabling people to recognize the existence of others, and, for the final stage, environment presence of virtual reality, which enables people to react as if they were in the real world, enhances presence. This study, based on these discussions about presence, defines presence as a cognitive condition in which people in an augmented reality environment feel virtual objects in a real world. In contrast, presence in virtual reality means a cognitive condition in which people feel that “I am in a virtual reality.” This differs from the presence of augmented reality, in which people actually feel that “there are virtual objects in the real world where I am.” 2.3 Flow If some factors such as technologies or challenges reach a certain level as people are engaged in various activities in their daily lives, people become engrossed in them. Flow is the condition in which people feel the present experience as an

24

Kye Bokyung

optimum experience of complete absorption. This concept was first presented in a paper titled, “Beyond Boredom and Anxiety” written by Csikszentmihalyi in 1975. He argued that absorption is a psychological and physical energization that people feel when they are completely absorbed in their activities. He also maintained that when a difficult task is combined with high quality competence, in-depth participation and optimal absorption are achieved, which is difficult to attain in the real world. Flow is a result of successive reactions stimulated by interactions, and it is essentially interesting and accompanies a complete loss of self-consciousness. Also, it is characterized by voluntary reinforcement (Novak & Hoffman, 1996). Flow has the following nine characteristics: (1) clear goals (2) immediate feedback (3) balance between challenge and skill level (4) concentration on the task at hand (5) action-awareness merging (6) loss of self-consciousness (7) altered sense of time (8) sense of control and (9) autotelic experience (Csikszentimihalyi, 1975). Csikszentimihalyi (1990) reorganized these nine aspects of flow and categorized them into four stages according to the passage of time: prelude to flow, threshold, experience, and result. Chen (1999) categorized these into the following five stages in his research about computer users. 2.4 Preceding researches and hypothetical concept models Studies on augmented reality have focused on technological approaches. The possibility of its application to diverse fields such as medical science, military and entertainment is being investigated on a laboratory level. However, few researches have proved augmented reality’s effectiveness through its adoption and utilization in the field of education given the fact that augmented reality has just recently been introduced. Therefore, this study examines preceding researches about virtual reality, which are the most similar to augmented reality in terms of media, as well as those on the web environment as a virtual space, and then analyzes the relations between related factors. Lavroff (1994) argued that Virtual Reality and augmented reality are media characterized by a strong presence based on virtuality, and that Virtual Reality has three characteristics (immersion, navigation, and manipulation) which determine presence. Immersion is not a feeling that participants feel when they observe something through a window. It is a feeling that they are actually experiencing a virtual world. This is, primarily, it is a function of hardware, highly dependent upon senses such as sight and hearing. Navigation is a condition in which participants can freely explore and interact in a cyber world created by computing technologies. The feeling of being able to freely walk around enables participants to feel as though the world is real. Manipulation, the third factor enhancing augmented reality system’s sense of reality, means the user's ability to

Augmented Reality Based Learning

25

manipulate the environment of virtual reality. Manipulation allows participants to open a virtual door or shoot a virtual enemy. While navigation is an interaction through which one can explore a virtual place on one’s own, manipulation is an interaction with objects as if they were in the real world, when participants can stimulate objects in virtual reality, such as moving, turning, and constructing them. Flauland (2002), Schubert et al. (1999), Sheridan (1992), and Slater & Wilbur (1995) have proved the influence of immersion, navigation and manipulation factors on presence through theoretical and exploratory researches. Strengthening presence in augmented reality based learning can create immersion in learning contents by enhancing the relationship between learning tasks and reality. Seo Haerim (2003), Koh Jaehyeok (2001), and Novak & Hoffman (1996) have proved that presence is an important factor in learning immersion. Although a large number of researches, including that of Larson (1988), have shown that presence is related to learning achievements, no research about augmented reality has been conducted. Webster, Trevino & Ryan (1993), Massimini & Carli (1988), Novak & Hoffman (1996), Kim Younghee, Kim Youngsoo (2006), Um Myeongyong et al. (2005), and Bricken & Byrun (1993) have proved that immersion had a positive influence on satisfaction levels through researches on web environment. And Larson (1998), Mayer (1978), Novak & Hoffman (1996), Kim Youngjin (2000), Baek Jaehyeon (2006), Park Seongik, Kim Yeonkyeong (2006), and Kim Heesu (2001) showed that learning immersion had a meaningful effect on the enhancement of understanding about learning contents. Also, Antonietti & Cantoia (2000), Gibson (1986), and Shim Gyuchul et al. (2003) have pointed out that encouraging learners who are utilizing Virtual Reality to conduct a high level of reasoning such as analysis and synthesizing had a meaningful influence on the application aspect of learning effects. These preceding researches indicate that augmented reality, as a new learning medium, can provide vivid learning experiences for learners not as an abstract symbol but through direct navigation and manipulation of three-dimensional materials. In addition, it is also expected to increase satisfaction levels in understanding and application abilities of learners by providing relevant learning contents and allowing learners to explore learning tasks. Figure 1. is the conceptual model of the relationship between variables which were selected based on theoretical background and on preceding researches of augmented reality based learning environment that were examined above.

26

Kye Bokyung

Figure 1. Concept Model Drawn from Preceding Researches

3 Research Methods This study was conducted on 290 fifth graders at two elementary schools in Gyeonggi province, where the school facilities and class environments were similar. A total of 272 questionnaires were analyzed after excluding the questionnaires of 4 absent students, 12 incomplete ones, and 2 untrustworthy ones. The program this study used was titled, “Journey of Water”, jointly developed by the Korea Education & Research Information Service(KERIS) and the digital experience center of Pohang University of Science and Technology. The "Journey of Water" program was designed for fifth graders and its contents were experiment activity type learning contents of tangible manipulation in which students could observe and experiment the circulation process of water including its evaporation, precipitation, and flowing. This study went through the process of designing a theoretical model through analyzing preceding researches, developing measuring tools, holding augmented reality based classes, conducting survey and achievement evaluations, and confirming a final model through the analysis of confirmatory factors and structural equation modeling. All the data obtained from the survey and achievement evaluation was processed using SPSS 12.0 for Windows, and AMOS 5.0 was used to verify a hypothetical model.

4 Research Results Confirmatory Factor Analysis(CFA) is a way to confirm a hypothesis model when researchers have knowledge about variables and factors (concepts) and theoretical background about them. This study used Maximum and Likelihood (ML), which supposes multivariate normality for confirmatory factor analysis, and evaluated fitness to check the optimum state of the structure concept and

Augmented Reality Based Learning

27

variable configuration. The fit index and its standard that were utilized for the result model were GFI (Goodness-of-Fit Index: over 0.9), AGFI (Adjusted Goodness-of-Fit Index: over 0.9), TLI (Tucker-Lewis Index: over 0.9), CFI (Comparative Fit Index: over 0.9), and RMSEA (Root Mean Square Error of Approximation: below 0.05). According to the first confirmatory factor analysis, most indexes exceeded the recommended standard (χ2=718.340, df=362, GFI=0.847, AGFI=0.816, TLI=0.904, CFI=0.914, RMSEA=0.060), showing that the overall fitness of the model was high. However, GFI, AGFI, RMSEA indexes fell short of the standard. Therefore, to exclude variables whose factor loading was less than 0.5 from the analysis, the following among factors were excluded one by one from the first measurement equation: NA1 among navigation factors, IM2 among sensory immersion, MA1 among manipulation factors, PR3 among presence factors, FL4 (specific feedback), FL7 (loss of self-consciousness), FL8 (loss of concept of time), and FL9 (self-purposive experience). After the measurement equation was modified in this manner, fit index of this equation changed to χ2=268.847, df=174, GFI=0.916, AGFI=0.900, TLI=0.962, CFI=0.968, RMSEA=0.045, an equation fitness superior to the first one, with improved indexes including GFI and AGFI. To verify the overall structure of the research model, the fitness of the model was analyzed with χ2 verification and fit index, as in the confirmatory factor analysis, and a significant result of χ2=318.086, df=217 was produced. And, according to the fit index evaluation result, the fit index was proved to be satisfactory with GFI=0.909 (recommended level is above 0.9), AGFI=0.885 (recommended level is above 0.9), TLI=0.963 (recommended level is above 0.9), CFI=0.968 (recommended level is above 0.9), RMSEA=0.041 (below 0.05 is good fitness, below 0.08 is reasonable fitness, below 0.1 is ordinary fitness). To improve the values of AGFI and NFI, which were somewhat lower than the recommended level, we explored an optimum model through the addition of free parameters. The results of exploration showed that the modification index of “manipulation satisfaction” was 24.479 and that of “manipulation application” was 11.419, both of which exceeded 10, a conservative standard, proving that there is room for modification in this model. Therefore, through the investigation of the theoretical background which could support these results, the routes that showed the direct effect of “manipulation- satisfaction” and “manipulation application” were added. The final model modified in this manner can be seen in Figure 2.

28

Kye Bokyung

Figure 2. Final model of the relationship between variables

As seen in Figure 2. among a total of eight hypothetical routes which were established in the beginning of the research, seven routes proved to be meaningful at the .001 level of significance, and one hypothetical route of manipulation>presence was proved not to have a meaningful influence at the .05 level of significance. In addition, new hypothetical routes (manipulation satisfaction, manipulation application) were added at the .001 and .01 level of significance, respectively. Based on this final model, for an analysis of the media utilization effectiveness model in an augmented reality based learning environment, the regression coefficient, standard error, and t-value were measured. As seen in Table 1, the results found no large standard error exceeding 2.5 which could cause problems when distinguishing models.

Augmented Reality Based Learning

29

Table 1. Effect analysis result of relations between factors

1. immersion(ξ1) 2. navigation(ξ2) 3. manipulation(ξ3) 4. presence(η1) 5. flow(η2)

**

Path → → → → →

6. flow(η2)

→

7. flow(η2)

→

8. comprehension(η4) 9. manipulation(ξ3) 10.manipulatio n(ξ3)

→

p