Human-Computer Interaction Series
Editors-in-chief John Karat IBM Thomas J Watson Research Center, New York, NY, U.S.A. Jean Vanderdonckt Université catholique de Louvain, Louvain-la-Neuve, Belgium Editorial Board Gaëlle Calvary, LIG-University of Grenoble 1, Grenoble, France John Carroll, School of Information Sciences & Technology, Penn State University, U.S.A. Gilbert Cockton, Northumbria University, Newcastle, U.K. Larry Constantine, University of Madeira, Portugal, and Constantine & Lockwood Ltd, Massachusetts, U.S.A. Steven Feiner, Columbia University, New York, U.S.A. Peter Forbrig, Universität Rostock, Rostock, Germany Elizabeth Furtado, University of Fortaleza, Fortaleza, Brazil Hans Gellersen, Lancaster University, Lancaster, U.K. Robert Jacob, Tufts University, Oregon, U.S.A. Hilary Johnson, University of Bath, Bath, U.K. Kumiyo Nakakoji, University of Tokyo, Tokyo, Japan Philippe Palanque, Université Paul Sabatier, Toulouse, France Oscar Pastor, University of Valencia, Valencia, Spain Fabio Pianesi, Bruno Kessler Foundation (FBK), Trento, Italy Costin Pribeanu, National Institute for Research & Development in Informatics, Bucharest, Romania Gerd Szwillus, Universität Paderborn, Paderborn, Germany Manfred Tscheligi, University of Salzberg, Salzberg, Austria Gerritvan der Veer, University of Twente, Twente, The Netherlands Shumin Zhai, IBM Almaden Research Center, California, U.S.A. Thomas Ziegert, SAP Research CEC Darmstadt, Darmstadt, Germany
Human–Computer Interaction is a multidisciplinary field focused on human aspects of the Development of computer technology. As computer-based technology becomes increasingly pervasive—not just in developed countries, but worldwide—the need to take a human-centered approach in the design and development of this technology becomes ever more important. For roughly 30 years now, researchers and practitioners in computational and behavioral sciences have worked to identify theory and practice that influences the direction of these technologies, and this diverse work makes up the field of human–computer interaction. Broadly speaking it includes the study of what technology might be able to do for people and how people might interact with the technology. In this series we present work which advances the science and technology of developing systems which are both effective and satisfying for people in a wide variety of contexts. The human–computer interaction series will focus on theoretical perspectives (such as formal approaches drawn from a variety of behavioral sciences), practical approaches (such as the techniques for effectively integrating user needs in system development), and social issues (such as the determinants of utility, usability and acceptability).
For other titles published in this series, go to www.springer.com/series/6033
Fabio Paternò Editor
Migratory Interactive Applications for Ubiquitous Environments
1 3
Editor Fabio Paternò CNR-ISTI HIIS Laboratory Via G. Moruzzi 1 56124 Pisa Italy
[email protected] ISBN 978-0-85729-249-0â•…â•…â•…â•… e-ISBN 978-0-85729-250-6 DOI 10.1007/978-0-85729-250-6 Springer London Dordrecht Heidelberg New York British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library Library of Congress Control Number: 2011921322 © Springer-Verlag London Limited 2011 Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms of licenses issued by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms should be sent to the publishers. The use of registered names, trademarks, etc., in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant laws and regulations and therefore free for general use. Product liability: The publisher makes no representation, express or implied, with regard to the accuracy of the information contained in this book and cannot accept any legal responsibility or liability for any errors or omissions that may be made. Cover design: deblik, Berlin Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)
Contents
1â•…Introduction ����������������������������������尓������������������������������������尓������������������������� ╇╅ Fabio Paternò 1.1â•… Overall View����������������������������������尓������������������������������������尓����������������� ╇╅ 1.2â•… Motivations����������������������������������尓������������������������������������尓������������������� ╇╅ 1.3â•… Objectives����������������������������������尓������������������������������������尓��������������������� ╇╅ 1.4â•… Technical and Architectural Aspects����������������������������������尓����������������� ╇╅ 1.5â•… Structure of the Book����������������������������������尓������������������������������������尓������ â•…
1 1 2 3 5 6
2â•…State of the Art in Migration ����������������������������������尓���������������������������������� ╇╅ 9 Fabio Paternò, Carmen Santoro and Rasmus Olsen 2.1â•… Introduction to Migration Frameworks����������������������������������尓������������� ╇╅ 9 2.2â•… Support for Application Migration����������������������������������尓�������������������� â•… 11 2.2.1â•… Middleware Support for Migration����������������������������������尓������� â•… 11 2.2.2â•… Network Mobility Support for Migration������������������������������� â•… 12 2.2.3â•… Context Management Support for Migration������������������������� â•… 14 2.3â•… Migratory Services for Games����������������������������������尓�������������������������� â•… 14 2.4â•… Advances Over the State of the Art����������������������������������尓������������������� â•… 15 2.4.1â•… Advances in Migratory User Interfaces����������������������������������尓 â•… 15 2.4.2â•… Advances in the Migration of the Application Logic�������������� â•… 18 2.4.3â•… Summary of Main Advances Over the State of Art���������������� â•… 19 References����������������������������������尓������������������������������������尓������������������������������ â•… 20 3â•…Migration Opportunities ����������������������������������尓������������������������������������尓���� â•… Agnese Grasselli, Alessandro Vangelista and Stefano Bolli 3.1â•… Setting the Scene����������������������������������尓������������������������������������尓����������� â•… 3.2â•… Multiscreen Ambition����������������������������������尓������������������������������������尓��� â•… 3.3â•… Migration Platform Value Chain����������������������������������尓����������������������� â•… References����������������������������������尓������������������������������������尓������������������������������ â•…
25 25 27 29 30
4╅The OPEN Migration Platform Architecture ����������������������������������尓������� ╅ 31 Miquel Martin 4.1╅ The Concept of Migration����������������������������������尓��������������������������������� ╅ 31 4.2╅ The Advantages of the OPEN Approach����������������������������������尓���������� ╅ 32 v
vi
Contents
4.3â•… Architectural Overview of the OPEN Platform����������������������������������尓 â•… 4.4â•… Making Applications OPEN-Aware����������������������������������尓������������������ â•… 4.4.1â•… The OPEN Adaptors����������������������������������尓����������������������������� â•… 4.4.2â•… Client and Server Side Applications����������������������������������尓����� â•… 4.4.3â•… Partial Migration����������������������������������尓����������������������������������� â•… 4.5â•… Platform Communication: The OPEN Dispatchers���������������������������� â•… 4.5.1â•… Communication Models����������������������������������尓������������������������ â•… 4.6â•… OPEN Platform Architecture����������������������������������尓���������������������������� â•… 4.7â•… OPEN Interfaces����������������������������������尓������������������������������������尓����������� â•… 4.7.1â•… Interface Design Philosophy����������������������������������尓����������������� â•… 4.7.2â•… Ensuring Data Consistency����������������������������������尓������������������� â•… 4.8â•… Conclusions����������������������������������尓������������������������������������尓������������������� â•… 5â•…User Interface Migration Based on the Use of Logical Descriptions ����������������������������������尓������������������������������������尓������� â•… Giuseppe Ghiani, Fabio Paternò and Carmen Santoro 5.1â•… Introduction����������������������������������尓������������������������������������尓������������������� â•… 5.2â•… Architecture����������������������������������尓������������������������������������尓������������������� â•… 5.2.1â•… OPEN Platform Integrated Orchestration������������������������������� â•… 5.2.2â•… Stand-Alone Web Migration Orchestration���������������������������� â•… 5.3â•… An Application Example of Total Web Migration������������������������������ â•… 5.4â•… An Application Example of Partial Web Migration���������������������������� â•… 5.5â•… Usability Evaluation����������������������������������尓������������������������������������尓����� â•… 5.6â•… Technical Migration Evaluation����������������������������������尓������������������������ â•… 5.7â•… Considerations and Open Issues����������������������������������尓����������������������� â•… 5.8â•… Conclusions����������������������������������尓������������������������������������尓������������������� â•… References����������������������������������尓������������������������������������尓������������������������������ â•… 6â•…Service Migration Network Support ����������������������������������尓��������������������� â•… Rasmus Olsen, Kim Højgaard-Hansen, Anders Nickelsen, Huan Cong Nguyen, Miquel Martin, Carmen Santoro, Björn Schindler and Simone Mazzei 6.1â•… Network and Deployment Scenarios����������������������������������尓���������������� â•… 6.2â•… Network Domain and Entities����������������������������������尓��������������������������� â•… 6.3â•… Deployment Scenarios����������������������������������尓������������������������������������尓�� â•… 6.4â•… Overview of the Network Support����������������������������������尓�������������������� â•… 6.5â•… Migration Orchestration and Orchestration Procedure����������������������� â•… 6.6â•… Context Management����������������������������������尓������������������������������������尓���� â•… 6.7â•… Internal Structure, Architecture and Interaction��������������������������������� â•… 6.7.1â•… Internal Structure and Architecture����������������������������������尓������� â•… 6.7.2â•… Interaction with the Context Management Framework���������� â•… 6.7.3â•… Adapting the Context Management Framework��������������������� â•… 6.8â•… Trigger Detection and Management����������������������������������尓����������������� â•… 6.8.1â•… Manual Migration Triggering����������������������������������尓��������������� â•… 6.8.2â•… Automatic Migration Triggering����������������������������������尓����������� â•…
33 35 35 36 38 38 40 40 41 43 43 44 45 45 47 48 49 49 50 54 56 57 58 58 61
61 61 62 63 64 66 67 67 68 69 72 73 74
Contents
6.8.3╅ Score-Based Trigger Decision Approach������������������������������ ╇╅ 6.8.4╅ Model-Based Trigger Decision Approach���������������������������� ╇╅ 6.9╅ Mobility Support����������������������������������尓������������������������������������尓��������� ╇╅ 6.9.1╅ Requirements for Mobility Support Module������������������������ ╇╅ 6.9.2╅ Terminal Mobility����������������������������������尓������������������������������� ╇╅ 6.9.3╅ Session Mobility����������������������������������尓��������������������������������� ╇╅ 6.9.4╅ Architecture of Mobility Support Module���������������������������� ╇╅ References����������������������������������尓������������������������������������尓���������������������������� ╇╅
vii
74 78 81 82 82 84 92 93
7â•…Dynamic Reconfiguration of Application Logic During Application Migration ����������������������������������尓������������������������������������尓������ ╇╅ 95 Holger Klus, Björn Schindler and Andreas Rausch 7.1â•… Introduction����������������������������������尓������������������������������������尓����������������� ╇╅ 95 7.2â•… The Application Logic Reconfiguration Module������������������������������ ╇╅ 96 7.3â•… ALR Application Components����������������������������������尓������������������������ ╇╅ 97 7.4â•… Application Logic Specification and Reconfiguration���������������������� ╇╅ 99 7.5â•… Related Work����������������������������������尓������������������������������������尓�������������� â•… 104 7.6â•… Conclusions and Future Work����������������������������������尓������������������������� â•… 106 References����������������������������������尓������������������������������������尓���������������������������� â•… 106 8â•…Design and Development of a Migratory Application Based on OPEN Migration Service Platform ����������������������������������尓����� â•… Giancarlo Cherchi and Francesca Mureddu 8.1â•… Introduction����������������������������������尓������������������������������������尓����������������� â•… 8.2â•… Aspects of a Migratory Application����������������������������������尓���������������� â•… 8.3â•… Guidelines for Making an Application OPEN-Compliant���������������� â•… 8.3.1â•… Application Logic����������������������������������尓������������������������������� â•… 8.3.2â•… User Interface����������������������������������尓������������������������������������尓�� â•… 8.3.3â•… Network����������������������������������尓������������������������������������尓���������� â•… 8.3.4â•… Context����������������������������������尓������������������������������������尓������������ â•… 8.3.5â•… Policy����������������������������������尓������������������������������������尓�������������� â•… 8.4â•… Perception and Awareness of the Migration Process������������������������ â•… 8.5â•… An Example of a Migratory Application: The Social Game������������� â•… 8.5.1â•… Scenario����������������������������������尓������������������������������������尓���������� â•… 8.5.2â•… Description����������������������������������尓������������������������������������尓������ â•… 8.5.3â•… Aspects����������������������������������尓������������������������������������尓������������ â•… 8.5.4â•… Architecture����������������������������������尓������������������������������������尓����� â•… 8.5.5â•… Examples of Migration����������������������������������尓����������������������� â•… 8.6â•… Conclusions����������������������������������尓������������������������������������尓����������������� â•…
109 109 109 111 113 115 116 117 118 119 120 121 125 126 130 132 135
9â•…Next-Generation Migratory Emergency Management Application ��� â•… 137 Kay-Uwe Schmidt, Veselina Milanova, Jörg Dörflinger and Susan Marie Thomas 9.1â•… Introduction����������������������������������尓������������������������������������尓����������������� â•… 137 9.2â•… Motivating Example����������������������������������尓������������������������������������尓��� â•… 138
viii
Contents
9.3â•… â•›Requirements����������������������������������尓������������������������������������尓����������� â•… 9.4â•… â•›Agile User Interfaces����������������������������������尓����������������������������������� â•… 9.5â•… â•›Agile User Interfaces Implemented����������������������������������尓������������� â•… 9.6â•… â•›Agile User Interfaces Evaluated����������������������������������尓������������������ â•… 9.7â•… â•›Related Work����������������������������������尓������������������������������������尓����������� â•… 9.8â•… â•›Conclusions and Future Work����������������������������������尓���������������������� â•… References����������������������������������尓������������������������������������尓�������������������������� â•… 10â•…Integration of User Interface Migration and Application Logic Reconfiguration: An Example in the Game Domain �������������� â•… Giuseppe Ghiani, Holger Klus, Fabio Paternò, Carmen Santoro and Björn Schindler 10.1â•… Introduction����������������������������������尓������������������������������������尓������������� â•… 10.2â•… Description of the PacMan Game����������������������������������尓���������������� â•… 10.3â•… Migration and the Main Architecture of the PacMan Game��������� â•… 10.4â•… Application Logic Reconfiguration����������������������������������尓������������� â•… 10.5â•… User Interface Migration����������������������������������尓����������������������������� â•… 10.6â•… State Persistence����������������������������������尓������������������������������������尓������ â•… 10.7â•…Integration of the User Interface Migration and Application Logic Reconfiguration����������������������������������尓������ â•… 10.8â•… Advantages of the OPEN Migration Platform������������������������������� â•… 10.9â•… Conclusions����������������������������������尓������������������������������������尓������������� â•… References����������������������������������尓������������������������������������尓�������������������������� â•… 11â•…The Usability Evaluation and the Programmability Assessment of Migration ����������������������������������尓������������������������������������尓� â•… Agnese Grasselli, Alessandro Vangelista and Stefano Bolli 11.1â•… What Does Testing a Migratory Middleware Platform Mean?������ â•… 11.2â•… Usability����������������������������������尓������������������������������������尓������������������� â•… 11.2.1â•… The ISO Definition of Usability����������������������������������尓����� â•… 11.3â•… Programmability����������������������������������尓������������������������������������尓������ â•… 11.3.1â•… Definition����������������������������������尓������������������������������������尓���� â•… 11.3.2â•… Programmability Assessment����������������������������������尓��������� â•… 11.3.3â•… Programmability Validation����������������������������������尓����������� â•… 11.4â•… Conclusion About Testing Activity����������������������������������尓�������������� â•… References����������������������������������尓������������������������������������尓�������������������������� â•…
139 139 141 142 146 147 147 149 149 150 150 151 154 157 159 160 161 161 163 163 164 164 169 170 171 173 174 174
Appendix ����������������������������������尓������������������������������������尓���������������������������������� ╅ 177 Appendix A: System Usability Scale����������������������������������尓��������������������� ╅ 177 Appendix B: Product Reaction Cards����������������������������������尓�������������������� ╅ 178 Index ����������������������������������尓������������������������������������尓������������������������������������尓����� ╅ 179
Contributors
Stefano Bolli╇ Sytel Reply, Milan, Italy Giancarlo Cherchi╇ Arcadia Design, Sestu, Italy e-mail:
[email protected] Jörg Dörflinger╇ SAP Research, Karlsruhe, Germany Giuseppe Ghiani╇ CNR-ISTI, HIIS Laboratory, Via G. Moruzzi 1, 56124 Pisa, Italy Agnese Grasselli╇ Vodafone Omnitel NV, Milan, Italy e-mail:
[email protected] Kim Højgaard-Hansen╇ Aalborg University, Aalborg, Denmark e-mail:
[email protected] Holger Klus╇ University of Clausthal, Clausthal, Germany e-mail:
[email protected] Miquel Martin╇ NEC Europe Ltd., Heidelberg, Germany e-mail:
[email protected] Simone Mazzei╇ Vodafone Omnitel NV, Milan, Italy e-mail:
[email protected] Veselina Milanova╇ SAP Research, Karlsruhe, Germany Francesca Mureddu╇ Arcadia Design, Sestu, Italy Huan Cong Nguyen╇ Aalborg University, Aalborg, Denmark e-mail:
[email protected] Anders Nickelsen╇ Aalborg University, Aalborg, Denmark e-mail:
[email protected] Rasmus Olsen╇ Aalborg University, Aalborg, Denmark e-mail:
[email protected] Fabio Paternò╇ CNR-ISTI, HIIS Laboratory, Via G. Moruzzi 1, 56124 Pisa, Italy e-mail:
[email protected] ix
x
Contributors
Andreas Rausch╇ Clausthal University of Technology, Clausthal-Zellerfeld, Germany Carmen Santoro╇ CNR-ISTI, HIIS Laboratory, Via G. Moruzzi 1, 56124 Pisa, Italy e-mail:
[email protected] Björn Schindler╇ Clausthal University of Technology, Clausthal-Zellerfeld, Germany e-mail:
[email protected] Kay-Uwe Schmidt╇ SAP Research, Karlsruhe, Germany Susan Marie Thomas╇ SAP Research, Karlsruhe, Germany Alessandro Vangelista╇ Vodafone Omnitel NV, Milan, Italy e-mail:
[email protected] List of Figures
Fig.€1.1↜ Example of multi-device environment mediated by the migration infrastructure����������������������������������尓������������������������������������尓�������������� ╅╇ 2 Fig.€3.1↜ Service integration across device types ����������������������������������尓�������� â•… 28 Fig.€4.1↜ Overall architecture, where applications are seen as clients of the migration platform ����������������������������������尓������������������������������ â•… 34 Fig.€4.2↜ OPEN servers may be specific to a location, and clients might be handed over between them ����������������������������������尓������������ â•… 35 Fig.€4.3↜ Applications implement or use the OPEN adaptors, and in doing so, present an OPEN client interface to the OPEN platform ���������� â•… 36 Fig.€4.4↜ Applications look like a single OPEN client to the platform, even when they have a server and a client part ������������������������������ â•… 37 Fig.€4.5↜ Different applications can choose to adapt differently to the OPEN platform to achieve migration support �������������������������� â•… 37 Fig.€4.6↜ Dispatchers in the OPEN server and client act as a communication hub ����������������������������������尓������������������������������������尓�� â•… 39 Fig.€4.7↜ OPEN platform architecture. Note that the mobility anchor point is left out for simplicity. (see Chap.€6 on mobility support for details) ����������������������������������尓������������������������������������尓���������������� â•… 41 Fig.€4.8↜ The interfaces offered to a component depend only on the direction the component is “facing” ����������������������������������尓������ â•… 42 Fig.€5.1↜ The desktop page of the airline company, considered in the total migration example ����������������������������������尓�������������������������������� â•… 50 Fig.€5.2↜ The pages generated on the mobile device after total migration ���� â•… 51 Fig.€5.3↜ The social game web application in the desktop device ���������������� â•… 52 Fig.€5.4↜ The selection of the chat and the betting area components ������������ â•… 53 Fig.€5.5↜ Two migrated components displayed in the target device (iPhone) ����������������������������������尓������������������������������������尓�������������������� â•… 53 Fig.€5.6↜ Efficiency results ����������������������������������尓������������������������������������尓������ â•… 56 Fig.€5.7↜ Satisfaction results ����������������������������������尓������������������������������������尓���� â•… 56 Fig.€6.1↜ OPEN network domain and major entities considered ������������������ â•… 62 Fig.€6.2↜ Overview of modules in the various OPEN network entities �������� â•… 63
xi
xii
List of Figures
Fig.€6.3↜ Module overview of key blocks for supporting migratory services ����������������������������������尓������������������������������������尓��������������������� â•… Fig.€6.4↜ Generic overview of migration process ����������������������������������尓�������� â•… Fig.€6.5↜ Example of configuration of a system. A change in configuration means a migration executed by the OPEN server �������������������������� â•… Fig.€6.6↜ Overview of internal components of a context agent, which are running on each node within the OPEN network domain �������������� â•… Fig.€6.7↜ Internal interaction between various components inside the context agent ����������������������������������尓������������������������������������尓������������ â•… Fig.€6.8↜ Query and subscription distribution among the context agents within the OPEN network domain ����������������������������������尓���������������� â•… Fig.€6.9↜ Outline of a support system for automatic configuration of context management framework with remote repositories of retrievers and processing units ����������������������������������尓���������������� â•… Fig.€6.10↜ Overview of external modules necessary for trigger management to carry out its task ����������������������������������尓������������������������������������尓���� â•… Fig.€6.11↜ Overview of internal modules in trigger management and how they interact with the context management framework to collect network context information and ALR to have configuration options and component requirements ����������������������������������尓���������� â•… Fig.€6.12↜ Examples of utility functions mapping system parameters to user perceived quality (MOS). Packet loss rate in the network or delay are examples of parameters ����������������������������������尓������������ â•… Fig.€6.13↜ Example utility functions for voice. Left is HQ stream (From Schwefel et€al. 2007), Right is LQ stream (created from left) ����������������������������������尓������������������������������������尓���� â•… Fig.€6.14↜ Example MOS utility from video quality under different network conditions (high lossâ•›=â•›25% packet loss, few lossâ•›=â•›5% packet loss) (From Klaue et€al. 2003) ����������������������������������尓������������������������������ â•… Fig.€6.15↜ Example application and scenarios modeled using a MDP ������������ â•… Fig.€6.16↜ Decision policies as given by solving the MDP, for different values of p ����������������������������������尓������������������������������������尓���������������� â•… Fig.€6.17↜ OPEN—aware home agent and its virtual subnet �������������������������� â•… Fig.€6.18↜ SOCKS-based session mobility. When a client device initiates a network connection it goes through the MAP and if a migration occurs the connection from the MAP to the AS is re-routed to the target device ����������������������������������尓������������������������������������尓�������� â•… Fig.€6.19↜ Third party control example using SIP ����������������������������������尓�������� â•… Fig.€6.20↜ Transferring SIP sessions using the REFER method ���������������������� â•… Fig.€6.21↜ A common SIP setup ����������������������������������尓������������������������������������尓 â•… Fig.€6.22↜ The elements for mobility support in OPEN ����������������������������������尓 â•… Fig.€6.23↜ Mobility support’s interaction with other OPEN components ������� â•… Fig.€7.1↜ ALR module within the OPEN migration platform architecture ���� â•… Fig.€7.2↜ Depiction of a running component within the application logic ���� â•… Fig.€7.3↜ Two example configurations of an application ���������������������������� ╇╅
64 65 66 68 69 70 71 72
73 75 76 77 79 80 86
88 89 90 91 92 93 96 97 98
List of Figures
xiii
Fig.€7.4↜ An example for a component descriptor ����������������������������������尓���� ╇╅ 98 Fig.€7.5↜ Graphical representation of a single Template . . . . . . . . . . . . . . . ╅ 100 Fig.€7.6↜ Examples of compliant and non-compliant components for Template T1 of Fig.€7.5 ����������������������������������尓������������������������������ ╅ 100 Fig.€7.7↜ An example of a component network specification with two Templates and an association between them �������������������������������� ╅ 101 Fig.€7.8↜ Example for constraints at Templates and associations between them ����������������������������������尓������������������������������������尓���������� ╅ 101 Fig.€7.9↜ Application logic specification with two configuration specifications ����������������������������������尓������������������������������������尓���������� ╅ 103 Fig.€7.10↜ Example for an XML-based application specification used in the prototypical implementation of ALR ����������������������������������尓���� ╅ 103 Fig.€8.1↜ Aspects of a migratory application ����������������������������������尓������������ ╅ 110 Fig.€8.2↜ General client-server architecture configuration �������������������������� ╅ 112 Fig.€8.3↜ Variation of client-server architecture configuration �������������������� ╅ 112 Fig.€8.4↜ Generic client-server architecture configuration with open dispatchers ����������������������������������尓������������������������������������尓������ ╅ 113 Fig.€8.5↜ Client-server architecture for mobility support ���������������������������� ╅ 117 Fig.€8.6↜ Scenario (part 1) ����������������������������������尓������������������������������������尓����� ╅ 122 Fig.€8.7↜ Scenario (part 2) ����������������������������������尓������������������������������������尓����� ╅ 123 Fig.€8.8↜ Scenario (part 3) ����������������������������������尓������������������������������������尓����� ╅ 124 Fig.€8.9↜ Screenshot of the social game ����������������������������������尓�������������������� ╅ 125 Fig.€8.10↜ Components selection menu in the social game �������������������������� ╅ 128 Fig.€8.11↜ The betting component is shown in gray in the interface when paused ����������������������������������尓������������������������������������尓������������ ╅ 129 Fig.€8.12↜ Chat and betting components terminated ����������������������������������尓��� ╅ 130 Fig.€8.13↜ Client-server architecture configuration of the social game �������� ╅ 131 Fig.€8.14↜ Details of social game client side architecture ������������������������������ ╅ 132 Fig.€8.15↜ Example of betting migration from a PC to a laptop �������������������� ╅ 133 Fig.€8.16↜ Example of partial migration from PC to PDA ���������������������������� ╅ 133 Fig.€8.17↜ Example of migration of IPTV from a laptop to a big screen ������ ╅ 134 Fig.€8.18↜ Example of migration of controls for sharing the social game interface ����������������������������������尓������������������������������������尓�������� ╅ 135 Fig.€9.1↜ High level architecture ����������������������������������尓�������������������������������� ╅ 140 Fig.€9.2↜ Client architecture ����������������������������������尓������������������������������������尓�� ╅ 141 Fig.€9.3↜ Application with activated flood component �������������������������������� ╅ 142 Fig.€9.4↜ Merged traffic and flood components after migration ������������������ ╅ 143 Fig.€9.5↜ The SUS questionnaire ����������������������������������尓������������������������������ ╅ 144 Fig.€9.6↜ Execution time for migration ����������������������������������尓���������������������� ╅ 145 Fig.€9.7↜ Distribution of SUS ratings among the 16 test participants ��������� ╅ 145 Fig.€10.1↜ The PacMan game considered in the example ����������������������������� ╅ 150 Fig.€10.2↜ Migration of the PacMan game ����������������������������������尓������������������ ╅ 151 Fig.€10.3↜ Application structure of the PacMan game ����������������������������������尓 ╅ 152 Fig.€10.4↜ The OPEN migration platform and the application logic ������������ ╅ 152 Fig.€10.5↜ The application specification of the PacMan game ���������������������� ╅ 153
xiv
List of Figures
Fig.€10.6↜ Component specification and application specification �������������� ╅ Fig.€10.7↜ The original desktop version of the PacMan �������������������������������� ╅ Fig.€10.8↜ The adapted iPhone version ����������������������������������尓������������������������ ╅ Fig.€11.1↜ Effectiveness analysis, data representation ����������������������������������尓 ╅ Fig.€11.2↜ Efficiency analysis, user distribution representation �������������������� ╅ Fig.€11.3↜ Efficiency analysis, user distribution representation with box plot ����������������������������������尓������������������������������������尓���������� ╅ Fig.€11.4↜ Evaluation of user satisfaction using SUS approach distribution ����������������������������������尓������������������������������������尓�������������� ╅ Fig.€11.5↜ Evaluation of user satisfaction using product reaction cards approach ����������������������������������尓������������������������������������尓�������� ╅ Fig.€11.6↜ Variables and rules adding ����������������������������������尓�������������������������� ╅ Fig.€11.7 ↜Evaluation chart for programmability test ����������������������������������尓�� ╅
154 155 157 166 166 167 169 170 171 174
List of Tables
Table€6.1↜渀ȀȕDefinition of OPEN entities and their role in the migration process............................................................................ â•… 62 Table€6.2↜渀ȀȕExample utility function for screen resolution in a given video application....................................................................................... â•… 76 Table€6.3↜渀Ȁ P(s, aâ•›=â•›‘nothing’, s′)........................................................................ â•… 79 Table€6.4↜渀 ╇ P(s, aâ•›=â•›‘migrate’, s′)........................................................................ â•… 79 Table€6.5↜渀ȀȕRewards R(s, a) for the example application calculated from the utility functions.......................................................................... â•… 80 Table€6.6↜渀Ȁ Comparison between MIP, mSCTP and SIP................................... â•… 84 Table€9.1↜╇╇ Task lists for both test participants.................................................. ╇ 144 Table€11.1↜渀 Classifier based on system usability score...................................... ╇ 168
xv
Chapter 1
Introduction Fabio Paternò
1.1â•…Overall View One important aspect of ubiquitous environments is to provide users with the possibility to freely move about and continue the interaction with the available applications through a variety of interactive devices (including cell phones, PDAs, desktop computers, digital television sets, and intelligent watches). Indeed, in such environments one big potential source of frustration is that people have to start their session over again from the beginning at each interaction device change. Migratory interactive services can overcome this limitation and support continuous task performances. This implies that interactive applications be able to follow users and adapt to the changing context of use while preserving their state. This book reports results based on the work carried out in the OPEN project (http://www.ict-open.eu/), which provides integrated solutions able to address three aspects: device change, state persistence and content adaptation. This is obtained through a middleware able to consider and integrate various aspects: adapt and preserve the state of the software application parts dedicated to interacting with end users; support mechanisms for application logic reconfiguration; and define suitably flexible mechanisms from the underlying network layers. The resulting middleware is able to interoperate with existing technologies. Thus, OPEN aims to offer an intelligent infrastructure able to: deliver seamless and transparent support to users in carrying out their tasks when changing available devices, even in multi-user interactive applications; provide and coordinate more reliable and dynamically changing/reconfiguring services; offer personalized user interaction by exploiting different interaction modalities and network technology by means of an infrastructure providing the necessary context information regarding the available devices, connectivity, users and related transformations for content adaptation. The OPEN project has defined middleware solutions developed in the context of example applications from various different domains (business applications and F. Paternò () CNR-ISTI, HIIS Laboratory, Via G. Moruzzi 1, 56124 Pisa, Italy e-mail:
[email protected] F. Paternò (ed.), Migratory Interactive Applications for Ubiquitous Environments, Human-Computer Interaction Series, DOI 10.1007/978-0-85729-250-6_1, ©Â€Springer-Verlag London Limited 2011
1
2
F. Paternò
gaming), to demonstrate the feasibility of the approach, the limited effort required to application developers, and its ability to enable new application services. There are many applications that can benefit from migratory interactive services. In general, applications that require time to be completed (such as games, business applications) or applications that have some rigid deadline and thus need to be completed wherever the user is (e.g.: online auctions). Other applications that can benefit from this flexible reconfiguration support are those that have to provide users with continuous support during the whole day through different devices (for example, in the assisted living domain).
1.2â•…Motivations Despite the multitude of different types of terminals available in the market, our lives have not yet become a multi-device experience since one source of constant frustration is that people cannot continue to perform their tasks when they move about and change their interaction device. This is due to the lack of migratory services
Fig. 1.1↜渀 Example of multi-device environment mediated by the migration infrastructure
1â•… Introduction
3
technology for the migration of applications in different usage scenarios. This book promises to fill this gap by providing a general and open migratory service platform solution based on a sound and innovative scientific approach developed by a multidisciplinary consortium combining the expertise of three technological world leaders (SAP, NEC, Vodafone), three well known research organizations (CNR-ISTI, University of Aalborg, and Clausthal University) and one SME (Arcadia). One important aspect of pervasive environments is to provide users with the ability to freely move about and continue the interaction with the applications in use through a variety of interactive devices with different interaction resources (e.g. cell phones, PDAs, desktop computers, digital television sets, intelligent watches, an example is in Fig.€1.1) and communication channels with different characteristics and performance (i.e. WiFi, Bluetooth, sensor networks, UMTS, …). In this regard, there has been a recent increase in interest in migratory interactive services. They provide users with the ability to change the interaction device and still continue their tasks through an interface adapted to the new platform. Migration can involve various types of devices. In some cases, migration can be used to improve the user’s experience by switching to a better suited devices (bigger screen, more graphical power, …) or to a more efficient communication channel/a communication channel that can guarantee better QoS (shorter delays, higher bandwidth).
1.3â•…Objectives In order to address the complex issues related to migration there is a need for a service platform able to consider and integrate various aspects: adapt and preserve the state of the software application parts dedicated to interacting with end users; support mechanisms for application logic reconfiguration; and define suitably flexible mechanisms from the underlying network layers. The resulting service platform should be able to interoperate with existing technologies. In general, a service is a computer-based entity that provides well-defined functionality, together with the policies that should control their usage. We also address services whose main goal is to support users, and thus include both the user interface software and the internal application logics. Services can communicate with each other. This activity might consist of bare data passing or more coordinated activities. Therefore, services have to be supported by an underlying infrastructure that allows them to coordinate with each other. Migratory interactive applications require that the user interface adapts itself to the resources and the existing services of the new device/environment; this can also imply dynamic (re)configuration of the overall application, which in turn involves establishing the required/requested connections between the available components/ services, which are user interface components, as well as non-user interface components providing application services. Migratory Interactive Services also imply an intelligent context-aware infrastructure able to capture the state of the user interface and application logic on the
4
F. Paternò
source device, transmit it to the target device (transformed if necessary) and then generate an appropriate user interface to the target device as well. The devices used for interaction can support various modalities (graphical, vocal, gesture, …), which can even be exploited in various combinations depending on the context of use. We also address partial/distributing migration, which is the ability to move from interacting with an application through a single device, to interaction through several coordinated devices using various modalities. This allows users to comfortably control, for example, videos displayed on a wall-sized screen through a vocal interface, or project a presentation stored in a personal device such as a PDA to a desktop-controlled maxi video screen in a conference hall while annotating the output through an intelligent whiteboard and maintaining control on the personal device. Migration of an interactive service to a set of devices detected and classified on the basis of the tasks supported by their resources can require complex processing. Migration can result in new capabilities for the whole pervasive application. Therefore, the application logic may also adapt by dynamically composing or decomposing the changing set of available application services resulting in dynamic adaptation of the applications in use at the functional level. This implies establishing the required/requested connections between the available components/services, which are user interface components as well as non-user interface components providing application services. Moreover, services that are no longer required in the newly established application have to be detached from the system. This attaching/ detaching process at runtime is called dynamic reconfiguration. Such dynamic reconfiguration requires a description of the services offered in such a way as to enable the integration of new components at runtime, even if they were not anticipated at application development time. Often the services required and offered by the two components are not identical, though they would be compatible. Therefore, there is a need for assessing the match between required and offered services in order to enable dynamic reconfiguration of the application. In some cases there may be multiple possibilities regarding one requested service. The choice can also be based on Quality of Service properties such as the available bandwidth or the delay that user is willing to wait. This adaptation requires a sufficient description of the QoS properties at the application layer in order to enable the infrastructure to reason about it during the run-time re-configuration. Distributed applications, e.g. client-server based, involve remote communications between the components. In this case, the mechanisms for migratory services also need to be closely inter-linked with adequate functionalities to provide seamless connectivity and service to the remote components. This can be achieved by either making the application mobility aware (e.g. the application server will recognize that the ‘new’ client IP address is supposed to replace the previous one in the ongoing session), or by using adequate mobility support mechanisms, either transparently at the network layer through a Mobile Internet Protocol (MIP), or at intermediate layers such as transport layer, or Session Initiation Protocol (SIP) layer, when present. Handover triggers require close coupling to the user interface adaptation in several cases, e.g., when the quality of the network connection changes substantially
1â•… Introduction
5
(e.g. video is no longer possible due to degrading throughput) or the network connection has changed from one access technology to another (e.g. from UMTS to WLAN). We will focus on scenarios associated with interactive migratory services because of their inherent complexity and potential impact on future services. Thus, the main objectives of this book are to describe solutions able to: • Offer seamless and transparent support to users in carrying out their tasks when changing devices as well as changing available services; • Offer more natural and personalized interaction obtained by exploiting different interaction techniques supporting the mobile user; • Exploit the wide availability of network technology to offer more reliable services in the context of migration with dynamically changing devices and services; • Propose a novel infrastructure in order to increase possible services and application scenarios in several contexts (services for citizen, business, games, new interactive and collaborative method in work or educational applications, and so on).
1.4â•…Technical and Architectural Aspects The migration concept can be implemented in various ways and exploiting various technologies. In a few words migration means adaptation to the context of use while preserving the state of the user session. In this section we introduce the possible types of user interface migrations, the possible underlying networking scenarios, and then some architectural aspects. Migration of an interactive application does not necessarily involve moving the entire user interface from one device to another one. A number of user interface migration types can be identified: • Total migration basically allows the user to change from one device to another one to interact with the application. In this case, the system is in charge of ensuring interaction continuity and supporting user interface adaptation to the different platforms. • Partial migration is the ability to migrate to the destination device only a portion of the interactive application, while the remaining portion remains in the source device. • In the distributing migration, the interactive application is totally distributed over two or more devices after migration. This is different from distributed user interfaces, for which the user interfaces are originally generated as distributed among various interaction resources connected to the same device. • The aggregating migration performs the inverse process: the interactive application of multiple source devices are grouped in the user interface of a single target device. • The multiple migration occurs when both the source and the target of the migration process are multiple devices.
6
F. Paternò
The Networking scenarios in which these migratory services are provided influence the possible solutions. Basic cases are listed in the following: • Cellular-type, infrastructure based networks, in which network controlled (or network supported) migration is possible. • Full ad-hoc/wireless multi-hop networks, e.g. resulting from direct Personal-Area-Network (PAN) to PAN communication. Migration solutions in this scenario cannot rely on the permanent presence of proxy servers, for example, and the dynamics in the network topologies as created by the device mobility will require distributed and robust solutions. • Mixed ad-hoc/infrastructure scenarios, such as for instance, created by Personal Networks or by cellular networks with ad-hoc coverage extensions. This could be the case in home automation application scenarios. This type of issues can be addressed only through a multi-layer architecture composed of different infrastructure layers. At the lowest level there is the network infrastructure, a connectivity software in charge of managing issues raised by the coexistence of heterogeneous networks, and therefore in charge of allowing them to communicate with each other while ensuring high levels of robustness, reliability and adequate performance, even by opportunistically re-configuring themselves. On a higher level there is the service infrastructure, which is essential for migrating services since it manages the dynamic discovery, provision, distribution, combination and reconfiguration of services required, state continuity, and generates usable interfaces for various input/output devices, so as to address the needs of individual users. A large part of the application industry will benefit from migratory services. On the supply side, today applications are designed with the platform/terminal where they will be used in mind. Allowing users to utilize such applications on a different platform/terminal is in general a very costly process. Migratory services will instead enable easy application portability. On the demand side, many applications involve tasks and services that are completed over a certain period of time (such as games, making reservations, online news, etc.) so users may change their interaction device in the meantime, or are constrained by rigid deadlines (such as business applications, online auctions, financial applications etc.) for which they must be accessed wherever the user is. Migratory services will enable a continuous and an unconstrained user experience for these classes of applications.
1.5╅Structure of the Book The book is structured into a number of chapters that provide a comprehensive and coherent view on the topics introduced. We start with Chap.€ 2 with a description of the State of the art from various viewpoints, then Chap.€3 reports a discussion on where it is useful to migrate, and
1â•… Introduction
7
what is specific to that environment and adaptation opportunities from a Mobile Operator perspective. Next, we have the chapters dedicated to the migration platform design. An introduction to the OPEN Migration Platform (MSP) architecture and the possible solutions is provided in Chap.€4. User interface migration based on the Use of Logical Descriptions is discussed in Chap.€5 while Service Migration Network Support is considered in Chap.€6. Then we have the Application Logic Reconfiguration based on application and component descriptions in Chap.€7. In terms of the application view of migration, we have Chap.€8 on design and development of a migratory application based on MSP, and Chap.€9 on migratory services in an emergency scenario. Chapter€10 is then dedicated to showing how to integrate support for User Interface and Application Logic Migration, using the PacMan game as case study. Lastly in Chap.€11, we discuss the usability evaluation and the programmability assessment of migration, followed by an indication of potential exploitation and some conclusions.
Chapter 2
State of the Art in Migration Fabio Paternò, Carmen Santoro and Rasmus Olsen
2.1â•…Introduction to Migration Frameworks In recent years, research on interactive migratory services has started by a number of R&T labs. For instance, one of the first works related to migration was the one of Bharat and Cardelli (1995), who put forward an initial solution for migrating entire applications (but without proving support for the user interface part), and which revealed to be problematic due to different execution environments and resources available in the involved devices. Among the earliest works we can also cite the one carried out by the HCI group at Stanford University, which aimed to generate interfaces for information appliances for extensible collections of services, namely ICrafter (Interface Crafter). ICrafter is a service framework for a class of ubiquitous computing environments known as interactive workspaces (Ponnekanti et€al. 2001). The main objective of ICrafter was to allow users of interactive workspaces to flexibly interact with the services contained in the workspace. This project was nevertheless limited to create support for controlling interactive workspaces by generating user interfaces for applications obtained by dynamic composition of elementary services, and thus did not provide support for migration and consequently continuity of task performance across different devices. A number of previous European projects have considered some of the research issues addressed in this proposal, such as FP5 CAMELEON (Cameleon 2004) and CONSENSUS (Consensus 2004). In CAMELEON, issues related to the design of multi-device interfaces were considered. This project developed a good conceptual framework for addressing such issues, but it was only able to develop tools to provide support mainly at design time for multi-device, form-based user interfaces However, often there is a need for interfaces that are able to support multimodality and interactive graphics in such a way as to adapt at run-time. CONSENSUS Project F. Paternò () CNR-ISTI, HIIS Laboratory, Via G. Moruzzi 1, 56124 Pisa, Italy e-mail:
[email protected] F. Paternò (ed.), Migratory Interactive Applications for Ubiquitous Environments, Human-Computer Interaction Series, DOI 10.1007/978-0-85729-250-6_2, ©Â€Springer-Verlag London Limited 2011
9
10
F. Paternò et al.
defined a device independent mark-up language (RIML, Renderer Independent Markup Language), which aimed to preserve the intent of the author and then transform the device-independent components into device specific ones through an adaptation system while retaining the author’s control over the final result/presentation. Although the CONSENSUS project involved many companies (such as SAP, Nokia and IBM) showing the industrial relevance of these issues, and also paid a lot of effort to locate its work within international standardisation bodies, the main lack can be found in the fact that its approach was limited to design time support and little was done for automatic adaptation at run-time. Moreover, in (Bharat and Cardelli 1995), an agent-based application migration approach has been presented, in which agents carrying pieces of code and the state of the migratory application are sent from one host to another, where a server allows the agent to rebuild the migrating application. However, such an approach is not general suitable for our goals, since it is not able to adapt to several kinds of platforms, most of which can be mobile devices with potentially limited resources of power, storage and processing. In the Pebbles project, carried out at CMU University in Pittsburgh (USA), a PUC (Personal Universal Controller) environment has been developed, which supports the downloading of logical descriptions of appliances and the automatic generation of the corresponding user interfaces (Nichols et€ al. 2002). However, the application area of this approach is limited to home appliances that require rather similar interfaces. Another relevant project is Aura, whose goal was to provide an infrastructure for the mobile user that configures itself automatically (De Sousa and Garlan 2002). Thus, when a user moves to a different platform, Aura attempts to reconfigure the computing infrastructure so that the user can continue working on tasks started elsewhere. In this approach, the different context of use is supported by the selection of a similar application achieving the same goal (for example, text editing can be supported through MS Word or Emacs depending on the resources of the device at hand). We aim to identify more flexible solutions where the application is still the same, but the interactive part is able to dynamically adapt to the new device and environment exploiting a wide variety of modalities. Some research work on migratory interfaces was described in (Bandelloni and Paternò 2004): however, that work was still far from identifying general solutions, which are for example able to handle migration through a variety of interactive devices supporting different combinations of modalities in such a way to preserve accessibility and usability. The migration onto one or multiple devices detected and classified regarding their possibilities in terms of task support requires complex and distributed processing. To achieve our goals in OPEN, we have analysed various types of software architectures able to support multimodal migration (considering both client-server and peer-to-peer solutions). It is important to find a solution which is able to better support users freely moving by dynamically generating multimodal user interfaces for mobile and stationary devices. This can be achieved through intelligent support able to provide usable results taking into account the users’ characteristics, the tasks that the users aim to accomplish, and the resources of the devices available (in terms of
2â•… State of the Art in Migration
11
interaction support, network capabilities, etc.). In the user interface generation process, our system will use transformations involving device-independent languages, which enable the system to find general solutions and limit the part of the generation that is device-dependent. In general, migratory services address a complex set of issues from the system viewpoint as well (Riva et€al. 2006). They share some ideas and leverage work done on process migration (Milojicic et€al. 2000), virtual machine monitors (Sapuntzakis et€al. 2002), mobile agents (White 1997), active networks (Wetherall 1999), and dynamic reconfiguration in distributed systems (Kramer and Magee 1985). Research works that can be seen as precursors of our context-aware migration model include process migration for load balancing and service component offloading (Messer et€al. 2002).
2.2â•…Support for Application Migration 2.2.1 Middleware Support for Migration In ubiquitous computing, system support is provided by platform services, including service registration and discovery protocols as the most characteristic ones. UPnP (UPnP 1999) and Jini (Jini) are two relevant examples of such protocols. One straight forward approach to the seamless integration of heterogeneous devices, services and resources is to build specific bridges or gateways between them. However, this approach has practical limitations imposed by the complexity of integrating many potentially different bridges without the existence of a common infrastructure. Therefore, a more generic and suitable approach can be taken if a specific and unifying platform layer is placed on top of the heterogeneous devices, services and resources. In this regard, the OSGi specification (OSGi) provides a suitable and general framework (Lee et€al. 2003). Other approaches adopt Jini as an integration platform, whilst ad-hoc solutions have also been proposed, e.g. (Allard et al. 2003). OSGi is oriented towards the implementation of residential gateways. It supports the integration and runtime combination of software components, known as bundles. The OSGi framework provides bundle installation, management and monitoring functionality which are of the utmost importance for both service providers and end users alike, and enables run time reconfiguration of components and their interaction useful for application reconfiguration. Typically a set-top box is used to host an OSGi framework runtime, and this residential gateway is in charge of integrating all the different devices, services and resources of the whole system, discovering components, checking dependencies, launching bundles on demand, and performing other related tasks. Common devices—a TV screen and a remote control—can be used as interfaces to the OSGi runtime, although many other options are also available through the use of different software bundles and communication media. Concerning user interfaces, there is a general trend towards the use of HTTP servers
12
F. Paternò et al.
as an evolution of the classical RPC model (Mattern and Sturm 2002). It is foreseeable that any mobile device will in the short-term be able to run an instance of a Web browser and support technologies that provide push mechanisms, such as Ajax (Garret 2005). Similar capabilities will also be present in future HDTV devices, and more devices will surely follow. Service migration is a fundamental functionality in intelligent environments and is particularly useful when paired with reliability and high availability features. Furthermore, component-based frameworks (e.g., Enterprise JavaBeans (JavaBeans) or Jini service containers) enable service download in ad-hoc networks, a key feature to provide adaptable service platforms for mobile end- user devices of any kind. Mobile devices, which accompany the user throughout their daily routines, should be integrated with the aforementioned residential gateway, so as to provide a coherent, seamless interaction with the system and perform valuable tasks for the user such as discovery of resources, interaction with applications and user monitoring and communication. Common problems many client devices share are related to their limited computational capabilities (due to size or power constrains). An approach to coping with these limitations is the one adopted by the Jini surrogate architecture, which allows a server to execute code on behalf of the device, in such a way that devices are only required to present the user interface to the application, leaving heavyweight tasks for the server. In addition, the OPEN platform for migratory services contains mechanisms for configurability and adaptability to support the seamless device integration. This means that the service platform functionality can be configured for different device classes and adapted at runtime to available device resources, i.e. based on on-the-fly installation of core services to a device recently added to the environment. Based on this adaptive service platform, service components can also be dynamically installed and migrated to support the application migration in conjunction with user interface migration. A new US commercial product, called Pluto (Pluto) shows that migration concepts start to raise industrial exploitation interest. However, Pluto seems still at a preliminary stage with respect to the migration potentialities: it is limited to redirect information streams depending on the user’s position while we think about migration also as something which is able to keep the state of the user input in the source device(s) and make it available seamlessly to the target device(s), for example in applications such as games, interactive TV, and collaborative work.
2.2.2 Network Mobility Support for Migration Network mobility support for migration includes the addressing and message routing between the devices participating in the migration as well as towards remote application end-points. Since we aim to make our migration platform transparent to the application server, regarding the support of connectivity to remote application end-points, several standard solutions exist that support such ‘transparent connectivity’ on different layers of the protocol stack. Most notably, mobile IP (mIP, version
2â•… State of the Art in Migration
13
4 (Perkins 2002) or 6 (Johnson et€al. 2004)) on the network layer, transport layer mobility e.g. (Riegel and Tuexen 2006), or mobility support via session control signaling, e.g. based on the Session Initiation Protocol (SIP) (Rosenberg et€al. 2002). Mobile IP (MIP) has been designed by the IETF as network layer protocol to hide the mobility of network nodes from the upper layers of the protocol stack. MIP does so by always showing the mobile nodes’ home address (HoA) to the higher layers instead of the actual mobile nodes’ location (the Care-of-Address, CoA). MIP has been originally designed for terminal mobility, i.e. the user’s terminal can change its access network while communicating but the actual device remains the same after a handover; MIP unfortunately does not offer means to operate a session handover (i.e. when an ongoing session is migrated from one device to another) or partial migration (at least a single media stream is migrated to a new device while part of the communication is maintained at the source device). One advantage of using MIP, though, is that only the nodes involved in the migration need to implement MIP, while the nodes that do not migrate a service but communicate with the migratory nodes do not need to implement MIP - this is particularly relevant when migratory nodes communicate with a ‘remote’ node such as a content server. At the session layer, SIP provides session initiation, modification and termination. At first, SIP was designed by the IETF (Rosenberg et€al. 2002) for multiparty video conference but was quickly used for managing IP-based multimedia sessions. 3GPP recently adopted SIP in release 5 of the UMTS specifications in order to implement a standardized platform for access control and session control, namely the IP-based Multimedia Subsystem (IMS) (TS23.228). On top of negotiating the end-to-end Quality of Service (QoS) and managing the media settings between the session endpoints, SIP can also be used for mobility. The original IETF SIP already defines the procedure for session mobility, i.e. maintaining the session settings after the session is moved to a new device. In addition to the candidate mobility support solutions mentioned above, more recently other approaches with strong architectural implications have been discussed, see e.g. the Host Identity Protocol (Moscovitz et€ al. 2007) that operates between network and transport layer. All the existing mobility support solutions require adaptation and extensions in order to utilize them efficiently in the OPEN platform for service migration: • Only individual flows may need to be re-directed in a QoS-aware manner, as only part of the application may be migrated. This re-direction may need to be transparent to the remote communication peers. • The presence of fixed mobility anchor points (Home Agent or similar) may not be assured in certain ad-hoc communication scenarios. • Mobility solutions and QoS-aware architectures often cannot be simply combined without resulting in severe inter-operability issues (see the example of mobile IP mobility support in a scenario of QoS- aware session signaling as in IMS (Renier et€al. 2006)). Furthermore, QoS information can trigger and control the OPEN migration scenarios, hence close coupling of the mobility support mechanism and the QoS-aware migration solution is required.
14
F. Paternò et al.
2.2.3 Context Management Support for Migration The Context Management support for Migration includes the management of context information that is used to trigger and support the migration procedure; this context includes user and device information, network/connectivity information, as well as service level information, namely the presence and capabilities of certain application parts on different end-systems. Then, the main tasks are to define protocols and algorithms to gather relevant context information, and to make it accessible to context-sensitive functionalities. General frameworks for context-sensitive networking have been developed in the past, e.g. the Context Toolkit (Dey and Abowd 2000) or the Ad-hoc Context Aware Network architecture (ACAN) (Khedr and Karmouch 2002). However, such frameworks have to be adapted to the OPEN migration case which in particular includes answers to the following questions: what context information is relevant for the migration scenarios, how can it be obtained/measured (including active or passive measurement procedures for communication performance), what network and application domains are relevant for the context management procedures? Adaptations and extensions to other special use-cases of context-sensitive networking are investigated in other EU projects, e.g. for the scenario of Personal Networks (MAGNET-b 2004; Ghader et€al. 2005). However, support of session and user interface migration has not been researched in these projects. One important part of the context information is the information about the service platform and application-level modules themselves. There is a set of off-the-shelf protocols and solutions that have been developed in the past, e.g. UPnP (UPnP), SLP (1999), Salutation (1999), Jini (Jini), but they all focus on service discovery, and while adaptation to delivering context according to the OPEN requirement is possible this will require major changes as to support the dynamic nature of context information.
2.3â•…Migratory Services for Games In general terms there is not much literature and technology available on the subject of migration services for games. As a matter of fact, despite the high demand in industry, trends from major technology providers seem not much in favour of portability. From one side, the console market (e.g.: Xbox, PlayStation, Wii) is a very closed market: console makers usually ask a game to be made exclusively for their specific platform. From the other hand, there is the attempt to expand games on mobile phones. However, the lack of mature technologies on mobile phones makes this job a very manual and costly one. To run a game a vast amount of service platform software is needed, the major part of it being the “game engine”. Since there are not yet “game engines” that are portable from a PC-like platform to a mobile phonelike platform, usually the porting of a game to a mobile phone simply requires to redevelop the game from scratch, possibly developing also the necessary service
2â•… State of the Art in Migration
15
platform part yourself. It is worth mentioning that for this reason, there are some specialised companies that port games, with a low level coding activity. A recent summary of migratory technology for games can be found in (Mehdi et€al. 2006), which, however, relies to some extent on a public deliverable published by FP6 project OLGA. Another FP6 integrated project, Games@Large (Games@Large) claims to support “ubiquitous game-play”. This project targets a different usage scenario than OPEN, as they state in their mission: “to research, develop and implement a new platform aimed at providing users with a richer variety of entertainment experience in familiar environments, such as their entire house, hotel room, cruise ships and Internet Cafe.” However, they do not provide any specific support for migration across different devices.
2.4â•…Advances Over the State of the Art In this section we first discuss some advances in specific areas important for the book and then we summarize the important overall advances that we plan to achieve with the results presented in the following chapters.
2.4.1 Advances in Migratory User Interfaces The increasing availability of various types of electronic interactive devices has raised interest in model-based approaches, mainly because they provide logical descriptions that can be used as a starting point for generating interfaces that adapt to the various devices at hand. In recent years, such interest has been accompanied by the use of XML-based languages, such as XIML (Puerta and Eisenstein 2002), UIML (Abrams et€al. 1999), UsiXML (Limbourg and Vanderdonckt 2004), RIML (Ziegert et€al. 2004), TERESA XML (Mori et€al. 2004), in order to represent the aforementioned logical descriptions. However, most of such approaches focus on providing device-adaptation support only in the design and authoring phase, whilst we believe that run-time support is equally relevant. The Personal Universal Controller (Nichols et€al. 2002) automatically generates user interfaces for remote control of domestic appliances. The remote controller device is a mobile device, which is able to download specifications of the functions of appliances and then generate the appropriate user interface to access them. XMLbased messages are exchanged to request a description of the appliance’s features and to signal events to and from the device. However, the process of discovering the device is far from automatic and it might compromise usability since the user needs to manually enter the device’s network address in the remote control application before any other action can be performed. Another approach addressing the problem of interacting with services by means of a variety of interactive platforms is XWeb (Olsen et€ al. 2000), whose goal is
16
F. Paternò et al.
to provide an interactive (and collaborative) client/server architecture that can be accessed from any interactive platform. Clients interact with the server by means of CHANGE requests on the server’s data tree rather than propagation of interactive events; clients are notified whenever server data are changed. Although our approach shares many concepts with this work (e.g.: the idea of platform independency, the adaptation to different devices), XWeb does not address the problem of migrating the user interfaces among different platforms. Bharat and Cardelli (Bharat and Cardelli 1995) addressed the migration of entire applications (which is problematic with limited-resource devices and different CPU architectures or operating systems) while we focus on the migration of the user interface. Newman and others (Newman et€ al. 2002) have developed the Speakeasy recombinant computing framework, which is a middleware exploiting mobile code to allow components in different devices to extend one another’s behaviour at runtime. However, they have not addressed the issue of adapting the services user interface to the interaction resources available in the device at hand, which we address through device-independent languages (and related transformations). WebSplitter (Han et€al. 2000) aims at supporting collaborative Web browsing by creating personalized partial views of the same Web page depending on the user and the device. Developers have to specify the Web content in XML and define a policy file indicating the tags content that can be presented depending on the user and the device. In addition, they have to define XSL modules in order to transform the XML content into HTML or WML. At run-time, a proxy server generates a presentation depending on user’s privilege and the access device. This approach has several drawbacks. Developers have to manage a plethora of low-level details to specify XML content, policy files, and XSL transformations, and support for continuing tasks across various devices is not provided. In ICrafter (Ponnekanti et€al. 2001), services beacon their presence by periodically sending broadcast messages. A control appliance then requests a user interface for accessing a service or an aggregation of services by sending its own description, consisting of the user interface languages supported (i.e. HTML, VoiceXML) to an entity known as the Interface Manager, which then generates the user interface and sends it back to the appliance. ICrafter also shares the multimodal approach to user interaction, but it does not support the transfer of the user interface from one platform to another, maintaining the client-side state of the interface. SUPPLE (Gajos et€al. 2005) generates adaptive user interfaces taking functional specifications of the interfaces, a device model and a user model as input. The remote solver server that acts as the user interface generator is discovered at bootstrap by the client devices, and they can thus request rendering of interfaces to it once it is discovered. However, discovery is limited to the setup stage of the system, and it does not monitor the runtime status of the system, thus loosing some of the benefits that could arise from a continuous monitoring activity. SUPPLE does not support migration of a user interface from one device to another, but only adaption among different platforms. Aura (Garlan et€al. 2002) provides support for migration but the adopted solution has a limited granularity. In Aura, for each possible service, various applications
2â•… State of the Art in Migration
17
are available and the choice of a particular application depends on the interaction resources available in the user device. Thus, for a given service, such as word processing, if a desktop PC is available then an application such as Microsoft Word can be activated, whereas in the case of a mobile device a light-weight text editor would be chosen. In Aura the available services (such as the aforementioned text editing service) are registered in a global registry that is the base for matching the service requests. However, with this solution, changing the currently used application whenever a change of device occurs can be rather disorienting for the users since they not only would change the device in use, but also the application. In this situation, the users might experience discontinuity within the interaction. A better solution would be, instead, to allow the users to use on the different devices the same application which opportunely adapts to the device in use to exploit at its best the interaction capabilities currently at hand. Luyten and Coninx (2005) present a system for supporting distribution of the user interface over a federation or group of devices. Migratability, in their words, is an essential property of an interface and marks it as being continuously redistributable. Their work shows the suitability of task model based design of user interfaces and XML-based user interface description languages for supporting user interface distribution and migration in device federations. The authors consider migration and distribution of only graphical user interfaces, while in OPEN we have a new solution supporting graphic, vocal and multimodal user interfaces migration. In addition, it is worth pointing out that some interesting possibilities might be offered by different kinds of migration. Indeed partial migration (namely, the possibility for the user to select the parts to be migrated) has interesting connections with some other topics like application customisation and even End User Programming and Development (Liebermann et€al. 2005). Indeed, if the users can select which parts of the UI to migrate, they are, to some extent enabled to “compose” and build their own customised applications. Moreover, partial migration can be related to some extent also to the issues connected with Distributed User Interfaces. In this area, a toolkit for developing distributed graphical UIs is described in (Melchior et€al. 2009). The toolkit is based on a widget distributed structure, composed of a ‘proxy’ of the widget (which remains stationary within the process that created the widget), and the ‘renderer’ of the widget (which is distributed, migratable and which the user can interact with). The toolkit is based on a peer-to-peer architecture in which a multi-purpose proxy is connected to one or more rendering engines able to render (partially or totally) a graphical user interface. In this solution the granularity of distribution is not limited to the user interface group’s level but can also extend to the widget level and even down to the different components of the widget itself. In the solution considered within the OPEN Project we have instead opted for a distribution down to the granularity of the single UI element which the user can interact with but no deeper, since we judged such fine granularity not very relevant from the user’s point of view since users are interested in moving some pieces of information sufficiently meaningful and not low-level details. In addition, this solution requires that the user interface be implemented using an extension of a specific toolkit (Tcl/Tk). Instead, we are interested in
18
F. Paternò et al.
solutions that mostly refer to standard technologies. For instance, when considering the migration of Web applications, we are interested in migrating any Web application developed with the standard Web languages (XHTML, CSS, and Javascript). Another related work was the one from Dearman and Pierce (2008), who tried to achieve a better understanding of why and how people use multiple devices in their everyday life. They found out that users employ a variety of techniques for accessing and managing information across the various devices, but at the same time they reported that there is still room for improvement, especially as far as the user experience is concerned. Indeed, participants reported managing information across their devices as the most challenging aspect of using multiple devices. Then, migratory interfaces can be an important support from this viewpoint. In OPEN, we plan to offer better user experience and help the user to use effectively their devices. A general reference model for user interfaces aiming to support migration, distribution and adaptation to the platform is proposed in (Balme et€al. 2004). This book aims to explain how to design and implement a software architecture that is able to support migration of user interfaces associated with applications hosted by different systems and among automatically discovered devices, therefore resulting in a more concrete solution to the issue of multimodal migration of user interfaces.
2.4.2 Advances in the Migration of the Application Logic 2.4.2.1â•…Adaptive Middleware Technologies There are numerous middleware solutions from the research community supporting proactive and reactive migration of application logic. These solutions are subsumed under the term adaptive middleware (Sadjadi 2003). They support not only the migration and (re-)wiring of various parts of the application but also the migration of the middleware itself. According to (Sadjadi 2003) the key paradigms applied to adaptive middleware include reflection, component-based design, aspect-oriented programming and software design patterns. OPEN uses component-based design. In fact it will use the syntactical as well as the behavioral component interface descriptions for the identification and automated wiring of components. Other frameworks (e.g. MoCA 2006) use only text-based properties for looking-up components and therefore do not provide type safety or more sophisticated semantic checks. Middleware solutions that support the migration of application can be further categorized in those that change the configuration of an application (i.e. the set of components that make up an application, the place of component execution and the established interconnections between the components) and in those that change the behavior of a component. OPEN focuses on the former in a reactive way. In other words it supports changing the configuration of an application (a) without the necessity of knowing all possible types of component services in advance and (b) without the explicit description of the migration behavior. OPEN will also support the component migration including the application state as well. In these cases components can react by using mechanisms for self-adaptation (e.g. reflection).
2â•… State of the Art in Migration
19
In (Oreizy et€ al. 1998) reactive system adaptation is also supported. The C2 (Taylor et€al. 1996) architectural style comes into play when one sees architectural connectors as well as components as first class entities. The authors have developed an infrastructure that allows adding and removing components and connectors at runtime by using a special scripting language. Moreover they support modifying associations between connectors and components. Finally, a special validator component checks whether the modifications are valid. The motivation is different than OPEN in this case. The authors aim to support runtime system maintenance and not automated migration of parts of the application. In (Trapp 2005) an approach to proactive system and component adaptation is described. A special-purpose executable modeling language enables the explicit description of the adaptation behavior of embedded systems. 2.4.2.2â•…Component Dependency Management Component dependency management refers to the management of the dependencies between service providing and service consuming components. The satisfaction of such dependencies is often subsumed under the term ‘wiring’. Wires can be compared to the C2 connectors discussed in the previous section. Dependency Injection also creates wires automatically. The main idea of Dependency Injection is that service requestors are passive. They do not look up services; they only define needed services, which are delivered by the infrastructure. This enforces separation of concerns and simplifies testing. One important feature is the migration support for application logic. In the book we show how to update the component wiring in the event of modification of the service landscape (i.e. service appearance, service unregistration, update). IoC containers (Fowler 2006) on the other hand typically configure an application during startup, without explicit support for reconfiguration at a later point in time. The latest version of Spring (2006) is an exception at this point.
2.4.3 Summary of Main Advances Over the State of Art To summarise, the book extends the state-of-the-art in presenting comprehensive support for service migration as an inherent feature of services in heterogeneous device environments. So, the main advances identified should enhance this situation in the following ways: • Ubiquitous access: Migration will enable access to any service from any available interface (TV, computer screen, mobile, PDA, etc.). In this way the user does not have to learn diverse procedures to access different services. Consistent interfaces will be available in all devices, with similar interaction procedures. • Seamless access: Since services are accessed through migratory interfaces, they can be dynamically transferred from one device to another maintaining the state
20
•
•
• •
•
F. Paternò et al.
of the user session. In this way, the user has seamless and ubiquitous access to the same information, wherever they are. Lower prices: Since all the services can be used through off-the-shelf devices and the software should be able to dynamically install and configure itself, by adapting to the available devices, installation and maintenance expenses should thereby be considerably reduced. Enhanced usability: As the context information is used to obtain migration triggers (e.g. as caused by degrading QoS) as well as to delimit the suitable candidate set of destination devices, these semi-automatic decisions will detach the end-user more strongly from the underlying technological aspects of the migration, hence increase usability. Personalization, the possibility to obtain services and the associated user interfaces that have been customized by the system according to user profiles, preferences and knowledge. User-decided customisations, apart from the possibility for the system to customize the migratable services depending on some user’s knowledge and preferences, the OPEN platform, thanks to the possibilities offered by partial migration, will also support to some extent End User programming, by enabling users to build/compose their own applications, Efficient use of network and device resources, since part of the system support for migration will be to provide adequate QoS-aware mobility solutions, that allow migration of parts of an application while other participating nodes are unaware of this migration, the OPEN applications can also involve devices with low capabilities, as they do not all need to provide the full OPEN service platform solution.
References Abrams, M., Phanouriou, C., Batongbacal, A., Williams, S., Shuster, J.: UIML: An applianceindependent XML user interface language. In: Proceedings of the 8th WWW Conference, Toronto, 11–14 May 1999 Allard, J., Chinta, V., Gundala, S., Richard III, G.G.: Jini Meets UPnP: An Architecture for Jini/ UPnP Interoperability, pp. 268–275. SAINT (2003) Balme, L., Demeure, A., Barralon, N., Coutaz, J., Calvary, G.: CAMELEON-RT: a software architecture reference model for distributed, migratable, and plastic user interfaces. In: Markopoulos P. et€al. (eds.) Proceedings of EUSAI ’04, Lecture Notes in Computer Science, vol.€3295, pp.€291–302. Springer Berlin (2004) Bandelloni, R., Paternò, F.: Migratory user interfaces able to adapt to various interaction platforms. Int. J. Hum. Comput. Stud. 60, 621–639 (2004) Bharat, K.A., Cardelli, L.: Migratory applications. In: Proceedings of User Interface Software and Technology (UIST ’95), pp.€133–142. Pittsburgh, 15–17 Nov 1995 Cameleon: Cameleon FP5 European project web site. http://giove.isti.cnr.it/projects/cameleon. html (2004) Consensus: FP5 European project web site. http://www.consensus-online.org (2004)
2â•… State of the Art in Migration
21
Dearman, D., Pierce, J.: It’s on my other computer!: Computing with multiple devices. In: Proceedings of the 26th Annual SIGCHI Conference on Human Factors in Computing Systems ACM CHI’08, Florence 5–10 April 2008, pp.€767–776. Dey, A.K., Abowd, G.D.: The context toolkit: Aiding the development of context-aware applications. In: Proceedings of Workshops on Software Engineering for Wearable and Pervasive Computing, Limerick, 6 June 2000 De Sousa, J., Garlan, D.: Aura: An architectural framework for user mobility in ubiquitous computing environments. In: Proceedings of the 3rd Working IEEE-IFIP Conference on Software Architecture, Montreal (2002) Fowler, M.: Martin Fowler’s web page on dependency injection. http://martinfowler.com/articles/ injection.html (2006). Accessed June 2006 Gajos, K., Christianson, D., Hoffmann, R., Shaked, T., Henning, K., Long, J.J., Weld, D.S.: Fast and robust interface generation for ubiquitous applications. In: Proceedings of UBICOMP’05: Ubiquitous computing, Lecture Notes in Computer Science, vol.€3660, pp.€37–55. Springer, Berlin Sept (2005) Games@Large FP6 European Project. http://www.gamesatlarge.eu Garlan, D., Siewiorek, D., Smailagic, A., Steenkiste, P.: Project Aura: toward distraction-free pervasive computing. IEEE Pervasive Comput. 21(2), 22–31 (April–June 2002) Garret, J.J.: Ajax: a new approach to web applications. Adaptive path, 18 Feb 2005. http://www. adaptivepath.com/publications/essays/archives/000385.php (2005) Ghader, M., Olsen, R.L., Genet, M.G., Tafazolli, R.: Service management platform for personal networks. In: Proceedings of 1st Summit 2005, Dresden (2005) Han, R., Perret, V., Naghshineh, M.: WebSplitter: Orchestrating multiple devices for collaborative web browsing. In: Proceedings of ACM Conference on Computer Supported Cooperative Work (CSCW), pp.€221–230. Philadelphia, 2–6 Dec 2000 JavaBeans technology. java.sun.com/products/ejb Jini network technology. http://wwws.sun.com/software/jini/ Johnson, D., Perkins, C., Arkko, J.: Mobility support in ipv6, RFC 3775. Internet Engineering Task Force (IETF), June 2004 Khedr, M., Karmouch, A.: Enhancing service discovery with context information. IEEE Canadian Conference of Electrical and Computer Engineering, Brazil (2002) Kramer, J., Magee, J.: Dynamic configuration for distributed systems. IEEE Trans. Softw. Eng. 11(4), 424–436 (1985) Lee, C., Nordstedt, D., Helal, S.: Enabling smart spaces with OSGi. IEEE Pervasive Comput. 2(3), 89–94 (2003) Lieberman, H., Paternò, F., Wulf, V.: End-user development. Springer, Netherlands (2005) Limbourg, Q., Vanderdonckt, J.: UsiXML: A user interface description language supporting multiple levels of independence. In: Matera, M., Comai, S. (eds.) Engineering advanced web applications, pp.€325–338. Rinton Press, Paramus (2004) Luyten, K., Coninx, K.: Distributed user interface elements to support smart interaction spaces. In: Proceedings of IEEE Symposium on Multimedia, Irvine, 12–14 Dec 2005 MAGNET-b: My personal adaptive global net. http://www.ist-magnet.org/ (2004) Mattern, F., Sturm, P.: From distributed systems to ubiquitous computing—the state of the art, trends, and prospects of future networked systems. In: Proceedings of the Symposium on Trends in der Informationstechnologie am Beginn des 21. Jahrhunderts, pp.€ 109–134, May 2002 Mehdi, Q., Kumar, P., Salim, A., Bechkoum, K.: Content adaptation and shared state distribution for multiplayer mobile games. In: Proceedings of 9th International Conference on Computer Games: AI, Animation, Mobile, Educational & Serious Games, CGames ’06, Dublin, 22–24 Nov 2006 Melchior, J., Grolaux, D., Vanderdonckt, J., Van Roy, P.: A toolkit for peer-to-peer distributed user interfaces: Concepts, implementation, and applications, EICS’09, pp.€69–78, Pittsburgh, 15–17 July 2009
22
F. Paternò et al.
Messer, A., Greenberg, I., Bernadat, P., Milojicic, D.S., Chen, D., Giuli, T.J., Gu, X.: Towards a distributed platform for resource-constrained devices. In: Proceedings of the 22nd International Conference on Distributed Computing Systems (ICDCS’02), pp.€43–51, Vienna, July 2002 Milojicic, D., Douglis, F., Paindaveine, Y., Wheeler, R., Zhou, S.: Process migration. ACM Comput. Surv. 32(3), 241–299, Sept 2000 MoCA: Homepage of the MoCA framework. http://www.lac.inf.puc-rio.br/moca/ (2006). Accessed Aug 2006 Mori, G., Paternò, F., Santoro, C.: Design and development of multi-device user interfaces through multiple logical descriptions. IEEE Trans. Softw. Eng. 30(8), pp.€507–520. IEEE Press, Aug 2004 Moscovitz, R., et al.: Host identity protocol, draft-ietf-hip-base-07 (work in progress). Internet Engineering Task Force (IETF), February 2007 Newman, M.W., Izadi, S., Edwards, W.K., Sedivy, J.Z., Smith, T.F.: User interfaces when and where they are needed: An infrastructure for recombinant computing. In: Proceedings of the UIST’02, Paris, 27–30 Oct 2002 Nichols, J., Myers, B.A., Higgins, M., Hughes, J., Harris, T.K., Rosenfeld, R., Pignol, M.: Generating remote control interfaces for complex appliances. In: Proceedings of ACM UIST’02, pp.€161–170, Paris, 27–30 Oct 2002 Olsen, D.R., Jefferies, S., Nielsen, S.T., Moyes, W., Fredrickson, P.: Cross-modal interaction using XWeb. In: Proceedings of UIST 2000: ACM SIGGRAPH Symposium on User Interface Software and Technology, pp.€191–200, San Diego, 2000 Oreizy, P., Taylor, R.N., Medvidovic, N.: Architecture-based runtime software evolution. In: Proceedings of the 20th International Conference on Software Engineering, pp.€177–186, Kyoto, 19–25 April 1998 OSGi Alliance. http://www.osgi.org/ Perkins, C.: IP mobility support for IPv4, RFC 3344. Internet Engineering Task Force (IETF), August 2002 Pluto.: http://plutohome.com/index.php?section=media_entertainment Ponnekanti, S.R., Lee, B., Fox, A., Hanrahan, P., Winograd, T.: ICrafter: A service framework for ubiquitous computing environments. In: Proceedings of UBICOMP 2001, Lecture Note in Computer Science, vol.€2201, pp.€56–75 (Atlanta, 2001) ISBN:3–540-42614–0. Springer, London (2001) Puerta, A., Eisenstein, J.: XIML: A common representation for interaction data. In: Proceedings of IUI 2002 (San Francisco, 13–16 Jan 2002), ACM, New York (2002) Renier, T., et al.: MIPv6 operations in IMS-based access networks. In: Proceedings of WPMC’06, San Diego, Sept 2006 Riegel, M., Tuexen, M.: Mobile SCTP, draft-riegel-tuexen-mobile-sctp-07.txt (work in progress). Internet Engineering Task Force (IETF), Oct 2006 Riva, O., Nzouonta, J., Borcea, C.: Reliable migratory services in ad hoc networks. IEEE Trans. Mob. Comput. 6(12), 1313–1328, Dec 2007 (2006) Rosenberg, J., et al.: SIP: Session Initiation Protocol, RFC 3261. Internet Engineering Task Force (IETF), June 2002 Sadjadi, S.M.: A survey of adaptive middleware software engineering and network systems laboratory. Michigan State University, USA Technical Report MSU-CSE 3–35, 2003 Salutation: Architecture specification (Part-1), the salutation consortium. Available: http://www. salutation.org/ (1999) Sapuntzakis, C.P., Chandra, R., Pfaff, B., Chow, J., Lam, M.S., Rosenblum, M.: Optimizing the migration of virtual computers. SIGOPS Oper. Syst. Rev. 36(SI), 377–390 (2002) SLP: Service Location Protocol svrloc—RFC2608, V2 ed., IETF, June 1999 Spring: Homepage of the spring framework. http://www.springframework.org/ (2006) Taylor, R.N., et al.: A component- and message-based architectural style for GUI software. IEEE Trans. Softw. Eng. 22(6), 390–406 June 1996 Trapp, M.: Modeling the adaptation behavior of adaptive embedded systems. München: Verlag Dr. Hut, 2005 Zugl.: Kaiserslautern, Techn. Univ. Diss. (2005)
2â•… State of the Art in Migration
23
TS23.228: 3rd generation partnership project, IP Multimedia Subsystem (IMS)—Stage 2, TS 23.228, v5.15.0, 3GPP, June 2006 UPnP: Universal plug’n’play. http://www.upnp.org/ (1999) Wetherall, D.: Active network vision reality: lessons from a capsule-based system. In: Proceedings of the 17th ACM Symposium on Operating Systems Principles (SOSP 1999), pp.€64–79, Charleston, Dec 1999 White, J.: Mobile agents. In: Bradshaw. (ed.) Software agents. MIT Press, Cambridge (1997) Ziegert, T., Lauff, M., Heuser, L.: Device independent web applications—the author once—display everywhere approach. In: Koch, N., Fraternali, P., Wirsing, M. (eds.) ICWE 2004, Lecture Notes in Computer Science, vol.€3140, pp.€244–255. Springer, Berlin (2004)
Chapter 3
Migration Opportunities Agnese Grasselli, Alessandro Vangelista and Stefano Bolli
3.1â•…Setting the Scene This section highlights key markets trends in order to outline future market scenarios where the OPEN project’s deliverables will be useful and directly applicable. These trends will generate opportunities in the ICT arena to create new services and Service Provider, such as Vodafone, leveraging on fixed and mobile assets, needs to keep pace with these trends. Broadband Access╇ • Broadband is moving beyond the PC; it will be about connecting numbers of devices, providing consumers with a range of multimedia services in an always-on digital world. • The ability of gadgets of all sizes to connect to the Internet and download or stream content will be more prevalent in coming years. • Connected portable devices, from netbooks to game consoles, are complementing “traditional” items such as mobile phones and desktop computers. • Mobile devices with increased power, faster communications capabilities and higher resolution displays are increasingly saturating everyday life. • Mobile phones will have more options for connecting to the mobile internet and the access will be more ubiquitous. • Wireless broadband networks mean that consumers can access broadband services within the home as well as on the move. • Wireless technology developments are paving the way for a far more flexible broadband environment that will increasingly allow users portable and nomadic access to content, applications and services. • The world of telecommunications is in constant flux, with fast, mobile networks enabling uninterrupted access to new internet-based products and service.
A. Grasselli () Vodafone Omnitel NV, Milan, Italy e-mail:
[email protected] F. Paternò (ed.), Migratory Interactive Applications for Ubiquitous Environments, Human-Computer Interaction Series, DOI 10.1007/978-0-85729-250-6_3, ©Â€Springer-Verlag London Limited 2011
25
26
A. Grasselli et al.
• Technical progress, increasing levels of network coverage, greater bandwidths allowing higher data speeds are factors which will expedite the progress of digitization and networking. Multimedia Services╇ • Broadband internet access is popping up in TVs—new TV sets have built-in networking connections requiring no additional set-top boxes for getting online. • Usage of online video services is rising quickly and HDTV is gaining ground. • Ownership of multiple computers is becoming more common. • IPTV has the potential to transform the TV experience, not just by offering more choice and flexibility, but by integrating aspects of communications and social computing. • The future home will be a place where consumers can watch what they want, when they want and by whichever device they want. • The average family might have a TV in every room, and each family member might be using them for something completely different; watching one of hundreds of channels, playing video games or watching DVDs or YouTube clips. • IPTV expands the range of what’s possible with TV. In addition to viewing video content, the TV can become a screen to view our personal digital photos or to make a video phone call. It also frees us from watching TV in the home; IPTV means we can take our content with us, wherever we go. • The broad range of devices available within the home shows there are many means of creating and accessing digital content. Social Computing╇ • In many areas of life, digital networking is taken for granted and the importance of these technologies will continue to increase. • Digitalization and networking ensure that anyone can stay in touch with people and things at any time. • Social media encourages the formation of a community around the game (as has been successfully demonstrated with EA’s Pogo for example) and enhances the game itself. • In 2008 nearly 2/3rds of young consumers had a social network profile (↜source: Brown 2008, Mobile Youth based on Pew IALP data). • The growth of digitalization and networking with broader bandwidth, faster speeds and more widespread use of mobile broadband connections will further influence and shape the way we communicate. Gaming╇ • Cross-platform gaming is gaining momentum, e.g., solutions across interactive TV, online and mobile platform, such as role play games with several online players. • Simplicity and feature accessibility creates significant changes in consumption.
3â•… Migration Opportunities
27
• Leisure activities will be strongly influenced by digitalization and networking; it will become common to listen to music or radio over the internet, access videos at any time or fill in waiting times by playing games on mobile handsets. • Mobile game revenues potentially have long term value as an advertising rather than direct revenue generation tool. • “Advergaming” passed $1 billion in 2007 and will double in value by 2011 (↜source: Brown 2008, Mobile Youth based on emarketer data). • In game advertising offers a useful option in developing a wider community strategy. • The community is often seen as a very important part of the game, particularly in the case of multi-player games. • In the Massively Multi-Player Online Game (MMOG) industry there is a saying “they come for the game but they stay for the community”. • Plenty of literature exists describing the benefits of a game having an active player community (Koivisto 2007). • Connected (online) mobile games will gain momentum with increasing network capabilities in terms of latency and throughput. • Some technologies that will become more common in mobile games such as the mobile phone camera, will encourage the development of games with new interaction styles. • Technology enhancement will drive flexible game interaction functionality such as the use of voice chat simultaneously with game playing, presence1 and messaging. • Technological advancements in the mobile phone such as hardware accelerated 3D graphics will drive the increasing quality of the mobile games. In this complex environment, better knowledge of migration scenarios will be a key topic to offer better user experience.
3.2â•…Multiscreen Ambition These market trends are changing the arena where service providers are going to design their new services. One of the main market drivers that could impact the OPEN project results playfield is the “multi-screen ambitions”, the ability to deliver the same service to any device, wherever and whenever required and with a recognizably similar (if not identical) user and developer experience. Next generation devices present the opportunity, for multi-screen players, to offer service integration across device types (see Fig. 3.1). 1╇ Presence: In ICT (Information and Communication Technologies) it’s a sort of “status information” that state communication partner’s information (such as telephonic capabilities, availability, location ...).
28
A. Grasselli et al.
Fig. 3.1↜渀 Service integration across device types
To play this game, Service Providers aim to expand their platform influence beyond mobile devices, extending the reach to the connected home and the extended home. Many players in the market are placing pieces of their planned policy day by day, realizing their strategy: • Microsoft is adding more weapons to its multi-screen armoury with the launches of Windows 7 for embedded devices and Silverlight for systems on chips (SOCs). • Google’s Android platform has begun cropping up in multiple additional device categories beyond smartphones, including TVs, tablets, PMPs, netbooks and even DAB radios. • Apple’s successful iPad launch seems to have proved a demand for connected tablet devices, whether or not they have yet a clear use case. • The Nokia/Intel-developed MeeGo device platform has the implicit backing of net-book, set-top box and PC vendors, as well as of Nokia itself, and looks likely to reach others.
3â•… Migration Opportunities
29
So, software platform and the surrounding ecosystem of developers are vital assets in making this vision.
3.3â•…Migration Platform Value Chain Migration Service Platform (MSP) plays a key role in the “multi-screen ambitions” in order to offer a completely immersive user experience. From the architecture point of view the platform plays as an “abstraction layer”, able to separate content deliver platforms from the end-user devices. In the end-to-end ecosystem, this Migration Service Platform will be a new “element” that gives a device agnostic access to the end user point of consumption. Depending on the various value chain viewpoints, OPEN MSP introduces different types of scenarios: • Contents Providers (BBC, Warner Brothers, SKY …) could add “as a new feature” a migration capability to their network: − This is like to address migration issues playing at application layer. − Each provider need to enhance its architecture and deal with TCO. • Carriers/Services Providers (Mobile Operator like Vodafone, AT&T, …) could introduce migration capabilities to their network as new service: − For the network providers, this is like to enrich “pipe services” with value added services like migration feature. − Network enhancement costs could be shared between end user and content providers. − For the service provider, it’s an additional instrument to delight more its customers offering experience continuity. − Carriers could also enrich their customers profile (which gives the ability to control access to services, usage and preference information) thanks to context information collected by migration platform. This could also be a selling proposition to 3rd parties (e.g. to help content providers understand what additional services consumers want; not only from contents point of view, but along the overall service consumption life time). • A new type of player could appear in the value chain: a Migration Provider − It offers “just” migration services. − In a cloud based paradigm (an emerging approach especially in mobile playfield), a Migration Provider offers “as a Service” migration capabilities to everyone who needs it. − From the end user point of view, it plays in horizontal way, across different carries networks. The introduction of a Migration Service Platform in the end-to-end service ecosystem also could impact developers and device manufacturer.
30
A. Grasselli et al.
In the design phase, developers have to consider new continuity capabilities offered by underline layers that could enrich the overall service/application and also could open the door for new type of end user interaction. Device manufacturer could introduce new hardware features leveraging on Migration Capabilities, or design new device interfaces considering service continuity as a commodity.
References Brown, G.: Mobile Youth based on Pew IALP data 2008 “Mobile Youth 2008 report (2)—marketing & advertising” 79 pg. http://www.mobileYouthNet.com/GrahamBrown (2008) Ericsson: Ericsson report: “IPTV and the connected home” [2009] What consumers want from advanced TV services and the connected home research study summary. http://www.ericsson. com/uk/ericsson/about/pressrel/Ericsson_Connected_Home_Summary_Report_300309_ RevA.pdf (2009) Koivisto, E.: Mobile games 2010, Nokia report 2007, Nokia Research Center Finland. http:// research.nokia.com/files/NRC-TR-2007-011.pdf (2007)
Chapter 4
The OPEN Migration Platform Architecture Miquel Martin
4.1â•…The Concept of Migration Migration is by any definition a complex operation, and its exact meaning depends on several design decisions spanning several aspects of the process. From an instantiation point of view, one might consider migration as the process by which a snapshot of the application and its state are removed from the source devices and injected into the target device (or devices)—an object cut & paste, if you will. While this is closest to a generic dictionary definition, it is exceedingly impractical in the context of software instances. Right at the start, the process is likely to fail unless the source and target devices are the exact same platform. Even then, the state of the different resources would cause incompatibilities that could lead to failure, even if the migrated application’s first step were to try and adapt. This calls for a relaxation of the concept of migration where applications support a call to serialize themselves (both code and state). The resulting binary object could then be handed over to a migration helper in the target device, responsible for deserializing the instance while at the same adapting it to the target device. This is as close as we could get to instance migration, and yet, it is not devoid of difficulties: given that the migration helper as described here is a generic component, the only adaptations it could make on the application instance would be just as generic to the device: switching user interface widgets or adjusting layouts to the screen size. We are then left with an application whose richness and complexity is limited by the intersection of the features of all the devices we want to migrate it to. In other words, our application will never be better than the simplest of the supported devices. This is not such a big problem for the migration of web based applications, the web being the reasonably standardized body of protocols it is, but is a fatal flaw for native applications. J2ME has long suffered from this problem, the only solution
M. Martin () NEC Europe Ltd., Heidelberg, Germany e-mail:
[email protected] F. Paternò (ed.), Migratory Interactive Applications for Ubiquitous Environments, Human-Computer Interaction Series, DOI 10.1007/978-0-85729-250-6_4, ©Â€Springer-Verlag London Limited 2011
31
32
M. Martin
involving polymorphic applications that would actually choose different modules depending on the device they run on. This has brought us, in the OPEN project, to further relax the concept of migration. In the case of what are referred to as native applications, we fully acknowledge the need for rich applications with different feature sets in each platform. With this in mind, we have chosen not to migrate application instances, but rather opt for extracting the application state, terminating the source instance, and starting a new native instance on the destination platform, with the same state, provided the destination platform has the sufficient capabilities. Such an approach requires a native version of the application for each supported platform where platform differences justify it, but this is already the industry’s approach, pushed by consumers to make the most out of each device they target. What must remain generic, however, is the application state, which needs to be understood by all implementations of the application, even if some don’t use certain parts of it. In summary, we support the requirement to fully exploit the potential of each platform, but do require both comparable application logics across devices and compatible state information models.
4.2â•…The Advantages of the OPEN Approach Given the OPEN understanding of the migration concept explained so far, developers are left with the task of implementing their application for their target platforms, and of equipping them with the ability to export their state. With these steps, however, application migration support is far from complete. At the very least, a coordinating entity is required that can communicate with both the source and target devices, and which can handle: • Instance lifecycle (termination in the source and instantiation in the target) • Extraction of application state from the source and injection into the target • Informing the user and getting his consent throughout the whole process If we consider these carefully, however, it soon becomes clear that these functionalities are bound to the platform, but independent of the concrete application. It therefore stands to reason that, by factoring out this functionality, we could build some sort of system daemon that supports the migration of all the applications in the device. In doing so, applications would not be required to implement these functionalities, needing only to support a lifecycle and a state transfer interface that would be operated by the said daemon. And so this is the OPEN migration approach: applications need only implement a minimal interface set, and the rest of the functionality will be handled by the OPEN platform. More concretely, this will involve a set of daemons running in the devices, called OPEN Adaptors, and a coordinating entity running on the server side, aptly called the OPEN Server. The sum of an application with the OPEN Adaptors is seen as a single entity by the server side, and is therefore called an OPEN Client. Please
4â•… The OPEN Migration Platform Architecture
33
note that one could also dispose of the OPEN Server by defining peer-to-peer capabilities between OPEN Clients, but this is left as future work. This then is the core advantage of the OPEN approach: migratory application development is freed from most of the complexities of the migration process by a set of common OPEN components running alongside them. By working on migration once, and using it in all applications, we are able to concentrate our efforts on a single design, and this gives us the possibility to consider additional functionalities which, when included in the OPEN modules, will benefit all the related applications. The following are additional functionalities of the OPEN platform, and they will be discussed in detail as we progress in the book: • Device discovery provides candidate migration targets, allowing users to choose the best one in an ad-hoc fashion, especially in dynamic or new environments where devices change over time (e.g. when using a public display in an airport) • Context management provides information about the user’s device and its surroundings, which helps the software to make informed decisions on issues like prioritization of migrations or even logic changes (e.g. do not show private information since the target device is publicly visible) • Application Logic Reconfiguration (ALR) allows applications to outsource certain decisions about how their internal logic is handled, considering that the ALR modules will have an overview of the environment, source and target devices, which is beyond what any single application instance has. • Trigger Management provides support for automatic triggers, where multiple aspects like application and user policies, device capabilities or network context are considered in order to flag migrations as beneficial to the user or not. • Mobility support will provide capabilities to handle a number of network related hurdles on behalf of the application, including network session persistence, and terminal mobility. • Partial migration, where a modular application can choose to migrate only some of its modules over to the target device. • User interface adaptation, which optimizes the UI to the capabilities and characteristics of the target device. And so the OPEN Platform can be considered as an Application Migration Helper Kit, in that it takes care of the standard migration operations, and further adds a number of advanced functionalities which would be too complex for each individual application to tackle on its own.
4.3â•…Architectural Overview of the OPEN Platform As described above, the OPEN platform provides a framework that enables an application to migrate between devices, be it by completely migrating, or by sending some of its parts over to another device (partial migration). Furthermore, the
34
M. Martin
Fig. 4.1↜渀 Overall architecture, where applications are seen as clients of the migration platform
platform provides the application with the necessary mechanisms to adapt to the new device, including adaptation of network connections, user interfaces, and restoration of the previous state. We have already introduced the concept of the OPEN Platform as a System Daemon split between a client and a server part, as illustrated in Fig.€4.1. Due to the generic approach we have followed, the OPEN server sees only any number of OPEN Clients, with no concern for the particular details of the applications behind them. By using the OPEN interfaces, applications can request migrations and access the functionalities provided by the platform. The OPEN server, in turn, can request other Clients to execute applications and restore/inject state, or trigger clients to perform any of the other advanced features. The OPEN server may reside on any device, so long as it is reachable by the OPEN Clients and fulfills the necessary hardware specifications. Within the scope of the project, a single OPEN Server is considered where all Clients are registered. It is, however, a small step towards more distributed deployments, such as the one shown in Fig.€4.2. In this example, an organization in an office building (domain 1) runs an OPEN Server, while a home user has set up an OPEN Server at her home (domain 2). OPEN Clients (marked OC) are registered to a particular OPEN Server, but might be handed over to other servers as the user moves from one domain to the other. The finer points of these mechanisms are not covered in this book, but numerous solutions exist, such as providing OPEN Server addresses as a field in the DHCP messages when entering the network. Likewise, a P2P approach without dedicated OPEN Servers would be possible in an environment where one node is elected by its peers to play the role of the OPEN Server, provided it has the necessary hardware requirements. Once the p2p overlay network is established, the OPEN mechanisms would continue to work in the fashion described in this document.
4â•… The OPEN Migration Platform Architecture
35
Fig. 4.2↜渀 OPEN servers may be specific to a location, and clients might be handed over between them
4.4â•…Making Applications OPEN-Aware One of the objectives of the OPEN platform is to abstract as much migration functionality as possible from the application, thus making it easy for developers to integrate with our solutions. An OPEN Client, which behaves as a Black Box as far as the platform is concerned, is actually composed of one or more devices which run the applications and call on the OPEN functionality.
4.4.1 The OPEN Adaptors Figure€4.3 illustrates the internals of an example OPEN Client. The Client is made up, in this case, of a Terminal (e.g. a mobile phone) which runs a native application and a set of OPEN Adaptors. The adaptors implement the part of the migration functionalities that are common across applications. They are meant to be reused across applications, so long as the platform permits it. In doing so, an application
36
M. Martin
Fig. 4.3↜渀 Applications implement or use the OPEN adaptors, and in doing so, present an OPEN client interface to the OPEN platform
needs only to make use of the adaptors to become migration capable. Naturally, there is still complexity involved in modifying the application logic to support the new migration lifecycle, but all other common tasks are extracted onto the adaptors. That said, an OPEN Client only needs to present an OPEN Client interface to the OPEN Server, so the use of the adaptors is actually optional. As long as applications respect the interface, they may override the default adaptors and re-implement their functionality internally. This is sometimes necessary to support certain types of application available nowadays, and especially those found in the web. Consider for instance: • Web based applications are split between their execution environment (a browser) and their actual content (HTML, JavaScript and embedded objects like Silverlight or Flash) • One might want to support certain migrations, such as web pages, without modifying the existing web servers. This requires pulling more functionality into the client Because of the richness of the application ecosystem, and to provide more flexibility to developers, applications can integrate with OPEN by using the OPEN Adaptors exclusively, or by re-implementing them and offering the OPEN Interface directly from the applications. Any combination in between, where applications reuse some adaptors, and override others, is possible as well.
4.4.2 Client and Server Side Applications Another crucial difference between application types is how many of them run on a server, and how many are actually client-based. Whenever state has to be kept, for instance, it might be necessary to harvest state information from both the client and the server. In another example, UI adaptation might require adaptation on the client side alone. The OPEN approach here is again that of flexibility. Assuming that the combined client and server side of an application can behave, as a whole, as an OPEN Client would, the OPEN Server will behave as expected. This simply means that the sum of the methods of the interfaces of the OPEN Adaptors should add up to implement all the methods of the OPEN Client interface. If this is the case, the platform will not concern itself with where each of the adaptors is actually running. Figure€4.4 illustrates this scenario.
4â•… The OPEN Migration Platform Architecture
37
Fig. 4.4↜渀 Applications look like a single OPEN client to the platform, even when they have a server and a client part
We have now described two of the choices that application developers have at their disposal when integrating with the OPEN Platform. The applications covered in this book, fall into different categories. Figure€4.5 shows how these applications map to the two parameters proposed. The Emergency Scenario, for instance, is a mostly server based, .NET and Silverlight application. As such, it is difficult for it to integrate with all of the available OPEN Adaptors, and therefore implements most of that functionality itself, from the server side.
Open Server functionality accessed mostly from the Application’s server side Pac-Man Game
Social Game Emergency Scenario
Open Server functionality accessed mostly from the Application’s client side
Ap us plica e t Ad the ions ap O tor pen s
Ap the plica t Op func ions im en tio Ad nal plem ap ity tor of ent the s
Fig. 4.5↜渀 Different applications can choose to adapt differently to the OPEN platform to achieve migration support
38
M. Martin
The Pac-Man game makes full use of the OPEN Adaptors, especially those for Application Logic Reconfiguration, and has some components on the server side. The Social Game, finally, makes use of most OPEN Adaptors, and uses a homogeneous mix of client and server side components.
4.4.3 Partial Migration The Social Game, additionally, illustrates the concept of partial migration, where an application distributes itself across multiple devices (i.e. portions of the application are migrated). Technically, this can be achieved by running the different application components in different devices, and then presenting themselves to the platform as individual OPEN Clients. Thanks to the initial registration information, the OPEN Platform will see the different “applications” running in each OPEN client for what they really are: components of a bigger application.
4.5â•…Platform Communication: The OPEN Dispatchers The OPEN Platform is built on the principle of modularity. The OPEN Server provides its functionality through multiple Server Components. The OPEN Client is made up of a number of Adaptors, and an application, which can be further split between a server and a client side, and then again into components. While this makes the OPEN Platform easier to customize to the specific needs of both the deployment devices and the integrating applications, it comes at a price. Consider, for instance, the OPEN Server Interface, which is made up by the sum of the interfaces of the OPEN Server components. How would an OPEN Client know which component to address, when a certain part of the interface was to be invoked? It is clear that communication hubs are required that factor out this complexity and provide some sort of directory service. The proposed solution in OPEN follows the dispatcher pattern, where a Dispatcher module acts as a single endpoint for a number of components and, at the same time, those components use the dispatcher to route their messages. Figure€4.6 shows how this concept is applied to the OPEN Platform: Notice how the highlighted dispatchers funnel the communication of the multiple components. We can identify three types of Dispatchers: • The OPEN Server Dispatcher: is unique for each OPEN deployment, and it is the endpoint for the whole OPEN Server. Any OPEN Client that needs to make a request to the server will do so through this Dispatcher, and all OPEN Server components will send their messages to the OPEN Clients through it.
4â•… The OPEN Migration Platform Architecture
39
Fig. 4.6↜渀 Dispatchers in the OPEN server and client act as a communication hub
• The OPEN Client Dispatcher: is present once per OPEN Client, and it represents the unique contact point through which the OPEN Server can send messages to the Applications and Adaptors it contains. Likewise, when an OPEN Client component needs to send a request to the OPEN Server, it does so through the Dispatcher. Note how, should an application be split between its server and client sides, the OPEN Client Dispatcher takes cares of sending the message where it is needed. • The OPEN Adaptor Dispatchers: are bound to a specific application/device pair. They are meant to act as a proxy for the whole functionality of the platform towards the applications, in a compact and easy way. As in the case of the Adaptors, applications can choose to use the existing dispatchers, or implement their own. Whichever the case, the Dispatcher must fulfill its functions, including the aggregation of the interfaces of the individual components, so that: • Applications see an OPEN Server Interface provided by the Adaptor Dispatcher • Adaptors see an OPEN Client Interface provided by the Adaptor Dispatcher • OPEN Servers see an OPEN Client Interface provided by the OPEN Client Dispatcher • OPEN Clients see an OPEN Server Interface provided by the OPEN Server Dispatcher. These interfaces are consistent with the directional view presented in Fig.€4.8. In using the dispatcher infrastructure, applications need to communicate only with a statically configured Dispatcher, and need not worry about the underlying network conditions. This is true to the point that components can now deal
40
M. Martin
exclusively with a ComponentIDs, regardless of the identifiers in the underlying layers (e.g. IP address, port or MAC Address). In other words, the specific service endpoints, which might change at runtime in mobility scenarios, remain hidden.
4.5.1 Communication Models The communication protocol for the OPEN Platform is XML-RPC. This assumes that components can listen for requests, and even receive asynchronous responses. This, however, can be problematic in at least the following cases: • Firewalls and Network Address Translators (NATs): can block incoming traffic, either through the enforcement of firewall policies, or because NATed components run on a private IP which is not routable from the other communication endpoint. • Application platforms which don’t support listening modes: the clearest case is that of web applications, which can make HTTP requests (even asynchronous ones using AJAX) but cannot listen to messages initiated by an external peer (with the notable exception of AJAX frameworks that support Comet functionality). To solve this problem, the OPEN Dispatchers support a polling mode on the server side. Messages that would usually be directly sent to the Client Interface are instead buffered at the dispatcher. The client end can then poll periodically to retrieve pending messages. The following points are worth stressing: • Only the OPEN Server interface offers a polling mode for asynchronous execution of the Client side interface • Because of that, all OPEN Client methods are asynchronous
4.6╅OPEN Platform Architecture The aim of this section is to put together all the concepts and ideas presented above, providing an overview of the OPEN Platform architecture. Figure€4.7 presents the overall architectural picture of the OPEN platform. A normal deployment will include one OPEN Server and any number of OPEN Clients. All communication between the different segments occurs through a Dispatcher, and the application can override any or all of the client side components. The Application Logic Reconfiguration (ALR), the Policy Enforcement and the Trigger management, do not require an adaptor counterpart, since they do not possess any client side functionality. While they do not exist as a software instance,
4â•… The OPEN Migration Platform Architecture
41
Fig. 4.7↜渀 OPEN platform architecture. Note that the mobility anchor point is left out for simplicity. (see Chap.€6 on mobility support for details)
they do have a logical significance, in that the Application seems to talk to the adaptor component on its way to the server. Because their functionality is limited to relaying messages between the application and the server components, we have chosen to implement this in the dispatcher instead. Trigger management is a special case in that manual migration triggers can be generated internally in the application, or using the OPEN Client Daemon; in this sense trigger generation is represented as a link to both the Trigger Management Adaptor and the OPEN Client Daemon. Finally, we would like to highlight the role of the Orchestrator component. As its name suggests, it is responsible for interacting with the rest of the platform components, and for coordinating the lifecycle of applications as they migrate. As a summary: • All OPEN Platform communication on the wire travels from and to a Dispatcher • Applications can choose to implement any of the OPEN Client functionality • In regard to platform communication, an OPEN Client will never talk directly to another OPEN Client: it will go through the OPEN Server
4.7â•…OPEN Interfaces All the functionality of the OPEN Platform is accessible through the OPEN Client and Server interfaces, both of which are implemented where necessary using XMLRPC.
42
M. Martin
These interfaces, however, are not limited to the OPEN Server and OPEN Client blocks, but throughout the platform: • The OPEN Server presents an OPEN Server interface to the OPEN Client but, • The OPEN Adaptors also present an OPEN Server interface towards the Applications • Likewise, the OPEN Client presents an OPEN Client interface to the OPEN Server, and • The Application also presents an OPEN Client interface to the Adaptors In other words, the platform uses only two interfaces: which one of them a component implements for its neighbor, and which one it can expect from that neighbor, depends exclusively on the direction we’re looking at: towards the server, all components implement the OPEN Server Interface; towards the client, the OPEN Client one. Figure€4.8 illustrates this concept.
Fig. 4.8↜渀 The interfaces offered to a component depend only on the direction the component is “facing”
4â•… The OPEN Migration Platform Architecture
43
Note that it is up to the application developer whether to override the OPEN Adaptors, or implement their functionality within the application. Whatever the choice, the aggregated interfaces provided by the Adaptors, and the Application, must always amount to a complete interface. The routing of external calls to the appropriate implementing component will be handled by the dispatchers, described in the next chapters.
4.7.1 Interface Design Philosophy The OPEN Interfaces are implemented using XML-RPC, an application independent Remote Procedure Call mechanism based on XML, which takes a method name and any number of name/value pairs as parameters. In order to simplify the implementation of the interfaces, we have striven for a minimalistic approach, with no method overloading and a minimal set of parameters. The complete listing of methods can be found in Appendixes A and B for the OPEN Client and OPEN Server interfaces respectively. The complexity needed for the platform, therefore, has been transferred to the Data Types that form the parameters. Appendix B has a complete listing of object types. Because certain clients cannot listen for Client interface methods, the OPEN Dispatchers support a polling mode. This, however, means that the Server components will not get a reply until the next polling cycle; so, to avoid unnecessary blocks, all OPEN Client interface methods must be asynchronous. This results in a number of OPEN Server methods which are in fact the Callback of their OPEN Client counterparts.
4.7.2 Ensuring Data Consistency All running components, applications and various other information, is kept at the OPEN Server. For example each application has an associated Application object, which contains information such as the components that form the application. On occasion, these objects might be sent over to the OPEN Client, where they might be changed. It is therefore important to keep track of the changes and ensure synchronization of the instances of the object. To achieve this, we have determined that objects may, in general, be modified only at the OPEN Server Orchestrator component. When client side processing is required, a method is used to send the object to the Client. Likewise a corresponding callback method receives the modified object. The OPEN Server must therefore relinquish its lock on the object from the moment it’s sent out, until it is returned. In doing so, we can guarantee that the Object will only exist at one place at any given time.
44
M. Martin
4.8╅Conclusions This chapter has elaborated the concept of migration as understood in the OPEN approach. Once a common understanding has been established, we have introduced the design philosophy and particularities of the OPEN architecture, without going into detail on the inner workings of each module. This will be covered in the following chapters, and exemplified with several applications. Acknowledgments╇ Thanks to the OPEN Partners for their contributions to the topics addressed to this Chapter.
Chapter 5
User Interface Migration Based on the Use of Logical Descriptions Giuseppe Ghiani, Fabio Paternò and Carmen Santoro
5.1â•…Introduction Nowadays people are ever more exposed to ubiquitous environments, which are characterized by the availability of various interactive devices with different interaction resources. Thus, the possibility to opportunistically exploit the resources that such contexts offer (e.g., moving from stationary to mobile devices) is invaluable for providing an effective interaction experience to the user. In this context, interactive migratory user interfaces offer the added value of enabling users to migrate across various types of devices while preserving the task continuity. This implies that there should be the possibility to select a target device and activate on it a version of the user interface adapted to its features with the same state as on the source device. The state of a user interface includes the values entered or selected by the user, the content, the cookies, etc. Various types of migration can be identified depending on the number of source and target devices or whether the entire user interface or only a part of it migrates. In particular, partial migration means moving only a portion of the interactive application (namely: some components) to another device in order to better exploit its interactive resources. The typical scenario is a user who is interacting with a desktop or large screen system and then for some reason has to leave, but wants to continue the interaction through a mobile device with only a part of the application. This can be either because of its complexity or because of some limitations in the mobile device (e.g. iPhones do not support Flash applications). This is particularly important with mashup-like applications, which tend to be particularly complex and made up of various perceivable components. Model-based approaches (see for example Eisenstein et al. 2001; Paternò et al. 2009a) have shown good potential in managing the complexity of multi-device environments. In such approaches there is usually a distinction between abstract (modality independent) and concrete (modality dependent) logical descriptions, F. Paternò () CNR-ISTI, HIIS Laboratory, Via G. Moruzzi 1, 56124 Pisa, Italy e-mail:
[email protected] F. Paternò (ed.), Migratory Interactive Applications for Ubiquitous Environments, Human-Computer Interaction Series, DOI 10.1007/978-0-85729-250-6_5, ©Â€Springer-Verlag London Limited 2011
45
46
G. Ghiani et al.
which makes it possible to better support interoperability across various types of devices and implementation languages. Indeed, such approaches generally work by progressively refining these descriptions, from higher, to more concrete ones, till the implementation level. Abstract descriptions enable the designer to specify user interfaces using a platform-independent vocabulary of UI objects (also called “interactors”). Therefore, at this level the UI specification includes generic selection objects, edit objects, … etc. Concrete UI descriptions further refine abstract objects by adding platform-dependent details. So, an abstract selection object can be refined into various concrete interactors, such as a pull-down menu, a radio button, a check-box. It is worth noting that the concrete UI objects still do not refer to any specific implementation language: such details are added by a final refinement step. Our approach aims to provide a general solution for Web applications implemented using (X)HTML, CSS, and Javascript. It can also support applications based on languages such as JSP, PHP, ASP because it considers one page at a time on the client side. Thus, it adapts only what is actually accessed by the user. Another advantage of the solution proposed is that it makes Web applications migratory regardless of the authoring environments used by the developers. Without requiring the use of any specific tool in the development phase, it enables the applications to migrate, even if the developers never considered migration. This is obtained through the use of reverse engineering techniques that create the logical descriptions of the Web pages accessed on the fly, which are then adapted for the target device. Lastly, an implementation with the state of the source version is dynamically generated. The subject of partial migration raises a number of issues, which have been addressed from different viewpoints in other work by the research community. Partial migration can be related, to some extent, to the issues connected with Distributed User Interfaces (DUI). To this regard, in (Melchior et€al. 2009) a toolkit for deploying distributed graphical UIs is presented. In our solution we opted for a distribution down to the granularity of the single interactor but no deeper, since we judged such fine granularity unimportant for our goals. In addition, this solution requires that the user interface be implemented using an extension of the Tcl/Tk toolkit, while we are interested in solutions that allow to partially migrating any Web application developed with standard W3C languages (XHTML, CSS) and JavaScript. The issue of distributing a user interface onto multiple devices is also analysed in (Pierce and Nichols 2008), with particular attention to how to leverage legacy applications to attain the new distributable features easily. However, it is worth pointing out that the relations on which this infrastructure is based includes a strong limitation that narrows the set of devices that can be interconnected to each other (only the personal devices of a user). Instead, fully migrable applications should be able to opportunistically exploit the devices in the environment (even the devices not owned by the users but accessible to them). The study in (Edwards et€al. 2009) describes an infrastructure (Obje) that supports building interoperable systems without having prior knowledge about them ahead of time. While the motivation of the Obje architecture was to provide an infrastructure for opportunistic interoperation in device-rich environments (as in migration) this approach basically addresses problems of interoperation rather than migration. In
5â•… User Interface Migration Based on the Use of Logical Descriptions
47
general, we can notice that while a number of model-based approaches have been put forward for the design of multi-device interfaces, and in particular for mobile applications (see for example Eisenstein et€al. 2001), none of them has shown a general solution able to work on any Web application implemented according to the W3C standards for supporting partial user interface migration from desktop to mobile systems. In this chapter we present a solution supporting UI migration (both total and partial), its main characteristics, the architecture of the migration platform supporting it, and also provide examples of migration for a Web application, in order to show its use and potentialities.
5.2â•…Architecture The starting point for the work presented in this paper was the solution described in (Paternò et€al. 2009a), which supports migration of only entire user interfaces (total migration), without providing any possibility to migrate only parts of them. In that work an architecture based on a number of modules was proposed supporting dynamic reverse and forward engineering. The modules identified were: the Reverse Engineering module, which builds the logical description of the source page considered; the Semantic Redesign, which transforms the source logical concrete description into another one, tailored for the target platform; the State Mapper associates the state of the current Web page to the logical description automatically generated for the target device; the Generator generates the corresponding implementation. Such implementation is then sent to the target device so that the user can immediately find the adapted page, with the state resulting from the interactions already carried out with the source device; the Proxy Server is an intermediate layer: as a proxy, it captures all the communications occurring from the user browser to the application server and vice versa; the Migration Orchestrator (as we will see in Chap.€6) handles the communications with the different modules involved in the migration. The Reverse Engineering part is able to build corresponding logical descriptions from (X)HTML, CSS and Javascript implementations. If the Web application contains Flash or Java applets, then the Reverse Engineering is not able to analyse its code. In this case, the applets are either replaced with alternative content provided by the application developers (such as images) or passed to the target device “as they are”, if the target browser is able to execute them. The Semantic Redesign module transforms the concrete description (specific for the source platform) to the one that refers to the target platform. The concrete deÂ�scriptions are independent of the implementation language, while the abstract deÂ� scriptions are even independent of the interaction modalities. In general, concrete descriptions assume the existence of some interaction modalities but are independent of the implementation language. At the abstract level there are, for example, concepts such as selection, edit, activate, while at the concrete level for a graphical device for example, the selection object can be refined into a list or a radio-button or a pull-down menu or other similar techniques. Such elements can be implemented
48
G. Ghiani et al.
in different languages. The abstract and concrete vocabularies contain concepts for structuring the user interface as well, such as grouping and relations. The semantic redesign transformation aims to map source concrete interface elements into ones that are more suitable for the interaction resources of the target device. The semantic redesign uses the abstract level to identify the type of interaction to support and then identify suitable, concrete refinements for them for the target platform. Thus, for example, in a desktop-to-mobile transformation the possible target concrete elements will be characterized by a more limited usage of screen space while preserving their semantics (i.e.: the effect that they have on the interactive application). The objective of the State Mapper is to update the concrete user interface for the target device (and which has been delivered by the semantic redesign module) with latest information regarding the state of the UI contained in the DOM file of the source page just before migration. After having obtained the new concrete user interface description for the target device (updated with information about the state), the Generator module builds the final user interface specified in an implementation language supported by the target device considered. The Proxy Server module plays a role whenever a browser on a client device requires access to a certain Web page. Indeed, every request to the application server is filtered by this module, which accesses the application server to obtain the page and also annotates it by including scripts, which will enable capturing the UI state. The previously described solution was not able to support partial migration because this feature implies the ability to select a subset of features and then migrate them to the target device. In order to obtain this we have again exploited the possibilities offered by the use of logical user interface descriptions. Indeed, in the MARIA language (Paternò et€al. 2009b) we use, it is possible to describe the logical structure of a user interface through interactor composition operators that indicate groups of logically connected elements or relations among such groups (e.g. a set of controls are associated with a certain form). When the user selects the migration, the migration server Orchestrator communicates with the Reverse module, which produces the Concrete User Interface (CUI) description associated to the current page and passes it to the Migration Orchestrator. If the user triggers a migration request when a subset of the components is selected in the page, then the migration is considered to be partial. In this case, the Migration Orchestrator requests a subset of the CUI from the Partial Migration module, according to the sub list of components selected by the user, and forwards it to the Semantic Redesign, thus skipping the Reverse phase (which, however, had been executed previously to create the original CUI of the entire desktop interface). The web migration support has been designed to be run as integrated with the OPEN Migration Service Platform or as a standalone support. In the following, more details are provided about the two possible modalities.
5.2.1 OPEN Platform Integrated Orchestration When the web migration support is run as part of the integrated OPEN platform, the migration orchestration is mainly provided by a dedicated module of the OPEN
5â•… User Interface Migration Based on the Use of Logical Descriptions
49
platform. Such module provides the migration support with the list of migratory components of the OPEN-aware web application. This is possible because the application had previously registered them and the Migration Orchestration had bound them with unique ids. The web migration support also notifies the platform orchestration about the components for which the migration has been requested and about the result of the migration (completed/failed). All the communications with the OPEN platform are made via XML-RPC according to the OPEN specifications. The advantage of such strategy is to exploit the features of the OPEN-awareness offered by the web application: for example, when the migration of a subset of components is completed and the application is notified by the Migration Orchestration, the components in the source device are deactivated.
5.2.2 Stand-Alone Web Migration Orchestration When run as stand-alone, the web migration is able to autonomously generate the list of migratory components. This is done by the reverse module that takes a web page and creates a logical description for it. Every page navigated via the web migration support is annotated by the migration proxy, which fills each of the main page components (such as DIV, FORM, TABLE, etc.) with a unique id (if it was not originally present). Such information is fundamental for the platform to identify the components selected by the user for the partial migration. The stand-alone web migration is then able to entirely perform the migration process. The advantage of this solution is that every valid web application can be migrated, even if it is not OPEN-aware (i.e.: even if it was not developed according to the OPEN specifications).
5.3╅An Application Example of Total Web Migration In this section we present an example of total migration involving a web site of an airline (http://www.ba.co.uk), through which the user is able to perform a search in order to get available flights for a business trip. Figure€ 5.1 shows a Web page for the considered example, visualised on a desktop device. As you can see, it is structured in different sections, in particular in the left hand side there is a form for arranging a trip. In the considered scenario, the user interacts with this site and at a certain point s/he decides to migrate to a mobile device since s/he has to leave. Then, s/he activates a migration to a mobile device. As a consequence, the current page is analysed by the OPEN Migration Platform, which will derive (through a reverse engineering step) a logical description of this page. This description will be used as input for another module that adapts it for a mobile device (this will be carried out by the Semantic Redesign module). From this new logical description, another UI specification for the mobile device will be derived, with the updated state included in it.
50
G. Ghiani et al.
Fig. 5.1↜渀 The desktop page of the airline company, considered in the total migration example
Figure€5.2 shows an example of the results of the transformations that the Semantic Redesign module performs. While in the desktop device the selection regarding the ticket type was implemented through a radio button, in the mobile device a pulldown menu has been used, since it supports the same activity while ‘consuming’ less screen space. Other modifications done by the Semantic Redesign module are, for instance, the resizing of the images, as well as the splitting of the original desktop page in more than one mobile page (if needed). As a consequence of the splitting, some additional links can be added in the split pages (see top-left part of Fig.€5.2, where a “back” link was added by this module in order to navigate back to the referring page).
5.4â•…An Application Example of Partial Web Migration In this section an example of partial Web migration is presented. The example considers a stand-alone web migration (which means that it can be carried out even on non OPEN-aware web applications). The application (called ‘Social Game’) is
5â•… User Interface Migration Based on the Use of Logical Descriptions
Fig. 5.2↜渀 The pages generated on the mobile device after total migration
51
52
G. Ghiani et al.
Fig. 5.3↜渀 The social game web application in the desktop device
a game that includes different parts: there is an IPTV in the top-left part, some additional info just beside the IPTV (e.g. live race positions), information on game positions, and a chatting area displaying the buddy list where users can connect and talk. In the bottom (left) part there is a betting area for selecting the driver to bet on as well as the desired amount, while the bottom right part displays the racing game. The application also involves interaction among multiple users (see Chap.€8). An example user interface can be seen in Fig.€5.3: The goal of the game is to finish a lap in the shortest time. Since the considered application is composed of several modules, it was judged as a good case study for assessing the features of the partial migration support. From the desktop device, the user migrates (see Fig.€5.4) both the betting area and the chat area to a mobile device (an iPhone) (partial migration desktopmobile). The interactive selection has been implemented with a mechanism that is supported directly in the concerned web page. Whenever the user clicks on a certain component or a part of the page (“onclick” event), this part of the page (or elementary component) will be automatically highlighted with a green background color within the page (see Fig.€5.4), in order to make the user aware of the selection currently done. It is worth pointing out that, in order to unselect a previously selected region/ component in the page, the user has just to click again on the same region/component, which will be removed from the list of components to be migrated. The resulting UIs on the mobile device are shown in Fig.€ 5.5. In this case, a partial migration will be done since the user interactively selects the parts of the UI that are of interest for him/her. In Fig.€5.5 you can see the resulting presentation that is displayed on the mobile device after partial migration. As you can see only the parts that have been
5â•… User Interface Migration Based on the Use of Logical Descriptions
Fig. 5.4↜渀 The selection of the chat and the betting area components
Fig. 5.5↜渀 Two migrated components displayed in the target device (iPhone)
53
54
G. Ghiani et al.
previously selected by the user are available on the mobile device, with the state preserved (you can note that all the selections that were done in the desktop component are still maintained in the mobile device after migration).
5.5â•…Usability Evaluation We conducted a usability evaluation test of our environment. The users involved were 8 male and 2 female, with average age of 30. Regarding the education level, 2 have bachelor degree, 5 had a Master degree, 3 had a PhD. All used internet desktop daily; as for the use of internet through a mobile device, 4 used it daily, 3 monthly, 1 yearly, 2 never accessed Internet through a mobile device. Furthermore, users were asked the frequency with which they played a game using a desktop device: 1 user played it on a weekly basis, 1 user reported a monthly frequency, 8 yearly. Finally, users were asked to report the frequency with which they played a game using a mobile device: 2 reported they had a monthly frequency, 4 yearly, 4 never played a game using a mobile device. The users were also asked to describe any other application/system in which they already found concepts similar to migration. The vast majority of participants (9 out of 10) answered that they never found such concepts in other systems. Only one person reported to have found some similarities in multiplayer games on desktop. Another question regarded which kinds of application the users believed the migration could be particularly useful for. One user declared that the migration could be useful especially for large web pages, and for any kind of “shareable” application (shared document editing, presentation viewing, …). More than one user mentioned the fact that business people, who generally have multiple devices, could mostly benefit from this system. Also, the migration support was found promising for communicating with friends/colleagues without interruptions. We also asked participants for quantitative evaluations, using a [1..5] scale where 1 is the worst score and 5 is the best one. Then, the user interface for selecting the target device for migration was assessed by them (Average value: 4.2; Standard deviation: 0.92). Two users pointed out that a major integration between the panel for selecting the device and the panel in which the application is shown would have been appreciated (in the tested version the panels were displayed in two different tabbed panels within the browser). Such users did not like the fact that the two interfaces were shown in two different tabs and then the relationship existing between them was not immediately evident. Another person suggested to use a photo of the device in order to let the user recognize more easily the target device for migration (this could be especially useful in case of multiple devices of the same category as in this case they would have the same icon). The user interface tested contained an icon and a name to identify each device, together with a link for accessing further information. Moreover, users had to evaluate the easiness in continuing the interaction in the target device, from the point they left in the source device (Average value: 4.1; Standard deviation: 0.73). Users said that they did not experience any problem in
5â•… User Interface Migration Based on the Use of Logical Descriptions
55
the desktop device. However, some of them pointed out that not having much familiarity with the mobile device could represent a detrimental factor regarding this aspect. However, this does not seem a big issue since people interested in migration to mobile devices are obviously familiar with such devices. One user pointed out that on mobile device the easiness could decrease because the page after migration might be very different from the original one. Connected to this aspect, another user appreciated that the migration platform was able to handle the information about the last element that received the focus, and preserve it when the resulting migrated UI was presented to the user: he said that this represents a sort of reference point within the pages resulting from the splitting in the new mobile device. Regarding the mechanism supporting the interactive selection of the UI portion to migrate, the users judged it as intuitive (Average value: 3.8; Standard deviation: 0.63). Nevertheless, several users gave recommendations for improving it. One suggestion was about the possibility for the user to enable/disable the automatic highlighting of the parts to be migrated, because some users found it a bit annoying when it occurred during normal navigation. It is worth noting that, after the evaluation exercise, we exploited this suggestion and implemented such feature and enhanced our support accordingly with the enable/disable option for the partial migration, as it can be seen in the initial part of the paper (see Fig.€5.2). All the users agreed on the usefulness of the partial migration, and thus on the possibility of selecting only the information they are really interested in. They highly appreciated its flexibility and capability in better coping with the interaction resources of devices with less capabilities than the desktop platform. Therefore, they highlighted its usefulness especially in cases when the migration is carried out from desktop to mobile. We also conducted further evaluations on more quantitative aspects of the migration. In particular, for the effectiveness of our migration platform we considered task failures (i.e.: when actions different from the ones required by the task list were executed by users). We conducted this test by directly observing the user while carrying out the activity. If we consider as 100 the number of all tasks executed by the entire user sample we obtained that no task caused the test abandonment and a very small percentage of tasks were not successfully completed (1.11%), all due to application errors. In particular, the deterioration of user experience was due to rare prototype bugs (which have since been corrected) regarding technological aspects and not to problems in the interface or presentation layer. Efficiency was the second quantitative indicator about usability related to the task execution time. This parameter was measured by recording the time that was needed for carrying out the tasks. In Fig.€5.6 the execution time distribution for the Social Game is reported. The plot considers the task list execution classes on x-axis (the classes depend on the performance time, which is reported in seconds) and the number of people that fall in each range on y-axis. Observing the distribution histogram we can see that all the data fall into two classes, which is a good result because it is a signal of the fact that different users carried out the task list more or less in a close time, and it is a positive indicator of the usability of the platform because it is perceived in the same way by different users.
56
G. Ghiani et al. Efficiency: task list distribution 7
People number
6 5 4 3 2 1 0
(0; 100)
(100; 200) (200; 350) (350; 500)
(500; 700) (700; 900) (900; +inf)
Task list execution range(s)
Fig. 5.6↜渀 Efficiency results
Number of people
Fig. 5.7↜渀 Satisfaction results
8 7 6 5 4 3 2 1 0
SUS distribution
Hard
Fairly Easy Fairly Hard Neutral Subjective rating of task difficulty
Easy
Regarding the satisfaction, we used the SUS questionnaire (Brooke 1986) to evaluate it. The System Usability Scale (SUS) approach is a simple, ten-item scale giving a global view of the experience proven by the user during the testing execution. The ten questions are standard and appears in a predefined order that alternates positive and negative question to mitigate the effects of undecided testers. A score is assigned to each answer, and a scoring algorithm is applied to obtain the overall usability value within the range of 0–100% (where 0% means “hard to use” while 100% means “easy to use”). Figure€5.7 shows the results we obtained to evaluate the user satisfaction about the partial migration experience. During this test the average SUS score obtained is 75% with a standard deviation of 11%, which is a very good result (especially if we consider the standard deviation associated to it).
5.6â•…Technical Migration Evaluation Apart from usability evaluation, some technical evaluations have been also carried out in order to understand the performance of the migration platform. In particular, a technical analysis was conducted in order to gain data on the performance of the
5â•… User Interface Migration Based on the Use of Logical Descriptions
57
overall migration platform and, down to a finer granularity, the performance of the various modules constituting the migration platform (e.g., the Proxy, the Semantic Redesign module, the Generator, etc.), in order to evaluate their impact on the overall performance. In order to do this, not only the times for migration were recorded for different web pages, but also the characteristics of the web pages themselves were analysed in order to understand to what extent some characteristics affect the different sub-modules. For instance, for the various modules of the migration platform the following parameters have been considered: number and size of CSS files, number and size of javascript files associated with the web page, number of images, size of the considered web page, number of nested nodes in the tree corresponding to the considered page, number of links existing within the page etc. Of course, some parameters are more relevant for some submodules, while other parameters are relevant for other submodules. For instance, the number of images included in the page is a parameter that is especially relevant for the Semantic Redesign module: indeed, for every image included in the page, the Semantic Redesign module has to evaluate if a resizing process is needed (if the size of the image is beyond a certain threshold), therefore, the time that the Semantic Redesign will need for this process will be proportional to this number. In addition, the number of images that have been actually resized by the Semantic Redesign is another parameter that has an impact on the performance of this module, since for each of such images the module has to perform the resizing action. For other submodules, the characteristics analysed might change. For instance, for the Proxy module, the time requested by this module on a page will be affected by the number of links that belong to the page, since the Proxy module has to change every link included in the page in such a way that all the links will pass through the Proxy.
5.7â•…Considerations and Open Issues There are some aspects that are not currently managed by the migration platform and others that highlighted the need of further improvements, especially after evaluation tests that were performed within the OPEN Project. One point is that the reverse engineering part is able to build corresponding logical descriptions from (X)HTML and JavaScript implementations. However, if the Web application contains Flash or Java applets, then this module is not able to analyse their code and correspondingly generate the logical description. Thus, in this case, such pieces of code are either replaced with alternative content provided by the application developers (such as images) or they are passed as they are to the target device, if it is able to execute them. Other aspects are connected with the preservation of the javaScript state of the web page. In some cases it revealed to be a bit hard to be managed. One example of this is the use of lexical closures (that can be found in some javaScript functions
58
G. Ghiani et al.
existing in a web page). Closures are a mechanism that is supported by javascript language and it is often exploited to deliberately hide the state of some variables (sometimes for security reasons). Due to the specificities of this mechanism, it was difficult to preserve the state enclosed in this kind of constructs. Moreover, one aspect that was highlighted as needing improvement in the migration platform was the meaningfulness of the labels of the links that are automatically added by the Semantic Redesign module when a web page splitting is carried out. For the moment, the labels for such links are built by directly getting them from the element names/identificators that are included in the original page for those elements. More intelligent techniques (e.g. like the analysis of additional data like for instance the words contained in the referring page) should aim to identify more meaningful names for such links in order to enable the user to better understand which content the linked page will actually contain. Other aspects that deserve further attention in the near future are those related to privacy and security in such migration environments.
5.8â•…Conclusions In this chapter, we presented our solution for supporting both total and partial migration of Web applications from desktop to mobile systems. Examples of web application were described to show how we support the possibility of totally migrating a page, or migrating a subset of the elements displayed and running in the source device. In the latter case users can directly select what parts should be migrated from the original Web page in an easy and intuitive manner. We also presented the results that we gained during an user evaluation where specific aspects were assessed (usability, effectiveness, efficiency, …).
References Brooke, J.: SUS—A quick and dirty usability scale. http://usabilitynet.net/trump/documents/ Suschapt.doc (1986) Edwards, W.K., Newman, M.W., Sedivy, J.Z., Smith, T.S.: Experiences with recombinant computing: exploring ad hoc interoperability in evolving digital networks. ACM Trans. Comput. Hum. Interact. 16(1), 1–44, Article 3 (2009) Eisenstein, J., Vanderdonckt, J., Puerta, A.R.: Applying model-based techniques to the development of UIs for mobile computers. In: Lester, J. (ed.) Proceedings of 5th ACM International Conference on Intelligent User Interfaces IUI 2001, pp.€69–76 (14–17 Jan 2001), ACM Press, New York (2001) Melchior, J., Grolaux, D., Vanderdonckt, J., Van Roy, P.: A toolkit for peer-to-peer distributed user interfaces: Concepts, implementation, and applications, EICS’09, pp.€69–78, Pittsburgh, 15–17 July 2009 Paternò, F., Santoro, C., Scorcia, A.: Ambient intelligence for supporting task continuity across multiple devices and implementation languages. Comput. J. 53(8), 1210–1228 (2009a)
5â•… User Interface Migration Based on the Use of Logical Descriptions
59
Paternò, F., Santoro, C., Spano, L.D.: MARIA: A universal language for service-oriented applications in ubiquitous environment. ACM Trans. Comput. Hum. Interact. 16(4), 19:1–19:30 Nov (2009b) Pierce, J.S., Nichols, J.: An infrastructure for extending applications’ user experiences across multiple personal devices. In: Proceedings of UIST 2008, pp.€101–110, Monterey, 19–22 Oct 2008
Chapter 6
Service Migration Network Support Rasmus Olsen, Kim Højgaard-Hansen, Anders Nickelsen, Huan Cong Nguyen, Miquel Martin, Carmen Santoro, Björn Schindler and Simone Mazzei
6.1â•…Network and Deployment Scenarios For the migration process a set of common operations and functionality can be identified. These are located in the Migration Service Platform in order to be able to reuse code, hereby reducing the coding effort required by application developers. These functions in particular focuses on networking issues and management of distributed information. In the following part of this chapter we elaborate these common functions, namely Mobility Support, Context Management, Migration Orchestration and Trigger Management functionalities.
6.2╅Network Domain and Entities The OPEN Network domain is separated from the application server-client domain, such that the interaction between the Application server and the OPEN domain remains unchanged, and that the OPEN architecture introduced in Chap.€ 4, does not make any assumption on the interaction with the application server. Figure€6.1 shows the main entities involved in the support of the migration process. It can be seen in Fig.€6.1 that there are three major OPEN aware entities, and only the application server is not assumed to be changed in any way. This keeps deployment of existing applications using server-client architecture simple, since only the client may need to be adapted to be integrated with the Migration Service Platform. In Table€6.1 an overview of the entity definitions are listed. The important characteristics of the platform is that the platform in a general use situation, it appears transparent to the user, until that user or an external event triggers a migration. Following the trigger the platform suspends the current ongoing activity and overtakes control of the involved clients inside the OPEN domain, R. Olsen () Aalborg University, Aalborg, Denmark e-mail:
[email protected] F. Paternò (ed.), Migratory Interactive Applications for Ubiquitous Environments, Human-Computer Interaction Series, DOI 10.1007/978-0-85729-250-6_6, ©Â€Springer-Verlag London Limited 2011
61
62
R. Olsen et al.
Fig. 6.1↜渀 OPEN network domain and major entities considered Table 6.1↜渀 Definition of OPEN entities and their role in the migration process OPEN Client Clients execute the applications. Applications executed on the clients can be OPEN Platform-aware or not aware OPEN Server An OPEN specific network element that supports the migration between clients OPEN Mobility An OPEN specific network element that ensures data streams at transport Anchor Point layer are directed to the correct device, transparent to the application server Application Any type of application server that provides services to client applications Server
whereafter it shifts the suspended application execution from one device to another and once ready, the platform releases the application activity for continued user interaction. From the user perspective, some key requirements to the performance of the network support platform exist, namely: • • • •
low migration delay as to interrupt the user as least as possible low network overhead in order to reduce communication and energy cost security and trust in that the process only involves the right and trusted devices autonomous migration trigger mechanism in order to be able to detect and trigger migrations based on potential issues not visible to users e.g. congested network. • low failure rates as any migration failure will at best annoy the user, and at worst lead to loss of data in the application
6.3â•…Deployment Scenarios The deployment of the OPEN platform is not entirely independent of the network scenario in which it is working. The focus and main characteristics of the OPEN platform is that the major network entities, the Mobility Anchor Point and OPEN
6â•… Service Migration Network Support
63
Server have fixed locations in the network and are available to the clients at all times. This is not the case in ad-hoc networks, for example where connectivity between the key entities is not always guaranteed, in enterprise networks where firewalls may become a challenge or in home networks were NAT problems have to be addressed in order to perform the operations necessary to carry out the migration process. As a part of the network support, the Migration Service Platform is also responsible for reconfiguration of the network e.g. by changing connection as to meet application requirements if needed.
6.4╅Overview of the Network Support A full overview of all modules for the Migration Service Platform, and their respective location among the various network entities are shown in Fig.€6.2. The following sections describe the key components within the Migration Service Platform, namely the Migration Orchestration, the Trigger Management, the Mobility Support and the Context Management. The remaining modules can be seen as auxiliary modules which are not directly needed for the migration process but supports the key components in some other way.
Fig. 6.2↜渀 Overview of modules in the various OPEN network entities
64
R. Olsen et al.
6.5╅Migration Orchestration and Orchestration Procedure The migration process is supported by a number of core components, which are illustrated in Fig.€6.3. On the right, a Migration Orchestration module is responsible for executing the migration process in a timely manner, e.g. starting, stopping and pausing applications, ensuring the state transfer is initiated at the right time between the right devices and so on. Hence, this module determines effectively how to migrate an application. In the middle, the module Trigger Management is responsible for the evaluation of the given circumstances with respect to possible migration objectives. Based on context information that describes the situation of involved objects and entities, the Trigger Management will issue a trigger to the Migration Orchestrator when it is time to migrate, i.e. in effect this block determines when and where to migrate. On the left, the Context Management Framework provides the necessary context information that allows the Trigger Management to make decisions about when to migrate. Context information is usually distributed and dynamic in nature, so a dedicated module that ensures access to any required information is needed. Hence, this module responds to the question what and how information used to make decisions upon the migration process is answered. Underlying those components, the mobility support works to ensure that the platform and the applications does not run into connectivity problems, before, during and after migration, and enables the Migration Orchestration to reconfigure the network as needed. Figure€6.4 shows a generic overview of how the OPEN entities interact with the client devices and the users of the OPEN platform. The interaction diagram assumes all entities are familiar with the IP address of the Migration Server, e.g. via DNS lookup, service discovery or a static configuration, and is only showing a very general and high level case whereas some application may require more or less interaction. Once the OPEN Server is started, it initializes all the internal modules, after which the OPEN Mobility Anchor Point can be started and registered at the OPEN
Policy Enforcement External context information
Context Management Framework
Authorize trigger Context information: Devices Connectivity/QoS properties Network context information
Trigger Management
Trigger/configurations: Target device(s) Application configuration
Application component configuration options Device capabilities Network performance/QoS measurements
Device Discovery
Devices
Application Logic Reconfiguration
Migration Orchestration
Application component configuration
Device configuration
Performance Monitoring
Fig. 6.3↜渀 Module overview of key blocks for supporting migratory services
6â•… Service Migration Network Support Device A Source
Device B Target
65 Mobility Anchor Point
OPEN Server
App. server
Registration Registration Launch app
Connect
Context Update Context Update Prepare Migration Get state of app Reconfiguration Set app. state Resume application Connect End migration
Disconnect Disconnect
Fig. 6.4↜渀 Generic overview of migration process
Server. Clients will now need to register information regarding device capabilities, OPEN enabled applications and their respective modules (if any) etc., which later will be used to identify capable migration candidates. When the application starts, a startup event will be pushed from the source client to the OPEN Server. This will make the OPEN Server start evaluating contexts of the different registered applications on different devices in order to decide if there are any opportunities for a better user experience regarding the application usage. If another OPEN Client applications on different devices becomes available for migration and the OPEN Server decides that migration is beneficial to the user, the OPEN Server will make the target device prepare for migration. This includes fetching the required application if it is not already present on the device, establishing a connection to the OPEN Mobility Anchor Point, and registering the application and components with the OPEN Server. Following this step, the OPEN Server will extract the application state from the source client device and transfer it to the application on the target device after which the application can be resumed on the target client device. The migration process can happen multiple times with multiple client devices and multiple users involved. At some point in time the user and the client devices can choose to leave the OPEN Platform by unregistering. Notice that the migration process may not always follow these exact steps. For example the state transfer may not be necessary or be so complex that it involves several activities, or the target device may not be visible for the OPEN Server before much later than shown in Fig.€6.4, whereas it makes no sense to do the registry at the time shown in the figure, but only when it becomes visible. In effect, the migration actually relates to the configuration of the system within the OPEN domain, so for example, a conversation between two people can be ex-
66
R. Olsen et al. Workstation Smart Phone
Workstation Smart Phone
Video
1
0
Video
0
0
Audio
1
0
Audio
0
1
Configuration 1
Configuration 2
Fig. 6.5↜渀 Example of configuration of a system. A change in configuration means a migration executed by the OPEN server
ecuted with both an audio and video stream or with an audio stream only, depending on the device capabilities. How the application is being executed we refer to as configuration, and it is up to the OPEN Server, by all its knowledge to decide upon which configuration the application should use (see Fig.€6.5). The configuration that has been decide by the OPEN Server, is then executed by whichever means possible by the OPEN Server, potentially involving the Mobility Anchor Point which assist keeping a session going while doing the change of configuration. In some cases, e.g. if the notion of a session is not really meaningful, this may not be needed, while in other cases it may be vital. Relating to the respective components, the Context Management Framework provides the necessary information to build these configurations in conjunction with the Application Logic Reconfiguration, while the Trigger Management determines when to change configuration and the Migration Orchestration decides how to execute the configuration change which potential involves the Application Logic Reconfiguration.
6.6╅Context Management Context information describes the situation of any entity and constitutes the information that is relevant to the entity itself and its interaction with the environment (Dey 2000). For the migration process, context information is a key indicator on when and how to migrate applications and services. Which information depends on the entity and its activity, whereas getting the information is a process that reuses a set of common functionality which are discovery and access to dynamic information distributed in the environment and accessed via a network infrastructure, being either wired, wireless or both. The common functionalities are search, discovery, access, distribution, caching. Thus context management relieves the application developer from these activities and allows him to focus on the usage of context in�formation (Bauer et€al. 2006). The Context Management Framework (CMF) originates from the Ist project MAGNET Beyond (MAGNET 2010) and was developed in Java. In OPEN, this framework even has been further developed to accommodate the diverse environment that the CMF is required to operate in.
6â•… Service Migration Network Support
67
6.7╅Internal Structure, Architecture and Interaction 6.7.1 Internal Structure and Architecture The major entity in the framework is the Context Agent, which implements the necessary functionality to ensure efficient gathering and distribution of context information. The framework assumes a context agent is running on each node within the operational domain. All agents have to be connected to an agent with the special role of knowing what context information is available within its network domain. We call this context agent for the Context Management Node (CMN). All context agents register their local information to this node type, and may use it to search for context information within the network domain. Further, the Context Management Node can connect to other Context Management Nodes via an overlay network of other Context Management Nodes, which allows a scalable setup of context agents. The Context Management Node can either be set via configuration files, or selected via a master selection algorithm. In OPEN the context agent running on the OPEN Server is a natural choice of being the Context Management Node, since all nodes involved in a migration are already aware of the OPEN Server network location, and therefore there is no need to decide upon a Context Manager. Thus, the selection of the Context Manager as the OPEN Server reduces needs for detection and election mechanisms for the Context Management Framework. Details on internal of the framework works are described in (MAGNET 2010; MAGNET D2.3.1; MAGNET D2.3.2). Figure€6.6 gives an overview of the internal components of a Context Agent. In the middle of the Context Agent, the Context Access Manager (CAM) is responsible for constructing and maintaining a list of information locally available for the Context Agent. This list also includes information of what information is available in the given network. On the right, the Context Aware Security Manager (CASM) ensures that all input/output is being checked with respect to privacy and security requirements, e.g. requests for location information may have associated privacy policies that may need to be enforced at the CASM. The Communication Module (NetCOM) ensures that internal data objects are (de)serialized and transported effectively between Context Agents. The Context Management Interface (CMI) ensures the interface between the internal components and the external client applications and services that use the Context Agent. This interface is defined by the so called Context Access LAnguage (CALA) which is an XML formatted query language (MAGNET D2.3.1). Then, there is the Context Agent Controller (CAC), which basically ensures correct configuration of the internal components via dedicated configuration files. Finally, there are the components Data Source Abstraction Manager (DSAM) and Processing Manager (PM) which maintain and control pluggable modules called Retrieves and Processing units. Retrievers are small software components which create the interfaces between some measurable information, e.g. ambient temperature, noise level, OS status, or network conditions and the internal of the Context Agent. Processing Units do not themselves interact with data sources in the same way as Retrievers do, but are still producing and offering information
68
R. Olsen et al. Queries
Responses
Subscriptions Notifications Modifications Context Agent
Context Management Interface (CMI) Context Agent Controller (CAC)
Storage Manager File
Memory
Processing Manager Processing Unit
Processing Unit
Data Source Abstraction Layer
Context Aware Security Manager (CASM)
Context Access Manager (CAM)
Communication Module (NetCom)
Management Interface (to other nodes and PN Agent) Information Interface (to other nodes and gateways)
DSA Manager Retriever
Retriever
Retriever
Retriever
Data Source (Sensors)
Data Source (OS Status)
Data Source (PHY/MAC Parameters)
Data Source (...)
Fig. 6.6↜渀 Overview of internal components of a context agent, which are running on each node within the OPEN network domain
via inference, logic, or other heuristic algorithms, and base their produced context information on already existing information in the framework. To get required information the Processing Units are requesting or subscribing to context information in the (nearly) same way as any other applications and services (the main difference is that Processing Units uses internal java objects and methods for communications, while applications and services uses the XML-RPC interface). Once they have the information needed, they execute which ever heuristic algorithm they implement in order to produce new types of context information. As a part of the processing component, there is also a Storage, which offers a simple database functionality to the applications, services and Processing Units that may need it.
6.7.2 Interaction with the Context Management Framework From the application point of view, the CMF provides a single interface, that is, an XML-RPC implementation of the CMI supporting the Context Access Language (CALA) (MAGNET D2.3.2). Both reactive as well as proactive subscription based access methods are supported, and for the latter both periodic update or event driven updates are supported if needed. The different methods of accessing context are done by different CALA type queries which include potentially settings, e.g. the update time interval for a periodic update. At this level, the workings of a context agent can be summarized as shown in Fig.€6.7. Not only can this interface be used to retrieve the information stored at the chosen Context Agent, but also in any other agent in the same CMF deployment. This
6â•… Service Migration Network Support
69
Application interface
Processing Units
Context Agent Core
Storage Component
Data Source Abstraction Layer
Sensor Wrappers (Retriever)
Hardware Sensors
Fig. 6.7↜渀 Internal interaction between various components inside the context agent
is due to the internal distribution of information availability in the Context Management Framework (MAGNET D2.3.2). When application requests information, the Retrievers, Processing Units and Storage of every Context Agent in the network are called upon to retrieve the needed data. Scalability issues are solved by exchanging the available data indexes among Context Management Nodes and thus preventing the activation of unnecessary Agents (see Fig.€6.8). A key feature with the interaction of the XML-RPC queries for the application, is that it is possible to scope queries via specific CALA formatted queries (MAGNET D2.3.2), so that applications have some control of where queries are focused (e.g. node local, same network domain, overlay network, in time and/or space). The above is merely a brief overview of the roles and functionalities of the internal components in the Context Management Framework, to get the full extent of the interaction and workings of the framework, see (MAGNET D2.3.1) and (MAGNET D2.3.2).
6.7.3 Adapting the Context Management Framework The mobility in the OPEN scenarios calls for the dynamic addition and removal of sensors. When an application is migrated, the devices in the new environment have to be taken into consideration: new sensors might be available which enrich the user interaction with the application or define its behavior. Likewise the user might
70
R. Olsen et al.
Queries And Subscriptions
Application interface
Processing Units
Context Agent Core
Storage Component
Data Source Abstraction Layer Sensor Wrappers (Retrievers)
Hardware Sensors
Fig. 6.8↜渀 Query and subscription distribution among the context agents within the OPEN network domain
unplug the keyboard when he takes a device away, or might walk off the line-ofsight of the display she/he was using. All these events can be monitored by the CMF, which would then work together with the Device Discovery module, or with other applications that require context information, like the Trigger Management. It is therefore clear that supporting the addition and removal of devices and sensors provides for greater adaptability and a tailored, as small as possible deployment at all times. Additionally, being able to reconfigure context agents, and the knowledge of the environment where they run, provide support for advanced features, such as running processing tasks in the most powerful or less battery-dependent devices, or perform operations that require high amounts of data as close to the data source as possible, thus minimizing bandwidth usage and therefore both cost and battery. Further, the argument of having multiple java component and applications running on only one Java Virtual Machine instead of one each is also a part of the considerations. For these reasons the existing framework is not sufficient to address these dynamic situations as envisioned, hence it was decided to shift the framework to an OSGi environment (OSGi) to utilize its aspects and functionalities in this framework to overcome the reconfiguration problem. In the following we describe how this is done and the perspectives of using OSGi. OSGi allows a flexible control of services, i.e. installing, running, stopping and uninstalling them at runtime which in turns gives additional possibilities for controlling active components as retrievers and/or processing units. This provides the building blocks for a support system that allows a dynamic component lifecycle management which was not possible in earlier version of the context management
6â•… Service Migration Network Support
71
system. Each component in a Context Agent is a service in the OSGi framework, including any processing units and retrievers that may be used. Each of them is marked with a version number, which enables the system to check if the required version of a given component is fulfilling requirements of other components or the system itself. It also allows, if checked against a repository to see if the newest or a specific version is being used, or updates are required. This may be needed for components focusing on specific platforms, OS, or specific driver/hardware. For example a storage component/service in a mobile terminal would most likely require a different version than one on a laptop, or a retriever that fetches data on a specific sensor interface. This concerns the core components, as well as the more dynamic components as retrievers and processing units. Figure€6.9 shows an example of how a context management support architecture would look like. In the infrastructure, e.g. the Internet, repositories of retriever and processing components are located from which Context Management Nodes may become aware of their availability. The repositories can be maintained by business entities offering context information, for example operators or small businesses. Once the key components of the Context Agent are running, needed retrievers and processing units can be downloaded and installed from the repositories known
Fig. 6.9↜渀 Outline of a support system for automatic configuration of context management framework with remote repositories of retrievers and processing units
72
R. Olsen et al.
a priori to the Context Management Node. Thereafter, they can be activated and the Context Agent can now offer the given context information. In this way, the Context Management Framework is able to provide information as requested by the platform and application, and if needed, reconfigure itself without further actions from the user, easing the use of the platform.
6.8â•…Trigger Detection and Management The purpose of the Trigger Management (TM) module is to decide the configuration of the migratory system and its applications in order to fulfill the requirements of the user and the applications. If changes are needed in the way applications are running or deployed, a migration is performed. To decide upon a configuration, TM collects information about possible configurations of applications, devices and networks. These configurations constitute the options to choose among. The decision is made by comparing the requirements of the options to the capabilities of the platform and the environment—i.e. the context of the configuration/applications. Figure€6.10 depicts the external modules necessary for TM to collect the described information. The knowledge about the requirements and context has different origins, e.g. • available configurations and requirements come from application logic reconfiguration. • device capabilities come from device discovery • network capabilities come from device discovery and performance monitoring • user preferences may be defined externally e.g. Default Delivery Context (2008) from W3C which specifies minimum required environment for experiencing the web. Every piece of context information going into TM is formatted as XML in a CMFobject. The TM decides which configuration to choose based on a set of configurations made available by the ALR (see Chap. 7). The ALR collects information from the registered device about which application components are registered, and decides which combinations of components—constituting applications—are viable in cerPolicy Enforcement External context information
Context Management Framework
Authorize trigger Context information: Devices Connectivity/QoS properties Network context information
Trigger Management
Trigger/configurations: Target device(s) Application configuration
Application component configuration options Device capabilities Network performance/QoS measurements
Device Discovery
Devices
Application Logic Reconfiguration
Migration Orchestration
Application component configuration Device configuration
Performance Monitoring
Fig. 6.10↜渀 Overview of external modules necessary for trigger management to carry out its task
6â•… Service Migration Network Support
73 Migration server Trigger management
Application Device
Source device
Trigger generator
Application Score generator
Device
Score aggregator
Capabilities
CMF retriever
CMF retriever
Performance monitoring
Performance monitoring
Network
Network
’closeBy’-score
’closeBy’-plugin
CA
Network properties Device capabilities
Score functions
CA
Application logic reconfiguration
Capabilities
Target device
Context notification: ’<source>closeBy’
Context Agent (CA)
Fig. 6.11↜渀 Overview of internal modules in trigger management and how they interact with the context management framework to collect network context information and ALR to have configuration options and component requirements
tain settings. A configuration is a set of application components on a set of devices, which is determined able to run by the ALR module. Deciding which configuration to use is done either manually based on the user input or automatically based on observations of contextual information. Figure€6.11 illustrates the internals of the TM module. The context information and the application configurations are input to the system mainly on the end terminals (some context information may come directly from system components) and the application configurations are provided by the ALR client-side components of the ALR framework as described in Chap.€5. The later sections describe the use of scores in order to rank configurations to make a decision; the placement of these score functions is also illustrated as an internal module on the server-side of the TM. TM does not have client-side components in this solution, as it is expected that all input from the client-side is communicated from the input suppliers (manual trigger from applications or migration client, or context providers) through the CMF. Output from TM is sent to the Migration Orchestrator for execution in the form of a configuration in the same format as received from the ALR.
6.8.1 Manual Migration Triggering Migration triggers can be generated manually by the user in the application or in the migration client. The manual form of triggering is for the most part based on by-passing trigger management, since manual trigger is interpreted as an intentional action from the user side. Another situation where the user is involved is when the TM module automatically generates triggers, as described in the following section. In this case, the user may
74
R. Olsen et al.
be asked to accept or reject a certain migration. If the user does not want to be asked each and every time a trigger is generated, authorization policies can be specified. Such policies can be managed in the context management system and the decision about a certain trigger could be followed by a request to remember the decision on the trigger, which means that in the future, the user would not be consulted when the particular trigger is generated. This could also be more expressive in the sense that the policy could be on trigger types, devices, device types or alike instead of individual triggers. This would be specified in the same semantic language as the configurations from the ALR, since a migration means to change from one configuration to another, and the question asked to the user regards whether these individual transitions (/migrations) are acceptable.
6.8.2 Automatic Migration Triggering The TM can make its decision on which configuration to use in several ways, characterized by level of complexity and how much information is used to make the decision. In the following, two methods are described; a score-based approach that has a low complexity but is not able to incorporate much information, and a Markov Decision Process (MDP)-based approach that has a higher complexity because it uses more information. In general, the decision making mechanism in the TM relates observed context information to requirements of the configurations of the applications to migrate. The TM chooses the configuration that has the best fit to the observed context, and if this configuration is another than the one currently running, a migration trigger is issued, with the selected configuration as target. The context information is collected via the CMF and provided by various monitoring components in the OPEN framework. In this chapter, the performance monitoring component is heavily used as context information provider, but context information could be provided by other components such as Device Discovery or the interface to service enablers in the network. The configurations, and thus requirements, are provided by the ALR.
6.8.3 Score-Based Trigger Decision Approach In the score-based approach, a score is calculated using a utility function for each configuration based on all its requirements and the state of the required parameters at a given time. The utility function assigns a value to the user perceived application quality of the requirement being met by the system, and the extent to which it is met. The system is represented by the observed context information. The quality is represented by any quality-of-experience parameter, for instance Perceptual Evaluation of
6â•… Service Migration Network Support
5 MOS
Fig. 6.12↜渀 Examples of utility functions mapping system parameters to user perceived quality (MOS). Packet loss rate in the network or delay are examples of parameters
75
0 System parameter
Speech Quality (PESQ) for speech applications or Perceptual Evaluation of Video Quality (PEVQ) or Peak Signal-to-Noise ratio (PSNR) for video applications. The utilities are all represented in the same scale using mean opinion score (MOS) values so every quality-of-experience parameter must be mapped into MOS values, if they are not already so. The MOS scale represents the user perceived quality of a given application in a scale from 1 to 5, 1 being very bad and 5 being excellent. As the quality is application specific, the mapping is performed by the application developer, and thus general to the migration framework. An example of mapping between PSNR and MOS is described in (Klaue et€al. 2003). The final utility functions taking system parameters as input and giving quality as output are also specified by the application developers. These functions can be simple binary expressions (>, ╛U(LQ), then we use HQ, otherwise we use LQ. Here hysteresis can be applied to avoid transitions too often, which could also disturb the user unnecessary. If we define the threshold from the utility functions governing HQ and LQ, we obtain a threshold value where the two graphs intersect, around 0.1. This threshold defines the two states N ( device = = BigScreenSize
Directionlf
IntelligentGhostLogic > logic = =“easy”
ordered Small Screen Config DirectionIf
device = = SmallScreenSize
> EasyGhostLogic
ghost logic component consists of further components which provide easy and intelligent ghost logic. The reconfiguration of the ghost logic component is done by the Open Adaptors (e.g. Context Agent see Chap.€4), which are connected to the Open Server Components. One of the Open Server Components is the Application Logic Reconfiguration (ALR) described in Chap.€7, which computes the best configuration out of the ALR view. At the PacMan Example a configuration of the application logic consists of the easy or intelligent ghost logic component. The specification of the application logic part (see Chap.€7) of the PacMan game is built up like depicted in Fig.€10.5. It consists of two configurations: 1. Big Screen Config 2. Small Screen Config The activation condition of the first configuration is a big screen size of the device. To activate the second configuration the device must have a small screen size. Both configurations contain a Template, described in Chap.€7, for a component, which has the following structural constraint: Each component which fits to this Template has to provide the interface DirectionIf. To fit to one of the Templates a component has also to fulfil the functional constraint of DirectionIf: logic equals ‘intelligent’ or logic equals ‘easy’. The component specification (see Chap.€7) of the ghost logic components is depicted Fig.€10.6. It contains a description of the following components: 1. IntelligentGhostLogic Component 2. EasyGhostLogic Component
154 Fig. 10.6↜渀 Component specification and application specification
G. Ghiani et al. Ghost Logic Component Specification IntelligentGhostLogic Component Provided Interface: DirectionIf Functional Property: logic = intelligent EasyGhostLogic Component Provided Interface: DirectionIf Functional Property: logic = easy
The two ghost logic components provide DirectionIf and an associated property: logicâ•›=â•›intelligent or logicâ•›=â•›easy. The Component 1 fits to the Template 1 and Component 2 fits to Template 2 of the application logic specification. At the reconfiguration process the ALR can switch between the possible configurations depending on the context information (in this case “device”). Component 1 is used at devices with a big screen. For small screens Component 2 is activated.
10.5╅User Interface Migration In the previous sections we analysed how the ALR adapts the application logic in order to cope with the changed conditions of the context (change of device) during a migration. Now we will focus on how the UI Migration part of the OPEN Migration Platform works in reaction to the same context change and considering the same application. So, we will focus on the type of UI adaptation the UI Migration module carries out after migrating the PacMan game from desktop to mobile. The PacMan game considered has been developed in XHTML1 and JavaScript (Flanagan 2006). Basically, the XHTML code specifies the presentation part of the application, while the JavaScript code handles the logic of the game. On the desktop platform an implementation of the game based on XHTML 1.0 has been used, while the version of the PacMan for the mobile device has been implemented in XHTML Mobile Profile2 language. The considered migration scenario is a desktop-to-mobile where a user starts to interact with the desktop version of the PacMan and then moves to a mobile device. After playing for a while the game on the desktop platform, the user decides to migrate towards a mobile device. Therefore, the user has to select, through the Migration Client, the platform towards which the migration will be activated, among the available devices (in our case a mobile device). After doing this, the application will be migrated to the new platform. As it is possible to see from Fig.€10.7, the desktop UI of the game includes different parts: 1╇ 2╇
http://www.w3.org/TR/xhtml1/ www.openmobilealliance.org/…/wap-277-xhtmlmp-20011029-a.pdf
10â•… Integration of User Interface Migration and Application Logic Reconfiguration
155
Fig. 10.7↜渀 The original desktop version of the PacMan
• the maze of the game with the different characters (the ghosts and the PacMan); • a part devoted to visualising the current state of the game (the number of PacMan lives still available, the current level of the game, the score), the ‘New Game’ button to start a new game, and the interaction elements for controlling the game; • the settings for specifying the general configuration of the game: the animation speed and the smoothness of the animation; • the possibility of using random maze layouts for the various levels, etc. Indeed, in the desktop version, the PacMan can be controlled either using the keyboard (e.g. arrow keys can be used to specify directions to steer the PacMan) or by using an imagemap (in Fig.€10.7 it is visualised in the bottom-right part just beside the maze). An imagemap is an image with clickable areas. In our case we have five of them: four are associated to the directions (north/south/east/west) for steering the PacMan, the remaining one (in the centre) is for pausing the game. Then, the user can click a specific region of the image or even pass the mouse over such a region (“onMouseOver” event) in order to activate a function allowing to set the new direction that the PacMan should take. The version of the PacMan game for the desktop platform is transformed by the migratory platform in order to obtain a new version adapted for the mobile device.
156
G. Ghiani et al.
In order to do this, a reverse engineering of the PacMan application for the web desktop platform is carried out by the OPEN Migration Platform in order to obtain its logical UI description, which will then be redesigned to finally generate the adapted user interface for the mobile device. Regarding the mobile device considered for migration in this example, it is worth pointing out that, due to the high variability of characteristics of mobile devices, we judged useful to distinguish two different categories among them. One category covers “traditional” mobile devices while another category covers those belonging to what we called“ iPhone-like” platform. With this term we mean devices that share the same characteristics of the Apple iPhone (e.g. Samsung Omnia, HTC Touch,…), e.g. they have a quite large screen size (e.g. starting from 480â•›×â•›320 pixel, to 960â•›×â•›640 pixels of the most recent ones), (multi-)touch based interaction, sensors such as accelerometers. For the example, we considered a device belonging to the “iPhone-like” platform (the Apple iPhone itself). When migrating to the iPhone, appropriate page splitting is performed. On the one hand, all the controls for the general configuration of the game are grouped together into a newly generated presentation which is accessed by activating the “Settings” button (see Fig.€10.8) in the first presentation. On the other hand, given the wide (relatively to mobile devices) screen available, and the high interactivity of the PacMan game, the design should not force the user to use zooming/scrolling during the game, therefore in this case the maze should fit entirely in the iPhone screen, without forcing the user to scroll the page for visualizing parts of the maze (see Fig.€10.8). The adaptation takes into account that touch-based interaction will be performed, thus the selectable elements should be large enough (or distant enough each other) to be touchable without creating confusion regarding what the selected element is. Therefore, when adapting the image map supporting the game controls to the iPhone platform, it will be ensured that the adaptation engine will provide controls that are large enough to support an easy-to-tap interaction. In addition, in the desktop version there were some actions activated by “onMouseOver” events (e.g. for moving the PacMan), which are not supported on the iPhone because there is no direct equivalent of this event on the iPhone (as well as other touch-based devices), then in the adapted version the onMouseOver events have been removed and the interaction is made more explicit (by tapping on the corresponding element). Moreover, in the desktop version of the game there were two possibilities for controlling the game (through the keyboard and also by using the imagemap). Since a touch-screen mobile device is used as migration target, the keyboard controls are considered by the adaptation engine not suitable for this type of platform and they are removed from the mobile version. Indeed, the iPhone does not have a physical keyboard, and triggering the appearance of the virtual keyboard would clutter the screen and obscure some parts of the user interface (e.g.: the maze), which is not very usable for the player who is currently using the mobile device. Therefore, in this case, the support for controlling the PacMan is obtained through touch-based interactions with the five buttons that have been made available on the iPhone platform (see Fig.€10.8) and that allow the user to control the game. Such buttons are
10â•… Integration of User Interface Migration and Application Logic Reconfiguration
157
Fig. 10.8↜渀 The adapted iPhone version
automatically derived from the original image_map of the desktop version, by selecting the different areas assigned by the image_map and mapping them to a set of buttons.
10.6â•…State Persistence Regarding the state of the game, it is preserved and then re-activated in the mobile device at the point it was when the migration was activated. Thus, the current PacMan and ghost positions are saved, together with the current level and score and the other settings that the player already selected in the desktop version, and the player can continue from this point when she accesses the iPhone version.
158
G. Ghiani et al.
More specifically, in this example, most of the state of the PacMan game is maintained in a number of global JavaScript variables and objects that are manipulated by functions declared within the 3 tag of the concerned page. In the PacMan example considered, the global JavaScript variables contain the information about the positions of ghost and PacMan, the dots already eaten, the current level and lifes still left, so they provide the information needed for restoring the state of the game and then allow the player to continue the game from the point s/he left off in the source device. However, it is worth pointing out that, since the migration transition is not performed instantaneously, the game has to be paused as soon as the migration is triggered, to let the migration transition take place. In this way, during the migration transition we prevent any further modification to the state of the application. Without pausing the game, it would enter in an inconsistent state, since the user would continue to interact with the source device, while the platform will not actually preserve the latest state (which is what the user would actually expect). This might cause confusion in the user, which should be avoided. In order to preserve the state of the web PacMan application a key role is played by the Window object, which represents the Web browser window and also defines the program namespace, since every JavaScript expression implicitly refers to the current window. Then, every object/variable defined in the global environment of a JavaScript code is simply a property of the global Window object. This is exploited for capturing the state of global JavaScript variables, by means of using the for… in4 statement, which provides a way to look at the (enumerable) properties of an object without knowing in advance the names of such properties. In this way it is then possible to capture the values assumed by the user-defined global JavaScript variables manipulated in the game. This is done by a method (↜saveState()) code summarised below: saveState(){ var globalJSONsâ•›=â•›{}; for (var prop in window) {
}
JSONobjectâ•›=â•›dojox.json.ref.toJson(window[prop], true); globalJSONs[prop]â•›=â•›JSONobject; … } return JSONobject;
(i)
As can be seen from the (i) code above, every property of the window object is saved in a JSON5 object. This object is a string, used to update the corresponding element in the global associative array (↜globalJSONs), which stores all the values of the JavaScript 3╇ The head element is a container that can include scripts, instruction for the browser where to find stylesheets, meta information, etc. 4╇ for (variable in object) {code to be executed} 5╇ http://www.json.org/
10â•… Integration of User Interface Migration and Application Logic Reconfiguration
159
global variables. In order to translate every global variable into a JSON string, a specific library was used (dojox.json.ref6). Within this library, the toJson function creates a JSON serialization of every property of the window object, and therefore it produces an array of couples , for each JavaScript global variable included by the programmer. This information contained in the JSON object is passed, through an AJAX (Holdener 2008) request, to a Java servlet7 (contained in the migration server) which is in charge of restoring the state in the page that will be uploaded in the target device. In order to do this, the servlet identifies the 8 element of the page and appends to the current content (if any) of the onload attribute (which occurs when an object has been loaded) of the element a new JavaScript function (↜restoreState()). The goal of this function is to update the JavaScript global variables contained in the page, by exploiting the serialization of the JavaScript global variables obtained in the previous steps). The key code of the restoreState() function is described below (ii): restoreState () { var globalJSONs╛=╛dojox.json.ref.fromJson(JSONstate);╛//╛parsing JSON state for (name in globalJSONs) { try{ value╛=╛dojox.json.ref.fromJson(globalJSONs[name]); win[name]╛=╛value; } catch.. {} } (ii) The restoreState() exploits the fromJson method provided by the dojox library dojox.json.ref mentioned before. This method enables to de-serialize the state contained in the globalJSON in order to update the page to be loaded on the target device. Afterwards, the servlet stores such updated page (which now contains the up-to-date state) in a specific location of the server and builds correspondingly the Universal Resource Locator (URL) from which the browser in the target device should load the page with the updated state.
10.7â•…Integration of the User Interface Migration and Application Logic Reconfiguration For the PacMan prototype several Open Clients (see Chap.€4) are used. The first Open Client is for the mobile device, the second Open Client is for the desktop device. The PacMan Game is running in a browser on the desktop and the mobile http://www.dojotoolkit.org/reference-guide/dojox/index.html http://java.sun.com/j2ee/tutorial/1_3-fcs/doc/Servlets.html 8╇ The body element contains the document’s content 6╇ 7╇
160
G. Ghiani et al.
device. The browsers of the devices are connected to the application server to get the PacMan Game implemented as JavaScript embedded in HTML. Another Open Client is for the ghost logic component. The browser with the active PacMan UI is connected to the ghost logic component. The Open Clients of the PacMan UI and the ghost logic component are also connected to the Open Server. When the PacMan game is migrated to another client (e.g. mobile to desktop device) the Open Adapters (see Chap.€4) reconfigure the PacMan UI and the ghost logic component. When the PacMan UI is migrated, the Context Agent commits the type of the target device. The ALR gets informed of all the updates of the device type value. At an update the ALR recovers the screen size value and computes a table of all possible configurations sorted by most suitably out of the ALR view. After that this configuration is forwarded to the Orchestrator (see Chap.€6) on the Server side (OS). OS forwards the configuration to the Orchestrator at the Client side (OC). The OC initiates the reconfiguration of the ghost logic component depending on the configuration.
10.8â•…Advantages of the OPEN Migration Platform To summarise, from the point of view of a user, the OPEN Migration Platform enables the migration of an application from one device to another, providing the user with the experience of seamlessly continuing to interact with the application at a device change. The UI migration module cares about the adaptation of the UI to the new device: UI objects are automatically adapted and arranged to provide the best possible usability after migration. State persistence allows the user to continue to interact with the application on the new device at the state reached before triggering migration. The ALR module cares about the adaptation of the application logic taking into account the characteristics of the new platform. For the PacMan example this means that, when migrating from a desktop to a mobile platform, the ghost logic of the game changes to a less difficult one. From the point of view of a developer, other advantages can be identified with respect to the various modules of the OPEN Migration Platform. Regarding the migration of the UI part, an advantage of the solution proposed is that it makes migratory Web applications regardless of the authoring environments used by the developers. Without requiring the use of any specific tool in the development phase, it enables the applications to migrate, even if the developers never considered migration. This is obtained through the use of reverse engineering techniques that create on the fly the logical descriptions of the Web pages accessed, which are then adapted for the target device (with the updated state included). The advantage of this is that every valid web application can be migrated, even if it is not OPENaware. Another benefit is that the adaptation of the UI is done automatically and dynamically by the OPEN Migration Platform: therefore, the developers do not have
10â•… Integration of User Interface Migration and Application Logic Reconfiguration
161
to implement different versions of the interactive part of the application depending on the different types of device to support for migration. Regarding the ALR part of the OPEN Migration Platform, a developer is facilitated by having different application logic components for different device types independent from the rest of the application. Indeed, separating the reconfiguration logic from the rest of the application reduces the complexity of the code, improves extensibility of the application and enhances code maintainability. Indeed, the alternative would be to implement a large number of branches in the application and for every new possible configuration the source code of the application should be changed. As it has been shown, the separation between the reconfiguration logic and the rest of the application is just what has been done for the PacMan application (in particular, for the implementation of the ghost logic). Indeed, the best configuration and the conditions to use the ghost logic components are described at the application specification, which can be easily changed.
10.9â•…Conclusions When a context change (in particular, a device change) occurs while a user is interacting with an application, a potential discontinuity is introduced by the changed conditions in the current context. The goal of migration is to cope with such issue and turn it into an opportunity to be suitably exploited by the user. To this aim, the OPEN Migration Platform provides mechanisms that automatically perform adaptation of the application. Such adaptation can affect different aspects of the application, not only the user interface but also the underneath logic. In this chapter we have examined how the OPEN Migration Platform is able to migrate an application while adapting in an integrated way both its user interface and logic to the characteristics of the target device. To this aim, we analysed a web-based PacMan game as an application case study, also highlighting the main benefits that the use of the OPEN Migration Platform can bring both to the user and to the developer of an interactive application.
References Flanagan, D.: JavaScript—the definitive guide, O’Reilly, Sebastopoll (2006) Holdener, A.T.: AJAX—the definitive guide, O’Reilly, Sebastopoll (2008)
Chapter 11
The Usability Evaluation and the Programmability Assessment of Migration Agnese Grasselli, Alessandro Vangelista and Stefano Bolli
11.1â•…What Does Testing a Migratory Middleware Platform Mean? The middleware can be defined as software enabling different processes to interact. In particular, as described in the previous chapters, the OPEN middleware allows applications to migrate: it provides a wide set of services/features for device change, state persistence and content adaptation. Testing the OPEN middleware platform is a big challenge: • It is difficult to test the platform itself, without using the on top applications. Example: in order to test the adaptation feature effectiveness, it is necessary an application that exploits the adaptation • The migration is a very innovative feature: the testing methodology needs to be designed taking into account migration characteristics for obtain good results. In the following paragraphs it is described the methodology that has been designed and applied to address the listed challenges. The testing activity addressed three main aspects: • Usability • Programmability • Technological evaluation In this chapter, the focus is on the Usability and Programmability evaluation, since these are the main interesting aspects.
A. Grasselli () Vodafone Omnitel NV, Milan, Italy e-mail:
[email protected] F. Paternò (ed.), Migratory Interactive Applications for Ubiquitous Environments, Human-Computer Interaction Series, DOI 10.1007/978-0-85729-250-6_11, ©Â€Springer-Verlag London Limited 2011
163
164
A. Grasselli et al.
11.2â•…Usability In order to evaluate whether a migratory middleware is usable from the end user point of view, it is necessary to use prototype applications developed on top of the middleware platform. In this way, it is possible to evaluate how the migratory features and services enabled by the platform are perceived by users.
11.2.1 The ISO Definition of Usability According to the international standard, ISO 9241-11 (1998), the main dimensions for usability evaluation are: 1. Effectiveness—Users’ capability of completing tasks and reach goals 2. Efficiency—Users’ required effort for complete tasks 3. Satisfaction—Users’ feelings and perceptions in using the tool The methodology proposed in the following paragraphs needs to cover all the three aspects of usability explained before, thus different kinds of quantitative data have to be collected during the same test execution. In particular, for each aspect of usability, a set of different measures is considered in order to provide useful and comparable results among different prototypes. 11.2.1.1â•…Task Lists Creation The key point for the usability evaluation is the task list. The task list drives the user through the test describing the different actions that need to be performed. For each prototype a predefined task list is created. All the usability evaluations are based on the information collected during the task list execution by the moderator. Please refer to (Jeffrey and Chisnell 2008) for more details on task list definition, moderator role and test participant characteristics. The usability test of application migration is a complex problem, because there is not a single application to evaluate, but a mechanism supporting: • Device change • Application UI adaptation and/or logic reconfiguration • State persistence In order to achieve an accurate evaluation of the OPEN platform, the task list should drive the user in all the listed activities. During each test, the user will execute tasks with the original application (e.g.: running on a PC) and the adapted one (e.g.: in the mobile device). When the user starts using an application and she/he completes her/his tasks, she/he also learns how to use the application. So, when she/he starts using the second version, she/he can be helped by this learning (for example when tested versions are very similar) or, in some cases, she/he can encounter more problems (for example, when tested versions are very different). This option is called “learning problem”.
11â•… The Usability Evaluation and the Programmability Assessment of Migration
165
In order to get more accurate results, it is necessary to have two different groups of users (say group A, and group B). Let’s suppose that the tested application can run over two devices: D1, and D2. The group A users will start using the application with device D1: they perform migration using the OPEN platform and then they use the device D2. At the end of each device usage (and after a migration to the other device), they will compile an evaluation questionnaire. However, for application prototypes that require a short task list, it is preferable to compile every evaluation questionnaire at the end of the test. The users of the group B will perform the same tasks performed by the group A users, but starting with the test of the device D2. In this way, the “learning problem” can be mitigated. This approach is an adaptation for the OPEN project of one of the methods proposed by (Jeffrey and Chisnel 2008) for the evaluation of multiple product version. 11.2.1.2â•…Effectiveness Effectiveness evaluation answers the question: can users complete tasks, reach goals with the product? The effectiveness evaluation is based on: • Number of tasks failed • Number of users abandoning the tasks In particular, a task is defined failed if during the task execution occurs one of the following cases: • Too much time to execute the task; • User executes a different action within respect to the one required by the task. An application failure (something goes wrong in the application and an error occurs) is not considered a failure for the usability evaluation because it does not depend on the user testing the application. Considering the number of tasks failed in respect to the overall number of executable tasks, it is possible to obtain a plot as the one reported in Fig.€ 11.1. It clearly shows the percentage of successful and unsuccessful tasks and also the percentage of tasks that, eventually, have been abandoned. The unsuccessful tasks are distinguished between user errors or application errors. 11.2.1.3â•…Efficiency Efficiency has to give insights on the effort required from the user to complete the task. The simplest and standard way to collect quantitative data is to record the time needed by the user to complete each task of the task-list. The collected data allows building a time distribution as shown in Fig.€11.2 for the whole task-list. Each bar shows the number of participants that completed the task within the related time interval.
166
A. Grasselli et al. Emergency
User Error 1.09%
Effectiveness: Successful tasks
Successful 97.28%
App. Error 1.63%
User Error
Application Error
Abandoned 0.00%
Successfull
Abandoned
Fig. 11.1↜渀 Effectiveness analysis, data representation
During the pilot test a reference time for the execution of each task is estimated, in order to evaluate if the recorded times during the test execution are compatible with the expected duration. In Fig.€11.2 the estimation time for task execution is shown by the dotted line. Some statistical indicators like average and standard deviation, lower bound and upper bound can also be easily computed to give a general overview of the system. Another useful representation for tasks time distribution is reported in Fig.€11.3, which uses the box plot paradigms (John et al. 1983), (Montgomery and Runger 2002) and (http://www.itl.nist.gov/ div898/handbook/eda/section3/boxplot.htm). The Box plot represents the median, the lower (25th percentile), and the upper (75th percentile) quartiles of recorded time in the vertical axis. In the plot the median is represented with a dotted line, while the rectangle extends itself from the lower to the upper quartile. This box represents the middle 50% of the data, and it is often called the “body” of the data.
Emergency Efficiency: Flood task distribution
5
People number
4 3 2 1 0
[0; 10)
[10; 25)
[25; 50)
[50; 75) [75; 100) [100; 150) [150; 200) [200; 300) [300; 400) [400; +inf) Task list execution range [s]
Fig. 11.2↜渀 Efficiency analysis, user distribution representation
11╅ The Usability Evaluation and the Programmability Assessment of Migration Fig. 11.3↜渀 Efficiency analysis, user distribution representation with box plot
70
167
Efficiency: Flood execution distributions Quartiles
60
Time [s]
50 40 30 20 10
tasks Tests
11.2.1.4â•…Satisfaction Satisfaction refers to what users think about the product’s ease of use. This is a very critical measure because a lot of different methods exists, usually based on questionnaires, to evaluate it. For example, in (QUE-06 2006) an exhaustive list of possible questionnaires to evaluate the user satisfaction are reported. Here are listed the most used ones, together with a brief description: • SUMI (http://sumi.ucc.ie) is a standard questionnaire based on fifty statement to which the user has to express the level of agreement. • WAMMI (http://www.wammi.com) is a methodology developed for web sites evaluation. • SUS (1986) is a widely used questionnaire. Above all the public domain questionnaire, this is the most mature and recommended. • QUIS (http://www.cs.umd.edu/hcil/quis/) is a tool designed by the Human-Computer Interaction Lab (HCIL). It contains questions addressing specific interface factors (e.g.: screen, system feedbacks...). • USE (http://www.stcsig.org/usability/newsletter/0110_measuring_with_use.html) is still in development by Arnie Lund. • CSQU (http://oldwww.acm.org/perlman/question.cgi?form=CSUQ) is based on 19 questions on a 7-point scale. Briefly, excluding the commercial and the not jet consolidated questionnaires, SUS, USE, CSUQ are suitable, but the wider used and suggested is SUS. Therefore, in order to apply a standard and easily repeatable approach to evaluate satisfaction parameters, the System Usability Scale (SUS) has been chosen. Together with SUS, a second methodology has been applied: Product Reaction Cards (PRC—Developed by and © 2002 Microsoft Corporation. All rights reserved.), and a detailed description is available in (Benedek and Trish 2002). It is an innovative evaluation approach that requires the user to select the words inside a big set that better describe the experience during the test, and ask the user to add a synthetic comment. The following paragraphs explain in detail the two considered approaches.
168
A. Grasselli et al.
Satisfaction—System Usability Approach The System Usability Scale (SUS) approach is a simple, ten-item scale giving a global view of the user experience during the testing execution. The ten questions are standard and are reported in Appendix A. For each question the user should indicate the level of agreement or disagreement (between 1 and 5) Questions are in a predefined order that alternates positive and negative questions so that it is possible to mitigate the effect of undecided testers. A score is assigned to each answer, and a predefined SUS scoring algorithm is applied in order to obtain the overall value of System Usability (SU) within the range 0–100%. In the considered scale 0% means “hard to use”, whereas 100% means “easy to use”. Further details about this approach and the scoring algorithm applied are available in (SUS 1986). In order to give readable results about the SUS, the following five classes of the same size have been identified starting from the SUS score. U = {‘hard’, ‘fairly hard’, ‘neutral’, ‘fairly easy’, ‘easy’}
In Table€11.1 the classification proposal is reported in detail. It is possible to plot the distribution of the System Usability results as shown in Fig.€11.4. Satisfaction—Product Reaction Card The Product Reaction Cards (PRC) approach was developed by Microsoft Corporation (Developed by and © 2002 Microsoft Corporation. All rights reserved.), and a detailed description is available in (Benedek and Trish 2002). It consists of 118 cards, each one containing one of the adjective listed in Appendix B. The word set is developed in a way that a correct mix of positive and negative concepts is available inside the set. At the end of the usability evaluation, the complete card set, ordered by ID, is given to the participant. Then, the main steps are: 1. The user selects the words that better describe the product or the perceived feelings. 2. Once the user picks the words, the moderator records the selected words. 3. Then, the user is asked to narrow his/her set down to 5 words. 4. The moderator asks the reasons why the user selected the final 5 words. Analyzing the user comments and the selected words, it is possible to classify the feedbacks inside three different classes: positive, negative, and neutral. A simple but very readable representation of these results can be shown with a plot as depicted in Table 11.1↜渀 Classifier based on system usability score
SU score range [0;20] [20;40] [40;60] [60;80] [80;100]
Usability class (U) Hard Fairly hard Neutral Fairly easy Easy
11â•… The Usability Evaluation and the Programmability Assessment of Migration
169
SUS distribution
8
Number of people
7 6 5 4 3 2 1 0
Hard
Fairly Hard Neutral Fairly Easy Subjective rating of task difficulty
Easy
Fig.€11.4↜渀 Evaluation of user satisfaction using SUS approach distribution
Fig.€11.5. Moreover, the most important outcome of this approach is the qualitative evaluation of the experience: it makes it possible to understand how the platform is perceived by the user. 11.2.1.5╅Competitor Analysis It can happen that some prototypes have features also available in commercial products. Competitor analysis consists of executing the same test and collecting the same data, both for the prototype whose usability validation is being done and for the products that implement similar features. Thus, comparing the results obtained from the prototypes and from the product is possible to have an idea about how the new proposal is evaluated, with in respect to the other ones. The percentage of successful tests is reported for the effectiveness, while the time consumed on task is shown for efficiency. In order to compare user satisfaction, ad-hoc questionnaire have been created in order to collect user feedback. To give a concrete example of how this kind of analysis can be applied in the OPEN project, the web page migration can be considered: actually, there are also other solutions that provide web page adaptation for mobile phones, e.g. the Opera Mini browser [http://www.opera.com/mobile/].
11.3â•…Programmability In order to evaluate how a migratory middleware is configurable and customizable, it is necessary to focus on the platform implementation. The aim of this class of tests is to evaluate if the platform’s migratory features can be easily extended or configured without any code implementation and recompilation.
170
A. Grasselli et al. Partial + Total Migration Product Reaction Card: 2nd iteration - Top 5 cards selected Excluding the strongly application related cards Positive
Negative
86%
0%
20%
40%
14%
60%
80%
100%
Fig. 11.5↜渀 Evaluation of user satisfaction using product reaction cards approach
11.3.1 Definition Programmability is the capability within hardware and software to accept a new set of variables and instructions that alter its behavior (http://encyclopedia2.thefreedictionary. com/programmability). In a middleware environment like OPEN, the following definitions are applied to variables and instructions: Variables are context information, while instructions are rules used to describe the migration process behavior, depending on the context information (see Fig 11.6). Both variables and instructions are the basic bricks that can be used to design specific behavior of the migration process, such as migration triggering (when to migrate) and orchestration (where, what, how to migrate), application logic reconfiguration and user interface adaptation. It is important to remember that new context variables can be added to identify the platform state, while new rules are new functions of the context variables used to describe and implement new platform behaviors. The high level idea described above can be better illustrated using the following practical example. Suppose that a mobile phone that provides battery and signal strength information is available. In the platform, migration triggering rules depending on these variables are defined. If other mobile phones can provide also the “user_location” (latitude and longitude), it is desirable: • to instruct the middleware in order to acquire the new “user_location” variable; • to define a new rule depending on this new variable. E.g.: if “user_location”=“home” then trigger the migration towards the TV. The programmability evaluation addressed the different components of the migration process, exploited by different middleware modules. For each module, the programmability evaluation will consider two different phases: • Programmability Assessment • Programmability Validation
11â•… The Usability Evaluation and the Programmability Assessment of Migration
171
Fig.€11.6↜渀 Variables and rules adding +
managed variables
=
new variable
+
managed rules
new managed variables
=
new rule
new managed rules
11.3.2 Programmability Assessment The first phase of the programmability evaluation process is the programmability assessment. With respect to the programmability, two different middleware tasks are distinguished: • Context variable collection and distribution: different middleware modules can participate in this task, e.g. considering the OPEN MSP, the Device Discovery, and the Context Management modules. For these modules, enabling the programmability means providing the capability of managing all the available variables without constraints (a possible constraint could be: the module accepts only numerical variables, it does not handle Boolean or String or the module accepts only the variables defined in a predefined structure). • Migration and adaptation rules definition: different modules apply rules for migration triggering and orchestration, application logic reconfiguration, and user interface adaptation. For these modules, enabling the programmability means providing some facilities to define the module behavior depending on the available context information. It could be useful to map these two tasks in the previous example, in which the platform should: • Instruct the middleware in order to acquire the new “user_location” variable (this aspect has been evaluated in the scope of “context variable collection and distribution”) • Define a new rule depending on this new variable. E.g.: if “user_location”=“home” then trigger the migration towards the TV (this aspect has been evaluated in the scope of “migration and adaptation rules definition”). The aim of the assessment phase is to understand for each OPEN middleware module which of the previous programmability aspects it should address, or if it
172
A. Grasselli et al.
should address both and, depending on it, understanding if appropriate facilities are available. In order to carry on this phase, the following Template has been used. Title: ID: Version
OPEN Programmability Assessment moduleName OPEN Programmability_moduleName_x Issue
Date
Author Module owner/Vodafone team
Module name moduleName General considerations
Reference prototypes
Specify if the module enables the programmability in its current implementation related to both programmability aspects: • context variable collection and distribution • migration and adaptation rules definition List of prototypes using the specified modules User: (developer, system manager, service provider…) Supported context variables type: (int, double, boolean…)
Synthetic description
Context variable collection and distribution
Language/tool available for the module behavior description
Manageable variables: (only predefined, all variables that can be represented by an integer number, all…) Available tools for configuration: • configuration file • graphic tool • workflow definition tool • other (specify) Only for modules addressing context variable collection and distribution. Please describe how the module is able to collect and make available context variables to other modules Only for modules addressing migration and adaptation rules definition. Please describe how the user can define the rules for module configuration
In the “Title” field the name of the module is indicated. The “ID” field is used to distinguish between the evaluation iteration performed for the first iteration prototype and the final evaluation. A different “version number” should be given to the document if substantive changes to the contents of the document have been made. Different “issue” numbers within a given version indicate minor changes, such as spelling and grammatical corrections. The synthetic description of the information expected for each field is described in italic. The “General consideration” field contains the information on the programmability aspect addressed by the module. The “Reference prototypes” field is filled listing the prototypes exploiting the features provided by the specified module.
11â•… The Usability Evaluation and the Programmability Assessment of Migration
173
The “Synthetic description” field is provide the following basic information: • User: who is the user of the provided facilities for programmability? (developer, system manager, service provider…) • Supported context variables type: which kinds of variable are supported? (int, double, boolean…) • Manageable variables: how many variables can be supported? (only predefined variables, all variables that can be represented by an integer number, all variables…) • Available tools for configuration (only for modules addressing rules definition): − − − −
configuration file graphic tool workflow definition tool other (specify)
The “Context variable collection and distribution” field is filled only for modules addressing this aspect of programmability. The module owner described how the module is able to collect and make available to other modules context variables. The “Language/tool available for the module behavior description” field is filled only for modules addressing migration and adaptation rules definition. The module owner described how the user can define the rules for module configuration.
11.3.3 Programmability Validation The Programmability Validation takes as inputs the modules that emerge from the assessment for providing programmability features. The final goal of this evaluation is to obtain a classification of the programmability features considering two dimensions, which are: • Extensibility. A middleware module is extensible if and only if it allows adding and removing rules and/or variables able to control its behavior without changing the module source code. • Configurability refers to the capability of a module to allow changing rules and variables, in order to change how it reacts to different situations. This evaluation also takes into consideration the tools available for the configuration (e.g.: the possibility of using external configuration files without changing the module source code). According to the dimensions defined above, it is possible to have a module and consequently a platform classification, which can be easily summarized using the plot reported in Fig.€11.7. The validation has been carried out starting from the module documentation, identifying the configuration features available: what can be configured and eventually extended in term of rules and variables managed. Starting from this analysis, a set of test cases have been produced and execute in order to provide evidence of the final module classification.
174
A. Grasselli et al.
Programmability Evaluation
EXTENSIBILITY
it is possible to both add and remove rules/variables
it is not possible neither to add nor to remove rules/variables it is not possible to edit the parameterization CONFIGURABILITY
it is possible to edit parameters of rules and variables already used
Fig.€11.7↜渀 Evaluation chart for programmability test
11.4â•…Conclusion About Testing Activity For the scope of this book, the focus has been done to the methodology applied, in order to address a wider set of readers. The methodology has been defined in order to better address the main OPEN middleware platform challenges: • It is difficult to test the platform itself, without using the on top applications. Example: in order to test the adaptation feature effectiveness, it is necessary an application that exploits the adaptation • The migration is a very innovative feature: the testing methodology needs to be designed taking into account migration characteristics for obtain good results. The work done can be used as reference for further testing activities involving platform with similar features and challenges.
References Benedek, J., Trish, M.: Measuring desirability: new methods for evaluating desirability in a usability lab setting. Microsoft Usability. Microsoft Corporation. http://www.microsoft.com/ usability/UEPostings/DesirabilityToolkit.doc (2002) CSUQ: http://oldwww.acm.org/perlman/question.cgi?form=CSUQ ISO 9241-11: Ergonomic requirements for office work with visual display terminals (1998)
11â•… The Usability Evaluation and the Programmability Assessment of Migration
175
Jeffrey, R., Chisnell, D.: Handbook of Usability Testing. Wiley, Indiana (2008) John, C., William, C., Beat, K., Paul, T.: Graphical Methods for Data Analysis. Duxbury Press, Wadsworth (1983) Montgomery, D.C., Runger, G.C.: Applied Statistics and Probability for Engineers, 3rd edn. Wiley, New York (2002) NIST/SEMATECH: e-Handbook of Statistical Methods. http://www.itl.nist.gov/div898/handbook/ eda/section3/boxplot.htm QUE-06: Usability Net.Questionnaire resources. http://www.usabilitynet.org/tools/r_questionnaire. htm (2006) QUIS: http://www.cs.umd.edu/hcil/quis/ SUMI: http://sumi.ucc.ie SUS: A Quick and Dirty Usability Scale. John Brooke, UK (1986) The Free Dictionary: http://encyclopedia2.thefreedictionary.com/programmability USE: http://www.stcsig.org/usability/newsletter/0110_measuring_with_use.html WAMMI: http://www.wammi.com
Appendix
Appendix A: System Usability Scale © Digital Equipment Corporation, 1986. Strongly disagree 1.
I think that I would like to use this system frequently
2.
I found the system unnecessarily complex
3.
I thought the system was easy to use
4.
I think that I would need the support of a technical person to be able to use this system
5.
I found the various functions in this system were well integrated
6.
I thought there was too much inconsistency in this system
7.
I would imagine that most people would learn to use this system very quickly
8.
I found the system very cumbersome to use
9.
I felt very confident using the system
10. I needed to learn a lot of things before I could get going with this system
Strongly agree
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
F. Paternò (ed.), Migratory Interactive Applications for Ubiquitous Environments, Human-Computer Interaction Series, DOI 10.1007/978-0-85729-250-6, ©Â€Springer-Verlag London Limited 2011
177
Appendix
178
Appendix B: Product Reaction Cards Developed by and © 2002 Microsoft Corporation. All rights reserved. The complete set of 118 product reaction cards Accessible Advanced Annoying Appealing Approachable Attractive Boring Business-like Busy Calm Clean Clear Collaborative Comfortable Compatible Compelling Complex Comprehensive Confident Confusing Connected Consistent Controllable Convenient
Creative Customizable Cutting edge Dated Desirable Difficult Disconnected Disruptive Distracting Dull Easy to use Effective Efficient Effortless Empowering Energetic Engaging Entertaining Enthusiastic Essential Exceptional Exciting Expected Familiar
Fast Flexible Fragile Fresh Friendly Frustrating Fun Gets in the way Hard to Use Helpful High quality Impersonal Impressive Incomprehensible Inconsistent Ineffective Innovative Inspiring Integrated Intimidating Intuitive Inviting Irrelevant Low Maintenance
Meaningful Motivating Not Secure Not Valuable Novel Old Optimistic Ordinary Organized Overbearing Overwhelming Patronizing Personal Poor quality Powerful Predictable Professional Relevant Reliable Responsive Rigid Satisfying Secure Simplistic
Slow Sophisticated Stable Sterile Stimulating Straight Forward Stressful Time-consuming Time-Saving Too Technical Trustworthy Unapproachable Unattractive Uncontrollable Unconventional Understandable Undesirable Unpredictable Unrefined Usable Useful Valuable
Index
A Access, 5, 13, 15, 16, 19, 20, 25–27, 29, 30, 34, 48, 64, 66–68, 81, 83, 88, 117, 118, 125–127, 129 Adaptation, 1, 4, 5, 7, 10, 13–16, 18, 19, 33, 34, 36, 95, 96, 99, 100, 104, 105, 109, 111, 113, 115, 116, 119, 128, 132–134, 149, 151, 154, 156, 160, 161, 163, 164, 169–172, 174 Adaptive, 12, 16, 18, 104, 106 Adaptor, 39–41, 112, 117, 118, 129 aggregating migration, 5 AJAX, 12, 40, 126, 127, 159 ALR, 33, 40, 72–74, 95–104, 106, 114, 153, 154, 160, 161 Application, 1–7, 10–15, 17–20, 29–40, 42, 43, 45–50, 52, 54, 55, 57, 58, 61–66, 68–70, 72–76, 78–80, 82–88, 91–93, 95–106, 109–120, 125–131, 133– 135, 138–143, 145–147, 149, 151–154, 156, 158, 160, 161, 163–165, 171, 174 Architecture, 6, 7, 12, 14, 16–18, 29, 34, 40, 41, 43, 46, 47, 61, 67, 71, 77, 92, 95, 96, 105, 111–113, 117, 120, 130–132, 138–141, 150 B Broadband, 25–27 C Cameleon, 9 Client, 4, 12, 16, 32, 34–43, 46, 48, 61, 62, 65, 67, 73, 76, 84, 85, 87, 88, 91, 92, 96, 111–113, 117, 118, 129, 131, 132, 139–141, 152, 154, 159, 160 client-server, 4, 10, 87, 111–113, 116, 117, 130, 131 Concrete User Interface, 48, 116
Context, 1, 3–5, 10, 11, 14, 20, 30, 31, 33, 45, 61, 64, 66–75, 77, 81, 95, 96, 98, 102, 105, 109–111, 113–121, 129, 130, 140, 143, 153, 154, 160, 161, 170–173 Context Management, 14, 33, 61, 64, 66–69, 71–74, 96, 102, 117, 118, 140, 171 Continuity, 5, 6, 9, 30, 45, 109, 111, 114, 116, 127, 132, 146 D Device Discovery, 33, 70, 72, 74, 92, 171 Dispatcher, 38–41, 96, 99, 112, 131 distributing migration, 4, 5 Dynamic, 3, 4, 6, 9, 11, 14, 33, 47, 64, 66, 69–71, 83, 84, 95, 104, 105, 115, 146, 147 E Evaluation, 54–58, 64, 75, 78, 138, 146, 147, 163–165, 167–174 F Framework, 9, 11, 16, 33, 64, 66–75, 78, 96, 117, 137, 140, 146 G Gaming, 2, 26, 121, 125 H HCI, 9 I iPhone, 52, 53, 156, 157 IPTV, 26, 52, 121, 125–127, 134 J Java a, 47, 57, 131 JavaScript, 18, 36, 46, 47, 57, 58, 104, 126, 154, 158–160 Jini, 11, 12, 14
F. Paternò (ed.), Migratory Interactive Applications for Ubiquitous Environments, Human-Computer Interaction Series, DOI 10.1007/978-0-85729-250-6, ©Â€Springer-Verlag London Limited 2011
179
Index
180 M MARIA, 48 Migration Framework, 9, 75 Migration Platform, 7, 12, 29, 30, 34, 47, 49, 55–58, 95, 96, 109, 139, 140, 150–152, 154, 156, 160, 161 Migratory services, 2, 4–7, 9, 11, 12, 14, 64, 137, 139, 147 Mobility Support, 4, 12, 13, 33, 41, 61, 64, 81–83, 92, 93, 116, 117, 129, 131, 132 Model, 11, 12, 15–18, 45, 46, 78, 81, 87, 97, 98, 105, 118, 137, 146, 147 Multi-Device, 2, 9, 45, 47, 111, 120 Multimedia, 13, 25, 26, 83, 85, 142 multiple migration, 5 Multiscreen, 28 N Network, 1, 3–7, 12–15, 20, 25, 27, 29, 33, 34, 39, 40, 61–64, 66–70, 72–78, 81–88, 90–92, 95, 97, 101, 110–114, 116–118, 121, 126, 127, 129, 131, 132, 134 O Open, 1, 3, 7, 10, 12–15, 17–20, 25, 28–30, 32–44, 48–50, 57, 61–63, 65–70, 74, 81–93, 95–99, 102, 106, 109, 111–113, 115–117, 120, 122, 126, 131, 132, 134, 135, 139–144, 150–154, 156, 159–161, 163–165, 169–172, 174 Orchestration, 48, 49, 61, 64, 66, 92, 132, 170, 171 OSGi, 11, 70, 71 P PacMan, 7, 95, 104, 149–161 Partial Migration, 5, 13, 17, 20, 33, 38, 45, 46, 48, 49, 52, 55, 56, 58, 110, 113, 116, 132, 133, 139, 142, 143, 146, 147 PDA, 4, 19, 95, 133 Peer-to-peer, 10, 17, 33, 63 Policy, 16, 29, 40, 74, 78, 80, 81, 110, 111, 113, 118, 119, 130, 139, 142 S scenarios, 3, 5, 6, 13, 14, 25, 28, 29, 39, 61, 62, 69, 76, 78, 79, 88, 104, 132, 137 Semantic Redesign, 47–50, 57, 58 sensor, 3, 71, 156 Server, 4, 10, 12, 13, 16, 32–34, 36–43, 47, 48, 61–63, 65–67, 73, 85–87, 91–93, 96, 99, 111–113, 121, 123, 127–129, 131, 139–142, 153, 159, 160 Service Provider, 11, 28, 25, 29, 30, 106, 172, 173
Session Mobility, 13, 81, 82, 84–92, 116, 117 Social Game, 38, 50, 52, 55, 120, 125–135 SOCKS, 87–89, 92, 117 Software, 1, 3, 6, 10, 11, 14, 18, 20, 29, 31, 33, 40, 68, 105, 106, 113, 114, 121, 126–128, 163, 170 State, 1, 3, 5, 6, 9, 10, 12, 15, 16, 18, 19, 31, 32, 34, 36, 45–49, 54, 57, 58, 64, 65, 74, 78, 80, 81, 92, 101, 102, 104, 106, 111, 114–116, 121, 127, 128, 132, 140, 141, 146, 155, 157–160, 163, 164, 170 T Terminal Mobility, 13, 33, 81–87, 116, 117 Toolkit, 14, 17, 46 Total migration, 5, 47, 49–51, 139 transformation, 48, 115, 147 Trigger, 13, 14, 33, 34, 40, 41, 62, 64, 72–74, 78, 97, 114, 170, 171 Trigger Management, 33, 40, 41, 61, 64, 66, 70, 72, 73, 76, 97, 114 TV, 11, 12, 19, 26, 27, 121, 143, 170, 171 U Ubiquitous, 1, 9, 11, 15, 19, 20, 25, 45, 99, 146 UMTS, 3, 5, 13 UPnP, 11, 14 Usability, 7, 10, 15, 20, 54–56, 58, 109, 142, 143, 145–147, 160, 163–165, 167–169 User, 1–6, 10, 12–20, 28–34, 45–49, 52, 54–56, 58, 61, 62, 65, 70, 72–75, 79, 81, 85, 87, 90, 91, 95, 97, 99, 100, 102, 104, 110, 111, 115, 116, 118–120, 123, 125–130, 132–134, 137–139, 141–143, 146, 149, 150, 152, 154–156, 158, 160, 161, 164–173 User experience, 6, 18, 28, 29, 55, 65, 115, 118, 132, 149, 168 User Interface, 3–5, 7, 9–12, 15–18, 20, 31, 33, 34, 45–48, 52, 54, 95, 97, 104, 110, 111, 115, 116, 128, 132, 138, 139, 141–143, 146, 149, 151, 154, 156, 159, 161, 167, 170, 171 W W3C, 46, 47, 72, 115, 128 Web browser, 12, 127, 158 Wireless, 6, 25, 66 X XML, 15–17, 40, 41, 43, 49, 67–69, 72, 98, 103, 113, 118, 131, 142 XSL, 16
