Time-to-Contact (Advances in Psychology, Volume 135)

TIME-TO-CONTACT Heiko Hecht Geert J.P. Savelsburgh Editors Elsevier TIME-TO-CONTACT ADVANCES IN PSYCHOLOGY 135 135...

Author: Heiko Hecht | Geert Savelsbergh

19 downloads 1421 Views 24MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

TIME-TO-CONTACT

Heiko Hecht Geert J.P. Savelsburgh Editors

Elsevier

TIME-TO-CONTACT

ADVANCES IN PSYCHOLOGY 135 135 Editor:

G.E. STELMACH

ELSEVIER Amsterdam -– Boston –- Heidelberg -– London -– New York -– Oxford –- Paris San Diego -– San Francisco -– Singapore –- Sydney –- Tokyo

TIME-TO-CONTACT

Edited by

Heiko Hecht Department of Psychology, University of Mainz, University Germany

Geert J.P. Savelsburgh Vrije Universiteit, Amsterdam, Amsterdam, The Netherlands

2004

ELSEVIER Amsterdam -– Boston –- Heidelberg -– London -– New York -– Oxford –- Paris San Diego –- San Francisco -– Singapore –- Sydney –- Tokyo

ELSEVIER B.V. Sara Burgerhartstraat 25 P.O. Box 211,1000 211, 1000 AE Amsterdam, The Netherlands

ELSEVIER Inc. 525 B Street Suite 1900, San Diego CA 92101-4495, USA

ELSEVIER Ltd The Boulevard Langford Lane, Kidlington, Langford Oxford OX5 1GB, UK

ELSEVIER Ltd 84 Theobalds Road London WC1X 8RR UK

ElsevierB.V. © 2004 Elsevier B.V. All rights reserved. This work is protected under copyright by Elsevier B.V., and the following terms and conditions apply to its use: Photocopying Single photocopies of single chapters may be made for personal use as allowed by national copyright laws. Permission of the Publisher and payment of a fee is required for all other photocopying, including multiple or systematic copying, copying for advertising or promotional purposes, puiposes, resale, and all forms of document delivery. Special rates are available for educational institutions that wish to make photocopies for non-profit educational classroom use.

Permissions may be sought directly from from Elsevier’s Elsevier's Rights Department in Oxford, Oxford, UK: phone (+44) 1865 1865 843830, fax 1865 853333, e-mail: [email protected]. [email protected]. Requests may also be completed on-line via the Elsevier homepage (+44) 1865 (http:/ /www.elsevier.com/locate/permissions). /www.else vier.com/locate/permissions). In the USA, users may clear permissions and make payments through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA; phone: (+1) (978) 7508400, fax: (+1) (978) 7504744, and in the UK through the Copyright Licensing Agency Rapid Clearance Service (CLARCS), 90 Tottenham Court Road, London W1P 0LP, UK; phone: (+44) 20 7631 5555; fax: (+44) 20 7631 5500. Other countries may have a local reprographic rights agency for payments. Derivative Works Tables of contents may be reproduced for internal circulation, but permission of the Publisher is required for external resale or distribution of such material. Permission of the Publisher is required for all other derivative works, including compilations and translations. Electronic Storage or Usage Permission of the Publisher is required to store or use electronically any material contained in this work, including any chapter or part of a chapter.

Except as outlined above, no part of this work may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without prior written permission of the Publisher. Elsevier's Rights Department, at the fax and e-mail addresses noted above. Address permissions requests to: Elsevier’s Notice No responsibility is assumed by the Publisher for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in verification of diagnoses the material herein. Because of rapid advances in the medical sciences, in particular, independent verification and drug dosages should be made.

First edition 2004 B r i t i of sh Congress Library Cataloging Cataloguing in Publication Data Library catalogue record is available from B r i t i of sh Congress. Library. A catalog from the Library British Library Cataloguing in Publication Data A catalogue record is available from the British Library. ISBN: ISSN:

0-444-51045-1 0-444-51045-1 0166-4115 (Series)

@ The paper used in this publication meets the requirements of ANSI/NISO ANSI/NISO Z39.48-1992 (Permanence of of Paper). Printed in The Netherlands.

V

Foreword: Time for Tau A catcher needs to anticipate the exact moment the ball will hit the hand in order to successfully close the fingers around the ball. The time remaining before the ball reaches the catcher, the time-to-contact (TTC), is determined by its distance, speed and acceleration. Lee (1976) described mathematically that expanding optical patterns can indeed contain accurate predictive temporal information about time to contact. That is, when the relative speed between object and observer remains constant, the inverse of the relative rate of dilation of the closed optical contour generated by an approaching object in the optic array specifies the remaining time to contact of the approaching ball. This optical variable was denoted x and its discovery initiated an incredibly large body of research into TTC judgments, which it is high time to evaluate In particular, it is time to make contact with the different theoretical approaches underlying the x concept. The purpose of the book is to explore and discuss the concept of TTC (not only x!) from very different theoretical perspectives. The first chapter is intended as an introduction and to outline the structure of the book. Thereafter, the book is divided into four sections, that is, foundations of TTC, different behavioral approaches to TTC estimation, TTC as perception and strategy and TTC and action regulation. Each section contains several chapters that provide up-to-date overviews of the respective sub-fields, critically evaluate the existing approaches and formulate alternative views. The 33 contributors to the book come from Australia, Canada, Europe, New Zealand, Japan and the United States of America. They include those with established reputations and young authors who are just beginning to make their mark. Together, they bring a wide-ranging perspective to bear on this rich and expanding field of study, and we thank them for their contribution to this project. In Chapter 1, Heiko Hecht and Geert Savelsbergh summarize the epistemological state of TTC theory. Then Barrie Frost and Hongjin Sun describe the functional biological bases of time-to-collision computation. Markus Lappe reviews the neural building blocks likely to be involved in this computation. Lucia Vaina and Franco Giulianini report an investigation on neural behavior in the face of approaching objects that disappear behind an occluder. The chapter by John Flach, Matthew Smith, Terry Stanard and Scott Dittman shows that control strategies, say in driving behind another car, are changed from a simple constant criterion to a relative criterion, say if retinal size increases break, if it decreases accelerate. When the approach velocity of the object is not constant, as in the case of controlled braking, the observer could make TTC estimates based on the second derivative of x or so-called x-dot. Chapter 6 by John Andersen and Craig Sauer is dedicated to such situations. James Tresilian calls for abandoning

VI VI

research on tau-hypotheses and recommends researchers to concentrate on how temporal control is achieved in interceptive tasks. Paulion van Hof, John van der Kamp and Geert Savelsbergh provide a developmental perspective of control of interceptive timing. In Chapter 9, David Regan and Rob Gray make the case that physiology cannot (yet) replace or render obsolete psychophysics. They suggest that tau-like information is used when available and no other information leads to the successful action more quickly. For this tau-like information edges and textures are particularly important. The latter are the topic of Klaus Landwehr's chapter. Patricia DeLucia considers tau-like information, including expansion rate, to be merely a heuristic. The heuristics serve to accommodate limitations of sensory processes. In a similar fashion, Mary Kaiser and Walt Johnson suggest to limit the domain of x theory. Chapter 13 by Rob Gray and David Regan clinches the case for the use of binocular information in TTC judgment. Simon Rushton takes the next step and sketches a general theory about the use of extraretinal information in interceptive timing. Chris Button and Keith Davids give an assessment of the state-of-the-art of acoustic t. They show that sound intensity seems to be the effective stimulus in most cases. In part IV of the book, Geoffrey Bingham and Frank Zaal argue that T cannot provide the online visual guidance that is required for catching at closer range. In chapter 17, Frank Zaal and Reinoud Bootsma provide a review of potential involvement of x in prehension. Geoffrey Bingham then explores the perception of phase stability and reminds us that timing variables have to be understood as part of a perception-action cycle. Simone Caljouw, John van der Kamp and Geert Savelsbergh, show that interceptive actions are regulated by multiple sources of information and in constant need for recalibration. Finally, Gilles Montagne, Aymar de Rugy, Martinus Buekers, Alain Durey, Gentaro Taga and Michel Laurent present a model of how x may be involved in the regulation of locomotion. We think the book will be of great interest to researchers, teachers and students in various fundamental and applied fields, including perception, human movement sciences, kinesiology, neuroscience, perception, developmental and biological psychology. Our special thanks go to several anonymous reviewers for their helpful comments on the original drafts. We would also like to express our gratitude to Fiona Barren, the technical editor of Elsevier for assisting in the preparation of this volume. Finally, we are most grateful to Petra Glaubitz who worked tirelessly copyediting and indexing the volume.

Heiko Hecht and Geert J. P. Savelsbergh Mainz and Amsterdam, October 2003

VII vn

Table of contents 1. Theories of time-to-contact judgment

1

Heiko Hecht and Geert J. P. Savelsbergh

Part I: Foundations of Time-to-Contact 2. The biological bases of time-to-collision computation

13

Barrie J. Frost and Hongjin Sun

3. Building blocks for time-to-contact estimation by the brain

39

Markus Lappe

4. Predicting motion: A psychophysical study

53

Lucia M. Vaina, and Franco Giulianini

5. Collisions: Getting them under control

67

John M. Flach, Matthew R. H. Smith, Terry Stanard and Scott M. Dittman

Part II: Different behavioral approaches to Time-to-Contact estimation 6. Optical information for collision detection during deceleration

93

George J. Andersen and Craig W. Sauer

7. Interceptive action: What's time-to-contact got to do with it?

109

James R. Tresilian

8. The information-based control of interceptive timing: A developmental perspective

141

Paulion van Hof, John van der Kamp and Geert J. P. Savelsbergh

9. A step by step approach to research on time-to-contact and time-to-passage

173

David Regan and Rob Gray

10. Textured Tau

229

Klaus Landwehr

Part III: Time-to-Contact as Perception and Strategy 11. Multiple sources of information influence time-to-contact judgments: Do heuristics accommodate limits in sensory and cognitive processes?

243

Patricia R. DeLucia

12. How now, broad Tau? Mary K. Kaiser and Walter W. Johnson

287

VIII

vm 13. The use of binocular time-to-contact information

303

Rob Gray and David Regan

14. Interception of projectiles, from when & where to where once

327

Simon K. Rushton

15. Acoustic information for timing

355

Chris Button and Keith Davids

Part IV: Time-to-Contact and Action Regulation 16. Why tau is probably not used to guide reaches

371

Geoffrey P. Bingham and Frank T. J. M. Zaal

17. The use of time-to-contact information for the initiation of hand closure in natural prehension

389

Frank T. J. M. Zaal and Reinoud J. Bootsma

18. Another timing variable composed of state variables: Phase perception and phase driven oscillators

421

Geoffrey P. Bingham

19. The fallacious assumption of time-to-contact perception in the regulation of catching and hitting

443

Simone Caljouw, John van der Kamp and Geert J. P. Savelsbergh

20. How time-to-contact is involved in the regulation of goal-directed locomotion

475

Gilles Montagne, Aymar De Rugy, Martinus Buekers, Alain Durey, Gentaro Taga and Michel Laurent

21. Subject Index

493

22. Author Index

497

IX

List of contributors George J. Andersen Department of Psychology, University of California, Riverside, CA, USA

Geoffrey P. Bingham Department of Psychology, Indiana University, Bloomington, IN, USA

Reinoud J. Bootsma UMR Mouvement et Perception, University de la Mediterranee, Marseille, France

Martinus Buekers Faculty of Physiotherapy and Physical Education, Motor Learning Lab, Katholieke Universiteit Leuven, Belgium

Chris Button School of Physical Education, Sport and Leisure Studies, University of Otago, Dunedin, New Zealand, University of Edinburgh, St. Leonard's Land, Edinburgh, UK

Simone Caljouw Perceptual-Motor Control: Development, Learning and Performance, Institute for Fundamental and Clinical Human Movement Studies, Vrije Universiteit, Amsterdam, The Netherlands

Keith Davids School of Physical Education, Sport and Leisure Studies, University of Otago, Dunedin, New Zealand

Patricia R. DeLucia Texas Tech University, Lubbock, TX, USA

Scott M. Dittman Visteon Inc., Detroit, MI, USA

Alain Durey (ft) Movement and Perception Lab, Faculty of Sport Science, Universite de la Mediterranee, Marseille, France

John M. Flach Wright State University, Dayton, OH, USA

Barrie J. Frost Department of Psychology, Queen's University, Kingston, Ontario, Canada

Rob Gray Department of Applied Psychology, Arizona State University, Mesa, AZ, USA

X X

Franco Giulianini Brain and Vision Research Laboratory, Department of Biomedical Engineering, Boston University, Boston, MA, USA

Heiko Hecht Institut fur Psychologie, Johannes Gutenberg-Universitat Mainz, Mainz, Germany Man-Vehicle Lab, Massachusetts Institute of Technology, Cambridge, MA, USA

Walter W. Johnson NASA Ames Research Center, Moffett Field, CA, USA

Mary K. Kaiser NASA Ames Research Center, Moffett Field, CA, USA

Klaus Landwehr Psychologisches Institut, Westfalische Wilhelms-Universitat Munster, Munster, Germany Bergische Universitat-Gesamthochschule Wuppertal, Wuppertal, Germany

Markus Lappe Psychologisches Institut, Westfalische Wilhelms-Universitat Munster, Munster, Germany

Michel Laurent Movement and Perception Lab, Faculty of Sport Science, University de la Mediterranee, Marseille, France

Gilles Montagne Movement and Perception Lab, Faculty of Sport Science, Universite de la Mediterranee, Marseille, France

David Regan Center for Vision Research, Department of Psychology BSB, York University, North York, Ontario, Canada

Aymar De Rugy Department of Kinesiology, The Pennsylvania State University, USA

Simon K. Rushton Department of Psychology, University of Cardiff, Cardiff, Wales, UK

Craig Sauer Department of Psychology, University of California, Riverside, CA, USA

XI

Geert J. P. Savelsbergh Institute for Fundamental and Clinical Human Movement Studies, Vrije Universiteit, Amsterdam, The Netherlands Centre for Biophysical and Clinical research into Human Movement, Manchester Metropolitan University, Manchester, UK

Matthew R. H. Smith Delphi Automotive Systems, Kokomo, IN, USA

Terry Stanard Klein Associates, Dayton, OH, USA

Hongjin Sun Department of Psychology, McMaster University, Hamilton, Ontario, Canada

Gentaro Taga Graduate school of Education, University of Tokyo & PRESTO, JST, Japan

James R. Tresilian School of Human Movement Studies, The University of Queensland, St Lucia, Australia

Lucia M. Vaina Brain and Vision Research Laboratory, Department of Biomedical Engineering, Boston University, Boston, MA, USA Harvard Medical School, Cambridge, MA, USA

John van der Kamp Institute for Fundamental and Clinical Human Movement Studies, Vrije Universiteit, Amsterdam, The Netherlands

Paulion van Hof Institute for Fundamental and Clinical Human Movement Studies, Vrije Universiteit, Amsterdam, The Netherlands

Frank T. J. M. Zaal Institute of Human Movement Sciences, Rijksuniversiteit Groningen, Groningen, The Netherlands

This Page is Intentionally Left Blank

Time-to-Contact –- H. H. Hecht Hecht and and G.J.P. G.J.P. Savelsbergh Savelsbergh (Editors) (Editors) Time-to-Contact © 2004 2004 Elsevier Elsevier B.V. B.V. All All rights rights reserved reserved ©

CHAPTER 1 Theories of Time-to-Contact Judgment

Heiko Hecht Universitat Mainz, Mainz, Germany Massachusetts Institute of Technology, Cambridge, MA, USA

Geert J. P. Savelsbergh Vrije Universiteit, Amsterdam, The Netherlands Manchester Metropolitan University, UK

ABSTRACT Tau-theory has become one of the best researched topics in perceptual psychology. A comprehensive look at its accomplishments and failures is long overdue. The current chapter provides a framework designed to help organize the vast literature on the topic. This is done by first outlining the historical roots of the concept and placing it within the context of ecological psychology, which it has long transcended. Then the theoretical significance of x theory will be assessed. Strangely, it can be regarded as most influential and theoretically productive while being utterly wrong at the same time. We make some suggestions as to how one could further exploit the concept without being trapped by its historical baggage. Finally, we speculate on the necessary ingredients for future theorizing on time-to-contact estimation.

2


1. A brief history of tau As an object approaches an observer (or vice versa) the temporal moment of its arrival is specified by optical variables. In other words, the time until the object collides with the observer or passes her eye-plane is given by the relative rate of change of the retinal extent of the object (for the collision case) or by the relative rate of change of the angle between the object and the observer's line of sight (for most passage cases). The remarkable feature of this optical specification, often referred to as tau or the symbol (x) typically used to denote it, lies in the fact that neither object distance nor object size are required if one were to base a time-to-contact (TTC) judgment or a time-to-passage (TTP) judgment on this optically specified information. The question became whether human observers do take advantage of this specification or whether they make separate velocity and distance estimates and combine these into a TTC judgment. The idea that the optical expansion pattern is informative about objects and events can be traced back to Gibson (1947/1982). Gibson suggested that in the case of a moving observer 'the rate of expansion of the image of any point or object is inversely proportional to the distance of that point or object from the observer' (Gibson, 1947, 1982, p.41). The often cited first derivation of T (see e.g. Kaiser & Johnson, this volume) was published as a footnote in Fred Hoyle's (1957) science fiction novel, "The Black Cloud". The much less cited first psychological study on the use of x (Van der Kamp, 1999) to our knowledge stems from Knowles and Carel (1958). These authors wrote the following abstract for the 66the annual convention of the American Psychological Association (August 27-September 3 in Washington): "An analysis of the geometry of head-on-collision situation shows that the relative change of visual angle per unit time determines the time remaining to collision. This experiment was designed to reveal whether observers could utilize this kind of information in the absence of other cues such as familiar size, distance, speed etc. It was shown that for periods up to about four seconds estimates were surprisingly accurate. Beyond four seconds the times were progressively underestimated." Finally, David Lee and his collaborators initiated a field of research by suggesting that observers will use x when it is indeed optically specified. This formulation of tau-theory has become one of the most seminal theories in perceptual psychology (see Lee, 1976; Lee & Reddish, 1981; Lee, Young, Reddish, Lough & Clayton, 1983). The x information for TTC judgments is usually referred to as local x, as the optical information specifying TTC is available in local object parameters,

Theories of time-to-contact judgment judgment

3

such as its retinal area, its retinal width or similar parameters. Accordingly, the T information for TTP judgments is referred to as global x because the object's (or the observer's) path of movement has to be determined from the global flow field. Tau is then given by the instantaneous rate of change of the retinal angle between that path and the object's center. A number of assumptions have to hold for x to yield accurate TTC information. For instance, the object must not change shape or size as it approaches and it must approach at constant velocity. In that case, TTC is given by TTC = x = G /(d0 / dt), where 0 is the optical variable in question (e. g. object diameter as projected onto the retina). Obviously, additional assumptions about the observer's competencies must be made to the effect that she can register the angle 9 and somehow compute its derivative with respect to time. Now tau-theory states that observers are able to do so and base their TTC estimates on x. Tau-theory further states that in order to make TTC judgments the pickup of optical x information is necessary and sufficient. That is, other extraneous information, such as the absolute distance of the object, will not be used. It is understood that the TTC judgments in question here are meant to be meaningful, such as estimating when to initiate an arm movement to catch a ball or when to release an object that is to hit a passing target. Tau theory has inspired so many researchers and continues to do so because it has offered an approach that steered clear of the behaviorist associationism as well as cognitive theorizing. After the dismissal of behaviorism, the cognitive turn (Neisser, 1967) suggested to many psychologists that visual estimations are brought about in a manner similar to other thought processes. According to rational thought or a physicist's approach, the estimation of ingredient variables speed and distance would stand at the beginning of TTC estimation. The beauty of tau-theory lies in the fact that these initial estimations are entirely superfluous. The direct availability of TTC through x elegantly circumnavigates the time-consuming computations that would be required when using the physicist's approach. In this sense, x theory falls within the domain of ecological psychology as laid out by James Gibson (1979). Tau is an informational invariant to which the visual system has direct access, presumably by means of dedicated neural circuitry. Depending on the makeup of this circuitry, what may look complex and intensely computational to the physicist may be very easy for the dedicated processor. The visual system has accordingly been likened to a smart perceptual device, such as a speedometer or a polar planimeter (Runeson, 1977). Their makeup produces a complex quantity as output although the device is simple. And if the nature of the perceptual system is making use of "smart" devices then research should be geared to finding thoseinvariants that are directly accessed by the smart device. Tau has become the prime example of an invariant that fulfills these criteria.

4


The fact that tau-theory emerged at a time when the ecological approach was still in its infancy and its proponents mainly occupied with honing their critique of cognitivist and computational theories, has made for a strange research strategy on the part of ecological psychologists. Rather than testing tautheory for its own sake, the programme was to prop up the prime example for the validity of the ecological approach. Thus, x became a test case for direct perception before it had been properly tested itself. Research results were amassed that showed how timed actions were rather accurate when assessed within an ecological context. The good performance was taken as proof for tautheory. This verificationist attitude, as understandable as it is, has delayed such obvious things as a rigorous psychophysical evaluation of TTC estimation until fairly recently (see e. g. Regan & Hamstra, 1993).

2. The tau hypothesis and its falsification In essence, x theory proper has to be considered as falsified. It is irreconcilable with empirical data in a twofold manner, because sometimes performance is worse than predicted and at other instances performance is surprisingly good even in the absence of x information. Recent work on TTC estimation has revealed that when x information is available, observers often fail to use it properly. For instance, absolute size of the object, rotation, and contrast all interfere with estimation accuracy when facing a clearly visible target (e.g., DeLucia, 1999; Smith, Flach, Dittman & Stanard, 2001). That is, the visual system often is unable to isolate x. On the other hand, judgments are remarkably robust when invariant information is no longer available. For instance, when dramatically reducing the density of an optic flow field consisting of single-pixel dots performance is affected little, suggesting that some image characteristics that covary with x are used as basis for TTC judgments (Kerzel, Hecht & Kim, 1999). Observers seem to adjust to many disturbances, which should not be surprising because they are typical rather than exceptional for ecological contexts. That is, the constancy assumptions that are part and parcel of x theory are more often violated than not. A theory of TTC estimation thus has to explain how such estimates are made in the face of non-rigid size changes during approach, in the face of orientation change of non-spherical objects as well as velocity variations. All of these would fool a x processor that is based on retinal edge distance. Note, however, that for all practical purposes more than one x parameter is specified. For instance, the retinal width of a vertical brick rotating around its center (pitch axis) that is approaching head-on at constant velocity, would yield accurate x information while its retinal length and area would lead to erroneous TTC estimates. Depending on rotation speed length changes could be nulled or


5

exaggerated. In terms of tau-theory then what happens if several different TTCs are specified simultaneously? As long as the theory does not state upon which of the taus the TTC estimate is based, performance can likely be attributed to one of the many taus that are present in ecological situations.

3. Tau's significance within and outside ecological psychology In sum, tau-theory can be regarded as most influential, especially within the framework known as the ecological psychology. Gibson's (1979) ecological approach to perception is also known as the direct perception perspective. The word 'direct' refers to the fact that objects, places and events in the environment can be perceived without the need of cognitive mediation to make perception meaningful. The importance of the ecological psychology contribution lies in its emphasis on the nature of information which forms the basis for perception and action. Gibson argued that information is embedded in the optic array, that is, the pattern of light coming from all directions of the environment to a point of observation. Changes in and persistence of patterns in the optic array can be shown to specify the environment. When such information is detected, objects, places and events can be perceived without the need for 'cognitive' processing. Moreover, these optical patterns are determined not only by objects, places and events in the environment, but also by movements of the observer. Examples of structured optical patterns are texture gradients, occlusion patterns (decretion and accretion of texture), motion perspective, and focus of expansion and rate of expansion. Within this perspective, information implies specificity, which is that the information is lawful related to its source (e.g. object, events) such that no other source could have generated that particular pattern (Burton & Turvey, 1990). Moreover, not only the relation between the source and information is specific, but also the relation between information and perception. In other words, the perception of dynamic events entails the detection of a single information variable specifying the event (Michaels & de Vries, 1998). Tau is an informational variable, which seems to pass this ecological exam becoming therefore very popular and incredible important in theory building and experiments. For example, that x is actually used in the control of catching was found in the 'deflating ball experiment' (Savelsbergh, Whiting, & Bootsma, 1991; Savelsbergh, Whiting, Pijpers, & van Santvoord, 1993). Participants were required to catch a ball that was large, small or deflating during approach. The deflating ball results in a lower rate of dilation specifying a larger time to contact, that is, a higher tau-value. The experiments showed that indeed the timing of the hand occurred later for the deflating ball, which indicates that the timing was clearly affected. This was anticipated from a x perspective, and thus, the deflating ball experiment provides qualitative evidence for the use of x in

6

Heiko Hecht and Geert J. P. Savelsbergh Savelsbergh

closing the hand. However, it was also found that the influence of the dilation manipulation was much more pronounced in the monocular viewing condition than in the binocular viewing condition (see also Savelsbergh, 1995). This suggests the contribution of binocular information as well. Additional empirical evidence stems from a replication of the 'deflating ball' study by Van der Kamp (1999), who used next to deflating balls, also inflating balls. The qualitative effects were in agreement with the Savelsbergh et al. (1991) findings, namely, the opening and closing of the hand occurred later (deflating) and earlier (inflating). However, the magnitude of these effects is much smaller than would be predicted on basis of x (see Figure 2 of Chapter 19). It is clear that other information, such as optical size and the rate of change of optical size covary with x, and may therefore also be involved. Together, however, instead of an exclusive regulation on basis of x, the deflating ball experiments point to the use of other information depending on the constraints of the task. Outside the domain of ecological psychology, x has received recognition in two very different ways. First, brain scientists have been inspired by tautheory to search for neural correlates of a x processor without a theoretical need to find it. They have been successful with animal studies (e. g. Sun & Frost, 1998). Second, un-ecological scientists have ventured with their own theoretical and methodological backgrounds into the ecological terrain of real-world perceptual tasks. This has facilitated behavioral studies on human observers that were sympathetic to the domain but critical to the tenets of tau-theory (e. g. Rushton, this volume). Being a prime example of an invariant, x featured surprisingly little in the debate on dynamic event perception (see Hecht, 1996). Direct perceptionists favor a direct availability of complex event characteristics through the kinematic motion information available in the optic array (Runeson & Frykholm, 1983) while others favor an information processing approach suggesting that the visual system uses flexible heuristics (Gilden, 1991). It appears that the status of x theory can considerably strengthen or weaken the direct position. The latter position as a framework for our thinking about timed events is seriously under attack as counterevidence against tau-theory continues to grow.

4. Exhausting the concept The above verdict that x theory is falsified does not imply, of course, that the theory should be thrown out without considering how it could be exploited further or even be salvaged (see Holzkamp, 1972, on the exhaustion principle). We can discern four ways to augment x theory, which is schematically represented in Figure 1.

Theories of time-to-contact judgment

7

tau Processor Motor response TTC estimate

Figure 1: Model of tau-theory. If optical x information is available, the system automatically generates a TTC estimate exclusively based on that information. The motor response is triggered when x has reached a given critical value. The x processor is necessary and sufficient for TTC estimates to be generated.

First, it can be amplified to include all possible x variables. This would be most conservative and basically chain together a number of x processors. All that needs to be added is a rule that determines which of the parallel tauparameters is used for TTC estimation. Second, the output of the x processor could be interpreted and modified by an added processing step. This has been suggested by Tresilian (1995) to the effect that fast actions, such as batting in baseball would receive little or no added processing while slower actions accommodate such processing. Other visual information, speculation or learned strategies can enter this added stage. This can explain errors in prediction motion (PM) tasks (see Figure 2).

tau processor

post processing

motor response

Figure 2: Tau theory augmented by a cognitive post-processing step that interprets the output of the x processor

8

Heiko Hecht and Geert Geert J. P. Savelsbergh Savelsbergh

Finally, a complex theory off TTC could incorporate the gamut of optical variables as well as heuristical rules that compete for input into the TTC estimate, thereby pulling the ecological footing from underneath x theory. The list of competitors can include all potential taus as well as simpler optical parameters, such as rate of expansion or retinal velocity, and non-optical information and strategies. This model is illustrated in Figure 3. tau processor *• optical ^ information ) •*-^..

..

tau processor

\

\

— " * ^ ,

TTC estimate

visual heuristic

motor response

/ '•"••-.

/

"•••-..

motor strategy

•-•-.,

extra-retinal information

Figure 3: Alternate model: The output of the tau-processor(s) is neither necessary nor sufficient. It exists together with other units that can inform a TTC estimate. If only a coarse estimate is needed quickly, for instance to make a decision to move out of harm's way, a simple heuristic may suffice to trigger a motor response. In other cases the tauprocessor may be engaged.

The model sketched in Figure 3 appears quite plausible, but it may in fact be too plausible to be of value. It has pretty much covered all the bases for TTC judgments. It will only start to be productive once all t processors are spelled out and possibly linked to some neural substrate. It also needs to be supplemented with rules stating under what circumstances extra-retinal or heuritstic information is considered. Once the model is filled-in in those terms it hardly deserves to be called x theory. Rather, it will be a complex theory of TTC judgment. The current book provides the building blocks for such a theory of TTC. The following paragraph provides a framework suggesting how one might relate the buildings blocks.

Theories Theories of time-to-contact time-to-contact judgment

9

5. The future of tau Given the multiple ways to exhaust the concept of x, as the research community we have to decide which strategy is best in the sense that it promises to yield the most innovative results. To do so, we need to take into consideration that the visual system may be considerably more flexible than monolithic theories can capture. The visual system may use different strategies depending on the accuracy that is required for a given action. It might use simple heuristical rules, or cheap tricks as the utilitarian notion that the visual system is using a bag of tricks would imply (Ramachandran, 1990). However, the system appears to be smarter than that. The visual system might systematically use different strategies at different times. If this were the case, we cannot discard the use of some variable for good based upon evidence that it is not used in some particular cases. This selective use of strategies as a function of task demands has been introduced as satisficing into cognitive theorizing (Simon, 1969, 1982). In analogy, the visual system can be taken to function as an entity bounded by its capacity limitations which create uncertainty about the future and about costs of information acquisition. It finds one solution to a perceptual problem, among a number of more or less costly solutions, that is sufficiently accurate and as easy to achieve as possible given the required accuracy level. When Regan and Gray (this volume) refer to task-relevant variables they can be interpreted to elaborate the notion of satisficing. A given optical variable that can in principle be used is in fact only used when the desired action elicits it. Only by experimentally holding constant all covariates of a given optical variable can subjects be forced to use the given variable. We need to re-evaluate existing studies under these premises. This way a large body of seemingly contradictory findings can be reconciled, such as some authors reporting excellent performance in natural settings while others find that non-tau variables, such as absolute size, retinal velocity etc. compromise TTC estimation. Likewise, we need modify the way experiments on TTC judgment are being designed. This re-conceptualization of TTC has already begun, as many of the chapters in this volume will prove.

10 10


REFERENCES Burton, G. & Turvey, M.T. (1990). Perceiving the lengths of rods that are held but not wielded. Ecological Psychology, 2,_295-224. DeLucia, P. R. (1999). Size-arrival effects: The potential roles of conflicts between monocular and binocular time-to-contact information, and of computer aliasing. Perception & Psychophysics, 61, 1168-1177. Gibson, J. J. (1947, 1982). Perception and judgements of aerial space and distance as potential factors in pilot selection training. In E. Reed & R. Jones (Eds.), Reasons for realism: Selected essays of James J. Gibson (pp.29-43.) Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers Gibson, J. J. (1979). The ecological approach to visual perception. Hillsdale, NJ: Lawrence Erlbaum. Gilden, D. L. (1991). On the origins of dynamical awareness. Psychological Review, 98, 554-568. Hecht, H. (1996). Heuristics and invariants in dynamic event perception: Immunized concepts or non-statements? Psychonomic Bulletin and Review, 3, 61-70. Hoyle, F. (1957). The black cloud. London: Heineman. Holzkamp, K. (1972). Kritische Psychologie: Vorbereitende Arbeiten. Frankfurt a. M.: Fischer Taschenbuch Verlag. Kerzel, D., Hecht, H. & Kim, N. G. (1999). Image velocity, not tau, explains arrival-time judgments from global optical flow. Journal of Experimental Psychology: Human Perception and Performance, 25, 1540-1555. Knowles W. B. & Carel, W. L. (1958). Estimating time-to-collistion. American Psychologist, 13, 405-506. Lee, D. N. (1976). A theory of visual control of braking based on information about time-tocollision. Perception, 5, 437-459. Lee, D. N. & Reddish, P. E. (1981). Plummeting gannets: A paradigm of ecological optics. Nature, 293, 293-294. Lee, D. N., Young, D. S., Reddish, P. E., Lough, S. & Clayton, T. (1983). Visual timing in hitting an accelerating ball. Quarterly Journal of Experimental Psychology, 35A, 333-346. Michaels, C. F. & de Vries, M. (1998). Higher order and lower order variables in the visual perception of relative pulling force. Journal of Experimental Psychology: Human Perception and Performance, 24, 526-546. Neisser, U. (1967). Cognitive Psychology. N.Y.: Appleton. Ramachandran, V. S. (1990). Interaction between motion, depth, color and form: the utilitarian theory of perception. In C. Blackmore (Eds), Vision: Coding and Efficiency, (pp. 347360), Cambrige: Cambrige University press. Regan, D. M. & Hamstra, S. (1993). Dissociation of discrimination thresholds for time to contact and for rate of angular expansion. Vision Research, 33,447-462. Runeson, S. (1977). On the possibility of "smart" perceptual mechanisms. Scandinavian Journal of Psychology, 18, 172-179.


11

Runeson, S. & Frykholm, G. (1983). Kinematic specification of dynamics as an informational basis for person-and-action perception; Expectation, gender recognition, and deceptive intention. Journal of Experimental Psychology: General, 112(4), 585-615. Savelsbergh, G. J. P. (1995). Catching "Grasping tau". Human Movement Science, 14, 125-127. Savelsbergh, G. J. P., Whiting, H.T.A. & Bootsma, R.J. (1991). 'Grasping' Tau. Journal of Experimental Psychology: Human Perception and Performance, 17,315- 322. Savelsbergh, G. J. P., Whiting, H.T.A., Pijper, R.I. & Santvoord, van A.A.M. (1993).The visual guidance of catching. Experimental Brain Research, 93, 148-156. Simon, H. A. (1969). The sciences of the artificial (2nd edition 1981). Cambridge, MA: MIT Press. Simon, H. A. (1982). Models of bounded rationality. Cambridge, MA: MIT Press. Smith M. R., Flach, J.M., Dittman, S.M. & Stanard, T. (2001). Monocular optical constraints on collision control. Journal of Experimental Psychology: Human Perception and Performance, 27, 395-410. Sun, H. & Frost, B. J. (1998). Computation of different optical variables of looming objects in pigeon nucleus rotundus neurons. Nature Neuroscience, 1, 296-303. Tresilian, J. R. (1995). Perceptual and cognitive processes in time-to-contact estimation: Analysis of prediction-motion and relative judgment tasks. Perception & Psychophysics, 57, 231245. Van der Kamp, J. (1999). The information-based regulation of interceptive timing. Nieuwegein: Digital Printing Partners Utrecht B.V.


Time-to-Contact Time-to-Contact – - H. Hecht and G.J.P. Savelsbergh (Editors) (Editors) © 2004 Elsevier B.V. All rights reserved

CHAPTER 2 The Biological Bases of Time-to-Collision Computation

Barrie J. Frost Queen's University, Kingston, Ontario, Canada

Hongjin Sun McMaster University, Hamilton, Ontario, Canada

ABSTRACT We begin the chapter by arguing that there may be several neural mechanisms that have evolved for computing time-to-collision (TTC) information as a way of controlling different classes of action. We then focus on single unit mechanisms responsible for processing the impending collision of a moving object towards a stationary observer. After discussing TTC processing in the invertebrate visual system, we describe our own work involving neurons in the pigeon nucleus rotundus that respond exclusively to visual information relating to objects that are approaching on a direct collision course, but not to visual information simulating observer's movement towards those same stationary objects. Based on the recorded neuronal responses to various manipulations of the stimuli, we classified these looming sensitive neurons into three different types of looming detectors based on the temporal differences in neuronal response relative to the moment of collision. We also described quantitative models for these looming detectors as a way of explaining their physiological response properties.

14 14

Barrie J. Frost Frost and Hongjin Sun

1. Introduction Information about the time-to-collision or time-to-contact (TTC) has important consequences for the survival of countless species and for their skilled interaction with both the inanimate and animate objects in their environments. As a consequence it appears very probable that there may be several neural mechanisms that have evolved to compute TTC information to control different classes of action, and even different mechanisms within the same animal for different functions. For example, it appears unlikely that mechanisms that have evolved in birds for avoidance of rapidly approaching objects such as predators, where critical and rapid evasive maneuvers are required, are the same mechanisms that control pinpoint landing on branches. In the former case the motion of the approaching predator will primarily determine TTC, whereas in the latter it is only the self-motion of the animal approaching the stationary branch that determines TTC. Of course there may be many instances where both self-motion and motion of another animal determine TTC. For an excellent review that puts TTC in a much broader context the reader is referred to a paper by Cutting, Vishton and Braren (1995). One way that may help conceptualize these factors is to subdivide the primary stimulus determinants of TTC on the one hand, and the general nature of responses controlled by the information on the other, and place these in a simple 2 x 2 table as illustrated in Table 1. Here we have divided the world simply into stationary and moving objects on the vertical axis, and the behavioural responses into approach and avoidance on the other. Examples of TTC studies falling in cell 1 (Stationary objects/Approach behaviour) are the

Source of LoomingStimulus

Behavioural Output

Self-motion towards stationary objects

1

• • •

Insect's landing Bird's landing Human or gerbil approaching towards target

2

• •

Avoiding obstacles Avoiding cliffs and drop offs

Moving objects (towards and away from the observer)

3

• • • •

Pursuit - prey capture Pursuing mates Flock formation Ball catching

4

• •

Predator avoidance Avoiding aggressive encounters

Approach

Avoidance

Table 1: Situations Requiring TTC Information

The Biological Bases of Time-to-Collision Computation

15

landing response of the milkweed bug, Oncopeltus fasciatus (Coggshall, 1972) and the fly (Wagner, 1982). The aerodynamic folding of gannet wings just prior to their entry into water (Lee & Reddish, 1981), birds landing on stationary perches (Lee, Davies, Green & Van der Weel, 1993) or human subjects braking to avoid collision with stationary barriers (Sun & Frost, 1997) or gerbil behaviour of running towards target (Sun, Carey & Goodale, 1992) are other examples that fall into this category. Examples of behaviour falling in cell 2 (Stationary objects/Avoidance) would involve negotiating paths through a cluttered environment where obstacles have to be avoided. This might include steering around barriers, and avoiding holes or sudden drop offs. There appear to be few studies of TTC detection in this category, but Cutting et al.'s (1995) study of path interceptions may be relevant. Prey capture by predators and pursuit chasing during mating could well satisfy entry into cell 3 (Moving objects/Approach), although not all studies of this behaviour have focused on TTC information. Ball catching behaviour, and batting in sports seem also to be appropriate exemplars of this category. Escape from rapidly approaching predators or threatening rivals in territorial mating would be prime example of cell 4 (Moving objects/Avoidance). Throughout the animal kingdom the sight of a rapidly approaching object almost universally signals danger and elicits an escape or avoidance response. When confronted with such a looming stimulus, the visual system must determine precisely the 3D flight path, and compute the TTC of the object, to provide the information necessary for eliciting and controlling the appropriate evasive action (Fishman & Tallaroco, 1961; Schiff et al., 1962; Schiff, 1965; Hayes & Saiff, 1967; Tronick, 1967; Bower et al., 1970; Ball & Tronick, 1971; Dill, 1974; Ingle & Shook, 1985; Yonas & Granrud, 1985). Our own work on neurons in the pigeon nucleus rotundus of pigeons clearly fits in this category because these neurons respond only to the direct collision course of approaching objects (Wang & Frost, 1992, Sun & Frost, 1998), and not to simulation of the movement of pigeons toward the same stationary objects (Sun & Frost, submitted). Also the work on locust looming detectors would fit this category because of the demonstrated elicitation of jumping and flying by the same expanding stimulus patterns that optimally excites the Lobula Giant Movement Detector (LGMD) and the Descending Contralateral Movement detector (DCMD) neurons (Rind & Simmons, 1992, 1999; Hatsopoulos, Gabbiani, Laurent, 1995). Of course it should be remembered that the necessity to compute TTC first requires that any object or surface be indeed on a collision course if the present path of the observing or approaching animal is maintained. Gibson (1979) in his classic work on ecological optics suggests that symmetrical expansion of the images of objects specifies direct approach along a course that will

16 16


ultimately result in collision with continuous motion. The advantage of using this strategy is that TTC can be computed using monocular information alone. It is possible that, for animals with well-developed binocular stereoscopic visual systems, subpopulations of neurons that respond to stereoscopic motion directions specifying object-motion paths directly toward the animal, might also be used for TTC computations. In this work and his other writings, Gibson also made the clear distinction between collisions with stationary objects occasioned by the motion of the observer (row 1 in Table 1) and other cases where it is the approaching object's motion itself that will result in collision if it continues along this path (row 2 in Table 1). In this chapter we will focus primarily on research that falls in cell 4 simply because it appears that most of the empirical studies about neural mechanisms of TTC have used stimulus arrangements that simulate events that fall into this category, that is, a rapidly approaching object on a direct collision course with the observing animal, and which might therefore require some sort of evasive action or avoidance response on the part of the animal. In the other category of object motion where the observing animal is trying to arrange a collision with the moving object such as the prey capture behaviour of dragonflies (Olberg, Worthington & Venator, 2000), similar processing mechanisms may occur.

2. TTC in the invertebrate visual system Flying insects have long been used as model systems because they exhibit spectacular aerial performance and accomplish this with relatively simple neural computational mechanisms. Moreover since the same neurons can be identified from animal to animal the neural circuitry is often amenable to analysis. Two such neurons, LGMD and DCMD in the locust, that are synaptically linked, have been shown to be selectively responsive to approaching objects (Rind & Simmons, 1992; Rind, 1997; Hatsopoulos et al., 1995). Neurons that respond to changes in depth have also been found in optic lobes of the hawk moth, Manduca sexta (Wicklein & Strausfeld, 2000), but these may be examples of neurons computing approach and recession for the control of self motion, in this case controlling the hovering flight in front of flowers during nectar collection, rather than for the computation of TTC. Because the DCMD neurons can be readily recorded extracellularly, have very large receptive fields, and respond well to the movement of objects, they have been studied extensively for many years. Schlotterer (1977) was the first to use approaching stimuli to show that DCMD neurons were more responsive to approaching objects than other 2D patterns of movement. Rind and her colleagues, and Laurent and his colleagues have extensively studied these neu-


17

rons using a variety of stimuli and confirmed that symmetrical expansion generated by an approaching stimulus object is the critical stimulus variable that optimally fires these cells. The allocation of the LGMD - DCMD neurons to cell 4 of our schema presented in Table 1 is justified by their connection to pre-motor interneurons and motor-neurons known to be involved in flying and jumping (Burrows & Rowell, 1973; Pearson et al, 1980; Simmons, 1980). This is further supported by the studies of Robertson and Johnson (1993a, 1993b) who have shown in tethered, flying locusts, that approaching objects elicit a steering avoidance response when the approaching object reaches a critical angular size, thus indicating that some thresholding probably occurs in this pathway. What are the critical features of a symmetrically expanding image that these locust DCMD neurons are responding to that generates their specificity to approaching objects? From an analysis of the several possible cues available in the monocular image Rind and her colleagues (Simmons & Rind, 1992; Rind & Simmons, 1992) have shown that these neurons do not register changes in overall luminance since they respond in a similar manner when light objects approach as when dark objects approach, and their responses were much smaller to sudden luminance change per se. Divergence of two lines moving in opposite directions, to partially represent the opposite contours of a symmetrically expanding object, also did not adequately stimulate DCMD neurons, but increasing the amount of edges in the Receptive Field (RF) and increasing the velocity of edges appeared to be the critical trigger features. Judge and Rind (1997; see also Rind & Simmons 1999) have shown that these locust looming sensitive neurons are very tightly tuned to the direct collision course. Stimulation of the locust retina in one area suppresses LGMD response to a second stimulus presented elsewhere in the visual field, thus indicating there are lateral inhibitory mechanisms operating. Indeed if the appropriate experiments were to be performed one might well find the RF characteristics are similar to those found in the tectofugal or collicular-pulvinar pathway of vertebrates where a directionally specific, double opponent RF organization occurs to ensure that these cells respond to moving objects, and not to the large patterns of optic flow produced by the animal's self motion (Frost 1978; Frost, Scilley & Wong, 1981; Frost, Cavanagh & Morgan, 1988, Sun, Zhao, Southall & Xu, 2002). From a functional point of view this also implies that the LGMD neurons might be interested in approaching objects that fall into cell 4 of our matrix, and not in the locust's approach toward stationary features in its environment. According to Rind and Simmons (1999) the specificity of the LGMD for approaching images is generated by a "critical race over the dendrites of the LGMD in the optic lobe". The two competitive forces in the race are the excitation produced by the moving edges of the expanding image, and lateral inhibition mediated by neurons in the medulla also synapse onto the LGMD. Rind and Bramwell (1996) have produced a neural network model which seems to support

18 18


this view and have also shown through electron microscopy that the anatomical arrangement of presynaptic connections to LGMD are compatible with this interpretation. Hatopoulus, Gabbiani and Laurent (1995) have also investigated the LGMD of locusts, and shown that this neuron fires with an increasing rate as an object approaches, then peaks, and drops off just before collision occurs. They have shown that the responses are typically brisker for fast moving or smaller objects, but the peak firing rate does not appears to solely depend on the approach speed or object size. They describe the peak as always exhibiting a constant latency after the time at which the object reaches a fixed angular threshold size on the eye (Gabbiani, et al, 1999). These authors have suggested that the behaviour of the LGMD is best described by the following equation: f(t)=Cx0'(t)Xeam

(1)

Here, 0 is visual angular subtense, C is a proportionality constant. In contrast to Rind and her associates view, these authors in recent papers (Gabbiani, et al, 2001; Gabbiani et al., 2002) have suggested that the LGMD postsynaptically multiplies an excitatory and inhibitory input via two different parts of LGMD neuron's dendritic tree. In order to provide evidence in support of this model these authors (Gabbiani et al., 2002) have selectively activated and deactivated pre-and post synaptic inhibition, and have found that it is post-synaptic inhibition that plays a critical role in shaping the temporal response profile of these neurons, and this indicates that the multiplication takes place within the LGMD neuron itself. These findings are noteworthy for two reasons: in the first place they show in a detailed way how these computations which provide information about looming object are accomplished within the neural machinery of the LGMD and its synaptic connections, and secondly they provide one of the first pieces of clear evidence for how multiplication (and division) is accomplished in the nervous system.

3. Neurons that compute tau in the pigeon brain A number of behavioral studies have revealed that the tectofugal pathway in vertebrates might be involved in processing the visual information necessary for generating such escape or avoidance action. Electrical stimulation experiments indicated that the anuran optic tectum is involved in triggering both prey-catching and also various kinds of avoidance behaviours (Ewert, 1984), and ablation of the optic tectum resulted in abolition of all visually guided preycatching and visual avoidance behaviour (Bechterew, 1984, cf: GriisserCornehls, 1984). Electrical or chemical stimulation of the superior colliculus in

The Biological Bases of Time-to-Collision Computation ofTime-to-Collision

19

rats also results in defensive-like reactions (Redgrave et al., 1981; Sahibzada et al., 1986; Dean et al., 1988) and is associated with large increases in blood pressure and heart rate (Keay et al., 1988). Pigeons with bilateral lesions of the optic tectum or/and the nucleus rotundus not only showed substantial impairment in intensity, colour, and pattern discrimination, but also exhibited severe deficits in visually guided orientation, escape or avoidance behaviour (Hodos & Karten, 1966; Hodos, 1969; Hodos & Bonbright, 1974; Jarvis, 1974; Bessette & Hodos, 1989). Wild rats with collicular lesions may ignore an approaching human (Blanchard et al., 1981) and similar results have been reported in hamsters and gerbils (Ellard & Goodale, 1986; Northmore et al., 1988). This evidence provides a vivid illustration of the importance of the tectofugal pathway in guiding orientation, detecting approaching objects, and generating escape or avoidance behaviours. Over the years several investigators have claimed that they have encountered cells that respond specifically to objects approaching the eye on a direct collision course. For example, as early as 1976 Griisser and Grusser-Cornehls (1976) and later Ewert (1984) reported that some frog and toad tectal neurons respond vigorously to stimuli moving on paths directly towards the eye. However from these early studies many of the appropriate controls were not performed to conclusively exclude the possibility that these neurons were simply responding to some aspect of the lateral motion of an approaching stimulus. It should be remembered that as an approaching image expands, obviously there will be 2D motion of the edges of the object and its textures and if the expansion is placed asymmetrically over a standard 2D directionally specific motion it could artifactually stimulate the neuron to give a false impression it is responsive to approaching stimuli. We also had encountered cells we thought were responding to the direct approach path of moving objects in 1983, but it was only when we had extremely well-controlled stimuli, which we could systematically vary in their simulated 3D paths, that we could finally convince ourselves that these neurons were indeed coding some aspect of 3D motion. In 1992 Wang and Frost showed that some neurons located in the dorsal posterior regions of the nucleus rotundus of pigeons responded specifically to the direct approach direction of a soccer ball pattern. Using the 3D imaging capabilities of a Silicon Graphics computer we were able to move this soccer-ball stimulus in any trajectory in 3D space. By performing very time consuming 3D tuning curves on these cells we were able to show that this rotundal subpopulation would only respond when the soccer ball stimulus was on a direct collision course with the bird's head. We chose a soccer ball because the spaceaverage mean luminance did not change as this stimulus expanded and contracted in size (especially when the object moved against a stationary background with the same texture pattern), and it provided many moving and expanding/contracting elements that might be necessary for these neurons to re-

20


spond. Earlier studies, and often some more recent ones, use a simple expanding square or circle where changes in luminance obviously occur concurrently with the expansion/contraction of stimuli, and this necessitates several other controls to rule out this variable as the major contributor to the responsiveness of the neuron. Also other studies have not specifically performed 3D tuning curves to quantify the true directional tuning of neurons of this type. Figure 1 shows the typical 3D tuning curves of one of these rotundal neurons.

Figure 1: A. A soccer-ball-like stimulus pattern consisting of black and white panels, was moved along simulated 3D trajectories 45° apart in spherical coordinates. The diagram illustrates the 4 planes along which stimuli were moved. B. A typical single neuron from the nucleus rotundus of pigeons exhibiting clear selectivity for a looming visual stimulus. Firing rate is plotted for the different directions of motion of the soccer-ball stimulus in 3D space. Each direction of motion was presented 5 times in a randomly interleaved sequence, and the values plotted represent the mean peak firing rate for each 3D direction. Note that in the standard X-Y (tangent screen plane, or Azimuth = 90°) plot, there is no indication of directional preference, and firing rate is quite low. However, for the 0 azimuthal plane (Zaxis) there is a strong preference for stimuli directly approaching the bird (0°). Polar tuning plots for directions specifying the azimuthal 45 and 135° planes likewise show no strong preference for any direction. Thus it is only the direct collision course or looming direction that produces an increased response in these neurons, and this pattern of activity was typical of the 27 neurons studied in the dorsal posterior area of the nucleus. (© Wang and Frost, 1992, Nature).


21

Even with 26 directions, 45 degrees apart these are relatively crude tuning curves, so in a few cases we have used a much narrower range of directions after having first performed the broad 3D tuning and found them to be very tightly tuned indeed (Sun & Frost, submitted). In fact the half-width and half-height of detailed tuning curves like those illustrated in Figure 2 is about 4 degrees, where the rotation is around a point halfway along the simulated 15 meter path taken by the approaching stimulus of 30 cm in size. This means in simple terms

Azimuthal direction of object motion (degrees)

Figure 2: Fine tuning of two cells located in the pigeon dorsal nucleus rotundus. First these cells were presented with a soccer ball stimulus that moved in 26 directions 45 degrees apart in 3D space, and they only responded to the direct collision course direction. The graphs shown here are the fine grained tuning curves and show that when the soccer ball, which traveled along a simulated path of 15 meters, was rotated by small amounts each time passing through the center of the path, the cells reduced their firing substantially. The few degrees of rotation of the path indicate that now the soccer ball would travel in a "near miss" and not collide with the bird.

that a stimulus that depicts a very "near miss" of the bird's head will only fire the neurons minimally, and one that is a clear "miss" will not evoke any response at all. Perhaps the most important defining character of these neurons' responses, in addition to their sensitivity to the direct collision course direction, was the constant time they fired before collision, irrespective of the size of the simulated approaching soccer-ball, or of its approach velocity (Wang & Frost, 1992). This indicated to us that these neurons might well be computing the optical variable tau that had been suggested by Lee (1976) to provide important information about the TTC with approaching objects.


22

We also found that there were a variety of times before collision that the population of neurons exhibiting these characteristics showed. This can be seen in Figure 3A. Distribution of Tc Values 8"

I 6

ll 800

900

1000

1100

1200

1300

1400 ms

to

B

Single-cell Variability 8"

=

S 6 "5

2

•

800

900

1000

1100

1200

1300

1400 ms

Tc

Figure 3: A. Distribution of different response onset time for 27 looming cells from the dorsal posterior zone of nucleus rotundus of pigeons. Although different cells exhibit different values of TTC, individual cells (B) show remarkable consistency even when velocity or size of stimulus is varied.

But for a particular neuron, the variation in its response onset was remarkably constant (see Figure 3B) on repeated trials and with stimuli of different sizes and velocities. This variation in the population is precisely what is needed if other factors, such as recognition of what the incoming object is, can jointly influence the time selected to perform escape responses of different sorts, each of which may have characteristic time requirements for their optimal deployment. Clearly the characteristics of these rotundal neurons suggested to us that they might be involved among other things in predator avoidance and thus fall in to cell 4 of our classification system shown in Table 1. To provide some evi-


23

dence for this we tested a few birds under a lightened anesthesia at the conclusion of their recording session. Under deep anesthesia no electromyelograhic signals (EMGs) are obtained from animals, but as the anesthetic is lightened and clearly before any pain stimuli can be experienced, it is possible to obtain good clear muscle responses. These responses do not result in any overt movement of the animal, but can be very useful in indicating what major muscle groups might be involved in a response system normally associated with a stimulus. In this case we recorded from the large pectoralis muscles that power the wings for flight. When we presented the approaching soccer-ball stimulus under these conditions we found that first the tau cells responded with their characteristically maintained burst of firing, then the pectoralis EMGs occurred some 200 milliseconds later, and then finally the heart rate went more slowly up to levels near 300% of the resting rate. These responses again were incredibly specific, and only occurred when the soccer-ball was on a direct collision course with the bird. Near misses and directions 180 degrees away showed no increased EMGs or increases in heart rate. Data typical of these experimental findings can be seen in Figure 4. Although only correlative, we feel this constellation of activity in these tau neurons, and the increased wing EMG and heart rate are indicative of a flight response elicited by the rapidly approaching ball. In a more detailed recent study Sun and Frost (1998) have again confirmed the presence of a population of neurons in nucleus rotundus of pigeons that only respond when the soccer-ball stimulus is on a direct collision course with the bird's head. Additionally, we found that these neurons only responded when our computer simulated the approach of a moving object towards the bird (stimuli falling into cell 4 of the matrix), and not when the complex stimulus pattern was configured to simulate the bird moving towards the same stationary soccer-ball (stimuli falling into cell 1, or possibly 2 of the matrix) (Sun & Frost, submitted).

24


B Visual response

Visual response

0 Heart rate

-I

*

1

Figure 4: Heart rate and pectoralis muscle EMGs recorded concurrently with single cell response rate from a looming selective rotundal neuron. Note that the "looming cell" begins firing first, then the muscle response occurs, and then heart rate increases dramatically when the soccer-ball looms toward the bird (A). B. No responses occur when the ball moves along the same path but in the opposite direction directly away from the bird. Bars under the visual response histograms indicate the duration of the visual stimulus. Data collection for the looming-selective neuron was terminated with stimulus offset. The neuronal data and EMGs represent the summed activity over 5 sweeps of the stimulus whereas the heart rate data represent the means and standard errors for the same 5 sweeps. Simulated size of stimulus was 30 cm, path length 15 m, and velocity 375 m/s. (© Wang and Frost, 1992, Nature).

To do this we placed the soccer ball against a stationary background, which consisted of a checkerboard pattern. When the soccer-ball was moved in a trajectory towards the bird (symmetrical expansion) while the background remained stationary, the neurons responded in the typical way and identically to the case where no background was present. However, when the background was moved along the same trajectory as the soccer-ball, so as to show a similar but delayed expanding pattern, the neurons did not respond. This latter configuration formed the precise simulation of the bird approaching a stationary soccer-ball that remained a constant distance in front of the background "brick wall". The stimulus conditions simulating a soccer ball approaching the bird, and the bird approaching a stationary soccer ball are shown in Figure 5.


L White stationary background

25

Image Change

Checker board stationary background

Background approaching with object

TIME 1

TIME 2

TIME 3

Figure 5: Schematic diagram of stimulus conditions presented to pigeons. The left portion of the figure illustrates the kind of object movement and its background in the simulated display, while the right portion represents the image change on the screen for the corresponding simulation illustrated on the left. In A, the soccer ball object is presented against a blank background, and the "path" simulates direct approach toward the bird. In B, the same looming object is presented against a stationary textured checkerboard background. In C, both object and textured background move (at the same speed) toward the bird, which simulates the bird's self-motion toward a stationary soccer ball. Note that the expansion pattern of motion of the object is identical in the three conditions. (© modified after Sun and Frost, 1998, Chapter).

It must be emphasized that the expansion pattern of the soccer-ball was identical in these to two cases of moving object and moving bird simulations, yet in the former the cells responded vigorously, while in the latter they were essentially silent. Figure 6 shows the responses of a tau neuron to an approaching soccerball, and also a simulation of the bird approaching a stationary soccer-ball where the rate of expansion is identical in both cases.

26


B 10cm

40"

c

Juo-t10cm

Q. (0

"5. (A

20cm

\j\i

n

.J.J...

2

Ji

10cm

Q. (A

20cm

20cm

30cm

30cm

jU

40cm

AJM

40cm

50cm

50cm

3

(A 4 0

1

0 sec

3

2

50cm

1

0 sec

3

2

1

0 sec

Figure 6: Comparison of the responses Peri-Stimulus Time Histograms (PSTHs) for a single tau neuron to a series of stimuli (soccer-ball) of varying sizes swept along the direct collision course towards the bird. Responses are the sum of 5 sweeps and are referenced to time zero, which is the time when the stimulus would have contacted the bird. The looming object was presented against a white non-textured background (A), a stationary textured checkerboard background (object-motion) (B), and a looming background moving at the same speed behind the object (C), as shown in Figure 5. The latter condition simulated the approach of the animal toward the stationary ball and background (self-induced motion). Responses were similar to those produced by the looming object against a blank background and a stationary checkerboard background. The magnitude of responses (maximal firing rate) were similar across different object sizes. Note that the neuron did not fire to the self-motion display, even though the soccer ball's image was expanding in the same way in B and C. This implies this tau neuron is exclusively selective for "object motion in depth". The simulated path for the object was 15 m in length and the simulated object size varied from a diameter of 10 cm to 50 cm. Velocity was 500 cm/s. (© modified after Sun and Frost, 1998, Chapter).


27

We have also found that not all of the neurons in nucleus rotundus appear to be computing the tau function (Sun & Frost, 1998). Histological examination indicated that those neurons were distributed in a larger anatomical region (dorsal rotundus) as opposed to dorsal posterior rotundus in our earlier discovery by Wang and Frost (1992). In fact, half of the neurons seem to be clearly responding in this fashion, that is, they suddenly start firing at a particular and constant time before the collision event and maintain this high firing rate throughout the remainder of the approach sequence. Roughly a quarter of the neurons that show selectivity to an approaching object show a response that begins earlier for larger objects, or soccer-ball stimuli approaching at slower velocities. In detailed mathematical arguments and quantification of the timing of the response, Sun and Frost (1998) show that these neurons are computing the rate of expansion (ROE), rho of the approaching object. Finally, the remaining quarter of the looming neurons appear to be computing the very same function which best describes the locust looming detector (Hatsopoulos et al., 1995; Gabbiani et al., 1999). An example of the response patterns of each of these three classes of neuron is shown in Figure 7. Sun and Frost (1998) show that on several multidimensional plots these three classes of neurons form very distinct and tight clusters which indicate that there is not some simple underlying continuum that we have arbitrarily divided into three separate groups, but that these are genuine types of neurons each computing the following three functions.

(l)Rho

p(t) = 0'(t)

(2)

(2) Tau

t(t)^-^-

(3)

U{t) (3) Eta

7j(t) = C x G'(t) X eaev

Select Object

\ ?

Derive a, da/dt (angular direction)

Isa = 0? Or da/dt •» 0? (constant angular direction)

Is T=o? (Constant expansion)

Is dv-ds = 0? (Constant expansion)

Figure 6: The REACT model, an information processing model, which indicates the specific sources of information that an observer may use for the detection of collision events.

Optical Information for for Collision Detection during Deceleration

107 107

9. Conclusions In this chapter we have focused on the theoretical and empirical research for detecting collisions during deceleration. We have reviewed two different analyses for collision detection under these conditions -x and the constant deceleration analysis. We also reviewed the empirical research examining collision detection during deceleration. It is important to note that the use of x or the constant deceleration analysis are only relevant for detecting collisions during linear trajectories and when speed is varied. These conditions represent on a small subset of scenarios in which collisions can occur. An exhaustive analysis of all impending collisions suggest that collision events are defined by four different variables - stationary or moving objects; constant or variable speed, linear or curvilinear paths, and single or multiple collision threats. A general model, the REACT model, was proposed that reviews the information available to an observer for detecting collisions under these circumstances. An important goal for future research will be to assess the sensitivity to the different sources of information proposed in the model.

108 108

George J. Andersen and Craig W. Sauer

REFERENCES Andersen, G. J., Cisneros, J., Atchley, P. & Saidpour, A. (1999). Effects of speed and edge rate in the detection of collision events. Journal of Experimental Psychology: Human Perception and Performance, 25, 256-269. Andersen, G. J., Cisneros, J., Saidpour, A. & Atchley, P. (2000). Age-related differences in collision detection during deceleration. Psychology & Aging, 15, 241-252. Atchley, P. & Andersen, G. J. (1998). The effects of age, retinal eccentricity, and speed on the detection of optic flow components. Psychology and Aging, 13, 297-308. Bootsma, R. J. & Oudejans, R. R. (1993). Visual information about time-to-collision between two objects. Journal of Experimental Psychology: Human Perception & Performance, 19, 1041-1052. Denton, G. G. (1980). The influence of visual pattern on perceived speed. Perception, 9, 393-402. Kaiser, M. K. & Mowafy, L. (1993). Optical specification of time-to-passage:Observers' sensitivity to global tau. Journal of Experimental Psychology: Human Perception & Performance, 19, 1028-1040. Kaiser, M. K. & Phatak, A. N. (1993). Things that go bump in the light: On the optical specification of contact severity. Journal of Experimental Psychology: Human Perception and Performance, 19, 194-202. Kim, N., Turvey, M. T. & Carello, C. (1993). Optical information about the severity of upcoming contacts. Journal of Experimental Psychology: Human Perception and Performance. 19, 179-193. Larish, J. F. & Flach, J. M. (1990). Sources of information useful for perception of speed of rectilinear self-motion. Journal of Experimental Psychology: Human Perception and Performance, 16, 295-302. Lee, D. N. (1976). A theory of visual control of braking based on information about time-tocollision. Perception, 5, 437-459. Lee, D. N. (1980). The optic flow field: The foundation of vision. Philosophical transactions of the Royal Society of London Series B, 290, 169-179. Yilmaz, E. H. & Warren, W. H. (1995). Visual control of braking: A test of the T hypothesis. Journal of Experimental Psychology: Human Perception and Performance, 21, 9961014.

Time-to-Contact –- H. Hecht and and G.J.P. Savelsbergh (Editors) (Editors) © © 2004 2004 Elsevier Elsevier B.V. B.V. All All rights rights reserved reserved

CHAPTER 7 Interceptive Action: What's Time-to-Contact got to do with it?

James R. Tresilian University of Queensland, St Lucia, Australia

ABSTRACT The time remaining before an object arrives somewhere - its time-to-contact (TTC) with that place - is a quantity that could be used to control the timing of interceptive and evasive actions directed at that object. This potential use for TTC information is the primary motivation for studying how people and animals perceive it. However, studies of interceptive actions and the role TTC information might play in controlling them have been very limited until recently - studies of the perceptual process of TTC estimation have been more popular. Unfortunately, TTC perception depends upon the task being performed and it seems impossible to understand TTC perception without understanding interceptive tasks. This chapter argues this case and discusses recent experiments that have attempted to determine how performance of interceptive actions - particularly their timing - depends upon task variables including the speed of the target, the distance to be moved to intercept it, the viewing time, the size of the target and of the intercepting manipulandum. Results demonstrate that the duration and velocity of interceptive movements are systematically and consistently affected by all these variables. The relationship between movement duration and the task variables derived from experimental results is interpreted as an empirical reflection of the 'rule' used by the central nervous system to preprogram movement duration. The role of TTC in the programming and initiation of interceptive movements is explicated.

110 110

James R. Tresilian

1. Introduction Time-to-contact - genetically understood to mean the time remaining before some event occurs, whether this event involves a real physical contact or not - is a quantity assumed to be critical for achieving the temporal control necessary for either bringing about a collision that would otherwise not occur (e.g., interception of a moving target) or avoiding one that probably would. This 'time-to-contact hypothesis' seems a reasonable one but it is not the only possibility - TTC is sufficient but not always necessary for accurate temporal control of collisions (for discussion see Tresilian, 1993). In this chapter I will limit discussion to that class of collision control tasks for which the TTC hypothesis is likely to hold. The TTC hypothesis raises two important questions. First, how does a person or an animal estimate the TTC that allows adequate temporal control in a particular task? In other words, what information is used? Second, how is the TTC estimate actually employed in the control and coordination of movement? In other words, what is the nature of the control strategy used to time an action? These questions are not independent of one another in the sense that the information one type of control strategy requires to achieve a specific temporal accuracy may be different from the information required by another strategy. Conversely, limits on the availability of TTC information will place constraints on the possibility of using different types of control strategy. The question of what information is used for temporal control has been the focus of most research effort; the question of how it is used has received much less attention. The major focus of this chapter will be on issues associated with this second question; but first it is important to make some introductory remarks concerning the other.

2. Information The issue of what information about TTC is used by people and animals was first approached from the perspective of J. J. Gibson's ecological psychology (e.g., Lee, 1974, 1976). Ecological psychology seeks in part to discover laws of perceiving and acting (see e.g., Turvey et al., 1981) and one component of this enterprise is the identification of specificational laws that define sources of perceptual information. A specificational law can be defined as a function (in the mathematical sense) that maps a distal stimulus quantity (such as the size of an object, its shape, distance or its TTC with something) into a proximal stimulus quantity. Such a function could be one-to-one - a single proximal stimulus quantity associated with a particular distal stimulus quantity (Figure 1A) - or one-to-many: many proximal stimulus quantities associated

Interceptive Action: What’s What's Time-to-Contact got to do with it?

111 Ill

with a particular distal quantity (Figure IB). In either case, the proximal stimulus quantity constitutes a source of specific stimulus information about the distal quantity if no additional functions exist that map other distal stimulus quantities to the same proximal quantities (no perceptual ambiguity exists). In Figure 1C many distal quantities are associated with the same proximal quantity, which is therefore ambiguous concerning the distal state of affairs that is actually present. In the absence of ambiguity the presence of a particular quantity in the proximal stimulus means that a particular distal stimulus condition exists.

Distal stimulus quantities

Proximal stimulus quantities

A. Each distal quantity is associated with a single proximal quantity

B.

Each distal quantity is associated with several proximal quantities

C.

Several distal quantities are associated with the same proximal quantity

Figure 1: Different relationships between distal (left) and proximal (right) stimulus quantities. A) One-to-one. B) One-to-many. C) Many-to-one.

112 112

James R. Tresilian Tresilian

The basic proposal of the ecological approach to perception is that under ecological conditions a specificational relationship exist between proximal and distal quantities (Gibson, 1979). That is, proximal quantities are related to distal quantities by relationships that are either one-to-one (Figure 1A) or one-to-many (Figure IB). It seems that early formulations only recognized the existence of functions of the one-to-one type (Figure 1A) and failed to recognize those of the one-to-many type (Figure IB, Cutting, 1986). Functions of the latter type introduce problems concerning selection between different information sources and combination of these sources (see Cutting, 1986; Tresilian, 1999 for discussion). Although established previously (e.g., Hoyle, 1957), it was Lee's (1974, 1976) well known geometrical analysis showing how TTC mapped to a simple quantity in the visual stimulus that served as the primary example of the ecological approach applied to a real perceptual problem (e.g., Turvey et al., 1981; Turvey & Carello, 1987). It was shown that the TTC of an approaching object maps to the reciprocal of the rate of expansion of the object's image (e) relative to the size of this image (s). That is, the distal stimulus (TTC) maps to the proximal stimulus variable s/e. As is well known, Lee denoted quantities of the type s/e with the symbol x (tau). A simple mathematical argument establishes that the optical geometry of collision situations leads to relationships of the form TTC = T

(1)

Obviously, the existence of such a relationship means that x provides TTC information. The interpretation of such relationships from the perspective of ecological psychology led to the development of what I have called the 'tauhypothesis' (see Tresilian, 1999 for review). The essence of this hypothesis is the uniqueness and universality of x: x is the only source of TTC information used by all animal species for the timing of movement in interception and avoidance behaviors. This hypothesis denies the existence of many-to-one 'mappings' from distal to proximal stimulus variables (Figure 1C) and is thus 'ecological' in Gibson's sense. However, it either overlooks the possibility of one-to-many mappings from distal to proximal stimulus quantities (Figure IB), or if such mappings exist it asserts they are not exploited by organisms. In addition, the hypothesis denies the task-specificity of information usage - it asserts that the same information is used for temporal control irrespective of the temporal accuracy or precision demanded by the task. Related to this is its denial of any species-specificity of information usage - all species use the same information. The x-hypothesis just described is truly extraordinary - it is putatively a completely general biological 'law' of perceptually guided behavior. Such a law certainly simplifies enormously what could, in principle, be an extremely


113

complicated taxonomy detailing different tasks, different species and the information used in each case. This is perhaps the reason why the T-hypothesis has been attractive and influential. It is therefore somewhat disappointing to find that neither empirical research nor logical analysis have provided any support for the T-hypothesis (see, e.g., Tresilian, 1993; Wann, 1996). Quite the opposite in fact: research over the past decade or so has been almost unanimous in its refutation of the T-hypothesis (see Tresilian, 1999, for review). None of this is to deny that tau is involved in the perception of TTC, or to claim that there are no generally applicable laws of perceptual guidance. However, if tau plays a role in TTC perception it is not the role proposed for it by the T-hypothesis and if there are general laws of perceptuo-motor behavior, the T-hypothesis is not one of them. I suggest that it is now time to abandon research into the T-hypothesis and concentrate on the problem of how temporal control is achieved in interceptive tasks. In the past, proposals concerning control have been closely bound up with the T-hypothesis and often confounded with it. This confounding has sometimes prevented independent evaluations of the T-hypothesis - which gives an answer to the question 'what information is used for temporal control?' - and hypotheses that answer the question 'how is information used in temporal control?' This chapter aims to present a discussion of recent research directed towards understanding how interceptive actions are temporally controlled.

3. Temporal control It is generally accepted that skilled voluntary movements involve both generative processes that can function without sensory information being available during execution and sensory guidance processes that rely on information being available during execution. A central generative process is conceived as a pattern generator - traditionally referred to as a 'motor program' the outputs of which constitute command signals for the peripheral neuromuscular system. The generative process will be called a motor pattern generator, abbreviated MPG in what follows. In this context, much research on interception has been directed at two complementary questions: First, what determines the time at which a person initiates the MPG? Second, how is movement execution guided by sensory information such that interception is achieved? The first of these questions will be the major focus here. The basic requirement for successful timing an interception is that the intercepting-effector (hand) arrive at an interception location at the same time as the object to be intercepted (target), give or take some temporal margin of error which will depend upon the task but can be as small as a few milliseconds (e.g., Regan, 1992; Watts & Bahill, 1990). Thus, the condition for successful

114 114

James R. Tresilian

interception at any point in time may be stated as follows: The time remaining until the hand reaches the interception location (its time-to-arrival or time-tocontact, TTC) must be equal to the time to arrival of the target at the interception location (to within the error tolerance of the task). If the action is not controlled on-line, then this condition must be met by initiating the act at just the right moment to ensure that its duration matches the target's TTC with the interception location. Many of the arguments that follow are developed under the assumption that the temporal evolution of the action is not controlled on-line. Although this assumption is unrealistic for acts lasting longer than 200 to 300 milliseconds, it is suggested that many of the conclusions reached are largely independent of whether or not movement timing is controlled on-line. An early proposal for the control of program initiation in interceptive actions was the operational timing (OT) hypothesis put forward by Tyldesley and Whiting (1975). The OT-hypothesis proposes that the temporal accuracy required for successful execution of interceptive actions is achieved by executing movement patterns of pre-determined (pre-programmed) duration following an earlier suggestion put forward by Poulton (1950). Thus, for every interceptive task that a person can perform (catching, playing a tennis stroke, hitting a baseball and so forth), there is an MPG that is responsible for producing the basic movement pattern that characterizes performance. The MPG is capable of determining the duration of any particular performance of the task. Temporal control is a matter of starting the pattern generating process at the right moment, the temporal structure of the movement having been determined in advance. Starting the process requires an initiating signal or 'GO' signal that activates the MPG at the appropriate moment. The GO signal is derived from some perceived quantity: when the value of this quantity exceeds some criterion value the GO signal is sent to the MPG. The central idea of the OT hypothesis is that the duration of the movement pattern (between its initiation and the time of target interception) is pre-programmed. Two versions of this type of OT hypothesis have been considered. In one version the criterion value of the perceptual variable used for activating the MPG associated with a particular interceptive task is assumed fixed. That is, the criterion is not something that can be varied by the nervous system so as to adapt performance to different task conditions. This version of the hypothesis seems to be the one initially favored by D. N. Lee (see, e.g., Lee, 1980) and later by others (e.g., Micheals et al, 2001; Smith et al., 2001; Van Donkelaar et al., 1992). In the other version the criterion value can be varied by the nervous system so as to adapt performance to different task conditions.

Interceptive Interceptive Action: What’s What's Time-to-Contact Time-to-Contact got got to to do do with with it? it?

115 115

3.1 The fixed criterion OT-hypothesis The basic functional structure of the fixed-criterion version is shown in Figure 2: the perceptual variable used for determining when to activate the MPG is derived from the stimulus input (perceptual processing). The value of this variable is then compared to the criterion, if it exceeds the criterion the GO signal is passed to the MPG. This activates the MPG which then outputs signals (motor commands) that descend to the peripheral neuromuscular system. Stimulus input

I

Perceptual processing

Initiating perceptual variable, P

P > criterion? If yes, issue GO signal

GO signal

Motor pattern generator

1

Central motor commands

Figure 2: Functional structure of the fixed-criterion version of the operational timing hypothesis described in the text.

Lee (1980) proposed that the GO signal is derived from information about the TTC of a moving target and an interception location (specifically from the variable x). Thus, the MPG is activated when the perceived value of TTC is equal to some criterion value. The basic idea would work as follows (refer to Figure 3). The TTC information in the stimulus reaches a criterion value (Tc) at time tl. It takes some time for the nervous system to detect that the criterion value has been reached and to activate the MPG - this is called the processing time in Figure 3. After the processing time has elapsed (at t2) the MPG begins to send signals to the neuromuscular system. It takes some amount of time for the effects of these signals to cause force generation in the muscles - called the transmission time in Figure 3. Time t3 marks the start of the movement pattern and the total time between tl and t3 is the time the system takes to react to the fact that the stimulus TTC information has reached the criterion - called the reaction time in the figure (RT = processing time + transmission time). The movement is completed (the intercepting effector reaches the interception location) at time t4 and so the time period (t4 -13) is the movement time (MT).

116 116


Movement starts

Stimulus TTC information = Tc Reaction time (RT)

MT ,,

Time -•

Transmission time

Processing time tl

Movement completed

t2

t3

t4

Figure 3: Sequence of events associated with the fixed criterion version of the operational timing hypothesis (see text for details).

In order for the interception to be successful the intercepting effector must reach the interception location at the same time as the target. This will happen if the criterion value of the TTC (Tc) is equal to RT + MT in Figure 3 (assuming that the perceived TTC is an accurate estimate of the true TTC). Since RT and Tc are both invariant, this strategy will only work if MT is also invariant: if MT is free to vary then RT + MT cannot always be equal to Tc. Thus, the fixed criterion OT hypothesis would seem to predict invariant MTs for specific interceptive tasks. Note, however, that this ignores the possibility that movement timing might be corrected during execution if errors are detected. The prediction of invariant MTs is, therefore, restricted to very rapid interceptive actions where there is insufficient time to modify the movement during execution (MTs less than about 150 to 200 milliseconds). A second type of prediction also follows from the idea that the criterion value is fixed. If this is true, then variations of the initiating variable at the time the movement starts should vary in a predictable way with variations in task conditions. Let us consider the example of variations in target speed. If TTC is used as the initiating variable, then the prediction is straightforward. Refer back to Figure 3: the criterion value of TTC is fixed at some value (Tc) and its value at the time of movement initiation is Tc + RT. Thus, the value of TTC at the moment the movement starts should be approximately constant since both Tc and RT are constant. If the initiating variable is not TTC but distance (of the target from the interception location) as some have suggested (e.g., Wann, 1996; see also Van Donkelaar et al, 1992, for a related idea) then the prediction is not


117 Ill

quite as simple. Refer to Figure 3 but assume the initiating variable is distance (D) and the criterion value is Dc. During the RT period the distance of the target from the interception location will have changed by an amount equal to the target's speed (V) multiplied by the RT. The distance at the time the movement starts is equal to Dc - V x RT. Thus, the distance of the target from the interception location at the time of movement initiation depends upon the speed of the target. As another example, if the initiating variable is the rate of expansion of the target's image then it can be shown that the expansion rate at the time of movement initiation should vary as the square root of target speed (assuming constant size targets; see Mattocks, Wallis & Tresilian, 2002). 3.2 Empirical investigation of the fixed criterion hypothesis A type of change in task conditions that has been found to lead to systematic changes in interception performance is variation of the speed of the moving target. It has been reported many times that people move more rapidly (shorter MTs and greater movement speeds) when intercepting fast moving targets than they do when intercepting slower moving targets (e.g., Bairstow, 1987; Brenner & Smeets, 1996; Carnahan & MacFadyen, 1996; van Donkelaar et al, 1992; Wallace, Stevenson, Weeks & Kelso, 1992). There are two possible reasons for this. It could be a result of there being less time available to complete the interceptive movement when the target moves faster: the person is constrained to move more rapidly when there is less time available. Alternatively, it could be that faster moving targets elicit more rapid response regardless of the available time. Mason and Carnahan (1999) recently observed that most of the studies that had reported an effect of target speed on the speed of interceptive movement had confounded target speed with the time for which the target was visible prior to interception (the viewing time), and hence the time available to make the interception. The confounding was such that the faster the target moved the shorter the viewing time. Mason and Carnahan (1999) reported the results of an experiment that unconfounded target speed and viewing time and found no effects of target speed on the interceptive movement. It was concluded that viewing time and not target speed was the determinant of the rapidity of the interceptive response. As we will see later (section 4), this conclusion is premature - both target speed and viewing time affect the speed of the response. Although it is difficult to explain more rapid responses when less time is available using the fixed criterion idea, the decrease in MT associated with faster moving targets could be explained by the hypothesis that a fixed criterion value of the target's image expansion rate is used. This hypothesis has been used to interpret recent experimental results (Micheals et al., 2001; Smith et al., 2001).

118 118


Image plane

Figure 4: Simple two-dimensional optical geometry of direct approach to an imaging system (eye) with focal length /. An object of size S (constant) approaches a stationary imaging system at constant speed V as shown. The object is instantaneously a distance D from the eye and the image of the object is at the same moment of a size s and is growing in size at a rate e. The equation given follows from the fact that the two triangles in the diagram are similar. Differentiation of this equation with respect to time establishes that e is proportional to SV/D2.

To see how a fixed criterion expansion rate can explain the variation in MT with target speed, consider the simple geometry shown in Figure 4. In these conditions it is easy to show that the expansion rate is proportional to SV/D2. If the criterion value of the expansion rate is constant, SV/D2 must take the same value irrespective of speed and so for a given target, higher target speeds (V) must be associated with greater distances D at the moment interceptive movements are initiated. However, D at initiation need only increase in proportion to the square root of V to maintain the same expansion rate value. This implies that the TTC at initiation (= distance at initiation/V) is inversely proportional to the square root of V and therefore smaller when V is larger. If the interception is to be successful, the duration of the interceptive movement needs to be shorter when the target moves faster. More precisely, the MT should vary in inverse proportion to the target speed. As will be described later, this is exactly what is found (Tresilian & Lonergan, 2001; Tresilian et al., 2001). However, to explain this finding using the fixed criterion hypothesis requires an additional hypothesis concerning the mechanism for modulating MT. It would appear that TTC information is necessary for correctly setting the MT even if expansion rate is the initiating variable.

Interceptive lnterceptive Action: What’s What's Time-to-Contact got to do with it?

119 119

Finally, Lee (1980) cited a study by Schmidt (1969) and another by Schmidt and McGowan (1972) that reported results on MT consistency as support for the fixed criterion hypothesis. Schmidt and McGowan's study demonstrated that MT variability in an interceptive task became increasingly smaller with practice. MT seemed to be getting progressively closer to a constant value as participants practiced. Schmidt's study (Schmidt, 1969) showed that MT variability in a simple coincidence anticipation task was very small - between about 4 and 13 milliseconds. Unfortunately, Lee (1980) failed to mention a major feature of Schmidt's data. The participants in Schmidt's experiment were required to slide a pointer through different distances from a start position to a target position. They had to move so that the arrival of the pointer at the target location was temporally coincident with the arrival of a moving object at a specified point on its path. Schmidt studied movements through four different distances (15, 30, 45, 60 cm) and found that MT increased reliably with the distance to be moved. Although this result may seem unsurprising it is nevertheless quite inconsistent with the fixed criterion hypothesis. Overall, there is reason to suggest that the fixed criterion hypothesis is inadequate to provide a convincing account of the flexibility of human performance in interceptive tasks. In the next section, a simple and rather obvious modification is described that may be all that is required to provide an adequate account. 3.3 The variable criterion OT-hypothesis The basic idea of the variable criterion version is that the nervous system can vary the duration of a movement pattern in order to adapt it to different conditions of execution. It is assumed that the duration (MT) of a particular interceptive act is pre-programmed based on factors related to the circumstances of performance. Based on the discussion presented in the last section, these factors include the distance to be moved to make the interception, the time available to make the interception and the speed of the target. The preprogrammed MT based on some set of factors (cp) will be denoted MT((p). Once the MT has been pre-programmed, the MPG needs to be activated so that the intercepting effector arrives at the interception location coincident with the arrival of the target. This requires, of course, that the MPG be activated at the right moment such that the time remaining before the target reaches the interception location (TTC) is equal to MT((p) + RT. Thus, the criterion value of the TTC information (Tc) used to activate the program needs to be set based on MT(cp). If a veridical TTC estimate is used then Tc = MT(cp) + RT.

120 120

James R. Tresilian

Internal factors Situational information Compute MT MT(cp) Stimulus. input

Perceptual processing

TTC information

TTC. During this period the target has moved through a distance equal to L + W and so the time window has a duration of (L + W)/V. Referring back to Figure 6, it should now be easier to see why moving with the target (parallel to it) decreases the temporal precision demand of interception: the speed of the target relative to the hand is equal to V - VH (where VH is the speed of the hand parallel to the target's motion) and so the time window is now (L+W)/(V-VH).


V

123 123

Path of target

target D Target becomes visible here Intercepting effector

Figure 7: Schematic diagram of the hitting task used in the experiments reviewed in the text. The target (length L) moves along a linear track with instantaneous speed V, the target's leading edge is shown thickened. The bat (width W) moves along a linear track perpendicular to that of the target. The bat moves through the region shaded light gray starting a distance D from the target's path. The target can be hit when any part of it is within this region. The target becomes visible when the leading edge is a distance Z from the bat.

The time window can be considered to define the temporal precision required to perform the task: if the centre of the target is in the middle of the strike zone at time tm then the target will be intercepted provided that the time the intercepting effector reaches the strike zone is within the range tm - co/2 < tm < tm+ co/2. One useful way to visualize this constraint is to use a space-time diagram as illustrated in Figure 8. In the diagrams in the figure, the abscissa represents spatial position along the direction of target motion and the ordinate represents time. In both diagrams the position of the leading edge of the target over time is indicated by a thick diagonal line and the position of the trailing edge by a thin diagonal line. The region of space occupied by the target at any instant is just the horizontal region between the points on the two diagonal lines corresponding to that instant. This region is shaded light gray for every instant and so the trajectory of the target is represented as a shaded parallelogram with width L and a slope equal to 1/V. A slower moving target (Figure 8, left panel) is associated with a more steeply oriented parallelogram than a faster moving target (Figure 8, right panel).

124 124

James R. Tresilian

Target's trailing edge

Target's leading edge

Figure 8: Space-time diagram for the task illustrated in Figure 7 for the case of constant target speed. The speed in the left panel is slower than that in the right panel. The target becomes visible at time to and sweeps out the parallelograms shown shaded. The spatial position of the bat along the direction of target motion does not change over time and so the vertical dashed lines define the region occupied by the bat. The bat and target can contact one another in the region shaded dark gray. The height of this region is the time window (co).

The strike zone defined by an effector of width W is shown bounded by vertical dashed lines and the region of the target's trajectory within the strike zone is shown as the darker shaded regions of the oriented parallelograms. The time window is just the height of this darker shaded region (co) and is smaller for the faster moving target (Figure 8, right panel) given that L and W are the same in both panels. If the abscissas are located at time to at which the target becomes visible, then the viewing times are as indicated (VT). In the examples shown in the Figure the viewing times are the same since the target becomes visible at a greater distance from the strike zone when the target moves faster (Z in Figure 8, right). The above shows that manipulations of L, W or V (independent of VT) act to vary the temporal precision demanded by the task (Figure 7) as defined by the time window. It can then be asked whether a person performing the task treats such manipulations as equivalent. For example, if the time window is

Interceptive Action: What’s What's Time-to-Contact Time-to-Contact got to to do with it? it?

125 125

decreased by a certain amount, does it matter whether the decrease is due to a change of bat size (W), of target length (L) or of target speed (V)? In sections 4.2 & 4.4, the results of some recent experiments that examined how performance variables were influenced by changes in W, L, V and D independent of VT are reviewed. 4.2 The effects of manipulating task constraints on interceptive aiming In two series of three experiments we have studied performance of the constrained aiming task illustrated in Figure 6. In the first series (Tresilian & Lonergan, 2002) the target accelerated from rest on a slightly inclined slope at approximately 1.2m/s2. The task was to move a hand held bat along a linear slide (tilted at the same angle as the slope down which the target moved) through one of three different distances (7.5, 20 & 38.5 cm) to intercept the target. The time for which the target was visible prior to reaching the strike zone was held approximately constant. The three experiments examined the effects of varying the time window in each of the three possible ways. In experiment 1 the bat width was varied and the target length, speed and acceleration were constant. In experiment 2 the target length was varied with the bat width, target speed and acceleration held constant. In experiment 3, the target speed upon entering the strike zone was varied, the target length and bat width were held constant. In each experiment the time windows were always approximately 35, 50 and 65 milliseconds. There were small errors in achieving these time windows but both the constant and variable errors were always less than 3 milliseconds. In each experiment participants were required to hit the target on at least 80% of trials in any given condition (15 trials in each of the nine conditions) and only trials on which the target was struck were analyzed. In the actual experiments the majority of participants (6 in each experiment) struck the target on over 90% of trials. Participants were instructed not to stop or make any reversals of direction when intercepting the target and all complied with this instruction. The main dependent variables in the experiments were the duration of the striking movements from initiation to the time the target was hit (MT) and the maximum speed reached during the movement (Vmax). The results in each experiment were fairly clear as all participants performed in essentially the same way. The results are summarized schematically in Figure 9. The results of experiments 1 and 2 were almost identical and displayed the pattern shown in the top panels of Figure 9 in which the effect of time window on MT was very small and not statistically reliable. Experiment 3 was showed a similar, but more exaggerated pattern where the effect of time window on MT was statistically reliable.

126 126

James R. Tresilian

/

CD

E

—

co = 35 ms co = 50 ms co = 65 ms

/

'

V

QJ >
speed • > speed O > speed • . a) Pattern expected when MT is plotted against target size if MT ~ [L + W]/V - V. b) pattern of the MT results in (a) plotted against time window, c) Pattern of MT against target size plot expected if MT 0 J

Aa

where Aa is an arbitrary small retinal area, dl is an element of the closed path enclosing area Aa and Vsini9 is the component of velocity normal to dZ (Green, 1967; Schey, 1973). This definition is framed in terms of the limiting case with area Aa tending to zero, but if we approximate for our case of an isotropically-expanding square retinal image of finite but small side length 26, then it can be straightforwardly shown that

to a first approximation, Hence TTC=2/divV

(7)

(Regan & Hamstra, 1993) In principle, therefore, for a moving observer the monocular changing-size mechanism could signal time of arrival at an external-world destination by extracting local divV from the retinal image flow pattern created by self-motion through the three-dimensional environment. Whether this occurs in practice is not known. Nevertheless it may be worth noting that by definition the variation of divV within the retinal image is independent of translational velocity produced by eye rotation, and there is psychophysical evidence that this independence is also shown (at least to a first approximation) by physiological relative motion detectors (see Figure 6).

198 198


A TEST SQUARE

B ADAPTING FLOW PATTERN

50 N

„

OS o03 _l LU Z> CC 0 3

Former location of locus 1

2

3

DISTANCE, X (degrees)

Figure 11: Looming detectors can be activated by the focus of expansion of a flow pattern. (A) Sensitivity to size oscillations of a 0.5 deg test square located a variable distance (X) from the point of fixation (M) were measured before and after adaptation. (B) Observers adapted to a radially expanding flow pattern for 10 min. This particular flow pattern had a sharp maximum of divV at the focus of expansion. (C) Depression of changing-size sensitivity as a function of distance X. Threshold elevations only occurred for points very close to the former location of the flow pattern's focus. From "Visually guided locomotion: psychophysical evidence for a neural mechanism sensitive to flow patterns", by D. Regan and K. I. Beverley, 1979. Science, 205, p.312. Copyright 1979 by the American Association for the Advancement of Science. Reprinted with permission.

A Step by Step Approach Time-to-Passage Approach to Research on Time-to-Contact and Time-to-Passage

199

2.9 A developmental hypothesis A predator's most efficient and least bothersome tactic is to kill before the prey is aware of the danger, and a silent approach form the rear is a simple way of achieving this aim when the prey has front-facing eyes. So it is not surprising that an expanding image, especially when located in the peripheral visual field, triggers a fast involuntary motor reflex in humans. The reflex is most evident when attention is distracted and the stimulus is unexpected, and includes a rapid eye movement to foveate the expanding image so as to allow estimation of TTC and identification of the approaching object. It has been proposed that selective visual sensitivity to the ratio 6/(d 6/dt), that is x, is not present from birth as is sensitivity to a looming stimulus (i.e. a rate of expansion (dO/dt). Rather, exposure to the radially-expanding pattern of optic flow associated with self-locomotion in early life progressively develops a neural mechanism whose sensitivity to 9/(d6/dt) is combined with insensitivity to both 0 and d6/dt (Regan & Beverley, 1979a; Regan & Vincent, 1995). This proposal is consistent with evidence that, in adults, the independence of the mechanisms for 6/(dd/dt) and for dd/dt is progressively lost as retinal eccentricity is increased (Regan & Vincent, 1995), because the stimulus necessary for the development of selective sensitivity to 8/(d6/dt) is much weaker in the peripheral visual field. In particular, if the direction of gaze roughly coincides with the direction of self-motion through a three-dimensional world cluttered with objects, the predominant effects of self-motion are that retinal images of nearby object in the central visual field expand with comparatively little translational motion and a value of 6/(d6/dt) that corresponds to TTC, whereas retinal images of nearby objects in far peripheral vision predominantly translate.

3. Neural mechanisms involved in estimating TTC on the basis of binocular information and the "Independent Processing" requirement 3.1 The binocular motion-in-depth mechanism and the importance of reference marks Wheatstone (1852) was the first to demonstrate that a rate of change of horizontal binocular disparity can, by itself, generate a compelling sensation of motion in depth such that the observer has the impression of collision at some future instant. In this section we review evidence that the human visual system contains a specialized mechanism that generates a sensation of motion in depth

200


when stimulated by changing-disparity, a mechanism that is almost totally insensitive to all the other variables so far tested. This last point complies with the "independent processing" requirement for a set of filters (Regan, 1982). Regan, Erkelens and Collewijn (1986a) reported that changing the target's disparity in the situation that no reference mark is visible produces only a weak sensation that the target is moving in depth or even no sensation of motion in depth at all. In other words, the effective binocular stimulus for motion-in-depth perception is a rate of change of relative rather than absolute disparity. A rate of change of absolute disparity more than 100 times larger than the rate of change of relative disparity that produced a just-noticeable sensation of motion in depth produced no motion-in-depth sensation at all for a large target and only a weak sensation for a point target. (For a discussion of absolute and relative disparity see Regan, 2000, pp.348-351). In Figure 12 the stationary reference was a plane covered with random dots. The observer fixated on this plane, and the accuracy of fixation was monitored by nonious lines. The target was a pair of identical bars, one viewed by the left eye and one by the right eye. The bars were seen in binocular fusion. The mean depth of the fused bar could be varied relative to the reference marks. Figure 12 shows that sensitivity to motion in depth (STEREO, continuous line) is greatest when the stationary marks are at the same depth as the changingdisparity target, and that sensitivity falls off steeply as the depth of the bar departs from that of the reference. Many observers have areas of the binocular visual field within which changing-disparity produces no sensation of motion in depth. These are called stereoscotoma (Richards and Regan, 1973). This total loss of motion-in-depth sensation is not accompanied by any loss of sensitivity to motion within a frontoparallel plane, nor any loss in sensitivity to a difference in static disparity (i.e., stereoacuity) (Regan, Erkelens & Collewijn, 1986b; Hong & Regan, 1989). These findings indicate that the binocular mechanism that supports the perception of motion in depth is separate from both the mechanism sensitive to frontal plane motion and the mechanism sensitive to static relative disparity. The finding that some observers can be blind to approaching motion in depth but sensitive to receding motion in depth, or vice versa, is evidence that the binocular motion-in-depth mechanism consists of separate "approaching" and "receding" submechanisms. This point is important for the model shown as Figure 3.

Approach to Research on Time-to-Contact and Time-to-Passage A Step by Step Approach

201

Stereo.

f

5

-20

-10

10

20

Static disparity, min arc

Figure 12: Sensitivity to changing disparity is greatest when the target and the stationary reference are at the same depth. A bar viewed by the left eye and an identical bar viewed by the right eye each executed 0.1 Hz sinusoidal oscillations either in antiphase (Stereo), giving a percept of motion in depth, or inphase (Binoc), giving a percept of motion within a frontoparallel plane. Ordinates plot the amplitudes of oscillation (of either bar) when the observer just detected motion. The observer fixated on a plane of random dots at zero disparity. Abscissae plot the mean disparity of the binocularly-fused bars. The horizontal dashed line shows the monocular threshold (one eye occluded). Adapted from "Some dynamic features of depth perception" by D. Regan and K.I. Beverley, 1973, Vision Research, 13, p.2371. Copyright 1973 by Elsevier Science Ltd. Reprinted with permission.

Observers can discriminate trial-to-trial variations in the speed of the motion in depth sensation evoked by changing-disparity while ignoring simultaneous changes in the direction of motion in depth and the distance moved (i.e., change of disparity A5) for both monocularly visible and cyclopean targets (Portfors-Yeomans & Regan, 1996, 1997). In Figure 13A-I the observer was instructed to signal after each single trial the target's speed, direction, and distance moved. Figure 13A-I shows that, in each of the three tasks, the observer's responses were based on the task-relevant variable, and the two taskirrelevant variables were ignored.

202 202

and Rob Gray Gray David Regan and

r B

100 80

| g 3 8

60

,«" * »

40

••-,-

20 0 100 80 CD ( 8

60 40 20 0

100 r Q

80

2 8.

60 40 20 0.5

1

Direction

1.5

0.8

1

1.2

Speed

1.4

0.8

1

1.2

Excursion

Figure 13: The direction, speed, and distance moved in depth by an approaching object are processed independently. The target was a bright square. The 216 test stimuli appeared to move in depth at different speeds and in different directions. The reference stimulus had the mean speed, mean direction and mean direction moved for the set of test stimuli (1.0 on the abscissae). Following each presentation of the reference and of one of the test stimuli the observer made three judgements. A-C: The percentage of "wider of the head than the reference trajectory" responses is plotted against the task-relevent variable (d0.5sec (Gray & Regan, 2000a, Appendix)

6. Estimation of TTC following adaptation It is not uncommon that an observer must estimate the TTC of an approaching object following exposure to visual stimuli that desensitizes local detectors of expansion. Given that such adaptation produces a weakened sensation of motion in depth (Regan & Beverley, 1978), Gray & Regan (1999a) asked whether adaptation to expansion has any effect on estimates of TTC based on either monocular or binocular TTC information. Following lOmin of adapting to a ramped increase in target size, estimates of absolute TTC were measured using the staircase procedure. Following adaptation to expansion, TTC estimates based on t alone were 15-27% longer as compared with a baseline condition where observer's adapted to a constant-sized target. It may not be intuitively obvious, given that object expansion is not involved in equation (1), why substantial overestimations of TTC (8-16%) occurred when estimates were based on binocular information alone (Figure 7). H Adapt Static Q Adapt Expansion

§

1

OBSERVER

Figure 7: Estimation errors for judgments of absolute TTC following lOmin adaptation to an expanding target. Solid bars show errors following adaptation to, a target with a constant disparity. Open bars are for a baseline condition in which the adaptation target remained at constant size. Estimates of TTC we based entirely on binocular information. Reproduced with permission from R. Gray and D. Regan (1999). Adapting to expansion increased perceived time to collision. Vision Research, 39, 3602-3607. Copyright 1999. Elsevier Science Ltd.

The Use Use of Binocular Time-to-Contact Information

317

This cross-adaptation effect can be understood in terms of the model of motionin-depth processing proposed by Regan & Beverley (1979a) - see Fig 3 - in which the signal generated by changing retinal image size is summed with the changing disparity signal before generation of the motion-in-depth signal (on the basis of which TTC is estimated). So, unlike the rotation and small object conditions described above, binocular information about TTC cannot be used to compensate for the inadequacy of X following adaptation to expansion. It is known that a radial flow of texture temporarily desensitizes local changing-size detectors located within approximately 0.5 deg of the focus of expansion (Regan & Beverley, 1979b; Beverley & Regan, 1982). Reasoning that a driver who gazes at a textured road while driving along a straight road produces a radial flow of texture on the retina we used an automobile simulator to find whether adaptation to the flow pattern affected judgements of TTC (Gray & Regan, 2000b). Following simulated highway driving on a straight empty road for 5 min, drivers initiated overtaking of a lead vehicle substantially later (220-510ms) than comparable maneuvers made following viewing a static scene. The implication of this lengthening of perceived TTC and consequent change in driving behaviour is that a driver who gazes straight ahead while driving in light traffic along a straight road might be at risk of rear-end collision when overtaking. As explained earlier, these errors of judgement would be present for judgements based both on binocular information and monocular information. Monitoring the scene ahead by shifting the gaze over the scene ahead would reduce errors (Gray & Regan, 2000b).

7. Processing of combinations of binocular and monocular information about TTC Regan and Beverley (1979a) pointed out that models of the processing of binocular and monocular information about TTC must take three factors into account. First is the ratio between the rate of expansion {d&dt) and rate of change of binocular horizontal disparity (d&dt) in the approaching object's retinal images. This ratio follows straightforwardly from geometrical optics - see equation (3). Second is the weighting given to these two variables by the visual pathway consequent on the difference between the dynamic operating characteristics of the mechanisms sensitive to d&dt and to changing-disparity respectively. Third is intersubject variability. (This is very large. Within the five observers studied by Regan and Beverley (1979a) the relative effectiveness of

318


d&dt and d&dt as stimuli for motion in depth varied by 80:1).6 We next discuss these factors. The crucial point brought out in equation (3) is that the ratio between d&dt and dS/dt does not depend on the object's distance nor on its speed. Equation (2) indicates that the magnitude of d&dt relative to the magnitude of d&dt is directly proportional to the approaching object's linear size. Consequently, for small objects the monocular correlate of TTC may be unimportant compared with the binocular correlate. A second implication of equation (2) is as follows. (For clarity we will defer a discussion of the different dynamic characteristics of the two systems). The distance at which binocular information starts to contribute to motion-indepth perception does not depend on the object's linear size, while the distance at which d&dt starts to contribute to motion-in-depth perception does depend on the object's linear size (S) scaled in units of I.7 In other words, for objects of different sizes approaching an observer's head in the z-direction at any given speed Vz, the distance at which binocular information starts to contribute to motion-in-depth perception is the same, while the distance at which d&dt starts to contribute to motion-in-depth perception depends on the ratio S/I. This last point can be illustrated by a numerical example. There are intersubject differences in the two thresholds, but for simplicity we will assume that they are both the same, and equal to 5 arc min/sec (see Fig.3, Beverley & Regan, 1979a). Consider an object 2m wide (about a car's width) so that, if I=6cm, S/I=33. Suppose that an observer is approaching this object with a closing speed of 5 m/sec (about 1 lmph). We have dS

IV7

— =—f l

(4)

dt D (Regan & Beverley, 1979a). Substituting into equation (4) we find that rate of change of relative disparity starts to generate a detectable percept of motion in depth when the object is at a distance of ca. 14m. We have dO SV7 (5)

6

Rushton and Wann (1999) proposed a model of relative weighting that takes no account of either intersubject variability or the known (large) differences in the dynamic characteristics of the mechanisms sensitive to monocular and binocular information about TTC. 7 Detection threshold for changing-disparity is not in general the same as the value for which motion in depth is just detected, and it is the second threshold that is relevant for estimating TTC. This distinction is demonstrated in Fig.3 of Regan and Beverley (1979a) and Fig.2 of Beverley and Regan (1979a).

The The Use Use of of Binocular Time-to-Contact Time-to-Contact Information

319 319

Regan & Beverley, 1979a). Substituting into equation (5) we find that the rate of expansion starts to generate a percept of motion in depth when the object is at a distance of ca. 2m. Looking back at equations (4) and (5) it can be seen why8 the ratio between these two critical distances is equal to y [S /1). Having discussed the differences between the distances at which binocular information and d&dt start to contribute to the estimation of TTC, we now consider the later stage when both are well above threshold. The crucial point now is that the dynamic characteristics of the mechanisms that process monocular and binocular information about TTC are quite different. The effect of this difference on the relative weighting of monocular and binocular information was measured experimentally by Regan and Beverley (1979a) and can be summarized as follows. Binocular information becomes relatively more effective in generating a sensation of motion in depth as approach speed is increased, and relatively less effective as viewing time decreases. In other words, the monocular cue is weighted more heavily at low speeds and brief viewing, while the binocular cue is weighted more heavily at high speeds and longer viewing durations. Both are large effects. A 64-fold increase of speed can increase the relative effectiveness of the binocular cue 16-fold (Beverley & Regan, 1979a, Fig.5). A 16-fold increase in viewing time can increase the relative effectiveness of the binocular cue fourfold. In addition, as already mentioned, the linear size of the approaching object determines the relative magnitude of the two cues. On the other hand, providing that both d&dt and dS/dt were well above threshold, there is evidence that the viewing distance has no effect or only a small effect on the relative effectiveness of the binocular and monocular cues to TTC: a very large change of binocular convergence (0 to 24 prism dioptres) had no effect on relative effectiveness in one observer and produced only a twofold difference in a second observer (Regan & Beverley, 1979a, Fig.6). In everyday life, there are often a very large difference between the effectiveness of monocular and binocular cues to TTC. Regan and Beverley (1979a) gave three examples, basing numerical calculations on the information present in the retinal images weighted by the measured dynamic characteristics of the relevant mechanisms in an individual observer. For an aircraft approaching a runway 100ft wide and 2000ft away at 140mph, the monocular cue to TTC with the runway would be 76 times more effective than the binocular cue for an inspection duration of 1.0 sec. For a cricket ball 50ft away approaching a batsman's head at 90mph the binocular cue would be 2.1 times more effective than the monocular cue for an inspection duration of 0.25 sec. For a fly 50cm away approaching the observer's head at 0.05 m/sec, the 8

This point is valid, though the situation is somewhat more complex, because relative sensitivity to changing-disparity and d&dt depends on Vz (Regan & Beverley, 1979a)

320


binocular cue would be 72 times more effective than the monocular cue for an inspection duration of 1.0 sec. Note, however, that these calculations are for one of the five observers studied, and intersubject variability was 80:1 within these five normally-sighted observers. Within the general population intersubject variability is presumably greater than 80:1.

8. Anecdotal and circumstantial evidence on the importance of binocular TTC information Early hints that binocular disparity could be important for judging TTC included the report of Bannister & Blackburn (1931). They designated 258 Cambridge undergraduates as either "poor" or "good" at ball games (including cricket). The group that was ranked as "good" had a larger mean interpupillary distance than the "poor" group. (A larger interpupillary separation in a larger disparity for a given depth separation between two objects). Using high-speed photography Alderson et al. (1974) found that when catching a ball with one hand the temporal order of finger flexions was disrupted when the lights were switched off 275 ms before the ball arrived, that is when the ball had reached 1.8m from the hand. Another kind of circumstantial evidence is the performance of professional baseball batters who have lost the use of one eye at some point during their playing career. Tony Conigliaro, of the Boston Red Sox was one of the most promising young ballplayers of the late sixties. In 1964, he hit 24 home runs and batted .290 as a 19 year-old rookie. He reached 100 career home runs at the age of 22 (the youngest AL player to reach that milestone at the time). But Conigliaro's promising career took a downturn after he developed a blind spot in one eye when he was hit by a pitch in 1967. Despite hitting 36 homeruns and being named "Comeback Player of the Year" in 1970, Conigliaro could not fully recover. In his last two seasons in the majors (1971 & 1975) he batted only .173 in 95 at-bats. Hofeldt, Hoefle, & Bonafede (1996) have provided direct evidence that visual processing of a rate of change of binocular information is important in hitting a baseball. Wearing a neutral density filter over one eye alters the perceived trajectory of a pendulum bob (Pulfrich, 1922). It is generally supposed that the effect is caused by delaying signals from one eye, thus distorting dynamic binocular information (perceptual delay increases as luminance decreases). Hofeldt and colleagues reasoned that wearing a filter over one eye might affect batting performance. Indeed it did. Using pitch speeds of 75-85 mph, they found a 55% drop in the number of contacts per swing when the hitter wore a filter over one eye. There was no decline in performance when lenses were worn over both eyes suggesting that the effect is not produced by a reduction in contrast. However it was not clear whether the degraded hitting performance was caused by timing error rather than by distortion in binocular


321

information about the trajectory of the ball (Beverley & Regan, 1973; PortforsYeomans & Regan, 1997). Field experiments have also provided hints that binocular information may be important in estimating TTC. Cavallo & Laurent (1988) had observers estimate the TTC with a stationary object while they sat in a car that was driven towards the object, and found that estimates were more accurate when observers viewed the object binocularly than when one eye was occluded. They reported that this improvement only occurred for near targets (nearer than roughly 75m), and on this basis concluded that the effect was is not simply due to having an extra source of ^information from the other eye. In a catching task, Savelsbergh et al. (1991)found that the precision of grasping, as indexed by the standard error over many trials, was better for binocular than for monocular viewing; and McLeod, McLaughlin, & Nimmo-Smith (1985) reported that an observer's accuracy in hitting a dropped squash ball was better when viewing was binocular instead of monocular. On the other hand, occluding one of the pilots' eyes during the approach to landing a jet or a propeller-driven aircraft had either no detrimental effect on landing performance, or performance even improved (Pfaffman, 1948; Lewis & Kriers, 1969; Lewis et al., 1973; Grosslight et al., 1978).

9. Summary and conclusions In this chapter we have reviewed several lines of evidence that support the hypothesis that binocular information can be important for the judgments of TTC. Previous research has provided clear evidence that the human visual system contains a neural mechanism sensitive to binocular information about TTC, and that observers can use this information on its own to make accurate estimates of absolute TTC with an approaching object and to discriminate small variations in relative TTC. Furthermore, it has been demonstrated that binocular TTC information can compensate for the ineffectiveness of Tin some real-world situations such as when the approaching object is small or when the approaching object is nonspherical and rotating (e.g., a tumbling American football). Finally, it has been demonstrated that binocular and monocular information about TTC are combined by the visual system to produce a significantly more accurate estimate of absolute TTC than is the case for estimates based on either cue alone. Collectively, these findings suggest that binocular information plays a larger role in collision avoidance and collision achievement than has been previously believed (e.g. Profitt & Kaiser, 1995). In this chapter we have also highlighted several methodological issues related to the investigation of TTC judgements. We describe several examples of situations in which an observer seemed to be able to make judgments of TTC,

322


but upon further analysis it was revealed that the judgment was actually based on some perceptual variable other than a visual correlate of TTC (e.g. total change in object size). Such findings raise questions about the validity of the results of TTC experiments that have made no provision for checking that observers ignored task-irrelevant optical variables. Past research in the area of time to contact has been heavily biased towards exploring the role of t in judging TTC (Tresilian, 1999), and consequently there are many important questions regarding binocular information about TTC that remain unanswered: How is the binocular information in equation (1) converted into a TTC signal? (Chapter 9 offers one possibility that is framed entirely in terms of retinal image information). Does the relative weighting of binocular and monocular TTC information differ for diverse eye-limb coordination tasks such as hitting, catching or running over rough ground? Is the relative weighting different for novice and experts performers? Does it change with practice? Clearly, the role of different information sources (other than f) in the judgment of TTC and the control of collisions is a fertile and developing research area.

The The Use Use of of Binocular Binocular Time-to-Contact Time-to-Contact Information Information

323 323

REFERENCES Alderson, G.J.K., Sully, D.J. & Sully, H.G. (1974). An operational analysis of a one-handed catching task using high-speed photography. Journal of Motor Behaviour, 6, 217-226. Bannister, H. & Blackburn, J. M. (1931). An eye factor affecting proficiency at ball games. British Journal of Psychology, 21, 382-384. Beverley, K.I. & Regan, D. (1973). Evidence for the existence of neural mechanisms selectively sensitive to the direction of motion in space. J. Physiol. 235, 17-29. Beverley, K. I. & Regan, D. (1979a). Separable aftereffects of changing-size and motion-in-depth: different neural mechanisms? Vision Res, 19(6), 727-732. Beverley, K.I. & Regan, D. (1980). Visual sensitivity to the shape and size of a moving object: implications for models of object perception. Perception, 9, 151-160. Beverley, K.I. & Regan, D. (1982). Adaptation to incomplete flow patterns: no evidence for "filling in" the perception of flow patterns. Perception, 11, 275-278. Bootsma, R. J. & Oudejans, R. R. (1993). Visual information about time-to-collision between two objects. J Exp Psychol Hum Percept Perform, 19(5), 1041-1052. Cavallo, V. & Laurent, M. (1988). Visual information and skill level in time-to-collision estimation. Perception, 17(5), 623-632. Gray, R. (2001). Behavior of college baseball players in a virtual batting task. Journal of Experimental Psychology: Human Perception and Performance., In press. Gray, R. & Regan, D. (1998). Accuracy of estimating time to collision using binocular and monocular information. Vision Res, 38(4), 499-512. Gray, R. & Regan, D. (1999a). Adapting to expansion increases perceived time-to-collision. Vision Res, 39(21), 3602-3607. Gray, R. & Regan, D. (1999b). Do monocular time-to-collision estimates necessarily involve perceived distance? Perception, 28(10), 1257-1264. Gray, R. & Regan, D. (2000a). Estimating the time to collision with a rotating nonspherical object. Vision Res, 40(1), 49-63. Gray, R. & Regan, D. (2000b). Risky driving behavior: a consequence of motion adaptation for visually guided motor action. J Exp Psychol Hum Percept Perform, 26(6), 1721-1732. Gray, R. & Thornton, I. M. (2001). Exploring the link between time to collision and representational momentum. Perception, 30(8), 1007-1022. Grosslight, J. H., Fletcher, H. J., Masterton, R. B. & Hagen, R. (1978). Monocular vision and landing performance in general aviation pilots: Cyclops revisited. Human Factors, 20, 127-133. Harris, J. M. & Watamaniuk, S. N. (1995). Speed discrimination of motion-in-depth using binocular cues. Vision Res, 35(7), 885-896. Heuer, H. (1993). Estimates of time to contact based on changing size and changing target vergence. Perception, 22(5), 549-563. Hofeldt, A. J., Hoefle, F. B. & Bonafede, B. (1996). Baseball hitting, binocular vision, and the Pulfrich phenomenon. Arch Ophthalmol, 114(12), 1490-1494.

324


Howard, I. P. & Rogers, B. J. (1995). Binocular Vision and Stereopsis (Vol. 29). Oxford: Oxford University Press. Hoyle, F. (1957). The Black Cloud. Middlesex, England: Penguin. Kohly, R. P. & Regan, D. (1999). Evidence for a mechanism sensitive to the speed of cyclopean form. Vision Res, 39, 1011-1024. Kohly, R. P. & Regan, D. (2002). Fast long-range interactions in the early processing of luminance-defined form. Vision Res, in press. Laurent, M., Montagne, G. & Durey, A. (1996). Binocular invariants in interceptive tasks: A directed perception approach. Perception, 25(12), 1437-1450. Lee, D. N., Lishman, J. R. & Thomson, J. A. (1982). Regulation of gait in long jumping. Journal of Experimental Psychology-Human Perception and Performance, 8,448-459. Lewis, C. E. Jr., Blakeley, W. R., Swaroop, R., Masters, R. L. & McMurty, T. C. (1973). Landing performance by low-time private pilots after the sudden loss of binocular vision— Cyclops II. Aerospace Med. 44, 1241-1245. Lewis, C. E. Jr. & Kriers, G. E. (1969). Flight research program: XIV. Landing performance in jet aircraft after the loss of binocular vision. Aerospace Med., 40, 957-963. McLeod, P., McLaughlin, C. & Nimmo-Smith, I. (1985). Information encapsulation and automaticity: evidence from the visual control of finely tuned actions. In M. I. Posner & O. S. M. Marin (Eds.), Attention and Performance (Vol. 11, pp. 391-406). New Jersey: Lawrence Erlbaum. Pfaffman, C. (1948). Aircraft landings without binocular cues. A study based on observations made in flight. Amer. J. Psychol. 61, 323-335. Portfors-Yeomans, C. V. & Regan, D. (1996). Cyclopean discrimination thresholds for the direction and speed of motion in depth. Vision Res, 36(20), 3265-3279. Portfors-Yeomans, C. V. & Regan, D. (1997). Discrimination of the direction and speed of motion in depth of a monocularly visible target from binocular information alone. J Exp Psychol Hum Percept Perform, 23(1), 227-243. Profitt, D. R. & Kaiser, M. K. (1995). Perceiving Events., Perception of Space and Motion.: Academic Press. Pulfrich, C. (1922). Die Stereoscopie im Dienste der isochromen und heterochromen Photometrie. Naturwissenschaft, 10, 553-564 Regan, D. (1992). Visual judgements and misjudgments in cricket, and the art of flight. Perception, 21(1), 91-115. Regan, D. (1995). Spatial orientation in aviation: visual contributions. J Vestib Res, 5(6), 455-471. Regan, D. & Beverley, K. I. (1978). Looming detectors in the human visual pathway. Vision Res, 18,415-421. Regan, D. & Beverley, K. I. (1979a). Binocular and monocular stimuli for motion in depth: changing-disparity and changing-size feed the same motion-in-depth stage. Vision Res, 19(12), 1331-1342. Regan, D. & Beverley, K. I. (1979b). Visually guided locomotion: psychophysical evidence for a neural mechanism sensitive to flow patterns. Science, 205, 311-313.


325

Regan, D. & Gray, R. (2000). Visually guided collision avoidance and collision achievement. Trends in Cognitive Sciences, 4(3), 99-107. Regan, D. & Hamstra, S. J. (1993). Dissociation of discrimination thresholds for time to contact and for rate of angular expansion. Vision Res, 33(4), 447-462. Regan, D. Beverley, K. I. & Cynader, M. (1979). The visual perception of motion in depth. Scientific American, 24(7], 136-151. Regan, D. Erkelens, C. J. & Collewijn, H. (1986a). Necessary conditions for the perception of motion in depth. Invest Ophthalmol Vis Sci, 27(4), 584-597. Rushton, S. K. & Wann, J. P. (1999). Weighted combination of size and disparity: a computational model for timing a ball catch. NatNeurosci, 2(2), 186-190. Savelsbergh, G. J., Whiting, H. T. & Bootsma, R. J. (1991). Grasping tau. J Exp Psychol Hum Percept Perform, 17(2), 315-322. Schiff, W. & Detwiler, M. L. (1979). Information used in judging impending collision. Perception, 8(6), 647-658. Scott, M. A., Li, F. X. & Davids, K. (1996). The Shape of Things to Come: Effects of Object Shape and Rotation on the Pick-up of Local Tau. Ecological Psychology, 8(4). Smith, M. R., Flach, J. M., Dittman, S. M. & Stanard, T. (2001). Monocular optical constraints on collision control. J Exp Psychol Hum Percept Perform, 27(2), 395-410. Todd, J. T. (1981). Visual information about moving objects. J Exp Psychol Hum Percept Perform, 7(4), 975-810. Tresilian, J. R. (1995). Perceptual and cognitive processes in time-to-contact estimation: analysis of prediction-motion and relative judgment tasks. Percept Psychophys, 57(2), 231-245. Tresilian, J. R. (1999). Visually timed action: time-out for 'tau'? Trends in Cognitive Sciences, 3(8), 301-310. Wann, J. P. (1996). Anticipating arrival: is the tau margin a specious theory? J Exp Psychol Hum Percept Perform, 22(4), 1031-1048. Watts, R. G. & Bahill, A. T. (1991). Keep Your Eye on the Ball: Curve Balls, Knuckleballs, and Fallacies of Baseball. New York: W.H. Freeman and Company. Wheatstone, C. (1852). Contributions to the physiology of vision II. Philos Trans R Soc Lond, 142, 259-266.


Time-to-Contact –- H. Hecht and G.J.P. Savelsbergh (Editors) (Editors) © 2004 Elsevier B.V. All rights reserved

CHAPTER 14 Interception of Projectiles, from When & Where to Where Once

Simon K. Rushton Cardiff University, Cardiff, UK

ABSTRACT In this chapter the where once model is introduced. The where once model is concerned with the prediction of the future egocentric position of a moving object. It is a very simple model based upon four visual variables to which humans have a documented sensitivity. The proposed model is a general purpose predictive algorithm that could be used in conjunction with a variety of perceptuo-motor control strategies and in a variety of circumstances.

328 328

Simon K. K. Rushton Rushton

1 Introduction & background What visual information is used during interception of an approaching projectile? For the majority of the past thirty years, researchers have attempted to answer this question by addressing themselves to two related questions: how does an observer know where the projectile is going and when it will get there? Work on the former has involved identifying possible sources of information about the trajectory of the projectile or the lateral distance at which it will pass the observer. Work on the latter has been concentrated on estimation of 'time-tocontact' (TTC) that is the number of seconds remaining before the projectile hits or passes the observer. Although not all researchers would necessarily acknowledge such an inspiration, this approach can be identified with the Gibsonian, or Ecological programme (Gibson, 1979). The Ecological programme has concentrated on finding patterns or relationships within the optic flow field that can be used to directly regulate action. A classic example of this is the 'focus of expansion' and the visual guidance of locomotion: The "flow-field" is the optic array sampled at a moving observation point by an eye or camera. It was noted that when an observation point translates through an environment that motion within the flowfield, appears to stream out from a single point. This point is the "focus of expansion" and it indicates the direction of travel of the observer (Grindley cited in Mollon, 1997; Gibson, 1958). Hence observers can identify the focus of expansion and determine its position relative to the image of their target (for example a doorway). If the two are not coincident then the observer can change their direction of locomotion appropriately. The critical point is that all the information necessary for the solution of the problem of visual guidance of locomotion is found within the optic array. There is no need to consider anything about the observer - the nature or state of their motor system, or visual system, or how the two are connected. Nor is there any need to build any form of representation of the outside world with which the observer interacts. In the case of projectile interception, tau (when) and the crossing distance (where) are the equivalents of the focus of expansion. Tau, f, is defined as the ratio of the current angular extent of an approaching object to its rate of change of extent (Lee, 1976; see equation [1] below). This ratio 'directly' specifies the number of seconds remaining before the object will collide with the observation point (assuming the velocity of the object remains constant), there is no need to determine distance and velocity. Tau is information that is available in the optic array at the observation point just waiting to be picked up by any creature (or device) with an appropriate perceptual system. In Lee's original example, braking a car, a driver can brake safely by simply monitoring and responding to T. Again, no knowledge of the perceptual or motor systems is required. So long as the observer can increase or decrease the braking, they can

Interception of Projectiles, from from When & Where to Where Once

329

"couple" their braking action directly to the information they receive from their visual system. Since Lee's original paper attempts have been made to extend the use of tau to other problems including interception of projectiles (eg Lee et al, 1983, ball punching). The crossing distance, X c , ratios are also informative patterns in the optic flow field waiting to be picked up. Several different crossing distance equations have been proposed (described below). Again, as with X and the focus of expansion, all other parameters (such as eye orientation) are either assumed not to be necessary, or in some cases minor details that are "obvious" or that can be specified later. The tau model has come under some attack. It is assumed that other chapters in this book provide an up to date summary, but to give a few selective examples, Wann (1996) questioned the data reported to support the use of X. Tresilian (1999) questioned the assumption that X is perceived 'directly', that is from the retinal ratio and suggested that other cues and strategies might be used in the perception of TTC. Michaels et al (2001) questioned whether interceptive timing is based upon X, they suggested that the initiation of an interception action might be based upon a simpler quantity, looming rate. However none of these researchers have challenged the basic when & where idea. The major challenge to the when & where idea came from Peper et al (1994) who proposed that interception is based upon continuous control. In their model, the observer never needs to explicitly know where a projectile is going. The observer can use a continuous control strategy that couples hand movement to optical information so moving their hand to the right place at the right time to intercept a ball (this strategy is covered in more detail later). The where once proposal outlined in this chapter starts with a consideration of the ecological problem for an observer who wants to interact with an object. The where once model might be termed an egocentric model in that it starts with the assumption that the critical problem that an observer must solve to successfully guide action is to determine where an object is relative to their body1, and if observer or the object is moving, to anticipate where the object will be once a period of time has elapsed. Knowing where an object will be once a given period of time has elapsed would be equally useful whether the observer is picking up a teacup, running to tackle a rugby player on the opposing team or catching a ball. In the first part of this chapter the problems with the when & where solution are reviewed. The when & where solution is found wanting, and a more traditional "secondary-school-physics" solution is considered, and relevant data discussed. In the final section the where once solution is introduced. The where once proposal starts from an 'egocentric' perspective on the problem and then 1

cf. the egocentric account of the visual guidance of locomotion (Rushton et al, 1998)

330

Simon K. Rushton

assembles a solution that draws heavily on data and ideas culled from the existing literature on interception, 7 and motion-in-depth.

2. When & where In this chapter I do not provide an extensive review of the when & where literature. If the reader requires a summary they can either consult other chapters within the book or review papers on TTC by Tresilian (1999) or the visual guidance of interception by Regan & Gray (2000). 2.1 When TTC is the number of seconds remaining before the projectile collides with the observation point. T is a first order approximation of TTC, based upon the assumption that the velocity of the projectile is constant (Lee, 1976).

where 6 is the size of the retinal image of the projectile, and 0 is the rate of change of retinal image size. Related information such as Time-To-Passage (TTP), the number of seconds remaining before the projectile crosses the frontoparallel plane containing the observation point has also been identified (Bootsma & Craig, 2002 is the most recent paper on this topic). Unfortunately interception normally occurs in front of the observation point. Therefore neither T nor TTP information provides sufficient information for interceptive timing under natural circumstances. The estimate of X or TTP needs to be "corrected" for interception before the observation point. First the distance of the interception point from the observation point must be known. Then either the velocity of the approaching projectile, or its instantaneous distance is required. However, use of either distance or velocity completely undermines the notion of "direct" perception. There are some cases for which potential "direct" solutions exist. One example would be when the hand and the projectile are fortuitously simultaneously in view. In this situation relative disparity could be used to directly estimate TTC with the hand. But such circumstances are special cases and are not representative of the normal circumstances under which projectiles are intercepted. Tau information is no more "sufficient" for interceptive timing than knowledge of only object distance, or only object speed.

Interception of Projectiles, from from When & Where to Where Once

331

2.2 Where Regan & Beverley (1980) pointed out that if we consider the speed of the lateral movement of an approaching ball within the left eye retinal image (or optic array) and compare it to the speed in the right eye retinal image, that the ratio of the speeds is correlated with the trajectory of the ball. When the ball is travelling straight towards the nose then the ratio will be -1:1, when the ball is travelling towards one of the eyes then the ratio will be 0:1 or 1:0. Bootsma (1991) showed that the lateral distance at which a projectile will cross the plane containing the eyes (the 'crossing distance'), X c , is:

2R

e

{)

where & is the angular lateral speed of the projectile at the Cyclopean eye and R is the radius of the projectile. Other variants of this equation based upon changing disparity have subsequently been proposed (Laurent, 1996, Regan & Kaushall, 1994). x

c _« i 0

(Vi

where <j) is the rate of change of binocular disparity and I the inter-ocular separation. Lastly, X c can be calculated from the relative left/right eye image velocities: Xr I

ocR / a , +1 dR/dL-l

...

Where d R is the angular speed of the ball within the image in the right eye and d L the speed in the left eye. Note the three equations above are approximations. Below (Figure 1) iso-ratio curves are shown in a plan view. The Cyclopean eye (mid-point between the left and right eye) of the observer is located at (0,0). In the three panels the ratio d/0 is constant (iso-ratio curves for d/<j) or d R / d L would look similar). To aid an explanation of how the curves were generated, let us first define (3 as the angle between straight-ahead (x=0) and the direction of travel of the projectile. We will also assume an inter-ocular distance, I , of 64 mm.

332

Simon K. Rushton

In the left panel ratio oc/6 was calculated for a projectile that is at (0,5) and has a trajectory angle, (3, of 10°. If the projectile continued along this trajectory it would cross the x-axis 0.88m to the right of the Cyclopean eye (Xc=5.tan(100)). The ratio a/9 is 13.8 (d/e = X c /I = 0.88/0.064). Using this value for a / 6 , iso-ratio lines were generated from a grid of starting points (marked by circles in each panel). At every point, on every line, within a panel, the ratio oc/6 is the same (left panel d/d = 13.8; middle panel d/8 = 0; right panel d/9 = 28.4). The consequence of this is that at every point within a panel the value of X c calculated using equations [2-4] is the same (left panel X c = 0.88m; middle panel X c = 0; right panel X c = -1.82m). Recall X c is the lateral distance at which the projectile will cross the x-axis (fronto-parallel plane containing the eyes) assuming its velocity, or more specifically its direction of travel, does not change. At any point on one of the iso-ratio lines, the current direction of travel is the tangent to the curve. In the left and right panels the tangents clearly do not converge at a single point on the x-axis. The tangents do cross the x-axis at the estimated position X c when the projectile is straightahead. However, the error in estimation of X c from equations [2-4] increases as function of eccentricity, nearness and lateral distance at which the projectile will pass the Cyclopean eye. This is unfortunate as it means an estimate of X c will in general become more and more inaccurate as it becomes more and more important - just before interception.

\

•

100 0

i i

• • r

200

3.

\ ':

;

-100 '•_

-200 -300 -300

r

-200

i

i

i

i

-100

i

i

i

i

i

i

t

i

100

i

i

i

200

300

Figure 4: Interceptive actions when wearing prisms. Plan view, dimensions measured in millimetres. Approximate position of Cyclopean Eye indicated by square at (0, -250), a fronto-parallel plane also shown. Ball travels from near (0, 500) towards observer. Finger of right hand is moved from left leg, near (-150, -150) up to intercept the ball.

Prisms perturb perceived direction and the angular error observed during reaching was directly proportional to the optical displacement of the prisms . Therefore we can conclude that the observers are making use of information about perceived instantaneous direction of the projectile. 4.1.3 Tau and a speed ratio Extensive work by Regan and colleagues has documented the sensitivity of the visual system to T . The criticisms of the "tau hypothesis", that tau is used to guide interceptive timing do not have any relevance to the extensive body of :

Head position was held constant so error was introduced in eye-orientation signal.

342

Simon K. Rushton

work by Regan that is concerned with psychophysical sensitivity. Therefore it is taken as demonstrated that the brain has access to X information. The speed ratio could be one of any of the previously identified ratios, d/, d R / d L or d / 9 . Although there is a lot of data is compatible with a sensitivity to a speed ratio, none of the published research fully discriminates between the three (the recent research on perception of trajectory I have conducted with Phil Duke indicates it is probably d/(<j) + 8)). However, for the sake of simplicity, in the section that follows the ratio d/<j> will be assumed (note the where once model works with any of the speed ratios): 4.2 Predicting future position 4.2.1 Tau and current distance - anticipating future distance We assume that the current distance of an object, D o , is known, and also T , the number of seconds before it will collide with the observation point. If the object is (or is assumed to be) travelling at a constant velocity then the distance that the object will travel towards the observer (Ad) in a given interval of time, At, is Ad = DQ.— T

(9)

and the distance from the observer after an interval At has elapsed, D t is D t = D 0 - A d = D 0 .(l-—) x

(10)

As binocular disparity is proportional to the reciprocal of distance, we can substitute l/

y> Xopen ^open'

X

close ^close >

Pvisio

geyX>y>

X

open

X

open'}'open ) \ open/

X X

J range v-*> ^ ' close ^

Pvisi ision (D)

= Vision

X

close I

J

close)

~ >"„,,)

(3a)

(3b)

Note the similarity between Equation (3a) and Equations (2c) and (2d). A closer look at these equations learns that the growing value of the visual variable r(D), through the variable /%&„„, decreases the strength of attraction of the point-attractor at xopen and increases the strength of attraction of the point attractor at xciose; f5VisWn and /?;„, play the same role in their respective equations. Obviously, a more thorough introduction into the model equations can be found in the original Schoner (1994a) paper.

Time-to-Contact –- H. Hecht and G.J.P. Savelsbergh (Editors) (Editors) © 2004 2004 Elsevier B.V. All All rights reserved reserved

CHAPTER 18 Another Timing Variable Composed of State Variables: Phase Perception and Phase Driven Oscillators

Geoffrey P. Bingham Indiana University, Bloomington, IN, USA

ABSTRACT In this chapter, we consider a perceptible variable that is related to x, but is different from x. The variable is phase, ((>.
444

Simone Caljouw, John van der Kamp and Geert Savelsbergh

1. The timing of catching and hitting Skilled athletes perform dynamic interceptive actions such as catching and hitting with ease and a high degree of accuracy. To intercept a ball, actors must contact the ball with the right spatial orientation and velocity of the hand, bat, or racket. The end-effector has to be at a certain location at a particular time. Therefore, precise coordination, between visual information about the target's trajectory and the kinematics of the end-effector is necessary to prepare for impact. Timing is very important in these interceptive actions. In catching, the ball will bounce off the hand or crush the fingers when the moment of hand closure is late or early, respectively. In the case of balls travelling with a speed of about 10 m/s, the precision with which the grasping action of the hand must be timed is estimated to be on the order of 15 ms (Alderson, Sully, & Sully, 1974). When striking fast balls, the temporal precision is even higher. Researchers reported temporal accuracy of about 6.5 ms for a forehand drive in table tennis (Bootsma & Van Wieringen, 1990) and 2.5 ms for batting in cricket (Regan, 1997). While hitting is similar to catching in a number of ways, there are some important differences. In catching, the contact with the ball should always be soft, because the fingers have to enclose the ball. On the other hand in hitting, the main objective is to transfer energy to the ball in order to transport it with a certain velocity in a particular direction. The ball is received and sent away in the same action. Hence, the actor has to ensure that the implement (or hand) travels with the right velocity at the moment of contact to transfer an appropriate amount of kinetic energy to the ball. In a review on batting in men's cricket, Stretch, Bartlett, & Davids (2000) described how a batter sometimes faces extremely severe spatio-temporal constraints. For example, in the leg glance, the frontal plane of the blade of the cricket bat (± 10 cm in width) has to be quickly turned almost 90 degrees to deflect the ball past the wicket keeper. Yet, skilled cricketers can successfully perform this intricate stroke against fast bowling speeds of around 40 m/s (Regan, 1997). With one of the most remarkable aspects of catching and hitting being the precision with which the actions are timed, it is perhaps not surprising that special attention has been paid to visual information specifying the temporal relation between an observer and the ball. Whiting & Sharp (1973) clearly showed that it was necessary to "keep an eye on the ball" for a successful timing of interceptive action. Catching performance was better when the visibility of the ball is available 320 ms instead of 160 ms before ball-hand contact. An organism-environment property thought to be of particular importance in the

The Fallacious FallaciousAssumption AssumptionofofTTC TTCPerception Perceptionininthe theRegulation RegulationofofCatching Catchingand andHitting Hitting 445 445

timing of interceptive actions is the time-to-contact (TTC), i.e. the time remaining before the ball reaches the observer. TTC can be defined as the distance of the ball at some instant of time divided by its velocity. Information about TTC is identified in the structure of optical expansion patterns generated by the approaching ball (see Section 2). This optical pattern is visually available and does not have to be computed from independently obtained perception of distance and velocity. In this Chapter we adopt Gibson's theory of direct perception or the ecological approach as a starting-point to understand how movement coordination is constrained by information (Gibson, 1979; Michaels & Carello, 1981). Gibson rejected the claim that observers have to construct a representation of the world from ambiguous stimuli (resulting from a 2D projection of the 3D world on the retina) by relying on constructive processes such as inference. In contrast, he argued that objects and events can be directly perceived and information does not have to be disambiguated. An abundance of information resides in optical patterns that are generated by (moving) objects and events as well as by movements of the observer. Perception is based on the detection of optical patterns that directly specify actor-environment properties. Specificity implies one-to-one mapping between physical properties and optical information. Given the constraints and regularities in the to be considered ecologies, information is lawfully related to its source (i.e. object, event) such that no other source could have generated the same optical pattern (Runeson, Jacobs, Andersson, & Kreegipuu, 2001). For many years, research on information-movement coupling in interceptive actions was dominated by the perspective that humans are able to detect a source of information that specifies the time it takes an object to reach an observer (TTC). By detecting the optical information, the actor perceives when an object is close enough to act upon it. In the present Chapter we will begin with an overview of this original TTC-model. We then review recent studies that show the flaws of this model and we outline that the TTC-model is insufficient to explain the information-based regulation of catching and hitting. In the final Section we will provide empirical evidence for the view that interceptive actions can be regulated by multiple sources of information. We will present alternative models and consider the implications for general issues in information-movement coupling.

446


2. The TTC-model Although it was already derived in 1958 that the optical expansion pattern of an approaching object specifies TTC (Knowles & Carel, 1958; Purdy, 1958), it was not until the early eighties that it became widely accepted. Within the ecological psychology framework, Lee (1976) demonstrated that TTC is directly available in the optic array. Hence, there is no need for actors to estimate the distance and velocity of an object. He mathematically showed that, given a constant approach velocity, the inverse of the relative rate of optical expansion, directly specifies TTC with the point of observation.1 This optical variable is denoted tau (i.e. T(, which results in time as the sole measurement unit. Van der Kamp et al. (1997) have systematically investigated the effects of ball size on the timing of one-handed catches. They found that participants opened and closed their hand at different times before contact. The larger the ball the earlier the hand opened and closed (see also Michaels et al., 2001). These results are in agreement with the "size-arrival" effect found by DeLucia and co-workers in TTC judgement studies using computer-simulated scenes (see Chapter 10 this volume). Studies that investigated the effect of approach speed also showed that participants did not initiate their movements at a constant TTC. The TTC at which participants initiated their movement consistently decreased with increasing speed (Bennett et al., 1999; Li & Laurent, 1994, 1995). Neither size nor speed effects are consistent with a TTCi strategy.

4. Alternative models for the control of interceptive actions We must conclude that there is hardly any empirical evidence for the original TTC-model as presented in Figure 1. At first, the axiom that exclusively x(cp) is detected to perceive TTQ for the regulation of interceptions is flawed. Furthermore, actors do not solely regulate their movements on basis of the perceived TTCi. Most research presented suggests that an explanation of the control of timing exclusively on the basis of a critical t(cp) strategy must be ruled out. Therefore we have to search for other possible models that might explain interceptive timing.

458


4.1 Direct action For many years the search for optical variables was constrained by information specifying TTCi. This was due to the implicit reasoning that the detection of information about an event entails the perception of this event, and it is the perception of this event that regulates the action (cf. Figure 1). But, as shown in the previous Section, actors do not always rely on perceived TTQ. This raises the issue whether, in the case of interceptive movements, the used information ought to specify TTC. The intermediate phase of perception may be superfluous. That is, information might directly regulate the timing without the need for perceiving TTC. This perspective results in the assumption that any information that is to some degree correlated to the approach of the ball, including optical variables that do not specify the environmental property TTC (i.e. lower order variables), might be used to regulate interceptive movements. For instance, several authors have recently proposed that the optical basis for the timing of interceptive action may be the "looming" variable or absolute rate of expansion (Michaels et al., 2001; Smith, Flach, Dittman, & Stanard, 2001; Van der Kamp et al., 1997, 1999). As described in the previous Section, it was found that the timing of catching a ball is affected by ball size and velocity. The larger the object and the faster the approach, the earlier (i.e. at a longer TTC) people initiated their action. Michaels et al. (2001) attempted to push the qualitative agreement between (p and the initiation of the action into a quantitative one. They presented techniques to determine the critical values and corresponding visuomotor intervals of optical variables. The optical variable used to initiate flexion of the arm turned out to be the absolute rate of expansion, rather than the relative rate of expansion. The optical variable that regulated the extension of the arm was less conclusive, since different variables (including (j) and x(q>)) were exploited by different participants. Also, the telestereoscope experiments showed that informational variables that do not specify TTCi contribute to the timing of catching. The telestereoscope manipulates the absolute value of the binocular variable target vergence by increasing the IO separation. Hence, when the relative rate of change of the binocular variable provides the information, wearing a telestereoscope will have no effect on the timing of the catch. The results showed that participants, while wearing the telestereoscope, closed their hand earlier around the ball (Bennett et al., 1999, 2000; Van der Kamp et al., 1999), suggesting that lower order variables contribute to the regulation of catching. In a recent article Michaels (2000) discussed the use of a lower order variable to control the timing of interceptive actions. Ecological psychologists expect a 1:1 mapping between informational variables and to-be-perceived

FallaciousAssumption Assumptionof ofTTC TTCPerception Perceptionin inthe theRegulation Regulationof ofCatching Catchingand andHitting Hitting 459 45 9 The Fallacious

properties. This relation holds for the optical quantity T(cp), which specifies TTCi. On the other hand, (j) is ambiguous with respect to object size, distance, velocity, and TTC. It specifies nothing. But, perhaps it is not necessary to first perceive an environmental property before acting upon it. The present proposal is that the initiation of interceptive actions may require the detection of a critical threshold value of an optical quantity; it does not require the perception of TTC. In the direct action model (schematised in Figure 6) the optical variable directly modulates action.

Critical value

Information

1

Action (Timing)

Figure 6: Schematic representation of the direct action model. In this model information directly modulates action, without the interposition of an unnecessary perceptual entity.

The direct action model, in line with the original TTC-model, assumes that the initiation of an interceptive movement is based on a critical value of a single informational variable. Most of the presented research in this Chapter violated regulation on the basis of x(cp), and indicated the exploitation of other informational variables (e.g. T((p,6), x(A), (p, A). What remains is a rather fragmented picture; it is impossible to provide a precise formulation of an alternative variable that can account for all of the observed effects. Gibson (1979) introduced the concept of a compound invariant, which is a unique combination of invariants. He argued that when a number of stimuli are completely covariant, when they always go together, they constitute a single stimulus (Gibson, 1979, p. 141). Obviously the question that has to be answered is "how do the variables combine to constitute a compound invariant"? At present we have no clear answer to this question. Note however that by no means the constituting informational variables have to be chosen such that in combination they inform precisely about TTC.

460


4.2 Directed action The original TTC model assumes that the pick up of informational variables is not flexible. After all, a sole variable could be used by many species to regulate many interceptive actions. Based on the evidence presented, we can conclude that multiple optical quantities are involved in the regulation of interceptive actions. But, how should the claim that timing is based on multiple information sources be understood? One might assume that temporal control is based on a single source of information, selected out of several available sources depending on the task at hand (Savelsbergh & Van der Kamp, 2000). For example, Van der Kamp et al. (1997) investigated what the role of monocular and binocular information was, in a natural interceptive task. They studied the timing of the opening and closing of the hand in response to an approaching ball. A size-arrival effect was found in the monocular condition but not in the binocular condition (cf. Michaels et al., 2001). This indicates that participants use different variables under different viewing conditions. Van der Kamp et al. (1997) argued that the size-arrival effect is consistent with the use of expansion information. No size-arrival effect was found in the binocular condition, which suggests that participants used a binocular variable that accounted for the constant TTC strategy (i.e. relative rate of target vergence). So, the human visual system is opportunistic and flexible in the pick up of variables that are present in the optic array (tau, optical expansion etc.). The idea that different information sources might be needed to control the rich diversity of interceptive tasks is in line with the present neuroscientific view that the visual system is composed of several functional subdivisions. The onedimensional view of much traditional research that the only role of the eyes and the associated cortical areas is to build a single representation of the world around us is in general rejected. For example, Milner and Goodale's (1995) model of the two-visual pathways can help us understand how the brain uses information depending on the task at hand. The dorsal stream, which addresses the motor areas of the brain more frequently, supports the visual control of goal directed actions. The ventral stream, mainly connecting to higher cortical areas such as memory, is used to identify characteristics of objects and events. These findings suggest that multiple information sources are detected with multiple perceptual systems as a function of task-constraints. That is to say, selection of information depends on constraints such as the available sources of information, environmental regularities and task demands. The process of selection is not an internal constructive process, such as inference, but relies on constraints in the task ecologies.

FallaciousAssumption Assumptionof ofTTC TTCPerception Perceptionininthe theRegulation Regulationof ofCatching Catchingand andHitting Hitting 461 461 The Fallacious

In contrast to the perspective that a single source of information is used depending on the task-circumstances is the view that various binocular and monocular optical information sources as well as oculomotor information simultaneously contribute to the regulation of timing. Tresilian (1994b; see also Wann & Rushton, 1995) proposed that timing patterns reflect the combination, by summation or multiplication, of several differently weighted sources of information. The weights are granted on basis of prior and current perceptual information about the accuracy of the information sources. For instance, both monocular and binocular information may be used to time interceptive movements when participants view the scene with two eyes. Both Heuer (1993), and Gray & Regan (1998), in line with Regan and Beverly (1976), argued that monocular and binocular information is summed and that binocular information becomes more dominant as the object size is small. And, comparably, the dipole model of Rushton and Wann (1999) combines estimates of TTCi, based on binocular (disparity) and monocular (expansion) information and applies different weights to both cues depending on their immediacy. When disparity information specifies that the ball will hit the point of observation in advance of the temporal estimate that is provided by expansion information, participants will rely on disparity information. Notice that Rushton and Wann follow the adjusted TTCi model (cf. Figure 4), which was found to be flawed in the previous paragraphs. Bennett et al. (2000) used the telestereoscope to test whether the contribution of binocular information is mediated by object size. They predicted an interaction effect between ball size and telestereoscopic viewing. If subjects are more reliant on binocular information when catching a small ball, they should initiate key aspects of the catch earlier under telestereoscopic viewing than with normal viewing. If subjects are less reliant on binocular information when catching a large ball, there should be less or no effect of telestereoscopic viewing on the timing of the catch. This experiment showed, as expected, that participants closed their hand earlier for the small and medium sized balls under telestereoscopic viewing compared to normal viewing. Following this perspective, adaptation to new task-constraints should be based on recalibration of the existing information-movement coupling instead of selecting an alternative coupling. To investigate whether adaptation occurs through selecting a different information-movement coupling or through recalibration of an existing information-movement coupling, another one-handed catching experiment with the telestereoscope was performed (Van der Kamp et al., 1999). A pre-exposure, exposure, post-exposure design was used. In the preexposure and post-exposure phase half of the participants were required to perform monocular catches and the other half had to perform binocular catches.

462


In the exposure phase both groups had to catch balls under binocular viewing while wearing the telestereoscope. If participants were able to transfer easily to the information-movement coupling they established in the post-exposure design (i.e. selection) no after-effects would occur. However, in the case of a recalibration process, after-effects would be expected. If recalibration was restricted to the manipulated information, only the binocular group would show after-effects, whereas after-effects might occur in the monocular group if recalibration involved multiple information sources. The results showed that the hand was closed later in the first three trials after removal of the telestereoscope. No differences in the after-effects were found between the monocular and binocular group. This experiment suggests that adaptation to the telestereoscope is due to recalibration of the coupling between information and movement, rather than a selection of other information. Moreover, the recalibration is not restricted to the manipulated binocular information but may encompass also monocular information sources. Proponents of the directed perception approach (Cutting, 1986, 1991; Laurent et al., 1996) argued that the perception of an environmental property (e.g. TTC) could be perceived by the detection and combination of different optical variables (e.g. i((p), T(A), and x(cp,6)). We come to terms with this approach in that interceptive actions may be based on multiple sources of information (cf. Figure 7) and these variables contribute more or less depending on the task-circumstances. However, we do not agree completely with proponents of the directed perception approach for several reasons. First, we reject the perspective that perception of the environmental property TTC is a vital step in the timing of an interceptive action. Second, we prefer a selforganized dynamic account of information use above a calculated combination of several differently weighted sources of information. The dynamical systems approach claims that optical information is one of the influences acting on the intrinsic dynamics of a system. So, we may assume that different information-related dynamics emerge from the use of different informational variables. If multiple sources of information are exploited, the entire dynamics of the system is given by a synergy in which the multiple information-related dynamics are assembled. For example, the dynamics of a system intercepting an approaching ball at the right time may be described by a synergy of the dynamics emerging from the use of expansion velocity and change of disparity. This process of assembling depends on different constraints; (i) Organismic constraints determine whether the particular information sources can be detected.


Task constraints

Multiple information q),A,T((p),T(e),t(A)etc.

I Action (Timing)

Critical value(s)

Figure 7: Schematic representation of the directed action model. On the basis of taskconstraints several co-varying variables are combined and/or selected that directly modulate action.

The change of disparity will not exert any influence on the dynamics of the system if the actor does not have stereoacuity. (ii) Environmental constraints determine which information is available to the system. If the approaching ball is far away binocular information is less salient (Collewijn & Erkelens, 1990). (iii) Task-constraints determine which information is relevant to perform a specific task. If a fronto-parallel moving ball, at a constant distance from the point of observation, has to be intercepted, the system will not be attracted to the dynamics of both information sources and the dynamics emerging from other information sources will contribute to the entire dynamics of the system. A selforganized dynamic account of information use does not suggest that actors weight and combine information sources, it suggests that the system settles in a state depending on several constraints. Allthough this dynamical process of combining information may be conveniently described in terms of addition and multiplication, it does not necessitate the brain solving this equation. To account for the direct influence of a multitude of variables on the timing of interceptive actions we introduce the term "directed action".

464


5. Another control strategy Accepting that empirical evidence for the original tau-model is at best scarce, one of two positions can be taken. One can either propose an alternative for the information used or one can reconsider the way in which the information is used to regulate the movement. The TTC-model was originally a predictive model, that is, it presumed that a movement was regulated on basis of information that specified when a future event (i.e. the interception) would occur. A critical value of perceived TTCi was used to initiate a ballistic movement of an invariant movement time and a constant visuo-motor interval (Tyldesley & Whiting, 1975). Predictive information might be useful in a visual masking task, where participants have to predict when a suddenly occluded object will arrive at a designated target area. In a natural interceptive task participants are able to view the ball until it is received into the hand. Hence, there is no need for information that accurately predicts the place and time of interception. The information continuously informs the actor about the current state of the actor-environment system and this information is used to update the movement online. In this section, different continuous control models will be presented. The first model is purely phenomenological and describes, with help of dynamical equations of motion, how the timing of the grasp component in prehension emerges on the basis of optical information generated by the reach component (Zaal, Bootsma, & van Wieringen, 1998; see also Chapter 17 of this volume). The other models described, are based on the Vector Integration to Endpoint model of Bullock and Grossberg (1988). They take into account a lateral catching task in which besides temporal information also spatial information is required (Dessing, Bullock, Peper, & Beek, 2002; Michaels, Jacobs, & Bongers, 2003; Peper, Bootsma, Mestre, & Bakker, 1994) 5.1 Dynamical systems approach Originally, the dynamical systems approach captured stability-related features of rhythmic movements. In the last decade, the dynamical systems theory has also been applied to describe the information driven change in the initiation and trajectory formation of discrete movements (Schoner, 1990, 1994). Schoner (1994) modelled, for instance, the wing retraction of the diving gannets (Lee & Reddish, 1981; see Section 2) as a dynamical system continuously guided by x(cp). The unfolded and folded wing positions are modelled as two stable point attractors, and the stability of the transition is regulated by a limit cycle. During the dive TTC information is generated and the optic quantity

The Fallacious FallaciousAssumption Assumptionof ofTTC TTCPerception Perceptionininthe theRegulation RegulationofofCatching Catchingand andHitting Hitting 465 465

l/t((p) regulates the (de)stabilization of the two postures. If l/x(cp) is greater than the critical value the folded wing posture destabilizes and simultaneously the unfolded wing posture stabilizes. As a result from the relative change in stability the system follows a limit cycle until it stabilizes onto the unfolded wing posture. In Chapter 17 of this volume Frank Zaal and Reinoud Bootsma describe how the Schoner model can be applied to the coordination of prehension. Zaal et al. (1998) modelled the hand opening and hand closing as point attractors that are connected by a limit cycle. The movement of the hand towards the object generates the optical information that changes the stability of the point attractors. When reaching towards an object, the angle subtended by the hand ((p) and the angle subtended by the gap between hand and object (6) decrease. So, the aforementioned optical variable x(cp,6) might be used to influence the stability of the hand opening and hand closing regimes. In this continuous dynamical information-movement coupling perspective, the initiation of hand closure does not have to occur at a constant value of T((p,0), although the threshold value determines when hand opening becomes unstable and hand closing stable. Note however that they did not examine whether x((p,0) was involved, they simply adopted TTCi as the relevant actor-environment property. 5.2 The required velocity strategy and vector integration to endpoint models Peper et al. (1994) formulated a continuous control model in order to account for their data. In Peper et al.'s experiment, participants had to catch balls that were swung towards them while their hand was restricted to move along the lateral axis. According to a predictive strategy the kinematics of the catch would be unaffected as long as the ball trajectories converge at the same time on the same interception point. In contrast, the kinematics of the catch varied systematically with approach angle, even when future time and place of interception were similar. The authors concluded that hand movements were continuously coupled to a variable that brought the hand to the right place at the right time. This kind of coupling does not demand accurate predictions, "accuracy is achieved during the unfolding of the act" (Peper et al., 1994, p.610). They proposed a strategy entailing a continuous regulation of hand displacement velocity on the basis of information that specifies the velocity required to ensure interception. The currently required velocity is specified by the ratio of the difference in lateral position of the hand and the ball divided by the time before the ball reaches the interception point. So, the model assumes that, besides picking up information that specifies TTCi, participants can also

466


pick up (kinaesthetic and/or optical) variables that specify the position and speed of the hand and the lateral position of the ball. The required velocity (RV) model has been elaborated (Bootsma, Fayt, Zaal, & Laurent, 1997) and empirically supported (Montagne, Fraisse, Ripoll, & Laurent, 2000; Montagne, Laurent, Durey, & Bootsma, 1999). Montagne and co-workers were especially interested in the movement reversals that occurred when the hand started at the interception point. There was a pattern to the reversals with respect to approach angle of the ball, in that the number of left-right reversals was higher for outward ball trajectories and the number of right-left reversals was higher for inward ball trajectories. Recently, however, it was shown that the RV-model and its extension, failed in simulating the behaviour they were designed to explain (Dessing et al., 2002). Dessing et al. (2002) remedied this deficiency by reformulating the RV-model in terms of a neural network based on the VectorIntegration-To-Endpoint (VITE) model developed by Bullock and Grossberg (1988). The VITE model was originally constructed to explain hand reaching towards stationary targets. The VITE model in its simplest form consists of four elements and its dynamic interactions. The target position vector stage represents the perceived location of the target. The present position vector stage represents the actual position of the hand. A difference vector is continuously established between the target position vector and the present position vector. It specifies the distance and direction that the hand has to travel to reach the target. A separate GO-signal gates this difference vector command that actuates the movement. The GO-signal is under voluntary control and if it becomes positive, the present position vector is updated and the movement continuous to unfold. A new present position vector is established in the direction of the target position vector at a rate proportional to the difference vector times the GO-signal. In this form, the VITE model explains trajectory formation during reaching movements to stationary targets in which the velocity of the hand becomes zero as it approaches the target. Intercepting moving targets is fundamentally different from intercepting stationary targets as the trajectory of the hand is not only spatially but also temporally constrained; the hand has to be at a certain target location at a particular time. This implies that time-dependent parameters have to be incorporated in the VITE-model. To reproduce the qualitative effects observed in the lateral interception experiments of Montagne et al. (1999; 2000), Dessing et al. (2002) translated the required velocity-model of Peper et al. in terms of the VITE model. First, the GO-signal was modulated according to the first-order TTC, in that the gain of the GO-signal increases as TTC becomes smaller (the RVITE-model). Second, to improve the control of the hand velocity, they included parallel to the difference vector a relative velocity vector (the RRVITE-model). This final model closes the gap between target position and

The Fallacious TTC Perception FallaciousAssumption AssumptionofofTTC Perceptionininthe theRegulation RegulationofofCatching Catchingand andHitting Hitting 467 467

hand position in the available time span, while avoiding superfluous movements given the ongoing movement of object and hand. All the above-mentioned continuous control models tried to tackle the question how information relates to movement trajectories, by assuming TTCi as an input variable. In the present Chapter it was demonstrated that besides TTCi, other (non-specifying) variables might be exploited to regulate interceptive timing. So, more in line with the approach taken here, an alternative solution to finding better fits between models and data may be to implement other input variables besides TTCi instead of elaborating and adjusting the existing continuous control models. An example of such an approach is the recent work of Michaels, Jacobs, and Bongers (2003). Instead of elaborating the simple VITE model to account for intercepting moving objects (Dessing et al., 2001), Michaels et al. (2003) changed the input variables of the simple VITE model. They implemented the optical variable lateral-velocity-to-expansion ratio to specify continuously changing target positions along the lateral axis. So, in their VITE model, the target position vector changes in accordance with the lateral-velocity-to-expansion ratio. A previous study of Jacobs and Michaels (2003) showed that hand movements in a lateral catching task, were better geared to the lateral-velocity-to-expansion ratio than to the momentary lateral ball position and TTCi. Momentary values of this ratio often do not specify the future passing distance of the ball in a way that permits catchers to make a ballistic movement to the predicted position. This variable can lead to successful interceptive movement only if continuously coupled to movement production with a properly calibrated control law. By implementing this new input variable into the simple VITE model they were able to quantitatively predict the individual hand trajectories accurately.

6. Summary and conclusions The present Chapter reviewed studies on the information-based regulation of interceptive timing. We started with a description of the TTCmodel. This model fitted well with the dominant view that the process of perception precedes the process of motor programming. From this perspective a performer is dependent on the accurate prediction of when an object arrives at a certain point in order to program the effector movement to the future interception point. So, interceptive timing was thought to be based on the perception of TTC, provided by the optical variable x((p). This original model was fatally flawed in many ways. For example, x(cp) was disqualified as being the only relevant source of information for catching and hitting tasks because (1)

468


the interception point does not coincide with the point of observation, (2) binocular vision enhances performance, and (3) the target will not always move with a constant velocity. However, the inexorable believe in the "informationperception-movement" sequence has led researchers to assume that information about TTC is indispensable in the regulation of interceptive actions. The emphasis on the process of perception constrained the search for alternative information sources. Besides x(cp) other sources were proposed that also specified TTC (e.g. x(A) and T(cp,9)). It is now well accepted that actors do not regularly control their actions on the basis of a constant TTC strategy. The finding that timing is dependent on object size and approach velocity eventuated in the conviction that interceptive actions may be based on the pick up of information sources that do not specify TTC. During natural interceptions actors can continuously view the ball up to and including the point of interception. They do not need to perceptually construct the ball's trajectory for prediction. Therefore, every source of information that is in some way confined to the approach of the ball might be used to regulate interceptions. Today, the original TTC-model is rejected and the original formulations are subjected to revision. The second part of this Chapter reviewed alternative models for the information-based regulation of interceptive actions. We no longer presented the perception of TTC as an intermediate phase in interceptive timing. Instead, we proposed that information directly regulates action. Furthermore, we suggested that not a sole variable, but multiple variables might constrain natural interceptive actions. The use of multiple information sources can be actualised by the processes of selection and combination of sources. In the event of selection, an information source is picked up depending on the task at hand, and actors adapt to different performance settings by a flexible change in information-movement couplings. In the event of combination, all sources contribute conjointly, and recalibration may occur when one of the sources is manipulated. Likely, both processes are involved, but more research is needed to understand the nature of multiple sources of information used to constrain natural interceptive actions The question "what information contributes to interceptive timing" cannot be posed in isolation from the question "how information is used to regulate timing". The original TTC model assumed a critical timing strategy, but there is also online regulation. The final Section of the present Chapter discussed different online models. Most modellers did not implement sources of information, but assumed TTCi as an input variable. Demonstrating that perception of TTCt is not an indispensable phase in the regulation of interceptive timing, it seems challenging to include other information sources into these online control strategies, as well.


To account for the direct influence of a multitude of variables on the timing of interceptive actions we introduced the term "directed action". With this denomination we referred to a dynamical systems perspective in which the entire dynamics of the system is given by a synergy in which the multiple information-related dynamics are assembled, depending on organismic, environmental, and task constraints. Taken as a whole the present chapter shows that we have to abandon the view that perception of TTC is an indispensable phase in the regulation of interceptive timing. Instead we have to increase our understanding of the large variety of information sources that support various functional interceptive behaviours. The issue of how information and control strategies differ as a function of task-constraints is highly important.

470


REFERENCES Alderson, G. J. K., Sully, D. J. & Sully, H. G. (1974). An operational analysis of a one-handed catching task using high-speed photography. Journal of Motor Behavior, 6(4), 217-226. Banister, H. & Blackburn, J. M. (1931). An eye factor affecting efficiency at ball games. British Journal of Psychology, 21, 382-384. Bennett, S. J., van der Kamp, J., Savelsbergh, G. J. P. & Davids, K. (1999). Timing a one-handed catch I. Effects of telestereoscopic viewing. Experimental Brain Research, 129(3), 362368. Bennett, S. J., van der Kamp, J., Savelsbergh, G. J. P. & Davids, K. (2000). Discriminating the role of binocular information in the timing of a one-handed catch. The effects of telestereoscopic viewing and ball size. Experimental Brain Research, 135(3), 341-347. Bootsma, R. J., Fayt, V., Zaal, F. T. J. M. & Laurent, M. (1997). On the information-based regulation of movement: What Wann (1996) may want to consider. Journal of Experimental Psychology: Human Perception and Performance, 23(4), 1282-1289. Bootsma, R. J. & Oudejans, R. R. D. (1993). Visual information about time-to-collision between two objects. Journal of Experimental Psychology: Human Perception and Performance, 19(5), 1041-1052. Bootsma, R. J. & Van Wieringen, P. C. W. (1990). Timing an attacking forehand drive in table tennis. Journal of Experimental Psychology: Human Perception and Performance, 16(1), 21-29. Bullock, D. & Grossberg, S. (1988). Neural dynamics of planned arm movements: emergent invariants and speed-accuracy properties during trajectory formation. Psychological Review, 95(1), 49-90. Cavallo, V. & Laurent, M. (1988). Visual information and skill level in time-to-collision estimation. Perception, 17(5), 623-632. Collewijn, H. & Erkelens, C. J. (1990). Binocular eye movements and the perception of depth. Reviews of Oculomotor Research, 4, 213-261. Cutting, J. E. (1986). Perception with an eye for motion. Cambridge, MA: The MIT Press. Cutting, J. E. (1991). Four ways to reject directed perception. Ecological Psychology, 3(1), 25-34. Dessing, J. C, Bullock, D., Peper, C. L. & Beek, P. J. (2002). Prospective control of manual interceptive actions: Comparative simulations of extant and new model constructs. Neural Networks, 15(2), 163-179. Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghthon Mifflin. Gray, R. & Regan, D. (1998). Accuracy of estimating time to collision using binocular and monocular information. Vision Research, 38(4), 499-512. Helmholtz, H. von (1962). Physiological optics. Dover, New York. Heuer, H. (1993). Estimates of time to contact based on changing size and changing target vergence. Perception, 22(5), 549-563.


Jacobs, D. M. (2001). On perceiving, acting and learning: Toward an ecological approach anchored in convergence. Doctoral Dissertation, Vrije Universiteit, Amsterdam. Johnson, J. (2002). One eyed jack. Retrieved October 2 2002, http://oneeyedjack. rodeoannouncer.com/index.htm. Judge, S. J. & Bradford, C. M. (1988). Adaptation to telestereoscopic viewing measured by onehanded ball-catching performance. Perception, 17(6), 783-802. Knowles, W. B. & Carel, W. L. (1958). Estimating time to collision. American Psychologist, 13, 405-406. Lacquaniti, F., Borghese, N. A. & Carrozzo, M. (1992). Internal models of limb geometry in the control of hand compliance. Journal of Neuroscience, 12(5), 1750-1762. Lacquaniti, F., Carrozzo, M. & Borghese, N. A. (1993). Time-varying mechanical behavior of multijointed arm in man. Journal of Neurophysiology, 69(5), 1443-1464. Lacquaniti, F. & Maioli, C. (1989). The role of preparation in tuning anticipatory and reflex responses during catching. Journal of Neuroscience, 9(1), 134-148. Laurent, M , Montagne, G. & Durey, A. (1996). Binocular invariants in interceptive tasks: A directed perception approach. Perception, 25(12), 1437-1450. Lee, D. N. (1976). A theory of visual control of braking based on information about time-tocollision. Perception, 5(4), 437-459. Lee, D. N., Lishman, J. R. & Thomson, J. A. (1982). Regulation of gait in long jumping. Journal of Experimental Psychology: Human Perception and Performance, 8, 448-459. Lee, D. N. & Reddish, D. E. (1981). Plummeting gannets: A paradigm of ecological optics. Nature, 293, 293-294. Lee, D. N. & Young, D. S. (1985). Visual timing of interceptive action. In D. J. Ingle, M. Jeanerod & D. N. Lee (Eds.), Brain mechanisms and spatial vision (pp. 1-30). Dordrecht: Martinus Nijhoff. Lee, D. N., Young, D. S., Reddish, D. E., Lough, S. & Clayton, T. M. (1983). Visual timing in hitting an accelerating ball. Quarterly Journal of Experimental Psychology, 35(a), 333346. Lenoir, M., Musch, E. & La Grange, N. (1999). Ecological relevance of stereopsis in one-handed ball catching. Perceptual and Motor Skills, 89, 495-508. Li, F. X. & Laurent, M. (1994). Effect of practice on intensity coupling and economy of avoidance skill. Journal of Human Movement Studies, 27, 189-200. Li, F. X. & Laurent, M. (1995). Occlusion rate of ball texture as a source of velocity information. Perceptual and Motor Skills, 81(3 Pt 1), 871-880. Mazyn, L., Lenoir, M., Montagne, G. & Savelsbergh, G. J. P. (2001). Do we need binocular depth vision to control the timing of a catch? In: N. Gantchev (Ed.), From basic motor control to functional recovery II: Towards an understanding of the role of motor control from simple systems to human performance. Sofia: Professor Marin Drinov Academic Publishing House.

472


Mclntyre, J., Zago, M., Berthoz, A. & Lacquaniti, F. (2001). Does the brain model Newton's laws? Nature Neuroscience, 4(7), 693-694. Michaels, C. F. (2000). Information, perception, and action: What should ecological psychologists learn from Milner and Goodale (1995)? Ecological Psychology, 12(3), 241-258. Michaels, C. F. & Beek, P. J. (1995). The state of ecological psychology. Ecological Psychology, 7(4), 259-278. Michaels, C. F. & Carello, C. (1981). Direct perception. Englewood Cliffs, NJ: Prentice-Hall. Michaels, C. F., Jacobs, D. M. & Bongers, R. M. (2003). Predicting lateral interceptive hand movements. (Submitted). Michaels, C. F., Zeinstra, E. B. & Oudejans, R. R. D. (2001). Information and action in punching a falling ball. Quarterly Journal of Experimental Psychology, 54A(1), 69-93. Milner, A. D. & Goodale, M. A. (1995). The visual brain in action. Oxford, UK: Oxford University Press. Montagne, G., Fraisse, F., Ripoll, H. & Laurent, M. (2000). Perception-action coupling in an interceptive task: First-order time-to-contact as an input variable. Human Movement Sciences, 19, 59-72. Montagne, G., Laurent, M., Durey, A. & Bootsma, R. J. (1999). Movement reversals in ball catching. Experimental Brain Research, 129(1), 87-92. Peper, C. L., Bootsma, R. J., Mestre, D. R. & Bakker, F. C. (1994). Catching balls: how to get the hand to the right place at the right time. Journal of Experimental Psychology: Human Perception and Performance, 20(3), 591-612. Purdy, W. C. (1958). The hypothesis of psychophysical correspondence in space perception. (Doctoral dissertation. Cornell University). Regan, D. (1997). Visual factors in hitting and catching. Journal of Sport Sciences, 15(6), 533558. Regan, D. & Beverley, K. I. (1979). Binocular and monocular stimuli for motion in depth: changing-disparity and changing-size feed the same motion-in-depth stage. Vision Research, 19(12), 1331-1342. Regan, D. & Hamstra, S. J. (1993). Dissociation of discrimination thresholds for time to contact and for rate of angular expansion. Vision Research, 33(4), 447-462. Regan, D. & Vincent, A. (1995). Visual processing of looming and time to contact throughout the visual field. Vision Research, 35(13), 1845-1857. Rind, F. C. & Simmons, P. J. (1999). Seeing what is coming: building collision-sensitive neurones. Trends in Neurosciences, 22(5), 215-220. Runeson, S., Jacobs, D. M., Andersson, I. E. K. & Kreegipuu, K. (2001). Specificity is always contingent on constraints: Global versus individual arrays is not the issue. Behavioral and Brain Sciences, 24(2), 240-241. Rushton, S. K. & Wann, J. P. (1999). Weighted combination of size and disparity: a computational model for timing a ball catch. Nature Neuroscience, 2(2), 186-190.

The Fallacious TTC Perception FallaciousAssumption AssumptionofofTTC Perceptionininthe theRegulation RegulationofofCatching Catchingand andHitting Hitting 473 473

Savelsbergh, G. J. P., Whiting, H. T. A. & Bootsma, R. J. (1991). Grasping tau. Journal of Experimental Psychology: Human Perception and Performance, 17(2), 315-322. Savelsbergh, G. J. P., Whiting, H. T. A., Burden, A. M. & Bartlett, R. M. (1992). The role of predictive visual temporal information in the coordination of muscle activity in catching. Experimental Brain Research, 89(1), 223-228. Savelsbergh, G. J. P., Whiting, H. T. A., Pijpers, J. R. & Van Santvoord, A. M. M. (1993). The visual guidance of catching. Experimental Brain Research, 93, 146-156. Savelsbergh, G. J. P. (1995). Catching "Grasping tau" comments on J. R. Tresilian (1994). Human Movement Science, 14, 125-127. Savelsbergh, G. J. P. & van der Kamp, J. (2000). Information in learning to co-ordinate and control movements: Is there a need for specificity of practice? International Journal of Sport Psychology, 31,1-18. Schoner, G. (1990). A Dynamic Theory of Coordination of Discrete Movement. Biological Cybernetics, 63(4), 257-270. Schoner, G. (1994). Dynamic theory of action-perception patterns - the time-before-contact paradigm. Human Movement Sciences, 13(3-4), 415-439. Shankar, S. & Ellard, C. (2000). Visually guided locomotion and computation of time-to-collision in the mongolian gerbil (Meriones unguiculatus): the effects of frontal and visual cortical lesions. Behavioral Brain Research, 108(1), 21-37. Smeets, J. B. J. & Brenner, E. (1994). The difference between the perception of absolute and relative motion: A reaction time study. Vision Research, 34(2), 191-195. Smith, M. R. H., Flach, J. M., Dittman, S. M. & Stanard, T. (2001). Monocular optical constraints on collision control. Journal of Experimental Psychology: Human Perception and Performance, 27(2), 395-410. Stretch, R. A., Bartlett, R. & Davids, K. (2000). A review of batting in men's cricket. Journal of Sport Sciences, 18(12), 931-949. Sun, H. J. & Frost, B. J. (1998). Computation of different optical variables of looming objects in pigeon nucleus rotundus neurons. Nature Neuroscience, 1(4), 296-303. Todd, J. T. (1981). Visual information about moving objects. Journal of Experimental Psychology: Human Perception and Performance, 7(4), 795-810. Tresilian, J. R. (1990). Perceptual information for the timing of interceptive action. Perception, 19(2), 223-239. Tresilian, J. R. (1993). Four questions of time to contact: a critical examination of research on interceptive timing. Perception, 22(6), 653-680. Tresilian, J. R. (1994a). Approximate information sources and perceptual variables in interceptive timing. Journal of Experimental Psychology: Human Perception and Performance, 20(1), 154-173. Tresilian, J. R. (1994b). Perceptual and motor processes in interceptive timing. Human Movement Sciences, 13, 335-373.

474


Tresilian, J. R. (1999a). Analysis of recent empirical challenges to an account of interceptive timing. Perception and Psychophysics, 61(3), 515-528. Tresilian, J. R. (1999b). Visually timed action: time-out for 'tau'? Trends in Cognitive Sciences, 3(8), 301-310. Tyldesley, D. A., & Whiting, H. T. A. (1975). Operational Timing. Journal of Human Movement Studies, 1, 172-177. Van der Kamp, J. (1999). The information-based regulation of interceptive action. (Doctoral dissertation, Vrije Universiteit, Amsterdam). Van der Kamp, J., Bennett, S. J., Savelsbergh, G. J. P. & Davids, K. (1999). Timing a one-handed catch II. Adaptation to telestereoscopic viewing. Experimental Brain Research, 129(3), 369-377. Van der Kamp, J., Savelsbergh, G. J. P. & Smeets, J. B. (1997). Multiple information sources in interceptive timing. Human Movement Science, 16(6), 787-821. Von Hofsten, C. (1983). Catching skills in infancy. Journal of Experimental Psychology: Human Perception and Performance, 9(1), 75-85. Wang, Y. & Frost, B. J. (1992). Time to collision is signalled by neurons in the nucleus rotundus of pigeons. Nature, 356(6366), 236-238. Wann, J. P. (1996). Anticipating arrival: Is the tau margin a specious theory? Journal of Experimental Psychology: Human Perception and Performance, 22(4), 1031-1048. Wann, J. P. & Rushton, S. (1995). Grasping the impossible: Stereoscopic virtual balls. In R. G. Bardy, R. J. Bootsma & Y. Guiard (Eds.), Studies in perception and action III (pp. 207210). Hilsdale, NJ: Lawrence Erlbaum Associates. Whiting, H. T. A. & Sharp, R. H. (1973). Visual occlusion factors in in a discrete ball catching task. Journal of Motor Behavior, 6, 11-16. Zaal, F. T. J. M., Bootsma, R. J. & van Wieringen, P. C. W. (1998). Coordination in prehension. Information-based coupling of reaching and grasping. Experimental Brain Research, 119(4), 427-435.

Time-to-Contact Time-to-Contact – - H. H. Hecht and G.J.P. Savelsbergh (Editors) (Editors) © © 2004 2004 Elsevier Elsevier B.V. B.V. All All rights rights reserved reserved

CHAPTER 20 How Time-to-Contact is Involved in the Regulation of Goal-Directed Locomotion Gilles Montagne Universite de la Mediterranee, Marseille, France

Aymar De Rugy The Pennsylvania State University, University Park, USA

Martinus Buekers Katolieke Universiteit Leuven, Leuven, Belgium

Alain Durey (U1) Universite de la M6diterrane'e, Marseille, France

Gentaro Taga University of Tokyo & PRESTO, JST, Japan

Michel Laurent University de la Mediterranee, Marseille, France

ABSTRACT This chapter is designed to show how time-to-contact (TTC) can be involved in the regulation of goal-directed locomotion. Two integrative models relying (at least partly) on the use of TTC are presented. The first one links a perceptual variable to a movement variable and allows an agent to get to the right place at the right time to catch a fly ball. The second one links a perceptual variable to the dynamics of the neuro-musculo-skelettal system and allows an agent to put a foot on a target placed on the floor. The status of TTC in these models as well as the respective contributions of these models to a better understanding of the mechanisms underlying goal-directed actions are discussed.

476

G. G. Montagne, A. De Rugy, M. Buekers, A. Durey, G. Taga and M. Laurent

1. Introduction Most of the actions we perform daily strive towards a goal. Generally this goal achievement relies on a close dialogue between the perceptual and the motor components of the action. Goal-directed locomotion is one of the paradigms frequently used by researchers interested in discovering the mechanisms underlying this dialogue. This last statement is corroborated by the increasing publication rate on this topic and, among others, by a recent special issue of Ecological Psychology, dedicated to «Visually Controlled Locomotion and Orientation» (vol. 10, 1998). It is worth mentioning that goal-directed locomotion groups numerous tasks characterized by various spatio-temporal constraints. We can easily differentiate heading tasks relying on the perception of self-motion direction (e.g., Warren & Hannon, 1988) from locomotion pointing tasks relying on the perception of the spatio-temporal vicinity of a target (e.g., Montagne et al., 2000a). This chapter is primarily concerned with the latter aspect, that is with tasks involving precise positioning of the foot or the body at the right place and the right time. These tasks include, for example, positioning the foot on a visible target on the floor while walking (De Rugy et al., 2001) or positioning the body at the right pace and the right time to intercept a fly ball (McLeod & Dienes, 1993). The regulation of goal-directed locomotion relies on the use of control mechanisms linking perceptual input to motor output. It is obvious that the kind of mechanisms described (and consequently the modeling of these mechanisms) depends highly on the metatheoretical background embraced. Warren (1998) differentiates control mechanisms relying on the construction and the use of internal representations of the world {model-based control) (Haruno et al., 2001) from those involving the use of information available in the perceptual flow {information-based control) (Warren et al., 2001). While the former model rests on a cognitive approach of perception and action (Marr, 1982), the latter rests on the ecological approach of perception and action (Gibson, 1979). Our work comes within the framework of the ecological approach of perception and action (Gibson, 1979). Gibson (e.g., 1958) emphasized the potential richness of the optic flow available to an agent engaged in a goaldirected locomotion task. He showed that each displacement gives rise to a specific optic flow. This specificity translates into optical invariants characterizing the displacement produced in relation to the environment. While underscoring this kinesthetic function of vision, Gibson opened the door to very parsimonious control mechanisms. Since the original proposal of Gibson, numerous theoretical and empirical studies have shown that (i) task-specific control information (Warren, 1998) exists (e.g., Lee, 1976) and that (ii) this information substrate can be used directly to control the action (e.g., Savelsbergh et al., 1991; De Rugy et al.,

How Time-to-Contact is Involved in the Regulation of Goal-Directed Locomotion

477 All

2001). The theoretical work implemented was designed to identify which information specifies a given property of the agent-environment system (Lee, 1976; Bootsma & Peper, 1992; Laurent et al., 1996). This specification process allows the agent to be informed about the validity of the movement/displacement produced in relation to the task at hand. The second kind of studies was designed to investigate whether the information described is involved in the regulation of action. Some nice experimental manipulations of the optic flow revealed that information can actually take part in the control of goal-directed action (e.g., Savelsbergh et al., 1991). Nevertheless, the results are controversial because it is not clear how the information participates in the regulation of movement/displacement (Bootsma et al., 1997; Montagne & Laurent, 2002). For example, numerous studies (e.g., McLeod & Dienes, 1993) have shown vertical optical information to be a good candidate for displacement control when one catches a fly ball. Unfortunately no model indicates how this information could be used in the control process. While there is no doubt that canceling optical acceleration guarantees success, nothing is sure about how the reference value (optical acceleration equals to zero) is reached and maintained. The present controversies will be overcome only when a testable conception of the way information is used to regulate behavior is developed (see Montagne et al., 1998 for a review). This chapter is designed to present two models showing how TTC can take part in the regulation of locomotion pointing tasks. Note that TTC is a property of the agent-environment system and that several sources of information specifying TTC are available in the optic flow. Consequently the two models presented can be implemented indifferently on the basis of the physical properties or of their optical counterparts.

2. The modeling 2.1 How to catch a fly ball Consider the situation illustrated in Figure 1. An agent runs to catch a fly ball that has been hit in a field defined by the external coordinate system R. The agent's movement relative to the ball can be most directly controlled within another coordinate system, R', which is anchored on the projection onto the horizontal plane (going through the head) of the ball's current position (Figure 1 and Figure 2) (von Hofsten, 1983; Zaal et al., 1999). Within this coordinate system, axis X' is defined by the origin of R' and the agent's location.

478 478

G. Montagne, Montague, A. De Rugy, M. Buekers, A. Durey, G. Taga and M. Laurent

Figure 1: Representation of the displacements of a fly ball and an outfielder. The displacements can be described in reference to a coordinate system (R') anchored to the moving ball or to an external coordinate system (R) anchored to a static background. The angle of gaze elevation is also represented ( 3 at time 13).

In the coordinate system R', the velocity of the agent (V(R)) can be defined (as each vector) by a direction and an amplitude. The "directional problem" is solved as soon as the velocity vector is brought into alignment with axis X'1 (t7 in Figure 2). That amounts to keeping the direction of axis X' invariant during the displacement. In fact, this is the case when, in the external coordinate system R, the agent adopts the same horizontal velocity as that of the ball (Vba]i(R)) (Figure 3). The agent is then left with the "amplitude problem"; he or she has to determine the amount of acceleration along the axis X' needed to succeed in the task. In this chapter we will assume that the agent is able to keep direction of axis X' invariant (e.g., Lenoir et al., 2002; Chardenon et al, 2002, under revision) and we will specifically examine the control mechanism underlying the solving of the "amplitude problem".

1

This leads to an apparent inconsistency that will be removed further on. The velocity of the agent in R corresponds to the sum of his/her velocity along X' and the horizontal velocity of the fly ball (Equation 4). Consequently a displacement directed towards the ball in R' (Figure 2) can correspond, depending on the amplitude of V(R.), to a displacement directed towards the interception point in R (Figure 3).

How Time-to-Contact is Involved in the Regulation of Goal-Directed Locomotion

479

tio

R'

Figure 2: Overhead view of the displacements of both the agent and the ball from initiation of the agent displacement (to) to ball interception (t10). In the coordinate system R\ the velocity of the agent (V (R)) can be defined (as each vector) by a direction and an amplitude. The "directional problem" is solved as soon as the velocity vector is brought into alignment with axis X' (t7 in this figure). The agent is then left with the "amplitude problem"; he or she has to determine the amount of acceleration along the axis X' needed to succeed in the task.

Our formalization is inspired by a model proposed by Peper and Bootsma (Peper et al., 1994; Bootsma et al., 1997; Bootsma, 1998) and tested empirically by Montagne and collaborators (Montagne et al., 1999, 2000b). According to this model, the control of a given action entails the regulation of the produced acceleration on the basis of a perceived difference between current and required behaviors. The same logic can apply to the task under consideration (Equation 1). The acceleration to be produced along the X' axis (A(R)) corresponds to the difference between the velocity required to succeed in the task (Vreq(R)) and the current velocity of the agent (V(R)) (Equation 1): ~

&^

req(R')

where a and ft are constants.

(1)

480

G. G. Montagne, A. De De Rugy, M. M. Buekers, A. Durey, G. G. Taga and M. Laurent

To arrive at the right place at the right time, the agent's required velocity along the X' axis (Vreq(R')) corresponds to the ratio between the current distance between the agent and the projection of the ball to the horizontal plane going through the eye and the time remaining before the ball reaches this plane (Equation 2). This required velocity (Vreq(R)) is optically specified by an information present in the optic flow produced by the combined displacements of the agent and the ball (Equation 2). D V

re?f/

=

°

=

TTC

8 2d{tan(0))/dt

m K}

Where Vreq(R) is the velocity required along the X' axis, D is the current distance between the agent and the projection of the ball to the plane going through the eye, TTC is the time remaining before the ball reaches the plane going through the eye, g is a gravitational constant, and $ is the angle of gaze elevation (Figure 1). On the other hand, the current velocity of the agent in the coordinate system R' (V(R>)) corresponds to the rate of change of the distance between the agent and the projection of the ball to the plane going through the eye. Once again the current velocity (V(R>)) is optically specified (Equation 3). The agent just has to know the ball size (r) in advance.

V

• \

Time-to-Contact (Advances in Psychology, Volume 135)

Advances in Psychology Volume 40 Current Issues in Theoretical Psychology

Advances in Psychology Volume 20 Trends in Mathematical Psychology

Advances in Experimental Social Psychology, Volume 23

Advances in Experimental Social Psychology, Volume 4

Advances in Experimental Social Psychology, Volume 1

Advances in Experimental Social Psychology, Volume 18

Advances in Psychology Volume 8 Discourse Processing

Advances in Experimental Social Psychology, Volume 34

Advances in Experimental Social Psychology, Volume 25

Advances in Psychology Research Volume 44

Advances in Psychology Research Volume 44

Advances in Psychology Research volume 80

Advances in Psychology Research. Volume 72

Advances in Psychology Research Volume 57

Advances in Experimental Social Psychology, Volume 20

Advances in Experimental Social Psychology, Volume 31

Advances in Experimental Social Psychology, Volume 6

Advances in Psychology Research Volume 57

Advances in Experimental Social Psychology, Volume 3

Advances in Experimental Social Psychology, Volume 24

Advances in Experimental Social Psychology, Volume 26

Advances in Experimental Social Psychology, Volume 27

Advances in Psychology Volume 26 Psychological Reality

Advances in Personality Psychology: Volume II

Advances in Experimental Social Psychology, Volume 22

Advances in Experimental Social Psychology, Volume 34

Advances in Experimental Social Psychology, Volume 42

Trends in Mathematical Psychology: Advances in Psychology

Advances in Imaging and Electron Physics, Volume 135 (Advances in Imaging and Electron Physics)

Advances in Experimental Social Psychology, Volume 39 (Advances in Experimental Social Psychology)

Time-to-Contact (Advances in Psychology, Volume 135)

Advances in Psychology Volume 40 Current Issues in Theoretical Psychology

Advances in Psychology Volume 20 Trends in Mathematical Psychology

Advances in Experimental Social Psychology, Volume 23

Advances in Experimental Social Psychology, Volume 4

Advances in Experimental Social Psychology, Volume 1

Advances in Experimental Social Psychology, Volume 18

Advances in Psychology Volume 8 Discourse Processing

Advances in Experimental Social Psychology, Volume 34

Advances in Experimental Social Psychology, Volume 25

Advances in Psychology Research Volume 44

Advances in Psychology Research Volume 44

Advances in Psychology Research volume 80

Advances in Psychology Research. Volume 72

Advances in Psychology Research Volume 57

Advances in Experimental Social Psychology, Volume 20

Advances in Experimental Social Psychology, Volume 31

Advances in Experimental Social Psychology, Volume 6

Advances in Psychology Research Volume 57

Advances in Experimental Social Psychology, Volume 3

Advances in Experimental Social Psychology, Volume 24

Advances in Experimental Social Psychology, Volume 26

Advances in Experimental Social Psychology, Volume 27

Advances in Psychology Volume 26 Psychological Reality

Advances in Personality Psychology: Volume II

Advances in Experimental Social Psychology, Volume 22

Advances in Experimental Social Psychology, Volume 34

Advances in Experimental Social Psychology, Volume 42

Trends in Mathematical Psychology: Advances in Psychology

Advances in Imaging and Electron Physics, Volume 135 (Advances in Imaging and Electron Physics)

Advances in Experimental Social Psychology, Volume 39 (Advances in Experimental Social Psychology)

Recommend Documents