Flying Insects and Robots

Flying Insects and Robots

Dario Floreano · Jean-Christophe Zufferey · Mandyam V. Srinivasan · Charlie Ellington

Editors

Flying Insects and Robots

123

Editors Prof. Dario Floreano Director, Laboratory of Intelligent Systems EPFL-STI-IMT-LIS École Polytechnique Fédérale de Lausanne ELE 138 1015 Lausanne Switzerland [email protected]

Dr. Jean-Christophe Zufferey Laboratory of Intelligent Systems EPFL-STI-IMT-LIS École Polytechnique Fédérale de Lausanne ELE 115 1015 Lausanne Switzerland [email protected]

Prof. Mandyam V. Srinivasan Visual and Sensory Neuroscience Queensland Brain Institute University of Queensland QBI Building (79) St. Lucia, QLD 4072 Australia [email protected]

Prof. Charlie Ellington Animal Flight Group Dept. of Zoology University of Cambridge Downing Street Cambridge, CB2 3EJ UK [email protected]

ISBN 978-3-540-89392-9 e-ISBN 978-3-540-89393-6 DOI 10.1007/978-3-540-89393-6 Springer Heidelberg Dordrecht London New York Library of Congress Control Number: 2009926857 ACM Computing Classification (1998): I.2.9, I.2.10, I.2.11 c Springer-Verlag Berlin Heidelberg 2009  This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Cover illustration: Flying insect image reproduced with permission of Mondolithic Studios Inc. Cover design: KünkelLopka, Heidelberg Printed on acid-free paper Springer is a part of Springer Science+Business Media (www.springer.com)

Preface

Flying insects represent a fascinating example of evolutionary design at the microscopic scale. Their diminutive size does not prevent them from perceiving the world, flying, walking, jumping, chasing, escaping, living in societies, and even finding their way home at the end of a long day. Their size and energy constraints demand extremely efficient and specialized solutions, which are often very different from those that we are accustomed to seeing in larger animals. For example, the visual system of flying insects, which features a compound eye comprising thousands of ommatidia – “little eyes” – represents a dramatic alternative to the design of our own eyes, which we share with all vertebrates and which has driven the design of today’s cameras. Do insect eyes differ from human eyes only superficially with respect to the optical and imaging characteristics, or do the nervous systems of their owners process the information that they receive in different ways? Several aspects of this question are explored in this book. The nervous system of flying insects not only coordinates the perception and motion of the animal at extremely high speed and in very dynamic conditions, but it actively monitors features in the surrounding environment, supports accurate landings in very tiny spots, handles recovery from high turbulence and collisions, directs the exploration of the environment in search of food, shelter or partners, and even enables the animal to remember how to return to its nest. Flying insects move their wings by using the whole thorax to produce fast, resonating, respiration-like contractions that result in the movement of the wing appendages, whose morphology and constituent materials then modify the basic, passive flapping motion through the air. As such, these creatures represent a fascinating source of inspiration for engineers aiming to create increasingly smaller and autonomous robots that can take to the air like a duck to water, and go where no machine has gone before. At the same time, robotic insects can serve as embodied models for testing scientific hypotheses that would be impossible to study in numerical simulations because of the difficulty in creating realistic visual environments, capturing the physics of fluid dynamics in very turbulent and low-speed regimes, reproducing the elastic properties of the active and passive materials that make up an insect body, and accurately modelling the perception-action loops that drive the behavior of the system. Despite much recent progress, both the functioning of flying insects and the design of micro flying robots are not yet fully understood, which makes this transdisciplinary area of research extremely fascinating and fertile with discoveries. This book brings together for the first time highly selected and carefully edited contributions from a community of biologists and engineers who share the same passion for v

vi

Preface

understanding the design principles of flying insects and robots. The book is the offspring of a stimulating meeting with the same title and organizers that was held in the summer of 2007 at Monte Verità in Switzerland. After the meeting, we decided to assemble a carefully edited volume that would serve both as a tutorial introduction to the field and as a reference for future research. In the months that followed, we solicited some of the participants, as well as additional authors whose research was complementary and would fit the book plan, to write chapters for a larger audience. The authors and the editors spent most of 2008 writing, revising, and cross-linking the chapters in order to produce a homogeneous, accessible, and yet up-to-date book. Approximately half of the book is written from a biological perspective and the other half from an engineering perspective, but in all cases the authors have attempted to use plain terminology that is accessible to both sides and they have made several links and suggestions that cut across the traditional divide between biology and engineering. The book starts with a description of today’s advanced methods used to study flying insects. After this, the reader is taken through a description of the perceptual, neuronal, and behavioral properties of flying insects and of their implications for the design of sensors and control strategies necessary to achieve autonomous navigation of miniature flying vehicles. Once this ground has been covered, the reader is gradually introduced to the principles of aerodynamics and control suitable for microsystems with flapping wings and to several examples of robots with fixed and flapping wings that are inspired by the principles of flying insects. We encourage readers to photocopy the figures of Chapter 15, cut out the drawings, and assemble a moving model of the thorax, which should provide an intuitive understanding of the typical workout routine of a flying insect. Two chapters venture into the area of robots that live and transit between the ground and the air. Although these chapters are more speculative from a biological perspective, they highlight the fact that flying insects are also terrestrial animals and that robots capable of a transition between terrestrial locomotion and flight can have several advantages. Finally, the book closes with two engineering chapters: one dedicated to energy supply and the possible use of solar cells to power micro aerial vehicles, and another to the technology available today and in the near future for realizing autonomous, flying micro robots. We sincerely hope that you will enjoy and learn from this book as much as we did throughout the entire creation and editing of this project. We would like to express our deep gratitude to the contributors of all the chapters, who enthusiastically presented their knowledge and achievements in such a small space and time, and who displayed amazing patience and dedication in revising their own material and that of their colleagues. Last, but not least, we would like to thank Ronan Nugent at Springer for welcoming this project, following it through all its production stages while accommodating many of our requests, and making sure that it is presented in a form that best fits its content both on the Internet and in the printed edition. Lausanne Brisbane Cambridge 12 June 2009

Dario Floreano Jean-Christophe Zufferey Mandyam V. Srinivasan Charlie Ellington

Contents

1 Experimental Approaches Toward a Functional Understanding of Insect Flight Control . . . . . . . . . . . . . . . . . Steven N. Fry

1

2 From Visual Guidance in Flying Insects to Autonomous Aerial Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mandyam V. Srinivasan, Saul Thurrowgood, and Dean Soccol

15

3 Optic Flow Based Autopilots: Speed Control and Obstacle Avoidance Nicolas Franceschini, Franck Ruffier and Julien Serres 4 Active Vision in Blowflies: Strategies and Mechanisms of Spatial Orientation . . . . . . . . . . . . . . . . . . . . . . . . . . Martin Egelhaaf, Roland Kern, Jens P. Lindemann, Elke Braun, and Bart Geurten

29

51

5 Wide-Field Integration Methods for Visuomotor Control . . . . . . . J. Sean Humbert, Joseph K. Conroy, Craig W. Neely, and Geoffrey Barrows

63

6 Optic Flow to Steer and Avoid Collisions in 3D . . . . . . . . . . . . Jean-Christophe Zufferey, Antoine Beyeler, and Dario Floreano

73

7 Visual Homing in Insects and Robots . . . . . . . . . . . . . . . . . . Jochen Zeil, Norbert Boeddeker, and Wolfgang Stürzl

87

8 Motion Detection Chips for Robotic Platforms . . . . . . . . . . . . . Rico Moeckel and Shih-Chii Liu

101

9 Insect-Inspired Odometry by Optic Flow Recorded with Optical Mouse Chips . . . . . . . . . . . . . . . . . . . . . . . . Hansjürgen Dahmen, Alain Millers, and Hanspeter A. Mallot 10 Microoptical Artificial Compound Eyes . . . . . . . . . . . . . . . . Andreas Brückner, Jacques Duparré, Frank Wippermann, Peter Dannberg, and Andreas Bräuer 11 Flexible Wings and Fluid–Structure Interactions for Micro-Air Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . W. Shyy, Y. Lian, S.K. Chimakurthi, J. Tang, C.E.S. Cesnik, B. Stanford, and P.G. Ifju

115 127

143

vii

viii

12

Contents

Flow Control Using Flapping Wings for an Efficient Low-Speed Micro-Air Vehicle . . . . . . . . . . . . . . . . . . . . . . Kevin D. Jones and Max F. Platzer

159

13

A Passively Stable Hovering Flapping Micro-Air Vehicle . . . . . . . Floris van Breugel, Zhi Ern Teoh, and Hod Lipson

14

The Scalable Design of Flapping Micro-Air Vehicles Inspired by Insect Flight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . David Lentink, Stefan R. Jongerius, and Nancy L. Bradshaw

185

Springy Shells, Pliant Plates and Minimal Motors: Abstracting the Insect Thorax to Drive a Micro-Air Vehicle . . . . . Robin J. Wootton

207

15

171

16

Challenges for 100 Milligram Flapping Flight . . . . . . . . . . . . . Ronald S. Fearing and Robert J. Wood

219

17

The Limits of Turning Control in Flying Insects . . . . . . . . . . . . Fritz-Olaf Lehmann

231

18

A Miniature Vehicle with Extended Aerial and Terrestrial Mobility . Richard J. Bachmann, Ravi Vaidyanathan, Frank J. Boria, James Pluta, Josh Kiihne, Brian K. Taylor, Robert H. Bledsoe, Peter G. Ifju, and Roger D. Quinn

247

19

Towards a Self-Deploying and Gliding Robot . . . . . . . . . . . . . Mirko Kovaˇc, Jean-Christophe Zufferey, and Dario Floreano

271

20

Solar-Powered Micro-air Vehicles and Challenges in Downscaling . . André Noth and Roland Siegwart

285

21

Technology and Fabrication of Ultralight Micro-Aerial Vehicles . . . Adam Klaptocz and Jean-Daniel Nicoud

299

Contributors

Richard J. Bachmann BioRobots, LLC, Cleveland, USA Geoffrey Barrows Centeye, Washington, DC, USA, [email protected] Antoine Beyeler Laboratory of Intelligent Systems, EPFL, Lausanne, Switzerland Robert H. Bledsoe United States Marine Corps and the Naval Postgraduate School, USA Norbert Boeddeker Department of Neurobiology and Center of Excellence ‘Cognitive Interaction Technology’, Bielefeld University, Bielefeld, Germany, [email protected] Frank J. Boria Department of Mechanical Engineering at the University of Florida, USA Nancy L. Bradshaw Faculty of Aerospace Engineering, Delft University of Technology, 2629 HS Delft, The Netherlands; Experimental Zoology Group, Wageningen University, 6709 PG Wageningen, The Netherlands Andreas Bräuer Fraunhofer Institute for Applied Optics and Precision Engineering, Albert-Einstein-Str. 7, D-07745, Jena, Germany Elke Braun Department of Neurobiology & Center of Excellence “Cognitive Interaction Technology”, Bielefeld University, D-33501 Bielefeld, Germany Floris van Breugel Cornell Computational Synthesis Lab, Cornell University, Ithaca, New York, USA, [email protected] Andreas Brückner Fraunhofer Institute for Applied Optics and Precision Engineering, Albert-Einstein-Str. 7, D-07745, Jena, Germany, [email protected] C.E.S. Cesnik Department of Aerospace Engineering, University of Michigan, Ann Arbor, Michigan, USA, [email protected] S.K. Chimakurthi Department of Aerospace Engineering, University of Michigan, Ann Arbor, Michigan, USA, [email protected] Joseph K. Conroy Autonomous Vehicle Laboratory, University of Maryland, College Park, MD, USA, [email protected] Hansjürgen Dahmen Cognitive Neurosciences, University of Tübingen, Germany, [email protected] ix

x

Peter Dannberg Fraunhofer Institute for Applied Optics and Precision Engineering, Albert-Einstein-Str. 7, D-07745, Jena, Germany Jacques Duparré Fraunhofer Institute for Applied Optics and Precision Engineering, Albert-Einstein-Str. 7, D-07745, Jena, Germany Martin Egelhaaf Department of Neurobiology & Center of Excellence “Cognitive Interaction Technology”, Bielefeld University, D-33501 Bielefeld, Germany, [email protected] Ronald S. Fearing Biomimetic Millisystems Lab, University of California, Berkeley, CA, USA, [email protected] Dario Floreano Laboratory of Intelligent Systems, EPFL, Lausanne, Switzerland, [email protected] Nicolas Franceschini Biorobotics Lab, Institute of Movement Science, CNRS & Univ of the Mediterranean, Marseille, France, [email protected] Steven N. Fry Institute of Neuroinformatics (INI) & Institute of Robotics and Intelligent Systems (IRIS) – Swiss Federal School of Technology Zürich, Switzerland Bart Geurten Department of Neurobiology & Center of Excellence “Cognitive Interaction Technology”, Bielefeld University, D-33501 Bielefeld, Germany J. Sean Humbert Autonomous Vehicle Laboratory, University of Maryland, College Park, MD, USA, [email protected] Peter G. Ifju Department of Mechanical and Aerospace Engineering, University of Florida, Gainsville, FL, USA, [email protected] Kevin D. Jones Naval Postgraduate School, Monterey, CA, USA, [email protected] Stefan R. Jongerius Faculty of Aerospace Engineering, Delft University of Technology, 2629 HS Delft, The Netherlands Roland Kern Department of Neurobiology & Center of Excellence “Cognitive Interaction Technology”, Bielefeld University, D-33501 Bielefeld, Germany Josh Kiihne United States Marine Corps and the Naval Postgraduate School, USA Adam Klaptocz Lab of Intelligent Systems, EPFL, Lausanne, Switzerland, [email protected] Mirko Kovac Laboratory of Intelligent Systems, EPFL, Lausanne, Switzerland, [email protected] Fritz-Olaf Lehmann Institute of Neurobiology, University of Ulm, Albert-Einstein-Allee 11, 89081 Ulm, [email protected] David Lentink Experimental Zoology Group, Wageningen University, 6709 PG Wageningen, The Netherlands; Faculty of Aerospace Engineering, Delft University of Technology, 2629 HS Delft, The Netherlands, [email protected] Y. Lian Department of Aerospace Engineering, University of Michigan, Ann Arbor, Michigan, USA, [email protected]

Contributors

Contributors

xi

Jens P. Lindemann Department of Neurobiology & Center of Excellence “Cognitive Interaction Technology”, Bielefeld University, D-33501 Bielefeld, Germany Hod Lipson Cornell Computational Synthesis Lab, Cornell University, Ithaca, New York, USA, [email protected] Shih-Chii Liu Institute of Neuroinformatics, University of Zürich and ETH Zürich, Zürich, Switzerland, [email protected] Hanspeter A. Mallot Cognitive Neurosciences, University of Tübingen, Germany, [email protected] Alain Millers Cognitive Neurosciences, University of Tübingen, Germany, [email protected] Rico Moeckel Institute of Neuroinformatics, University of Zürich and ETH Zürich, Zürich, Switzerland, [email protected] Craig W. Neely Centeye, Washington, DC, USA, [email protected] Jean-Daniel Nicoud Didel SA, Belmont, Switzerland, [email protected] André Noth Autonomous Systems Lab, ETHZ, Zürich, Switzerland, [email protected], [email protected] Max F. Platzer Naval Postgraduate School, Monterey, CA, USA, [email protected] James Pluta United States Marine Corps and the Naval Postgraduate School, USA Roger D. Quinn Department of Mechanical and Aerospace Engineering at Case Western Reserve University, Cleveland, USA, [email protected] Franck Ruffier Biorobotics Lab, Institute of Movement Science, CNRS & Univ of the Mediterranean, Marseille, France, [email protected] Julien Serres Biorobotics Lab, Institute of Movement Science, CNRS & Univ of the Mediterranean, Marseille, France, [email protected] W. Shyy Department of Aerospace Engineering, University of Michigan, Ann Arbor, Michigan, USA, [email protected] Roland Siegwart Autonomous Systems Lab, ETHZ, Zürich, Switzerland, [email protected] Dean Soccol ARC Centre of Excellence in Vision Science, Queensland Brain Institute, University of Queensland, St. Lucia, QLD 4072, Australia, [email protected] Mandyam V. Srinivasan ARC Centre of Excellence in Vision Science, Queensland Brain Institute, University of Queensland, St. Lucia, QLD 4072, Australia, [email protected] B. Stanford Department of Mechanical and Aerospace Engineering, University of Florida, Gainsville, FL, USA, [email protected]

xii

Wolfgang Stürzl Department of Neurobiology and Center of Excellence ‘Cognitive Interaction Technology’, Bielefeld University, Bielefeld, Germany, [email protected] J. Tang Department of Aerospace Engineering, University of Michigan, Ann Arbor, Michigan, USA, [email protected] Brian K. Taylor Department of Mechanical and Aerospace Engineering at Case Western Reserve University, Cleveland, USA Zhi Ern Teoh Cornell Computational Synthesis Lab, Cornell University, Ithaca, New York, USA, [email protected] Saul Thurrowgood ARC Centre of Excellence in Vision Science, Queensland Brain Institute, University of Queensland, St. Lucia, QLD 4072, Australia, [email protected] Ravi Vaidyanathan Department of Mechanical Engineering at the University of Bristol, Bristol, UK; Department of Systems Engineering at the Naval Postgraduate School, CA, USA, [email protected] Frank Wippermann Fraunhofer Institute for Applied Optics and Precision Engineering, Albert-Einstein-Str. 7, D-07745, Jena, Germany Robert J. Wood Harvard Microrobotics Laboratory, Harvard University, Cambridge, MA, USA Robin Wootton School of Biological Sciences, Exeter University, Exeter EX4 4PS, UK, [email protected] Jochen Zeil ARC Centre of Excellence in Vision Science and Centre for Visual Sciences, Research School of Biological Sciences, The Australian National University, Biology Place, Canberra, ACT 2601, Australia, [email protected] Jean-Christophe Zufferey Laboratory of Intelligent Systems, EPFL, Lausanne, Switzerland, [email protected]

Contributors

Chapter 1

Experimental Approaches Toward a Functional Understanding of Insect Flight Control Steven N. Fry

Abstract This chapter describes experimental approaches exploring free-flight control in insects at various levels, in view of the biomimetic design principles they may offer for MAVs. Low-level flight control is addressed with recent studies of the aerodynamics of free-flight control in the fruit fly. The ability to measure instantaneous kinematics and aerodynamic forces in free-flying insects provides a basis for the design of flapping airfoil MAVs. Intermediate-level flight control is addressed by presenting a behavioral system identification approach. In this work, the motion processing and speed control pathways of the fruit fly were reverse engineered based on transient visual flight speed responses, providing a quantitative control model suited for biomimetic implementation. Finally, high-level flight control is addressed with the analysis of landmark-based goal navigation, for which bees combine and adapt basic visuomotor reflexes in a context-dependent way. Adaptive control strategies are also likely suited for MAVs that need to perform in complex and unpredictable environments. The integrative analysis of flight control mechanisms in free-flying insects promises to move beyond isolated emulations of biological subsystems toward a generalized and rigorous approach.

S.N. Fry () Institute of Neuroinformatics, University of Zurich and ETH Zurich; Institute of Robotics and Intelligent systems, ETH Zurich e-mail: [email protected]

1.1 Introduction Flying insects achieve efficient and robust flight control despite size constraints and hence limited neural resources [6, 12]. This is achieved from closely integrated and often highly specialized sensorimotor control pathways [19], making insects an ideal model system for the identification of biological flight control mechanisms, which can serve as design principles for future autonomous micro-air vehicles (MAVs) [18, 16, 38, 62]. While the implementation of biomimetic design principles in MAVs and other technical devices is inherently appealing, such an approach has its pitfalls that can easily lead to misconceptions [54]. A first problem relates to the immense complexity of biological systems, in particular flight control mechanisms. The multimodal sensorimotor pathways represent a high-dimensional control system, whose function and underlying physiology are understood only partially. A second problem relates to the often substantially different spatial and temporal scales of insects and MAVs. A meaningful transfer of a control mechanism identified in a small insect to its typically much larger robotic counterpart is non-trivial and requires detailed knowledge of the system dynamics. For example, it is not obvious how to control a robot based on motion processing principles derived from insects [41, 62], which perform maneuvers much faster and based on completely different locomotion principles than their robotic counterparts. This chapter presents recent experimental approaches aimed at a functional understanding of insect flight control mechanisms. To this end, flight control in insects is addressed at various levels, from the

D. Floreano et al. (eds.), Flying Insects and Robots, DOI 10.1007/978-3-540-89393-6_1, © Springer-Verlag Berlin Heidelberg 2009

1

2

biomechanics of flapping flight to flight control strategies and high-level navigational control. The experimental approaches share in common a detailed analysis of the time-continuous processes underlying the control of free flight under highly controlled and yet meaningful experimental conditions.

1.1.1 Chapter Overview • Low level: Biomechanics. A formidable challenge for the design of MAVs is to generate sufficient aerodynamic forces to remain aloft, while controlling these forces to stabilize flight and perform maneuvers. The sensorimotor system of insects has evolved under size constraints that may be quite similar to those of MAVs. Consequently, the biological solutions enabling flight in these small animals may provide useful design principles for the implementation of MAVs. The first example in this chapter addresses lowlevel flight control with a detailed description of free-flight biomechanics in the fruit fly Drosophila [29, 30]. 3D high-speed videography and dynamical force scaling were combined to resolve the movements of the wings and the resulting aerodynamic forces. The time-resolved analysis reveals aerodynamic and control requirements of insect flight, which are likewise essential to MAV design [60, 17, 16, 58], also see Chaps. 11–16. • Intermediate level: Visuomotor reflexes. To navigate autonomously in a cluttered environment, MAVs need to sense objects and produce appropriate responses, such as to avoid an impending collision. Insects meet this challenge with reflexive responses to the optic flow [33], i.e., the perceived relative motion of the environment during flight. The so-called optomotor reflexes mediate various visual flight responses, including attitude control, collision avoidance, landing, as well as control of heading, flight speed, and altitude. Optomotor reflexes provide a powerful model system to explore visual processing and flight control principles, reviewed in [12, 21, 46, 50], also see Chaps. 2, 4, and 17. The second example in this chapter describes a rigorous system analysis of the fruit fly’s visual flight speed response using TrackFly, a wind tunnel

S.N. Fry

equipped with virtual reality technology [27]. The identification of the control dynamics in the form of a controller provides a powerful strategy to transfer biological control principles into the robotic context, including MAVs [17, 16], also see Chap. 3. • High level: Landmark navigation. Autonomous MAVs should ultimately be able to flexibly solve meaningful tasks, such as navigate through cluttered, unpredictable, and potentially dangerous environments, and safely return to their base. Here, too, insects can serve as a model system, as some species show the impressive ability to acquire the knowledge of specific locations in their environment (e.g., nest, food site), which they repeatedly visit over the course of many days [56], also see Chaps. 2 and 7. The third example in this chapter describes an experimental approach aimed to explain landmarkbased goal navigation in honey bees from a detailed analysis of individual maneuvers. Goal navigation is explained with basic sensorimotor control mechanisms that are combined and modified through the learning experience. The ability to achieve robust, adaptive, and flexible flight control as an emergent property of basic sensorimotor control principles offers yet more interesting options for the design of autonomous MAVs with limited built-in control circuits.

1.2 Low-Level Flight Control – Biomechanics of Free Flight A detailed knowledge of flight biomechanics provides the foundation for our understanding of biological flight control strategies – or their implementation in MAVs. Flight control is ideally studied in free flight, in which the natural flight behavior of an insect can be measured under realistic sensory and dynamic flight conditions. To understand how a flapping insect stabilizes its flight and performs maneuvers, the underlying mechanisms must be studied at the level of single wing strokes – not an easy task, considering the tiny forces and short timescales involved. The example described in this section shows how the application of 3D high-speed videography and dynamic force scaling using a robotic fly wing allowed such a detailed analysis of free-flight biomechanics to

1

Experimental Approaches to Insect Flight Control

3

be performed in the fruit fly Drosophila, a powerful model system for the design of flapping airfoil MAVs [18, 17, 16, 58].

1.2.1 Research Background The aerodynamic basis of insect flight has remained enigmatic due to the complexities related to the intrinsically unsteady nature of flapping flight [57]. A solid theoretical basis for quantitative analyses of insect flight aerodynamics was provided by Ellington’s influential theoretical work based on time-averaged models [22]. At the experimental level, dynamically scaled robotic wings provided the technological breakthrough allowing aerodynamic effects to be explored empirically at the timescale of a single wing stroke [44].

by a black cylindrical cup filled with vinegar, the flies approached the center of the chamber, where they often hovered before landing on the cup or instead performed a fast turning maneuver (saccade) in order to avoid colliding with it. These flight sequences were filmed using three orthogonally aligned high-speed cameras, whose lines of sight intersected in the middle of the flight chamber. Next, the wing and body kinematics were extracted using a custom-programmed graphical user interface (Fig. 1.1B). The wing kinematics were then played through a dynamically scaled robotic wing (Robofly, Fig. 1.1C) to measure the aerodynamic wing forces (arrows in Fig. 1.1B) throughout the filmed flight sequence. Combining the kinematic and force data allowed a direct calculation of instantaneous aerodynamic forces, torques, and power (see below). 1.2.2.1 Hovering Flight

1.2.2 Experiments To perform a time-resolved biomechanical analysis of free flight, the wing and body movements of fruit flies were recorded using 3D high-speed videography (Fig. 1.1A). For this, hungry flies were released into a small flight chamber (side length 30 cm). Attracted

B

A

Hovering flight offers itself for an analysis of the aerodynamic requirements of flapping flight without the complications resulting from body motion. The analysis of such a hovering sequence, consisting of six consecutive wing strokes, is shown in Fig. 1.2. The precisely controlled wing movements are characterized by a high angle of attack and maximal force

C Motors

HS

Wing Head

C

Mineral oil

Fly’s path

HS HS

Abdomen Model wing

LED

Aerodynamic force 0.8 m

Fig. 1.1 Measurement of kinematics and forces. (A) Setup: Flies were filmed with three orthogonally aligned high-speed (5000 fps) cameras (HS). Arrays of near-infrared light emitting diodes (LEDs) were used for back-lighting. Flies were attracted to a small cylindrical cup (C), in front of which they were filmed within the small overlapping field of view of the cameras (shown as a wire-frame cube). (B) Kinematic extraction: Wing and body kinematics were measured using a graphical user interface, which allowed a user to match the wing silhouettes and the

positions of the head and abdomen to obtain their 3D positions. Arrows show the aerodynamic forces measured using Robofly. (C) “Robofly”: Plexiglas wings (25 cm in length) were flapped in mineral oil at the appropriate frequency to match the Reynolds number of the fly’s flapping wings in air (and hence the fluid dynamics). The up-scaled fluid dynamic forces were measured with strain gauges on the wings and the aerodynamic forces acting on the fly’s wings (shown in B) were back-calculated. Figure modified from [30]

4

S.N. Fry

A

Net force (N) Lift Thrust

0

20 10 Time (ms)

30

D

Measured

Model

1 0.5

5

Pitch torque (10–8 Nm)

5 0 –5

1.5

0

Down Up

Angle Devi- Stroke of ation angle attack (deg)

90 0 –90

Flight forces (10–5 N)

C

0

–5

B 1 × 10–5 N

E

500

Down

Up

Power (W kg–1)

Total Aerodyn. 0 Intertial –200

Down

Up

Stroke cycle

Fig. 1.2 Hovering flight. (A) Wing kinematics and flight forces. Data from six consecutive wing beats are shown. For a definition of the stroke angles refer to [30]. (B) Wing motion and forces. Successive wing positions during a hovering stroke cycle are shown with matchstick symbols (dots show the leading edge). Instantaneous aerodynamic forces are shown with arrows. Axes indicate horizontal (± 90◦ ) and vertical (± 10◦ ) stroke positions. The arrow shown with the fly shows the aerodynamic force averaged over the stroke cycle. (C) Quasi-steady analysis of

instantaneous aerodynamic force. The measured force is shown together with the force predicted by a quasi-steady model [20]. (D) Instantaneous pitch torque. Pitch torque (black line ± S.D.) oscillates around a mean of zero during hovering. (E) Instantaneous specific flight power (in W/kg muscle mass). Traces show total mechanical power (±S.D.), which is composed of the aerodynamic and inertial power required for wing motion. Figure modified from [30]

production during the middle of the downstroke and the early upstroke (Fig. 1.2A, B). As expected for hovering, the drag of the downstroke and upstroke cancels itself out, while the mean lift offsets the fly’s weight. The aerodynamic forces generated by the wing movements are largely explained with a quasi-steady model that takes into account translational and rotational effects of the wing motion (Fig. 1.2C). The main discrepancy between the modeled and measured forces is a phase delay, which is likely due to unsteady effects (e.g., wing–wake interactions) not considered here. A further requirement of stable hovering flight is a precise balance of the aerodynamic torques over the course of a wing stroke. To maintain a constant body pitch, for example, the substantial torque peaks

generated throughout the stroke cycle must cancel each other out precisely, as shown in Fig. 1.2D. Finally, the instantaneous power was calculated directly from the scalar product of wing velocity and the forces acting on the wings (Fig. 1.2E). The power associated with aerodynamic force production peaks around the middle of each half-stroke, when aerodynamic forces and wing velocity are maximal. Conversely, the power required to overcome wing inertia reverses its sign during each half-stroke due to the deceleration of the wings toward the end of each half-stroke. The total mechanical power, the sum of these two components, is positive for most part of the wing stroke. Power is negative when the wings decelerate while producing little aerodynamic force, which occurs briefly toward

1

Experimental Approaches to Insect Flight Control

5

the end of each half-stroke. During this phase, the mechanical power could be stored elastically and partially retrieved during the subsequent half-stroke to reduce the total power requirements. The potential reduction of flight power in fruit flies, however, is quite limited (in the order of 10%).

1.2.2.2 Maneuvering

B

Yaw torque High Low

30 deg

The physical constraints may be similar for MAVs and flies operating at similar size scales, and the flight control mechanisms evolved in insects can therefore provide valuable design principles for MAVs. The application of high-speed videography and aerodynamic force measurements using dynamically scaled robots provides detailed insights into the requirements of insect flight control that can help identify important design constraints for MAVs. This analysis reveals critical aspects of flight control in Drosophila that need to be considered also for MAV design. Precise and fast wing actuation appears most critical for flight control. As shown by the example of pitch torque, the instantaneous torques produced by the wings vary considerably and must be precisely balanced over the course of a stroke cycle. As shown by the analysis of yaw torque during turning maneuvers, even subtle changes in wing motion are sufficient to induce fast turns within a few wing beats. Precise and fast sensorimotor control loops are obviously required for flight control using similar morphologies. The experiments also indicate less critical features of flapping flight control, at least at the size scale of the fruit fly. Wing stiffness and surface structure, for example, may be relatively unimportant under certain conditions. The simple Plexiglas wing used in Robofly was sufficient to reproduce the required aerodynamic forces without the need to mimic the quite complicated structure of the fly’s wing. The feasibility of a stiff,

15 10

2 1

5 0

0

–5

–1

–10

Stroke plane angle

–20 –10

Fig. 1.3 Maneuvering. (A) Changes in wing kinematics associated with yaw torque production during saccades. Wing tip trajectories were measured during free-flight turning maneuvers (saccades). Wing tip trajectories associated with high and low aerodynamic yaw torques are shown as light and dark gray lines,

3

Stroke amplitude

0 10 20 Time (ms)

–2 30

Stroke plane angle (deg)

A

Stroke amplitude (deg)

This approach was taken further to explore how flies modify their wing kinematics during flight maneuvers. Flight sequences containing saccadic turning maneuvers were filmed and the aerodynamic wing forces again measured using Robofly. Figure 1.3A shows wing tip trajectories during such maneuvers, labeled according to the yaw torque they produced. At the onset of a turn, the outside wing tilts backward and its amplitude increases (light gray tip trajectories). Conversely, the inside wing tilts forward and its amplitude decreases (dark gray trajectories). The resulting difference in yaw torque generated by the two wings is sufficient to accelerate the fly to over 1000◦ /s within about five wing strokes [29]. The changes in stroke plane angle and stroke amplitude over the course of a saccade are shown in Fig. 1.3B. To maximally accelerate at the onset of the saccade, the difference in stroke amplitude between the outside and inside wing is only around 5◦ , while the stroke plane angle differs by a mere 2◦ . Even during extreme flight maneuvers, therefore, the changes in wing kinematics are quite small.

1.2.3 Conclusions

40

respectively. (B) Bilateral changes in wing kinematics during a turning maneuver. At the onset of a turn, the outside wing increases the stroke amplitude and stroke plane angle (backward tilt) relative to the inside wing. Figure modified from [29]

6

light-weight wing structure for flapping lift production was recently demonstrated in [59]. As a further simplification, quasi-steady mechanisms of aerodynamic force production dominate in the production of aerodynamic forces of flapping wings, at least in the fruit fly. Unsteady effects, such as wing–wing and wing–wake interactions, play a minor role, such that simple analytical tools can be applied at least in first approximation. Finally, elastic storage plays a comparatively small role given the small mass of the wings, and therefore does not present a significant design constraint. In conclusion, the measurement of instantaneous wing positions in free flight, together with the aerodynamic forces measured in the robotic wing, is sufficient to robustly quantify several relevant aspects of flight biomechanics. The example of the fruit fly, itself a powerful model for MAV design [61, 60], reveals a suitable strategy to get flapping MAVs off the ground. The next substantial challenge is to actively stabilize flight, for which exceedingly fast and precise wing control is required. The impressive advances in flight biomechanics provide a solid foundation for biomimetic MAV design that takes into account the requirement for flight control.

1.3 Intermediate-Level Flight Control – Visuomotor Reflexes An intermediate level of flight control involves reflexive responses to sensory input. On the one hand, they can mediate corrective maneuvers to recover from disturbances and unstable flight conditions to increase dynamic system stability. On the other hand, they can mediate flight maneuvers to suitably respond in an unpredictable environment. For example, an MAV equipped with motion sensors can sense an object appearing in front and respond with an avoidance maneuver to prevent a collision. Equipped with the appropriate sensors and flight control strategies, autonomous MAVs can navigate more safely and efficiently within cluttered and unpredictable environments (Chaps. 3 and 8). The extremely efficient and robust visuomotor reflexes of insects can provide design principles for biomimetic flight control strategies in autonomous MAVs. The second part of this chapter describes behavioral experiments aimed at a rigorous system

S.N. Fry

identification of visuomotor control pathways in the fruit fly. The characterization of biological control pathways in the form of a control model allows more direct and meaningful transfer of biological flight control principles into a robotic context, including MAVs [17, 16].

1.3.1 Research Background Pioneering experiments explored the transfer properties of optomotor turning reflexes in insects. This was achieved with a simple preparation, in which insects were stimulated using a rotating drum and their intended turning responses measured using elegant techniques [36, 23, 34]. The response tuning of optomotor turning reflexes provides the foundation for a cohesive theory of optic flow processing in insects to this day, reviewed in [6, 4, 37, 5], also see Chaps. 4 and 5. While tethering provides a simple method to deliver stimuli without influence of the behavioral reactions (referred to as open loop in the biological literature), the results of such experiments are difficult to interpret functionally [49]. Tethering disrupts various reafferent feedback circuits, which leads to significant behavioral artifacts [30] and prevents the analysis of flight control under realistic dynamical conditions. Visual reflexes have also been extensively explored in free flight, reviewed in [12, 6, 46], in which case the visual input is coupled to the insect’s flight behavior (natural closed-loop condition) [45, 49]. This coupling hinders a rigorous system identification because the stimuli are no longer under complete experimental control. Nevertheless, data obtained in this way can provide valuable insight into flight control mechanisms [40, 11, 3], Chap. 2. A simpler behavioral analysis becomes possible from measuring free-flight behavior under steady-state conditions [15, 47, 2], but cannot address questions relating to flight control dynamics.

1.3.2 Experiments A functional understanding of biological flight control principles that can be meaningfully transferred into MAVs requires careful consideration of the

1

Experimental Approaches to Insect Flight Control

multimodal reafferent pathways. Below I describe a recent experimental approach aimed at a system identification of visuomotor pathways that can serve as design principles for MAV control.

1.3.2.1 System Analysis of Visual Flight Speed Control Using Virtual Reality Display Technology To perform a system identification of the fruit fly’s motion-dependent flight speed control pathways, a wind tunnel was equipped with virtual reality display technology (TrackFly [27], Fig. 1.4). An automated procedure was implemented to induce flies to fly to the center of the wind tunnel and then stimulate them with horizontally moving sine gratings of defined temporal frequency (TF), spatial frequency (SF), and contrast. To hold the linear image velocity (defined as TF/SF) constant in the fly’s eyes, the grating speed was adjusted continuously to compensate for the displacement of the fly (one-parameter open-loop paradigm). The automated high-throughput system allowed a large data set of visual responses to be measured for a broad range of temporal and spatial frequencies. The results show that fruit flies

M LP

Cam

Cam Projector

Wind Fly M

LP

Fig. 1.4 TrackFly. A wind tunnel was equipped with a virtual reality display system (only working section of the wind tunnel is shown). Visual stimuli were presented to free-flying flies in open loop, i.e., the pattern offset was adjusted to the fly’s position along the wind tunnel in real time. M: Mirror; LP: Light path; Cam: Video camera. For details see [27]

7

respond to the linear velocity (TF/SF) of the patterns, which serves as a control signal for flight speed [27]. The visual tuning properties of visual flight speed responses differ from optomotor turning responses, which instead show a response maximum at a particular temporal frequency (TF) of displayed patterns [34]. Next, system identification procedures were applied to obtain a controller that was able to reproduce the transient open-loop response properties, i.e., reproduce the transient changes in flight speed after onset of the optic flow stimulus. The controller was then used to predict the speed responses under more realistic visual closed-loop conditions, and the results confirmed with data obtained from flies tested in closed loop. A detailed quantitative account of the procedures and data is published elsewhere ([27], [28], Rohrseitz and Fry, in prep.).

1.3.3 Conclusions The reflexive flight control pathways of insects can provide powerful control architectures for biomimetic MAVs. For a meaningful interpretation of the biological measurements, however, the behavioral context and relevance of multimodal feedback must be carefully considered. Free-flight experiments are ideal to explore flight control under realistic flight conditions, but the difficulty of delivering arbitrary stimuli in a controlled manner is a hindrance for detailed behavioral system identification. The described approach works around this limitation by allowing a particular parameter (here: pattern speed) to be presented in open loop, without disrupting the remaining stimuli. From the measured transient responses, linear pattern velocity was identified as the relevant control parameter for visual flight speed control. Based on this high-level understanding, the underlying visual computations and neural structures can be further explored. Next, the transient responses were used to reverse engineer the control scheme underlying flight speed control (Fig 1.5). The measurements performed in open and closed loop are quantitatively explained by a proportional control law, which is simpler still than a PID controller recently suggested for insects [24], Chap. 3. A rigorous system identification approach in biology provides a functional understanding of the

8

S.N. Fry

Wind tunnel TF, SF

Pattern slip speed

Speed signal Vision

Flight speed command

Control

Locomotor limit

Open/closed loop

Flight speed

Fig. 1.5 Control model. The fly’s flight speed responses are quantitatively explained by a simple control model. The visual system computes pattern velocity as input to a controller of flight

speed, which is constrained by measured locomotor limits. The simulated flight speed responses in open and closed loop (note switch symbol) were verified experimentally using TrackFly

underlying neuromotor pathways and characterizes their dynamics in a concrete control model. The control strategy can then be meaningfully transferred into MAVs even if the underlying neuromotor mechanisms remain only partially known.

1.4.1 Research Background

1.4 High-Level Flight Control – Landmark-Guided Goal Navigation High-level flight control strategies are ultimately required to enable MAVs to perform meaningful tasks, for which sensory input must be processed in a contextdependent way. For example, an MAV could rely on the same visual objects encountered along its flight path during the outbound trip and to return to its home base, requiring a context-dependent processing of the visual input. Some insect species reveal the amazing ability to return to quite distant places that they previously visited. Honey bees, for example, learn the location of a rewarding food source, which they repeatedly visit to collect food for the hive (also see Chaps. 2 and 7). The third example of this chapter describes experiments exploring the basic control principles underlying such complex, context-dependent behaviors. Landmark-based goal navigation is explained with visuomotor control mechanisms that are modified through learning experience. Complex flight behaviors result as an emergent property of basic flight control strategies and their interactions with the environment. Similar control strategies could allow MAVs with limited resources to likewise perform well in complex, real-world applications.

The mechanisms by which flying insects use landmarks to return to a learned place was pioneered by Tinbergen’s (1932) [52] classic neuroethological studies in the digger wasp. His approach, followed by many later researchers (e.g., for flying honey bees [1, 7]; review [55]), was to induce search flights in experimentally modified visual surroundings and conclude from the search location the internal visual representation of the visual environment. A similar approach in honey bees performed half a century later led to the influential snapshot model [8], which explained goal-directed flight control to result from a comparison between the current retinal image and a template image formerly stored at the goal location. The ways in which insects represent locations as visual memories and use these to return to a learned place are studied experimentally in increasing detail, see recent reviews in [13, 10]. Not least due to its algorithmic formulation, the snapshot and related models found widespread appeal in the robotic community and were further explored in numeric [43] and robotic [39, 42] implementations, reviewed in [25, 53].

1.4.2 Experiments The experiments giving rise to the snapshot model were first replicated and extended to explore landmarkbased goal navigation in more detail [31]. The results were suggestive of alternative visuomotor control strategies, which were subsequently explored with detailed analyses of individual approach flights using more advanced video tracking techniques [26].

Experimental Approaches to Insect Flight Control Ai

Initial training Aiii

9

Aii

B

Aiv

Flight durations (s)

1

Feeder

15 10 5 0 1

20 40 60 80 100 120

Flights

Fig. 1.6 Learning experiments. (A) Successive approach flights of a single bee. The experiments were performed in a cylindrical tent (Ø 2.4 m). The bee entered on the left and flew to an inconspicuous feeder (location indicated with a stippled cross-hair) 0.5 m in front of a black paper square attached to the back wall (shown as a black bar on the right). (i) Initial training. The bee was trained by displacing a temporary feeder (location shown

with arrows) closer toward the final feeder position on consecutive foraging trips. Lines show the flight trajectories toward the temporary and final feeder positions. (ii) Flights 1–20. (iii) Flights 51–70. (iv) Flights 101–120. (B) Duration of successive flights. With increasing experience, flight duration decreased to about 2 s after about 50 flights. Figure modified from [32]

Detailed measurements of approach flights of bees to a goal location were made, taking care not to disrupt their natural behavior. To explore the relevance of learning, every single approach flight of a single bee was measured (Fig. 1.6). By moving a temporary feeder stepwise through a uniform flight tent, the bee was trained to fly toward a permanent feeder located in front of a single landmark (Fig. 1.6 Ai). The approach flights of this inexperienced bee were slow and quite convoluted. The first 20 flights of the same bee with the permanent feeder were faster, but still revealed turns and loops reminiscent of search flights (Fig. 1.6 Aii). The bee’s approaches became progressively faster and smoother as it gained experience during successive foraging trips (Fig. 1.6 Aiii). After about 100 flights, the bee approached the feeder with straight and fast trajectories. Duration of successive flights decreased from about 5–10 s to 2–3 s (Fig. 1.6B). Next, individual bees were trained using landmark settings that differed in position, number, and color of the landmarks. First, a bee was trained with a single black cylinder (•) located just to the right of the feeder (Fig. 1.7A, left; feeder location is marked with crosshairs). The bee approached the cylinder and performed occasional left turns, roughly aimed at the feeder position. During these approaches, the bee held the cylinder roughly in the frontal-right visual field (Fig. 1.7A, right). A different bee trained with the cylinder further to the right side still held the cylinder in the

right visual field and performed more convoluted flight paths toward the goal (Fig. 1.7B). In the identical situation, a second bee approached the feeder with a different approach pattern, but like the first bee held the cylinder in the right visual field (Fig. 1.7C). This simple rule was even used by bees trained with two differently colored cylinders (marked L and R in Fig. 1.7D). The bees simply relied on one of the cylinders (in this case the right cylinder) for the initial approach, again holding it in the right visual field. The detailed structure of an approach flight, together with the measured body axis direction, is shown in (Fig. 1.7E). These and other experiments provide a coherent view on the visuomotor strategies employed by bees to locate a goal in different environmental situations. Bees with little experience with a landmark setting (Fig. 1.6 Ai, Aii) or in the absence of a suitable (i.e., near-frontal) landmark (Fig. 1.7C) perform search-like flights. If a landmark is suitably located behind the goal (Fig. 1.6), an experienced bee will simply head toward it and find its goal. To do so, the bee needs to fixate the landmark in a frontal position, as symbolized with the curved arrows in Fig. 1.8A. If the bee tends to keep the landmark in the right visual field (Fig. 1.8B), it will tend to make turns to the left, as required for the final goal approach (Fig. 1.7). Finally, the bee can rely on one of several landmarks during its goal approach, if it associates it with the appropriate retinal position and/or turning direction (Fig. 1.8C). These results are

10

S.N. Fry

B

1

0 180

Frequency

Frequency

A

90

0

90

180

1

0 180

D

1

L

Frequency

Frequency

C

R 0 180

90

0

90

90

0

90

180

Azimuth distribution (deg)

Azimuth distribution (deg) 1

L

R

0 180

180

90

0

90

180

Azimuth distribution (deg)

Azimuth distribution (deg)

E

Fig. 1.7 Approach flights and landmark azimuth during approach flights. (A) Left: 40 successive approach flights of an individual bee with a cylinder (•) positioned at an angular distance of 15◦ from the feeder (+). Right: Distribution of landmark positions in the bee’s visual field. (B) Cylinder placed 40◦ to the right of the feeder. (C) As in (B), with data from a differ-

A

Landmark

ent bee. (D) Approach flights of two individual bees in the presence of two cylinders of different colors. The bees headed toward the right (R) cylinder. (E) Typical example of a bee’s approach flight. The bee’s position (dots) and body axis direction (lines) were measured at 50 Hz using a pan-tilt tracking system [26]. Data are subsampled for clarity. Figure modified from [32]

B

Feeder

C 2

1

1

1 1

2 1

1

2

Flight path

Fig. 1.8 Visuomotor guidance model. (A) Frontal landmark. Bees fixate the landmark frontally. (B) Lateral landmark. Bees perform biased turns to the left, keeping the cylinder in the right visual field. (C) Two landmarks. Bees use one landmark dur-

ing the approach. Lines show hypothetical flight paths. Curved arrows symbolize a learned visuomotor association. For details see [32]

consistent with various experiments performed in both flying and walking insects, e.g., [9, 35].

process. Relatively unstructured, search-like flights are observed in bees with limited experience with the landmark setting. A suitable landmark is used to direct the flight toward the goal, while successful flight motor patterns are reinforced by operant learning to increase the efficiency and reliability of the approach flights. Control strategies based on a flexible and adaptable employment of basic control loops may also be suited for MAVs with limited storage and processing capacity to enable successful landmark navigation. A

1.4.3 Conclusions Landmark-based goal navigation in honey bees is explained as an emergent property of basic visuomotor reflexes, which are modified by a continuous learning

1

Experimental Approaches to Insect Flight Control

possible scenario could consist of exploring unfamiliar terrain and subsequently patrolling along suitable routes. The flexible use of comparatively basic visuomotor control strategies can more likely meet the high requirements for fast and robust flight control required by MAVs than a single complex and hard-wired algorithm. Flexible adaptation to varying environmental conditions and increasing experience provides a powerful strategy for flight control in complex and unpredictable environments.

1.5 Closing Words Insects perform complex flight control tasks despite their small size and presumably limited neural capacity. The fact that insects nevertheless excel in their flight performance is explained with a close integration of specialized sensorimotor pathways. While it is intriguing to take inspiration from flying insects for the design of autonomous MAVs, the high complexity of an insect’s multimodal flight control system renders this task non-trivial and prone to misconceptions [54, 14, 50]. It may therefore be hardly fruitful – and indeed counter-productive – to take superficial inspiration from biology and implement the purported principles in robots without due care. Instead, biologists and engineers should take advantage of the fact that insects can achieve superior flight control with possibly quite basic, but highly integrated control principles. To this end, detailed biological studies are required that address flight control mechanisms at various levels, including biomechanics, neural processing, sensorimotor integration, and high-level behavioral strategies. The experimental approaches described in this chapter show that advanced concepts and technologies can help provide the functional understanding of biological flight control principles required for meaningful biomimetic implementations in MAVs. Not only can engineers profit from rigorous biological research of flight control, but the concepts and tools applied in engineering [17, 16] can likewise be meaningfully applied to explore biological control principles in a rigorous and quantitative way [51, 48, 49, 27]. The control principles thus identified in insects can then be transferred more easily into a

11

robotic environment with appropriate consideration of the behavioral context and scaling issues. The presented research examples motivate closer interactions between biological research of flight control mechanisms and engineering design of MAVs. Such interactions promise significant benefits to both fields, in that biologists can aim at more rigorous quantitative analyses of flight control and engineers can aim at more meaningful biomimetic implementations. Indeed, this aim will likely have been reached when the common fascination about flight control becomes the defining element of a coherent, interdisziplinary research effort. Acknowledgments I wish to thank to reviewers for useful comments, Chauncey Grätzel for advice on the writing, Jan Bartussek, Vasco Medici and Nicola Rohrseitz for useful comments and discussions. The work described in this chapter was funded by the following institutions: Human Frontiers Science Program (HFSP), Swiss Federal Institute of Technology (ETH) Zurich; Swiss National Science Foundation (SNSF), University of Zurich and Volkswagen Foundation.

References 1. Anderson, A.: A model for landmark learning in the honeybee. Journal of Comparative Physiology A 114, 335–355 (1977) 2. Baird, E., Srinivasan, M.V., Zhang, S., Cowling, A.: Visual control of flight speed in honeybees. Journal of Experimental Biology 208(20), 3895–3905 (2005) 3. Boeddeker, N., Kern, R., Egelhaaf, M.: Chasing a dummy target: Smooth pursuit and velocity control in male blowflies. Proceedings of the Royal Society of London Series B Biological Sciences 270(1513), 393–399 (2003) 4. Borst, A., Egelhaaf, M.: Principles of visual motion detection. Trends in Neurosciences 12(8), 297–306 (1989) 5. Borst, A., Haag, J.: Neural networks in the cockpit of the fly. Journal of Comparative Physiology A 188(6), 419–37 (2002) 6. Buchner, E.: Behavioral analysis of spatial vision in insects. In: M.A. Ali (ed.) Photoreception and Vision in Invertebrates, pp. 561–621. Plenum Press, New York (1984) 7. Cartwright, B.A., Collett, T.S.: How honey bees use landmarks to guide their return to a food source. Nature 295, 560–564 (1982) 8. Cartwright, B.A., Collett, T.S.: Landmark learning in bees: Experiments and models. Journal of Comparative Physiology A 151, 521–543 (1983) 9. Chittka, L., Kunze, J., Shipman, C., Buchmann, S.L.: The significance of landmarks for path integration in homing honeybee foragers. Naturwissenschaften 82, 341–343 (1995)

12 10. Collett, T.S., Graham, P., Harris, R.A., Hempel de Ibarra, N.: Navigational memories in ants and bees: Memory retrieval when selecting and following routes. Advances in the Study of Behavior 36, 123–172 (2006) 11. Collett, T.S., Land, M.F.: Visual control of flight behaviour in the hoverfly Syritta pipiens L. Journal of Comparative Physiology 99, 1–66 (1975) 12. Collett, T.S., Nalbach, H.O., Wagner, H.: Visual stabilization in arthropods. Reviews of Oculomotor Research 5, 239–63 (1993) 13. Collett, T.S., Zeil, J.: Places and landmarks: An arthropod perspective. In: S. Healy (ed.) Spatial Representation in Animals, pp. 18–53. Oxford University Press, Oxford, New York (1998) 14. Datteri, E., Tamburrini, G.: Biorobotic experiments for the discovery of biological mechanisms. Philosophy of Science 74(3), 409–430 (2007) 15. David, C.T.: Compensation for height in the control of groundspeed by Drosophila in a new, ’barber’s pole’ wind tunnel. Journal of Comparative Physiology A 147, 485–493 (1982) 16. Deng, X.Y., Schenato, L., Sastry, S.S.: Flapping flight for biomimetic robotic insects: Part II - Flight control design. IEEE Transactions on Robotics 22(4), 789–803 (2006) 17. Deng, X.Y., Schenato, L., Wu, W.C., Sastry, S.S.: Flapping flight for biomimetic robotic insects: Part I - System modeling. IEEE Transactions on Robotics 22(4), 776–788 (2006) 18. Dickinson, M.H.: Bionics: Biological insight into mechanical design. Proceedings of the National Academy of Sciences of the United Statesof America 96(25), 14, 208–9. (1999) 19. Dickinson, M.H.: Insect flight. Current Biology 16(9), R309–R314 (2006) 20. Dickson, W.B., Dickinson, M.H.: The effect of advance ratio on the aerodynamics of revolving wings. Journal of Experimental Biology 207(24), 4269–4281 (2004) 21. Egelhaaf, M., Kern, R.: Vision in flying insects. Current Opinion in Neurobiology 12(6), 699–706 (2002) 22. Ellington, C.P.: The aerodynamics of hovering insect flight. Philosophical Transactions of the Royal Society B: Biological Sciences 305, 1–181 (1984) 23. Fermi, G., Reichardt, W.: Optomotorische Reaktionen der Fliege Musca domestica. Kybernetik 2, 15–28 (1963) 24. Franceschini, N., Ruffier, F., Serres, J.: A bio-inspired flying robot sheds light on insect piloting abilities. Current Biology 17(4), 329–335 (2007) 25. Franz, M.O., Schöllkopf, B., Mallot, H.A., Bülthoff, H.H.: Where did i take that snapshot? Scene based homing by image matching. Biological Cybernetics 79, 191–202 (1998) 26. Fry, S.N., Bichsel, M., Müller, P., Robert, D.: Tracking of flying insects using pan-tilt cameras. Journal of Neuroscience Methods 101(1), 59–67 (2000) 27. Fry, S.N., Rohrseitz, N., Straw, A.D., Dickinson, M.H.: TrackFly: Virtual reality for a behavioral system analysis in free-flying fruit flies. Journal of Neuroscience Methods 171(1), 110–117 (2008) 28. Fry, S.N., Rohrseitz, N., Straw, A.D., Dickinson, M.H.: Visual control of flight speed in Drosophila melanogaster. Journal of Experimental Biology 212, 1120–1130 (2009)

S.N. Fry 29. Fry, S.N., Sayaman, R., Dickinson, M.H.: The aerodynamics of free-flight maneuvers in Drosophila. Science 300(5618), 495–498 (2003) 30. Fry, S.N., Sayaman, R., Dickinson, M.H.: The aerodynamics of hovering flight in Drosophila. Journal of Experimental Biology 208(12), 2303–2318 (2005) 31. Fry, S.N., Wehner, R.: Honey bees store landmarks in an egocentric frame of reference. Journal of Comparative Physiology A 187(12), 1009–1016 (2002) 32. Fry, S.N., Wehner, R.: Look and turn: Landmark-based goal navigation in honey bees. Journal of Experimental Biology 208(20), 3945–3955 (2005) 33. Gibson, J.J.: The visual perception of objective motion and subjective movement. 1954. Psychological Review 101(2), 318–23 (1994) 34. Götz, K.G.: Optomotorische Untersuchung des visuellen Systems einiger Augenmutanten der Fruchtfliege Drosophila. Kybernetik 2, 77–92 (1964) 35. Graham, P., Fauria, K., Collett, T.S.: The influence of beacon-aiming on the routes of wood ants. Journal of Experimental Biology 206(3), 535–541 (2003) 36. Hassenstein, B., Reichardt, W.: Systemtheoretische Analyse der Zeit-, Reihenfolgen- und Vorzeichenauswertung bei der Bewegungsperzeption des Rüsselkäfers Chlorophanus. Zeitschrift für Naturforschung 11b, 513–524 (1956) 37. Hausen, K.: Decoding of retinal image flow in insects. Reviews of Oculomotor Research 5, 203–35 (1993) 38. Jeong, K.H., Kim, J., Lee, L.P.: Biologically inspired artificial compound eyes. Science 312(5773), 557–561 (2006) 39. Lambrinos, D., Möller, R., Labhart, T., Pfeifer, R., Wehner, R.: A mobile robot employing insect strategies for navigation. Robotics and Autonomous Systems 30, 39–64 (2000) 40. Land, M.F., Collett, T.S.: Chasing behaviour of houseflies (Fannia canicularis). Journal of Comparative Physiology A 89, 331–357 (1974) 41. Liu, S.C., Usseglio-Viretta, A.: Fly-like visuo-motor responses on a robot using aVLSI motion chips. Biological Cybernetics 85(6), 449–457 (2001) 42. Möller, R.: Insect visual homing strategies in a robot with analog processing. Biological Cybernetics 83, 231–243 (2000) 43. Möller, R.: Do insects use templates or parameters for landmark navigation? Journal of Theoretical Biology 210, 33– 45 (2001) 44. Sane, S.P., Dickinson, M.H.: The aerodynamic effects of wing rotation and a revised quasi-steady model of flapping flight. Journal of Experimental Biology 205(8), 1087–1096 (2002) 45. Schuster, S., Strauss, R., Götz, K.G.: Virtual-reality techniques resolve the visual cues used by fruit flies to evaluate object distances. Current Biology 12(18), 1591–4 (2002) 46. Srinivasan, M.V., Zhang, S.: Visual motor computations in insects. Annual Review of Neuroscience 27, 679–96 (2004) 47. Srinivasan, M.V., Zhang, S., Lehrer, M., Collett, T.S.: Honeybee navigation en route to the goal: Visual flight control and odometry. Journal of Experimental Biology 199(1), 237–44 (1996) 48. Tanaka, K., Kawachi, K.: Response characteristics of visual altitude control system in Bombus terrestris. Journal of Experimental Biology 209(22), 4533–4545 (2006) 49. Taylor, G.K., Bacic, M., Bomphrey, R.J., Carruthers, A.C., Gillies, J., Walker, S.M., Thomas, A.L.R.: New experimen-

1

50.

51.

52.

53.

54. 55.

56.

Experimental Approaches to Insect Flight Control tal approaches to the biology of flight control systems. Journal of Experimental Biology 211(2), 258–266 (2008) Taylor, G.K., Krapp, H.G.: Sensory systems and flight stability: What do insects measure and why? Advances in Insect Physiology 34, 231–316 (2008) ˙ Taylor, G.K., Zbikowski, R.: Nonlinear time-periodic models of the longitudinal flight dynamics of desert locusts Schistocerca gregaria. Journal of the Royal Society Interface 2(3), 197–221 (2005) Tinbergen, N.: über die Orientierung des Bienenwolfs (Philantus triangulum Fabr.). Zeitschrift für Vergleichende Physiologie 16, 305–334 (1932) Vardy, A., Möller, R.: Biologically plausible visual homing methods based on optical flow techniques. Connection Science 17(1), 47 – 89 (2005) Webb, B.: Validating biorobotic models. Journal of Neural Engineering 3(3), R25–R35 (2006) Wehner, R.: Spatial vision in arthropods. Handbook of Sensory Physiology, vol. VII/6C, pp. 287–616. Springer, Berlin, Heidelberg, New York, Tokyo (1981) Wehner, R.: Arthropods. In: F. Papi (ed.) Animal homing, pp. 45–144. Chapman & Hall (1992)

13 57. Weis-Fogh, T.: Quick estimates of flight fitness in hovering animals, including novel mechanisms for lift production. Journal of Experimental Biology 59, 169–230 (1973) 58. Wood, R.J.: Design, fabrication, and analysis of a 3DOF, 3 cm flapping-wing MAV. Intelligent Robots and Systems, 2007. IROS 2007, pp. 1576–1581 (2007) 59. Wood, R.J.: The first takeoff of a biologically inspired atscale robotic insect. Robotics, IEEE Transactions on 24(2), 341–347 (2008) 60. Wu, W., Shenato, L., Wood, R.J., Fearing, R.S.: Biomimetic sensor suite for flight control of a micromechanical flying insect: Design and experimental results. Proceedings of the 2003 IEEE International Conference on Robotics and Automation (ICRA 2003), vol. 1, pp. 1146–1151. IEEE Press, Piscataway, NJ (2003) 61. Wu, W.C., Wood, R.J., Fearing, R.S.: Halteres for the micromechanical flying insect. Proceedings of the IEEE International Conference on Robotics and Automation, ICRA 2002 1, 60–65 (2002) 62. Zufferey, J.C., Floreano, D.: Fly-inspired visual steering of an ultralight indoor aircraft. IEEE Transactions on Robotics 22(1), 137–146 (2006)

Chapter 2

From Visual Guidance in Flying Insects to Autonomous Aerial Vehicles Mandyam V. Srinivasan, Saul Thurrowgood, and Dean Soccol

Abstract Investigation of the principles of visually guided flight in insects is offering novel, computationally elegant solutions to challenges in machine vision and robot navigation. Insects perform remarkably well at seeing and perceiving the world and navigating effectively in it, despite possessing a brain that weighs less than a milligram and carries fewer than 0.01% as many neurons as ours does. Although most insects lack stereovision, they use a number of ingenious strategies for perceiving their world in three dimensions and navigating successfully in it. Over the past 20 years, research in our laboratory and elsewhere is revealing that flying insects rely primarily on cues derived from image motion (“optic flow”) to distinguish objects from backgrounds, to negotiate narrow gaps, to regulate flight speed, to compensate for headwinds and crosswinds, to estimate distance flown and to orchestrate smooth landings. Here we summarize some of these findings and describe a vision system currently being designed to facilitate automated terrain following and landing.

2.1 Introduction Insect eyes differ from vertebrate or human eyes in a number of ways. Unlike vertebrates, insects have immobile eyes with fixed-focus optics. Therefore, they

M.V. Srinivasan () Queensland Brain Institute and ARC Centre of Excellence in Vision Science, University of Queensland, St. Lucia, QLD 4072, Australia e-mail: [email protected]

cannot infer the distances to objects or surfaces from the extent to which the directions of gaze must converge to view the object, or by monitoring the refractive power that is required to bring the image of the object into focus on the retina. Furthermore, compared with human eyes, the eyes of insects are positioned much closer together and possess inferior spatial acuity [16]. Therefore, the precision with which insects could estimate range through binocular stereopsis would be much poorer and restricted to relatively small distances, even if they possessed the requisite neural apparatus [15]. Not surprisingly, then, insects have evolved alternative strategies for dealing with the problems of visually guided flight. Many of these strategies rely on using image motion, generated by the insect’s own motion, to infer the distances to obstacles and to control various manoeuvres (Franceschini, Chap. 3 of this volume) [10, 16, 21, 22]. This pattern of image motion, known as “optic flow”, is used in many ways to guide flight. For example, distances to objects are gauged in terms of the apparent speeds of motion of the objects’ images, rather than by using complex stereo mechanisms [12, 17, 26]. Objects are distinguished from backgrounds by sensing the apparent relative motion at the boundary [18]. Narrow gaps are negotiated safely by balancing the apparent speeds of the images in the two eyes [11, 19]. The speed of flight is regulated by holding constant the average image velocity as seen by both eyes [1, 5, 27]. This ensures that flight speed is automatically lowered in cluttered environments and that thrust is appropriately adjusted to compensate for headwinds and tail winds [2, 27]. Visual cues are also used to compensate for crosswinds. Bees landing on a horizontal surface hold constant the image velocity of the surface

D. Floreano et al. (eds.), Flying Insects and Robots, DOI 10.1007/978-3-540-89393-6_2, © Springer-Verlag Berlin Heidelberg 2009

15

16

as they approach it, thus automatically ensuring that flight speed is close to zero at touchdown [24]. Foraging bees gauge distance flown by integrating optic flow: they possess a visually driven “odometer” that is robust to variations in wind, body weight, energy expenditure and the properties of the visual environment [4, 6, 7, 23]. In this chapter we concentrate on two aspects of visually guided navigation in flying insects, which are heavily reliant on optic flow. One relates to landing on a horizontal surface. The other concerns a related behaviour, terrain following, which involves maintaining a constant height above the ground and following the local fluctuations of terrain altitude.

2.2 Landing on a Horizontal Surface We begin by summarizing the results of experiments that were conducted a few years ago in our laboratory to elucidate the visual mechanisms that guide landing in honeybees. How does a bee land on a horizontal surface? When an insect makes a grazing landing on a flat surface, the dominant pattern of image motion generated by the surface is a translatory flow in the front-toback direction. What are the processes by which such landings are orchestrated? Srinivasan et al. [24] investigated this question by video-filming trajectories, in three dimensions, of bees landing on a flat, horizontal surface. Two examples of landing trajectories, reconstructed from the data, are shown in Fig. 2.1a,b. A number of such landings were analysed to examine the variation of the instantaneous height above the surface (h), instantaneous horizontal (forward) flight speed (Vf ), instantaneous descent speed (Vd ) and descent angle (α). These variables are illustrated in Fig. 2.1c. Analysis of the landing trajectories revealed that the descent angles were indeed quite shallow. The average value measured in 26 trajectories was ca. 28◦ [24]. Figure 2.2a,b shows the variation of flight speed with height above the surface, analysed for two landing trajectories. These data reveal one of the most striking and consistent observations of this study: Horizontal speed is roughly proportional to height, as indicated by the linear regression on the data. When a bee flies at a horizontal speed of Vf (cm/s) at a height of h (cm), the angular velocity ω of the image of the surface directly

M.V. Srinivasan et al.

beneath the eye is given by ω=

Vf rad/s h

(2.1)

From this relationship it is clear that, if the bee’s horizontal flight speed is proportional to her height above the surface (as shown by the data), then the angular velocity of the image of the surface, as seen by the eye, must be constant as the bee approaches it. This angular velocity is given by the slope of the regression line. The angular velocity of the image varies from one trajectory to another, but is maintained at an approximately constant value in any given landing. An analysis of 26 landing trajectories revealed a mean image angular velocity of ca. 500◦ /s [24]. These results reveal two important characteristics. First, bees landing on a horizontal surface tend to approach the surface at a relatively shallow descent angle. Second, landing bees tend to hold the angular velocity of the image of the ground constant as they approach it. What is the significance of holding the angular velocity of the image of the ground constant during landing? One important consequence is that the horizontal speed of flight is then automatically reduced as the height decreases. In fact, by holding the image velocity constant, the horizontal speed is regulated to be proportional to the height above the ground, so that when the bee finally touches down (at zero height), her horizontal speed is zero, thus ensuring a smooth landing. The attractive feature of this simple strategy is that it does not require explicit measurement or knowledge of the speed of flight or the height above the ground. Thus, stereoscopic methods of measuring the distance of the surface (which many insects probably do not possess) are not required. What is required, however, is that the insect be constantly in motion, because the image motion resulting from the insect’s own motion is crucial in controlling the landing. The above strategy ensures that the bee’s horizontal speed is zero at touchdown, but does not regulate the descent speed. How does the descent speed vary during the landing process? Plots of descent speed versus height reveal a linear relationship between these two variables, as well. Two examples are shown in Fig. 2.2c,d. This finding implies that landing bees (i) adjust their forward (i.e. flight) speed to hold the

2

Flying Insects and Autonomous Aerial Vehicles

17

Fig. 2.1 (a, b) Three-dimensional reconstruction of two typical landing trajectories, from video films. Vertical lines depict the height above surface. (c) Illustration of some of the variables analysed to investigate the control of landing. h (cm): height

above surface; Vf (cm/s): horizontal (forward) flight speed; Vd (cm/s): vertical (descent) speed; α (deg or rad): descent angle [α=(tan-1 (Vf /Vd )]. Adapted from [24]

image velocity of the ground constant and (ii) couple the descent speed to the forward speed, so that the descent speed decreases with the forward speed and also becomes zero at touchdown. These results reveal what appears to be a surprisingly simple and effective strategy for making grazing landings on flat surfaces. A safe, smooth landing is ensured by following two simple rules: (a) adjusting the speed of forward flight to hold constant the angular velocity of the image of the surface as seen by the eye and (b) making the speed of descent proportional to the forward speed, i.e. flying at a constant descent angle. This produces landing trajectories in which the forward speed and the descent speed decrease progressively as the surface is approached, both approaching zero at touchdown. What are the advantages, if any, of using this landing strategy? We can think of three attractive features. First, the strategy is very simple because it does not

require explicit knowledge of instantaneous height or flight speed. Second, forward and descent speeds are adjusted in such a way as to hold the image velocity constant. This is advantageous because the image velocity can then be maintained at a level at which the visual system is most sensitive to deviations from the “set” velocity, thereby ensuring that the control of flight is as precise as possible. An alternative strategy, for instance, might be to approach the surface at constant flight speed, decelerating only towards the end. Such a constant speed approach, however, would cause the image velocity to increase rapidly as the surface is approached and to reach levels at which the image velocity measurements may no longer be precise enough for adequate flight control. This situation would be avoided by the bee’s landing strategy, which holds the image velocity constant. Third, an interesting by-product of the bee’s landing strategy is that the projected time to touchdown is constant throughout the

18

M.V. Srinivasan et al.

Fig. 2.2 (a, b) Variation of horizontal flight speed (Vf ) with height (h) above the surface for two different landing trajectories. (c, d) Variation of descent speed (Vd ) with height (h) above the surface for two different landing trajectories. The

straight lines are linear regressions through the data, as represented by the equations; r denotes the regression coefficient. Adapted from [24]

landing process (details in [24]). In other words, if, at any time during the landing process, the bees were to stop decelerating and continue downward at constant velocity, the time to contact the ground would be the same, regardless of where this occurs in the landing trajectory. From the landing trajectories, one calculates a projected time to touchdown of about 0.22 s. Thus, it appears that landing bees allow themselves a “safety margin” of a fifth of a second to prepare for touchdown if they were to abandon the prescribed landing strategy at any point, for whatever reason, and proceed towards the ground in the same direction without further deceleration.

the ground and to fly parallel to the surface by following any fluctuations of height (terrain following). The strategy would now be not to fly towards a target on the ground, but instead towards a distant target or the horizon. This will ensure a level flight attitude. Flight speed is held constant by maintaining a constant forward thrust (which should be effective, at least in still air). The height above the ground is regulated by adjusting the altitude so that the magnitude of the optic flow generated by the ground is constant. If the forward flight speed is Vf and the desired height above the ground is h, the magnitude of the optic flow that is to be maintained is given by Eq. (2.1) above as simply the ratio of Vf to h. If the measured optic flow is greater than this target value, it signifies that the altitude is lower than the desired value and a control command is then generated to increase altitude until the target magnitude of optic flow is attained. If the optic flow is lower than the target value, the opposite control command is issued, to again restore the optic flow to its desired level. This strategy has been tested successfully in simulations, in tethered robots in the laboratory and in some

2.3 Terrain Following A simple modification of the landing strategy described above can be used, in principle, to regulate the height above the ground during cruising flight. Here the aim is to maintain a constant height above

2

Flying Insects and Autonomous Aerial Vehicles

freely flying model aircraft (Franceschini, Chap. 3 of this volume; Zufferey et al., Chap. 6 of this volume) [3, 13, 14, 25, 28].

19 A’ η

O

r

2.4 Practical Problems with the Measurement of Optic Flow When flying at low altitudes or during landing, the image of the ground can move very rapidly, making it difficult to obtain accurate estimates of optic flow. Here we describe a specially shaped mirror surface that, first, scales down the speed of image motion as seen by the camera and, second, removes the perspective distortion (and therefore the distortion in image velocity) that a camera experiences when viewing a horizontal plane that stretches out to infinity in front of the aircraft. Ideally, the moving image that is captured by the camera through the mirror should exhibit a constant, low velocity everywhere, thus simplifying the optic flow measurements and increasing their accuracy (see Fig. 2.3).

Video camera

Reflective surface

Ground plane

Fig. 2.3 Illustration of a system for visually guided terrain following and landing. The optical system is shown on an enlarged scale relative to the aircraft in order to illustrate its configuration

2.5 A Mirror-Based Vision System for Terrain Following and Landing We seek a mirror profile that will map equal distances along the ground in the flight direction to equal displacements in the image plane of the camera. The geometry of the camera–mirror configuration is shown in Fig. 2.4. A is the camera image of a point A on the ground. We want A (the image of A) to move at a con-

Camera image

Mirror surface

θ

f

Camera nodal point

Ground

P

β

A

Fig. 2.4 Geometry of camera/mirror configuration for the design of mirror profile. Modified from [20]

stant velocity in the image plane of the camera, independent of the position of A in the ground plane. That is, we require dη dt = L, where η is the distance of the point A from the centre of the camera’s image plane, and L is the desired constant velocity; f is the focal length of the camera. The derivation of a mirror profile that meets these objectives is given in [20] and we shall not repeat it here. Instead, we use an example profile to illustrate the performance of the mirror. Figure 2.5 shows one example of a profile of the reflective surface and includes the computed ray paths. In this example the camera faces forward, in the direction of flight. The nodal point of the camera is at (0.0). The image plane of the camera is to the left of this point and is not included in the figure, but its reflection about the nodal point (an equivalent representation) is depicted by the vertical line to the right of the nodal point. Parameters used in this design are as follows: V = 1000.0 cm/s; h = 100.0 cm; r0 = 10.0 cm; f = 3.5 cm; L=2.0 cm/s, where V is the speed of the aircraft, h is the height above the ground, L is the image velocity, f is the focal length of the camera and r0 is the distance from the nodal point of the camera to the tip of the reflective surface. If the camera was looking directly downwards at the ground, the image velocity of the ground would have been 35 cm/s; with the mirror, the image velocity is reduced to 2.0 cm/s. Thus, the mirror scales down the image velocity by a factor of 17.5. The curvature of the mirror is highest in the region that images the ground directly beneath the aircraft, because this is the region of the ground that moves at the highest angular velocity with respect to the camera,

20

M.V. Srinivasan et al.

Fig. 2.5 Example of computed mirror profile. Adapted from [20]

5

Camera nodal point Mirror profile

y

0

–5

–10

Forward –15 5

10

15

20

25

x

and which therefore requires the greatest reduction of motion. We see from Fig. 2.5 that equally spaced points on the ground along the line of flight map to equally spaced points in the camera image. This confirms the correct operation of the surface. Figure 2.6a illustrates the imaging properties of the mirror, positioned with its axis parallel to and above a plane carrying a checkerboard pattern. Note that the mirror has removed the perspective distortion (foreshortening) of the image of the plane. The scale of the mapping depends upon the radial direction. Compression is lowest in the vertical radial direction and highest in the horizontal radial direction. Figure 2.6b shows a digitally remapped view of the image in Fig. 2.6a, in which the polar co-ordinates of each pixel in Fig. 2.6a are plotted as Cartesian coordinates. Here the vertical axis represents radial distance from the centre of the image of Fig. 2.6a and the horizontal axis represents the angle of rotation about

Fig. 2.6 (a) Illustration of imaging properties of mirror and (b) remapped version of image in Fig. 2.6a. Adapted from [20]

a

the optic axis. The circle in Fig. 2.6a maps to the horizontal line in Fig. 2.6b. Regions below the line represent areas in front of the aircraft and regions above it represent areas behind. Thus, the mirror endows the camera with a large field of view that covers regions in front of, below and behind the aircraft. A consequence of the geometrical mapping produced by the mirror is that, for straight and level flight parallel to the ground plane, the optic flow vectors will have constant magnitude along each radius but will be largest along the vertical radius and smallest (zero) along the horizontal radius. This is illustrated in Fig. 2.7, which shows the optic flow vectors that are generated by a simulated level flight over an ocean. Figure 2.7a shows the flow field in the raw image and Fig. 2.7b the flow field in the remapped image. In the remapped image, all flow vectors are oriented vertically. The magnitudes of the vectors are constant within each column and

b

2

Flying Insects and Autonomous Aerial Vehicles

21

Fig. 2.7 Optic flow field generated by simulated level flight over an ocean. (a) The flow in the raw image and (b) the flow in the remapped image. Adapted from [15]

a

b

decrease progressively with increasingly lateral directions of view. These vectors provide information on the height above the ground, the topography of the ground and the ranges to objects in the field of view (which is quite large). If the system undergoes pure translation along its optic axis, as in Fig. 2.5, the magnitude of the optic flow vector at each point on the unwarped image will be inversely proportional to the distance of the viewed point from the optic axis of the system. Thus, the flow vectors will provide information on the profile of the terrain in a cylindrical co-ordinate system relative to the aircraft. The mapping that is provided by the mirror should be particularly useful for aircraft guidance. If a cylinder of “clear space” is desired for obstaclefree flight along a given trajectory, the maximum permissible flow magnitude is determined by the speed of the aircraft and the radius R of this cylinder (see Fig. 2.8). This simplifies the problem of determining in advance whether an intended flight trajectory through the environment will be collision-free and of making any necessary adjustments to the trajectory to ensure safe flight. The system will provide information on the height above the ground, as well as the distance of any potential obstacles, as measured from the optical axis.

2.6 Height Estimation and Obstacle Detection During Complex Motions If the aircraft undergoes rotation as well as translation, the optic flow vectors that are induced by the rotational components of aircraft motion will contaminate the range measurement. They must be subtracted from the total optic flow field, to obtain a residual flow field that represents the optic flow that is created only by the translational component of aircraft motion. The rotations of the aircraft can be estimated through gyroscopic signals. Since each component of rotation (yaw, roll, pitch) produces a known, characteristic pattern of optic flow which depends on the magnitude of the rotation but is independent of the ranges of objects or surfaces in the environment, the optic flow vector fields that are generated by the rotations can be predicted computationally as a sum of the characteristic optic flow fields for yaw, pitch and roll, each weighted by the magnitude of the measured rotation about the corresponding axis. This composite rotational flow field must then be subtracted from the total flow field to obtain a residual flow field that represents the flow due to just the translational component of motion. The

Collision-free radius R

Fig. 2.8 Illustration of collision-free cylinder mapping achieved by the system. Modified from [15]

Vision system

Flight axis Maximum permissible flow magnitude in mirror image

22

M.V. Srinivasan et al.

distances to objects and surfaces from the optical axis can then be computed from this residual field.

2.7 Hardware Realization and System Tests The mirror profile shown in Fig. 2.5 was machined in aluminium on a numerically controlled lathe. It was mounted on a bracket, which also carried an analogue video CCD camera (320 × 240 pixels) with its optical axis aligned with the axis of the mirror. The nodal point of the camera’s lens was positioned 10 cm from the tip of the mirror, as per the design specifications (see above). In order to extract the range of objects during complex motions, it is necessary to first determine the flow signatures, or “templates”, that characterise the patterns of optic flow during pure yaw, pure roll and pure pitch. This was done by using a robotic gantry to move the vision system in a richly textured visual environment. The environment consisted of a rectangular arena 3.05 m long, 2.2 m wide and 1.13 m tall (Fig. 2.9a). The walls and floor of the arena were lined with a texture composed of black circles of five different diameters (150 mm, 105 mm, 90 mm, 75 mm and 65 mm) on a white background. The rich visual texture permitted dense and accurate measurements of the optic flow in the lower hemisphere of the visual field. A raw image of the arena, as acquired by the system, is shown in Fig. 2.10a. An unwarped version of this image is shown in Fig. 2.10b. The gantry was used to position the optical axis of the system at a height of 650 cm above the floor. Optic

Fig. 2.9 (a) View of vision system, carried by a robotic gantry in a visually textured arena and (b) plan view of a curved trajectory used to test the system. Adapted from [15]

Mirror

a

flow templates for yaw, pitch and roll were obtained for the remapped images by using the gantry to rotate the vision system by small, known angles (ranging from 0.25◦ to 2.5◦ , in steps of 0.25◦ ) about each of the three axes, in turn. Measurements were repeated with the vision system positioned at several different locations in the arena and the results were pooled and normalized to obtain reliable and dense estimates of the optic flow templates for a 1◦ rotation. (In theory, the rotational optic flow templates should be independent of the position or attitude of the vision system within the arena.) The optic flow was computed using a correlation algorithm [8]. The resulting rotational templates for yaw, roll and pitch are shown in Fig. 2.11, for rotations of 1◦ in each case. For small angular rotations, the magnitude of the flow vector at any given point in the visual field should increase approximately linearly with the magnitude of the rotation, and the direction of the flow vector should remain approximately constant. This has been verified in [15]. Thus, we can legitimately scale the optic flow templates for the measured rotations in yaw, pitch and roll, in order to subtract out the contributions of each of these rotational components in a flow field that is generated by complex motion. The next step was to test performance when the system executed compound motions that combined translation and rotation. The aim was to investigate whether the system could determine the range to objects in the environment (and specifically, the height above the ground) while executing complex motions. This investigation was performed by using the gantry to move the system along a trajectory along a curved trajectory, as shown in Fig. 2.9b.

Camera

b

2

Flying Insects and Autonomous Aerial Vehicles

23

Fig. 2.10 Raw (a) and unwarped (b) images of the arena as viewed by the system. Adapted from [15]

a

b

Fig. 2.11 Measured rotational templates for yaw, roll and pitch

Yaw

Roll

Pitch The height of the system was held constant at 650 mm above the floor throughout the trajectory. The trajectory consisted of a sequence of stepwise motions in the horizontal plane. The optical axis of the system was always aligned along the instantaneous direction of translation, i.e. it was parallel to the local tangent to the trajectory. Each step, in general, consisted of an elementary translation (of 50 mm) along the optical axis of the vision system, followed by an elementary yaw rotation of known, but variable, magnitude (ranging from +2.9◦ to –1.5◦ ). A visual frame was acquired from the camera at the end of each elementary step (translation or rotation). Figure 2.12a shows the optic flow generated by a compound step, consisting of an elementary translation followed by an elementary rotation (yaw). This is the

flow that would be generated between two successive frames if the system had moved along a smooth curve. Figure 2.12b shows the flow that would have been induced by the rotational component of motion that occurred during this compound step. This flow pattern was computed by weighting the template for yaw rotation by the (known) magnitude and polarity of the yaw that occurred during the compound step. (During flight, this yaw information would be obtained from a rate gyro.) Figure 2.13a shows the optic flow that is obtained when the rotational component of the flow (from Fig. 2.12b) is subtracted from the total flow that is generated by the compound step (Fig. 2.12a). This residual flow should represent the flow that is induced solely by the translational component of the system’s motion. It

24

M.V. Srinivasan et al.

Fig. 2.12 (a) Flow measured after a compound step consisting of translation followed by yaw and (b) flow induced by the rotational (yaw) component, calculated from the template for yaw. Adapted from [15]

a

a

b

b

Fig. 2.13 (a) Residual flow obtained by subtracting the yaw-induced flow (Fig. 2.12b) from the measured flow (Fig. 2.12a). (b) Flow induced by the translatory component of motion in the compound step. Adapted from [15]

is evident that all the vectors in the residual flow field are parallel and vertical, as would be expected during pure translation. Furthermore, this pattern of optic flow is in excellent agreement with the pattern of flow that is actually generated by pure translation at this par-

Fig. 2.14 Results of extending the de-rotation procedure to roll as well as yaw. The figure shows one frame of a motion sequence in which the vision system executed a trajectory involving translation, yaw and roll. The upper left panel shows the raw image. The upper right panel shows one frame of the unwarped image and the instantaneous raw optic flow. The lower right panel shows the result of de-rotation, by using the roll and yaw templates to subtract the flow components induced by the measured roll and yaw

ticular location in the arena. This latter flow pattern, shown in Fig. 2.13b, is the flow measured between the two frames that bracketed the translatory segment of the compound step. A comparison of Fig. 2.13a,b reveals that these flow patterns are virtually identical.

Flying Insects and Autonomous Aerial Vehicles

This result, which was obtained consistently for all the compound steps in the trajectory, demonstrates that the pattern of optic flow that is generated during a small but complex motion can be successfully “de-rotated” to extract the optic flow that is created purely by the translatory component of motion that occurred during the period. The results of extending this de-rotation procedure to rotations about two axes – yaw and roll – are summarized in Fig. 2.14. Here again, after subtracting the optic flow components induced by yaw and roll, the residual optic flow is purely translatory, thus demonstrating the validity of the de-rotation procedure.

25

a 12

10

Flow magnitude (pixels)

2

8

6

4

2

0 –50 –40 –30 –20 –10

2.8 Extracting Information on Range and Topography

0

10

20

30

40

50

Lateral viewing angle (deg)

b The final step is to examine whether the residual component of the optic flow (the translation-induced component) can be used to obtain accurate information on the range and profile of the terrain over which flight occurs. With reference to Figs. 2.7b and 2.13, for horizontal flight over level ground, the magnitudes of the translation-induced optic flow vectors should be a maximum in the central column (corresponding to the ground directly beneath the aircraft) and should fall off as a cosine function of the lateral angle of view (in the columns to the left and right; [20]). This should be true for any row of vectors. This prediction is tested in Fig. 2.15a, which shows the variation of translatory flow magnitude with lateral angle for flow vectors in any given row, in data sets such as that shown in Fig. 2.13a. The results show the magnitude profiles for the vectors in the second row from the bottom, computed for each step of the trajectory. The second row represents a view that is oriented at approximately 90◦ to the axial direction (a lateral view). A least-squares analysis reveals that each of the profiles approximates a cosine function quite well. The thick red curve shows the mean of the cosine functions fitted to each of the profiles obtained along the trajectory. If the speed of the aircraft is known, the amplitude of this curve provides information on the height above the ground (the amplitude is inversely proportional to height). The mean amplitude of the curves is

Fig. 2.15 (a) Variation of flow magnitude with lateral viewing angle in the residual flow field at each of the steps along the trajectory of Fig. 2.9b. Adapted from [15]. (b) Magnitudes of the de-rotated optic flow computed along the entire trajectory of a motion path involving translation and roll

9.076 pixels and the standard deviation is 0.062 pixels, indicating that the estimate of flight height is consistent and reliable throughout the trajectory of Fig. 2.9b. Figure 2.15b shows the magnitudes of the de-rotated optic flow computed along the entire trajectory of a complex motion path involving translation and roll. It

26

is evident that, in each frame, the profile of the optic flow magnitude closely approximates a cosine function. The peak of the function wanders to the left or the right from frame to frame because the vision system is undergoing roll as well as translation, and the optic flow data are plotted in relation to the co-ordinates of the vision system.

2.9 Preliminary Flight Tests Figure 2.16 shows a view of a prototype of the vision system mounted on the underside of a model aircraft. Figure 2.17 shows views of the raw camera image

M.V. Srinivasan et al.

(left) and the unwarped image (right) with the computed optic flow vectors. The horizon (in the raw image as well as the unwarped image) distorts in predictable ways depending upon the roll and pitch attitudes of the aircraft. The horizon profile can thus be used to estimate the roll and pitch of the aircraft, when flying at high altitudes over reasonably flat terrain. Figure 2.18a shows one frame of the unwarped camera image during a test flight, with the computed raw optic flow field. Figure 2.18b shows the same frame, with optic flow computed after de-rotation in yaw, pitch and roll, using information from the aircraft’s gyroscopes. The optic flow vectors in the de-rotated field are all very close to vertical, indicating that the de-rotation procedure is successful and accurate.

2.10 Conclusions and Discussion

Fig. 2.16 View of vision system mounted on the underside of a model aircraft

This study has described the design of a vision sensor, based partly on principles of insect vision and optic flow analysis, for the measurement and control of flight height and for obstacle avoidance. A video camera is used in conjunction with a specially shaped reflective surface to simplify the computation of optic flow and extend the range of aircraft speeds over which accurate data can be obtained. The imaging system also provides a useful geometrical remapping of the environment, which facilitates obstacle avoidance and

Fig. 2.17 Test flight. Views of the raw camera image (left) and the unwarped image (right) with the computed optic flow vectors

2

Flying Insects and Autonomous Aerial Vehicles

27

Fig. 2.18 Test flight. (a) One frame of unwarped camera image, with computed raw optic flow field. (b) Same frame, with optic flow computed after de-rotation in yaw, pitch and roll

a computation of three-dimensional terrain maps. By using calibrated optic flow templates for yaw, roll and pitch, accurate range information can be obtained even when the aircraft executes complex motions. In principle, the vision system described here can be used in the platforms described in (Franceschini, Chap. 3 of this volume Zufferey et al., Chap. 6 of this volume) to facilitate vision guidance during landing or when flying at high speeds close to obstacles or close to the ground. Future work with this system will involve open-loop and closed-loop tests on flying vehicles, to examine the utility of this approach for autonomous guidance. Acknowledgements This work was supported partly by the US Army Research Office MURI ARMY-W911NF041076, Technical Monitor Dr Tom Doligalski, US ONR Award N00014-041-0334, an ARC Centre of Excellence Grant CE0561903 and a Queensland Smart State Premier’s Fellowship to MVS.

References 1. Baird, E., Srinivasan, M.V., Zhang, S.W., Cowling, A.: Visual control of flight speed in honeybees. The Journal of Experimental Biology 208, 3895–3905 (2005) 2. Barron, A., Srinivasan, M.V.: Visual regulation of ground speed and headwind compensation in freely flying honey bees (Apis mellifera L.). Journal of Experimental Biology 209, 978–984 (2006) 3. Barrows, G.L., Chahl, J.S., Srinivasan, M.V.: Biologically inspired visual sensing and flight control. The Aeronautical Journal, London: The Royal Aeronautical Society 107(1069), 159–168 (2003) 4. Dacke, M., Srinivasan, M.V.: Honeybee navigation: Distance estimation in the third dimension. Journal of Experimental Biology 210, 845–853 (2007) 5. David, C.T.: Compensation for height in the control of groundspeed by Drosophila in a new, “Barber’s Pole” wind tunnel. Journal of Comparative Physiology 147, 485–493 (1982)

b

6. Esch, H., Burns, J.E.: Honeybees use optic flow to measure the distance of a food source. Naturwissenschaften 82, 38– 40 (1995) 7. Esch, H., Zhang, S.W., Srinivasan, M.V., Tautz, J.: Honeybee dances communicate distances measured by optic flow. Nature (London) 411, 581–583 (2001) 8. Fua, P.: A parallel stereo algorithm that produces dense depth maps and preserves image features. Machine Vision and Applications 6(1), 35–49, December (1993) 9. Horridge, G.A.: Insects which turn and look. Endeavour N.S. 1, 7–17 (1977) 10. Horridge, G.A.: The evolution of visual processing and the construction of seeing systems. Proceedings of the Royal Society of London. Series B 230, 279–292 (1987) 11. Kirchner, W.H., Srinivasan, M.V.: Freely flying honeybees use image motion to estimate object distance. Naturwissenschaften 76, 281–282 (1989) 12. Lehrer, M., Srinivasan, M.V., Zhang, S.W., Horridge, G.A.: Motion cues provide the bee’s visual world with a third dimension. Nature (London) 332, 356–357 (1988) 13. Neumann, T.R., Bülthoff, H.: Insect inspired visual control of translatory flight. In: Kelemen J, Sosik P (eds.) Proceedings of the ECAL 2001, pp. 627–636. Springer, Berlin (2001) 14. Ruffier, F., Franceschini, N.: Optic flow regulation: the key to aircraft automatic guidance. Robotics and Autonomous Systems 50, 177–194 (2005) 15. Soccol, D., Thurrowgood, S., Srinivasan, M.V., A vision system for optic-flow-based guidance of UAVs. Proceedings, Ninth Australasian Conference on Robotics and Automation. Brisbane, 10–12 December (2007) 16. Srinivasan, M.V.: How insects infer range from visual motion. In: F.A. Miles, J. Wallman (eds.) Visual Motion and its Role in the Stabilization of Gaze. Elsevier, Amsterdam, pp. 139–156 (1993) 17. Srinivasan, M.V., Lehrer, M., Zhang, S.W., Horridge, G.A.: How honeybees measure their distance from objects of unknown size. Journal of Comparative Physiology A 165, 605–613 (1989) 18. Srinivasan, M.V., Lehrer, M., Horridge, G.A.: Visual figureground discrimination in the honeybee: the role of motion parallax at boundaries. Proceedings of the Royal Society of London. Series B 238, 331–350 (1990) 19. Srinivasan, M.V., Lehrer, M., Kirchner, W., Zhang, S.W.: Range perception through apparent image speed in freely-flying honeybees. Visual Neuroscience 6, 519–535 (1991)

28 20. Srinivasan, M.V., Thurrowgood, S., Soccol, D.: An optical system for guidance of terrain following in UAVs. Proceedings, IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS ’06), Sydney, 51–56 (2006) 21. Srinivasan, M.V., Zhang, S.W.: Visual control of honeybee flight. In: M. Lehrer (ed.) Orientation and Communication in Arthropods, pp. 67–93. Birkhäuser Verlag, Basel (1997) 22. Srinivasan, M.V., Zhang, S.W.: Visual navigation in flying insects. In: M. Lappe (ed.) International Review of Neurobiology, Vol. 44, Neuronal Processing of Optic Flow, pp. 67–92. Academic Press, San Diego (2000) 23. Srinivasan, M.V., Zhang, S.W., Altwein, M., Tautz, J.: Honeybee navigation: nature and calibration of the ‘odometer’. Science 287, 851–853 (2000) 24. Srinivasan, M.V., Zhang, S.W., Chahl, J.S., Barth, E., Venkatesh, S.: How honeybees make grazing landings on flat surfaces. Biological Cybernetics 83, 171–183 (2000)

M.V. Srinivasan et al. 25. Srinivasan, M.V., Zhang, S.W., Chahl, J S, Stange, G, Garratt, M.: An overview of insect inspired guidance for application in ground and airborne platforms. Proceedings of the Institution of Mechanical Engineers. Part G, Journal of Aerospace Engineering 218, 375–388 (2004) 26. Srinivasan, M.V., Zhang, S.W., Chandrashekara, K.: Evidence for two distinct movement-detecting mechanisms in insect vision. Naturwissenschaften 80, 38–41 (1993) 27. Srinivasan, M.V., Zhang, S.W., Lehrer, M., Collett, T.S.: Honeybee navigation en route to the goal: visual flight control and odometry. The Journal of Experimental Biology 199, 237–244 (1996) 28. Zufferey, J.C., Floreano, D.: Fly-inspired visual steering of an ultralight indoor aircraft. IEEE Transactions on Robotics 22, 137–146 (2006)

Chapter 3

Optic Flow Based Autopilots: Speed Control and Obstacle Avoidance Nicolas Franceschini, Franck Ruffier and Julien Serres

Abstract The explicit control schemes presented here explain how insects may navigate on the sole basis of optic flow (OF) cues without requiring any distance or speed measurements: how they take off and land, follow the terrain, avoid the lateral walls in a corridor, and control their forward speed automatically. The optic flow regulator, a feedback system controlling either the lift, the forward thrust, or the lateral thrust, is described. Three OF regulators account for various insect flight patterns observed over the ground and over still water, under calm and windy conditions, and in straight and tapered corridors. These control schemes were simulated experimentally and/or implemented onboard two types of aerial robots, a micro-helicopter (MH) and a hovercraft (HO), which behaved much like insects when placed in similar environments. These robots were equipped with optoelectronic OF sensors inspired by our electrophysiological findings on houseflies’ motion-sensitive visual neurons. The simple, parsimonious control schemes described here require no conventional avionic devices such as rangefinders, groundspeed sensors, or GPS receivers. They are consistent with the neural repertory of flying insects and meet the low avionic payload requirements of autonomous micro-aerial and space vehicles.

N. Franceschini () Biorobotics Lab, Institute of Movement Science, CNRS & Univ. of the Mediterranean, Marseille, France e-mail: [email protected]

3.1 Introduction When an insect is flying forward, an image of the ground texture and any lateral obstacles scrolls backward across the ommatidia of its compound eye. This flowing image set up by the animal’s own forward motion is called the optic flow (OF). Recent studies have shown that freely flying insects use the OF to avoid collisions [94, 11, 88], follow a corridor [53, 83, 3, 4, 80], cruise, and land [93, 84, 78, 86]. The OF can be described as a vector field where each vector gives the direction and magnitude of the angular velocity at which any point in the environment moves relative to the eye [54]. Several authors have attempted to model the control systems at work in insects during free flight, focusing on specific aspects such as speed control [15, 84], distance or speed servoing [55, 17, 20], course control, and saccadic flight behavior (Egelhaaf et al., Chap. 4 of this volume) [39, 66]. Freely flying flies navigate by making pure translations alternating with fast, saccade-like turns [39, 94, 11, 92]. This idiosyncratic flight behavior was interpreted [94, 60, 11] as an active means of reducing the image flow to its translational components (that depends on the distances to objects [54]). A biorobotic project was launched in the mid-1980s to investigate how a fly could possibly navigate and avoid collisions based on OF cues. The prototype Robot-Fly (“robot mouche”) that we developed [60, 26] was a 50-cm high, fully autonomous wheeled robot carrying a compound eye driving 114 OF sensors with an omnidirectional azimuthal field of view (FOV). This reactive robot sensed the translational OF while moving straight ahead until detecting an obsta-

D. Floreano et al. (eds.), Flying Insects and Robots, DOI 10.1007/978-3-540-89393-6_3, © Springer-Verlag Berlin Heidelberg 2009

29

30

cle, which triggered a quick eye and body turn of a suitable amplitude (during which time vision was inhibited). Since the Robot-Fly traveled at a constant speed (50 cm/s), the OF measured during any translation easily gave the object range, and the robot was thus able to dodge and slalom to its target lamp through a random array of posts [26]. Despite the success of this early neuromimetic robot, flying insects would obviously have to use OF cues differently, since they are not in mechanical contact with the ground and cannot therefore easily estimate their groundspeed. How might flies, or micro-aerial vehicles (MAVs) cope with the severe disturbances (obstacles, wind, etc.) they encounter? Here we summarize our attempts to model the visuomotor control systems that provide flying insects with a means of close-range autonomous piloting. First, we focus on ground avoidance in the vertical (longitudinal) plane (Sect. 3.3). We then discuss the ability to avoid corridor walls (Sect. 3.4), which has been closely analyzed in honeybees. Independent vertical and horizontal flight control systems (Egelhaaf et al., Chap. 4 of this volume) (see also [59]) were suggested by the performance of flies, which control their movements along the three orthogonal axes independently [94], while keeping their head transiently fixed in space [92, 96] via compensatory head roll and pitch mechanisms [40, 41]. Experimental simulations were performed and our control schemes were tested on two fly-by-sight aerial robots: a micro-helicopter (MH) (Fig. 3.5a) and a miniature hovercraft (HO) (Fig. 3.8a). These aerial robots use neuromorphic OF sensors [7, 25, 8] inspired by the elementary motion detectors (EMDs) previously studied at our laboratory in houseflies [67, 22, 27]. These sensors are briefly described in Sect. 3.2.

3.2 From the Fly EMDs to Electronic Optic Flow Sensors Conventional cameras produce images at a given frame rate. Each image is scanned line by line at a high frequency. Many authors working in the field of computer vision have presented algorithms for analyzing the OF field based on scanned camera images. Although an

N. Franceschini et al.

OF algorithm has been implemented onboard a highly miniaturized, slow but fully autonomous indoor MAV (Zufferey et al., Chap. 6 of this volume), none of the OF algorithms to be found in the insect brain actually start with a retinal image scanned like a television image. Insects analyze the OF locally, pixel by pixel, via a neural circuit called an “elementary motion detector” (EMD). Further down the neural pathways, well-known collator neurons called “lobula plate tangential cells” (LPTCs) integrate the outputs of large numbers of EMDs and analyze the OF field generated by the animal’s locomotion. Some of them transmit electrical signals via the neck to thoracic interneurons directly or indirectly responsible for driving the wing, leg muscles, or head muscles. Other LPTCs send relevant signals to the contralateral eye (see [87, 37, 9, 90]). To determine the functional principles underlying an EMD, the responses of an LPTC neuron were recorded (H1, Fig. 3.1b) while two neighboring photoreceptors in a single ommatidium were being stimulated using a high-precision instrument (a hybrid between a microscope and a telescope: Fig. 3.1d), where the main objective lens was a single ocular facet (diameter ∼ = 50 μm) = 25 μm, focal length ∼ (Fig. 3.1a). By illuminating the two photoreceptors successively, a motion occurring in the visual field of the selected ommatidium was “simulated”. The H1 neuron responded with a vigorous spike discharge to this “apparent motion,” provided the motion was mimicked in the preferred direction (compare top and bottom traces in Fig. 3.1c) [67]. By applying various sequences of light steps and/or pulses to selected receptor pairs, an EMD block diagram was obtained and the dynamics and nonlinearity of each block were characterized [22, 27, 23]. In the mid-1980s, we designed a neuromorphic OF sensor inspired by the results of these electrophysiological studies [7, 25]. By definition, the OF is an angular speed ω corresponding to the inverse of the time Δt taken by a contrasting feature to travel between the visual axes of two adjacent photodiodes separated by an angle Δϕ (Fig. 3.2a). Our OF sensor’s scheme is not a “correlator” [38, 65] but rather a “feature-matching scheme” [91], where a given feature (here, a passing edge) is extracted and tracked in time. Each photodiode signal is first band-pass filtered

3

Optic Flow Based Autopilots: Speed Control and Obstacle Avoidance

31

Fig. 3.1 (a)–(c) Experimental scheme used to analyze the principles underlying elementary motion detectors (EMDs) in flies, using single neuron recording and single photoreceptor stimulation. (d) The triple-beam incident light “microscope-telescope” used to deliver a sequence of 1 μm light spots to two neigh-

boring photoreceptors (the fly’s head is indicated by the arrow). The microelectrode (c) recorded the electrical response (nerve impulses) of the motion-sensitive neuron H1 to this “apparent motion” (from [27])

(Fig. 3.2b), mimicking the analog signals emitted by the large monopolar neurons present in the fly lamina [97]. The next processing step consists in performing hysteresis thresholding and generating a unit pulse. In the EMD version built in 1989 for the Robot-Fly (Fig. 3.2d), the unit pulse from one channel sampled a long-lived decaying exponential function generated by the other channel, via a simple nonlinear circuit called a minimum detector (Fig. 3.2b), giving a monotonically increasing output VEMD with the angular velocity ω = Δϕ/Δt (Fig. 3.2b) [8]. The thresholding makes the EMD respond whatever the texture and contrast encountered, contrary to what occurs with “correlator” EMDs [65, 19] (see also Egelhaaf et al., Chap. 4 of this volume). A very similar EMD principle was developed, independently, a decade later by Koch’s group at CALTECH and termed the “facilitate and sample” velocity sensor [47]. These authors patented an aVLSI chip based on this principle [77], another variant of which

was recently presented (Moeckel & Liu, Chap. 8 of this volume). Our OF sensor actually comprises two parallel EMDs, each of which responds to either positive or negative contrast transitions, as in the fly EMD (cf. Figs. 15 and 16 in [27]). The circuit responds equally efficiently to natural scenes [61]. Our current OF sensors are still based on our original “travel time” principle [7, 25, 8] but for the sake of miniaturization, the signals are processed using a mixed analog + digital approach [75] and the time Δt is converted into ω via a lookup table (Fig. 3.2c). Although they are much larger than any aVLSI (or fly’s) EMDs, our current OF sensors (Fig. 3.2e,f) are small and light enough to be mounted on MAVs. Several OF sensors of this type can also be integrated into a miniature FPGA [1, 2]. A different kind of OF sensor was recently designed and mounted on a model aircraft [6, 35]. Optical mouse sensors have also been used as OF sensors (Dahmen et al., Chap. 9 of this volume) [36].

32

N. Franceschini et al.

Fig. 3.2 Principle (a, b) of the elementary motion detector (EMD) inspired by our electrophysiological findings (cf. Fig. 3.1) (after [7, 25, 8]) and (d) completely analog version (mass 5 g) made in 1989 with small mounted device (SMD) technology

(from [8]). (c, e) Hybrid version (mass 0.8 g) based on a microcontroller (from [75]). (f) Similar hybrid version (size 7 mm × 7 mm, mass 0.2 g) built using low-temperature co-fired ceramics technology (LTCC) (from [64])

3.3 An Explicit Control Scheme for Ground Avoidance

Flies and bees are able to measure the translational OF, ω, irrespective of the spatial texture and contrast encountered [15, 83, 3], and some of their visual neurons respond monotonically to ω [45]. Neurons facing downward can therefore act as ventral OF sensors and directly assess the groundspeed-to-groundheight ratio Vx /h (Fig. 3.3). Before Gibson introduced the OF concept [32], Kennedy established that an insect sees and reacts to the OF presented to its ventral viewfield [49] (see also [13]). This flowing visual contact with the ground is now known to be essential for insects to be able to orient upwind and migrate toward an attractive source of odor [49, 48] or pheromones [51]. Based on field experiments on locusts, Kennedy developed the “optomotor theory” of flight, according to which locusts have a “preferred retinal velocity” with respect to the

3.3.1 Avoiding the Ground by Sensing the Ventral Optic Flow The ventral OF perceived in the vertical plane by airborne creatures (including aircraft pilots) is the angular velocity ω generated by a point on the underlying flight track [33, 95]. Based on the definition of the angular velocity (Fig. 3.3A), the ventral OF ω is the ratio between groundspeed Vx and groundheight h as follows: ω = V x /h[rad.s−1 ]

(3.1)

3

Optic Flow Based Autopilots: Speed Control and Obstacle Avoidance

33

Fig. 3.3 (A) The ventral OF is an angular speed ω [rad s–1 ] corresponding to the groundspeed-to-groundheight ratio. (B) The OF sensor comprises a microlens and two photoreceptors separated by a small angle ϕ (see Fig. 3.2a) driving an EMD. The

latter delivers a signal ωmeas ∼ =∼ = ϕ/t = Vx /h, which serves as a feedback signal in the OF regulator (Fig. 3.4A). The onedimensional random texture is a magnified sample of that shown in Figs. 3.5b and 3.6A (from [28])

ground below [49, 50]. In response to wind, for example, they adjust their groundspeed (or groundheight) to restore the velocity of the ground feature images. Kennedy’s theory has been repeatedly confirmed during the last 30 years. Flies and bees seem to maintain a constant OF with respect to the ground while cruising or landing [13, 62, 84, 86, 4]. The problem is how they achieve this remarkable feat, since maintaining a given OF is a kind of chickenand-egg problem, as illustrated by Eq. (3.1): an insect may hold its perceived OF, ω, constant by controlling Vx (if it knows h) or by controlling h (if it knows Vx ). Moreover, it could maintain an OF of say 1 rad/s (i.e., 57◦ /s) by flying at a speed of 1 m/s at a height of 1 m or by flying at a speed of 2 m/s at a height of 2 m: there exists an infinitely large number of possible combinations of groundspeeds and groundheights generating the same “preferred OF”. Kennedy’s “theory” therefore lacked an explicit control scheme elucidating:

Our first attempt to develop a control scheme [55] was not very successful, as we were cornered by the chicken-and-egg OF problem mentioned above and by the assumption prevailing in those days that insect navigation involves estimating distance [53, 60, 83, 26, 82]. In the experimental simulations described in 1994, for example [55], we assumed that flying insects (and robots) are able to measure their groundspeed Vx (by whatever means), so that by measuring ω they would then be able to assess the distance h from the ground (Eq. (3.1)) and react accordingly to avoid it. Although this stratagem – which is in line with the Robot-Fly’s working principles (Sect. 3.1) – may be acceptable for aerial vehicles able to gauge their own groundspeed [5, 31], it does not tell us how insects function. In 1999, we established (via experimental simulations) how a seeing helicopter (or an insect) might manage to follow a terrain and land on the sole basis of OF cues without measuring its groundspeed or groundheight (see Figs. 4 and 5 in [57]). The landing maneuvers were performed under the permanent feedback control of an OF-sensing eye, and the driving force responsible for the loss of altitude was the decrease in the horizontal flight speed which occurred when the rotorcraft (or the insect) was about to land, either voluntarily or because of an unfavorable headwind. The landing trajectory obtained in these simulations [57] resembled the final approach of bees landing on a flat surface [84]. The 840-g rotorcraft we constructed was able to jump over 1-m-high obstacles (see Fig. 8 in [58]).

1. 2. 3. 4.

the flight variables really involved the sensors really required the dynamics of the various system components the causal and dynamic links between the sensor(s) and the variable(s) to be controlled 5. the points of application and the effects of the various disturbances that insects may experience 6. the variables insects control to compensate for these disturbances

34

N. Franceschini et al.

3.3.2 The “Optic Flow Regulator”

3.3.3 Micro-Helicopter (MH) with a Downward-Looking Optic Flow Sensing Eye More recently we developed a genuine “OF-based autopilot” called OCTAVE (which stands for optical altitude control system for aerial vehicles) that enables a micro-helicopter to perform challenging tasks such as takeoff, terrain following, reacting suitably to wind, and landing [70–74]. The idea was to integrate an OF sensor into a feedback loop driving the robot’s lift so as to compensate for any deviations of the OF sensor’s output from a given set point. This is what we call the OF regulator for ground avoidance. The term “regulator” is used here as in control theory to denote a feedback control system striving to maintain a variable constantly equal to a given set point. The variable regulated is often a temperature, a speed, or a distance, but here it is the variable ω [rad/s] corresponding to the Vx :h ratio, which can be sensed directly by an OF sensor. The OF sensor produces a signal ωmeas (Fig. 3.3b) that is compared with an OF set point ωSet (Fig. 3.4A). The error signal ε = ωmeas − ωSet drives a controller adjusting the lift L, and hence the groundheight h, so as to minimize ε. All the operator does is to set the pitch angle  and hence the forward thrust, and hence the airspeed (see Fig. 3.4A): the OF regulator does the rest, keeping the OF, i.e., the Vx :h ratio, constant. In the steady state (i.e., at t = ∞), ωmeas ∼ = ωSet and the groundheight h becomes proportional to the groundspeed Vx : h = K · V x (with K = 1/ωSet = constant)

(3.2)

Fig. 3.4 (A) The OCTAVE autopilot consists of a feedback control system, called the optic flow regulator (bottom part), that controls the vertical lift, and hence the groundheight, so as to maintain the ventral OF, ω, constant and equal to the set point ωset whatever the groundspeed. (B) Like flies [13] and bees [21],

To test the robustness of the OF regulator, we implemented it on a micro-helicopter (MH) equipped with a ventral OF sensor [70]. The robot (Fig. 3.5a) is tethered to the arm of a flight mill driven in terms of its elevation and azimuth by the MH’s lift and forward thrust, respectively (Fig. 3.5b). Any increase in the rotor speed causes the MH to lift and rise, and the slightest (operator mediated) forward tilting induces the MH to gain speed. The flight mill is equipped with ground-truth azimuthal and elevation sensors with which the position and speed of the MH can be measured with great accuracy in real time. The MH is equipped with a minimalistic two-pixel ventral eye driving a single EMD (Fig. 3.2e). The latter senses the OF produced by the underlying arena covered by a richly textured, randomly distributed pattern in terms of both the spatial frequency and the contrast m (0.04<m ωSetFwd /2), the agent will reach a final ordinate y∞ (Eq. (3.10)) that is smaller than half the value of the corridor width. This means that the flying agent will hug one wall, thus generating wall-following behavior. • At all sideways OF set point values that are smaller than half the forward OF set point (i.e., ωSetSide < ωSetFwd /2), the agent will not reach the final ordinate y∞ predicted by Eq. (3.10) without causing a change in sign in the error signal εSide (see Eq. (3.6)). The agent will therefore be forced to maintain the OF value at ωSetFwd /2, i.e., it will consistently navigate along the midline of the corridor, thus showing centering behavior. Oscillations in the trajectory are likely to occur, however, because of the repeatedly changing sign of the error signal εSide (Eq. (3.6)).

3.4.5 Simulation Results: Flight Paths Along Straight or Tapered Corridors The implementation of the complete LORA III visuomotor autopilot on the robotic hovercraft is under-

41

way, and only computer-simulated experiments will be presented here [79]. The computer simulations include the dynamic hovercraft model on both the surge and sway axes, the controllers with their PD or PI compensation, actuator saturations, full optical transfer function of the lens/photoreceptor system of each eye, detailed interactions between each photoreceptor and the random patterns lining the two walls, and complete processing within each OF sensor, according to the EMD principle described in Fig. 3.2c. These simulations were carried out on a standard PC equipped with the MatlabTM /Simulink toolbox at a sampling frequency of 1 kHz. This sampling frequency is the same as the sampling rate of the microcontroller installed onboard the LORA robot (Fig. 3.8a).

3.4.5.1 Wall-Following Behavior Along a Straight Corridor Figures 3.10, 3.11, and 3.12 show some examples of the computer-simulated behavior of the hovercraft equipped with the LORA III autopilot (Fig. 3.9) [79]. The first visual environment simulated was a straight corridor (3 m long, 1 m wide). The right and left walls were lined with a random pattern of various gray vertical stripes covering a 1-decade contrast range (from 4% to 38%) and a 1.1-decade spatial frequency range (from 0.068 c/◦ to 0.87 c/◦ reading from the corridor midline). From the three simulated trajectories shown in Fig. 3.10a, the hovercraft can be seen to navigate without ever colliding with the walls. It reached the same steady speed regardless of its initial ordinate y0 at the entrance to the corridor (Fig. 3.10b). All three trajectories correspond to the case where the sideways OF set point was larger than half the forward OF set point (ωSetSide = 230◦ /s > 150◦ /s = ωSetFwd /2). Under these conditions, the HO adopted wall-following behavior, as predicted in Sect. 3.4.4, and followed the right or left wall, depending on the initial ordinate y0 . The HO consistently generated a steady-state clearance of 0.25 m from either wall (left wall: squares and crosses; right wall: full dots) and a “safe” forward speed of Vx∞ =1 m/s. Steady-state clearance and speed define a similar “operating point” to that calculated from Eqs. (3.9) and (3.10): taking ωSetFwd = 300◦ /s, ωSetSide = 230◦ /s, and a corridor width D = 1 m give an operating point Vx∞ = 0.94 m/s and y∞ = 0.23 m.

42

N. Franceschini et al.

Fig. 3.10 (a) Wall-following behavior of the hovercraft (HO) (time marks on the flight paths are at 0.3-s intervals). The HO moves to the right at a forward OF set point ωSetFwd = 300◦ /s (3.28 V) and a sideways OF set point ωSetSide = 230◦ /s (2.21 V), starting at various ordinates y0 (squares: y0 =0.90 m, crosses: y0 =0.50 m, full dots: y0 = 0.10 m). (b) Irrespective of its initial ordinate, the HO ends up following one wall with a clearance

of 0.25 m, at a forward speed Vx∞ = 1 m/s. (c, d) The sum of the lateral OFs measured and the sum of the actual OFs both eventually equal the forward OF set point ωSetFwd = 300◦ /s. (e, f) The larger value of the OFs measured and that of the actual OFs both eventually equal the sideways OF set point ωSetFwd = 230◦ /s (from [79])

Whether the hovercraft followed the right or the left wall depended on the sign of the error signal ε Side (Eq. (3.6)). The hovercraft’s initial ordinate y0 was treated like a disturbance (see Fig. 3.9), which was rejected by the dual OF regulator. It can be seen from Fig. 3.10d,f that both the sum and the larger value of the lateral OFs eventually reacts the OF set points of 300◦ /s and 230◦ /s, respectively.

reached the sideways OF set point, ωSetSide (which was set at 90◦ /s or 110◦ /s or 130◦ /s). This is because these required values of ωSetSide were all smaller than half the value of the forward OF set point (i.e., ωSetSide < ωSetFwd /2 = 150◦ /s). These low values of ωSetSide relative to ωSetFwd forced the hovercraft to center between the two walls, as predicted at the end of Sect. 3.4.4. In addition, the robot’s ordinate can be seen to oscillate about the midline, due to the ever-changing sign of the error signal εSide (Eq. (3.6)) – as also predicted at the end of Sect. 3.4.4. The LORA III dual OF regulator may therefore even account for the striking oscillatory flight pattern observed in honeybees when they fly centered along a corridor (see Fig. 2a in [53]).

3.4.5.2 “Centering Behavior”: A Particular Case of “Wall-Following Behavior” Figure 3.11 illustrates the opposite case, where the OFs generated on either side (150◦ /s: Fig. 3.11d) never

3

Optic Flow Based Autopilots: Speed Control and Obstacle Avoidance

43

Fig. 3.11 “Centering behavior” as a particular case of “wallfollowing behavior”. (a, b) Simulated trajectories of the hovercraft (HO) moving to the right along a straight corridor at a forward OF set point ωSetFwd = 300◦ /s, with various sideways OF set points (crosses: ωSetSide = 130◦ /s, open dots: ωSetSide = 110◦ /s, full dots: ωSetSide = 90◦ /s). Initial condition y0 = 0.25 m, time marks as in Fig. 3.10. The HO can be seen to consis-

tently end up centering between the two walls at a forward speed of 1.3 m/s. (c, d) The larger of the two OFs measured and the larger actual OF both eventually equal 150◦ /s (=ωSetFwd /2). In attempting to reach either of the three sideways OF set points, the LORA III autopilot triggers changes in the sign of the error signal εSide (Eq. (3.6)), causing oscillations about the midline (see (a)) (from [79])

The error signal ε Side is consistently minimum along the midline, but the OF cannot become smaller than 150◦ /s (i.e., ωSetFwd /2). The hovercraft can be seen to have reached the steady-state forward speed of Vx∞ = 1.3 m/s (Fig. 3.11b) in all three cases because all three values of ωSetSide were below 150◦ /s. In all three cases, the steady-state operating point of the hovercraft was similar to that predicted in Sect. 3.4.4: at ωSetFwd = 300◦ /s and ωSetSide < 150◦ /s, a 1-m-wide corridor gives Vx∞ =1.31 m/s and y∞ = DR∞ = DL∞ = 0.5 m (Eqs. (3.9) and (3.10)). The lateral OFs measured on either side reached 150◦ /s (= ωSetFwd /2) in all three cases (Fig. 3.11c), and their sum was therefore equal to 300◦ /s (=ωSetFwd ). Similar centering behavior occurred at all values of ωSetSide such that ωSetSide ≤ ωSetFwd /2. Centering behavior can therefore be said to be a particular case of wall-following behavior.

3.4.5.3 Flight Pattern Along a Tapered Corridor In the last set of computer-simulated experiments presented here, the environment was a 6-m-long tapered corridor with a 1.24-m-wide entrance and a 0.5-mwide constriction located midway (Fig. 3.12). The right and left walls were lined with a random pattern of gray vertical stripes covering a 1-decade contrast range (from 4% to 38%) and a 1.5-decade spatial frequency range (from 0.034 c/◦ to 1.08 c/◦ reading from the corridor midline). As shown in Fig. 3.12a and b, whatever the position of its initial ordinate y0 , the HO automatically slowed down on approaching the narrowest section of the corridor and accelerated when the corridor widened beyond this point. In Fig. 3.12a, the HO adopted wallfollowing behavior because ωSetSide > ωSetFwd /2 =

44

N. Franceschini et al.

Fig. 3.12 Automatic navigation along a tapered corridor, requiring no data on the corridor width or the tapering angle α (marks on trajectories as in Fig. 3.10). Again, the hovercraft’s behavior is entirely determined by its two OF set points: ωSetFwd = 300◦ /s and ωSetSide = 230◦ /s. (a) Simulated trajectories of the HO moving to the right along the corridor (tapering angle α = 7◦ ) with three initial ordinates y0 (open dots: y0 = 0.90 m, crosses: y0 = 0.60 m, full dots: y0 = 0.30 m). (b) The for-

ward speed decreases and increases linearly with the local corridor width and the distance x traveled. (c, d) The sum of the two lateral OFs measured (and that of the actual OFs computed with Eq. (3.4) and (3.5)) is maintained constant and is equal to 300◦ /s (=ωSetFwd ). (e, f) The side control system effectively keeps whichever lateral OF is larger at a constant value of approximately 230◦ /s (=ωSetSide ) (from [79])

150◦ /s (see Eq. (3.10)). It can be seen from Fig. 3.12c that the forward control system succeeded in keeping the sum of the two lateral OFs measured constant and equal to the forward OF set point ωSetFwd = 300◦ /s. Likewise, the side control system succeeded in keeping the larger of the two lateral OFs measured virtually constant and equal to the sideways OF set point ωSetSide = 230◦ /s (Fig. 3.12e). Not only the initial ordinate y0 but also the (gradually changing) ordinates yR and yL are regarded by the LORA III autopilot as output perturbations, which are rejected by the dual OF regulator (see the points of application of these disturbances in Fig. 3.9).

The ensuing forward speed profile along the tapered corridor is particularly instructive (Fig. 3.12b): the HO’s forward speed Vx tends at all times to be proportional to the distance traveled x, as observed with the flight path of bees flying freely along a tapered corridor [84]. This plot of Vx = dx/dt versus x actually defines a phase plane, in which the linear change in speed observed with the distance traveled means that the speed Vx (t) is bound to vary as an exponential function of time [79]: sign(α)·(t−t )/τ (α)

Vx (t) = Vx (t0 ).e

0

(3.12)

3

Optic Flow Based Autopilots: Speed Control and Obstacle Avoidance

where the time constant τ (α) is a monotonic function of the tapering angle α, as follows: ωSetFwd 2 · tan |α| · ωSetSide · (ωSetFwd − ωSetSide ) (3.13) Thus, without having any knowledge of the corridor width D or the tapering angle α, the HO (or the bee) is therefore bound to slow down exponentially with a time constant τ (α) when entering the narrowing section (α < 0) of the corridor and to speed up exponentially with the same time constant τ (α) after leaving the constriction (α > 0), in accord with the observed honeybee’s flight path in a tapered tunnel [84]. The behavioral effects of two other types of disturbances affecting the LORA III autopilot were also studied [79]. With a large gap in a wall (as in the bee experiments shown in Fig. 3.7d) the HO was not flummoxed and kept on following the remaining wall [79]. When a wall pattern was moved at a constant speed VP in the direction of travel (as in the original bee experiments [53]), the robot moved closer to the moving wall, and vice versa [79]. The reason is that the robot’s relative speed with respect to the moving wall became Vx –VP instead of Vx in Eq. (3.8), causing a predictable shift in the robot’s operating point, Vx∞ , y∞ as computed from Eqs. (3.9–3.10). τ (α) =

3.5 Conclusion 3.5.1 Is There a Pilot Onboard an Insect? In this chapter, we have recounted our attempts to specify the types of operations that insects may perform to guide their flight on the sole basis of optic flow (OF) cues. The OCTAVE principle differs markedly from another current OF-based navigation strategy, where OF sensing needs to be completed by groundspeed sensing (based on a GPS, for example) to estimate the groundheight (see Eq. (3.1)) so as to follow terrain and land safely [5, 31]. The OCTAVE and LORA III autopilots harness the power of the translational OF more parsimoniously because they do not need to measure or estimate any distances or groundspeeds and therefore do not require any sensors other than OF sensors. The purpose of these autopilots is not to reg-

45

ulate any distances or groundspeeds. The only variable they need to regulate (i.e., maintain constant) is the OF – a variable that represents a speed-to-distance ratio and that can be directly measured by a dedicated sensor called an OF sensor. OCTAVE and LORA III autopilots include three interdependent OF regulators in all, which control the lift, lateral thrust, and forward thrust, on which the groundheight, lateral positioning, and groundspeed, respectively, depend. The block diagrams (Figs. 3.4 and 3.9) show which variables need to be measured, which ones are controlled, and which ones are regulated, as well as the point of application of the various disturbances. They also give the causal and dynamic relationships between these variables. These three feedback control loops may enable an agent having no mechanical contact with the ground to automatically attain a given groundspeed, a given groundheight and a given clearance from the walls in a simple environment such as a corridor, without any need for speed or range sensors giving explicit speed or distance data. In a tapered corridor, for example, the hovercraft (HO) automatically tunes both its clearance from the walls and its groundspeed to the local corridor width (Fig. 3.12), although it is completely “unaware” of the exact corridor width, the forward speed, and the clearance from the walls. The behavior depends wholly on three constants which are the three OF set points ωSetVentr , ωSetSide, and ωSetFwd . Experimental simulations and physical demonstrators showed that difficult operations such as automatic takeoff, ground avoidance, terrain following, centering, wallfollowing, suitable reaction to headwind, groundspeed control, and landing can be successfully performed on the basis of these three OF regulators. These simple control schemes actually account for many surprising findings published during the last 70 years on insects’ visually guided performances (details in [28]), including honeybees’ habit of landing at a constant slope [86] and their flight pattern along a tapered corridor [84]. Our novel finding that bees do not center systematically in a corridor but tend to follow a wall, even when the opposite wall has been removed (Fig. 3.7b–d), cannot be accounted for by the optic flow balance hypothesis [53, 83] and is convincingly accounted for by the LORA III model, where “centering behavior” [53, 83] turned out to be a particular case of “wall-following behavior” (Sect. 3.4.5.2). LORA III (Fig. 3.9) would give the bee a safe speed and safe clearance from the walls, whereas OCTAVE

46

(Fig. 3.4) would give the bee safe clearance from the ground – a clearance commensurate with its forward speed, whatever the speed. These explicit control schemes can therefore be viewed as working hypotheses. Onboard insects, the three OF set points may depend on either innate, internal, or external parameters. Recent findings have shown that the forward speed of bees flying along a corridor depends not only on the lateral OF but also partly on the ventral OF [4], which suggests that the ventral and lateral OFs should not perhaps be handled as separately as with OCTAVE and LORA III. Recent experimental simulations have shown how an agent might fly through a tunnel by relying on the OF generated by all four sides: the lateral walls, ground, and roof [61]. Indoor experiments on the autonomously flying microplane MC2 showed that when the plane makes a banked turn to avoid a wall, the ventral OF may originate partly from the lateral wall and the lateral OF partly from the ground (Zufferey et al., Chap. 6 of this volume). This particularity may not concern insects, however, since they compensate for the banking and pitching of their thorax [40, 41] by actively stabilizing their “visual platforms” [92, 96]. A high-speed, one-axis oculomotor compensatory mechanism of this kind was recently implemented onboard a 100-g aerial robot [52]. The electronic implementation of an OF regulator is not very demanding (nor is its neural implementation), since it requires only a few linear operations (such as adding, subtracting, and applying various filters) and nonlinear operations (such as minimum and maximum detections). OF sensors are the crux of OF regulators. Our neuromorphic OF sensors deliver an output that grows monotonically with the OF ω, regardless of the spatial frequency and contrast encountered (see Sect. 3.2), much like honeybees’ velocity-tuned (VT) neurons [45]. Most importantly, the OF regulator scheme greatly reduces the dynamic range constraints imposed on OF sensors, since it tunes the animal’s behavior all the time so that the OF will deviate little from the OF set point [28]. In Fig. 3.6C, for example, the MH holds ωmeas virtually constant throughout its journey, despite the large groundspeed variations that occur during takeoff and landing, for instance (see Fig. 3.6B). This and other examples (see Figs. 3.10c,e, 3.11c, and 3.12c,e) show that the 1-decade dynamic range (from 40◦ /s to 400◦ /s [75]) of our OF sensors is up to the tasks.

N. Franceschini et al.

3.5.2 Potential Aeronautics and Aerospace Applications The control schemes presented here rely on the OF to carry out reputedly difficult tasks such as taking off, terrain following, landing, avoiding lateral walls, and groundspeed control. These simple control schemes are restricted so far to cases where the OF is sensed either ventrally or laterally, perpendicular to the heading direction. The field of view (FOV) of the eyes and the provocatively small number of pixels (2 pixels per eye) and EMDs (one EMD per eye) obviously need to be increased when dealing with navigation in more sparsely textured environments. Increasing the number of motion sensors was found to improve the goal-directed navigation performances of an airship in noisy environments [46], and we recently described how an additional forward-looking EMD might enable our MH to climb steeper rises by providing the autopilot OCTAVE with an anticipatory feedforward signal [74]. It will also be necessary to enlarge the FOV and control the heading direction (about the yaw axis) to enable the HO to successfully negotiate more challenging corridors including L-junctions and T-junctions. The more frontally oriented visual modules required for this purpose could be based on measuring the OF divergence [56–58], a procedure that flies seem to use when they land and trigger body saccades [93, 78, 89]. In the field of aeronautics, these systems could serve to improve navigation aids and automatic maneuvers. Steady measurement of the ventral OF could prevent deadly crashes by warning pilots that the current altitude is “too low for the current groundspeed” – without requiring any altitude and speed measurements [29]. An OCTAVE OF regulator implemented onboard an aircraft would enable it to gradually take off “under automatic visual control”, to veto any attempt to descend to a groundheight not commensurate with the current groundspeed, and to make it land safely [72, 29]. These systems could also potentially be harnessed to guiding MAVs indoors or through complex terrains such as mountains and urban canyons. Since these control systems are parsimonious and do not rely on GPS or bulky and power-hungry emissive sensors such as FLIRs, RADARs, or LADARs, they meet the strict constraints imposed on the bird and insect scale in terms of their size, mass, and consumption.

3

Optic Flow Based Autopilots: Speed Control and Obstacle Avoidance

For the same reasons, these autopilots could potentially be adapted to micro-space vehicles (MSVs) performing rendezvous and docking missions in space or exploration missions on other celestial bodies. A Martian lander equipped with a more elaborate OCTAVE autopilot could perform smooth automatic landing (see Sect. 3.3 and [29, 72]). A flying reconnaissance rover equipped with more elaborate OCTAVE and LORA III autopilots could take off autonomously and explore an area, skimming the ground and hugging the walls of a canyon, and adapting its groundspeed and clearance from the walls automatically to the width of the canyon. The orbiter (or base station) would simply have to send the rover a set of three low-bandwidth signals: the values of the three OF set points [24]. Acknowledgments We are grateful to S. Viollet, F. Aubépart L. Kerhuel, and G. Portelli for their fruitful comments and suggestions during this research. G. Masson participated in the experiments on bees and D. Dray in the experimental simulations on LORA III. We are also thankful to Marc Boyron (electronics engineer), Yannick Luparini, and Fabien Paganucci (mechanical engineers) for their expert technical assistance and J. Blanc for revising the English manuscript. Serge Dini (beekeeper) gave plenty of useful advice during the behavioral experiments. This research was supported by CNRS (Life Science; Information and Engineering Science and Technology), an EU contract (IST/FET – 1999-29043), and a DGA contract (2005 – 0451037).

7.

8.

9.

10.

11.

12.

13.

14. 15.

16.

References 1. Aubépart, F., Franceschini., N.: Bio-inspired optic flow sensors based on FPGA: Application to micro-air vehicles. Journal of Microprocessors and Microsystems 31, 408–419 (2007) 2. Aubépart, F., El Farji, M., Franceschini, N.: FPGA implementation of elementary motion detectors for the visual guidance of micro-air vehicles. Proceedings of the IEEE International Symposium on Industrial Electronics (ISIE’2004), pp. 71–76. Ajaccio, France (2004) 3. Baird, E., Srinivasan, M.V., Zhang, S.W., Cowling., A.: Visual control of flight speed in honeybees. Journal of Experimental Biology 208, 3895–3905 (2005). 4. Baird, E., Srinivasan, M.V., Zhang, S.W., Lamont R., Cowling. A.: Visual control of flight speed and height in the honeybee. In: S. Nolfi et al. (eds.) From Animals to Animats 9, (SAB 2006), LNAI 4095, 40–51 (2006) 5. Barber, D.B.; Griffiths, S..R.; McLain, T.W.;& Beard, R.W.: Autonomous landing of miniature aerial vehicles. Proceedings American Institute of Aeronautics and Astronautics Conference. (2005) 6. Barrows, GL, Neely, C., Miller, K.T.: Optic flow sensors for MAV navigation. Fixed and flapping wing aerodynamics

17.

18.

19.

20.

21.

22.

47

for micro-air vehicle applications. Progress in Astronautics and Aeronautics 195, 557–574 (2001) Blanès, C.: Appareil visuel élémentaire pour la navigation à vue d’un robot mobile autonome. M.S. thesis in Neuroscience, Université d’Aix-Marseille II, Marseille (1986) Blanès, C.: Guidage visuel d’un robot mobile autonome d’inspitation bionique. PhD thesis, Polytechnical Institute, Grenoble. Thesis work at the Neurocybernetics lab, CNRS, Marseille, France (1991) Borst, A., Haag, J.: Neural networks in the cockpit of the fly. Journal of Comparative Physiology A188, 419–437 (2002) Buchner, E., Götz, K.G.: Evidence for one-way movement detection in the visual system of Drosophila. Biological Cybernetics 31, 243–248 (1978) Collett, T., Nalbach, H.O., Wagner, H.: Visual stabilization in Arthropods; In: F.A. Miles, J. Wallman (eds.) Visual motion and its role in the stabilization of gaze, pp. 239–263. Elsevier (1993) Coombs, D., Roberts, K.: Centering behavior using peripheral vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, USA (1993) David, C.: The relationship between body angle and flight speed in free-flying Drosophila. Physiological Entomology 3, 191–195 (1978) David, C.: Height control by free-flying Drosophila. Physiological Entomology 4, 209–216 (1979) David, C.: Compensation for height in the control of groundspeed by Drosophila in a new, ‘barber’s pole’ wind tunnel. Journal of Comparative Physiology 147, 1432–1351 (1982). David, C.: Visual control of the partition of flight force between lift and thrust in free-flying Drosophila. Nature 313, 48–50 (1985) Dickson, W., Straw, A., Poelma, C., Dickinson, M.: An integrative model of insect flight control. Proceedings of the 44th American Institute of Aeronautics, Astronautics and Aerospace Meeting, Reno, USA (2006) Duchon, A.P., Warren, W.H.: Robot navigation from a Gibsonian viewpoint. Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics (SMC), San Antonio, USA. 2272–2277 (1994) Egelhaaf, M., Borst, A.: Movement detection in Arthropods. In: F.A. Miles, J. Wallman (eds.) Visual motion an its role in the stabilization of gaze. Amsterdam: Elsevier, 53–77 (1993) Epstein, M., Waydo, S., Fuller, S.B., Dickson, W., Straw, A., Dickinson, M.H., Murray R.M.: Biologically inspired feedback design for Drosophila flight. Proceedings of the American Control Conference (ACC), NY, USA, pp. 3395– 3401 (2007). Esch, H., Natchigall, W., Kogge, S.N.: Correlations between aerodynamic output, electrical activity in the indirect flight muscles and flight positions of bees flying in a servomechanically controlled flight tunnel. Journal of Comparative Physiology A 100, 147–159 (1975) Franceschini, N.: Early processing of colour and motion in a mosaic visual system. Neuroscience Research 2, 517–549 (1985)

48 23. Franceschini, N.: Sequence-discriminating neural network in the eye of the fly. In: F.H.K. Eeckman (ed.) Analysis and Modeling of Neural Systems, pp.142–150. Norwell, USA, Kluwer Acad. Pub. (1992) 24. Franceschini, N.: Towards automatic visual guidance of aerospace vehicles: from insects to robots. Proceedings of the ACT Workshop on Innovative Concepts « A Bridge to Space » ESA-ESTEC, Noordwijk, The Netherlands, Acta Futura 4: 8–23 (2008) 25. Franceschini, N., Blanes, C. and L. Oufar, L.: Appareil de mesure, passif et sans contact, de la vitesse d’un objet quelconque. Technical Report ANVAR/DVAR, N◦ 51549, Paris, France (1986) 26. Franceschini, N., Pichon, J.M., Blanès, C.: From insect vision to robot vision. Philosophical Transactions of the Royal Society of London. Series B 337, 283–294 (1992) 27. Franceschini, N., Riehle, A., Le Nestour A.: Directionally selective motion detection by insect neurons. In: D.G. Stavenga, R.C. Hardie (eds.) Facets of Vision, Chapt. 17, pp. 360–390. Berlin, Springer (1989) 28. Franceschini, N., Ruffier, F., Serres, J.: A bio-inspired flying robot sheds light on insect piloting abilities. Current Biology 17, 329–335 (2007) 29. Franceschini, N., Ruffier, F., Viollet, S., Boyron, M.: Steering aid system for altitude and horizontal speed, perpendicular to the vertical, of an aircraft and aircraft equipped therewith. International Patent PCT FR2003/002611 (2003) 30. von Frisch, K.: Tanzsprache und Orientierung der Bienen, Berlin, Springer (1965) 31. Garratt M.A., Chahl, J.S.: Vision-based terrain following for an unmanned rotorcraft. Journal of Field Robotics 25, 284–301 (2008) 32. Gibson, J.J.: The perception of the visual world. Boston, Houghton Mifflin (1950) 33. Gibson, J.J., Olum, P., Rosenblatt, F.: Parallax and perspective during aircraft landings. The American Journal of Psychology 68, 372–395 (1955) 34. Götz, K.G.: Flight control in Drosophila by visual perception of motion. Kybernetik 4, 199–208 (1968) 35. Green, WF, Oh, Y., Barrows, G.: Flying insect inspired vision for autonomous axial robot maneuvers in near earth environments. Proceedings of the IEEE International Conference on Robotics and Automation vol. 3, pp 2347–2352 (2004) 36. Griffiths, S., Saunders, J., Curtis, A., Barber, B., McLain, T., Beard, R.: Maximizing miniature aerial vehicles – obstacle and terrain avoidance for MAVs. IEEE Robotics and Automation Magazine 13, 34–43 (2006) 37. Hausen K., Egelhaaf, M.: Neural mechanisms of visual course control in insects. In: D.G. Stavenga, R.C. Hardie (eds.) Facets of Vision, chapt. 18, pp. 391–424. Berlin: Springer (1989) 38. Hassenstein, B., Reichardt, W.: Systemtheoretische Analyse der Zeitreihenfolgen und Vorzeichen-Auswertung bei der Bewegungsperzeption des Rüsselkäfers Chlorophanus. Zeitschrift für Naturforschung 11b, 513–524 (1956) 39. Heisenberg, M., Wolf, R.: Vision in Drosophila. Genetics of microbehavior. Berlin: Springer (1984) 40. Hengstenberg R.: Mechanosensory control of compensatory head roll during flight in the blowfly Calliphora ery-

N. Franceschini et al.

41.

42.

43.

44.

45.

46.

47.

48.

49.

50.

51. 52.

53.

54. 55.

56.

57.

throcephala Meig. Journal of Comparative Physiology A 163, 151–165 (1988) Hengstenberg R.: Control of head pitch in Drosophila during rest and flight. Göttingen Neurobiology Meeting 305 (1992) Heran, P., Lindauer, M.: Windkompensation und Seitenwindkorrektur der Bienen beim Flug über Wasser. Zeitschrift für Vergleichende Physiologie 47, 39–55 (1963) Humbert, J.S., Hyslop, H., Chinn, M.: Experimental validation of wide-field integration methods for autonomous navigation. Proceedings of IEEE/RSJ International Conference Intelligent Robots and Systems (IROS). San Diego, USA, 2144–2149 (2007) Humbert, J.S., Murray, R.M., Dickinson, M.H.: Sensorimotor convergence in visual navigation and flight control systems. In: Proceedings of 16th IFAC World Congress. Prague, Czech Republic (2005) Ibbotson, M.R.: Evidence for velocity-tuned motionsensitive descending neurons in the honeybee. Proceedings of the Royal Society of London, Series B 268, 2195–2201 (2001) Iida, F.: Goal-directed navigation of an autonomous flying robot using biologically inspired cheap vision. Proceedings of the 32nd International Symposium on Robotics (2001) Kramer, J., Sarpeshkar, R., Koch, C.: Pulse-based analog VLSI velocity sensors. IEEE Trans. Circuits and Systems II : Analog and digital signal processing 44, 86–101 (1997) Kellog, F.E., Fritzel, D.E., Wright, R.H.: The olfactory guidance of flying insects IV. Drosophila. Canadian Entomology, 884–888 (1962) Kennedy, J.S.: Visual responses of flying mosquitoes. Proceedings of the Zoological Society of London 109, 221–242 (1939) Kennedy, J.S.: The migration of the desert locust (Schistocerca gregaria Forsk.) I. The behaviour of swarms. Philosophical Transactions of the Royal Society London, B 235, 163–290 (1951) Kennedy, J.S., Marsh D.: Pheromone-regulated anemotaxis in flying moths. Science 184, 999–1001 (1974) Kerhuel, L., Viollet, S., Franceschini, N.: A sighted aerial robot with fast gaze and heading stabilization. Proceedings of the IEEE International Conference on Intelligent Robots Systems, San Diego, pp. 2634–2641 (2007) Kirchner, W.H. ; Srinivasan, M.V.: Freely moving honeybees use image motion to estimate distance. Naturwissenchaften 76, 281–282 (1989) Koenderink, J.J.: Optic flow. Vision Research 26, 161–179 (1986) Mura, F., Franceschini., N.: Visual control of altitude and speed in a flying agent. In: D. Cliff et al. (eds.) From Animals to Animats III, pp. 91–99. Cambridge: MIT Press (1994) Nelson, RC, Aloimonos, Y. Obstacle avoidance using flow field divergence. IEEE Transactions on Pattern Analysis and Machine Intelligence 11, 1102–1106 (1989) Netter, T., Franceschini, N.: Neuromorphic optical flow sensing for Nap-of-the-Earth flight. In: D.W. Gage, H.M. Choset (eds.) Mobile Robots XIV, SPIE, Vol. 3838, Bellinghan, USA 208–216 (1999)

3

Optic Flow Based Autopilots: Speed Control and Obstacle Avoidance

58. Netter, T., Franceschini, N.: A robotic aircraft that follows terrain using a neuromorphic eye. Proceedings of the IEEE International Conference on Intelligent Robots & Systems (IROS) Lausanne, Switzerland, 129–134 (2002) 59. Neumann T.R., Bülthoff, H.: Insect inspired visual control of translatory flight. Proceedings ECAL 2001, pp. 627–636. Berlin: Springer (2001) 60. Pichon, J-M., Blanès, C., Franceschini, N.: Visual guidance of a mobile robot equipped with a network of self-motion sensors. In: W.J. Wolfe, W.H. Chun (eds.) Mobile Robots IV, Bellingham, U.S.A.: SPIE Vol. 1195, 44–53 (1989) 61. Portelli, G., Serres, J., Ruffier F., Franceschini, N.: An Insect-Inspired Visual Autopilot for Corridor-Following. Proceedings of the IEEE International Conference on Biomedical Robotics & Biomechatronics, BioRob 08, Scottsdale, USA, pp. 19–26 (2008) 62. Preiss, R.: Set-point of retinal velocity of ground images in the control of swarming flight of desert locusts. Journal Comparative Physiology A 171, 251–256 (1992) 63. Preiss, R., Kramer, E.: Control of flight speed by minimization of the apparent ground pattern movement. In: D. Varju, H. Schnitzler (eds.) Localization and orientation in Biology and Engineering, pp. 140–142 Berlin, Springer (1984) 64. Pudas, M, Viollet, S, Ruffier, F, Kruusing, A, Amic, S, Leppävuori, S, Franceschini, N.: A miniature bio-inspired optic flow sensor based on low temperature co-fired ceramics (LTCC) technology, Sensors and Actuators A133, 88–95 (2007) 65. Reichardt, W.: Movement perception in insects. In: W. Reichardt (ed.) Processing of Optical Data by Organisms and by Machines. New York, Academic Press (1969) 66. Reiser, M, Humbert, S., Dunlop, M., Vecchio, D., Murray, M., Dickinson, M.: Vision as a compensatory mechanism for disturbance rejection in upwind flight. American Control Conference 04. Proceedings IEEE Computer Society pp. 311–316 (2004) 67. Riehle, A, Franceschini, N.: Motion detection in flies: parametric control over ON-OFF pathways. Experimental Brain Research 54, 390–394 (1984) 68. Riley, J.R., Osborne, JL.: Flight trajectories of foraging insects: observations using harmonic radar. In: D.R. Reynolds, C.D. Thomas (eds.) Insect Movement: Mechanisms and Consequences, Chap. 7, pp. 129–157. CAB International (2001) 69. Rossel, S., Wehner, R.: How bees analyze the polarization pattern in the sky. Experiments and model. Journal of Comparative Physiology A154, 607–615 (1984) 70. Ruffier, F., Franceschini, N.: OCTAVE, a bioinspired visuomotor control system for the guidance of Micro-Air Vehicles, In: A. Rodriguez-Vazquez, D. Abbott, R. Carmona, (eds.) Bioengineered and Bioinspired Systems, Vol. 5119, pp. 1–12. Bellingham, U.S.A.: SPIE (2003) 71. Ruffier, F., Franceschini, N.: Visually guided micro-aerial vehicle: automatic take-off, terrain following, landing and wind reaction. Proceedings of the IEEE International Conference on Robotics & Automation (ICRA), New Orleans, USA, pp. 2339–2346 (2004) 72. Ruffier, F., Franceschini, N.: Automatic landing and takeoff at constant slope without terrestrial aids. Proceedings of the 31st Eur. Rotorcraft Forum, AAF/CEAS, Florence, pp. 92.1–92.8. (2005)

49

73. Ruffier, F., Franceschini, N.: Optic flow regulation: the key to aircraft automatic guidance. Robotics and Autonomous Systems 50, 177–194 (2005) 74. Ruffier, F., Franceschini, N.: Aerial robot piloted in steep relief by optic flow sensors. Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS), Nice, pp.1266–1273 (2008) 75. Ruffier, F., Viollet, S., Amic, S., Franceschini, N.: Bioinspired optical flow circuits for the visual guidance of micro-air vehicles. Proceedings IEEE, ISCAS, III, pp. 846–849. Bangkok, Thaïland (2003) 76. Santos-Victor, J., Sandini, G., Curotto, F., Garibaldi, S.: Divergent stereo in autonomous navigation: from bees to robots. International Journal of Computer Vision 14, 159– 177 (1995) 77. Sarpeshkar, R.; Kramer, J.; Koch, C.: Pulse domain Neuromorphic Circuit for Computing Motion, US patent Nb 5,781,648 (1998) 78. Schilstra, C., van Hateren J.H.: Blowfly flight and optic flow. I. Thorax kinematics and flight dynamics. Journal of Experimental Biology. 202, 1481–1490 (1999) 79. Serres, J., Dray, D., Ruffier, F., Franceschini, N.: A visionbased autopilot for a miniature airvehicle: joint speed control and lateral obstacle avoidance. Autonomous Robots 25, 103–122 (2008) 80. Serres, J., Masson, G., Ruffier, F., Franceschini, N.: A bee in the corridor : centering and wall following. Naturwissenschaften 95, 1181–118 (2008) 81. Spork, P., Preiss, R.: Control of flight by means of lateral visual stimuli in gregarious desert locusts, Schistocerca gregaria. Physiological Entomology 18, 195–203 (1993) 82. Srinivasan, M.V.: How insects infer range from visual motion. In: F.A. Miles, J. Wallman (eds.) Visual motion and its role in the stabilization of gaze. Amsterdam, Elsevier (1993) 83. Srinivasan, M.V., Lehrer, M., Kirchner, W.H., Zhang, S.W.: Range perception through apparent image speed in freely flying honeybees. Visual Neuroscience 6, 519–535 (1991) 84. Srinivasan, M.V., Zhang, S.W., Lehrer, M., Collett., T.: Honeybee navigation en route to the goal: visual flight control and odometry. Journal of Experimental Physiology 199, 237–244 (1996) 85. Srinivasan, M.V., Chahl, J.S., Weber, K., Venkatesh, S., Nagle, M.G., Zhang, S.W.: Robot navigation inspired by principles of insect vision. Robotics and Autonomous Systems 26, 203–216 (1999) 86. Srinivasan, M.V., Zhang, S.W., Chahl, J.S., Barth, E., Venkatesh, S.: How honeybees make grazing landings on flat surface. Biological Cybernetics 83, 171–183 (2000) 87. Strausfeld, N.: Beneath the compound eye: Neuroanatomical analysis and physiological correlates in the study of insect vision. In: D.G. Stavenga, R.C. Hardie (eds.) Facets of Vision., Berlin, Springer, chapt. 16, pp. 316–359 (1989) 88. Tammero, L.F., Dickinson, M.H.: Collision-avoidance and landing responses are mediated by separate pathways in the fruit fly, Drosophila melanogaster. Journal of Experimental Biology 205, 2785–2798 (2002) 89. Tammero, L.F., Dickinson, M.H.: The influence of visual landscape on the free flight behavior on the fruit fly

50

90.

91. 92.

93.

N. Franceschini et al. Drosophila melanogaster. Journal of Experimental Biology 205, 327–343 (2002) Taylor G.K., Krapp, H.G.: Sensory systems and flight stability: what do insects measure and why? In: J. Casas, S.J. Simpson (eds.) Advances in Insect Physiol. 34: Insects mechanisms and control, pp. 231–316 Amsterdam, Elsevier (2008) Ullman, S.: Analysis of visual motion by biological and computer systems, Computer 14, 57–69 (1981) van Hateren, H., Schilstra, C.: Blowfly flight and optic flow, II. Head movement during flight. Journal of Experimental Biology 202: 1491–1500 (1999) Wagner, H.: Flow-field variables trigger landing in flies. Nature 297, 147–148 (1982)

94. Wagner, H.: Flight performance and visual control of flight of the free-flying housefly Musca domestica, I. Organisation of the flight motor. Philosophical Transactions of the Royal Society of London, Series B312, 527–551 (1986) 95. Whiteside, T.C., G.D. Samuel.: Blur zone. Nature 225, 94– 95, (1970) 96. Zeil, J., Boeddeker, N., Hemmi, J.M.: Vision and the organization of behavior, Current Biology 18, 320–323 (2008) 97. Zettler, F., Weiler, R.: Neuronal processing in the first optic neuropile of the compound eye of the fly. In: F. Zettler, R. Weiler (eds.) Neural principles in vision, pp. 226–237. Berlin, Springer (1974)

Chapter 4

Active Vision in Blowflies: Strategies and Mechanisms of Spatial Orientation Martin Egelhaaf, Roland Kern, Jens P. Lindemann, Elke Braun, and Bart Geurten

Abstract With its miniature brain blowflies are able to control highly aerobatic flight manoeuvres and, in this regard, outperform any man-made autonomous flying system. To accomplish this extraordinary performance, flies shape actively by the specific succession of characteristic movements the dynamics of the image sequences on their eyes (‘optic flow’): They shift their gaze only from time to time by saccadic turns of body and head and keep it fixed between these saccades. Utilising the intervals of stable vision between saccades, an ensemble of motion-sensitive visual interneurons extracts from the optic flow information about different aspects of the self-motion of the animal and the spatial layout of the environment. This is possible in a computationally parsimonious way because the retinal image flow evoked by translational self-motion contains information about the spatial layout of the environment. Detection of environmental objects is even facilitated by adaptation mechanisms in the visual motion pathway. The consistency of our experimentally established hypotheses is tested by modelling the blowfly motion vision system and using this model to control the locomotion of a ‘Cyberfly’ moving in virtual environments. This CyberFly is currently being integrated in a robotic platform steering in three dimensions with a dynamics similar to that of blowflies.

M. Egelhaaf () Department of Neurobiology & Center of Excellence “Cognitive Interaction Technology”, Bielefeld University, D-33501 Bielefeld, Germany e-mail: [email protected]

4.1 Virtuosic Flight Behaviour: Approaches to Unravel the Underlying Mechanisms Anyone who observes a blowfly landing on the rim of a cup or two flies chasing each other will be fascinated by the breathtaking aerobatics these tiny animals can produce (Fig. 4.1). While the human eye is hardly capable of even following their flight paths, the pursuer fly is quite capable of catching its speeding target. During their virtuosic flight manoeuvres blowflies can make up to 10 sudden, so-called saccadic turns per second, during which they reach angular velocities of up to 4,000◦ /s [58, 70]. During their flight manoeuvres blowflies rely to a great extent on information from the displacements of the retinal images across the eyes (‘optic flow’). This visual motion information is then transformed in a series of processing steps into motor control signals that are used to steer the flight course. The analysis of neural computations underlying behavioural control often rests on the implicit assumption that sensory systems passively pick up information about their surroundings and process this information to control the appropriate behaviour. This concept, though useful from an analytical point of view, misses one important feature of embodied and situated behaviour: normal behaviour operates under closed-loop conditions and all movements of the animal may shape the sensory input to a large extent. Although we will mainly concentrate in this chapter on the sensory side of the action–perception cycle and, in particular, the processing of visual motion information, our approach is distinguished by envisioning the blowfly as a dynamic system embedded in continuous interactions with its environment.

D. Floreano et al. (eds.), Flying Insects and Robots, DOI 10.1007/978-3-540-89393-6_4, © Springer-Verlag Berlin Heidelberg 2009

51

52 Fig. 4.1 Five snapshots from a high-speed film sequence of a blowfly landing on the rim of a cup. The fly turns by almost 180◦ within approximately 140 ms

M. Egelhaaf et al. 0,0s

0,026s

0,036s

0,086s

0,14s

Since it is currently not possible to probe during flight behaviour into the neural circuits processing visual motion information, behavioural and neuronal analyses are done separately. Both types of analyses are interlinked by employing for visual stimulation in experiments at the neural level reconstructions of the image sequences free-flying blowflies have previously seen during their virtuosic flight manoeuvres as well as targeted manipulations of such sequences. Neural analysis can, for methodological reasons, unravel only subsystems at a time, such as in our case a population of motion-sensitive nerve cells in the blowfly visual system. The functional role of the analysed subsystems for the performance of the entire system, i.e. in our case for visually guided behaviour, can therefore hardly be assessed appropriately by experimental analysis alone. Reasons are the non-linearity of most computational mechanisms, the recurrent organisation of many neuronal subsystems and the resulting complex dynamics of neuronal activity, as well as the closed-loop nature of behaviour. These constraints can only be overcome by closely linking and complementing experiments with computational approaches: experimentally established hypotheses are modelled in our CyberFly project, both in software and hardware, and put into the context of the entire system interacting with its environment.

The following aspects of visually guided behaviour of blowflies and the underlying neural computations will be addressed: (1) Flight activity of blowflies will be scrutinised and segregated into sequences of individual prototypical components. The corresponding behaviourally generated visual input, a consequence of the closed action–perception cycle, will then be reconstructed; (2) we will pinpoint what information about self-motion and the outside world is provided by populations of output neurons of the visual motion pathway; and (3) the experimentally established hypotheses on the mechanisms of motion computation, on the coding properties of this population of cells and on how this neural population activity is used to control behaviour are challenged with our CyberFly model under openloop and closed-loop conditions.

4.2 Active Vision: The Sensory and Motor Side of the Closed Action–Perception Cycle Optic flow induced on the eyes during locomotion does not only provide information about self-motion of the animal but is also a potent visual cue for spatial

4

Active Vision in Blowflies

information. Optic flow is the most relevant source of spatial information in flying animals which probably do not have other distance cues at their disposal. Some insects use relative motion very efficiently to detect objects and to infer information about their height [10, 34, 45, 61]. When the animal passes or approaches a nearby object, the object appears to move faster than its background. Motion can thus provide the perceived world with a third dimension. Locusts and mantids, for instance, are known to judge the distance of prey objects based on motion parallax actively generated by peering movements [37]. Moreover, honeybees assess the travelled distance or the distance to the walls of a flight tunnel on the basis of visual movement cues [1, 14, 15, 27, 60, 62, 63, 66]. Hummingbird hawkmoths hovering in front of flowers use motion cues to control their distance to them [16, 33, 53]. Several insect species, such as wasps and honeybees, perform characteristic flight sequences in the vicinity of their nest or of a food source. It has been concluded that the optic flow on the eyes is shaped actively by these characteristic flight manoeuvres to provide spatial information [44, 72–74] (see also Chaps. 1, 7 and 17). Blowflies employ a characteristic flight and gaze strategy with strong consequences for the optic flow patterns generated on the eyes: Although blowflies are able to fly continuous turns while chasing targets [2, 3], they do not show smooth turning behaviour during cruising flight or in obstacle avoidance tasks. Instead, they keep their gaze almost straight for short flight segments and then execute sharp fast turns, commonly referred to as saccades (Fig. 4.2A,B). These saccades only last for about 50–100 ms. During saccades blowflies may reach rotational velocities of up to 4000º/s and change their body orientation by up to 90◦ [58]. The head is actively moved so that gaze shifts are even shorter and more precise than those of the body. Active head movements considerably improve stabilisation of the gaze direction between saccades [70]. This behaviour can be interpreted as an active vision strategy stabilising the gaze rotationally as much as possible [31, 57]. The temporal pattern of saccades and intersaccadic intervals as well as the amplitude of saccades may vary systematically depending on the behavioural context. For instance, blowflies, even when flying in a straight tunnel, do not fly in a straight line, but perform a sequence of alternating saccades.

53

The frequency and amplitude of these vary with the width of the flight tunnel (Kern et al. in prep.). As a consequence of the saccadic strategy of flight and gaze control the rotational and translational components of the optic flow are largely segregated at the behavioural level. In blowflies, this strategy is not obviously reflected as sharp bends in the path of the body’s centre of mass. Changes in orientation do not immediately result in changes in flight direction due to sideward drift after a body saccade, presumably caused by the inertia of the blowfly [58]. In this regard blowflies appear to differ from the much smaller fruitflies ([65]; Chap. 17). So far, we mainly decomposed behaviour into just two prototypical components, saccades and straight flight segments during the intersaccadic intervals. This segmentation was mainly based on the horizontal components of movement. Does the segmentation of behaviour into distinct prototypical movements generalise if we consider all six degrees of freedom of locomotion? To answer this question we decomposed behavioural free-flight sequences into their constituent prototypical movements by clustering algorithms. Clustering approaches have been successfully applied, for instance, to classify basic skills and behaviours within observation data of humans and also in computer science and robotics when controlling the movements of artificial agents [56, 67, 68]. In contrast to classical behavioural analysis, clustering methods (such as k-means; [28]) allow us to analyse vast amounts of data in a relatively short time. The large database enables us to assess the relative frequency and the characteristic sequences of prototypical movements occurring in different behavioural contexts. We applied k-means clustering to the six-dimensional set of translational and rotational velocities obtained from cruising flight sequences in an indoor flight arena. We could identify nine stable prototypical movements as behavioural building blocks during cruising behaviour of blowflies. Even if all six degrees of freedom of self-movement are taken into account (Fig. 4.2B,C), the prototypical movements can be classified into two distinct main classes, i.e. rotational movements, on the one hand, and translational movements, on the other hand (Fig. 4.2D). Within the translation class of prototypes, for instance, prototypes reflecting almost pure forward translation and

54

A

500 ms

5 cm

1000 ms

0 ms

5 cm

1200 ms rot. vel. [deg /s ]

B 2000

yaw

0

pitch roll

–2000

C transl. vel. [m /s ]

1 forward

0

sideward lift

–1

D

10 nb prot. move

Fig. 4.2 Flight sequence of a free-flying blowfly and its decomposition into prototypical movements. (A) Section of the flight sequence. The position of the head and its orientation as seen from above are indicated every 10 ms by a dot and a line, respectively (behavioural data: courtesy J.H. van Hateren, University of Groningen, NL). (B) Rotational velocities (yaw, pitch, roll) of the head for the flight sequence shown in (A). Note the saccadic structure of rotational movements. (C) Translational velocities (forward, sideward, lift) for the same flight sequence as shown in (A). The translational velocities predominantly change on a slower timescale than the rotational velocities and do not show an obvious saccadic structure. (D) Prototypical movements into which the flight sequence shown in (A) can be decomposed. Four saccadic (two rightward and two leftward saccadic prototypes) and five translational prototypes were identified. Each prototype is characterised by a distinct combination of rotational and translational movements

M. Egelhaaf et al.

sacc. left

5 0 0

prototypes characterised by a strong sideward component can be distinguished. Organising behaviour as a sequence of prototypical movements leads to a tremendous complexity reduction which is likely to be favourable for both motor control and sensory information processing. The prototypical movements are thought to be selected from a limited pool of possible movement prototypes according to a strategy that depends on the respective behavioural context. This strategy may simplify motor control tremendously. Because of the closed-loop nature of behaviour, the different types of prototypical movements go along with retinal image displacements which are characterised by distinct spatiotemporal features. We could show that the sensory

intersacc. sacc. right

400

800 time [ms]

1200

input is shaped by the very nature of the prototypical movements in a way that greatly facilitates the neuronal analysis of complex sensory information. The corresponding optic flow is either mainly rotational (i.e. during saccades) or translational (i.e. during the intersaccadic intervals). The behavioural segregation of rotational and translational self-movements enables flies to gather spatial information about the three-dimensional layout of their environment during intersaccadic flight sections by relatively simple computational means. The optic flow component resulting from translational selfmotion depends on distance of environmental objects from the observer, whereas the rotational optic flow component is independent of the distance [35]. Thus,

4

Active Vision in Blowflies

only the translational optic flow component contains spatial information. It should be noted that, although translational optic flow contains information about the three-dimensional layout of the environment, it does not directly provide information about metric distances. Rather, spatial information derived from optic flow is only relative, because it depends on (i) the velocity of the observer, (ii) his/her distance to objects in the surroundings and (iii) the location of the objects in the visual field relative to the direction of translation. Although it is mathematically possible to decompose optic flow fields into their rotational and translational components [8, 50, 54], blowflies and other insects appear to avoid the heavy computational effort by a smart behavioural strategy that keeps largely apart rotational and translational flow components from the bottom up. It is not yet clear how this behavioural strategy is accomplished. In particular, the underlying head–body coordination is demanding, because it leads to almost perfect gaze stabilisation between saccades within only few milliseconds, while the body still shows residual slow rotational movements [70]. We can only surmise that feed-forward control and/or mechanosensory information may play a major role, whereas visual feedback might be too slow on the relevant short timescale. In conclusion, the active flight and gaze strategy of blowflies as well as those of many insect species may have been shaped during evolution by requirements of image motion processing. This behavioural strategy can help to reduce the complexity of the sensory input by structuring the movements in an adaptive way. Active vision strategies may thus facilitate the extraction of spatial cues by smart, i.e. relatively simple, mechanisms. Such mechanisms may also be relevant when engineering lightweight autonomous air vehicles.

4.3 Extracting Spatial Information from Actively Generated Optic Flow The blowfly visual motion pathway is well adapted to make use of the structured visual input resulting from the saccadic flight and gaze strategy when extracting spatial information from optic flow. As the blowfly visual system is optimised for reliable performance in

55

virtuosic flight behaviour and is amenable to a broad spectrum of neuronal and behavioural methods it has proved to be a good model system for tracing the computations which serve to process image motion proceeding from the eyes [7, 10, 13, 20]. Retinal image displacements are not perceived directly by the eye. Rather, the photoreceptors in the retina register just a spatial array of brightness values continuously changing in time. From this, the nervous system has to go through a series of steps to evaluate information on the image movements. In the blowfly visual system motion is initially processed in the first and second visual areas of the brain by successive layers of retinotopically arranged columnar neurons. One major function of the first visual area is to remove spatial and temporal redundancies from the incoming retinal signals and to maximise the transfer of information about the time-dependent retinal images by adaptive neural filtering [29, 43, 69]. There is evidence that direction selectivity is first computed by retinotopically arranged local movement detectors in the most proximal layers of the second visual area (review in [9, 64]). The performance of such local movement detection circuits can be accounted for by a computational model, the correlation-type motion detector, often referred to as elementary motion detector (EMD) ([6, 11, 12, 36, 64]; for hardware implementations of local movement detectors, see Chap. 8). This model explains neuronal responses to a wide range of motion stimuli including those that are experienced during highly aerobatic flight manoeuvres [48]. EMDs correlate the brightness data of adjacent light-sensitive cells receiving appropriately filtered brightness signals from neighbouring points in visual space and subtract the outputs of two such correlation units with opposite preferred directions. Movement is signalled when the input elements report the same brightness value in immediate succession. During this process, each motion detector reacts with a large excitatory signal to movement in a given direction and with a negative, i.e. inhibitory, signal to motion in the opposite direction. The responses of EMDs depend not only on image velocity but also on the contrast, the spatial frequency content and orientation of the pattern elements [5, 12]. As a result of these coding properties the representations of local motion information in biological systems, such as the blowfly, are likely to differ considerably from the veridical retinal velocities forming the optic flow (review in [10, 12]).

56

M. Egelhaaf et al.

Since behaviourally relevant information is contained in the global features of optic flow rather than in the local velocities, EMD signals from large areas of the visual field need to be combined. Accordingly, in a variety of animal groups ranging from insects to primates, neurons sensitive to optic flow were found to have large receptive fields (reviews in [5, 42]; see also

Chap. 5). In blowflies, spatial pooling is accomplished by an ensemble of individually identifiable motionsensitive neurons, the so-called tangential cells (TCs) [7, 10, 13, 24, 38]. TCs are thought to spatially pool on their large dendrites the output signals of many local motion detectors received via excitatory and inhibitory synapses. The local motion detectors are activated

100 cm

1

N=4 n = 132

Response [rel. units]

0.8

0.6

0.4

0.2

0

−0.2

102 103 Minimal distance to arena wall [mm]

Fig. 4.3 Significance of three-dimensional layout of the environment for responses of HS cells during intersaccadic intervals is shown by manipulating the distance of the fly to the walls of the flight arena. This was accomplished by increasing the size of the virtual arena while keeping the flight trajectory the same as in the original arena (see sketches of top views of the arenas above the data plot). HS cells were stimulated with the image sequences as would have seen by free-flying blowflies in the original flight arena and different virtual flight arenas. The data are based on six different trajectories (one of them shown

from different perspectives in inset). In addition, the translational movements were completely omitted and the fly only rotated with its natural rotation velocity in the centre of the arena (right data point). The mean response amplitudes between saccades decrease substantially when the distance of the animal to the arena wall gets smaller. For a minimal distance of approx. 1 m, virtually the same intersaccadic responses are obtained as without any translational movement at all. Data are averages based on 4 HSN cells and a total of 132 stimulus repetitions

4

Active Vision in Blowflies

either by downward, upward, front-to-back or back-tofront motion. However, the actual preferred directions change somewhat across the visual field according to the geometrical lattice axes of the compound eye [41, 52]. Two classes of TCs, the 3 HS cells [22, 23, 40] and the 10 VS cells [25, 26, 39], have been analysed in particular detail with respect to encoding the optic flow generated by self-motion of the fly. Whereas the dominant preferred direction of motion of HS cells is ipsilateral front-to-back motion in different overlapping areas of the visual field [23, 40], the dominant preferred direction of motion of VS cells is downward motion in overlapping neighbouring vertically oriented stripes of the visual field [17, 39]. HS and VS cells are output elements of the visual motion pathway and thought to be involved in controlling visually guided orientation behaviour. We could recently show that the populations of HS and VS cells make efficient use of the saccadic flight and gaze strategy of blowflies to represent spatial information. The head trajectories of flying blowflies and thus – because of their immobile eyes in the head capsule – the gaze direction could be determined in a laboratory setting with the help of a magnetic coil system [58, 70]. Using the knowledge of the threedimensional layout and wall patterns of the flight arena, the retinal image sequences were calculated at a high temporal resolution [47]. Under outdoor conditions, the flight paths and body orientations of freeflying blowflies were recorded with a pair of highspeed cameras. The retinal image sequences were assessed by moving a panoramic camera along the same path with a robotic gantry [4]. We used these behaviourally generated retinal image sequences as visual stimuli in electrophysiological experiments on HS and VS cells. On the basis of such experiments, the functional properties of these cells were interpreted in a different conceptual framework than in previous analyses: rather than being primarily viewed as sensors for determining rotational self-motion from the retinal optic flow patterns, they were concluded to provide also spatial information. Since blowflies keep their gaze virtually constant between saccades leading to prominent translational optic flow if environmental objects are sufficiently close to the eyes, the HS and VS cells can extract information about the spatial layout of the environment [4, 30–32]. For instance, the intersaccadic depolarisation level depends on the distance of the

57

blowfly to environmental structures (Fig. 4.3). As we could show in recent experiments, the sensitivity of HS cells to retinal velocity increments caused by nearby objects is even enhanced by motion adaptation, i.e. after the fly is exposed to repeated optic flow patterns for sometime [46].

4.4 A CyberFly: Performance of Experimentally Established Mechanisms Under Closed-Loop Conditions Given the ability of flies to perform extraordinary acrobatic flight manoeuvres, it is not surprising that there have been various attempts to implement fly-inspired optic flow processing into simulation models and on robotic platforms (for review [19, 51, 71, 75]; see also Chaps. 5 and 6). Although these approaches usually employed simplified model versions of the visual motion pathway, in all these studies the sensorimotor loop was closed. Most of them used optic flow information to stabilise the agent’s path of locomotion against disturbances or to avoid collisions with walls. On the other hand, only few attempts in robotics make use of the fly’s saccadic strategy of locomotion and of the implicit distance information present in translatory optic flow between saccades to implement obstacle avoidance [18, 55, 59, 75]. However, most of these agents generate very low dynamic movements compared to blowflies. In a recent study we implemented a saccadic controller that receives its sensory input from a model of the blowfly’s visual motion pathway and takes the specific dynamic features of blowfly behaviour into account. The model of the sensory system providing the input to the controller has been calibrated on the basis of experimentally determined responses of a major motion-sensitive output neuron of the blowfly’s visual system, one of the HS cells, to naturalistic optic flow, i.e. the visual input of flies in free-flight situations [48]. Apart from an array of retinotopically organised spatiotemporal filters which mimic the overall signal processing in the peripheral visual system, the core elements of the sensory model are elementary motion detectors of the correlation type (EMDs; see Sect. 4.3). These are spatially pooled by two elements – one in

58

M. Egelhaaf et al.

either hemisphere of the simulated brain – corresponding to the equatorial HS cell (HSE). To simulate the behaviour of a blowfly, the output signals of the sensory model are transformed into motor signals to generate behavioural responses. The properties of the motor controller determine how these motor signals are transformed into movements of the animal. By simulating the system in a closed control loop, hypotheses about the functional significance of the responses of sensory neurons and different types of sensorimotor interfaces can be tested. So far, the main task of this CyberFly has been to avoid collisions with obstacles, one of the most fundamental tasks of any autonomous agent [49]. Coupling the differential signal of the sensory neurons proportionally to the generated yaw velocities does not lead to sufficient obstacle avoidance behaviour. A more plausible sensorimotor interface is based on a saccadic controller modelled after the flight behaviour of blowflies. Here, the responses of simulated HSE cells in both halves of the visual system are processed to generate the timing and amplitude of saccadic turns. Timing and direction are determined by applying a threshold operation to the neuronal signals. The amplitudes of saccades are computed from their relative difference. The threshold used to initiate a saccade is very high just after the end of a preceding saccade, to prevent the CyberFly from generating

a)

saccades at unrealistically high frequencies. From this start value the threshold continually decreases, increasing the readiness for the generation of a new saccade with extending intersaccadic interval. Even with sideward drift after saccadic turns as is characteristic of real blowflies [58], the CyberFly is able to successfully avoid collisions with obstacles. The implicit distance information resulting from translatory movements between saccades is provided by the responses of model HS cells in the two halves of the visual system and appears to be crucial for steering the CyberFly safely in its environment (Fig. 4.4). A limitation of this simple mechanism is its strong dependence on the textural properties of the environment. A strong pattern dependence is also suggested by behavioural experiments of flight behaviour of Drosophila ([21]; see also Chap. 17). Currently, we are analysing in combined electrophysiological and behavioural experiments on blowflies as well as by model simulations the reasons for the sensitivity of the CyberFly to changes in the textural properties of the environment. Moreover, the current CyberFly, which operates so far only in the horizontal plane, is being elaborated to a fully three-dimensional model and implemented on a robotic gantry platform. These elaborations will be based on the recent knowledge of prototypical movements comprising all three degrees of freedom of rotation and translation as well as by

b) HSE

HSE R

L

>?

>?

|L−R| |L|+|R|

turn left

turn right

amplitude pattern− generator

pattern− generator

yaw rotation velocity

Fig. 4.4 Structure and performance of CyberFly: (a) Responses of simulated motion-sensitive neurons (HSE) are processed to generate the timing and amplitude of saccadic turns. Timing and direction are determined by a threshold operation, and amplitudes are computed from the contrast of the neuronal signals. A pattern generator replays yaw velocity templates computed from

behavioural data. (b) Example for the trajectories generated by CyberFly (grey marker: start). The big markers indicate position (spheres) and viewing directions. The texture of the cylindrical simulation environment is shown with reduced contrast to enhance perceptibility

4

Active Vision in Blowflies

taking larger populations of motion-sensitive tangential cells (HS cells and VS cells) into account. One aim of these modelling studies is to assess which aspects of the saccadic flight and gaze strategy of blowflies and of the underlying neural mechanisms will turn out to be particularly relevant and advantageous for flight performance and, thus, might prove suitable for implementation in micro-air vehicles.

4.5 Conclusions Although blowflies and many other insects are only equipped with a tiny brain, they operate with ease in complex and ever-changing environments. In this regard, they outperform any technical system. It is becoming increasingly clear that visually guided orientation behaviour of blowflies is only possible because the animal actively reduces the complexity of its visual input and the mechanisms underlying visual information processing make efficient use of this complexity reduction. By segregating the rotational from the translational optic flow generated during normal cruising flight, processing of information about the spatial layout of the environment is much facilitated. By adapting the neural networks of motion computation to the specific spatiotemporal properties of the actively shaped optic flow patterns evolution has tuned the blowfly nervous system to solve apparently complex computational tasks efficiently and parsimoniously. Biological agents such as blowflies generate at least part of their power as adaptive autonomous systems through efficient mechanisms acquiring their strength through active interactions with their environment and not by simply manipulating passively gained information about the world according to a predominantly predefined sequential processing scheme. These agent– environment interactions lead to adaptive behaviour in environments of a wide range of complexity. By cunningly employing the consequences of a closed action– perception loop animals’ even tiny brains are often capable of performing extraordinarily well in specific behavioural contexts. Model simulations and robotic implementations reveal that the smart biological mechanisms of motion computation and of controlling flight behaviour might be helpful when designing micro-air vehicles that may carry an on-board processor of only a relatively small size and weight.

59

References 1. Baird, E., Srinivasan, M.V., Zhang. S., Cowling, A.: Visual control of flight speed in honeybees. The Journal of Experimental Biology 208, 3895–3905 (2005) 2. Boeddeker, N., Egelhaaf, M.: Steering a virtual blowfly: Simulation of visual pursuit. Proceedings of the Royal Society of London B 270, 1971–1978 (2003) 3. Boeddeker, N., Kern, R., Egelhaaf, M.: Chasing a dummy target: Smooth pursuit and velocity control in male blowflies. Proceedings of the Royal Society of London B 270, 393–399 (2003) 4. Boeddeker, N., Lindemann, J.P., Egelhaaf, M., Zeil, J.: Responses of blowfly motion-sensitive neurons to reconstructed optic flow along outdoor flight paths. Journal of Comparative Physiology A 25, 1143–1155 (2005) 5. Borst, A., Egelhaaf, M.: Principles of visual motion detection. Trends in Neurosciences 12, 297–306 (1989) 6. Borst, A., Egelhaaf, M.: Detecting visual motion: Theory and models. In: F.A. Miles, J. Wallman (eds.) Visual motion and its role in the stabilization of gaze. Amsterdam: Elsevier (1993) 7. Borst, A., Haag, J.: Neural networks in the cockpit of the fly. Journal of Comparative Physiology A 188, 419–437 (2002) 8. Dahmen, H.J., Franz, M.O., Krapp, H.G.: Extracting ego-motion from optic flow: limits of accuracy and neuronal filters. In: J.M. Zanker, J. Zeil (eds.) Computational, neural and ecological constraints of visual motion processing. Berlin, Heidelberg, New York, Springer (2000) 9. Douglass, J.K., Strausfeld, N.J.: Pathways in dipteran insects for early visual motion processing. In: J.M. Zanker, J. Zeil (eds.) Motion vision: computational, neural, and ecological constraints. Berlin, Heidelberg, New York, Springer; 67–81 (2001) 10. Egelhaaf, M.: The neural computation of visual motion. In: E. Warrant, D.-E. Nilsson (eds.) Invertebrate vision. Cambridge, UK, Cambridge University Press (2006) 11. Egelhaaf, M., Borst, A.: Motion computation and visual orientation in flies. Comparative Biochemistry and Physiology 104A, 659–673 (1993) 12. Egelhaaf, M., Borst, A.: Movement detection in arthropods. In: J. Wallman, F.A. Miles (eds.) Visual motion and its role in the stabilization of gaze. Amsterdam, London, New York, Elsevier, pp. 53–77 (1993) 13. Egelhaaf, M., Kern, R., Kurtz, R., Krapp, H.G., Kretzberg, J., Warzecha, A.-K: Neural encoding of behaviourally relevant motion information in the fly. Trends in Neurosciences 25, 96–102 (2002) 14. Esch, H.E., Burns, J.M.: Distance estimation by foraging honeybees. The Journal of Experimental Biology 199, 155–162 (1996) 15. Esch, H.E. Zhang, S., Srinivasan, M.V., Tautz, J.: Honeybee dances communicate distances measured by optic flow. Nature 411, 581–583 (2001) 16. Farina, W.M., Varjú, D., Zhou, Y: The regulation of distance to dummy flowers during hovering flight in the hawk moth Macroglossum stellatarum. Journal of Comparative Physiology 174, 239–247 (1994)

60 17. Farrow, K., Borst, A., Haag, J.: Sharing Receptive Fields with Your Neighbors: Tuning the Vertical System Cells to Wide Field Motion. Journal of Neuroscience 25, 3985– 3993 (2005) 18. Franceschini, N., Pichon, J.M., Blanes, C.: From insect vision to robot vision. Philosophical Transactions of the Royal Society of London. Series B 337, 283–294 (1992) 19. Franz, M.O., Mallot, H.A.: Biomimetic robot navigation. Robots and Autonomous Systems, 133–153 (2000) 20. Frye, M.A., Dickinson, M.H.: Closing the loop between neurobiology and flight behavior in Drosophila. Current Opinion in Neurobiology 14, 729–736 (2004) 21. Frye, M.A., Dickinson, M.H.: Visual edge orientation shapes free-flight behavior in Drosophila. Fly 1, 153–154 (2007) 22. Hausen, K.: Motion sensitive interneurons in the optomotor system of the fly. I. The Horizontal Cells: Structure and signals. Biological Cybernetics 45, 143–156 (1982) 23. Hausen, K.: Motion sensitive interneurons in the optomotor system of the fly. II. The Horizontal Cells: Receptive field organization and response characteristics. Biological Cybernetics 46, 67–79 (1982) 24. Hausen, K., Egelhaaf, M.: Neural mechanisms of visual course control in insects. In: D. Stavenga, R.C. Hardie (eds.) Facets of vision. Berlin, Heidelberg, New York, Springer; 391–424 (1989) 25. Hengstenberg, R.: Common visual response properties of giant vertical cells in the lobula plate of the blowfly Calliphora. Journal of Comparative Physiology 149, 179–193 (1982) 26. Hengstenberg, R., Hausen, K., Hengstenberg, B.: The number and structure of giant vertical cells (VS) in the Lobula plate of the blowfly, Calliphora erythrocephala. Journal of Comparative Physiology 149, 163–177 (1982) 27. Hrncir, M., Jarau, S., Zucchi, R., Barth, F.G.: A stingless bee (Melipona seminigra) uses optic flow to estimate flight distances. Journal of Comparative Physiology Series A 189, 761–768 (2003) 28. Jain, A.K., Dubes, R.C.: Algorithms for clustering data. Prentice-Hall (1988) 29. Juusola, M., French, A.S., Uusitalo, R.O., Weckström, M.: Information processing by graded-potential transmission through tonically active synapses. Trends in Neurosciences 19, 292–297 (1996) 30. Karmeier, K., van Hateren, J.H., Kern, R., Egelhaaf, M.: Encoding of naturalistic optic flow by a population of blowfly motion sensitive neurons. Journal of Neurophysiology 96, 1602–1614 (2006) 31. Kern, R., van Hateren, J.H., Egelhaaf, M.: Representation of behaviourally relevant information by blowfly motionsensitive visual interneurons requires precise compensatory head movements. Jounal of Experimental Biology 209, 1251–1260 (2006) 32. Kern, R., van Hateren, J.H., Michaelis, C., Lindemann, J.P., Egelhaaf, M.: Function of a fly motion-sensitive neuron matches eye movements during free flight. PLOS Biology 3, 1130–1138 (2005) 33. Kern, R., Varjú, D.: Visual position stabilization in the hummingbird hawk moth, Macroglossum stellatarum L.: I. Behavioural analysis. Journal of Comparative Physiology Series A 182, 225–237 (1998)

M. Egelhaaf et al. 34. Kimmerle, B., Srinivasan, M.V., Egelhaaf, M.: Object detection by relative motion in freely flying flies. Naturwiss 83, 380–381 (1996) 35. Koenderink, J.J.: Optic Flow. Vision Research 26, 161–180 (1986) 36. Köhler, T., Röchter, F., Lindemann, J.P., Möller, R.: Bioinspired motion detection in an FPGA-based smart camera module Bioinspiration & Biomimimetics 4 (1:015008), 2009 doi: 10.1088/1748-3182/4/1/015008 37. Kral, K., Poteser, M.: Motion parallax as a source of distance information in locusts and mantids. Journal of Insect Behavior 10, 145–163 (1997) 38. Krapp, H.G.: Neuronal matched filters for optic flow processing in flying insects. In: M. Lappe (ed.) Neuronal processing of optic flow. San Diego, San Francisco, New York, Academic Press, pp. 93–120 (2000) 39. Krapp, H.G., Hengstenberg, B., Hengstenberg, R.: Dendritic structure and receptive-field organization of optic flow processing interneurons in the fly. Journal of Neurophysiology 79, 1902–1917 (1998) 40. Krapp, H.G., Hengstenberg, R., Egelhaaf, M.: Binocular contribution to optic flow processing in the fly visual system. Journal of Neurophysiology 85, 724–734 (2001) 41. Land, M.F., Eckert, H.: Maps of the acute zones of fly eyes. Journal of Comparative Physiology Series A 156, 525–538 (1985) 42. Lappe, M. Ed: Neuronal processing of optic flow. San Diego, San Francisco, New York, Academic Press (2000) 43. Laughlin, S.B: Matched filtering by a photoreceptor membrane. Vision Research 36, 1529–1541 (1996) 44. Lehrer, M.: Small-scale navigation in the honeybee: Active acquisition of visual information about the goal. The Journal of Experimental Biology 199, 253–261 (1996) 45. Lehrer, M., Srinivasan, M.V., Zhang S.W., Horridge, G.A.: Motion cues provide the bee’s visual world with a third dimension. Nature 332, 356–357 (1988) 46. Liang, P., Kern, R., Egelhaaf, M.: Motion adaptation facilitates object detection in three-dimensional environment. Journal of Neuroscience 29, 11328–1332 (2008) 47. Lindemann, J.P., Kern, R., Michaelis, C., Meyer, P., van Hateren, J.H., Egelhaaf, M.: FliMax, a novel stimulus device for panoramic and highspeed presentation of behaviourally generated optic flow. Vision Research 43, 779–791 (2003) 48. Lindemann, J.P., Kern, R., van Hateren, J.H., Ritter, H., Egelhaaf, M.: On the computations analysing natural optic flow: Quantitative model analysis of the blowfly motion vision pathway. Journal of Neuroscience 25, 6435–6448 (2005) 49. Lindemann, J.P., Weiss, H., Möller, R., Egelhaaf, M.: Saccadic flight strategy facilitates collision avoidance: Closedloop performance of a cyberfly. Biological Cybernetics 98, 213–227 (2007) 50. Longuet-Higgins, H.C., Prazdny, K.: The interpretation of a moving retinal image. Proceedings of the Royal Society of London. Series B 208, 385–397 (1980) 51. Neumann, T.R.: Biomimetic spherical vision. Universität Tübingen; 2004. 52. Petrowitz, R., Dahmen, H.J., Egelhaaf, M., Krapp, H.G.: Arrangement of optical axes and the spatial resolution in the compound eye of the female blowfly Calliphora.

4

53.

54. 55.

56.

57. 58.

59. 60.

61.

62.

63.

64.

Active Vision in Blowflies Journal of Comparative Physiology Series A 186, 737–746 (2000) Pfaff, M., Varjú, D.: Mechanisms of visual distance perception in the hawk moth Macroglossum stellatarum. Zoology Jb Physiology 95, 315–321 (1991) Prazdny, K.: Ego-Motion and Relative Depth Map from Optical-Flow. Biological Cybernetics 36, 87–102 (1980) Reiser, M.B., Dickinson, M.H.: A test bed for insectinspired robotic control. Philosophical Transactions of the Royal Society of London. Series A 361, 2267–2285 (2003) Schack, T.: The cognitive architecture of movement. International Journal of Sport & Exercise Psychology 2, 403– 438 (2004) Schilstra, C., van Hateren, J.H.: Stabilizing gaze in flying blowflies. Nature 395, 654 (1998) Schilstra, C., van Hateren, J.H.: Blowfly flight and optic flow. I. Thorax kinematics and flight dynamics. The Journal of Experimental Biology 202, 1481–1490 (1999) Sobey, P.J.: Active navigation with a monocular robot. Biological Cybernetics 71, 433–440 (1994) Srinivasan, M.V., Lehrer, M., Kirchner, W.H., Zhang, S.W.: Range perception through apparent image speed in freely flying honeybees. Visual Neuroscience 6, 519–535 (1991) Srinivasan, M.V., Lehrer, M., Zhang, S.W., Horridge, G.A.: How honeybees measure their distance from objects of unknown size. Journal of Comparative Physiology Series A 165, 605–613 (1989) Srinivasan, M.V., Zhang, S., Altwein, M., Tautz, J.: Honeybee navigation: Nature and calibration of the “odometer”. Science 287, 851–853 (2000) Srinivasan, M.V., Zhang, S.W., Lehrer, M., Collett, T.S.: Honeybee navigation en route to the goal: Visual flight control and odometry. The Journal of Experimental Biology 199, 237–244 (1996) Strausfeld, N.J., Douglass, J.K, Campbell, H.R., Higgins, C.M.: Parallel processing in the optic lobes of flies and the occurrence of motion computing circuits. In: E. Warrant,

61

65.

66.

67.

68.

69.

70.

71.

72.

73.

74.

75.

D.-E. Nilsson (eds.) Invertebrate vision, pp. 349–398 Cambridge, Cambridge University Press (2006) Tammero, L. F., Dickinson, M.H.: The influence of visual landscape on the free flight behavior of the fruit fly Drosophila melanogaster. The Journal of Experimental Biology 205, 327–343 (2002) Tautz, J., Zhang, S., Spaethe, J., Brockmann, A., Si, A., Srinivasan, M.: Honeybee odometry: performance in varying natural terrain. PLOS Biology 2, 915–923 (2004) Thoroughman, K.A., Shadmehr, R.: Learning of action through adaptive combination of motor primitives. Nature 407, 742–747 (2000) Thurau, C., Bauckhage, C., Sagerer, G.: Synthesizing movements for computer game characters, pp. 179– 1863175 Heidelberg, Springer (2004) van Hateren, J.H.: Processing of natural time series of intensities by the visual system of the blowfly. Vision Research 37, 3407–3416 (1997) van Hateren, J.H., Schilstra, C.: Blowfly flight and optic flow. II. Head movements during flight. The Journal of Experimental Biology 202, 1491–1500 (1999) Webb, B., Harrison, R.R., Willis, M.A.: Sensorimotor control of navigation in arthropod artifical systems. Arthropod Structure & Development 33, 301–329 (2004) Zeil, J.: Orientation flights of solitary wasps (Cerceris, Sphecidae, Hymenoptera). I. Description of flights. Journal of Comparative Physiology Series A 172, 189–205 (1993) Zeil, J.: Orientation flights of solitary wasps (Cerceris; Sphecidae; Hymenoptera). II. Similarities between orientation and return flights and the use of motion parallax. Journal of Comparative Physiology Series A 172, 207–222 (1993) Zeil, J., Kelber, A., Voss, R.: Structure and function of learning flights in bees and wasps. The Journal of Experimental Biology 199, 245–252 (1997) Zufferey, J.-C., Floreano, D,: Fly-Inspired Visual Steering of an Ultralight Indoor Aircraft. IEEE Transactions on Robotics 22, 137–146 (2006)

Chapter 5

Wide-Field Integration Methods for Visuomotor Control J. Sean Humbert, Joseph K. Conroy, Craig W. Neely, and Geoffrey Barrows

Abstract In this chapter wide-field integration (WFI) methods, inspired by the spatial decompositions of wide-field patterns of optic flow in the insect visuomotor system, are reviewed as an efficient means to extract visual cues for guidance and navigation. A control-theoretic framework is described that is used to quantitatively link weighting functions to behaviorally relevant interpretations such as relative orientation, position, and speed in a corridor environment. The methodology is demonstrated on a micro-helicopter using analog VLSI sensors in a bent corridor.

5.1 Introduction In recent years robotics research has seen a trend toward miniaturization, supported by breakthroughs in microfabrication, actuation, and locomotion [24], as described in Chap. 16. Engineers are on the verge of being able to design and manufacture a variety of microsystems; however, the challenge is to endow these creations with a sense of autonomy that will enable them to successfully interact with their environments. Scaling down traditional paradigms will not be sufficient due to the stringent size, weight, and power requirements of these vehicles (Chap. 21). Novel sensors and sensory processing architectures will need to be developed if these efforts are to be ultimately successful.

J.S. Humbert () Autonomous Vehicle Laboratory, University of Maryland, College Park, MD USA e-mail: [email protected]

Recently there has been considerable interest [1] (see Chaps. 2, 3, and 6) in utilizing optic flow for navigation as an alternative to the more traditional methods of computer vision [12] and machine vision [4]. The general approach has been to extract qualitative visual cues from optic flow and use these directly in a feedback loop. One example is the detection of expansion, which can be used as an indication of an approaching obstacle. In [22] it was shown that simple models of integrated expansion on the left versus the right eyes of fruit flies accounted well for saccadic behavior of freely flying animals in an arena. This technique was successfully implemented by Zufferey et al. (Chap. 6) where reflexive obstacle avoidance was demonstrated on lightweight, propeller-driven airplane. Navigation methods based on optic flow are mostly inspired by the insightful work of Srinivasan et al. (Chap. 2) who postulated a well-known heuristic, the centering response, observed in honeybees as they traversed a corridor. This heuristic states that in order to negotiate a narrow gap, an insect must balance the image velocity on the left and right retinas, respectively. Local navigation utilizing this centering technique has been described in recent approaches [19, 15], as well as by Franceschini et al. (see Chap. 3). An excellent review and summary of earlier work is given in [8]. In this chapter the method of wide-field integration is reviewed, an analogue to tangential cell processing inspired by the spatial decompositions of optic flow in the insect visuomotor system. The concept is based on extracting information for navigation by spatially decomposing wide-field patterns of optic flow with sets of weighting functions. Outputs are interpreted as encoding information about relative speed and proximity with respect to obstacles in the surrounding

D. Floreano et al. (eds.), Flying Insects and Robots, DOI 10.1007/978-3-540-89393-6_5, © Springer-Verlag Berlin Heidelberg 2009

63

64

J.S. Humbert et al.

environment, which are used directly for feedback. The methodology described herein is demonstrated on a ground vehicle and a micro-helicopter navigating corridor environments of varying spatial structure. In both examples, weighted sums of the instantaneous optic flow field about the yaw axis are used to extract relative heading and lateral position (and, in addition, relative speed for the ground vehicle). The resulting closedloop responses replicate the navigational heuristics described by Srinivasan in Chap. 2.

5.2 The Insect Visuomotor System The insect retina can be thought of as a map of the patterns of luminance of the environment. As an insect moves, these patterns become time dependent and are a function of the insect’s relative motion and proximity to objects through motion parallax. The rate and direction of these local image shifts, taken over the entire visual space, form what is known as the optic flow field (see Chap. 6, Sect. 6.3). Local estimates of optic flow are thought to be computed by correlation-type motion detectors in the early portion of the visual pathway. Subsequently, these outputs are pooled by approximately 60 tangential neurons in the third visual neuropile of each hemisphere (see Chap. 4, Sect. 4.3), a region referred to as the lobula plate

(Fig. 5.1A). Extraction of wide-field visual information occurs at the level of these tangential cells, which spatially decompose complicated motion patterns over large swaths of the visual field into a set of feedback signals used for stabilization and navigation [5, 2]. Descending cells, which receive dendritic input from tangential cells, connect to the neurons in the flight motor to execute changes in wing kinematics. Tangential cells respond selectively to stimuli by either graded shifts in membrane potential or changes in spiking frequency, depending on class [9]. Shifts are depolarizing (or spiking frequency increases) if the motion pattern is in the preferred direction and hyperpolarizing (or spiking frequency decreases) in the null direction [10, 11]. In addition to direction selectivity, tangential cells also exhibit spatial selectivity within their receptive fields [10, 11, 18, 17], as shown in Fig 5.1B. Due to their motion sensitivity patterns, which are similar to the equivalent projected velocity fields for certain cases of rotary self-motion, it has been postulated that tangential cells function as direct estimators of rotational velocity [17]. However, recent work has shown that translational motion cues, which are the source of proximity information, are also present in the outputs of cells that were previously thought to be used only for compensation of rotary motion (Chap. 4, Sect. 4.2). This suggests that cell patterns might be structured to extract a combination of relative

Motion Detection

A

B

Lamina Medulla

VS1 Lobula Plate

Dendritic Arborization

Tangential Cells

Photoreceptors

VS1 Cell Sensitivity Pattern

Descending Cells

Fig. 5.1 (A) Visuomotor system structure, (B) VS1 tangential cell and associated wide-field sensitivity pattern. The data were extracted and replotted from [17]

elevation (deg)

To Flight Motor 50

0

−50 0

50

100

azimuth (deg)

150

5

Wide-Field Integration Methods for Visuomotor Control

65

speed and proximity cues, rather than direct estimates of the velocity state. Hence, significant progress has been made in understanding structure, arrangement, and synaptic connectivity [5]; however, the exact functional role that each of these neurons hold in the flight control and navigation system of the fly remains a challenging and open question.

5.3 Wide-Field Integration of Optic Flow The goal of this research is to investigate how the spatial decompositions of optic flow by tangential cells, described here as wide-field integration, might be used in closed-loop feedback to explain the visualbased behaviors exhibited by insects. The approach is based on a novel premise: tangential cells are not used to directly estimate self-motion quantities of a flying organism as in traditional implementations [7, 6]. Rather, it is assumed their purpose is to detect departures from desired patterns of optic flow, generating signals that encode information with respect to the surrounding environment. The resulting set of signals can be used in a feedback loop to maintain a safe distance from obstacles in the immediate flight path. The intuition behind this approach is shown in Fig 5.2. Forward motion of a vehicle constrained to move in 3 DOF (forward and lateral translation along

with yaw rotation) in the horizontal plane generates an optic flow pattern with a focus of expansion in the front field of view, a focus of contraction in the rear, with the largest motion on the sides. If plotted as a function of the angle δ around the retina, this is approximately a sine wave (Fig. 5.2A). Perturbations from this equilibrium state of a constant forward velocity u0 along the centerline of the tunnel introduce either an asymmetry in this signal for lateral displacements δy (Fig. 5.2B) or a phase shift for rotary displacements δψ (Fig. 5.2C). If the forward speed is increased by δu, the amplitude of this signal increases (Fig. 5.2D), and if the vehicle is rotating at angular velocity ψ˙ about the yaw axis a DC shift in the signal of equal magnitude occurs. Therefore, the amplitude, phase, and asymmetry of the pattern of optic flow around the yaw axis encode important information that could be used for navigation and speed regulation. The structure of the visuomotor pathway of insects (Fig. 5.1A) gives us a hint as to how extracting this type information from patterns of optic flow might be achieved simply and quickly. Tangential cells, which parse the complicated patterns of optic flow generated during locomotion, exhibit either a shift in membrane potential or an increase in spike frequency when they are presented with a wide-field pattern they are tuned to. Essentially, each cell makes a comparison between its preferred pattern sensitivity and the pattern of the stimulus. Mathematically, this process can be represented as an inner product u,v, analogous to the dot

B

δy

γ

z xb

yb

x

Lateral Displacement

y

C

δψ

δψ γ

δy Fig. 5.2 Perturbations of azimuthal optic flow and their correlations to the relative state of the vehicle. Amplitude, phase, and asymmetry of the azimuthal optic flow pattern encode relative proximity and speed with respect to obstacles in the environment

a

δψ

Orientation Shift

Optic Flow Pattern

A

D

Left

δu

γ γ

Right

Centered Motion

Forward Speed Increase

γ

66

J.S. Humbert et al.

product between vectors, which tells us how similar two objects u and v are. In the case of Fig. 5.2, the patterns are assumed to reside in L2 [0,2π], the space of 2π-periodic and square-integrable functions, where the inner product is given by ˙ i = zi (x) = Q,F



2π

˙ ,x) · Fi (γ ) dγ . Q(γ

(5.1)

0

˙ ,x) is the measured optic flow about the yaw Here Q(γ axis, Fi (δ) is any square-integrable weighting function such that (5.1) exists, and zi (x) is the resulting output which is a function of the relative state x (proximity and velocity) of the insect with respect to the environment. This expression, which could represent either a shift in membrane potential or a change in spiking frequency, gives a number that is maximum if the patterns line up, negative if the pattern has the same structure but is in the opposite direction, and zero if the patterns are completely independent (orthogonal) of one another. The resulting set of outputs represents a decomposition of the motion field into simpler pieces that encode perturbations from the desired pattern. Navigation behavior can be achieved by implementing a feedback loop which attempts to maintain a sine wave pattern of optic flow (Fig. 5.2A) on the circular imaging surface. Departures from the desired relative state create spatial perturbations that can be extracted with the above tangential cell analogue (5.1). For example, an estimate of the phase shift is given by ˙ against an F(γ) = cos γ weighting funcintegrating Q ˙ against tion, relative speed results from integrating Q an F(γ) = sin γ weighting function, and asymmetry can be extracted with an F(γ) = cos 2γ weighting function. These correspond to the first cosine a1 , first sine b1 , and second cosine a2 Fourier harmonics of the optic flow signal. These low-order spatial harmonics yield the orientation, speed, and lateral position relative to the corridor and can be used as feedback commands to maneuver the vehicle accordingly to maintain the desired sine wave pattern of optic flow. Rotational motion ψ˙ about the yaw axis also introduces an asymmetry between the left and right values of the optic flow signal; however, since the result is a pure DC shift, the F(γ) = cos 2γ weighting extracts only the portion of the asymmetry due to a lateral displacement δy. To extract the rotational motion ψ˙

directly, a function which is a combination of a DC shift and a cos 2γ weighting would be required as a lateral displacement δy also introduces a change in the DC value of the signal. Something similar occurs when attempting to extract the relative heading. A pure orientation offset δψ creates a phase shift in all the harmonics of the optic flow signal; however, a lateral velocity δv will introduce a shift in only the low-frequency harmonics. This is not an issue for platforms which have sideslip constraints such as a ground vehicle [14]. For other platforms such as hovercraft or helicopters, a coupling between the orientation ψ and lateral position y degrees of freedom is introduced which has a detrimental effect on the achievable closed-loop bandwidth in the lateral degree of freedom [15]. The extension of the proposed wide-field integration methodology to 3D environments and 6-DOF vehicles has shown that this issue can be resolved by utilizing optic flow about either the pitch or roll axis, which can be used to unambiguously decouple the two distinct motions [16]. The primary advantage of this framework is that it can be used to quantitatively link weighting functions to interpretations such as relative orientation, position, and speed, as shown above. For standard obstacles (one ˙ wall, two walls, cylinders, etc.) one can express Q in closed form, analytically compute (5.1), and subsequently linearize about the state corresponding to desired equilibrium pattern of optic flow. This results in an output equation y = Cx, which is a linear function of the relative state. Analysis tools from control theory, such as output LQR [21], can then be applied to derive gains for desired stability and performance. Results from an experimental implementation on a wheeled robot [13] are shown in Fig. 5.3. Optic flow is computed using an OpenCV implementation of the Lucas–Kanade algorithm around a 360◦ ring from imagery generated by a vertically mounted camera which points upward at a parabolic mirror. Spatial harmonics (integral weightings) of optic flow, encoding perturbations from the equilibrium pattern, are used as control inputs to maneuver the vehicle. Regulation of the sine wave pattern on the mirror is achieved by using these signals as feedback to force or torque accordingly. Therefore, this simple visuomotor control architecture (wide-field integration) gives rise to the centering and clutter navigational heuristics observed in insect behavior [20].

5

Wide-Field Integration Methods for Visuomotor Control

A

67

B

Finish

rad/s

y (m)

1

0

1

0.1 0 −0.1

Start

0

−0.2

2

x (m)

0

2

4

6

8

10

Time (s) 0.6

5

b1

0.5

4

y (m)

a1 a2

0.2

2

C

Fourier Coefficients

0.3

u b1

(rad/s)

0.4

3

0.3 2 0.2

Fwd Speed

1

u (m/s)

0.1 0 −1

0

0

1

0

1

2

3

4

Distance Along Tunnel (m)

x (m)

Fig. 5.3 (A) Autonomous corridor navigation of a ground vehicle using wide-field integration techniques. (B) Navigation of a 90◦ bend. The mean values of the position in the corridor along with all 20 individual trials are plotted for a combined lateral and rotational offset, along with the corresponding Fourier harmon-

ics a1 and a2 of the optic flow signal. (C) Converging–diverging corridor navigation. The first sine harmonic b1 of the optic flow is held constant over 20 runs by modulating the forward speed, resulting in a speed decrease as the vehicle navigates the narrow gap

5.4 Application to a Micro-helicopter

closed using optic flow-derived estimates. No altitude control is required as the helicopter is sufficiently stable for a constant thrust in ground effect. Optic flow is measured in a tangential ring arranged around the azimuth of the helicopter (Fig. 5.4B). Due to a lack of available payload for the camera and parabolic mirror configuration, VLSI sensors are used for optic flow estimation. The increased noise inherent in the VLSI implementation of the optic flow algorithms (see Chap. 8) along with the increased vibrations present on the helicopter reduces the quality of the optic flow estimates. It is demonstrated, however, that the wide-field integration methodology is sufficient to affect navigation capabilities.

The preceding experimental implementation of optic flow-based navigation on the ground vehicle has been extended to the case of an electric micro-helicopter (Fig. 5.4A). The goal in this demonstration is to provide autonomous navigation of a bent corridor environment using estimates of relative heading and lateral position derived from weighted sums of the instantaneous optic flow field about the yaw axis. Pitch, roll, and forward speed control of the helicopter is accomplished with an external marker-based visual tracking system while the heading and lateral control loops are

A

Fig. 5.4 (A) E-Sky Honeybee micro-helicopter, and (B) optic flow sensor ring with six ARZ-Lite sensors from Centeye, Inc

B

Micro-Helicopter

Optic Flow Sensor Ring

68

J.S. Humbert et al.

Fig. 5.5 Closed-loop block diagram for WFI-based navigation. Inner loop pitch and roll control are accomplished using Vicon measurements while optic flow-derived outputs are used to close the outer navigation loop

For a helicopter, the high-order dynamics and a large degree of coupling require additional consideration, thus making implementation more complex than that of the ground vehicle. Additionally, the motion is unconstrained such that sideslip, forward speed, and rotation can all be varied independently. A state-space model of the helicopter dynamics, linearized about the hover condition, has been identified in prior work [3]. This identified model confirms that the lateral and longitudinal dynamics are highly coupled and of high order. Additionally, the heading degree of freedom is highly sensitive to the yaw input and disturbances due to the low rotational damping and inertia. This degree of freedom is immediately modified via onboard integrated yaw rate feedback to add additional damping. While the helicopter is clearly not a planar constrained vehicle in general, given a fixed altitude and non-aggressive maneuvers, a planar analysis and implementation can be assumed. A ViconTM visual tracking system provides an offboard feedback capability that reduces the effective dynamics of the vehicle. Direct measurements of the vehicle state x = (x,y,ψ,u,v,r) are available at 350 Hz at 10 ms latency and provide ground truth data for the experiment. Attitude feedback for the roll and pitch degrees of freedom greatly reduces the cross-coupling effects inherent in the dynamics, thus allowing the lateral and longitudinal degrees of freedom to be considered separately. The resulting set of reduced order dynamics is given by v˙ = Yv v + gφr u˙ = Xu u − gθr ,

(5.2)

where Yv and Xu are aerodynamic damping derivatives [23], g is gravity, and φr and θ r are the commanded orientation values for roll and pitch, respectively. For low frequencies these can be considered to be the same as the actual roll and pitch values, φ and θ . The corresponding transfer function form, given only for the lateral degree of freedom, is v φr

=

g . s + Yv

(5.3)

The effective lateral and longitudinal transfer functions can be further modified via a simple proportional velocity tracking feedback using a Vicon estimate for the lateral velocity v (Fig. 5.5). Given a control gain Kv and a commanded velocity vr , the resulting transfer function for the lateral degree of freedom is v vr

=

Kv g . s + (Yv + Kv g)

(5.4)

A transfer function of identical form is valid for the longitudinal degree of freedom. The velocity tracking loop further reduces the effect of the cross-coupling in the vehicle dynamics and adds effective damping. In the current experiment (Fig. 5.6) the forward commanded velocity ur is fixed to a constant value u0 = 0.6 m/s, whereas the desired lateral velocity reference vr can be used as a control input to laterally maneuver the helicopter. Heading control is also augmented using the visual tracking system to supply rate tracking capability. The open-loop heading dynamics are given as

5

Wide-Field Integration Methods for Visuomotor Control

Fig. 5.6 (A) Bent corridor test environment, (B) ˙ azimuthal optic flow pattern Q measured at time t = 2.7, (C) helicopter trajectory through the corridor along with the relative heading given by the angle of the L-shaped markers with respect to the vertical, and (D) time history of the controlled inputs uy and uψ .

69 Test Corridor

A

B

rad/s

0.5

−1

y C

v

u ψ

uy

meters

6

1 0 0

0

uψ

2

0.5

4

6

8

t = 2.7 sec

0

−0.5

−2

−1

0 meters

r˙ = Nr r + Nμy μy ,

(5.5)

where r is the yaw rate, Nr is the yaw damping, Nμy is the control sensitivity, and μy is the tail motor input. The term Nr is naturally aerodynamic in origin; however, we include in this term the effects of the onboard yaw rate damping as well. Simple proportional feedback is again used to provide yaw rate tracking capability (Fig. 5.5), resulting in the following transfer function, from desired reference rate, rr , to the actual rate, r is:

s + (Nr + Kr Nμy )

2 4 azimuth (rad)

−1

t = 2.7 sec

Kr Nμy

0

Controlled Inputs

D

Trajectory

1

=

0 −0.5

−1

r rr

Optic Flow (@ t = 2.7sec)

1

.

(5.6)

The desired yaw velocity reference rr can be used as a control input to modulate the heading of the helicopter. We express the tangential component of the optic flow on a circular-shaped sensor that is constrained to 3-DOF motion in the horizontal plane for a general configuration of obstacles as follows: ˙ ,x) = −r + μ(γ ,x) (u sin γ − v cos γ ). Q(γ

(5.7)

This expression is a 2π -periodic function in the viewing angle δ and the state of the vehicle x = (x,y,ψ,u,v,r). The function μ(γ ,x) = 1/d(γ ,x) is defined as the

1

2

3

0

2

4 time (sec)

6

8

nearness, where d(γ ,x) is a continuous representation of distance to the nearest point in the visual field from the current pose of the vehicle within the environment. For the case of a straight corridor, the nearness function μ(γ ,x) is independent of the axial position and can be expressed in closed form as a function of the lateral position y, the relative body frame orientation ψ, and the tunnel half-width a: ⎧ sin (γ + ψ) ⎪ ⎪ 0≤γ +ψ 0) a = 0.191 m (distance from d1 to the center of rotation, defined by aFD1 = bFD2 ) b = 0.268 m (distance from d2 to the center of rotation, defined by aFD1 = bFD2 ) M = 0.025 kg (machine mass) g = 9.8 m2 /s (acceleration due to gravity)

where d1,2,w are of the form 12 ρCd A, ρ is the fluid density, Cd is the drag coefficient, and A is the crosssectional area. In order to determine the stability of the system we can put it into state space, and examine the eigenvalues, in addition to some test case scenarios. The

13 A Passively Stable Hovering Flapping MAV

181

Fig. 13.12 System response for (a) a stable system, with the center of mass located below the pivot point. The system started at (0,0) with a tilt angle of 10◦ at time=0 (open circle) and proceed to stabilize, ending at the black circle. (b) unstable system response with the center of mass located above the pivot point. The system started at (0,0) with a tilt angle of 10◦ at time=0 (open circle) and did not stabilize, ending at the black circle

Ft = 1.05 ∗ M ∗ g (thrust slightly higher than the hovering condition) c = 0.00833 m (distance from center of rotation to center of mass) Analyzing this system yields the eigenvalues: {0, −1589.1, −2.2, −0.7, 0, −0.4}. The two zero-value eigenvalues are due to the rigid body modes. This can be verified by adding a spring in the x and y dimensions, causing all the eigenvalues to be negative.

13.5 Performance Results The design can be operated under a variety of configurations, depending on the need for longer flight times or increased payload capacity for tools such as sensors and cameras. Power, lift, and flapping frequency were measured using a digital multimeter, a scale, and strobe light (Fig. 13.2). Under maximum lift conditions the craft operates at a Reynolds Number of roughly 8,000.

182

F. van Breugel et al.

Fig. 13.13 Operating characteristics. Lift and frequency for various power arrangements. The nonlinearity at 7.5 W is likely due to a second oscillatory mode resembling a standing wave that appeared at this power. Furthermore the motor efficiency becomes nonlinear as the power increases. We operated the machine just below this point, at 6.9 W

Our current mode of operation uses two 3.7 V, 90 mA h Li-Poly batteries in series to provide a nominal output voltage of 7.4 V. At these voltages the flapping transitions into a second oscillatory mode: a standing wave with nearly zero lift production. In order to minimize this effect we implemented a 0.81/2 resistance to reduce the voltage. This also reduced the stress on the motors and lowered the maximum height achieved. Operating conditions were measured to 6.5 V at 1.07 A for a total lift of 25 g capable of 33 s flight. The poor performance is likely due to the motors being driven far beyond their recommended operating characteristics, lack of a voltage regulator, and unoptimized wing structure (see Sect. 13.5.1) (Fig. 13.13).

13.5.1 Future Design Changes In order to improve the machines performance there are a number of issues that need to be addressed. First, several of the components (motors, batteries, and wires) are being operated at a much higher power than is recommended. With the addition of an efficient voltage regulator and by minimizing the length and amount of wiring we can further reduce wasted power. Beyond component optimization the wing shape and materials need to be examined more carefully. During our design process we systematically tried various wing shapes, sizes, and material properties, yet were unable to find a simple method to help us optimize the wing

13 A Passively Stable Hovering Flapping MAV

shape. Once we found the presented design to work we moved on to other areas of the machine. The first step is to design reliable experiments for measuring lift with systematic changes to the wings. This is made difficult due to the fact that the machines’ flapping characters change when it is tethered to a stiff stand. We will need to investigate a number of attributes including size, geometry, aspect ratio, mass distribution, flexibility, and flexibility distribution. Having a more flexible trailing than leading edge may help increase lift by reducing the formation of trailing edge vortices which destroy the lift enhancing leading edge vortices [14]. Different mechanisms of flapping can also be explored. Currently, the motor is attached directly to the wings simulating a direct method of flapping. Perhaps indirect mechanisms of flapping as observed in the Drosophila could be used. Drosophila expands and contracts its thorax to produce its flapping motion, using elastic energy storage for increased efficiency. Perhaps the use of a more biomimetic actuator would further improve our efficiency. We also began to explore altitude control as a first step into implementing active control of the flapper by controlling each motor individually. We used a four-channel receiver from Plantraco (Micro9-S-4CH 1.1 g Servo Rx w/ESC) and a Plantraco transmitter (HFX900 M2). We converted Blue Arrow 2.5 g servos from positional control to speed control to make use of the four-channel receiver. The difficult design constraint here was that we needed four-channels of control, while most lightweight electronics are made for two–three control outputs. On testing, we found that the performance of the modified servos was unstable. There was also an appreciable drop in voltage from the batteries to the motors. This resulted in poor lift generation. Our next approach was to connect the four motors in parallel and wire them to the throttle output which had a 2 A output. (At this point we could have changed the design to use a different receiver and transmitter with just a single channel.) With an external power source of 7 V at maximum throttle, the receiver would consistently let out a sharp beep and the motors would start to reduce their flapping frequency as opposed to maintaining a frequency suitable for substantial lift generation. This could be due to the overloading of the throttle which was designed for only one motor. Here we are loading it with four motors. Control could also be achieved by altering the stroke plane as opposed to altering the power of the motors. Poten-

183

tially, the four connections of the module connector to an individual module could be replaced with material that mimics muscle. Looking at Fig. 13.8a, if the connector arms are replaced with an actuator that contracts like a muscle, the stroke plane can be tilted when the “artificial muscle” contracts. Shape memory alloys (SMAs) and piezo actuators come to mind. However, SMAs generate substantial amounts of heat and piezo actuators require a high voltage to achieve appreciable strain. Therefore, issues of weight and input power need to be addressed before such an artificial muscle can be used in the flapper.

13.6 Conclusion While the performance of the machine presented here needs to be improved, it demonstrates several design considerations relevant to others interested in stable and controllable hovering flapping MAVs. The quadflapper design allows for active control using the thrust from the wings, rather than additional rudders and actuators. Since our machine operates within the same aerodynamic flow regime of insects, these design principles should scale favorably to insect-sized machines. While for real-world applications active control is critical, in designing and optimizing the thrust portions of small-scale flapping MAVs the use of sails for passive stability may prove to be much simpler than trying to build an active controller small enough that it can be carried on board. Ultimately we hope hovering flapping flight will open the door to new applications and provide further insight into the mechanisms underlying insect and hummingbirds remarkable flying abilities. Acknowledgments We would like to thank our funding sources for supporting this project, including Cornell Presidential Research Scholars, the NASA Space Consortium, and the NASA Institute for Advanced Concepts.

References 1. Arrow, B.: Wing bird rc flying bird (2005). http:// flyabird.com/wingbird.info.html 2. Berman, G., Wang, J.: Energy-minimizing kinematics in hovering insect flight. Journal of Fluid Mechanics 582, 153–168 (2007)

184 3. Chronister, N.: The ornithopter zone – fly like a bird – flapping wing flight. http://www.ornithopter.org/ 4. Combes, S.A., Daniel, T.L.: Into thin air: contributions of aerodynamic and inertial-elastic forces to wing bending in the hawkmoth manduca sexta. The Journal of Experimental Biology 206, 2999–3006 (2003). 10.1242/jeb.00502. http://jeb.biologists.org/cgi/content/abstract/206/17/2999 5. Dalton, S.: Borne on the Wind. Reader’s Digest Press, New York (1975) 6. DeLaurier, J., SRI/UTIAS: Mentor project (2005). http://www.livescience.com/technology/041210_project_ ornith%opter.html 7. Dickinson, M.H., Lehmann, F.O., Sane, S.P.: Wing rotation and the aerodynamic basis of insect flight. Science 284, 1954–1960 (1999). 10.1126/science.284.5422.1954. http://www.sciencemag.org/cgi/content/abstract/284/5422/ 1954 8. Dickson, W., Dickinson, M.H.: Inertial and aerodynamic mechanisms for passive wing rotation. Flying Insects and Robotics Symposium, p. 26 (2007) 9. Ellington, C.: The novel aerodynamics of insect flight: applications to micro-air vehicles. The Journal of Experimental Biology 202, 3439–3448 (1999). http://jeb. biologists.org/cgi/content/abstract/202/23/3439 10. Ellington, C.P., van den Berg, C., Willmott, A.P., Thomas, A.L.R.: Leading-edge vortices in insect flight. Nature 384, 626–630 (1996). 10.1038/384626a0. http://dx. doi.org/10.1038/384626a0 11. Fearing, R., Wood, R.: Mfi project (2007). http://robotics.eecs.berkeley.edu/ronf/MFI/index.html 12. Jones, K., Bradshaw, C., Papadopoulos, J., Platzer, M.: Bioinspired design of flapping-wing micro air vehicles. Aeronautical Journal 109, 385–393 (2005)

F. van Breugel et al. 13. Keennon, M., Grasmeyer, J.: Development of two mavs and vision of the future of mav design. 2003 AIAA/ICAS International Air and Space Symposium and Exposition: The Next 100 Years (2003) 14. Lehmann, F.O.: When wings touch wakes: understanding locomotor force control by wake wing interference in insect wings. The Journal of Experimental Biology 211, 224–233 (2008). 10.1242/jeb.007575. http://jeb. biologists.org/cgi/content/abstract/211/2/224 15. Lehmann, F.O., Sane, S.P., Dickinson, M.: The aerodynamic effects of wing-wing interaction in flapping insect wings. The Journal of Experimental Biology 208, 3075–3092 (2005). 10.1242/jeb.01744. http://jeb. biologists.org/cgi/content/abstract/208/16/3075 16. Lentink, D., Team, D.: Delfly (2007). http://www.delfly.nl/ 17. Michelson, R.: Entomopter project (2003). http://avdil.gtri. gatech.edu/RCM/RCM/Entomopter/EntomopterPro%ject. html 18. Michelson, R., Naqvi, M.: Extraterrestrial flight. Proceedings of von Karman Institute for Fluid Dynamics RTO/AVT Lecture Series on low Reynolds Number Aerodynamics. Brussels, Belgium (2003) 19. Wang, J.: Dissecting insect flight. Annual Review of Fluid Mechanics 37, 183–210 (2005). http://arjournals. annualreviews.org/doi/abs/10.1146/annurev.f%luid.36. 050802.121940?cookieSet=1&journalCode=fluid 20. Woods, M.I., Henderson, J.F., Lock, G.D.: Energy requirements for the flight of micro air vehicles. Aeronautical Journal 105, 135–149 (2001) 21. Wowwee: Wowwee flytech dragonfly toy 2007). http:// www.radioshack.com/product/index.jsp?productId= 2585632%&cp&cid=

Chapter 14

The Scalable Design of Flapping Micro-Air Vehicles Inspired by Insect Flight David Lentink, Stefan R. Jongerius, and Nancy L. Bradshaw

Abstract Here we explain how flapping micro air vehicles (MAVs) can be designed at different scales, from bird to insect size. The common believe is that micro fixed wing airplanes and helicopters outperform MAVs at bird scale, but become inferior to flapping MAVs at the scale of insects as small as fruit flies. Here we present our experience with designing and building micro flapping air vehicles that can fly both fast and slow, hover, and take-off and land vertically, and we present the scaling laws and structural wing designs to miniaturize these designs to insect size. Next we compare flapping, spinning and translating wing performance to determine which wing motion results in the highest aerodynamic performance at the scale of hummingbirds, house flies and fruit flies. Based on this comparison of hovering performance, and our experience with our flapping MAV, we find that flapping MAVs are fundamentally much less energy efficient than helicopters, even at the scale of a fruit fly with a wing span of 5 mm. We find that insect-sized MAVs are most energy effective when propelled by spinning wings.

14.1 Introduction Recently, micro-air vehicles (MAVs) have gained a lot of interest of both aerospace engineers and biologists studying animal flight. Such small planes are of special

D. Lentink () Experimental Zoology Group, Wageningen University, 6709 PG Wageningen, The Netherlands; Faculty of Aerospace Engineering, Delft University of Technology, 2629 HS Delft, The Netherlands e-mail: [email protected]

interest because they have many promising civil and military applications: from inspection of buildings and other structures to silent and inconspicuous surveillance. These small sensor platforms with a wingspan of less than 10 in. can potentially be equipped with various micro-sensors ranging from multiple microphones and cameras to gas detectors. But how can one design such small planes best? One major problem is aerodynamics. MAVs have a size and flight speed comparable to insects and small birds, which are much smaller and slower than airplanes. The aerodynamic effect of low speed and small size is quantified by the Reynolds number. The Reynolds number is the ratio of inertia and viscous stress in the flow and ranges roughly from 10 to 100,000 for insects to birds, and MAVs, whereas it ranges from 300,000 to 100,000,000 for airplanes. The low Reynolds number aerodynamics of MAVs is therefore more similar to that of flying birds and insects than that of airplanes. Only little is known about the aerodynamics in the low Reynolds number domain, which is studied mainly by biologists. Many engineers have therefore looked for biological inspiration for the design of their MAVs. Of special interest are insect-sized MAVs that can actually fly like insects using flapping wings: ornithopters. It is not widely known that several bird-sized, freely flying, ornithopters have been built and successfully flown even before Otto Lilienthal and the Wright brothers took off into the air with their, now, conventional airplanes. Ever since there have been few, but successful, amateur ornithopter enthusiasts that have developed many bird-sized ornithopters. The most successful predecessor of DelFly, a flapping MAV which we present here, is the AeroVironment Microbat, which could fly

D. Floreano et al. (eds.), Flying Insects and Robots, DOI 10.1007/978-3-540-89393-6_14, © Springer-Verlag Berlin Heidelberg 2009

185

186

for 42 s [14]. An up-to-date historic overview of flapping MAVs can be found on www.ornithopter.org. Developing a working insect-sized ornithopter that can both hover, fly fast and take off and land vertically like insects can remain, however, an open challenge. Not only because of small scale but also because we only know since roughly 10 years how insects can generate enough lift with their wings to fly [4, 3, 5, 18]. Biologists have found that a key feature that enables insects to fly so well is the stable leading edge vortex that sucks their wings upward, which augments both wing lift and drag. Building MAVs at the size of insects is even more challenging, because the critical components for successful flight cannot yet be bought at small enough size, low enough weight and high enough efficiency. Further, special production processes and design strategies are needed to build micro-flapping wings that function well at the length scale of insects. The ultimate dream of several engineers and biologists is to build a fruit fly-sized air vehicle. Here we present an integrated design approach for micro-air vehicles inspired by insect flight that really fly and can be scaled from bird size to insect size.

14.2 The Scalable Wing Aerodynamics of Hovering Insects To quantify if the aerodynamics of hovering insect wings are scalable from bird size to insect size we chose to study hovering flight, because it is the most

Fig. 14.1 Flapping fly wings generate more lift than translating fly wings. Stroke-averaged lift–drag coefficient polar of a translating (dark grey triangle), simple flapping (light grey circles) and realistically flapping (star) fruit fly wing at Re = 110. The angle of attack amplitude ranges from 0◦ to 90◦ in steps of 4.5◦ [11]

D. Lentink et al.

power and lift demanding flight phase. Under hover conditions we measured the aerodynamic forces generated by a fruit fly wing model at Reynolds numbers (Re) of 110 (fruit fly sized), 1400 (house fly sized) and 14,000 (hummingbird sized). We performed these experiments with a robotic insect model at Caltech, RoboFly [5], in collaboration with Michael Dickinson (for details and methods, see [11]). The RoboFly setup consists of a tank filled with mineral oil in which we flapped a fruit fly-shaped wing using both measured and simplified fruit fly kinematics. The simplified fruit fly kinematics consists of sinusoidal stroke and filtered trapezoidal angle of attack kinematics. The stroke amplitude of 70◦ (half the full amplitude defined in [2]) is based on the measured kinematics of six slowly hovering fruit flies [6]. The angle of attack amplitude was varied from 0◦ to 90◦ in steps of 4.5◦ , which encloses the full range of angle of attacks relevant for the flight of flies (and other insects). The lift and drag measurements at fruit fly scale, Re = 110, show that flapping fruit fly wings generate roughly twice as high lift coefficients as translating ones (lift coefficient is equal to lift divided by the product of the averaged dynamic pressure and wing surface area; the drag coefficient is similarly defined). RoboFly force measurements using actual kinematics of slow hovering fruit flies reveal that fruit flies indeed generate much more lift by flapping their wing than generated when the same wing is simply translated, like an airplane, Fig. 14.1. The force coefficients measured for fruit fly kinematics overlap with the lift–drag coefficient polar generated with the simplified fruit fly flap kinematics, Fig. 14.1. The elevated lift and drag forces generated by a fruit fly wing are due to a stably attached leading edge vortex (LEV) on top of the wing, which sucks the wing upward, Fig. 14.2. A stable LEV that explains the elevated lift forces generated by hovering insects was first found for hawkmoths [3]. Dickinson et al. [5] measured the actual unsteady forces generated during a flap cycle of a fruit fly wing, which showed that indeed much, up to 80%, of the total lift can be attributed to the ‘quasi-steady’ lift contribution of the stable leading edge vortex. To test if the aerodynamics of flapping fly wings is indeed scalable we flapped the same wing, using the same kinematics, in less viscous oil (house fly scale, Re =1400) and finally water (humming bird scale, Re =14,000), which is even less viscous. We found that the lift–drag coefficient polars did not change much, the main effect we found is

14 The Scalable Design of Flapping MAVs

187

Fig. 14.2 (a) Cartoon of the leading edge vortex that is stably attached to the wing of a fruit fly. (b) Flow visualization of this stable LEV on a fruit fly wing at Re=110. Visible are air bubbles

that swirl into the LEV after they were released from the leading edge of a fruit fly wing immersed in a tank with mineral oil [11]

that the lift coefficients generated at fruit fly scale are a bit smaller and the minimum drag coefficient a bit higher, due to viscous damping Fig. 14.3. This shows that the aerodynamic forces generated by flapping fly wings can be estimated well across this whole Reynolds number range using the coefficients measured at either Re = 110, 1400 or 14,000, multiplied by the average dynamic pressure and wing surface area that correspond with the scale of interest (average dynamic pressure is calculated using a blade element method [2]).

14.3 Design Approach: Scale a Flapping MAV That Works Down to Smaller Sizes

Fig. 14.3 The forces generated by a flapping fly wing depend weakly on Reynolds number. Stroke-averaged lift–drag coefficient polar of a model flapping fruit fly wing (light grey circles) at Re = 110, 1400 and 14,000. The angle of attack (amplitude) ranges from 0◦ to 90◦ in steps of 4.5◦ . The lift–drag coefficient polars only weakly depend on Re, especially for angles of attack close to 45◦ , which correspond approximately with maximum lift. The polars at Re = 1400 and 14,000 are almost identical [11]

Model airplane enthusiasts have demonstrated that small, lightweight airplanes can be built with wingspans that range from 70 to 10 cm, of which the lightest weigh around 1 g (e.g. www.indoorduration. com). The biggest challenge might be the availability of high-performance micro-components that build up the flight system: micro-radio controllers (RC), actuators, motors, batteries, etc. (see the extensive list of suppliers in Appendix 1). Keeping in mind that both aerodynamics and structure are not limiting to scaling ornithopters, it is advantageous to develop a relatively large, well-flying ornithopter inspired by existing ornithopter designs and insect flight. This scale should be chosen such that the required electronic and mechanical components are both commercially available and affordable. An artist impression of this approach is shown in Fig. 14.4. Based on this approach we first designed and built DelFly, a 35 cm span (22 g) flapping ‘MAV’. It can fly both fast and perform slow hovering flight for maximal 15 min, while streaming video (2005). Next we scaled the DelFly design down to 28 cm span (16 g), DelFly II (2006), which can take off and land vertically. DelFly II can hover for 8 min and fly fast for 15 min, while streaming video. Recently Nathan Chronister scaled the DelFly to 15 cm span (3.3 g); this model can both hover and fly loopings. The group of Yoshiyuki Kawamura has made the next step in an earlier phase (2006–2007) and scaled down the DelFly design to 10 cm span (2.3 g) which flies for a couple of minutes [8]. This 10 cm span ornithopter is the first successful insect-sized flapping MAV.

188

D. Lentink et al.

trolled and too briefly, they have two big advantages. First, they are inherently stable designs. Second, the in-flight torque of the rubber band can be determined using a torque meter constructed of a thin, calibrated, piano-steel wire, and the windings in the rubber band can be counted easily; the result is illustrated in

Fig. 14.4 Artist impression of micro-aerial vehicle design inspired by insect flight

14.4 DelFly: A Flapping ‘MAV’ That Works The aim of the DelFly project was to design a stable, radio-controlled, flapping MAV that could fly for 15 min and function as a sensor platform. The DelFly mission was to detect a person walking with a red suitcase. For this DelFly is equipped with an onboard colour camera, of which the images were streamed live to a base station with situation awareness software. The team consisted of 11 bachelor students of Delft University of Technology supervised by scientists and engineers from Wageningen University, Delft University of Technology and Ruijsink Dynamic Engineering. To kick start the project the students started out with flight testing three existing rubber-band-powered ornithopter designs: a monoplane, biplane and tandem design, Fig. 14.5. The fight test procedures are described in Appendix 2. Although these ornithopters cannot be considered MAVs, because they fly uncon-

Fig. 14.5 Three different rubber-powered ornithopter configurations flight tested for the design of DelFly. (a) Falcon, a monoplane ornithopter available at www.ornithopter.org. A single pair of wings is powered by a rubber band. (b) Luna, a biplane ornithopter available at www.ornithopter.org. Its two wings form a cross; the two lower legs of the cross are actuated. As a result the upper and lower left wings flap towards and away from each other (same for the right wing). (c) Tandem wing ornithopter, custom built inspired by an existing Swedish design. The two tandem wings flap in anti-phase, the front wing is actuated and the hind wing flaps 180◦ out of phase in reaction to this. The front wing is connected to one side of the rubber band and the hind wing is connected to the fuselage to which we attached the other side of the rubber band

14 The Scalable Design of Flapping MAVs

Fig. 14.6 Rubber band torque vs. number of windings. After putting in a certain number of winds into a rubber band (e.g. 1000) it starts to unwind at the right side of this graph at peak torque. We first let the rubber unwind such that the torque flattens off and then start performing a test flight. Knowing the start and end torque and the number of windings in the rubber band (counted during (un)winding) the in-flight torque can be estimated well, especially with the movies we made of every flight, for which we used the recorded flapping sound of the ornithopter to determine its flap frequency. High-quality rubber band is the key to good flight performance and useful measurements: www.indoorduration.com

Fig. 14.6. During the flight test of the three ornithopter configurations we measured the mass, torque (at the start and end of flight), flap frequency (audio data) and average flight velocity (video data). Finally we estimated the rocking amplitude at the front of the ornithopter (video data), where we planned to fit the camera. Using torque and flap frequency we computed the average power consumption. The measured and calculated flight variables can be found in Table 14.1. Based on the measurements we found that it was most difficult to trim the tandem design such that it flew well, and we therefore eliminated this configuration. The two competing configurations were the monoplane and biplane configurations. The

189

monoplane appeared to be the most efficient flyer, but flew at relatively high speed and rocked significantly. Therefore, we chose the slower biplane configuration which rocked least, which is critical for a camera platform. Having reliable performance estimates is critical for sizing the electronic components, actuators, motor, gearbox and battery. For the sizing we used simple scaling laws to obtain realistic torques, frequencies, power consumption and flight speed estimates for an arbitrary size, using the measured data in Table 14.1. In our scaling we assume that the flight path is the same for both the original (indicated with ‘1’) and the newly scaled ornithopter (indicated with ‘2’). To determine the new horizontal velocity U∞ we use the fact that lift equals weight during horizontal flight as follows: & 2 mg ∝ CL 1 2ρU∞ S → U∞,2 = U∞,1



m2 S1 m1 S2

0.5 ,

(14.1) where m is the mass, CL the lift coefficient which we assume to be independent of Reynolds number (Fig. 14.3), ρ the air density (constant), U∞ the forward flight velocity of the ornithopter and S the wing surface area. We use the proportional instead of equal sign, because the proportionality remains valid for other flight conditions such as climbing and turning. The required power P is proportional to weight times speed (because drag scales with weight) as follows: U∞,2 m2 → P2 U∞,1 m1  1.5  0.5 (14.2) m2 S1 = P1 , m1 S2

P ∝ mgU∞ → P2 = P1

where g is the gravity constant. To determine the flap frequency we need to know how forward speed U∞ and the flap frequency f are related; for this we assume that the advance ratio J [2, 12] of the ornithopter remains constant:

Table 14.1 Flight test results of the three ornithopter configurations Variable Unit mg S m2 bm T Nm f Hz

V m/s

PW

Rocking mm

Monoplane 6.7 0.043 0.41 0.0075 3.7 1.7 0.18 80 Biplane 8.0 0.074 0.35 0.0064 6.7 1.3 0.28 ±0 Tandem 10.9 0.066 0.35 0.013 7.9 1.5 0.66 – The tandem configuration was difficult to trim and did not fly well as a result. We averaged over flight tests during which we judged the ornithopter to be trimmed well; fly stable. The variables are as follows: m, mass; S, total wing area; b, wing span; T, torque needed to drive the wings; f, flap frequency; V, flight speed; P, required power; rocking, rocking amplitude at the front of the ornithopter

190

D. Lentink et al.

U∞,2 0,1 R1 U∞ = const → f2 = f1 → f2 4f 0 R U∞,1 0,2 R2     m2 S1 0.5 0,1 R1 , = f1 m1 S2 0,2 R2 (14.3) where 0 is the stroke amplitude in radians (half the total amplitude defined in [2]) and R the single wingspan (radius). If we assume that the stroke amplitude is constant, which is true for isometric scaling, this relation becomes straightforward. What if measurements are performed for hovering flight when U∞ = 0? To scale both hovering and forward flight continuously we suggest to use the wing tip speed V in Eqs. (14.1) and (14.2) instead of U∞ , which is within good approximation [10, 12]: J=

V≈

' 2 + (4f  R)2 . U∞ 0

(14.4)

Based on the calculated power and the specs of the battery pack we can now calculate the average motor current as follows: I2 =

P2 . VLiPo

(14.5)

where I2 is the motor current of the newly scaled ornithopter and VLiPo the voltage of the lithiumpolymer battery pack. The total flight time in seconds can now be estimated as follows: t2 = 3.6

CLiPo VLiPo , P2

(14.6)

where CLiPo is the capacity of the lithium-polymer battery pack (mA h). Based on the required power, the voltage of the battery pack and the flap frequency, a motor can now be selected (see www.didel.com for pager motor selection charts). Based on the rpm of the selected motor RPMmotor,2 , the required gearbox ratio red2 is as follows: red2 =

RPMmotor,2 , 60f2

(14.7)

Using a spreadsheet and Eqs. (14.1)–(14.17) the ‘components off the shelf’ (COTS) are chosen which means components are bought as light and small as currently available in retail. We illustrated the main components chosen to build DelFly in Fig. 14.7 (these are representative, not actual, photos). In 2005 these

were the most lightweight components available (see the extensive list of suppliers in Appendix 1). The weight, power consumption and other performance indices of these components determined the smallest possible dimensions of DelFly at which it could fly for 15 min and stream live video. The main component of DelFly is the battery. In order to choose a suitable battery two criteria are important: the capacity and the maximum discharge rate. The first parameter, the required capacity, is determined by the required flight duration and the power consumption of the electric systems. The second, the maximum required discharge rate, is determined by the maximum power required by the total electrical system. The latter is a problem with most lightweight batteries. Therefore, we selected a battery with a least power to weight ratio and a sufficient maximum discharge rate. The lightest available battery fulfilling these requirements is a 140 mA h lithium-polymer battery as seen in Fig. 14.8. It could discharge up to five times its capacity, 700 mA, which is enough. The biggest energy consumer is the motor that powers the flapping wings of DelFly. We chose a brushed pager motor, because of its availability. Brushless motors are more efficient, but the available motors cannot handle the periodic loading due to the flapping wings. The motor drives a gearbox (see Fig. 14.7) to reduce the RPM of the motor to match the required flapping frequency of the wings. A dedicated crankshaft, conceptually the same as that of the Falcon biplane, connects the gearbox to the two lower legs of the X-wing and drives both lower wings in phase. The left lower wing is directly connected to the right upper wing and vice versa. Therefore both sides of the X-wing flap synchronously towards each other and away from each other (buying the actual kits helps to get a good three-dimensional picture of this system). We re-designed the flapping mechanism itself using the freely available Java software at www.ornithopter.org/software.shtml. For controlling DelFly we used standard model aircraft RC radio equipment. The remote control sends control signals to DelFly’s onboard receiver (see Fig. 14.7). This in turn translates the control signal to a power signal to the coil actuators (see Fig. 14.7), which we connected to the control surfaces. DelFly also has a camera onboard for two reasons. First, a camera is a useful sensor to obtain images of its surrounding. Second, the camera in combina-

14 The Scalable Design of Flapping MAVs

Fig. 14.7 Illustration of the main components of DelFly (and DelFly II): (a) brushless electric motor (www.bsdmicrorc.com; modified for use on DelFly II); (b) colour camera (www.misumi.com.tw); (c) plastic gear box (www.didel.com);

Fig. 14.8 Control loop for vision-based awareness of DelFly

191

(d) micro-actuators (www.bsdmicrorc.com/); (e) receiver (www.plantraco.com); and (f) lithium-polymer battery (www. atomicworkshop.co.uk)

192

D. Lentink et al.

Fig. 14.9 Three-dimensional CAD drawing of the DelFly design (a) and DelFly flying in the European Alps (b)

tion with dedicated vision software can be part of the control loop. DelFly has a camera onboard that sends video signals via the transmitter to the receiver on the ground. This signal enters a central processing unit, usually a personal computer or laptop, via its video card. Dedicated software is installed on the central processing unit, which analyses the video signal to detect objects like a red suitcase or compute the optical flow. Based on the image analysis the base station then sends updated control signals to DelFly via the RC radio to the receiver on board of DelFly. This continuous control loop for vision-based awareness is illustrated in Fig. 14.8. All the components combined resulted in the design shown in Fig. 14.9a. The combined mass of all (electronic) components is approximately 12.5 g. To carry this load and bear the corresponding aerodynamic and inertial loads we designed a lightweight structure of approximately 4.5 g. The structure consists primarily of carbon fibre rods. The wings leading edge spars are made of a balsa–carbon (unidirectional) sandwich that is much less stiff in flight direction than in flap direction. This stiffness asymmetry is essential to make the wing deform well aero-elastically. The wing is covered with transparent Mylar foil of 7 g/m2 . We used cyano-acrylate for gluing carbon–carbon, epoxy for gluing carbon–balsa and transparent Pattex hobby glue, diluted 1:1 with acetone, for gluing carbon– Mylar. DelFly turned out to be an easy to control and very stable flapping MAV, Fig. 14.9b. Its main drawback is that it can get into a spiral dive when turning too tightly and too fast at too low angle of attack, because the actuators are slightly underpowered for this flight condition

(finding strong and light enough actuators remains a challenge for MAV design).

14.5 DelFly II: Improved Design After designing and building DelFly within a student project we professionalized the DelFly design and better quantified its aerodynamic performance. For this we were supported by TNO (The Netherlands), which resulted in the DelFly II design, shown in Figs. 14.10 and 14.11. DelFly II weighs about 14 g without payload and 17 g with payload, a camera and video transmitter. The wingspan is reduced from 35 cm to 28 cm and its length is reduced from 40 cm to 28 cm, such that it fits in a 30 cm diameter sphere. The most important difference between DelFly and DelFly II is its symmetric driving mechanism and the customrefitted brushless motor. This brushless electric motor has enough power to enable DelFly II to vertical takeoff and landing, shown in Fig. 14.12. The motors’ efficiency of roughly 60% enables it to fly longer than normal, using a pager motor instead will still allow DelFly II to take off and land vertical and hover at the cost of some flight duration. The brushless motor we used had to be refitted with different windings, magnet configuration and controller software such that it could cope with the highly varying drive torque of the flapping wings. Note that brushed pager motors do not need special modifications to drive a flapping wing and are therefore a time efficient and inexpensive solution (see the extensive list of suppliers in Appendix 1).

14 The Scalable Design of Flapping MAVs

193

Fig. 14.10 Detailed photos of DelFly II. (a) Main components of DelFly II. The front part of the carbon fuselage is a sandwich of carbon cloth (65 g/m2 ) with a Rohacell core (lowest density available). The transparent Mylar foil weighs 7 g/m2 . (b) Rudder connection mechanism for the controls. (c) Brushless electric motor (e-motor) and the symmetric driving mechanism of

both lower legs of the X-wing. The rotary motion of the gears is converted into a translating motion through the carbon fibre rods (cr-rods) that are fitted with a small bearing consisting of a flattened brass rod with a hole drilled in it. (d) Wing root with cellotape reinforced Mylar film and rapid prototype wing hinge (www.quickparts.com)

14.6 DelFly II: Aerodynamic Analyses

getting insight into the aerodynamics of aero-elastic insect wings. Based upon the coefficient that we measured for DelFly under hovering conditions, which is around 2, we believe that DelFly employs at least two of the high-lift mechanisms that are found in insect flight. First we think that DelFly creates a stable leading edge vortex (Fig. 14.2), like the AeroVironment Microbat [14]. Second we think DelFly benefits from the clap and fling mechanism of aero-elastic wings, which is utilized by small insects and butterflies [17]. The wings of DelFly clap and fling when the upper and lower wings come together as the X-wings close. The peeling motion of the aero-elastic DelFly wings resembles the wings of butterflies during take-off, Fig. 14.13. First the wings ‘clap’ together at the ‘start’ of the flap cycle, at 0%, after which they ‘peel’ apart at 12.5% through

Insects have limited control over the wing shape, because their muscles stop at the base of their wings. The aero-elastic wings of insects are therefore thought to be passively stabilized. The aero-elastic wings of ornithopters like DelFly have passively stabile aeroelastic wings too, which deform strongly under loading. But what forces mediate DelFly’s wing deformation, aerodynamic loading or wing inertia? And how much power is lost with accelerating and decelerating a flapping wing continuously? What flap angles result in the best hover performance, and how high are the lift coefficients generated by a strongly deforming aeroelastic wing? The answers to these questions are likely to be as relevant for optimizing DelFly as they are for

194

D. Lentink et al.

Fig. 14.11 Dimensions and building material of DelFly II. All the above are drawn to scale and angles are realistic: top view (a), wing dimensions (b) and build in dihedral (c). Note that cr stands for carbon fibre rod, whenever available we used hollow carbon rods. All carbon fibre components are available at www.dpp-pultrusion.com. (d) DelFly II in hovering flight, note that this is a slightly different model than shown in Fig. 14.10.

Photo credit: Jaap Oldenkamp. Not shown is the X-shaped landing gear of DelFly II, build out of the leading edge carbon rods of the horizontal and vertical rudder. The landing X-rods are pulled together with two thin wires (e.g. nylon or Kevlar) and as a result they bend. The base of the landing gear has smaller dimensions than the wing span (dimensions are not very critical)

37.5% of the flap cycle. The clap and fling is essentially a combination of two independent aerodynamic mechanisms that should be treated separately. First during the clap the leading edges of the wing touch each other before the trailing edges do, progressively closing the gap between them. Second during the fling the wings continue to pronate by leaving the trailing edge stationary as the leading edges fling apart. A lowpressure region is supposedly generated between the wings and the surrounding fluid rushes in to occupy this region. This initializes the build up of circulation [17]. Experiments of Kawamura et al. [8] with a 10 cm

span DelFly biplane model and a similar monoplane model showed that the clap and fling indeed increases thrust up to roughly 50%. The thrust–power ratio, a measure of efficiency, is also roughly 50% higher for the biplane configuration. Another explanation as to why insects might clap and peel their wings is that they simply try to maximize their stroke amplitude to maximize wing lift (for the same flap frequency). The lift force is proportional to velocity squared and therefore amplitude squared; maximization of the stroke amplitude will therefore significantly enhance the total flight forces [17].

14 The Scalable Design of Flapping MAVs

195

a

b

Fig. 14.12 Compilation of video images that illustrate the vertical take-off (a) and vertical landing (b) capabilities of DelFly II. The clearly visible light square is the reflective battery pack (LiPo)

14.6.1 DelFly Models Used for Aerodynamic Measurements We studied the aerodynamic performance and aeroelastic deformation of DelFly II wings to maximize its lift and minimize its power consumption in hovering flight, the most power-consuming flight mode. These studies were performed at Wageningen University in collaboration with Delft University of Technology [1]. For these studies two DelFly II models were used: DelFly IIa, powered by a strong brushed motor (simplified aluminium construction). This model was used for high-speed camera imaging DelFly IIb, powered by a 3.5 V brushless motor of which the frequency is controlled by varying the current (realistic carbon fibre and plastic construction). This model was used for all except one performance

Fig. 14.13 High-speed video image sequence of the clap and fling of DelFly II wings in air at 14 Hz and 30◦ flap angle. The images are snapshots starting at the beginning of upstroke up to the start of the downstroke: 0%, 12.5%, 25%, 37.5% and 50% of the flap cycle

measurement. We used a brushed motor for the performance analysis shown in Fig. 14.15, because it allowed us to test for higher flap frequencies. Figure 14.14a shows the DelFly IIb model mounted on a six-component force transducer. This transducer is capable of accurately measuring forces and torques (i.e. moments) in three directions with a resolution of

196

D. Lentink et al.

Fig. 14.14 (a) DelFly IIb mounted on the six-component transducer, φ indicates the flap angle. The oblique line indicates the wing position at the start of a flap cycle (upstroke) when the leading edge of the wing is at maximum deflection, i.e. 50% of the

flap cycle, where the downstroke starts. (b) Wing hinge with different drive-rod positions that result in the different flap angles for which we measured the hover performance of DelFly

Table 14.2 Flap angle as a function of flap angle position. The final design of DelFly II has an even larger flap angle, for which special wing hinges were designed Flap angle position φ 3 4 5 6 7 8 9

17.5◦ 19.5◦ 21.5◦ 24◦ 27◦ 30◦ 36◦

approximately 0.5 g. The flap angle is indicated by φ. Different flap angles are obtained by connecting the drive rod to the different connection points at the wing’s hinge, shown in Fig. 14.14b, the actual values are given in Table 14.2.

14.6.2 Lift as a Function of Flap Frequency at a Constant Flap Angle of 36◦

Fig. 14.15 Lift vs. wing beat frequency at a constant flap angle of 36◦ . The line indicates the linear trend, whereas a quadratic trend is expected (lift is proportional to frequency squared). We think the linear trend with increasing frequency results from the increasingly higher lift forces that deform the wing more and more, and therefore reduce the angle of attack of the wing, which lowers lift force (because lift is proportional to the angle of attack)

We found that lift increases linearly with wing beat frequency between 14 and 20 Hz at a constant flap angle of 36◦ , Fig. 14.15. This flap angle was chosen because this flap angle most closely resembles the flap angle of DelFly in flight. The frequency range was determined by the maximum power output of the brushed

motor fitted on DelFly II B. Figure 14.15 shows that this model needs to flap at a frequency of 20 Hz to lift the payload (total mass 17 g) during hovering and a frequency of 17.2 without payload (14 g). The final DelFly II design flaps at lower frequencies, because it has a larger flap angle than we could test with the

14 The Scalable Design of Flapping MAVs

here-used hinge. Note that the net wing speed is proportional to flap angle times frequency and that this product is roughly the same for our DelFly IIb and the final design, because both operate at similar lift coefficients.

14.6.3 Lift and Power as a Function of Flap Angle at a Flap Frequency of 14 Hz Both lift and power increase with flap angle, but the more important lift over power ratio reaches a plateau, Fig. 14.16. The lift over power ratio is a measure of how effectively DelFly generates lift. The required power measurements include the aerodynamic power as well as the power needed to drive the motor, the drive train and overcome the inertia of the complete mechanism (accelerate and decelerate it). Both the lift and the required power increase with flap angle, because wing speed increases with flap angle (at constant frequency). Theoretically lift is proportional to the flap angle squared, and aerodynamic power to the flap angle cubed. Because the experiment is carried out at constant flap frequency the flap velocity, and therefore wing lift, increases significantly with flap angle. The increasing lift deforms the aero-elastic wing increasingly more. These significant deformations might explain why the lift over power ratio is not

Fig. 14.16 Lift, power and efficiency of DelFly: (a) lift vs. flap angle at a constant wing beat frequency of 14 Hz; (b) power vs. flap angle at a constant wing beat frequency of 14 Hz; and (c) the ratio of lift to power vs. flap angle at a constant wing beat

197

proportional to the inverse of flap angle, predicted by theory. The measurements suggest that the most efficient flap angle is higher than 30◦ for DelFly II flapping at 14 Hz. The Reynolds number in these experiments varied from 3700 to 7600. Based on Fig. 14.3 we do not expect that this difference in Reynolds numbers affects the wings’ lift coefficients much. The dimensionless lift and power coefficients of DelFly II flapping at 14 Hz are shown in Fig. 14.17. The lift coefficients have high values, CL = 1.8–2.5, compared to the lift coefficients of translating wings at similar Reynolds numbers of which the maximum lift coefficient is approximately 1. The power coefficients are calculated by dividing power by the product of dynamic pressure, wing speed and wing surface area using a blade element method [2]. It is striking that these coefficients are an order of magnitude greater than the force coefficients. Based on scale arguments we expected that both coefficients are of order 1, O(1). An explanation for this inconsistency might be in the fact that we used total power instead of aerodynamic power. In order to determine aerodynamic power we had to separate it from the power needed to drive the motor, the drive train and overcome inertial forces. We determined the aerodynamic power by measuring the required power under vacuum conditions and subtracting that power from the power needed to flap at the same frequency in air. For these experiments we designed and built a custom vacuum chamber with a minimum pressure of 10 Pa.

frequency of 14 Hz. The lift–power ratio is a measure of ‘efficiency’. The dots give the results of the individual measurements per sequence and the crosses denote the mean value of the four runs

198

D. Lentink et al.

Fig. 14.17 Lift (a) and power (b) coefficients vs. flap angle at a constant wing beat frequency of 14 Hz. Power coefficients vary much more with flap angle than lift coefficients

14.6.4 Power Requirement and Wing Deformation in Air Versus Vacuum The power losses of a flapping wing range from roughly 80% at a flap angle of 24◦ to 50% for a flap angle of 36◦ . To isolate the aerodynamic power, the power measured in near vacuum was subtracted from the power measured in air: Paero = Pair – Pvac . We plotted the aerodynamic power as a percentage of total power as a function of flap frequency in Fig. 14.18. The results suggest that the power losses of DelFly are strongly dependent on the flap angle, but not so much on frequency. The larger flap angle results in the highest percentage of aerodynamic power, which partly explains why a large flap angle results in high flap performance, Fig. 14.16c. If we correct the power coeffi-

cient for a power loss of 80% at a flap angle of 24◦ and 50% at 36◦ , we obtain power coefficients of approximately 5 and 7.5 which are much closer to the values found for flapping fruit fly wings of 2–4 at similar angles of attack in Fig. 14.3. The remaining differences can still be explained by mechanical losses, because we were unable to correct for the effect of variable motor efficiency. The much lower torque in vacuum can drastically alter the efficiency of the brushless motor, we hope to better quantify this in a future study. Finally we wanted to know how inertial versus aerodynamic forces deform the aero-elastic wing of DelFly. Using the vacuum chamber we made highspeed video images of the flapping foil in vacuum and air, Fig. 14.19. The wings deformation is signif-

Fig. 14.18 Percentage of aerodynamic power compared to total power increases with flap angle and depends weakly on wing beat frequency: (a) flap angle of 24◦ and (b) flap angle of 36◦

14 The Scalable Design of Flapping MAVs

199

Fig. 14.19 Comparison of wing deformation under aerodynamic plus inertial wing loading in air (left) and inertial wing loading in vacuum (right) at 14 Hz and 30◦ flap angle. The images are snapshots at the start of the upstroke (0%), 12.5%

and end (50%) of the flap cycle, where the downstroke starts. In vacuum the upper and lower foils stick together (12.5%), most likely due to electrostatic stickiness

icantly higher in air than in vacuum, hence we conclude that the aero-elastic wing deformation largely determines DelFly II’s wing shape. It confirms that there is a direct coupling between wing load and the wings aerodynamic angle of attack, because the wing deforms towards lower angles of attack under loading (compare air and vacuum at 0% and 50% flap cycle). We think that the apparent wing peeling at 12.5% of the flap cycle is due to the electrostatic stickiness of

the upper and lower foil. This could be tested in future studies by using non-electrostatic foils. Based on our aerodynamic analysis we conclude that inertial and friction losses in DelFly-like designs are high and need attention. One solution could be to use elastic energy storage in a spring, tuned to the average flapping frequency of the wing. Ideally the spring has a variable stiffness such that its natural frequency continuously matches the flapping frequency.

200

14.7 Bio-Inspired Design of Insect-Sized Flapping Wings Our aim is to work towards fly-sized micro-air vehicles that manoeuvre well. DelFly is still big, slow and sluggish. DelFly potentially could be controlled more directly using the flapping wings instead of the airplane tail for control. Because we focus on flapping MAVs that can actually fly, the size of every new design is limited by the smallest and lightest available components, which limits miniaturization. A more direct approach is to start at a smaller scale and figure out how such small fly-sized flapping mechanisms, wings and actuators can be designed and constructed. This approach, pioneered in the lab of Ronald Fearing, has significantly increased our understanding of designing and building micro-flapping structures. Recently, the first wire-supported micro-mechanical insect successfully lifted off from the ground in the lab of Robert Wood [21]. Building upon the existing work in the field we wondered how we could improve the flat wing design of micro-mechanical insects and DelFly wings at the scale of insects. The current DelFly wing is made of a D-shaped carbon fibre leading edge spar of constant thickness. A flexible Mylar foil forms the wing’s surface and is stiffened by two carbon rods. The foil and carbon rods form a flat airfoil that is cambered by aerodynamic forces and, to a lesser extent, inertial forces during flapping. Aero-elastic tailoring of DelFly’s wing has been done by trial and error using a strobe. At some point we even applied variable wing tension (left versus right) for thrust vectoring, but all these measures are primitive compared to the aero-elastic tailored wings of insects. Insect-sized flapping MAVs could benefit from stiffer wings for the same weight, because their shape can be controlled and tailored more directly. Wing venation, like found for insect wings, could potentially minimize or even stop wing tear, which is a problem with DelFly wings. Images of DelFly in flight often show wrinkling in trailing edges that affect its aerodynamic performance, which could be prevented by making the trailing edge stiffer. This keeps the wing foil in shape during flapping and prevents the wing from tearing. We used dragonfly wings as an inspiration to develop design principles for such stiffer micro-wings with venation-like tear-stoppers.

D. Lentink et al.

If we take a close look at insect wings we find that they are not flat but corrugated and wing thickness varies both span- and chord-wise. The wing structure of an insect has therefore a much richer architecture than the non-corrugated and constant thickness DelFly II wings. Compared to flat wings, corrugation improves the strength and stiffness of insect wings, because it increases the moments of area of the wing sections [16, 22, 23]. Corrugation also mediates the aero-elastic properties and vibration modes (natural frequencies) of the wing and, finally, corrugated wings are lighter for the same stiffness [16], while performing well aerodynamically [15, 16, 13, 9, 19]. We found inspiration for improving the wings of DelFly in the front wing of a dragonfly, Sympetrum vulgatum of the order Odonata [7]. Odonata is a primitive order of insects; their four wings possess relatively complex venation patterns. The fossil record shows that these venation patterns exist even in large dragonfly wings of up to 70 cm span, whereas current dragonflies have wing spans smaller than 10 cm, this suggests that dragonfly wings and their aerodynamic function are scalable. Although they are primitive insects, dragonflies have evolved into well-flying insects. Fast manoeuvres, silent hovering and even inflight hunting and mating are commonly shown by these aerobatic artists. S. vulgatum flaps its wings at a frequency of approximately 35 Hz. We first digitized a front wing of S. vulgatum, Fig. 14.20a, using a micro-CT scanner, Fig. 14.20b. The nearly 4000 cross-sectional images per wing create an accurate three-dimensional digital reconstruction of the wing. This reconstruction allowed us to quantify both the vein and shell thickness of the dragonfly wing, Fig. 14.20c1,c2. In our next step we simplified the scans by converting the wing geometry into beam and shell elements with the same geometric properties as the scan. The result is an accurate and efficient finite element model of both wings. For this dedicated processing software was written using Matlab 7.0 (MathWorks). All the elements, load data and boundaries were automatically written to an input file for Abaqus, a finite element solver for structural analysis. Using a blade element model [2], we calculated the aerodynamic and inertial loads on the wing during hovering flight, Fig. 14.20d1,d2. Using both the simplified finite element model and the calculated wing loading we determined the wings deformation, internal stresses

14 The Scalable Design of Flapping MAVs

201

Fig. 14.20 Design wheel of an insect-sized flapping wing based on the forewing of a dragonfly (Sympetrum vulgatum): (a) original dragonfly forewing; (b) micro-CT scan of the dragonfly wing; (c1) thickness distribution of the wing veins; (c2) thickness distribution of the wing membranes; (d1) maximum aerodynamic load on the wing during hovering, computed with wing

and flight data using a blade element method; (d2) maximum inertia loads on the wing during hovering; (e) maximum wing deformation during hovering; (f1) maximum internal loads during hovering; (f2) synthesized and simplified load paths in S. vulgatum forewing; and (g) the bio-inspired design of an insectsized flapping wing

and vibration modes, Fig. 14.20e. We calculated the average load paths over a stroke cycle to estimate which veins and shells contributed most to the stiffness and strength of the wing, Fig. 14.20f1. Based on the average load paths we used engineering judgement to eliminate veins that carried only little load and connect the veins such that they formed continuous load paths, Fig. 14.20f2. In our final design step we used the simplified load paths to come up with a conceptual corrugated wing for a flapping MAV, Fig. 14.20f3 (details can be found in [7]). The main features of the conceptual corrugated wing design are its corrugation at the leading

edge. Both thickness and corrugation height decrease towards the wing tip where less stiffness is needed. Between the leading edge beams we suggest to apply ribs, to prevent the beams from buckling. Controlled buckling might be a very interesting failure mode when ultimate forces are applied on the wing. The wing design consists of several thin ribs that connect the leading and trailing edge and form ‘rib-enclosed compartments’ that can stop wing tearing. By carefully designing the corrugation profile, the location of supporting ribs and the thickness distribution of the ‘veins’, the wing can be customary tailored to perform well aero-elastically.

202

14.8 Production of the Bio-Inspired Wings for an Insect-Sized MAV Although one can cut very nice two-dimensional wings out of carbon sheet using a laser cutter, this method is not well suited for building complex threedimensional structures. We envision that a miniature three-dimensional weaving machine in combination with a mould could solve this problem [7]. Such a machine could weave carbon fibres into a three-dimensional vein network that forms the wing structure. We think an even more promising solution could be to weave thin tungsten wires and damp boron on it, a super-stiff and strong metalloid. Boron fibres are already used to stiffen the lightest planes for their size: 65 cm span at a weight of 1 g (see www.indoorduration.com). These fibres are made by vacuum coating boron on tungsten wires, hence the process is already an industry standard for making simple two-dimensional structures. Within our project we focussed on a low-cost demonstration project using carbon fibres. We chose cyano-acrylate as a matrix for the carbon fibres, because its low viscosity results in good wetting properties of the carbon fibres. To cover the wings we chose one-sided (OS) film of 0.5 μm thickness glued with diluted Pattex (similar to DelFly wings). The wings are produced by stretching dry carbon fibres on a threedimensional mould, Fig. 14.21. Crossing fibres are checked for sufficient contact surface between each other to ensure correct bonding. If all the dry fibres are stretched on the mould an infusion process starts by applying cyano-acrylate drops attached to the round head of a pin to the fibres. Drops of cyano-acrylate are also placed on the intersections of fibres to connect them firmly. We immediately noticed the advantage of the low-viscous cyano-acrylate; the fibres absorb the glue and spread it through the fibre by capillary force fast and easily. Pushing a pin against the fibre gives insight as to which part is sufficiently infused and which part is still dry. The wing is trimmed to its final shape after infusion and consolidation of all the fibres. Finally OS film was mounted on the carbon fibre structure using thinned glue. This process is very labour intensive, but it can be automated using a miniature three-dimensional weaving and glue machine, but even better would be to damp boron on three dimensionally woven tungsten wires.

D. Lentink et al. Fig. 14.21 Developing process of insect-sized flapping wings: (a) design of an insect-sized flapping wing inspired by a dragonfly forewing; (b1) cross section of re-designed wing; (b2) cross section of original DelFly wing; (c) cross section of Sympetrum vulgatum forewing at approximately 30% of wingspan. (d–g) Building method: (d) stretch carbon fibres on a three-dimensional mould; (e) consolidate structure by tipping drops of glue on the fibres with a pin; (f) trim consolidated wing to its final shape; (g) end result: insect-inspired wing made of carbon fibre. (h) Mock-up of future DelFly micro-equipped with wings with an advanced three-dimensional structure, which makes them stiff for their weight. Shown: tandem configuration of which the forewing and hindwing can flap 180◦ out of phase, like a dragon fly during slow flight

14 The Scalable Design of Flapping MAVs

14.9 Less Is More: Spinning Is More Efficient than Flapping an Insect Wing Having demonstrated the scalable design of flapping MAVs we conclude that the biggest challenge for fruit fly-sized air vehicles is the development of highperformance micro-components. Another challenge is

Fig. 14.22 The forces generated by a spinning fly wing depend weakly on Reynolds number. Stroke-averaged lift–drag coefficient polar of a simple spinning fruit fly wing (triangles) at Re = 110, 1400 and 14,000. The angle of attack (amplitude) ranges from 0◦ to 90◦ in steps of 4.5◦ . The lift–drag coefficient polars only weakly depend on Re, especially for angles of attack up to 45◦ , which correspond approximately with maximum lift. The polars at Re = 1400 and 14,000 are almost identical [11]

Fig. 14.23 The aerodynamic performance of a spinning fly wing is higher than that of a flapping fly wing. Stroke-averaged power factor vs. glide-number polar of a flapping versus spinning fruit fly wing at Re = 110, 1400 and 14,000. Aerodynamic power is proportional to the inverse of the power factor, the highest power factor represents maximum performance. These per-

203

energy efficiency, because existing insect size flapping MAVs are inefficient. This inefficiency is demonstrated by the low flight duration that ranges from 1 to 15 min at relatively low wing loading. The smallest mechanical insects can still not take off without power cables attached to batteries on earth. Based on aerodynamic measurements on both insect wings and DelFly II we found that the aerodynamic performance of flapping insect wings is low. We further found, similar to others [20], that simple spinning insect wings also generate a stable leading edge vortex and corresponding elevated lift and drag forces for Re = 110– 14,000, Fig. 14.22. In fact we deliberately depicted a stable LEV on a spinning (not flapping) fruit fly wing in Fig. 14.2, because it is surprisingly similar to the one generated when the wing flaps. This observation inspired us to explicitly compare the hover efficacy of both flapping and spinning fruit fly wings at Reynolds numbers ranging from fruit flies (Re = 110) to small birds (Re = 14,000); published in Lentink and Dickinson [11], Fig. 14.23. Through this unique constant Reynolds number and equal wing shape comparison within one experiment we found that spinning insect wings outperform flapping ones up to a factor 2. This suggests that helicopter-like MAVs fitted with insectlike wings could potentially be, up to a factor 4, more energy efficient than flapping insect-like MAVs. Such helicopter-like MAVs can generate similarly elevated lift forces compared to flapping wings using a stable leading edge vortex. The estimated factor 4 efficiency improvement results from the combined effect

formance polars are based on the flapper data in Fig. 14.2 and the spinner data in Fig. 14.22. Note that fruit flies, at Re = 110, flap at approximately the same maximum performance level obtained with the more simple flap kinematics; they flap well (based on [11])

204

D. Lentink et al.

of a factor 2 difference in aerodynamic power and a factor 2 difference due to inertial power loss. Our study suggests, therefore, that combining both the spinning motion of helicopters and the wing shape of insects might give best of both worlds: the high lift from a stably attached leading edge vortex and the high efficiency of a spinning wing. Our study predicts therefore that a fruit fly-sized and winged air vehicle will be most efficient when fitted with spinning wings.

Acknowledgments We would like to thank the following persons and everyone else who helped us out with the research and designs presented in this chapter. Insect flight research: Michael Dickinson, William Dickson, Andrew Straw, Douglas Altshuler, Rosalyn Sayaman and Steven Fry at Caltech. DelFly design: Michel van Tooren, Rick Ruijsink, Christophe de Wagter, Bart Remes, René lagarde, Ayuel Kacgor, Wouter Roos, Kristien de Klercq, Christophe Heynze, Gijs van der Veen, Pieter Moelans, Anand Ashok, Daan van Ginneken, Michiel Straathof, Bob Mulder and Meine Oosten at TU Delft. Johan van Leeuwen at Wageningen University. Eric den Breejen, Frank van den Bogaart, Klamer Schutte, Judith Dijk and Richard den Hollander at TNO Defence, Security, and Safety. DelFly research: Eric Karruppannan, Evert Janssen, Jos van den Boogaart, Henk Schipper, Ulrike Müller, Gijs de Rue and Johan van Leeuwen at Wageningen University. Rick Ruijsink at TU Delft. Insect wing research: Elke van der Casteele at SkyScan. Adriaan Beukers at TU, Delft. Mees Muller and Johan van Leeuwen at Wageningen University.

Appendix 1 Suggested Web Sites for Ordering Micro-Components and Materials Components/materials

Web site

Smallest RC system Ornithopter kits Motors, gears, etc. Micro-RC shop Micro-camera systems Micro-RC shop Miniature carbon fibre rods Lightweight indoor airplanes Micro-RC shop Micro-RC shop (e.g. Mylar) Alternative LP batteries Rapid prototyping

www.microflierradio.com www.ornithopter.org www.didel.com www.bsdmicrorc.com www.misumi.com.tw www.plantraco.com www.dpp-pultrusion.com www.indoorduration.com www.peck-polymers.com www.wes-technik.de www.atomicworkshop.co.uk www.quickparts.com

Appendix 2 Tested Parameters Parameter

Test method

Flight speed

Measured using both a stopwatch and video-analysis of a straight flight. The flight is performed along a reference red-white tape to measure distance and climb angle. The number of video frames is also used to measure time The flapping frequency was determined by examining audio-peaks in the audio track of the video recording. The audio file was filtered using GoldWave software The power is derived from the flapping frequency (video data) and the torque of the rubber band (torque meter in combination with counting the number of windings in the rubber band) Measured using video-analysis and red-white tape as a reference for rocking amplitude

Flapping frequency

Power

Rocking

References 1. Bradshaw, N.L., Lentink, D.: Aerodynamic and structural dynamic identification of a flapping wing micro air vehicle. AIAA conference, Hawaii (2008) 2. Ellington, C.P.: The aerodynamics of insect flight I-VI. Philosophical Transactions of the Royal Society of London. Series B 305, 1–181 (1984) 3. Ellington, C.P., van den Berg, C., Willmott, A.P., Thomas, A.L.R.: Leading-edge vortices in insect flight. Nature 384, 626–630 (1996) 4. Dickinson, M.H.: The effects of wing rotation on unsteady aerodynamic performance at low Reynolds numbers. The Journal of Experimental Biology 192, 179–206 (1994) 5. Dickinson, M.H., Lehmann, F.O., Sane, S.P.: Wing rotation and the aerodynamic basis of insect flight. Science 284, 1954–1960 (1999) 6. Fry, S.N., Sayaman, R., Dickinson, M.H.: The aerodynamics of free-flight maneuvers in Drosophila. Science 300, 495–498 (2003) 7. Jongerius, S.R., Lentink, D.: Structural analysis of a dragonfly wing. Journal of Experimental Mechanics, special issue on Locomotion (2009) 8. Kawamura, Y., Souda1, S., Nishimoto, S., Ellington, C.P.: Clapping-wing Micro Air Vehicle of Insect Size. In: N. Kato, S. Kamimura (eds.) Bio-mechanisms of Swimming and Flying. Springer Verlag. (2008) 9. Kesel, A.B.: Aerodynamic characteristics of dragonfly wing sections compared with technical aerofoils. The Journal of Experimental Biology 203, 3125–3135 (2000) 10. Lentink, D., Gerritsma, M.I.: Influence of airfoil shape on performance in insect flight. American Institute of Aeronautics and Astronautics 2003–3447 (2003)

14 The Scalable Design of Flapping MAVs 11. Lentink, D., Dickinson, M.H.: Rotational accelerations stabilize leading edge vortices on revolving fly wings. The Journal of Experimental Biology accepted (2009) 12. Lentink, D., Dickinson, M.H.: Biofluid mechanic scaling of flapping, spinning and translating fins and wings. The Journal of Experimental Biology accepted (2009) 13. Okamoto, M., Yasuda, K., Azuma, A.: Aerodynamic characteristics of the wings and body of a dragonfly. The Journal of Experimental Biology 199, 281–294 (1996) 14. Pornsin-Sirirak, T.N., Tai, Y.C., Ho, C.H., Keennon, M. (2001). Microbat-A Palm-Sized Electrically Powered Ornithopter. NASA/JPL Workshop on Biomorphic Robotics. Pasadena, USA 15. Rees, C.J.C.: Form and function in corrugated insect wings. Nature 256, 200–203 (1975a) 16. Rees, C.J.C.: Aerodynamic properties of an insect wing section and a smooth aerofoil compared. Nature 258, 141– 142 (13 November 1975) doi: 10.1038/258141aO Letter (1975b) 17. Sane, S.P.: The aerodynamics of insect flight. The Journal of Experimental Biology 206, 4191–4208 (2003)

205 18. Srygley, R.B., Thomas, A.L.R.: Unconventional liftgenerating mechanisms in free-flying butterflies. Nature 420, 660–664 (2002) 19. Tamai, M., Wang, Z., Rajagopalan, G., Hu, H., He, G.: Aerodynamic performance of a corrugated dragonfly airfoil compared with smooth airfoils at low Reynolds numbers. 45th AIAA Aerospace Sciences Meeting and Exhibit. Reno, Nevada, 1–12 (2007) 20. Usherwood, J.R., Ellington, C.P.: The aerodynamics of revolving wings I-II. The Journal of Experimental Biology 205, 1547–1576 (2002) 21. Wood, R.J.: The first takeoff of a biologically-inspired atscale robotic insect. IEEE Trans. on Robotics 24, 341–347 (2008) 22. Wootton, R.J.: Geometry and mechanics of insect hindwing fans: a modelling approach. Proceedings of the Royal Society of London. Series B 262, 181–187 (1995) 23. Wootton, R.J., Herbert, R.C., Young, P.G., Evans, K.E.: Approaches to The Structural Modelling of Insect Wings. Philosophical Transactions of the Royal Society 358, 1577–1587 (2003)

Chapter 15

Springy Shells, Pliant Plates and Minimal Motors: Abstracting the Insect Thorax to Drive a Micro-Air Vehicle Robin J. Wootton

Abstract The skeletons of the wing-bearing segments of advanced insects show unexploited potential in the design of biomimetic flapping MAVs. They consist of thin, springy, composite shells, cyclically deformed by large, enclosed muscles to flap the wings as first-order levers over lateral fulcra. The wings are light, flexible, membrane-covered frameworks, with no internal muscles, whose deformations in flight are encoded in their structure; they are ‘smart’ aerofoils. Both thorax and wings are apparently resonant structures, storing energy elastically, and tuned to deform appropriately at their operating frequencies. The form of the basic wing stroke is determined structurally, but is modulated by a series of controlling muscles, contracting tonically to alter the positions of skeletal components over the course of several stroke cycles. Fuel economy through lightness, low wing inertia and cyclic energy storage are all desirable in a flapping MAV. Furthermore, the insects’ peculiar combination of structural automation with modulation has great potential in achieving versatile kinematics with relatively few actuators. Aspects of the thoracic functioning of an advanced fly can be simulated in a simple card flapping model, combining the properties of a closed four-bar linkage with the elastic lateral buckling of a domed shell. Instructions for building this are included. Addition of further degrees of freedom, along with biomimetic smart wings, would seem to allow other crucial kinematic variables to be introduced and controlled with minimum actuation, and ways are suggested how this might be achieved in a sophisticated mechanism. R.J. Wootton () School of Biosciences, Exeter University, Exeter EX4 4QD, UK e-mail: [email protected]

15.1 Introduction Insects are superlative micro-air vehicles. This is widely recognised by engineers who have chosen to propel small flying robots by means of flapping wings, and insect flight specialists have been extensively consulted and sometimes actively involved in MAV development. Their contributions, however, have so far mainly been in the areas of flapping flight aerodynamics and of flight control. The mechanical processes and components to drive MAVs – the actuators, transmission and effectors – have tended to follow orthodox rather than biomimetic technology: electric motors or piezoelectric actuators driving novel, often beautifully ingenious mechanisms of stiff components, linked by bearings, and flapping rigid or simple flexible wings ([1, 2, 4, 12, 14–18, 22, 30, 31, 36] and see [4] for a useful classification of types); see also Chaps. 13 and 14. Insect ‘technology’ is very different. Muscle has no real parallel in our own engineering; and the insect flight skeleton, which provides both the transmission and the wings themselves, is a system of thin, springy shells, plates and frameworks. The thoraxes of advanced insects – flies, wasps, moths, true bugs – can be thought of as flexible monocoques which are deformed cyclically by the muscles, flapping the wings as first-order levers over lateral fulcra. The wings themselves are smart, deformable aerofoils, whose instantaneous shape through the stroke cycle is determined, largely automatically, by the interaction of their structural elasticity and the inertial and aerodynamic forces they are experiencing. Again there are no obvious parallels in our own technology – yet.

D. Floreano et al. (eds.), Flying Insects and Robots, DOI 10.1007/978-3-540-89393-6_15, © Springer-Verlag Berlin Heidelberg 2009

207

208

There are good reasons for this. Deformation of thin shells and plates under loading often involves buckling, which can be highly non-linear, is difficult to model and is often destructive. Furthermore, cyclical deformation can lead quickly to fatigue failure. These disadvantages can to some extent be overcome by using appropriate polymers and composites and by new computer modelling and optimisation techniques; but in most engineering contexts traditional methods are adequate and considerably more straightforward to use. Insects, however, overcome these difficulties with ease, and thereby gain considerable advantages. First, thin composite shells and plates have low mass; an advantage in any flying machine, but especially valuable in a flapping system, where the inertia of the moving parts and particularly of the wings needs to be minimised. Second, they are often springy, made of resilient materials in three-dimensional structures that deform elastically and are capable of elastic energy storage – again invaluable in an oscillating system. Elastic storage is a component of some existing MAV mechanisms (e.g. [16, 30, 31]), but achieved in more conventional ways. Third, they seem fairly insensitive to scaling effects; in some prominent groups similar mechanisms operate over a very wide size range, from the unnervingly large to the near-microscopic, with little apparent difference in morphology. These properties should all commend themselves to MAV designers, and it seems possible that insect solutions may have much to offer in the development of small robotic flapping mechanisms. This chapter will therefore explore the possibilities and implications of adopting and adapting aspects of the insect thoracic skeleton in designing the transmissions of small, versatile, manoeuvrable low-speed MAVs with some insect flight characteristics and will also examine how an understanding of insect wing functioning might lead to the development of more effective flapping aerofoils than have so far been employed.

15.2 Some Requirements of a Small, Versatile Flying Machine: How Do Insects Manage?

R.J. Wootton

of this chapter; we are concerned with design requirements of the last two. These need the following.

15.2.1 Low Mass Flapping flight is expensive, especially at low speeds. Power is at a premium; and minimising both weight and the inertial cost of flapping is a major design consideration. The cuticular skeleton of insects, which also serves as a skin forming almost the entire interface between the insect and its surroundings, is suitably light. It consists mainly of an extraordinarily versatile array of composite materials, chemically and structurally related but varying markedly from place to place in the orientation of the fibrous component, which consists of microfibrils of the carbohydrate chitin, and in the composition and degree and nature of cross-linking of the protein molecules that provide the matrix of the composite material. These variations provide a range of mechanical properties – stiffness, toughness, hardness, strength, resilience – apparently optimised everywhere for the forces encountered and for the many local functions that the cuticle serves [20, 23]. In the skeleton of the thorax itself cross-linked cuticle provides areas of rigid, but often springy, plates and curved shells (‘sclerites’), which may locally be reinforced by internal ridges or thinned for greater pliancy. Between and continuous with them are areas of soft, compliant cuticle, which may locally include tensile, tendon-like bands joining muscles to plates or plates to plates and sometimes bands or pads of rubbery, elastomeric protein. The wings themselves consist almost entirely of cuticle, which provides both the supporting, usually tubular, veins and the membrane, whose thickness varies greatly but can be less than 1 μm. With no internal muscles and little contained fluid the wings are usually extremely light with low moments of inertia – important, since flapping frequencies of several hundred hertz are common – and the potential for inertial energy loss is substantial. Rigidity is enhanced by relief: corrugation and camber, raising the second moment of area of cross sections, with little extra cost in material and mass.

15.2.2 Appropriate Kinematics Flying machines, whether natural or man-made, need a power source, actuators, a control system, a transmission and effectors. The first three are outside the scope

Appropriate wing kinematics are essential in flapping flight. These become more complex at low speeds or

15 Springy Shells, Pliant Plates and Minimal Motors

209

more precisely at lower values of the advance ratio, J. This is a measure of the ratio of forward speed to flapping speed and is given by J = V/2nR, where V is the forward velocity,  the stroke amplitude in radians, n the stroke frequency and R the wing length [7]. At high J values the wings meet the air at positive angles of attack on both downstroke and upstroke. In fast forward flight the wings of birds and bats undergo minimal change in shape and attitude, and this is also true of the relatively small range of insects, including some butterflies, that fly fast with slow strokes. For this reason it is relatively easy to build a simple fast-flying model whose wings simply flap up and down, and several are available as toys, showing remarkably lifelike flight. The best known are the inexpensive Tim and Timmy birds, by Schylling Toys, whose simply supported wings are flapped by an ingenious variant on the open four-bar linkage driven by wound synthetic rubber loops, widely used in ‘ornithopters’ by aeromodellers. However, as the value of J decreases, changes in wing attitude and/or shape between the two halfstrokes become increasingly necessary if the upstroke is not to exert adverse forces on the wings, with a net downward component. Slow flight at low J values, and hovering, where J=0, hence require the wings to twist and sometimes to change shape between the halfstrokes in order to either minimise upstroke forces or direct them favourably. Furthermore, in many insects wing twisting, appropriately timed, is itself responsible for generating bursts of high vorticity and hence high lift around the points of stroke reversal, in a range of unsteady aerodynamic mechanisms that may be essential to support the insects’ weight and in the fine control of accelerations and manoeuvres [5, 8, 26]. Insects achieve these kinematic skills by unique methods. In the great majority of species, flapping is achieved by cyclic contraction of opposing sets of ‘indirect’ muscles, inserted on thoracic sclerites remote from the wings, rather than on the wings themselves as in flying vertebrates. These muscles alternately pull the top of the thorax down, levering the wings up, and cause it to bow upwards by compressing it longitudinally, levering the wings down (Fig. 15.1a,b). Other movements of the whole wing – promotion and remotion, some basal twisting, control of the wing stroke path, folding and extension – are

Fig. 15.1 Diagrammatic cross section of a generalised insect thorax showing the action of the flight muscles: (a) upstroke and (b) downstroke

achieved by a series of ‘direct’ muscles, inserted on, or connected by tensile cuticle bands to sclerites at the extreme wing base. In the great majority of species, the indirect muscles powering the wing stroke are of a physiologically and histologically distinct type, known as ‘asynchronous’ or ‘fibrillar’ muscle, and unique to insects, whose contraction frequency is not limited by the frequency of the incoming nerve impulses. This allows the insects to operate if necessary at stroke frequencies far higher than those with orthodox muscle only, which are constrained to values below c. 100 Hz. The direct muscles, on the other hand, appear always to be of the orthodox kind; and in insects with asynchronous power musculature, changes in the wing stroke variables which the former control are achieved by slower, tonic contractions that modify the positions of the basal sclerites during the course of several stroke cycles. There is some evidence that the direct muscles can influence the shape of the wings at their extreme base, but wing shape is otherwise passively determined from instant to instant during the stroke cycle by the interaction of the wings’ elasticity with the inertial and aerodynamic forces that they are receiving. These alterations in shape – twisting, flexion, change of cross section – are an integral part of the flight process, and they are effectively programmed into the wings’ structure: in the relief, the form and arrangement of the supporting veins, the distribution of rigid and flexible components. The wings are in effect ‘smart’ aerofoils, combining remote and automatic shape control in ways which seem to occur nowhere else in nature or in technology [32, 33, 35].

15.3 Biomimetic Possibilities Biological systems have limitations as well as advantages – see Vogel [24] for a thoughtful analysis. They

210

have no access to bulk metals or (above the level of bacterial flagella) to rotational joints, so that movement involving rigid components is restricted to oscillation and reciprocation. Their actuators can only pull, not push. Most importantly, their designs are constrained by their ancestry; they have always to develop from those of their immediate predecessors. They cannot, as engineers can, learn from, adopt, combine and adapt the designs of widely separate groups. Engineers contemplating biomimetic design solutions therefore face a sequence of decisions. Is the naturally occurring solution to the problem in question the best available? If so, is it feasible and sensible to adopt it? If so, to what extent? Copying the principles of insect thoracic skeletons appears to have real potential in MAV design. It is at present beyond us to emulate the extraordinary versatility of insect cuticle, but combining appropriate tough, fatigue-resistant polymers and composites in an optimally designed thin shell would seem to be a feasible, and ultimately simple and reproducible way of building an effective low mass transmission. MAV wings, too, could usefully copy the smart properties of insect wings; the greater the degree of structurebased automation, the less information needs to be processed from instant to instant, and the fewer actuators are required. Minimising the number of actuators is a major consideration in MAV development, and insects here give us no cause for optimism. In dragonflies, insects which instinctively appeal to biomimetic engineers because of their appropriate size and superlative flight skills, no fewer than 50 muscles are involved in flight. Even flies (Diptera), which operate with a single pair of aerofoils on one thoracic segment, use 38 muscles [28]. Here at least, close biomimicry seems impracticable. The implications of so many actuators in terms of weight and control in an MAV are obvious, and different solutions are needed.

15.3.1 Kinematic Requirements A fully manoeuvrable flapping MAV may need to be capable of varying and controlling most or all of the following: A. Flapping amplitude: Most insects whose flight has been studied appear to use amplitude change as a means of varying the strength of the net aero-

R.J. Wootton

B.

C.

D.

E.

dynamic force, though the relationship between amplitude and flight velocity is far from simple [6]. Stroke path: The principal reason to vary the trajectory of the wing stroke is to control the direction and centring of the mean aerodynamic force vector of each stroke, or in low frequency flappers each half-stroke, relative to the centre of mass of the insect; hence inducing and controlling movement around the three rotational axes of space: pitch, roll and yaw. Control of pitch is particularly important, in influencing the angle of the stroke plane to the horizontal: the stroke plane angle – see below. Most insects studied are capable of modifying their stroke path to a considerable extent [3, 7, 10, 25, 27]. The ‘figure of eight’ shape that appears in much of the earlier literature is only one of a range of such paths, and the emphasis on achieving this that has guided some recently published flapping mechanisms is probably quite unnecessary. Stroke plane angle: In many insects the flapping movement of the wing approximates to a plane. The angle of this plane – calculated as the slope of the linear regression of the vertical component of the wing tip path on the horizontal axis [7] – is a major determinant of the direction of the mean force vector, and hence of flight velocity. The degree and timing of wing twisting during the stroke: Twisting, as we have seen, is essential at low values of J, and its timing can be important in unsteady lift mechanisms. The capacity for lateral stroke asymmetry in amplitude, stroke path, the degree or timing of twist, or in any combination of these, to facilitate manoeuvres.

Control of another possible kinematic variable – stroke frequency – is probably less important. Frequency appears to vary little in individual insects and is probably kept close to the resonant frequency of oscillation, where energy expenditure is minimised [6, 13]. For economy any flapping mechanism in an MAV should be designed as a resonant system, and its frequency would then be relatively invariate.

15.4 An Appropriate Thorax Design for Abstraction – Higher Flies Insect thoraxes are extremely diverse, and the mechanics of very few have been studied in detail. While there

15 Springy Shells, Pliant Plates and Minimal Motors

a

211

b

Fig. 15.2 The indirect flight muscles of the blowfly Calliphora: (a) the wing elevators and (b) the wing depressors

is nothing to preclude the abstraction and combining of principles from several different groups, I shall base the following on the blowfly Calliphora, whose flapping mechanics have been investigated in particular depth [9, 19, 28, 29]. Figure 15.2 shows semidiagrammatically the positions of the flapping muscles in a sagittal section of the blowfly mesothorax – the flapping body segment. Wing elevation is achieved by contraction of an array of large, paired dorsoventral muscles (Fig. 15.2a) inserted dorsally on the notum – the thoracic roof – and ventrally low down on the pleura – the sides of the thorax. Wing depression is brought about by contraction of huge dorsal longitudinal muscles (Fig. 15.2b), inserted broadly on the anterior part of the domed notum, and posteriorly on the vertical back of the notum and an internally extending plate, the phragma. These, together with a pair of small lateral muscles, not illustrated, are ‘indirect’; remote from the wing itself. Three further pairs of small muscles, not shown, serve to tension the thoracic box, and no fewer than 13 further pairs, inserted directly on the

complex of small sclerites around the wing hinge, are concerned with wing folding, in controlling the positioning and attitude of the wing base and in modifying the form of the stroke. It would be absurd to attempt to copy such a complex actuation system in an MAV. The correct approach would seem to develop a simplified transmission based on the thoracic skeleton and to consider from first principles what actuation would be necessary to achieve the necessary kinematics. The following account of the deformation of the mesothorax largely follows that in [19, 29]. Figure 15.3a shows the mesothorax from the side. The four round dots represent transverse axes, about which the thorax distorts cyclically under the action of the indirect, flapping muscles. The positions of the anterior and ventral axes are approximate; their presence is betrayed by the cyclic widening and contracting of the adjacent pointed clefts, filled with soft cuticle. Posteriorly two more clefts also widen and narrow by the cyclic elevation and lowering of the processes X and Y.

Fig. 15.3 (a) Side view of the mesothorax of Calliphora, showing the three principal processes, X, Y and Z, to which the wing attaches and the four rotational axes about which the thorax

deforms in flight. (b, c) The mesothorax treated as a four-bar linkage, with three out-of-plane coupler bars. (b) Side view and (c) from above

212

These are flexibly linked to the base of the wing; and the latter rests on the process Z, which acts as the fulcrum. In flapping, the up and down movements of X impart some twist to the wing base, so that the latter tends to pronate as the wing is depressed and to supinate as it rises. The mesothoracic flapping mechanism can be simplified as a three-dimensional four-bar linkage, with fixed coupler bars projecting inwards from three of the four, their ends representing the three processes, X, Y and Z, to which the wing is attached (Fig. 15.3b,c). Z, the fulcrum, is more laterally situated than X and Y; and X is positioned slightly more laterally than Y. The proportions of the bars and the positions of X, Y and Z are critical. If correct, low-amplitude compression at the points indicated by arrows raises X and Y relative to Z and would depress an attached wing. X moves higher than Y, which would tend to twist the wing base and pronate the wing; and Y also moves slightly posteriorly, and closer to X, tending to promote the wing. Hence, as a direct consequence of the structure of the mechanism, a single movement depresses, pronates and promotes the wing; and conversely tension at the indicated points would raise, retract and supinate the wing – all in keeping with actual wing kinematics. Since a four-bar linkage has only one degree of freedom, the system should theoretically be operable with a single actuator, replacing the four sets of indirect muscles in the insect itself. In the fly the four bars are actually thin shells, and in no case do the ends of the flexible clefts coincide with the axes of rotation, so that some lateral buckling is inevitable. The notum in particular is domed, and Ennos [9] found that contraction of the dorsal longitudinal muscles caused the sides to buckle outwards at the process Y, assisting in wing depression. This is an elastic process, and a potential site of cyclic energy storage. Figure 15.4 shows a cardboard flapping model that combines these properties and can be worked by simple pressure of thumb and forefinger. Figure 15.5 is a flat design for the model, with instructions for building it. If properly constructed the mechanism produces an automated flapping cycle with a degree of appropriately timed promotion, remotion and pronatory and supinatory twisting and twisting. For a manoeuvrable MAV, however, a mechanism with a single degree of freedom is inadequate. The only kinematic variables that could be altered in flight

R.J. Wootton

Fig. 15.4 The thorax modelled as a four-plate linkage with lateral elastic buckling

are stroke amplitude and, within the limits imposed by resonance, frequency. To be able to change the stroke path and the stroke plane angle, additional freedom and actuation are required. Here another insect trick could if necessary be adopted: the tonic contraction of the direct, controlling muscles, acting slowly by altering the positions of thoracic components over several flapping cycles. Two approaches, available to engineers but not to insects, appear to be worth exploring. 1. The stroke path could be made adjustable by introducing active movement to the lateral fulcra. 2. An extra three-dimensional, shell-like bar could be added to the linkage, increasing its mobility to 2 and theoretically necessitating one additional actuator. Interaction of the two actuators, operating at the same frequency, should allow precise instantaneous control of the relative motions of Y and Z, and hence of the wing tip path, and using one actuator ‘tonically’ to position the mechanism over several cycles is also an option. One feature of the four-bar system is lost. Experiment shows that active twisting of the wing base can no longer be appropriately coordinated with the flapping cycle, so that the bar corresponding to the process X, which causes twisting in the four-bar model, needs to be lost. Promotion and remotion, the other functions of X, are components of the stroke path and hence controllable by the newly acquired mobility of the system. Active wing twisting would now need separate actuation. However, this may not be necessary. Most of the

15 Springy Shells, Pliant Plates and Minimal Motors

213

anterior

A

a

a x

y

y x

b

b

Fig. 15.5 Cut-out design for the thorax model: (A–D) cut-out proformas and (E) diagrammatic detail from Fig. 15.4, showing the positions of the cords at the articulation. Assembly instructions: Use thin card, about the thickness of a standard index card. A glue stick is best for adhesive. Paperclips and small staples are useful for the temporary support of glued surfaces, and fine forceps or pliers help in making the wing articulations. Carefully score along the broken lines. Those with longer dashes are actual fold lines, the smaller dashes facilitate the operation of the mechanism. (A) The dorsal component of the model, corresponding to the insect’s notum. Bend the triangular tabs at each end until they overlap as shown and glue them together, forming two triangular, boat-shaped ends. Fold in the tabs marked ‘a’ and ‘b’, but do not glue them at this stage. (B) The ventral component. Glue together the triangular/square tabs at each end, as indicated, creating diamond-shaped, boat-shaped ends, with a transverse crease across each diamond. Fold back the tabs marked ‘c’, but do not glue them at this stage. Flex the model along the line ‘d–d’ through approximately 90◦ and glue the tabs marked ‘e’ to the shaded squares. (C) The leading edge spar of the wing, and two should be made. Crease along all the broken lines, fold along the middle one and mould the result into a v-shaped cross section, with the small, cut-out rectangle on

10cm

the concave side of the V, near the pointed end. Glue all but the shaded area. Join the dorsal and ventral components of the body (A) and (B) at the ends by gluing the protruding, flexible, triangular part of the diamond-shaped ends of (B) inside the triangular ends of (A), making sure that both components are similarly orientated. Allow to dry. Cut four lengths of thin cord, or strong thread, each ca. 4 cm long. On each side, glue ca. 1.5 cm along the fold line of tab a, with the free length protruding posteriorly at point y, corresponding to point Y on the fly thorax in Fig. 15.3. Glue tab a in place, anchoring the cord. Bring the pointed end of the wing spar (C) up to point y and glue the next section of the cord along the concave face, behind the innermost layer, bringing it out through the cut-out rectangular slot. Complete the gluing of the spar base, which should now be capable of free rotation about y. Glue the remaining end of the cord down the fold line of tab c in the ventral component of the model. The upper end of tab c, point z, forms the fulcrum for the wing, corresponding to point Z in Fig. 15.3. Both y and z should be in flexible contact with the wing spar, with very little cord showing. Glue tab c in place. Glue ca. 1.5 cm of the next piece of cord along the fold line of tab b, with the free end protruding anteriorly at point x. Glue the tab in place. Cut two of the shapes marked (D) from paper and glue the straight sides

214

R.J. Wootton

C

D E

anterior

B e

d

c z

d

e c

z

Fig. 15.5 (Continued) to the wing spars. The model is now virtually complete. The wing spars should be level, projecting transversely and slightly above the horizontal. Glue the free ends of the cord extending from point x to the underside of the posterior part of the wing, leaving enough cord between x and the wing to allow free movement. Trim off any surplus cord. To

operate the model, hold the anterior part of the ventral component between the thumb and forefinger of one hand. Support the most ventral part of the model (at d–d) with the thumb of the other hand and gently squeeze repeatedly with the forefinger at the posterior extremity of the ventral component

torsion in insect wings takes place within the span, and appears often to be driven solely by inertial forces on the wings, perhaps with an aerodynamic component. It may well be that in a suitably constructed and tuned wing no active twisting is required. There remains the need to introduce lateral asymmetry to the wing stroke, in the form of differential amplitude or torsion, in order to manoeuvre. For either of these, additional actuation is essential. Four approaches deserve consideration.

A. In the movable fulcrum option described above, asymmetry would be provided by separate actuation of the two sides. B. It may be possible to achieve the necessary effects by actively stiffening or warping the shell on one side through several stroke cycles. Pressure applied to one side of the model in Fig. 15.4 can alter the form of the stroke on that side, indicating that it may be possible to optimise this effect by careful design.

15 Springy Shells, Pliant Plates and Minimal Motors

C. Another solution available to engineers, though not to insects, may be to allow active movement of the centre of mass of the body relative to the centre of aerodynamic force. D. The effects may be achievable by remote control of the instantaneous shape of the wings themselves. We will explore the last solution.

15.5 Wing Biomimicry MAV designers have so far paid little attention to the biomimetic possibilities that insect wings offer. Approaches to modelling their unique properties, which combine structural automation of their kinematics with remote control, have been explored by Wootton et al. [35], with reference to much earlier work; see also Chap. 11. Insects make extensive use of relief to stiffen their wings. This not only minimises mass but also provides differential flexibility in different planes. For example, a wing with longitudinal pleats is flexible along axes parallel to the pleats, but rigid to bending across them. Transverse bending is only possible if the pleats can be flattened or if the pleats on the inside of the curve can bow upwards into the plane of those on the outside [21]. A wing with a cambered section is more rigid than one with a flat section, and asymmetric in its response to bending forces. Force applied

Fig. 15.6 A cardboard (a) and paper (b) wing, demonstrating how three flexion lines can allow and control supinatory twisting in the upstroke. Explanation in the text

215

from the convex side flattens the section and allows non-destructive bending with a fairly large radius of curvature, but applied from the concave side tends to increase the camber and hence the rigidity, and eventually leads to local buckling and destructive failure [32]. If the force is centred behind the wing’s torsional axis, this asymmetric response to bending leads to asymmetry in resistance to twisting, which combines bending and torsion; and this simple property appears to be extensively used by insects to facilitate passive twisting in the upstroke while resisting it in the downstroke [11, 34]. In many insects with broadly supported wing bases, or whose fore and hind wings are coupled together, upstroke twisting of the distal part of a cambered wing is achieved by ventral bending along an oblique line of flexibility. The amount of bending and angle of twist are related to the height of the wing camber proximally to the line of flexion, and this seems to be actively controllable by muscles at the wing base. Bending and torsion of a cambered or pleated wing involve elastic deformation, and it seems that wings too are resonant structures. Like the thoracic box they need to deform correctly at their working frequency, and one can identify morphological features that seem adapted to tune them to do so. These principles can readily be modelled physically and could certainly be used in designing wings for an MAV. Figure 15.6 shows one such model. Support is provided by the stippled area, which is made of thin card. The leading edge section is curved ventrally and

216

is crossed by two oblique lines of flexibility, a–a and b–b, made by cutting part way through the card with a sharp blade. The broad basal stippled area is also cambered and is crossed by a longitudinal flexion line c–c that crosses b–b. The rest of the wing is made of paper. The wing has interesting and unexpected properties, best understood by making the model. The curved section of the leading edge makes it resistant to bending when force is exerted on the ventral, concave side, as it would be in a downstroke. If the force is applied to the convex side, behind the wing’s torsional axis, as it would be in an upstroke, the leading edge bends slightly about a–a, and the wing twists readily towards the tip and could easily assume a positive angle of attack and generate useful upward force. Bending about b–b greatly enhances the twisting, but this is controllable by varying the camber of the basal area around c–c. When the base is nearly flat, bending at both a–a and b–b allows the distal part of the wing to twist dramatically (Fig. 15.6b). Steeper basal camber limits bending to a–a and the wing twists far less. A wing so designed would twist automatically to some extent in the upstroke, but the extent could be controlled over a wide range by simple basal actuation.

15.6 Conclusion It seems possible that a complete system comprising a transmission with the properties of a five-bar linkage in a resonant, springy shell, together with a pair of smart wings with actively variable basal camber, could drive an MAV having mechanical control over all the kinematic variables that we have identified as essential for versatile flight, with a rather small number of appropriately designed and located actuators. The mechanism in Figs. 15.4 and 15.5 and the wing in Fig. 15.6 are naïve examples; a sophisticated design would be a deformable monocoque, optimised using modern modelling software, with similar optimised wings. Such a system would have the added advantages of low weight and inertia and relative economy through cyclic elastic energy storage and release. It should moreover be fairly easy to build and replicate, and be capable in time of progressive miniaturisation, as smaller motors, power stores and control circuits become available.

R.J. Wootton

References 1. Avadhanula, S., Wood, R.J., Steltz, E. Yan, J., Fearing, R.S.: Lift force improvements for the Micromechanical Flying Insect. IEEE International Conference on Intelligent Robots and Systems 1350–1356 (October 2003) 2. Banala, S., Agrawal, S.K.: Design and optimization of a mechanism for out-of-plane insect wing-like motion with twist. Transactions ASME, Journal of Mechanical Design 127, 817–824 (2005) 3. Betts, C.R.: The kinematics of Heteroptera in free flight. Journal of Zoology B 1, 303–315 (1986) 4. Conn, A.T., Burgess, S.C., Ling, S.C.: Design of a parallel crank-rocker flapping mechanism for insect-inspired micro air vehicles. Proceedings of the Institution of Mechanical Engineers C. Journal of Mechanical Engineering Science 221(10), 1211–1222 (2007) 5. Dickinson, M.H., Lehmann, E.O., Sane, S.P.: Wing rotation and the aerodynamic basis of insect flight. Science 284, 1954–1960 (1999) 6. Dudley, R.: The Biomechanics of Insect Flight. Princeton University Press. Princeton, N.J. (2000) 7. Ellington, C.P.: The aerodynamics of hovering insect flight. III. Kinematics. Philosophical Transactions of the Royal Society London B 305, 41–78 (1984) 8. Ellington, C.P.: The aerodynamics of hovering insect flight. IV Aerodynamic mechanisms. Philosophical Transactions of the Royal Society London B 305, 79–113 (1984) 9. Ennos, A.R.: A comparative study of the flight mechanism of Diptera. Journal of Experimental Biology 127, 355–372 (1987) 10. Ennos, A.R.: The kinematics and aerodynamics of the free flight of some Diptera. Journal of Experimental Biology 142, 49–85 (1989) 11. Ennos, A.R.: Mechanical behaviour in torsion of insect wings, blades of grass, and other cambered structures. Procedings of the Royal Society London B 259, 15–18 (1995) 12. Galinski, C., Zbikowski, R.: Insect-like flapping wing mechanism based on a double spherical Scotch yoke. Journal of the Royal Society Interface 2(3), 223–235 (2005) 13. Greenwalt, C.H.: The wings of insects and birds as mechanical oscillators. Proceedings of the American Philosophical Society 104, 605–611 (1960) 14. Khan, Z.A., Agrawal, S.K.: Design of flapping mechanisms based on transverse bending mechanisms in insects. Proceedings of the 2006 IEEE International Conference on Robotics and Automation, Orlando, Florida, 2323–2328 (2006) 15. Madangopal, R., Khan, Z.A., Agrawal, S.K.: Biologically inspired design of small flapping wing air vehicles using four-bar mechanisms and quasi-steady aerodynamics. Journal of Mechanical Design 127(4), 809–816 (2005) 16. Madangopal, R., Khan, Z.A., Agrawal, S.K.: Energeticsbased design of small flapping-wing micro air vehicles. IEEE/ASME Transactions on Mechatronics 11(4), 433– 438 (2006) 17. McIntosh, S.H., Agrawal, S.K., Khan, Z.A.: Design of a mechanism for biaxial rotation of a wing for a hovering

15 Springy Shells, Pliant Plates and Minimal Motors

18.

19.

20.

21.

22.

23.

24. 25.

26.

vehicle. IEEE/ASME Transactions on Mechatronics 11(2), 145–153 (2006) Mukerjee, S., Sanghi, S.: Design of a six-link mechanism for a micro air vehicle. Defence Science Journal 54, 271– 276 (2004) Nachtigall, W.: Mechanics and aerodynamics of flight. In: G.J. Goldsworthy, C.H. Wheeler (eds.) Insect Flight, pp. 1–28. CRC Press Inc. Boca Baton (1989) Neville, A.C.: Biology of Fibrous Composites: Development Beyond the Cell Membrane. Cambridge University Press, Cambridge, U.K. (1993) Newman, D.J.S., Wootton, R.J.: An approach to the mechanics of pleating in dragonfly wings. Journal of Experimental Biology 125, 361–372 (1986) Steltz, E., Wood, R.J., Avadhanula, S., Fearing, R.S.: Characterization of the Micromechanical Flying Insect by optical position sensing. Proceedings of the 2005 IEEE International Conference on Robotics and Automation, Barcelona 1–4, 1252–1257 (2005) Vincent, J.F.V.: Insect cuticle: a paradigm for natural composites. In: J.F.V. Vincent, J.D. Currey (eds.) The Mechanical Properties of Biological Materials, pp. 183– 210. Symposia of the Society for Experimental Biology 34. Cambridge University Press, Cambridge UK (1980) Vogel, S.: Cats’ Paws and Catapults. 382 pp. W.W. Norton and Company New York (1998) Wakeling, J.M., Ellington, C.P.: Dragonfly flight. II. Velocities, accelerations and kinematics of flapping flight. Journal of Experimental Biology 200, 557–582 (1997) Weis-Fogh, T.: Quick estimates of flight fitness in hovering animals, including novel mechanisms for lift production. Journal of Experimental Biology 59, 169–230 (1973)

217 27. Wilmott, A.P., Ellington, C.P. The mechanics of flight in the hawkmoth Manduca sexta. I. Kinematics of hovering and forward flight. Journal of Experimental Biology 200, 2705–2722 (1997). 28. Wisser, A., Nachtigall, W.: Functional-morphological investigations on the flight muscles and their insertion points in the blowfly Calliphora erythrocephala (Insecta, Diptera). Zoomorphology 104, 188–195 (1984) 29. Wisser, A., Nachtigall, W.: Mechanism of wing rotating regulation in Calliphora (Insecta, Diptera). Zoomorphology 111, 111 (1987). 30. Wood, R.J.: Design, fabrication and analysis of a 3 DOF, 3 cm flapping-wing MAV. IEEE/RSJ IROS, San Diego, CA, (October 2007). 31. Wood, R.J.: The first take off of a biologically-inspired atscale robotic insect. IEEE Transactions on Robotics 24 (2), 341–347 (2008). 32. Wootton, R.J.: Support and deformability in insect wings. Journal of Zoology London 193, 447–468 (1981) 33. Wootton, R.J.: Functional morphology of insect wings. Annual Review of Entomology Palo Alto 37, 113–140 (1992) 34. Wootton, R.J.: Leading edge section and asymmetric twisting in the wings of flying butterflies. Journal of Experimental Biology 180, 105–117 (1993) 35. Wootton, R.J., Herbert, R.C., Young, P.G., Evans, K.E.: Approaches to the structural modelling of insect wings. Philosophical Transactions of the Royal Society London B 358, 1577–1587 (2003) 36. Zbikowski, R., Galinski, C., Pedersen, C.B.: Four-bar linkage mechanism for insectlike flapping wings in hover: Concept and an outline of its realization. Journal of Mechanical Design 127, 817–824 (2005)

Chapter 16

Challenges for 100 Milligram Flapping Flight Ronald S. Fearing and Robert J. Wood

Abstract Creating insect-scale flapping flight at the 0.1 gram size has presented significant engineering challenges. A particular focus has been on creating miniature machines which generate similar wing stroke kinematics as flies or bees. Key challenges have been thorax mechanics, thorax dynamics, and obtaining high power-to-weight ratio actuators. Careful attention to mechanical design of the thorax and wing structures, using ultra-high-modulus carbon fiber components, has resulted in high-lift thorax structures with wing drive frequencies at 110 and 270 Hz. Dynamometer characterization of piezoelectric actuators under resonant load conditions has been used to measure real power delivery capability. With currently available materials, adequate power delivery remains a key challenge, but at high wingbeat frequencies, we estimate that greater than 400 W/kg is available from PZT bimorph actuators. Neglecting electrical drive losses, a typical 35% actuator mass fraction with 90% mechanical transmission efficiency would yield greater than 100 W/kg wing shaft power. Initially the micromechanical flying insect (MFI) project aimed for independent control of wing flapping and rotation using two actuators per wing. At resonance of 270 Hz, active control of a 2 degrees of freedom wing stroke requires precise matching of all components. Using oversized actuators, a bench top structure has demonstrated lift greater than 1000 μN from a single wing. Alternatively, the thorax structure can be drastically simplified by using passive wing rotation and a

R.S. Fearing () Biomimetic Millisystems Lab, Univ. of California, Berkeley, CA, USA e-mail: [email protected]

single-drive actuator. Recently, a 60 mg flapping-wing robot using passive wing rotation has taken off for the first time using external power and guide rails.

16.1 Motivation and Background Flies (order Diptera) are arguably the most agile objects on earth, including all things man-made and biological. They can fly in any direction, make 90◦ turns in tens of milliseconds, land on walls and ceilings, and navigate very complex environments. It is natural then to use flies as inspiration for a small autonomous flying robot. However, this bio-inspiration must be done with care. There are certain aspects of insect morphology and physiology which would not make sense to replicate (reproduction, for example). So our bio-inspiration paradigm hopes to observe natural systems and extract the underlying principles. Then we apply our most advanced engineering techniques in concert with these principles to achieve a desired goal. Insects control flight with a three degrees of freedom (DOF) wing motion and either one or two pairs of wings. This discussion focuses on two-wing insects for two reasons: first, the agility of Dipteran insects is arguably rivaled only by a few species of Odonata. Second, the mechanical complexity of four wings is simply greater than that of two. The three DOF wing trajectory consists of flapping, rotation, and stroke plane deviation. Flapping (upstroke and downstroke) defines the stroke plane. Rotation consists of pronation and supination about an axis parallel to the spanwise direction. The final DOF is stroke plane deviation; however, this will not be considered due to the fact that hovering Dipteran wing motions can be approximately characterized with only two rotational axes.

D. Floreano et al. (eds.), Flying Insects and Robots, DOI 10.1007/978-3-540-89393-6_16, © Springer-Verlag Berlin Heidelberg 2009

219

220

R.S. Fearing and R.J. wood

(a)

insect flight [18]. Finally, dynamically scaled robotic insect wings have resulted in approximate quasi-steady empirical models using lift and drag coefficients to hide the unsteady terms [8]. These empirically derived models are used throughout the design of a robotic fly due to their relative simplicity. These models provide the engineer with a first-order approximation to the forces and moments expected from a pair of flapping wings. This chapter will describe the design and fabrication of two classes of robotic flies, shown in Fig. 16.1, using characteristics and models derived from insect flight.

16.2 Design of High-Frequency Flapping Mechanisms

(b) Fig. 16.1 Prototype flapping-wing MAVs, with integrated air frame, thorax, and piezoelectric actuators, but offboard power. (a) UC Berkeley micromechanical flying insect (130 mg). (b) Harvard microrobotic fly (60 mg)

This periodic wing trajectory exists at a Reynolds number of approximately 100–1000 and thus the flow around the wings is mostly separated. Biologists have identified key features of the flow patterns of hovering Diptera and collectively called these ‘unsteady aerodynamics’ [8, 17]. No closed-form analytical description of the unsteady aerodynamics exists due to the challenges in capturing all fluid interactions with non-trivial airfoil deformations. Moreover, the vast diversity in wing morphology (e.g., shapes, textures, anisotropic compliance) offers a further impediment to a simple description of flapping-wing flight [5, 6]. Similarly, numerical simulations (solving the Navier– Stokes equations) have proven difficult for broad studies of multiple simultaneous unsteady aerodynamics phenomena. However, simplified wing models and kinematics have been used to explain some aspects of

Diptera have two sets of flight muscles: direct and indirect [10] as shown in Fig. 16.2. The indirect flight muscles control flapping and provide the vast majority of power for flight [9]. The direct flight muscles insert directly on the pleural wing process via basalar sclerites [13]. It is thought, therefore, that the direct flight muscles are involved with control of pronation and supination of the wing. Details of insect wing drive systems are provided in Chap. 15.

Fig. 16.2 Simplified drawing of a Dipteran thorax. The indirect flight muscles (dorsoventral and dorsolongitudinal) create the upstroke and downstroke, respectively. The direct flight muscles insert on the base of the wing hinge at the pleural wing process (adapted from [10])

16 Challenges for 100 Milligram Flapping Flight

221

(a) transmission

airfoil

(b)

airframe

actuator

Fig. 16.3 (a) Half of original four actuator thorax with two independent DOF per wing [14]. (b) Simplified thorax with single-drive actuator and passive wing rotation [19]

The first design for a flapping-wing MAV (Fig. 16.3a) combines power and flight control actuators to provide direct control of pronation and supination. However, it is speculated that dynamic forces acting on the wing during flight also contribute to wing rotation [12]. The second design for a flapping-wing MAV (Fig. 16.3b) uses this latter assumption and relies on passive wing rotation.

16.2.1 Four Actuator Thorax A wing drive mechanism was designed to provide simultaneous control of wing flapping and rotation angles using a two-input two-output transmission system shown in Fig. 16.3a. To minimize reactive power required to drive the wing inertia, the thorax is designed to operate near mechanical resonance as described by Avadhanula et al. [2]. Each wing is driven

by two piezoelectric bimorph bending actuators [22], which provide an unloaded displacement of ± 250 μm and blocked force of ± 60 mN [22]. The transmission is designed [14, 15, 20, 3] to convert this high-force small displacement to an ideal wing stroke of ± 60◦ , with an equivalent transmission ratio of approximately 3000 rad m−1 . The MFI structure in Fig. 16.3a uses two stages of mechanical amplification followed by a differential element to couple the individual actuator motion into wing flapping and rotation. The first stage slidercrank converts actuator linear displacement into ± 10◦ input to the planar four bar. The four bar has a nominal amplification of 6:1, providing an ideal ± 60◦ output motion. Finally, the two planar four bars are coupled into a spherical five-bar differential element [2, 3], an approximation to the insect wing hinge. The differential element converts angle difference between the four bars into wing rotation, such that a 22◦ angle difference gives rise to a 45◦ wing rotation. The original goal of this design was to achieve independent control of flapping and rotation, providing much greater control moments than even real insects can obtain. As discussed in Sect. 16.6.1, wing inertial and aerodynamic coupling effects dominate the available actuator control effort, making independent control difficult to achieve. The lessons learned from the four actuator MFI motivated the design of a structure with greatly reduced complexity, described next.

16.2.2 Single Actuator Thorax with Passive Rotation The design of a flapping-wing MAV based on passive rotation is shown in Fig. 16.3b. Here a central power actuator is responsible for controlling flapping while pronation and supination are passive. The power actuator thus acts to deliver a maximal amount of power to the wing stroke in an analogous fashion to the indirect flight muscles of the Dipteran thorax. Passive rotation is achieved with a flexure hinge at the base of the wing at the interface between the wing and the transmission. A custom-fabrication method (described in Sect. 16.3) enables the designer to create flexures with arbitrary geometries. Incorporated into the wing hinge flexure are joint stops which limit the rotational motion. Therefore, if adequate inertial and aerodynamic loads are experienced by the wing during flapping, the wing

222

will rotate to a pre-determined angle of attack for each half-stroke. The statics and dynamics of passive rotation are equally important. Using a pseudo-rigid-body model of the wing hinge flexure, it is simple to estimate the effective torsional stiffness of the wing hinge. Thus for an expected loading we can estimate the maximum rotation angle during flapping. Furthermore, the geometry of the flexure defines the limits of rotation by the joint stops. In order to achieve quasi-static rotation, it is important to also consider the dynamics of the rotational DOF. We design the first rotational resonance to be significantly higher than the flapping resonance by tuning the materials and geometry of the wing and flexure hinge. In this way, the baseline trajectory is mechanically hard coded into the structure and flapping and rotation can be accomplished simultaneously with a single actuator. Derivations from this baseline trajectory – to control body moments – will be accomplished with smaller actuators which subtly alter the transmission of the thorax in a similar manner as Diptera [13].

16.3 Fabrication Using Smart Composite Manufacturing Because of the scale of the components, we require a ‘meso’-scale manufacturing method. ‘Meso,’ in this use, refers to scales in between two heavily investi regimes: ‘macro’-scale (traditional machining) and MEMS. More traditional large-scale machining processes are inappropriate for a robotic insect for two fundamental reasons. First, the required resolution, on the order of 1 μm, would be difficult to achieve with standard machining tools. Second, as the components become smaller, the ratio of surface area to volume increases, and thus surface forces such as friction begin to dominate the dynamics of motion. This latter point implies that more traditional mechanisms for coupling rotations (e.g., sleeve or ball bearings) would exhibit increased inefficiency at the scale of insect joints. Alternatively, researchers have created articulated robotic structures using surface [23] and bulk micromachining [11] MEMS processes. However, MEMS devices are limited in terms of material choice, geometry, and actuation. Furthermore, MEMS process steps typically

R.S. Fearing and R.J. wood

involve cost-prohibitive infrastructure and significant time delays. For all of these reasons, we require a novel way to construct the articulated and actuated mechanical/electromechanical/aeromechanical components of a robotic insect. This must be fast, inexpensive, repeatable, and result in structures that can have dramatic deformations (> ± 60◦ ), long fatigue life (>10 M cycles), and high power density. The solution is a multi-step micromachining and lamination process called smart composite microstructures (SCM [21]). In this process, select materials (metals, ceramics, polymers, or fiber-reinforced composites) are first laser micromachined into arbitrary 2D geometries, as shown in Fig. 16.4. This typically involves thin sheets of material and a UV (frequencytripled Nd:YVO4 , 355 nm) or green (frequencydoubled Nd:YAG, 532 nm) computer-controlled laser. Once each material is cut, they are properly aligned and cured to form the laminate. Alignment can use a number of techniques including folding, fluid surface tension, and mechanical aligners using vision and registration marks (similar to mask aligners). A common constituent lamina material is carbon fiber prepreg. This is a composite material of ultra high modulus fibers with a catalyzed but uncured polymeric matrix. During curing (at elevated temperatures using a modified vacuum bagging process), the matrix flows and makes bonds with the various layers in the laminate. Using this process, we can create laminates with a well-defined spatially distributed compliance (e.g., flexures) which can be folded into any 3D shape with any number of degrees of freedom. Moreover, by including electroactive materials into the laminate – PZT for example – we can create actuators and actuated structures. This process is the basis for all of the mechanical and aeromechanical components of our robotic flies. It is enabling for the demanding application of a robotic fly and is potentially impactful for a number of other meso-scale robotics applications.

16.4 Actuation and Power Providing adequate power for lift and thrust is critical for hovering devices. For Dipteran insects, power of 70–100 W kg−1 of body mass is estimated [16], with power plant power density of approximately 200

16 Challenges for 100 Milligram Flapping Flight Fig. 16.4 (a) Smart composite manufacturing process using laser micromachining and lamination. Gaps are cut in carbon fiber which define flexure joint locations, then an intermediate layer of polyimide is used as the flexure layer, and finally a second layer of carbon fiber is laminated to form the complete structure. (b) Example parts for UCB MFI thorax

223

(a)

(b) W kg−1 . Traditional electromagnetic motors, ubiquitous in larger robotic systems, are inappropriate for actuation of a robotic insect. This is due to the scaling arguments made in Sect. 16.3. Additionally, there are practical limitations to the current density in smaller electromagnetic windings which exacerbate the poor scaling of such motors. (The limits of current available actuators are discussed in Chap. 14 and 21.) Furthermore, a simple periodic (or even harmonic) motion is required to drive the wings. Therefore, any rotary motion would require a kinematic linkage to convert rotations to the flapping motions. Clamped-free piezoelectric bending bimorph actuators were chosen for the MFI based on the desired metrics of high power density, high bandwidth, high efficiency, and ease of construction [22]. These actuators are constructed using the same method as with the articulated mechanisms; only here some of the constituent layers are piezoelectric. Figure 16.5 shows a cross section of the actuator. Each of the layers is laser

micromachined, aligned, and cured in a similar manner as the transmission. Although initial lift results using piezoelectric actuators were promising [3], verifying actual power output from the actuators is critical for identifying V1 PZT

V2

PZT

CFRP

S-glass

Fig. 16.5 Composite piezoelectric bimorph actuator cross section and example actuators at multiple scales

224

R.S. Fearing and R.J. wood

Fig. 16.6 Dynamometer for testing piezoelectric power output at resonance [16]

x1

x2

ksp

DUT

Driver (b)

(a) possible transmission losses or aerodynamic inefficiencies. Extrapolation of actuator performance from DC measurements [22] predicted higher power than was actually observed. Hence, a miniature dynamometer system was developed [16] to measure real actuator output power for simulated damping loads at resonance. Figure 16.6a, b shows the setup for the dynamometer, which uses precision optical sensors to measure the displacement of the drive actuator and

Energy Output per Cycle (µJ)

the device-under-test (DUT). Force is measured from the extension of the connecting spring, and equivalent damping is set by adjusting the driver phase. As seen in Figure 16.7, with a 10.1 mg actuator, energy density of 1.89 J kg−1 was obtained. With operating frequency for the MFI of 275 Hz, power density of 470 W kg−1 is obtained, with internal mechanical losses of approximately 10%.

16.5 Airfoils

20 16 Improved Actuators

12 8 Standard Actuators

4 0

Sensors

0

40

80

120

160

200

240

DUT Motion Amplitude (µm)

Fig. 16.7 Direct measurements of actuator output energy for various simulated loads at resonance

Another crucial component of a robotic fly is the airfoils. Insects wings exhibit a huge diversity in shape, size, venation pattern, and compliance. (Wing and aerodynamic issues are considered further in Chaps. 11, 12, and 14.) It is currently unknown how these morphological features affect flight: Are some of the features of insect wings due to bio-material limitations or are they instead an indicator of beneficial performance? Due to the complexity of this question, here current airfoils are designed to match key features of appropriately sized Diptera (aspect ratio, second moment of area, length, etc.) while remaining as rigid and lightweight as possible. To achieve this, carbon fiber ‘veins’ are laser micromachined and aligned to a

16 Challenges for 100 Milligram Flapping Flight

225

Fig. 16.8 Passive wing hinge with joint stop [19]

thin film polymer membrane (1.5 μm thick polyester) and cured. The outline of the wing shape is then cut with a final micromachining step which results in the wings shown in Fig. 16.8. These airfoils weigh less than 600–700 μg.

16.6 Results The SCM process and piezoelectric actuators have enabled lightweight thorax designs with both active and passive wing rotation. Active control of wing rotation is possible, but is very sensitive to near exact matching of each half of the thorax structure. Passive wing rotation, while still requiring precise tuning of wing hinge stiffness and rotational inertia properties, is more tolerant of manufacturing process variation.

16.6.1 Dynamic Challenges for Active Control of Flap and Rotation One of the motivations for active control of wing rotation is the potential to achieve enhanced rotational lift effects at the end of wing strokes [8]. Figure 16.9a shows several candidate wing rotation profiles, including a simple sinusoidal profile and higher harmonics in rotation, to generate a faster wing rotation at the end of each half-stroke. Using a dynamic model of the thorax and wing [14], the required actuator forces can be predicted, as shown in Fig. 16.9b. Interestingly, all the trajectories generate approximately the same lift (within 10%), but the trajectories with the faster rotation require five times greater actuation forces, which exceeds the capabilities of available actuators. This result indicates that a passive rotation, which approxi-

mates a sinusoidal rotation, may provide adequate lift forces with minimal power.

16.6.2 MFI Benchtop Lift Test A benchtop, one-wing version of the MFI was tested using over-sized actuators [14]. Through careful tuning of the amplitude, phase, and frequency of the two actuators (four parameters), an operating point with decent wing rotation was found as shown in Fig. 16.10. Tuning is quite critical, and due to driving at the resonant frequency, controllability is reduced. At 275 Hz, with flap angle ± 35◦ and rotation ± 45◦ , a net lift force of 1400 μN was measured using a precision scale. It is interesting to note that the small wing stroke, large wing rotation, high wing-beat frequency used are more bee-like than fly-like [1]. In addition, the high frequency allows better power density from the actuators, and short wing stroke reduces strain on the four bar joints.

16.6.3 Flapping-Wing MAV with Passive Rotation An alternative design simplifies the thoracic mechanics and uses passive rotation to achieve the required flapping and rotation trajectories. This entails similar components as the active rotation version, but uses only a single actuator and eliminates the differential mechanism. Once the four mechanical and aeromechanical components of the fly (actuator, transmission, airfoils, and airframe) are complete, they are integrated to form the structure in Fig. 16.1b. The first metric of interest is the trajectory that active flapping with

226 400

60

3

40 20

0

3

2

−20 −40 −60

(a)

Actuator Voltage (V)

1 Rotation angles (°)

Fig. 16.9 Comparison between desired wing rotations and required actuator voltage. Quick rotations, such as Case 1, require unachievable actuator power, suggesting only slow rotations may be achievable. Case 2 is a pure sinusoid trajectory and Case 3 is a sinusoidal drive voltage. Slow rotations could be achieved by passive aerodynamic and compliant mechanisms [14]

R.S. Fearing and R.J. wood

0

1

2

3

4

Time (ms)

5

200 0 −200

(b)

1

−600 −800 −1000

6

2

−400

0

1

2

3

4

5

6

Time (ms)

Fig. 16.10 (a) Benchtop testing of UC Berkeley MFI [14] with wing beat at 275 Hz. (b) Measured wing trajectory in stroke plane

(a)

(b) passive rotation can create. This is evaluated by simply driving the wings open loop at the flapping resonant frequency (approximately 110 Hz) and observing the wing motion with a high-speed camera. It was observed that the trajectory is nearly identical to

Diptera in hover (see Fig. 16.11a). The second metric of interest is the thrust produced. This was evaluated by fixing the structure to a custom single-axis force transducer and yielded an average thrust-to-weight of approximately 2:1.

16 Challenges for 100 Milligram Flapping Flight

227

Fig. 16.11 Takeoff of Harvard microrobotic fly

16.6.4 Benchtop Takeoff with Passive Rotation The final metric for this initial fly is a demonstration of takeoff. The fly was fixed to guide wires which restrict the motion of the fly to purely vertical, the other five body degrees of freedom were constrained. The wings were again driven open loop and the fly ascended the guide wire as shown in Fig. 16.11b. This shows the ability to produce insect-like wing motion with an integrated insect-size robot and that these wing motions produce lift forces of similar magnitude as a similarly sized fly. However, this does not show onboard power, integrated sensors, or automatic control and therefore there are numerous open research questions which need to be addressed to meet the goal of an autonomous robotic insect.

16.7 Conclusion The two main challenges remaining before free-flying robot flies can be created are flight control and compact power sources. For control, flight stabilization has been shown in simulation [7], and MEMS sensors (body attitude and rate) of the appropriate mass and power are close to off the shelf. While small devices are inherently highly maneuverable due to high angular accelerations (and hence potentially unstable), recent work described in Chaps. 17 points to high damping during turns which may simplify some control issues. Conventional computer vision systems are still too

computationally intensive and slow to use on an insectsize flying robot; however, bio-inspired navigation techniques such as optical flow sensing as described in Chaps. 3, 5, and 6 are low mass and can provide crucial flight control information, such as obstacle avoidance. Power sources currently are the biggest obstacle to 100 mg free flight. The required power source at the 50 mg size is still about an order of magnitude smaller than commercially available practice. The power required for free flight is estimated in Fig. 16.12. For a 100 mg flyer, 10 mW of wing power would provide 100W kg−1 of body mass. Considering thorax losses, and assuming efficient charge recovery [4] from the piezoelectric actuator(s), 27 mW of battery power should be sufficient, which corresponds to a reasonable battery power density of about 600 W kg−1 which can be obtained with current LiPoly battery technology (albeit in a 1 g battery rather than the 50 mg battery desired here). Several key challenges for flapping flight at the 0.1 gram size scale have been met. In particular, thorax kinematics have been designed which can drive wings at high frequency. A new fabrication process, smart composite microstructures (SCM), has enabled lightweight, high-strength, dynamic mechanisms with dozens of joints which can operate at hundreds of Hz, yet weigh only tens of milligrams. These structures have low losses, less than 10%. A low-inertia high stiffness wing has been shown to generate high lift forces. The SCM process has also enabled high-power density piezoelectric actuators, which have demonstrated sufficient power density for lift off of a tethered robot fly.

228

R.S. Fearing and R.J. wood

Fig. 16.12 Estimated power budget for free flight of microrobotic fly

We expect that free flight of fly-sized robots should be realizable in the next few years. Acknowledgments The authors acknowledge the key work of collaborators S. Avadhanula and E. Steltz on thorax and actuator design and characterization. Portions of this work were supported by NSF IIS-0412541. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation (NSF).

References 1. Altshuluer, D., Dickson, W., Vance, J., Roberts, S., Dickinson, M.: Short-amplitude high-frequency wing strokes determine the aerodynamics of honeybee flight. Proceedings of the National Academy of Sciences (USA) 102, 18, 213–18, 218 (2005) 2. Avadhanula, S., Wood, R., Campolo, D., Fearing, R.: Dynamically tuned design of the MFI thorax. IEEE International Conference on Robotics and Automation. Washington, DC (2002) 3. Avadhanula, S., Wood, R.J., Steltz, E., Yan, J., Fearing, R.S.: Lift force improvements for the micromechanical flying insect. IEEE/RSJ International Conference on Intelligent Robots and Systems, 2007 IROS 2007 (Oct. 28–30, 2003) 4. Campolo, D., Sitti, M., Fearing, R.: Efficient charge recovery method for driving piezoelectric actuators in low power applications. IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control 50, 237–244 (Mar. 2003) 5. Combes, S., Daniel, T.: Flexural stiffness in insect wings I. Scaling and the influence of wing venation. Journal of Experimental Biology 206 (17), 2979–2987 (2003) 6. Combes, S., Daniel, T.: Flexural stiffness in insect wings II. Spacial distribution and dynamic wing bending. Journal of Experimental Biology 206 (17), 2989–2997 (2003) 7. Deng, X., Schenato, L., Sastry, S.: Model identification and attitude control for a micromechanical flying insect including thorax and sensor models. IEEE Int. Conf. on Robotics and Automation. Taipei, Taiwan (2003)

8. Dickinson, M., Lehmann, F.O., Sane, S.: Wing rotation and the aerodynamic basis of insect flight. Science 284, 1954– 1960 (1999) 9. Dickinson, M., Tu, M.: The function of dipteran flight muscle. Comparative Biochemistry and Physiology vol. 116A, pp. 223–238 (1997) 10. Dudley, R.: The Biomechanics of Insect Flight: Form, Function and Evolution. Princeton University Press (1999) 11. Ebefors, T., Mattsson, J., Kälvesten, E., Stemme, G.: A walking silicon micro-robot. The 10th Int. Conf. on SolidState Sensors and Actuators (Transducers ’99), pp. 1202– 1205. Sendai, Japan (1999) 12. Ennos, A.: The inertial cause of wing rotation in Diptera. Journal of Experimental Biology 140, 161–169 (1988) 13. Miyan, J., Ewing, A.: How Diptera move their wings: A reexamination of the wing base articulation and muscle systems concerned with flight. Philosophical Transactions of the Royal Society of London B311, 271–302 (1985) 14. Steltz, E., Avadhanula, S., Fearing, R.: High lift force with 275 hz wing beat in MFI. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems. IROS 2007. pp. 3987–3992 (October 29 2007–November 2 2007) 15. Steltz, E., Avadhanula, S., Wood, R., Fearing, R.: Characterization of the micromechanical flying insect by optical position sensing. IEEE International Conference on Robotics and Automation. Barcelona, Spain (2005) 16. Steltz, E., Fearing, R.: Dynamometer power output measurements of piezoelectric actuators. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems. IROS 2007. pp. 3980– 3986 (October 29 2007–November 2 2007) 17. Sunada, S., Ellington, C.: A new method for explaining the generation of aerodynamic forces in flapping flight. Mathematical Methods in the Applied Sciences 24, 1377–1386 (2001) 18. Wang, Z., Birch, J., Dickinson, M.: Unsteady forces and flows in low reynolds number hovering flight: two-dimensional computations vs robotic wing experiments. Journal of Experimental Biology 207, 449–460 (2004) 19. Wood, R.: Design, fabrication, and analysis of a 3dof, 3 cm flapping-wing MAV. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, 2007. IROS 2007. pp. 1576–1581 (October 29 2007-November 2 2007)

16 Challenges for 100 Milligram Flapping Flight 20. Wood, R., Avadhanula, S., Menon, M., Fearing, R.: Microrobotics using composite materials: The micromechanical flying insect thorax. IEEE Int. Conf. on Robotics and Automation. Taipei, Taiwan (2003) 21. Wood, R., Avadhanula, S., Sahai, R., Steltz, E., Fearing, R.: Microrobot design using fiber reinforced composites. Journal of Mech. Design 130 (5) (2008)

229 22. Wood, R., Steltz, E., Fearing, R.: Optimal energy density piezoelectric bending actuators. Journal of Sensors and Actuators A: Physical 119 (2), 476–488 (2005) 23. Yeh, R., Kruglick, E., Pister, K.: Surface-micromachined components for articulated microrobots. Journal of Microelectrical Mechanical Systems 5 (1), 10–17 (1996)

Chapter 17

The Limits of Turning Control in Flying Insects Fritz-Olaf Lehmann

Abstract This chapter provides insights into the turning flight of insects, considering this specific behavior from experimental and numerical perspectives. The presented analyses emphasize the need for a comparative approach to flight control that links an insect’s maneuverability with the physical properties of its body, the properties and response delays of the sensory organs, and the precision with which the muscular system controls the movements of the wings. In particular, the chapter focuses on the trade-off between lift production and the requirement to produce lateral forces during turning flight. Such information will be useful not only for a better understanding of the evolution and mechanics of insect flight but also for engineers who aim to improve the performance of the future generation of biomimetic micro-air vehicles.

17.1 Introduction Insects display an impressive diversity of flight techniques such as effective gliding, powerful ascending flight, low-speed maneuvering, hovering, and sudden flight turns [1–9]. Flies, in particular, are capable of extraordinary aerial behaviors aided by an array of unique sensory specializations including neural superposition eyes and gyroscopic halteres [10–12]. Using such elaborate sensory input, flies steer and maneuver

F.-O. Lehmann () Institute of Neurobiology, University of Ulm, Albert-Einstein-Allee 11, 89081 Ulm e-mail: [email protected]

by changing many aspects of wing kinematics including angle of attack, the amplitude and frequency of wing stroke, and the timing and speed of wing rotation [13–18]. The limits of these kinematic alterations, and thus the constraints on the aerial maneuverability of a fly, depend on several key factors including the maximum power output of the flight muscles, mechanical constraints of the thoracic exoskeleton, and the ability of the underlying neuromuscular system to precisely control wing movements [19–22]. What we experience as flight behavior of a flying insect reflects the output of a complex feedback cascade that consists of receptors to collect sensory information, the central nervous system and thoracic ganglion to process this information and to produce locomotor commands, and the mechano-muscular system to drive the wings (see also Chaps. 1– 7). Changes in flight behavior result from changes in any of these components such as alterations in the sensory input or in the bilateral symmetry of flight force production caused by wing damage. Due to the predominant role of the compound eyes for navigation, orientation, and flight stability, the vast majority of investigations on flight control in the past have been done on the question of how changes in the visual input change aerial behavior and thus on the question of how the nervous system processes visual information [23–31]. Flight control and maneuverability of flies have been studied by a variety of methods under both free and tethered flight conditions. Although tethered flight reflects only a small fraction of an insect’s total behavioral repertoire in free flight, this technique has proven useful in elucidating the organization of the flight control system in flies such as optomotor behaviors in response to rotating and expanding visual flow fields and object orientation behaviors [32–39]. A major

D. Floreano et al. (eds.), Flying Insects and Robots, DOI 10.1007/978-3-540-89393-6_17, © Springer-Verlag Berlin Heidelberg 2009

231

232

F.-O. Lehmann

A

B

Lift /yaw R

Side slip/ pitch

CoP

b1 b3 b2

CoG Thrust / roll III1

III2-4 1.0 mm

I2

I1 0.3 mm

Fig. 17.1 Free flight body posture and flight muscles in the 1.2 mg fruit fly Drosophila. (A) Rotational axes and forces, (B) subset of the 17 pairs of flight control muscles involved in amplitude control for yaw turning. Neural activation of the basalare

b1–b3 causes an increase in stroke amplitude. Muscle spikes in the first (I1–I2) and third (III1–III4) pterale typically result in a decrease in amplitude. R, wing length; CoP, aerodynamic center of pressure on wing; CoG, the fly’s center of gravity

disadvantage of tethered studies, however, is the lack of adequate feedback from sensory organs such as the halteres and the antennae. Moreover, tethered flight studies in flight simulators require elaborate computational algorithms for feedback simulation, in order to model the physical behavior of the insect body similar to what would occur in free flight. Thus, despite the difficulty in reconstructing body and wing motion in freely flying animals and in assessing sensory stimuli, free flight measurements are crucial because they capture the behavior of an animal in a more natural context and under natural closed-loop feedback conditions between the fly’s sensory and motor system. Besides the control for adjusting translational forces such as thrust and body lift, yaw turning during maneuvering flight has attracted considerable interest, because it determines flight heading and is thus of augmented ecological relevance for foraging behavior and search strategies in an insect. This chapter attempts to summarize some of the most important factors for yaw turning control in an insect, such as the time course of yaw torque production and thus the temporal changes in wing motion, the constraints on sensory feedback, and the physics of turning. The chapter especially highlights the significance of the ratio between the body mass moment of inertia and the frictional damping between the body structures and the surrounding air. Experimental results and numerical predictions will further demonstrate some of the most important trade-offs in flight control and also show how the maximum locomotor capacity constrains stability and maneu-

verability at elevated muscle mechanical power output during flight of the small fruit fly Drosophila melanogaster (Fig. 17.1).

17.2 Free Flight Behavior and Yaw Turning In many insects, including the fruit fly, straight flight in a stationary visual environment is interspersed by sudden flight turns termed flight saccades. Flight saccades are maneuvers in which the fruit fly quickly changes flight heading between 90◦ and 120◦ within 15–25 wing strokes (75–125 ms, Figs. 17.2 and 17.3) [3, 5, 7]. Due to their high dynamics, these maneuvers differ from other forms of turning flight such as continuous, smooth turning behavior with angular velocities well below 1000◦ s−1 . There is an ongoing debate on the exact angular velocity profile during saccadic turning, because this profile critically depends on at least three factors: the time course of yaw torque production, the moments of body inertia, and the frictional damping on body and wings [40]. Although the maximum angular velocity of approximately 1600◦ s−1 within a saccade is independent of the forward speed in fruit flies, flight saccades are supposedly not uniform maneuvers in terms of a fixed motor action pattern. Instead, measurements have shown that the total turning angle within a flight saccade varies between 120◦ and 150◦ , potentially depending on the visual input the animals have experienced prior to the turn [40].

17 The Limits of Turning Control in Flying Insects

A

B

Stop

Fv

Panorama Ft

D

Fh Fl Start

CoR

40 mm

Fg

rp Flight path

D

C 28 Lateral force (F l)

21

30

Lift (F v)

14

25

7 0 Thrust (F h)

33

Force (μN)

Total flight force (µN) Flight forces (µN)

Fig. 17.2 Flight behavior and forces in freely flying fruit flies. (A) Saccades (white dots) of a 2.4 s flight path within a cylindrical, 170 mm high, free flight arena surrounded by a random-dot visual panorama. Sample time = 8 ms. (B) Forces during flight, (C) time traces of forces acting on the animal (upper traces) and total flight force production (lower trace). Gray dots indicate the times at which saccades occur in (A). (D) Total force and ratio between force components within flight recordings that fell within the top 10% maximum of total force. Fv , vertical force (lift); Fl , lateral force (centripetal); Fh , horizontal force (thrust); Ft , total force; Fg , gravitational force. D, body drag; rp , flight path radius; CoR, center of radius

233

Maximum locomotor capacity Total flight force (F t)

20

Fl

15 10

Fv

26

Fh

5

19

0

12

Body weight

5

0

0.4

0.8

1.2

1.6

2.0

2.4

Ascending order F t

Flight time (s)

A potential threat on an animal during fast yaw turning at elevated forward velocity is the occurrence of centrifugal forces that cause side-slip motion [41]. Drifting during turning is a serious problem when the insect must quickly change its flight course, for example, in response to an approaching obstacle. Besides the centering response, obstacle avoidance

behavior is of great ecological significance because it allows an insect to safely cruise through dense vegetation and also to escape from predators [42, 43]. Thus, to avoid sideslipping and to stay on track within the saccade, insects must compensate for centrifugal forces by producing centripetal forces. Many insects achieve this behavior by performing bank (roll) turns,

90–120°

A 0 ms

B

20 ms

40 ms

Amplitude envelope

Downstroke

– ΦA –ΦP

ΦS

ΦS

60 ms

Fig. 17.3 Free flight saccade visualized by high-speed video. (A) Images (top view) show body orientation and wing position at ventral (0–60 ms) and dorsal (80–120 ms) stroke reversals. (B) Mean stroke amplitude (Φ S ) supports body weight. Changes

100 ms

120 ms

Upstroke

+ ΦP –ΦA +ΦP

ωdt

80 ms

+ ΦP

+ ΦP

ωdt

–ΦP

in stroke amplitude on each body side result from passive components (Φ P ) due to body rotation and active components (Φ A ) due to changes in the activity of flight control muscles. Cycling period of a stroke cycle is ~5 ms (200 Hz)

234

in which the mean flight force vector tilts sideways, toward the inner side of the flight curve [44]. In fruit flies, side-slipping movements are rare, and even at their maximum forward velocity of 1.22 ms−1 the animals are apparently able to completely compensate for centrifugal forces while turning [5]. In terms of total force balance, lateral force production is of particular significance, because in the fruit fly the production of centripetal forces consumes up to 70% of the locomotor reserves [5, 19, 22].

17.3 Forces and Moments During Turning Flight The physical parameters predominantly determining changes in forward-, upward-, and side-slip velocities of an insect are the frictional damping coefficient on body and wings and body inertia [3, 40]. Turning rate, by contrast, is determined by the frictional damping coefficient and the mass moment of inertia of the body. The former measure determines air friction during body motion, where a higher frictional damping coefficient results in a lower peak turning velocity at constant torque production. Mass moment of inertia, in turn, determines how quickly the animal may alter its angular velocity around the three vertical and horizontal body axes: yaw, pitch, and roll (Fig. 17.1A). Elevated inertia, moment of inertia, and frictional damping potentially favor stable flight because these factors reduce angular and translational accelerations (inertia) and also maximum angular and translational velocities (friction) in an insect. A major benefit of passive frictional damping is that it reduces the computational load needed to process sensory feedback signals by the nervous system and decreases the required precision for wing control (see Sect. 17.4.2.3). The benefit of frictional damping for flight stabilization is also shown in birds [45]. For example, wing-based damping in wing amplitude asymmetry-driven turning is an important mechanism for roll dynamics during aerodynamic reorientation, because the roll damping coefficient in cockatiels is two to six times greater than the coefficient typical of airplane flight dynamics [45]. However, high moment of inertia and frictional damping also limit flight agility [1−9]. Flight behavior is thus a compromise between the need to stabilize the ani-

F.-O. Lehmann

mal body and the need to allow quick maneuvers, for example, in order to escape from predation or to avoid collisions with nearby objects in the environment.

17.3.1 Modeling Friction and Moment of Inertia In general, an insect’s instantaneous angular velocity during yaw turning ω at a given time t results from the three major components: the torque T produced around the vertical body axis, the moment of inertia I given by the body mass distribution of the animal, and the frictional damping coefficient C of body and wings [3, 40]. This relationship thus becomes T(t) = I ω(t) ˙ + Cω(t).

(17.1)

A rough estimate of the moment of inertia may be derived when assuming that the shape of the insect body can be approximated by a long thin cylinder with a given length l that rotates around its vertical axis at 50% length (Fig. 17.1A). Inserting body length and body mass mb , the moment of inertia of an insect may be easily derived from the simple equation

I=

mb l2 . 12

(17.2)

Both the mass moment of inertia and the frictional damping coefficient CB of such a cylinder are relatively small; in the fruit fly the two measures yield only 0.52 pNms2 and 0.52 pNms, respectively. By contrast, the wings’ contribution to friction during yaw rotation is more complicated because this measure depends on the complex aerodynamics of the flapping wings. In general terms, frictional damping due to flapping wings depends on the differences in profile drag between the up- and downstroke and the left and right wing. Drag, in turn, depends on stroke frequency, amplitude, the ratio between up- and downstroke, wing length, mean drag coefficient, and also on the location of the center of pressure on each wing (Fig. 17.1A). The combined damping coefficient for an insect is the sum of wing-based damping and body damping CB , and we can write this sum as

17 The Limits of Turning Control in Flying Insects

C=

¯ Dif rˆCP (R + 0.5 W) D + CB , ω¯

235

(17.3)

in which the numerator is equal to the frictional moment given by the product between mean differ¯ Dif and ence in profile drag between the two wings D the length of the moment arm, where R is wing length, W is thorax width, and rˆCP is the relative location to the wing’s center of pressure [40]. The dominator is mean angular velocity during turning. ¯ Dif is challenging, because this measure Deriving D depends on wing velocity in each half stroke. Mean wing velocity is proportional to the product between stroke frequency and stroke amplitude. The latter measure, however, changes during turning due to two processes: first, passively in the global coordinate system, as the result of body rotation, and second, actively as a result of the bilateral difference in wing beat Motion of visual pattern:

A

amplitude used for yaw torque production (Fig. 17.3). Experiments in both tethered animals flying in a reality flight simulator and in free flight show that during counterclockwise (clockwise) turning flies increase (decrease) its wing beat amplitude on the right body side and decrease (increase) the amplitude on the left side (Fig. 17.4A). The first step in deriving the damping coefficient for yaw turning is thus to derive wing velocity in each half stroke. According to Ellington, the mean wing velocity of a wing segment in each half stroke u¯ (r) at normalized distance r (0–1) from the wing base is proportional to the product between dimensionless wing velocity, the dimensionless wing velocity profile durˆ ˆt, stroke frequency, mean ing up- and downstroke dφ/d stroke amplitude Φ S , and wing length R [46]. Assuming that both the dimensionless velocity profile and stroke frequency remain constant during turning and

Upward

Downward

Left

Right

40° control range

Amplitude envelope

WKg–1

240

97

230

87

220

77

210

67

200

57

190 37

27

180 140

150

47

160

170 180 Stroke amplitude (deg)

Fig. 17.4 Stroke amplitude control by tethered fruit flies flying inside a virtual reality flight simulator. (A) The flies try to compensate for the motion of the visual panorama displayed inside the arena by actively modulating its stroke amplitude. (B) In vivo working range (kinematic envelope) of a tethered fruit fly responding to visual stimulation while actively controlling the azimuth velocity of the visual panorama using the bilateral difference between left and right stroke amplitude. At maximum

C 180

6 5

170 4 3

160

2 150 1 140 0.4

0 0.6

0.8

1.0

1.2

Variance of amplitude (degrees)

Stroke frequency (Hz)

B 250

Stroke amplitude (deg)

140–160° to support body weight

1.4

Flight force/ body weight

flight force production (arrow), the control of moments is compromised because the animal is restricted to a unique combination between stroke amplitude and frequency [50]. Hyperbolic lines represent mechanical power isolines of the indirect flight muscle [22]. (C) Mean stroke amplitude (closed circle) and temporal variance (open circle) of left and right stroke amplitudes plotted against relative flight force production. Gray areas indicate standard deviations

236

F.-O. Lehmann

only stroke amplitude changes due to passive Φ P rotation of the animal body and active steering Φ A , the velocity at the wing’s center of pressure and for a half stroke can be written as n ˆ ˆt| (ΦS + ΦP + ΦA ) rˆCP R. u¯ = 12 |dφ/d j

(17.4)

Due to the bilateral symmetry between left and right wing and the ratio between up- and downstroke, passive and active amplitudes have different signs in each half stroke (Fig. 17.3). If we consider a counterclockwise turn of the animal, Φ P is positive for the left (right) wing during the up (down)stroke and negative during the down (up)stroke (Fig. 17.3B). Since a counterclockwise turn requires an increase in stroke amplitude on the right body side and a decrease on the left side, the active component Φ A is negative (positive) in the left (right) wing in both half strokes. Another modification in the above equation is the effective frequency in each half stroke that depends on the time periods of up- and downstroke. Effective frequency is the ratio between stroke frequency and relative time spent during the up- and downstroke (ˆt = fraction of downstroke in a complete stroke cycle {0–1}). Thus, the parameter j in the above equation is equal to ˆt during the downstroke and amounts to 1 − ˆt during the upstroke. In a second step, we employ a numerical model that converts velocity estimates into mean wing pro¯ The most simple approach to derive D ¯ file drag D. for each wing is to use Ellington’s 2D quasi-steady aerodynamic model based on wing velocity squared and to lump unsteady aerodynamic effects, such as the development of a leading edge vortex, together into a mean drag coefficient [5, 46]. Despite ignoring 3D flow conditions, modified versions of this approach have successfully been used in insect flight research (for Drosophila see [18]). By combining the various velocity estimates with the quasi-steady model, we may derive the desired drag residual during turning from the differences in drag between left and right wing within the entire stroke cycle, whereby drag is positive during the downstroke and negative during the upstroke. We finally get the following expression: 

1 ¯ Dif = ρ C¯ D,Pro S ˆtu¯ 2L,D + 1 − ˆt u¯ 2R,U D 2 

− 1 − ˆt u¯ 2L,U − ˆtu¯ 2R,D ,

(17.5)

Table 17.1 Description of modeling parameters for the fruit fly [40] Symbol Description Value S N ω¯ ˆt P,U P,D A rˆCP R W P C¯ D,Pro S ˆ ˆt| |dφ/d

Wing stroke amplitude for weight support Wing stroke frequency Saccadic turning rate Relative duration of downstroke Passive wing stroke difference upstroke Passive wing stroke difference downstroke Mean active wing stroke difference Normalized distance to center of pressure Wing length Thorax width Density of air Mean profile drag coefficient of wing Surface area of one wing Dimensionless wing velocity

140◦ (2.44 rad) 218 Hz 1600◦ s−1 (27.9 rad s−1 ) 0.538 4.0◦ (0.069 rad) 3.1◦ (0.054 rad) 5◦ (0.087 rad) 0.7 2.47 × 10−3 m 1.0 × 10−3 m 1.2 kg m−3 1.46 2.0 × 10−6 m2 4.4

where u¯ L,D (¯uR,D ) and u¯ L,U (¯uR,U ) are wing velocities for the down- and upstroke of the left (right) wing, respectively (see Table 17.1 for abbreviations).

17.3.2 The Consequences of High Frictional Damping The ratio between moment of inertia and aerodynamic damping (I/C) is a critical measure for the control of body dynamics by the neuromuscular system of an insect. At high ratio when flight is dominated by the forces due to the distribution of body mass, turning acceleration is low but turning rate steadily increases when the animal produces rotational moments. At low ratio, by contrast, angular acceleration is high but angular velocity saturates with increasing turning rate [40]. These relationships have two consequences: Insects with an elevated moment of body inertia compared to frictional damping potentially benefit from an increase in flight-heading stability, but may lose control at high turning rates at which the transfer function

17 The Limits of Turning Control in Flying Insects

237

of the sensory structures becomes highly non-linear. Moreover, once the animal has initiated yaw moments, turning rate decreases only slowly when torque production vanishes. In this case, an insect must actively brake in order to terminate its turning by producing counter torque as shown in Fig. 17.5B. By contrast, small insects such as fruit flies that rely on relatively low I/C ratios are comparatively unstable around their rotational axes because their angu-

A

Straight flight

lar acceleration is high at low torque production [40]. These insects need to have a precise flight apparatus by allowing either a higher spatial resolution in wing amplitude control and other kinematic parameters, or a higher temporal resolution by decreasing the lack in response time of sensory feedback. High damping, however, allows an animal to passively terminate its turn without applying any kind of counter torque. In insects flying at relatively high damping coefficients,

Straight flight

Passive braking

Torque production

I/C = 0.01

B

Straight flight

Torque production

Passive rotation

Active braking

Straight flight

I/C = 1

C

D

Yaw torque (nNm)

Turning rate (103degs–1)

3

6 cycles 3

2

2

1

1

0

0

–1

–1

1

1 Torque 0

0 Counter torque

–1

–1 0

40

80

Time (ms) Fig. 17.5 Numerical modeling of yaw turning at two ratios between mass moment of inertia and frictional damping (I/C) on body and wings [40]. (A) Changes in stroke amplitude at small I/C ratio, corresponding to what is predicted for the fruit fly. (B) The fly must actively terminate yaw turning (counter torque) at high I/C ratio when frictional damping is considered on the body alone [3]. (C) Turning rates at low (black) and high (gray) frictional damping (cf. A and B) in response to a single 30 ms 0.8 pNms yaw torque pulse in the fruit fly. (D) Counter torque at

120

0

40

80

120

Time (ms) the end of the saccade (lower graph) produces negative turning rates when the model includes frictional damping on wings (gray, upper graph). Without frictional damping (black, upper graph), the production of counter torque reduces turning rate but does not totally terminate saccadic rotation. Values for the fruit fly are I = 0.52 pNms2 ; C = 0.52 and 54 pNms for body damping alone (in B) and combined frictional damping on body and wings (in A), respectively

238

F.-O. Lehmann

the production of high counter torque might even partly annihilate directional changes initiated at the beginning of the flight turn (Fig. 17.5D). When inserting the values of Table 17.1 for the fruit fly into the numerical model (Eq. 17.5), we obtain a total frictional damping coefficient C of 54 pNms due to the flapping wings that is approximately 100 times the value estimated from the body alone (0.52 pNms). Combining all estimates of moment of inertia, torque production, and damping in the fruit fly, the damping term in torque equation (17.1) is roughly 4–16 times larger than the inertia term. This finding is also confirmed by an elaborate 3D computational fluid dynamic (CFD) study on Drosophila yaw turning [47], suggesting a CFD torque profile similar to that shown in Fig. 17.6B (no active braking), rather than to a biphasic torque profile that includes active braking [3]. Consequently, in fruit flies friction plays a larger role for yaw turning behavior than moment of inertia.

17.4 Balancing Aerodynamic Forces During Maneuvering Flight Based on the assumption that flight in fruit flies is dominated by frictional forces rather than by inertia, the forces acting on the fly body during maneuvering flight may be derived from a simple numerical model for force balance. The development of such a model is beneficial for several reasons: First, it gives insight into how total aerodynamic forces are distributed among horizontal-, vertical-, and lateral components; second, it allows predictions of the relationship between flight path curvature and forward velocity; and third, it allows estimations of the maximum locomotor capacity in a freely flying animal. The following sections describe the various force components required for flight and show how force and velocity components may be determined in a freely cruising animal.

17.4.1 Forces and Velocities

Turning rate (103 degs–1)

1.6

120

1.2

90

0.8

60

0.4

30 0

0.0 0

Yaw torque (nNm)

B

40

80

120

Turning angle (deg)

A

160

3 8 Stroke cycles

2 1 0 Counter torque

–1 0

40

80

120

160

Time (ms) Fig. 17.6 Modeling of yaw torque during a saccadic turn of a freely flying fruit fly. (A) Torque development is calculated using a simplified velocity profile of turning rate (black) and turning angle (gray), measured in freely flying flies (cf. Fig. 17.2A) [5]. (B) At its natural frictional damping coefficient of 54 pNms on body and wings, the fruit fly does not depend on active braking (i.e., the production of counter torque) to terminate the saccade (gray line). Assuming damping on the fly body alone (1% of wing damping coefficient), however, active braking is required to terminate the saccade (black)

Flight velocity and thus flight direction of an insect depends on the ratio between vertical force (body lift), horizontal force (thrust), and lateral force (sideslip) multiplied by normalized friction and on the moments around these vectors: yaw (vertical axis), roll (horizontal axis), and pitch (lateral axis, Fig. 17.1A). While forces for translation are considered to play a major role in force balance of the fruit fly, the forces needed to generate torque are negligible. This assumption is fostered by torque measures obtained during optomotor behavior in tethered flies flying inside a virtual reality flight arena. Under these conditions, fruit flies typically vary yaw torque by not more than ±1.0 nNm (Fig. 17.5C,D) [48]. At a moment arm of 65% wing length from the wing base for the wing’s center of pressure (equal to the center of force, Fig. 17.1A) [49], this moment requires forces of not more than 0.50 μN or approximately 3% of the maximum flight force in this animal. Thus, the production of moments around the three body axes should require only minor modification in instantaneous force production by the flapping wings. For the above reason, we may model flight, assuming that total flight force Ft produced by both wings is equal to the vector sum between vertical-, horizontal-,

17 The Limits of Turning Control in Flying Insects

239

and lateral forces (Fv , Fh , and Fl , respectively), written as ' (17.6) FT = Fh2 + Fv2 + Fl2 .

The fly’s turning velocity ω is given by the product between horizontal velocity and path curvature,

To transform these forces into velocity estimates, we use a simplified approach assuming frictional damping at low Reynolds number. Reynolds number for body motion depends on forward velocity and thus varies between values close to zero at slow forward flight and, in the fruit fly, approximately 64 at maximum cruising speed (mean body width = 0.8 mm, kinematic viscosity of air = 15 × 10−6 m2 s−1 ). Although these values suggest a conventional ‘force–velocitysquared’ relationship, measurements on tethered flies show that flight force production is also linearly correlated with wing velocity (non-squared) given by the product between stroke amplitude and frequency (Reynolds number = 120–170) [18]. We may thus derive a reasonable approximation of forces acting on the fly body by using Stoke’s law and normalized friction [40]. Consequently, to estimate maximum flight velocities at a given flight path curvature of the insect, we replace thrust in Eq. (17.6) by body drag given as the product between normalized friction on body and wings and forward speed and lift by the sum of gravitational force and drag on the fly body when moving in the vertical. Lateral force, needed to keep the animal on track during yaw turning, is equal to centrifugal force because sideslip is negligibly small in fruit flies [40]. The latter force equals the product between the path radius rp , horizontal velocity, and body mass mb . The three forces are then

The most critical measure in these equations is normalized friction on body and wings during forward flight, because this parameter determines maximum horizontal and vertical flight velocities. In contrast to yaw turning, estimations of normalized friction for body translation are susceptible to major errors for two reasons: First, friction on the body depends on body posture which changes with flight speed as shown by David [50] and, second, the orientation of the wings continuously changes within the stroke cycle with respect to the oncoming air. It is thus advantageous to calculate normalized friction from Eq. (17.7) using an estimate of maximum forward velocity of the insect derived from behavioral experiments and an estimate for maximum thrust derived from maximum locomotor capacity. In case of the fruit fly, reconstructions of the flight path in freely flying individuals revealed a maximum horizontal velocity of approximately 1.22 m s−1 at level flight. By contrast, estimations of maximum locomotor capacity of the fruit fly are more challenging. There are at least two ways to derive this measure: First, from load lifting experiments in which freely flying animals are scored on their ability to lift up small weights as shown by Marden [51], and second, from direct force measurements in tethered flies [18, 22]. In load lifting experiments, locomotor reserve and thus maximum thrust is equal to the load that the animal is able to lift up, while in tethered flight experiments maximum locomotor capacity is equal to maximum force production when the animal is stimulated under visual open-loop optomotor conditions in a flight simulator with a vertically oscillating stripe grating. For the fruit fly, both approaches yield similar estimates for maximum thrust of approximately 4.9 μN and normalized friction in Drosophila thus amounts to approximately 4.0 μNm−1 s (Eq. 17.7).

Fl = mb u2h rp−1 ,

Fh = kuh ,

and

Fv = kuv + mb g, (17.7–17.9)

respectively, where g is the gravitational constant. If we now replace the force terms in Eq. (17.6) by the expressions in Eqs. (17.7–17.9), maximum horizontal and vertical flight velocities (uh and uv , respectively) at various flight conditions can be derived from the following two equations: 1 uh = mb and

%  

1 ' 4 4 2 2 2 2 2 2 rp k − 4rp mb Fv − Ft − rp k , 2 (17.10) '

uv =

FT2 − ku2h − m2b u4h rp−2 − mb g k

ω = uh rp−1 .

(17.12)

17.4.2 Trade-Offs Between Locomotor Capacity and Control High aerial maneuverability of a flying insect may be useful in a large variety of behavioral contexts

.

(17.11)

240

including predator avoidance, prey catching, mating success, and male–male competition. A well-known example of predator avoidance is, for example, the evasive flight reaction of noctuid moths when they detect the ultrasound of predating bats [52]. Stability and maneuverability are two sides of the same coin, and the system that allows high stability in an insect also controls and constrains maneuverability. The ability of an insect to provide aerodynamic forces in excess of its body weight thereby appears to be a key factor for high maneuverability. Studies, for example, on butterfly take-off behavior show that a critical measure for high aerodynamic performance is the ratio between flight muscle and body mass [53]. In other insect species, such as dragonflies, this ratio also depends on age. Young dragonflies are typically poor flyers but gain muscle mechanical power output during adult growth. Maximum power reserves for both flight force production and steering performance are exhibited at maturity. At this stage, dragonflies defend a territory, and aerial competition determines their mating success [54]. In the fruit fly, these trade-offs between power output and flight control can be predicted by a force balance model and verified by experiments under free and tethered flight conditions. In the following sections we thus focus on the relationships between muscle performance, mechanical constraints of the thorax, and flight control in the fruit fly, i.e., (1) the trade-off between lift, thrust, and lateral force production during turning flight and its consequences for flight velocity; (2) the collapse of steering envelope at maximum locomotor performance; and (3) the significance in precision, with which the neuromuscular system is able to control bilateral stroke amplitudes during yaw turning. Altogether, these issues highlight the problems and limits of maneuvering flight in the small fruit fly and also show how our numerical model (Sect. 17.4.1) may explain the various behaviors we observe in flying flies.

17.4.2.1 The Trade-Off Between Lift, Thrust, and Lateral Forces In the previous section we learned about the relationships between forward, upward, and turning velocity and how these parameters depend on total force production. Free flight experiments in fruit flies show that the lateral force needed to keep the animal on track

F.-O. Lehmann

during a flight saccade clearly outscores thrust and lift and almost reaches 28 μN (Fig. 17.2C). Thus, shorttime total flight force production even reaches 32 μN, which is approximately three times the body weight of the animal (~1.2 mg) and above the value typically measured under tethered flight conditions. By contrast, thrust only contributes moderately to total force (900◦ s−1 , Fig. 17.2A). In response to this visual stimulation, fruit flies typically minimize the retinal slip on their compound eyes

17 The Limits of Turning Control in Flying Insects

241

A

B 13.1 μN

16.3 μN

500 21.0 μN 250 32.4 μN

0 0.0

0.5

1.0

1.5

Horizontal velocity

2.0

(ms–1)

1.3

Flight velocity (ms–1)

Minimum flight path radius (mm)

750

Zero climbing velocity

0.8

uh

0.3

ul

uv

–0.2 –0.7 –1.2 0

50 100 150 200 Flight path radius (mm)

Fig. 17.7 Numerical modeling of force balance in freely flying fruit flies. (A) Minimum flight path radius at a given forward velocity and level flight shown for four estimates of total flight force. Maximum flight forces are 16.3, 21.0, and 32.4 μN in tethered flight, load lifting free flight, and free flight under optomotor stimulation, respectively. (B) Alteration in maximum vertical

climbing velocity (uv ) at a mean forward cruising speed (uh ) of 0.6 ms−1 and assuming 16 μN maximum flight force. Gray area indicates path radii at which the fly loses flight altitude while turning. Lateral speed (ul ) is equal to zero because of side-slip compensation

similar to what has been observed in tethered flies [16, 31, 32, 34, 35]. Consequently, in the attempt to match forward velocity to translational velocity and turning rate to angular velocity of the rotating visual environment, the animals continuously move in concentric circles around the arena center [5]. Under these conditions, the flies’ vertical speed approaches zero (constant altitude) while forward velocity and turning rate typically vary out-of-phase (Fig. 17.8). The numerical model in Eq. (17.6) predicts this behavior because at maximum locomotor capacity and constant flight altitude, any increase in lateral force production should lead to a corresponding decrease in thrust.

Moreover, since an increase in total flight force requires an increase in mechanical power output of the asynchronous flight musculature, transgenic flies with reduced muscle mechanical power output exhibit larger flight path radii during turning than wild-type animals flying at similar forward speed. We noticed this behavior in a myosin light chain mutant (MLC2) and several fly lines in which the phosphorylation capacity of the muscle protein flightin (fln) had been modified by point mutation (F.-O. Lehmann, unpublished data) [55]. Flightin is a multiply phosphorylated myosin-binding protein found specifically in indirect flight muscles (IFM) of Drosophila. When flown in a flight simulator and scored on maximum locomotor capacity, both strains show reductions in stroke frequency and partly also in stroke amplitude, whereas muscle and aerodynamic efficiency are similar among the two transgenic strains and wild type files.

Peak saccade velocity 2.50

0.4 1.25 0.2 0

Turning velocity (103 degs–1)

Horizontal velocity (ms–1)

0.6

17.4.2.2 Collapse of Steering Envelope at Maximum Locomotor Performance

0 1.0

1.1

1.2 1.3 Flight time (s)

1.4

1.5

Fig. 17.8 Trade-off between horizontal velocity (gray, left scale) and turning velocity (black, right scale) in a freely flying fruit fly at elevated flight force production. Horizontal velocity and turning rate co-vary during saccadic turning (gray areas) presumably due to the production of elevated centripetal forces

Since propulsion and control reside in the same locomotor system, flight control in insects is constrained by the mechanical limits of the thoracic exoskeleton that generates wing motion. The relationship between propulsion and control is of fundamental consequence because it predicts a complete loss in control at maximum locomotor force production. In general,

242

locomotor reserves of an insect function as power reserves to boost horizontal or vertical flight velocities but also allow the insect to modulate wing kinematics. Since many insects control lift and yaw moments by changing stroke amplitude (not dragonflies that apparently more often use changes in angle of attack and wing phasing for steering) [1, 8, 56–60], flying with amplitudes near the mechanical limits should impair stability and maneuverability [18, 61, 62]. In the fruit fly, the collapse in kinematic envelope may be quantified under tethered flight conditions in a closed-loop virtual reality flight simulator, in which the ability of the animal is scored to modulate stroke frequency and stroke amplitudes at different flight forces. In these experiments, the flies actively stabilize the azimuth velocity of a visual object (black bar) displayed in the panorama using the bilateral difference in stroke amplitude between both wings. While steering toward the visual target, the flies modulate mean stroke amplitude on both body sides in response to the up- and down motion of a superimposed, openloop background pattern [22]. At maximum flight force production, the temporal deviation of stroke amplitude and frequency approaches zero, indicating that the animal is restricted to a unique combination of mean wing velocity (the product between amplitude and frequency) and mean lift coefficient (Fig. 17.4B,C) [62]. Ignoring the potential contribution of other kinematic parameters, this collapse of the kinematic envelope during peak force production should greatly attenuate maneuverability and stability of animals in free flight. A possible mechanism that helps small insects to remain stable around their roll and yaw axes at elevated force production is the wings’ high-profile drag. As already outlined in the previous sections, even without any active control, frictional damping on the flapping wings is high enough to terminate yaw turning within 3–5 stroke cycles. Consequently, high frictional damping in the fruit fly helps to ensure stable flight conditions in cases in which force control by the neuromuscular system fails due to the mechanical limits of the thoracic box for wing motion.

17.4.2.3 Significance of Muscle Precision and Response Time of Sensory Feedback Another parameter that limits stability and maneuverability is the muscular precision of the flight appara-

F.-O. Lehmann

tus, including the temporal delay of sensory feedback. In Sect. 17.3.1 we discussed the relationship between moments of inertia, frictional damping on body and wings, turning velocity, and yaw torque production in a freely maneuvering insect. In terms of free flight stability and turning behavior, however, it is of interest to derive angular velocity for turning from Eq. (17.1). A time-variant form for yaw turning velocity at time t is (   I I C+ . (17.13) ω(t) = T(t) + ω(t − 1) dt dt Since torque production is proportional to the product of the bilateral difference instroke amplitude between both wings and an experimentally derived conversion factor (fruit fly: 2.9 × 10–10 Nm deg−1 ) [61], we may link the ability of an insect to keep track during turning and to precisely steer toward a visual object to its temporal changes in stroke amplitude. In free flight, changes in stroke amplitude are due to the interplay between the mechano-sensory system (halteres, antennae, and campaniform sensilla), the visual system (compound eyes and ocelli), and the muscular system. In contrast to insects with synchronous power muscles, such as dragonflies, in flies, power and control reside in two different muscle systems: the asynchronous indirect flight muscles (IFM) and the synchronous direct flight control muscles. IFM provide the power to overcome inertia and drag during wing flapping and fill up most of the thorax. By contrast, reconfiguration of the wing hinge for flight control lies in the function and interplay of 17 tiny flight control muscles. Flight control muscles typically produce positive work but also function as active springs that absorb mechanical muscle power produced by the IFM [63]. There is multiple electrophysiological evidence that three groups of control muscles play a key role in stroke amplitude control of flies: the basalare muscles b1–b3 and the muscles of the first (I1, I2) and third axillare (III1–4, Fig. 17.1B) [13–17, 64]. The flight control muscles in flies receive input from two major sensory organs: the gyroscopic halteres and the compound eyes. The halteres are condensed hindwings, driven by own power and flight control muscles [12]. Halteres beat in anti-phase with the wings and sense changes in Coriolis forces when the animal body rotates [10–12, 65]. They project on control muscle motoneurons via fast electrical synapses [66]. In fruit flies, halteres encode angular velocities

17 The Limits of Turning Control in Flying Insects

243

TMax = IωV /tV + CωV ,

(17.14)

in which ωV is the limit of angular speed allowing the fly to determine its angular rotation, and the ratio ωV /tV is the maximum angular acceleration between the upper limit of the angular speed and the sensormotor reaction time tV of the fly. Consequently, the term TMax indicates the maximum torque allowed for heading stabilization during active flight control within the limits of the sensory apparatus. Considering Eq. (17.14) from an evolutionary perspective, we may predict the following phenomenon: Insects exhibiting small delays in sensory information processing and large damping coefficients on wings and body reduce their need to develop a muscular system with high accuracy for wing motion, and thus high accuracy for yaw torque control. By contrast, a flight system with large response delays at small frictional damping requires a very precise muscular apparatus to avoid instabilities during flight. Understanding these trade-offs is of great relevance for the design of biomimetic micro-air vehicles that need to gain stability due to the function of both their electrical control circuitries and the actuators that drive wing flapping. Surprisingly, in fruit flies flying at their natural frictional damping coefficient, vision-mediated flight requires very high precision of wing amplitude con-

25 Amplitude difference (deg)

of up to at least 700◦ s−1 [67] during turning and provide fast feedback within at least a single wing stroke of approximately 5 ms [68]. By contrast, the vision system in insects suffers from a long delay scattered around 30 ms. The latter value was estimated from behavioral studies on male–female chases in houseflies, Musca [69]. Most of this delay appears to be due to the time-to-peak response of the photo-transduction process, i.e., 12 and 41 ms for the dark- and lightadapted state of the housefly’s compound eye, respectively [70]. The values reported for fruit flies are similar to those of houseflies and range from 20 to 50 ms bump latency [71]. To incorporate the properties of the sensory feedback into our analytical framework, we may modify Eq. (17.1) by inserting the response time and the upper threshold for detecting rotational body movements by the sensory organs [5, 40]. Replacing the various terms, we obtain a time-invariant equation:

Thresholds for motion detection

20 15 10 5 0 10

100

1000

10000

Damping coefficient (pNms) Fig. 17.9 Precision of steering control in tethered flying fruit flies. Data show the absolute difference between left- and rightwing stroke amplitude of flies (i.e., proportional to yaw torque), produced in order to actively stabilize yaw heading toward a black stripe displayed inside a flight simulator. The shaded area indicates the upper limits of the visual system that allow visual control of the stripe according to Eq. (17.14) (50 and 100% response thresholds of the insect’s elementary motion detector, EMD). The dotted line indicates crossing of the regression line with the x-axis. Natural damping coefficient of the fruit fly amounts to 54 pNms. Means ± S.E. N= 47 flies. See also [40]

trol for flight-heading stability within a range from 0.25◦ to 0.85◦ for each wing (Eq. (17.14), Fig. 17.9). Behavioral tests on tethered flies flying under visionmediated closed-loop conditions in a flight simulator, however, show that the neuromuscular system is not able to control stroke amplitude below a threshold of 1–2◦ [40]. Consequently, the tethered animals fail to safely control yaw moments based on vision alone, suggesting additional feedback coming from the halteres in freely flying animals. However, a fruit fly exhibiting a 5 ms response time of the visual system (delay of a single stroke cycle) might keep instantaneous angular velocity of the body below the threshold of the visual system even at its natural damping coefficient, because the required changes in stroke amplitude for flight stabilization would range between 0.6◦ and 1.9◦ and would thus be within the scope of visually mediated yaw control. Nevertheless, despite the recent progress in understanding the feedback control cascade in flies, the exact contribution of each sensory system for force control in a freely cruising animal still needs to be determined.

244

17.5 Synopsis Heading toward the construction of robotic biomimetic micro-air vehicles based on flapping wing design is challenging (see other chapters in this book), and a detailed and integrative view on flight control in insects is thus of great interest. In this chapter, we focused on the interplay between the physical forces and the neuromuscular control cascade during yaw turning in a small fly. Our analysis shows that aerial behavior results from the combined properties of the flight feedback cascade, including the physiological limits of the nervous structures, the constraints on flight muscle function, and the physics of body dynamics during flight. Consequently, if we experimentally break the connection between the various functional hierarchies for flight control in an insect, we potentially face the risk that the system changes its locomotor state and our analyses are restricted to descriptions of functionally isolated sub-components of the flight apparatus. For example, if we ignore frictional damping during turning flight in Drosophila, we tempt to conclude that flight stability resides in a fast and very precise system for sensory information processing and wing control. By contrast, including passive stabilization of the insect body due to high frictional damping, the computational load on the central nervous system may decrease as well as the required speed of sensory information processing and the precision of wing control. The ultimate challenge in understanding flight control and the limits of maneuverability, however, lies in the large variety of insect species and thus locomotor designs. A major goal for understanding both the biology of flight and the design of biomimetic micro-air vehicles must thus be to derive a comprehensive view on flapping flight that includes the various forms of senso-motor control. Eventually, this approach seems to be beneficial not only to evaluate the various forms of flight in insects but also to comprehend the physics and neuromuscular function of wing control in other flying animals, such as birds and bats.

References 1. Alexander, D.E. : Wind tunnel studies of turns by flying dragonflies. The Journal of Experimental Biology 122, 81–98 (1986)

F.-O. Lehmann 2. Ennos, A.R.: The kinematics and aerodynamics of the free flight of some Diptera. The Journal of Experimental Biology 142, 49–85 (1989) 3. Fry, S.N., Sayaman, R., Dickinson, M.H.: The aerodynamics of free-flight maneuvers in Drosophila. Science 300, 495–498 (2003) 4. Marden, J.H., Wolf, M.R., Weber, K.E.: Aerial performance of Drosophila melanogaster from populations selected for upwind flight ability. The Journal of Experimental Biology 200, 2747–2755 (1997) 5. Mronz, M., Lehmann, F.-O.: The free flight response of Drosophila to motion of the visual environment. The Journal of Experimental Biology 211, 2026–2045 (2008) 6. Rüppell, G.: Kinematic analysis of symmetrical flight manoeuvres of odonata. The Journal of Experimental Biology 144, 13–42 (1989) 7. Wagner, H.: Flight performance and visual control of flight of the free-flying housefly (Musca domesticaL) II Pursuit of targets. Philosophical Transactions of the Royal Society of London. Series B 312, 553–579 (1986) 8. Wang, H., Zeng, L., Liu, H., Chunyong, Y.: Measuring wing kinematics, flight trajectory and body attitude during forward flight and turning maneuvers in dragonflies. The Journal of Experimental Biology 206, 745–757 (2003) 9. Zbikowski, R.: Red admiral agility. Nature 420, 615–618 (2002) 10. Nalbach, G.: The halteres of the blowfly Calliphora I kinematics and dynamics. Journal Comparative Physiology A 173, 293–300 (1993) 11. Nalbach, G.: Extremely non-orthogonal axes in a sense organ for rotation: Behavioral analysis of the dipteran haltere system. Neuroscience 61, 149–163 (1994) 12. Pringle, J.W.S.: The gyroscopic mechanism of the halteres of Diptera. Philosophical Transactions of the Royal Society of London. Series B 233, 347–384 (1948) 13. Balint, C.N., Dickinson, M.H.: Neuromuscular control of aerodynamic forces and moments in the blowfly, Calliphora vivina. The Journal of Experimental Biology 207, 3813–3838 (2004) 14. Dickinson, M.H., Lehmann, F.-O., Götz, K.G.: The active control of wing rotation by Drosophila. The Journal of Experimental Biology 182, 173–189 (1993) 15. Dickinson, M.H., Lehmann, F.-O., Sane, S.: Wing rotation and the aerodynamic basis of insect flight. Science 284, 1954–1960 (1999) 16. Götz, K.G., Hengstenberg, B., Biesinger, R.: Optomotor control of wing beat and body posture in Drosophila. Biol Cybernetics 35, 101–112 (1979) 17. Heide, G.: Flugsteuerung durch nicht-fibrilläre Flugmuskeln bei der Schmeißfliege Calliphora. Z Vergl Physiologie 59, 456–460 (1968) 18. Lehmann, F.-O., Dickinson, M.H.: The control of wing kinematics and flight forces in fruit flies (Drosophila spp). The Journal of Experimental Biology 201, 385–401 (1998) 19. Casey, T.M., Ellington, C.P.: Energetics of insect flight. In: W. Wieser, E. Gnaiger (eds.) In Energy Transformations in Cells and Organisms, pp. 200–210. Stuttgart, Thieme (1989) 20. Harrison, J.F., Roberts, S.P.: Flight respiration and energetics. Annual Review of Physiology 62, 179–205 (2000)

17 The Limits of Turning Control in Flying Insects 21. Lehmann, F.-O.: The constraints of body size on aerodynamics and energetics in flying fruit flies: an integrative view. Zoology 105, 287–295 (2002) 22. Lehmann, F.-O., Dickinson, M.H.: The changes in power requirements and muscle efficiency during elevated force production in the fruit fly, Drosophila melanogaster. The Journal of Experimental Biology 200, 1133–1143 (1997) 23. Borst, A., Egelhaaf, M.: Principles of visual motion detection. Trends in Neurosciences 12, 297–306 (1989) 24. Dill, M., Wolf, R., Heisenberg, M.: Visual pattern recognition in Drosophila involves retinotopic matching. Nature 365, 751–753 (1993) 25. Egelhaaf, M., Borst, A.: Motion computation and visual orientation in flies. Comparative Biochemistry and Physiology 104A, 659–673 (1993) 26. Franceschini, N., Riehle, A., Nestour, A.: Directionally selective motion detection by insect neurons. In: Stavenga, Hardie (eds.) In Facets of vision, pp. 361–390. Berlin Heidelberg, Springer (1989) 27. Kirschfeld, K.: Automatic gain control in movement detection of the fly. Naturwissenschaften 76, 378–380 (1989) 28. Krapp, H.G., Hengstenberg, B., Hengstenberg, R.: Dentritic structure and receptive-field organization of optic flow processing interneurons in the fly. American Physiological Society. Journal of Neurophysiology 79 1902–1917 (1998) 29. O’Carroll, D.: Feature-detecting neurons in dragonflies. Nature 362 541–543 (1993) 30. Reichardt, W.: Evaluation of optical motion information by movement detectors. Journal of Comparative Physiology A 161, 533–547 (1987) 31. Tammero, L.F., Dickinson, M.H.: Spatial organization of visuomotor reflexes in Drosophila. The Journal of Experimental Biology 207, 113–122 (2004) 32. Blondeau, J., Heisenberg, M.: The three dimensional optomotor torque system of Drosophila melanogaster. Journal of Comparative Physiology A 145, 321–329 (1982) 33. Borst, A., Bahde, S.: Comparison between the movement detection systems underlying the optomotor and the landing response in the housefly. Biological Cybernetics 56, 217– 224 (1987) 34. Duistermars, B.J., Chow, D.M., Condro, M., Frye, M.A.: The spatial, temporal and contrast properties of expansion and rotation flight optomotor responses in Drosophila. The Journal of Experimental Biology 210, 3218–3227 (2007) 35. Egelhaaf, M.: Visual afferences to flight steering muscles controlling optomotor responses of the fly. Journal of Comparative Physiology A 165, 719–730 (1989) 36. Götz, K.G., Wandel, U.: Optomotor control of the force of flight in Drosophila and Musca II Covariance of lift and thrust in still air. Biological Cybernetics 51, 135–139 (1984) 37. Heide, G., Götz, K.G.: Optomotor control of course and altitude in Drosophila is achieved by at least three pairs of flight steering muscles. The Journal of Experimental Biology 199, 1711–1726 (1996) 38. Heisenberg, M., Wolf, R.: Reafferent control of optomotor yaw torque in Drosophila melanogaster. Journal of Comparative Physiology A 163, 373–388 (1988) 39. Kaiser, W., Liske, E.: Die optomotorischen Reaktionen von fixiert fliegenden Bienen bei Reizung mit Spektrallichtern. Journal of Comparative Physiology 80, 391–408 (1974)

245 40. Hesselberg, T., Lehmann, F.-O.: Turning behaviour depends on frictional damping in the fruit fly Drosophila. The Journal of Experimental Biology 210, 4319–4334 (2007) 41. Schilstra, C., van Hateren, J.H.: Blowfly flight and optic flow I. Thorax kinematics and flight dynamics. The Journal of Experimental Biology 202, 1481–1490 (1999) 42. Egelhaaf, M., Borst, A.: Is there a separate control system mediating a “centering response” in honeybees. Naturwissenschaften 79, 221–223 (1992) 43. Srinivasan, M.V., Lehrer, M., Kirchner, W.H., Zhang, S.W.: Range perception through apparent image speed in freely flying honey bees. Visual Neuroscience 6, 519–535 (1991) 44. Ennos, A.R.: The kinematics and aerodynamics of the free flight of some Diptera. The Journal of Experimental Biology 142, 49–85 (1989) 45. Hedrick, T.L., Usherwood, J.R., Biewener, A.A.: Low speed maneuvering flight of the rose-breasted cockatoo (Eolophus roseicapillus) II Inertial and aerodynamic reorientation. The Journal of Experimental Biology 210, 1912– 1924 (2007) 46. Ellington, C.P.: The aerodynamics of insect flight VI Lift and power requirements. Philosophical Transactions of the Royal Society of London. Series B 305, 145–181 (1984) 47. Ramamurti, R., Sandberg, W.C.: A computational investigation of the three-dimensional unsteady aerodynamics of Drosophila hovering and maneuvering. The Journal of Experimental Biology 210, 881–896 (2007) 48. Heisenberg, M., Wolf, R.: Vision in Drosophila. SpringerVerlag, Berlin (1984) 49. Ramamurti, R., Sandberg, W.C.: Computational study of 3-D flapping foil flows 39th Aerospace Sciences Meeting and Exhibit, 605 (2001) 50. David, C.T.: The relationship between body angle and flight speed in free flying Drosophila. Physiological Entomology 3, 191–195 (1978) 51. Marden, J.H.: Maximum lift production during take-off in flying animals. The Journal of Experimental Biology 130, 235–258 (1987) 52. Roeder, K.D., Treat, A.E.: The detection and evasion of bats by moths. Am Sci 49, 135–148 (1961) 53. Almbro, M., Kullberg, C.: Impaired escape flight ability in butterflies due to low flight muscle ratio prior to hibernation. The Journal of Experimental Biology 211, 24–28 (2008) 54. Marden, J.H., Fitzhugh, G.H., Wolf, M.R.: From molecules to mating success: Integrative biology of muscle maturation in a dragonfly. American Scientist 38, 528–544 (1998) 55. Barton, B., Ayer, G., Heymann, N., Maughan, D.W., Lehmann, F.-O., Vigoreaux, J.O.: Flight muscle properties and aerodynamic performance of Drosophila expressing a flightin gene. The Journal of Experimental Biology 208, 549–560 (2005) 56. Norberg, R.A.: Hovering flight of the dragonfly Aeshna juncea L. In: T.Y.-T. Wu, C.J. Brokaw, C. Brennen (eds.) Kinematics and Aerodynamics, vol. 2, pp. 763–781. NY, Plenum Press (1975) 57. Reavis, M.A., Luttges, M.W.: Aerodynamic forces produced by a dragonfly. AIAA Journal 88:0330, 1–13 (1988)

246 58. Wakeling, J.M., Ellington, C.P.: Dragonfly Flight II. Velocities, accelerations, and kinematics of flapping flight. The Journal of Experimental Biology 200, 557–582 (1997) 59. Usherwood, J.R., Lehmann, F.-O.: Phasing of dragonfly wings can improve aerodynamic efficiency by removing swirl. Journal of the Royal Society, Interface 5, 1303–1307 (2008) 60. Thomas, A.L.R, Taylor, G.K., Srygley, R.B., Nudds, R.L., Bomphrey, R.J.: Dragonfly flight: Free-flight and tethered flow visualizations reveal a diverse array of unsteady liftgenerating mechanisms, controlled primarily via angle of attack. The Journal of Experimental Biology 207, 4299– 4323 (2004) 61. Götz, K.G.: Bewegungssehen und Flugsteuerung bei der Fliege Drosophila. In: W. Nachtigall (ed.) BIONA-report 2 Fischer, Stuttgart (1983) 62. Lehmann, F.-O., Dickinson, M.H.: The production of elevated flight force compromises flight stability in the fruit fly Drosophila. The Journal of Experimental Biology 204, 627–635 (2001) 63. Tu, M.S., Dickinson, M.H.: Modulation of negative work output from a steering muscle of the blowfly Calliphora vicina. The Journal of Experimental Biology 192, 207–224 (1994) 64. Lehmann, F.-O., Götz, K.G.: Activation phase ensures kinematic efficacy in flight-steering muscles of Drosophila

F.-O. Lehmann

65.

66.

67.

68.

69.

70.

71.

melanogaster. Journal Comparative Physiology 179, 311– 322 (1996) Nalbach, G., Hengstenberg, R.: The halteres of the blowfly Calliphora II Three-dimensional organization of compensatory reactions to real and simulated rotations. Journal Comparative Physiology A 174, 695–708 (1994) Fayyazuddin, A., Dickinson, M.H.: Haltere afferents provide direct, electronic input to a steering motor neuron of the blowfly, Calliphora. Journal of Neuroscience 16, 5225– 5232 (1996) Sherman, A., Dickinson, M.H.: A comparison of visual and haltere-mediated equilibrium reflexes in the fruit fly Drosophila melanogaster. The Journal of Experimental Biology 206, 295–302 (2003) Hengstenberg, R., Sandeman, D.C.: Compensatory head roll in the blowfly Calliphora during flight. Proceedings of the Royal Society of London. Series B 227, 455–482 (1986) Land, M.F., Collett, T.S.: Chasing Behaviour of houseflies (Fannia canicularis). Journal of Comparative Physiology A 89, 331–357 (1974) Howard, J., Dubs, A., Payne, R.: The dynamics of phototransduction in insects: A comparative study. Journal of Comparative Physiology A 154, 707–718 (1984) Hardie, C.R., Raghu, P.: Visual transduction in Drosophila. Nature 413, 186–193 (2001)

Chapter 18

A Miniature Vehicle with Extended Aerial and Terrestrial Mobility Richard J. Bachmann, Ravi Vaidyanathan, Frank J. Boria, James Pluta, Josh Kiihne, Brian K. Taylor, Robert H. Bledsoe, Peter G. Ifju, and Roger D. Quinn

Abstract This chapter describes the design, fabrication, and field testing of a small robot (30.5 cm wingspan and 30.5 cm length) capable of motion in both aerial and terrestrial mediums. The micro-air– land vehicle (MALV) implements abstracted biological inspiration in both flying and walking mechanisms for locomotion and transition between modes of operation. The propeller-driven robot employs an undercambered, chord-wise compliant wing to achieve improved aerial stability over rigid-wing micro-air vehicles (MAVs) of similar size. Flight maneuverability is provided through elevator and rudder control. MALV lands and walks on the ground using an animalinspired passively compliant wheel-leg running gear that enables the robot to crawl and climb, including surmounting obstacles larger than its own height. Turning is accomplished through differential activation of wheel-legs. The vehicle successfully performs the transition from flight to walking and is able to transition from terrestrial to aerial locomotion by propeller thrust on a smooth horizontal surface or by walking off a vertical surface higher than 6 m. Fabricated of lightweight carbon fiber the ~100 g vehicle is capable of flying, landing, and crawling with a payload exceeding 20% its own mass. To our knowledge MALV is the first successful vehicle at this scale to be capable of both aerial and terrestrial locomotion in real-world terrains and smooth transitions between the two.

R.D. Quinn () Department of Mechanical and Aerospace Engineering at Case Western Reserve University, Cleveland, USA e-mail: [email protected] Video of the robot during field testing may be observed at: http://faculty.nps.edu/ravi/BioRobotics/Projects.htm.

18.1 Introduction Advances in fabrication, sensors, electronics, and power storage have made possible the development of a wide range of small robotic vehicles capable of either aerial or terrestrial locomotion. Furthermore, insights into animal locomotion principles and mechanisms have significantly improved the mobility and stability of these vehicles. For example, the utility and importance of bat-inspired passively compliant wings for fixed wing micro-air vehicles (MAVS) have been demonstrated for aircraft with wingspans as small as 10 cm [1]. Likewise, highly mobile ground vehicles using animal-inspired compliant legs have been constructed with body lengths as short as 9 cm that can run rapidly over obstacles in excess of their own height [2]. This chapter describes the design, fabrication, and testing of a novel small vehicle (dubbed the micro-air– land vehicle (MALV)) that is capable of both aerial and terrestrial locomotion. Robot morphology is inspired by neuromechanics in animal locomotion, integrating passive compliance in its wings, joints, and legs, such that it may fly, land, walk on the ground, climb over obstacles, and (in some circumstances) take to the air again all while transmitting sensor feedback. Experimental testing has demonstrated that the robot can be made rugged enough for field deployment and operation. To our knowledge, MALV is the only existing small vehicle capable of powered flight and crawling, climbing obstacles comparable to its height and transitioning between locomotion modes. In the longer term, the design architecture and locomotion mechanisms are expected to lead to a family of vehicles with multiple modes of locomotion that can be scaled to a range

D. Floreano et al. (eds.), Flying Insects and Robots, DOI 10.1007/978-3-540-89393-6_18, © Springer-Verlag Berlin Heidelberg 2009

247

248

of functions. Applications targeted include surveillance, reconnaissance, exploration, search/rescue, and remote inspection.

18.1.1 Overview and Design Approach In a biological organism, execution of a desired motion (e.g., locomotion) arises from the interaction of descending commands from the brain with the intrinsic properties of the lower levels of the sensorimotor system, including the mechanical properties of the body. Animal “neuromechanical” systems successfully reject a range of disturbances which could otherwise induce instability or deformation of planned trajectories [3]. The first response to minimize such effects, in particular for higher frequency disturbances such as maintaining posture over varying terrestrial substrates [4] and unexpected gusts in flight [1, 5], is provided by the mechanical properties (e.g., structures, muscles, and tendons) of the organism. In legged locomotion, for example, a fundamental role is played by compliance (i.e., springs and dampers) in joints and structures that store and release energy, reduce impact loads, and stabilize the body [4] in an intrinsic fashion and thus greatly simplify higher level control [6, 7]. Reproduction of the dynamic properties of muscle and the intrinsic response of the entire mechanical system [8] have been an impediment to the successful realization of animal-like robot mobility over a variety of substrates and through different mediums (e.g., air and land). It is these intrinsic properties of the musculoskeletal system which augment neural stabilization of the body of an organism. Although biological inspiration offers a wealth of promise for robot mobility, many constituent technologies are not at a state of maturity where they may be effectively implemented for small autonomous robots. Existing power, actuation, materials, and other robotic technology have not developed to the point where animal-like neuromechanics may be directly integrated into robotic systems. Given this challenge, the majority of biologically inspired legged and flying robots have been confined to laboratory or limited field demonstrations. A method to surmount this, known as abstracted biological inspiration [9], focuses principally on the delivery of critical performance characteristics to the engineering system. Abstracted biological inspiration

R.J. Bachmann et al.

attempts to extract salient biological principles and to implement them using available technologies. This approach founded the basis of the design methodology aimed at delivering capabilities of flight locomotion, crawling locomotion, and transition between each to MALV.

18.1.1.1 Organization of Chapter The remainder of this section describes past work in flying and crawling robots individually and some of the small body of research in robots with multimodal mobility. Section 18.2 delineates the biologically inspired structures for flight and walking utilized for MALV. Section 18.3 details the specific design process and fabrication of the vehicle, while Sect. 18.4 provides details on the performance characteristics of the robot based on exhaustive system testing in facsimile field operations. Section 18.5 enumerates the conclusions of the research and envisioned future work.

18.1.2 Micro-ground Vehicles Two major factors remain significant challenges to the deployment and field utility of terrestrial microrobots. First, the relative size of real-world obstacles (e.g., stairs, gravel, terrain fluctuations) makes movement a daunting task for small robots. For example, RHex [10], at approximately 50 cm in length, is the shortest existing ground robot to our knowledge that can climb standard stairs without jumping [11–13], flying, or using gripping mechanisms on its feet [14, 15]. Second, power source miniaturization has not kept pace with other critical technologies, such as actuation, sensing, and computation. A wide array of vehicles have been constructed that attest to the difficulty of designing field-deployable terrestrial mobile micro-robots. Khepera robots have a 5 cm wheelbase, onboard power, and an array of sensors [16]. Although they are widely used by group behavior researchers, their 1.4 cm diameter wheels restrict them to operation on very smooth, flat surfaces. Millibots [17] use tracks, but it is not clear that they offer significant advantages since it is difficult

18 A Miniature Vehicle with Extended Aerial and Terrestrial Mobility

to implement a modern track suspension at this small scale. A small hexapod has been developed by Fukui et al. [18] that runs in a tripod gait using piezoelectric actuators. However, small joint excursions also limit the vehicle to relatively flat surfaces. Birch et al. [19] developed a 7.5 cm long hexapod inspired by the cricket and actuated by McKibben artificial muscles. Though capable of waking using 2 bars of air pressure it has not yet carried its own power supply. Sprawlita [20] is a 16 cm long hexapod that uses a combination of servomotors and air cylinders. Although Sprawlita attains a top speed of 4.5 body lengths per second, which is fast compared to existing robots of similar size, a necessary operating air pressure of 6 bars makes it unlikely that the robot will become autonomous in its current form. Abstracted biological inspiration has spawned a group of highly mobile robots, called WhegsTM [21] and Mini-WhegsTM [22]. To our knowledge, MiniWhegsTM is the fastest small terrestrial vehicle that is also capable of surmounting large obstacles relative to its size. Using a single drive motor, the 9 cm long robot attains a speed of 10 body lengths per second and can easily run over 3.5 cm tall obstacles – higher than the top of its body. The more recently developed iSprawl [23] also uses single motor propulsion and benefits from abstracted biological principles. It has run even faster, 15 body lengths per second, although its obstacle climbing ability is more restricted because of the small excursions of its feet.

18.1.3 Micro-air Vehicles (MAVs) The majority of research to develop practical (nonrotary) winged MAVs can be categorized into three fundamental approaches. The first and most widely used is to configure the airframe as a lifting body or flying wing using conventional propeller-driven thrust in a manner similar to larger aircraft. In this approach, the emphasis is to increase the relative area of the lifting surface while decreasing drag, directly addressing the decrease in the aerodynamic efficiency and putting less emphasis on issues of stability and control. Research groups have used optimized rigid wings and accepted the need for stability augmentation systems or superior pilot skill to deal with intrinsically unsteady behav-

249

ior. Among the most successful examples of rigidwing MAVs designed with this approach is Aerovironment’s “Black Widow” [24], an electric 15 cm flying wing. Virtually every component on the aircraft is custom built, including a sophisticated gyro-assisted control system. Other successful examples of rigid-wing designs include the “Trochoid” [25] and the “Microstar MAV” [26]. Both of these also have gyro-assisted stabilization systems, without which these lifting bodies would be difficult to control. This approach differs significantly from natural flight: Birds and bats have welldefined wings and a fuselage, and we find no examples of lifting bodies or flying wings in animals that produce thrust and fly for extended periods rather than simply gliding. The second approach that is being explored for MAV design draws on direct biological inspiration through mimicry of insect- or bird-like beating wings [27–30] (Chaps. 11, 12, 13, 14, 15, 16). Flapping wings can produce both lift and thrust. Researchers have demonstrated flapping wing MAVs that can fly and even hover [31] (Chap. 14) using the “clap and fling” mechanism as described by Ellington [27]. However, these MAVs are susceptible to failure in even light winds and their payload capacity is very small. This approach remains attractive for future work, in particular for low-speed, low-wind applications such as inside buildings. In a third approach [32–37], the lifting surface is allowed to move and deform passively like animal wings, which leads to more favorable aerodynamic performance in a fluctuating low Reynolds number environment. These findings helped lead to a flexible wing concept, which has been applied by Ifju to successful MAVs over the past 8 years [1, 38–41]. Based upon this abstracted biologically inspired mechanism flight vehicles have been developed that utilize conventional propeller-driven thrust in combination with an adaptive-shape, compliant wing that responds to flight conditions and also develops a stable limit-cycle oscillation during flight.

18.1.4 Multi-mode Mobility While the aerial and terrestrial vehicles described above represent significant enhancements in their respective fields, their utility is limited by their

250

dependence on a single locomotion modality. At present, very few robots have been developed that are capable of multiple modes of locomotion, with the majority of work focusing on swimming/crawling robots. One example is Boxybot [42], which uses a vertically oriented tail and two horizontally oriented fins for aquatic propulsion. By reversing the orientation of one or both of the fins, turning moments or reverse thrust can be generated. Continuous rotation of the fins produces a sort of “pronking” terrestrial movement. A watertight version of RHex [43] has also been equipped with fin-like legs that allow it to swim under water. The neuromechanical design of a more recent amphibious robot is based upon salamanders and it can run on land and swim using the same central pattern generator [44]. To the knowledge of the authors, there are few published works with the stated goal of both aerial and terrestrial locomotion. The Entomopter [45] uses reciprocating chemical muscle [46] to produce flapping motion of its four wings. We are not aware of data on the vehicle’s terrestrial capabilities or performance results for either locomotion mode. The recently developed Microglider (Chap. 19) also locomotes both on the ground and in the air and implements a biologically inspired wingfolding mechanism. However, it hops into the air and then glides rather than being able to fly for extended periods as is the intended purpose of MALV and the Entomopter. Nature has repeatedly demonstrated the need for multiple modes of locomotion, especially for small animals such as insects. Pure terrestrial locomotion may be impractical at this scale simply because of the distances that must be traveled to search for food, mates, etc. However, mono-modal aerial locomotion is also undesirable because it is impossible to stay airborne indefinitely, a variety of conditions (winds, etc.) make it difficult to land at exactly the desired location, and walking is far more energy efficient than flying for traveling short distances. Utility for small robots often reflects the exact same problem domain as small animals; multiple modes of locomotion would represent a generational leap in their capability. Flight capacity could allow a vehicle to travel long distances and approach a general target area, while crawling locomotion would allow a range of additional possibilities (close inspection, surveillance, performance of tasks, etc.) unachievable by any vehicle existing today.

R.J. Bachmann et al.

18.2 Biologically Inspired Structures for Flying and Walking As stated earlier, abstracted biological inspiration focused on functionality of the MALV with technology presently available. Its mechanical design incorporated neuromechanical flight (deformable wing) and walking (compliant wheel-legs and axle joints) mechanisms, which were key to MALV’s locomotion capacity. The challenge was designing mechanisms for functionality in both modes while preserving as much mobility as possible in each individually.

18.2.1 Terrestrial Locomotion The implementation of biological locomotion principles holds considerable promise for terrestrial locomotion. Legged animals exist and thrive at a wide range of sizes and are capable of overcoming obstacles that are on the order of their own size. Animal legs behave as if they have passive spring-like compliant elements when they are perturbed [6]. Alexander describes three uses for springs in legged locomotion, including energy absorption [4]. Jindrich and Full demonstrated this in an experiment wherein a cockroach was suddenly perturbed, too quickly for its nervous system to react [7]. It was shown that the passive compliance in its legs stabilized its body. Similar stability benefits are achieved through compliant elements in the legs of the RHex robot [10], which preceded WhegsTM . Locomotion studies on cockroaches have elucidated several critical behaviors that endow the insect with its remarkable mobility [47]. During normal walking, the animal uses a tripod gait, where adjacent legs are 180◦ out of phase. The cockroach typically raises its front legs high in front of its body, allowing it to take smaller obstacles in stride, but when climbing larger obstacles, the animal moves adjacent legs into phase, thus increasing stability. These benefits have been captured in design through the WhegsTM concept (Fig. 18.1), which served as the basis for the terrestrial locomotion of MALV. WhegsTM has spawned a line of robots that implement abstracted biological inspiration (based on the cockroach) for advanced mobility. Compliance is implemented into WhegsTM legs in two ways: radially for

18 A Miniature Vehicle with Extended Aerial and Terrestrial Mobility

251

Fig. 18.1 The wheel-leg provides a compromise between the efficiency and ease of propulsion of a wheel and the terrain mobility of legs

shock absorption (one of the three uses of springs in legged locomotion cited by Alexander [4]) and torsionally for gait adaptation, leading to improved traction and stability [9]. Torsional compliance allows for a single motor to drive six three-spoke wheel-leg appendages in such a manner as to accomplish all of the locomotion principles discussed above. WhegsTM robots are also scalable, with successful robots being developed with body lengths ranging from 89 cm down to 9 cm. This concept has been extended to Mini-WhegsTM (Fig. 18.2) that offer a combination of speed, mobility, durability, autonomy, and payload for terrestrial micro-robots. Mini-WhegsTM robots are extremely fast (10 body lengths per second) in comparison to most other legged robots and can climb obstacles taller than the top of their bodies [22]. The wheel-leg appendage

1

Fig. 18.2 Photograph depicting the relative sizes of a MiniWhegsTM and a Blaberus giganteus cockroach. Scale is in centimeters (figure courtesy of Andrew Horchler)

results in a natural “high stepping” behavior, allowing the robot to surmount relatively large obstacles. These vehicles have tumbled down concrete stairs and been dropped from heights of over 10 body lengths, without damage. Mini-WhegsTM have also carried over twice their body weight in payload [22].

18.2.2 Compliant Wings for Aerial Locomotion A rigid leading edge, chord-wise compliant wing design is the basis for MALV’s aerial locomotion. The compliant wing inspired by flying animals has several advantages over similarly sized rigid-wing vehicles [1]. Delayed stall allows the vehicles to operate at slower speeds. Improved aerodynamic efficiency reduces the payload that must be dedicated to energy storage. Passive gust rejection significantly improves stability. It has been well established that the aerodynamic efficiency of conventional (smooth, rigid) airfoils is significantly compromised in the Reynolds number (Re) range between 104 and 106 . This Re range corresponds to the class of craft referred to as micro-air vehicles [48]. In fact, the ratio of coefficient of lift (CL ) to coefficient of drag (CD ) drops by nearly 2 orders of magnitude through this range. With smooth, rigid wings in this Re range, the laminar flow that prevails is easily separated, creating large separation bubbles, especially at higher angles of attack [49]. Flow separation leads to sudden increases in drag and loss of efficiency. The effects of the relationship discussed above are very clear in nature. Consider, for example, the behaviors of birds of various sizes. Birds with large

252

wingspan, with a fixed wing Re > 106 , tend to soar for prolonged periods of time. Medium-sized birds utilize a combination of flapping and gliding, while the smallest birds, with a fixed wing Re < 104 , flap continuously and rapidly to stay aloft. Other major obstacles exist for flight at this scale [1]. Earth’s atmosphere naturally exhibits turbulence with velocities on the same scale as the flight speed of MAVs. This can result in significant variations in airspeed from one wing to the other, which in turn leads to unwanted rolling and erratic flight. The small mass moments of inertia of these aircraft also adversely affect their stability and control characteristics; even minor rolling or pitching moments can result in rapid movements that are difficult to counteract. A rigid leading edge, chord-wise compliant wing addresses these issues for MALV’s aerial locomotion capabilities. Through the mechanism of passiveadaptive washout, a chord-wise compliant wing (first implemented in the University of Florida (UF) [1]) overcomes many of the difficulties associated with flight on the micro-air vehicle scale. Adaptive washout is a behavior of the wing that involves the shape of the wing passively changing to adapt to variations in airflow. For example, an airborne vehicle may encounter a turbulent headwind, such that the airspeed over only the right wing is suddenly increased. The compliant wing structure responds to the instantaneous lift generated by the gust to deform in a manner similar to Fig. 18.3. This is referred to as passive-adaptive washout and results in a reduction in the apparent angle of attack and a subsequent decrease in lifting efficiency, as compared to the non-deforming wing. However, because the air velocity over the deformed wing is higher, it continues to develop a nearly equivalent

Fig. 18.3 The chord-wise compliance of a flexible wing allows for passive-adaptive washout, increasing stability of the aircraft

R.J. Bachmann et al.

lifting force as the flat wing. Similarly, as the airflow over the wing stabilizes, the wing returns to its original shape. This behavior results in a vehicle that exhibits exceptionally smooth flight, even in gusty conditions; our own MALV flight tests have been conducted in the presence of winds that precluded the flight of larger (2+ m wingspan) rigid-wing aircraft. While quantification of this effect is difficult, feedback from highly skilled pilots has confirmed the efficacy of flexible wings in gusty environments.

18.3 MALV Design and Development 18.3.1 Methodology 18.3.1.1 Locomotion Mechanisms The function of MALV is to carry a sensor payload, fly a long distance, and then land and move on the ground for a short distance, all while relaying sensor (e.g., visual) information to a remote location. For flight, a flexible wing was first selected over rigid or flapping wings to provide the best combination of controllability, payload capacity, speed, and efficiency (for long-distance missions) in the critical size range. Next, a range of terrestrial locomotion mechanisms were assessed for integration onto a MAV. One possibility was to attach free-spinning wheels to the fuselage of a MAV and use its propeller to drive the vehicle on the ground and in the air. Our experiments demonstrated that such a vehicle can land and takeoff from smooth firm terrain. However, this device had extremely poor ground mobility on rugged terrain. The propeller had a strong tendency to collide with obstacles, thus severely restricting ground mobility. Ground locomotion also suffered due to the fact that the forward thrust of the propeller was, out of necessity, above the wheel axle, creating a torque that pitched the vehicle forward. Therefore, when the wheels contacted an obstacle, the vehicle would often pitch forward nose first rather than actually moving forward. A possible alternative to this involved directly powering wheels attached to the fuselage, yet the vehicle’s mobility would still be limited as it would not be able to climb obstacles even a small fraction of its own height. Legged locomotion mechanisms were judged to be too complicated, delicate,

18 A Miniature Vehicle with Extended Aerial and Terrestrial Mobility

and heavy at this time for use on a vehicle capable of both flight and crawling. We therefore chose to integrate flexible wing and wheel-leg (WhegsTM ) locomotion mechanisms to design MALV.

18.3.1.2 Multi-modal Mobility Trade-Offs A design analysis was performed to determine how to best integrate flight and ground mobility mechanisms. If a crawling robot was simply attached to the bottom of a MAV, the resulting vehicle would be too heavy to fly unless its wingspan were greatly increased. Therefore, a trade-off analysis was done to determine the most important parameters for ground locomotion and MAV locomotion. The successful MALV design preserves those parameters as much as possible. Less important design parameters were compromised to improve overall vehicle performance. In the case of a conflict where parameters important for flight were severely deleterious to ground mobility, a morphing mechanism was employed to resolve the problem. In the trade-off analysis flight was assumed to be the limiting condition because of energy demands and the larger payload enabled by crawling structures. In flight, legs increase drag and reduce controllability. Furthermore, their associated mass reduces payload and can alter flight stability. On the ground, wings, propellers, and tails limit payload and impede mobility in confined spaces. The fuselage of an aerial vehicle tends to be long to increase its stability, but on the ground a long chassis causes the vehicle to high center on obstacles. On the ground, more legs can increase a vehicle’s stability and mobility, but in the air they add drag and mass. These design inconsistencies broadly fell into two categories: mass and geometry. Wheel-legs were judged vital to the ground mobility of MALV. However, their implementation was reconsidered to improve the overall performance of the vehicle. Past wheel-leg robots in small sizes typically had four wheel-legs driven by one propulsion motor and its front wheel-legs are steered. The front wheel-legs are most important because they reach in front of the vehicle and on top of obstacles in the vehicle’s path to lift and pull the vehicle forward. WhegsTM are designed this way to model the front feet of cockroach, which lift high and in front of the animal to overcome obstacles [47]. To reduce mass and complexity, testing of the ground mobility of a MALV was executed with two

253

wheel-legs instead of four. The wheel-legs were placed in the front and to the side of the propeller. The rear of the fuselage dragged on the ground. We found that a MALV with this configuration could move forward over obstacles similar in height to a comparably sized Mini-WhegsTM robot. Additionally, the fuselage provided a tail-like action that prevented the robot from flipping onto its back, which happens when a purely terrestrial vehicle attempts to surmount obstacles very tall relative to its height. The drawback to this design is that MALV’s mobility in reverse on rugged terrain was poor because the fuselage impacts irregularities and impedes motion. However, the weight savings justified the two-wheel-leg design. Wheel-leg steering vs. differential steering was also contrasted in traded off design studies. Wheel-leg steering requires the wheel-legs to be placed further outboard on the wings so they do not strike the propeller when they are turned. Either design requires two motors. The differential steer design was chosen because no steering mechanism is required; the design is simpler and MALV can turn more sharply in this configuration. Efficient hybrid designs can reduce mass by integrating structural, sensor, actuator, and power components. In MALV, the fuselage of the aircraft is also the chassis of the ground vehicle. Its shape has been changed to meet design criteria for flight and ground systems. MALV uses the same cameras in flight and ground locomotion to transmit video to a remote base. The same motor could be used for both flight and ground mobility in a manner akin to insects using large muscles to drive their body–coxa leg joints and flap their wings [50, 51]. However, this idea was abandoned for the first-generation robot because the complexity overrode possible mass reduction. A transmission would be needed because the wheel-legs must turn much more slowly than the propeller. The propeller shaft and wheel-leg axles are perpendicular, which also increase transmission complexity. Furthermore, a clutch would be needed to switch from propeller to wheel-leg drive. For these reasons we chose to use different motors for flight and ground mobility. Wings provide a geometric inconsistency that cannot be compromised. Wings are clearly essential for flight, but are an impediment for ground locomotion especially when MALV is moving through narrow spaces. Birds and insects fold their wings when they are on the ground to eliminate impediments to motion.

254

R.J. Bachmann et al.

Fig. 18.4 User interface of the MAVLab design software

An insect-like wing-folding mechanism was therefore developed for MALV that mimics this action. It can stow its wings on its back when moving on the ground. 18.3.1.3 Design Summary Simply attaching wheel-legs to a MAV was inadequate to design an efficient MALV. As is described above and in more detail in the following, a range of parameters were optimized for trade-off in the vehicle design space, compromised where necessary, and implemented following a system’s approach focused on coupling between mechanisms for the design of each component. The resulting MALV achieves its goals of air and ground locomotion, but, because of necessary design trade-offs it is not yet as agile on the ground as Mini-WhegsTM and has less payload capacity and is less controllable than a flexible wing MAV.

18.3.2 MALV Design Implementation Based upon weight estimates and initial flight testing, the lift capacity of a 30 cm wingspan MAV was determined to be sufficient to carry the additional weight associated with components needed for terrestrial locomotion.1 Analysis of existing technology led to the

1 Initial performance specifications also called for a wingspan