Optomechatronics: Fusion of Optical and Mechatronic Engineering

1969_half 9/16/05 11:11 AM Page 1 Fusion of Optical and Mechatronic Engineering Mechanical Engineering Series Frank ...

Author: Hyungsuck Cho

212 downloads 1633 Views 15MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

1969_half 9/16/05 11:11 AM Page 1

Fusion of Optical and Mechatronic Engineering

Mechanical Engineering Series Frank Kreith & Roop Mahajan - Series Editors Published Titles Distributed Generation: The Power Paradigm for the New Millennium Anne-Marie Borbely & Jan F. Kreider Elastoplasticity Theor y Vlado A. Lubarda Energy Audit of Building Systems: An Engineering Approach Moncef Krarti Engineering Experimentation Euan Somerscales Entropy Generation Minimization Adrian Bejan Finite Element Method Using MATLAB, 2nd Edition Young W. Kwon & Hyochoong Bang Fluid Power Circuits and Controls: Fundamentals and Applications John S. Cundiff Fundamentals of Environmental Discharge Modeling Lorin R. Davis Heat Transfer in Single and Multiphase Systems Greg F. Naterer Introductor y Finite Element Method Chandrakant S. Desai & Tribikram Kundu Intelligent Transportation Systems: New Principles and Architectures Sumit Ghosh & Tony Lee Mathematical & Physical Modeling of Materials Processing Operations Olusegun Johnson Ilegbusi, Manabu Iguchi & Walter E. Wahnsiedler Mechanics of Composite Materials Autar K. Kaw Mechanics of Fatigue Vladimir V. Bolotin Mechanics of Solids and Shells: Theories and Approximation Gerald Wempner & Demosthenes Talaslidis Mechanism Design: Enumeration of Kinematic Structures According to Function Lung-Wen Tsai Multiphase Flow Handbook Clayton T. Crowe Nonlinear Analysis of Structures M. Sathyamoorthy Optomechatronics: Fusion of Optical and Mechatronic Engineering Hyungsuck Cho Practical Inverse Analysis in Engineering David M. Trujillo & Henry R. Busby Pressure Vessels: Design and Practice Somnath Chattopadhyay Principles of Solid Mechanics Rowland Richards, Jr. Thermodynamics for Engineers Kau-Fui Wong Vibration and Shock Handbook Clarence W. de Silva Viscoelastic Solids Roderic S. Lakes

1969_title 9/16/05 11:11 AM Page 1

Fusion of Optical and Mechatronic Engineering

Hyungsuck Cho

Boca Raton London New York

A CRC title, part of the Taylor & Francis imprint, a member of the Taylor & Francis Group, the academic division of T&F Informa plc.

1969_Discl.fm Page 1 Monday, September 26, 2005 11:04 AM

Published in 2006 by CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2006 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group No claim to original U.S. Government works Printed in the United States of America on acid-free paper 10 9 8 7 6 5 4 3 2 1 International Standard Book Number-10: 0-8493-1969-2 (Hardcover) International Standard Book Number-13: 978-0-8493-1969-3 (Hardcover) Library of Congress Card Number 2005050570 This book contains information obtained from authentic and highly regarded sources. Reprinted material is quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable efforts have been made to publish reliable data and information, but the author and the publisher cannot assume responsibility for the validity of all materials or for the consequences of their use. No part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC) 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe.

Library of Congress Cataloging-in-Publication Data Cho, Hyungsuck Optomechatronics / by Hyungsuck Cho p. cm. Includes bibliographic references and index. ISBN 0-8493-1969-2 (alk. paper) 1. Mechatronics. 2. Optical detectors. TJ163.12.C44 2005 670.42'7--dc22

2005050570

Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com Taylor & Francis Group is the Academic Division of Informa plc.

and the CRC Press Web site at http://www.crcpress.com

Author

Hyungsuck Cho gained his B.S. degree at Seoul National University, Korea in 1971, an M.S. degree at Northwestern University, Illinois in 1973, and a Ph.D. at the University of California at Berkeley, California in 1977. Following a term as Postdoctoral Fellow in the Department of Mechanical Engineering, University of California, Berkeley, he has joined the Korea Advanced Institute of Science and Technology (KAIST) in 1978. He was made a Humboldt Fellow in 1984-1985, won Best Paper Award at the International Symposium on Robotics and Manufacturing, USA in 1994, and the Thatcher Brothers Awards, Institute of Mechanical Engineers, UK in 1998. Since 1993, he has been an associate editor or served on the editorial boards of several international journals, including IEEE Transactions on Industrial Electronics, and has been guest editor for three issues, including IEEE Transactions IE Optomechatronics in 2005. Dr. Cho wrote the handbook Optomechatronic Systems: Technique and Application, has contributed chapters to 10 other books and has published 435 technical papers, primarily in international journals. He was the founding general chair for four international conferences and the general chair or co-chair for 10 others, including the SPIE Optomechatronic Systems Conference held in Boston in 2000 and 2001. His research interests are focused on optomechatronics, environment perception and recognition for mobile robots, optical vision-based perception, control, and recognition, and application of artificial intelligence/ machine intelligence. He has supervised 136 M.S. theses and 50 Ph.D. theses. For the achievements in his research work, he was made POSCO professor from 1995 to 2002.

Preface In recent years, optical technology has been increasingly incorporated into mechatronic technology, and vice versa. The consequence of the technology marriage has led to the evolution of most engineered products, machines, and systems towards high precision, downsizing, multifunctionalities and multicomponents embedded characteristics. This integrated engineering field is termed optomechatronic technology. The technology is the synergistic combination of optical, mechanical, electronic, and computer engineering, and therefore is multidisciplinary in nature, thus requiring the need to view this from somewhat different aspects and through an integrated approach. However, not much systematic effort for nurturing students and engineers has been made in the past by stressing the importance of the multitechnology integration. The goal of this book is for it to enable the reader to learn how the multiple technologies can be integrated to create new and added value and function for the engineering systems under consideration. To facilitate this objective, the material brings together the fundamentals and underlying concepts of this optomechatronic field into one text. The book therefore presents the basic elements of the engineering fields ingredient to optomechatronics, while putting emphasis on the integrated approach. It has several distinct features as a text which make it differ somewhat from most textbooks or monographs in that it attempts to provide the background, definition, and characteristics of optomechatronics as a newly defined, important field of engineering, an integrated view of various disciplines, view of systemoriented approach, and a combined view of macro– micro worlds, the combination of which links to the creative design and manufacture of a wide range of engineering products and systems. To this end a variety of practical system examples adopting optomechatronic principles are illustrated and analyzed with a view to identifying the nature of optomechatronic technology. The subject matter is therefore wide ranging and includes optics, machine vision, fundamental of mechatronics, feedback control, and some application aspects of micro-opto-electromechanical system (MOEMs). With the review of these fundamentals, the book shows how the elements of optical, mechanical, electronic, and microprocessors can be effectively put together to create the fundamental functionalities essential for the realization of optomechatronic technology. Emphasizing the interface between the relevant disciplines involving the integration, it derives a number of basic optomechatronic units. The book

then goes on in the final part to deal, from the integrated perspectives, with the details of practical optomechatronic systems composed of and operated by such basic components. The introduction presents some of the motivations and history of the optomechatronic technology by reviewing the technological evolution of optoelectronics and mechatronics. It then describes the definition and fundamental concept of the technology that are derivable from the nature of practical optomechatronic systems. Chapter 2 reviews the fundamentals of optics in some detail. It covers geometric optics and wave optics to provide the basis for the fusion of optics and mechatronics. Chapter 3 treats the overview of machine vision covering fundamentals of image acquisition, image processing, edge detection, and camera calibration. This technology domain is instrumental to generation of optomechatronic technology. Chapter 4 presents basic mechatronic elements such as sensor, signal conditioning, actuators and the fundamental concepts of feedback control. This chapter along with Chapter 2 outline the essential parts that make optomechatronics possible. Chapter 5 provides basic considerations for the integration of optical, mechanical, and electrical signals, and the concept of basic functional modules that can create optomechatronic integration and the interface for such integration. In Chapter 6, basic optomechatronic functional units that can be generated by integration are treated in detail. The units are very important to the design of optomechatronic devices and systems, since these produce a variety of functionalities such as actuation, sensing, autofocusing, acousticoptic modulation, scanning and switching visual feedback control. Chapter 7 represents a variety of practical systems of optomechatronic nature that obey the fundamental concept of the optomechatronic integration. Among them are laser printers, atomic force microscopes (AFM), optical storage disks, confocal microscopes, digital micromirror devices (DMD) and visual tracking systems. The main intended audiences of this book are the lower levels of graduate students, academic and industrial researchers. In the case of undergraduate students, it is recommended for the upper level since it covers a variety of disciplines, which, though fundamental, involve various different physical phenomena. On a professional level, this material will be of interest to engineering graduates and research/field engineers who function in interdisciplinary work environments in the fields of design and manufacturing of products, devices, and systems. Hyungsuck Cho

Acknowledgments I wish to express my sincere appreciation to all who have contributed to the development of this book. The assistance and patience of Acquiring Editor Cindy Renee Carelli, have been greatly appreciated during the writing phase. Her enthusiasm and encouragement have provided me with a great stimulus in the course of this book writing. In addition, I would like to thank Jessica Vakili, project coordinator, Fiona Woodman, project manager, and Tao Woolfe, project editor of Taylor and Francis Group, LLC, for ensuring that all manuscripts were ready for production. I am also indebted to my former Ph.D students, Drs. Won Sik Park, Min Young Kim and Young Jun Roh for their helpful discussions. Special thanks go to Hyun Ki Lee and all my laboratory students, Xiaodong Tao, Deok Hwa Hong, Kang Min Park, Dal Jae Lee and Xingyong Song who have provided valuable help in preparation of the relevant materials and proofreading the typed materials. Finally, I am grateful to my wife, Eun Sue Kim, and my children, Janette and Young Je, who have tolerated me with patience and love and helped make this book happen.

Contents 1. Introduction: Understanding of Optomechatronic Technology.............1 2. Fundamentals of Optics................................................................................31 3. Machine Vision: Visual Sensing and Image Processing .....................105 4. Mechatronic Elements for Optomechatronic Interface........................173 5. Optomechatronic Integration ....................................................................255 6. Basic Optomechatronic Functional Units ...............................................299 7. Optomechatronic Systems in Practice .....................................................447 Appendix A1

Some Considerations of Kinematics and Homogeneous Transformation......................................565

Appendix A2

Structural Beam Deflection............................................573

Appendix A3

Routh Stability Criterion ...............................................577

Index .....................................................................................................................581

1 Introduction: Understanding of Optomechatronic Technology CONTENTS Historical Background of Optomechatronic Technology ................................ 4 Optomechatronics: Definition and Fundamental Concept ............................. 8 Practical Optomechatronic Systems............................................................ 9 Basic Roles of Optical and Mechatronic Technologies .......................... 12 Basic Roles of Optical Technology..................................................... 13 Basic Roles of Mechatronic Elements................................................ 15 Characteristics of Optomechatronic Technology .................................... 16 Fundamental Functions of Optomechatronic Systems.................................. 20 Fundamental Functions ...................................................................................... 21 Illumination Control.................................................................................... 21 Sensing........................................................................................................... 24 Actuating ....................................................................................................... 24 Optical Scanning .......................................................................................... 24 Visual/Optical Information Feedback Control ....................................... 24 Data Storage.................................................................................................. 25 Data Transmission/Switching ................................................................... 25 Data Display ................................................................................................. 25 Optical Property Variation.......................................................................... 26 Sensory Feedback-Based Optical System Control .................................. 26 Optical Pattern Recognition ....................................................................... 26 Remote Operation via Optical Data Transmission................................. 27 Material Processing...................................................................................... 27 Summary ............................................................................................................... 27 References ............................................................................................................. 28 Most engineered devices, products, machines, processes, or systems have moving parts and require manipulation and control of their mechanical or dynamic constructions to achieve a desired performance. This involves the use of modern technologies such as mechanism, sensor, actuator, control, microprocessor, optics, software, communication, and so on. In the early 1

Optomechatronics

value (performance)

2

optical element

electrical/ electronics


optical element

software

software



optical element

software


mechanical mechanical mechanical mechanical mechanical mechanical element element element element element element

1800

1970

2000

year FIGURE 1.1 Key component technologies contributing to system evolution (not to scale).

days, these have been operated mostly via mechanical elements or devices which caused inaccuracy and inefficiency, thus resulting in difficulty in achieving a desired performance. Figure 1.1 and Figure 1.2 show how the key technologies such as mechanical, electrical, and optical have contributed to the evolution of machines/systems in terms of “value or performance” as years have passed [6]. As can be seen from the figure, tremendous efforts have been made to enhance system performance by combining electrical and electronic hardware with mechanical systems. A typical example is a gear-trained mechanical system controlled by a hardwired controller. This mechanical and electronic, called mechatronic configuration, consisted of two kinds of components: mechanism, and electronics and electric hardware. Because of 10k 10k

mechanism mechanical automation

− 411 +

analog control

μ-processor embedded M/C

optomechatronically embedded system

internet based (teleoperation)

mechatronic technology optomechatronics technology

FIGURE 1.2 Evolution of machines.

Introduction: Understanding of Optomechatronic Technology

3

the hard-wired structural limitation of this early mechatronic configuration, flexibility was not embedded in most systems in those days. This kind of tendency lasted until the mid 1970s when microprocessors came into use for industrial applications. The development of microprocessors has provided a new stimulant for industrial evolution. This brought about a big change, the replacement of many mechanical functions with electronic ones through the role of microprocessor. This evolutionary change has opened up the era of mechatronics, and has raised the autonomy level of machines and systems, at the same time increasing versatility and flexibility. The autonomy and flexibility achieved thus far, however, have a growth limited to a certain degree, since both the hardware and software of the developed mechatronic systems have not been developed so much as to have the capability of realizing many complicated functions autonomously while adapting to changing environments. In addition, information structure has not been developed to have real-time access to appropriate system data and information. There may be several reasons for this delay. The first one may be that in many cases mechatronic components alone may not achieve desirable function or performance as specified for a system design. The second one is that, although mechatronic components alone can work, the results achieved may not be as good as required because of their low perception and execution capability and also inadequate integration between hardware and software. In fact, in many cases measurements are difficult or not even feasible due to inherent characteristics of the systems. In some other cases, the measurement data obtained by conventional sensors are not accurate or reliable enough to be used for further processing. They can sometimes be noisy, necessitating some means of filtering or signal conditioning. The difficulties listed here may limit the enhancement of the functionality and performance of the mechatronic systems. This necessitates the integration of the mechatronic technology with other systems. In recent years, optical technology has been increasingly incorporated at an accelerated rate into mechatronic systems, and as a result, a great number of mechatronic products, machines and systems with smart optical components have been introduced into the market. There may be some compelling reasons for this changing trend. One reason is that the optical technology possesses unique characteristics such as noncontact/noninvasiveness visual perception, and insensitivity to electrical noise. As shown in Figure 1.1, the contribution of the optical technology is growing and enhances the system value and performance, since optical elements incorporating mechatronic elements embedded in the system provide some solutions to the difficult-to-solve technical problems. This emerging trend demonstrates that the optically integrated technology provides enhanced characteristics in such a way that it creates new functionalities that are not achievable with conventional technology alone, exhibits higher functionalities since it makes products and systems function on an entirely different principle or in a more efficient manner, and produces high precision

4

Optomechatronics

and reliability since it can facilitate or enable in-process monitoring and control of the system state. Besides, the technology makes it feasible to achieve dimensional downsizing and allows system compactness, since it has the capability of integrating sensors, actuators, and processing elements into one tiny unit.

Historical Background of Optomechatronic Technology The root of the development of the optically integrated mechatronic technology may be easily found when we revisit the historical background of the technological developments of mechatronics and optoelectronics. Figure 1.3 shows the development of mechatronic technology in the upper line above the arrow and that of optical engineering in the lower line [8]. The real electronic revolution came in the 1960s with the integration of transistor and other semiconductor devices into monolithic circuits following the invention of the transistor in 1948. Then the microprocessor was invented in 1971 with the aid of semiconductor fabrication technology and made a tremendous impact on a broad spectrum of technological fields. In particular, the development created a synergistic fusion of a variety hardware and software technologies by combining them with computer technology. The fusion made it possible for machines to transform analog signal into digital, to compute necessary calculations, to make decisions based upon the computed results and software algorithms, and then to take appropriate action according to the decision and to accumulate knowledge/data/information within their own memory domain. This new functionality has endowed machines and systems with characteristics such as flexibility and adaptability, and the importance of this concept has been recognized among industrial sectors, which accelerated ever wider applications. In the 1980s, the semiconductor technology also created micro-electro-mechanical systems (MEMSs), and this brought about a new dimension of machines and systems, micro-sizing their dimensions. Another technological revolution, so-called opto-electronic integration, has continued during the last 40-plus years ever since the laser was invented in 1960. This was made possible with the aid of advanced fabrication methods such as chemical vapor deposition, molecular-beam epitaxy, and focused-ion-beam micro-machining. These methods enabled integration of optical, electro-optic, and electronic components in a single compact device. The charge coupled device (CCD) image sensor developed in 1974 not only introduced computer vision technology but also opened up a new era of optical technology along with optical fiber sensors which appeared from 1976. The optical components and devices that were developed possessed a number of favorable characteristics, including: (1) noncontact/noninvasive; (2) easy to transduce; (3) having a wide sensing range; (4) insensitive to

FIGURE 1.3 History of optomechatronics. Source: Cho, H.S. and Kim, M.Y., IEEE Transaction on Industrial Electronics, 52:4, 932–943, 2005. q 2005 IEEE.

Introduction: Understanding of Optomechatronic Technology 5

6

Optomechatronics

electrical noises; (5) distributed sensing and communication; and (6) high bandwidth. Naturally, these favorable optical characteristics began to be integrated with those of the mechatronic elements and this integration helped achieve systems of higher performance. When a system or a machine is integrated in this way, namely optically, mechanically, and electronically, it is called an optomechatronic system. Looking back at the development of this system shown in Figure 1.3, we can bring to mind a number of practical examples. The lithography tool that fabricates ICs and other semiconductor devices belongs to this system category: it functions through a series of elaborate mirrors in addition to a light beam, optical units and a stepper servo mechanism that precisely shifts the wafer from site to site. Another representative system is the optical pick-up device mass-produced from 1982. The pickup system reads information off the spinning disc by controlling both the up-and-down and side-to-side tracking of a read head which carries a low-power diode laser beam focused onto the pits of the disc. Since the early days, a great number of optomechatronic products, machines or systems have come out at an increasingly accelerated rate, for the effects that can be achieved with the properties of optical components are significant. As shown in Figure 1.3, through the advancement in microsystems and the advent of MEMS, optomechatronic technology has brought about a new technology evolution, that is, a marriage of optics with microsystems or MEMS. A variety of the components or systems that belong to this category have been developed, and the atomic force microscope (AFM), optical MEMS, and optical switch are some examples among them. As seen from the historical perspective, the electronic revolution accelerated the integration of mechanical and electronic components and later, the optical revolution created the integration of optical and electronic components. This trend enabled a number of conventional systems having very low level autonomy and very low work performance to evolve into those having improved autonomy and performance. Figure 1.4 illustrates practical systems currently in use that evolved from their original old versions. Printed circuit board (PCB) inspection was carried out by the naked eyes of human workers using a microscope until recently, but is now performed by a visual inspection technique. The chip mounter or surface mounting device (SMD), originally mostly performed in a mechanical way based on CAD drawing, is now being carried out by integrated devices such as a part position estimator, visual sensors, and a servo control unit. The coordinate measuring machine (CMM) appeared as a contact then noncontact device, then became a digital electromagnetic type and then an optical type. In recent years, the CMM is actively being researched to introduce it as an accurate, reliable, versatile product, in which a sensor integration technique is to be adopted, as can be seen from the figure. The washing machine shown in Figure 1.6d also evolved from a mechanically operated machine to one having optical sensory feedback and intelligent control function. Table 1.1 illustrates the evolution of some of

touch probe

FIGURE 1.4 Illustrative evolutions.

(c) coordinate measuring machine

(a) PCB inspection

Optical probe

lamp

(d) projector

fluorescent magnetic yolk panel CRT projector

electron gun

electron beam

(b) chip / SMD mounting

mechanical positioning mounter

camera

illumination

LCD projector

B

G

R lens DLP projector

digital micro mirrors

PCB

mounter nozzle

Visual positioning mounter

Fiducial

Introduction: Understanding of Optomechatronic Technology 7

8

Optomechatronics

TABLE 1.1 Evolution in Various Products Technology/Product Data storage disc Printer Projector IC chip mounter

PCB inspection Camera

Coordinate measuring machine (CMM)

Technological Trend Mechanical recording ! magnetic recording·optical recording Dot printer ! thermal printer/ink jet printer ! laser printer CRT ! LCD·DLP projector Partially manual automated assembly ! mechanically automated assembly ! assembly with visual chip recognition Naked eye inspection ! optical/visual inspection Manual film camera ! motorized zoom, auto exposure, auto focusing ! digital camera (CMOS, CCD device) Touch probe ! optical probe ! touch probe þ visual/optical sensing

Source: Cho, H.S. and Kim, M.Y., IEEE Transactions on Industrial Electronics, 52:4, 932–943, 2005. q 2005 IEEE.

products through the presence of optomechatronic technology. In the sequel, we shall elaborate more on this issue and utilize a number of practical systems to characterize optomechatronic technology.

Optomechatronics: Definition and Fundamental Concept It can be observed from the previous discussions that the technology associated with the developments of machines/processes/systems has continuously evolved to enhance their performance and to create new value and new function. Mechatronic technology integrated by mechanical, electronic/electrical, and computer technologies has been certainly taking an important role for such evolution, as can be seen from the historical time line of technology evolution. To make them evolve further towards systems of precision, reliability, and intelligence, however, optics and optical engineering technology needed to be integrated into mechatronics, thus compensating for some limitations in the existing functionalities and creating new ones. The optomechatronics centered in the middle of Figure 1.5 is, therefore, a technology integrated with the optical, mechanical, electrical, and electronic technologies. The technology fusion in this new paradigm is termed optomechatronics or


9

Optics

Optom ech at

r

Op t

ics tron lec oe

ics on

Optomechatronics

Mechanics

M e c h a tr o nic s

Electronics

FIGURE 1.5 The optomechatronic system.

optomechatronic technology [6, 7]. Figure 1.5 shows the integrated technologies that can be achieved by three major technologies: optical, electrical, and mechanical. We can see that optomechatronics can be achieved with a variety of different integrations. We will see in Chapter 5 that these three important combined technologies, optoelectronics, optomechanics, and mechatronics, will be the basic elements for optomechatronics integration. In this section, to provide a better understanding of, and insight into, the system we will illustrate a variety of optomechatronic systems being used in practice and briefly review the basic roles of optical and mechatronic technologies. Practical Optomechatronic Systems Examples of optomechatronic systems are found in many engineering fields such as control and instrumentation, inspection and test, optical, manufacturing, consumer and industrial electronics, MEMS, automotive, and biomedical applications. Here, we will take only some examples of such fields of application. Cameras and motors are typical products which are operated by optomechatronic components. For example, a smart camera [3] is equipped with an aperture control and a focusing adjustment together with an illuminometer to perform well regardless of the ambient brightness change. With this system configuration, new functionalities are created for the enhancement of the performance modern cameras. As shown in Figure 1.6a, the main components of a camera are several lenses, an aperture, a shutter, and a film or an electrical image cell such as CCD or complementary metal oxide semiconductor (CMOS). Images are focused and exposed on the film or the electrical image cell via a series of lenses which effect zooming and focusing of an object. Moving the lenses with respect to the imaging plane results in changes in magnification and focusing points. The amount of light entering

FIGURE 1.6 Illustrations of optomechatronic systems.

(g) fiber scope device for inspection for microfactory

2 mm

SMA coil spring

bending part SMA actuator

image guide fiber light guide fiber

sweeper

laser half-mirror

Photo-detector lens

=Closed(up)

lens

fibers

Non-interrupted signal

mirror

disk

squeeze roll

(i) pipe welding process

ng ldi n we ctio e r i impeder d

contact tip

laser

(f) n×n optical switching system

=Open(down)

cutoff signal MEMS Mirror

input fiber array

(c) optical storage disk

mirror (X-Y scanner)

(h) rapid prototyping process

liquid photopolymer

focusing lens

platform

elevator motorized actuators

PCB

illuminator and camera

micro-positioner

macro-positioner

(e) vision guided micro positioning system

picker

part rack

laser source

output of wash sensor

phototransistor wash sensor drainpipe mechanical part

(d) modern washing machine with optical sensory feedback

motor

light

3-axis piezo electric stage

AFM Tip

AFM Cantilever

(b) atomic force microscope

infrared LED

water supply valve

washing tank water

upper lid

(a) camera

y

position sensitive detector z y

laser

10 Optomechatronics


11

through the lenses is detected by a photosensor and is controlled by changing either the aperture or shutter speed. Recently, photosensors or even CMOS area sensors are used for autofocusing with a controllable focusing lens. A number of optical fiber sensors employ optomechatronic technology whose sensing principle is based on detection of modulated light in response to changes in the physical variables to be measured. For example, the optical pressure sensor uses the principle of the reflective diaphragm, in which deflection of the diaphragm under the influence of pressure changes is used to couple light from an input fiber to an output fiber. An atomic force microscope (AFM) is composed of several optomechatronic components: a cantilever probe, a laser source, a position-sensitive detector (PSD), a piezo-electric actuator and a servo controller, and an x-y servoing stage, as shown in Figure 1.6b, which employs a constant-force mode. In this case, the deflection of the cantilever is used as input to a feedback controller, which, in turn, moves the piezo-electric element up and down in z, responding to the surface topography by holding the cantilever deflection constant. This motion yields a positional variation of light spot at the PSD, which detects the z-motion of the cantilever. The position-sensitive photodetector provides a feedback signal to the piezo-motion controller. Depending upon the contact state of the cantilever, the microscope is classified into contact AFM, intermittent AFM, or noncontact AFM. The optical disc drive (ODD) or optical storage disc is an optomechatronic system as shown in Figure 1.6c. The ODD is composed of an optical head that carries a laser diode, a beam focus servo that dynamically maintains the laser beam in focus, and a fine track voice coil motor (VCM) servo that accurately positions the head at a desired track. The disc substrate has an optically sensitive medium protected by a dielectric overcoat and rotates under a modulated laser beam focused through the substrate to a diffractionlimited spot on the medium. Nowadays a washing machine effectively utilizes optoelectronic components to improve washing performance. It has the ability to feedback control the water temperature within the washing drum and adjust the washing cycle time, depending upon the dirtiness inside the washing water area. As shown in Figure 1.6d, the machines are equipped with an optomechatronic component to achieve such a function. To detect water contamination a light source and a photo-detector are installed at the drain port of the water flowing out from the washing drum, and this information is fedback to the fuzzy controller to adjust washing time or water temperature [44]. The precision mini-robot equipped with a vision system [4] is carried by an ordinary industrial (coarse) robot as shown in Figure 1.6e. Its main function is fine positioning of the object or part to be placed in a designated location. This vision-guided precision robot is directly controlled by visual information feedback, independently of the coarse robot motion. The robot is flexible and low cost, being easily adaptable to change of batch run size, unlike the expensive, complex-and-mass production oriented equipment. This system can be effectively used to assemble wearable computers which

12

Optomechatronics

require the integration of greater numbers of heterogeneous components in an even more compact and light-weight arrangement. Optical MEM components are miniature mechanical devices capable of moving and directing a light beam as shown in Figure 1.6f. The tiny structures (optical devices such as mirrors) are actuated by means of electrostatics, electromagnetics, and thermal actuating devices. If the structure is an optical mirror, the device can move and manipulate light. In optical networks, optical MEMS can dynamically attenuate a switch, compensate, and combine and separate signals, all in an optical manner. The optical MEMS applications are increasing and classified into five main areas: optical switches, optical attenuators, wavelength tunable devices, dynamic gain equalizers, and optical add/drop multiplexes. Figure 1.6g illustrates a fine image fiberscope device [42] which can perform active curvature operations for inspection of a tiny, confined area such as a micro-factory. A shape memory alloy (SMA) coil actuator enables the fiberscope to move through a tightly curvatured area. The device has a fine image fiberscope of 0.2 mm outer diameter with light guides and 2000 pixels. Laser-based rapid prototyping (RP) is a technology that produces prototype parts in a much shorter time than traditional machining processes. One use of this technology is stereo-lithography apparatus (SLA). Figure 1.6h shows the SLA which utilizes a visible or ultraviolet laser and a position servo mechanism to selectively solidify liquid photo-curable resin. The process machine forms a layer with a cross-sectional shape that has been previously prepared from computer-aided design (CAD) data of the product to be produced. By repeating the forming layers in a specified direction, the desired three-dimensional shape is constructed layer by layer. This process solidifies the resin to 96% of full solidification. After building, in a post-curing process the built part is put into an ultraviolet oven to be cured up to 100%. There are a number of manufacturing processes requiring feedback control of in-process state information that must be detected by optoelectronic measurement systems. One such process is illustrated here to help readers to understand the concept of the optomechatronic systems. Figure 1.6i shows a pipe-welding process that requires stringent weld quality control. A structured laser triangulation system achieves this by detecting the shape of a weld bead in an on-line manner and feeding back this information to a weld controller. The weld controller adjusts the weld current according to the shape of element being made. In this situation, no other effective method of instantaneous weld quality measurement can replace the visual in-process measurement and feedback control described here [21]. Basic Roles of Optical and Mechatronic Technologies Upon examination of the functionalities of a number of optomechatronic systems, we can see that there are a number of functions that can be carried


13

out by optical technology. The major functions and roles of optical technology can be categorized into several functional domains as shown in Figure 1.7 [5]. Basic Roles of Optical Technology (1) Illumination: illumination, which is shown in Figure 1.7a, provides the source of photometric radiant energy incident to object surfaces. In general, it produces a variety of different reflective, absorptive, and transmissive characteristics depending on the material properties and surface characteristics of the objects to be illuminated. The illumination source emits spectral energy from a single wavelength which those produces a large envelope of wavelength. (2) Sensing: optical sensors provide fundamental information on physical quantities such as force, temperature, pressure, and strain as well as on geometric quantities such as angle, velocity, etc. This information is obtained by optical sensors using various optical phenomena such as reflection, scattering, refraction, interference, diffraction, and so on. Conventionally, optical sensing devices are composed of a light source, photonic sensors, and optical components such as lenses, beam splitter, and optical fiber as shown in Figure 1.7 b. Recently, numerous sensors have been developed using optical fiber for its advantages in various applications. Optical technology can also contribute to material science. The composition of chemicals can be analyzed by spectrophotometry, which recognizes the characteristic spectrum of light that could be reflected, transmitted, and radiated from the material of interest. (3) Actuating: light can change physical properties of materials by increasing the temperature of the material or affecting the electrical environment. The materials which can be changed by light are lead zirconate titanate (PZT) and SMA. As shown in Figure 1.7c, the PZT is composed of ferroelectrics material, in which the polar axis of the crystal can be changed by applying an electric field. In optical PZT, an electric field is induced in proportion to the intensity of light. The SMA is also used as an actuator. When SMA is illuminated by light, its shape is changed as a memorized shape due to the increase of temperature. On the other hand, when the temperature of SMA is decreased, its shape is recovered. The SMA is used in a variety of actuator, transducer, and memory applications. (4) Data (signal) storage: digitized data composed of 0 and 1 can be stored in media and read optically as illustrated in Figure 1.7d. The principle of optical recording is using light-induced changes in the reflection properties of a recording medium. That is to say, the data are carved in media by changing the optical properties in

FIGURE 1.7 Basic roles of optical technology.

14 Optomechatronics


(5)

(6)

(7)

(8)

15

the media with laser illumination. Then, data reading is achieved by checking the reflection properties in the media using an optical pickup sensor. Data transmitting: light is a good medium for delivering data for its inherent characteristics such as high bandwidth unaffected by external electromagnetic noise. Laser, a light source used in optical communication, has high bandwidth and can contain a lot of data at a time. In optical communication, the digitized raw data such as text or picture are transformed into light signals and delivered to the other side of the optical fiber and decoded as the raw data. As indicated in Figure 1.7e, the light signal is transferred within the optical fiber without loss by total internal reflection. Data displaying: data are effectively understood by end users by visual information. In order to transfer data to users in the form of an image or graph, various display devices are used such as cathode ray tube (CRT), liquid crystal display (LCD), light emitting diode (LED), plasma display panel (PDP), etc. As illustrated in Figure 1.7f, they are all composed of pixel elements consisting of three basic coloremitting cells that are red, green, and blue light. Arbitrary colors can be made by the combination of these three colors. Computing: optical computing is performed by using switches, gates, and flip-flops in their logic operation just like digital electronic computing. Optical switches can be built from modulators using optomechanical, optoelectronic, acousto-optic, and magneto-optic technologies. Optical devices can switch states in about a picosecond or a thousandth of billionth of a second. An optical logic gate can be constructed from the optical transistor. For an optical computer, a variety of circuit elements besides the optical switch are assembled and interconnected, as shown in Figure 1.7g. Light alignment and waveguide are two big problems in the actual implementation of the optical computer. Material property variation: when a laser is focused on a spot using optical components, the laser power is increased on a small focusing area. This makes the highlighted spot of material change its state as shown in Figure 1.7h. Laser material processing methods utilize a laser beam as the energy input, and can be categorized into two groups: (1) the method for changing the physical shape of the materials, and (2) the method for changing the physical status of the materials.

Basic Roles of Mechatronic Elements The major functions and roles of mechatronic elements in optomechatronic systems can be categorized into the following five functional domains: sensing, actuation, information feedback, motion or state control, and embedded intelligence with microprocessor [6].

16

Optomechatronics

First, transducer technology used for sensing nowadays enjoys the integrated nature of mechatronics. The majorities of sensors belong to this category. Second, the drivers and actuators produce a physical effect such as a mechanical movement or a change of property and condition. Third, one of the unique functions of mechatronics is to feedback information for certain objectives. Fourth, the control of motion or state of systems is a basic functionality that can be provided by mechatronics. Last, a mechatronic system implemented with a microprocessor provides many important functions, for example, the stored or programmed control, the digital signal processing, and the design flexibility for the whole system. In addition, advantages of integration within a small space and the low power consumption are attractive features. Characteristics of Optomechatronic Technology Based upon what we have previously observed from various optomechatronic systems, we can summarize the following characteristic points: (1) they possess one or more functionalities to carry out certain given tasks; (2) to produce such functionalities, several basic functional modules are required to be appropriately combined; (3) to achieve combining functions in a desired manner, a certain law of signal (energy) transformation and manipulation needs to be utilized that converts or manipulates one signal to another in a desired form, using the basic mechanical, optical, or electrical one; (4) optomechatronic systems are hierarchically composed of subsystems, which are then composed of units or components. In other words, elements, components, units, or subsystems are integrated to form an optomechatronic system. As we have seen from various illustrative examples of optomechatronic systems discussed above, optomechatronic integration causes all three fundamental signals to interact with each other as shown in Figure 1.8a. Here, three signals imply three different physical variables originated from optical, mechanical, and electrical disciplines. For instance, an optical signal includes light energy, ray and radiation flux, mechanical signal, energy, stress, strain, motion and heat flux, and electrical signal, current, voltage, and magnetic flux, and so on. Depending on how they interact, the properties of the integrated results may be entirely different. Therefore, it is necessary to consider the interaction phenomenon from the point of view of whether the interaction may be efficiently realizable. Optomechatronic integration can be categorized into three classes, depending on how optical elements and mechatronic components are integrated. As indicated in Figure 1.8b, the classes may be divided into the following. (1) Optomechatronically fused type In this type, optical and mechatronic elements are not separable in the sense that if either are removed from the system that they constitute, the system cannot function properly. This implies that those two separate


17

physical inter action

(a) optomechatronic interaction

(b) integration type

dimensional interaction

(c) interaction between three different dimensional worlds FIGURE 1.8 Characteristic features of optomechatronic technology.

elements are functionally and structurally fused together to achieve a desired system performance. Figure 1.9a shows the systems integrated according to this type and include an optically ignited weapon system (left) and a photostrictive actuator (right). The constituent element of each system is shown in the figure where O, M, and E stands for optical, mechanical, and electrical, respectively. (2) Optically embedded mechatronic type In this type, an optical element is embedded into a mechatronic system. The optical element is separable from the system, but the system can function with a decreased level of performance or can function in an entirely different manner. The majority of engineered optomechatronic systems belong to this category, such as servomotor, washers, vacuum cleaners, monitoring and control systems for machines and manufacturing processes, robots, cars, and so on. The anatomy of the servo motor is shown in Figure 1.9b. (3) Mechatronically embedded optical type This type can be found basically from an optical system whose construction is integrated with mechanical and electrical components. Many optical systems require positioning or servoing optical elements and devices to manipulate and align a beam and to control the polarization of the beam. Typical systems that belong to “positioning or servoing” include cameras, optical projectors, galvanometers, series parallel scanners, line

18

Optomechatronics

FIGURE 1.9 Illustration of types of optomechatronic integration. Source: Cho, H.S. and Kim, M.Y., IEEE Transactions on Industrial Electronics, 52:4, 932–943, 2005. q 2005, IEEE.

scan polygons, optical switches, fiber squeezer polarization controllers, and so on. In some other systems, an acoustic wave generator driven by a piezo-element is used to create a frequency shift of a beam. Another typical system is the passive (off-line) alignment of optical fiber–fiber or fiber– waveguide attachment. In this system, any misalignment is either passively or actively corrected by using a micropositioning device to maximize their coupling between fiber – fiber or fiber wave guide. The anatomy of a zoom lens that belongs to this type is shown in Figure 1.9c and consists of five basic elements. Traditional optomechatronic systems usually operate in macro-worlds, if we term a macro-world as a large system whose size can range from mm to a few meters. As briefly mentioned before, due to the miniaturization trend and availability of MEMS and nano-technologies, optomechatronic integration is becoming popular, and thus accelerating the downsizing of engineering products. Optomechatronic integration is therefore achieved in two different physical scales as shown in Figure 1.8c: macro- and micro-nano scales. This length scale difference produces three different types of integration: integration in macro-scale (left), integration in micro/nanoscale (right) and integration in mixed scales (middle). The middle part of the figure implies an employment of combined macro- and micro/nano-systems for the integration. Integration of this type can be found from a variety


19

* atom nanotube virus & bacteria optical lithography

human hair optical fiber 10

10−5 10

−3

MEMS

10−11

10−10 10−9

STM AFM

x-ray nanomanipulation thin film zone plate ring spacing

10−8 10−7

10−6

biological cell

FET

DNA CNT

NEMS

visible light

−4

infrared

* scale in meters

FIGURE 1.10 Scales and dimensions of some entities that are natural and man-made.

of physical systems such as an optical disc (macro) having a micro fineadjustment mechanism, a laser printer that may be operated by micro-lens slits, an AFM, etc. Figure 1.10 depicts scales of some natural entities and some man-made ones. MEMS parts range from hundreds mm to few mm and NEMS parts from few hundreds to tens nm. Nanomanipulation such as atomic lettering using scanning tunneling microscope (STM) is of the order of approximately nm or less. It is known that most of the microscopes currently available have used extremely high resolution. In particular, the extreme case of the smallest scale can be found from STM and AFM whose resolutions range between , 0.1 nm and , 0.5 nm, respectively. One important factor we need to consider in dealing with micro-systems is the so-called “size or scale effect” due to which the influence of the same physical phenomena becomes variable with the length scale of a system. To take some examples: (1) Inertial (volume) force that influences a macro-scale system becomes less significant in micro-systems. Instead, surface effects (force) become rather dominant. This is because in general the inertial force is proportional to volume (L 3), whereas the surface force is in proportion to surface area (L 2). (2) In optical and mechatronic interaction, deflections on the order of wavelengths of light do matter in a microsystem but not in a macrosystem.

20

Optomechatronics

(3) Atomic attractive and repulsive forces become significant in microsystems. (4) The smaller geometrical size material gets stronger. Example (1) may mislead us into neglecting the less significant terms in analyzing microsystems. However, in the case of dynamic analysis and control of the systems, they may not be negligible when the accuracy of the positioning is in high precision or when the range of dominant force operated on the systems is compatible with that of the less significant terms.

Fundamental Functions of Optomechatronic Systems From the discussions in the previous sections, we now elaborate on what type of fundamental functions the optomechatronic integration can produce.

Information feedback control Mechanism design

Precision actuators Optical / Visual Motion control

Artificial intelligence Micro processor

sensors, actuators data display materials processing information processing recognition transmission / switching micro elements

Signal processing Pattern recognition

MEMs

Sensor fusion

FIGURE 1.11 Enabling technologies for optomechatronics.

Sensors & measurement


21

There are a number of distinct functions which originate from the basic roles of optical elements and mechatronic elements discussed. When these are combined, the functional forms may generate the fundamental functions of the optomechatronic system. Figure 1.11 illustrates the technology element needed to produce optomechatronic technology. In the center of the figure, those elements related to optical elements are shown, while mechatronic elements and artificial intelligence are listed on its periphery. These enabling technologies are integrated to form five engines that drive the technology. These include: (1) actuator module, (2) sensor module, (3) control module, (4) signal/ information processing module, and (5) decision making module. It is noted that the integration of these gives a specified functionality with certain characteristics. A typical example is the sensing of an object surface with an AFM. The AFM requires a complicated interaction between sensing element (laser sensor), actuator (piezo-element), controller, and other relevant software. This kind of interaction between modules is very common in optomechatronic systems, and produces the characteristic property of the systems.

Fundamental Functions A number of fundamental functions can be generated by the fusion of optical mechatronic elements. These include (1) illumination control, (2) sensing, (3) actuating, (4) optical scanning, (5) visual/optical information feedback control, (6) data storage, (7) data transmission/switching, (8) data display, (9) optical property variation, (10) sensory feedback based optical system control, (11) optical pattern recognition, (12) remote monitoring/control, and (13) material processing [5]. We will discuss these fundamental functions below.

Illumination Control Illumination needs to be adjusted depending on the optical surface characteristics and surface geometry of objects in order to obtain a good quality image. The parameters to be adjusted include incident angle, and the distribution and intensity of light sources. Figure 1.12a illustrates a typical configuration of such an illumination control system. The system consists of a guardant ring fiber light source, a paraboloid mirror, a toroid mirror, and a positioning mechanism to adjust the distance between the object surface and the parabola mirror. The incident angle is controlled by adjusting the distance, while the intensity of the light source in each region is controlled independently.

object

3-facet mirror xs

object

PSD B

ys

laser zs beam

glass grid

adhesive film

pressure transducer diode strip

(e) motion control : placing diode strip into a glass

camera

detector

opticsservo

(f) data storage/ retrieval

laser

disk

servo

6-D position & orientation laser beam position

tracking control

(b) measurement of a 6-degrees-of-freedom motion of arbitrary objects

PSD A

vacuum-based end effector fiducial mark

cartesian robot

laser source PSD C

FIGURE 1.12 The functionalities achieved by optomechatronic integration.

(d) mirror based scanning

laser

mirror

parabolic mirror

mirror

mirror

lens

laser

(a) illumination control

object lens

half-mirror

camera

mirror

(g) optical cross connect operation by mirror control

scanning mirror

servoing mirror array

(c) shape memory-based optical actuating

spring

motion

light

22 Optomechatronics

23

FIGURE 1.12 Continued.


24

Optomechatronics

Sensing Various types of optomechatronic systems are used to measure various types of physical quantities such as displacement, geometry, force, pressure, temperature, target motion, etc. The commonly adopted feature of these systems is that they are composed of optical, mechanical moving, servoing, and electronic elements. The optical elements are usually the sensing part divided into optical fiber and nonfiber optical transducers. Recently these elements have been accelerating the sensor fusion in which other types of sensors are fused together to obtain necessary information. Figure 1.12b shows a six-dimensional (6D) sensory system employing the optomechatronic principle. The sensor is composed of a 6D microactuator, a laser source, three photosensitive devices (PSD), and a three-facet tiny crystal (or mirror) that transmits the light beam into three PSD sensors. The sensing principle is that the microactuator is so controlled that the top of the tiny mirror is always positioned within the center of the beam. In other words, the output of each PSD sensor is kept identical regardless of object motion. Another typical problem associated with optical sensing is focusing, which needs to be adjusted depending upon the distance between the optical lens and the surface of the objects to be measured. Actuating A number of actuators belong to this category, both optical-based and mechatronic actuation-based. Optical-based actuation utilizes transformation of optical energy into mechanical energy that can be found from a number of practical examples. Figure 1.12c shows one such actuator in which optical energy is used as a heat generator which drives a temperaturesensitive material (e.g., SMA) to move. This actuator can be used to accurately control the movement of mechanical elements. Optical Scanning Optical scanning is used to divert light direction with time in a prescheduled manner and generates a sequential motion of the optical element such as a light source. For high-speed applications, a polygon mirror or galvanometer is effectively used as shown in Figure 1.12d. To operate a scanning system, the scanning step and the scanning angle have to be considered carefully. The scanning ability is useful in applications ranging from laser printing to materials processing. Visual/Optical Information Feedback Control Visual/optical information is very useful in the control of machines, processes, and systems. In this functionality the information obtained by optical sensors is utilized for changing variables operating the systems. A great number of optomechatronic systems require this type of information


25

feedback control. Positioning controlling the position of an object, moving it to a specified location. When a visual sensor is used for the feedback, the motion control is called “visual servoing.” Accurately placing diode strips into a glass grid by using a robot is shown in Figure 1.12e as an illustrative example. A number of practical examples can be found from motion tracking such as following the motion of a moving object and following a desired path based on the optical-sensor information. Many more typical examples can also be found from mobile robot navigation, e.g., optical-based dead reckoning, map building, and vision-based obstacle avoidances. A pipe-welding process shown in Figure 1.6i necessitates an optical information feedback control. The objective of this process control is to regulate the electric power needed for welding in order to attain a specified weld quality. Therefore, the process state relevant to the quality is detected and its instantaneous information is fedback in real-time manner. This principle can also be found from the washing machine shown in Figure 1.6d. In this system, the optical sensor detects the dirtiness of the water and this information is fedback to a controller to adjust the washing time or to adjust the water temperature inside the drum. Data Storage Data storage retrieval is performed by a spinning optical disc and controlled optical units whose main functions are beam focusing and track following, as illustrated in Figure 1.12f. Conventionally, the recording density is limited by the spot size and wavelength of the laser source. Recently, new approaches to increase the recording density are being researched such as near-field optical memory and holographic three-dimensional storing methods. Data Transmission/Switching Optical data switching is achieved by an “all optical network” to eliminate the multiple optical-to-electrical-to-optical (O-E-O) conversions in conventional optical networks [35]. Figure 1.12g illustrates a multi-mirror servo system in which mirrors are controlled to connect a light path any to any ports by using servo controllers. This kind of system configuration is also applied to switch micro-reflective or -deflective lens actuated in front of optical filters. Data Display Digital micro-mirror devices (DMDs) make projection displays by converting white-light illumination into full-color images via spatial light modulators with independently addressable pixels [30]. As schematically illustrated in Figure 1.12h, the DMD developed by Texas Instruments is a lithographically fabricated MEMS system composed of hundreds of thousands of titling aluminum-alloy mirrors (16 mm £ 16 mm) which

26

Optomechatronics

functions as a pixel in the display. Each mirror is attached to an underlying sub-pixel called the yoke, which, in turn, is attached to a hinge support post by way of an aluminum torsion hinge. This allows the mirror to rotate about the hinge axis until the landing tips touch the landing site. This switching action occurs from þ 108 to 2 108 and takes place in several msec. Optical Property Variation Optical properties such as frequency, wavelength, and so on can be tuned by making light waves interact with mechatronic motion. Figure 1.12i illustrates a frequency shifter which consists of an acoustic wave generator, a piezoelectric element, and a high birefringence fiber. In-line fiber optic frequency shifters utilize a traveling acoustic wave to couple light between the two polarization modes of a high-birefringence fiber, with the coupling accompanied by a shift in the optical frequency. Sensory Feedback-Based Optical System Control In many cases, optical or visual systems are operated based on the information which is provided by external sensory feedback, as shown in Figure 1.12j. The systems that require this configuration include: (1) zoom and focus control system for video-based eye-gaze detection using an ultrasonic distance measurement, (2) laser material processing systems whose laser power or laser focus is real-time controlled depending upon the process monitoring, (3) visually-assisted assembly systems in cooperating with other sensory information such as force, displacement, and tactile, (4) sensor fusion systems in which optical/visual sensors incorporate with other sensory systems, and (5) visual/optical systems that need to react to acoustic sound and other sensory information tactile, force, displacement, velocity, etc. Optical Pattern Recognition Three-dimensional pattern recognition shown in Figure 1.12k, uses a laser and a photorefractive crystal as the recording media to record and read holograms [38]. The photorefractive crystal stores the intensity of the interference fringe constructed by the beam reflected from a 3D object and a plane-wave reference beam. The information on the shape of the object is stored in the photorefractive crystal at this point, and this crystal can be used as a template to be compared with another three-dimensional object. To recognize an arbitrary object, an optical correlation processing technique is employed. When an object to be recognized is placed in the right position from the original object with the same orientation, the recorded hologram diffracts the beam reflected from the object to be compared, and the


27

diffracted wave propagating to the image plane forms an image that represents the correlation between the Fourier transforms of the template object beam and the object to be compared. If a robot or a machine is trying to find targets in unknown environments, this system can be effectively used to recognize and locate them in a real-time manner. In this case a scanned motion of the optical recognition system is necessary over the whole region of interest. Remote Operation via Optical Data Transmission Optical data transmission is widely used when data/signals obtained from sensors are subject to external electrical noise or when the amount of data needed to be sent is vast, or when operation is being done at a remote site. Operation of systems at a remote site is ubiquitous nowadays. In particular, internet-based monitoring, inspection, and control are becoming pervasive in many practical systems. Visual servoing of a robot operated in a remote site is a typical example of such a system, as shown in Figure 1.12‘. In the operation room, the robot controls the position of a vibration sensor to monitor the vibration signal with the aid of vision information and transmits this signal to a transceiver. Another typical example is the operation of the visual servoing of mobile robots over the internet. Material Processing Material processing can be achieved by the integration of a laser optical source and a mechatronic servo mechanism. The system produces a material property change or cut surface and heat-treated surface of work pieces. In addition to conventional laser machining, laser micro-machining is becoming popular due to its reduced cost and accuracy. MEMS fabrication, drilling and slotting in medicine, wafer dry cleaning, and ceramic machining employ such micro-machining technology. Figure 1.12m shows a laser surface hardening process [46], which is a typical example of an optical-based monitoring and control system. The laser of high power (4 kW) focused through a series of optical units hits the surface of a workspace and changes the material state of the workpiece. Maintaining a uniform thickness of the hardened surface (less than 1 mm) is not an easy task since it depends heavily upon several process parameters such as the workpiece travel speed, laser power, and the surface properties of the material. Here, the indirect measurement of the coating thickness is made by an infra-red temperature sensor and feedback to the laser power controller.

Summary In recent years, integration of optical elements into mechatronic systems has increasingly accelerated since it produces a synergistic effect, creating new

28

Optomechatronics

functionalities for the systems or enhancing the performance of the systems. This trend will certainly be the future direction of current mechatronic technology and contribute to the advent of a new technological paradigm. This chapter has focused on helping readers to understand optomechatronic technology and systems in which optical, mechanical, electrical/electronics, and information engineering technologies are integrated. In particular, the definition and fundamental concepts of the technology have been introduced. Based upon these, it has been possible to identify types of optomechatronic systems and the fundamental functions that can be created by the integration. Altogether, thirteen functionalities have been identified. They are expected to lead to future developments of such technologies as mechatronics, instrumentation and control, adaptive optics, MEMS, biomedical technology, information processing storage as well as communication. From this point of view, optomechatronics will be a prime technological element that will lead the future direction of various technologies.

References [1] [2] [3] [4] [5] [6] [7] [8]

[9] [10] [11] [12] [13]

Backmutsky, V. and Vaisman, G. Theoretical and experimental investigation of new mechatronic scanners, Proceeding of the IEEE Nineteenth Convention of Electrical and Electronics Engineers in Israel, pp. 363– 366, 1996. Bishop, R.H. The Mechatronics Handbook. CRC Press, Boca Raton, FL, 2002. Cannon Co. Ltd, http:www.canon.com, 2002. Chen, M. and Hollis, R. Vision-guided Precision Assembly. 1999, http://www-2. cs.cmu.edu/afs/cs/project/msl/www/tia/tia_desc.html. Cho, H.S. and Park, W.S. Determining optimal parameters for stereolithography process via genetic algorithm, Journal of Manufacturing Systems, 19:1, 18 – 27, 2000. Cho, H.S. Characteristics of optomechatronic systems, Opto-mechatronic Systems Handbook, chap. 1, CRC Press, Boca Raton, FL, 2002. Cho, H.S. and Kim, M.Y., Optomechatronic technology: the characteristics and perspectives, IEEE Transaction on Industrial Electronics, 52:4, 732– 743, 2005. Ebisawa, Y., Ohtani, M. and Sugioka, A. Proposal of a zoom and focus control method using an ultrasonic distance-meter for video-based eye-gaze detection under free-head conditions, 18th Annual Conference of the IEEE Engineering in Medicine and Biology Society, Vol. 2, pp. 523– 525, Amsterdam, Netherlands, 1997. Geppert, L. Semiconductor lithography for the next millennium, IEEE Spectrum, April, 33 – 38, 1996. Han, K., Kim, S., Kim, Y. and Kim, J. Internet control architecture for internetbased personal robot, Autonomous Robots, 10, 135– 147, 2001. Haran, F.M., Hand, D.P., Peters, C. and Jones, J.D.C. Real-time focus control in laser welding, Measurement Science and Technology, Vol. 7, 1095– 1098, 1996. Higgins, T.V. Optical storage lights the multimedia future, Laser Focus World, September/October, Vol. 31, 1995. Ishi, T. Future trends in mechatronics, JSME, International Journal, Series, III, 33:1, 1 – 6, 1990.

Introduction: Understanding of Optomechatronic Technology [14]

29

Jeong, H.M., Choi, J.J., Kim, K.Y., Lee, K.B., Jeon, J.V. and Park, Y.E., Milli-scale mirror actuator with bulk micromachined vertical combs, Proceedings of the Transducer, Sendai, Japan, pp. 1006– 1011, 1999. [15] Kamiya, M., Ikeda, H. and Shinohara, S. Data collection and transmission system for vibration test, Proceedings of the Industry Application Conference, 3, 1679–1685, 1998. [16] Kayanak, M.O. The age of mechatronics, IEEE Transactions in Industrial Electronics, 43:1, 2 – 3, 1996. [17] Kim, W.S. and Cho, H.S. A novel sensing device for obtaining an omnidirectional image of three-dimensional objects, Mechatronics, 10, 717– 740, 2000. [18] Kim, J.S. and Cho, H.S. A robust visual seam tracking system for robotic arc welding, Mechatronics, 6:2, 141– 163, 1996. [19] Kim, W.S. and Cho, H.S. A novel omnidirectional image sensing system for assembling parts with arbitrary cross-sectional shapes, IEEE/ASME Transactions in Mechatronics, 3:4, 275– 292, 1998. [20] Knopf, G.K. Short course note, SC 255 Opto-Mechatronic System Design, SPIE’s Photonic East, Boston, USA, 2000. [21] Ko, K.W. Cho, H.S., Kim, J.H and Kong, W.I. A bead shape classification method using neural network in high frequency electric resistance weld, Proceeding of World Automation Congress, Alaska, USA. [22] Krupa, T.J. Optical R&D in the Army Research Laboratory, Optics&Photonics News, June, 16 – 39, 2000. [23] Larson, M.C. and Harris, J.S. Wide and continuous wavelength tuning in a vertical cavity surface emitting laser using a micromachined deformable membrane mirror, Applied Physics Letters, 15, 607– 609, 1996. [24] Larson, M.C. Tunable Optoelectronic Devices. 2000, http://www-snow.stanford. edu/~larson/research.html. [25] Lim, T.G. and Cho, H.S. Estimation of weld pool sizes in GMA welding process using neural networks, Proceedings of Institute of Mechanical Engineers, 207, 15 – 26, 1993. [26] Madou, M. Fundamentals of Microfabrication. CRC Press, Boca Raton, FL, 1997. [27] Mahr Co. Ltd, Multiscope 250/400. 2001, http://www.mahr.com/en/content/ products/mess/mms/ms250.html. [28] McCarthy, D.C. Hands on the wheel, cameras on the road, Photonics Spectra, April, 78 – 85, 2001. [29] Mitutoyo Co. Ltd, Quick Vision. 2001, http://www.mitcat.com/e-02.htm. [30] McDonald, T.G. and Yoder, L.A. Digital micromirror devices make projection displays, Laser Focus World, August, 1997. [31] McKee, G. Robotics and machine perception, SPIE’s International Technical Group Newsletter, 9, 2000. [32] Park, I.O., Cho, H.S. and Gweon, D.G. Development of programmable bowl feeder using a fiber optic sensor, 10th International Conference on Assembly Automation, Tokyo, Japan, 1989. [33] Park, W.S., Cho, H.S., Byun, Y.K., Park, N.Y., and Jung, D.K. Measurement of 3-D position and orientation of rigid bodies using a 3-facet mirror, SPIE International Symposium on Intelligent Systems and Advanced Manufacturing, pp. 2– 13, Boston, USA. [34] Pugh, A. Robot Sensors, Tactile and Non-Vision, Vol. 2. Springer-Verlag, Berlin, 1986.

30

Optomechatronics

[35] Robinson, S.D. MEMS technology — micromachines enabling the all optical network, Electronic Components and Technology Conference, pp. 423 – 428, Orlando, FL, 2001. [36] Rogers, C.A. Intelligent materials, Scientific American, September, 122–127, 1995. [37] Roh, Y.J., Cho, H.S. and Kim, J.H. Three dimensional volume reconstruction of an object from X-ray images, Conference on SPIE Optomechatronic Systems Part of Intelligent Systems and Advanced Manufacturing, Vol. 4190, pp. 181–191, Boston, USA, 2000. [38] Shin, S.H. and Javidi, B. Three-dimensional object recognition by use of a photorefractive volume holographic processor, Optics Letters, 26:15, 1161– 1163, 2001. [39] Shiraishi, M. In-process control of workpiece dimension in turning, Annals of the CIRP, 28:1, 333– 337, 1979. [40] Takamasu, K. Development of a nano-probe system, Quarterly Magazine Micromachine, 35, 2001. [41] Toshiyoshi, H., Su, J.G., LaCosse, J. and Wu, M.C. Micromechanical lens scanners for fiber optic switches, Proceedings of the Third International Conference on Micro Opto Electro Mechanical Systems (MOEMS 99), pp. 165– 170, Mainz, Germany, , 1999. [42] Tsuruta, K., Mikuriya, Y. and Ishikawa, Y. Micro sensor developments in Japan, Sensor Review, 19, 37 – 42, 1999. [43] Veeco co. Ltd, Contact AF. 2000, http://www.tmmicro.com/tech/modes/ contact.htm. [44] Wakami, N., Nomura, H. and Araki, S. Fuzzy logic for home appliances, Fuzzy Logic and Neural Networks, pp. 21.1 – 21.23, McGraw Hill, New York, 1996. [45] Wilson, A. Machine vision speeds robot productivity, Vision Systems Design, October, 2001. [46] Woo, H.G. and Cho, H.S. Estimation of hardened layer dimensions in laser surface hardening processes with variations of coating thickness, Surface and Coatings Technology, 102:3, 205– 217, 1998. [47] Zankowsky, D. Applications dictate choice of scanner, Laser Focus World, December, 1996. [48] Zhang, J.H. and Cai, L. An autofocusing measurement system with a piezoelectric translator, IEEE/ASME Transactions in Mechatronics, 2:3, 213– 216, 1997. [49] Zhou, Y., Nelson, B.J. and Vikramaditya, B. Integrating optical force sensing with visual servoing for microassembly, Journal of Intelligent and Robotic Systems, 28, 259– 276, 2000.

2 Fundamentals of Optics CONTENTS Reflection and Refraction ................................................................................... 33 Lenses .................................................................................................................... 36 Refraction at a Spherical Surface............................................................... 37 Multiple Lenses and System Matrices.............................................................. 46 The System Matrices.................................................................................... 50 Computer Ray Tracing ................................................................................ 52 Aperture Stops and Pupils ................................................................................. 53 Aberration ............................................................................................................. 55 Spherical Aberration.................................................................................... 56 Coma Aberration.......................................................................................... 56 Field Curvature ............................................................................................ 57 Astigmatism .................................................................................................. 57 Distortion....................................................................................................... 59 Polarization ........................................................................................................... 61 Coherence.............................................................................................................. 64 Interference ........................................................................................................... 65 Young’s Experiment .................................................................................... 67 Fabry-Perot Interferometer ......................................................................... 69 Michelson Interferometer ........................................................................... 72 Diffraction.............................................................................................................. 74 Double Slits ................................................................................................... 79 Multiple Slits................................................................................................. 81 Circular Aperture......................................................................................... 84 Diffraction Grating....................................................................................... 89 Optical Fiber Transmission................................................................................. 92 Gaussian Beam Optics ........................................................................................ 95 Problems................................................................................................................ 97 References ........................................................................................................... 103

Physical optics is rooted on the fact that light corresponds to the electromagnetic radiation in the narrow slice of spectrum. Optics concerns

31

32

Optomechatronics

with the radiation within this range visible to the human eye but often extends to the region of spectrum near visible region. Classification due to variation in wave length is summarized in Figure 2.1. Visible radiation ranges from 0.35 to 0.75 mm, depending upon types of color of the light their wave length varies. The ultraviolet region extends from 0.01 to 0.35 mm, while infrared radiation extends from the lower end of the visible spectrum to nearly 300 mm. Being conduced as one of the electromagnetic waves, light radiates from a source, propagating straight through a medium at a velocity, c, in a form of combination of electric and magnetic waves oscillating at a frequency v and with a wave length l: The simplest form of such combination of waves is a monochromatic (single color) light wave that is composed of sinusoidal components of electric and magnetic fields, propagating in free space along the direction z as represented in Figure 2.2. The wave here is a plane wave and the electric field is varying parallel to the propagation direction. As can be seen from the figure, the electric and magnetic fields are vector quantities, and thus their magnitude and direction need to be specified. However, both fields are oriented at right angles to each other in nonabsorbing media. Only the direction and magnitude of the electric field need to be specified. We will return to this subject with a little bit more detail when we discuss polarization of light in this chapter. The speed of propagation of the wave in

wavelength (m) 10−10

10−8

frequency (HZ) X-ray X -ray

UV UVlight light

1×1018

1×1016

visible 1×1014

10−6 infrared

1×1012

10−4

10−2

μ-waves μ -waves

FIGURE 2.1 The electromagnetic spectrum: optical radiation.

1×1010

Fundamentals of Optics

33 y E x B

B

E

E

B

B

E

z

FIGURE 2.2 Propagation of a plane-polarized electromagnetic wave.

free space (vacuum) is approximately given by c ¼ 2:998 £ 108 m/sec. The speed in air is just a little bit smaller than this, while that in glass and water is always much smaller than that in vacuum. Throughout this book we will confine ourselves to optical problems concerning visible and monochromatic light. In the first part of this chapter, we will treat a light wave as if it travels from its source along straight lines without any wave motion. Using this treatment, we therefore cannot explain such phenomena as the interference, polarization, diffraction and so on that occur due to wave motion of the light. The regime of this treatment is called “geometric optics” within which the wavelength or wave motion effect is considered negligible compared to the dimensions of the optical system interacting with light waves. In the later part of this chapter, we will treat wave optics, in which wave motion of light will be included, to describe the above phenomena that cannot be explained by geometric optics.

Reflection and Refraction A light wave transmitting through a lens or a prism is typical of wave refraction. A light wave reflecting from the surface of a mirror or a coated reflective surface is typical of wave reflection. To understand these phenomena we will review the laws of reflection and refraction. When a light wave travels through several homogeneous media in sequence, its optical path constitutes a sequence of discontinuous line segments. A simple case of this situation occurs when a light ray is incident upon a plane surface separating two media. There are several laws governing the direction of light propagation at the interface. Here, we will discuss briefly

34

Optomechatronics

FIGURE 2.3 Reflection ur and refraction ut at the boundary of two different media.

the two most frequently used laws, reflection and refraction. When a light wave is incident on an interface lying in the plane of incidence shown in Figure 2.3, a certain fraction of the wave is absorbed or transmitted and the remainder is reflected. This is illustrated using a plane wave in Figure 2.4a. As can be seen from the figure, three waves lie in the plane of incidence and those waves are conveniently represented by three corresponding rays

incident light

normal

A

normal

reflected light

reflected light

incident light

C

qi medium 1 medium 2

B D

refracted light

(a) a beam of plane waves enters the interface

(b) reflection

normal incident light

qi

ni qt

nt refracted light

(c) refraction FIGURE 2.4 A plane wave front incident on the boundary between two media.

qr


35

AB; BC; BD: A ray is a line that indicates the direction of wave propagation and is perpendicular to the wave front. Therefore, we will transform this figure into those represented only by rays as shown in Figure 2.4b. The law of reflection states that the angle of reflection equals the angle of incidence. This means that

ui ¼ ur

ð2:1Þ

This is a special case of light reflection. In general cases, most interfaces contain both specular and diffuse surfaces. Diffuse surfaces cause the reflected light to scatter in all directions radiating an equal amount of power per unit area per unit solid angle. The condition given in Equation 2.1 is valid for the specular reflection. When an incoming plane light passes through the boundary between two homogeneous media, the direction of it will bend because of its speed change. The amount it is bent is dependent upon the absolute refraction index of each media, which is defined by n ¼ c=v, where c is speed of light in a vacuum and v is the speed in the medium. The law of refraction, called Snell’s law, states that the refraction angle ut obeys the following relation, ni sin ui ¼ nt sin ut

ð2:2Þ

where ni is the refraction index of the incident medium, while nt is that of the transmitting medium. When ni , nt , that is, the ray enters a medium with higher index (optically more dense), it bends toward the normal. In contrast, when ni . nt , (from optically more to less dense) it bends away from the normal. There are some special cases where further thoughts on Snell’s law are necessary. To explain this, let us suppose that incident light coming from a point source located below an interface travels across the upper medium of lower relative index as shown in Figure 2.5. When ui ¼ 0, that is, an incident normal to the surface ray is transmitted straight across the boundary without change of direction (case 1). However, as the incident angle ui becomes larger and larger, the angle of refraction becomes large. (case 1)

(case 2)

(case 3)

(case 4)

qt nt

qt

qr = qi = 0

nii

qi

S FIGURE 2.5 Various types of refraction phenomena.

qi

90° qc

qi

qi

36

Optomechatronics

TABLE 2.1 Refractive Index of Various Materials Material Vacuum Air (208C) Water vapor Optical glass Quartz Crystal Plexiglas Lucite Zinc crown glass Light flint glass Heavy flint glass Heaviest flint glass Sugar solution (30%) Polymer optical fiber (polymethyl ethacrylate PMMA) Polymer optical fiber (polystyrene) Polymer optical fiber (polycarbonates)

Refractive Index (n) 1.0000000 1.000292 1.00024 1.5 2 01.85 1.4868 2.0 1.488 1.495 1.517 1.575 1.65 1.89 1.38 1.49 1.59 1.5 2 1.57

Finally, there will be a critical angle of incidence that will make ut ¼ 908 (case 3). This angle is called the “critical angle, uc ” which is given by

uc ¼ sin21

nt ni

ð2:3Þ

When the angle of incidence ui . uc , the incident ray exhibits total internal reflection, as shown in case (4). This type of reflection is effectively used for fiber optic transmission, which will be discussed at the end of this chapter. The refraction index of some materials is listed in Table 2.1.

Lenses Figure 2.6 shows when a point source S produces a spherical wave propagating toward a convex or concave lens. The waves expand and intercept the thin lens. In case of the convex lens, the wave fronts contract and converge to a point I, where real image is formed. For the concave lens, the wave fronts expand further so that a virtual image I is created at the backside of the lens. This point is known as a focus of the bundle of rays. This implies that the change in the curvature of the wave fronts occurs due to the refracting power of the lens. One other thing to note is that, in both lenses,


37

S

I image point

object point

S object point I image point

(a) convex lens

(b) concave lens

FIGURE 2.6 Interference of a spherical wave with two typical lenses.

if a point source is placed at the image point I, the corresponding image will be located at S. These two points S and I are called “conjugate points”. Refraction at a Spherical Surface A spherical surface is commonly used as a lens surface and can be either concave or convex. Figure 2.7 shows a concave surface of radius R where two rays from a point source S are emanating as indicated by an arrow. One is an axial ray impinging a spherical interface centered at C, which is normal to the surface at its vertex point V. The source is located at a distance so from V. The other ray incident at a point P with an angle ui is refracted with an angle ut , as indicated by an arrow. According to Snell’s law, this ray refracts with the relation, n1 sin ui ¼ n2 sin ut

ð2:4Þ

qt

P

h

qi j

V

n2

s0

n1

FIGURE 2.7 Refraction at a spherical interface.

si

j¢ S R

α I

C

38

Optomechatronics

These two rays appear to be emanating from a point I which is called the image point, whose distance is si from V. The exact relationship between so and si is rather complicated to derive, but we will simplify by assuming ! ! SP ¼ SV

and

! ! IP ¼ IV

ð2:5Þ

by using small angle approximation i.e., sin u . u; cos u . 1: From this relation, it follows from Equation 2.4 that n1 ðw 2 aÞ ¼ n2 ðw0 2 aÞ

ð2:6Þ

Taking the tangents for the angles in the above equation and utilizing the assumption given in Equation 2.5, Equation 2.6 further leads to n1 n n 2 n2 2 2 ¼ 1 ð2:7Þ so si R It is noted that the object distance, so is positive whereas si , the distance of the image (virtual), is negative and R is negative according to the sign convention for spherical refraction surfaces and thin lenses. This equation then can be generalized to include the case of a convex surface. The general form of Equation 2.7 can be expressed as n1 n n 2 n1 þ 2 ¼ 2 ð2:8Þ so si R The above equation is valid for rays that are incident to the surface near to the optical axis such that w and h are small. These rays are known as paraxial rays. The optics dealing with such rays arriving at the paraxial region is called Gaussian optics, named after K. F. Gauss (1777 –1855), the first to develop this formulation. In the preceding discussion, we have dealt with a single spherical surface. In order to relate the source points and image point P we now apply this method to determine the locations of the conjugate points for a thin lens, which represents the thin-lens equation. In this case, two refractions at two spherical surfaces are involved. Referring to Figure 2.8, the first refractions at a spherical interface of radius R1 yields n1 n n 2 n1 þ 2 ¼ 2 so1 si1 R1

ð2:9Þ

where so1 is the distance of the source S from the vertex point V1 and si1 is the distance of the image from the vertex point V1 : At the second spherical surface of radius R2 n2 n n 2 n2 þ 1 ¼ 1 ð2:10Þ so2 si2 R2 Here, the image point P0 acts as a source object point for the second surface, which is considered the second object. Now since the lens thickness is


39

R1

R2 P'

P C2

S

s i1

n1

V1

so1

O n2

V2

n1

C1 si2

d

so2

FIGURE 2.8 A point source passing through a spherical lens having two different spherical interfaces.

negligible for a thin lens, i.e., d ! 0, so2 becomes so2 ¼ 2si1

ð2:11Þ

Combining Equation 2.9, Equation 2.10 and Equation 2.11 leads to 1 1 n 2 n1 þ ¼ 2 so si n1

1 1 2 R1 R2

ð2:12Þ

where so1 ¼ so and si2 ¼ si are substituted. It is noted that so and si is measured from either the vertices V1 and V2 or the lens center, O: When a source is located at infinity, we can see that the image distance becomes the focal length f : The focal length of the thin lens is therefore defined as the image distance for an object at infinity. In this case, Equation 2.12 can be rewritten as 1 ðn 2 n1 Þ 1 1 2 ¼ 2 f n1 R1 R2

ð2:13Þ

This equation is referred to as the lens maker’s formula. When a surrounding medium is air, n1 becomes approximately unity, i.e., n1 . 1: In this case, the lens formula becomes 1 1 1 ¼ ðn2 2 1Þ 2 f R1 R2 By using Equation 2.12 and Equation 2.13, the thin lens equation can be expressed as 1 1 1 þ ¼ so si f

ð2:14Þ

40

Optomechatronics

Equation 2.14 is called the Gaussian lens formula. In this formulation we have assumed that all light rays make small angles with the optical axis, which were termed paraxial rays. To help understand the formation of the focal point F, we use Figure 2.9, which illustrates how wave fronts of plane waves pass through the convex and concave lenses. It can be seen that the thicker portion of the lenses causes light to be delayed, thus resulting in convergence for a convex lens and divergence for a concave lens as shown in Figure 2.9a and Figure 2.9b, respectively. In the case of the convex lens, all rays that pass through the lens meet at the focal point F, whereas in the case of the concave lens all the rays that pass through diverge afterwards, as if all the rays appear to emanate from the focal point F: This focal point is called the second of image-side focus. Figure 2.10 summarizes the effect of positive and negative lenses on incident plane wave. The convex lens causes a divergent ray either to converge or diverge less rapidly. In particular, in a situation when a point source S is located at the focal point, the refracted ray becomes parallel, as shown in Figure 2.10d. A concave lens causes a convergent ray either to converge (not shown here) or to diverge (Figure 2.10b). It causes a divergent beam to become more divergent as shown in Figure 2.10c to e. When a convergent beam incident to the lens is directed toward the focal point F, it becomes parallel to the optical axis, as can be seen in Figure 2.10a. As we can see, a convex lens makes a convergent beam converge more rapidly or makes a divergent beam diverge with less speed or converge. In contrast, a concave lens behaves in exactly the opposite way. The output beam properties shown here are made only by a single lens with various angles of incident beam. The output beam properties may be made different depending on how we make the incoming beam incident to the lens. Also, they may be made variable depending on what types of lens or combination of lenses we use in order to manipulate a given incident beam. All of these concerns are involved with optical system design. The beam expander is a good illustration that we can combine two lenses in a proper way to make a beam expand with a desired magnification, as

plane wave

plane wave

F

(a) convex lens FIGURE 2.9 A plane wave transmitting through a lens.

F

(b) concave lens


41 F

F

(a)

F

.

F

s

s

F

F

F

(b)

s’

s’

s

s’ F

s

(e)

F

.

s’

(c) s

(d)

F

F

s

F

F

F

s

s’ F

F

.

s’ F

F

s F

s’

F

concave lens

convex lens FIGURE 2.10 The effect of location of point light source.

shown in Figure 2.11. The figure shows two cases of beam expanding. The first one uses two convex lenses and the other one uses one concave and one convex lens. In either case, the lenses have a common focal point. Depending on the focal length of the lenses, the beam diameter at the output is made different. By similar triangles, it is seen that D2 ¼

f2 D f1 1

ð2:15Þ

This implies that a desired output beam diameter D2 can be easily obtained by properly choosing those three variables.

L2

L1

L1 D1

D2

D1

D2 f1

(a)

f1

f2

L2

(b)

FIGURE 2.11 Transmissive beam expanding by combination of two lenses.

f2

42

Optomechatronics

An object is normally a collection of point sources. To find its image, it is desirable to locate the image point corresponding to each point of the object. However, it is sufficient to use a few chief rays in order to determine the object image. To illustrate this, ray diagrams for convex and concave lens chief rays are shown in Figure 2.12. It can be seen that the use of this diagram can facilitate the determination of the location, size, and orientation of an image produced by a lens. The size and location of an image is of particular interest, since they determine magnification. Following the sign convention, the transverse distance above the optical axis is considered positive, while the distance below the axis negative, as can be seen for the integrated image. It follows from the figure that the transverse magnification is expressed by MT ¼

hi s ¼2 i ho so

ð2:16Þ

where ho and hi are the transverse distance, height of the object, and that of its image, respectively. The minus sign for si accounts for an inverted real image as formed by a single thin lens. From Figure 2.12a we can see that in case of a convex lens, the sign and magnitude of the MT depend upon where

A ho

Fi Fo

B

A′

O

f

hi B′

f si

so (a) real image formed by a convex lens

A B′

ho B

Fo

so

A′

hi

O si

(b) virtual image formed by a concave lens FIGURE 2.12 Location and transverse lateral magnification of object image.

Fi


43

an object to be image is located. When it is located within the range 1 . so . 2f , 21 , MT , 0: In case of a diverging lens, shown in Figure 2.12b, we can see MT . 0 for si , 0, since in this case all images are virtual and erect. They are always smaller than the object, and lie closer to the lens than the object. The discussed characteristics of the images of real objects are summarized in Table 2.2. When the object is located in front of a convex lens in the range 1 . so . 2f , MT becomes 21 , MT , 0: When it is located so ¼ 2f , it becomes MT ¼ 21: When it is located in the range f , so , 2f , the image is real and inverted and MT becomes magnified MT , 21: Let us take some examples for thin lenses. When an object is located 6 cm in front of a convex lens having focal length 10 cm, the location and magnitude of the image to be formed can be calculated by using the formulas in Equation 2.14 and Equation 2.16, respectively. The image location is obtained by si ¼

6 £ 10 ¼ 215 6 2 10

which indicates that the image is virtual and lies to the left of the lens. The magnification of the image is obtained by MT ¼ 2

si ð215Þ ¼2 ¼ 2:5 so 6

which indicates that the image is erect. In a similar way, the location and size of the image of an object located 10 cm in front of a concave lens of focal length 6 cm can be obtained as follows: location image size

si ¼

10 £ ð26Þ 260 ¼ 10 2 ð26Þ 16

MT ¼ 2

si ð260=16Þ 3 ¼2 ¼ so 10 8

The above simple calculations show that we can vary magnification of an image formed by a single lens by changing its distance from an object. If the lens moves toward the object, by d lying within a range, the image will become larger, but if shifted away from the object, it will become TABLE 2.2 Image Characteristics of Real Objects Lens Convex

Concave

Object Location 1 . so . 2f so ¼ 2f f , so , 2f so , f Any

Image Characteristics

Remarks

Real inverted 21 , MT , 0 Real inverted MT ¼ 21 Real inverted MT , 21 Virtual erect MT . 1 Virtual erect 0 , MT , 1

si . 0 si . 0 si . 0 si , 0 si , 0

44

Optomechatronics

h f

f

h' f < s o < 2 f so - d

d

si

h h'

f

f so = 2f

d

h' s o = 2f

si

f

h

h'

f so + d

2f < s o > (∆q)min

(b) q ≈ (∆q)min

FIGURE 2.46 Images formed due to diffraction for two different angular separating of sources.

Previously, we have dealt with the case when the incident wave fronts of light make a right angle with the plane of grating surface but here will deal with an arbitrary incident angle. The common form of diffraction grating is an amplitude grating with a rectangular profile of transmittance as shown in Figure 2.47. At its surfaces, light modulation occurs due to the alternate opaque and transparent regions. This can be obtained by vacuum deposition of many opaque (metallic) strips on a glass plate, which forms a periodic array of narrow slits of equal distance. If parallel light beam strikes this grating with an oblique incident angle, ui , diffracted beams from successive slits interfere constructively and destructively. Referring to the right handside of Figure 2.47b, the net path difference between waves from successive slits is given by

d1 2 d2 ¼ dðsin um 2 sin ui Þ

ð2:87Þ

mth order m= +1 m= +1 m= 0 diffracted light m= 0 m= −1 incident light m= −1

(a) diffraction geometry

(b) diffracted light

FIGURE 2.47 Diffraction of light by a transmission amplitude grating.

δ2

qi

qm

δ1 d


91

R B

grating lens

R B

white 1 light

f

m = +2

m = +1

m=0 B R B R

m = −1

m = −2

focal plane FIGURE 2.48 Spectra produced by white light.

from which the grating equation can be expressed as dðsin um 2 sin ui Þ ¼ ml;

m ¼ 0; ^1; ^2…

ð2:88Þ

for all waves in phase. In Figure 2.47b only three diffracted waves ðm ¼ 0; ^1Þ are plotted for simplicity. We can see that the diffraction angle differs from diffraction order. When m ¼ 0, the diffraction angle of the zeroth order of interference becomes identical to that of the incident wave, i.e., um ¼ ui : At this direction, the diffracted wave produces the maximum irradiance for all wavelengths l, as we have seen in the previous sections. Higher orders, however, the direction of um varies with wavelength l. These properties are used to measure the wavelength of light and do spectral analysis. Figure 2.48 illustrates how white light can be decomposed into different colors having different wavelength by a diffraction grating. To achieve this a collecting lens is employed to focus each collimating beam of wavelengths to a point in the focal plane. R and B denote the red (R) and blue (B) ends of the light color spectrum. Figure 2.49 shows a reflection phase grating. The gratings have highly reflecting faces of periodically varying optical thickness. Those modify the phase of incident light while the amplitude remains unchanged. This is in contrast with the amplitude grating. As shown in the figure, optical path difference is expressed as given in Equation 2.87 and therefore, the same grating equation as given in Equation 2.88 can be applied.

92

Optomechatronics incident light

qi

m = −1 m=0

diffracted light m = −1 m = +1 m=0

d1 d d2 qm

mth order

m = +1

(a) diffracted lights

(b) diffraction geometry

FIGURE 2.49 Reflection phase grating.

Optical Fiber Transmission Ever since the concept of optical fiber has been used as a means of transmission of signal or data, they have been widely used as transmission media in optical measurement as well as optical communication. In addition, those are becoming extremely important for laser beam delivery for instrumentation and surgical applications. This is due to low, loss transmission, high capacity of carrying information and immunity to electromagnetic interference. An optical fiber is composed of two concentric dielectric cylinders; the core (inner cylinder) and the cladding (outer cylinder). The core having a refractive index nco is clad with the cladding of lower refractive index ncl i.e., nco . ncl : The optical fiber is made based on the phenomenon of total internal reflection within this fiber structural arrangement. Let us elaborate this principle in more detail. As shown in Figure 2.50, rays strike the interface of the core and cladding. Applying Snell’s law at the interface, we have nco sin uco ¼ ncl sin ucl

ð2:89Þ

From this relation we can easily see that, when ucl ¼ 908, the refracted ray travels along the boundary as shown in the middle of Figure 2.50a. The corresponding ucl that yields this phenomenon is called the critical angle uc as discussed previously. As indicated in the right hand side of Figure 2.50a, when uco . uc , all rays striking the interface will be totally reflected back into the core by means of many internal reflections. Thus, the fiber becomes a light guide, as shown in Figure 2.50b. This critical angle uc can be determined from


93

θt

ncl nco

θi

θi θi

θc

when θi = θc critical angle

when θi < θc refraction + reflection

θi

θi

when θi > θc total internal refraction

(a) reflection and refraction at various ray incident angles ncl nco θi

θ > θc refracted ray

ni cladding

(b) total internal reflection of rays within an optical fiber FIGURE 2.50 Light propagation in optical fiber.

Equation 2.89

uc ¼ sin21

ncl nco

To yield this critical angle, there will be a range of an angle of incidence for a ray to enter the core. Let us determine the range of the incident angle. If the fiber is surrounded by a medium having a refractive index ni , Schnell’s law gives for some refractive angle u. ni sin ui ¼ nco sin u

ð2:90Þ

If it is assumed here that for this incident angle ui , the refracted ray hits the interface with angle u, as indicated in Figure 2.50b. The angle u to be greater than uc must satisfy the following condition cos u $

ncl nco

ð2:91Þ

From Equation 2.90 and Equation 2.91 we see that the condition for the occurrence of total internal reflection is limited by sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi nco ncl 2 12 sin ui # ni nco

94

Optomechatronics

Since in air ni < 1:0 the maximum ui condition is given by qffiffiffiffiffiffiffiffiffiffiffiffi ð2:92Þ ½ui max ¼ sin21 n2co 2 n2cl qffiffiffiffiffiffiffiffiffiffiffiffi If the above quantity n2co 2 n2cl is defined as the numerical aperture (NA) of the fiber, the above equation be rewritten as ½ui

max

¼ sin21 NA

For example, if nco and ncl are 1.51 and 1.49, respectively, then ½ui max of a fiber in air becomes approximately 14.188. Therefore, NA of the fiber can be an indication of the maximum angle of incidence. Fiber with a wide variety of numerical apertures ranging from 0.2 up to 1.0 is being used, depending on applications. Figure 2.51 depicts the three major types of fiber configuration being used currently. The single mode index fiber depicted in Figure 2.51a has a very narrow core, typically less than 10 mm in diameter. Therefore, it can have only a single mode in which the rays travel parallel to the center through the fiber. The multimode step index fiber shown Figure 2.51b has a core of relatively large diameter, 50 to 200 mm, making it advantageous to launch light into

(a) single-mode step-index fiber

(b) multi-mode step-index fiber

(c) multi-mode graded-index fiber FIGURE 2.51 Major refraction index configurations of optical fibers.


95

fibers using LED sources. In addition, it can be easily terminated and coupled. The multimode graded index fiber as shown in Figure 2.51c has a core diameter of 20 to 90 mm. The core employs a nonuniform refractive index, which has higher value at the center decreasing parabolically toward the interface. Due to this index variation, the rays smoothly spiral around the center axis instead of a zigzag path, as seen for the case of the multi mode index fiber. Power losses occur when light propagates through the fiber. It depends upon fiber material, scattering by impurities and defects for glass fiber and absorption for plastic fiber. The attenuation in optical power is usually defined by Pl ¼ 10 log10

Wo Wi

ð2:93Þ

where Pl is expressed in dB, and Wi and Wo are the input and output power, respectively.

Gaussian Beam Optics In modern optics, lasers are the indispensable light source for measurement, communication, data recording, material processing and so forth. In Table 2.4, various lasers are compared with regard to the type, state, wavelength and power range. In most of their applications, it is necessary to shape the laser beam using optical elements. The laser for most cases operates and emits a beam in the lowest order transverse electromagnetic (TEM) mode. In this mode, the beam is a perfect plane wave and has a Gaussian wave front whose characteristics are radically symmetric and can be described by Gaussian function. Although in actual lasers it is not perfectly Gaussian, in Helium Neon (HeNe) and argon-ion lasers it is very close. Figure 2.52 illustrates

TABLE 2.4 Comparison Between Various Laser Sources Type

State

Wave Length (nm)

He–Ne Argon CO2 Semiconductor Excimer (KrF) Nd–YAG He–Cd Ruby

Gas Gas Gas Solid Gas Solid Gas Solid

632.8 488.0, 514.5 10600 850 –910 248 1047–1064 442, 325 694.3

Power Range 0.5– 50 mW 0.5– 10 W 0.5– 10 kW ,10 kW 1–300 W 1 mW– 10 kW 50–150 mW Pulse (,400 J) (0.2 —5 ms)

96

Optomechatronics

r

1 irradiance contour e2

r(z)

I0 e2

I0

r0

I0 e2

0

q

2r0 zr

z

FIGURE 2.52 Spot size variation with propagation distance: Gaussian beam.

the characteristics of Gaussian laser beam such as beam irradiance and divergence profile. As can be seen from the figure, the beam has maximum beam intensity I0 at its center and decreases with distance from the axis. The beam diameter slightly differs depending upon how it is defined. Three widely used definitions are: (1) full power beam diameter, (2) half power beam diameter, and (3) 1=e2 excluded power beam. The full power, beam diameter is the diameter of the laser beam core that contains the total beam power, except the power portion outside 8s which is eight times the standard deviation of the distribution. This corresponds to 99.97% of the total power. The half power beam diameter contains 50% of the total beam power but excluded the rest of the 50% power lying outside 2:35s: The 1=e2 excluded beam diameter is the diameter that contains the beam power exclusive of 13.5% ð1=e2 Þ of the total power. This power corresponds to 4s: This diameter is commonly adopted when referring to the diameter of a laser beam. The divergence angle u of a Gaussian laser beam as indicated in the figure is the angle by which the beam spreads from the center of the waist. This angle is essentially the angle of the asymptotic cone made by 1=e2 irradiance surface. Let us consider the geometry of this beam spread. If rðzÞ is denoted by the radius of the contour at the distance z from the beam waist, it can be expressed by " rðzÞ ¼ r0 1 þ

lz pr20

!2 # 12

ð2:94Þ

where r0 , called the radius of beam waist, is the radius of the irradiance contour at the location where the wave front is flat. From Equation 2.94 we can see that r0 is the radius of the beam waist at z ¼ 0 and that rðzÞ is symmetrical about z ¼ 0: Also, it can be seen that at rðzÞ ¼ r0 the radius of curvature rðzÞ is infinite. For large value of z, that is, at the far field, the equation is largely determined by the term ðlz=pr0 Þ2 , since !2 lz q1 ð2:95Þ pr20


97

Using the asymptotic line approaching z ¼ 0 and small angle assumption, we can obtain asymptotic angle u from Equation 2.95 tan21 u . u .

l pr0

ð2:96Þ

This angle is measured at a distance far away from the laser that is z q zr where zr stands for the Rayleigh range which is equal to pr20 =l: The range is defined by the distance from the center of the waist to the propagated location at which the beam wave front has a minimum radius of curvature rmin ðzÞ on either side of the beam waist. Let us have an example to illustrate how much the laser beam will spread for a given HeNe laser having wavelength 632.8 nm and diameter 1.0 mm. From Equation 2.96, we have

u.

632:8 £ 1026 ¼ 4:03 £ 1024 rad pð0:5Þ

Once this is obtained, beam spread rðzÞ can be easily determined at any location of z. Equation 2.94 can be applied to the case when a Gaussian laser beam enters a lens with small divergence angle and is focused at its focal point. Since the lens is located in the far field relative to the focus point as compared with the size of the small focused beam, the focused beam radius can be approximated by r0 ¼

lf pD

ð2:97Þ

where D is the aperture diameter of the lens.

Problems P2.1. Suppose that light wave is incident on the surface of a glass having refractive index nt ¼ 1:5 with an angle u ¼ 458, as shown in Figure P2.1.

45° qg

qa

FIGURE P2.1 A ray transmitting through a glass.

glass

98

Optomechatronics

(1) Determine the refractive angle ug : (2) What will be the transmittive angle ua ? P2.2. A meniscus convex lens made of a glass ðnt ¼ 1:5Þ has its radius of curvature of 28 cm at one side and 50 cm at the other side as shown in Figure P2.2a.

1.5m

1.5m

(a)

(b)

FIGURE P2.2 Meniscus convex lens.

(1) Determine the focal length of this lens. (2) If the same lens is flipped over in respect to the direction of the incident beam as indicated in Figure P2.2b, determine the focal length. (3) In both cases, (1) and (2), determine the image point on the optical axis when an object is placed 1.5 m in front of the lens. P2.3. A bundle of parallel rays transmits through (a) convex lens (b) two lenses composed of a convex lens and a planar convex as indicated in Figure P2.3. Draw a ray diagram in each case in detail and discuss the results of the ray path. f and f 0 are the focal lengths of the two cases.

L1

L1 L2

f

(a)

(b)

f′

FIGURE P2.3 Ray diagram.

P2.4. Design a beam expander using any combination of two types of lenses (concave and convex). The expander is required to expand a beam of 1.5 cm to 9.0 cm. Assume that the absolute value of the focal length of the lens is less than 9.0 cm.


99

P.2.5. There are several ways to measure the effective focal length of a lens. One way is to use a collimator. Draw a configuration of this focal collimator, a focal length measurement system, and explain how it works in principle. P2.6. The beam expander is a frequently used device to expand beam diameter. The nonfocusing reflective beam expander is practical for most applications due to simplicity of its configuration. Draw three such beam expanders and explain how they work. P2.7. The image size of an optical system can be conveniently described by a lateral magnification factor, m ¼ h0 =h, where h and h0 are the object height and image height, respectively. If so and si are the object and image distances from the lens, respectively, prove that so and si can be expressed by so ¼

1 21 f m

si ¼ ð1 2 mÞ f P2.8. Consider the astigmatism discussed in Figure 2.23. Suppose that rays focus as a line at fT and fs as shown in Figure P2.8. If the lens is assumed to have a tangential focus ð fT Þ 17.8 cm and a sagittal focus ð fs Þ, 19.2 cm, what will be the diameter of the circle of least confusion. The circle is assumed to be located at the point having an average of the optical powers of the two focal lengths. The diameter of the lens is D: lens

circle of least confusion

D fT

fs

FIGURE P2.8 Location of circle of least confusion.

P2.9. A telescope is an optical device that enlarges the apparent size of an object located at a distance away. Such a device can be configured by combining two thin lenses, L1 and L2, shown in Figure P2.9. The two lenses are separated by distance d, and f1 and f2 are the focal lengths of lenses L1 and L2, respectively. (1) Locate the focal point of each lens and draw optical paths by which light reaches the eye for the two cases shown in Figure P2.9a and b.

100

Optomechatronics L1

L1

L2

L2

eye

(a)

eye

(b)

d L1

d L3

L2

(c) FIGURE P2.9 Inverted and erected images in a telescope.

(2) In Figure P2.9a, the image obtained is inverted. In order to obtain the erected image, repeat the same problem (1) for an optical system shown in Figure P2.9c. In this case, what will be magnification? P2.10. Figure P2.10 shows an aperture placed in an optical system composed of two identical convex lenses. Discuss the effect of the aperture stop on distortion. lens

aperture stop FIGURE P2.10 The effect of aperture stop.

P2.11. Suppose we wish to grind the rim of the lens so that the line between the centers of curvature of two lens surfaces coincides with the mechanical axis, which is defined by the ground edge of the lens. Given an accurate tubular tool on a rotating spindle where the lens can be fasten with wax or pitch, (1) configure an optical system for this grind system and (2) describe in detail how the system works.


101

P2.12. Consider that an unpolarized light wave propagates through a polarizer with vertical transmission axis and then a second polarizer with its transmission axis making an angle of 608 with the vertical axis, as shown in Figure P2.12. What will be the polarization angle and the intensity of the transmitted light?

60°

unpolarized light

polarizer

polarizer

FIGURE P2.12 Polarizing light with a polarizer.

P2.13. Counting the number of fringes observed in a Michelson interferometer is used to measure a displacement of an object. When the object is displaced by 0.81 mm, a shift of 2000 fringes is observed. Compute the wavelength of the light source. P2.14. Figure P2.14 shows an arrangement of Fabry-Perot interference. The arrangement utilizes a collimation lens between the source and the interferometer. Discuss what types of fringes will occur. Give the reasons for this. d

S

source

collimating lens

interferometer focusing lens screen

FIGURE P2.14 Fabry-Perot interferometer.

P2.15. When an incident beam contains two wavelengths which are slightly different from each other, the transmitted irradiance of each fringe will be shown in Figure P2.15. If one wavelength l0 is 500 nm and the other differs only by Dl ¼ 0:0052 nm, what will be the spacing of the Fabry-Perot

102

Optomechatronics T

fringe 1 fringe 2

1.0

0.5

∆js ∆l l0

∆ϕ

FIGURE P2.15 Two overlapping wavelength components.

etalon when the mth order of one component coincides with the ðm þ 1Þth order of the other? P2.16. A Fabry-Perot etalon has a reflection coefficient of rr ¼ 0:92 and a plate separation of 1.5 cm. The incident light has two slightly different wavelengths around 490 nm. Determine (1) coefficient of finesse (2) maximum order of interference (3) minimum resolvable wavelength difference Dl: P2.17. The optical system shown in Figure P2.17 called “Lloyd’s mirror” consists of a point source S, a plane mirror M placed near the source, and image plane Si :

point source S

image plane Si

M mirror FIGURE P2.17 An optical system composed of point source S and a plane mirror M.

(1) Draw the relevant optical paths in order to describe the image in the image plane Si : (2) What type of image will appear in the plane Si : P2.18. Suppose that a parallel beam of 632.8 nm light propagates through a single slit. If the distance between the slit and screen is 1 m, determine the width of the slit in order to have the spread of the central maxima, 30 cm.


103

P2.19. Suppose that a coherent light wave having the wavelength l ¼ 640 nm passes through double slits whose separation distance is 0.2 mm. The screen is located 1 m away from the slits. Calculate the width of the central maxima. P2.20. In double slit diffraction, the relationship between separation and width is given by d ¼ 5w: (1) Plot the interference term vs. a (2) Plot the diffraction term vs. b (3) Plot the irradiance vs. spatial angle, u P2.21. From Figure 2.40 and Figure 2.41, we can see that the waves appear much sharper for the case of multislits than that of double slits. Give a physical reason in some detail. pffiffi P2.22. Show pthat ffiffi at Rayleigh range zr the beam spreads by a factor of 2, that is, rðzr Þ ¼ 2r0 : P2.23. Suppose that a laser beam spreads out from the beam waist. Determine the beam radius at a location z ¼ 80 m for a beam of radius 0.3 mm laser diode l ¼ 810 nm. P2.24. Suppose that a HeNe laser of radius 5 mm enters a lens having focal length 50 mm. Compute an approximate focus spot diameter.

References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]

Andonovic, I. and Uttamchandani, D. Principle of Modern Optical Systems, Vol. 1. Artech House, Boston, MA, 1989. Bernacki, B.E. and Mansuripur, M. Causes offocus-error feedthrough in optical-disk systems: astigmatic and obscuration methods, Applied Optics, 33:5, 735–743, 1994. Fan, K.C., Chu, C.L. and Mou, J.I. Development of a low-cost autofocusing probe for profile measurement, Measurement Science and Technology, 12:12, 2137–2146, 2001. Gasvik, K.J. Optical Metrology, 2nd Ed., Wiley, New York, 1996. Heavens, O.S. and Ditchburn, R.W. Insight into Optics, Wiley, New York, 1991. Hecht, E. Optics, 4th Ed., Addison Wesley, Reading, MA, 2001. Mansuripur, M. Classical Optics and its Applications, pp. 222– 239, Cambridge Press, Cambridge, 2002. Code V, Optical Research Associates, 2001. O’Shea, D.C. Elements of Modern Optical Design, pp. 24 – 75, Wiley, New York, 1985. Pedrotti, F.J. and Pedrotti, L.S. Introduction to Optics, pp. 43 – 348, Prentice Hall, Englewood Cliffs, NJ, 1992.

104 [11]

Optomechatronics

Pluta, M. Advanced light microscopy, Principles and Basic Properties, pp. 110 – 129, Vol. 1. Elsevier, Amsterdam, 1988. [12] Rossi, B. Optics, pp. 122– 149, Addison-Wesley, Reading, MA, 1957. [13] Smith, W.J. Practical Optical System Layout: and Use of Stock Lenses, McGraw-Hill Professional, New York, 1997.

3 Machine Vision: Visual Sensing and Image Processing CONTENTS Image Formation................................................................................................ 109 Imaging Devices................................................................................................. 112 Image Display: Interlacing ....................................................................... 115 Image Processing ............................................................................................... 115 Image Representation........................................................................................ 116 Binary Image............................................................................................... 117 Gray Scale Image ....................................................................................... 117 Histogram.................................................................................................... 118 Histogram Modification............................................................................ 118 Image Filtering ................................................................................................... 121 Mean Filter .................................................................................................. 123 Median Filter .............................................................................................. 124 Image Segmentation .......................................................................................... 127 Thresholding............................................................................................... 127 Iterative Thresholding ............................................................................... 131 Region-Based Segmentation ..................................................................... 133 Edge Detection ................................................................................................... 136 Roberts Operator........................................................................................ 138 Sobel Operator............................................................................................ 140 Laplacian Operator .................................................................................... 142 Hough Transform .............................................................................................. 148 Camera Calibration............................................................................................ 152 Perspective Projection ............................................................................... 155 Problems.............................................................................................................. 165 References ........................................................................................................... 172

Visual sensing is prerequisite for understanding the structure of a 3D environment, which requires assessing the shape and position of objects. The sensing is to acquire the image of the environment through a visual or an 105

106

Optomechatronics

imaging system, usually a camera system. When the system sees the scene containing objects, the amount of light energy it acquires depends upon a number of factors, such as shape, optical properties of objects, background, and illumination conditions. Often, these factors are critical to determine the amount of light impinging on the object’s surface. Depending upon these factors, the camera system takes in a certain amount of light reflected and scattered from the scene through its optical system. The received light energy in general determines the intensity of light sensed by the camera, which is the physical quantity that the vision system handles to interpret the scene. In proportion to the intensity of the received light, the imaging hardware device of the camera transforms the physical quantity into an electrical signal. This electrical video signal is captured, processed, and digitized by the frame grabber in a discrete fashion so that a computer can understand the digitized values. After digitization, this signal is sent to a computer for image processing. Utilizing the acquired image, the image processing algorithms carry out several steps of image modification and transformation with a view to extract the information necessary to understand the environments. This information includes a shape, a surface texture, a size of objects, location, orientation of objects, and distance from the vision sensor to objects. Based on this information, a so-called high level algorithm classifies the objects into several groups of dissimilar objects and finally carries out the pattern recognition tasks, if necessary. Machine vision technique comprises a series of these tasks described above. The complete steps needed for image processing, analyzing, and understanding are depicted in Figure 3.1. Let us reiterate in more detail the image analysis part. The objective of the image analysis is to understand the nature of an image by acquiring and interpreting the information necessary to solve a variety of engineering problems. It is composed of several hierarchical steps: (1) (2) (3) (4)

Preprocessing Intermediate processing Feature extraction Scene understanding.

Preprocessing is a stage of selecting an interested region of the image and enhancing the acquired image to be analyzed by eliminating artifacts from the image, filtering noise in the image, making image quantization such as thresholding, and so on. In many cases, the original image acquired usually contains noise, broken lines, blurs, low contrast, and geometric distortion caused by the lens. These unwanted defects make it difficult to accurately extract high-level information such as points, edges, surfaces, and regions. It is, therefore, necessary to eliminate or reduce noise and blurring for the improvement in contrast and the correction of distortion. The processing of carrying out these tasks is called image preprocessing and should yield an improvement to the original image.

Machine Vision: Visual Sensing and Image Processing

107

FIGURE 3.1 Steps for visual information processing.

Since the image usually contains a great deal of pixel data, it is difficult to directly use them to analyze the nature of the acquired scene. To this end, the task in the next stage concerns the extraction of the higher-level information mentioned above, based upon the results obtained by preprocessing. Utilizing the preprocessed image, the intermediate processing stage achieves edge detection, segmentation, and transformation of the image into another domain. Of course, the quality of preprocessing greatly affects that of this stage. In the step for feature extraction, selection of the features is carefully made by considering which information will be useful to best represent the characteristics of objects in the segmented image. The purpose of using the features of the image, rather than the raw image itself, is to make searching and acquiring the representative information easier for processing, thus reducing the amount of time for image analysis. There are many types of features used to extract information as listed in Table 3.1. The frequently used features are geometrical shape, texture, and color selection as clarified in the table. Feature extraction is, therefore, the process

108

Optomechatronics

TABLE 3.1 Commonly Used Feature Sets for High Level Image Processing Feature Set Geometrical shape

Parameter

Description

Center of mass

Area moment

Perimeter

Width

h w

Vertex

Aspect ratio

b a

Principle axis

Elliptic ratio

b a

Circularity Gradient Symmetry Histogram

Orientation Measure for symmetry

Histogram shape (energy, entropy)

Color

Color difference

Color brightness

Texture

Power (spectrum)


109

of acquiring hyper-level image impunity, which is application dependent. In the final stage, these extracted features are utilized for the interpretation of the scene, which is an indispensable stage for inspection, measurement, and recognition of the objects. There are two types of the vision system: active and passive, which are needed to capture the image of a 3D environment. Active vision acquires a 3D image by projecting the light energy onto a scene. In contrast, the passive method utilizes images acquired in the visible region of light spectrum with no use of any external light projection. Due to this advantage, this method has been applied to various types of environments. In this chapter, we will treat the principle of an image formation, an imaging device or a sensor that acquires images, and some fundamental image processing techniques, which include filtering, thresholding, and edge detection. Camera calibration will also be discussed, whose process enables us to interpret the acquired image output in terms of a real world coordinate system. Throughout this chapter, we will limit ourselves to a passive vision method, a monocular system, and a monochrome image.

Image Formation Light emits energy in the ultraviolet (UV), visible, and infrared (IR) portions of the electromagnetic spectrum. UV light is invisible with a very short wavelength in the range of 10 to 400 nm. Visible light is electromagnetic radiant energy with a short wavelength between 400 and 700 nm. In contrast, infrared light has a rather long wavelength between 0.7 and 100 mm. Brightness is related to the radiant flux of a visible light emitted from a source. Depending on how particular objects in the scene to be imaged are illuminated and how they reflect light, the imaging system presents the image of different brightness. For a given illumination condition and scene, the imaging system will present an image with brightness distributed over the scene in a specified manner. The distribution of brightness contains such important information as shape, texture of surface, and color. In addition to these, it will also contain the information on the position and the orientation of the objects, which can be obtained by determining the geometric correspondence between points in the image and those in the scene. The brightness of the imaged surface depends upon optical reflectance characteristics of imaging surface which are very critical to acquiring images. They are, however, not so simple to analyze since the surfaces reflect light in many different ways according to their texture and wavelength. As illustrated in Figure 3.2, there are three typical types of reflectance pattern. They are the specular surface, which reflects light at an angle equal to the incident angle, the Lambertian surface, which uniformly scatters light in such a way that the light has consistent luminance scattered without

110

Optomechatronics specular diffuse

incident light

specular

diffuse

FIGURE 3.2 Radiance depending on surface condition.

dependency on the view angle, and finally, the specular diffuse surface, which scatters incident light with some directivity. When a reflected light contains specular properties to some extent, it sometimes requires special attention and, therefore, normally needs a special illumination condition and imaging methods. Another important parameter to imaging is the field of view, since it determines the angle within which an imaging system can see. The angle is defined by the angle of the cone of directions encompassed by the scene to be imaged, and depends upon the focal length of the system. Normally, a telephoto lens has a narrow field of view due to its large focal length relative to the size of the object to be imaged, while an ordinary lens has relatively broader range of field of view. Before discussing sensing and image processing, let us consider how the image of an object is formed within the image plane and discuss the geometric correspondence between points in the scene and points in the image. There are two projection models to relate the image to the scene: perspective and orthographic. In the perspective projection shown in Figure 3.3a, each point in the image corresponds to a particular direction from which a ray passes through a pinhole. Let P denote a point in the scene with camera coordinates ðxc ; yc ; zc Þ, and Pi denote its image described in the coordinates ðxi ; yi ; zi Þ: If f is the distance of the image plane from the pinhole, we have zi ¼ f : From the optical geometry, the relation between the two coordinate systems are given by xi y f ¼ i ¼ xc yc zc

ð3:1Þ

and from this we have xi ¼ fxc =zc

yi ¼ f yc =zc

ð3:2Þ


111

image plane yi

xi

zi = f

xc

yc

Oi

object zc

P (xc, yc, zc) zc

Oc Pi(xi, yi)

(a) perspective model image plane yi

xi

Oi

xc

yc

P(xc, yc, zc) Oc

Qi(xi, yi)

object

zc Q(xc, yc, zc)

(b) orthographic model FIGURE 3.3 Image formation models.

This implies that as zc increases, the image point of the object point gets closer to the center of the image plane. Orthographic projection shown in Figure 3.3b involves the projections of two parallel lines to the optical axis on the image plane. Consider again a point Pc ðxc ; yc ; zc Þ in the object located at z ¼ zc : The perspective projection model given in Equation 3.1 can be rewritten as xi ¼ 2mxc

ð3:3Þ f where lateral magnification is defined by m ¼ 2 : zc When the depth of the scene is small relative to the average distance of the surface from the camera lens, the magnification m is almost constant. In this case, we can simplify Equation 3.3 by normalizing the coordinate system with m ¼ 2 1, and thus, the orthographic projection is defined by xi ¼ xc

yi ¼ 2myc

yi ¼ yc

ð3:4Þ

This equation implies that, unlike perspective projection, the orthographic projection is not sensitive to changes in depth and can ignore the depth information. This approximation can often be useful for the optical system whose focal length and camera distance are large to compare to the size of the object, as in the case for a microscope observing microparts.

112

Optomechatronics

Imaging Devices There are primarily three types of imaging sensors presently in use: a charge coupled device, a charge injection device, and a photo diode array. The most commonly used device is the charge coupled device (CCD) and, thus, we will describe it in brief. Since AT&T Bell Laboratories invented the CCDs in the late 1960s, the growth has been exploited in almost all application areas. They are currently being used as indispensable imaging detectors of visible, ultraviolet, and x-ray light, and have now become the central element in an imaging system. The CCD visual sensor is composed of an array of tiny photosensitive semiconductor devices called “pixels”. When light is incident on the devices, a charge is generated by absorption of photons and accumulated during a certain period of time. The amount of the charge accumulation is dependent on the intensity of light. The accumulated charge is stored and converted to an electrical signal to be used for image analysis when it is processed properly. To briefly understand how CCDs work, let us consider a unit of CCD cell called “pixel” as shown in Figure 3.4. A semiconductor (p type) has two different states of electrons: One is where the electrons are free to move around in response to an electric field and the other is where the electrons are prevented from moving around. When light enters the silicon surface and photons are absorbed, this causes electrons constrained in motion to move freely, leaving behind a hole, an occupied site. This generates charge, i.e., electron– hole pairs. The figure shows the state of the accumulated charges in a well by introducing an electric field with a voltage þ V through an optically transparent electrode. This state is a result of charge creation by incident light. The amount of the charge here represents the integration of light. After accumulating the charge during a specified integration time, incident lights

transparent electrical contact

+V

e– – e– e e– – e– e e–

depletion region FIGURE 3.4 Generation of charge packet by photons.

p-type semiconductor


113

the charge collected at each pixel needs to be transferred to the adjacent electrode for a sequential move to a readout port. Figure 3.5 illustrates a charge transfer in three phases, using seven electrodes and cycling three voltages (V ¼ 0, 5, and 10 V) on the rectangular grid of electrodes. In Figure 3.5a, the state of the charge due to CCD exposure to incident light is illustrated. A certain amount of charge is accumulated with the electrode voltage V ¼ 10 in the packets A3 and A6. When the voltages on the adjacent wells are increased from 0 to 10 V, the charges become shared between the electrodes E3 and E4 for the packet A3 and E6, and E7 for the packet A6, as shown in Figure 3.5b. Removing the voltages from E3 and E6, the charges in the packets A3 and A6 are completely removed and transferred to the respective adjacent electrodes as indicated in Figure 3.5c. The principle of the transfer of charge from pixel to pixel presented above can be applied to the case for the CCD array configured in Figure 3.6. The CCD system here contains a large pixel array, multiple vertical registers, and usually one horizontal shift register. The charge transfer flow is sequenced V1 applied voltage V2 V3

0 5 10 E1

pixel electrode

E2

E3

E4

E5

E6

E7

0 A3

10

A6

packet

packet

(a) end of exposure V1 V2 V3

10 0 5

A1

A7

A4

(b) charge transfer V1 V2 V3

5 10 0

A2

A5

(c) end of transfer FIGURE 3.5 Charge transfer in a “three phase” CCD.

114

Optomechatronics CCD array configuration

pixel

vertical shift register

output stage

image data

Horizontal CCD Shift Register FIGURE 3.6 Charge transfer configuration for read-out.

as follows. As indicated by dotted lines and arrows, the accumulated charges are transported down to the horizontal shift register by the vertical shift registers. The horizontal register collects one line at a time and moves each packet of charge in a serial manner far right to the output stage. This is called an interline transfer CCD method. Figure 3.7 shows a typical image signal pushed by shift registers. Image signal level is represented by a solid line ranged from 0 to þ 0.714 V, where þ 0.714 V indicates the white level while 0 V the black level. The horizontal synchronization intervals are denoted by the negative pulses with 10.9 m sec. The dotted lines before and after a pulse indicate the last pixel of a line and the first pixel of the next line, respectively. + 0.714V

last pixel of line

white level

video signal 0 – 0.286V

blanking level

first pixel of line

horizontal sync interval 10.9 ms

active line time 52.95ms 1 video line 63.49ms

horizontal line timing FIGURE 3.7 Horizontal synchronization.

black level


3 5

start of odd field frame 2 4

. . . .

477 479

. . . .

480 Active lines

retrace

in the interlaced frame

1

115

478 480 reset to even field frame

FIGURE 3.8 Interlaced image display.

Image Display: Interlacing Interlacing is a method of displaying an image signal by use of the alternate line field structure required by the RS-170 format standard. According to the format standard which is administered by the Electronics Industries Association (EIA), the image on the monitor has 480 lines divided into two fields of 240 lines. As shown in Figure 3.8, they consist of a field containing the odd numbered line depicted as solid lines and a field containing even number lines depicted as dotted lines. Each field is scanned in 1/60th of a second, so that a complete image can be scanned per 1/30 sec. The frequency of scanning is therefore 525 horizontal cycles per 1/30 sec. The time duration for a complete cycle is 63.5 m sec, the forward scan and return trace requiring 52.1 and 11.4 msec, respectively.

Image Processing The image signal sensed by CCD cells is transferred to the acquisition unit called a frame grabber. This captures video signals, buffers or stores the images, and simultaneously displays the image on an external monitor. Figure 3.9 is a simplified version of the grabber and shows its components. The major components are: a video multiplexer that takes in video signals up to N channels, an analogue to digital (A/D) converter that converts analogue

116

Optomechatronics

connected to PC/104 Plus

image grabber

CCD camera

video 1 video 2 …

. video n

video MUX

A-to-D converter amplifier

composite sync HSync VSync

On-board processing unit buffer memory

sync generator

clock syncronization

external trigger

BUS interface

internal clock PCI, ISA and PC104+ BUS Line

FIGURE 3.9 Architecture of an image grabber. Source: Datatranslation Co. Ltd (www.datx.com) and Meteor Co. Ltd (www.natrox.com).

video signal to digital signal, an onboard processing unit that processes various real-time tasks, a frame buffer that stores the image data and buffers its flow, and the interfacing unit that allows to transfer the processed image data onto a host computer (CPU) through bus lines such as PCI, ISA, PC 104þ, and so on. We will discuss some of the characteristic features of a CPU, a multiplexer, an A/D converter, and the data bus later in Chapter 4.

Image Representation Brightness of the image acquired by the camera is defined on every pixel within the CCD array, which is a function of spatial coordinates. To describe the brightness an image function is usually used, which is a mathematical representation of an image that a camera produces on a CCD cell. The function denoted by f ðx; yÞ defines the brightness of the gray level at the spatial location ðx; yÞ: In other words, the brightness of the image at the pixel point ðx; yÞ is represented by f ðx; yÞ: When digitized by the frame grabber, it can be represented by f ði; jÞ where i and j are the ith row and jth column of


117

FIGURE 3.10 Gray image and binary image.

the 2D CCD array cell, respectively. Two types of image will be considered here: binary and gray scale. Binary Image The binary images have two-valued gray levels and are produced from the gray scale image by using a threshold technique to segment the image into regions. Let us consider a 256 £ 256 image of a single object in the field of view of a camera, with a background having different brightness, as shown in Figure 3.10. The object and the background are seen to have approximately uniform brightness. Since, in this case, the object appears darker than the background does, it is easy to segment these into two regions, 0 for the object and 1 for the background. The image represented by this two-valued function is called binary image. Such an image can be obtained by thresholding the gray level image, which we will discuss later. If the pixel intensity is larger than that of the set threshold, it is regarded to possess “1”. On the other hand, if the intensity is below it, it is regarded to possess “0”. The binary image is easier to digitize and store than a full gray-level image, but some information is lost. Therefore this image is usually used for simple image analysis or analysis of simple image. Gray Scale Image In contrast to the binary information, gray scale images have a number of different brightness levels. The levels can be expressed by using eight bit/pixel which expresses 255 different levels. The levels often become

118

Optomechatronics

FIGURE 3.11 Various shapes of histogram.

a burden of computation. And they also have values in the range 0 to 63, or 0 to 1023, corresponding to six or ten bit digitization, respectively. Histogram Histogram representation of an image is a fundamental tool for image analysis and enhancement. The histogram is a plot of the relationship between gray scale value and the number of pixels at that value and, thus, it is a discrete function. As we shall see later, the shape and the magnitude of the histogram are indicative of the characteristic nature of the image, and they are very useful in determining a threshold. Figure 3.11 shows three different histograms where an image has a variety of different distributions over gray level. Shown in Figure 3.11a is the bimodal distribution in which two groups of gray level distributions are somewhat distinct. In this case, the whole image is usually divided into a binary image, that is, black “0” and white “1”. This is one of the simple forms of histogram, which makes it easy to do image analysis. Figure 3.11b shows a very narrow, clustered histogram, which is bad from the viewpoint of image quality. This is because the image has very low contrast, thus making it difficult to discriminate different objects within the image. On the other hand, the histogram shown in Figure 3.11c indicates a spread of histogram over the wide range of gray values, which is good from the view point of image contrast. In this case, it is said to have a high contrast. Histogram Modification When the contrast of an image is low due to poor lighting, incorrect set-up of the camera, or various other reasons, the image is not easily understandable. This is because the differences in the intensity between pixel values are small, so that objects and the background in the image cannot be discerned. In this case, it is necessary to improve the contrast of a poor contrast image.

max.

Ia

Ib

gray level

No. of pixels

(a)

min.

(b)

intensity after enhancement

No. of pixels


119

Ib′ z′

Ia′ Ia Ia′

intensity

Ib′

gray level

z I b

z

intensity before enhancement

(c) Intensity transformation relationship

FIGURE 3.12 Contrast enhancement.

In general, there are two commonly adopted methods of modifying the contrast as shown in Figure 3.12. Let us assume that the original pixel intensities lie in the range Imin # I # Imax ; as shown in Figure 3.12a. This indicates that the image has most of its pixels in the range between Ia and Ib : The first method shown in Figure 3.12b is histogram stretch which extracts a certain range of the intensity values of the interested pixels and modifies them in such a way that a new intensity distribution can make the original image more visually discernable. Let us assume that we are interested in mapping the intensity level range of the original image shown in Figure 3.12a to a new intensity distribution shown in Figure 3.12b. Suppose that we wish to transform the original intensity distribution into a modified distribution. In order words, if a new intensity value z0 is mapped from the corresponding value z, intensity z Ia # z # Ib

!

intensity z0 I 0a # z0 # I 0b

To achieve this, a transformation formula for a histogram stretch can be derived from Figure 3.12c. z0 ¼

I 0b 2 I 0a ðz 2 Ia Þ þ I 0a Ib 2 Ia

ð3:5Þ

The second method called “histogram equalization” shown in Figure 3.13 transforms the original image such that an equal number of pixels is

Optomechatronics

No. of pixels

120

0

dr

r

gray level

No. of pixels

(a)

(b)

0

intensity

ds

s

gray level

FIGURE 3.13 Histogram equalization.

allocated throughout the intensity range of interest. The effect of the histogram equalization is to improve the contrast of an image. In other words, there will be an increase in brightness differences of pixel values near the intensity values at which there are many pixels. In contrast to this, there will be a decrease in brightness differences near the intensity values with few pixels. To illustrate the concept in brief, let us suppose that gray levels described in the original image shown in Figure 3.13a can be transformed in a new gray level distribution having an equal number of pixels within a certain gray value range as illustrated in Figure 3.13b. We wish to find such transformation s ¼ TðrÞ where r and s are the variables representing gray level in the original image and that in the new image, respectively. If the probability density function Pr ðrÞ is assumed to represent the gray level pattern in the range ½r; r þ dr shown in Figure 3.13a, then the transformation will be made to obtain Pr ðsÞ in the range Pr ½s; s þ ds : Because the total number of pixels in the two images will remain equal, we have Pr ðsÞds ¼ Pr ðrÞdr This equation is a basic equation to the histogram equalization process which is essentially to make all the gray levels in the original image equally


121

probable. By letting Pr ðsÞ be a constant C, we have ðs 0

C ds ¼

ðr 0

Pr ðrÞdr

The above equation leads us to obtain the range of the gray value having an equal probability s¼

1 ðr P ðxÞdx C 0 r

where x is the dummy variable. This is illustrated in Figure 3.13b. Other histogram modifications such as histogram stretching, shrink, and sifting can also be performed in a similar manner. Histogram shrink is the opposite of a histogram stretch and decreases image contrast by compressing the gray level. Histogram slide is to translate the histogram in order to make an image either darker or lighter, while retaining the relationship between gray level values.

Image Filtering As mentioned earlier, noise more or less occurs in the image, corrupting a true intensity variation. Filtering an image is to recover its intrinsic visual information when it contains unwanted noise. This can be done by transforming the image intensities in order to remove noise or by performing image enhancement. There are two types of filtering depending upon whether the transformation is made in spatial or frequency domain: spatial domain filter and frequency domain filter. We will discuss briefly only spatial filters such as mean filter and median filter, which are frequently used to remove noise. Before getting into further details of image analysis, it is very helpful to become familiar with the convolution concept in order to understand filtering and edge-detecting procedures. Figure 3.14 indicates three cases of convolution masks. They show the 1 £ 2, 2 £ 1, 2 £ 2, and 3 £ 3 masks defined in an image frame. In the figure, M are the masks operated on the specified pixels, which are also called detection operators. And mij indicates the weighting values placed on the respective pixels. As indicated for a (3 £ 3) mask in Figure 3.15, the convolution process proceeds as follows; Overlay the mask on the image frame, multiply the corresponding pixel value one by one, sum all these results, and place this result at the center of its mask, as shown in the figure. This process continues until it is completed with the final pixel of the image. In a mathematical form, the output of the summation of the convolution results

122

Optomechatronics i 2x2 convolution mask

i

1x2 convolution mask Mx = m11 m12 j

M =

f(i,j ) f (i+1,j ) f(i,j+1)

j

f(i,j )

m11 m12 m21 m22

f(i+1,j )

f(i,j+1) f(i+1,j+1)

My = m11

2x1 convolution mask m 21 image frame

(a) 1×2 and 2×1 convolution masks

(b) 2×2 convolution mask

i

3x3 convolution mask

M= f(i-1,j-1) f(i,j-1) f(i+1,j-1) j

f(i-1,j)

f(i,j )

image frame

m11 m12 m 13 m21 m22 m 23 m31 m32 m 33

f(i+1,j)

f(i-1,j+1) f(i,j+1) f(i+1,j+1)

(c) 3×3 convolution mask FIGURE 3.14 Convolution masks.

3x3 mask

. mask center

FIGURE 3.15 The procedure of applying convolution mark.

image frame


123

can be written as gði; jÞ ¼ f ði; jÞ p mði; jÞ ¼

p n X X

f ði 2 k; j 2 lÞmðk; lÞ

ð3:6Þ

k¼1 l¼1

where: p denotes the convolution of f ði; jÞ with mði; jÞ; gði; jÞ is the sum of results of the multiplication of coincident values, f ði; jÞ represents the input image, and mði; jÞ is called the convolution mask. For the case of a 3 £ 3 mask, the output gð2; 2Þ in the above is computed by gð2; 2Þ ¼ m11 f11 þ m12 f12 þ m13 f13 þ m21 f21 þ m22 f22 þ m23 f23 þ m31 f31 þ m32 f32 þ m33 f33

ð3:7Þ

This procedure is explicitly indicated in the figure. Referring to Equation 3.7, the convolution values at pixel ði; jÞ can be easily computed by the following manner; For 1 £ 2 and 2 £ 1 masks Mx ¼ m11 f ði; jÞ þ m12 f ði þ 1; jÞ;

My ¼ m11 f ði; jÞ þ m21 f ði; j þ 1Þ

ð3:8Þ

For 2 £ 2 masks M½ f ði; jÞ ¼ m11 f ði; jÞ þ m12 f ði þ 1; jÞ þ m21 f ði; j þ 1Þ þ m22 f ði þ 1; j þ 1Þ

ð3:9Þ

For a 3 £ 3 mask M½ f ði; jÞ ¼ m11 f ði 2 1; j 2 1Þ þ m12 f ði; j 2 1Þ þ m13 f ði þ 1; j 2 1Þ þ m21 f ði 2 1; jÞ þ m22 f ði; jÞ þ m23 f ði þ 1; jÞ þ m31 f ði 2 1; j þ 1Þ þ m32 f ði; j þ 1Þ þ m33 f ði þ 1; j þ 1Þ

ð3:10Þ

Mean Filter A mean filter is one of the simplest linear filters and is essentially an average filter that operates on neighboring pixels within an m £ m window. It replaces the center pixel with an average value of the image intensities of the pixels. A convolution mask travels through the image for computation and replacement of the computed average. This can be mathematically expressed by X ~ jÞ ¼ 1 fði; f ðk; lÞ N ðk;lÞ[w

ð3:11Þ

124

Optomechatronics

where N is the total number of pixel within the window W. For example, if a 3 £ 3 mask is given as 1 9

1 9

1 9

1 9

1 9

1 9

1 9

1 9

1 9

~ jÞ becomes then, fði; jþ1 iþ1 X X ~ jÞ ¼ 1 fði; f ðk; lÞ 9 k¼i21 l¼j21

ð3:12Þ

It is noted that the sum of the coefficients of the mask is 1. Median Filter A median filter is effective in coping with impulse noise, while retaining original image details. This is because it does not consider any values which are significantly different from the typical values in the convolution mask. To see how it works, consider Figure 3.16 where a mask (3 £ 3) of the filter is shown. It looks similar to the convolution mask. We can see that the process is not a weighted sum, but employs a nonlinear operation. In this method, all intensity values are sorted in an ascending order for pixels within the mask. Then the value of the middle pixel is selected as the new value for pixel ði; jÞ:

input image

filter window (3x3 case) 14 8 0 4 9 35 6 5 27

FIGURE 3.16 The concept of median filtering.

output image ordered pixels 0 4 5 6 8 9 14 27 35


125

This process is repeated in a successive fashion until all pixel operations are undertaken. The figure shows a 3 £ 3 filter window arriving at a designated location. Putting all the intensity values in ascending order and taking the middle value among them yields the intensity value, “9”. For a 3 £ 3 mask, filtered output value G½f ði; jÞ can be written by G½f ði; jÞ ¼ Med{ f ði 2 1; j 2 1Þ; f ði; j 2 1Þ; f ði þ 1; j 2 1Þ; f ði 2 1; jÞ; f ði; jÞ; f ði þ 1; jÞ; f ði 2 1; j þ 1Þ; f ði; j þ 1Þ; f ði þ 1; j þ 1Þ}

ð3:13Þ

where Med{L} indicates the operator that takes the value of the middle pixel. Often 5 £ 5 and 9 £ 9 masks are used but their filtering methods are the same as that of the above 3 £ 3 operator. Let us consider the original natural image composed of a human face and a bookshelf as shown in Figure 3.17a in order to see the performance of the median filter. It contains salt and pepper noise of 20% intentionally added to the original. A 3 £ 3 median filter is operated on it. As can be seen from Figure 3.17b, the noise is almost removed. The result indicates that this filter is very effective to impulsive noise. In order to obtain the above filtered image by processing the median filter on this image, Matlab M-files are used. In the M-files we write Matlab-specific statements and execute a series of such written statements. Digital image processing methods using Matlab toolbox are well documented in Ref. [7]. Throughout this chapter we will use M-files to create the Matlab statements. The followings are the Matlab codes for obtaining median filtering (Box 3.1).

FIGURE 3.17 Median filtered image.

126

Optomechatronics

Box 3.1. FILTERING : MEDIAN FILTER Matlab source: Median Filter clear all % reading image File image ¼ imread(‘tao.bmp’); % add salt & pepper noise n Percent ¼ 20; [y, x ] ¼ size(image); nMaxHeight ¼ round(y p n Percent/100.0); nMaxWidth ¼ round(x p n Percent/100.0); for I ¼ 1:nMaxHeight, for J ¼ 1:nMaxWidth, cx ¼ round(rand(1) p (x 2 1)) þ 1; cy ¼ round(rand(1) p (y 2 1)) þ 1; aaa ¼ round(rand(1) p 255); if aaa . 128 image(cy,cx) ¼ 255; else image(cy,cx) ¼ 1; end end end % median filtering for i ¼ 1:x for j ¼ 1:y if(i ¼ ¼ 1 ll j ¼ ¼ 1 ll i ¼ ¼ x ll j ¼ ¼ y) % boundary image_out( j,i) ¼ image( j,i); % not changed else for l ¼ 1:3 for k ¼ 1:3 window(l þ (k 2 1) p 3) ¼ image( j þ l 2 2,i þ k 2 2); end end %sorting for l ¼ 1:8 for k ¼ 2:9 if (window(l) . window(k)) temp ¼ window(k); window(k) ¼ window(l); window(l) ¼ temp; end end


127

end image_out( j,i) ¼ window(5); % medain value end end end % showing image figure subplot(1,2,1); imshow(image); subplot(1,2,2); imshow(image_out);

Image Segmentation Thresholding Thresholding is the simplest and still one of the most effective methods to segment objects from their background. Segmentation is an image processing method that splits the image into a number of regions, each having a high level of uniformity in brightness or texture. It reduces a gray scale image to binary black or white pixels. In the binary image, objects appear as black figures on a white background, or as white figures on a black background. One method to extract the objects from the background is to use a thresholding method in which the object and the background pixels have gray levels grouped into two dominant modes. Any point ðx; yÞ having gray level greater than a threshold value, T; that is, f ðx; yÞ . T; belongs to an object point while the point for f ðx; yÞ , T is called a background point. In this case, we need only one threshold value to classify the two, but in other cases, multiple thresholds may be needed when the image contains brightness information rather complex so that it cannot be ideally made binary. In the case of an image containing only two principal brightness regions, a global thresholding is the simplest technique, which partitions the image histogram by using a single threshold, T. Consider the gray level histogram shown in Figure 3.18a, which corresponds to an image f ðx; yÞ: It is composed of light objects on a dark background, having the object and the background pixels grouped into two dominant modes in terms of gray level. The thresholding problem is to find a threshold Tsuch that the two modes are well separated. If a thresholded image, gðx; yÞ; is defined by ( gðx; yÞ ¼

1

if f ðx; yÞ . T

0

if f ðx; yÞ # T

) ð3:14Þ

then the pixels labeled 1 correspond to the objects composed of the object points, whereas the pixels labeled 0 correspond to the background composed

Optomechatronics

number of pixels (N)

number of pixels (N)

128

(a) single threshold

T

T1

intensity

T2

(b) multiple threshold

intensity

FIGURE 3.18 Idealized gray level histogram.

of the background points. By labeling the image pixel by pixel in this way, segmentation is achieved, depending on whether the gray level of the pixel is greater or less than the T value. To illustrate the threshold technique, two scenes with a 256 £ 256 image will be considered. As shown in Figure 3.19, clearly the binary image of the

histogram

number of pixels

10000 background 5000 object 0

0

50

(a) the image of a tree leaf and its corresponding histogram

T1=180

T2 = 150

(b) the processed images with two different thresholds FIGURE 3.19 Thresholding process using single thresholding value.

100 150 intensity

200

250


129

tree leaf differs due to variation of the threshold value. Consider the image of a tree leaf whose corresponding histogram is shown in Figure 3.19a. From the histogram, the leaf is seen to roughly have the intensity range of 0 to 150, while that of the background is 150 to 255. Based on the thresholding operation given in Equation 3.14, the image is thresholded with two different values of T as shown in Figure 3.19b. It can be seen that when the image is thresholded at T ¼ 180; a clear binary image can be obtained. However, as T moves toward a smaller value, for example, 150, the leaf image starts to be deteriorated, showing a slight difference from that of the original. This is because, in this case, the pixels having intensity larger than 150 are classified to background (white region) rather than to part (black). A more complicated scene is the latter case, of which the original image is shown in Figure 3.20a. Thresholding at T ¼ 128 is used to obtain its binary image. It can be observed that, due to the similar gray level among the objects, the background, and the shadow within the scene, the binary image shown in Figure 3.20b does not exactly represent the original face image, but somehow retains some important features. The following is the Matlab source code for obtaining such image (Box 3.2). Returning to Figure 3.18b it shows two types of objects on a dark background, which is a more general case. In this case, we need to define two thresholds T1 and T2 as shown in the figure. The three modes separated by T1 and T2 are: object 1

if f ðx; yÞ . T2

object 2

if T1 , f ðx; yÞ , T2

background

if f ðx; yÞ , T1

FIGURE 3.20 Gray-to-binary image transformation.

ð3:15Þ

130

Optomechatronics

Box 3.2. (THRESHOLD ) Matlab source: Threshold clear all % reading image File image ¼ imread(‘tao.bmp’); [y,x ] ¼ size(image); threshold ¼ 128; % defined by user % threshold for i ¼ 1:x for j ¼ 1:y if image( j,i) . 128 image_th( j,i) ¼ 255; else image_th( j,i) ¼ 0; end end end % showing image figure subplot(1,2,1), imshow(image) subplot(1,2,2), imshow(image_th)

The images having a multimode histogram of this type can be frequently found in most of the real scenes. For instance, when an IC chip is placed on a table, the gray level of this image may form three different distinct ranges; IC body, lead, and background. In this case, image analysis is more difficult than the case of the distinct histogram distribution. Choice of threshold value in this case needs to be made depending upon the information to be extracted from the image. Figure 3.21 shows the images of various electronic parts and bolts. The objects include three chips with white characters engraved on their top surfaces and the image is observed to contain several shaded areas around their bodies. As can be observed from the figure, the contour and the shape of each object become clearer at T ¼ 50 than those of the other two threshold values ðT ¼ 100; 150Þ; and the shadow effect is small. In the case of the electronic chip having several leads of brighter color, the contours are hardly seen from the image. When T is increased to 100, the white characters become clearly recognized and the shadow effect becomes significant. However, at T ¼ 150; the shadow significantly influences the whole object image and, thus, the object and the shadows can not be distinguished easily. This example clearly shows a choice of threshold values should be appropriately made depending upon the objective of the given task.


131

FIGURE 3.21 Thresholding of a variety of electronic parts having different imaging characteristics.

In general, multilevel thresholding becomes a difficult task when the number of separation of regions of interest is large. An approach to this type of problems is to use a single, adaptive threshold. The difficulty also arises when an image is contaminated by some noise, which is frequently encountered in actual machine vision environments. In this case, optimal thresholding techniques can be used in such a way that the probability of the classification error, that is, erroneously classifying an object point as a background point and vice versa, is minimized with respect to T. Iterative Thresholding The previous discussions reveal that there is a need for an optimal thresholding technique which yields the best binary image quality. Some of the techniques to achieve this objective as well as to work for two principal brightness regions include: (1) Iterative threshold method (2) Optimal thresholding using maximum likelihood. The iterative method does not require an estimated function of a histogram and, therefore, works for more general cases, whereas the second method requires an analytical function of the histogram for optimization. In the following, the algorithms for the method will be introduced, based upon

132

Optomechatronics

heuristic approach. Let us assume that an image to be segmented has two brightness regions. The algorithm begins then: (1) Choose an initially estimated value of the threshold value T. (2) Partition the image into two groups R1 and R2 using the T value. (3) Calculate the mean gray value fR1 ðx; yÞ and fR2 ðx; yÞ of the partitioned group R1 and R2 ; respectively. X f ðx; yÞ fR1 ðx; yÞ ¼

fR2 ðx; yÞ ¼

ðx;yÞ[R1

Pixel number of the R1 region X f ðx; yÞ

ð3:16Þ

ðx;yÞ[R2

Pixel number of the R2 region

(4) Choose a new threshold to have T¼

fR1 ðx; yÞ þ fR2 ðx; yÞ 2

ð3:17Þ

(5) Repeat step (2) through step (4) until the mean gray values fR1 and fR2 do not change as iteration proceeds. 2500 frequency

2000 1500 1000 500 0

(a) original image

T= 100

0

50

100 150 intensity (b) histogram of the image

T= 130.7

(c) binary images obtained from the iterative thresholding process FIGURE 3.22 The iterative thresholding process.

200

T= 147

250


133

This algorithm discussed above is illustrated with the example of the electronic part and electric resistance. To begin with, we choose the initial value of T ¼ 100; which is indicated by a histogram in Figure 3.22a. According to Equation 3.16 and Equation 3.17 we can calculate fR1 ðx; yÞ and fR2 ðx; yÞ They are determined by fR1 ðx; yÞ ¼ 186:1

fR2 ðx; yÞ ¼ 75:2

ð3:18Þ

which enables us to obtain a new threshold value. We then use a new value Tð1Þ ¼ 130:7 and its image is indicated in Figure 3.22b for the next iteration. Repeating in this way, we obtain Tð2Þ ¼ 142; Tð3Þ ¼ 145; and finally a steady state T value, Tð4Þ ¼ 147; as shown in Figure 3.22c. It can be seen that, for this single and simple object, the convergence rate is very fast, reaching its optimal value only after four iteration steps. Region-Based Segmentation In the previous section, thresholding was used to partition an image into the regions based on the distribution of the pixel intensity. Another way of segmenting the image is to find the regions directly. Region splitting and merging, and region growing belong to this segmentation category. Here, we will discuss a basic formulation based on splitting and merging technique. As shown in Figure 3.23, the image window contains seven different regions ðR1 ; R2 ; R3 ; R4 ; R5 ; R6 ; R7 Þ having different intensity color and texture. The purpose of this segmentation method is to divide the region into seven segments having identical properties. To generalize this concept, let us define R to represent the entire image region referring to the figure. Then, a segmentation is regarded as a process partitioning R into n subregions R1 ; R2 ; …; Rn : The conditions that must be satisfied during the process are then; ð1Þ

n

< Ri ¼ R

i¼1

ð2Þ Ri is a connected region; ði ¼ 1; 2; …nÞ: ð3Þ Ri > Rj ¼ f

;i and j; i – j:

ð4Þ PðRi Þ ¼ true;

i ¼ 1; 2; …n:

ð5Þ PðRi < Rj Þ ¼ false;

i–j

ð3:19Þ

In the above, f is the null set, and PðRi Þ is a logical predicate over the points in set Ri : Condition (1) implies that every pixel must be in a region while condition (2) indicates connectivity in that all points in a region must be connected. Condition (3) requires that there should be no jointed regions between regions Ri and Rj within R. In other words, they are disjointed between them. Condition (4) indicates that all points in Ri must have the

134

Optomechatronics

FIGURE 3.23 Seven divided different image regions.

same properties such as the intensity and the color. For instance, PðRi Þ ¼ True; if all pixels in Ri have an identical intensity. Condition (5) implies that the property of Ri must be different from that of Rj : Region splitting and merging is, therefore, to subdivide an image into a set of disjointed regions while satisfying the conditions given in the above. During this process, splitting and merging the regions are iteratively carried out. Consider an image frame R composed of four subregions, R1 ; R2 ; R3 ; R4 ; as shown in Figure 3.24. If R1 and R2 satisfy the conditions in Equation 3.19, where R3 and R4 do not, then R3 and R4 need to be segmented into eight subregions, each into four. Once again, these eight regions are segmented into smaller subregions. This segmentation process continues until all the subregions satisfy the conditions. To illustrate this basic concept, a binary image is depicted in the left hand side of Figure 3.25a, which consists of a single object (black) and the background (white). Within each region, the intensity will be assumed to be identical. Then, in the first step, we divide the image into four quadrants, R1 ; R2 ; R3 ; and R4 : We see that two quadrants R2 and R4 satisfy the predicate while the other two do not; PðR1 Þ ¼ false; PðR2 Þ ¼ true; PðR3 Þ ¼ false; PðR4 Þ ¼ true

ð3:20Þ

Therefore, PðR2 Þ and PðR4 Þ are not changed. In the next step, we further divide two “false” regions into subquadrants as shown in the figure. The followings hold for R1: PðR11 < R13 Þ ¼ true; PðR14 Þ ¼ true

ð3:21Þ


R1

R2

R3

R4

R31

R32

R41

R42

R33

R34

R43

R44

135

R

R

R

R

R

R

R

R

R

R

R

R

R

R

R

R

331 333

332 334

341 341

341 341

431 431

431 431

441 441

441 441

(a) partioning the image region R R

R1

R2

R31

R4

R3

R32

R33

R331 R332 R333 R334

R34

R41

R42

R43

R341 R342 R343 R344 R431 R432 R433 R434

R44

R441 R442 R443 R444

(b) tree representation FIGURE 3.24 Segmentation for the split and merge algorithm.

Combining Equation 3.20 and Equation 3.21, we obtain PðR2 < R4 < R11 < R13 Þ ¼ true for “0” intensity level PðR14 Þ ¼ true for “1” intensity level Following the same procedure shown in the above for the subquadrant R13 in Figure 3.25b, we obtain PðR2 < R4 < R11 < R13 < R31 < R33 < R34 Þ ¼ true for “0” intensity level PðR14 UR32 Þ ¼ true for “1” intensity level

ð3:22Þ

One more segmentation step yields the following final conditions. PðR2 < R4 < R11 < R13 < R31 < R33 < R34 < R121 < R122 < R123 Þ ¼ true PðR14 < R32 < R124 Þ ¼ true

ð3:23Þ

The final result, which combines all of these results, is obtained by merging the regions satisfying the conditions specified in Equation 3.19.

136

Optomechatronics Segmentation R1

R2

split

(a) R11

split

(b)

R4

R3 R12

R13

R14

R31

R32

R33

R34

split

R121

R122

R123

R124

FIGURE 3.25 Illustration of the region based segmentation.

Edge Detection In the previous section, we have discussed various methods of image segmentation, discriminating one region from another in the image of interest. Edge detection is one such method for analyzing and identifying image contents, and presents the most common approach for detecting discontinuities in gray level. In this section, we will consider the detection of edges and discuss its basic concepts, the processing steps associated with edge detection, and the edge detectors. As illustrated in Figure 3.26, an edge in the image is the boundary between two regions with relatively distinct gray level value. In other words, the gray level difference is distinct, exhibiting a significant local change in the image intensity as shown in Figure 3.26a. The variation in the gray level f ðx; yÞ in the x-y image plane can be expressed as Gx and Gy in Figure 3.26b. As we go along from point A to point B through C in the image shown in Figure 3.26c, we can see that f ðx; yÞ slowly decreases to somewhere near point C. The gray level has a drastic change and afterwards it changes very little until it


137

y

y

Gy

B

a Gx

C

C

A

G

x

(a) line A-B f(x,y)

x

(b) image f (x,y) G [ f ( x , y )]

A

C

B

(c) profile of a line A-B

A

C

B

(d) magnitude of first derivative

FIGURE 3.26 Edge characteristics.

hits point B. If f ðx; yÞ is differentiated with respect to spatial points along the line A – C – B, the magnitude of the derivative Gðx; yÞ is expressed as shown in Figure 3.26d. Clearly, the maximum value of Gðx; yÞ occurs at point C. This information provides a useful cue in edge detection, as we shall see later. In general, the edge varies with spatial location in the image. There are several standard edge profiles defined depending on its shape: step, ramp line, and roof. Almost all general edge profiles are composed of these standard profiles. Along with these profiles, there usually occurs discontinuity in the image intensity. To mathematically characterize the edge, let us consider the image frame shown in Figure 3.26a of which intensity distribution is defined by f ðx; yÞ: An edge is then characterized by the gradient of the image intensity having two components: magnitude and direction. The image gradient is defined by " G½ f ðx; yÞ ¼

Gx Gy

#

2 6 6 ¼6 4

›f ›x ›f ›y

3 7 7 7 5

ð3:24Þ

The magnitude of the gradient is expressed by qffiffiffiffiffiffiffiffiffiffiffi G½ f ðx; yÞ ¼ G2x þ G2y

ð3:25Þ

138

Optomechatronics

For simplicity of implementation, the gradient magnitude is approximated by addition of the gradient components as; G½ f ðx; yÞ ¼ lGx l þ lGy l

ð3:26Þ

where the direction of the gradient at location ðx; yÞ denoted by aðx; yÞ is given by

aðx; yÞ ¼ tan21

Gy Gx

ð3:27Þ

which is defined with respect to the x axis. It is noted that the above Equation 3.26 and Equation 3.27 require computation of the partial derivatives ›f =›x; ›f =›y at every pixel location. However, the edge taken by CCD camera is discrete in nature and, thus, needs to be numerically approximated by different equations as follows: Gx . f ði þ 1; jÞ 2 f ði; jÞ

Gy . f ði; j þ 1Þ 2 f ði; jÞ

ð3:28Þ

where i and j refer to the ith pixel in x direction and jth pixel in negative direction, respectively. As already discussed, in order to calculate all gradient magnitude values within the image frame of interest, we locate the operator at the left uppermost corner as an initial point and calculate a gradient magnitude at that location. To obtain the next value, we move the mask to the next pixel location. In other words, one value is obtained at each point. This procedure is repeated until the computation is completed for all pixels located within the frame. Gradient operators are in fact the convolution masks that enable us to compute the gradient vector of a given pixel and thus detecting edges, examining small neighborhoods. Table 3.2 summarizes the operators being used for edge detection. The operator approximating the first derivative (Roberts, Sobel) will be examined first and then the second derivative operator (Laplacian). And then the operator called Laplacian of Gaussian (LoG) which combines a Gaussian filter and Laplacian operator will be treated. The edge operators discussed in the above have different characteristics responding differently to edge and noise. For instance, some operators may be robust to noise but miss some critical edges while others may be sensitive to noise although they detect most edges. This necessitates a performance metric for edge detection operator. One such metric called the Pratl Figure of Merit is frequently used for the comparison purpose, but we will not deal with the further details here. Roberts Operator The Roberts operator is a very simple gradient operator that uses a 2 £ 2 neighborhood of the current pixel. The operator is identical to that shown in


139

TABLE 3.2 Properties of Various Edge Operators Detection

Typical Operator/Mask

Roberts

Gmx = Pwett

1

0

0

−1

Gmy =

−1 −1 −1

Gmx = Sobel

1

0 −1

0

1

0

0

Gmy = −1

0

1

1

1

1

−1

0

1

−1

0

1

Gmy = −2 0

2

Gmx = 0

0

0

1

2

1

−3 −3 5

−1

0 −1

∆ Robinson compass mask

−1

−3 5

0

0 −1

0

−1 0

1

5

or

−1

8

−1

−1 −1 −1 −2 −1 0

… G = −1 0 −1 m7

Gmo = −2 0 2 −1 0 15

0

1

m1 m2

Magnitude: maximum of convolution of eight major compass masks Direction: maximum value of eight compass mask Rotationally symmetric direction: sign of the result from two adjacent locations

−1 −1 −1

4 −1

qffiffiffiffiffiffiffiffiffiffiffi Magnitude: m21 þ m22

1

−3 −3 −3

−3 −3 5

2=

0

qffiffiffiffiffiffiffiffiffiffiffi Magnitude: m21 þ m2 m Direction: tan21 2 m1

Direction: tan21

… G = −3 0 5 m7

Gmo = −3 0 5

Laplacian

Edge point only

0 −1

0

−1 −2 −1

Kirsch compass mask

Edge Magnitude/Direction

2

Magnitude: maximum of convolution of eight major compass masks Direction: maximum value of eight compass mask

Figure 3.14b and is written by

Gmx =

1

0

0

−1

Gmy =

0

−1

1

0

ð3:29Þ

Using this operator, Equation 3.26, and Equation 3.28, the magnitude of the gradient value at pixel ði; jÞ is obtained by G½ f ði; jÞ ¼ lf ði; jÞ 2 f ði þ 1; j þ 1Þl þ lf ði; j þ 1Þ 2 f ði þ 1; jÞl

ð3:30Þ

Note that, due to its geometric configuration, the above approximate difference values are assumed to be computed at the interpolated point

140

Optomechatronics

ði þ 1=2; j þ 1=2Þ: As can be observed, this operator is very sensitive to noise because only 2 £ 2 pixels are used to approximate the gradient. Sobel Operator This operator uses a 3 £ 3 mask that considers the neighborhood pixels for the gradient computation as shown in Figure 3.14c. It is expressed by

Gmx =

−1

0

1

−2

0

2

−1

0

1

Gmy =

1

2

1

0

0

0

−1

−2

−1

ð3:31Þ

255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 85

85 255 255 255 255 255 255

255 255 255 255 255

85

41

41

255 255 255 255

85

41

0

0

41

255 255 255 85

85 255 255 255 255 255 85 25 5 255 255 255

41

0

0

0

0

41

255 255

85

41

0

0

0

0

0

0

85 255 255 255 41

85 255 255

255 255

41

0

0

0

0

0

0

0

0

41 255 255

255 255

85

85

85

85

85

85

85

85

85

85 255 255

255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255

(a) original mosaics mimicking a real image 0

0

0

0

0

0

0

0

0

0 -170 -170 170 170

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0 -170 -510 -510 -170 0

0

0

0

0

0

0

0

0

0 -170 -554-812 812 -554 -170 0

0

0

0

0

0

0

0

0 -170 -554 -683 -469 -469 -683 -554 -170 0

0

0

0

0 -170 -554-683 -425 -208 -208 -425 -683 -554 -170 0

0

0 -170 -554 -384 384 554 170

0

0 -170 -554 -683 -299 299 683 554 170

0

0

0

0

0

0 -170 -554- 683-425 -126 126 425 683 554 170

0

0

0

0

0

0

0

0 -170 -554-6 83-425 -167 -41 41 167 425 683 554 170

0

0 -170 -554-683 -425 -167 -41 -41 -167 -425 -683 -554 -170 0

0 -554 -853 -425 -167 -41

0

0

41 167 425 853 554

0

0 -214 -513 -425 -167 -41

0 -768 -894 -167 -41

0

0

0

0

41 167 85 768

0

0

0 -554 -595 -41

0

0

0

0

0

0

41 595 554

0

0

214 683 979 102010201020102010201020 979 683 214

0

0 -170 -170 0

0

0

0

0

0

0

0

0

0

170 510 680 680 680 680 680 680 680 680 510 170

0

0

0

0

0

0

0

0

0

0

0

0

0

0

170 170 0

0

Gx (b) the gradient values Gx and Gy FIGURE 3.27 Sobel operation of a series of mosaic images.

0

0

0

0

-41 -167 -425 -513 -214 0

44 173 299 340 340 340 340 299 173 44

0

0

0

0

0

Gy

0

0

0

0

0

0

0

0

0


0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0 0

0

0

0

0

240 537 537 240

0 0

240 783 898 898 783 240

0

0

0

0

0

0

141

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

240 783 965 556 556 965 783 240

240 783 965 601 243 243 601 965 783 240

0 0

0

0

0

0

240 537 537 240

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

240 783 898 898 783 240

240 783 965 556 556 965 783 240

240 783 965 601 243 243 601 965 783 240

0

240 783 965 601 236 57

57 236 601 965 783 240

0

0

240 783 965 601 236 57

57 236 601 965 783 240

0

0

593 995 601 236 57

0

57 236 601 995 593

0

0

593 995 601 236 57

0

57 236 601 995 593

0

0

768 895 240 301 340 340 340 340 301 240 895 768

0

0

768 895 240 301 340 340 340 340 301 240 895 768

0

0

593 905 979 102010201020102010201020 979 905 593

0

0

593 905 979 102010201020102010201020 979 905 593

0

0

240 537 680 680 680 680 680 680 680 680 537 240

0

0

240 537 680 680 680 680 680 680 680 680 537 240

0

0

0

0

0

0

0

0

0

0

0

(c)

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

(d)

|G|

0

0

0

240 537 537 240

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

240 783 965 556 556 965 783 240

240 783 965 601 243 243 601 965 783 240

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

T= 500

0

240 783 898 898 783 240

0

0

0

0

240 537 537 240

240 783 898 898 783 240

240 783 965 556 556 965 783 240

240 783 965 601 243 243 601 965 783 240

0

240 783 965 601 236 57

57 236 601 965 783 240

0

0

240 783 965 601 236 57

57 236 601 965 783 240

0

0

593 995 601 236 57

0

57 236 601 995 593

0

0

593 995 601 236 57

0

57 236 601 995 593

0

0

768 895 240 301 340 340 340 340 301 240 895 768

0

0

768 895 240 301 340 340 340 340 301 240 895 768

0

0

593 905 979 102010201020102010201020 979 905 593

0

0

593 905 979 102010201020102010201020 979 905 593

0

0

0

240 537 680 680 680 680 680 680 680 680 537 240

0

0

0

0 0

(e)

0

240 537 680 680 680 680 680 680 680 680 537 240 0

0

0

0

0

0

0

T= 800

0

0

0

0

0

(f)

0

0

0

0

0

0

0

0

0

0

0

0

0

0

T= 1000

FIGURE 3.27 Continued.

The magnitude of the gradient at the center pixel ði; jÞ is calculated in the following manner; Gx ¼ ð f13 þ 2f23 þ f33 Þ 2 ð f11 þ 2f21 þ f31 Þ Gy ¼ ð f11 þ 2f12 þ f13 Þ 2 ð f31 þ 2f32 þ f33 Þ

ð3:32Þ

where G values are easily computed by substituting Equation 3.32 into Equation 3.26. Some characteristics observed from this operator are as follows; First, the operator places an emphasis on pixels located closer to the center pixel. Second, due to this operator configuration, it provides both differencing and smoothing effects. As a result, this operator is one of the most popularly used edge operators. We will consider an example of obtaining the edges of an input image composed of mosaics by using the Sobel operator for a simple image similar to the triangle in Figure 3.27. In the figure the number inside each

142

Optomechatronics

FIGURE 3.28 Sobel and LoG operated images.

pixel represents the image intensity of which gray level is in the range of 0 to 255. By application of the Sobel operator, we wish to obtain its edges when the threshold value of Gðx; yÞ is kept at three different values, 500, 800, and 1000. Utilizing Equation 3.32, Gx ðx; yÞ and Gy ðx; yÞ can be computed and the results are shown in the figure. Combining these two values yields G½ f ði; jÞ as indicated in the figure. The result shows that when T ¼ 500, at the edge pixels, the gradient values are the largest, as expected. We can proceed to the same computational procedure using Equation 3.32 and arrive at the results shown in the figure. It is noted that the threshold value, T, greatly influences the determination of the edge: At T ¼ 800, the edge shape of the triangle is reserved but, at T ¼ 1000, it is completely lost. The Sobel operation is also carried out on the face image considered previously in Figure 3.20a. Figure 3.28a shows the edgedetected results obtained at T ¼ 60: We can see that the detected edges obtained are composed of thick lines and appear to well represent the original image. The Matlab codes to detect the edges by the Sobel operation are presented in the below (Box 3.3). Laplacian Operator So far, we have discussed the gradient operators that use the first derivative of the image intensity. The underlying principle of this method is that the peak of the first derivative occurs at the edge. The Laplacian operator, however, uses the notion that, at edge points where the first derivative becomes extreme, there will be zero crossing in the second derivative. To illustrate the concept, let us introduce the Laplacian of a 2D function f ðx; yÞ in


143

Box 3.3. EDGE DETECTOR : SOBEL Matlab source: Sobel operation clear all % reading image File image ¼ imread(‘tao.bmp’); threshold ¼ 60; % defined by user [y,x ] ¼ size(image); imgs ¼ double(image);% changing data type % Sobel operation for i ¼ 1:x for j ¼ 1:y if(i ¼ ¼ 1 ll j ¼ ¼ 1 ll i ¼ ¼ x ll j ¼ ¼ y) % boundary image_sobel( j,i) ¼ 255; % white else sx ¼ 2 (imgs( j 2 1,i 2 1) þ 2 p imgs( j,i 2 1) þ imgs( j þ 1,i 2 1)) þ imgs( j 2 1,i þ 1) þ 2 p imgs( j,i þ 1) þ imgs( j þ 1,i þ 1); sy ¼ 2 (imgs( j 2 1,i 2 1) þ 2 p imgs( j 2 1,i) þ imgs( j 2 1,i þ 1)) þ imgs( j þ 1,i 2 1) þ 2 p imgs( j þ 1,i) þ imgs( j þ 1,i þ 1); M ¼ sqrt(sx^2 þ sy^2); if(M . threshold) image_sobel( j,i) ¼ 0; % black else image_sobel( j,i) ¼ 255; % white end end end end % showing image figure subplot(1,2,1); imshow(image); subplot(1,2,2); imshow(image_sobel);

an image frame, which is a second order derivative. It is defined as 72 f ðx; yÞ ¼

›2 f ›2 f þ ›x2 ›y2

ð3:33Þ

Consider a 1D edge profile shown in Figure 3.29a of which intensity varies along x direction. Its first and second derivatives are also shown in the figure. As can be observed in Figure 3.29b, the pixels having their first derivative

144

Optomechatronics

above a threshold shown in a dotted line will be all regarded as the edge. This, however, gives a somewhat rough estimation of the edge. If the second derivative is utilized as shown in Figure 3.29c, only a local maximum of the first derivative will be regarded as the edge because it becomes zero in this case. Therefore, all we need to do in the case of the Laplacian operator is to find a zero crossing of the second derivative. This implies that finding the zero crossing position is much easier and more accurate than finding an extreme value of the first derivative. This operator can also be expressed in a digitized numerical form by using different equations in the x and y directions,

›2 f ›Gx ¼ ¼ f ði þ 1; jÞ 2 2f ði; jÞ þ f ði 2 1; jÞ ›x ›x2

ð3:34Þ

›2 f ›Gy ¼ ¼ f ði; j þ 1Þ 2 2f ði; jÞ þ f ði; j 2 1Þ ›y ›y2

Adding these two equations according to the relation given in Equation 3.33 yields the Laplacian operator given in the following form

2=

0

1

0

1

−4

1

0

1

0

ð3:35Þ

∆

f (x, y)

(a)

x peak

f' (x, y)

(b)

a

b

threshold

x

f'' (x, y) zero crossing

(c) FIGURE 3.29 Laplacian operator finding zero crossing of f 00 (x,y).

x


145

From the Laplacian operated image, we can find the edge by finding the position of zero crossing. It can be described as IF { f ½i; j . 0 AND ðf ½i þ 1; j , 0

OR

f ½i; j þ 1 , 0Þ}

{ f ½i; j , 0 AND ðf ½i þ 1; j . 0

OR

f ½i; j þ 1 . 0Þ}

OR ð3:36Þ

THEN f ½i; j ¼ edge Although the Laplacian operator responds very sharply to variation in the image intensity, it is very sensitive to noise due to its second derivative action. Therefore, the edge detection by finding the zero crossing of the second derivative of the image intensity may yield erroneous results. To avoid this the Laplacian of Gaussian (LoG) filter, in which the Gaussian filtering is combined together with the Laplacian operator, is popularly used to filter out noise before the edge enhancement. If the Gaussian filter is denoted by 2

2

gði; jÞ ¼ exp2ði þj Þ=2s

2

ð3:37Þ

Convoluting this equation with f ði; jÞ; we have ~ jÞ ¼ ½gði; jÞ p f ði; jÞ fði;

ð3:38Þ

The output of the LoG is then ~ jÞ ¼ 72 ½gði; jÞ p f ði; jÞ fði;

ð3:39Þ

The above notation implies the discrete version of the continuous function as shown in Equation 3.28. The derivative of convolution yields ~ jÞ ¼ ½72 gði; jÞ p f ði; jÞ fði; In the above the LoG is denoted by ! 2 2 2 i2 þ j2 2 2s2 2 7 gði; jÞ ¼ exp2ði þj Þ=2s 4 s

ð3:40Þ

ð3:41Þ

The edge detection using the LoG filter is summarized in the following three steps: (1) Smoothing operation by a Gaussian filter (2) Enhancing the edges using Laplacian (3) Detecting zero crossing. The same face image discussed in Figure 3.20a is used here to see the effect of the LoG filtering. Its parameter has a mean value ¼ 0 and s ¼ 2.0.

146

Optomechatronics

The filtered result is shown in Figure 3.28b. The image is slightly different from the image obtained by Sobel in that there are many thin lines appearing in the image. This is because the image produced by the LoG operator is the result of the edge detection together with the thinning operation. The Matlab codes for this filter are listed in the below (Box 3.4). Box 3.4. EDGE DETECTOR : LAPLACIAN

OF

GAUSSIAN

Matlab source: Laplacian of Gaussian Operation clear all % reading image file image ¼ imread(‘tao.bmp’); [y,x ] ¼ size(image); % image size % making a log filter mask N ¼ 13; % filter size sigma ¼ 2.0; % sigma half_filter_size ¼ round(N/2); siz ¼ (N 2 1)/2; std2 ¼ sigma^2; [xx,yy ] ¼ meshgrid(2 siz:siz, 2 siz:siz); arg ¼ 2 (xx. p xx þ yy. p yy)/(2 p std2); h ¼ exp(arg); h(h , eps p max(h(:))) ¼ 0; sumh ¼ sum(h(:)); if sumh , ¼ 0, h ¼ h/sumh; end; % now calculate Laplacian h1 ¼ h. p (x. p x þ y. p y 2 2 p std2)/(std2^2); op ¼ h1 2 sum(h1(:))/prod(N); % make the filter sum to zero op ¼ op 2 sum(op(:))/prod(size(op)); % make the op to % sum to zero imgs ¼ double(image);% changing data type % Laplacian operation for i ¼ 1:x for j ¼ 1:y if(i , half_filter_size ll j , half_filter_size ll i . x-half_filter_size-1 ll j . y-half_filter_size-1) % boundary imgs2( j,i) ¼ 255; % white else M ¼ 0; for k ¼ 1:N


147

for l ¼ 1:N M ¼ M þ op(k,l) p imgs( j þ k-half_filter_size, i þ l-half_filter_size þ 2); end end imgs2( j,i) ¼ M; end end end m ¼ y 2 1; n ¼ x 2 1; % image size rr ¼ 2:m 2 1; cc ¼ 2:n 2 1; % The output edge map: e ¼ repmat(false, m, n); b ¼ repmat(false, m, n); b ¼ imgs2; thresh ¼ 0.15 p mean2(abs(b(rr,cc))); %thresh ¼ 1.0; % Look for the zero crossings: þ 2 , 2 þ and their transposes % We arbitrarily choose the edge to be the negative point [rx,cx ] ¼ find(b(rr,cc) , 0 & b(rr,cc þ 1) . 0… &abs(b(rr,cc) 2 b(rr,cc þ 1)) . thresh); % [2 þ ] e((rx þ 1) þ cx p m) ¼ 1; [rx,cx ] ¼ find(b(rr,cc 2 1) . 0 & b(rr,cc) , 0… &abs(b(rr,cc 2 1) 2 b(rr,cc)) . thresh); % [þ 2 ] e((rx þ 1) þ cx p m) ¼ 1; [rx,cx ] ¼ find(b(rr,cc) , 0 & b(rr þ 1,cc) . 0… &abs(b(rr,cc) 2 b(rr þ 1,cc)) . thresh); % [2 þ ]0 e((rx þ 1) þ cx p m) ¼ 1; [rx,cx ] ¼ find(b(rr 2 1,cc) . 0 & b(rr,cc) , 0… &abs(b(rr 2 1,cc) 2 b(rr,cc)) . thresh); % [þ 2 ]0 e((rx þ 1) þ cx p m) ¼ 1; % Most likely this covers all of the cases. Just check to see if there % are any points where the LoG was precisely zero: [rz,cz ] ¼ find(b(rr,cc) ¼ ¼ 0); if , isempty(rz) % Look for the zero crossings: þ 0 2 , 2 0 þ and their %transposes % The edge lies on the Zero point zero ¼ (rz þ 1) þ cz p m; % Linear index for zero points zz ¼ find(b(zero 2 1) , 0 & b(zero þ 1) . 0… &abs(b(zero 2 1) 2 b(zero þ 1)) . 2 p thresh); % [2 0 þ ]0 e(zero(zz)) ¼ 1; zz ¼ find(b(zero 2 1) . 0 & b(zero þ 1) , 0… &abs(b(zero 2 1) 2 b(zero þ 1)) . 2 p thresh); % [þ 0 2 ]0

148

Optomechatronics

e(zero(zz)) ¼ 1; zz ¼ find(b(zero 2 m) , 0 & b(zero þ m) . 0… &abs(b(zero 2 m) 2 b(zero þ m)) . 2 p thresh); % [2 0 þ ] e(zero(zz)) ¼ 1; zz ¼ find(b(zero 2 m) . 0 & b(zero þ m) , 0… &abs(b(zero 2 m) 2 b(zero þ m)) . 2 p thresh); % [þ 0 2 ] e(zero(zz)) ¼ 1; end % normalization image_laplacian ¼ (1 2 e) p 255; % showing image figure subplot(1,2,1); imshow(image); subplot(1,2,2); imshow(image_laplacian);

Hough Transform The Hough transform, developed by Hough (1962) is an effective way of segmenting objects with known shape and size within an image. It is designed specifically to find lines. An advantage of this approach is the robustness of segmentation and the result of the presence of imperfect data or noise. To introduce the underlying concepts of the transform, let us consider the simple problem of detecting a straight line in an image. Referring to Figure 3.30a, let us suppose that a straight line AB composed of infinitely many points including Pi ¼ ðxi ; yi Þ: Let the general equation of the line passing through the point Pi be yi ¼ mxi þ c On the other hand, infinitely

c

image space Pi+n

(xi+2, yi+2) (xi+1, yi+1) (xi,yi)

Pi

parameter m-c space …

(xi+n, yi+n) y = mx + c c = –mx + y

(a) multiple points on a line FIGURE 3.30 A line defined in two different spaces.

x

c = –Xi+nm + Yi+n

…

y

c'

m'

c = –Xim + Yi c = –Xi+1m + Yi+1 c = –Xi+2m + Yi+2 m

(b) the corresponding points mapped into parameter space


149

many lines will pass through ðxi ; yi Þ as long as it will satisfy the equation yi ¼ mxi þ c

ð3:42Þ

for some values of m and c. This implies that the above equation can also be defined in the parameter space composed of the parameters m and c. For example, all the straight lines going through the point Pi can be represented in the m – c space by c ¼ 2xi m þ yi

ð3:43Þ

Similarly, straight lines going through the point Piþn can be described in the same space by c ¼ 2xiþn m þ yiþn

ð3:44Þ

If those two points were on the same line in the image, the two line equation will be intersected at a point denoted by a dot ðm0 ; c0 Þ as indicated in Figure 3.30b. This indicates that any straight line in the image is mapped to a single point in the m – c parameter space, and any part of this line is mapped into the same point. Lines of any direction in an image may pass through any of edge pixels. In some cases, if we consider the slope and the intercept value of those lines to be bounded within a certain range ðmmax ; mmin Þ and ðcmax ; cmin Þ, the parameter space may be digitized into a subdivision of the space, the so called accumulator array composed of the cells of a rectangular structure as shown in Figure 3.31. These cells clearly depict the relation between the image space (x, y) and the parameter space ðm; cÞ: A(0,L-1)

A(K-2, L-1)

c

A(K-1, L-1) cmax

… A(K-1, L-2)

…

…

… A(0,1) cmin

A(0,0) A(1,0) mmin

FIGURE 3.31 Discretization of parameter space (m, c).

A(K-1,0)

… mmax

m

150

Optomechatronics

A cell Aðm; cÞ has the parameter values m and c in the parameter space. For each image point ðxl ; yl Þ in the image plane, when the lines of the allowed directions pass through this pixel, their corresponding parameters m and c are determined by the equation c ¼ 2xl m þ yl , yielding different Aðm; cÞ values. If a line is present in the image in the form of the equation y ¼ m0 x þ c 0 , the value of the Aðm0 ; c 0 Þ will be increased many times. For instance, if the points composing a line are detected M times, the value in Aðm; cÞ will be M, which indicates that M points in the image plane lie on the line y ¼ m0 x þ c 0 : Therefore, lines existing in the image will produce large values of the appropriate cells. This results in local maxima in the accumulator space. In the following, the Hough algorithm for line detection is summarized: Step 1: Quantize the parameter space (m, c) within the range mmin # m # mmax and cmin # c # cmax : Step 2: Form an accumulator array Aðm; cÞ and initialize it to some value. Step 3: Increment the value of an accumulator array Aðm; cÞ in relation to the points composing lines in the image. It is noted that the accuracy of the colinearity of the points lying on a line is dependent upon the number of discrete cells in the ðm; cÞ plane. If, for every point ðxl ; yl Þ, the m axis is quantized into K number, then the number of the corresponding c values will be K. This requires nK computation for each cell of Aðm; cÞ when n points in the image are involved. It is also noted that the larger the K value, the lower the resolution in the image space. The next task is to select a threshold that serves as a criterion to decide whether the value contained in a specific block can be a line or not. For this, we then examine the quantized blocks that contain more points than the threshold value. The blocks that pass this criterion are marked as a line in the processed image. The Hough transform method suffers when the line approaches the vertical (as m ! 1 and c ! 1). One way to avoid this difficulty is to represent a line as x sin u þ y cos u ¼ r

ð3:45Þ

which is depicted in Figure 3.32a. Construction of the accumulator array, in this case, is identical to the previous method used for slope – intercept representation (m, c). Here, again, a straight line in the image plane is transformed to a single point in the ðr; uÞ plane as shown in Figure 3.32b. Suppose that L collinear points are lying on a line x cos ui þ y sin ui ¼ r . This yields L sinusoidal curves that intersect at ðri ; ui Þ in the parameter space. It is noted here that the range of u lies within þ 908 and 2 908, and ranges from 0 to N where N is the image size, N £ N. When u ¼ 0, a line is horizontal and has a positive r . Similarly, when u ¼ 908, the line is vertical and has a positive r in þ x direction and when u ¼ 2908, r has a negative x-intercept.


ρ

y

151

A(0,L-1)

A(K-2, L-1) A(K-1, L-1) …

rmax

A(K-1, L-2) …

…

θ

…

ρ

x

(a) parameter space

rmin

A(0,1) A(0,0) A(1,0)

…

qmin

qmax

A(K-1,0) θ

(b) accumulation cell

FIGURE 3.32 Hough transform in (r, u) plane.

Hough transform to ðr; uÞ the plane follows the same procedure as discussed in the case of ðm; cÞ plane. The only difference is to use Equation 3.45 in order to determine r and u for each pair of ðx; yÞ in the image. Let us take two examples to understand how the transform actually works on the actual images. One example is made of a synthetic image composed of several polygons and the other is a real image of a camera placed upside down. Figure 3.33 illustrates the first example image in which 11 line edges are obtained by an edge operator. To these line images, the Hough transform algorithm is applied and the transformed result is plotted in r – u space. If those lines were perfectly represented by a line, they should be plotted as points in the space. However, the figure shows several streaks instead of points, since the extracted line edges are not a thin line and contain some noise as a result of edge processing. In the last figure, the original lines are successfully reproduced by checking the cells in the accumulator that has the local maximum value. Figure 3.34 shows an image of a camera and its frame from which we wish to find out some straight lines. It is noted from the extracted edges that the image contains several curved lines as well as straight lines. In addition, there appear several broken lines and noises. These lines and noises are all transformed into Hough r – u space. From this result, the straight lines are reproduced in the last figure. We can see that only a few lines are found from the transform although there appear many lines in the original input image. This is because the extracted edges contain noises and thick lines, in addition to circular arcs. The main advantage of using Hough transform lies in the fact that it is insensitive to missing parts of lines, image noise, and other parts of the image, exhibiting nonline characteristics. For example, a noisy or rough straight line will not yield a point in the parameter space but be transformed into a cluster of points. In this way, these are discriminated from straight lines in the image.

152

Optomechatronics

(b) extracted edges

(a) original image

50

50

100

100

150

150

200

200

250

250

300

300

350

350 50

100

150

200

250

(c) the result of Hough transformation

300

350

400

50 100 150 200 250 300 350 400

(d) detected lines

FIGURE 3.33 A Hough transformation of the image composed of polygon objects.

Camera Calibration In the pinhole camera model, a small hole is punched at the optical lens center in the camera coordinate system, through which some of the rays of light reflected by the object pass to form an inverted image of the object on the image plane. This is a camera perspective model as shown in Figure 3.35a, as discussed at the beginning of this chapter. Here, the camera lens is drawn in a dotted line to neglect its presence. A pinhole does not focus and, in fact, limits the entrance of incoming light, requiring long exposure time. No focusing required means that all objects are in focus, which, in turn, implies that the depth of field of the camera is ideally unlimited. However, the pinhole to film (image plane) distance affects the sharpness of the image and the field of view. In actual cameras, the pinhole is sufficiently opened by using a converging lens to avoid the disadvantage of the pinhole model as shown in Figure 3.35b. However, the actual lens


153

(b) extracted edges

(a) original image

50

50

100

100

150 150

200 250

200

300

250

350

50

100

150

200

250

300

(c) the result of Hough transformation

350

300

50

100

150

(d) detected lines

FIGURE 3.34 A Hough transformation of a real object image.

model causes focusing and aberration problems in reality. A difference from the previous model is that, in this model, the lens focuses and projects light rays from an object into the image plane, which is located in a specified distance from the camera lens. Disregarding aberration problem, let us consider the focus variation of the image plane. If an object is located at z ¼ s from the lens as shown in the figure, the following Gaussian lens formula holds 1 1 1 ¼ 0 þ f f s

ð3:46Þ

where f and f 0 are the focal length of the lens and the distance of the image plane from the center point of the lens, respectively. This f is called the effective focal length of the camera or camera constant. As we can see, when

154

Optomechatronics

virtual image object optical center

image sensor

zc real image real image

(a) pin hole model

optical lens

world point P

image plane zc image point P’

f

s

f’

(b) lens model FIGURE 3.35 The perspective camera model.

the object distance, s, varies from infinity to some distance close to the lens, f 0 slightly deviates from the focal length, f : This deviation may cause an error of projection, which cannot be tolerated in the case of the accurate depth measurement at close range. In this case, this parameter must be accurately determined by a camera calibration technique. To analyze the imaging process with a real image rather than the inverted one, we will deal with a virtual image plane denoted in Figure 3.35a. For simplicity, we call this “image plane” in the sequel. The objective of the image analysis is then how we map the image of an object acquired in the image plane into the world coordinate system. Referring to Appendix A1, the mapping conveniently can be described in the following form. up ¼ ½Mp Xw

ð3:47Þ

where up is the image vector of the object described in the image pixel coordinates and is projected from the 3D world coordinates Xw, and Mp is the resulting projection matrix. This is obtained by a perspective model discussed at the beginning of this chapter.


155

Perspective Projection A perspective transformation called the imaging transformation projects 3D points on to a plane. In other words, the transformation changes a 3D aggregate of the objects into a plane surface. To describe this in detail, we need to define four coordinate systems; With reference to Figure 3.36, the first one is the world coordinate system, {Xw } where an object to be imaged in 3D scene is located, the second one is the camera coordinate system {Xc } of which origin is located at the optical lens center, the third is the pixel coordinate system, {up } denoted by the coordinates of the pixels in the digital image, and finally, the fourth is the image coordinate system {ui }; which describes the same image in a different manner from the one described in the pixel coordinate. As shown in Figure 3.37, the image coordinates ðui ; vi Þ are located at the center of the pixel coordinate system ðu0 ; v0 Þ: As shown in the figure, the relationship between the pixel coordinate and image coordinate systems is, therefore, given by up ¼ u0 þ ku ui

vp ¼ v0 þ k v vi

ð3:48Þ

where ku and kv are the inverse of the horizontal and the vertical effective pixel sizes, su and sv, respectively, ku and kv are expressed in units of pixel £ m 21 while su and sv are interpreted as the size in meters for the horizontal and vertical pixels, respectively. Utilizing these coordinate systems, the whole imaging process can be described in the following sequence: coordinate system : coordinate points :

{Xw } 2 3 Xw 6 7 6 Yw 7 4 5

!

{Xc } 2 3 Xc 6 7 6 Yc 7 4 5

!

Zw

Yc

Yw

Xw

Zw Ow world coordinates

{ui } "

!

Zc

Xc

Oc camera coordinate

!

oi

op

up

FIGURE 3.36 The coordinate system for image analysis.

vi

#

{up } "

!

up

#

ð3:49Þ

vp

object (Xc,Yc, Zc) world point: P

(ui,vi)

vi

vp

ui

!

ui

optical axis

virtual image plane

Zc

156

Optomechatronics vi

(uo,vo) o image coordinates

vp

pixel coordinates

ui

up

ku : 1/ su kv : 1/ sv su : horizontal effective pixel size

pixel

sv

su

sv : vertical effective pixel size FIGURE 3.37 Pixel and image coordinates.

The first step to obtain the image interpretation is to obtain a relationship when a point ðXc ; Yc ; Zc Þ in the camera coordinate systems is projected onto the image coordinates ðui ; vi Þ: As indicated in Figure 3.36, under the assumption of the ideal projection, the transformation can be described by the projection equation ui ¼

f Xc ; Zc

vi ¼

f Yc Zc

ð3:50Þ

where f is the focal length of the camera. Note that these equations are nonlinear, since the depth variable Zc is at the denominator of each equation. These relationships can be utilized directly to avoid any complexity that arises from the nonlinearity. We may use alternatively homogeneous coordinates, which enable us to deal them in linear matrix from. As discussed in Appendix A1, the homogeneous coordinates of a point with respect to camera coordinates ðXc ; Yc ; Zc Þ are defined as ðlXc ; lYc ; lZc ; lÞ, where l is a nonzero constant. Conversion of homogeneous coordinates back to Cartesian coordinates can be made by dividing the first three homogeneous coordinates by l. Similarly, we can define the homogeneous coordinates of a point with the pixel coordinates ðup ; vp Þ by ðsui ; svi ; sÞ, where s is a nonzero constant. Then, the relationship in Equation 3.50 called


157

perspective matrix can be conveniently expressed in linear form, 2 3 3 Xc 3 2 2 7 f 0 0 0 6 sui 6 7 76 Yc 7 7 6 6 6 svi 7 ¼ 6 0 f 0 0 76 7 ~c or u~ i ¼ ½Hc X 56 7 5 4 4 6 Zc 7 0 0 1 0 4 5 s 1

ð3:51Þ

~ c in the where u˜i and X˜c are the augmented coordinate vectors of u~ i and X homogenous coordinate space. Using the rigid body transformation, Xc can be related to the world coordinates by the equation 2

lXc

3

2

r11

r12

r13

r22

r23

TX

32

XW

3

2

r T1

76 7 6 6 7 6 TY 7 76 YW 7 6 r T2 76 7¼6 76 7 6 6 ZW 7 6 r T3 r32 r33 TZ 7 54 5 4 0 0 1 1 0T " # .. ~ c ¼ ½Hw X ~ w ¼ ·R· · · .· · ·T· · X ~w X . 0 .. 1 6 7 6 6 lY 7 6 r 6 c 7 6 21 6 7¼6 6 7 6 6 lZc 7 6 r31 4 5 4 0 l

TX

32

XW

3

76 7 76 7 TY 76 YW 7 76 7 76 7 6 ZW 7 TZ 7 54 5 1 1

or ð3:52Þ

where R denotes the 3D rotation matrix, T denotes the 3D translational vector, and ri (i ¼ 1,2,3) is a row vector. Finally, from the relationship given in Equation 3.52, the relationship between the image and the pixel coordinates in the homogeneous coordinate space, 32 3 3 2 2 ui ku 0 u 0 sup 76 7 7 6 6 6 svp 7 ¼ 6 0 kv v0 76 vi 7 or u~ p ¼ ½Hu u~ i ð3:53Þ 54 5 5 4 4 s

0

0

1

1

Therefore, the overall imaging process is described by the relationship, using Equation 3.50 through Equation 3.53. 32 3 2 T XW 2 32 3 r 1 TX 7 76 ku 0 u0 f 0 0 0 6 76 7 6 6 76 76 r T2 TY 76 YW 7 7 6 7 7 6 7 6 u~ p ¼ 6 0 k v 0 f 0 0 or v 0 54 4 56 T 7 76 ð3:54Þ 6 r 3 TZ 76 ZW 7 54 5 0 0 1 0 0 1 0 4 1 0T 1 ~W u~ p ¼ ½H X where [H] is the 3 £ 4 camera projection matrix, which is denoted by 3 2 au 0 u0 7 6 7 ð3:55Þ H ¼ ½C ½R ½T ¼ 6 4 0 av v0 5½R ½T 0

0

1

158

Optomechatronics

where au and av are the image scaling factors, given by au ¼ ku f and av ¼ kv f ; and aspect ratio is given by au =av ¼ ku =kv : The above relationship indicates that au and av are dependent upon the distance of the focal length and the sizes of the pixels in the horizontal and vertical directions, respectively. It is noted that the parameters au ; av ; u0 ; and v0 do not depend on the position and orientation of the camera, and are, therefore, called the intrinsic parameters. The above equation can be rewritten by 2

H11

H14

3

6 u~ p ¼ 6 4 H21

H12

H13

H22

H23

7 ~ H24 7 5Xw

H31

H32

H33

H34

ð3:56Þ

When the above 12 elements are scaled by the value H34 , we have 11 variables to determine. The resulting equation is obtained by 3

2

2

up 7 6 0 6 7 6 s0 6 4 vp 5 ¼ 4 H21

H012

H013

H022

H023

H031

H032

H033

1

H011

3 2 3 Xw 7 6 6Y 7 7 6 7 w 7 6 H024 7 7 56 6 Zw 7 5 4 1 1

H014

ð3:57Þ

where s0 is given by s=H34 : This yields the camera calibration equation from which the relationship between the pixel coordinates and the world coordinates can be established. To illustrate the relationship in more detail, let us consider a point, say ith point, in the world coordinates {X} and the corresponding point imaged in the pixel coordinates {up }: Use of Equation 3.57 leads to 3 H011 7 6 6 0 7 6 H12 7 7 6 7 6 6 0 7 6 H13 7 7 6 7 6 6 0 7 6 H14 7 7 6 7 6 6 0 36 H21 7 7 2 3 i i 7 6 2up Zw 6 u 7 76 0 7 4 p 5 56 H22 7 ¼ 7 6 vp 7 2vip Ziw 6 6 0 7 6 H23 7 7 6 7 6 6 0 7 6 H24 7 7 6 7 6 6 H0 7 6 31 7 7 6 7 6 6 H0 7 6 32 7 5 4 2

2 6 4

i Xw Yiw Ziw 1

0

0

0

0

0

0

i 0 2uip Xw 2uip Yiw

i i 0 Xw Yiw Ziw 1 2vip Xw 2vip Yiw

H033

ð3:58Þ


159

In deriving the above equation, we have used the following relationship; s0 ¼ H031 Xw þ H032 Yw þ H033 Zw þ 1 which can be obtained from Equation 3.57. It needs to be pointed out that, at least, we need 12 equations like these that can be obtained by six calibration points. The equation can be written in a more compact form, Aq ¼ b

ð3:59Þ

where A represents the left-hand side matrix of the H vector and b represents the right-hand side of the equation. Solving for q yields q ¼ ½AT A

21

AT b ¼ Aþ b

ð3:60Þ

where Aþ is the pseudo inverse of A. Here, we have 11 unknowns for qs to be determined and, therefore, need at least six calibration points. Before we proceed to solve Equation 3.60, let us consider the transformation from a point in the image coordinates Pi to a point in world coordinates without a camera perspective model. Referring to Figure 3.38, this can be obtained by considering the absolute position vector of point Pw ðxw ; yw ; zw Þ with respect to the world coordinates, which is equivalent to point Pi ðui ; vi Þ: The position vector is expressed by rw ¼ rc þ ri þ rp

ð3:61Þ

where rw is the absolute position vector of point Pw expressed with respect to the world coordinates, rc is the position vector of the center of the camera with respect to the center of the world, ri is the position vector of the image coordinates with respect to the camera center Oc, and finally rp is the position vector of Pi with respect to the center of the image coordinates, Oi : The camera shown in the figure has usually two rotational motions denoted by the angles a and b, which are called, respectively, tilt and panning. The sequence of positioning point Pi in the image coordinates to the corresponding point Pw in the world coordinates is to obtain the camera transformation matrix in Equation 3.54. The first translation gives 2

1

6 60 6 H1 ¼ 6 6 60 4 0

0 0 1 0 0 1 0 0

xw

3

7 yw 7 7 7 7 zw 7 5 1

ð3:62Þ

160

Optomechatronics Zc β

image coordinates {I}

pan-tilt device Xc α camera coordinates {C}

rp

ri Oi

Oc

Yc

Pi (ui ,vi ,0)

u

v

z optical axis

rc Zw

Ow Yw

Pw(xw, yw, zw)

Xw

world coordinates {W }

FIGURE 3.38 Geometry for camera coordinate transformation.

where xw, yw and zw are the positioning vectors of the center of the camera coordinates. The two rotations, tilting and panning, yield the following transformation: For panning (b) 2

cos b

6 6 sin b 6 H2 ¼ 6 6 6 0 4 0

2sin b

0

cos b

0

0

1

0

0

0

3

7 07 7 7 7 07 5 1

ð3:63Þ


For tilting (a) 2 1 0 6 6 0 cos a 6 H3 ¼ 6 6 6 0 sin a 4 0

0

0 2sin a cos a 0

0

161

3

7 07 7 7 7 07 5 1

ð3:64Þ

The translation from the origin of the camera coordinates with respect to that of the image coordinate is expressed by 3 2 1 0 0 xi 7 6 60 1 0 y 7 6 i7 7 ð3:65Þ H4 ¼ 6 7 6 6 0 0 1 zi 7 5 4 0 0 0 1 where ðxi ; yi ; zi Þ is the position vector of the image coordinate center. As can be seen from the figure, the orientation of the image coordinates is different from that of the camera coordinates. Therefore, the frame needs to be rotated 908 about Zc axis, which yields 2 3 0 21 0 0 6 7 61 0 0 07 6 7 7 H5 ¼ 6 ð3:66Þ 6 7 60 0 1 07 4 5 0 0 0 1 Then, another rotation about YC yields 2 3 0 0 1 0 6 7 6 0 1 0 07 6 7 7 H6 ¼ 6 6 7 6 21 0 0 0 7 4 5 0 0 0 1

ð3:67Þ

If i rp is the position vector of Pi , the transformation of the Pi into the Pw is obtained by w

rp ¼ ½H1 ½H2 ½H3 ½H4 ½H5 ½H6 i rp

ð3:68Þ

This yields the relationship between Pw ðxw ; yw ; zw Þ in the world coordinates and Pi ðui ; vi ; 0Þ in the image plane. We have seen that a point defined in the world coordinates (known) can be mapped into a point defined in the image coordinates (measured) from Equation 3.68. To obtain the transformation matrix H, we need to solve Equation 3.58 or Hij s. However, H contains the intrinsic and the extrinsic parameters, which are

162

Optomechatronics

difficult to obtain by a direct measurement. Furthermore, at least six points located in the world coordinates and the corresponding points in the image coordinates need to be exactly known. This necessitates a measurement technique called “camera calibration.” The calibration method requires geometrically known points called calibration marks. The choice of the marks depends upon the conditions that the camera calibration needs to be carried out, for example, the light conditions, the coloring of the mark and the accuracy of the calibration. The choice of the marks depends upon the conditions under which the camera calibration condition is carried out. Such conditions include the light conditions, the coloring of the mark, and the accuracy of the calibration. Figure 3.39 shows an actual geometrical configuration set for the camera calibration. The calibration rig is composed of an object to be calibrated, a CCD camera, and an optical rail on which the camera can travel within a certain distance. The object contains an evenly spaced pattern of circles filled in white and is located in a known location along the Z axis from the camera position of which center is at Oc The first step is to obtain the image of the pattern and calculate the locations of the center of the circle given in the pixel coordinates, based on the acquired image. Figure 3.40 shows two acquired images taken by the camera at two different locations at Zw ¼ 0 mm and Zw ¼ 250 mm. In this procedure, any information on the circle such as the features of the circle must be given for the calculation. Here, the centroids of the circles are used. As indicated in Figure 3.41, to determine this, we need to use the entire image processing procedures discussed in the previous section, selection of region of interest (ROI), image segmentation, and enhancement, detection, and determination of the mass center of each circle. Table 3.3 indicates the coordinate values of the centroids of the circles with respect to the world coordinate system.

Yw

Xw

Xc

camera

ical

optical center, Oc

opt Yc

FIGURE 3.39 Camera calibration set-up.

Zw

Zc

rail


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

(a) image 1 ( Zw=0mm, “circle_0.raw ”)

163

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

(b) image 2 ( Zw=250mm, “circle_250.raw”)

FIGURE 3.40 The pattern images acquired at two different locations.

Utilizing the locations of the circles obtained in the pixel coordinate values and the world coordinate values, we wish to compute the intrinsic parameters and the calibration transformation matrix given in Equation 3.54. Plugging these data into Equation 3.54, we obtain the following calibration

input image

selection of ROI (region of interest)

segmentation by labeling

edge detection

mass center using edge of each circle

FIGURE 3.41 The procedure to determine the centers of the circle patterns.

164

Optomechatronics

TABLE 3.3 The Centroid of the Circles with respect to World Coordinate System Feature Point

X (mm)

Y (mm)

1-1 1-2 1-3 1-4 1-5 1-6 1-7 1-8 1-9 1-10 1-11 1-12 1-13 1-14 1-15 1-16 1-17 1-18 1-19 1-20 1-21

2120 280 240 0 40 80 120 2120 280 240 0 40 80 120 2120 280 240 0 40 80 120

200 200 200 200 200 200 200 160 160 160 160 160 160 160 120 120 120 120 120 120 120

2-1 2-2 2-3 2-4 2-5 2-6 2-7 2-8 2-9 2-10

280 240 0 40 80 280 240 0 40 80

160 160 160 160 160 120 120 120 120 120

matrix H. 2 0 H11 6 6 0 H¼6 6 H21 4 H031

H012

H013

H022

H023

H032

H033

H014

Z (mm) Feature Point Image 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Image 2 250 250 250 250 250 250 250 250 250 250

3

2

1:8813

7 6 7 6 0:0021 H024 7 7¼6 5 4 0 1

X (mm)

Y (mm)

1-22 1-23 1-24 1-25 1-26 1-27 1-28 1-29 1-30 1-31 1-32 1-33 1-34 1-35 1-36 1-37 1-38 1-39 1-40 1-41 1-42

2 120 2 80 2 40 0 40 80 120 2 120 2 80 2 40 0 40 80 120 2 120 2 80 2 40 0 40 80 120

80 80 80 80 80 80 80 40 40 40 40 40 40 40 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

2-11 2-12 2-13 2-14 2-15 2-16 2-17 2-18 2-19 2-20

2 80 2 40 0 40 80 2 80 2 40 0 40 80

80 80 80 80 80 40 40 40 40 40

250 250 250 250 250 250 250 250 250 250

0:0054

20:3737

21:8833 20:2707 0

20:0012

Z (mm)

321:0669

3

7 7 427:9406 7 5 1 ð3:69Þ


165

Examination of Equation 3.57 indicates that this result enables us to relate a point Pp ðup ; vp Þ in the pixel coordinates to a point Pw ðxiw ; yiw ; ziw Þ in the world coordinates.

Problems P3.1. Figure P3.1 shows an image having a certain gray-level histogram. If we wish to modify the original histogram such that it is stretched over a certain range and that it slides with some offset from the original histogram, as shown in the figure, explain what kind of changes in the image will occur for both cases.

(a) stretching

(b) sliding FIGURE P3.1 Histogram modification.

P3.2. Finding the connected components in an image is common practice in machine vision. A labeling algorithm finds all connected components in the image and assigns a label to all points in the same component. For an image shown in Figure P3.2, explain how the labeling algorithm can work to segment each individual component. P3.3. Binary image varies greatly depending on which threshold value is used. Figure P3.3 shows an image of a tree leaf. Obtain the binary images at T1 ¼ 150 and T2 ¼ 180 and explain the results by comparing the obtained binary images.

166

Optomechatronics

255 255 255 255 255 255 255 255 255 255 255 255 255 255 25 25 255 255 255 255 255 255 255 25 25 25 25 255 255 255 255 255 25 25 25 25 25 25 255 255 255 255 25 25 25 25 25 25 255 255 255 255 255 25 25 25 25 255 255 255 255 255 255 255 25 25 255 255 255 255 255 255 255 255 255 255 255 255 255 255 FIGURE P3.2 A mosaic image.

FIGURE P3.3 Image of a tree leaf.

P3.4. In Figure P3.4(a) and (b), two simple series of mosaics of 1s and 0s mimicking gray level image are given in an 8 £ 10 window. We wish to determine their edge using Robert and Laplacian operators. (1) Obtain the gradient vector Gx and Gy of the image (a), using a Robert operator and find the edge, using T ¼ 220: (2) Obtain the Laplacian operation of the image (b) and find the zerocrossing line from the result.


255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 25 25 25 25 255 255 255

167

20 20 20 20 20 80 80 80 80 80 20 20 20 20 20 80 80 80 80 80

255 255 255 25 10 10 25 255 255 255

20 20 20 20 20 80 80 80 80 80

255 255 255 25 10 10 25 255 255 255

20 20 20 20 20 80 80 80 80 80

255 255 255 25 25 25 25 255 255 255

20 20 20 20 20 20 20 20 20 20

255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255

(a)

20 20 20 20 20 20 20 20 20 20

(b)

FIGURE P3.4 A series of mosaic images.

P3.5. The image of an array resistance is shown in Figure P3.5. Use the Sobel operator to detect the edge at three different arbitrary values of threshold.

FIGURE P3.5 Image of an array resistance.

P3.6. Figure P3.6 contains two different noise levels. Obtain the filtered images of the images using a median filter with mean ¼ 0.0 and standard deviation ¼ 2.0. P3.7. A line in the x-y coordinate frame is shown in Figure P3.7. Let points P1 and P2 be located on the line. (1) Explain why the polar coordinate representation of lines, r ¼ x cos u þ y sin u, is more suitable for line detection using the Hough transform than the standard line representation, y ¼ mx þ c: (2) Explain how the standard line representation y ¼ mx þ c can be converted to the polar coordinate representation, r ¼ x cos u þ y sin u:

168

Optomechatronics

FIGURE P3.6 Images of a pair of cutters with two different noise levels.

y

P2 P1

(−1,1+ 3) (− 2,1) O

x

FIGURE P3.7 A line in x-y coordinate frame.

(3) For the line shown in the figure, show the standard line representation and the polar coordinate representation of the line. P3.8. Let an image plane be with 300 £ 400 pixel size as shown in Figure P3.8. For extracting the line information from the image acquired in this image plane, a Hough transform is generally used. Here, we would like to get the specified resolution of the extracted line, and Dr ¼ 1 pixel. (1) Design the accumulator array AðuÞ: Assume that the image quantization effect is negligible. Determine m and c values where m and c are the horizontal and vertical sizes of the accumulator, respectively. (2) On this image plane, a line denoted by AB is acquired as shown in Figure P3.8. Which cell of the accumulator is corresponding to the line?


169 q = ( 400, 300 )

vp

lA ( 100, 200 )

( 300, 100 ) B up FIGURE P3.8 A line in image coordinate frame.

(3) When this line is reconstructed from the corresponding accumulator cell, how much is the parametric error of the line equation? P3.9. A camera is located relatively to the world coordinate frame {W}, as shown in Figure P3.9. We wish to map a point in world coordinates on to the corresponding points in the image coordinates frame. (1) The transformation between the world coordinates and the camera coordinates can be expressed by using the euler angle (f,u and c)

Yc Xc Zc {C} Zw T = [a, b, c]

f = 90° q = 90°

Yw Xw

{W}

FIGURE P3.9 A configuration of camera and world coordinate frames.

y = 90° T= [0, –100, –100]

170

Optomechatronics

and translation ða; b; cÞ and is given by 2

Xc

3

6 7 6Y 7 6 c7¼ 6 7 6 7 6 Zc 7 4 5 1

c

2 6 6 6 6 6 6 4

Tw

R … 0

32

Xw

3

76 7 .. Yw 7 . T7 76 6 7 76 7 76 .. 7 6 Z . …7 54 w 7 5 .. . 1 1

where c Tw is given by 2

cos f cos u cos c 2 sin f sin c 2cos f cos u sin c 2 sin f cos c cos f sin u a

3

7 6 6 sin f cos u cos c þ cos f sin c 2sin f cos u sin c þ cos f cos c sin f sin u b 7 5 4 2sin u cos c

sin u sin c

cos u

c

Let the intrinsic parameters of the camera be given by f ¼ 16 mm, ku ¼ kv ¼ 50 mm21, u0 ¼ 320, v0 ¼ 240. (1) Find the transformation matrix H in Equation 3.52. (2) If a point Pw is located at Xw ¼ ½200; 600; 200 in the world coordinates, what is the corresponding up in the pixel coordinates? ~w P3.10. Consider a perspective transformation having u~ p ¼ HX where 2

h11

h14

3

6 H¼6 4 h21

h12

h13

h22

h23

7 h24 7 5

h31

h32

h33

h34

and let q1 ¼ h11 h12 q4 ¼ h14 h24 h34 :

h13

q2 ¼ h21

h22

h23

q3 ¼ h31

h32

h33

(1) Show that u0 and v0 can be expressed as u0 ¼ q1 qT3 ; v0 ¼ q2 qT3 : qffiffiffiffiffiffiffiffiffiffiffiffiffi qffiffiffiffiffiffiffiffiffiffiffiffi (2) Show that au ¼ q1 qT1 2 u20 av ¼ q2 qT2 2 v20 : P3.11. Consider the camera calibration configuration presented in the section “Camera Calibration.” A pattern image is obtained as shown in Figure P3.11.


171

FIGURE P3.11 A pattern image obtained a Zw ¼ 150 mm.

(1) Determine the coordinate values of the centroids of the circles from this image. (2) Compare this result with the one obtained by the camera calibration matrix. The absolute coordinate values of the circles are shown in Table P3.1. Discuss the error caused by calibration.

TABLE P3.1 World Coordinate Values of the Centers of the Circle Patterns Feature Point

X (mm)

Y (mm)

Z (mm)

Feature Point

X (mm)

Y (mm)

Z (mm)

2 120 2 80 2 40 0 40 80 120 2 120 2 80 2 40 0 40 80 120

80 80 80 80 80 80 80 40 40 40 40 40 40 40

150 150 150 150 150 150 150 150 150 150 150 150 150 150

Image 3-1 3-2 3-3 3-4 3-5 3-6 3-7 3-8 3-9 3-10 3-11 3-12 3-13 3-14

2120 280 240 0 40 80 120 2120 280 240 0 40 80 120

160 160 160 160 160 160 160 120 120 120 120 120 120 120

150 150 150 150 150 150 150 150 150 150 150 150 150 150

3-15 3-16 3-17 3-18 3-19 3-20 3-21 3-22 3-23 3-24 3-25 3-26 3-27 3-28

172

Optomechatronics

References [1] Ballard, D.H. and Brown, C.M. Computer Vision, Prentice Hall, Englewood Cliffs, NJ, 1982. [2] Bishop, R.H. The Mechatronics Handbook, CRC Press, Boca Raton, 2002. [3] Corke, P.I. Visual Control of Robots, Research Studies Press Ltd., Taunton, UK, 1996. [4] Datatranslation PCI Frame Grabber Product Catalog, Datatranslation Co. Ltd., http://www.datatranslation.com/top_products_pages/imaging-PCI.htm, 2005. [5] Fu, K.S., Gonzalez, R.C. and Lee, C.S.G. Robotics Control, Sensing, Vision, and Intelligence, McGraw Hill lnc., New York, 1987. [6] Gonzalez, R.C. and Woods, R.E. Digital Image Processing, Addison Wesley Publishing Co., Reading, MA, 1992. [7] Gonzalez, R.C., Woods, R.E. and Eddins, S.L. Digital Image Processing Using MATLAB, Prentice hall, Englewood Cliffs, NJ, 2004. [8] Horn, B.K.P. Robot Vision, The MIT Press, McGraw-Hill Book Co., 1986. [9] Jahne, B., Haupecker, H. and Geipler, P. Handbook of Computer Vision and Applications, System and Applications, Vol. 3. Academic Press, New York, 1999. [10] James, R. and Carstens, P.E. Automatic Control Systems and Components, Prentice Hall, Englewood Cliffs, NJ, 1990. [11] Jain, R., Kasturi, R. and Schunck, B.G. Machine Vision, McGraw-Hill, New York, 1995. [12] Metero Frame Grabber Product Catalog, Matrox Co. Ltd., http://www.matrox. com/imaging/products/frame_grabbers.cfm, 2005. [13] Petrou, M. and Bosdogianni, P. Image Processing: The Fundamentals, Wiley, New York, 1999. [14] Ridler, T.W. and Calvard, S. Picture thresholding using an iterative selection method, IEEE Transactions SMC, 8:8, 630–632, 1978. [15] Stadler, W. Analytical Robotics and Mechatronics, McGraw-Hill, New York, 1995. [16] Umbaugh, S.E. Computer Vision and Image Processing, Prentice Hall PTR, Upper Saddle, NJ, 1998. [17] Watson, D.M. Lab Manual, Astronomy 111, The University of Rochester, Rochester, NY, 2000. [18] Xie, M. Fundamentals of Robotics, World Scientific Publishing Co. Pte. Ltd., Singapore, 2003.

4 Mechatronic Elements for Optomechatronic Interface CONTENTS Sensors ................................................................................................................. 175 Capacitive Sensor....................................................................................... 176 Differential Transformer............................................................................ 177 Piezoelectric Sensors.................................................................................. 179 Pyroelectric Sensor..................................................................................... 185 Semiconductor Sensors; Light Detectors................................................ 186 Photodiode .................................................................................................. 189 Other Photodetectors................................................................................. 191 Photovoltaic Detectors ....................................................................... 192 Avalanche Photodiode....................................................................... 192 Signal Conditioning........................................................................................... 193 Operational Amplifiers ............................................................................. 193 Inverting Amplifier ............................................................................ 193 Noninverting Amplifier .................................................................... 194 Inverting Summing Amplifier.......................................................... 195 Integrating Amplifier......................................................................... 195 Differential Amplifier......................................................................... 196 Comparator ......................................................................................... 196 Signal Processing Elements ...................................................................... 197 Filters .................................................................................................... 197 Digital-to-Analog Conversion .......................................................... 198 Analog-to-Digital Converters ........................................................... 199 Sample and Hold Module................................................................. 200 Multiplexer .......................................................................................... 201 Time Division Multiplexing.............................................................. 201 Wheatstone Bridge ............................................................................. 202 Isolator.................................................................................................. 203 Microcomputer System ............................................................................. 204 Microcomputer ................................................................................... 204 Input/Output Interface ..................................................................... 207

173

174

Optomechatronics

Microcontrollers.................................................................................. 207 Sampling of a Signal.................................................................................. 207 Actuators ............................................................................................................. 210 Electric Motors............................................................................................ 210 Piezoelectric Actuator ............................................................................... 212 Voice Coil Motor (VCM) ........................................................................... 216 Electrostatic Actuator ................................................................................ 218 Microactuators.................................................................................................... 220 Shape Memory Alloy (SMA) Actuator................................................... 221 Magnetostrictive Actuator ........................................................................ 222 Ionic Polymer Metal Composite (IPMC) Actuator............................... 223 Signal Display..................................................................................................... 224 Dynamic Systems and Control ........................................................................ 225 Dynamic Systems Modeling .................................................................... 226 Thermal Systems ................................................................................ 227 Spring-Mass-Damper System ........................................................... 227 Fluid System........................................................................................ 228 Optical Disk......................................................................................... 229 Dynamic Response of Dynamical Systems............................................ 230 System Transfer Function ......................................................................... 233 First Order System ............................................................................. 234 Second Order System......................................................................... 235 Higher Order System......................................................................... 236 Laplace Transforms Theorems ......................................................... 236 Open Loop vs. Feedback Control .................................................... 237 System Performance .................................................................................. 238 Basic Control Actions ................................................................................ 241 System Stability .......................................................................................... 244 Problems.............................................................................................................. 245 References ........................................................................................................... 252

As mentioned in the Introduction, mechatronics is an integrated discipline in which optical, mechanical, electrical, and computer technologies are embedded together. Therefore, this discipline encompasses a variety of technical fields ranging from mechanism design, sensor and measurement, signal conditioning and processing, drive and actuator, system control, microprocessor, and so on, as shown in Figure 1.11. These are considered to be the key areas of mechatronics. In this section, we will deal with some detailed fundamental methodologies involved with the technical fields, but, in particular, those that may be effectively combined with optical elements for optomechatronic integration. In the first part, sensors and actuators are introduced. In recent years, a great deal of new materials and transduction methods have been developed. In addition, an abundance of micro sensors and actuators have

Mechatronic Elements for Optomechatronic Interface

175

appeared in recent years owing to development of microfabrication methods; these sensors and actuators are expected to grow at a very fast rate due to down-sizing of macroparts and wide applications of MEMS parts. We will discuss here some of these micro sensors and actuators in brief; the majority of these micro sensors and actuators here means the sensors and actuators that operate on orders of mm range or less, regardless of their physical size. Most sensors to be dealt with have a small range of measurement but high resolution such as capacitive and piezoelectric sensors. Other sensors based on semiconductor technology such as light detecting, piezoresistive are also treated because of the variety of their use in optomechatronic integration. Actuators to be discussed here cover a variety of actuating principles including piezoelectric, capacitive, electromagnetic, material phase transformation, magnetostrictive, and so on. Most of these actuators have a relatively small range of actuation. We will also discuss DC motor which is a rotary actuator and has been used in a variety of optomechatronic systems. We shall make use of some of these actuators in order to produce optomechatronic integration in the later part of this book. In the second part, we will discuss the components for signal conditioning such as operational amplifiers, filters, comparators, multiplexer, microcomputer, and then signal sampling via analog to digital (A/D) converter, sending-out output signal from microcomputer via digital to analog (D/A) converter. In the last part, we will finally discuss some of elementary concepts of system modeling, transfer function, system response, and basic feedback controllers.

Sensors Sensing is the most fundamental technique that senses the physical variables being measured. The sensor is a physical element that does this, and contains one or more transducers within it. The transducer converts or transforms one form of energy to another form in a detectable signal form: The energy includes various types of form-mechanical, electrical, optical, chemical, and so forth. In view of this, the transducer is an essential element of a sensor which needs to be contained within it. The transformed raw signal usually contains low amplitude, noise contaminated, narrow sensing range, nonlinearity, and so on. To make these raw signal forms conditioned into desirable forms, signal conditioning units are usually necessary. Some sensors contain these conditioning units as a part of their body but some are equipped with these units outside. The recent trend, however, due to downsizing of products, is that most of the necessary elements are put together within a single body. Semiconductor sensors belong to one of the groups that lead this trend. This trend can also be found from intelligent sensors in which dissimilar multiple sensors are

176

Optomechatronics

embedded together to measure a variety of different physical quantities. In this section, focusing piezoelectric sensors and light detecting sensors, we will discuss fundamental concepts of several sensors that are frequently used for optomechatronic systems. Capacitive Sensor When two isolated conductive objects or plates are connected to the positive poles of a battery, the plates will receive equal amounts of opposite charge. If the plates are disconnected from the battery, they will remain charged as long as they are in a vacuum. A pair of these plates called a capacitor has a capability of holding electrical charge whose capacitance depends on the magnitude of charge, q, and the potential difference between the plates, V. The capacitance of the parallel plate capacitor is given by q ð4:1Þ C¼ V In terms of the permittivity constant of free space, and the relative permittivity of dielectric material, we can write the capacitance between the plates as C¼

10 1r A d

ð4:2Þ

where 10 is the permittivity of free space, 1r is the relative permittivity of the dielectric material between two plates, A is the area of the plates, and d is the distance between the plates. The relative permittivity is often called the dielectric constant which is defined by 1=10 ¼ 1r, 1 being defined by the

TABLE 4.1 Dielectric Constants of Some Materials (T 5 258C) Material

1r

Vacuum Air Water Benzene Rubber (silicon) Plexiglas Polyesters Ceramic (alumina) Polyethylene Silicon resins Epoxy resins Nylon Lead nitrate

1 1.00054 78.5 2.28 3.2 3.12 3.22 to 4.3 4.5 to 8.4 2.26 3.85 3.65 3.50 37.7


w

V

177

V

d x

(a) area A overlaps

d

(b) gap changes

V

d

dielectric material

(c) dielectric material moves FIGURE 4.1 Change of capacitance between two plates.

permittivity of the material. The dielectric constants of some materials are given in Table 4.1. As shown in Figure 4.1 there are three ways of achieving capacitance variation. In Figure 4.1a, the displacement of one of the plates changes the area of overlap. Figure 4.1b shows the case of varying the distance d between the plates, whereas in Figure 4.1c the displacement of the dielectric material causes a change in capacitance. As one example of such changes, let us suppose that the plate separation d is changed by the amount Dd, and then the resulting change in C will be given by DC ¼ 2

10 1r A Dd d þ Dd d

which indicates a nonlinear relationship between them. This type of sensor has several advantages: noncontact, high accuracy, and high resolution. In addition, stability is another advantage, because it is not influenced by pressure or temperature of the environment. Differential Transformer The differential transformer employing the principle of induction shown in Figure 4.2 provides an AC voltage output proportional to the displacement of the magnetic core passing through the windings. It is essentially a mutual inductance device which consists of three coils symmetrically spaced along an insulated circular tube. The center coil which is the primary coil is energized from an AC power source and the two identical end coils which are the secondary coils are connected together in series. The connection is made in such a way that its outputs are produced out of phase with each

178

Optomechatronics output voltage Ve

magnetic core secondary coil 2

input displacement

primary coil

secondary coil 1

input voltage i FIGURE 4.2 Schematic view of linear variable differential transformer (LVDT).

output voltage

other. When the magnetic core is positioned at the central location the voltage induced (electromotive force emf) in each of the secondary coils will be of the same magnitude, and thus the resulting net output is zero. When the core moves farther from the central position, there will be an increase in the output within a certain limit on either side of the null position. Figure 4.3 illustrates the characteristic curve of output voltage vs. core displacement. The phase difference between the outputs of the two regions is 1808, i.e., out of phase, as can be seen from the figure. Recalling Faraday’s law

A

O

A O

FIGURE 4.3 Typical output voltage of LVDT.

B

core displacement

B core displacement


179

of induction, the voltage (emf) induced in a secondary coil due to change in current i flowing in the primary coil is equal to Ve ¼ 2M

di dt

ð4:3Þ

where M is the coefficient of mutual inductance between two coils. The minus sign indicates the direction of the induced voltage. Since there are two secondary coils in the vicinity of the primary coil, we have two induced emfs for the coils. Thus, if an input current i ¼ i0 sinvt flows through the primary coil, each emf is given by Ve1 ¼ C1 sinðvt 2 wÞ

Ve2 ¼ C2 sinðvt 2 wÞ

where Ve1 and Ve2 denote the emf of the secondary coil one and two, respectively, the constants, C1 and C2, are determined by the position of the core relative to the secondary coils one and two, and w is the phase difference between the primary and secondary voltages. As can be seen from Figure 4.2, the two secondary emfs are connected in series, and therefore the output voltage becomes Ve1 2 Ve2 ¼ ðC1 2 C2 Þsinðvt 2 wÞ It is noticed that the sign and magnitude of ðC1 2 C2 Þ depend on where the magnetic core position is located at the instant of its motion. If C1 ¼ C2 , then the core is located at the null position. If C1 . C2 , the portion of the core positioned in the secondary coil 1 is greater than that in the coil 2, whereas C2 . C1 is the reverse case. This sensor has several advantages for displacement measurement. One of the advantages is noncontact, which eliminates friction and thus gives high resolution. Piezoelectric Sensors Certain materials become electrically polarized when they are subject to mechanical strain. This effect is known as the piezoelectric effect discovered by the French Curie brothers in 1880. The reverse effect is called the inverse piezo effect which is widely used in piezo actuators. Piezoelectric materials can be categorized into single crystals, ceramics, and polymers. Notable among the crystals materials are quartz and Rochelle salt. Barium titanate, lead zirconate titanate (PZT), and ZnO belong to ceramic materials, and polyvinylidene fluoride (PVDF) is an example of a piezoelectric polymer. The piezoelectric effect stays up to the Curie point, since it is dependent upon the temperature of the material. Above that point, the ceramic material has a symmetric polycrystal structure, whereas below the point it has nonsymmetric crystal structure. When it is under the symmetric state, it cannot produce the piezoelectric effect since the centers of the positive and negative charge sites coincide. When it exhibits nonsymmetric structure due

180

Optomechatronics

V

(a) warmed up state

(b) placed in strong electric field V

FIGURE 4.4 Thermal poling of piezo material.

to a phase transformation of the material, the centers of the positive and negative charge sites no longer coincide. In this case, the material exhibits the piezoelectric effect. The Curie point depends on the material, for example, piezo ceramics (150 – 4008C). There are several different procedures for producing the piezoelectric effect called “polarizing” procedure. In the case of material such as PZT and barium titanate, the procedure starts with heating it to a temperature above the Curie point (1208C). This step makes some electric dipoles within the materials which were initially randomly oriented align in a desirable direction. Then, a high DC voltage is applied across the faces, and the material is cooled down while keeping the same electric field across its thickness. Finally, when the electric field is removed, its polarization stays permanent as long as the polarized material is kept below the Curie point. Figure 4.4 shows two states of the electric field in the PZT developed in the course of poling, as indicated with arrows. In Figure 4.4a, the PZT is electrically neutral when no heat is applied, whereas the state of the PZT in Figure 4.4b is electrically charged in a certain direction shown with arrows due to shift of atoms inside the material, when electric field is applied. This phenomenon leads to the development of electric charge on its surface in response to a mechanical deformation: one face of the material becomes positively charged and the opposite face negatively charged, resulting in an electric potential as shown in Figure 4.5a. In Table 4.2, various properties of piezoelectric materials at T ¼ 208C are listed. The charge developed is proportional to the applied force, F, and is expressed as q ¼ Sq F

ð4:4Þ

where Sq is the piezoelectric constant or charge sensitivity of the piezoelectric material and depends upon the orientation of the crystals material and the way in which the force is applied. The charge sensitivity coefficient is listed in Table 4.3. To pick up an accumulated electric charge, conductive electrodes are applied to the crystal at the opposite sides of the material cut surface. From this point of view, the piezoelectric transducer is considered to be a parallel plate capacitor. The capacitance of the piezoelectric material


181

force cable

A + + + + + + + + + +

piezoelectric transducer

electrodes

h

charge charge amplifier amplifier

_ _ _ _ _ _ _ _ _ _

(a)

(b)

force

Cf

Rf

i

Cp

Rp

−

Cc

i

+

(c) piezoelectric transducer cable

charge amplifier

Cp

Rp

Vo

(d)

FIGURE 4.5 Piezoelectric sensor connected to charge amplifier.

between the plates is given by Cp ¼

10 1r A h

ð4:5Þ

where 1r is the relative permittivity of the piezoelectric material, 10 is the permittivity of the free space, and A and h are the area and thickness of the material, respectively. Since Cp ¼ q=V where V is the potential difference between the electrode plates, combining Equation 4.4 and Equation 4.5

TABLE 4.2 Various Properties of Piezoelectric Materials (T 5 208C)

Material Quartz Barium titanate PZT Polyvinylidene fluoride

Density r, 103(kg/m3)

Young’s Modulus E, (109N/m)

Dielectric Constant 1r

Piezoelectric Constant Cp (pC/N)

1.78 5.7 7.5 1.78

77 110 830 0.3

4.5 1700 1200 12

2.3 78 110 d31 ¼ 20, d32 ¼ 2, d33 ¼ 30

182

Optomechatronics

TABLE 4.3 Typical Charge Sensitivities Sq of Piezoelectric Materials Material Quartz SiO2 Single crystal Barium titanate (BaTiO3) Ceramic, poled polycrystalline

Orientation

Sq (pC/N)

X-cut (length longitudinal) X-cut (thickness longitudinal), Y-cut (thickness shear) Parallel to polarization Perpendicular to polarization

2.2 22.0, 4.4 130 256

Source: Dally, J.W., et al. Instrumentation for engineering measurements, John Wiley and Sons Inc., 1984. Reprinted with permission of John Wiley & Sons, Inc.

leads to V¼

Sq h F 10 1r A

ð4:6Þ

The above equation implies that the developed electric voltage is proportional to the charge sensitivity, thickness, and applied force. The developed potential difference is usually very small, and thus it needs to be amplified by an amplifier called a “charge amplifier” as indicated in Figure 4.5b. When this sensor is connected via a cable to this amplifier, the equivalent electric circuit of the piezoelectric transducer and the amplifier circuit may be as shown in Figure 4.5c. The whole electric circuit can be modeled as a charge generator electrically connected to the capacitance Cp and the leakage resistance Rp , the cable represented as a single capacitance Cc , and the charge amplifier composed of Cf and Rf . If the cable and the amplifier circuits are neglected, the equivalent circuit can be modeled as shown in Figure 4.5d. A polyvinylidene fluoride (PVDF) film sensor shown in Figure 4.6 measures a force acting on the cantilever beam which in this case is a concentrated force. The force sensing principle of the sensor utilizes the same concept as that of the piezoelectric effect, as shown in Figure 4.6a. When the PVDF cantilever structure deforms as a result of the applied force, the deformation will produce an electrical charge due to the piezo effect. In general, the relationship between the behavior of the piezoelectric material and electrical behavior is known to exhibit a very complicated phenomenon. According to the property of the linear piezoelectric materials the following relationship holds Di ðxÞ ¼ dij sj ðxÞ þ 1ik Ek ðtÞ

ð4:7Þ

In the case of the PVDF film sensor, Di is the amount of charge on the PVDF per unit area, Ek is the electric field strength, dij is the piezoelectric coefficient


183

F electrode Vo

PVDF film electrode F

(a) force sensing principle PVDF layer A

F h

3 1

w 2

(b) PVDF sensor configuration FIGURE 4.6 PVDF-based piezoelectric force sensor.

of the material, 1ik is the permittivity of the PVDF, and sj ðxÞ is the applied stress. The double indices in dij indicate the direction of the generated charge (first index i) and that of the applied stress (second index j). The double indices in 1ik indicate the direction of the generated charge (first index i) due to the electric field strength acting in the kth direction. It is noted that in the case of piezoelectric materials, the majority of the dielectric constants are zero except; 111, 122, 133, i.e. 1ik ¼ 0 when k – i. According to these properties, for the configuration shown in Figure 4.6b Equation 4.7 can be expressed as D3 ðxÞ ¼ d31 s1 ðxÞ þ 133 E3 ðtÞ Then the charge generated in the PVDF can be determined by integrating the above equation ð q ¼ D3 ðxÞdA where A is the film area. When the above integration is carried out by using

s1 ðxÞ ¼

hð‘ 2 xÞ FðtÞ 2I

induced due to a concentrated force, F(t), the charge developed for the PVDF configuration is obtained by q¼

d31 Ah‘ FðtÞ þ 133 E3 ðtÞA 4I

ð4:8Þ

It can be observed that the developed charge is composed of the contribution due to the applied force and the induced electric field. It is noted here that h denotes the film thickness only, and I denotes the moment

184

Optomechatronics

of inertia which accounts for total thickness consisting of the electrode, film, and surface coating. This charge generates a current dq=dt through the resistance of the PVDF Rp, so the output voltage Vo ðtÞ of the sensor can be determined by Vo ðtÞ ¼ Rp

dq dt

Because E3 ðtÞ ¼ 2

dVo ðtÞ V ðtÞ ¼2 o dh h

the above equation can be rewritten as Rp Cp

dVo ðtÞ dFðtÞ þ Vo ðtÞ ¼ Kp dt dt

ð4:9Þ

where Cp is the capacitance of the PVDF film given by Cp ¼ the constant given by Kp ¼

133 A and Kp is h

Rp d31 Ah‘ 4I

Referring to Figure 4.5d, we can confirm that the above equation represents the equivalent electrical circuit produced by PVDF film. Piezoresistive sensors utilize materials that exhibit a change in resistance when subjected to an external force. A semiconductor gauge usually exhibits large piezoresistive effect. The common material is silicon doped with p-type or n-type material; the p-type yields a positive gauge factor whereas the n-type gives a negative one. Some details of this type of semiconductor material will be discussed in the next subsection. The strain gauges are attached usually to a beam or a diagram as shown in Figure 4.7a. Another type of piezoresistive force sensor shown in Figure 4.7b uses piezoresistive ceramic composite as a sensing element. As shown, it does not use any flexible member in its configuration but uses a bulk material which has both structural strength and force detection function. In recent years, piezoelectric sensors and actuators in micro scale have been well developed due to advancement of microfabrication methods such as deposition of thin films and patterning techniques. For example, applied force piezoresistor piezoresistor

Si substrate

(a) beam-type FIGURE 4.7 Schematics of piezoresistive sensors.

applied force

electrode

(b) bulk-type

ceramic composite

electrode


185

deposition of thin layers of polymers and ceramics over silicon is a commonly adopted method to obtain silicon based piezoelectric structure. This can be easily carried out by sputtering ion beam or vapor deposition methods. The structural shapes commonly used for deposition are rigid substrate, diaphragm, and two layer bimorph. Discussion of these methods will not be made further, for those involved with complex physical phenomena are beyond the scope of this book. Pyroelectric Sensor

polarization

The pyroelectric materials are crystalline materials capable of generating electric charge in response to heat flow. This effect is very closely related to the piezoelectric effect discussed previously. In fact, many crystalline materials exhibit both pyroelectric and piezoelectric properties. When polarized materials are exposed to a temperature change, its polarization, which is the electric charge developed across the material, varies with temperature of the material. A typical relationship is shown in Figure 4.8a. Polarization decreases with increase in temperature and becomes zero or almost zero near the Curie temperature point. The reduction in polarization results in decrease in charge at the surface of material. As a result, a surplus of charge occurs at the surfaces because there are more charges retained at surface before temperature changes. The pyroelectric materials therefore act

temperature, T

(a) polarization of a pyroelectric material in response to heat flow heat flow out

electrodes

pyroelectric material

i

Cp

Rp

heat absorbing layer heat flow in

(b) sensor configuration FIGURE 4.8 Pyroelectric sensor.

(c) equivalent circuit

186

Optomechatronics

as a capacitor. When electrodes are deposited on opposite sides, as shown in Figure 4.8b, these thermally induced surplus charges can be collected by electrodes. The configuration is the pyroelectric sensor that measures the change in the charge that occurs in response to change in temperature. From the relation shown in Figure 4.8a, the sensor will produce the corresponding charge changes Dq in response to the change in temperature DT in the following manner: Dq ¼ kq DT where kq is called pyroelectric charge coefficient and exhibits a nonlinear function of temperature. From the sensor configuration, we can see that the pyroelectric detector does not need any external excitation signal, but requires only an electric circuit that connects the electrodes. The equivalent electric circuit of a pyroelectric sensor is shown in Figure 4.8c. It consists of a capacitor Cp charged by the heat flow induced excess charge and the resistance Rp of the input circuit. Semiconductor Sensors; Light Detectors Semiconductors are crystalline solid material and have several interesting properties: (1) their electrical properties lie in between those of conductor and insulator; roughly speaking, conductors have very low electrical resistivity (a pure metal), and isolators have very high resistivity. (2) their resistance decreases rapidly with increasing temperature, that is, they have a negative coefficient of resistance. For example, silicon is found to have resistivity that decreases about 8%/8C. (3) In a pure state, they have very high resistance such that few electrons are available to carry electric current. Conductors that contain no impurities are called intrinsic semiconductors. However, when impurities are introduced to the pure semiconductor materials, they are called extrinsic semiconductors. The most interesting property of these impure semiconductors is that their conductivity changes, the amount of the change being dependent on the property of the impurity to be added. The impure materials added into the semiconductor materials are called dopants, and the process of adding impurities is called doping. In extrinsic semiconductors, there is current flow within the materials under the action of an applied field. In this situation, bound electrons arranged in energy bands are separated from particular atoms in the materials, and as a result, positively charged holes are formed and will move in the direction opposite to the flow of electrons; a hole is a missing electron in one of the bands. Therefore, current flow in semiconductor materials is due to the contribution of (1) positively charged holes and (2) negatively charged electrons. These two cases are illustrated in Figure 4.9. The p-type semiconductors are created due to an excess of holes. In this case, the contribution to current flow is due to presence of holes. Boron, aluminum,

Mechatronic Elements for Optomechatronic Interface pure semiconductor

flow of holes flow of current

(a) p-type semiconductor

metal electrode

187 pure semiconductor

metal electrode

flow of electron flow of current

(b) n-type semiconductor

FIGURE 4.9 Two types of semiconductors.

gallium are the impurities that serve as p-dopants. In contrast, the n-type semiconductors are created excess of electrons in which current flow is due to presence of electrons. As shown in the figure, it is noted that the current flow in the same direction as that of whole flow. Most semiconductor devices involve a junction at which p-type and n-type doping meet. In other words, these are placed in contact with one another. The resulting device is called a p –n junction. Through this junction, current flows easily from the p-type to the n-type conductor, since positively charged holes easily enter the n-type conductor, whereas electrons (negatively charged) easily enter the p-type conductor. This process is known as diffusion process. In the case of the current flow from the n-type to p-type, there is much greater resistance. There are two types of connection when a voltage source is applied to the junction; forward biased when the p-type is connected to the positive side of the source, and reverse biased when n-type is connected to the positive side, as shown in Figure 4.10a and Figure 4.10b. Figure 4.10c depicts the characteristics of a voltage –current for the p –n junction. In the forward biased connection, current does not flow till a voltage reaches the threshold value Vth , but above the threshold voltage, the current rapidly increases with the voltage. In contrast to this trend, in reverse biased connection, the current is almost zero, reaching, 2i0 , which is called reverse saturation current. This small current flow occurs due to the fact that there exist a small number of electrons in p-type and small number of holes in n-type. The saturation current exists within a wide range of reverse bias voltage. However, when the voltage increases slightly beyond a certain value called the breakdown voltage, Vbr, the junction of the semiconductor diode breaks down. As a result, the current increases rapidly without any further increase in voltage as shown in the figure. Combining the voltage vs. current relation, for both regions of applied voltage, the resulting curve looks as illustrated in Figure 4.11. It is noted that the curve depends upon the material properties and temperature. For instance, the reverse saturation current i0 is of the order of nanoampares for silicon but milliampares for germanian. The current equation in a diode that

(c)

current (i ), mA

Vth

voltage (V ), volt

metal electrode

FIGURE 4.10 Types of connections and their characteristics of voltage vs. current.

(a)

diode

n-type

(b)

Vbr

p-type

io

diode

current (i ), mA

p-type

metal electrode

voltage (V ), volt

n-type

188 Optomechatronics

189

current (i ), mA


Vbr

−io

voltage (V ), volt

FIGURE 4.11 The characteristic curve of the voltage vs. current (silicon).

can represent the above relationship may be approximately expressed by i ¼ i0 exp

qV kT

21

ð4:10Þ

where V is the applied voltage, i0 is the saturation current, q is the charge of an electron, T is the temperature in degree Kelvin, and k is the Boltzman constant. Photodiode Semiconductor junctions are sensitive to light as well as heat. If a p – n junction is reversely biased, it has high resistance. However, when it is exposed to light, photons impinging the junction can excite bound electrons and create new pairs of electrons and holes on both sides of the junction. Consequently, these separate and flow in opposite directions: electrons flow toward the positive side of the voltage source, whereas holes flow toward the negative side. This results in photocurrent, ip which is directly proportional to the irradiance of the incoming light, Ir ip ¼ CI Ir

ð4:11Þ

where CI is the proportionality constant depending on the area of the diode exposed to incoming light. Combining Equation 4.10 and Equation 4.11 yields i ¼ i0 exp

qV kT

2 1 2 ip

ð4:12Þ

190

Optomechatronics

dark current voltage (v), volt

Ir = 0 Ir = I1 Ir = I2

I1 < I2 < I3

current (i ), mA

Ir = I3

FIGURE 4.12 Voltage vs. current characteristics of a photodiode.

The characteristics of the voltage vs. current relation of the photodiode are shown in Figure 4.12. As can be seen, the photo current ip is much larger than the dark current i0 ðIr ¼ 0Þ and gets larger with increasing light irradiance or intensity. This indicates that if we operate the photodiode within the range of a reversed biased voltage we can make the output current directly proportional to the irradiance or intensity of incident light. Since total current is composed of photo current and dark current (reverse leakage), the dark current needs to be kept as small as possible to have high sensitivity of a photodiode. This can be done by cooling photodiodes to very low temperatures. A variety of photodiode detectors are available for converting a light signal to the corresponding electrical signal for position detection. One form of such detector types is multidiode units which are deposited on a common silicon substrate. There are two types of diode configuration, two or four separate diodes. As shown in Figure 4.13, these are all separate identical diodes insulated from each other and produce a photocurrent in proportion to the area upon which the light beam is incident. A two-separate diode type can be used to determine one dimensional displacement whereas a guardant type can be used for measurement of two dimension displacement of the light beam. When the light beam is illuminated on an equal area of each diode, that is, the beam is centered, the output of each diode will be identical. This case yields i1 ¼ i2 for the two diode type, and ix1 ¼ ix2 ¼ iy1 ¼ iy2 for the quadrant and therefore no output will be produced out of the operational amplifier. However, when the illumination is unequal, this will produce


191

i1 OP amp

OP amp

ix

1

∆x

ix

2

i2

iy

∆y

1

iy

OP amp

2

(a) split cell detector

(b) quadrant detector

FIGURE 4.13 Photodiode detectors.

unequal output from each diode, which in turn will produce the output of the amplifier. Other Photodetectors Photo detectors detect electromagnetic radiation in the spectral range from ultraviolet to infrared. These are categorized into two groups: photon detectors and thermal detectors. The photon detectors operate from the ultraviolet to mid infra spectral ranges, whereas thermal detectors separate in the spectral range of the mid and far infra. Photon detectors utilize the “photoelectric” principle that when light is incident on the surface of semiconductor materials the photonic energy is converted to kinetic energy of the electrons. There are three major mechanisms which produce this type of phenomenon. These include the photoemissive, photoconductive, and photovoltaic; photoemissive detectors produce an electrical signal in proportion to the incident light. As shown in Figure 4.14a, it consists of a cathode (negative), the emissive surface deposited on the inside of a glass tube, and an anode (positive) collecting surface. If a proper circuit is used, light incident on the cathode can cause electrons to be liberated and

light photoconductive layer incident light

e e e

electrode anode h

cathode

(a) photo emissive type FIGURE 4.14 Basic structure of photo detectors.

electrode

(b) photoconductor type

192

Optomechatronics

emitted. These emitted electrons reach the anode and make current flow through the external circuit. When the output current is not sufficient even at high intensity of light, adding successively higher electrodes called “dynodes” will provide substantial amplification. This device is known as a photomultiplier tube. Photoconductive detectors made of some bulk semiconductors without a p – n junction sensitively respond to light. As shown in Figure 4.14b, these consist of a photoconductive thin layer fabricated from semiconductor materials such as cadmium selemium (CdSe), cadmium sulfide (CdS), lead telluride (PbTe), and an electrode set at both ends of the conductor. The electrical resistivity of these materials decreases when they are exposed to light. As discussed earlier, in photodiode due to the decrease in resistivity, the output current of the conductor can be quite large. The change in resistance DR is related to the change in conductance Ds; DR ¼

1 ‘ hDs w

where ‘, w and h are the length, width and thickness of the conductor, respectively. A photoconductor often uses a bridge circuit to detect the intensity of light illuminated on the n-type layer. The change in resistivity due to incident light can be effectively detected by this circuit. Photovoltaic Detectors These detectors supply the voltage or current without any external power source, when they are exposed to light. The detector commonly known as a solar cell consists of a sandwich of dissimilar materials such as an iron base coated with a thin layer of iron selenide. When the cell is exposed to light, a voltage is developed across the sandwitch. Semiconductor junction also belongs to one of such materials. A photodiode discussed in Figure 4.12 exhibits the photovoltaic effect, when it is operated in the region where i is negative and V is positive (fourth quadrant of the figure). In this region, the active area of the p –n type photodiode junction is illuminated without any external bias voltage, and a current is produced in proportion to the intensity of the incident light for essentially zero voltage across the diode. Avalanche Photodiode As already discussed in Figure 4.11, an avalanche photodiode operates in the region of near breakdown voltage Vbr : At this breakdown voltage, known as the Zener voltage, current rapidly increases with a small increase in reverse bias. In other words, at near Vbr, even a small change in the illumination on the photodiode causes a large change in the photocurrent.


193

Signal Conditioning Operational Amplifiers An operational amplifier (op-amp) is a complete amplifier circuit supplied as an integrated circuit on a silicon chip and is a basic signal conditioning element. The amplifier has an extremely high gain with typical value higher than 105, and therefore can be regarded as infinite for the purpose of circuit analysis. Figure 4.15 shows the physical sizes of op-amps along with the schematic diagram of its internal circuit representation. We will limit our discussion here to the ideal op-amp which has the following characteristics; (1) high input impedance mega ohms to giga ohms, (2) low output impedance (of the order of 100 V) considered to be negligible, (3) extremely high gain (G ¼ 105 (100 dB) typical value). Figure 4.16 shows six different types of amplifier circuits frequently used in instrumentation and signal conditioning. Inverting Amplifier Since an operational amplifier has a high intrinsic gain and very high input impedance, it is operated essentially with zero input current and voltage (ia ¼ 0, Va ¼ 0). Figure 4.16a shows the inverting mode of the op-amp which has negative feedback. At the junction of the two resistors we have i1 ¼

Vin 2 Va R1

inverting input noninverting input

output

(a) photograph of OP amplifiers

(b) operational amplifier circuit

FIGURE 4.15 Typical operational amplifiers. Source: National Semiconductor Corporation (www.nsc.com).

194

Optomechatronics

i2

Rf

Rf

R1 i1

Vin

a

a

Vout

Rf

R1

V1

Vout

(b) non-inverting amplifier

(a) inverting amplifier

V2

R1

Vin

C R

R2

Vout

(c) summing amplifier

Vin

Vout

(d) integrating amplifier Rf

Rf C

Vin

(e) differentiator

V1 V2 Vout

R1 R2

Vout

(f) differentiator

FIGURE 4.16 Basic operational amplifier circuits.

i2 ¼

Va 2 Vout Rf

ð4:13Þ

where R1 and Rf are the input and feedback resistors, respectively. By Kirchhoff’s current law, i1 ¼ i2 þ ia Hence Vout R ¼2 f Vi R1

ð4:14Þ

The above relationship implies that the voltage gain of the op-amp is determined simply by the ratio of the input and output resistors.


195

Noninverting Amplifier This mode of op-amp operation causes a voltage of the same polarity to appear at the amplifier output. The general use of this mode is schematically illustrated in Figure 4.16b. It has both inverting and noninverting inputs and amplifies the voltage difference between the two inputs. Let us assume that current flows through the inverting terminal input and feedback registers. The voltage at the junction “a” is then equal to Va ¼

R1 V R1 þ Rf out

Since there is no current flow, i.e., ia . 0, this gives Va . Vin : Therefore, we have Vout R ¼1þ f Vin R1

ð4:15Þ

The closed loop gain is again determined by the ratio of resistors. The characteristic of this noninverting input is that the circuit input impedance is the input impedance of the amplifier itself rather than the impedance denoted by amplifier input resistor. Inverting Summing Amplifier The objective of this mode is to obtain the output voltage in terms of the input voltages V1 and V2 : As in the case of the inverting amplifier, the use of Kirchhoff’s law at the summing junction shown in Figure 4.16c yields i1 þ i2 ¼ ia þ i

ð4:16Þ

In terms of voltage drops, the above equation can be written as Vout V V ¼2 1 þ 2 Rf R1 R2 since ia ¼ 0: It is clear that this relationship can be extended to an arbitrary number of input ports. In the case of fixed feedback resistance Rf , the gains of the individual input can be adjusted by varying the input resistances R1 and R2 : When R1 ¼ R2 ¼ Rs , the output of this summing mode given in the above equation is obtained by Vout ¼ 2

Rf ðV þ V2 Þ Rs 1

This relationship can be effectively used when we want to average the input voltages from multiple sensors. Integrating Amplifier An integrating amplifier is used to integrate the incoming input voltage and utilizes a capacitor as shown in Figure 4.16d. The expression for the output

196

Optomechatronics

voltage Vout can be obtained by considering the current flow through the capacitor. Since the charge q ¼ CVout and i ¼ dq=dt, we have Vin dVout ¼ 2C R dt

ð4:17Þ

Solving for the output voltage Vout we have Vout ¼ 2

1 ðt V dt RC 0 in

The output is proportional to the integral of input voltage, Vin : Differential Amplifier A differential amplifier is used to obtain the difference between two input voltages. Two types of the differentiator are shown in Figure 4.16e and Figure 4.16f. The one shown in Figure 4.16e is similar to the integrating amplifier except that the position of resistor and capacitor are interchanged. An expression for the output voltage Vout of this amplifier mode can be developed by using Equation 4.17. In another type of the differentiator shown in Figure 4.16f, there is no current flowing through the amplifier. The voltage potential Va will then be given by Va R2 ¼ V2 R1 þ R2

ð4:18Þ

As with the previous amplifiers, the current flowing through the feedback resistance Rf must be equal to that through the input resistance R1 : Thus V1 2 Va V 2 Vout ¼ a R1 R2

ð4:19Þ

Combining Equation 4.18 and Equation 4.19, we have Vout ¼

R2 ðV 2 V1 Þ R1 2

Comparator When a comparison between two signals is required, an operational amplifier itself can be used as a comparator. As shown in Figure 4.17, a voltage input V1 is applied to the inverting input while another input V2 is applied to the non inverting input. Because of the high gain of the amplifier, only a small difference signal of the order of 0.1 mV is required for the output to compare which of two voltages is the larger. The right hand side of the figure shows the output voltage vs. the input voltage difference. When non inverting input is greater than inverting input, then the output Vout swings from the negative 2Vs to the positive Vs , and vice versa.


197 output Vs

V1 0

V2

input

−Vs FIGURE 4.17 Comparator.

Signal Processing Elements Filters Filters are used to inhibit the presence of a certain band of undesirable frequencies from a dynamic final signal, permitting others to be transmitted, as illustrated in Figure 4.18a. The range of frequencies passed by a filter is a signal containing all frequency components

particular frequency components of interest

electronic filter

practical

0 fC (b) low pass

practical

fC1

(d) band pass FIGURE 4.18 Various signal filters.

f

ideal

signal magnitude 0

signal magnitude

ideal

fC2

f

ideal practical

fC 0 (c) high pass

signal magnitude

signal magnitude

(a) signal filtering

0

f

ideal practical

fC1

(e) band stop

fC2

f

198

Optomechatronics

called the pass band and the range not passed as the stop band. In actual practice, it is not possible to completely remove the unwanted frequency contents due to the dynamic characteristics of the signal. Figure 4.18b to Figure 4.18e show four different ideal filters classified according to the ranges of frequencies. A low pass filter is the one that transmits all frequencies from zero up to some specified frequency, that is, frequencies below a prescribed cut-off frequency. In contrast, a high pass filter transmits all frequencies from a cut-off frequency up to infinity. A band pass filter transmits all frequencies within a specified band whereas a band stop filter rejects all frequencies within a particular band. As can be observed from the figure, the sharp cut-off of the ideal filter cannot be realized, because all filters exhibit a transition band over which the magnitude ratio decreases with the frequency. The rate of transition is known as the filter roll off. Digital-to-Analog Conversion The purpose for a digital-to-analog (D/A) conversion is to convert the information contained in a binary word to a DC output voltage, which is an analog signal. The signal represents the weighted sum of the nonzero bits in word. This indicates that the leftmost bit is the most significant bit (MSB) having the maximum value which is twice the weight of the next less significant bit, and so on. In general, the weighting scheme of an M bit word is given as below bit

M21

bit 2

bit 1

bit 0

2M21

···

22

21

20

2M21

···

4

2

1

Under this scheme, the leftmost bit, M 2 1 is known as the MSB, since its contribution to the numerical value of the word is the largest relative to the other bits. Bit 0 is known as LBS. The number of the digital word determines the resolution of the output voltage. Suppose that the word has M bits and expresses a full scale of an output voltage Vout : In this case, a change of one bit will cause the corresponding change in the output voltage Vout of Vout =2M : The above conversion relationship can be realized electronically in a number of ways one of which is shown in Figure 4.19. This is a simple DAC known as the weighted register DAC and employs an inverting summing amplifier. In the figure Rf is the feedback resistance, R is the base resistance related to the input resistances, and bi ði ¼ 1; 2· · ·MÞ is 0 or 1. The bi is “off” when connected to the ground and “on” is when connected to VR : When all bis are on, DAC gives analog output voltage which is proportional to an input parallel digital signal; bM21 ; bM22 · · ·b2 ; b1 ; b0 : Summing all currents at the summing junction, we have i0 þ i1 þ · · · þ iM21 ¼ ia þ if

ð4:20Þ

Mechatronic Elements for Optomechatronic Interface VR

199

Rf bM-1

20 R

bM-2

21R

bM-3

22R

b1

2 M-2 R

b0

2 M-1 R

if Vout

Va ia

FIGURE 4.19 Digital-to-analog converter.

Equation 4.20 can be rewritten using Ohm’s law as bM21

VR 2 Va V 2V V 2V V 2 Vout þ bM22 R 1 a þ · · · þ b0 RM21 a ¼ ia þ a 0 Rf 2 R 2 R 2 R

Since Va ¼ 0, the output voltage is obtained by Vout ¼ 2

Rf VR R

bM21 b b1 b0 þ M22 þ · · · þ M22 þ M21 20 21 2 2

¼2

Rf V N R R

ð4:21Þ

where N is denoted by N¼

bM21 b b1 b0 þ M22 þ · · · þ M22 þ M21 20 21 2 2

ð4:22Þ

Equation 4.22 indicates that N is the binary number output to D/A converter and the output of the D/A converter Vout is proportional to N. In the above, bM21 is the MSB, bM22 the next significant bit, and b0 is the least significant bit (LSB). If M ¼ 8, there are 8 bits in the converters, switch number seven corresponds to MSB, while switch number 0 corresponds to LSB. When switch numbers one, six, and seven are closed, the bits in Equation 4.22 become 11000012, which corresponds to 9710. Analog-to-Digital Converters The analog-to-digital (A/D) converter is a device that samples an analog input voltage and encodes the sampled voltage as a binary word. A number of different types of A/D converters are available that offer a range of different performance specifications. The most common are successive approximation, ramp, dual ramp and flash. The simplest form of A/D converter is the successive approximation. Figure 4.20 shows the converter circuit in which a D/A converter is used along with a comparator and a time and control unit. This circuit utilizes the output of a counter as the input to a D/A converter. The output of the

200

Optomechatronics clock n bit register

analog input

counter

DAC

parallel digital output FIGURE 4.20 Configuration for successive approximation ADC.

counter is obtained by counting a sequence of pulses generated in a binary fashion by a clock. The D/A converter then converts this value into an analog value, which is compared with the input analog value by the comparator. When the output of the D/A converter equals or exceeds the input analog voltage, counting of the pulses from the clock is stopped by closing a timing and control unit (gate). The value of the counter is then the desired digital value. For example, initially the counter makes a guess that an input analog voltage is greater than or equal to a certain voltage, say, 5 V. The unit then sets MSB’s flip-flop to a logical 1 state. The binary word 1000 is fed to the D/A converter, which feeds 5.0 V to the comparator. If the comparator senses that the analog voltage is greater than the voltage from the D/A converter, the counter leaves the MSB’s flip-flop set to a logical 1 state. Because the first guess is too small to represent the analog input, the counter then makes a second guess, setting the second bit to a 1. Sample and Hold Module The operation of analog-to-digital conversion takes some time. Therefore, it is necessary to hold an instantaneous value of analog input while conversion takes place. This is done by using a sample and hold device whose circuit is shown in Figure 4.21. The circuit consists of an electronic switch to take the

y(t)

Vin

C

(a) sample and hold circuit FIGURE 4.21 Sample/hold device.

original input signal

Vout

(b) sample and hold signal

t


201

sample, a capacitor for the hold, and an operational voltage follower. The operation principle works, during sample and hold states as follows. In the sample state, if the switch is closed, the capacitor is charged with the applied analog input voltage. The voltage follower makes the output voltage follow the input voltage: in the hold state, if the switch is opened, the capacitor retains its charge. The amplifier maintains the output voltage equal to the input voltage at the instant of time until the switch was opened. Therefore, the output voltage is held constant at the value of the input voltage at the instant of time until the switch is closed again. Multiplexer When sampling several input signals from a number of sources it is sometimes desirable to use a single A/D converter by switching one input signal to another by means of a multiplexer circuit such as that shown in Figure 4.22. The multiplexer is an electronic switching device that enables sampling of the input signal sequentially rather than in parallel. Multiplexing therefore is used to enable a single channel to be shared between a number of signal sources. Time Division Multiplexing Figure 4.23 shows a schematic diagram of a time division multiplexer with four channels, 0 1 2 3. When the four input signals are present, the multiplexer selects a signal as an input to a sample/hold device, and then it needs to know which signal is connected to the output line. Thus, each channel has its own binary address signal. For example, if the address signal is 10, the channel line two is switched on to the output line. In this manner, the multiplexer output has a series of sampled signals taken from different channels at different times, as illustrated in the figure. It is noted that the channels are addressed in the order, 0, 1, 2, and 3, and that DT is the sampling

V8 V7 V6 input signals V5 V4 V3 V2 V1

S/H 8-channel mux devices

FIGURE 4.22 Multiplexer with simultaneous sampling.

A/D

multiplexer switch

202

Optomechatronics

channels

0

multiplexer (MUX)

∆T

1

0 1 2 3 4

2

multiplexed signal

3 signal conditioners

∆T

solid state switch

address decoder

S/H

0 1 2 3 4

sample & hold signal

control signal

FIGURE 4.23 Time division multiplexing.

interval. When the multiplexed signals are fed to the sample hold device, it produces the signal types shown in the right hand side of the figure. Wheatstone Bridge The bridge circuit is the most common unit which can be used for obtaining the output of transducers. Of all possible configurations, the Wheatstone bridge is used to the greatest extent. It converts a resistance change in response to a voltage change. As shown in Figure 4.24, the bridge consists of four resistance arms with a constant electrical source (DC), and a meter M. Due to a constant supply voltage this type is called a constant voltage Wheatstone bridge. When the bridge is balanced, i.e., Vout is zero, no current may flow through the meter, which gives iout ¼ 0: Then, the current i1 must be equal to i2 , whereas the current i3 must be equal to i4 : In addition, the potential difference across R1 must be equal to that across R3 , or i1 R1 ¼ i3 R3 and similarly, for the potential difference across R2 and R4 , i2 R2 ¼ i4 R4 :

B R1

R2 i2

i1

A

i3

R4

R3 D

Vs FIGURE 4.24 Constant-voltage Wheatstone bridge.

C

i4

M


203

We therefore obtain the condition for balance R1 R ¼ 3 R2 R4 When this bridge undergoes change in one of the resistances, the balanced condition cannot be satisfied. This means that the output voltage Vout is no longer zero. The Vout under this condition can be obtained in the following manner: Since Vout ¼ VAB 2 VAD , the use of Ohm’s law yields Vout ¼ Vs

R1 R3 2 R1 þ R2 R3 þ R4

ð4:23Þ

If resistance R1 changes by an amount DR1 , Vout will accordingly change to Vout þ DVout : This relation can be written by Vout þ DVout Vs

8 > >
DR R R3 1 2 > : 1þ þ 1þ R1 R1 R4

9 > > = > > ;

ð4:24Þ

When all resistances are initially equal and Vout ¼ 0, the relation in Equation 4.24 may be simplified as DVout DR1 =R ¼ Vs 4 þ 2ðDR1 =RÞ

ð4:25Þ

where R ¼ R1 ¼ R2 ¼ R3 ¼ R4 , which is a widely used form. Equation 4.25 shows that this type of resistance bridge exhibits nonlinear behavior, but can be considered to be linear, since usually DR1 is very small for most applications. Isolator When a circuit carrying high current or high voltage is connected to the next circuit, the connection may cause the possibility of damage. This can be protected in various ways. One means of such protection is to use an isolator between two circuits. The isolator is a circuit that completely isolates circuits, while keeping them electrically isolated from each other. Figure 4.25 shows a typical isolator circuit using an operational amplifier, and optical signals. The noninverting amplifier used here is a voltage follower discussed previously whose gain is unity. This implies that circuit 2 follows the output of circuit 1 which is the input to the op-amp. The electrical signal at the output of circuit 1 is converted into an optical signal by a light emitting diode (LED). The converted light signal is in turn converted again into the corresponding electrical signal at the output of circuit 2 by using a photo detector such as a photodiode or phototransistor.

204

Optomechatronics

photodiode LED circuit 2

circuit 1 Vin

Vout

light

optoisolator

(a) an isolator using op−amp

(b) optoisolator

FIGURE 4.25 Isolation via optoisolator.

Microcomputer System The microcomputer or microprocessor has become a vital and indispensable tool to solve almost all engineering problems, in practice, signal conditioning, modern complex control or automation problems. Furthermore, when it is interfaced with sensors, actuators, and artificial intelligent tools, it makes modern machines, processes or products or systems smart so that human intervention can be greatly reduced. This trend will be accelerated in the future, because computers will be equipped with smart software in addition to their capabilities interfacing with intelligent hardware tools and systems. In this section, we will briefly discuss the basics of microcomputers in terms of hardware structure. Microcomputer Basically, most of the computers consist of the following essential units and circuitry: (1) (2) (3) (4)

the central processing unit (CPU) memory bus input and output interfaces and devices.

The CPU is a microprocessor, a semi conductor VLSI (very large scale integrated circuit) chip. The main objective is to recognize and carry out program instructions by fetching and decoding them. As shown in Figure 4.26, it consists of control units, an arithmetic and logic unit (ALU), registers accumulator, program counter, and other auxiliary units such as memory controller, interrupt controller, and so on. The control unit determines the timing and sequence of operations involving fetching a program instruction from memory and executing it. The ALU is a logic circuit which can perform arithmetic and logical operations such as addition,


EPROM EPROM EEPROM EEPROM

address bus

205

control bus

RA M ROM

microprocessor ROM RAM

device select lines

I/O interfore unit

data in

data out

data bus

FIGURE 4.26 Schematic diagram of a micro-computer.

subtraction, and logical AND/OR. The microprocessor has several registers such as the status registers containing information on the latest processed results carried out in the ALU, the address register containing the address of data to be manipulated, the instruction register storing an instruction received from the data bus and some other registers. The accumulator is the data register where data to be sent to the ALU is temporarily stored. The program counter is a register which contains the address of the instruction to be executed next. The memory unit is used for several purposes such as storing the instructions simply in numbers and holding data to be processed, immediate answers and the final results of a calculation. The size of the memory unit is specified by the number of storage locations available, for example, 1 k memory (10 bit address signal); 210 ¼ 1024, 64 k ¼ 216 ¼ 65536 locations. ROM (read-only memory) is a permanent memory where no data can be written, but the data can only be read. In this memory, programs that include computer operating systems and dedicated microprocessor applications are stored. RAM (random access memory) is the memory which information can be written into or read out of. It is used for temporary deposits and withdrawals of data or information required for the establishments of programs. The stored information is lost when the power supply is switched off. EPROM is the erasable and programmable ROM. The program is stored by applying voltages to the specified pins of the integrated circuit and producing a pattern of charged or uncharged cells. Erasing the pattern can be done by optical means using ultraviolet light. A bus is a transmitting media consisting of a group of lines. It ties the CPU memory and input – output (I/O) device together by three sets of wire lines. The first wire set, data bus, is used to carry data between the CPU and the

206

Optomechatronics

memory or I/O channels. This bus also transports the instructions from the memory to the CPU with a view to make them executed. The data bus is bidirectional, allowing signals to flow in and out from the CPU. The second set of wires is an address bus that carries a set of signals known as addresses. This address signal indicates which particular memory location is to be selected so that data may be transferred between the CPU and that location. In this situation, only the location is opened to the communication from the CPU. It is noted that on the address bus data always flows from the CPU to the external devices. Most of the popular CPUs typically employ a 16 bit line address; having 64 K locations; 216 ¼ 65536, but some other types use 20 or 24 address lines. The third set is a control bus which carries a variety of timing and control signals out from the CPU. This bus also carries control signals generated by the control and timing units to the CPU, which synchronize and control each internal operation inside the computer. For example, read and write control signals generated by the CPU are used to indicate which way the data is to flow across the data bus and when the data is to be transferred from memory to the CPU and vice versa. As shown in Table 4.4, the binary states of simple logic signals can be used to tell which mode is enabled. In and out control signals generated by the CPU specify whether an input or an output operation is to be performed. In addition, these specify when the addresses and data are valid for operation, as indicated in the table. The CPU interrupts the program being performed and begins execution of the Int signal when it receives an interrupt signal (Int). The functions of the TABLE 4.4 Control Signals for Some Modes of Operation Operation Mode Read 0 0 1 1

Write

Control Signal Function

0 1 0 1

No memory read/write operation requested Memory address and data are valid for write operation Memory address is valid for read operation Not assigned In/Out Control Signal

0 0 1 1

0 1 0 1

No I/O is to be operated I/O address and data are valid for sending out signals I/O address is valid for taking in external signals. Not assigned INT Control Signal

0 1

Not interrupt requested Interrupt requested


207

interrupt signal are performed according to the operation mode, as shown in the table. It is noted that all of these signals are carried by the control bus. Input/Output Interface Input/output devices are connected to the CPU through three bus lines. This enables the CPU to interact with the devices to perform a basic operation or to send out the executed results. All of these interactions occur through the I/O ports whose pins are used for external connections of inputs and outputs. Ports are places where loading and unloading of data take place. The ports can be classified into input port or output port or programmable port depending on whether the signal is input or output. The peripheral (I/O) devices are keyboard, monitor, line printer, memory, A/D, and D/A. In control and instrumentation applications, the inputs may be signals from sensors. These signals are sampled via A/D for computer. In the case of instrumentation, the computed results may be used for analysis of the process being instrumented. In process control, the results will often be converted to analog signal via a D/A converter in order to meet the specification required by actuators involved with control. Microcontrollers The microprocessors and microcontrollers are similar but as far as applications are concerned, these are slightly different in architecture. Microprocessors are primarily used for high speed computing applications. In contrast, the microcontroller is designed to satisfy the applications of signal processing, engineering measurements, industrial automation, system control and so forth. The microcontroller is a digital integrated circuit in which several functions are all brought together on a single chip which includes the CPU, ROM, RAM, ADC, DAC, and serial and parallel ports. From this point of view, the microcontroller is the integration of a microprocessor with memory I/O interface timer, and the converters (ADC and DAC). Sampling of a Signal Sampling of a continuous time signal is a basic process for computer assisted analysis, instrumentation, and control of processes and systems because of the discrete-time nature of the digital computer. Sampling a signal means to replace the signal by its values in a discrete set of points. If sampling is done at an equal time interval Dt, as indicated in Figure 4.27, then the sampling instants are equally spaced in time, therefore the kth sampling time is expressed by tk ¼ kDt;

k ¼ 0; 1; 2· · ·n

ð4:26Þ

If a time-varying continuous signal is represented by f ðtÞ, the sampled version of the signal may be expressed by f ðtk Þ: This is called a sampled

208

Optomechatronics

signal amplitude

f(t )

O 0 ∆t 2∆t 3∆t

k∆t (tk )

(t1) (t2) (t3)

time (t )

FIGURE 4.27 Sampling a continuous signal f ðtÞ:

sequence often referred to as a time series. In this case, f ðtk Þ is said to be sampled with a frequency of fs which is called sampling frequency or sampling rate; fs ¼

1 Dt

ð4:27Þ

amplitude (V)

amplitude (V)

amplitude (V)

amplitude (V)

To see the effect of the sampling frequency, let us consider a 0.16 Hz sine wave which is plotted versus time over time period 20 sec, as shown in 1 0 –1

0

5

10

15

20

5

10

15

20

5

10

15

20

10

15

20

(a) original signal (frequency: 0.16 HZ)

time (t ), sec

1 0 –1

0

time (t ), sec

(b) fs = 1 HZ

1 0 –1

0

(c) fs = 0.37 HZ

time (t ), sec

1 0 –1

0

5

(d) fs = 0.25 HZ

FIGURE 4.28 Sampling a sinusoidal signal with various frequencies.

time (t ), sec


209

Figure 4.28a. This sine wave is sampled with three different frequencies and the results are shown in Figure 4.28b to Figure 4.28d: (b) fs ¼ 1 Hz (c) fs ¼ 0:37 Hz (d) fs ¼ 0:25 Hz: It is apparent that as the sampling frequency decreases the sampled signal loses the original shape of the continuous sine wave. If sampling frequency is very low, that is, sampling rate is too slow, then the sampled signal appears to be entirely different from the original, having a frequency much lower than that of the original. From this observation we can see that the frequency content of the original signal can be reconstructed accurately only when the sample frequency is higher than the twice the highest frequency contained in the analog signal. That is, fs . 2fmax

ð4:28Þ

where fs is the maximum frequency in the analog signal. This is known as Nyquist criterion. In terms of sampling time interval, Dt ,

1

ð4:29Þ

2fmax

In converting a continuous signal to discrete signal form Equation 4.28 provides a criterion for the minimum sampling frequency. Similarly, Equation 4.29 represents a criterion for the maximum time interval. Whenever a signal at a sample rate is less than 2fs , the resulting discrete signal will appear false differently from the original signal due to misinterpretation of the high frequency content of the original. This phenomenon is referred to as aliasing and the false frequency is called the aliasing frequency. Aliasing occurs when fa ,

1 f 2 s

ð4:30Þ

where fa is the aliasing frequency given by fa ¼ lfmax 2 nfs l

for integer n

ð4:31Þ

For example, suppose that a signal is composed of f ðtÞ ¼ A1 sin 2pð120Þt þ A2 sin 2p ð120Þt If this signal is sampled at a rate of 125 Hz, the frequency content of the resulting discrete signal would be fa ¼ l120 2 125l ¼ 5 Hz

210

Optomechatronics

Therefore, the sampled signal appears to be oscillating at frequency of 5 Hz. The aliasing problem can be prevented when a low-pass filter is placed before the signal enters ADC.

Actuators Electric Motors In view of operation principle, electric motors may be categorized into three basic types of actuators; DC, AC, and stepper motor. DC motors utilize DC current source to produce mechanical rotation, while AC motors consume alternating electrical power to generate such motion. In DC motor, depending upon what type of magnetic fields (permanent magnetic type or electromagnet wound field type) is used, these are divided into two; permanent magnetic DC motor and electromagnetic type or wound field DC motor. Stepper motors steps by a specified number of degrees according to each pulse the motor receives from its controller. We will here discuss DC motors only. Figure 4.29a depicts a schematic of a DC servomotor, in which the basic components are included. The stator has magnets which produce an electrical field across the rotor. The rotor is wound with coils through which current is supplied through the brushes. The brush and commutator work together. They are in contact and rub against each other during rotation, as shown in Figure 4.29b. Due to this arrangement, the rotating commutator sends the current through the armature assembly so that current passes through the rotor. Two principles are involved in the motor motion: the motor law and generator law. In case of the motor law, Fleming’s left hand rule is applied to indicate the directions of the electric current. When all three fingers are kept

rotor

stator magnet stator magnet

stator magnet

motor shaft

magnetic flux

coil

ia F

N

brush commutator

B

FIGURE 4.29 Electric DC motor.

stator magnet

ia

S

brush coil (electric conductor)

commutator ia

ia Va

(a) schematic of DC motor

F

(b) principle of DC motor rotation


211

at right angles to each other, magnetic force (F) is indicated by the thumb, magnetic flux (B) by the index finger, current ðia Þ by the middle finger. Because the current passing through the rotor coils (conductor) is in the magnetic field generated by the stator, the rotor will receive the electromagnetic force called the “Lorentz force” which is given by F ¼ B‘c ia

ð4:32Þ

where B is the magnetic flux density in tesla, ia is the armature current in amperes, and ‘c is the effective length of the coil in meters. The above equation converts electrical energy within a magnetic field into mechanical energy so that a rotor can do work. On the other hand, the law of generator as mentioned describes a relation describing the effect of mechanical motion on electricity. The law states that when the rotor coils (conductor) are moving in the magnetic field in the direction perpendicular to the speed v, it generates an electromotive force (emf) Ve in volts which is expressed by Ve ¼ B‘c v

ð4:33Þ

This Ve can cause the corresponding current to flow in an external electric circuit. Based on the observation of these two laws, we can model the dynamics of the motor rotating in the magnetic field by considering the torque acting on the rotor. T ¼ 2B‘c ia r ¼ kt ia where r is the radius of the rotor, and kt is the torque constant. The voltage generated due to the rotation of the rotor in the magnetic field is expressed by Ve ¼ 2B‘c r

du du ¼ 2kb dt dt

ð4:34Þ

where du=dt is the angular velocity and kb is the back emf constant. Referring to Figure 4.30, we have for the electrical side La

dia du þ Ra ia ¼ Va 2 kb dt dt

ð4:35Þ

where Va is the supply voltage to the armature and for the mechanical side I

d2 u du þb ¼T dt dt2

ð4:36Þ

where I is the moment of inertia of the motor, b is the viscous damping coefficient of the motor, and T is torque induced by a supply voltage. When all parameters are known, the angular speed and position of the motor can be determined from the above Equation 4.35 and Equation 4.36. As we shall see later, the motor parameters I, kb , and b determine the dynamic characteristics of the motor.

212

Optomechatronics Ra

La

Va

I

T

ia Ve

(a) electrical model

b

θ T

(b) mechanical model

FIGURE 4.30 Schematic diagram of an armature controlled DC motor.

Piezoelectric Actuator As mentioned earlier, a piezoelectric material deforms or produces a force when it is subject to an electrical field. This concept is a basis of piezoelectric actuator. The relationship between electrical behavior and mechanical behavior has already been treated in piezoelectric transducer but we will briefly discuss it here again to describe basic deformation modes. The relationship can be approximately described when piezoelectric materials are free of applied loads. It is given by S ¼ dij E

ð4:37Þ

where S is the strain in the piezo material, E is the electric field strength, and dij is the coupling coefficient between S and E, which is the piezoelectric charge constant. To get familiar with the physical meaning of the relation, let us consider two basic deformation modes as shown in Figure 4.31. The unchanged dimension of the piezoelectric material is shown in Figure 4.31a, whereas Figure 4.31b and Figure 4.31c, respectively, show the axially and transversely elongated states due to the electric field E applied as indicated in the specified direction. According to the indicated directions of both electric field and strain, dij in Equation 4.37 can be identified. The axial (thickness) mode has dij ¼ d33 , because in this case both directions are directed along axis 3. The transverse mode (longitudinal) has dij ¼ d31 , because the electric field is in the direction of axis 3, while the strain direction is along axis 1.

electrode E 3

h 2

h+∆h 1

(a) original (unchanged)

(b) axial expansion

FIGURE 4.31 Piezoelectric effect: longitudinal and transverse modes.

electrode E _ +

+ _

+∆

(c) transverse expansion


213 do di

z

+x −y

+y d

−x −x −y

(a) stack

(b) bimorph

(c) tube actuator

FIGURE 4.32 Bimorph configuration.

The other deformation modes such as thickness shear and face shear can also be treated in this way. Generally, a single piezoelectric actuator employing the above configuration produces a very small deformation, and therefore a variety of structural configurations are available, which yield relatively large deformations compared to that of single structure. Figure 4.32 indicates three such actuators which include stack, bimorph, and tube types. The stack type is composed of N axial piezo actuators and thus increases N times the deformation of single actuator. The bimorph actuator consists of two similar transverse actuators. It has a cantilever beam configuration which has a large bending mode when subjected to transverse load. These actuators deform in opposite way and therefore produce a large bending. This bimorph actuator is of great importance in piezoelectric microactuator to be discussed later. The cylindrical hollow tube type has outer diameter do and inner diameter di : It is radially polarized and the outer electrode surface is divided into the quadrant. Each of two surfaces has an electrode; þ x and 2 x or þ y and 2 y. When two opposite electrodes are subject to opposite electric field, it bends. Thus, these give two directional motions in both x and y directions. On the other hand, the z motion is generated when all electrodes are subjected to the same electric field. Based on the linearized portion of the relationship given in Equation 4.37, Table 4.5 describes the deformation of various configurations of piezo actuators discussed in the above. The first two denote the single actuator while the other three represent multiple actuators. In the table Vp is an applied voltage and dij is the coupling piezoelectric coefficient. From the relations the applied force can be computed by considering the fact that the piezo generated static force Fp is proportional to the deformation. From Equation 4.6 the force is related to Fp ¼ CF VP where CF is the force sensitivity constant and given by 1 1A CF ¼ 0 r Sq h

ð4:38Þ

214

Optomechatronics

TABLE 4.5 The Displacement of the Piezoelectric as a Function of the Applied Voltage VP Configuration

Mode

Deformation

Geometry electrode E

Axial

Longitudinal (axial)

_ +

D‘ ¼ d33 VP +∆ ‘

electrode E

Transversal

Transversal

Dz ¼

Stack

Longitudinal

D‘ ¼ nd33 VP

Transversal

L2 d ¼ 3 2 d31 VP h

h

d31 VP

_ +

z+∆z z

Bimorph

`

h

δ

do

di

Tube

Transversal

d¼

2L d V d0 2 di 31 P

+x −y

+y

−x −x −y

The relationship in Equation 4.38 shows that the actuation force is proportional to applied voltage. From the discussions we have made in the above it can be summarized that, when a piezoelectric material undergoes a change in geometry due to an applied voltage, the amount and direction of its deflection depend upon the type of piezo electric material and the geometry of the material. Due to this complexity involved, the piezo material exhibits unfavorable nonlinearity between strain and applied voltage, hysteresis, creep, and aging. To reduce or eliminate this inherent nonlinearity there are several methods developed to date. The most frequently used methods are table-loop-up based on feedback control compensation. In general, feedback compensation results in fairly accurate correction of nonlinearity, leaving only 1% nonlinearity. A general feedback control concept will be discussed shortly. So far, we have considered the static relationship between the deformation and electric field under the assumption that the piezoelectric element will reach equilibrium after undergoing an initial deformation as given in Equation 4.37. However, the piezo element is essentially a moving element, so its motion will be governed by certain dynamic characteristics. The piezoelectric ceramics has inherently a spring element kp and an internal

Mechatronic Elements for Optomechatronic Interface electric load applied

215

electrode z z

Vp

bp

kp

Fp= CF (Vp - Ve )

piezoelectric ceramic

(a) electric load applied to a piezoelectric element

(b) equivalent mechanical model

FIGURE 4.33 Equivalent mechanical model of the piezoelectric actuator.

damping element, bp, and thus these need to be taken into account when its dynamic motion is considered in some particular cases. Figure 4.33a shows its equivalent dynamic model composed of a spring-mass-damper system. When an external load Fl is applied to a piezoelectric element, the dynamics of the element can be written as mp

d2 z dz þ bp þ kp z ¼ Fp þ Fl 2 dt dt

ð4:39Þ

where z is the displacement of the piezo actuator in the vertical direction, mp, bp and kp are the effective mass, damping, and elastic stiffness of the piezo ceramics, and Fp is the effective force developed due to the applied net input voltage to the element that can be obtained from the piezoelasticity theory. The first term in the right-head side of Equation 4.39 results from the inverse piezo-effect Fp : Figure 4.33b indicates that although an input command voltage Vp is applied to the actuator, this does not produce the corresponding strain due to the “piezo effect” by the external load. Upon consideration of this effect, effective Fp can be written as Fp ¼ CF ðVp 2 Ve Þ where CF is a piezoelectric force sensitivity constant converting from electric voltage to force, Vp and Ve are the command input voltage and the voltage induced due to external load, respectively. The above Equation 4.39 can be rewritten as mp

d2 z dz þ kp z ¼ CF ðVp 2 Ve Þ þ Fl þ bp dt dt2

ð4:40Þ

This equation of motion describes the displacement of the actuator for a given input voltage, Vp , and external load, Fl. The natural frequency and damping ratio are determined by the system parameters mp, bp and kp. The displacement of the actuator is usually very small and usually ranges within a few mm.

216

Optomechatronics displacement x current i

coil force F

coil magnet yoke

+

R

L

Ve

V

N S

i

−

magnetic flux

(a) voice coil motor

(b) electrical circuit of the VCM leaf spring

yoke coil

base

moving part magnet

(c) actual system configuration FIGURE 4.34 Configuration of voice coil motor.

Voice Coil Motor (VCM) The name “voice-coil motor” originates from the fact that the inventor got the idea for this motor from a speaker. Figure 4.34a illustrates a simple construction of a typical actuator system driven by a VCM, which can freely move in the direction parallel to a mounting rail. The motor is composed of a moving coil (bobbin), two magnets, and a spring-damper system that nullifies the effect of any external disturbance and stabilizes the bobbin motion as well. For clarity, the spring and damper are not shown. The working principle of VCM is the same as that of the DC motor discussed previously and revisited here for better understanding: When a current flows in the coil, this causes Lorentz force which moves the coil. The direction of motion is determined by the direction of the current in the coil and that of the magnetic flow according to Flemming’s left hand rule. In more detail, the magnet M1 generates a magnetic field in the direction of the arrow (x direction) as indicated in the figure. When the bobbin is placed within a magnetic field with magnetic field density B, force F in the x vertical direction is generated according to Fleming’s left hand rule. The force F is given by F ¼ nBi‘c

ð4:41Þ

where n is the number of coil turns, B is the magnetic flux density in tesla, i is the current in ampere flowing in the direction perpendicular to the paper and ‘c is the coil effective length in meters. Figure 4.34b shows the electrical


217

circuit of the VCM which is written by L

di þ Ri ¼ V þ Ve dt

ð4:42Þ

where L and R are the inductance and resistance of the coil, respectively, V is the electrical voltage applied to the coil and Ve is the back emf of the bobbin, respectively. The inductance L can be obtained by Faraday’s and Ampere’s laws and is given by L¼

10 n2 Ac ‘c

where Ac is the cross-sectional area of the coil, and 10 is the permeability of the air. And the resistance R is defined by R¼

r ‘c Ac

where r is the resistivity of the conductor. The back emf is given by Ve ¼ 2nB‘c

dx dx ¼ 2kb dt dt

ð4:43Þ

where dx=dt is the velocity of the bobbin in the upward direction. Rewriting Equation 4.42 by use of Equation 4.43, we have L

di dx þ Ri þ kb ¼V dt dt

ð4:44Þ

Therefore, when the bobbin is supported by a spring as shown in Figure 4.34c, the dynamic equation governing the motion of the bobbin can be written by m

d2 x dx þb þ kx ¼ F dt dt2

ð4:45Þ

where m is the mass of the bobbin, and b existing between the guide and bobbin and k are the damping coefficient and the stiffness of the bobbin, respectively. The actuating force generated by the VCM can adjust the displacement of the bobbin to a desired accuracy, depending upon the choice of these values. We will return to VCM in more detail in Chapter 6. Typical magnetic configurations of linear VCM are shown in Figure 4.35. These are either cylindrical or rectangular in configuration. They have some differences in such characteristics as gap flux density, demagnetization effect, and leakage flux between the magnets and other elements, shielding the leakage. For instance, the outer magnet long coil shown in Figure 4.35a has high leakage between the center yoke and the magnets, whereas inner magnet long coil shown in Figure 4.35b has low leakage factor due to its configuration. Figure 4.35c has low leakage factor and coil inductance in comparison with that of the configuration shown in Figure 4.35d which is the

218

Optomechatronics

N

side yoke

side yoke

S

coil center yoke

N S permanent magnet (a) outer magnet long coil

coil center yoke

N S

(b) inner magnet long coil

N

S

N

S

N

S

N

S

(c) steel plate pole

(d) enclosed magnet long coil

FIGURE 4.35 Configuration of various voice coil motors.

enclosed magnet configuration. This configuration outperforms the previous ones in leakage and uniformity of magnetic field along the path of the coil. Electrostatic Actuator This actuator utilizes an electrostatic force generated by two electrically charged parallel plates. The electrostatic energy stored in this actuator is expressed by U¼

1 CV 2 2

ð4:46Þ

where V is the potential difference between the electrodes and C is the capacitance between two electrodes given by C ¼ 10 1r

A d

ð4:47Þ

In the above, A is the area of opposing electrodes and d is the separation of the electrodes. Consideration of this relation has been already given in the capacitance sensor. Utilizing the relation in Equation 4.46, we can obtain the force acting in an arbitrary direction as Fr ðrÞ ¼

›UðrÞ ›r

ð4:48Þ

Let us elaborate the concept of the electrostatic actuator in two directions. Figure 4.36 illustrates two directional forces that can be actuated by this drive mechanism: (1) overlapped area driving force, (2) gap closing


V

219

V z

z x

(a) overlapped area driving force

(b) gap closing force

kc

bc

V

zs

z

(c) dynamic model of the electrostatic actuator FIGURE 4.36 Electrostatic force actuator.

force. The area driving force shown in Figure 4.36a can be derived using Equation 4.48 and written as Fx ¼

10 1r wV 2 2z

ð4:49Þ

where V is the input voltage applied to the set of the plates, x is the overlap distance whose coordinate is shown in the figure, and w is the width of actuator. The gap closing force per gap is illustrated in Figure 4.36b. The force can be obtained in a similar way as derived in the case of the comb driving force, which can be derived as Fz ¼ 2

10 1r wV 2 2z2

ð4:50Þ

It is noted that the gap closing force increases drastically for a given input voltage V as z decreases. Equation 4.49 and Equation 4.50 present the relationship between the input voltage and the output for a specified gap z or overlap distance x. When this actuator gives an actuating force to a mechanical element, one of the capacitor upper plates moves in z or x directions [19]. Figure 4.36c illustrates the case when the capacitor generates a z-directional force while constrained to a spring kc and a damping element bc. Upon consideration of this dynamic configuration, the equation governing the dynamics of the capacitor plate may be written as mc

d2 z dz 1o AV 2 þ k ðz 2 z Þ ¼ 2 þ b c 0 c dt dt2 2z2

ð4:51Þ

220

Optomechatronics

where A is the charge area of the capacitor and z is the gap between the two plates, which is measured downwards from the equilibrium position, z0. It is given by z ¼ z0 2 zs where z0 is the equilibrium position at which no spring force and capacitor charge occur, and zs is the displacement caused by the spring. This indicates that when z ¼ 0, the displacement of the plate, z, becomes zero. It is important to observe that the driving force appearing in the right hand side of the above equation is a function of displacement z. Examination of this equation shows that at equilibrium the electrostatic force pulling the plate down and the spring force must be equal. This implies that the net force Fn which is the difference between two forces must be zero, which is given by Fn ¼ 2

1AV 2 2 kc ðz 2 z0 Þ 2z2

In order to see the stability of this point, we now differentiate Fn with respect to z, we obtain dFn 1AV 2 2 kc ¼ dz z3

ð4:52Þ

The stability of this point must be dependent upon the sign of the above equation. In order for the capacitor plate to be at a stable equilibrium, the variation of the net force with respect to displacement z must be dFn =dz , 0, which leads to kc .

1AV 2 z3

ð4:53Þ

Detailed discussion on the behavior of the plate dynamics is left for a problem in Chapter 4. We will discuss the dynamic characteristics of this actuator in detail in Chapter 7.

Microactuators There are two categories of microactuators: one is the actuator that drives mechanical elements within micro range regardless of its physical size. The other is the actuator whose physical scale is limited to micro dimension. A number of these microactuators have been developed in the past. Microactuators to be discussed here will include only three actuators that operate on orders up to a few hundred mm. These actuators are made of shape memory alloy, magnetostrictive material, and ionic polymer metal composite. Table 4.6 compares a group of microactuators in terms of deformation range, frequency ranges, and force and so on.


221

TABLE 4.6 Properties of Various Micro-Displacement Actuators Actuator Technology Piezoelectric (BM500) Shape memory (Nitinol) Magnetostrictive (Terfenol-D) Electrostrictive (PMN)

Typical Displacement

Force

Hysteresis

Frequency Range

100 mm (L ¼ 5 cm)

20 kN

8 , 15%

,30 kHz

500 mm (L ¼ 5 cm)

500 N

10 , 30%

,5 Hz

100 mm

1.1 kN

,10%

,4 kHz

65 mm

9 kN

1 , 4%

,1 kHz

Operating Temperature Range 220 , 2508C (Tc ¼ 3658C) up to 4008C (thermal actuation) up to 3008C (Tc ¼ 3808C) 0 , 308C (Tc)

Source: Prasad, E. Sensor Technology limited, 2002.

Shape Memory Alloy (SMA) Actuator SMAs such as titanium nickel alloy (TiNi) are smart materials, which possess thermoelastic martensitic transformation exhibiting shape recovery phenomenon when heated: The alloy has two main phases associated with the upper memory recovery, austenite (high temperature phase) and martensite (low temperature phase). Physically speaking, this means that, when a SMA is stretched from its undeformed state at some temperature below its transformation temperature (high temperature phase), it has the ability to return to the undeformed original shape upon heating. This characteristic results from a transformation in its atomic crystal structure. As shown in Figure 4.37, the martensite state possesses the twinned structure easily to deform while the austenite state has the very rigid cubic structure. This atomic structural change from the twinned martensite to the austenite causes the SMA to generate a large force which can be a very useful property as an actuator. Figure 4.38a depicts

crystal lattice of austenite

FIGURE 4.37 Phase transformation of shape memory alloys.

twinned crystal lattice of martensite

Optomechatronics

contraction,

∆

martensite (finish) austenite (start) cooling

∆T

heating

austenite (finish)

tensile force, F

222

cooling

∆T heating

martensite (start) Tm

temperature (T )

0

Ta

(a) contraction vs. temperature

temperature (T )

Ta

(b) tensile force vs. temperature

FIGURE 4.38 Typical transform hysteresis of shape memory alloys.

the physics involved with phase transformation and shows the relationship between the deformation and the temperature applied. As temperature rises, the deformed material in martensite phase starts to change its phase to austenite. At the temperature Ta , the phase of material is completely turned into austenite. At this phase, the material recovers its length (shape), whose deformation is reversed. It is noted that during this period, elongation is a slightly nonlinear function of temperature. When the material in austenite phase cools from the high temperature Ta , it does not follow up the relationship curve, but the lower curve, slightly deviated from the upper curve. This indicates it undergoes a typical hysteresis effect. As the temperature keeps decreasing, the phase of the material is completely changed into martensite, at temperature Tm : At this phase, the material reaches the deformed state again. This shape recovery property creates a wide range of application areas as an actuator. The drawbacks of this actuator are: nonlinearity due to the hysteresis and slow response. Some of these adverse effects may be partially compensated by applying some advanced control algorithms. In MEMS applications, a variety of methods depositing this SMA film over silicon are used. In this case, actuation is due to the recovery of residual tensile stress in the film. Figure 4.38b shows the relationship of tensile stress vs. temperature for TiNiCu film material. As the film material deposited at temperature Ta cools down, its phase is changed from austenite to martensite. Below temperature Tm , it can be seen that the thermal stress in the film is almost relaxed. The reverse transformation occurs when it is heated from Tm to Ta and causes the film to recover its original shape. Magnetostrictive Actuator Magnetostrictive materials found in 1970s transduce magnetic energy to mechanical energy when these are subjected to an electromagnetic field


223

as indicated in Figure 4.39a. These materials also generate electromagnetic fields when they are deformed by an external force. Therefore, magnetostrictive materials can be used for both actuation and sensing due to this bidirectional coupling. The advantage of this actuator is that it readily responds to significantly lower voltage as compared with piezoelectric actuators (200 to 300 V). Magnetostrictive materials can be deposited over a silicon micromachined cantilever beam as shown in Figure 4.39b when they are used as a micro actuator. The actuator consists of a thin film and a silicon beam. When it is subjected to a magnetic field, the magnetostrictive film expands and causes the beam to bend in the vertical direction. The deflection d is found to show a dependency of field strength (telsa). Typical beam deflection [22] is found to be 100 mm at 0.01 telsa for a 10 mm thick terfenol-D film over a silicon thickness 50 mm and length 20 mm. The recent applications of these materials can be found from a variety of applications: adaptive optics, high force linear motor, active vibration or noise control, and industrial and medical sonar and pump. Ionic Polymer Metal Composite (IPMC) Actuator The IPMC materials possess the susceptibility to interactions with externally applied electric fields and also to their own internal field structure. Due to this property, when an electrical field is applied, the hydrated cations in the materials move to negatively charged electrode side. As a result, the IPMC strip undergoes internal volume change and thus bends towards the anode, negatively charged side, as shown in Figure 4.40. The advantages of this actuator are light weight, relatively large displacement, low input voltage (4 to 7 V) and fast response (msec to sec). The deformation and actuation force are found to be dependent on the applied voltage and geometry of the material. This material can be used as

displacement

(a)

terfenol-D rod

field coil

terfenol-D layer

d (b)

Si

FIGURE 4.39 Configuration of a magnetostrictive actuator.

displacement

224

Optomechatronics

+ – + – + – + – + –

V

+

+ + + + + + – – – – – + – – –

V

FIGURE 4.40 Behavior of an ionic polymer due to an applied electric field.

a sensing element as well. When it is bent, the resulting internal stresses developed cause shifting of mobile charges, which can be detected by a low power amplifier. This constitutes an IPMC sensor.

Signal Display The display element is usually the final data representation element. There are a number of data presentation devices from simple pointer scale and recorders to very large scale display such as cathode-ray-tube (CRT) and plasma display panel (PDP). Here, a brief discussion will be made, being limited to optical or image display devices. As shown in Figure 4.41, optical displays units can be conveniently categorized into small scale alphanumeric displays and large displays that all work in the digital domain. As far as displaying principle is concerned, it is interesting to see that all of these can be grouped into three categories. The first method is to use a light emitting source such as LED. The image optical intensity is directly controlled by controlling input current. The CRT as shown in Figure 4.41a uses this principle, in which deflected electrons controllable by a deflection system within the device interact with semiconductor material such as phosphors formed in dots. Plasma panels in Figure 4.41b are composed of an array of cells known as pixels. The electrodes between two glass substrates produce a gas in a plasma stage, which is made to react with phosphors in each pixel in discharge region. The resulting reaction causes each pixel to produce a desired image at the screen. The second one does not emit light but uses light incoming from other sources. In the liquid crystal device (LCD) depicted in Figure 4.41c, orientation of molecules remains not random but in certain directions, making the optical effect predominant in those directions. This property makes it feasible to modify the crystal structure and thus optical effects by applying electric fields to the LCD. The last display device is to use a signal manipulation technique which is totally different from the conventional display methods in the above. As shown in Figure 4.41d it is based on MEMS technology, and has two types


screen

anode focusing

225

bus electrode

front panel

dielectric phosphor cathode ray tube

x-defection

back panel

y-defection

address electrode

(a) CRT

(b) PDP 7 segment electrode patterns

.. . . .. .

5 liquid crystal 6 7 material glass plate with electrode pattern

reflected light incident light

4

3

21

(c) LCD panel displaying numerals

mirror

(d) DMD schematic

FIGURE 4.41 Large scale display units.

of display, one utilizing optical switching and the other utilizing grating diffraction. The device using optical switching is called a digital mirror device (DMD). The device contains hundreds of millions of tiny mirrors which are embedded in one chip, and is based on micro-electro mechanical system technique (MEMS). Its main role is to steer a light beam into a screen (display unit) in order to project an image in a desired manner by a simple mirror switching action. The innovative concept lies in the fact that in contrast to (LCD) the color content of the image and intensity of the projected image can be flexibly controlled by their tiny motion and switching time. Another device that belongs to this category utilizes grating diffraction. It is called grating light value (GLV) and is composed of a number of pixels which in turn are composed of six diffraction grating made deformable by an actuator. From the aspect of technology integration, DMD projector and GLV are of optomechatronic nature, while LCD is based on an optical-electrical (OE) combination having no moving or deflecting projection of light. We will discuss DMD and GLV in more detail in Chapter 7.

Dynamic Systems and Control Previously, we have seen a variety of physical systems from simple sensors to quite complicated complex systems. Many such systems involve

226

Optomechatronics

phenomena of more than one of the disciplines such as mechanical, electrical, optical and so on. In this case, we need to consider the physical interaction between the discipline involved, which adds more complexity such as nonlinearity, hysteresis and saturation to physical behaviors of the systems. Looking at the systems from a different view, many systems are considered to be static but some are inherently dynamic in nature. In many cases, static condition assumed for static systems often may not be valid due to some physical conditions involved, but may often need to be treated as dynamic systems, as we have already illustrated for the sensors and actuators treated in the previous sections. When it comes to analysis and control of these dynamical systems, characterization of their behaviors is ultimately necessary. To do this physical modeling we first need to take into consideration the properties of physical variables involved and their interaction. Based on this modeling, a control system is to be designed next. Undoubtedly, the performance of the control largely depends on accuracy of the physical model and therefore, modeling needs to be done as accurately as possible. Since most of physical systems encountered in engineering are not ideally simple, some approximation is usually made to make them a linear system which makes controller design relatively much simpler than those not approximated. In this section, we will briefly deal with system modeling, system transfer function, which represents the input – output relationship, and some elementary control techniques. Dynamic Systems Modeling Modeling refers to description of system dynamics in a mathematical form. It involves understanding of the physics involved and mathematical formulation of the physical behavior. Therefore, a fundamental step in building a dynamic model is writing the equations of motion for the system to be modeled. In doing this, we may often find it difficult to completely and accurately describe the system dynamics in detail due to the uncertainty and complexity associated with physical interactions between variables. In this situation, we resort to an identification method with which the unknown or time varying parameters of the system can be identified either from experiments or simulations. There are a number of types of system that can be identified depending on how they behave. These are summarized in Figure 4.42. The lumped model uses lumped variables to represent the systems dynamics while the distributed system uses the partial differential equation in order to describe the system dynamics in both space and time without lumping them. Since the classification given in the figure is fairly standard, one may easily find the description of each model in any control-related books. The systems we will deal with belong to a lumped, continuous and deterministic model. We have already modeled the dynamics of various actuators and sensors. Therefore, we will not illustrate many actuator systems, but instead consider


227

system model

lumped system

discrete system

linear system

deterministic system

time varying system

distributed system

continuous system

nonlinear system

stochastic system

time invariant system

FIGURE 4.42 Types of dynamical systems.

three distinct systems in nature: thermal, vibration and fluid positioning system. Thermal Systems Consider a thermal system as shown in Figure 4.43a. It is assumed that the system at temperature T is exposed to the surrounding environment at temperature T0 : These temperatures are assumed to be uniform throughout the system. When this system has a heat energy inflow, heat energy flow rate through it will be governed by q¼

1 ðT 2 T0 Þ Rt

where q is the heat energy flow rate, and Rt is the thermal resistance. The net inflow into the substance within system causes variation of the temperature of the system. When T0 is constant, the relationship given above can be written as dT 1 ¼ q dt Ct where Ct is the thermal capacity. The above equation indicates that for a given rate of change of heat flow or rate of energy storage q, if the thermal capacitance of the system is large, the rate of temperature change becomes low. The reciprocal of thermal resistance is called “heat conductance.” Spring-Mass-Damper System Many practical systems can be modeled as a spring-mass-damper system if some assumptions are made properly. Figure 4.43b illustrates one such system composed of a two-mass system. From the free-body diagram shown in the figure, we can model the system as m1

d2 x 1 dx1 dx2 ¼ 2b 2 dt dt dt2

2 k1 ðx1 2 x2 Þ þ F

ð4:54Þ

228

Optomechatronics flow in thermal system

valve (orifice)

heat inflow q1(t ) q2 (t ) heat outflow T(t ) T0(t )

(a) thermal system

piston Q A

x

(c) fluid system F x1

m11 k1

F

m m11

b m21 m

k2

k1 (x1–x2) b dx1– dx2 dt dt

x2

k1 (x1–x2) b dx1– dx2 dt dt

m m211 k2x2

(b) vibration system FIGURE 4.43 Various systems for dynamic modeling.

m2

d2 x 2 dx1 dx2 2 ¼b dt dt dt2

þ k1 ðx1 2 x2 Þ 2 k2 x2

It is important to note that the displacements x1 and x2 are taken from their equilibrium position that arises due to gravitational force. Therefore, this force was not included in the above model. Fluid System The fluid flow shown in Figure 4.43c is designed to move a piston in a desired fashion in the x-direction. The physical laws involved with this flow are continuity, resistance and force equilibrium for the piston. When the fluid flows through the orifice (valve) and enters into the empty area of the piston, due to flow resistance at the valve, the mass flow rate Q becomes Q¼

1 pffiffiffiffiffiffiffiffiffiffiffiffi 1 pffiffiffiffi ðP1 2 P2 Þ ¼ DP R R

ð4:55Þ

where R is the orifice flow resistance, and P1 and P2 are the fluid pressures at both sides of the orifice. The flow should meet the continuity law Ardx ¼ Qdt


229

where A is the piston area and r is the fluid density. Due to this fluid flow, the equation governing the motion of the piston is obtained by ADP ¼ m

d2 x dx þb dt dt2

ð4:56Þ

where m and b are the mass and damping coefficient of the piston, respectively. Examination of Equation 4.55 and Equation 4.56 signifies that the model derived is highly nonlinear due to flow through the orifice. However, mass flow rate can be linearized near the normal operation point, Q0 , DP0 , R0 , which will result in a linearized equation. Optical Disk The optical disk shown in Figure 4.44 has two main servo systems composed of a tracking system (radial) and an autofocusing system (vertical). The track following servo is divided into a fine track servo and a coarse motion track servo which are schematically illustrated in the square box. The fine servo system is mounted on the optical pickup and actuated by VCM. As can be seen from the figure, track-following accuracy is largely dependent on this fine servo system. The operation principle is as follows: the laser beam coming out of the laser diode passes through the oblique glass plate (mirror) which is collected by the objective lens. Depending on the location of the lens in the vertical direction, the collected beam may or may not be focused onto the disk surface. The beam reflected from the disk pits travels back through the glass plate and finally is detected by the photodiode. The objective of the track following systems here is to position the laser spot in radial direction with sufficient precision. We will discuss this in detail in Chapter 6.

x

Ω

optical disk

k

objective lens

fine motion unit

b

photo diode

laser diode mirror coarse motion

FIGURE 4.44 Track following system of an optical disk.

230

Optomechatronics

Let us consider the motion of the fine servo system. It is composed of a mass, a damper, and a spring attached to the VCM. The equation of the motion of the fine servo system is described by m

d2 x dx þb þ kx ¼ F dt dt2

ð4:57Þ

Here, m is composed of the mass of the lens and VCM, b is the damping coefficient, and F is the force induced by disk run-out and imbalance of the disk. The unbalance force comes in due to the fact that the disk itself is not perfectly balanced. It should be noted that, although F is very small, this influences the overall accuracy of track following, since track following accuracy is required to be highly precise. Dynamic Response of Dynamical Systems Once the system dynamics is modeled in a mathematical form, we now need to investigate how the system will respond to a given input. As we have modeled a variety of dynamical systems in the previous sections, we know that the complexity may differ from system to system. The simplest system we have observed is a first order system which is the case of the thermal system. And the next is a second order system, which the rest of the illustrative systems belong to. In this manner, dynamic systems may be in general represented by an nth order system whose dynamics is given by a0

dn x dn21 x dm u dm21 u þ a1 þ · · · þ a x ¼ b þ b þ · · · þ bm u n 0 1 n m dt dt dt1 dtm21

ð4:58Þ

where ai and bi are the system parameters and input parameters, respectively. Here, it is important to note that ai and bi may have two different types; time-invariant and time-variant. The two forms make a lot of difference in system analysis: The time invariant case is allowed to utilize the Laplace transformation method as we shall discuss later, whereas the latter case is not permitted, thus making system analysis much more difficult. At this stage we now identify the given system dynamic model as an inputoutput model, which forms the basic of control system. By defining u as an input and x as an output, we will start analyzing system response with a first order system. Let us first consider a first order system which is described by a differential equation of the form a0

dx þ a1 x ¼ b1 u dt

where uðtÞ is an input, and a0 , a1 and b1 are constants. If we let a0 =a1 ¼ t and


231

1.2 t=1

step response, x (t )

1.0 0.8

t=4

0.6

t=3 t=2

0.4 0.2 0.0

0

2

4

6 8 time (t), sec

10

12

14

FIGURE 4.45 Step response for a first order system according to change of time constant.

b1 =a1 ¼ 1 for simplicity, then, this equation is modified as

t

dx þx¼u dt

ð4:59Þ

where t is called “time constant.” When a unit step input is applied to the system, i.e., u ¼ 1, ðt . 0Þ the steady state solution of this equation is obtained by xðtÞ ¼ ð1 2 e2t=t Þ

ð4:60Þ

which signifies that for a unit step input and zero initial condition the final value of xðtÞ reaches unity at a steady state, i.e., at t ¼ 1: The characteristics of the first order system differ largely by time constant, t: When t ¼ t the response becomes xðtÞ ¼ ð1 2 e21 Þ: At this time xðtÞ reaches 0.63 of its steady state value as shown in Figure 4.45. In a time of 3t, xðtÞ rises to 95% of the steady state value. Therefore, it is important to note that time constant t can be a measurement indicating speed of the response. When t p 1, the response is very fast, but, when t is relatively large, the response gets sluggish. The second order system which has the most practical importance in real situations takes the form of the following; a0

d2 x dx þ a1 þ a2 x ¼ b1 u dt dt2

where a0 , a1 , a2 and b1 are constants. This can be modified as d2 x dx þ v2n x ¼ b0 v2n uðtÞ þ 2jvn 2 dt dt

ð4:61Þ

232

Optomechatronics

where 2jvn ¼ a1 =a0 , and a2 =a0 ¼ v2n , b1 =a0 ¼ b0 v2n : In the above, vn is the natural frequency of the system, and j is the damping ratio. The roots of the characteristic equation p2 þ 2jvn p þ v2n ¼ 0 are given by p1 ; p2 ¼ 2jvn ^ pffiffiffiffiffiffiffiffi 2 vn j 2 1: Examination of the roots indicates that the equation can have three different roots depending upon j value; for underdamped case, j , 1, for critically damped case j ¼ 1, for overdamped case j . 1. When the roots j , 1 become complex conjugate, the response of the system becomes for a unit step input and zero initial conditions " ( )# j jvn t cos vd t þ pffiffiffiffiffiffiffiffi sin vd t xðtÞ ¼ b0 1 2 e 1 2 j2 pffiffiffiffiffiffiffiffi where the damped natural frequency is given by vd ¼ vn 1 2 j2 : The above equation reduces to " # e2jvn t xðtÞ ¼ b0 1 2 pffiffiffiffiffiffiffiffi cosðvd t 2 wÞ ð4:62Þ 1 2 j2 j w ¼ tan21 pffiffiffiffiffiffiffiffi 1 2 j2 Figure 4.46 illustrates typical responses for several values of j for b0 ¼ 1:0: As j becomes smaller, the response becomes more oscillatory with larger amplitude while speed of response gets faster. It is seen, however, that the maximum value of the response corresponding to the first peak value is increasingly higher with decrease of j value. As we further decrease in j we can anticipate that a sinusoidal response will eventually appear when j ¼ 0: 1.6 x=0.25

1.4 x=0.55

step response, x (t)

1.2

x=0.75

1.0

x=0.10

0.8 0.6

x=0.15

0.4 0.2 0.0

0

2

4

6 8 time (wn t )

10

12

FIGURE 4.46 Step response for a second order system according to changes of j value.

14


233

When j ¼ 1, the roots are equally at 2vn , and the response becomes xðtÞ ¼ b0 ½1 2 e2vn t 2 vn te2vn t

ð4:63Þ

The response shows no oscillation but approaches the steady state value with slower speed than that of the case j , 1: When j . 1, two roots are real and distinct, and the response becomes xðtÞ ¼ b0 ½1 þ Aep1 t þ Bep2 t

ð4:64Þ

where p1 and p2 are the roots of the system. In this case, the response gets sluggish with even slower speed than the case where j ¼ 1: System Transfer Function As we have seen previously, the response of the system dynamics relating an input to an output has been obtained by solving the differential equation analytically. When the system dynamics come to higher order, the solution method must resort to a numerical approach. Since an analytical solution tool is not easily available, this time domain approach makes it difficult to understand the input –output relationship as well as the characteristics of system dynamics until a numerical solution comes out. To avoid this difficulty we usually transform a differential equation describing system dynamics into an algebraic equation in Laplace domain or s-domain. This transformation method is called Laplace transform which is an indispensable tool for control system analysis and design. This is because the s-domain transformation provides us with a transfer function that relates the input and output of a dynamic system. Once this is obtained in s-domain, we carry out the necessary calculation in algebraic form. To elucidate the concept in more detail, let us suppose that the input and output variables are transformed into Laplace domain (s-domain), respectively, as L½xðtÞ ¼ XðsÞ;

L½uðtÞ ¼ UðsÞ

where L denotes the Laplace transform of the variable in ½· : Utilizing the transformation, we may write the nth order dynamic equations in Equation 4.58, in a Laplace transformed form. The transfer function of the system describing the relationship between the input and output, can be written as GðsÞ ¼

XðsÞ UðsÞ

ð4:65Þ

where GðsÞ denotes the transfer function. It is important to notice that in transforming the differential equation all initial conditions are assumed to be zero. A block diagram representation GðsÞ is given in Figure 4.47. From this relationship, the response XðsÞ in s-domain can be easily obtained by XðsÞ ¼ GðsÞUðsÞ

ð4:66Þ

234

Optomechatronics G(s) 1 ts+1

U(s)

X(s)

(b) first order U(s)

G(s)

X(s) G(s)

(a) generic representation U(s)

wn2 s2

+ 2xwns + w n2

X(s)

(c) second order FIGURE 4.47 A block diagram representation of system transfer function.

if UðsÞ is given. Therefore, the important properties of system dynamics such as stability and characteristics of system response can be analyzed by manipulating the above algebraic equation in s. If necessary, we can easily go back to the time domain from the s-domain in order to do some analysis simply by inverse-transforming the above equation in s L21 ½XðsÞ ¼ L21 ½GðsÞUðsÞ ¼ L21 ½GðsÞ p L21 ½UðsÞ where the symbol p indicates the convolution operation. Let us find out how we obtain these transfer functions by revisiting the differential equations we have dealt with previously. First Order System Consider again the dynamic system described by a first order differential equation given in Equation 4.59. The equation is rewritten here for subsequent discussion

t

dx þx¼u dt

Laplace transform of this is given by

t sXðsÞ þ XðsÞ ¼ UðsÞ which for zero initial condition leads to the following transfer function; GðsÞ ¼

XðsÞ 1 ¼ UðsÞ ts þ 1

A block diagram representation is shown in Figure 4.47b. When UðsÞ is a unit step input, then XðsÞ is given by XðsÞ ¼

1 sðts þ 1Þ


235

It can be shown that Laplace inverse transforming of this equation yields the same differential equation given in Equation 4.60. It is worthwhile to remembering that time constant t determines the speed of the system response, as already discussed. Second Order System Consider the second order system described by the equation given in Equation 4.61 which is rewritten as d2 x dx þ 2jvn þ v2n x ¼ b0 v2n uðtÞ 2 dt dt When we assume Laplace transforming of the above equation with all zero initial conditions we obtain s2 XðsÞ þ 2jvn sXðsÞ þ v2n XðsÞ ¼ b0 v2n UðsÞ from which the transfer function is obtained by GðsÞ ¼

XðsÞ b0 v2n ¼ 2 UðsÞ s þ 2jvn s þ v2n

ð4:67Þ

The block diagram of GðsÞ is described in Figure 4.47c for b0 ¼ 1: As we have already discussed, the roots of denominator of GðsÞ ¼ 0, that is, s2 þ 2jvn s þ v2n ¼ 0 yield three different cases, depending upon j value, as before. pffiffiffiffiffiffiffiffi When j , 1; Two roots are at p1 , p2 ¼ 2jvn ^ jvn 1 2 j2 which are complex conjugates. By using these roots, Equation 4.67 can be rewritten as GðsÞ ¼

v2n ðs1 þ p1 Þðs þ p2 Þ

ð4:68Þ

When UðsÞ is a unit step input, XðsÞ is obtained by XðsÞ ¼

v2n pffiffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffi s s þ jvn þ jvn 1 2 j2 s þ jvn 2 jvn 1 2 j2

By Laplace inverse transforming of this, we have exactly the same response equation given in Equation 4.62. When the system is critically damped, i.e., j ¼ 1 two roots are equally at p ¼ p1 ¼ p2 ¼ 2vn , GðsÞ in this case is given by GðsÞ ¼

v2n ðs þ pÞ2

from which XðsÞ is obtained for UðsÞ ¼ 1=s by XðsÞ ¼

b0 v2n sðs þ vn Þ2

The time domain response of this is identical to that given in Equation 4.63. When j . 1 two roots are distinct and real and located at

236

Optomechatronics

pffiffiffiffiffiffiffiffi p1 ,p2 ¼ 2jvn ^ vn j2 2 1, we have GðsÞ ¼

b0 ·v2n sðs þ p1 Þðs þ p2 Þ

XðsÞ is then given by XðsÞ ¼

b0 ·v2n pffiffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffi sðs þ jvn þ vn j2 2 1Þðs þ jvn 2 vn j2 2 1Þ

The Laplace inverse transform of this equation yields the same time domain response as given in Equation 4.64. Higher Order System When a system is described by the nth order differential equation given in Equation 4.58, a general form of its transfer function is represented by m Y

ðs þ z1 Þðs þ z2 Þ· · ·ðs þ zm Þ j¼1 ¼ n GðsÞ ¼ Y ðs þ p1 Þðs þ p2 Þ· · ·ðs þ pn Þ i¼1

ðs þ zj Þ ð4:69Þ ðs þ pi Þ

if the denominator and nominator are factored out by use of n poles, pi ði ¼ 1,2· · ·nÞ and m zeros zj ðj ¼ 1,2· · ·mÞ: The pi and zj may include the second pffiffiffiffiffiffiffiffi order complex conjugate terms s þ jvn ^ vn j2 2 1: It is noticed that a system to be realizable needs to satisfy the condition n q m: The Laplace transform of this general equation can be made in a similar way to the case of the second order system. This is done by use of the partial fraction method which decomposes the transfer function into several simple first or second order terms. Once the transfer function can be decomposed like this, then control system analysis and design will be much easier than handling the unfactored form. Laplace Transforms Theorems There are several theorems on the Laplace transform that are useful in the analysis and design of control systems. Here, we present them without proof. Final value theorem: If f ðtÞ and df ðtÞ=dt are Laplace transformable, and if f ðtÞ approaches a finite value as t ! 1, then lim f ðtÞ ¼ lim sFðsÞ

t!1

s!0

it is noted that when f ðtÞ is a sinusoidal function such as cos vt, and sin vt, this theorem is not valid, since for these functions lim f ðtÞ does not exist. t!1 This theorem is frequently used to determine the steady state error of the response of a control system.


237

Initial value theorem: If f ðtÞ and df ðtÞ=dt are Laplace transformable and if lim sFðsÞ exists, then s!1 f ð0þ Þ ¼ lim sFðsÞ s!1

where t ¼ 0þ implies the positive side of time at t ¼ 0, and this definition together with t ¼ 02 is useful for a function having discontinuity at some instant of time. This theorem is useful to determine the initial value of the response of a control system at t ¼ 0þ , and thus the slope of the response at that time. The initial value theorem and final value theorem enable us to predict the system behavior in the time domain without actually transforming Laplace transformed functions back to time functions. Open Loop vs. Feedback Control A system normally consists of a series of several transfer function elements as shown in Figure 4.48a. In this case, the block diagram is reduced to obtain the overall transfer function GðsÞ XðsÞ ¼ G1 ðsÞG2 ðsÞG3 ðsÞ ¼ GðsÞ UðsÞ

ð4:70Þ

This equation signifies that once the desired output value is specified we can determine one-to-one correspondence between the input and output. For instance, when a desirable xd ðtÞ is given, we can provide the system with an input UðtÞ so as to make the response xðtÞ ! xd ðtÞ as time t goes to infinity. This is true only when the system is not subject to external disturbance, which will be discussed shortly. This implies that this type of control is called open loop control and does not use any measurement information. In contrast to this, the feedback or closed loop control utilizes an instantaneous measurement information obtained by a sensor or sensors for feedback, as shown in Figure 4.48b. The overall system transfer function can

U(s)

G2(s)

G1(s)

G 3(s)

X(s)

Xd(s)

G(s) equivalent

(a) open loop control FIGURE 4.48 Open loop control vs. feedback control.

G(s)

X(s)

original

original

U(s)

+_

X(s)

Xd(s)

G(s) 1 + G(s) equivalent

(b) feedback control

X(s)

238

Optomechatronics

be obtained upon consideration of error variables which is defined by EðsÞ ¼ Xd ðsÞ 2 XðsÞ Since XðsÞ=EðsÞ ¼ GðsÞ, we finally have XðsÞ GðsÞ ¼ Xd ðsÞ 1 þ GðsÞ

ð4:71Þ

Comparison of Equation 4.70 and Equation 4.71 enables us to examine two basic differences between the open loop control and closed loop control. The first one is that the closed loop control is much less sensitive to the variation of system parameters. To illustrate this let GðsÞ be changed by DGðsÞ from its original function. Then open loop control :

XðsÞ ¼ ½GðsÞ þ DGðsÞ UðsÞ

closed loop control :

XðsÞ ¼

GðsÞ þ DGðsÞ X ðsÞ 1 þ GðsÞ þ DGðsÞ d

In the open loop system, the response XðsÞ is varied by the amount DGðsÞUðsÞ, while in closed loop control XðsÞ is varied approximately by DGðsÞ X ðsÞ 1 þ GðsÞ d Therefore, sensitivity to variation of system parameters is greatly reduced by a factor 1=ð1 þ GðsÞÞ: The second one is that the closed loop control is much less sensitive to external disturbance. To illustrate this let DðsÞ be the Laplace transform of a disturbance dðtÞ, as indicated Figure 4.49, XðsÞ given in Equation 4.70 is modified as open loop control :

XðsÞ ¼ GðsÞUðsÞ þ DðsÞ

closed loop control :

XðsÞ

GðsÞ 1 X ðsÞ þ DðsÞ 1 þ GðsÞ d 1 þ GðsÞ

We can see again that the effect of the disturbance can be drastically reduced with closed loop control. However, one thing to take into careful consideration is that the closed loop system often becomes unstable even if the system is open loop stable. Therefore, the closed loop control system design should take into consideration system stability. In conclusion, the effects of system variation and disturbance on system response can be controllable or even eliminated with some types of control action we will discuss later. System Performance As we have seen already, the system response of the second order system is largely dependent on damping ratio, j and natural frequency, vn : In other words, these system variables determine the response characteristics at both


239 D(s)

(s)

+

G(s)

+

X(s)

open loop

D(s) X d(s)

+_

G(s)

+

+

X(s)

closed loop FIGURE 4.49 Inclusion of an external disturbance.

transient and steady states. This indicates that, once the desired characteristics are specified, we need to determine the corresponding system variables, j and vn according to the specification. The problem is how we define such characteristics. In many practical cases, response characteristics of a dynamical or controlled system are specified in terms of time domain parameters that are indicative of system performance. Because the transient response of a practical system often exhibits damped oscillatory behavior before reaching state, we will define time domain parameters by using an oscillatory transient response curve we have observed from the second order system. In defining such parameters, it is common to use a unit step input: in fact, there are several types of test signals such as impulse, unit step, ramp, parabolic, sinusoidal and random signal as shown in Figure 4.50. The choice of the test signal is dependent upon what type of response a system is required to produce. For example, if the system is a positioning system, the step type test signal may be useful to test its response. Referring to Figure 4.51 which shows a typical response curve of dynamic system for a unit step input, we may specify the time domain parameters commonly used for evaluating the response characteristics. For transient response: (1) rise time ðtr Þ : the time required for the response to rise from 10 to 90%, 5 to 95% or 0 to 100% of its final value, (2) peak time ðtp Þ : the peak time is the time required for the response to reach the first peak of the overshoot,

240

Optomechatronics

(1) impulse

(2) step

(4) parabolic

(3) ramp

(5) sinusoidal

(6) tracking

FIGURE 4.50 Various test signals to evaluate system response.

(3) maximum overshoot ðMp Þ : this quantity is defined by Mp ¼

xðtp Þ 2 xð1Þ £ 100% xð1Þ

For steady state response:

1.2

Mp

response, x (t )

1.0

2%~5%

0.8 0.6 0.4 0.2 0.0 tr

tp

ts

FIGURE 4.51 Characterization of a system response for a unit step input.

time (t )


241

(4) settling time ðts Þ : the time required for the response to reach and stay within a range of 2 to 5% of the final value. (5) steady state error: the error which is the deviation from the final value at steady state eð1Þ ¼ 1 2 xð1Þ According to this specification, we may calculate the following time domain parameters for the second order system. tp ¼

pffiffiffiffiffi2 p 4 3 , Mp ¼ e2ðj 12j Þp , ts ¼ ð2% criterionÞ or ð5% criterionÞ ð4:72Þ vd jvn jvn

From this specification, it is noted that all time domain parameters are a function of j and vn : Clearly, these parameters are a family of parameters that can characterize the system performance. As long as we keep all the parameters ðtp ,Mp ,ts Þ as small as possible, we can achieve the best desirable response, if there is no steady state error. The second order system given in Equation 4.67 has no steady state error if b0 ¼ 1: In general, steady state error appears in most control systems, although it is small. In some cases this small error may not be tolerable which requires careful design of a controller that may reduce or eliminate error altogether. One important thing to be noticed is that there is always a trade off between rise time ðtp Þ and maximum overshoot ðMp Þ because shorter rise time implies larger overshoot, and vice versa. A higher order system can also be specified using these parameters only in certain simple situations. Normally, we need to make an appropriate assumption for system analysis and design or to determine the parameters by obtaining the response numerically (Simulink). Basic Control Actions Most dynamical systems do not exhibit desirable response characteristics unless they are controlled properly. For instance, some systems may not satisfy the transient response requirements while some others may not satisfy the steady state response requirements. In severe cases, they may be inherently unstable for some range of operation. The effective way of correcting the unwanted response characteristics is to use a feedback control method. As we have already pointed out before, this method utilizes the information on the instantaneous state of systems, which is acquired by sensors. Based upon the sensor information, system actuators adjust their manipulating signal so as to reduce or eliminate the error that occurs at any instant of time and eventually the desired response. Figure 4.52 illustrates a generic form of a block diagram of a control system. It is composed of a controller, GC ðsÞ which produces a command signal to an actuator depending upon the error variable, an actuator that actuates the system,

242

Optomechatronics

according to the command signal from the controller, a dynamic system GP ðsÞ to be controlled and finally a sensor GS ðsÞ that measures the instantaneous state of the system. All of these elements are of paramount importance to obtain desired system performance, because each affects system response characteristics. In particular, sensors and actuators must be suitably chosen so that their dynamic characteristics meet the overall system specification. In addition, the control signal should be properly generated by the controller, taking into account the dynamic characteristics of the system, actuators, sensors, and disturbances. There are a variety of controller types that may be categorized into several groups but here we will discuss in brief only three classical controllers; proportional (P) controller, proportional plus integral (PI) controller, proportional plus integral plus derivative (PID) controller. Upon examination of the three controllers listed above, it can be observed that combination of three basic actions enables those controllers to generate appropriate command signals. These are proportional (P), derivative (D), and integral (I) actions. Figure 4.53 shows those three basic actions. Proportional control action is based upon the proportional relationship between the controller output uðtÞ and the error signal eðtÞ which is described by uðtÞ ¼ kp eðtÞ

ð4:73Þ

or in Laplace-transformed quantities UðsÞ ¼ kp EðsÞ

ð4:74Þ

where kp is termed the proportional gain. This control action produces a signal that is linearly proportional to the error between the desired set value and the actual output value. Therefore, this type of action may be said to be one of the simplest control actions among the three. In integral action, the value of controller output uðtÞ is changed at a rate proportional to the integral

disturbance D(s) command signal

X d(s) desired value

+_

error controller E(s) Gc(s)

actuating signal actuator

U(s)

Ga(s)

Gs(s)

measured variable

sensor

FIGURE 4.52 A generic representation of a control system.

M(s)

+

+ Gp(s)

X(s) actual output


desired value

Gc(s)

+_

243

Ga(s)

sensor signal

kp

+_

(a) proportional

+_

ki s

(b) integral

+_

kds

(c) derivative

FIGURE 4.53 Basic control actions.

of the error, whose relationship is given by ðt UðsÞ k uðtÞ ¼ ki eðtÞdt or ¼ i EðsÞ s 0

ð4:75Þ

where ki is the integral constant. This action provides a signal that is a function of all past values of error rather than just the current value. The action is effective in the steady state rather than in the transient state, because in the steady state, the error may be accumulated as time increases. The last type is the derivative action which has the form uðtÞ ¼ kd

de dt

or

UðsÞ ¼ kd s EðsÞ

ð4:76Þ

This control action depends upon the rate of change of the error. As a result, a controller with derivative action exhibits an anticipatory error. As can be observed, this action is effective during the transient period but not at steady state where not much variation in response occurs. It has the effect of stabilizing the response. The PI controller is composed of a proportional (P) action and integral action (I). The control signal is given by ðt k uðtÞ ¼ kp eðtÞ þ ki eðtÞdt ð4:77Þ or UðsÞ ¼ kp þ i EðsÞ s 0 As can be seen, the controller gains kp and ki play the role of a weighting factor on each term. If kp . ki , the proportional controller contributes to the command signal more than the integral part does. On the other hand, if ki . kp , the integral action contributes more. This controller may be effective when a system needs faster response and requires no steady state error due to constant disturbance. Electronic implementation of this controller in a single circuit is shown in Figure 4.54a. Refer to the basic circuits illustrated in Figure 4.16.

244

Optomechatronics

R2 R1

C1

R1

R3 R2

– +

C1

(a) PI controller

– +

(b) PD controller R2

C1

C2

– +

R1

(c) PID controller FIGURE 4.54 Electric implementation of three controllers.

The proportional plus derivative (PD) controller may be represented by uðtÞ ¼ kp eðtÞ þ kd

deðtÞ dt

or

UðsÞ ¼ ðkp þ kd sÞEðsÞ

ð4:78Þ

This controller may be effective when a controlled system requires proper amount of damping without too much loss of response speed. Figure 4.54b shows the circuit implementing this controller. The most general type of controller is basically a three mode controller which is the PID controller defined by ðt deðtÞ or uðtÞ ¼ kp eðtÞ þ ki eðtÞdt þ kd dt 0 ð4:79Þ ki þ kd s EðsÞ UðsÞ ¼ kp þ s When a system to be controlled requires a proper amount of damping, a proper speed and no steady state error to constant external disturbances, this control action may be effective. Again, it is noticed that kp , ki and kd can be interpreted as the weighting factor of each action for the controller. For instance ki q kp , kd implies that integral action contributes greatly in comparison to the other two actions. Figure 4.54c shows a simple PID controller combined in a single amplifier circuit. System Stability Stability of a system is a property that needs to be checked first, because if a system is unstable, then the response characteristics we have discussed in the above is not meaningful at all. System stability can be checked by


245

Im [s]

neurally stable Re[s] = 0; s = ± j ω stable region Re[s] < 0

unstable region Re[s] > 0 O

Re[s]

FIGURE 4.55 Stability in complex s-plane.

examining the denominator of GðsÞ given in Equation 4.69. The stability requires that all roots satisfying the characteristic equation (denominator set equal to zero) must lie in the left-half s plane as indicated in Figure 4.55. To be more specific, all roots that satisfy DðsÞ ¼ ðs þ p1 Þðs þ p2 Þ…ðs þ pn Þ ¼ 0

ð4:80Þ

must be strictly in the left hand side of the complex plane excluding jv axis. In other words, all pi s must have negative real part. If any one of pi s does not meet this requirement, the system is said to be neutrally stable or unstable. This stability check can be made by using the Routh Stability Criterion in Appendix A3. For a neutrally stable system, the roots lie in the jv axis and therefore its response exhibits a typical sinusoidal oscillation with constant amplitude but does not increase with time. For an unstable system, its response to any input will increase monotonically with time or oscillate with increasing amplitude. An important remark to be made here is that whether a linear system is stable or unstable is a property of the system itself, but not a property of external input, unless the input is dependent upon the system response.

Problems P4.1. Consider a capacitance sensor composed of two parallel plates separated by a dielectric material. Its original overlapping length is ‘: Suppose that the upper plate is moved by D‘ as shown in Figure P4.1. Using Equation 4.2, determine the sensitivity function defined by S ¼ DC=D‘:

246

Optomechatronics moving w

fixed d

FIGURE P4.1 A schematic of a capacitance sensor with changing area.

P4.2. When a dielectric material between two plates moves by ‘ in the direction of the x axis, as shown in Figure P4.2, obtain the total capacitance of this sensor. 1r1 and 1r2 are the permittivity of the relative dielectric constants of material 1 and material 2, respectively. plate dielectric εr1

dielectric εr2

x

FIGURE P4.2 A capacitive sensor with dielectric area change.

P4.3. A piezoelectric sensor consists of a piezoelectric transducer, a connecting cable and an external circuit, as shown in Figure P4.3. Assume that it is modeled as a spring-mass-damper system having stiffness kp , damping coefficient bp and mass mp :

applied force, F z

RL i

Cp

Cc

VL

piezoelectric transducer piezoelectric transducer

cable

load

FIGURE P4.3 Piezoelectric sensor configuration.

Suppose that the cable is represented by a pure capacitance element Cc and that the external circuit is represented by a pure load RL : (1) Write down the equation of motion of the sensor in the z direction for a given force F.


247

(2) If the transducer is modeled by a current generator with a capacitance Cp , write down the equation for the electric circuit shown in the figure. (3) Obtain the transfer function between the force F and the output voltage VL : P4.4. Suppose that there is a point light source (object) S distant from a detector (D)

object S FIGURE P4.4 A detector receiving a bundle of rays from an object surface.

detector, D, as shown in Figure P4.4. When high efficiency of sensing is required, suggest a method to increase the efficiency. R2

R1

C

Vin

Vout

FIGURE P4.5 Integral operation by op-amp.

P4.5. Shown in Figure P4.5 is an electronic circuit of the integral operation. Obtain the equation describing the input– output relationship. P4.6. In micro electro-mechanical-systems (MEMS) or micro actuator and sensors, clamped beam structure has a variety of interesting deformable beam h fixed beam

d

V

δ fixed

FIGURE P4.6 A capacitive clamped–clamped type actuator.

248

Optomechatronics

applications. One such application can be found from a capacitive type actuator as shown in Figure P4.6. The two beams are separated initially by d: The upper beam is deformable, while the lower beam is fixed. If the electric field V is applied to the deformable beam, determine the force per unit length acting on the beam element. In the figure, ‘, w and h are the length, width, and thickness of the beam, respectively. P4.7. A spring of mass m is made of shape memory alloy (SMA). The objective of the use of the SMA is to give the mass an oscillatory motion in a proper way (See Figure P4.7).

SMA spring

x m FIGURE P4.7 SMA actuated spring motion.

(1) Describe how the vibratory motion can be produced. (2) Based on this principle, express roughly the equation of the motion, taking into consideration the material hysteresis effect. P4.8. Figure P4.8 shows the use of a Wheatstone bridge to measure strain. When the deformation of the strain gauge occurs, it will cause the gauge resistance to change, DR: If the bridge has equal resistance R, show that Vout is determined by Vout ¼ ðVS Rf =RÞðDR=2Þ: In the figure Vs is the supply voltage and Rf is the feedback resistance of the amplifier. P4.9. A galvanometer recorder utilizes a UV light as a light source, a mirror, and a moving strip of photosensitive paper. The mirror is attached to a moving coil assembly which is suspended by a torsional string and a viscous damper. The coil is moving in a magnetic field under the same principle as that of the electric DC motor. While the paper is moving in the x direction, the mirror reflects an incident beam to a point Pðx; yÞ on the paper as depicted in Figure P4.9.


249

VS Rf R1

R2 i1

i2

i3

i4

R+∆R

i5

i6 na Vout

R4

ng Rf

strain gauge

R+∆R

FIGURE P4.8 Bridge arrangement with a differential amplifier for strain measurement.

(1) If the coil assembly has a moment of inertia I, and is suspended by a torsional spring, kt , and a viscous damper, b, write down the equation governing the mirror motion u: Assume that the magnetic flux density is B, n is the number of coil turns, A is the cross sectional area of the magnet, and i is the current flowing through the coil.

rotational damper

b

θ

mirror

photo sensitive paper

magnetic field B

magnet N tortional spring

P (x,y)

moving

S

i kt

x y S UV light source

FIGURE P4.9 Strip chart-galvanometer recorder.

250

Optomechatronics

(2) Suppose that the mirror is rotated by u from an initial angle uð0Þ ¼ 0, with zero angular velocity du=dtð0Þ ¼ 0: Write down the transfer function between the current iðtÞ and uðtÞ: Discuss the mirror motion in terms of natural frequency and damping ratio of the mirror system. (3) If the mirror is rotated by small amount Du from its initial angular position, what will be the relationship between y and Du ? Assume that the initial angle of incident beam is at u ¼ ui with respect to some reference angle. P4.10. A closed loop control system is shown in Figure P4.10. For control purposes, the microprocessor takes in the information obtained by a sensor whose signal is contaminated by noise as shown in the figure. Then the signal is sampled once within a period Dts and based upon this sampled signal, the controller generates a command signal to an actuator which drives the system. (1) Suppose that the signal of the output variable has a frequency range 10 Hz # fs # 50 Hz, and noise frequency ranges fn $ 60 Hz: What type of filter can be used to reduce noise? What kind of considerations should be given to design such a filter? (2) To ensure a good performance of the control system, what frequency of the control action is recommended? Assume that the control action ð fc Þ is carried out at least twice within one sampling frequency. P4.11. Consider a mirror driven by a gap closing actuator supported by a spring whose stiffness is k as depicted in Figure P4.11. When m is total mass of the mirror unit, z is the displacement from the equilibrium state in downward direction, V is the applied voltage to the actuator, and z0 is the original gap, the equation of motion of the mirror is the same as given Equation 4.51 in the text. (1) Discuss the behavior of the mirror motion depending upon the applied voltage. (2) Describe the behavior of the mirror motion when the mirror approaches the lower plate. (3) Show a block diagram of a feedback control system to maintain the mirror position at a desired position, and explain its control concept. P4.12. Consider the piezoelectric sensor treated in Problem P4.3. If the dynamics of the sensor is given such that its transfer function is described by GðsÞ ¼

VðsÞ tv2n s ¼ FðsÞ ðts þ 1Þðs2 þ 2jvn þ v2n Þ

A/D

controller

measured value

FIGURE P4.10 A signal processing and control for a digital control system.

(b) output signal with noise

O

amplitude

(a) feedback system

xd

desired value

sampled signal

sensor

D/A

actuator

system

time (t )

actuation signal

x( t )

actual output variable

Mechatronic Elements for Optomechatronic Interface 251

252

Optomechatronics

mirror

m leaf spring k

v

z actuator

FIGURE P4.11 A mirror driven by an electrostatic actuator.

(1) Discuss the effect of t on the system response as t decreases from a large value to a very small value. In the case where t # 1, what will be the type of response characteristics? (2) Plot the response of the sensor system to a unit step input of force FðsÞ ¼ 1=s, using the Simulink model in Matlab. The parameters are given in the followings: vn ¼ 1:6 £ 105 rad=sec, j ¼ 0:01, t ¼ 2:0 msec: (3) Determine the steady state error for (a) a unit step input force and (b) a unit ramp input force.

References [1]

Ando, T. Laser beam scanner for uniform halftones, Printing Technologies for Images, Gray Scales and Color, SPIE, Vol. 1458. 1991. [2] Auslander, D.M. and Kempf, C.J. Mechatronics — Mechanical System Interfacing. Prentice Hall, Englewood Cliffs, 1996. [3] Benech, P., Chamberod, E. and Monllor, C. Acceleration measurement using PVDF, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 43: 5, 838– 843, 1996. [4] Bentley, J.P. Principles of Measurement Systems, 4th ed. Prentice Hall, Englewood Cliffs, 2004. [5] Bolton, W. Mechatronics Electronic Control Systems and in Mechanical Engineering. Addison Wesley/Longman, Reading MA, London, 1999. [6] Bradley, D.A., Dawson, D., Burd, N.C. and Loader, A.J. Mechatronics. Chapman and Hall, London, 1991. [7] Dally, J.W., Riley, W.F., and McConnell, K.G. Instrumentation for Engineering Measurements. John Wiley and Sons Inc, New York, 1984. [8] Fraden, J. Handbook of Modern Sensor. American Institute of Physics, New York, 1993. [9] Fukuda, T., Hattori, S., Arai, F. and Matsuara, H. Optical servo systems using bimorph PLZT actuators, The American Society of Mechanical Engineering (ASME), 46, 13 – 19, 1993.

Mechatronic Elements for Optomechatronic Interface [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20]

[21] [22]

253

Fukuda, T. Performance improvement of optical actuator by double side irradiation, IEEE Transactions on Industrial Electronics, 42, 455– 461, 5 October, 1995. Gweon, D.G., Special Topics in Dynamics and Control, Lecture Note, MAE850, Korean Advanced Institute of Science and Technology, 2004. Kenjo, T. and Sugawara, A. Stepping Motors and Their Microprocessor Controls, 2nd ed. Oxford Science Publications, Oxford, 1995. Lin, R., Shape Memory Alloys and Their Application, http://www.standford. edu/, richlin1/sma/sma.html, 2005. Luxon, J.T., Parker, D.E., 2nd ed., Industrial Lasers and Applications. Prentice Hall, Englewood Cliffs, 1992. Mahalik, N.P. Mechatronics: Principles, Concepts and Applications. McGraw-Hill Inc, New York, 2003. Near, C.D., et al. Sensor Technology Limited (Product Information), SPIE, Vol. 1916. pp.396 – 404, Oxford, 1993. Pluta, M.. Advanced light Microscopy Principles and Properties. Elsevier, Amsterdam, 1988. Sashida, T. and Kenjo, T. An Introduction to Ultrasonic Motors. Oxford Science Publications, 1993. Senturia, S.D. Microsystem Design. Kluwer Academic Publishers, Dordrecht, The Netherlands, 2001. Shen, Y., et al. A high sensitivity force sensor for microassembly: design and experiments advanced intelligent mechatronics, Proceedings of IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Kobe, Japan, Vol. 2. 2003, pp. 703– 708. Stadler, W. Analytical Robotics and Mechatronics. McGraw-Hill Inc, New York, 1995. Tabib-Azar, M. Microactuators. Kluwer Academic Publisher, Dordrecht, 1998.

5 Optomechatronic Integration CONTENTS Basic Considerations for Integration .............................................................. 256 Basic Functional Modules ................................................................................ 263 Signal Transformation ............................................................................... 263 Signal Manipulation .................................................................................. 264 Signal Sensing............................................................................................. 266 Signal Actuation ......................................................................................... 269 Signal Transmission ................................................................................... 270 Signal Display............................................................................................. 272 Optomechatronic Interface or Integration ..................................................... 273 Basic Two-Signal Integration.................................................................... 273 Fundamental Optomechatronic Integration: Illustrations................... 277 Generic Forms for Optomechatronic Interface ..................................... 279 Integrability................................................................................................. 288 Signal Flow.................................................................................................. 289 Integration-Generated Functionalities............................................................ 291 Problems.............................................................................................................. 294 References ........................................................................................................... 297

In the previous chapters we have reviewed some of the fundamentals of the two engineering fields, optical and mechatronic engineering. During the process of the review, we have seen that basic principles and concepts, and their relevant theories and tools, can be combined to interact with each other to create new concepts or technical elements. In fact, we have seen from the illustrations discussed in Chapter 1 that a variety of the integrated components and systems result from those interactions among optical, mechanical, and electrical technologies. Further, we have seen that such integrations can be derivable from certain principles of physical interaction. Judging from the observations, we can summarize the following characteristic points:

255

256

Optomechatronics

Hierarchical structure of integration: Optomechatronic integration is hierarchically composed of several levels of integration, so its architecture is composed of functional coordination level, structural integration level, and organization level. Fundamental functionalities: Functionality is a task-oriented functional entity required for a system to execute a given task specified by the designer. Such a functionality may be produced by combining several fundamental functional modules. Basic functional modules: The functional modules are the functionoriented modules required to produce a functionality in a specified manner. These functional modules are combined together to create the basic functionalities required by optomechatronic integration. Integration interface: There are certain laws for putting the functional modules together. The integration must consider the interfacing mechanism between the optical, mechanical, and electrical components. At the same time, it must take into consideration the signal flow that represents physical phenomena that occur during the interaction involved with the integration. In this chapter, we will have a more detailed look into physical optomechatronic components and systems, in order to investigate their characteristics regarding the composition of their structure and the fundamental functionalities. To this end, we will consider the basic functional modules that create such functionalities, and study the signal flow in order to facilitate the composition of the modules needed for functionality generation. Here, signal is meant in the broad sense by energy or information. Based upon the analysis of the functional modules, we will discuss the nature of integration in order to find common modalities that might exist among various integrations of the three engineering fields. From these findings, we finally derive the optomechatronic integration and analyze the characteristics of the integration, which include integrability, integration structure, and information signal flow. To better understand the underlying concept, we will illustrate a variety of examples of physical systems from the view point of integration.

Basic Considerations for Integration Looking into the details of optomechatronic technology from this viewpoint, we can see that it involves interaction among three engineering fields, optical, mechanical, and electrical. The interaction between the three signals will differ, depending upon the type of the integration and the strength of contribution of each technical element. The dependency will, therefore, determine the nature of optomechatronic integration. If we can formulate the relationship, the type, and strength of the integration vs. the characteristics

Optomechatronic Integration

257

of the integration, developing integration methodology will become easy for a given set of problems. However, as we might anticipate, it is an extremely difficult task to formulate and analyze the relation even with a heuristic manner. Nevertheless, we will make an effort to dig out the underlying natures of the integration by introducing basic functional modules that can be the basis for the integration, and the analysis of the structure of the integration. As listed in Table 5.1, three engineering fields (O, E, M) have distinct physical variables originating from their own physical phenomena. Integration of these fields implies that these unique, distinct variables are interacting with each other to affect some of the interacting variables, or create new physical variables. As indicated in Figure 5.1, the unique optical variables include light energy, ray intensity, and radiation flux, the mechanical variables include mechanical energy, motion, deformation, strain, and fluid/heat flow, and electrical variables include current, voltage, charge, and magnetic flux. We will represent these unique variables in the following manner: mechanical variables as a mechanical signal, optical

TABLE 5.1 Signals and Basic Elements of Mechanical, Electrical, and Optical Engineering Mechanical Force ( f ) Velocity (v) Power ( p) (mechanical) Energy (E)

Electrical

Optical

Signal Current (i) Voltage (v) Power ( p) (electrical) Energy (E)

Ray (r) Intensity (I ) Irradiance (R) (radiant, luminous) Radiant energy (E )

Basic elements Mass (m) m

Capacitor (c)

Prism

Spring (k)

Coil (L)

Beam splitter

Damping (b)

Resistor (R)

Lens

Gear

Transformer

Cam

Alternator

Belt

Inductor

Bellows

Semiconductor p

Mirror N

S

Stop Aperture

n

Gratings

258

Optomechatronics

energy, ray, intensity, radiation flux ti

na

l

op

ca

sig

l

ical chan me

energy, current, voltage, magnetic flux charge

al ctric ele

signal

energy, motion, strain, fluid/heat flow

signal

FIGURE 5.1 Optical, mechanical, and electrical signals.

variables as an optical signal, and electrical variables as an electrical signal. The signals are expressed in the following form: optical signal ¼ O (energy, ray, intensity, radiation flux…) mechanical signal ¼ M (energy, motion, fluid flow deformation, strain…) electrical signal ¼ E (energy, current, voltage, charge, magnetic flux…) The interaction can occur with various modalities, depending largely upon: (1) which variables are interacting, (2) how many variables are interacting, (3) how and when they are interacting, and (4) finally, the sequence and the duration of the interaction. It will be a formidable, or even an impossible, job to formularize the integration modalities. Rather we will try to find a relevant integration concept, taking into consideration the fundamental questions mentioned above. To start with, we will assume that all three signals contain six or fewer fundamental functional modules, as shown in Figure 5.2. These elements comprise transformation, manipulation, sensing, actuation, transmission, storage, and display of a signal. Let us discuss them one by one, in detail, to identify their characteristics. Signal transformation is the conversion of one signal form into another as a result of a change of physical elements. Signal manipulation is the action of diverting or modifying a signal. Sensing is the measurement of the state of a signal. Signal actuation is the control of a signal in order to maintain it at a certain value. Signal transmission is the action of transmitting a signal to a desired location. Finally, signal display is the presentation of a signal in a presentation element. Optomechatronic integration may often involve the combination of multiples of these functional modules, but, in a simple example, may


259

FIGURE 5.2 Basic technological elements for optomechatronic technology.

involve only one single module. For instance, the optical disk shown in Figure 1.6c has several physical components that carry out their own tasks: an optically-encoded rotating disk, a tracking unit that searches a certain data location, a track following unit, and auto focusing and reading units. The rotating disk is a signal storage device that has a signal storage module (optically recorded element), and a signal transformation module (motor) that makes the disk rotate by an electric motor at a specified rotation speed. The tracking unit has a signal transforming component (moving optical head) and a sensing module (optical element) that measures the data track. The auto-focusing and reading unit has a signal control module and a sensing module as well. All of these modules involved here represent optomechatronic integration. Taking another example, the optical switching shown in Figure 1.6f has one functional module, “signal modulation,” which is achieved by optomechatronic integration composed of optical elements (light, mirror) and a mechatronic element (actuator) that actuates the mirror. From these two examples, it may be said that the optomechatronic integration, indeed, is necessary, not only for combining functional modules, but also for making each individual technical component work as desired. The principle of the optomechatronic integration concept departs from this concept. Figure 5.3 depicts a standard procedure which indicates the integration processes involved. According to the procedure, once the design specification is given, the required functionalities are to be checked first, in order for the desired tasks to be carried out by the system or device to be designed. The next thing to do is to identify all the functional modules that might be involved in producing the desired functionalities. Since all those

260

Optomechatronics

design specification

create the functionality

identify the functional modules

integration of optomechatronic components

system integration

verify the performance good

no

yes end FIGURE 5.3 The procedure of optomechatronic integration.

modules may not be feasible for the intended integration, optomechatronic integrability must be checked to investigate its feasibility. After the integrability is confirmed, then optical, mechanical, and electrical hardware elements are to be integrated, so as to meet various requirements set by each individual module or the combined ones. The final stage is to verify the performance of the integration. The steps for optomechatronic design are similar to those standard procedures for designing most engineered products or systems. However, there are several differences. In creating the functionalities, the optomechatronic designer can consider adding new functionalities that may not be realized by mechatronic design alone. Taking a simple example, when we wish to machine a precision mechanical part as specified, we may need to accurately measure its dimensions and surface roughness during the machining operation. This, however, may not be an easy task with the current mechatronic sensor technology. By utilizing optomechatronic concepts, this task becomes feasible by building a non-contacting optical sensor that can work with a rotating and translating machining element (mechatronic element). This implies that, in effect, the optical sensing device creates a new functionality, “optical feedback control,” for the machine


261

system. When the optical device is combined with a control unit, the combined system constitutes a complete optical-based feedback control system, which drastically improves the quality of the machined part. In the stage of identifying which modules are necessary to achieve the required functionalities, there will be some difference between the two design methods. A simple difference is that the use of an optical signal may not require signal manipulation or processing (e.g., filtering), while an electrical signal may. However, the major difference comes at the stage of the integration. As discussed earlier, the optomechatronic approach strives to find the aggregated elements in order to achieve the required functionalities more effectively than the mechatronic approach alone can do. Let us consider a typical optomechatronic system, which is an opticalbased heat-treatment of the surface of a mechanical part, as shown in Figure 5.4. In the heat treatment, a moving mechanism (translating, or rotating, or a combination of both) for either a material or optical element is needed to cover the whole area to be heat treated — if we assume that the material is not heated all at once over the whole area. Under this configuration, there may be five divided functionalities involved to achieve a desired quality of heat treatment, although they will not be explained in detail here. Those functionalities are light beam generation for heating the material, control of the actuator for precise light intensity, control of the stage motion in a desired manner (moving range, moving sequence), sensing or monitoring the heat-treated state of the material, and feedback control of this information to regulate the heat-treated state in a desired manner. In order to create these functionalities, we may consider the relevant functional modules, as illustrated in Figure 5.5. To produce an appropriately segmented mirror

mirror

4 kW CW CO2 Laser resonator laser beam lens

sensor fixture IR sensor

coated specimen A

c: >exp now controlled...

measurement & control hardened track XY-table

FIGURE 5.4 Laser surface hardening process.

262

Optomechatronics

light source generation

moving the material or optical head

sensing/control of the heat treated state

FIGURE 5.5 Identifying the functional modules necessary for a heat treatment process.

conditioned optical signal with required power, we first need “signal transformation” that produces an optical signal generated by an electrical signal, optical signal modulation that converts the light beam into a desired form, and optical light control that focuses the beam in order to not only obtain high quality of the heat-treated workpiece, but also to prevent the waste of optical power needed for heat treatment. Note that optical beam control needs its own functional modules to carry out processing, manipulation, and control of the beam injected into the material surface. Another important component of this system is the moving mechanism. Because the moving element should be actuated and positioned as accurately as possible in a desirable manner, it needs a precision electricalto-mechanical actuator, a signal transformation module (which needs signal control), and sensing modules to produce accurate motion of either the material or optical head. Signal sensing and control of the treated material state, therefore, are the important functions of the system that affect treatment quality. The signal sensing has its own signal transformation part that can measure state variables, such as the treated quality, and the dimension of the treated zone. The control of the treated material state requires two signals to be regulated; one is the light power, and the other is the velocity of the moving element. In regulating these variables, the signal sensing part needs two individual sensing devices. The first is the power measurement sensor, and the second is the velocity measurement sensor. It is noted that feedback control will not be classified as a basic functional


263

module, but will be identified as a functionality having two functional modules; actuation and sensing.

Basic Functional Modules Signal Transformation In Chapter 1, we observed that a number of systems have signal or energy transformation from one form to another among optical, mechanical and electrical signals. For instance, a digital camera has three important components: CCD cell, auto-zooming by an electric servomotor, and autofocusing by an ultrasonic motor. The CCD cell transduces light information into an electrical signal as a sensor, the ultrasonic motor transduces electrical signal into a mechanical signal as an actuator (as the electric motor does). In actual optical, mechanical, and electrical systems, there are many such signal transformers that convert one signal to another, depending upon the causality between input and output. Table 5.2 configures such transforming elements that convert from one signal form to another. In this table, some of the elements transforming the same kind of signals are not included — for example, optical – optical (OO), electrical –electrical (EE), and mechanical– mechanical (MM) — although they are also important for optomechatronic integration. The first transduction type, optical-to-electrical, denoted by the symbol ðTEO Þ, produces electricity from optical input, which can be derived from several phenomena, such as the photovoltaic phenomenon. The second type, TABLE 5.2 Basic Signal Transformations Transformation

Phenomenon

Symbol

Signal flow

Typical Device

Photovoltaic, pyroelectric, photo emissive Emission of photon

TEO

O ! TEO ! E

Photo diode

E TO

E E ! TO !O

Optical-to-mechanical

Photovoltaic phase transformation

O TM

O O ! TM !M

Mechanical-to-optical

Triboluminiscent

M TO

M M ! TO !O

Light emitting diode Optical actuator, shape memory actuator

Electrical-to-mechanical

Electromagnetism, piezo electric Induction, piezo electric

E TM

E E ! TM !M

TEM

E M ! TM !E

Optical-to-electrical

Electrical-to-optical

Mechanical-to-electrical

Electric motor, piezo actuator Electric generator, piezo sensor

264

Optomechatronics

E Þ; is the reverse case of the first. It produces light from electrical-to-optical ðTO electric input, and is popularly used for either light source or signal conversion for transmission. The third category of the transformation, which O Þ; yields mechanical signals out of optical is optical-to-mechanical ðTM signals. This conversion can be found from optical actuators which utilize some physical phenomena, such as inverse piezo-electric and, material M phase transformation. The fourth one, ðTO Þ mechanical-to-optical, can be found from smart luminiscent materials that generate light when they are E Þ is commonly used in most of subject to stress or friction. The fifth one ðTM the mechatronic devices, including motion generating devices such as electric motors and piezo actuators. The last transformation type ðTEM Þ produces electrical signals that can be observed from mechanical signals from phenomena such as electric induction and piezo electricity. At this point, we can now consider a variety of signal transformation methods by combining some of the basic forms presented in the table. We notice that a number of signal transformations can be obtained as the result of such integration. For instance, suppose that we wish to combine the first, E : The integration then yields: TEO and the fifth, TM

optical ! TEO !electrical ! TME !mechanical which indicates that the optical input signal is transformed into the mechanical output signal. Mathematically, this can be expressed by: E O TEO þ TM ! TM

ð5:1Þ

An important observation we can make from this Equation is that, no matter what type of the intermediate transformation is involved (e.g., electrical signal), the final result is written by the relation between input signal (optical) and output signal (mechanical). Avariety of cases that can illustrate this mode can be found from practical systems. One such example is to adjust an optical system based upon the measured information by a photo detector. Signal Manipulation In electrical signal processing, signal manipulation often needs to be converted into another form while retaining its original information, depending upon the application. The manipulation of optical beams is a process which a number of applications must utilize in order to obtain targeted outputs needed for processing transmission, sensing, or actuating communication via an optical signal. Steering, switching, alteration of amplitude and phase, change of wave form, filtering within a certain frequency band, and noise reduction of optical signals are some of the examples that belong to optical beam manipulation. In this subsection, we will discuss the nature of beam manipulation from the viewpoint of the interaction between optical and mechatronic elements, by dividing the manipulation into two major types, beam modulation and scanning.


265

In Table 5.3, various manipulation methods are summarized in the views of the basic operation principle. Almost all methods are operated based on optical manipulation. Mme oo , indicated in the table, denotes the manipulation of the optical beam by both mechanical and electrical elements. The “me” in the bottom of the symbol implies that manipulation is conducted by electrically driving the mechatronic elements. A variety of cases that can illustrate this mode can be found from practical systems. One such example is manipulation of an optical system by diverting its optical signal by means of a mechatronic element. Mm oo and Me oo imply that the manipulation is made by purely mechanical and electrical means, respectively. Let us consider two important methods that belong to signal manipulation in the sequel. (1) Optical signal modulation: In the optical engineering field, the modulation method is also common practice in order to convert an optical signal into a form suitable for certain applications. The major need for optical modulation comes when we want to: (1) (2) (3) (4)

control the intensity and phase of a light beam. control the frequency and wave form of a light beam. impress information on to a carrier signal. have noise reduction.

The technical fields that frequently need modulation can be found among sensing, actuation, control, communication, display, and so on. The modulation can be made primarily by choppers, acousto-opto modulators, electrooptical modulators, photo-elastic modulators, and spatial light modulators. The mechanical optical chopper is typical of optomechatronic choppers,

TABLE 5.3 Types of Signal Manipulation Optical !

o

Mme o

! Optical

Optical !

o

Me o

! Optical

Manipulation type

Basic principle

Acousto-optical modulator Mechanical chopper Photo-elastic modulator Electro-optical modulator Mechanical beam scanner Acousto-optic scanner Electro-optical scanner Mechanical signal manipulation

Bragg deflection Beam chopping by blade Changes in refraction index Changes in refraction index Reflection Bragg deflection Reflection Electromechanical motion

Symbol Mme oo Mme oo Mme oo Me oo Mme oo Mme oo Me oo Me m m

Mme oo optical modulation by mechatronic principle, Me oo optical modulation by electrical method, Me m m mechanical signal manipulation by electrical means.

266

Optomechatronics

as illustrated in Figure 5.6a). In the figure, the optical beam (optical element) incorporates with the motion of a mechatronic unit (rotating chopper blade), and thereby provides a modulated beam as desired. (2) Optical scanning: Scanning is necessary when a light beam or field of view needs to be directed to a certain point or area of an optical field. To achieve scanning, there are two methods, reflective and refractive. In Figures 5.6b,c, these are shown schematically to illustrate the interaction of an optical element (beam) with a mechatronic unit (rotating mirror or prism). In the reflective method, mirrors are utilized, while lens and prisms are used in the refractive method. In terms of scan pattern, there are two classes of the method — fixed pattern or random access. All of these methodologies involved with scanning affect the accuracy and time of scanning. There are three optical scanners most popularly used for practical applications: mechanical scanners, acoustooptical scanners, and electro-optical scanners. Upon examination of the signal manipulation methods shown in the figure, we can see that all manipulations are of optomechatronic nature. Signal Sensing The sensing module is the element that measures physical variables of the process or system being measured. In general, sensing modules operate with two different methods when classified, depending upon which modules are involved. One utilizes signal transformation, while the other uses signal modulation. In other words, sensing can be made by using either signal transformation or signal modulation, as listed in Table 5.2 and Table 5.3, respectively. Therefore, there may be a variety of sensing types classified into several different categories, depending upon the input –output signal form, and the physical phenomena used for sensing.

ψ

ω mirror

incident beam

ω

prism ω

incident beam light beam

reflected beam

refracted beam

(c) refractive scanning

2ω

(a) optical chopper

(b) reflective scanning

FIGURE 5.6 Optical scanning by optomechatronic principle.

(d) scan pattern


267

Basically, all the signal transformation and manipulation modes can be used for a sensing element. Combinations of each individual module are also feasible for sensing. The problem is how effectively and accurately each form of sensing can be made. The sensing modules employing the transformation modules TEO , TEM and the modulation modules Mm oo , Me oo are popularly used in actual practice. The sensing module ToM can be used to measure a mechanical signal in terms of an optical signal. For example, when some structures are in fracture or mechanical strain, they produce optical signal. This phenomenon is called “triboluminiscience” as shown in the left hand side of Figure 5.7a. Another luminiscience that belongs to this transformation module is photo luminiscience as shown in the right hand side of Figure 5.7a. The figure shows a portion of a structural surface coated with pressure sensitive paint, which is essentially photoluminescence material. When this structure surface is stressed or strained, the intensity of light reflecting from the surface is varied, depending upon the pressure applied to E seems to be rare, but in the past has been the surface. The sensing module TM actively used in the control of mechatronic devices, or systems such as pneumatic or hydraulic systems.

TABLE 5.4 Signal Sensing and the Related Input – Output Variables Sensing Mode Optical !

TEO

Mechanical !

Electrical !

Phenomenon

! electrical

Photovoltaic, photo emissive

Photo diode storage tube

TEM

Deflection thermo effect, piezoelectric

Strain gauge, piezo sensor, pyro sensor

Mechanical motion

Fly ball governor

TME

! electrical

! mechanical

o

Optical ! Mm o ! optical o

Optical ! Me o ! optical e

Electrical ! Mo e ! electrical Optical !

TEO

Mechanical !

Typical Device

!

TEM

TOE !

! optical

TOE

! optical

Phase, intensity, wave, spectrum

—

—

Fabry-Perot sensor

—

—

Photo emissive —

Image intensifier —

Some other transformation and manipulation modules and their combinations are feasible for sensing, which are not shown here due to space limitation.

268

Optomechatronics

Table 5.4 lists various modules of signal sensing whose principles are originated from signal transformation and modulation. Here, a sensing module measuring the same kind of signal is excluded, as was the case of the signal transformation. As far as transformation type sensing modules are concerned, there are six different types of sensing, whose symbols are identical to those of signal transformation. Among these the sensing E O not shown in the table may become an effective means of modules TO , TM sensing as opposed to the others TEO and TEM , when sensing involves special environments that require such conditions as being inflammable, being contamination-free to noise, and needing fast direct feedback. For instance, if a sensing environment does not permit the sensing module TEO to measure O may be a substitute for the measurement. In this the optical signal, then, TM M case, the module TE should also be replaced by the module Mm oo , which can measure a mechanical signal by modulation of the optical signal. A two-mode sensor module is also feasible by combining two transformation modules together, which was the case discussed in Table 5.4. One such combined sensor module can be seen in an image intensifier, which E as shown in Table 5.4. Figure 5.7b has the signal flow denoted by TEO ! TO illustrates the concept of a typical image intensifier. The intensifier takes a scene with dim incident light and produces a visible image. It consists of an emissive photo cathode surface which causes electrons to be emitted when receiving photons on to a surface, and a phosphor screen to receive the amplified image. In more detail, when photons produced by a dimly lit scene are incident on the cathode surface, the cathode causes electrons to be

λi

λe

emitted light

oxygen-sensitive probe molecules

fracture mechanical loading

structure

base coationg

structure

(a) single-mode transformation sensor photo cathode

induced surface corrugation phosphor screen

structure

photorefractive material light beam 1

collecting lens

(b) two-mode sensor

o

(c) optical actuator ; T M

FIGURE 5.7 Sensors and an actuator employing signal transformation.

light beam 2


269

emitted. Under a certain voltage application, the electrostatic lens then focuses the generated electrons on to a phosphor screen. These photo electrons can excite a phosphor on the screen and intensify the illumination of the image. Therefore, the signal flow occurring during the transformation process that produces this image amplification can be written by: E O ! TEO ! TO !O

The other form of a two-mode sensor module is the case of signal flow E : This module senses a mechanical motion by electrical means, TEM ! TO but converts the sensed electrical signal to the corresponding optical signal: E M ! TEM ! TO !O

ð5:2Þ

In this manner, some other multimode sensor modules can also be produced, with even more than two transformation modules. A vast number of sensors employ the modulation method. Here, we list only three kinds: mechanical modulation of optical signal, which is denoted by Mm oo , electrical modulation of optical signal denoted by Me oo, and optical modulation of electrical signal, denoted by Mo ee : The module Mm oo uses the modulation of phase, intensity, wave and spectrum of optical signal that may be caused by mechanical motion. Especially, optical fiber sensors which are popularly used today in various engineering fields adopt this concept, as we shall see later. The sensing module Me oo can be found from the sensors that utilize the interaction of the optical signal with the electro magnetic field. The sensing module, Mo ee is the reverse case of the above modulator, and utilizes modulation of electrical signals by means of optical signals. Signal Actuation Actuating a signal to an arbitrary state is the most common form of the control technology, as we have seen already. As it might be confusing, a signal here means the physical variable to be actuated, and comprises three different types (optical, mechanical and electrical), as we may recall from the previous discussion. Signal actuation is defined here by “actuation of mechanical signal,” to discriminate this type from other types of transformation and modulation modules. O A number of signal actuators employing TM have been developed. One such actuator is shown in Figure 5.7c. The profile of a structural surface that may be altered optically is shown. This optically induced surface deformation employs the principle that photosensitive materials induce stresses and strains when they are exposed to light beam. As shown in the figure two light beams interfere and form a periodic change in the refractive index in a film of a photosensitive material. Due to the changes in the index periodic deformation on the surface of the material results in, whose amplitude is within nm range. This periodic surface corrugations give rise to actuation and can be controllable by the optical interferometric light pattern.

270

Optomechatronics

E Þ is the most commonly-used The electrical-to-mechanical module ðTM actuation mode that actuates mechanical signal by electrical signal. On the other hand, the optical-to-mechanical module is the actuation of a mechanical signal by an optical signal. According to this definition, we can consider two important actuation modules of the transformation type; O E optical-to-mechanical TM and electrical-to-mechanical, TM : When it is necessary for signal actuation to become a remote operation, it involves transmission of a signal, either an optical or electrical. But when sending an electrical signal to a remote site may be not desirable due to noise contamination or a safety problem, an optical means may be a better solution for the transmission. Let us consider the case when an actuator to be operated is located at a remote site. The computer-generated electrical signal is converted into an optical signal, which is then transmitted in an optical signal form. The actuation module at the remote site is involved with the integration of the transformation of the second, the first, and the fifth modes listed in Table 5.2. The resulting transformation mode yields the following mathematical form: E O E E TO þ TRO O þ TE þ TM ! TM

ð5:3Þ

where transmission mode TRO O is included to express signal transmission explicitly. This equation implies that the final transformation mode is operated in the form of the electrical-to-mechanical mode, which is the actuation module of transformation type. A typical example system of this case is an optically operated valve control of a pneumatic system located in a remote site. An electrical signal corresponding to a desired value of the pneumatic valve position is converted into the corresponding optical signal by a LED and is sent to a remote site. This transmitted optical signal is then converted into an electrical signal, which in turn, operates positioning of the servo valve. Depending upon the valve position, the pneumatic servo system is operated. We will treat this in more detail at the end of this chapter. Signal Transmission Signal transmission is a basic form of transporting information or data to or from one location or another. The distance between two locations may be very short, like the length within MEMs parts, short like that of small sensors, or very long like that of communication systems. It is mainly transmitted by means of optical or electrical signal modulation. The transmission should be immune from external noise, exhibit low attenuation, and safe from hazardous interference. For this reason, although application dependent, optical transmission has replaced many of the application areas dominated by electrical transmission. Table 5.5 shows three types of signal transmission, electrical through optical to electrical, optical to


271

TABLE 5.5 Types of Signal Transmission Source Signal

Signal Transmission

Signal Flow

Electrical Optical Optical

Operation site ! remote site E Electrical ! optical–optical E ! Me ee ! TO ! TRO O !O o Optical–optical O ! Mme o ! O ! TRO O !O O Optical–mechanical O ! Mme oo ! O ! TRO O ! TM ! M

Electrical Optical Mechanical

Remote site ! operation site E Electrical ! optical–optical E ! Me ee ! TO ! TRO O !O E Optical ! optical O ! TO ! TRO ! O O E Mechanical –optical M ! TEM ! TO ! TRO O !O

optical, and optical to mechanical. The first transmission case is useful when remote operation of mechatronic devices or systems is needed. A simplified optical data transmission system by optical signal is depicted in Figure 5.8. It consists of three major parts, transmitter, optical fiber, and receiver. The transmitter contains a laser or an LED, and appropriate control circuitry: The laser and LED here is a signal transformer that converts an electrical to an optical signal. In a case where the data source is optical, transmitting site

receiving site

laser diode

modulator

optical fiber

optical

(a) electrical-to-optical transmission transmitting site modulator

receiving site optical fiber

optical

optical switch

(b) optical-to-optical transmission transmitting site

receiving site optical

modulator optical switch

optical fiber

(c) optical-to-mechanical transmission FIGURE 5.8 Optical transmission configuration.

mechanical elements

mechanical

272

Optomechatronics

the transforming unit is not necessary and the signal can be sent directly to the input optical fiber unit. When a signal from the signal source is fed into the transmitter, the signal is modulated by the transmitter. Then, the modulated light is launched into the fiber and transmitted to the receiving unit that retrieves the transported information or data. A light detector such as a photo diode (PD) takes the role of receiving signals. The received signal usually goes through a signal processing such as amplification and noise filtering. From the discussion above, it can be seen that in the remote operation shown in Figure 5.8a, signal is transformed from electrical to optical, modulated, transmitted, and transformed back to electrical at the receiving site, and then properly processed and conditioned according to applications. In terms of signal mathematics, the transmission procedure can be written: E E ! Me ee ! TO ! TRO O!O

ð5:4Þ

When a control operation at the sender site is needed, the resulting signal at the remote site is transmitted back to the sender side. In this case, the form of signal mathematics is propagated in a reverse direction as: E e O ˆ TRO O ˆ TO ˆ M e e ˆ E

ð5:5Þ

The second type is a direct optical-to-optical transmission as shown in Figure 5.8b. The input to the fiber is an optical signal modulated by an optical switch Mme oo , and the output signal is the optical signal type, which can be used for a variety of applications. It is noted that some means of providing an optical signal to the operation site is necessary. The signal flow mathematics for this case is expressed by: O ! M meoo ! O ! TRO O!O

ð5:6Þ

The third type is optical-to-mechanical, as indicated in Figure 5.8c. In this case, the transmitted optical signal is directly interacted with mechanical elements at the optic – mechanical interface, thus producing a mechanical signal in a desired manner. The signal flow mathematics is described by: O O ! M meee ! O ! TRO O ! TM ! M

ð5:7Þ

This type is a seemingly unachievable transmission type, but can be found from a practical example where the transmitted optical signal operates on a fluid nozzle which controls fluid flow. This alters a deflection of the jet from the center position. Some of the details will be discussed in the last section of this Chapter. Signal Display The display element is usually the final data representation element. A simple one is a bathroom scale that contains a typical signal display element of the LCD type. There are a number of data presentation devices, from simple pointer scales and recorders to very large scale display devices such


273

small-scale alphanumeric

light-emitting diodes (LED)

liquid crystal display (LCD)

large-scale displays (electronic display)

liquid crystal cathod-ray display (LCD) Tube (CRT)

image MEMs based gas plasma discharge intensifier projector

plasma display panel

field emission display

electrolume scene

light emitting diode

FIGURE 5.9 Classification of optical data presentation devices.

TABLE 5.6 Various Signal Display Modes Display Mode

Display Principle

Typical Device

E M ! TEM ! TO !O E E ! TEE ! TO !O E O ! TEO ! TO !O o O ! Mme o ! O O ! Me oo ! O

Photoemissive Photosemissive Photosemissive Optical switching Polarization

Weight indicator (scale) Cathode ray tube, LED, PDP Image intensifier, image converter Digital micromirror, grating light valve (GCV) Liquid crystal display (LCD)

as cathode-ray-tubes (CRT) and plasma display panels (PDP), as shown in Figure 5.9. Since brief discussions on the principles of some of these display units have been made in Chapter 4, we will not discuss them here. As shown in the figure, optical display units can be conveniently categorized into small-scale alphanumeric displays and large displays that all work in a digital domain. As far as the displaying principle is concerned, it is interesting to see that all of these can be grouped into three categories as shown in Table 5.6. The first method is to use a light emitting source like a light-emitting diode (LED). The second one is the polarization method which does not emit light, but uses light incoming from other sources. The last display method is to use a signal manipulation technique.

Optomechatronic Interface or Integration Basic Two-Signal Integration In the previous chapters, we have seen a variety of different types of integration. From the observations of such integrations, we realize that

274

Optomechatronics

to become an optomechatronic element requires an optical element (O), a mechanical element (M), and an electrical element (E) all to be properly integrated together while obeying certain laws. As we shall see later, not all types of the integrations made by these three elements will yield an optomechatronic nature. Integration exhibiting such a nature needs to meet certain conditions called optomechatronic integrability. Among all of the integrations, the basic types are the integration of two signal elements, namely, OE, EO, ME, EM, OM, and MO, if homogeneous integration such as EE, OO and MM are excluded. For example, the photo diode having the transformation TEO in Table 5.1 has the capability of transforming an optical to electrical signal, and does not have mechatronic elements in it. This is an optoelectronic integration that can be denoted by OE. The electric motor (ME) has the capability of transforming an electrical signal to a mechanical signal. And the optical actuator employing shape memory alloy (OM) has the capability of converting optical signals to mechanical. Similar arguments can also be applied to the rest of integrations, such as a laser diode (EO), electric generator (EM), and motion induced light generator (MO). The two signal elements can also be observed from some modulators and manipulators upon examination of the modules listed in Table 5.2 and Table 5.3. For instance, optical transmission itself does not interact with mechatronic elements in transmitting optical signals. It represents an optoelectronic element (OE). Because this two-signal element forms the basis of optomechatronic integration, let us take some more practical illustrations. Figure 5.10 depicts various forms of two-signal integrations of OE, EO, EM, ME, OM, and MO. The light emitting diode (LED) in Figure 5.10a emits electromagnetic radiation over a certain band wavelength when forward biased. This integration represents “EO,” the order of which identifies the input– output relation. As discussed in “Signal Conditioning”, in Chapter 4, a light emitting diode (LED), like an electronic diode, directs electricity flow in one direction. When electricity flows from the cathode (positive lead) to the anode (negative lead), wire encapsulated in the LED body emits light, thus releasing photons. This component has a capability of transforming an electric signal to an optical signal. By using causality relation, this phenomenon can be expressed as electric signal as an input and optical signal as an output. The photo diode shown in Figure 5.10b is a photon detector, which is a semiconductor device in which electrons can be excited from the balance band to the conduction band when incident light hits the junction surface. This integration can be represented by OE, which means that an optical signal as an input produces electrical signal as an output. The two conversion elements in the above can be symbolically represented as: laser emitting diode:E þ O ! EO photo diode:O þ E ! OE Unless specified, we will describe this as OE.


reflector

275 motion

electrode

illumination

reflector p type

negative positive

LED chip

n type

piezoelectric material

(b) photo diode

(a) light emitting diode

(c) piezoelectric transducer

motion core S

rotation

coil

arm

separation

S

motion

N

stator

(d) variable capacitance

(e) linear motion variable inductor

stator

rotor

(f) electric motor light

displacement piezoelectric material displacement

optical fiber

bevel painted on black

electrode field coil

deformation directio

terfenol-D rod

(g) magnetostrictive actuator

(h) piezoelectric actuator

diaphram

cavity

gas

light

(j) optical actuator

(i) moving fiber deflection

light

electrostatic bonding

optical fiber

shape memory alloy ( Ti Ni )

glass

light

(k) optical-based shape memory alloy

(l) optical sensor

FIGURE 5.10 Illustrations of various forms of two-signal integration.

Figure 5.10c depicts the phenomenon of piezoelectricity. When a piezoelectric material is subject to external force or pressure which is a mechanical signal, it produces electricity. Figure 5.10d shows the capacitance of an electric capacitor subjected to motion. One plate of the capacitor is made to move freely, while the other plate is fixed. Thus, depending upon the motion applied, this device achieves modulation of electric signal due to change in the capacitance. Figure 5.10e illustrates a typical arrangement of a variable inductor that converts mechanical signal to electrical signal when a coil is subject to a mechanical motion of the core.

276

Optomechatronics

The piezoelectric transducer in the above exhibits the transformation of mechanical signal to electrical signal, while the other devices modulate their signal when they are provided with external mechanical motion. From the point of view of energy conversion, they convert one energy to a different energy form, which, in this case, is mechanical-to-electrical signal conversion. However, from the view point of a functional module, we can call this “signal transformation” rather than “signal modulation.” Symbolically, the above three conversion elements are then described by: M þ E ! ME

ð5:8Þ

Again, the first element M denotes mechanical signal, while the second element E is electrical signal. The meaning of the equation implies a combination of mechanical signal generating elements and electrical signal generating elements, denoted by electromechanical or mechatronic elements. Figure 5.10f shows an electromagnetic motion generator when a coil wound element is exposed to an electromagnetic field, and, subject to the change of the field strength, the element rotates according to the applied electrical signal. The magnetostriction actuator shown in Figure 5.10g utilizes the material property that causes a material to change its length when subjected to an eletro-magnetic field. Thus, this transforms electromagnetic signal to mechanical displacement signal. Another actuator similar to this type is the piezo-electric (PZT) actuator shown in Figure 5.10h. This device transforms electrical signal input to mechanical displacement output. The above elements are produced purely by combination of mechanical and electrical elements, and can be expressed by: E þ M ! ME

ð5:9Þ

Figure 5.10i, Figure 5.10j and Figure 5.10k illustrate optical actuators that transform heat generated by lightwaves to mechanical deformation. The deforming optical fiber works on the following principle. One end of an optical fiber is bevel cut and painted black. When the light illuminates at the other end of the fiber, it exhibits a photo-thermal phenomenon. The optical actuator transforms heat to mechanical signal indirectly, because it uses the expansion of gas to generate a mechanical deformation. The cavity is filled with gas that expands when heated with a light beam. This expansion in the deflection of the diaphragm generates vertical direction. Shape memory alloys such as Ti Ni exhibit thermoelastic martensitic transformation, which yields shape-recovery characteristics upon heating. This results in the generation of mechanical motion. The signal transformation for these types of actuation devices is expressed by: O þ M ! OM

ð5:10Þ

The MO integration shown in Figure 5.10l is an optical fiber sensor configuration in which mechanical deformation provides changes in


277

crystal mirror

photodetector sound undefracted

beam

light beam

diffracted piezoelectric actuation material

beam

(a) mechanical

(b) acousto-optical modulator

scanner

light

actuator motion

photo detector laser artifact

angular displacement light source

(c) optical chopper

deformable glass mirror

wave

PZT actuator stack electric leads

(e) grating sensor

laser

(d) optical encoder

(f) deformable mirror

atom or cell

(g) optical twizer

FIGURE 5.11 Illustrative examples of optomechatronic integration.

intensity of a light beam. This type of integration is described by: M þ O ! MO

ð5:11Þ

Fundamental Optomechatronic Integration: Illustrations As presented in Equation 5.8 through Equation 5.11, we have discussed three types of integration, including optical plus electrical (OE or EO), optical plus mechanical (OM or MO), and mechanical plus electrical (ME or EM). The OE (EO) is called optoelectronic integration. Similarly, the OM (MO) and ME (EM) are called optomechanical and mechatronic, respectively. When all of three basic elements, optical, mechanical, and electrical (O, M, E), are combined together, the combination may produce optomechatronic integration. The integration can be achieved with a variety of combinatorial types, which we have seen in Chapter 1. Figure 5.11 depicts various different types of optomechatronic integrations. The first three devices are optical modulators. It can be seen that all of these devices do not change the form of the original input signal, but retain its optical signal form. They modulate direction, wave form, and amplitude, as we shall discuss in detail in Chapter 6. Figure 5.11a is a polygon scanner that diverts a light beam in a desired direction. This scanner consists of a light source (OE) and a rotating mirror (ME) which constitutes optomechatronic integration (OPME). An acousto-opto modulator, shown in Figure 5.11b, modulates the optical signal in such a way that it diverts a light beam in a certain direction by combining light waves with sound waves generated by excitation of a piezoelectric material. It is composed of a light source (OE) and an acoustic generator (ME), resulting in optomechatronic integration (OPME). Figure 5.11c shows an optical chopper, discussed already, that modulates a light beam

278

Optomechatronics

having a wave form and frequency different from the original signal. A light source (OE) and a rotating wheel (ME) constitutes an essential element of optomechatronic integration (OPME). Summarizing all of the above integration types, the modulators can be expressed by: OE þ ME ! OPME The sensors shown in Figure 5.11d,e exhibit all optomechatronic integrations. Shown in Figure 5.11d is an optical angle encoder, which is composed of a light source (OE), a rotating coded wheel (ME), and a separate photo sensor (OE) for each track. It has alternating transparent and opaque areas patterned on the rotating disk. A grating sensor shown in Figure 5.11e also runs on the optomechatronic principle [4]. It measures an artifact topology by using controlled motion of an optical grating device (ME), through the grating of which light (OE) is diffracted differently depending on the surface topology. When the configuration in Figure 5.10l is combined with a photo sensor, the combined device becomes an optical fiber sensor based on the modulation principle. It modulates the amplitude of a light beam passing through the optical fiber (OE), which can be deformed by bimetal displacement due to temperature variation (M). This type of integration can be regarded as a signal transformation, since it converts a mechanical signal (bimetal strip displacement) into an optical signal (light beam amplitude). In summary, the integrated forms of the sensors discussed above are expressed in the following: optical encoder: OE þ ME þ OE ! OPME optical grating sensor: OE þ ME þ OE ! OPME optical fiber sensor: OE þ M ! OPME Two devices shown in Figure 5.11f,g are related to manipulation or actuation that can run on optomechatronic principles. The deformable mirror shown in Figure 5.11f is a mirror that can be locally deformable by multiple-stacked piezo actuators (ME). The state of deformation is controlled by translation of the actuators, depending on the distortion characteristics of incoming light waves (OE). The optical twizer shown in Figure 5.11g is a device that can directly manipulate atoms or nanometer objects by optical means. The motion of small nanoobjects (M) is controlled by the standing light wave generated by a laser light (OE). Therefore, this device can trap objects in the minima of the standing wave condition. From the integration viewpoint, two optical devices are expressed by: deformable mirror: OE þ ME ! OPME optical twizer: OE þ M ! OPME Figure 5.12 shows various types of optomechatronic-based manufacturing processes. Figure 5.12a is an inspection process of measuring the parts on a moving conveyor (ME) by using a CCD camera (OE). The laser welding system in Figure 5.12b is also based on the optomechatronic-based processing concept that the workpiece to be welded by the laser head (OE) is


279 laser

camera

optical module/ wave guide

illumination

welding head parts conveyor

(a) inspection of moving parts

work piece

LED

lens

6 axis stage

motor

displacement

(b) laser welding

(c) optical packaging

FIGURE 5.12 Various types of optomechatronic integration used for manufacturing processes.

controlled according to the joining path plan (ME). The laser head is equipped with an autofocusing device, which is composed of a set of lenses (O) and an actuation mechanism (ME). This device itself, therefore, exhibits optomechatronic characteristics. An optical packaging process shown in Figure 5.12c uses a laser beam (OE) to align the photo diode by using a multiaxis adjustable stage (ME). Depending upon the degree of the alignment between the laser beam and the photo diode, the amount of light intensity detected by a photo sensor (OE) becomes different. The manufacturing processes considered above have the following integration form: vision-based inspection: ME þ OE ! OPME laser welding: OE þ O þ ME þ O þ ME ! OPME alignment process: OE þ ME þ OE ! OPME Based on the discussions we have made above, it is clear that optomechatronic features can be found from either one single functional module or more than two modules combined together to generate an output signal. In the case of a single module, optical and mechatronic signals within it are properly integrated to generate an output signal. When multiple functional modules are combined to generate an output signal, the condition for optomechatronic integration should be such that at least one module must exhibit the nature of one optical module, and one mechatronic module must be included for the combination. Generic Forms for Optomechatronic Interface The above observation enables us to define the basic forms of configuration for optomechatronic integration. Figure 5.13 describes two such types, classified according to the change of the signal type between the input and output ports. Figure 5.13a indicates the case when the output signal has a different signal type as a result of integration. Most of the functional modules discussed in the previous section, except modulators, belong to this class.

280

Optomechatronics interface

integration module

input signal signal type :

modulating signal

interface

integration module

input signal

output signal

output signal

input signal = output signal

input signal ≠ output signal

(a) transformation type

(b) manipulation type

FIGURE 5.13 Basic types of optomechatronic integration.

This is called “transformation type” optomechatronic integration. In contrast, Figure 5.13b indicates that the type of output signal is not changed as a result of integration, retaining the same signal as that of the input signal, as can be seen from the modulators discussed in the previous section. This type is termed “modulation type” integration. Let us further examine details of each type of optomechatronic integration. As can be seen from Figure 5.14, the transformation type integration has basically three families of configurations. The thick line indicated in the figure denotes the interface between the input and output signals. The first configuration is the combination of a single-signal with one transformation module, which is shown in Figure 5.14a. When there appears no interacting external signal, a single signal transformer alone interface

O

E

EM

M

OM

O

O

O

M

E

M

MO

M

E

OE

E

ME

M

M

E

functional module

EO

E

O

O

(a) a signal with one transformation module O

TE O

OE

E

E

O

OM

EM

TO M

E

M

TM

TE

ME

EO

O

O

E

E

EM

M

TM

OM

TO M

M

M

TE

TO

MO

O

M

(b) combination of two transformation modules O E M

OPME OME

MO

O

O

O or M or E

(c) coupling between three different signals FIGURE 5.14 Interface for an optomechatronic transformation type integration.

ME

TE

OE

E

E

M

M

E

M

O

TM

E

TM

E

TO

EO

O


281

cannot be of optomechatronic nature. The reason is that with such one transformer optomechatronic integration cannot be physically feasible, since an input signal cannot be transformed into another at the interface without an additional transforming unit. For instance, when an optical signal is present at the interface of a mechatronic transformer (ME, EM), the coupling result should produce either a mechanical or electrical signal, so that the transformer can accommodate an optical signal. However, an optical signal cannot be coupled with either one of the two signals, since there is no unit transforming the optical signal to either a mechanical or electrical signal. Therefore, an interacting signal to the transformer must be present that has a signal type different from those inputs and outputs involved with the transformation. In other words, it must occur under the presence of a signal type different from those of the transformer. Figure 5.14a shows all six feasible configurations of integration. The first integration implies the case when an electrical-to-mechanical (E – M) transformation occurs under the influence of an optical signal. Due to the presence of an optical signal, the transformation may result in different characteristics. As we shall see later in “Optomechatronic Actuation” in Chapter 6, an interesting example of this case can be found in which the presence of an optical signal influences an electrostatic force between two capacitive plates. The last integration type in the figure describes an E –O transformation under the influence of a mechanical signal. Likewise, all the rest of the integrations shown in the figure can be physically interpreted. The second configuration is the combination of two functional modules shown in Figure 5.14b. This is essentially a multiprocess signal-transforming element. Some examples of such transformations are illustrated. For example, optically driven motion of mechatronic elements belong to the configuration shown in the first figure. The transformation yields mechanical signal motion to a mechanical element by applying an electrical signal transformed from an optical signal. A practical case of this configuration is a photodiode-driven PZT motion. Optical signal generation by an electrical signal, which in turn is produced by a mechanical signal, is another example of the integration shown in the last figure. A PZT driven laser diode well represents this configuration. Typical examples that belong to this type of integration are given in Table 5.7. The mathematical forms of the transformation integrations given in Figure 5.14 are expressed by: E O TEO þ TM ! TM

E O E TO þ TM ! TM

M TO þ TEO ! TEM

O þ TEM ! TEO TM

E M E þ TO ! TO TM

E M TEM þ TO ! TO

The third configuration is the combination of three single signal elements, denoted by OPME, as shown in Figure 5.14c. This configuration produces an output, any one of the three optical, mechanical, or optical signals. The integration can be made in any order, depending upon the physical

282

Optomechatronics

TABLE 5.7 Symbols and Mathematics for Various Types of Integrations Type of Integration

Symbol

Integration Mathematics

Typical Phenomenon, Device

reflector

Electricalto-optical

EO

reflector

E!O

negative positive

E þ O ! OE

LED chip

illumination p type Optical-toelectrical

OE

O!E O þ E ! OE

n type electrode

Mechanical-toelectrical

ME

motion

M!E M þ E ! ME

piezoelectric material motion

Mechanical-toelectrical

ME

M!E

S

separation

M þ E ! ME

core Mechanical-toelectrical

ME

M!E M þ E ! ME

coil

arm motion

continued


283

TABLE 5.7 CONTINUED Type of Integration

Symbol



rotation

Electrical-tomechanical

EM

E!M E þ M ! EM

S

N

stator

rotor

stator

displacement Electrical-tomechanical

EM

E!M E þ M ! EM

field coil terfenol-D rod displacement

Electrical-tomechanical

EM


E!M

electrode

E þ M ! EM

light

Optical-tomechanical

OM

O!M

optical fiber

bevel painted on black

O þ M ! OM

deformation direction diaphram

cavity


OM

O!M

electrostatic bonding

gas

O þ M ! OM

light

glass

continued

284

Optomechatronics

TABLE 5.7 CONTINUED Type of Integration

Symbol


OM



M!O M þ O ! MO

optical fiber

TABLE 5.8 Integration Type and Mathematic for Various Optomechatronic Integrations Integration Type

Symbol


Typical Phenomenon Device, Processes

Object manipulation/actuation

mirror EO ! ME ! OH

OPME

OE ! ME ! OPME

beam

crystal

undefracted

EO ! M ! O

OE ! M ! OPME

diffracted beam

sound

actuation

EO ! ME ! O


OE þ ME ! OPME

light beam

continued


285

TABLE 5.8 CONTINUED Integration Type

Symbol

O ! ME ! O


Typical Phenomenon Device, Processes light

O þ ME þ O ! OPME

deformable glass mirror

PZT actuator stack

electric leads

OE ! ME

laser

OE þ ME ! OPME

wave

Sensing

EO ! ME ! OE

OPME

atomor cell photo detector

OE þ ME þ OE ! OPME

angular displacement

light source actuator motion

EO ! ME ! OE


det laser

artifact

Manufacturing system or process laser

EO ! ME ! O ! ME

OPME

OE þ O þ ME þ O þ ME ! OPME

welding head work piece

motor

displacement

continued

286

Optomechatronics

TABLE 5.8 CONTINUED Integration Type


Symbol

Typical Phenomenon Device, Processes camera

EO ! ME ! OE


illumination

parts conveyor

EO ! ME ! OE


LED

optical module/wave guide 6 axis stage lens

phenomena involved. The mathematics of this integration is expressed by: O þ M þ E ! OPME In Table 5.8, typical devices that satisfy this relationship are summarized together with integration mathematics. The manipulation type shown in Figure 5.15 has basically six configurations. As we have already seen above, this type produces the same type of input signal as output, by utilizing a signal manipulator in the rectangular box. All manipulators are of the transformation type, operated on a twosignal integration element. The first configuration of the manipulation type shown in Figure 5.15a is a mechatronic type, since optical signal manipulation is achieved by mechatronic signal transformation. The second type in Figure 5.15b is an optomechanical type, whereas the last one is O

E O

EM

O

E

ME

E

M

M

M O

OM

E

O

(a) mechatronic type

E

MO

EO

M

O E

(b) optomechanical type

M

OE

(c) optoelectronic type

FIGURE 5.15 Interface for an optomechatronic modulation type integration.

M


287

the optoelectronic type in Figure 5.15c. The mathematics involved with the manipulation-type integration may be expressed by: O ! Memoo ! O

E ! Momee ! E

m M ! Meom !M

O ! Mmeoo ! O

E ! Mmoee ! E

m M ! Moem !M

A physical example of the first signal integration type can be observed from the optical modulator and mechanical beam scanner, as discussed in the “Basic Functional Modules” section. The other integration types may be conceivable, and may be found from some practical examples. In actual practice, optomechatronic integration can take place with the combination of more than two transformation modules. For example, the photo diode-actuated electromechanical device at a remote site belongs to this case. The mathematics of this integration can be given by: O E E E þ TRO TO O þ TE þ T M ! TM E denotes the signal transformation from electrical to optical It is noted that TO from a laser diode, TRO O denotes signal transmission to a remote site in the form of an optical signal, and finally, the signal transformation is back to a mechanical signal. When a piezoelectric sensor signal at a remote site is optically transmitted to a central control station, and then is transformed back to an electrical signal, the mathematics involved in this the integration can be expressed by: E O E þ TRO TEM þ TO O þ TE ! T M

The first term indicates the transformation of a signal from mechanical to electrical by the piezoelectric sensor, the next two terms involve signal transmission, and the last term indicates the transmitted optical signal back to an electrical signal at the control station. Another integration type can take place when a signal transformer and manipulator are combined together. A combination of a signal transformation and a signal manipulator occurs when laser light impinges on to a deformable mirror. The resulting integration is a light distributor, whose mathematics is written by E TO þ Mmeoo ! O

where the first term indicates a transformation for laser light and the second indicates the optical signal modulation by a mechatronic actuator. Another example of this category is an electric motor with an optical encoder. In this integration, the rotational motion of the motor is generated by a signal transformer, and then is measured by the encoder for feedback. The interface diagram for this interface cannot be seen in Figure 5.14 or Figure 5.15 because this interface requires two separate types of modules; one is

288

Optomechatronics

transformation and the other is modulation. In this case, the plus sign for mathematical expression can not be physically feasible, since the causality at the interface between two signal functional modules is not met. To this end, we use the multiplication sign to describe the mathematical expression as: E E ! TM £ Mmeoo ! O

Integrability So far, we have seen a number of different combinations conceivable for optomechatronic integration. However, it can be argued that not all integrations listed here can be expected to make the resulting device or system physically work. From this point of view, it appears that there need to be several considerations for devising a plausible interface between the integrated signals, to make such integrations physically realizable. Two important considerations can be made regarding the integrability. These are the interfaciability, and the output power requirement. The interfaciability refers to causality considerations, whereas output power requirement refers to the ability to produce the power that can be usable at the output side. Causality is the relationship between the signal at the input side and the signal at the receiving unit when they are interconnected at the interface. In other words, the input signal to the interface should be transformable, in order that the signal receiving unit can accommodate it. For example, if the integration shown in Figure 5.14a is considered, each input signal should be interfaciable to the port of each transforming unit. If this is not the case, the integration cannot possibly happen. The same arguments can be applied to the other integration cases shown in Figure 5.14b,c. The other consideration of integrability is the magnitude of energy or power attainable at the output port for a given input signal level. If the output magnitude is very small, the output signal may be affected by noise excitation, or require a very large input signal to attain a relatively large output signal. For a signal within some limited range, this kind of integration becomes problematic, although causality is met. Let us take one interface for an integration, for example, the second type O ! M, in Figure 5.14a. From what we have seen from the piezoelectric actuator, if the required power of the optical signal is too large to attain a desired mechanical signal, this integration module may not be appropriate or feasible. The same requirement on output power will be needed for the other two cases in Figure 5.14b,c. Photo diode-actuated mechatronic elements at remote sites are one such example that may need stringent power requirements. The transmitted optical signal is normally attenuated, and its transmitted optical power may not be large to provide sufficient input to the photo diode. In other words, sufficient power to drive the mechatronic element may not be attainable either due to attenuation during transmission, TRO O , or power loss during transformation. Another example can be found from the devices, such as optically driven mechatronic elements, acousto –optical modulators ðMmeoo Þ:,


289

and mechanical scanners ðMmeoo Þ: As we will discuss in the next chapter, all of these devices will need to consider efficiency in their output power. Signal Flow An optomechatronic device or system may be full of functional modules or single signal elements. Figure 5.16 indicates the two generic configurations exhibiting an optomechatronic nature that can be made by optic – mechatronic interface. The arrow indicates the flow of information from one module to another. The open loop structures shown in Figure 5.16a represents the case when the interfacing would occur in a series, that is, all modules are combined in series to generate an output signal. It has three basic forms, which are series, parallel, and series-parallel. The series structure has the configuration of functional modules that are coupled in series such that the output signal of each module flows from one to another. In the parallel structure, however, two modules are put together in parallel, such that their output signals are added together. The series-parallel structure is different from the two in that the divided signals from the proceeding module enter into the two different modules next to it. A device of this type is conceivable when an optical signal is split into two by a beam splitter. Then, each split signal enters into its own autofocusing functional module. The configurations shown in Figure 5.16b have a feedback structure in that the final output is fed back to at least one of the modules within the system. Two feedback structures shown in Figure 5.16b differ from their feed forward (open) loop. The basic case has a single path, while the other form has a multiple path (in this instance, two paths). An example can be illustrated with a camera with autofocusing capability, as mentioned before. In this example, one of the relevant modules in the series has a feedback loop within the integration, as module

module

module

signal

module

+

module

signal

− module

+ + module module

signal

+

module +

module module

(a) open-loop configuration

module module module

(b) feedback configuration

FIGURE 5.16 Basic open loop and feedback configurations for signal flow.

signal

290

Optomechatronics

transformer E

O Er

TEO

o me o

M

+_

E

Er +

_

Md

+

TME

M

O

transformer PSD sensor TEO

laser

E

transformer

(b) measurement of machined workpiece surface optical actuator

TOE

E

o meo

TME

TME

(a) camera

O

TOE

TOM

Mme oo

optical power

gripper

TMM

M

optical mirror gripper

UV light optical fiber iris (valve)

O M

T

optical actuator

TEM

mirror

mirror supporter

motor

sensor

(c) force feedback control of a two-fingered gripper FIGURE 5.17 Illustrations of the closed-loop system.

shown in Figure 5.17a. The optical signal (O) is continuously adjusted to make a feedback control of a lens location focused until a certain image can be focused to a satisfactory level. Another example is an optical power control which has been previously discussed in a laser-controlled machining system. In this example, focusing is made by adjusting the position of the optical system in the laser power system. The schematic diagram illustrated in Figure 5.17b shows an optical sensor-based control for mechatronic system, as discussed in “Optomechatronics: Definition and Fundamental Concept” in Chapter 1. Basically, it has three transforming elements and one modulator: The mechatronic E element denoted by TM has a signal transforming element (from electrical E has a transformation to mechanical signal); a laser diode denoted by TO (from electrical to optical signal); a photo sensor TEO has a signal transforming element (from optical to electrical signal). The module in the middle E represents the interaction with the mechatronic and optical elements. TM The modulation module, Mmeoo ; receives a laser beam produced by a laser E , and impinges it upon the surface of the mechatronic element (e.g., source TO rotating workpiece). The incident beam reflects differently and the reflected beam enters to the photo sensor, TEO depending on the machined surface texture of the workspace being machined. This signal is feedback to compare E with the desired electrical signal. Based on the difference signal, TM module generates an electrical command signal to a driving motor, which acts as a modulation module Mme oo .


291

The cascade feedback form shown in Figure 5.17c has basically the same structure as that of the basic feedback form, except the cascade element in the open-loop path. One nature of this feedback form can be found in the case when two optical actuators actuate a mechanical element composed of two grippers which grasp an object with a desired force, as depicted in the figure. The gripping force signal generated by a force sensor can be used to detect the state of gripping, and fed back to the controller element of the controlling optical power coming from a light source. The operation principle of this optically-actuated two-figured gripper is as follows: A O E , actuates two optical actuators TM proper amount of optical power TO and then two grippers accordingly produce small mechanical deformations M Þ due to the gripping force. This force (mechanical signal, M) of an object ðTM is detected by a force sensor ðTEM Þ and fed back to an optical power controller force to compare with the desired mechanical signal ðMd Þ:

Integration-Generated Functionalities In the previous sections we have defined basic functional modules that generate functionalities required for optomechatronic systems, discussed the integrability of the modules to create optomechatronic characteristics, and then studied the signal flow graph that represents the flow of information from input port to final output port. All of these considerations are necessary to design a certain functionality specified for optomechatronic devices, processes, or systems. Our interest here is to know how they are generated by a single module, or by combining the functional modules. In this subsection, we will briefly treat these subjects and illustrate how functional modules are used to produce the required functionalities for an optomechatronic device or system. Any engineering system that includes devices, machines, or processes must possess one or more functionalities in order to perform the desired tasks as specified by the designer. In essence, such a functionality is created by an individual module or combination of functional signal modules. Depending upon how many modules are involved, or how they are interacted with each other, the functionality can have different attributes. Various functionalities created in this way can be combined to make systems exhibit a certain functional behavior. In general, a system or device will need a variety of functionalities to perform a given task. The fundamental functionalities are the common functionalities that are frequently utilized everywhere. Let us consider here some of the fundamental functionalities listed in the Introduction. Figure 5.18 shows a variety of configurations to create a functionality. This configuration is basically similar to those shown for basic configuration in Figure 5.16. Figure 5.18a shows the case when a single module is needed to generate a functionality. The series configuration in Figure 5.18b shows the case when signal flow occurs in a forward direction,

292

Optomechatronics

functional module

input signal

functionality

(a) single module input signal

functional module

functional module

functional module

…

functionality

(b) series input signal

input signal

functional module

input signal + +

functionality

+

−

functional module

functional module

functional module

(c) parallel-series

functional module

(d) feedback

FIGURE 5.18 Various configurations to generate a functionality.

FIGURE 5.19 Functional modules for a pneumatic pressure control system.

functionality


293

while the parallel series utilizes a scheme of signal processing in parallel, as shown in Figure 5.18c. Lastly, Figure 5.18d shows a feedback scheme to produce a functionality. Figure 5.19 illustrates the remote control of a pneumatic power system located in a remote site. This system is all optically operated, so that it uses all-optical operation modes, e.g., optical actuators and sensors. The pneumatic, flapper-nozzle valve is controlled by an optical actuator, as considered in the previous discussion. The operation starts when the highpower optical signal transmitted through an optical fiber enters the optical actuator. The actuator, when activated, causes the flexible membrane to move closer towards, or farther from, the nozzle, which accordingly yields a change in backpressure; closer means higher back pressure, while farther means lower backpressure. This change is amplified by a pneumatic gain block. The amplified pneumatic pressure operates a spool valve, and actuates a pneumatic cylinder connected to a system to be controlled. This control system, therefore, belongs to the feedback form shown in Figure 5.18d. Figure 5.20 illustrates the diffraction-based sensing concept that can be used for an atomic force microscope (AFM), which will be treated in Chapter 7. The sensing uses the principle that the variation of grating distance results in the variation of diffraction angle of a laser beam (first order), as discussed in “Diffraction” in Chapter 2. The sensing tip is

actuator motion

photo detector

comb drive

laser

spring AFM tip

artifact

sample (a) diffraction grating sensor measurement function

input signal

+_

actuation module (comb drive)

measurement signal

sensor system

sensing module

(b) functional module diagram FIGURE 5.20 Functional modules for a diffraction grating sensor.

output distance

294

Optomechatronics

supported by a diffraction grating whose end is connected to a comb drive actuator. Since the grating element inherently possesses a spring element due to its material and structural composition, the grating element becomes compressed when the whole sensing unit is scanned through and encounters rough artifact. Note that the support element is fixed at a constant height, as shown in the figure. When laser light is interfaced with this variable grating, it results in the angle variation of the diffracted beam, depending upon how the grating element is displaced.

Problems P5.1. The following are some single-stage transformation modules that interact with a single signal considered in Figure 5.14a. Illustrate a physical device or system corresponding to each case (Figure P5.1).

O

E

ME

M

E

O

O

OM

M

EM

E

M

FIGURE P5.1 Single –stage transformation modules.

P5.2. Illustrate a physical element, device, or system that works according to each of the following: two-stage module composed of transformation and/or modulation type as shown in Figure P5.2.

E

E

O

TO

O

TM

M

E

E

M

TE

E

o

Mme o

O

(2)

(1) M

O

TO

E

TO

(3)

O

O

O

TM

M

M

TE

E

(4)

FIGURE P5.2 Two–stage transformation modules.

P5.3. Repeat the same problem for the multi-stage modules composed of signal transformation and signal modulation as depicted in Figure P5.3.


295 O

E

E

TO

O

o

Mme o

O

TE

TM

E

M

O

E

(1) O M

M

TE

E

M

E

TM

o

Mm o

O

TE

(2) FIGURE P5.3 Multi–stage modules composed of signal transformation and modulation.

P5.4. The following signal flow graph indicates a feedback control system in which the position of an optical modulation type device ðMme oo Þ: is varied depending on the magnitude of electrical output of a transformation module ðTEO Þ: The transformation module receives the modulator’s optical output as its input as can be seen from Figure P5.4. (1) Describe a detailed operation principle of the feedback system. (2) Devise an optomechatronic system that obeys this signal flow.

E

E

TO

O

o

Mme o

O

O

TE

E

E

TM

FIGURE P5.4 The signal flow graph for an optomechatronic system.

P5.5. Consider a remote operation of the pneumatic pressure control system shown in Figure 5.19 and redrawn here for this problem in Figure P5.5. (1) Write down signal flow mathematics for this system. (2) Draw a complete signal flow graph describing remote operation of this system. Assume that the sender (control) site uses an LED for signal transformation. P5.6. Figure P5.6 shows an optically-ignited mechatronic system discussed in the Introduction. Repeat the same problem as given in P5.5 (1) and (2).

296

Optomechatronics

FIGURE P5.5 Remote operation of a pneumatic pressure control system.

laser path

sapphire window/case/seal projectile

collimator

propellant charge

FIGURE P5.6 Optically-ignited mechatronic weapon system.

P5.7. Figure P5.7 shows a remote operation of an inspection system, operated by a robot, also discussed in the Introduction. The transmission is carried out by an optical signal. (1) What type of transceiver in the control site do you recommend? (2) What type of transceiver in the remote site do you recommend?A transceiver contains a module for transmission and a module for receiving. control site

remote site

monitor

transceiver

transceiver fiber PC

FIGURE P5.7 Data transmission for remote operations.

sensor


297

(3) Write down the signal flow mathematics for this system for each. (4) Draw a complete signal flow graph describing the remote operation of this system. Assume that there is no feedback operation at the control site, that is, one-way operation. P5.8. Laser welding process considered for discussion in “IntegrationGenerated Functionalities” in Chapter 5 is shown in Figure P5.8. (1) Describe in detail how it works to join two metal plates. (2) Based on this figure, draw a complete signal flow graph to carry out the welding task. laser

welding head work piece motor displacement FIGURE P5.8 A laser welding system.

References [1] Cho, H.S. Characteristics of Optomechatronic Systems, Chapter 1, Optomechatronic Systems Handbook. CRC Press, 2002. [2] Cho, H.S. and Kim, M.Y., Optomechatronic technology: the characteristics and perspectives, special issue on optomechatronics: fusion of optical and mechatronic engineering, IEEE Transaction on Industrial Electronics, 52:4, 932– 943, 2005. [3] Hockaday, B.D. and Waters, J.P. Direct optical-to-mechanical actuation, Applied Optics, 29:31, 4629– 4632, 1990. [4] Kim, S.G., Optically sensed in-plane AFM tip with on-board actuator, Lecture No. 6.777, Design and Fabrication of Microelectromechanical Device, Final report of Design Project, MIT, 2002. [5] Liu, K. and Jones, B.E. Pressure sensors and actuators incorporating optical fibre links, Sensors and Actuators, 17:314, 501– 507, 1989. [6] Ogata, K. Modern Control Engineering, International Edition. Prentice-Hall, Englewood Cliffs, NJ, 1970.

6 Basic Optomechatronic Functional Units CONTENTS Optomechatronic Actuation............................................................................. 301 Silicon Capacitive Actuator ...................................................................... 302 Optical Piezoelectric Actuator ................................................................. 304 Mathematical Model of the Photon-Induced Currents................ 305 Induced Strains of the PLZT ............................................................ 306 Photo-Thermal Actuator ........................................................................... 311 Optomechatronic Sensing................................................................................. 316 Optical Sensor............................................................................................. 316 Fabry-Perot Etalon.............................................................................. 318 Fiber Optic Sensors .................................................................................... 321 Automatic Optical Focusing ............................................................................ 326 Optical System Configuration.................................................................. 327 Optical Resolution ..................................................................................... 328 Axial Resolution ................................................................................. 329 Feedback Control of the Objective Lens ................................................ 330 Effect of External Disturbance.......................................................... 336 Focus Measurement ........................................................................... 338 Acoustic-Opto Modulator ................................................................................ 339 Deflector ...................................................................................................... 341 Frequency Shifter ....................................................................................... 345 Tunable Wavelength Filtering .................................................................. 347 Efficiency of Modulation and Speed....................................................... 347 Optical Scanning ................................................................................................ 348 Galvanometer ............................................................................................. 349 Feedback Control of Galvanometer ........................................................ 356 Polygonal Scanner...................................................................................... 363 Correcting Scan Errors....................................................................... 367 Optical Switch .................................................................................................... 367 Thermally Actuated Mirror...................................................................... 369 Electrostatically Actuated Mirror Control ............................................. 371 Lens Controlled Switching ....................................................................... 375 Zoom Control ..................................................................................................... 377

299

300

Optomechatronics

Zooming Principle ..................................................................................... 377 Zoom Control Mechanism........................................................................ 383 Visual Autofocusing .......................................................................................... 386 Image Blurring............................................................................................ 386 Focus Measure............................................................................................ 389 Illumination Control.......................................................................................... 399 Illumination Methods................................................................................ 400 Illumination Control.................................................................................. 403 Illumination Quality Measure.................................................................. 408 Autofocusing with Illumination Control ............................................... 410 Visual (Optical) Information Feedback Control............................................ 411 Visual Feedback Control Architectures .................................................. 414 Fixed Camera Configuration ............................................................ 417 Eye-In-Hand Configuration .............................................................. 420 Feedback Controller Design ............................................................. 421 Optical Signal Transmission............................................................................. 428 Signal Transmission ................................................................................... 428 Power Transmission and Detection ........................................................ 431 Problems.............................................................................................................. 432 References ........................................................................................................... 443 In the previous chapter, we have seen that signal elements (mechanical, optical, and electrical) can be combined in a variety of ways to produce basic functional modules which include transformation, manipulation, transduction, actuation, transmission, storage, and display of the signal. These modules were often shown to be optomechatronic in nature, or may be formed by other natures such as optoelectronic, mechatronic, and so on, as we have discussed in Chapter 5. Due to the presence of optomechatronic interaction, the integration was found to create a variety of different types of functional modules, adding more attributes to the existing ones. Any engineering system that includes devices, machines, or processes must possess one or more functionalities in order to perform the desired tasks as specified by the designer. In essence, such a functionality is created by an individual module or combination of functional signal modules. Depending upon how many modules are involved or how they interact with each other, the functionality may have different attributes. The functionalities created in this way can be combined to make a system exhibit a certain functional behavior as specified by the designer. In general, a system or device will need a certain number of functionalities to perform a given task. But, in case the device carries out a number of different tasks, it will need a number of different functionalities. Generalizing this notion to the case of many systems or devices, we see that a number of functionalities may be frequently used for certain tasks. The fundamental functionalities are those functionalities that are common everywhere in optomechatronic-related engineering fields. In this chapter,

Basic Optomechatronic Functional Units

301

we will consider the fundamental functionalities produced by the modules that are the basis of creating optomechatronic properties. As mentioned briefly in Introduction, these properties include: (1) optomechatronic actuation, (2) optomechatronic sensing, (3) optical autofocusing, (4) AO modulator, (5) optical scanning, (6) optical switching, (7) zoom control, (8) visual autofocusing, (9) illumination control, (10) visual servoing, and (11) optical signal transmission. Because these functionalities are frequently used in most optomechatronic systems, we will focus on understanding their basic concept, hardware configuration, and important factors associated with their performance.

Optomechatronic Actuation The actuating function, one of the fundamental functions needed for actuating optomechatronic systems, is the actuation produced by a mechatronic actuator that drives optical systems, or by an optical drive that actuates mechatronic systems. The types of optomechatronic actuation can be grouped into three classes. The first class is the optical actuators which employ the optically-driven actuation principle. In this case, the energy supplied for actuation comes from a light source, which provides displacement or force to moving mechanisms. A variety of this type of actuator has been developed for different applications. The second class is the mechatronic actuators with embedded optical units: Here, mechatronic actuators implies all non-optical actuators. An electrical motor with an optical encoder is a typical example. The third class encompasses the actuators that drive an optical or optomechatronic system. In this chapter, we will discuss some of the optically driven actuators. Optical actuators function when light is either directly or indirectly transformed into mechanical deformation that generates small displacements of micro- or nanometer scale. Depending upon the interaction of light energy with an absorbing surface, a variety of effects can occur. According to their nature, two types of optical actuations are considered here: photoelectric actuation and photo-thermal actuation. Photo-electric actuation utilizes the conversion of the variations in light intensity into the change in the electric power by means of a p –n junction of semiconductor devices, or piezoelectric material resulting from the generation of photo electrons. The other form of conversion uses the change in capacitance in a capacitor type of actuator configuration. In addition to these types, optical actuators that belong to this photo-electric class use a variety of other conversion methods, including photo-conductivity modulation, photodiode, direct optical manipulation (optical twizer), and so on. In contrast to this principle, photo thermal actuation utilizes the conversion of the variation in light energy into the change in thermal energy.

302

Optomechatronics

Silicon Capacitive Actuator Silicon micro actuator is one of the photo-electric actuators that utilize the photon-generated electrons. This actuator uses change in the stored charges in a capacitor, which results in change in the electrostatic pressure acting on a cantilevered beam. This actuator therefore does not use piezoelectric or thermal effects for silicon microstructures. As shown in Figure 6.1a, the actuator is composed of a cantilever beam (Si) and a ground plate (Cu) on an insulating glass, which forms a parallel plate capacitor [33]. This parallelplate capacitor is given by a simple relation according to Equation 4.2 A d

C0 ¼ 10 1r

where A is the area of the cantilever facing the ground plate, 10 is the permittivity of the free space, 1r is the permittivity constant of the medium between the gap, which in this case is air, and d is the gap between the cantilever and the ground plate. If an electric voltage V0 is applied to the capacitor through a resistor R, the stored charge is obtained by q0 ¼ C0 V0

ð6:1Þ

Now, let us consider the optical actuator subject to the electrical potential V, whose gap is not constant due to the cantilevered beam being clamped at one end. The stored charge will then produce an electrostatic force, which in turn causes a deformation of the cantilever. Since the beam has a clamped-free condition at its ends, the deflection d at the tip can be described by

d ¼ kq q2

ð6:2Þ

where kq is a constant related to the geometry and material property of the cantilever. This relation can be derived from a simple beam theory from which the deflection of a cantilever beam is governed by d4 z 4 ¼ pðxÞ dx4

cantilever (Si) d

d

e ee

V

light ground plane(Cu) glass

R (a) actuator configuration

FIGURE 6.1 Semiconductor capacitive optical actuator.

total current (i). pA

ð6:3Þ

O

light intensity

DC bias voltage, Vb

(b) variation of total current with DC bias voltage

Basic Optomechatronic Functional Units

303

where z is the vertical displacement at the location x along the beam, and pðxÞ is the distributed loading due to the applied voltage. In the above equation, the cantilever has small deflection and the thickness h p ‘: Attention should be given in the above equation that, due to deflection, the electrostatic force is no longer uniform or constant, but depends on x. Nevertheless, we will assume for simplicity a constant loading condition. According to the electrostatic force theory, voltage causes a constant electrostatic force between the ground plate and the undeformed cantilever by the relation given by pðxÞ ¼

10 V 2 2d2

ð6:4Þ

as given in Equation 4.49. Substituting Equation 6.4 into Equation 6.3 and evaluating zðxÞ at the end of the cantilever, we have zðxÞlz¼l ¼ d
> < Vi ¼ U > > : Umin

is defined by if U $ Umax if Umin , U , Umax if U # Umin

Combining all of the elements mentioned above, we can draw a complete block diagram for a feedback control system as illustrated in Figure 7.40. The controller utilizes an error signal eðtÞ ¼ x3d ðtÞ 2 x3 ðtÞ, the difference between the reference and the feedback signals of the displacement in the x-axis. The aim of the control objective is to eliminate this error, that is, to position the stage in x direction to a desired location with smaller overshoot but faster rise time without any steady state error. From observation of the control system block diagram, we can see that the system has nonlinearity with saturation. To analyze this, we need to utilize a controller such as a describing function method, adaptive control, and so on. However, we will apply a simple linear PID controller for simplicity in order to see the scanning motion control. With this PID controller we now determine

x3d

e

GC (s) controller

U

Vi

Umin

Vi UmaxU

FIGURE 7.40 Schematic of the x-axis position control system.

Ga (s) driver

VO

Fcf

Fp

G (s) stage

x3

Optomechatronic Systems in Practice

509

TABLE 7.7 The Parameters Used for the x-Stage Simulation of an AFM Parameters

Symbol

Unit

Value

Force coefficient The first mass The second mass The third mass Stiffness Resistance of drive Capacitance of piezoactuator Saturation voltage

Fcf m1 m2 m3 k R C Vsat

Nm/V kg kg kg N/m V F V

30 £ 1023 8.68 £ 1023 8.68 £ 1023 1.28 £ 1021 5 £ 1025 6.7 £ 1023 4.8 £ 1029 100

the response characteristics for a variety of controller gain parameters. The parameters used for this simulation are listed in Table 7.7 and the Simulink model is shown in Figure 7.41. Figure 7.42 represents the response of the stage for a step displacement of 5 mm. Figure 7.42a shows the response for three different proportional gains at a fixed set of integral and derivative gains, ki ¼ 5 £ 103 , and kd ¼ 5 £ 1021 : It can be seen that, as the proportional gain increases, the response becomes oscillatory, but the steady state error is seen to rapidly decrease. The effect of the integral gain is shown in Figure 7.42b for a fixed set of gains, kp ¼ 2 £ 103 and kd ¼ 5 £ 1021 : As the integral gain increases, the response becomes very fast without overshoot. In general the x-axis response shows a satisfactory response in both speed and steady state characteristics. In this analysis, the motion coupling due to the monolithic stage mechanism is not considered. In the actual stage, the x-stage motion will be somewhat influenced by y-direction motion due to the characteristic property of the monolithic stage mechanism. However, the degree of the y-stage influence is found to be very small in terms of its amplitude due to the design for decoupling between axes. The same trend is found from the of y-stage motion. Although, the stage motion in the

target Xr

e

1000000 Gain1

PID Controller

Vi

U Saturation

1 π R C.s+1 Actuator

Scope2

Scope1

FIGURE 7.41 Simulink model for the x-stage simulation.

Vo

30 e-3 Gain2

F

2

π

M1.s +2 K den (s) System

X3

510

Optomechatronics

5

4

displacement (x), μm

displacement (x), μm

5

kp = 3x103 kp = 2x103 kp = 1x103

3 2

ki = 5x104 ki = 5x103 ki = 5x10

3 2 1

1 0

4

0

1

time (t ), ms

2

3

(a) effect of the proportional gain, ki = 5×103, kd = 5×10-1

0

0

1

time (t ), ms

2

3

(b) effect of the integral gain, kp= 2×103, kd = 5×10-1

FIGURE 7.42 The response of the x-stage for various gain parameters.

z direction is not analyzed here, it is found to exhibit motion characteristics similar to that of the x-stage. The z-motion stage that moves the cantilever in the direction vertical to the sample surface during the x-y scan motion should also exhibit extremely high accuracy motion in response to the cantilever deflection.

Confocal Scanning Microscope The concept of confocal microscopy was first proposed by Minsky in the 1950’s in an effort to improve the limited resolution of conventional microscopes by reducing the amount of scattered light from the sample being inspected that has been observed from conventional microscopes. However, due to lack of suitable lasers and precision stages the technique could not be practically realized until the early 1980s. Confocal microscopy is now a well-established technique measuring high resolution surfaces of the topography of objects in biomedical applications, material science, semiconductor quality monitoring and control, forensic applications, and so forth. Measurement Principle As shown in Figure 7.43a, the microscope is composed of a point-wise light source (usually a laser), a precision scanning mechanism, a light detector with a pinhole sensing the reflected light from a specimen (object), and the necessary optical units.


511

laser source lens lens condenser lens objective lens

collective lens

source

pinhole

sample plane

objective lens sample

(a) confocal (reflective type)

image plane

focal plane out of focus

(b) conventional

FIGURE 7.43 Comparison between confocal and conventional microscopes.

Due to the basic difference in its optical arrangement a confocal microscope yields image “characteristics” different from those that can be obtainable by a standard microscope. The operation principle with the confocal microscope is as follows. When the object to be imaged moves out of the focal plane, the resulting defocused image will be very weak in intensity and beyond some locations it will disappear rather than blurring. This is because the light reflected from the object out of focus is defocused at the pinhole located in front of a detector and (dotted line), therefore not much light will pass through it, thus giving no appreciable contribution to imaging at the detector. It is noted here that the image intensity in this case decreases as the object image is defocused. Due to this property, a confocal microscope tends to yield sharper images than a standard microscope does. By contrast, in a conventional optical microscope as shown in Figure 7.43b, the whole area of the sample, including the focal plane, is simultaneously illuminated by an extended source through a condenser lens full-field illumination. Therefore, the information from each illuminated point in the object is simultaneously imaged on to a screen or the retina of the eye. Due to this configuration, much of the light illuminated from the regions of the objective plane above and below the selective focal plane are simultaneously collected by the objective lens. This causes an out-of-focus blur to form the image. However, as we recall the principle of autofocusing treated in Chapter 6, “Automatic Optical Focusing,” the blurring can be eliminated or reduced by keeping the position of the image plane at a focal plane. There are two types of confocal system; the reflective type and the transmission type shown in Figure 7.44. In both models, one objective lens is used for illumination, while one collector lens is used for detection. The transmission geometry is particularly useful for imaging transparent objects

512

Optomechatronics axial response from a mirror objective beam splitter

source .

z object

scanned

objective

collector scanned

collector

point detector

point source specimen plane

(a) transimitive type

pinhole detector

(b) reflective type

FIGURE 7.44 Two basic types of the confocal system.

in a highly scattering medium. For weakly scattering objects, the threedimensional (3D) imaging properties are not much improved, because of strong unscattered light in transmission. Its 3D point spread function is found to be almost identical with nonfocal imaging. Due to this, the majority of confocal systems operate in the reflection mode under the same operation principle but with a slightly varied configuration. The reflective type, as mentioned above, produces a strong signal only at the in-focus axial position of the object, producing a very weak one at the defocused position. It is noted that the property of this reflective type is identical to that of the transmission mode if an object is placed in the focal plane. Although confocal concept has been already explained, let us elucidate it again for a better understanding of this subject. Figure 7.45 depicts a simplified schematic to explain the principle of confocal detection. The basic steps employed are to project a focused beam of light into a specimen, to scan a beam of light over the specimen, and at the same time detect the light reflected from its surface by a light detector device with a pinhole or single-mode optical fiber. In more detail, (1) the laser beam enters a collimating optical lens from the left-hand side. (2) The collimated light is first transmitted to the right by a beam (solid line). (3) This transmitted light is then focused on to a spot by a focusing lens. Depending on the location of the object to be measured, the beam is focused in the focal plane of the objective (location b), positively (location c) and negatively focused (location a). (4) The reflected light travels back through the objective lens, enters the beam splitter where the light directs down towards a collecting lens. The collective lens then focuses the light on to the detector aperture. The ray paths indicated in


lens

513 out of focus (c) in focus (b) out of focus (a)

beam splitter objective lens

laser

p´

object p

p´´

x-y stage

z stage

collecting lens d´´

(a) (b) (c)

d d´

pinhole detector

FIGURE 7.45 A simplified schematic to explain the principle of confocal detection.

dotted and dashed lines show how lights from an out-of-focus object are propagated though the optical system. The figure shows three focal points illuminated according to the object location in an axial direction (z direction) (a), (b), (c). The focused point, d in the pinhole is due to the focused point, p at the in-focus object position (b). Likewise, the defocused point p0 at the location (c) forms the defocused point d0 in the detector, while the defocused point p00 at the location (a), the defocused point d.00 The detector signal reaches its maximum if the object is at the focal point of the objective lens. If the illumining spot is defocused a larger area is illuminated with a much weaker intensity. This property implies that by changing the focal plane in the direction of the optical axis while carrying out successive x-y scans, a 3D image of a sample can be recorded as a result of the confocal rejection of out-of-focus signal. In the case of a thick sample, its 3D image can be recorded with slicing of the sample, since the reflection system can record a series of sections of the image at different depths of the sample and thereby build up complete 3D information of it in a 3D world. By use of this confocal scanning, height information of 3D samples whose variations are in the order of 1 nm can be imaged. In addition, oblique sections at any angle can also be accurately measured. Beam Scanning As in the case of AFM, the scanning mechanism is a basic device needed to build up the image point by point. The mechanism consists of a z-motion scanning stage that maintains a constant objective-sample separation and an

514

Optomechatronics transmitive sample pinhole

light source

D y

light source

x

detector

(a) sample scanning

pinhole

A'

D'

A

D

light source

planar detector array planar lenslet array

detector

(b) beam scanning

(c) multiple simultaneous beam scanning

z pinholes

FIGURE 7.46 Types of scanning in confocal microscopes.

x-y stage that performs an x-y raster pattern scanning. All three elements, sample, objective, and laser beam shown in Figure 7.46 can be scanned. Objective scanning shown in Figure 7.46a scans samples relative to a fixed, stationary focus spot. It has a relatively low speed of scanning due to the large inertia of its mechanical configuration and requires relatively high positioning accuracy. This scanning method has difficulty maintaining uniform illumination across the field of view (FOV), and therefore, has not been popularly used. Due to the limitations associated with the above method, the beam scanning method has been widely adopted for the majority of commercial uses. This scanning method scans the laser beam relative to a stationary object. It deflects the beam by some scanning mechanism such as in two different schemes, point-by-point (Figure 7.46b) and multiple simultaneous scanning (Figure 7.46c), schematically shown in the figure. The point-by-point method is beam scanning which utilizes one illumination pinhole and one detection hole. It employs a 2D vector scanner for the x and y directions. When the illumination pinhole moves from point A to A0 , accordingly this causes the confocal image to move image point D to D0 upward as can be seen from the right-hand side of the figure. The scanning system should cover the pupil size of the objective lens of the microscope and angle of view, and meet requirements such as scanning resolution, accuracy and speed. The scanned optical quality depends not only upon the type of scanner, but also on scanning optics that needs to be matched with the microscope. As discussed previously, the scanning resolution here again is limited by diffraction theory, and the larger aperture size of the scanner will increase the resolution. As scanning devices galvanometer mirrors and acousto-optic


515

laser source

A-O scanner deflected beam

beam splitter galvanometer x-y mirrors

(b) acousto-optic scanner aperture

detector

object

rotating mirror

(a) rotating x-y mirrors A-O scanner

(c) hybrid scanner FIGURE 7.47 Schematic diagram of the x-y scanner used for confocal microscopes.

cells are used, while a polygon mirror is rarely used since it cannot provide satisfactory scanning accuracy and reliability due to wobbling while rotating and fabrication defects. There are several ways to achieve this scanning which includes methods as illustrated in Figure 7.47, viz., the use of (1) rotating mirror, (2) acousto-optic deflector, (3) hybrid scanner, and (4) a single-mode optical fiber in replace of the pinhole. Although these methods are based upon point by point scanning, they are very fast in scanning. The figure shows a schematic overview illustrating three beam scanning schemes. In the case of the galvanometer mirror (Figure 7.47a), as mentioned previously, the laser beam passing through an illuminating aperture is reflected from a beam splitter. The reflected beam then impinges upon the mirror surfaces of the two scanning galvanometers; the scanning optics projects the deflecting point of the scanner on the pupil of the objective lens and the deflected beam resulting from the scanning motion is focused and then illuminates the sample through the objective lens. The reflected light returns through the microscope along the same ray-path as the projection light. The light passed back is transmitted by the beam splitter and finally imaged on to the detector aperture. This operating principle is also applied to the case of adopting acousto-optic scanners and hybrid methods. When the sample image of some large area needs to be formed simultaneously, an array of a large numbers of pinholes would be required for both pinholes of illumination and detection. This method is called a “tandem scanning optical microscope” (TSOM) and uses a large number of pinholes for real-time simultaneous scanning of a certain area of the object. We will return to this method in the next subsection.

516

Optomechatronics objective lens objective lens detector

rotating mirror sample detector

rotating mirror

light source

sample

light source

(a) objective lens with partially filled with light d1

(b) objective lens with completely filled with light

d2

d3 sample

detector rotating mirror

objective lens auxiliary lens

light source

(c) auxiliary lens system FIGURE 7.48 Two types of beam scanning using a rotating mirror.

Figure 7.48 shows two typical types of rotating mirror based-beam scanning. If the beam is deflected by a mirror and is incident on an objective lens, the lens may be either partially or completely filled with light. If the lens with partially filled with light is used, as in Figure 7.48a, the numerical aperture is smaller than that in case of using a full aperture, this results in loss of resolution. If the lens is fully covered with light as shown in Figure 7.48b, a large portion of the light will be lost. These drawbacks can be resolved by using auxiliary lenses as shown in Figure 7.48c. The rotating mirror must be placed in a zero-deflection plane, a distance d1 in front of the auxiliary lens where d1 ¼ ðd2 þ d3 Þ

d2 d3

ð7:33Þ

where d2 is the focal length of the auxiliary lens. The auxiliary lens here contributes to the complete filling of the objective lens without a loss of light. The schematic configuration of a confocal scanning microscope that has two-axis scanning mirrors is shown in Figure 7.49. It is basically the same configuration as shown in Figure 7.48c in that it utilizes an auxiliary lens. This configuration satisfies the above condition in Equation 7.33. A confocal system that employs an acoustic-optical deflector (AOD) for beam steering is shown in Figure 7.50. The laser light irradiating from a laser diode is expanded through a beam expander. When the expanded parallel


517 auxiliary lens

motor objective lens

sample

x scanning mirror

z stage

y scanning mirror motor beam splitter

detector pinhole lens lens

laser source FIGURE 7.49 Confocal laser scanning microscope with a two-axis rotating mirror scanner.

beam enters the AOD, the parallel beam is deflected with a certain angle depending on the voltage applied to a piezoelectric transducer. If the deflected angle by the AOD is given by ud, then sin ud ¼

lfa 2va

ð7:34Þ

tube lens

scan lens AOD

beam wave splitter plate

y z

laser collector

f1

f2

sample objective

collective

photo diode

FIGURE 7.50 Confocal scanning microscope using AOD.

518

Optomechatronics

where va is the sound velocity and fa is the frequency of sound. This deflected beam goes through a scan lens and becomes parallel again by a tube lens. In this condition, it follows that if the focal lengths of the scan and tube lenses are given as f1 and f2 , respectively, then the magnification ratio should meet the following condition M¼

f2 f1

ð7:35Þ

This parallel beam then enters an objective lens, which eventually is focused on a point within a specimen. It should be noted that between the tube and objective lenses there are a polarized beam splitter and a 14 l wave plate, which makes it possible to discriminate the refractive beam from the incident beam. This is because the wave plate makes the beam refracted from the object polarized into a right angle to that of the light from the source. Therefore, all the light refracted from the object is transmitted by the beam splitter, then enters a collective lens and is eventually focused on the photodiode through a slit. The role of the scan lens and the tube lens is to expand and collimate the beam so that the objective lens aperture is filled with the beam. To fulfill this condition, the diameters of the two beams passing through the lenses should satisfy M¼

Dt Ds

where the size of the beam incident to the scan lens is Ds and the incident to the tube lens is Dt , which is another way of expressing M. The role of the AOD here is to make the position of the focused beam vary along the vertical axis (y-axis), as can be seen from the coordinate system attached to the object. This phenomenon results from the fact that AOD deflects the incident beam to a desired angle with which the deflected beam is incident to the objective lens, thus making the resulting beam focused on a point on the vertical axis different from that which can be obtained without any deflection. Again, the deflection angle of the AOD ud , is determined by Equation 7.34. In order to vary the scanning position we simply change the frequency of the acoustic wave or the RF according to Equation 7.34. Although, in the above, scanning is explained only in one dimension, the same principle can be applied to the case of scanning, the x-y plane in 2D. It is noted that the intensity of the beam collected on the sensor depends on how accurately focused the beam is on the object. This necessitates the need of the feedback control of the object position in an axial direction. Nipkow Disk The operating principle of TSOM may be easily understood in more detail from Figure 7.51. The system employs the Nipkow disk which is an opaque disk containing many thousands of pinholes drilled or etched in spiral patterns. The use of many interlacing spiral patterns is to provide

Optomechatronic Systems in Practice Nipkow disk light source

519

eyepiece

W

perforated pinholes collective lens mirror

mirror

mirror mirror

beam splitter objective object

(a) optical arrangement

(b) Nipkow disk made of thousands of pin holes

FIGURE 7.51 Tandem scanning optical microscope (TSOM).

the confocal optical microscope with the raster scan. Typically, the diameter of the pinhole is of the order of a few tens of mm and spaced at ten pinhole diameters. The pinhole spacing is made to ensure that there is no interaction between the images on the object formed by the individual pinholes. As shown in the figure, the optical arrangement is made in such a way that a collective lens focuses the incoming light on to the pinholes on a portion of the rotating disk. This beam from the source illuminates and passes through several thousands holes located in the left-hand side of the figure. The light then travels via several optics and again is focused by a collective lens. Each point of light reflected or scattered by the sample is focused by the same lens. The focused light returns through a conjugate set of pinholes located on the opposite side of the disk. This light can be seen in the eyepiece. From the discussions on the confocal principle shown in Figure 7.45, it is noted that only the light reflected from the focused region of the object returns through the pinholes via the optical agreement, while the light from the defocused part of the object does not. In effect, this is equivalent to achieving several thousand confocal microscopes all returning in parallel in a real-time manner. It is apparent that performance of the TSOM depends upon the diameter ðdÞ spacing of the pinholes ðsÞ; and rotation speed of the disk ðVÞ: The pinhole size certainly influences the range (depth) resolution while its spacing affects the interaction between the light reflected from the object through the individual holes. Too close spacing will make the reflected lights interfere with each other. The rotation speed influences the image capturing rate.

520

Optomechatronics

Too slow rotation speed will not yield benefits of processing the images at high speed. The performance influenced by these factors is further limited by the mechanical precision required for disk fabrication, alignment between the input and output pinholes, optical complexity which requires a large number of optical components and low light efficiency. The stage carrying the sample needs to be operated at high positioning accuracy and speed to have high resolution and high measurement throughput. Positioning accuracy is determined by the accuracy of the constructed stages and that of the position controller. The accuracy of the stage itself is determined by fabrication accuracy and its alignment with guide systems such as lead screw and bearing. The control accuracy is dependent upon the performance of the servo actuator and its control algorithm. Since performance of the control accuracy requires very high precision, the control algorithm must be able to compensate for all the uncertainty and nonlinearity involved with the stage, servo system, and alignment system. It also must take into consideration the external loads, such as vibration from the stage itself, as well as from the ground. Since the stage control is illustrated for the case of AFM in the previous section, the discussion of the control system for the scan stage will not be treated again here. System Resolution Much of the resolution of the focused system is discussed in the section in Chapter 6, “Automatic Optical Focusing.” The important fact is that lateral and axial resolutions are limited by diffraction phenomena. As we may recall, the minimum resolvable length in the lateral direction perpendicular to the optical axis is given by Dx ¼

0:61l NAObj

for an objective lens of a confocal microscope having a numerical aperture NAObj, and a wavelength of l of the light beam. On the other hand, the resolution of axial direction is found to be expressed by using the para-axial approximation as Dz ¼

2ln ðNAObj Þ2

where n is the refractive index of the immersing medium adjacent to the objective lens. Another way of describing the axial resolution is to use the sharpness of the axial response of the image. Based on a high aperture theory for a plane refractive object, axial intensity is found to vary with normalized axial position as shown in Figure 7.52a. It shows the dependency of the axial response of the confocal system of the dry objective lens. In the figure, the vertical axis shows normalized intensity (I) while the horizontal axis

FWHM

–2

–1

0

1

normalized axial position

521

normalized resolution


2 z /l

(a) plane object

axial resolution (plane)

1st Airy disc normalized radius (rn)

(b) axial resolution vs. circular aperture

FIGURE 7.52 Axial resolution of the confocal system.

indicates the axial position normalized with respect to the wavelength of the illuminating light. The measure of the axial resolution is indicated by the value of FWHM in the figure. The FWHM is the full width intensity at which the light power becomes one half of the maximum power. The smaller value of FWHM indicates a shaper response curve. It is found that, as NA becomes higher, the response from a planar object gets sharper, which indicates higher axial resolution. That is, the width of the response decreases with the increase of NA. However, the signal intensity which is the area integrated under the curve becomes smaller with higher NA. Figure 7.52b depicts the normalized confocal axial resolution for circular confocal aperture size. The normalized radius, rn is expressed by rn ¼ ð2pa=lÞsin u where a is the radius of the aperture. As can be seen from the figure, the normalized axial resolution drops sharply near the Airy disk indicated by a vertical line as the confocal aperture increases. Since signal intensity increases with the aperture radius, selection of the confocal aperture therefore requires a trade-off between resolution and intensity in the axial direction. From this point of view an optimal aperture size of the pinhole needs to be properly determined. Focus Measure and Control We have dealt with the autofocusing problem in the previous chapter, in the sections “Automatic Optical Focusing” and “Visual Autofocusing,” in which autofocusing has been achieved based on feedback control of various focus measures. In optical autofocusing discussed in Chapter 6, “Automatic Optical Focusing,” the principle of aberration called “astigmation” was employed in determining a focus measure. In “Visual Autofocusing” in Chapter 6, we

522

Optomechatronics

discussed several focus measures for an online focus control that utilizes the characteristics of the image intensity of the object to be focused. These can be basically utilized for this confocal microscope unless the time for computing the focus measure based on measurement is considerably long or resolution of the detector such as CCD camera and photodetector is relatively low in comparison to that required by the system. Here, we will briefly discuss some other factors associated with visual autofocusing that have not been discussed previously. The focus measures denoted in Equation 6.78 to Equation 6.79 are not directly comparable, since they are not of the same scale in the same quantity. To this end, focus assessment is needed in order to determine which focus measure is the best among a variety. The so-called “progressive sharpness” is one such assessment determined by Ff ¼ Ak

3 X k¼1

ldi 2 dj lk

for

i–j

ð7:36Þ

where di and dj are the two intersecting distances giving an equal focus measure in the central lobe of a focus measure, k represents the index denoting the equidistant focus level from the maximum value, and A is the normalized constant. For example, if the maximum value is assigned 1.0 at k0 ¼ 0, the focus measures are 0.75 at k1 ¼ 1, 0.5 at k2 ¼ 2, 0.25 at k3 ¼ 3, and so on. Another assessment is to evaluate the robustness of the measure relative to noise. The assessment measure can be determined by Ff ¼

DF=Dz F

ð7:37Þ

where F is the focus measure defined in Equation 6.79, and Dz is the distance from the sample. These measures are usually computed within a predefined window size. Focus control is the problem in adjusting the sample distance. The controller will drive the stage in the direction required to maximize the value. This is equivalent to find the best location at each instant of time which gives the maximum value of the measure. In order to achieve this, the control system needs to move the stage and then evaluate whether the focus measure increases or decreases can determine this. The hill climbing method is one of the effective methods as we have used before. Figure 7.53 illustrates one typical method of focus control for the confocal microscope in order to obtain the desired focus. It utilized a focus value of the image obtained from the sample plane. Based on the error occurring between the desired and actual values, the controller computes the drive signal in order to position the z-stage (optical axis) at a desirable location. When PID control is adopted, the control drive signal utilizes the same control law given previously. Here, eðtÞ is given by eðtÞ ¼ Ffd 2 Ff ðtÞ


523

beam splitter

z

laser

piezoactuator

object

driver

pinhole

controller focus value

detector

–

+ desired focus value

FIGURE 7.53 Focus control of the confocal system based on the focus measure value.

where Ffd is the desired focus value and Ff ðtÞ is the value obtained at time t, either from Equation 7.36 or Equation 7.37. If Ffd is evaluated and given prior to positioning control, the control signal can be carried out within each sampling period. The positioning step size for each control step is different for the two control strategies; rough positioning and fine positioning. In rough positioning, the actuation mechanism moves the stage in large increments, when far from focus, while the fine mechanism moves it in much smaller steps when close to the focus point. In fine positioning, the increment of the moving z-stage at each step should be far less than submicron, , mm. Figure 7.54 depicts a typical image of a sample obtained by this principle

2.10 [μm] A′

0.90 775.00

750.00

1.50

0.30 –0.30

A 0.00

–0.90 0.0

FIGURE 7.54 Surface topology measured by a confocal microscope. Source: Company presentation, Nano Focus AG Co. Ltd.

155

310

465

620

775 [μm]

524

Optomechatronics

where an autofocusing control scheme is utilized. It shows an nm scanning accuracy.

Projection Television MEMs-based projection displays stem from the principle of optomechatronics in that the controller and microdevices are interacted with incoming light according to the input video schedule of light modulation. The interaction occurs when light is steered as desired by means of the control of micromirrors. There are two MEMs-based approaches in the control of light for projection display; reflective display and diffractive display. The reflective type shown in Figure 7.55a utilizes a large 2D array of micromirrors fabricated monolithically over a single integrated circuit that reflect light according to their controlled surface angle. The mirrors in the array are all controlled by individual electro-static actuators to various positions. The diffractive type shown in Figure 7.55b, on the other hand, uses a linear array of diffraction gratings which is a much smaller device than the reflective 2D arrays. One pixel of the image is composed of six deformable beams and all beams are actuated by the undivided electrostatic actuators (Here only four beams are shown). When actuated, incoming light diffracts at a certain specified diffraction angle, otherwise stays reflective without diffraction as shown in Figure 7.55b. Let us now study some of the details of silicon nitride beam

A

1

mirror electrode

yoke address pad

2

3

4

A'

address pad

silicon dioxide supports

to projector θ

θ

θ

mirror

to projector

θ deflected beam state solid beam array

(a) digital micro mirror device FIGURE 7.55 Optomechatronic MEMS-based projection display.

(b) grating light value


525

the projectors such as operation principle, system configuration, and control of the dynamics of the mirrors and beams that interact with light. Digital Micromirror Device The digital micromirror device (DMD) is a device that makes projection displays by converting white light illumination into full-color images. This is achieved by using spatial light modulators with independently addressable micromirrors. The device was invented by Texas Instruments in 1987, which is the outgrowth of work that began a decade earlier on micromechanical, analog light modulators [26]. Figure 7.56 shows a picture illustrating the configuration of the DMD display system. It consists of a display light processing (DLP) board, a DMD, a light source, a RGB color filter, optical units, a projection lens, and a screen. The board (chip) contains tiny micromirrors, memory, and processor. The device is a reflective spatial modulator composed of an array of rotatable monolithically fabricated aluminum micromirrors, and has anywhere from 800 to more than 1,000,000 mirrors, depending on the size of the array. These mirrors work as pixels in display as can be seen from Figure 7.57. The mirrors can be rotated within ^ 108 by mechanical stops which are digitally operated in two modes, on or off. The ^ 108 rotation angles are converted into high contrast brightness variations by use of a dark-field projection system.

FIGURE 7.56 A picture of the configuration of the DMD display system. Source: DLP projector overview, Texas instrument Co. Ltd.

526

Optomechatronics

pixels on screen

projection lens l ight absorber

light source

–10

+10 DMD micro mirrors

FIGURE 7.57 Operation of two micromirrors.

The working principle of the DMD display can be described with the aid of Figure 7.56 and Figure 7.57. When white light from a source enters, a condenser lens collects the light from a source which is forced down onto a color wheel filter. This rotating wheel combines the red, green, and blue video signals in sequence, depending on the content of the coded video data generated by the chip. These signals are sent to the array of mirrors. When a mirror is rotated to ^ 108 “on state,” it reflects the incoming light into the pupil of the projection lens. The lens then produces an enlarged image of each DMD mirror on a projection screen. In this case, the mirror appears bright at the projection screen. A mirror rotated to 2 108 or “off” state reflects the incoming light to a light absorber, which indicates that the light misses the pupil of the projection lens. Contrary to the previous case, the mirror appears dark at the projection screen. In more detail, when light is incident to the two mirrors, the left mirror directs it through the projection lens and on to the screen, generating one square white image. The right mirror, however, does not direct the light through the lens, projecting it to a light absorber shown in the right-hand side of the figure. This case produces a square dark image on the designated part of the screen. Architecture of a DMD Cell Figure 7.58 illustrates an exploded view of a structure of a DMD mirror basically composed of a mirror, a yoke and hinge mechanism, a metal threelayer and CMOS static RAM cell. The mirror is connected to an underlying yoke by a support post and suspended over an air gap by two mechanically


527

mirror

landing tips mirror address electrode

torsion hinge yoke

mirror−10dug mirror+10dug

yoke address electrode bias/reset bus

landing sites

hinge

yoke

spring tip

CMOS substrate

CMOS memory

(a) exploded view

(b) assembled view

FIGURE 7.58 The structure of a DMD mirror. Source: Van Kessel, P.F. et al. Proceedings of IEEE, 86:8, 1998, q 2005 IEEE.

operated torsional hinges. This enables the yoke to freely rotate within a specified angle which is in this case 2108 # u # 108: The limit of rotation is achieved by a mechanical stop, the landing tips of the yokes make a stop at the landing sites of the metal three layer. Underlying the mirror are a pair of address electrodes, mirror address and yoke address, which are connected to the CMOS memory layer. The mirror and yoke are electrically and mechanically connected to a bias reset bus built in the metal three layer, one of the three motorization layers. This bus interconnects the mirror and yoke to a bond pad at the chip parameter. A simplified cross-sectional view of the assembled structure of the DMD mirror is shown in Figure 7.59a. The rotational motion of the mirror, u, is provided by the torque electrostatically generated by voltage control. This torque acts upon the DMD unit cell composed of the mirror, the substructure including yoke, hinge unit, and electrode. The mirror is electrically grounded through its support post which is electrically connected when external voltage is applied. This voltage is called bias voltage. When a voltage is applied to the electrode in the right-hand side, due to the induced electrostatic force, the mirror tilts toward the right landing site (not shown here), until it reaches the mechanical limit. This results in a þ 108 “on” state, inclined at 108 to the horizontal plane. When a voltage is applied to the left side of the electrode, the mirror reaches another mechanical limit, yielding a tilting angle 2 108 “off” state. As mentioned above, the motion of the DMD exhibits binary behavior by choosing either one of the two stable positions, þ 108 or 2 108, depending on the magnitude of the applied voltage ðV1 ; V2 Þ to the electrodes of both sides and bias voltage Vb applied to the mirror. The tilt motion should be generated in a desired fashion for switching and therefore there needs to

528

Optomechatronics incident light

reflected light

mirror address electrode

address electrode address pad

yoke

address pad

(a) geometry of the DMD incident light reflected light

mirror mirror

mirror

(b) An illustration of mirror rotation in sequence FIGURE 7.59 Geometry and rotational motion of the DMD.

be a strategic (logical) operation of the three voltages to achieve such motion. The method involved with the operation of applying the three voltages will be briefly described by use of Figure 7.59b. Three figures illustrate a series of sequential motion, composed of three different states of tilt motion. Let the voltage V1 and V2 be applied at the left-hand electrode and right-hand electrode, respectively. Whenever they satisfy the following condition V1 2 Vb , V2 2 Vb the tilting motion will begin in a clockwise direction. When at some instant the applied voltage is such that V1 2 Vb p V2 2 Vb the mirror will continue to tilt rapidly. Finally, when the voltage difference between the two is decreased, V1 2 Vb , V2 2 Vb the mirror will have a much smoother motion than before to be ready to land at the right-hand side landing site. Based on the foregoing discussions, we can see that there may be a variety of logic for applying the voltages V1 ; V2 ; and Vb in order to produce an electrode torque that can generate the desired tilting motion. The electrostatic force generating this torque is found to primarily depend on the voltage difference across the air gap between the mirror and its


529

electrode, V, the air gap at a given element area, h and 10 is the permittivity of air. The electrostatic pressure, that is, the infinitesimal electrostatic force per unit area exerted on an area dA of the mirror or yoke is given by dF 1 V ¼ 10 dA 2 h

2

ð7:38Þ

where 10 is given by 8.54 £ 10212 F/m. From Figure 7.60, the electrostatic torque acting on the mirror is written as ð dF dA ð7:39Þ t¼ x dA Substitution of Equation 7.38 into Equation 7.39 yields,

t¼

ððð 1 V x10 h A 2

2

dA

ð7:40Þ

where x is the distance from the rotation axis to the applied infinitesimal force, and y is the coordinate perpendicular to the x-axis. If the width of the electrode is assumed to be uniform, and further, tilting angle u is assumed to be small, Equation 7.40 can be rewritten as ( ) ð‘ x1 w V2 0 t¼ dx ð7:41Þ 2 ðho 2 xuÞ2 0 where w and ‘ are the width and length of the address electrode, respectively, and ho is the original gap between the mirror and the electrode. The above relationship implies that, even if V is held constant, the electrostatic force is varied due to rotation of the mirror. According to this phenomenon, when a mirror approaches the land site of the þ 108 side, it experiences much larger torque than a mirror in other positions except the other region, 2 108 side, as can be seen from Equation 7.41. To obtain a good quality of projection image the mirrors should not provide any undesirable motion to the optical path. This means that the angular motion of the mirrors u must have good dynamic characteristics ∆F electrostatic force

x

mirror

x

θ h0

address electrode yoke address pad FIGURE 7.60 Geometry to calculate the electrostatic torque.

address pad

address electrode

530

Optomechatronics

when they are switched from one state to the other. The motion should be fast, exhibiting small overshoot, no steady-state error, and no oscillation at the steady state. We therefore need to consider the mirror dynamics for the optical system design. If the mirror system has parameters associated with dynamics such that the moment of inertia is I, tensional spring constant is kt , damping inherently possessed by the microstructure is b, the dynamic equation governing the tilting motion of the mirror (one pixel) is given by [11] I

d2 u du þ kt u ¼ t þb dt dt2

ð7:42Þ

Here, it is assumed that there exist no other inputs except torque. Equation 7.42 represents a nonlinear dynamic equation. Because the dynamics is rather complex due to t, we will approximate t by expanding it into a Taylor series expansion about u0 : The torque in Equation 7.41 then can be rewritten as;

t ðV; uÞ ¼ t ðV; u0 Þ þ þ

dt ðV; uÞ dt

d2 t ðV; uÞ dt2

u¼u0

u¼u0

ðu 2 u0 Þ

ðu 2 u0 Þ2 þ high order terms in ðu 2 u0 Þ

when u0 ¼ 0; we can express the above equation as

t ðV; uÞ ¼

1 1 0 w ‘2 V 2 1 10 w‘3 V 2 3 1 0 w ‘4 V 2 2 u þ u þ ··· þ 4 3 8 h20 h30 h40

ð7:43Þ

If we substitute this equation into Equation 7.42 and retain up to the first order term in u, then the resulting equation is expressed by I

d2 u du þ kt u ¼ t^ ðV; uÞ þb 2 dt dt

ð7:44Þ

where t^ ðV; uÞ is given by

t^ ðV; uÞ

Optomechatronics: Fusion of Optical and Mechatronic Engineering

Micro-Optomechatronics (Optical Science and Engineering)

Optical fiber fusion splicing

Micro-optomechatronics

Handbook of Optical Engineering

Modern Optical Engineering: The Design of Optical Systems (Optical and Electro-Optical Engineering Series)

Electro-optical Displays (Optical Science and Engineering)

Optical Methods of Engineering Analysis

Physics of Optoelectronics (Optical Science and Engineering)

Optical Fiber Fusion Splicing (Springer Series in Optical Sciences)

Modern optical engineering

Modern Optical Engineering

Modern optical engineering

Modern optical engineering

Fusion

Fusion

Fusion

Microwave and RF Engineering (Microwave and Optical Engineering)

Computational Methods for Electromagnetic and Optical Systems (Optical Engineering)

Modern Optical Engineering. The Design of Optical Systems

Fusion

Fusion

Fusion

Fusion

Numerical Simulation Of Mechatronic Sensors And Actuators

Fusion

Fusion

Fusion

Fusion

Mechatronic Systems, Sensors, And Actuators

Fusion

Optomechatronics: Fusion of Optical and Mechatronic Engineering

Micro-Optomechatronics (Optical Science and Engineering)

Optical fiber fusion splicing

Micro-optomechatronics

Handbook of Optical Engineering

Modern Optical Engineering: The Design of Optical Systems (Optical and Electro-Optical Engineering Series)

Electro-optical Displays (Optical Science and Engineering)

Optical Methods of Engineering Analysis

Physics of Optoelectronics (Optical Science and Engineering)

Optical Fiber Fusion Splicing (Springer Series in Optical Sciences)

Modern optical engineering

Modern Optical Engineering

Modern optical engineering

Modern optical engineering

Fusion

Fusion

Fusion

Microwave and RF Engineering (Microwave and Optical Engineering)

Computational Methods for Electromagnetic and Optical Systems (Optical Engineering)

Modern Optical Engineering. The Design of Optical Systems

Fusion

Fusion

Fusion

Fusion

Numerical Simulation Of Mechatronic Sensors And Actuators

Fusion

Fusion

Fusion

Fusion

Mechatronic Systems, Sensors, And Actuators

Fusion

Recommend Documents