CARRIER ETHERNET Providing�the Need�for�Speed
AU6039.indb 1
2/13/08 9:19:35 AM
OTHER TELECOMMUNICATIONS BOOKS FROM AUERBACH Active and Programmable Networks for Adaptive Architectures and Services Syed Asad Hussain ISBN: 0-8493-8214-9 Ad Hoc Mobile Wireless Networks: Principles, Protocols and Applications Subir Kumar Sarkar, T.G. Basavaraju, and C. Puttamadappa ISBN: 1-4200-6221-2
Introduction to Mobile Communications: Technology, Services, Markets Tony Wakefield, Dave McNally, David Bowler, and Alan Mayne ISBN: 1-4200-4653-5 Millimeter Wave Technology in Wireless PAN, LAN, and MAN Shao-Qiu Xiao, Ming-Tuo Zhou, and Yan Zhang ISBN: 0-8493-8227-0
Comprehensive Glossary of Telecom Abbreviations and Acronyms Ali Akbar Arabi ISBN: 1-4200-5866-5
Mobile WiMAX: Toward Broadband Wireless Metropolitan Area Networks Yan Zhang and Hsiao-Hwa Chen ISBN: 0-8493-2624-9
Contemporary Coding Techniques and Applications for Mobile Communications Onur Osman and Osman Nuri Ucan ISBN: 1-4200-5461-9
Optical Wireless Communications: IR for Wireless Connectivity Roberto Ramirez-Iniguez, Sevia M. Idrus, and Ziran Sun ISBN: 0-8493-7209-7
Context-Aware Pervasive Systems: Architectures for a New Breed of Applications Seng Loke ISBN: 0-8493-7255-0
Performance Optimization of Digital Communications Systems Vladimir Mitlin ISBN: 0-8493-6896-0
Data-driven Block Ciphers for Fast Telecommunication Systems Nikolai Moldovyan and Alexander A. Moldovyan ISBN: 1-4200-5411-2
Physical Principles of Wireless Communications Victor L. Granatstein ISBN: 0-8493-3259-1
Distributed Antenna Systems: Open Architecture for Future Wireless Communications Honglin Hu, Yan Zhang, and Jijun Luo ISBN: 1-4200-4288-2
Principles of Mobile Computing and Communications Mazliza Othman ISBN: 1-4200-6158-5
Encyclopedia of Wireless and Mobile Communications Borko Furht ISBN: 1-4200-4326-9
Resource, Mobility, and Security Management in Wireless Networks and Mobile Communications Yan Zhang, Honglin Hu, and Masayuki Fujise ISBN: 0-8493-8036-7
Handbook of Mobile Broadcasting: DVB-H, DMB, ISDB-T, AND MEDIAFLO Borko Furht and Syed A. Ahson ISBN: 1-4200-5386-8
Security in Wireless Mesh Networks Yan Zhang, Jun Zheng, and Honglin Hu ISBN: 0-8493-8250-5
The Handbook of Mobile Middleware Paolo Bellavista and Antonio Corradi ISBN: 0-8493-3833-6
Wireless Ad Hoc Networking: Personal-Area, Local-Area, and the Sensory-Area Networks Shih-Lin Wu and Yu-Chee Tseng ISBN: 0-8493-9254-3
The Internet of Things: From RFID to the Next-Generation Pervasive Networked Systems Lu Yan, Yan Zhang, Laurence T. Yang, and Huansheng Ning ISBN: 1-4200-5281-0
Wireless Mesh Networking: Architectures, Protocols and Standards Yan Zhang, Jijun Luo, and Honglin Hu ISBN: 0-8493-7399-9
AUERBACH PUBLICATIONS www.auerbach-publications.com To Order Call: 1-800-272-7737 • Fax: 1-800-374-3401 E-mail:
[email protected] AU6039.indb 2
2/13/08 9:19:35 AM
CARRIER ETHERNET Providing�the Need�for�Speed
GILBERT HELD
Boca Raton London New York
CRC Press is an imprint of the Taylor & Francis Group, an informa business
AN AUERBACH BOOK
AU6039.indb 3
2/13/08 9:19:35 AM
Auerbach Publications Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487‑2742 © 2008 by Taylor & Francis Group, LLC Auerbach is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Printed in the United States of America on acid‑free paper 10 9 8 7 6 5 4 3 2 1 International Standard Book Number‑13: 978‑1‑4200‑6039‑3 (Hardcover) This book contains information obtained from authentic and highly regarded sources Reason‑ able efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The Authors and Publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www. copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC) 222 Rosewood Drive, Danvers, MA 01923, 978‑750‑8400. CCC is a not‑for‑profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging‑in‑Publication Data Held, Gilbert, 1943‑ Carrier Ethernet : providing the need for speed / Gilbert Held. p. cm. ISBN 978‑1‑4200‑6039‑3 (hardback : alk. paper) 1. Ethernet (Local area network system) 2. Metropolitan area networks (Computer networks) I. Title. TK5105.8.E83H448 2008 004.6’8‑‑dc22
2007049111
Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the Auerbach Web site at http://www.auerbach‑publications.com
AU6039.indb 4
2/13/08 9:19:36 AM
Dedication One of the advantages associated with living in a small town for almost 30 years is the commute to work. After having lived in New York City and the suburbs of Washington, D.C., moving to Macon, Georgia, provided me with over ten hours per week of additional time that I could devote to writing manuscripts and preparing presentations. Over the past 30 years that I have lived in Macon, I was fortunate to be able to teach over a thousand graduate students locally and perhaps ten thousand or more students who came to various seminars I taught throughout the United States, Europe, Israel, and South America. Many of those students were highly inquisitive and their questions resulted in a mental exercise for this old professor as well as second, third, and even fourth editions of some of the books I authored. In recognition of the students who made teaching truly enjoyable, this book is dedicated.
AU6039.indb 5
2/13/08 9:19:36 AM
AU6039.indb 6
2/13/08 9:19:36 AM
Contents Preface............................................................................................................ xv Acknowledgments........................................................................................xvii About the Author..........................................................................................xix
1
Introduction to Carrier Ethernet............................................................1 Defining Carrier Ethernet.............................................................................1 Overview..................................................................................................2 Rationale..................................................................................................2 Expanded Use of Ethernet LANs.........................................................2 Frame Compatibility...........................................................................3 Low Cost.............................................................................................3 High Network Access Speeds..............................................................4 Mass Market for Technology...............................................................4 Business Continuity.............................................................................4 Enabling Technologies..................................................................................5 Copper and Fiber Infrastructure..............................................................5 ADSL..................................................................................................6 ADSL2 and ADSL2+..........................................................................6 SHDSL................................................................................................8 VDSL..................................................................................................8 VPNs.......................................................................................................9 Types of VPNs.....................................................................................9 Protocols..............................................................................................9 Service Provider Provisioned VPNs...................................................10 VLANs..................................................................................................11 Broadcast Domain Reduction............................................................12 Facilitate Subnet Creation..................................................................12 Reduce Hardware Requirements.......................................................13 Traffic Control...................................................................................13 Types of VLANs................................................................................13 vii
AU6039.indb 7
2/13/08 9:19:36 AM
viii n Contents
MPLS.....................................................................................................13 Overview...........................................................................................14 Architecture.......................................................................................14 Operation..........................................................................................14 Applications................................................................................................17 Interconnecting Distributed Offices.......................................................17 Providing Real-Time Backup.................................................................17 Voice, Video, and Data Support.............................................................17 Challenges to Carrier Ethernet...................................................................18 Total Cost of Operation.........................................................................18 Packet Overhead....................................................................................18 Management and Troubleshooting.........................................................19 Reliability...............................................................................................19 Security..................................................................................................20 QoS........................................................................................................20
2
AU6039.indb 8
Data Networking Concepts...................................................................21 Transport Technologies...............................................................................21 LANs.....................................................................................................22 WANs....................................................................................................22 Characteristics...................................................................................23 Wireless.............................................................................................23 Data Protocols............................................................................................25 Ethernet.................................................................................................25 Evolution...........................................................................................25 IEEE Involvement.............................................................................26 Network Interfaces.....................................................................................29 Network Equipment...................................................................................29 Network Interface Cards........................................................................30 Hubs......................................................................................................30 Operation..........................................................................................31 Passive versus Intelligent Hubs...........................................................31 Switches.................................................................................................32 Operation..........................................................................................32 Advantages........................................................................................32 Evolution...........................................................................................33 Routers...................................................................................................33 Operation......................................................................................... 34 Advantages....................................................................................... 34 Capabilities....................................................................................... 34 Firewall..................................................................................................35 Placement..........................................................................................35 Operation..........................................................................................36
2/13/08 9:19:37 AM
Contents n ix
VPN Appliances.....................................................................................36 Operation..........................................................................................36 Advantages........................................................................................37 Combining Functions........................................................................37 Network Facilities.......................................................................................37 T1..........................................................................................................37 The DSO Time Slot...........................................................................38 T-Carrier Hierarchy...........................................................................38 Channelized versus Non-Channelized...............................................38 SONET.................................................................................................39 Optical Carrier Levels........................................................................39 Framing.............................................................................................41 Utilization.........................................................................................43
3
AU6039.indb 9
The Flavors of Ethernet.........................................................................45 Metcalfe’s Original Design.........................................................................45 Bus-Based Network Structure............................................................... 46 The DIX Standard................................................................................. 46 DIX Version 2.0............................................................................... 46 IEEE 802.3 Standardization.......................................................................48 Division of Effort...................................................................................48 Physical Layer Effort..........................................................................48 Network Layer Effort.........................................................................51 Data Link Layer.................................................................................51 IEEE Changes from DIX.......................................................................53 802.3 Frame Format..........................................................................53 Sub-Network Access Protocol................................................................54 The CSMA/CD Protocol.......................................................................54 Frame Size..............................................................................................54 Early Ethernet.............................................................................................55 The 10 Mbps Ethernet Family................................................................55 10BASE-5..........................................................................................55 10BASE-2..........................................................................................56 10BROAD-36...................................................................................56 10BASE-T.........................................................................................56 Network Characteristics....................................................................59 5-4-3 Rule.........................................................................................59 FOIRL and 10BASE-F......................................................................59 Fast Ethernet............................................................................................. 60 100BASE-T........................................................................................... 60 Layer Subdivision............................................................................. 60 100BASE-TX.........................................................................................62 Network Configuration.....................................................................63
2/13/08 9:19:37 AM
n Contents
Coding..............................................................................................63 Repeaters...........................................................................................63 100BASE-T4......................................................................................... 64 100BASE-T4 Repeater Hub...............................................................65 100BASE-T2......................................................................................... 66 Auto-Negotiation.................................................................................. 66 LIT Pulses........................................................................................ 66 FLP Pulses.........................................................................................67 Parallel Detection Function...............................................................67 The Base Page....................................................................................67 The Next Page Function....................................................................69 Extended Next Page Function...........................................................71 Priorities............................................................................................73 Option Considerations......................................................................73 Fiber.......................................................................................................74 100BASE-FX.....................................................................................74 100BASE-SX.....................................................................................75 100BASE-BX.....................................................................................75 Gigabit Ethernet.........................................................................................75 Fiber-Based Gigabit Ethernet.................................................................76 1000BASE-SX...................................................................................76 1000BASE-LX...................................................................................76 Fiber Auto-Negotiation.....................................................................76 1000BASE-ZX and LH.....................................................................78 Copper-Based Gigabit Ethernet..............................................................78 1000BASE-CX..................................................................................78 1000BASE-T.....................................................................................78 Summary...............................................................................................79 10 Gigabit Ethernet................................................................................... 80 GbE versus 10 GbE............................................................................... 80 Layers and Interfaces..............................................................................81 XGMII..............................................................................................81 XAUI.................................................................................................82 XGMII..............................................................................................82 MAC.................................................................................................83 PCS...................................................................................................83 PMA..................................................................................................83 PMD.................................................................................................83 WAN Physical Layer..........................................................................83 10 GbE over Copper............................................................................. 84 10GBASE-CX4.................................................................................85 10GBASE-T......................................................................................85
AU6039.indb 10
2/13/08 9:19:37 AM
Contents n xi
Ethernet in the First Mile...........................................................................87 Architectures..........................................................................................87 Physical Layer Interfaces....................................................................88 Applications...........................................................................................89 Advantages........................................................................................89 Use of Dual Fibers............................................................................ 90 Use of Single Fibers...........................................................................91 EPON...............................................................................................91 MPCP...............................................................................................94
4
AU6039.indb 11
Frame Formats.......................................................................................99 Basic Ethernet...........................................................................................100 The Ethernet II/DIX Frame.................................................................100 Preamble Field.................................................................................100 Destination Address Field................................................................100 Source Address Field........................................................................101 Type Field........................................................................................101 Data Field........................................................................................102 Frame Check Sequence Field...........................................................102 The 802.3 Frame..................................................................................103 Length Field....................................................................................103 Preamble Field Modification............................................................103 Type/Length Field Values................................................................103 The 802.2 Header................................................................................104 Subnetwork Access Protocol............................................................104 LLC Header Operation...................................................................105 The SNAP Frame.............................................................................105 IPX over Ethernet............................................................................106 Full Duplex and the Pause Frame..........................................................107 Advantages...........................................................................................108 Flow Control........................................................................................108 PAUSE Frame..................................................................................108 Overview.........................................................................................109 Frame Fields....................................................................................109 VLAN Tagging........................................................................................109 The 802.1Q Standard........................................................................... 110 Advantages........................................................................................... 110 Frame Format....................................................................................... 110 SNAP Frames....................................................................................... 111 Frame Determination........................................................................... 111 Fast Ethernet............................................................................................ 111 4B5B Coding.......................................................................................112
2/13/08 9:19:38 AM
xii n Contents
Delimiters............................................................................................112 Interframe Gap.................................................................................... 114 Gigabit Ethernet....................................................................................... 114 Carrier Extension................................................................................. 114 Half-Duplex Use.............................................................................. 115 Frame Bursting.................................................................................... 115 Jumbo Frames...................................................................................... 116 Operation........................................................................................ 116 Length Rationale............................................................................. 116 Advantages...................................................................................... 117 Problems and Solutions.................................................................... 117 Performance.............................................................................................. 118 Basic Ethernet...................................................................................... 118 SNAP Frames....................................................................................... 119 Gigabit Ethernet................................................................................... 119 Frame Rates.........................................................................................120 Mathematical Oddities....................................................................121 Frame Rate Computations...............................................................122 Gigabit Constraints.........................................................................124
5
AU6039.indb 12
LAN Switches......................................................................................127 Bridge Operations.....................................................................................127 Transparent and Translating Bridges....................................................127 Plug-and-Play Operation.................................................................128 Bridge Operation.............................................................................128 Intelligent Switching Hubs...................................................................131 Basic Components...........................................................................131 Buffer Memory................................................................................131 Delay Times....................................................................................131 Parallel Switching.................................................................................133 Switch Operations....................................................................................133 Switching Techniques...........................................................................134 Cross-Point Switching.....................................................................134 Operation........................................................................................134 Latency............................................................................................135 Store-and-Forward...............................................................................135 Filtering Capability..........................................................................135 Operation........................................................................................135 Delay Time......................................................................................136 Hybrid.................................................................................................137 Switch Port Address Support................................................................137 Port-Based Switching.......................................................................137
2/13/08 9:19:38 AM
Contents n xiii
Segment-Based Switching................................................................138 Applications.....................................................................................139 Considering Port Capability.................................................................140 Basic Switching................................................................................141 Multi-Tier Networking....................................................................141 Interconnecting Dispersed Offices...................................................142 Virtual LANs............................................................................................143 Characteristics......................................................................................143 Construction Basics.............................................................................143 Implicit versus Explicit Tagging...........................................................144 Using Implicit Tagging........................................................................144 Explicit Tagging................................................................................... 145 The IEEE 802.1Q Standard............................................................. 145 Vendor Implementation................................................................... 151
6
Carrier Ethernet Services....................................................................157 Overview.................................................................................................. 157 The Metro Ethernet Forum.................................................................. 157 Requirements for Use...........................................................................158 VLAN Tagging...............................................................................158 The 802.1P (Priority) Standard........................................................160 Latency Considerations....................................................................160 Fiber Connectivity...........................................................................163 Transporting Ethernet in a Service Provider Network..............................164 Operating over Other Transports.........................................................164 Comparison to Other Layer 2 Protocols...............................................165 Ethernet Topologies.............................................................................165 Carrier Ethernet Service Types.............................................................165 E-LINE...........................................................................................166 E-LAN............................................................................................166 E-TREE...........................................................................................167 Encapsulation Techniques....................................................................167 VLAN Stacking...............................................................................168
7
Service Level Agreements and Quality of Service...............................175 The Service Level Agreement....................................................................176 Metrics................................................................................................. 176 Availability...................................................................................... 176 Latency............................................................................................181 Jitter.................................................................................................181 MTTR............................................................................................182 Installation Time.............................................................................182
AU6039.indb 13
2/13/08 9:19:38 AM
xiv n Contents
Bandwidth Provisioning..................................................................183 Packet Loss......................................................................................183 Guaranteed Bandwidth...................................................................183 SLA Problems...........................................................................................183 OAM Overview........................................................................................184 OAM and Ethernet..............................................................................184 Ethernet OAMs....................................................................................184 Functions.........................................................................................185 Testing.............................................................................................185 Link-Layer OAM.............................................................................185 Service OAM...................................................................................187 Quality of Service Overview.....................................................................187 Soft versus Hard QoS...........................................................................188 Soft QoS..........................................................................................188 Hard QoS........................................................................................189 QoS Actions......................................................................................... 191 Classification................................................................................... 191 Policing............................................................................................ 191 Queuing..........................................................................................192 Scheduling.......................................................................................192 Cisco ML-Series Card.....................................................................194 Index............................................................................................................197 Numbers Index.............................................................................................203
AU6039.indb 14
2/13/08 9:19:39 AM
Preface Similar to a fine watch, the technology behind the original Ethernet specification continues to move forward. From a 10 Mbps transmission technology, Ethernet was enhanced several times over the past 40 years. From the original 10 Mbps coaxial cable, BUS-based technology, Ethernet evolved first into a 10-Mbps twisted-wire, hub-based technology, shortly thereafter followed by Fast Ethernet, which extended the data rate to 100 Mbps. By the early 1990s Gigabit Ethernet made its appearance, which was then followed by 10 Gigabit Ethernet. Today, work is progressing on extending the data rate of Ethernet further up the Gigabit range. Although a significant portion of the preceding Ethernet technologies were oriented towards moving data over local area networks, both Gigabit and 10 Gigabit Ethernet include the ability to transmit data over optical fiber at long distances; this provides customers with the ability to interconnect buildings in a campus environment easily while communications carriers gradually began deploying the technology into their metropolitan area networks as a low-cost overlay network to provide customers with inter-site connectivity. Originally referred to as Metropolitan Area Ethernet (MAE) and today primarily referred to as Carrier Ethernet, this technology represents the focus of this book and in addition results in communications carriers providing a transmission technology that can be used to significantly enhance the data rate between customer sites. Because new technology is rarely an island, we will discuss the major components of the technology behind Carrier Ethernet prior to focusing our attention upon a detailed investigation of Carrier Ethernet services. Thus, in this book we will examine data networking concepts, the difference between so-called “flavors” of Ethernet, the Ethernet frame, and the manner by which switches operate. In addition, we will refresh our knowledge of virtual LANs (VLANs), virtual private networks (VPNs), Multi-Protocol Label Switching (MPLS), and other technologies used to provide a Carrier Ethernet service tailored to the requirements of subscribers. Using this information as a base will provide readers with a firm background in Ethernet and its relevant technologies that will allow one to obtain a maximum benefit from the portion of this book that covers Carrier Ethernet technology in detail.
xv
AU6039.indb 15
2/13/08 9:19:39 AM
xvi n Preface
Once we complete our discussion of the technology associated with Carrier Ethernet services, we will then conclude this book by turning our attention to another important topic: Service Level Agreements. Because Carrier Ethernet represents a service, we need to understand the structure of a Service Level Agreement as it represents a contract that will enhance our organization’s use of this evolving service. Because Carrier Ethernet technology deals with such important issues as obtaining a quality of service for the movement of voice and real-time video and the creation of VLANs to facilitate the movement of data, we will also discuss each of these important topics in this book. Thus, the reader of this book will be exposed to both the different versions of Ethernet and the technologies that result in the use of Carrier Ethernet being rapidly implemented by many organizations as a mechanism to interconnect separated locations in a manner that allows high data transfers at a reasonable cost. As a professional author who has spent approximately 30 years working with different flavors of Ethernet technology, I welcome reader feedback. Please feel free to write to me in care of my publisher whose address is on the jacket of this book; or you might choose to send an e-mail to
[email protected]. Because I periodically travel overseas, it may be a week or more before I can respond to specific items in the book. Please feel free also to provide your comments concerning both the material in this book as well as topics you may want to see in a new edition. Although I try my best to “place my feet” literally in the shoes of the reader to determine what may be of interest, I am human and make mistakes. Thus, let me know if I omitted a topic you feel should be included in this book or if I placed too much emphasis on another topic. Your comments will be greatly appreciated.
AU6039.indb 16
2/13/08 9:19:39 AM
Acknowledgments As the author of many books, a long time ago I realized that the publishing effort is dependent upon the work of a considerable number of persons. First, an author’s idea concerning a topic must appeal to a publisher who is typically inundated with proposals. Once again, I am indebted to Rich O’Hanley at Auerbach Publications for backing my proposal to author a book focused upon a new type of Ethernet communications. As an old-fashioned author who periodically travels, I like to use the original word processor — a pen and paper — when preparing a draft manuscript. Doing so ensures that I will not run out of battery power nor face the difficulty of attempting to plug a laptop computer into some really weird electric sockets I encountered while traveling the globe. Unfortunately, a publisher expects a typed manuscript, and Auerbach Publications is no exception. Thus, I would be remiss if I did not acknowledge the fine efforts of my wife, Beverly J. Held, in turning my longhand draft manuscript into a polished and professionally typed final manuscript that resulted in the book you are now reading. Once again, I would like to acknowledge the efforts of Taylor & Francis/Auerbach Publications employees in Boca Raton, Florida. From designing the cover through the editing and author queries, they double-checked this author’s submission and ensured that this book was ready for typesetting, printing, and binding. To all of you involved in this process, a sincere thanks.
xvii
AU6039.indb 17
2/13/08 9:19:39 AM
AU6039.indb 18
2/13/08 9:19:39 AM
About the Author Gilbert Held is an internationally recognized author and lecturer who specializes in the applications of computer and communications technology. He is a frequent lecturer and conducts seminars on topics such as LAN/WAN internetworking, data compression, and PC hardware and software. Held is the author of more than 40 books on computers and communications technology and has won several awards for his technical excellence in writing.
xix
AU6039.indb 19
2/13/08 9:19:39 AM
AU6039.indb 20
2/13/08 9:19:40 AM
Chapter 1
Introduction to Carrier Ethernet Similar to other books written by this author, the purpose of an introductory chapter is to provide readers with general information about the topic of the book. This chapter is no exception as we will commence our familiarity of Carrier Ethernet with a definition. Once this is accomplished we will discuss the rationale for this relatively new technology, briefly tour several key aspects of the technology, and then discuss some of the applications that can benefit from the use of Carrier Ethernet. Because a coin has two sides, this author would be remiss if he did not point out some of the challenges to this evolving technology. Thus, in concluding this chapter we will turn our attention to some of the challenges faced by communications carriers offering a Carrier Ethernet service as well as by end users looking to utilize this service.
Defining Carrier Ethernet Carrier Ethernet can be simply defined as “a high-speed Ethernet transport mechanism for metropolitan area networking.” Because of this, the terms “Carrier Ethernet” and “Metropolitan Area Ethernet” are often used synonymously; however, in this book we will refer to the technology as Carrier Ethernet due to the fact that it is primarily a communications carrier service offering, although it is possible for an end user to install a Carrier Ethernet infrastructure in a campus environment.
AU6039.indb 1
2/13/08 9:21:59 AM
n Carrier Ethernet: Providing the Need for Speed
Overview Carrier Ethernet defines the use of Ethernet frames as a transport facility, enabling such frames to transport IP packets or even ATM (Asynchronous Transfer Mode) cells. Because Ethernet is scalable, with 10-G Ethernet now many years old while higher data rates are on the standards horizon, the technology can be viewed as presenting a challenge to a traditional Synchronous Optical Network (SONET) telephony infrastructure. However, because SONET rings are designed for providing near-immediate recovery in the event of a cable cut or another type of communications failure, it is this author’s opinion that Carrier Ethernet will complement SONET and in many cases be carried via a SONET connection between communications carrier offices.
Rationale The advent and expansion of the use of Carrier Ethernet results from a series of inter-related issues. Those issues, which are listed in Table 1.1, will be briefly discussed in this section.
Expanded Use of Ethernet LANs The so-called “LAN wars” during the later part of the 1980s through the mid1990s are now history. During that period Ethernet battled IBM’s Token-Ring for LAN supremacy. Similar to the VHS versus Beta videotape recorder battle a decade earlier, one technology survived while the other technology was relegated to history. In the battle for LAN supremacy Ethernet won the LAN wars many years ago. Although there are still some universities, research laboratories, government agencies, and commercial organizations that operate Token-Ring networks, their days are numbered. Due to the increase in Internet access and the use of graphics in e-mails the relatively low data rate of the 16-Mbps Token-Ring network is not sufficient for most modern communications networks. Thus, operators of Token-Ring Table 1.1 Rationale for Carrier Ethernet Expanded use of Ethernet LANs Frame compatibility Low cost and high access speeds Mass market for technology Simplifies business continuity
AU6039.indb 2
2/13/08 9:21:59 AM
Introduction to Carrier Ethernet n
networks have been replacing their infrastructure with Fast Ethernet and Gigabit Ethernet LANs. Within a few years it is more than likely that the only Token-Ring networks in use will operate in museums. Today over 90 percent of all LANs are based upon Ethernet technology.
Frame Compatibility A logical evolution of the use of end-to-end Ethernet technology is to enable data to flow between locations connected via a Metropolitan Area Network (MAN) as Ethernet frames. In doing so, this action would eliminate the necessity to convert Ethernet frames into ATM cells or another transport facility and then re-convert them back into their original format. Due to the growth in the transport of realtime data conveying voice and video, the elimination of frame-to-cell-to-frame or other conversions can have a beneficial effect on the reconstruction of voice or video at their destination location. Simply put, the avoidance of conversion lowers delay time, which is a key metric in determining if a digitized voice stream can be transported and converted back to an analog format without experiencing distortion.
Low Cost Most organizations go through a budgetary process where they allocate various funds for different projects into the future. One of the projects typically budgeted in an IT environment is for network upgrades. In the original LAN wars mentioned earlier in this chapter, Ethernet won over Token-Ring for a variety of reasons, with one of the primary benefits of Ethernet being its low cost; a second key benefit was its ability to scale upward. Concerning the latter, an organization operating a legacy 10-Mbps Ethernet LAN could either upgrade the network to a 100-Mbps Fast Ethernet network or selectively use switches to connect the existing network to a backbone network operating at a much higher data rate. Similarly, a Fast Ethernet network operating at 100 Mbps could be upgraded to a Gigabit Ethernet network or the end user could selectively use Gigabit LAN switches with some Fast Ethernet ports that could be employed to connect the existing network to a faster high-speed Gigabit Ethernet backbone. These network scenarios enable data to flow end-to-end as Ethernet frames. This significantly reduces the cost associated with training network personnel as well as the cost of diagnostic equipment. In addition, because the use of LAN switches enables portions of a network to be selectively upgraded, this allows the cost associated with a network upgrade to be spread over more than one budgetary period. When we discuss the use of Carrier Ethernet to interconnect two or more locations within a metropolitan area, similar cost savings are obtainable due to the ease in connecting existing Ethernet LANs via a Carrier Ethernet service. Thus, the
AU6039.indb 3
2/13/08 9:21:59 AM
n Carrier Ethernet: Providing the Need for Speed
low cost associated with connecting LANs to a Carrier Ethernet service represents another reason for considering the use of this service.
High Network Access Speeds The ability to connect locations via Carrier Ethernet implies the transport of data at high speeds. Thus, the use of Carrier Ethernet enables locations within a metropolitan area to be connected to one another via access lines that operate at high data rates. When transporting delay-sensitive data such as real-time voice and video minimizing network ingress and egress times can be quite beneficial. A second area that deserves mention is the use of Carrier Ethernet as a replacement for lower-speed T1 and T3 transmission systems. A T1 line was originally developed to transport 24 digitized voice conversations, and by the early 1990s was primarily used as a 1.544-Mbps data pipe to connect stations on a LAN to the Internet. Similarly, the T3 transmission system was originally developed to transport 28 T1 lines, each transporting 24 digitized calls. Today, a majority of local loop T3 lines are used to provide large organizations with Internet access at a data rate approaching 45 Mbps. Through the use of Carrier Ethernet it becomes possible to obtain an access line operating at a gigabit data rate.
Mass Market for Technology A fifth driving factor behind the acceleration in the use of Carrier Ethernet is the mass market for Ethernet technology. Having won the LAN wars many years ago, Ethernet in a variety of flavors represents the dominant technology for moving data over local area networks. This results in Ethernet providing an economy of scale for developing such products as LAN switches, router ports, and network adapters. Because Carrier Ethernet is based on Ethernet, the mechanism required to connect Ethernet LANs to a carrier Ethernet service does not represent a quantum leap in technology. Instead, the connection can occur using off-the-shelf products, which enables a mass market of equipment to be usable. This in turn drives down the cost of interconnecting Ethernet LANs via a Carrier Ethernet service, resulting in the use of the service becoming more appealing.
Business Continuity Until 9/11 many small- and medium-sized corporations discussed the need for continuity of operations, but did not invest the necessary funds to achieve a high level of backup. The world changed after 9/11 and today business continuity is a major operational goal of business. Through the use of Carrier Ethernet it becomes relatively easy for one office to back up its data onto the data storage residing at another office. Thus, one of the
AU6039.indb 4
2/13/08 9:22:00 AM
Introduction to Carrier Ethernet n
Table 1.2 Technologies Enabling Carrier Ethernet Copper and fiber infrastructure VPNs VLANs MPLS
benefits obtained from the high speed provided by Carrier Ethernet is to enable off-site updates to occur in a timely fashion. In addition, organizations can use Carrier Ethernet to transmit backup data to off-site storage repositories, providing another option for business recovery that can be tailored to changing data patterns and either supplement or complement conventional backup strategies where tapes or disks are transported to an off-site storage facility. Now that we have an appreciation for a few of the driving forces contributing to the growth in the use of Carrier Ethernet, we will turn our attention to some of the technology issues that enable the relatively high data rate of this new version of Ethernet to be used effectively.
Enabling Technologies In this section we will examine a core series of relatively new technologies that enable organizations to effectively use Carrier Ethernet. Table 1.2 lists four key technologies that enable the use of Carrier Ethernet to become a viable transport technology for interconnecting locations at a high data rate within a metropolitan area.
Copper and Fiber Infrastructure Over the past decade significant improvements in the data transmission rate obtainable via copper wires occurred while many communications carriers strung fiber into buildings or to the curb where copper was used to deliver high-speed data for relatively short distances into the home. Concerning the use of copper, although conventional modems are only capable of reaching a data rate of approximately 56 Kbps, such modems only use approximately 4 KHz of the bandwidth of copperbased wiring. In actuality, the available bandwidth of twisted-pair copper wiring is over 1 MHz. However, because the telephone network was originally developed to transport voice, low and high pass filters are used to form a passband of approximately 4 KHz, limiting the ability of modems to transmit data at high speed.
AU6039.indb 5
2/13/08 9:22:00 AM
n Carrier Ethernet: Providing the Need for Speed
Voice
Upstream
4 KHz 25.875 KHz
Downstream
138 KHz
1 MHz+
Figure 1.1 ADSL frequency use on copper wire
ADSL Recognizing the availability of a significant amount of unused bandwidth on copper wiring resulted in telephone companies altering their last-mile connections to take advantage of frequencies from approximately 40 KHz to or beyond 1 MHz. In doing so, they used their current copper-based local loop, which runs from a telephone exchange to the customer presence, to become capable of transporting both voice and data. To do so, the telephone company initially installed Asymmetric Digital Subscriber Line (ADSL) modems at the customer premises and a rack-mounted Digital Subscriber Line Multiplexer (DSLAM) at the central office, with the latter serving multiple subscribers. Through the use of Frequency Division Multiplexing (FDM) the ADSL modem created two frequency bands above the voice band, enabling both voice calls and data transmission to occur simultaneously over a common copper-wire connection. Figure 1.1 illustrates the general frequency division that occurs when ADSL is implemented on a telephone copper wire. Note that the lower 4 KHz is used for voice. In comparison, the larger bandwidth devoted to data transmission supports downstream (central office to subscriber) transmission while the lower amount of bandwidth devoted to data transmission is used to support upstream (subscriber to central office) communications. This partition of upper frequency into two different sized bands results in an asymmetric data rate and is designed to support typical Internet access where short amounts of upstream transmissions in the form of URLs are followed by lengthy downstream transmissions in the form of Web pages.
ADSL2 and ADSL2+ Since the adoption of ADSL standards in 1998 there have been several enhancements to the technology, most notable being ADSL2 and ADSL2+. Table 1.3 provides a comparison of the original ADSL2 and ADSL2+. Note that the International Telecommunications Union (ITU) standard G.992.5 Annexes J and M shift the upstream/downstream frequency split from 138 to 276 KHz as a mechanism to boost upstream data rates. In addition, the “all digital-loop” variation of
AU6039.indb 6
2/13/08 9:22:02 AM
Introduction to Carrier Ethernet n
Table 1.3 Comparing Maximum Operating Rates Technology
Standard
Downstream Rate Upstream Rate (Mbps) (Mbps)
ADSL
ANSI T1.413
8
1.0
ADSL2
ITUG.992 3⁄4
12
1.0
ADSL2
ITUG.992 3⁄4 Annex J
12
3.5
RE-ADSL2
ITUG.992 3⁄4 Annex L
5
0.8
ADSL2+
ITU G.992.5
24
1.0
RE-ADSL2+
ITU G.992.5 Annex L
24
1.0
ADSL2+
ITU G.992.5 Annex M
28
3.5
ADSL2 and ADSL2+ defined in Annexes I and J, which define ADSL2+_ISDN both without and with overlapped spec- Table 1.4 Distance versus trum, support an additional 256 Kbps of Downstream Data Rate upstream data when the 4-KHz bandDistance Maximum Data Rate (feet) (Mbps) width allocated for voice is reallocated for ADSL. 1500
= 8100?
Yes
No < = 1500
DIX/Ethernet II
vLAN Tagged 802.3 Frame 2 bytes Following Length/ Type
= AA or AB SNAP Frame
= FF IPX Encapsulated
Figure 4.8 Determining the frame type
of Stream Delimiter (SSD) and End of Stream Delimiter (ESD) were used to surround the frame.
4B5B Coding The SSD results from the use of 4B5B encoding under which groups of 4 bits are mapped into groups of 5 bits. Because there are 32 possible combinations of 5 bits and 16 combinations of 4 bits, the 16 groups of 5 bits with the most transitions are used to provide as many transitions provide clocking information for the signal. Table 4.1 illustrates the manner by which groups of 4 bits can be mapped into groups of 5 bits, enabling enough transitions so that the clock signal can be recovered at a receiver. Because there are 16 “unused” characters, they can be used to detect errors for a special purpose, such as the SSD and ESD.
Delimiters Figure 4.9 illustrates the 100BASE-TX frame. Note that this frame differs from the 802.3 frame through the addition of a byte to mark the beginning and end of the frames. Because at 100 Mbps the frames are known as streams, this resulted in the names assigned to the two delimiters.
AU6039.indb 112
2/13/08 9:22:48 AM
Frame Formats n 113
Table 4.1 Mapping 4B5B 4b Binary
bytes: 7 S S D
1
Preamble S F D
5b Binary
0000
0
11110
0001
1
01001
0010
2
10100
0011
3
10101
0111
4
01010
1001
5
01011
1010
6
0110
1011
7
01111
1000
8
10010
1001
9
10011
1010
A
10110
1011
B
10111
1100
C
11010
1101
D
11011
1110
E
11100
1111
F
11101
N/A
Idle
11111
6
6
2
46–1500
4
Destination Address
Source Address
Length
Data
Frame Check Sequence
E S D
SSD Start of Stream Delimiter ESD End of Stream Delimiter
Figure 4.9 The 100BASE-TX frame format
AU6039.indb 113
2/13/08 9:22:48 AM
114 n Carrier Ethernet: Providing the Need for Speed
Interframe Gap Another difference between the 802.3 frame and the 100BASE-TX frame concerns the interframe gap. At 10 Mbps the interframe gap is 9.6 μs between frames, and at 100 Mbps idle codes are used to mark a 0.96-μs interframe gap. The SSD 5B symbols are 11000 10001, and the ESD has the 5B symbols 01101 00111. Both SSD and ESD fields can be considered to fall within the interframe gap of Fast Ethernet frames. Thus, computation between Ethernet/IEEE 802.3 and Fast Ethernet becomes simplified as the latter has an operating ten times the former and an interframe gap one tenth the former.
Gigabit Ethernet The introduction of the IEEE 802.3z standard for Gigabit Ethernet was accompanied by several changes to the Ethernet frame format. At a data rate of 1 Gbps maintaining a minimum frame length of 64 bytes (72 when the preamble and start of frame delimiter fields are considered) would reduce the network diameter to approximately 20 m. While this distance might be suitable for connecting a switch to a group of stations within close proximity of one another, it is not suitable for supporting horizontal wiring within a building where a 10-m distance is allowed from a wall faceplate to the desktop. To enable Gigabit Ethernet to support a network diameter of up to 200 m, a technique referred to as carrier extension was added to the technology.
Carrier Extension Carrier extension results in an extension of the Ethernet slot time from 64 bytes (512 bits) to a new value of 512 bytes (4096 bits). To accomplish this extension frames less than 512 bytes in length are padded with special carrier extension symbols. Note that under Gigabit Ethernet the minimum frame length of 64 bytes is not changed. All frames less than 64 bytes in length are first padded out to a minimum of 64 bytes. The carrier signal placed on the network is then extended to provide a minimum carrier length of 512 bytes. The preceding discussion of frame length is based upon the IEEE use of technology and does not consider the 8 bytes associated with the preamble and start of frame delimiter fields. Figure 4.10 illustrates the Gigabit Ethernet frame to include the location where non-data symbols are added. Note that the FCS is calculated only on the original, non-extended frame. At the receiver the extension symbols are removed before the FCS value is checked.
AU6039.indb 114
2/13/08 9:22:48 AM
Frame Formats n 115 bytes: 7 Preamble
1
6
6
2
46–1500
4
S F D
Destination Address
Source Address
Length
Data
Frame Check Sequence
E X T E N S I O N
64 bytes minimum 512 bytes minimum Duration of carrier event
Figure 4.10 Carrier extension on a Gigabit frame
Half-Duplex Use Carrier extension is only applicable to half-duplex transmission. This is because full-duplex transmission eliminates the possibility of collisions. Because carrier extension can significantly degrade the performance associated with short-packet transmission a second modification to the Gigabit Ethernet frame referred to as frame bursting was developed.
Frame Bursting Frame bursting represents a Gigabit Ethernet technique developed as a mechanism to compensate for performance degradation associated with carrier extension. Under frame bursting a station with more than one frame to send can transmit multiple frames if the first is successfully transmitted. If the first frame is less than 512 bytes in length carrier extension is applied to that frame. Succeeding frames in the burst are limited until a burst limit of 65536 bit times (8192 bytes) is reached. An interframe gap period is inserted between each frame in the burst. However, instead of allowing the medium to be idle between frames, the transmitting station fills the interframe gaps with nondata symbols that maintain an active carrier and which are then discarded by receiving stations. Bursting is only applicable to Gigabit and higher Ethernet speeds when transmission is half duplex. Figure 4.11 illustrates an example of Gigabit Ethernet frame bursting. In this example the first frame in the burst is transmitted successfully with an extension and is assumed to have additional frames that need to be transmitted. Thus, transmission continues until either all frames are transmitted or 8192 byte times are reached, whichever comes first.
AU6039.indb 115
2/13/08 9:22:49 AM
116 n Carrier Ethernet: Providing the Need for Speed
Frame w/extension
IFG
Frame
IFG
…
Frame
Burst Limit
Figure 4.11 Frame bursting
Jumbo Frames Without considering the use of a VLAN tag nor an Ethernet SNAP frame the maximum data field in a frame is 1500 bytes. Although this amount of data was sufficient during the 1970s when text-based e-mail prevailed we will fast-forward to today. The inclusion of signature blocks in e-mail, the attachment of photographs and motion picture files, and a general increase in organizational data storage resulted in the common occurrence of large e-mails and lengthy file transfers. Moving such data in 1500-byte fields can also result in a processing load on computers that hinders other multi-tasking operations. For example, consider moving a 1gigabyte file from a server to a workstation via a Gigabit Ethernet connection. This action would require the processing of approximately 666667 frames that could consume 20 to 40 percent of the processing power of a computer just to handle associated network interrupts. Based upon the preceding, in 1998 Alteon Networks proposed an initiative to increase the maximum length of the Ethernet data field from 1500 bytes to 9000 bytes. Although this initiative was not adopted by the IEEE, it was implemented by a large number of hardware vendors as a jumbo frame option.
Operation In the Alteon Networks proposal which was adopted as an option by several hardware vendors, the Ethernet data field is extended by a factor of six, from a maximum of 1500 to 9000 bytes. This extension can reduce the number of frames required to move a file by a factor of six, increasing application throughput while decreasing host CPU utilization. Because the resources used by a server to handle network traffic are proportional to the number of frames transmitted and received, using larger frames improves performance when compared to the use of smaller frames.
Length Rationale One of the key considerations in sizing a jumbo frame was the CRC-32 algorithm. To maintain a level of compatibility with Ethernet, jumbo frames only changed the size of the data field. Due to the manner by which the CRC-32 algorithm operates, the probability of an undetected error is relatively unchanged until frames exceed
AU6039.indb 116
2/13/08 9:22:50 AM
Frame Formats n 117
12000 bytes. Thus, to maintain the same undetected bit error rate jumbo frames should not exceed 12000 bytes. In the opposite direction, certain applications have a low maximum size for a network file. For example, the maximum size for a Network File System (NFS) datagram is approximately 8000 bytes. Thus, a jumbo data frame of 9000 bytes appears to be a good compromise.
Advantages In addition to reducing the number of frames required to transport files and network overhead, the use of jumbo frames can result in other benefits. Those additional benefits can include a reduction in fragmentation, enhancing TCP throughput as well as the efficiency of switches and routers. Reducing fragmentation reduces fragmentation overhead. This will result into a lower overhead associated with CPU processing. In a TCP environment, throughput has been shown to be directly proportional to the Maximum Segment Size (MSS). Because the MSS is equal to the Maximum Transmission Unit (MTU) less the TCP/IP headers, you can enhance throughput by increasing the Ethernet data field, which enables larger packets to be transported. Concerning switches and routers, because their efficiency is primarily a function of how much time they spend examining headers, a reduction in frames needing processing will make these hardware devices more efficient.
Problems and Solutions One of the problems associated with the use of jumbo frames is intermediate hardware that uses a 1500-byte MTU. Because the smallest MTU used by any device in a given network path determines the maximum MTU for all traffic traveling along that path, frames traversing such hardware cannot be expanded. Thus, an organization with connections to other offices and the Internet may only be able to use a small portion of a local switch for jumbo frames. Although replacement routers and switches could be purchased, they are not inexpensive and in certain situations economics might prevent more than a token replacement of hardware each year. One method that can be used to implement jumbo frames is to isolate the use of such frames to a portion of one’s network that has compatibility with such frames. To accomplish this, the IEEE 802.1Q VLAN tagging specification could be used to enable jumbo frames and standard Ethernet frames to be separate from each other, even when traversing the same physical link if the endpoints support jumbo frames. Thus, through the use of VLAN tagging jumbo compatible hardware could communicate with each other using jumbo frames, while communicating with other network devices using standard Ethernet frames. Then, as funds become available to upgrade additional equipment the VLAN can be modified.
AU6039.indb 117
2/13/08 9:22:50 AM
118 n Carrier Ethernet: Providing the Need for Speed
Performance In concluding this chapter we will examine the effect of the frame overhead on different types of Ethernet. In doing so, we will first assume that the preamble and start of frame delimiter fields as well as the destination address, source address, length/type and frame check sequence fields result in 26 bytes of overhead.
Basic Ethernet For a basic Ethernet frame a 1-byte character carried in the data field must be padded by the addition of 45 fill characters so that a minimum of 46 bytes are transmitted as data. In this situation the overhead required to carry a 1-byte character is 26 plus 45, or 71 bytes. Now, consider the situation in which you have 46 bytes of data to transmit. Here the 46 bytes of data would not require the addition of pad characters, because the frame length would be 64 bytes (72 when considering the preamble and start of frame delimiter fields) which is the minimum frame length. Thus, 64 bytes of data would result in a frame overhead of 26 bytes. Table 4.2 summarizes the overhead associated with an Ethernet nonSNAP frame as the number of bytes of information varies from 1 to a maximum of 1500 bytes that can be carried in the data field of the frame. As indicated in Table 4.2 Basic Ethernet Frame Overhead Information Carried in Ratio of Frame Overhead Percent Data Field (bytes) to Frame Length Overhead
AU6039.indb 118
1
71/72
98.61
10
62/72
86.11
20
52/72
72.22
30
42/72
58.33
45
27/72
37.50
46
26/72
36.11
64
26/90
28.89
128
26/154
16.88
256
26/282
9.22
512
26/538
4.83
1024
26/1050
2.48
1500
26/1526
1.70
2/13/08 9:22:50 AM
Frame Formats n 119
Table 4.2, the percentage of overhead (computed as the ratio of bytes of overhead to frame length times 100) can vary considerably, ranging from a high of 98.61 percent to a low of 1.7 percent when the maximum length data field is used to transfer information.
SNAP Frames The preceding computations were for a basic Ethernet frame. When the length/ type field value is hex 8100 the frame is a SNAP frame. This means that the 8 bytes of additional LLC and SNAP subfields reduce the data field to a value between a minimum of 38 bytes and a maximum of 1492 bytes. Because the extra LLC and SNAP subfields are in the data field, they count towards having a minimum frame length of 64 bytes and the data field does not have to be padded whenever the data field transports 30 or more characters of information. Because the additional 8 bytes of LLC and SNAP subfields represent overhead, a 72-byte frame contains 34 overhead characters plus pads. Thus, the transmission of a 1-byte character requires 37 pad characters or 71 overhead characters. Similarly, a 10-byte message in the data field requires 28 pad characters plus 34 fixed overhead bytes (26 + 8 for the LLC and SNAP subfields) for a total of 62 overhead characters. Thus, the overhead associated with a basic Ethernet frame and a SNAP frame are the same until the data field transports more than 37 characters. At this point the overhead from the LLC and SNAP subfields exceeds the padding, resulting in additional overhead. For example, when 38 data characters are transported LLC, SNAP, and data fields now carry 46 bytes in a frame length of 72 bytes. Thus, the overhead is 34 bytes/72 bytes or 47.22 percent. When 64 data bytes are transported the overhead is 26 + 8 or 34 bytes, while the frame length is 26 + 8 + 64 or 98 bytes, resulting in an overhead of 34/98 or 34.69 percent.Table 4.3 summarizes the overhead associated with Ethernet SNAP frames.
Gigabit Ethernet The previous computations do not consider the overhead associated when halfduplex Ethernet is used. As previously noted in this chapter, packets less than 512 bytes in length (not including their 8-byte header) are extended in length through the use of carrier extension symbols. This means that a packet transporting a 1-byte character is first extended by the addition of 45 padding bytes, further extended by the addition of 45 padding bytes, and then further extended by 448 carrier extension symbols. Thus, a Gigabit Ethernet frame transporting 1 data byte contains 520 bytes, of which 519 represent overhead when operating in half duplex. To illustrate the additional overhead computations, we will assume a frame is transporting 64 bytes of data. In this case there is no requirement for padding
AU6039.indb 119
2/13/08 9:22:50 AM
120 n Carrier Ethernet: Providing the Need for Speed
Table 4.3 Ethernet SNAP Frame Overhead Information Carried in Ratio of Frame Overhead Percent Data Field (bytes) to Frame Length Overhead 1
1/72
98.61
10
62/72
86.11
20
52/72
72.22
30
42/72
58.33
37
36/72
50.00
38
34/72
47.22
45
34/79
43.03
46
34/80
42.50
64
34/98
34.69
128
34/162
20.98
256
34/290
11.72
512
34/546
6.23
1024
34/1058
3.21
1492
34/1492
2.27
characters. However, because the frame must be 512 characters in length without considering the header, this means there must be 430 carrier extension symbols appended to the frame (448 − 18). If you add the normal overhead bytes to the 430 carrier extensions, you obtain a total overhead of 456 bytes. Table 4.4 summarizes the overhead associated with transporting information in Gigabit Ethernet frames as the number of bytes of data varies from 1 to a maximum of 1500. Note that this table is only applicable for half-duplex operations as fullduplex Gigabit Ethernet does not require carrier extensions.
Frame Rates One of the problems associated with Gigabit Ethernet is its use as a shared-media half-duplex network. Although its cost is low, the use of carrier extension to ensure a minimum frame length of 512 bytes can severely degrade performance. In fact, under certain conditions, half-duplex Gigabit Ethernet can represent at best a marginal improvement over Fast Ethernet.
AU6039.indb 120
2/13/08 9:22:50 AM
Frame Formats n 121
Table 4.4 Gigabit Ethernet Half-Duplex Frame Overhead Information Carried in Ratio of Frame Overhead Percent Data Field (bytes) to Frame Length Overhead 1
519/520
98.61
10
510/520
98.08
20
500/520
96.15
30
490/520
94.23
40
480/520
92.31
45
475/520
91.35
46
474/520
91.15
64
456/520
87.69
128
392/520
75.38
256
264/520
50.77
512
136/520
26.15
1024
26/538
4.83
1500
26/1050
2.48
***
26/1526
1.70
Mathematical Oddities Simple mathematics tells us that 100 is ten times 10 and 1000 is ten times 100. If we apply logic and simple mathematics to the operating of Ethernet, Fast Ethernet, and Gigabit Ethernet, we would expect the operating rate of each local area network to differ by a similar amount. While logic and simple mathematics are indeed true and result in operating rates increasing by an order of magnitude from Ethernet to Fast Ethernet to Gigabit Ethernet, what can we say about the ability of each network to transport data? Does Fast Ethernet provide ten times the throughput of Ethernet? Similarly, does Gigabit Ethernet provide ten times the throughput of Fast Ethernet? As we will note shortly, the answer to the second question in certain networking situations is negative and the reason for the answer has important considerations for network managers and LAN administrators considering halfduplex Gigabit technology. Because many readers may be from Missouri, which is known as the “Show Me” state, we will examine the basic frame format of Ethernet as a starting point to determine the frame rate obtainable on different types of
AU6039.indb 121
2/13/08 9:22:51 AM
122 n Carrier Ethernet: Providing the Need for Speed
Ethernet networks. By first computing the frame rate on a 10 Mbps network, we can use the resulting computations as a base to examine the effect of increasing the LAN operating rate upon the effective data transfer capability obtainable on Fast Ethernet and half-duplex Gigabit Ethernet. Readers should note that the effective data transfer rate represents the quantity of data per unit of time transferred over a network. Because all protocols have a degree of overhead in the form of headers and trailers wrapped around information formed into frames (Layer 2) or packets (Layer 3), the effective data rate is always lower than the operating rate of a transmission facility. Although most people only consider the LAN operating rate, the effective data transfer rate is a more important metric as it governs the ability of a network to transport data.
Frame Rate Computations Previously, Figure 4.1 illustrated the basic format of an Ethernet frame when the frame is placed onto a network. Although many publications reference a minimum frame length of 64 bytes, those publications are referencing the length of the frame prior to its placement onto a network. Once the latter occurs, 8 bits are added to each frame for synchronization in the form of a 7-byte preamble field and a 1-byte start of frame delimiter field. This results in a minimum Ethernet frame length of 72 bytes. In examining Figure 4.1, note that the data field ranges from a minimum of 46 bytes to a maximum length of 1500 bytes. When a maximum length frame (nonVLAN tagged) is formed, its length is 1518 bytes in the network adapter. However, when placed on the network the addition of 8 bytes in the form of the preamble and start of frame delimiter fields included for synchronization results in a maximum frame length of 1526 bytes. Thus, all Ethernet frames other than those in error range between 72 and 1526 bytes in length when flowing on a network. The only exceptions are jumbo frames and the use of VLAN tagging, with the latter adding 4 bytes to the length of the frame. For our computations, we will consider the maximum frame length to be limited to 1526 bytes, representing a conventional non-tagged frame. Because the length of an Ethernet frame can vary between 72 and 1526 bytes, the frame rate obtainable on a network will vary. However, because the minimum and maximum frame lengths are known we can compute the frame rate in terms of each metric, which will provide the maximum and minimum frame rates on a network. Because frame rate is inversely proportional to frame length, the maximum frame length will enable us to compute the minimum frame rate, and the minimum frame length obtainable on a network will provide us with the ability to compute the maximum frame rate. At a 10-Mbps operating rate Ethernet requires a dead time of 9.6 μs between frames. The bit duration at a 10-Mbps operating rate is 1/107 or 100 ns. Based
AU6039.indb 122
2/13/08 9:22:51 AM
Frame Formats n 123
upon the preceding we can compute the maximum number of frames per second (fps) for maximum length and minimum length frames. For example, consider the maximum length frame of 1526 bytes. Here the time per frame becomes 9.6 μs +1526 bytes * 8 bits/byte * 100 ns/bit, which results in a frame time of 1.23 ms. Thus, in 1 s there can be a maximum of 1/1.23 ms or 812 maximum length frames, each capable of transporting 1500 bytes of data. This means that if all frames were maximum length frames, the effective data transfer capability would be 812 fps * 1500 bytes/frame * 8 bits/byte or 9.744 Mbps. We can revise the preceding computations to determine the number of minimum length frames that can flow on a 10 Mbps Ethernet network. For a minimum length frame of 72 bytes, the time per frame is 9.6 μs +72 bytes * 8 bits/byte * 100 ns/bit, which results in a frame duration of 67.2 * 10−06 s. Thus, in 1 s there can be a maximum of 1/67.2* 10−06 or 14880 minimum length 72-byte frames, each capable of transporting between one and 46 data characters. This means that if all frames were minimum length frames the effective data transfer rate would range between 14880 frames/s * 1 byte/frame * 8 bits/byte or 119040 bps to 14880 frames/s * 46 bytes/frame * 8 bits/byte or 5.48 Mbps. Based upon the preceding computations, a file transfer between two stations on an Ethernet network that completely fills each frame’s data field to its maximum capacity of 1500 bytes will result in an effective data transfer of 9.744 Mbps, approaching the 10-Mbps LAN operating rate. However, as file transfers are replaced by interactive queries that result in a lesser number of data characters transported in each frame, the effective data transfer rate decreases. On a probably absurd level, if each frame transported one data character the effective data transfer rate would be reduced to 119040 bps. Even when each minimum length frame is filled with 46 data characters, the effective data transfer capacity is only slightly over half the network’s 10-Mbps operating rate. Fast Ethernet uses the same frame format as Ethernet, but the dead time between frames and bit duration are one tenth Ethernet’s 10-Mbps metrics. Thus, the frame rate for maximum and minimum length frames are ten times that of Ethernet. That is, Fast Ethernet supports a maximum of 8120 maximum length 1526-byte fps and a maximum of 148800 minimum length 72-byte fps. Similarly, the effective data transfer capability of Fast Ethernet is ten times that of Ethernet. Thus, we can summarize the expectations of network managers and LAN administrators concerning the performance of Fast Ethernet in comparison to Ethernet as you truly get what you expect. Table 4.5 compares the frame rates and effective data transfer capability of Ethernet and Fast Ethernet. Although we might reasonably expect Gigabit Ethernet to extend Fast Ethernet’s frame rate and data transfer capability by a factor of ten, this is not the case in certain situations. Those situations involve the use of Gigabit Ethernet in a shared-media environment when the Gigabit Ethernet basic frame length is less than 512 bytes.
AU6039.indb 123
2/13/08 9:22:51 AM
124 n Carrier Ethernet: Providing the Need for Speed
Table 4.5 Ethernet versus Fast Ethernet Frame Rates and Effective Data Transfer Capability Average Frame Size Frame Rate (bytes) (fps)
Effective Data Transfer (bps)
ETHERNET 1526
812
72
14880
9.744 Mbps 119.04 kbps to 5.48 Mbps
FAST ETHERNET 1526
8120
72
148800
97.44 Mbps 1.19 Mbps to 54.8 Mbps
Gigabit Constraints The use of Gigabit in a shared-media environment requires a station on the network to be able to hear any resulting collision on the frame it is transmitting before it completes the transmission of the frame. This means that the transmission of the next-to-last bit of a frame that results in a collision should allow the transmitting station to hear the collision voltage increase prior to the transmission of the last bit. Thus, the maximum allowable Ethernet cabling distance is limited by the bit duration associated with the network operating rate and the speed of electrons flowing on the network. When Ethernet operates at 1 Gbps, the allowable cabling distance would normally have to be reduced from Fast Ethernet’s 200-m diameter to approximately 10 m or 33 ft. This would result in a major restriction on the ability of Gigabit Ethernet to be effectively used in a shared-media, half-duplex environment. To overcome this cabling limitation, a carrier extension scheme was proposed by Sun Microsystems. This scheme, which results in the extension of the time an Ethernet frame is on the wire, became part of the Gigabit Ethernet standard for half-duplex operations. Under the Gigabit Ethernet carrier extension scheme, the IEEE standard requires a minimum length frame of 512 bytes to be formed for shared-media, half-duplex operations. This means that the resulting frame when placed onto the network must be a minimum of 520 bytes in length due to the addition of a 7-byte preamble field and a 1-byte start of frame delimiter field. Previously, Figure 4.10 illustrated the Gigabit Ethernet carrier extension scheme associated with ensuring the flow of extended minimum length frames. In examining Figure 4.10, note that the timing extension occurs after the end of the standard Ethernet frame. The actual carrier extension occurs in the form of special symbols
AU6039.indb 124
2/13/08 9:22:51 AM
Frame Formats n 125
that result in the occurrence of line transitions and inform other stations “listening” to the network that the wire is in use. The carrier extension extends each Gigabit frame time to guarantee a minimum 512-byte slot time (520 bytes on the network) for half-duplex operations. Note that the increase in the minimum length of frames does not change the contents of any frame. Instead, carrier extension technology only alters the time the frame is on the network. Thus, compatibility is maintained between the original Ethernet frame and frames flowing on half-duplex Gigabit Ethernet. It should also be noted that Gigabit Ethernet carrier extension technology is not applicable for non-shared media environments, such as transmission on fiber or workstation connections to full-duplex switch ports. This is because no collisions are possible on such network environments, alleviating the necessity to employ carrier extension technology to ensure each frame flows on the network for sufficient duration to enable the transmitting station to recognize the occurrence of a collision within a 200-m diameter. Although carrier extension technology enables the cable length of a halfduplex Gigabit Ethernet network to be extended to a 200-m diameter, the extension is not without cost. The primary cost is one of additional overhead and lower data throughput because extension symbols added to the end of short frames waste bandwidth. To obtain an appreciation for the manner by which carrier extension technology wastes bandwidth, consider the requirement to transmit a 64-byte record. When using Ethernet or Fast Ethernet, the record would be encapsulated within 26 bytes of overhead, resulting in a frame length of 90 bytes. When using Gigabit Ethernet as the transport mechanism, the minimum length frame must be 520 bytes when flowing on the network. Thus, the frame must be extended through the addition of 430 (520 − 90) carrier extension symbols. To further complicate bandwidth utilization, when the amount of data to be transported by a frame is less than 46 bytes, nulls are added to the data field to produce a 64-byte minimum length (72 bytes on the wire) frame prior to extending the frame via carrier extension symbols. Thus, a simple query, such as “Mr Watson I hear you,” which consists of 20 characters including spaces between words, would be placed in the data field, padded with 26 nulls under each version of Ethernet to form a minimum length frame. However, under Gigabit Ethernet the frame would be extended further through the addition of 448 carrier extension symbols to obtain a minimum 512-byte slot time or 520 bytes when the frame flows on the network. In this example, the ratio between actual information transported and total characters transmitted changes from 20 per 72 on Ethernet and Fast Ethernet to 20 per 520 on Gigabit Ethernet! Under Gigabit Ethernet, the minimum 64-byte slot time (72 bytes on the network) requires the use of 448 carrier extension symbols. To examine the effect upon the data transport capability of Gigabit Ethernet, we can compute the frame rate in a manner similar to prior computations. However, because the minimum length frame is 520 bytes, we will use that value instead of 72 bytes to compute the maximum frame rate. In doing so, the dead time between frames becomes 0.096 μs and
AU6039.indb 125
2/13/08 9:22:52 AM
126 n Carrier Ethernet: Providing the Need for Speed
the bit duration becomes 1 ns. Thus, the time per minimum length frame becomes 0.096 μs + 520 bytes * 8 bits/byte * 1 ns/bit or 4.256 * 10−06 s. Then, in 1 s a total of 1/4.256 μs or 234624 minimum length frames can flow on a Gigabit Ethernet shared-media network. Note that the frame rate ratio between Gigabit Ethernet and Fast Ethernet is 234624/148800 or 1.58 and not the ten to one you would expect. Concerning the effective data transfer capacity, each minimum length Gigabit Ethernet frame can transport between 1 and 498 data characters because pads and carrier extension symbols are used to ensure a minimum length frame of 520-bytes flows on the network. This means that the effective transfer rate for minimum length frames ranges between 234962 fps * 1 byte/frame and 234962 fps * 498 bytes/frame. Expressed as a conventional data rate, we obtain an effective data transfer rate between 1.88 Mbps and 93.6 Mbps. Only as frame lengths increase beyond the minimum length does half-duplex Gigabit Ethernet provide an enhanced data transfer beyond that obtainable by Fast Ethernet. The preceding computations illustrate that the effective data transfer rate when using Gigabit Ethernet is highly dependent upon the average frame length expected to flow on the network. Although a shared-media Gigabit Ethernet backbone is probably well suited for use in an ISP environment where browsing graphics fill most frames to their limit, the technology may not be particularly well suited for organizations where a high proportion of traffic is in the form of interactive query– response. Thus, it is important to investigate fully the technology as well as the manner by which you propose to use the technology prior to making an investment in the technology.
AU6039.indb 126
2/13/08 9:22:52 AM
Chapter 5
LAN Switches The purpose of this chapter is to provide readers with detailed information concerning the operation and utilization of Ethernet LAN switches. To do so, we will first examine the basic operation of a bridge, because a LAN switch in effect represents a multiport bridge that supports concurrent operations. Once this is accomplished, we will examine the three basic types of Ethernet switches and their switching methods. Because the first series of Ethernet LAN switches operated at Layer 2 in the International Standards Organization (ISO) Open System Interface (OSI) Reference Model, we will first focus our attention upon Media Access Control (MAC) address switching. Building upon MAC address switching, we will also examine higher layer switching as a prelude to discussing virtual LANs (VLANs). Once the latter is accomplished, we will conclude this chapter by examining how modern Ethernet LAN switches can function as gateways from the edge of a customer network into different types of carrier network to include a Carrier Ethernet service.
Bridge Operations Because the first series of LAN switches were based upon the manner by which bridges operate, we will examine the operation of bridges in this section. Bridges operate at the data link layer of the OSI Reference Model, using MAC addresses as a mechanism for controlling the flow of frames through the bridge.
Transparent and Translating Bridges As the need to link networks increased, two types of bridges were developed: transparent and translating. A transparent bridge connects similar networks and a translating 127
AU6039.indb 127
2/13/08 9:22:52 AM
128 n Carrier Ethernet: Providing the Need for Speed
Lan A
A
C
B
1 Bridge 2
Lan B
D
E
F
Figure 5.1 Using a transparent bridge to connect two Ethernet networks
bridge is used to connect two dissimilar networks, such as an Ethernet LAN to a Token-Ring LAN. Because the LAN wars are over and Ethernet is the clear winner, we will focus our attention upon the operation of transparent bridges.
Plug-and-Play Operation A transparent bridge is a plug-and-play device. That is, you plug each LAN connection into a bridge port and power-on the bridge. Thereafter, the bridge uses a backward-learning algorithm, which we will describe shortly to determine where frames should be routed. Thus, the bridge automatically begins to “play,” hence the term “plug-and-play.”
Bridge Operation To illustrate the operation of a transparent bridge, consider Figure 5.1 which illustrates the interconnection of two bus-based Ethernet networks. For simplicity the MAC address of each station on each LAN is indicated by a letter instead of a 48bit MAC address. A transparent bridge operates in a promiscuous mode, which means that it reads every frame transmitted on each network to which it is connected. Once the bridge is connected to each network and powered-on, it operates according to the three Fs, flooding, filtering, and forwarding, using a backward-learning algorithm. To illustrate the bridge operating process, we will assume the following activities take place. First, station A transmits to station E, which then responds to station A. Next, station B transmits a frame to station A, followed by station A transmitting a frame to station E. When the bridge is powered-on, its port/address table is empty. Thus, we can view the contents of the port/address table as follows:
AU6039.indb 128
2/13/08 9:22:53 AM
LAN Switches n 129
Port
Address
-
-
Flooding Based upon our prior assumption, station A transmits a frame which flows on LAN A and reaches the bridge on its port 0. Here the destination address in the frame is E and the source address is A. The bridge examines its port/address table to determine where to route the frame whose destination address is E. Because the bridge was just powered-on, its port/address table is empty. Thus, not knowing where to route a frame results in the bridge flooding the frame. That is, the bridge transmits the frame out of all ports other than the port it was received on. In this example the frame is output on port 1 and is then received by station F. In addition to flooding the frame, the bridge examines the frame’s source address and the port it entered the bridge. The bridge uses this data to determine if it should update its port/address table. Because the port/address table was empty, the bridge enters the source address and the port it arrived on into its port address table. Thus, the port address table begins to build and is now: Port
Address
0
A
Forwarding Next, it was assumed that station E responds to station A. As the frame transmitted from station E arrives at the bridge via port 1, the bridge looks into its port/address table and notes that address A is associated with port 0. Thus, the bridge forwards the frame onto LAN B via port 0. In addition, the bridge notes that the source address of the new frame is E and that address is associated with port 1. Thus, the bridge updates its port/address table as follows: Port
Address
0
A
1
E
Filtering Next it was assumed that station B on LAN A transmits a frame to station A on that network. As the frame flows on the network, it reaches port 0 of the bridge. The bridge notes that the destination address of the frame (A) is on port 0 from the entries in its port/address table. Thus, the bridge filters the frame so that it does not flow through it. In addition, the bridge examines the source address of the frame
AU6039.indb 129
2/13/08 9:22:53 AM
130 n Carrier Ethernet: Providing the Need for Speed
Table 5.1 Steady-State Port/Address Table Port
Address
0
A
0
B
0
C
1
D
1
E
1
F
(B) and, because it is not in the port/address table, adds the station address and port it arrived on. Thus, the port/address table entries are now as follows: Port
Address
0
A
0
B
1
E
As more and more frames flow over each network, the bridge continues to learn addresses via a backward-learning algorithm by examining the source address of frames entering different bridge ports. This eventually results in the bridge learning the address of each station on each network and the port of the bridge connected to the network. The result of this learning process is the completion of a steady-state port address table. Table 5.1 shows an example of a steady-state port address table for the bridge shown in Figure 5.1. We can summarize the operation of a transparent bridge as follows: 1. If the destination and source addresses are on the same network, discard the frame. 2. If the destination and source addresses are on different networks, forward the frame. 3. If the destination network is not known, flood the frame. When bridges were first developed during the 1970s, memory was relatively expensive and processing capabilities were a fraction of what they are today. In those “good old days” the port/address table included a time stamp for each entry. After a predefined amount of time, the entry was purged, in effect reducing the memory required to store entries as well as the processing required to search the table. If we fast-forward to the present time, many bridges and switches include a
AU6039.indb 130
2/13/08 9:22:53 AM
LAN Switches n 131
time stamp for each entry; however, either users will be able to control when entries are purged or the device will allow more entries than its early 1970s-era “cousins.”
Intelligent Switching Hubs Bridges were originally developed for use with shared-media LANs, such as 10BASE-5 and 10BASE-2 networks. Thus, the bandwidth constraints associated with shared-media networks are also associated with bridges. That is, a transparent bridge can route only one frame at a time received on one or more ports, with multiple-port frame broadcasting occurring during flooding or when a broadcast frame is received. Recognizing the limitations associated with the operation of bridges, equipment vendors incorporated parallel switching technology into devices referred to initially as intelligent switching hubs and now more commonly known as switches. Such devices were developed based upon technology used in matrix switches, which for years were successfully used in telecommunications operations. By adding buffer memory to store address tables as well as buffering frames, the switch could read the destination address of frames entering the device on multiple ports and either directly switch the frames to their destination or buffer them in memory until the destination port becomes available.
Basic Components Figure 5.2 illustrates the basic components of a four-port intelligent switch. Each switch consists of Buffers and Address Tables (BAT), logic, and a switching fabric that permits frames entering one port to be routed to any port in the switch. The destination address in a frame is used to determine the associated port with that address via a search of the port/address table, with the port address used by the switching fabric for establishing the cross-connection.
Buffer Memory The amount and placement of buffer memory depends upon the type of switch and its design. Some switches use a small amount of buffer memory to hold frames arriving at each port until their destination address is read. Other switches may store frames as they are input to compute a Cyclic Redundancy Check (CRC) and determine if the frame is error free and, if not, discard the frame. In addition, some switches may store frames in a central memory area instead of using distributed memory.
Delay Times Switching occurs on a frame-by-frame basis, with the cross-connection torn down after being established to route a frame. Thus, frames can be interleaved from two
AU6039.indb 131
2/13/08 9:22:53 AM
132 n Carrier Ethernet: Providing the Need for Speed
Port
BAT
Port
BAT
Port
BAT
Port
BAT
BAT: Buffer and Address Table
Figure 5.2 Basic components of an intelligent switch
or more ports to a common destination port with a minimum of delay. For example, consider a non-VLAN tagged frame whose maximum length is 1526 bytes to include preamble and start of frame fields. At a 10-Mbps operating rate, each bit time is 1/107 sec or 100 ns. For a 1526-byte frame, the minimum delay time if one frame precedes another frame being routed to a common destination becomes
1526 bytes × (8 bits/byte) × (100 ns/bit) = 1.22 ms
As you might expect, blocking delay decreases as the data rate increases. Table 5.2 lists the blocking delay times for a maximum length non-VLAN frame for Ethernet at 10 Mbps through 10 Gbps. Table 5.2 Blocking Delay Times for a 1526-Byte Frame Ethernet Version
AU6039.indb 132
Blocking Delay
10 Mbps
1.22 ms or 1220 ns
100 Mbps
0.122 ms or 122 ns
1 Gbps
0.0122 ms or 122 ns
10 Gbps
0.00122 ms or 1.22 ns
2/13/08 9:22:54 AM
LAN Switches n 133
The previous computed delay times represent blocking resulting from frames entering on two ports having a common destination and should not be confused with another delay time referred to as latency. Latency represents the delay associated with the physical transfer of frames from one port via the switch to another port and is based upon the architecture of the switch, which adds additional delays above and beyond the delay associated with the physical length of the frame being transported through the switch. In comparison, blocking delays depends upon the number of frames from different ports attempting to access a common destination port and the method by which the switch is designed to respond to blocking. Some switches have large buffers for each port and service ports in a round-robin fashion when frames from two or more ports attempt to access a common destination port. Other switches may be designed to recognize that port buffer occupancy results in a priority service scheme based upon the occupancy of the port buffers in the switch.
Parallel Switching The ability to support parallel switching is the key advantage obtained from the use of a switch. This feature permits multiple cross-connections to occur between source and destination ports at the same time. To illustrate this capability, we will return our focus to Figure 5.2 and assume that four 100BASE-T networks were connected to each port of the four-port switch. Assuming a station on two LANs communicates with stations on the other two LANs, two simultaneous cross-connections are supported, each at 100 Mbps. This results in an increase in bandwidth to 200 Mbps. Thus, from a theoretical perspective, an N-port switch where each port operates at 100 Mbps provides a potential throughput of up to N/2 * 100 Mbps. Thus, by connecting either individual workstations or network segments to switch ports, you can overcome the operating rate limitation of an Ethernet network. In addition, because the cross-connection through a switch represents a dedicated connection, there will never be a collision. Thus, the collision detection wire-pair can be used to provide a full-duplex capability, which can dramatically enhance the operation of servers connected to a switch. Now that we have an appreciation for the key advantages associated with a switch, we will turn our attention to the manner by which switches operate.
Switch Operations In this section we will examine two different types of switch operations. First, we will examine the three basic types of switching techniques as well as discuss the latency associated with each technique. Once this is accomplished, we will then focus our attention upon switching methods, obtaining an understanding of why
AU6039.indb 133
2/13/08 9:22:54 AM
134 n Carrier Ethernet: Providing the Need for Speed
some switches are limited to connecting individual workstations to a switch port and others enable network segments to be connected to a port.
Switching Techniques There are three basic types of switching techniques: cut-through or “on-the-fly,” store-and-forward, and hybrid. The latter alternates between the first two methods based upon the frame error rate.
Cross-Point Switching A cross-point or cut-through switch has the minimum amount of latency due to the manner by which it operates. As a frame enters a switch port, the switch uses the destination address as a decision criteria to obtain a port destination from a look-up table. Once a port destination is obtained, the switch initiates a cross-connection, resulting in the frame being routed to a destination port. Switches, like bridges, initially operated only at the MAC layer and used a backward-learning process to construct a port/address table. Thus, a Layer 2 switch follows the three Fs: flooding, forwarding, and filtering frames.
Operation Figure 5.3 illustrates the basic operation of a cross-point or cut-through switch. When cut-through switching occurs, the frame enters a switch port and its destination address is read (1) prior to the entire frame being read and buffered. The destination address is forwarded to a look-up table (2) to determine the port destination address, which is used by the switching fabric to initiate a cross-connection to the destination port (3). Because this switching method only requires the storage of a
Ports
1
1 2
2 Switching Fabric
3
4
Lookup Table
3
Figure 5.3 Cross-point switching
AU6039.indb 134
2/13/08 9:22:55 AM
LAN Switches n 135
small portion of a frame until the switch can read the destination address and perform its table look-up operation to initiate switching to an appropriate output port, latency through the switch is minimized.
Latency You can consider latency as functioning as a brake on the transmission of frames whose effect depends upon the application being routed via the switch. For example, in a conventional client–server data transfer environment the transmission of a frame by a workstation results in a server response. Thus, the minimum wait time without considering the server processing delay is two times latency for each client–server exchange, lowering the effective data transfer rate. If a VoIP application had digitized voice routed through a switch, the additional latency could represent a different type of problem. That is, as snippets of 20-ms portions of a digital conversation are passed through a switch, the latency delay could result in the reconstructed sound appearing awkward, which can be more of a problem than a lowering of the effective throughput of a switch. Because a cross-point switching technique results in a minimum amount of latency, the effect upon different applications routed through this type of switch is minimal.
Store-and-Forward A store-and-forward switch first stores a frame in memory prior to initiating its switching fabric to move the frame to its destination. Once the entire frame is stored, the switch checks the integrity of the frame by computing a local CRC on the contents of the frame and comparing its computed CRC against the CRC stored as part of the frame. If the two match, the frame is considered to be error free and will be switched to its destination. Otherwise, the frame is considered to have one or more bits in error and is sent to the great bit bucket in the sky.
Filtering Capability In addition to allowing frames to be error checked, their storage permits filtering against various frame fields to occur. Thus, frames transporting certain protocols could be routed to predefined destination ports or other fields could be used to create switching rules.
Operation Figure 5.4 illustrates the basic operation of a store-and-forward switch. In this example the switch is shown using shared buffer memory to store frames prior to
AU6039.indb 135
2/13/08 9:22:55 AM
136 n Carrier Ethernet: Providing the Need for Speed
1 1 2
Ports
2
3
Shared Memory Buffer
3 Switching Fabric
Lookup Table
4 4
Figure 5.4 Store-and-forward switching
their switching to a port affiliated with the destination address of the frame. The store-and-forward switch first reads the frame’s destination address (1) as it flows into the shared memory buffer (2). As the frame is being read into memory, a lookup operation using the frame’s destination address (3) occurs to obtain the destination port address. Once the entire frame is stored in memory, a CRC is performed and one or more filtering operations can be initiated. If the CRC indicates that the frame is error free, it is then forwarded from memory to its destination (4), otherwise the frame is discarded.
Delay Time Without considering a VLAN frame, the minimum length of an Ethernet frame is 72 bytes on a LAN and 64 bytes when stored in memory, because the preamble and start of frame delimiter (SFD) fields are not stored. Thus, the minimum oneway delay when a station or LAN operating at 10 Mbps is connected to a switch port becomes
96 ns + 64 bytes × 8 bits/byte × 100 ns/bit or 9.6 × 10−6 + 512 × 100 × 10−9 or 60.8 × 10−6 sec
In the previous computation, 9.6 ns represents the Ethernet interframe gap, and 100 ns/bit is the bit duration of a 10 Mbps Ethernet LAN. Thus, the minimum one-way latency of a store-and-forward Ethernet switch is 0.000062 sec, and a round-trip minimum latency is twice that duration. For a maximum-length, nonVLAN Ethernet frame with a data field of 1500 bytes, the frame length that is stored in a switch is 1518 bytes. Thus, the one-way maximum latency becomes
AU6039.indb 136
2/13/08 9:22:56 AM
LAN Switches n 137
Table 5.3 Minimum and Maximum Ethernet Frame Delays Ethernet Operating Minimum Frame Maximum Frame Rate Length Delay Length Delay
10 Mbps
60.6 ns
1224.00 ns
100 Mbps
6.06 ns
122.40 ns
1 Gbps
0.606 ns
12.24 ns
10 Gbps
0.0606 ns
1.22 ns
96 ns + 1518 bytes × 8 bits/byte × 100 ns/bit or 9.6 × 10−6 + 12144 × 100 × 10−9 or 1224 × 10−6 sec
Table 5.3 indicates the delay times for minimum and maximum length Ethernet frames operating at 10 Mbps through 10 Gbps. As you might expect, as the data rate increases the delay time, resulting from storing frames decreases.
Hybrid A third type of switch that supports both cut-through and store-and-forwarding is the hybrid switch. The hybrid switch monitors the frame error rate in the cutthrough mode of operation. If the error rate exceeds a predefined level, the switch changes its mode of operation to store-and-forwarding, enabling frames with one or more bits in error to be discarded. The major benefit of a hybrid switch is the fact that it provides minimal latency when the frame error rate is low and discards frames by adapting to a store-andforward switching method. Then, the switch can discard frames when the error rate exceeds a predefined threshold.
Switch Port Address Support Switches fall into two general categories based upon their port address support: port-based switching and segment-based switching.
Port-Based Switching A switch which performs port-based switching supports only a single address per port. This restricts switching to one device per port while requiring a minimal amount of
AU6039.indb 137
2/13/08 9:22:56 AM
138 n Carrier Ethernet: Providing the Need for Speed
•
Workstation 1
•
•
•
Workstation N
Port Based Switch
Server 1
•
•
•
•
Server N
Figure 5.5 Port-based switching
memory in the switch. In addition, the table look-up process is faster because this type of switch limits support to one device address per port. Figure 5.5 illustrates an example of the use of a port-based switch. In this example M user workstations use the switch to contend for access to N servers, where M >> N. If a switch operating at 10 Mbps per port has N ports, then the maximum throughput is (10 Mbps * N)/2, because up to N/2 simultaneous cross-connections can be supported. However, in the real world the number of workstations (M) connected to a switch greatly exceeds the number of servers (N) connected. Because the vast majority of communications occurs as client–server data exchanges, this means that the number of servers normally governs the throughput of the switch. Both cut-through and store-and-forward switches can employ port-based switching. When a cut-through switch uses port-based switching, its latency is minimized because only one address is stored per port, reducing the time required to search the device’s port/address table for routing frames to their destination.
Segment-Based Switching A segment-based switch supports the switching of multiple addresses per port. This means that you can connect individual workstations or network segments to each switch port. Figure 5.6 illustrates the use of a segment-based switch in a modern Ethernet environment where conventional 10BASE-T and 100BASE-T hubs long ago replaced the
AU6039.indb 138
2/13/08 9:22:56 AM
LAN Switches n 139
Corporate Server
Corporate Server
Segment Based Switch
Conventional Hub
WS
•
•
•
Conventional Hub
Dept. Server
WS
•
•
•
Dept. Server
Figure 5.6 Segment-based switching
use of coaxial cable bus-based network segments. Although two conventional hubs are shown attached to two segment-based switch ports, you can also connect individual workstations and servers to such ports. In fact, the top portion of Figure 5.6 illustrates the connection of two “corporate” servers to the segment-based switch. In examining Figure 5.6, note that each conventional hub functions as a repeater and forwards each frame transmitted to the hub to the switch port to which it is connected, regardless of whether or not the frame requires the resources of the switch. The switch examines the destination address of each frame against addresses in its look-up table, forwarding those frames that warrant forwarding, and also performing filtering and flooding as necessary. Because a segment-based switch stores more than one address per port, the search of its port/address table is more time consuming than an equivalent port-based switch. Thus, you can expect a worst-case latency to occur when you use a store-and-forward segment-based switch.
Applications To obtain an appreciation of the applications that can use Ethernet switches a few words concerning the types of Ethernet switch ports are justified. Originally,
AU6039.indb 139
2/13/08 9:22:57 AM
140 n Carrier Ethernet: Providing the Need for Speed
Corporate Server
Corporate Server
Fat Pipe
100 Base-T Ports
Segment Based Switch
10 Base-T Ports Conventional Hub
WS
•
•
•
Dept. Server
Conventional Hub
WS
•
•
•
Dept. Server
Figure 5.7 Using Fast Ethernet ports to service servers
Ethernet switch ports were limited to supporting 10BASE-T and each port operated at 10 Mbps. When Fast Ethernet ports were originally incorporated into Ethernet switches, their cost precluded each port operating at 100 Mbps. Instead, a few high-speed ports were used for server connections and 10BASE-T ports were used for connecting to individual workstations and LAN segments. An example of this architecture is shown in Figure 5.7. In examining Figure 5.7, note the two 100BASE-T connections to the server located in the upper right corner of the referenced illustration. The use of two or more connections from a switch to a workstation, server, or other device is referred to as a fat pipe. Normally, fat pipes are used to enhance the data transfer between a switch and a server. Although switches that support 1 GbE and 10 GbE are available, the per-port cost can be expensive, making the use of fat pipes an attractive, cost-effective alternative.
Considering Port Capability Because queries from workstations flow at 10 Mbps to servers that respond at 100 Mbps, dual-speed switches included additional buffers as well as a flow control mechanism to regulate the flow of data between devices connected to the switch
AU6039.indb 140
2/13/08 9:22:58 AM
LAN Switches n 141
that operates at different data rates. With the growth in the manufacture of 10/100BASE-T NICs that lowered cost, most switches today support 10/100 Mbps data rates as a minimum. In addition, the development of Gigabit Ethernet and 10 GbE resulted in switches being manufactured that support the use of a few such ports as well as some switches that only operate at 1 Gbps on each port. Because there are many types of GbE and 10 GbE technologies, it is important to ensure that you select the correct type of high-speed port(s) that will satisfy your organization’s transmission requirements. For example, the difference in the transmission range of various type of GbE fiber can cause an application to fail unless the correct type of fiber is available on a switch. Now that we have an appreciation for the necessity to carefully select the type of switch ports, we will turn our attention to the manner by which switches can be used to support different applications.
Basic Switching For small organizations a centralized switch may be sufficient to support the entire organization. An example of the use of a 10/100 Mbps Ethernet switch used to connect both existing individual workstations and workstations located on legacy segments to corporate servers is shown in Figure 5.8. Note that the failure of this switch would in effect disable network communications for the entire organization. Due to this, most switches can be obtained with redundant power supplies and central logic that results in a cutover if a primary element should fail.
Multi-Tier Networking In some modern organizations, departments such as engineering and accounting may have their own servers, while at the corporate level one or more mail servers support the entire organization, and a large router is used to provide Internet access for all employees. In this type of environment, a multi-tier network consisting of interconnected switches can be considered to satisfy organizational networking requirements. Figure 5.9 illustrates the generic use of a two-tiered, Ethernet switch-based network. Here the switches at the lower tier are used to provide individual departments with connectivity to their local server as well as access to a corporate e-mail server and a router that provides a connection to the Internet. One of the major benefits of the use of a multi-tier switching arrangement is the fact that the failure of a switch or inter-switch connection enables employees connected to other departmental switches to continue to perform the majority of their work. For example, if the higher-tier switch of one departmental switch should fail, employees connected to other departmental switches could continue to access their departmental servers.
AU6039.indb 141
2/13/08 9:22:58 AM
142 n Carrier Ethernet: Providing the Need for Speed
Server
Server
Fat Pipe
100 Base-T Ports
10/100 Mbps Ethernet Switch
10 Base-T Ports Conventional Hub
WS
•
•
•
Conventional Hub
WS
WS
•
•
•
WS
Figure 5.8 Using a single switch in a small department or corporation
Interconnecting Dispersed Offices With the development of the Ethernet in the First Mile (EFM) IEEE 802.3ah standard, it has become possible to access a Carrier Ethernet service that enables data to flow between organizational locations as Ethernet frames. This ability can significantly improve network performance as it does away with protocol conversion to the network layer, enabling throughput performance to be enhanced. Figure 5.10 illustrates the use of an Ethernet switch that includes a 1000BASEBX10 port, which supports 1000-Mbps Ethernet transmission over an individual single-mode fiber at distances up to 10 km. Thus, if a communications carrier serving an office provides a single-mode fiber into the building, that fiber could be used as an EFM connection to the Carrier Ethernet service, which is shown in the middle of the figure. Similarly, other offices within the metropolitan area can also use switches with one or more 1000BASE-BX10 or other EFM-compatible ports to access the Carrier Ethernet service. In this manner offices could communicate with one another with a minimum of latency due to data transported end-to-end at Layer 2.
AU6039.indb 142
2/13/08 9:22:58 AM
LAN Switches n 143
Mail Router Server • • •
Switch Higher Tier Lower Tier
Switch
Switch
DS
DS Legend: DS Departmental Server
Workstation
Figure 5.9 Using a two-tiered Ethernet switch-based network
Virtual LANs One of the key benefits associated with the use of switches is the support many devices provide for the operation of different types of VLANs. In this section, we will first examine the basic characteristics of a VLAN and its rationale for use. Once this is accomplished, we will examine the different types of VLANs and the VLAN IEEE standard, concluding with a review of the advantages associated with the use of VLANs.
Characteristics A VLAN can be considered to represent a broadcast domain. This means that a transmission generated by one station on a VLAN is only received by those stations predefined by some criteria to be in the domain.
Construction Basics A VLAN is constructed by the logical grouping of two or more network nodes on a physical topology. To accomplish this, logical grouping requires the use of a “VLAN-aware” switch.
AU6039.indb 143
2/13/08 9:22:59 AM
144 n Carrier Ethernet: Providing the Need for Speed
Carrier Ethernet Service
1000 Base-BX 10 or another EFM port
Ethernet Switch
Figure 5.10 Using Ethernet to link geographically separated locations within a metropolitan area
Implicit versus Explicit Tagging Two methods can be used to form a VLAN: implicit tagging and explicit tagging. Implicit tagging eliminates the need for a special tagging field to be inserted into frames. Examples of implicit tagging can include the use of MAC addresses, port numbers, protocols transported by a frame, or another parameter that permits nodes to be grouped into a broadcast domain. In comparison, explicit tagging requires the addition of a field into a frame or packet. One of the initial disadvantages associated with explicit tagging is the fact that it increases the length of an Ethernet frame. This resulted in a degree of incompatibility between equipment until a significant base of VLAN-aware switches and network adapters reached the market. In the remainder of this section we will examine the use of implicit tagging prior to discussing the IEEE VLAN standard.
Using Implicit Tagging As previously mentioned, MAC addresses, port numbers, and protocols when used to create VLANs can be considered to represent implicit tagging methods. To illus-
AU6039.indb 144
2/13/08 9:23:00 AM
LAN Switches n 145
trate the creation of VLANs based upon implicit tagging, consider the eight-port Ethernet switch shown in Figure 5.11. In this example the ports are numbered 0 through 7, and the MAC address of each device connected to the switch for simplicity is indicated as A0 through A7. We will further assume that the clients with MAC addresses A0 through A5 will be associated with the first VLAN, and clients with the MAC addresses A6 through A8 will be associated with the second VLAN. Also note that of the two servers shown connected to the switch, one has an address such that it resides in the first VLAN, and the address of the second server places that device in the second VLAN. In examining Figure 5.11, note that each VLAN represents a logical grouping of ports on top of the physical topology of the network. Each group of ports represents an independent broadcast domain, such that frames transmitted by a workstation or server on one domain remain constrained to that domain. In addition to MAC-based implicit tagging, it is also possible to create implicittagged VLANs based upon the port where a frame enters a switch or the protocol carried in a frame. For port-based VLANs an administrator could simply use the switch console to assign a group of ports to each VLAN. The creation of protocolbased VLANs can be more difficult, because the switch has to look deeper into each frame to determine the protocol being transmitted.
Explicit Tagging Explicit tagging involves the insertion of a VLAN tag into frames. Prior to the introduction of the IEEE 802.1Q standard several vendors developed proprietary methods to support VLAN creation. Two examples of those proprietary methods are Cisco’s ISL (Inter-Switch Logic) and 3Com’s VLT (Virtual LAN Trunk). Because the IEEE 802.1Q standard is today used by most, if not all, vendors offering VLANs, we will turn our attention to that standard.
The IEEE 802.1Q Standard The IEEE 802.1Q standard defines a method for tagging Ethernet frames with VLAN membership information as well as defining how VLAN bridges (switches) operate. Under the IEEE 802.1Q standard, compliant switch ports can be configured to transmit tagged or untagged frames. A tag frame contains fields for both VLAN and 802.1P priority information that can be transmitted between 802.1Q-compliant switches, enabling a VLAN to span multiple switches. However, it is important to ensure that intermediate as well as endpoint or edge switches are all 802.1Q compliant, as many NICs and legacy switches do not support VLANs. If such devices receive a tagged frame, they will more than likely discard it as they will either not understand why the value in the legacy length field is incorrect or that the maximum length of a frame was increased from 1518 to 1522 bytes.
AU6039.indb 145
2/13/08 9:23:00 AM
146 n Carrier Ethernet: Providing the Need for Speed
Server A7
Server A5
0
1
vLan 1
2
vLan 2
Switch
3
4
5
6
A3
A1 A2
A6 A6
A4
vLan 1 = Ports 0, 2, 3, 4, 5
7
vLan 2 = Ports 1, 5, 7
Figure 5.11 Establishing a MAC-based VLAN
The VLAN Tag Figure 5.12 illustrates the VLAN Ethernet frame. Note that the VLAN tag, which consists of two bytes, is inserted between the source address and length fields. The first two bytes in the VLAN tag are referred to as the tag protocol identifier (TPID), which has a defined value of hex 8100, indicating that the frame carries IEEE 802.1Q/ 802.1P data. Preamble
SFD Destination Address
Hex 8100
Source Address
802.1Q Tag Type
Tag Control Information
Length /Type
Data
FCS
bits: 3 1 12 User_Priority CFI vLAN ID CFI Canonical Form Indicator
Figure 5.12 The Ethernet VLAN frame
AU6039.indb 146
2/13/08 9:23:01 AM
LAN Switches n 147
The second two bytes in the VLAN tag are referred to as the tag control information (TCI) field. This field consists of three subfields: user priority (3 bits), canonical format indicator (1 bit), and VLAN ID (VID), with the latter 12 bits in length. The user priority field provides eight (23) priority levels. The CFI 1bit subfield is used for compatibility between Ethernet and Token-Ring networks. The CFI is always set to 0 for Ethernet switches. If the CFI is set to 1, the frame should not be forwarded to an untagged port. The third subfield, VID, allows up to 4096 (212) VLANs to be identified; however, a VID of 0 is used to identify priority frames, and a value of hex FFF is reserved, reducing the maximum number of VLANs to 4094.
VLAN Traffic The VLAN standard defines three types of traffic: untagged, priority tagged, and VLAN tagged. Untagged frames do not have a VLAN tag. Priority-tagged frames are VLAN-tagged frames that have a valid priority setting and a VID (VLAN ID) set to zero, also referred to as a NULL VID. The third type of traffic, VLAN tagged, contains a VLAN tag with a non-zero VID field value.
802.1P Signaling The three priority bits enable network traffic to be prioritized at the MAC layer. The use of the 802.1P standard allows users to prioritize traffic into various classes; however, no bandwidth reservations are established. Switches, routers, servers, and even desktop computers that support VLAN tagging can set these priority bits. Although there is no standard that maps the priority field values to priority queues, the IEEE provided recommendations concerning how traffic classes correspond to priority values. From these recommendations, network managers can then assign 802.1P values to queues supported by switches, routers, and other devices. For example, 802.1P values of 0 through 3 could be assigned to a low-priority queue, priority levels 4 through 6 could be assigned to a medium-priority queue, and priority 7 could be assigned to a high-priority queue.
VLAN Operations The IEEE 802.1Q standard is limited with respect to the manner by which it handles untagged frames. Under the 802.1Q standard, only a per-port VLAN solution is employed for untagged frames. This means that assigning untagged frames to VLANs only considers the port from which they were received. Each port has a parameter called a permanent virtual identification (PVID) that specifies the VLAN assigned to receive untagged frames. IEEE 802.1Q-compliant switches can
AU6039.indb 147
2/13/08 9:23:01 AM
148 n Carrier Ethernet: Providing the Need for Speed
be configured either to admit only VLAN-tagged frames or all frames, with the latter including untagged frames. Initially, each physical port on an 802.1Q switch is assigned a PVID value that represents its native VLAN ID whose default value is VLAN 1. All untagged frames are assigned to the VLAN specified in the PVID parameter and the PVID is considered as a tag. However, because the frame is untagged and the PVID is tagged, this enables both tagged (VLAN-aware) and untagged (VLAN-unaware) devices to co-exist on a common network infrastructure. Figure 5.13 illustrates how VLAN-aware and -unaware stations can co-exist. In examining Figure 5.13, note the two VLAN-unaware end stations shown in the lower left portion of the figure. Because they are VLAN-unaware, they will be associated with VLAN C, assuming that the PVIDs of the VLAN-aware switches in this example are set equal to VLAN C. The VLAN-unaware stations only transmit untagged frames, which informs the VLAN-aware devices that receive such frames to assign them the VLAN C.
Ingress Rules Each frame received by a VLAN switch can only belong to a single VLAN. This is accomplished by associating a VID value with the received frame. If the VID is set to 0 (null VLAN ID), then the tag only transports priority information and is not used for configuring a PVID. In fact, if the port is set to a value admitting only VLAN-tagged frames, such frames are then dropped. If the VID is not the null VLAN ID, then the VLAN identifier parameter is used. However, a VID of hex FFF is reserved and is not configurable as a PVID. All frames that are not discarded as a result of the application of ingress rules are sent to the forwarding process and learning process prior to exiting the switch based upon 802.1Q egress rules.
The Forwarding Process The forwarding process allows frames to be forwarded for transmission on some ports, referred to as “transmission ports,” and discarded without being transmitted to other ports. Each port can be considered as a potential transmission port only if it is in a forwarding state, the frame was received on a port that was in a forwarding state, and the port considered for transmission is not the same port on which the frame was received. Once a frame is permitted to be forwarded, it is subject to filtering. Filtering can occur based upon the destination MAC address in the frame, its VID, information in the filtering database for that MAC address and VID or a default filtering behavior specified for the potential transmission.
AU6039.indb 148
2/13/08 9:23:01 AM
LAN Switches n 149
vLAN A PVID = A
PVID = C
vLAN A PVID = A
vLAN Aware Switch
Access Ports
PVID = C
PVID = B vLAN B
vLAN Aware Switch
Access Ports PVID = B vLAN B
PVID = C vLAN Unaware Endstation
vLAN Unaware Switch vLAN C vLAN Unaware Switch
PVID = C
PVID = B
vLAN Aware Endstation vLAN B
PVID: Permanent Virtual Identifier
Figure 5.13 Coexistence of VLAN-aware and -unaware stations
Frame Queuing The forwarding process can result in frames being queued prior to flowing to a destination port. Both unicast and group-addressed frames with a given user priority are either sent in the order of receipt or assigned to storage queues based on their user priority. The user priority can be mapped to one to eight traffic classes. The latter occurs when a VLAN switch supports priority queuing. Table 5.4 illustrates IEEE-recommended user priority to traffic class mappings. In examining Table 5.4, note that up to eight traffic classes are supported by the traffic class tables, which enables separate queues for each level of user priority. Traffic classes are numbered 0 through N − 1, where N represents the number of traffic classes associated with a defined outbound port. If the forwarding process does not support expedited classes of traffic, the user priority value is mapped to traffic class 0, which corresponds to non-expedited traffic. Once a frame is queued it will normally be processed and transmitted through a port. However, there are several situations where the frame can be removed from the port: if the buffering time is guaranteed and then exceeded, if the bridge transit time reaches the time at which the frame would be transmitted, and if the associated port leaves the forwarding state.
AU6039.indb 149
2/13/08 9:23:02 AM
150 n Carrier Ethernet: Providing the Need for Speed
Table 5.4 Recommended User Priority to Traffic Class Mappings Number of Available Traffic Classes 1
2
3
4
5
6
7
8
0
0
0
1
1
1
1
2
1
0
0
0
0
0
0
0
0
2
0
0
0
0
0
0
0
1
3
0
0
0
1
1
2
2
3
4
0
1
1
2
2
3
3
4
5
0
1
1
2
3
4
4
5
6
0
1
2
3
4
5
5
6
7
0
1
2
3
4
5
6
7
User priority 0 (default)
Frame Selection Once frames are ready for transmission, an algorithm is used for their selection. This algorithm results in the selection of frames for a given supported traffic class from the corresponding queue only if all queues corresponding to numerically higher values of traffic classes supported by the port are empty at the time of selection.
Egress Rules Once a frame is selected, egress rules are applied to control the exit of frames from switch ports. A transmitting port only transmits VLAN tagged or untagged frames. This means a VLAN-compliant switch cannot transmit priority-tagged frames. Thus, a station that transmits a priority-tagged frame via a switch will receive a response that is either VLAN tagged or untagged, with the actual response dependent upon the state of the untagged set for the VLAN concerned.
The Learning Process Similar to how bridging learns MAC addresses and ports on which frames enter the device, a VLAN switch observes the MAC source address of frames received on each port and their VID. As the source MAC address and VID are learned, the learning process updates the information in a filtering database. If the filtering database is full, an existing entry is removed to allow the new entry to be
AU6039.indb 150
2/13/08 9:23:02 AM
LAN Switches n 151
entered into the database. Normally a default aging time of 300 sec purges rarelyused entries. The filtering database can contain both static entries entered via a management configuration process and dynamically learned entries. Such entries can be used to control the forwarding of frames with particular VIDs, the inclusion or removal of tag headers in forwarded frames, and the use of the source MAC address, destination MAC address, VID, and the port on which a MAC address was learned to perform some predefined action.
Vendor Implementation Although the IEEE 802.1Q standard defines a solid base for establishing and using VLANs, many vendors implemented subsets of the standard as well as tailored the capability of their switches to add other features, such as VLAN creation based upon IP addressing, IP applications, or another category beyond that available by the 802.1Q standard. In the remaining portion of this section, we will briefly examine how the 3Com SuperStack family of switches supports VLANS.
SuperStack Switch The SuperStack switch is limited to supporting up to 16 VLANs using the 802.1Q standard. Each switch port can be associated with any single VLAN defined or placed in multiple VLANs at the same time using 802.1Q tagging. Prior to the switch being able to forward traffic, you need to define the following information about each VLAN: VLAN name such as Engineering or Marketing 802.1Q VLAN ID used to identify the VLAN when 802.1Q tagging is used across your network Local ID used to identify the VLAN within the switch which corresponds to the VLAN ID used in legacy 3Com devices The 3Com switch supports VLT tagging. This tagging is a proprietary 3Com system that allows a VLAN to be placed in all VLANs defined for a switch.
The Default VLAN A new or initialized switch contains a single VLAN, which is the default VLAN. The VLAN has an 802.1Q VLAN ID of 1 and a local ID of 1. Initially, all ports are placed in the default VLAN, and it is the only VLAN that can be used to access 3Com management software of the switch.
AU6039.indb 151
2/13/08 9:23:02 AM
152 n Carrier Ethernet: Providing the Need for Speed
Defining VLANs For the first example we will assume we are creating VLANs using implicit (untagged) connections. Suppose we have a 24-port switch connected to four end stations and two servers as shown in Figure 5.14. Using the VLAN setup page, we would first define VLAN 2, because VLAN 1 is the default VLAN and already exists. Next, we would edit the port settings using an untagged VLAN list box so that ports 1, 3, and 13 of the switch are placed in VLAN 1 and ports 10, 12, and 24 of the switch are placed in VLAN 2.
Untagged Connections with Hubs The VLAN shown in Figure 5.14 can be considerably expanded by connecting switch ports to hubs instead of to individual end stations. If we assume that the switch now has a Layer 3 module installed, it can pass traffic between VLANs. In the example shown in Figure 5.15, a hub is connected via its port number 13 to port number 1 on a switch that has a Layer 3 switching module installed. In this example the switch has ports 1 and 14 assigned to VLAN 1, and ports 2, 7, and 24 are assigned to VLAN 2. Using the switching module, VLANs 1 and 2 can communicate with each other. The configuration of the VLAN shown in Figure 5.15 again requires the use of the VLAN list box on the 3Com port setup page of the switch’s Web interface, resulting in ports 13 and 14 placed in VLAN 1 and ports 2, 7, and 24 placed in VLAN 2. To enable communications between the two VLANs, the Layer 3 module would then be configured. Finally, port 13 on the hub would be cabled to port 13 on the switch.
Stations in vLAN 1
Server in vLAN 1
Stations in vLAN 2
Server in vLAN 2
Figure 5.14 Creating two VLANs on a 3Com SuperStack 24-port switch
AU6039.indb 152
2/13/08 9:23:03 AM
LAN Switches n 153
Stations in vLAN 1
Stations in vLAN 2
Conventional Hub
Switch with layer 3 Switching module
Server in vLAN 1
Server in vLAN 2
Figure 5.15 Expanding a VLAN and enabling inter-VLAN communications
802.1Q Tagged Connections Although untagged VLANs are fine to use with a single switch in a 3Com environment, they are not able to support multiple switches. Thus, in a network with more than one switch where VLANs are distributed among multiple 802.1Q tagging must be used. This requirement enables VLAN traffic to be passed along the trunks routed to connect switches. Figure 5.16 illustrates the interconnection of two VLAN switches using 802.1Q tags. In this example each switch has end stations that are in VLAN 1 and VLAN 2. In addition, each switch has a server for a VLAN, with all stations in VLAN 1 requiring the ability to connect to the server attached to switch 1 and all stations in VLAN 2 require the ability to connect to the server attached to switch 2. In examining the two switches shown in Figure 5.16, note that the untagged VLANs are configured as previously discussed. To provide inter-switch communications, port 26 on switch 1 is assigned to VLANs 1 and 2, so that all traffic will be passed over the trunk to switch 2. On switch 2, port 25 would be assigned to VLANs 1 and 2, enabling all VLAN traffic to flow to switch 1. Once this is accomplished, port 26 on switch 1 would be connected to port 25 on switch 2, allowing end stations in both VLANs to communicate with their applicable servers that can be on the same switch or on a different switch.
Supporting 802.1Q Learning In our concluding example, we will examine the use of three switches that support the 802.1Q learning process. In the example shown in Figure 5.17, each end
AU6039.indb 153
2/13/08 9:23:04 AM
154 n Carrier Ethernet: Providing the Need for Speed Stations in vLAN 1 and vLAN 2 (untagged) Switch 1
Port 26 vLANS 1 and 2 Server in vLAN 1 (untagged)
802.1Q tagged Stations in vLAN 1 and vLAN 2 (untagged)
Switch 2
Port 25 vLAN 1 and 2 802.1Q tagged
Server in vLAN 2 (untagged)
Figure 5.16 Using 802.1Q tagged connections for interconnecting switches
station informs the network that it is to receive traffic for certain VLANs and the switches automatically place the end stations in those VLANs. In addition, the trunks between switches are automatically configured to forward traffic that contains unknown 802.1Q tags. To support the configuration shown in Figure 5.17, you would configure switch 1 so the end stations belong to VLANs 1, 2, and 3. Similarly, you would configure switch 2 so the stations are assigned to VLANs 4, 5, and 6. Next, you would enable 802.1Q learning on each switch and configure the Layer 3 module to allow communications between VLANs 1 through 6. To complete the network you would then connect port 26 on switch 1 to port 1 of switch 3, and port 25 of switch 2 to port 2 on switch 3. The 3Com SuperStack switch is limited to supporting 16 VLANs and the 802.1Q standard supports up to 4094. Thus, if a network contains stations that support 802.1Q, the 3Com switches may have to forward traffic that uses unknown 802.1Q tags. Such traffic is automatically forwarded if a 3Com SuperStack switch has 802.1Q learning enabled, as assumed in Figure 5.17. In this example each switch is assumed to have 802.1Q learning enabled so they can place stations in a VLAN. In addition, 802.1Q learning enables applicable VLAN traffic to reach a station from anywhere in the network. Thus, traffic from VLANs 1, 2, and 3 as well as unknown tags are forwarded to switch 3 from switch 1, and traffic from
AU6039.indb 154
2/13/08 9:23:04 AM
LAN Switches n 155 Switch 1: Stations configured to belong to vLANs 1, 2, 3 802.1Q learning enabled
vLANs 1, 2, 3 802.1Q tagged/ unknown tagged forwarded
Switch 2: Stations configured to belong to vLANs 4, 5 and 6 802.1Q learning enabled
vLANs 4, 5, 6 802.1Q tagged/unknown tagged forwarding Switch 3: Layer 3 module using 802.1Q learning
Figure 5.17 Using 802.1Q learning
VLANs 4, 5, and 6 as well as unknown tags are forwarded from switch 2 to switch 3. Under 802.1Q learning, stations transmit a packet with a known multicast address to the entire network. This traffic informs other devices that the station should receive traffic for specific VLANs. Then, when a packet arrives at a port on a switch with 802.1Q learning enabled, the switch places the receiving port in the specified VLANs and forwards the packet to all other ports. When the frame arrives at another switch with 802.1Q learning enabled, that switch also places the receiving port in the VLAN specified and forwards the frame to all other ports, enabling VLAN information to be propagated throughout the network, which enables VLAN traffic to reach stations from anywhere in the network.
Advantages of Use Previously it was mentioned that a VLAN represents a broadcast domain. One of the key advantages associated with the use of VLANs is the ability to tailor operations by adjusting the broadcast domain. That is, you can increase the number of broadcast domains, but reduce the number of stations in each to correspond to employee groupings, departments, or even floor locations. This in turn can reduce network traffic and even increase security as a member of one domain can be restricted from
AU6039.indb 155
2/13/08 9:23:05 AM
156 n Carrier Ethernet: Providing the Need for Speed
being able to view traffic in another domain. Other advantages associated with the use of VLANs include facilitating adds, moves, and changes and reducing the effort required to segment networks.
AU6039.indb 156
2/13/08 9:23:05 AM
Chapter 6
Carrier Ethernet Services
Overview In less than a decade Ethernet has rapidly evolved into a full duplex, point-to-point Layer 2 protocol that no longer has the possibility of experiencing collisions. Operating at data rates from 10 Mbps to 10 Gbps on a variety of copper and fiber media the original distance limitations of Ethernet have been significantly exceeded to the point where the technology can be used to connect geographically separated offices. Recognizing a new source of revenue, communications carriers have gradually responded to customer requirements and have begun to offer metropolitan Ethernet, which in the past few years was renamed Carrier Ethernet. As its name implies, Carrier Ethernet, represents a service offered by a communications carrier: the transportation of data by Ethernet frames at Layer 2 in the ISO Open System Interconnection Reference Model. In this chapter, we will turn our attention to Carrier Ethernet. In doing so, we will obtain an overview of the technology and we will discuss why many communications carriers do not provide Carrier Ethernet service at present. Then we will go into more detail, examining how Ethernet and MPLS can be combined as well as how we can access Carrier Ethernet.
The Metro Ethernet Forum The concept for Carrier Ethernet dates to 2001 when the Metro Ethernet Forum (MEF) was formed to develop business services for customers of communications 157
AU6039.indb 157
2/13/08 9:23:05 AM
158 n Carrier Ethernet: Providing the Need for Speed
carriers that would be accessed primarily over optical metropolitan networks as a mechanism to connect Enterprise LANs. One key objective of the MEF is Carrier Ethernet services. In fact, the forum’s objective is stated as: “Carrier Ethernet services shall be delivered over native Ethernet-based Metro and Access networks and can be supported by other transport technologies.” The MEF defines Carrier Ethernet as a “ubiquitous, standardized, carrier-class service defined by five attributes that distinguish Carrier Ethernet from familiar LAN-based Ethernet.” Those attributes include standardized services, scalability, service management, reliability, and Quality of Service (QoS). Since 2001 the MEF has developed a series of 16 specifications to define one or more Carrier Ethernet attributes, including standardized services, service management, reliability, QoS, and scalability.
Requirements for Use In extending Ethernet from the LAN into the metropolitan area, native Ethernet protocols need extensions to become scalable, obtain a QoS capability and resiliency as well as provide Operation, Administration, and Maintenance (OAM) support, with the latter of key importance for communications carriers to monitor provisioning and maintain their network infrastructure. Over the past decade two trends have emerged for transporting Ethernet into and through metropolitan area networks: (1) protocol extensions and (2) encapsulating Ethernet within another transport technology such as MPLS. Unlike an Ethernet LAN, which is dedicated for use by an organization or organizational departments, Carrier Ethernet needs to have the ability to provide service to different organizations. Thus, the first requirement for developing a Carrier Ethernet service is for the service to support multiple customers. This requirement was satisfied by the use of a relatively old Ethernet technology that was originally intended to represent an enterprisewide technology, the virtual LAN (VLAN). By tagging Ethernet frames of customers it becomes possible for the Carrier Ethernet service provider to allow different customers to use the same Ethernet infrastructure without incurring a security risk. To provide readers with a review of VLAN tagging, this author will be a bit redundant instead of simply referencing an earlier portion of this book.
VLAN Tagging Under the IEEE 802.1Q standard four bytes are inserted into each Ethernet frame. For convenience, Figure 6.1 represents a duplicate of Figure 4.7, which indicates where the four bytes are inserted as well as the three subfields of the tag control information field.
AU6039.indb 158
2/13/08 9:23:05 AM
Carrier Ethernet Services n 159 bytes: 7
1
Preamble
6
SFD Destination Address
6
2
Source Address
bits:
2
Tag Tag Type Control hex 8100 Information
2
??
Length/ Type
User Priority
CFI
vLAN ID
3
1
12
??
Data 42-1500
4 FCS
Figure 6.1 The 802.1Q frame format
In examining 802.1Q bytes note that the first two, which are in the tag type field, are set to hex 81-00. The following 2 bytes of tag control information consist of a 3-bit user priority subfield, a 1-bit Canonical Format Indicator (CFI) and a 12bit VLAN ID. The first two bytes are always set to hex 81-00 to identify the frame as an 802.1Q frame. The second 2 bytes, which are subdivided into three subfields, identify a priority level for the frame, whether bit order is canonical or non-canonical (or can have additional significance based upon the MAC Protocol), and the third subfield identifies the VLAN. This 12-bit field permits 212 − 2 or 4094 unique VLANs. Although 4094 unique VLANs are more than adequate for many metropolitan areas, this number is not sufficient for large cities where tens of thousands of traditional T1 and T3 access lines could be replaced by Carrier Ethernet. Thus, the IEEE modified the 802.1Q specification to significantly enhance the number of definable VLANs. That modification is referred to as the IEEE 802.1ad specification, titled “Provider Bridges.” This specification is actually an amendment to the IEEE 802.1Q-1998 standard. The purpose of this amendment, according to the IEEE, is to enable an architecture and bridge protocols, compatible and interoperable with existing bridged local area network protocols and equipment, to provide separate instances of the MAC services to multiple independent users of a bridged local area network in a manner that does not require cooperation among the users, and requires a minimum of cooperation between the users and the provider of the MAC service. By following the specifications in this amendment a service provider can now offer the equivalent of separate LAN segments, bridged or virtual bridged LANs, to a number of users, over the provider’s bridged network.
AU6039.indb 159
2/13/08 9:23:06 AM
160 n Carrier Ethernet: Providing the Need for Speed
While work proceeded on the IEEE 802.1ad amendment various Layer 2 encapsulation schemes have been either proposed or performed by vendor equipment to address the scalability issue. Such schemes include VLAN stacking, MAC address stacking, and MPLS Layer 2 encapsulation. These techniques will be discussed later in this chapter.
The 802.1P (Priority) Standard One of the key requirements of Carrier Ethernet is to provide a QoS capability. While QoS is not directly possible, indirectly the first subfield in the tag control information field is used to indicate the priority of a frame. It is important to note that the priority value corresponds to a Class of Service (CoS) and does not directly provide a QoS. This difference is significant and deserves a degree of explanation. For QoS to work, a path is set up from the network access to network egress such that each device along the path provides a certain level of service at a specified data rate. As the path is established, each device obtains the ability to accept or reject the proposed connection based upon its available resources, similar to the routing of a call through the telephone network that can result in a fast busy signal when too many customers attempt to dial long distance from a given serving switch. If we focus our attention upon Ethernet, we would note that it was developed as a connectionless technology. This means that it is not possible to predefine a path for a service nor to pre-allocate bandwidth along a path. Instead, QoS mechanisms would be used to prioritize frames belonging to different traffic classes while switches and routers would use queuing to favor certain traffic classes over other classes. Unfortunately, this will not guarantee an end-to-end bandwidth and QoS, although it will prioritize traffic based upon different classes of service. In comparison to a QoS environment where a path is set up, in a CoS environment frames are simply marked by a sender to indicate their priority and do not have to follow a specific path through a network. This means network devices do not receive the ability to refuse higher-priority connections, in effect making the network administrator responsible for ensuring that the network is not over-committed with high-priority traffic. For example, if you have a 1 Gbps link and only have 200 Mbps of priority traffic that will flow over the link, the use of CoS will not cause any problem. Thus, CoS provides a mechanism for expediting various types of network traffic as long as the network administrator takes care to provision such traffic with recognition of the capability of the link speeds in the network.
Latency Considerations Although file transfer and e-mail are not adversely affected by latency, the same cannot be said concerning such real-time applications as videoconferencing and VoIP. When considering the use of Carrier Ethernet, it is important to examine the delays
AU6039.indb 160
2/13/08 9:23:07 AM
Carrier Ethernet Services n 161 Customer A Location B Customer Switch
Carrier Switch Carrier Switch
Carrier Switch
ec nn
Customer A Co Location A EFM Customer Switch
E Gb
n
tio
Carrier Switch
Li
nk
Carrier Switch
Service Provider Network Carrier Ethernet Service
Customer B Location A Customer Switch
Figure 6.2 Latency on a Carrier Ethernet network
along the route or path between the access and egress locations. Figure 6.2 illustrates the use of a Carrier Ethernet network to transport frames between two locations. In examining Figure 6.2, it is assumed that the Carrier Ethernet network is used to interconnect two geographically separated customer networks at 1 Gbps. The customer switch at each location is assumed to support 10/100/1000 Mbps, with the latter including a 1000BASE-BX10 interface that provides a bidirectional transmission rate of 1 Gbps over a single-mode fiber at distances up to 10 km.
Switch Latency Switch latency, which represents the time it takes for a frame to flow from an input port to an output port, depends upon several factors to include but not limited to the type of switch (cut-through or store-and-forward), average frame length, switch congestion, and port operating rates. Table 6.1 indicates the average latency computed from an analysis of ten vendor switches that included either twelve autosensing 10/100/1000BASE-T and four 1000BASE-SX ports or eight 1000BASE-SX ports. The table indicates the effect of both unloaded and loaded conditions when the frame was 64, 512, and 1518 bytes in length. Here an unloaded switch is assumed to represent a switch that has a load at 50 percent or less of its maximum throughput, while it is considered to be loaded when data is flowing into N/2 ports, where N represents the total number of switch ports.
AU6039.indb 161
2/13/08 9:23:07 AM
162 n Carrier Ethernet: Providing the Need for Speed
Table 6.1 Switch Latency (msec) 64
512
1518
Unloaded Loaded Unloaded Loaded Unloaded Loaded 6.15
33.78
12.48
125.06
27.04
132.06
In examining the data contained in Table 6.1, several comments are warranted. First, the entries represent the average delays associated with ten switches. Second, an increase in the average frame length resulted in an increase in latency regardless of the load on the switch. Third and most important, the load on a switch can have a significant effect upon latency, with a loaded switch on the average having a latency between slightly over five times that of an unloaded switch when frames are 64 bytes in length to over ten times that of an unloaded switch when frames are 1518 bytes in length. Because VoIP applications almost always use relatively short frames we can focus our attention upon the delays associated with 64-byte frames. For our simple example shown in Figure 6.2, assuming each switch is unloaded would result in a cumulative switch delay of 6.15 * 4 or 24.6 μs.
Access and Egress Delays The previously computed switching delays represent only a portion of the end-toend delay we need to consider. Other delays include the access and egress delays as well as the delay associated with moving data through the carrier network. The access or ingress delay represents the time required to transport the frame from the customer presence into the Carrier Ethernet network. If we assume a 1Gbps EFM (Ethernet in the First Mile) connection, then each bit requires 1/109 or 1.0 * 10−9 sec. If a 64-byte frame is transported, adding the header and trailer results in an additional 26 bytes, for a total of 90 bytes or 720 bits. Thus, the access or egress time to transfer the frame becomes 720 bits * 1. 1.0 * 10 −9 bits/sec = 0.72 μs. If we assume the egress connection also operates at 1 Gbps, then the total ingress and egress delays are 0.72 * 2 or 1.44 μs.
Frame Transport Because Carrier Ethernet makes use of an optical carrier, the transport of data occurs through the network at the speed of light. This means that the delay associated with the transport of frames through the network can be simplified by only considering the number of switches frames flow through and then computing the time required to place the frames onto the optical carrier. Returning to Figure 6.2, two carrier switches are shown on the path from location A to location B. Thus, at 1
AU6039.indb 162
2/13/08 9:23:08 AM
Carrier Ethernet Services n 163
Gbps the frame transport time is 0.72 μs. Adding up the switch delays, ingress and egress delays, and the frame transport delays, the total frame delay becomes 24.6 μs + 0.72 μs + 0.72 μs or 26.04 μs. Because the transport of digitized voice requires a cumulative delay of less than 150 ms the network shown in Figure 6.2 does not appear to represent a problem. However, it should be noted that the prior example did not consider the fact that a higher priority data stream could lock out a lower priority data stream from being processed by routers or switches for a period of time that could significantly increase latency. In addition, not all carriers have optical solutions for EFM access that operate at 1 Gbps. In fact, some communications carriers offer copper-based solutions, such as 2BASE-TL, which provides a maximum data rate of 2 Mbps at distances up to 2700 m. When 2BASE-TL is used for access to a Carrier Ethernet network, the delay becomes 720 bits * 1/2 * 106 or 300 * 10−6 μs, which is 0.3 ms. Thus, even if the egress line is 2BASE-TL the total access and egress delays would be 0.6 ms. When added to the switch delays and frame transport delays the total latency should still be significantly under 150 ms or 150,000 μs when frames are prioritized. Thus, the use of Carrier Ethernet coupled with EFM access and egress is well suited for transporting VoIP and other real-time applications as long as a method exists to prioritize such frames.
Fiber Connectivity There are a large number of commonly available methods to access a Carrier Ethernet network. Those methods include both copper- and fiber-based EFM technology as well as unused fiber and various types of switch and router modules connected to EFM and dark fiber transports.
Dark Fiber “Dark fiber” is a term used to define previously installed but currently unused fiber. Because of economics, many carriers install fiber bundles into office buildings and only initially use a fraction of the fibers in the bundle. Then, as traffic requirements increase, the carrier may “light” various dark fiber strands.
Gigabit Interface Converters Although dark fiber and various EFM copper and fiber methods provide a transport facility, they require an interface to equipment. One of the most popular types of equipment interface is the Gigabit Interface Converter (GBIC). In a Cisco environment the GBIC can be plugged into certain models of Ethernet switches and router ports, after which you would connect the optical cable to the GBIC. Table 6.2 lists three types of optical fiber for which you can obtain a GBIC from Cisco. At the
AU6039.indb 163
2/13/08 9:23:08 AM
164 n Carrier Ethernet: Providing the Need for Speed
Table 6.2 Common GBICs for Optical Fiber IEEE
Wavelength (nm) Fiber Type
Maximum Distance (km)
1000BASE-SX
850
Multi-mode
0.2–0.5
1000BASE-LX/LH
1310
SMF, NDSF
10
1000BASE-ZX
1550
SMF, NDSF
70–100
Note: SMF = single-mode fiber, NDSF = non-dispersion shifted fiber, NZ-DSF = non-zero dispersion shifted fiber.
time this book was written a Cisco 1000BASE-ZX GBIC module was available for approximately $1700 and a 1000BASE-LX/LH long haul, long-wavelength module was obtainable for slightly more than $100. In comparison, a D-Link GBIC for 1000BASE-SX could be obtained for $160. A single-mode fiber is an optical fiber designed to transport a single ray of light. Because such fibers do not significantly exhibit dispersion, they can transport light pulses at greater distances than multi-mode fiber. Unfortunately, lasers have a range of optical wavelengths that result in the wavelengths becoming spread out in time as they traverse the fiber. This spreading of wavelengths is referred to as chromatic dispersion. Standard single-mode fiber has a near-zero dispersion at 1310 nm and represents most of the optical cable installed during the l980s. This type of cable is also referred to as non-dispersion shifted fiber. In comparison, non-zero dispersion shifted fiber is fabricated to support high power signals over long distances as well as dense wavelength division multiplexing.
Transporting Ethernet in a Service Provider Network Over the past decade carrier networks evolved from a reliance on copper-based technology to fiber-based technology. However, the selection of fiber does not provide a uniform transport mechanism. Instead, transport technologies were developed ranging from coarse and dense wavelength division multiplexing to SONET/SDH, ATM, Frame Relay, Switched Ethernet, Resilient Packet Ring (RPR), MPLS, and IP to transport data. Thus, the infrastructure of many communication carriers resembles a fruit salad of equipment acquired over a relatively long period of time.
Operating over Other Transports In discussing Carrier Ethernet, it is important to note that the term does not imply that Ethernet is used end to end. Although Ethernet can be used as a transport
AU6039.indb 164
2/13/08 9:23:08 AM
Carrier Ethernet Services n 165
medium, it can also run over different types of transport facilities. Those facilities can include SONET, Resilient Packet Ring, and even MPLS.
Comparison to Other Layer 2 Protocols The transmission of Ethernet at Layer 2 has several key differences when compared to such Layer 2 protocols as ATM and Frame Relay. The latter two provide an intelligent forwarding mechanism at Layer 2, which in effect is a routing protocol. In comparison, switched Ethernet at Layer 2 has no such intelligence. Instead, frames are processed according to the 3 Fs: filtering, forwarding, and flooding of frames based upon their destination MAC addresses. Because subscribers are associated with MAC addresses that cannot be grouped into a sequence of addresses (like IP addressing where addresses can be subdivided into a network address, subnet address, and host address), each MAC address must be learned and maintained in tables. This represents another limitation associated with Carrier Ethernet, especially if a failure occurs in a metropolitan area serving a large number of subscribers. Then, equipment relearning MAC addresses would require a considerable amount of time, which would adversely affect carrier resilience. This means that IP or MPLS or another transport mechanism was preferred because they could be scaled to support tens of thousands of customers as well as provide routing intelligence and carrier resilience.
Ethernet Topologies When data is transmitted via Ethernet there are two general topologies that can be used: point-to-point and ring. The selection of an Ethernet topology is normally based upon the existing carrier infrastructure. For example, if a SONET/SDH network already exists, Ethernet could be laid over the physical SONET/SDH ring. If a fiber ring exists, Ethernet components could be daisy-chained by interconnecting Gigabit switches to form a ring or coarse WDM could be used to provide a series of point-to-point Gigabit Ethernet circuits over the physical fiber ring. Thus, in many cases the current carrier infrastructure will be the driving force concerning how Ethernet is transported in the metropolitan area.
Carrier Ethernet Service Types Although we speak of Carrier Ethernet as a service provider transport facility, we previously noted that an Ethernet service can be based upon virtually any transport technology, such as Ethernet over SONET, MPLS, and so on. Another method used by Carrier Ethernet service providers to differentiate their Carrier Ethernet offerings is by defining their offerings by service type. Thus, a specific Carrier
AU6039.indb 165
2/13/08 9:23:08 AM
166 n Carrier Ethernet: Providing the Need for Speed
Router
Carrier Ethernet Network
Router
Figure 6.3 E-LINE service
Ethernet service type that actually represents a topology can also be transported as a native Ethernet or carried by another transport facility. Presently, three types of Ethernet Services can be offered by service providers: E-LINE, E-LAN, and E-TREE. Each type corresponds to a specific type of network topology or architecture. Of the three, E-LINE and E-LAN are well defined; E-TREE can be considered to represent an evolving work in progress for which standards have yet to be finalized.
E-LINE E-LINE represents a point-to-point Ethernet connection. This connection can be used to interconnect two geographically dispersed offices within a metropolitan area, as illustrated in Figure 6.3. E-LINE can be considered to represent a leased line replacement that offers a much higher bandwidth than such conventional telco services as T1 and T3 connections. In examining Figure 6.3, note that the most common endpoint to an E-LINE service is to use routers with the correct optical interface at each edge of the carrier network. Because most modern switches and routers accept various optical modules, interfacing an E-LINE service offering is readily available.
E-LAN While E-LINE service is used to connect two locations, E-LAN represents an Ethernet service that can be used to connect multiple locations. Thus, you can view ELAN as resembling a multi-point service that provides an “any-to-any” connection, similar to a VLAN operating over any type of public network. The primary use of an E-LAN Ethernet service is to provide an interconnection capability between multiple organizational sites within a metropolitan area. Figure 6.4 illustrates an E-LAN Ethernet service type.
AU6039.indb 166
2/13/08 9:23:09 AM
Carrier Ethernet Services n 167
Router
Router Carrier Ethernet Network
Router
Router
Figure 6.4 E-LAN Ethernet service type
E-TREE A third type of Ethernet Service can be considered to represent a point-to-multi-point transmission service. Referred to as E-TREE, this service is similar to an EPON Ethernet topology and is also commonly referred to as “hub and spoke,” “root to leaf,” or “leaf to root.” E-TREE represents a future service that may be widely used once standards are developed. Perhaps the primary use of E-TREE will be a multiplexed connection to an ISP, with branches flowing to different organizational sites. Figure 6.5 illustrates the topology of an E-TREE Carrier Ethernet service.
Encapsulation Techniques Previously in this chapter we noted that the IEEE 802.1Q standard is limited to supporting 4094 VLANs. Although the support of 4094 VLANs represents a physical constraint, there is also another problem associated with directly using VLAN tagging. That problem is the fact that switches in the core of a Carrier Ethernet network would have to learn all of the MAC addresses of each host in any customer VLAN (c-VLAN). This could result in extremely large MAC address tables that would be maintained by core switches, resulting in what is referred to as a MAC address table explosion. Thus, a switch failure might result in a considerable delay due to recovery operations requiring the switch to relearn a considerable number of MAC addresses. Another problem associated with the use of VLAN tags is the possibility that two or more customers may select the same VLAN identifier (VID). If this occurs the service provider must be able to differentiate between them within the Carrier Ethernet domain. Based upon these problems a variety of solutions in the form of encapsulation schemes have been either implemented or proposed to provide a
AU6039.indb 167
2/13/08 9:23:10 AM
168 n Carrier Ethernet: Providing the Need for Speed
Router Carrier Ethernet Network
Router
Router
Router
Figure 6.5 E-TREE
more scalable Layer 2 service. Such schemes insert additional tags or fields in the customer Ethernet-generated frames at ingress nodes that are removed at the egress nodes. Some of the encapsulation schemes include VLAN stacking and the use of MPLS-based Ethernet encapsulation. As we will note shortly, there are different approaches to each scheme.
VLAN Stacking There are two methods of VLAN stacking that involve adding additional tagging fields to each Ethernet frame. The first method which involves the use of VLAN (802.1Q) tags is commonly referred to as Q-in-Q tagging, which was standardized in the IEEE 802.1ad specification. A second stacking method occurs via the use of a virtual-Metropolitan Area Network (VMAN) tag instead of a service provider Q-tag.
Q-in-Q Tagging The first method of VLAN stacking results in an additional Q-tag being inserted into customer Ethernet frames at the ingress switch of a Carrier Ethernet domain. This action results in a frame having two Q-tags, one referred to as a provider (P) tag, and the second represents the customer ( c ) tag. Figure 6.6 illustrates an example of VLAN stacking, which is more formally defined as a Q-in-Q Ethernet frame under the IEEE 802.1ad specification. Through the use of stacked VLAN tags (Q-in-Q), it becomes possible to define up to 16,777,216 labels, resulting in a much more scalable network. Because customer equipment is not supposed to understand the Q-in-Q frame format, the addition of the second Q-tag occurs at the ingress to the provider network and is then removed by the egress switch in the provider’s network.
AU6039.indb 168
2/13/08 9:23:10 AM
Carrier Ethernet Services n 169
Preamble
SFD DA SA
P-Ethertype
P-TCP
C-Ethertype
C-TCI
T/L
FCS
P-Ethertype Provider Ethertype P-TCI Provider Tag Control Information Contains P-vLAN ID C-Ethertype Customer Ethertype C-TCI Customer Tag Control Information Contains C-vLAN
Figure 6.6 IEEE 802.1ad VLAN stacking
Interpretation of Stacked Tags There are two ways of interpreting stacked Q-tags. In the first method only the VID of the outer tag, which is inserted by an ingress switch, is used by the core Ethernet switches to identify the C-VLAN across the domain. The second method combines the VID fields of both the customer and provider to support a much larger number of C-VLANs.
Tagged versus Raw Mode Edge routers in the service provider’s network can operate in one of two modes: tagged or raw. In a tagged mode, frames with different VLAN IDs can belong to different customers, or if they belong to the same customer they may require different treatment by the service provider. For example, some frames with specific VLAN tags could be forwarded via different paths or even mapped to different CoS classes for custom QoS treatment. In comparison, when an edge switch operates in a raw mode VLAN tags are not used to define a service to the network. Instead, the tag is part of the customer VLAN structure and is transparently passed through the network without processing.
Virtual MAN Tag Encapsulation A second type of VLAN stacking occurs through the use of a virtual Metropolitan Area Network (VMAN) tag instead of a provider Q-tag. The VMAN tag functions similar to the outer Q-tag assigned by the service provider in Q-in-Q encapsulation. However, the provider obtains control over the 24-bit VID instead of having to combine the provider and customer VIDs. Thus, the use of VMAN tag encapsulation makes it possible to transport traffic from more than 4094 VLANs over the MAN. Figure 6.7 illustrates VLAN stacking with a non-Q-tag. The M-bit is set to 0 if the 3-byte domain identifier (DI) is derived by mapping the customer tag and port, while a value of 1 indicates that the DI is derived by mapping of the port. Concerning the 24-bit domain identifier, the value represents the VMAN-ID assigned by the service provider to a C-VLAN and has significance only within a given service
AU6039.indb 169
2/13/08 9:23:11 AM
170 n Carrier Ethernet: Providing the Need for Speed
Preamble
SFD
DA
SA
New Non-Q Ethernet vLAN Type Tag
M
802.1Q hex 8100
T
vLAN Tag
Version (6)
Reserved (5)
Priority (3)
Original Ethernet Type
Data
FCS
1 byte 1 byte
Control
1 byte
Domain Identifier
3 bytes
Figure 6.7 VLAN stacking with non-Q-tag
provider domain. The T-bit is set to 0 for client data, and a value of 1 indicates control data is being passed. Both of the previously mentioned VLAN stacking methods enable subscribers to maintain their own C-VLAN structure. Although two stacked levels are the most common methods used, some vendor equipment enables up to 8 stacked VLANs. In addition, while both methods address the C-VLAN scalability issue they do not directly address the MAC address table explosion issue. Instead, a technique referred to as scalable Ethernet bridging which is also known as MACin-MAC (M-i-M), can be used.
M-i-M Tagging In the M-i-M tagging method, as its name implies, the service provider’s domain transports the customer (C-VLAN) frames based on the provider edge node’s MAC address. Specifically, each ingress node inserts two additional MAC address fields (destination and source) that have local significance into customer frames, as illustrated in Figure 6.8. In examining Figure 6.8, note that although only two provider MAC fields are shown as being added to the customer frame, other fields such as a Q-tag may be included in the stacked M-i-M header. As data flows through a service provider network using M-i-M stacked addresses, each core switch only needs to learn the edge switch MAC addresses, which significantly reduces MAC address table entries as well as search times needed to locate an entry.
Data Flow To illustrate the flow of data using M-i-M consider Figure 6.9. In this example the service provider has two edge nodes (PE1 and PE2) and five core switches (PCore
AU6039.indb 170
2/13/08 9:23:12 AM
Carrier Ethernet Services n 171 Customer Frame
Provider DA SA
Provider DA
SA vLAN Tag
Data
Original Ethertype
FCS
Provider Fields
Figure 6.8 Provider M-i-M MAC encapsulated Ethernet frame
P Core 2
P Core 1
MAC Table for PE 2 P Core 3 Port 1
MAC Address Table for PE 1 Port Address 0
M1…M4
1
M5…M9
PE 1
Rest of Frame M1 M6 PE1 PE2
Address
0
M5…M9
1
M1…M4
PE 2
Port 0
Port O
Rest of Frame M1 M6
C-Switch M4 M3
M1
Port
M2
C-Switch M5
M9 M8
M6
M7
Figure 6.9 Data flow using M-i-M tagging
1 through PCore 5). We will assume a customer has two locations within a metropolitan area that they wish to interconnect using a Carrier Ethernet service. Four computers with addresses M1 through M4 are at one location, and five computers with addresses M5 through M9 are located at a second. Figure 6.9 illustrates the flow of data from one customer location to the other location through the provider network. Note that PE1 and PE2 represent the outgoing port numbers of the provider edge switches.
AU6039.indb 171
2/13/08 9:23:13 AM
172 n Carrier Ethernet: Providing the Need for Speed
Stack Problems For each of the three previously mentioned VLAN stacking methods, it is important to separate customer use of the Spanning Tree Protocol from the provider’s network. Otherwise, changes on the customer’s spanning tree could affect the provider’s spanning tree, resulting in unintended situations occurring. Currently IEEE standards do not support tagged Bridge Protocol Data Units (BPDUs). Thus, it may become necessary for some customer BPDUs to be tagged and transported across the service provider’s network to enable customer VLAN sites to function properly. Another area where VLAN stacking will need modification concerns the definition of new Ethertype values. Such values would enable switches to distinguish between encapsulated provider frames and regular 802.1Q customer frames. This would enable provider switches to be automatically configured, and alleviate the need to manually configure switches, minimizing potential problems resulting from the misconfiguration of switches.
MPLS Layer 2 Encapsulation In concluding our examination of encapsulation techniques we will focus our attention on the use of MPLS Layer 2 encapsulation, referred to as Martini encapsulation in honor of the editor of a number of Internet RFCs to include RFC 4448 titled “Encapsulation Methods for Transport of Ethernet over MPLS Networks.” When Layer 2 services are configured over MPLS, Layer 2 traffic is encapsulated in MPLS frames and then transported via MPLS tunnels through an MPLS network. The encapsulation occurs when traffic reaches the edge of the service provider’s network, while de-capsulation occurs when traffic exits the network. The advantage of MPLS encapsulation includes using a large base of existing network equipment. In addition, this Layer 2 VPN technique takes advantage of MPLS label stacking under which more than one label can be used to forward traffic through an MPLS network. Under MPLS Layer 2 encapsulation two labels are used. One label represents a point-to-point virtual circuit, and the second label represents the tunnel through the network. As traffic is encapsulated, the ingress Label Switch Router (LSR) assigns it to a virtual circuit label. This label identifies the VPN, VLAN, or connection endpoint while the egress LSR uses the virtual circuit label to determine how to process the frame. Between the ingress and egress routers the core routers use the tunnel label to determine the path that data flows through the network. Figure 6.10 illustrates MPLS Layer 2 encapsulation of a tagged Ethernet frame. Note that the two MPLS labels that are inserted into customer Ethernet frames are based on the destination MAC address, port, and 802.1Q information. As previously mentioned, the tunnel label provides information required for transporting frames through the provider network. LSRs in the network only use information
AU6039.indb 172
2/13/08 9:23:13 AM
Carrier Ethernet Services n 173 Customer Frame
VC Label
Tunnel Label
DA
SA
vLAN Tag
Original Ethertype
Data
FCS
Encapsulated Fields
Figure 6.10 MPLS Layer 2 encapsulation
in the tunnel label to switch labeled frames across the MPLS domain. At the hop prior to the egress Label Edge Router (LER) the tunnel label is removed, leaving the VC (virtual circuit) label, which is used by the egress LER to determine how to process the frame and where to deliver it on the destination network via outputting the frame on an outgoing port. Due to MPLS tunneling the VC label is not visible until the frame reaches the egress LER. Thus, two labels (VC and tunnel) are necessary under MPLS encapsulation. The MPLS encapsulation method provides a general comparison with the previously described Ethernet extensions as follows: first, the VC label can be considered to correspond to the Q/VMAN tag and the tunnel label would correspond to the M-i-M extensions (SA/DA). Under MPLS, the Label Distribution Protocol (LDP) and Border Gateway Protocol (BGP) would be used to distribute the labels while the LSRs would be used to establish the required Label-Switched Paths (LSPs). The MPLS labels in effect perform the same function as the stacked Q/VMAN tags, enabling more than 4094 customers to be supported by the Carrier Ethernet network operator. Through the use of Martini encapsulation Ethernet frames can be transported as a virtual private LAN service (VPLS). VPLS supports the connection locations while emulating a single bridged domain over a managed IP/MPLS network. Thus, all the services in a VPLS will appear to be on the same LAN regardless of their actual location. Due to the wide availability of MPLS in carrier networks, this can be a very effective technology for transporting Ethernet. Because MPLS operates over a wide variety of transport facilities, this also means that Ethernet with Martini encapsulation can be transported over such transport facilities as T1, T3, SONET, and, in fact, just about any physical network. The methods mentioned in this chapter provide a mechanism to transport Ethernet over communications carrier facilities. However, in doing so both customers and service providers must consider the hidden “cell tax” in which the insertion of various tags used to encapsulate Ethernet reduces overall efficiency. Thus, it is
AU6039.indb 173
2/13/08 9:23:13 AM
174 n Carrier Ethernet: Providing the Need for Speed
important for both customers and network operators to consider the effect of encapsulation and additional labels upon efficiency, especially if Ethernet is being used to transport VoIP, which in turn is transported as Ethernet over MPLS. When this occurs the headers and labels can result in actual data only representing a very small portion of the bandwidth. This obviously will affect the cost of the carrier service.
AU6039.indb 174
2/13/08 9:23:14 AM
Chapter 7
Service Level Agreements and Quality of Service In this concluding chapter in this book, we will focus our attention upon two of the most difficult to implement but certainly very important aspects associated with using a Carrier Ethernet service: the Service Level Agreement (SLA) and Quality of Service (QoS). The SLA represents a contract between the service provider and the end user which defines a variety of network parameters. In comparison, QoS represents a series of resource reservation control mechanisms that can provide different levels of performance to different users or different types of user data. As you might expect, some type of QoS is typically incorporated into an SLA. In this chapter we will first define in some detail what an SLA means in an Ethernet environment and why it can be difficult to implement. As we discuss the SLA in an Ethernet environment we will become aware of the fact that different service providers offering a Carrier Ethernet service may not provide the same devices that enable similar measurements. In addition, when data flows from one service provider’s network through another network to its destination it may not be possible to obtain a meaningful SLA. In concluding this chapter we will turn our attention to QoS, which governs the ability of real-time applications such as VoIP, streaming video, and teleconferences to operate correctly. As we discuss QoS we will note that Ethernet’s class of Service (CoS) must be used as a basis for obtaining a QoS.
175
AU6039.indb 175
2/13/08 9:23:14 AM
176 n Carrier Ethernet: Providing the Need for Speed
The Service Level Agreement In this section we will obtain an appreciation for the meaning of the term “SLA” to include the parameters that are typically defined within a contract for service. An Ethernet SLA represents an agreement or contract between the service provider and the end user or customer. The SLA is provided on an end-to-end basis, which sets up a series of parameters and values the service provider will meet or be penalized if it fails to meet the terms of the agreement.
Metrics Table 7.1 lists some of the metrics that can be included in a Service Level Agreement and which we will elaborate upon in the next series of sections in this chapter.
Availability Availability can be defined at both a component and system level, with the latter permitting an end-to-end computation. In this section we will first define the term and then examine its applicability to a Carrier Ethernet transmission facility.
Component Availability The availability of an individual component can be expressed in two ways that are directly related to one another. First, as a percentage, availability can be defined as Table 7.1 Service Level Agreement Metrics Availability Latency Intra-metro Inter-metro Route specific Jitter Mean time to repair (MTTR) Installation time Bandwidth provisioning Packet loss Guaranteed bandwidth
AU6039.indb 176
2/13/08 9:23:14 AM
Service Level Agreements and Quality of Service n 177
the operational time of a device divided by the total time, with the result multiplied by 100. This is indicated by the following equation:
A% = (operational time / total time) * 100
where A% is availability expressed as a percentage. As an example of availability consider a Carrier Ethernet service that operates continuously, 24 hours per day, 7 days per week. Over a 1-year period the network should be available for use for 365 days × 24 hours per day or 8760 hours if we assume a non-leap year. Now suppose the network was not available four times during the year, with the average downtime being two hours. Thus, the network is then operational 8760 hours less 8 hours or 8752 hours. Using our availability formula we obtain:
A% = (8752 / 8706) * 100 = 99.9%
MTBF and MTTR Because some service providers express availability using Mean Time Before Failure (MTBF) and Mean Time to Repair (MTTR), we will briefly turn our attention to these metrics and how they provide a computation that determines availability. MTBF represents the average operational time of a device or facility prior to its failure. Thus, MTBF can be considered as being equivalent to the operational time of a device or facility. Once a failure occurs it must be fixed or repaired. The interval from the time the device or facility fails until it is repaired is known as the time to repair, and the average of each repair time is known as the Mean Time to Repair (MTTR). Because the total time is the sum of MTBF + MTTR, we can rewrite our availability formula as follows:
A% = MTBF / (MTBF + MTTR) * 100
It is important to remember the “M” in MTBF and MTTR, as you must use the average or mean time before failure and average or mean time to repair. Otherwise, your calculations are subject to error. For example, if the use of a Carrier Ethernet network became unavailable halfway through the year, you might be tempted to assign 4380 hours to the MTBF. Then you would compute availability as follows:
A% = 4380 / (4380 + 8) * 100 = 99.91%
The problem with this computation is the fact that only one failure occurred, which results in the MTBF not actually representing a mean. Although the
AU6039.indb 177
2/13/08 9:23:14 AM
178 n Carrier Ethernet: Providing the Need for Speed
computed MTBF is correct for a specific device or facility, the MTBF would be different for a second or third device or facility that when taken together provide an end-to-end transmission facility. Thus, if you are attempting to obtain an availability level for a network consisting of lines, switches, and routers you need to compute an average or mean level of availability through the use of an average MTBF. Then, the next logical question is how to obtain average MTBF information. Fortunately, most vendors and network operators provide the MTBF information for products they manufacture and services they offer instead of waiting for a significant period of time to obtain appropriate information.
Considering Published Statistics Although many published MTBF statistics can be used as is, it is important to note that certain statistics can represent extrapolations that deserve a degree of elaboration. For example, when a vendor introduces a new switch or router and quotes an MTBF of 50,000 or 100,000 hours, they obviously have not operated that device for that length of time. Instead, they either extrapolated MTBF statistics based upon improvements made to a previously manufactured product, or based their statistics on the MTBF values of individual components such as line cards, power supplies, and logic boards. If you are reading the brochure of a vendor or service provider and notice an asterisk next to an MTBF figure and the footnote indicates extrapolation, you might consider determining additional information. After all, if the MTBF of some device is indicated as 100,000 hours, or almost 12 years, why is the warranty period typically a year or less? In such situations you may want to consider using the warranty period as the MTBF value instead of an extended MTBF value. Concerning the MTTR, this interval may also be included on vendor literature and may also require a degree of modification to be realistic.
Considering Travel Time The MTTR figure is based upon the time required to repair a device or facility once a person is on site. Thus, you need to consider the location where equipment resides on an end-to-end transmission path and the travel time to potential failure locations. For example, assume your Ethernet switch will provide a connection to a Carrier Ethernet service provider via an EFM (Ethernet in the First Mile) optical transmission facility. If the specification sheet for the Ethernet switch lists an MTBF of 16,500 hours and an MTTR of two hours, the latter may not be accurate unless your organization has on-site maintenance support. Otherwise, you need to add travel time to the MTTR to obtain a more realistic value. For example, assume the Ethernet switch is located in a suburb of Atlanta and it takes a maintenance person two hours to travel to your organization’s location. Then, a more realistic
AU6039.indb 178
2/13/08 9:23:14 AM
Service Level Agreements and Quality of Service n 179
MTTR would result from adding the expected travel time to the vendor’s MTTR metric provided in a product specification sheet. Now that we have an appreciation for MTBF and MTTR, we will turn our attention to how system availability is computed.
System Availability In communications a system is considered to represent a collection of devices and line facilities which form a given topology. In a Carrier Ethernet environment system availability represents the end-to-end availability from source to destination. The Carrier Ethernet service provider will install demarcation points at each customer location. The demarcation point, which we will discuss in more detail later in this chapter, represents the location from which the communications carrier takes responsibility. Thus, an availability metric provided by the communications carrier represents an availability level between two demarcation points. Because both equipment and different types of line facilities occur on the end-to-end path the level of availability actually more formally represents system availability. Due to the fact that the end-to-end data flow over networks structures represent devices and lines connected in series we will next examine how the availability level of this type of topology is computed.
Components Connected in Series To illustrate the computation of system availability we need a network to analyze. Thus, the top portion of Figure 7.1 shows the path of transmission from an end user’s Ethernet switch through a service provider’s Carrier Ethernet network to a second location within a metropolitan area. In this example we will assume that
L2
End User Switch
L1
S2
S1
Demarc
L3 S3
L4
End User Switch
Demarc
Figure 7.1 Network components in series
AU6039.indb 179
2/13/08 9:23:15 AM
180 n Carrier Ethernet: Providing the Need for Speed
the service provider DEMARC (demarcation) line terminates just in front of each end-user switch. The DEMARC or demarcation represents the boundary between the service provider or carrier’s network and the customer’s or end user’s network. The purpose of the DEMARC is to define the endpoints of the carrier’s responsibility as well as to enable the carrier to test and monitor its network up to the DEMARC located at the customer’s presence. The latter is extremely important as it enables the service provider to determine where problems exist and to dispatch technicians to alleviate such problems. An Ethernet Demarcation Device (EDD) represents a relatively new network component now manufactured by a few vendors for service providers. The EDD provides Operation, Administration, and Maintenance (OAM), loopback capabilities, a variety of statistics, and enables the measurement and tracking of such endto-end SLA parameters as latency, jitter, and packet loss. In examining Figure 7.1 note that L1 through L4 represent different types of transmission facilities and S1 through S3 represent three service provider switches. The network illustrated in Figure 7.1 represents a series of devices and facilities that could be drawn with respect to their availability level as shown below:
A1
A2
An
When n components to include lines are connected in series the availability of those components as a system is computed by multiplying together the availabilities of each of the n individual components. Mathematically, this is expressed as follows for n components: n
A=
i=1
Ai
To illustrate the use of the prior formula, we will assume that each switch in the carrier network has an availability level of 99.9 percent and each of the six links have an availability level of 99.98 percent. Then, the end-to-end availability level becomes:
(0.999)2 * (0.9998)6 = 0.9968
Although a 99.68 percent level of availability may appear quite high it is important to remember that during a year with 8,760 hours or 525,600 minutes this means that you can expect 525,600 * (1 − 0.9968) or 1,682 minutes of downtime or approximately 28 hours of outage. Whether this is good or bad depends upon when the outage occurs (day or night), its duration, and what your organization was attempting to transmit right before the outage occurred.
AU6039.indb 180
2/13/08 9:23:16 AM
Service Level Agreements and Quality of Service n 181
Based upon the preceding, it is important to note that a 99+ percent level of availability can still result in a long-duration outage once a year or a series of shorter-duration outages. Thus, end users must carefully examine the availability portion of an SLA to determine when the level of availability is guaranteed and what recourse they have, if any, if the service provider fails to exceed the level of availability defined in the SLA.
Latency Latency represents the delay experienced by data flowing though a network. That delay results from routers examining headers to transfer frames from one path to another as well as delays associated with frames in the form of binary signals flowing across line segments connecting routers and switches. In the wonderful world of Carrier Ethernet all line segments possibly other than the access line are optical, minimizing latency because data flows at the speed of light.
Application Considerations It is important to compare a service provider’s latency against your organization’s application requirements and data flow. Concerning the former, real-time voice and video require a lesser degree of latency and file transfers and Web page browsing can tolerate a higher degree of latency.
Types of Latency There are several types of latency a service provider may quote. Those types of latency include intra-metro, inter-metro, and route-specific latency. Normally the SLA type of latency is defined as a monthly average.
Jitter Jitter represents the unwanted variation between successive signals. In a communications environment jitter usually occurs due to routers and switches having variable loads when processing a sequence of frames transmitted through the device to a common destination. Because a router or switch has many ports, the activity on the device at a particular point in time governs its ability to process the next frame in a sequence of frames flowing through the device, resulting in random delays.
Jitter Buffers For non-real-time applications jitter really does not matter. However, when real-time voice or video is transferred too much jitter can result in distortions to recreated
AU6039.indb 181
2/13/08 9:23:16 AM
182 n Carrier Ethernet: Providing the Need for Speed
voice and video. Thus, most VoIP and teleconferencing systems include jitter buffers where the variations in frames reaching their destination can be compensated for by storing frames and then removing them via a predefined timing scheme. The storage area is referred to as a jitter buffer, which counters jitter by enabling a continuous playout of audio or video. Most if not all readers of this book have indirectly used a jitter buffer although they may not be aware of the fact. Each time you use Microsoft’s Windows Media Player, Apple’s QuickTime Player, or a similar program to view a video you will probably notice the “Buffering…” message displayed in the lower left or right corner of the program. What this message tells you is that to display a smooth video the program is placing data in its jitter buffer so that it can extract such data with precise timing. This action enables video to be displayed and audio to be sent to the speakers without the intermittent delays that occur between frames as they traverse a network. The maximum amount of network jitter should always be less than the capacity of the jitter buffer. For example, if the jitter buffer is set up to store 20 ms of audio, then the network jitter should be less than 20 ms.
MTTR Previously we noted that availability can be computed in terms of MTBF and MTTR. Some service providers in addition to specifying network availability also specify an MTTR. When the MTTR is specified in an SLA it is usually computed by summing the total network downtime during the month and dividing that number by the total number of service interruptions that occurred during the month. The result, expressed in minutes, becomes the specified MTTR. Obviously, the lower the MTTR, the more responsive the service provider to customer problems. However, customer locations can also affect the MTTR. For example, Los Angeles is a large metropolitan area with numerous businesses that have multiple locations within the city limits, and the city is significantly spread out from a downtown area to a literal maze of other areas where business office parks are located. Thus, the MTTR quoted for a company with locations spread out from the city center to the suburbs can be expected to exceed the MTTR for a company with offices concentrated in one general location.
Installation Time One important metric that many end users forget to discuss is the guarantee provided by the Carrier Ethernet service provider for installing the service. Many Carrier Ethernet service providers will include an on-time installation metric in their SLA. That metric is typically specified in terms of days after a contract is signed.
AU6039.indb 182
2/13/08 9:23:16 AM
Service Level Agreements and Quality of Service n 183
Bandwidth Provisioning Although we like to think that we have a reasonable prediction of the bandwidth our organization needs, many times new or improved applications come along that relegate our predictions to the waste basket. When this situation occurs we need to inform our Carrier Ethernet provider that our organization requires additional bandwidth. Because the connection to the service provider is usually through an EFM connection operating on a 1GbE optical line, it is relatively easy for the service provider to accept more information from the customer per unit of time. What is a bit more difficult is for the service provider to provision the internal network to receive and route additional customer data through the existing network infrastructure. When incorporated into an SLA the service operator normally specifies a period of time in hours for provisioning additional bandwidth.
Packet Loss Many Carrier Ethernet service providers currently do not include a packet loss metric in their SLA. The reason for this is the fact that unless a true QoS exists, the service provider is hesitant to guarantee a packet loss level that can be adversely affected by users beyond their control. The exception to this occurs when the service provider has sufficient bandwidth to accommodate traffic peaks, allowing a packet loss value to be included in an SLA.
Guaranteed Bandwidth Some Ethernet service providers offer customers a guaranteed bandwidth. To accomplish this without a QoS capability results from the fact that the service provider has provisioned sufficient bandwidth to enable the customer to burst traffic up to a certain bit rate without packet loss occurring.
SLA Problems Currently Carrier Ethernet service providers are basically using a series of old metrics to provide parameters in their contracts. Unlike SONET/SDH, T-carrier circuits, and other transport facilities that have built-in protocols that provide the ability to monitor the performance of a facility, tried and true Ethernet lacks this capability. In addition, although many network facilities include the ability of non-intrusive testing as well as an OAM capability, until recently Ethernet lacked both of these features. Thus, new or modified Ethernet protocols and frames were required to obtain data that service providers can use to test, administer, and maintain their expanding Ethernet infrastructure. Fortunately, two types of OAM were recently
AU6039.indb 183
2/13/08 9:23:16 AM
184 n Carrier Ethernet: Providing the Need for Speed
being developed to provide this capability for Ethernet, one by the ITU and the IEEE and the second by the EFM Task Force. The former can be considered to represent a fully featured OAM mechanism that can provide an end-to-end OAM capability and which will provide performance measurements; the latter is limited to the monitoring of continuity on a single link providing a limited number of statistics and as such is designed for supporting access applications. To provide readers with an insight into the need for OAM and how it is being implemented within Ethernet we will probe a bit deeper into this topic.
OAM Overview Operation, administration, and maintenance is used to describe the monitoring of a network by service providers. Through the use of OAM a service provider can detect network problems, activate such network fault prevention measures as the rerouting of data, and in general respond to various alarms. Because OAM can provide service providers with detailed metrics of network performance as well as the ability to respond to faults it allows them to offer SLAs that go beyond basic measurements.
OAM and Ethernet Ethernet was originally developed as a local area network with clusters of co-located stations sharing access to a common medium. As Ethernet evolved little thought was given to adding an OAM capability as testing was limited to a local environment and the service provider was in fact the company that operated the network. With the introduction of Carrier Ethernet networks the need for an OAM capability changed. The service provider now needed a method to detect and overcome faults as well as to measure network performance so SLAs could be guaranteed.
Ethernet OAMs Previously we noted that there are two Ethernet OAMs, one developed for EFM applications, the other provides an end-to-end capability. The EFM OAM was developed by the IEEE 802.3ah Task Force in the 802.3 Working Group. Thus, it is often referred to as the 802.3ah or EFM OAM. This is a link-layer OAM, which was recently incorporated into the main 802.3 Ethernet specification as Clause 57. At the time this book was written, work on the service layer OAM for Carrier Ethernet was being performed by the IEEE under the 802.1ag specification and the ITU-T Y.1731 draft. In addition, the Metro Ethernet Forum (MEF) is also working on an Ethernet Service OAM, with all three organizations cooperating with each other. Because these three organizations are looking at Ethernet OAM from
AU6039.indb 184
2/13/08 9:23:16 AM
Service Level Agreements and Quality of Service n 185
a service level while EFM OAM is focused at the link level, the resulting protocols should be complementary and eventually work simultaneously.
Functions The primary function of an OAM protocol is to detect network faults. When a fault reaches a certain time threshold an alarm should be generated to make the network operator aware of the situation. To detect some types of network outages, special messages are periodically transmitted that are either looped back at a demarcation or responded to by equipment. Such messages are known as continuity checks (CC) as they test to ensure a path is available. A response not received within a certain period of time indicates a loss of service, and an alarm is usually generated.
Testing Testing can be divided into two general categories known as in-service (non-intrusive) and out-of-service (intrusive). In an Ethernet OAM environment frames transmitted that do not disrupt the normal operation fall into the first category. In comparison, a frame that, for instance, caused a demarcation point to be placed into a loopback for testing would result in intrusive testing.
Link-Layer OAM Ethernet’s link-layer OAM was developed for reaching customer locations from a service provider’s central office. Functions performed by this link-layer OAM include placing remote devices into and out of a loopback, querying the configuration of the remote device, and setting flags to indicate critical events. Because this OAM is limited to a single link it cannot provide data for an end-to-end service guarantee. Thus, the information provided about the state of the link is minimal.
Messages Link-layer OAM messages are transmitted as slow protocol frames referred to as OAM Protocol Data Units (OAMPDUs). The reason for the term “slow protocol” is the fact that no more than 10 OAMPDU frames per second can be transmitted, limiting the time when other traffic cannot flow on the link. Figure 7.2 illustrates the EFM OAM frame format. Note that all slow protocols use an Ether Type value of hex 88-09 and that the link-layer OAM is defined by a value of hex 03, which appears in the first byte of the MAC client payload. The destination address (DA) is a specific multicast address that is link constrained because
AU6039.indb 185
2/13/08 9:23:17 AM
186 n Carrier Ethernet: Providing the Need for Speed
Preamble
Destination Address
Source Address
Type hex 8809
Sub Type hex 03
Flags 2B
Code 1B
Data 42-1496
FCS 4B
Figure 7.2 EFM OAM frame format
Table 7.2 EFM OAM Codes Information
EFM OAMPDUs only traverse a single link. Thus, they are never forwarded by bridges or switches.
Event notification Variable request and response Loopback control Organization-specific
Codes Presently five codes are defined by the linklayer OAM. Information is encoded by setting the Code field value followed by encoding information. Table 7.2 lists the five codes currently defined for EFM OAM.
Information The information code OAMPDUs can be used to discover remote devices (auto-discovery) and exchange information about their capabilities, provide fault notifications, and serve as a heartbeat generator. Concerning the latter, OAMPDUs must be transmitted at least once per second if there are no other pending messages.
Event Notification As the name of the code implies, event notification frames report various link statistics. Statistics can be reported for a specific period of time or as a cumulative total since the counter was last reset.
Variable Request and Response Variable request frames are used by the service provider to obtain the configuration of customer equipment. To do so, the variable request will request specific SNMP MIB variables. The customer response to such requests occurs in variable response frames. Because Ethernet frame delivery is not guaranteed, even OAMPDUs may be transmitted several times to enhance the probability of reception.
AU6039.indb 186
2/13/08 9:23:17 AM
Service Level Agreements and Quality of Service n 187
Loopback Control A fourth type of OAM frame is used to enable or disable intrusive loopback in a remote device when enabled statistics from local and remote clients can be queried.
Organization Specific The last type of OAM frame allows an organization to tailor a new OAMPDU to satisfy a specific requirement.
Flags If we return our attention to Figure 7.2 we can note that two bytes or 16 bits can be toggled. Referred to as flags, each OAMPDU has one or more bits set to denote critical events as well as signal whether a remote device is ready to receive OAM messages.
Service OAM The IEEE 802.1ag specification provides an end-to-end service, enabling up to eight OAM levels as well as allowing customers and service providers to run independent OAMs. By default, users are allocated three levels of OAMs, service providers are allocated two levels, and operators three levels. OAM frames that are part of a higher level will be forwarded transparently by lower level devices. One of the key problems in developing a service OAM for Ethernet is the fact that the protocol is connectionless. Another problem is the fact that previously developed OAM protocols were primarily designed to support point-to-point connections. In comparison, Ethernet represents a multipoint-to-multipoint protocol. Solving these problems is no trivial matter, which probably explains why the IEEE 802.1ag specification is a work in progress. Although this author believes that Ethernet service OAMs will eventually be fairly common it is important to remember also that with sufficient thrust pigs will fly. What this author is implying is that the effort, coding, and devices required to establish a service OAM capability for Ethernet may have a level of complexity and cost that negates its full usefulness.
Quality of Service Overview In this section we will turn our attention to one of the most important metrics normally missing from a formal Service Level Agreement: QoS. Because QoS is not currently incorporated into most Carrier Ethernet SLAs, it is important to understand the rationale behind this omission and how some service providers compensate for it.
AU6039.indb 187
2/13/08 9:23:17 AM
188 n Carrier Ethernet: Providing the Need for Speed
As we noted earlier in this book, Ethernet provides a class of service (CoS) by using 3 bits within the IEEE 802.1Q VLAN header in the form of 3 bits. The 3 CoS bits enable 23 or 8 classes of service to be defined. In an IP environment where the Internet Protocol is transported over Ethernet a packet traverses both Layer 2 and Layer 3, so it is relatively easy to maintain QoS. This is because the IPv4 header’s Type of Service (ToS) field uses 3 bits to provide up to eight classes of service. Thus, IP’s ToS can be mapped to Ethernet’s CoS, and vice versa. However, for QoS to be set on an end-to-end basis requires configuring each device such as routers and switches to classify, police, and operate their queues in a similar manner. Thus, we need to turn our attention to the manner by which different devices can be configured to provide a QoS capability. In doing so, we will first note the differences between soft and hard QoS, as the former does not provide a true bandwidth guarantee. Today, Carrier Ethernet service providers offer “soft QoS” and primarily use the large bandwidth available in their Carrier Ethernet infrastructure to provide customers with the bandwidth they require.
Soft versus Hard QoS In some literature, readers will encounter the terms “soft” and “hard” QoS. As we will note shortly, the former term is more of a marketing mechanism as it does not guarantee that bandwidth will be reserved through a network for a specific application or series of applications.
Soft QoS Soft QoS refers to a QoS model where traffic is provided with precedence over other traffic within a network. To achieve a soft QoS capability the network operator uses multiple CoSs and assigns specific traffic to a specific CoS. Thus, a soft QoS actually does not represent a true QoS as it does not reserve bandwidth through a network. Instead, a soft QoS represents a priority scheme which, when bandwidth is sufficient for all customers, provides a similar traffic flow to a true QoS capability. The use of a soft QoS enables a service provider to differentiate between different types of network traffic. This in turn enables the service provider to offer multiple types of services, such as Platinum, Gold, Silver, Copper, and Bronze, where Platinum costs more than Gold, Gold costs more than Silver, and so on. Similarly, Platinum provides a higher priority through the network and Bronze has the lowest priority. Although soft QoS enables traffic differentiation it does not allow the carrier to guarantee specific bandwidth or provide a packet loss value within an SLA. Today, most Carrier Ethernet providers get around this limitation by having a significant amount of bandwidth available in their infrastructure that vastly exceeds cumulative customer demands. In effect, the existing Carrier Ethernet networks can be
AU6039.indb 188
2/13/08 9:23:18 AM
Service Level Agreements and Quality of Service n 189
considered as over-engineered so that network traffic does not reach a point where data may have to be discarded. Unfortunately, as the popularity of Carrier Ethernet increases, so will both the number of customers and the quantity of data that they will transmit. As the use of the Carrier Ethernet network increases it will become harder for service providers to over-engineer their network to prevent traffic from being discarded. When that level of utilization is reached many network operators will more than likely attempt to migrate to a hard QoS method.
Hard QoS A hard QoS requires paths to be pre-provisioned through a network while resources such as switches and routers are allocated to guarantee bandwidth for different applications. To accomplish this, hard QoS requires a connection-oriented approach with the ability to reserve bandwidth prior to an application’s commencing. Protocols that provide a QoS capability include Asynchronous Transfer Mode (ATM), Frame Relay, the Resource Reservation Protocol (RSVP), X-25, and some ADSL modems. The use of MPLS provides eight QoS classes that can be considered to represent QoS in that end-to-end delay and packet loss can be defined. In comparison, the IEEE 802.1P standard, which is included in the VLAN 802.1Q tag, provides a CoS that does not allocate bandwidth to be allocated nor define packet loss or delay. Because Ethernet is a connectionless technology in its current state it cannot provide a hard QoS capability. However, the MEF has ratified a series of technical specifications and has begun working on new technical specifications that may eventually enable service providers to support hard QoS services to their customers. In fact, during 2004 the MEF ratified MEF5 (Traffic Management Specification, Phase 1), which defines traffic management specifications for Carrier Ethernet to deliver hard SLA-based broadband services.
MEF QoS The Metro Ethernet Forum approaches QoS from a service definition point of view instead of a protocol or implementation method. In doing so, the MEF defined the requirements for different Ethernet services and the manner by which they should be measured. Under MEF10, which merged MEF1 (Service Model) and MEF5, the MEF now defines a traffic profile titled MEF Ethernet Service Attributes, Phase 1. Table 7.3 lists the attributes that can be used to define a traffic profile and their meanings.
Bandwidth Allocation An examination of the entries in Table 7.3 shows that it is possible to provide users with inexpensive non-guaranteed bandwidth while guaranteeing a portion of bandwidth. The non-guaranteed bandwidth in the form of the Excess Information Rate
AU6039.indb 189
2/13/08 9:23:18 AM
190 n Carrier Ethernet: Providing the Need for Speed
Table 7.3 MEF Traffic Profile Attributes Attribute CIR
Meaning Defines the average rate in bps of ingress service Frames up to which the network delivers service frames and meets the performance objectives defined by the CoS service attribute
Committed Burst Size Limits the maximum number of bytes available for a (CBS) burst of ingress service frames sent at the UNI speed to remain CIR-conformant EIR
Defines the average rate in bps of ingress service frames up to which the network may deliver service frames without any performance objectives
Excess Burst Size (EBS)
Limits the maximum number of bytes for a burst of ingress service frames transmitted at the UNI speed to remain EIR compliant
Frame Delay
The delay experienced by frames transmitted over the network from ingress to egress
Frame Delay Variation The variation in the offset of frames by time from when they should appear, representing frame jitter Frame Loss Ratio
The ratio of the number of frames lost divided by the number of frames transmitted
(EIR) represents a best-effort delivery. In comparison, the Committed Information Rate (CIR) represents a guaranteed delivery of bandwidth. Service providers can assign traffic profiles for Ethernet users to the Network User Interface (NUI) per Ethernet Virtual Circuit (EVC) or per combined EVC and IEEE 802.1P CoS. Figure 7.3 illustrates three methods by which a service provider can assign bandwidth at the ingress to their network. In each of the three examples EVCs are assumed to be established. In the first example shown in the left portion of Figure 7.3 three EVCs share a bandwidth profile established for the User Network Interface (UNI). In the middle portion of Figure 7.3 a bandwidth profile is established for each EVC, and the right portion of the figure shows how a bandwidth profile can be established using the CoS on an individual EVC. Readers familiar with Frame Relay and ATM more than likely have noted the similarity of the MEF traffic profile attributes to the ones in the two mentioned protocols. Similar to those protocols, traffic up to the CIR is guaranteed and would thus experience a very low frame loss ratio. Traffic between the CIR and EIR will be delivered based upon the present available bandwidth in the network, but can be dropped in the event congestion occurs in the network.
AU6039.indb 190
2/13/08 9:23:18 AM
Service Level Agreements and Quality of Service n 191 EVC1 cos 0, 1, 2, 3 U N I
EVC1 EVC2 EVC3
Bandwidth Profile per Ingress UNI
U N I
EVC1 EVC2 EVC3
Bandwidth Profile per EVC
U N I
cos 4, 5 cos 6, 7 EVC3
Bandwidth Profile per EVC and 802.1p COS
Figure 7.3 Ingress bandwidth profiles
Now that we have an appreciation for the MEF approach to QoS from a service definition approach, instead of altering the Ethernet protocol, which would not be desirable, we will examine how network operators can use existing routers and switches to expedite traffic based upon their CoS.
QoS Actions When using applicable routers and switches network engineers need to configure such devices to perform a series of function to provide QoS capability through the network. At the ingress location equipment must first classify data, then ensure that only up to a predefined amount of bandwidth is allowed to enter the network. Through policing and marking data that exceeds a predefined CIR, data can be either transmitted, marked, and transmitted or dropped based upon other activity occurring in the network. At the egress location QoS actions include queuing and scheduling data for delivery. To obtain an appreciation for these actions, we will briefly discuss each.
Classification As data enters the service provider’s network it is classified. In a Carrier Ethernet environment classification is based upon the CoS. Depending upon the equipment used and its configuration frames could be placed into up to eight classes.
Policing As previously mentioned, one of the functions of policing is to ensure that no more than a predefined amount of bandwidth is allowed into the network. Thus, policing can be used to enforce a CIR established for a customer on a particular network interface.
AU6039.indb 191
2/13/08 9:23:19 AM
192 n Carrier Ethernet: Providing the Need for Speed
Depending upon the equipment used and its configuration, policing can allow data into the network that exceeds the CIR. Such data can be considered to represent an overflow that can be marked and dropped by the network, if required.
Queuing As data flows through the network to the egress it can be placed into queues according to its prior marking at the egress into the network. That marking can be the CoS in the IEEE 802.1Q customer VLAN tag (c-tag), the provider VLAN tag (p-tag), or another tag used by the service provider for facilitating the flow of data through their network. Routers within the service provider’s network place data into output queues based upon a predefined tagging mechanism such as the customer priority value in the c-tag. For example, a priority of 5 to 7 could be assigned to frames transporting VoIP, teleconferencing, and other real-time applications that require low latency. Then, a priority of 3 or 4 might be assigned to near-real-time applications, such as bank teller terminal transactions, and a priority of 0 to 2 could be assigned to such applications as file transfers that are minimally affected by latency. Because the configuration of router queues requires a scheduling operation to become effective we will turn our attention to this topic.
Scheduling Scheduling determines how frames placed in queues are extracted and exit each egress port, in effect controlling the servicing of queue entries. One of the earliest methods associated with scheduling was the round robin servicing of queues. For example, with three queues (Q1, Q2 , Q3) associated with a port, data from Q1 would be extracted first, followed by data from Q2 , and so on. The problem with this extraction method is the fact that it does not prioritize the servicing of queues. A second scheduling method that prioritizes the servicing of queues is a round robin priority extraction method. Under this method a weight is assigned to each queue for extraction. For example, Q1, which might represent a low latency queue, has data sampled seven out of every ten samplings of the Q buffers. Q2, which could represent a buffer for holding near-real-time data, has a sampling rate twice that of Q3 that holds data with a CoS between 0 and 2. Figure 7.4 illustrates a weighted queue scheduling technique similar to the one just described. In examining Figure 7.4 note that if an entry does not exist in the queue, the scheduler then proceeds to the next highest weighted queue. Although this technique enables a priority to be associated with each queue, it assumes that the frame lengths of entries in each queue are the same. Thus, a pure weighted round robin
AU6039.indb 192
2/13/08 9:23:19 AM
Service Level Agreements and Quality of Service n 193
Port Buffers
Q1
Q2
Q3
70%
20%
10%
Output
Figure 7.4 Weighted queue scheduling example
scheduling technique, where weights are assigned to queues regardless of their content, can be considered as unfair. One solution to this level of unfairness that takes into consideration that different queues can contain different frame lengths is referred to as deficit round robin (DRR) by Cisco Systems. Thus, we will briefly turn our attention to this scheme.
Deficit Round Robin Deficit round robin (DRR) scheduling solves the problem associated with different queues having different frame lengths. DRR represents a modified weighted round robin scheduling scheme that was proposed in 1995. Under DRR a maximum packet size number is subtracted from the packet length. Any packets that exceed that number are then held back until the next visit of the scheduler. In comparison to weighted round robin that serves every non-empty queue, DRR serves packets at the head of every non-empty queue for which the deficit counter is greater than the packet’s size. If it is lower, then the deficit counter is increased by a value called a “quantum.” If a queue is not able to send a packet in a previous round due to an excessive packet length the remainder from the previous amount of credits a queue receives in each round (quantum) is added to the quantum for the next round.
Weighted DRR Another queuing method used by devices provides the ability to assign a weight to each queue while considering the length of frames is the weighted deficit round robin (WDRR). WDRR extends the use of a quantum from DDR to provide weighted throughput for each queue. Under WDRR queues have different weights and the quantum assigned to each queue in its scheduled round is proportional to its relative weight of the queue among all the queues serviced by the scheduler.
AU6039.indb 193
2/13/08 9:23:19 AM
194 n Carrier Ethernet: Providing the Need for Speed
Cisco ML-Series Card In a Cisco optical network environment this company manufactures a series of optical network systems (ONSs) whose functionality is enhanced through the use of its ML-Series cards. Two of the key functions of those cards is to provide queuing and scheduling capabilities.
Queue Assignment There are three methods by which queues can be assigned on an ML-Series card: (1) by the use of the “Priority” command during the configuration process; (2) through the use of the “Bandwidth” command; and (3) by allowing queues to be assigned automatically. Through the use of a weighting structure traffic can be scheduled at 1/2048 of the port rate. This action equates to approximately 488 Kbps for traffic exiting a Gigabit Ethernet port, 293 Kbps for traffic exiting an OC-12c port, and approximately 49 Kbps for traffic exiting a Fast Ethernet port. Using an ML-Series card it is common to create three types of queues as shown in Figure 7.5. The first type of queue is a low latency queue, which would be assigned a committed bandwidth of 100 percent, ensuring that data placed in that queue is serviced without delay. To limit the bandwidth used by this type of queue, you would need to assign a strict policy that would limit ingress traffic for each low latency queue. The second type of queue shown in Figure 7.5 is for unicast frames addressed to specific addresses. Similar to low latency queues, the unicast queues are created through an output service policy on egress ports. Each unicast queue is assigned a committed bandwidth, with the weight of the queue determined by the normalization of committed bandwidth of all defined unicast queues for a particular port. Any traffic in excess of the committed bandwidth on any queue is then treated by the scheduler based on the relative weight of the queue. Assignment Through the Use of ‘Priority’ Command
Through the Use of ‘Bandwidth’ Command
Automatically
Queue Low Latency Queues
Unicast Queues
Multicast/Broadcast Queues
Figure 7.5 Cisco ML-Series card queues
AU6039.indb 194
2/13/08 9:23:20 AM
Service Level Agreements and Quality of Service n 195
The third type of queue is a multicast/broadcast queue. That is, multicast and broadcast frames are automatically placed in such queues. Because Ethernet frames use the CoS bits as markers, those bits can denote both prioritized and discard-eligible frames. Then, when congestion occurs and a queue begins to fill, the first frames that will be dropped are those with a discardeligible setting in the CoS field. In comparison, committed frames will not be dropped until the total committed load exceeds the interface output.
Other Card Features In addition to performing ingress and egress priority marking the Cisco ML-Series card provides support for queue in queue. This enables a Carrier Ethernet service provider to transparently transport customer VLANs (C-VLANs) entering any specific port at the entrance to the service provider’s network over the service provider’s network. Thus, without reserving bandwidth through a network, you can prioritize traffic and expedite such traffic through queuing to provide a traffic flow that attains most of the qualities of a true QoS environment.
AU6039.indb 195
2/13/08 9:23:20 AM
AU6039.indb 196
2/13/08 9:23:20 AM
Index A Access delay, 162 Access point (see AP) Address resolution protocol (see ARP) ADSL, 6–7 ADSL2/ADSL2+, 6–7 Alteon Networks**, 116 Alien FEXT, 48 Alien NEXT, 48 Aloha packet network, 26, 45 American National Standards Institute (see ANSI) ANSI, 26 Any Transport over MPLS, (see ATOM) AP, 23–24 Application gateway, 36 ARP, 51 Asynchronous Transfer Mode (see ATM) ATM, 2–3, 15 ATOM, 10 Attachment Unit Interface (see AUI) AUI, 55 Auto-discovery, 98 Auto MDI/MDI-X, 58–59 Auto-negotiation, 66–74, 76–78 Auto-negotiation priorities, 73 Availability, 176–181
B Bandwidth, 183,189–190 Basepage, 67, 77 Basic Service Set (see BSS) Blocking delays, 132–133
Boggs, David, 25 Bridge operations, 127–131 Broadcast address, 101 Broadcast domain, 12 BSS, 24 Bus-based network structure, 46, 57 Business continuity, 4–5
C Cable categories, 49–50, 65, 78–79, 85–86 Carrier Ethernet, 1–16, 18–20, 26–27, 157–174 Carrier extension, 114–115 Access method, 26 Applications, 17–18 Challenges, 18–20 Definition, 1 Enabling technologies, 5–16 Encapsulation techniques, 167–174 Frame transport, 162–163 IEEE involvement, 26–27 Metro Ethernet Forum, 157–158 Overview, 2 Rationale, 3–5 Service types, 165–167 Topologies, 165 Carrier Sense Multiple Access with Collision Detection (see CSMA/CD) CBS, 190 Channelized versus non-channelized, 38–39 Cheap-net, 56 CIR, 190 Circuit-level gateway, 36 Class I repeater, 63–64, 74–75 Class II repeater, 63–64, 74–75
197
AU6039.indb 197
2/13/08 9:23:21 AM
198 n Index Committed burst size (see CBS) Committed information rate (see CIR) CRC, 30 Cross-point switching, 133–134 CSMA/CD, 26, 53, 54 Cut-through switching (see cross-point switching) Cyclic Redundancy Check (see CRC)
D Dark fiber, 163 Data Networking Concepts, 21–43 Deficit round robin scheduling, 193 DEMARC (demarcation) line, 180 Distribution system, 24 DIX frame, 100–104, 108 DIX standard, 25–26, 46–47, 53, 108 DSO Time slot, 38 Dual fiber, 90
E EBS, 190 Echo cancellation, 27 EFM, 87–98 Egress delay, 162 EIR, 189–190 E-LAN, 166–167 E-LINE, 166 EPON, 29, 91–98 ESS, 24 E-TREE, 167–168 Ethernet DIX standard, 25–26 Evolution, 25 First Mile, 28–29 Frame formats, 96–126 Performance, 118–119 Ethernet in the First Mile (see EFM) Ethernet II, 46, 100–103 Ethernet over Passive Optical Network (see EPON) Ethertype, 47 Excess Burst Size (see EBS) Excess Information Rate (see EIR) Explicit tagging, 144–145 Extended service set (see ESS)
AU6039.indb 198
F Far-end cross talk (see FEXT) Fast Ethernet, 3,27, 33, 60–66, 11–114 Fast Link Pulse (see FLP) FEXT, 48 Fiber-optic cable, 22, 27, 49–50, 90–91 Fiber-Optic Inter-Repeater Link (see FOIRL) Fiber-to-the-curb (see FTTC) Fiber-to-the-Neighborhood (see FTTN) Filtering, 129–130 Firewall, 35–36 Flooding, 129 Flow control, 108–109 FLP, 67 FOIRL, 59–60 Forwarding, 129 Frame bursting, 115 Frame check sequence, 102–103 Frame delay, 190 Frame delay variation, 190 Frame formats, 99–126 Frame loss ratio, 190 Frame size, 54–55 FTTC, 7 FTTN, 7 Full-duplex, 30, 52–53, 107–108
G GBIC, 163–164 Generic Routing Encapsulation, 37 Gigabit Ethernet, 3, 22, 27, 33, 75–80 Gigabit Interface Converter (see GBIC) Gigabit Media Independent Interface (see GMII) GMII, 77 Graded- index multi-mode fiber, 50–51, 74 10 Gigabit Ethernet, 22, 28, 40, 80–87, 114–117, 119–126 100 Gigabit Ethernet, 29, 87
H Half-duplex, 52, 108, 115 Hard QoS, 189 Hubs, 30–31, 57–58, 65 Hybrid switching, 137
2/13/08 9:23:21 AM
Index n 199
I
M
ICMP, 35 IEEE 802.3 standardization, 48–64 IETF, 51 Implicit tagging, 144–145 Intelligent hub, 31–32 Intelligent switching hub, 131–133 Internet Control Message Protocol (see ICMP) Internet Engineering Task Force (see IETF) IPSec, 9–11, 37 IPX over Ethernet, 106–107 Iso Ethernet, 68
MAN, 3, 22 Managed hub, 30 MDI, 58, 62 MDI-X cable, 58 Mean time before failure (see MTBF) Mean time to repair (see MTTR) Media access control, 48, 52–53 Medium Dependent Interface (see MDI) Medium Independent Interface (see MII) MEF QoS, 189 Meta Ethernet Forum, 157–158 Metcalfe, Robert, 25, 45–46 Metropolitan Area Ethernet, 1 Metropolitan Area Network (see MAN) MII, 61 M-i-M tagging, 170–172 Modular connectors, 57–59 MPCP, 94 MPLS, 13–16, 172–174 MTBF, 177–180 MTTR, 177–180, 182 Multi-mode fiber, 49–50 Multi-Point Control Protocol (see MPCP) Multi-Protocol Label Switching (see MPLS)
J Jam signal, 26 Jitter, 181 Jitter buffers, 181–182 Jumbo frames, 116–117
K Keep-alive (see LIT and NLP)
L Label Edge Router (see LER) Label Forwarding Information Base (see LFIB) Label Switch Path (see LSP) Label Switch Router (see LSR) Lampson, Butler, 25 LAN, 21–23 Latency, 161–162, 181 Layer 2 operations, 10, 12, 14–16, 32 Layer 3 operations, 10, 13, 14–16, 32 Layer 2 Tunneling Protocol (see L2TP) LER, 14, 16 LFIB, 16 Link Integrity Test (see LIT) LIT, 66–67 Local Area Network (see LAN) Logical link control, 48, 52, 104–105 LSP, 14 LSR, 14, 16 L2TP, 9–11
AU6039.indb 199
N Near-end crosstalk (see NEXT) Network Interface Card (see NIC) Network-to-Network Interface (see NNI) NEXT, 48 Next page function, 69–72 NIC, 30, 56 NLP, 66–67 NNI, 29 Normal Link Pulses (see NLP) NWay (see auto-negotiation)
O OAM, 183–187 OC-192c, 81 OLT, 92–98 ONU, 92–98 Open System Interface (see OSI)
2/13/08 9:23:21 AM
200 n Index Operational, administration, and maintenance (see OAM) Optical Line Terminator (see OLT) Optical Network Unit (see ONU) Optical splitter, 92 OSI, 51
P Packet loss, 183 Passive hub, 31–32 Pause frame, 108–109 PCS, 77, 83 Peer-to-peer, 23 Performance, 118–126 Ethernet, 118–119 Gigabit Ethernet, 119–126 Physical Medium Dependent (see PMD) Physical Medium Independent (see PMI) PMD, 77, 83 PMI, 77 Policing, 191–192 Port-based switching, 137–138 Pulse Coding Sublayer (see PCS)
Q Q-in-Q tagging, 168–170 QoS, 187–195 Quality of Service (see QoS) Queing, 192–195
R Refractive index, 50 RJ-45 (Registered jack), 57–58, 62, 79 Repeater, 63–64, 74–75 Roaming, 24 Router, 33–35
S Secure Sockets Layer (see SSL)
AU6039.indb 200
Segment-based switching, 138–140 Service level agreements, 175–187 Service Set Identifier (see SSID) Shim header, 14–15 SHDSL, 8 Single-mode fiber, 49–50 SNAP, 54, 104–106, 111 Soft QoS, 188–189 SONET, 2, 39–43, 83–84 SSID, 24 SSL, 9–11, 37 Star topology, 57 Step-index multi-mode fiber, 50 Store-and forward switching, 135–137 Sub-Network Access Protocol (see SNAP) Switch, 32–33, 127–156 Switch applications, 139–144
T T-Carrier Hierarchy, 38 Technology ability field, 68–69 Thacker, Chuck, 25 Thick-net, 55 Thin-net, 56 TIA/EIA-568 standard, 49, 62 Translating bridge, 127–128 Transparent bridge, 127–128 Transport technologies, 21–24 T1, 37–38
U UNI, 29, 190 User-to-Network Interface (see UNI)
V VDSL, 8, 22, 89 Virtual LANs (see VLANs) Virtual Private Network (see VPN) VLANs, 11–13, 109–113, 143–156, 158–160, 167–170 VPN, 9–11 VPN appliance, 36–37
2/13/08 9:23:21 AM
Index n 201
W
X
WAN, 21–23 WAN-PHY, 40 Wide Area Network (see WAN) Wireless, 23–24, 28
XGAUI, 82–83 XGMII, 81–82
AU6039.indb 201
2/13/08 9:23:21 AM
AU6039.indb 202
2/13/08 9:23:22 AM
Numbers Index 5-4-3 rule, 59 4B5B code, 112–113 64B/66B code, 80, 83 8B6T code, 66 8B/10B code, 77–78, 83 2 BASE-TL, 80 10 BASE-2, 27, 30, 56 10 BASE-5, 27, 30, 55–56 10 BASE-F, 27, 59 10 BASE-FB, 60 10 BASE-FL, 59–60, 75 10 BASE-FP, 60 10 BASE-T, 27, 30, 33, 49, 56, 66–67 10 PASS-TS, 88 100 BASE-BX, 75 100 BASE-BX10, 88 100 BASE-LX10, 88 100 BASE-SX, 75 100 BASE-T, 49, 60, 108 100 BASE-TX, 27, 49, 60, 62–63, 112–114 100 BASE- T2, 60, 66–67, 69 100 BASE-T4, 27, 60, 64–66 1000 BASE-BX10, 88 1000 BASE-CX, 27, 78–79 1000 BASE-LH, 78, 164 1000 BASE-LX, 27, 76–77, 164 1000 BASE-LX10, 88, 90 1000 BASE-PX10, 88 1000 BASE-PX20, 88
1000 BASE-SX, 27, 76–77 1000 BASE-T, 27, 49, 69, 71–72, 78 1000 BASE-ZX, 78, 164 10 BROAD-36, 56 10 GBASE-CX4, 84–85 10 GBASE-ER, 28 10 GBASE-LR, 28 10 GBASE- LX4, 28 10 GBASE-SR, 28 10 GBASE-ZR, 28 10 Gigabit Attachment Unit Interface (see XAUI) 10 Gigabit Media Independent Interface (see X GMII) 100 GBASE-T, 49, 67, 84–87 802.2 Header, 104 802.1D, 27 802.1P, 148, 160 802.1Q, 109–111, 145–155, 153–156, 158–160, 167–170 802.3 frame, 103–104 802.3ab, 27–28 802.3ae, 80 802.3ag, 184 802.3ah, 28, 184 802.3an, 85 802.3x, 108 802.3z, 27
203
AU6039.indb 203
2/13/08 9:23:22 AM
AU6039.indb 204
2/13/08 9:23:22 AM
