3D Integration for NoC-based SoC Architectures (Integrated Circuits and Systems)

Integrated Circuits and Systems

Series Editor Anantha P. Chandrakasan Massachusetts Institute of Technology Cambridge, Massachusetts

For further volumes: http://www.springer.com/series/7236

Abbas Sheibanyrad€•Â€Frédéric Pétrot€•Â€Axel Jantsch Editors

3D Integration for NoC-based SoC Architectures

1  3

Editors Abbas Sheibanyrad TIMA Laboratory 46, Avenue Felix Viallet 38000 Grenoble France [email protected]

Axel Jantsch Royal Institute of Technology Forum 120 SE-16440 Kista Sweden [email protected]

Frédéric Pétrot TIMA Laboratory 46, Avenue Felix Viallet 38000 Grenoble France [email protected]

ISSN 1558-9412 ISBN 978-1-4419-7617-8â•…â•…â•…â•… e-ISBN 978-1-4419-7618-5 DOI 10.1007/978-1-4419-7618-5 Springer New York Dordrecht Heidelberg London Library of Congress Control Number: © Springer Science+Business Media, LLC 2011 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

â•…

Preface

3D integration technologies, 3D-Design techniques, and 3D-Architectures are emerging as truly hot and broad research topics. As the end of scaling the CMOS transistor comes in sight, the third dimension may come to the rescue of the industry to allow for a continuing exponential growth of integration during the 2015–2025 period. As such 3D stacking may be the key technology to sustain growth until more exotic technologies such as nanowires, quantum dot devices and molecular computers become sufficiently mature for deployment in main stream application areas. The present book gathers the recent advances in the domain written by renowned experts to build a comprehensive and consistent book around the topics of threedimensional architectures and design techniques. In order to take full advantage of the 3D integration, the decision on the use of in-circuit vertical connection (predominantly Through-Silicon-Vias (TSVs) and Inductive Wireless Interconnects) must come upfront in the architecture planning process rather than as a packaging decision after circuit design is completed. This requires taking the 3D design space into account right from the start of the system design. Previously published books about this active research domain focus on fabrication technologies and physical aspects, rather than network and system-level architectural concerns. In contrast, the present book covers almost all architectural design aspects of 3D-SoCs, and as such can be useful both for introducing the current research topics to researchers and engineers, giving a basis for education and training in M.Sc. and Ph.D. programs. The book is divided into three parts. The first part, which contains two chapters, deals with the promises and challenges of 3D integration. Chapter€1 introduces 3D integration of Integrated Circuits, and as an objective, discusses performance enhancement, as well as new integration capabilities, enabled technology platforms, and potential applications made possible by 3D technology. Chapter€2 elaborates on the promises and limitation of 3D integration by studying the limits of performance under different memory distribution constraints of various 2D and 3D topologies in current and future technology nodes. The second part of the book consists of four chapters. It discusses technology and circuit design of 3D integration. Chapter€3 focuses on the available solutions and open challenges for testing 3D Stacked ICs (3D-SICs). It provides an overview v

vi

Preface

of the manufacturing steps of TSV-based 3D-SICs relevant for the testing issues. Chapter€4 reviews the process of 3D-IC designing exploiting Through-Silicon-Via technology, and introduces the notion of re-architecting systems explicitly to exploit high density TSV processes. Chapter€5 investigates physical properties of NoC topologies for 3D integrated systems. It describes an enhanced physical analysis methodology, providing a means to estimate early in the design cycle the behavior of a 3D topology for an integrated system interconnected with an on-chip network. Chapter€6 characterizes the performance of multiple 3D NoC architectures in the presence of realistic traffic patterns through cycle-accurate simulation and establishes the performance benchmark and related design trade-offs. The last part of the book includes five chapters that globally concern system and architecture design of 3D integration. Chapter€7 makes a case for using asynchronous circuits to implement 3D-NoCs. It claims that asynchronous logic allows for serializing vertical links, leading to the definition of innovative architectures which, by reducing the number of TSVs, can address some critical issues of 3D integration. Chapter€8, by supporting both unicast and multicast traffic flows, considers the problem of designing application-specific 3D-NoC architectures that are optimized for a given application. Chapter€9 presents methodologies and tools for automated 3D interconnect design, focusing on application-specific 3D-NoC synthesis which consists of finding the best NoC topology for the application, computing paths for the communication flows, assigning network components onto the layers of the 3D stack, and placing them in each layer. Chapter€10 describes constructing 3D-NoCs based on inductive-coupled wireless interconnect in which the data modulated by a driver are transferred between two inductors placed at exactly the same position of two stacked dies. Chapter€11 discusses how 3D technology can be implemented in GPUs. It investigates the problems and constraints of implementing such a technology and proposes architectural designs for a GPU that implements 3D technology and evaluates these designs in terms of fabrication cost, power consumption and thermal profile. We greatly appreciate all the authors’ hard work that led to their valuable contributions, whose patience and support are the basis for the production of this book. Their innovations and comprehensive presentations make this book novel, unique and useful. We also thank the publisher for his strong support and quick actions that made a speedy and timely publication possible. We sincerely hope that you will find reading this book as exciting and informative as we have done when selecting and discussing the content.  Abbas Sheibanyrad  Frédéric Pétrot  Axel Jantsch

Contents

I╅╇╛╛3DI Promises and Challenges ����������������������������������尓������������������������������ ╅╇ 1 1╅╇╛Three-Dimensional Integration of Integrated Circuits—an Introduction ����������������������������������尓����������������������������������� ╅╇ 3 Chuan Seng Tan 2╅╇╛The Promises and Limitations of 3-D Integration ������������������������������� â•… 27 Axel Jantsch, Matthew Grange and Dinesh Pamunuwa II╅╛╛╛Technology and Circuit Design ����������������������������������尓��������������������������� â•… 45 3╅╇╛Testing 3D Stacked ICs Containing Through-Silicon Vias ����������������� â•… 47 Erik Jan Marinissen 4â•…╇╛Design and Computer Aided Design of 3DIC ����������������������������������尓���� â•… 75 Paul D. Franzon, W. Rhett Davis and Thor Thorolfsson 5â•…╇╛Physical Analysis of NoC Topologies for 3-D Integrated Systems ����� â•… 89 Vasilis F. Pavlidis and Eby G. Friedman 6â•…╇╛Three-Dimensional Networks-on-Chip: Performance Evaluation ���� ╇ 115 Brett Stanley Feero and Partha Pratim Pande IIIâ•… System and Architecture Design ����������������������������������尓������������������������� ╇ 147 7╅╇╛Asynchronous 3D-NoCs Making Use of Serialized Vertical Links ���� ╇ 149 Abbas Sheibanyrad and Frédéric Pétrot 8╅╇╛Design of Application-Specific 3D Networks-onChip Architectures ����������������������������������尓������������������������������������尓������������ ╇ 167 Shan Yan and Bill Lin

vii

viii

Contents

╇ 9╅3D Network on Chip Topology Synthesis: Designing Custom Topologies for Chip Stacks����������������������������������尓�������������������� ╅ 193 Ciprian Seiculescu, Srinivasan Murali, Luca Benini and Giovanni De Micheli 10╅3-D NoC on Inductive Wireless Interconnect����������������������������������尓���� ╅ 225 Hiroki Matsutani, Michihiro Koibuchi, Tadahiro Kuroda and Hideharu Amano 11╅Influence of Stacked 3D Memory/Cache Architectures on GPUs����� ╅ 249 Ahmed Al Maashri, Guangyu Sun, Xiangyu Dong, Yuan Xie and Narayanan Vijaykrishnan Index ����������������������������������尓������������������������������������尓������������������������������������尓����� ╅ 273

Contributors

Ahmed Al Maashriâ•… The Pennsylvania State University, University Park, PA, USA Hideharu Amanoâ•… Keio University, Yokohama, Japan Luca Beniniâ•… The University of Bologna, Bologna, Italy W. Rhett Davisâ•… The North Carolina State University, Raleigh, NC, USA Giovanni De Micheliâ•… EPFL, Lausanne, Switzerland Xiangyu Dongâ•… The Pennsylvania State University, University Park, PA, USA Brett Stanley Feeroâ•… ARM Inc., Austin, TX, USA Paul D. Franzonâ•… The North Carolina State University, Raleigh, NC, USA Eby G. Friedmanâ•… University of Rochester, Rochester, NY, USA Matthew Grangeâ•… Lancaster University, Lancaster, UK Axel Jantschâ•… Royal Institute of Technology, Sweden Michihiro Koibuchiâ•… Japanese National Institute of Informatics, Tokyo, Japan Tadahiro Kurodaâ•… Keio University, Yokohama, Japan Bill Linâ•… University of California, San Diego, CA, USA Erik Jan Marinissenâ•… IMEC, Leuven, Belgium Hiroki Matsutaniâ•… The University of Tokyo, Tokyo, Japan Srinivasan Muraliâ•… EPFL, Lausanne, Switzerland Dinesh Pamunuwaâ•… Lancaster University, Lancaster, UK Partha Pratim Pandeâ•… Washington State University, Pullman, WA, USA Vasilis F. Pavlidisâ•… EPFL, Lausanne, Switzerland Frédéric Pétrotâ•… TIMA Laboratory, Grenoble, France

ix

x

Contributors

Ciprian Seiculescuâ•… EPFL, Lausanne, Switzerland Abbas Sheibanyradâ•… TIMA Laboratory, Grenoble, France Guangyu Sunâ•… The Pennsylvania State University, University Park, PA, USA Chuan Seng Tanâ•… Nanyang Technological University, Singapore Thorlindur Thorolfssonâ•… The North Carolina State University, Raleigh, NC, USA Narayanan Vijaykrishnanâ•… The Pennsylvania State University, University Park, PA, USA Yuan Xieâ•… The Pennsylvania State University, University Park, PA, USA Shan Yanâ•… University of California, San Diego, CA, USA

â•…

Part I

3DI Promises and Challenges

â•…

Chapter 1

Three-Dimensional Integration of Integrated Circuits—an Introduction Chuan Seng Tan

1.1â•…Background and Introduction Imagine a situation where you need to travel between your home and office every day. You need to put up with time lost during commute as well as paying for the fuel. One possible solution is to have your home in another floor of your office building. In this way, all you need to do is to go up and down between floors and you can save time and cost. This simple idea can similarly be applied to boost the overall performance in future integrated circuits. For the past 40 over years, higher computing power was achieved primarily through commensurate performance enhancement of transistors as a result of continuously scaling down the device dimensions in a harmonious manner. This has resulted in a steady doubling of device density from one technology node to another as famously described by Moore’s Law. Improvement in transistor switching speed and count are two of the most direct contributors to the historical performance growth in integrated circuits (particularly in silicon-based digital CMOS). This scaling approach has been so effective in many aspects (performance and cost) that integrated circuits have essentially remained a planar platform throughout this period of rigorous scaling. As performance enhancement through geometrical scaling becomes more challenging and demand for higher functionality increases, there is tremendous interest and potential to explore the third dimension, i.e., the vertical dimension of the integrated circuits. This was rightly envisioned by Richard Fenyman, physicist and Nobel Laureate, when he delivered a talk on “Computing Machines in the Future” in Japan in 1985. His original text reads: “Another direction of improvement (in computing power) is to make physical machines three dimensional instead of all on a surface of a chip. That can be done in stages instead of all at once—you can have several layers and then add many more layers as time C. S. Tan () School of Electrical and Electronic Engineering, Nanyang Technological University, 50 Nanyang Avenue, 639798 Singapore Tel.: +65-67905636 e-mail: [email protected] A. Sheibanyrad et al. (eds.), 3D Integration for NoC-based SoC Architectures, Integrated Circuits and Systems, DOI 10.1007/978-1-4419-7618-5_1, ©Â€Springer Science+Business Media, LLC 2011

3

4

C. S. Tan

goes on” [1]. The need for 3D integration has become clear going forward and it was reiterated by Dr. Chang-Gyu Hwang, President and CEO of Samsung Electronics’ Semiconductor Business, when he delivered a keynote speech at the 2006 International Electron Devices Meeting (IEDM) in San Francisco entitled “New Paradigms in the Silicon Industry” [2]. Some important points of his speech are quoted: “The approaching era of electronics technology advancement—the Fusion Era—will be massive in scope, encompassing the fields of information technology (IT), bio-technology (BT), and nano-technology (NT) and will create boundless opportunities for new growth to the semiconductor industry. The core element needed to usher in the new age will be a complex integration of different types of devices such as memory, logic, sensor, processor and software, together with new materials, and advanced die stack technologies, all based on 3-D silicon technology.”

1.2â•…Motivations and Drivers This section examines the role of 3D integration in ensuring that performance growth enjoyed by the semiconductor industry as a direct result of geometrical scaling, coupled with the introduction of performance boosters in more recent nodes, as predicted by Moore’s Law can continue in the future. Scaling alone has met with diminishing return due to fundamental and economics scaling barriers (non-commensurate scaling). 3D integration explores the third dimension of IC integration and offers new dimension for performance growth. 3D integration also enables integration of disparate chips in a more compact form factor and it is touted by many as an attractive method for system miniaturization and functional diversification commonly known as heterogeneous integration.

1.2.1  Sustainable IC Performance Growth Beginning with the invention of the first integrated circuit in 1958 by Kilby and Noyce, the world has witnessed sustainable performance growth in IC. The trend is best exemplified by the exponential improvement in computing power (measured in million instructions per second, MISP) in Intel’s micro-processors over the last 40€years as shown in Fig.€1.1 [3]. This continuous growth is a result of the ability to scale silicon transistor to smaller dimension in every new technology nodes. The growth continued, instead of hitting a plateau, in more recent nodes thanks to the addition of performance boosters (e.g., strained-Si, high-κ and metal gate, etc) on top of conventional geometrical scaling. Scaling doubles the number of transistors on IC (famously described by “Moore’s Law”) in every generation and allows us to integrate more functions on IC and to increase its computing power. We are now in the Giga-scale integration era featured by billions of transistors, GHz operating frequency, etc. Going forward to Tera-scale integration, however, there are a number of imminent show-stoppers (described in the next section) that pose serious threat to continuous performance

1â•… Three-Dimensional Integration of Integrated Circuits—an Introduction Fig. 1.1↜渀 The evolution of computing performance. (Source: Intel)

5

1.E+06 TIPS

1.E+05 1.E+04 MIPS

1.E+03 1.E+02 1.E+01 1.E+00 1.E-01 1.E-02 1960

1980

2000

2020

enhancement in IC and a new paradigm shift in IC technology and architecture is needed to sustain the historical growth. It is widely recognized that the growth can be sustained if one utilizes the vertical (i.e., the third) dimension of IC to build three-dimensional IC, a departure from today’s planar IC as projected in Fig.€1.2. Three-dimensional integrated circuits (3D IC) refers to a stack consists of multiple ultra-thin layers of IC that are vertically bonded and interconnected with through silicon via (TSV) as shown in Fig.€1.3. In 3D implementation, each block can be fabricated and optimized using their respective technologies and assembled to form a vertical stack. 3D stacking of ultra-thin ICs is identified as an inevitable solution for future performance enhancement, system miniaturization, and functional diversification.

Performance

Architectural 3D Integration Business as usual Performance Boosters (Cu/low-k, Strain, HK/MG, High- channels, 3D structures) Business as usual

Geometrical Scaling

1990s

2000s

Year

Fig. 1.2↜渀 Historical IC performance growth can be sustained with a new paradigm shift to 3-D integration

6

C. S. Tan

Fig. 1.3↜渀 A conceptual representation of 3D IC

Device/Interconnect

Through Silicon Via

Layer 4 Layer 3 Layer 2

Bonding Interface

Layer 1 (Substrate)

1.2.2  Show-Stoppers and 3-D Integration as a Remedy 1.2.2.1â•…Transistor Scaling Barriers There are at least two barriers that will slow down or impede further geometrical scaling. The first one relates to the fundamental properties of transistor in extremely scaled devices. Experimental and modeling data suggest that performance improvement in devices is no longer commensurate with ideal scaling in the past due to high leakage and parasitic hence they consume more power. This is shown in Fig.€1.4 by Khakifirooz and Antoniadis [4]. The intrinsic delay of n-MOS is shown to increase beyond the 45€nm despite continuous down scaling of the transistor pitch.

RO Stage Delay Intrinsic NMOS

Delay (ps)

10

1

Fig. 1.4↜渀 The intrinsic delay in n-MOS transistor is projected to increase in future nodes despite continuous down scaling of the device pitch [4]

45 nm Target @ 32 nm

10

100 Technology Node (nm)

600

Frequency

0

300

1,500 1,000 500 0

Frequency

7

900

1â•… Three-Dimensional Integration of Integrated Circuits—an Introduction

0

1 2 3 4 Normalized Leakage Current

5

0.7

0.8 0.9 1.0 1.1 1.2 1.3 Normalized Threshold Voltage

Fig. 1.5↜渀 Measured leakage current and threshold voltage in 65€nm devices reported by IBM [6]

Another issue related to scaled devices is variability [5]. Variability in transistor performance and leakage is a critical challenge to the continued scaling and effective utilization of CMOS technologies with nanometer-scale feature sizes. Some of the factors contributing to the variability increase are fundamental to the planar CMOS transistor architecture. Random dopant fluctuations (RDFs) and line-edge roughness (LER) are two examples of such intrinsic sources of variation. Other reasons for the variability increase are the advanced resolution-enhancement techniques (RETs) required to print patterns with feature sizes smaller than the wavelength of lithography. Transistor variation affects many aspects of IC manufacturing and design. Increased transistor variability can have negative impact on product performance and yield. Figure€1.5 shows measurement data reported by IBM on its 65€nm devices that clearly show variation in leakage current and threshold voltage. Variability worsens as we continue to scale in future technology nodes and it is a severe challenge. The second barrier concerns the economic aspect of scaling. The development and manufacturing cost has increased from one node to another making scaling a less favorable option in future nodes of IC. 3D integration on the other hand achieves device density multiplication by stacking IC layers in the third dimension without aggressive scaling. Therefore it can be a viable and immediate remedy as conventional scaling becomes less cost effective. 1.2.2.2â•…On-Chip Interconnect While dimensional scaling has consistently improved device performance in terms of gate switching delay, it has a reverse effect on global interconnect latency [7]. The global interconnect RC delay has increasingly become the circuit performance limiting factor especially in the deep sub-micron regime. Even though Cu/low-κ multilevel interconnect structures improve interconnect RC delay, they are not a long-term solution since the diffusion barrier required with Cu metallization has a finite thickness that is not readily scaled. The effective resistance of the interconnect

8

C. S. Tan

2-D wires C0

C1

C2

C3

C4

C5

C4

C3

C0

C1

C2

C5

3-D wires

Wire Counts

a

b

3-D

2-D

Wire Length

Fig. 1.6↜渀 a Long global wires on IC can be shortened by chip partitioning and stacking. b 3D integration reduces the number of long wires on IC

is larger than would be achieved with bulk copper, and the difference increases with reduced interconnect width. Surface electron scattering further increases the Cu line resistance, and hence the RC delay suffers [8]. When chip size continues to increase to accommodate for more functionalities, the total interconnects length increases at the same time. This causes a tremendous amount of power to be dissipated unnecessarily in interconnects and repeaters used to minimize delay and latency. On-chip signals also require more clock cycles to travel across the entire chip as a result of increasing chip size and operating frequency. Rapid rise in interconnects delay and power consumption due to smaller wire cross-section, tighter wire pitch, and longer lines that transverse across larger chips is severely limiting IC performance enhancement in current and future nodes. 3D IC with multiple active Si layers stacked vertically is a promising method to overcome this scaling barrier as it replaces long inter-block global wires with much shorter vertical inter-layer interconnects as shown in Fig.€1.6. 1.2.2.3â•…Off-Chip Interconnect (Memory Bandwidth Gap) Figure€1.7a depicts the memory hierarchy in today’s computer system in which the processor core is connected to the memory (DRAM) via power-hungry and slower off-chip buses on the board level. Data transmission on these buses experiences

1â•… Three-Dimensional Integration of Integrated Circuits—an Introduction Fig. 1.7↜渀 a Memory Hierarchy in today computer system. b Direct placement of memory on processor improves the data bandwidth

On-chip Wires

Core

9

off-chip Buses

Main Memory (DRAM)

Cache

a TSV

Memory Processor Core

b

severe delay and consumes significant amount of power. The number of available bus channels is also limited by the amount of external pin count available on the packaged chips. As a consequence, the data bandwidth suffers. As the computing power in processor increases in each generation, the limited bandwidth between processor core and memory places a severe limitation on the overall system performance [9]. The problem is even more pressing in multi-core architecture as every core will demand for data supply. To close this gap, the most direct way is to shorten the connections and to increase the number of data channels. By placing memory directly on processor, the close proximity shortens the connections and the density of connections can be increased by using more advanced CMOS processes (as opposed to packaging/assembly processes) to achieve fine-pitch TSV. This massively parallel interconnection is shown in Fig.€1.7b. Table€1.1 is a comparison between 2D and 3D implementations in terms of connection density and power consumption. Clearly, 3D can provide bandwidth enhancement (100X increment at the same frequency) at lower power consumption (10X reduction). Effectively, this translates into 1,000X improvement in bandwidth/power efficiency, an extremely encouraging and impressive number.

Table 1.1↜渀 Comparison of 2D and 3D implementations € 2D Connections density 5 tomorrow “Chip scale” 1–10 2–50 >5 (Cu) 0 (W)

Capacitance Up to 1€pF, typically 1), np (5.22b)

 where APE is the area of the processing element. The area of all of the PEs and, consequently, the length of each horizontal link are assumed to be equal. For those cases where the PE is implemented in multiple physical planes (↜np€ >€ 1), a coefficient coef is used to consider the effect of the interplane vias on the reduction in wirelength due to utilization of the third dimension. This coefficient is based on the layout of a crossbar switch designed [24] with the FDSOI 3-D technology from MIT Lincoln Laboratory (MITLL) [25]. In the following section, expressions for the power consumption of a network with delay constraints are presented.

5â•… Physical Analysis of NoC Topologies for 3-D Integrated Systems

97

5.3.2  Power Consumption Model for 3-D NoC Power dissipation is a critical issue in three-dimensional circuits. Although the total power consumption of a 3-D system is expected to be lower than that of an equivalent 2-D circuit (since the global interconnects are shorter [26]), the increased power density is a challenging issue for this novel design paradigm. Therefore, those 3-D NoC topologies that offer low power characteristics are of significant interest. The different power consumption components for interconnects with repeaters are briefly discussed in this section. Due to specified performance characteristics, a low power design methodology with delay constraints for the interconnect within an NoC is adopted from [19]. An expression for the power consumption per bit of a packet transferred between a source destination node pair is used as the basis for characterizing the power consumption of an NoC for the proposed topologies. The power consumption components of an interconnect line with repeaters are: (a) Dynamic power consumption is the dissipated power due to the charge and discharge of the interconnect and input gate capacitance during a signal transition, and can be described by 

2 Pdi = as_noc f (ci li + hi ki C0 )Vdd ,

(5.23)

where f is the clock frequency and as_noc is the switching factor [27]. (b) Short-circuit power is due to the DC current path that exists in a CMOS circuit during a signal transition when the input signal voltage changes between Vtn and Vdd€+€Vtp. The power consumption due to this current is described as shortcircuit power and is modeled in [28] by 

Psi =

2 2 4as_noc fId0 tri Vdd ki h2i , Vdsat GCeffi + 2H Id0 tri hi

(5.24)

where Id0 is the average drain current of the NMOS and PMOS devices operating in the saturation region and the value of the coefficients G and H are described in [29]. Due to resistive shielding of the interconnect capacitance, an effective capacitance is used in (5.23) rather than the total interconnect capacitance. Note that resistive shielding results in a smaller capacitive load as seen from the interconnect driver (i.e., Ceff€ ≤€ Ctotal). This effective capacitance is determined from the methodology described in [30, 31]. (c) Leakage power is comprised of two power components, the subthreshold and gate leakage currents. Subthreshold power consumption is due to current flowing where the transistor operates in the cut-off region (below threshold), causing Isub current to flow. The gate leakage component is due to current flowing through the gate oxide, denoted as Ig. The total leakage power can be described as





Pli = hi ki Vdd (Isub0 + Ig0 ),

(5.25)

98



V. F. Pavlidis and E. G. Friedman

where the average subthreshold Isub0 and gate Ig0 leakage current of the NMOS and PMOS transistors is considered in (5.25).

The total power consumption with delay constraint T0 for a single line of a crossbar switch Pstotal, horizontal buss Phtotal, and vertical buss Pvtotal is, respectively, 

Pstotal (T0 − ta ) = Pdi + Psi + Pli ,

(5.26)



Phtotal (T0 ) = Pdi + Psi + Pli ,

(5.27)



Pvtotal (T0 ) = Pdi + Psi + Pli .

(5.28)

The power consumed by the arbitration logic is not considered in (5.26)–(5.28) since most of the power is consumed by the crossbar switch and the buss interconnect, as discussed in [32]. Note that for a crossbar switch, the additional delay of the arbitration logic poses a stricter delay constraint on the switch. The minimum power consumption with delay constraints is determined by the methodology described in [19], which is used to determine the optimum size h*powi and number k*powi of repeaters for a single interconnect line. Consequently, the minimum power consumption per bit between a source destination node pair in an NoC with a delay constraint is 

Pbit = hopsPstotal + hops2−D Phtotal + hops3−D Pvtotal .

(5.29)

Note that the proposed power expression includes all of the power consumption components in the network, not only the dynamic power. The effect of resistive shielding is also considered in determining the effective interconnect capacitance. Additionally, the effect of temperature on each of the power dissipation components is considered. Furthermore, since the repeater insertion methodology described in [19] minimizes the power consumed by the repeater system, any additional decrease in power consumption is only due to the network topology. In the following section, a technique for analyzing the power dissipation of a PE and the related rise in temperature within an NoC system is described.

5.4â•…Thermal-Aware Analysis Methodology The 3-D NoC topologies and related timing and power models presented in the previous section emphasize performance improvements achieved by including the third dimension within the on-chip network. Vertical integration, however, can significantly enhance the performance of the PEs [33, 34], in addition to the performance of the network. Redesigning a planar PE into several physical planes can greatly decrease the power consumed by the PEs [35]. This reduction, in turn, lowers the temperature rise within the stack leading to tangible benefits in the overall behavior of the system. For example, since the temperature rise is limited, the corresponding increase in the interconnect resistance is lower, decreasing the interconnect delay within the NoC. Excessive increases in leakage power will also be avoided. In this

5â•… Physical Analysis of NoC Topologies for 3-D Integrated Systems

99

section, a methodology for analyzing the overall behavior of a 3-D system interconnected with an on-chip network is presented. Since the systems described in this chapter are presumed to be homogeneous, the power consumed by the entire system can be straightforwardly determined if the power dissipated by an elemental unit, as depicted in Fig.€5.1, is known. The power consumed by a PE and the two physical links that surround the PE is another useful metric for characterizing the different 3-D NoC topologies, 

Ptotal = PPE + 2Lp Pbit ,

(5.30)

where PPE is the power consumed by a single PE. PPE includes all of the different energy components described in Section 5.3.2. Different expressions, however, can be used to describe these components. The dynamic power consumption PPE_dyn is separated into the power constituents, Pg and Pint, for driving the capacitance of the logic gates and interconnects, respectively. The dynamic power consumption can therefore be written as   2  PPE_dyn = Pg + Pint = fPE · αs Ntot Cgate + Cint_loc + Cint_semi + Cint_glob Vdd , (5.31)

where Ntot is the number of gates within a PE, as is the switching activity within a PE, and fPE is the operating frequency of the PE. Note that this frequency is typically different from the clock frequency fc of the NoC. Cint_loc, Cint_semi, and Cint_glob, are, respectively, the total capacitance of the local, semi-global, and global interconnects driven by the Ntot gates within the PE. The leakage and short-circuit power dissipation of a PE can be determined similarly to (5.24)–(5.25). The leakage power of the PE is 

PPE_leak = Weff (Isub0 + Ig0 ) · Vdd ,

(5.32)

where Weff is the total width of the transistors within a PE. Assuming that the PEs consist of four transistor gates and the transistors are sized to equalize the rise and fall transitions, the width of the four devices is a function of the minimum feature size. Multiplying this width by the number of gates within the PE, Weff is obtained. To determine the interconnect capacitance described in (5.31), the distribution of the interconnect within a PE is required. An enhanced wire distribution model for either 2-D or 3-D circuits based on [36] is utilized in this analysis. This model is further integrated into the methodology described by [37] to produce the number of tiers (i.e., local, semi-global, global) and the pitch of each metal layer. A pair of metal layers routed orthogonally is assumed to comprise a tier. A specific constraint for the interconnect delay is set for each tier to determine the maximum interconnect length that can be placed within that tier. This constraint is set to 25 and 90% of the clock frequency fPE for the local and other tiers, respectively. An interconnect is considered to be placed on the next tier whenever the length of the wire does not satisfy the aforementioned constraint. Although the effect of temperature is considered in these power expressions, the methodology described in [37] does not consider thermal issues. To consider

100

V. F. Pavlidis and E. G. Friedman Q3

I3 ∆T3

∆T3 = Q3(R3+R2+R1) + Q2(R2+R1) + Q1R1

∆T2 = Q3(R2+R1) + Q2(R2+R1) + Q1R1

Q2

Q1

R3 ∆T2 R2 ∆T1

∆T1 = R1(Q3+Q2+Q1)

R1

V3 = I3(R3+R2+R1) + I2(R2+R1) + I1R1 I2

R3 V2 = I3(R2+R1) + I2(R2+R1) + I1R1

I1

R2 V1 = R1(I3+I2+I1) R1

Fig. 5.2↜渀 An example of the duality of thermal and electrical systems

these important 3-D circuit effects, a first-order thermal model of a 3-D circuit has been integrated into this methodology. This model assumes a one-dimensional (1-D) flow of heat throughout a 3-D stack [38, 39]. Note that the model includes both the heat generated by the devices and interconnect. The assumption of a 1-D heat flow is justified as the path towards the heat sink exhibits the lowest thermal resistance, facilitating the removal of heat from the circuit to the ambient. This path is the primary path for the heat to flow in a 3-D circuit [39]. By exploiting the electro-thermal duality, as illustrated in Fig.€5.2, the temperature in each plane and each metal layer within a physical plane of a 3-D circuit can be determined. The number of metal layers, metal pitch within these layers, temperature rise, and power dissipation for a PE can all be determined early in the design cycle by utilizing this thermal-aware interconnect architecture design methodology. The analysis method is depicted in Fig.€5.3. The input parameters are the target technology node, the number of gates Ntot within the PE, the number of physical planes np used for the PE, and the clock frequency fPE. Most of the interconnect and related parameters can be extracted for the target technology node. The complete methodology proceeds as follows: 1. For the clock frequency fPE and nominal temperature, an initial number of metal layers and the interconnect pitch are determined to satisfy the aforementioned delay constraints. 2. For this interconnect architecture and assuming the same average current density for the wires on each metal layer, the rise in temperature is determined. 3. Based on this increase in temperature, the electrical wire resistance, leakage power, and other temperature dependent parameters are updated. The interconnect delay is again evaluated against the input delay constraints. If these specifications are not satisfied, a new interconnect architecture is produced. 4. The iterative process terminates once the circuit has reached a steady state temperature and the delay constraints for each tier is satisfied. The output is the number of metal layers, interconnect pitch, and temperature of the circuit.

5â•… Physical Analysis of NoC Topologies for 3-D Integrated Systems Fig. 5.3↜渀 Analysis flow diagram that produces a first-order interconnect architecture and the steady-state temperature of a PE consisting of np physical planes within a 3-D stack of n planes (↜np€≤€n)

101

fPE, np, Ntot

Determine # of tiers, interconnect pitch, and distribution

Calculate T

Update wire resistance, temperature dependent parameters, leakage power

Are delay specs. met?

NO

YES

NO

Is ΣT < 10–5 °C?

YES Report steady-state temp., interconnect pitch, number of tiers per PE

Once the temperature and interconnect length at each tier have been determined, the power consumed by the PE is readily determined from (5.30)–(5.32). In this manner, the topology that produces the lowest power dissipation for the entire 3-D system rather than the physical link of the on-chip network is determined. Following this procedure, the different 3-D topologies discussed in Section€5.2 are evaluated in terms of the latency and power dissipation of the NoC, the total power dissipation of the system, and the rise in temperature. Various tradeoffs inherent to these topologies are also discussed for a NoC-based 3-D system.

102

V. F. Pavlidis and E. G. Friedman

5.5â•…Latency, Power, and Temperature Tradeoffs in 3-D NoC Topologies The improvement in the performance of traditionally planar on-chip networks by introducing the third-dimension is discussed in this section. Those topologies that produce the highest network performance, the lowest power dissipation of the network and system, and the lowest rise in temperature are evaluated. Different topologies are demonstrated that satisfy each of these objectives. Consequently, the analysis methodology presented in the previous section can be a useful tool to evaluate the improvement in a specific objective offered by a 3-D topology. The effect of the 3-D topologies on the speed, power, and temperature rise of a network is discussed in Sections€5.5.1, 5.5.2, and 5.5.3, respectively. The latency, power, and rise in temperature of a 2-D mesh network with the same number of nodes is used as a reference for the comparisons throughout this section. In all of the on-chip networks, a 45€ nm technology node, as described in the ITRS, is assumed [15]. Consequently, specific interconnect and device parameters, such as the minimum pitch of the horizontal interconnects, the maximum power density, and the dielectric constant k are adopted from this report. A TSV technology is assumed to provide the vertical interconnects, with a TSV pitch of 5€μm and an aspect ratio of five [33]. Finally, a synchronous on-chip network is assumed with a clock frequency of fc€=€1€GHz. A small set of parameters are considered as variables throughout the analysis of the 3-D topologies. These variables include the network size, number of physical planes making up the 3-D system, clock frequency of the PEs fPE, and area of the PEs APE. The range of these variables is listed in Table€5.1. For multi-processor SoC networks, sizes of up to N€=€256 are expected to be feasible within the near future [7, 40], whereas for NoC with a finer granularity, where each PE corresponds to a hardware block of approximately 100 thousand gates, network sizes over a few thousands nodes are predicted at the 45€nm technology node [41]. Furthermore, to apply the thermal model discussed in [38, 39], the nominal temperature is considered to be 300€K and the side and topmost surfaces of the 3-D stack are assumed to be adiabatic. In other words, the heat is assumed to flow from the uppermost to the lowest plane of the multi-plane system.

Table 5.1↜渀 Parameters of the investigated NoCs

Parameter N APE (mm2) fPE (GHz) Max. number of planes, nmax

Values 16, 32, 64, 128, 256, 512 0.64, 0.81, 1.00, 2.25 1, 2, 3 8

5â•… Physical Analysis of NoC Topologies for 3-D Integrated Systems

103

5.5.1  Latency of 3-D NoCs Utilizing the third dimension to implement an on-chip network (i.e., 2-D IC – 3-D NoC topology) by simply stacking the PEs (i.e., n3€>€1, np€=€1) and using network switches with seven ports decrease the average number of hops for packet switching, thereby improving the network latency. Alternatively, utilizing the third dimension to decrease the length of the physical links between adjacent network switches (i.e., 3-D IC – 2-D NoC topology) also reduces the latency of the network. The reduction in buss length is achieved by implementing the PEs in multiple physical planes (i.e., n3€=€1, np€>€1), thereby reducing PE area. Finally, a hybrid topology (i.e., 3-D IC – 3-D NoC topology), which uses the third dimension both for the onchip network and the PEs (i.e., n3€>€1, np€>€1), can result in the greatest reduction in latency. Note that the effect of the various topologies on only the speed of the network, described by fc, is considered, while the operating frequency of the PEs fPE can be different. Although this approach is convenient from an architectural perspective, certain physical design issues can arise due to the multiple clock domains that can co-exist in these topologies. Each of these topologies faces different synchronization challenges. A multi-plane on-chip network can be implemented with various synchronization schemes ranging from a fully synchronous approach (as assumed herein) to an asynchronous network. A potent non-synchronous approach that has been used for PE-to-network communication in planar systems is the globally asynchronous locally synchronous (GALS) approach. An immediate extension of the GALS approach into three physical dimensions could include a number of clock domains equal to the number of planes comprising a 3-D circuit. This synchronization scheme is suitable for the 2-D IC – 3-D NoC topology. Alternatively, for the 3-D IC – 2-D NoC topology where the network is planar, a synchronous clocking scheme is a simpler and efficient solution. The synchronization challenge for this topology is related to the multi-plane PEs. The primary issue is how to efficiently propagate the clock signal across a PE occupying several planes. Recently, several synthesis techniques that produce bounded skew and testable clock trees prior and after bonding have been reported [42]. In addition, preliminary experimental results on multi-plane clock networks demonstrate that operating frequencies in the gigahertz regime are feasible, while the clock skew is manageable [24]. Although functional testing for each plane of a multi-plane PE remains an important problem, early results are encouraging. For the 3-D mesh networks discussed in this chapter, fully synchronous schemes are assumed due to the simplicity of these approaches. In this way, the effect of the topological characteristics rather than the synchronization mechanism on the performance of the network is evaluated, which is the objective of the physical analysis flow discussed in this chapter. The latency for different network sizes and a PE area of 0.64€mm2 and 2.25€mm2 is illustrated in Fig.€5.4a, b, respectively. Note

104 12

20

2D ICs - 2D NoCs 2D ICs - 3D NoCs 3D ICs - 2D NoCs 3D ICs - 3D NoCs

18 16

2D ICs - 2D NoCs 2D ICs - 3D NoCs 3D ICs - 2D NoCs 3D ICs - 3D NoCs

14

8

Latency [ns]

Latency [ns]

10

V. F. Pavlidis and E. G. Friedman

6 4

12 10 8 6 4

2

2 0

a

4

5 6 7 8 Number of nodes log2N

9

0

b

4

5

6

7

8

9

Number of nodes log2N

Fig. 5.4↜渀 Zero-load latency for various network sizes where (a) APE€ =€ 0.64€ mm2 and (b) APE€=€ 2.25€mm2 for fPE€=€1€GHz

that the temperature rise due to stacking or folding the PEs is also considered by the methodology presented in Section€5.4. The latency of the 2-D IC – 3-D NoC topology decreases with increasing network size. For example, for N€=€16, the third dimension does not improve the latency, since the number of hops required for packet switching is small. The increase in the delay of the network switch (due to the increase in the number of switch ports) can outweigh the decrease in latency due to the reduction in the number of hops, which is negligible for this network size. As the network size increases, however, the decrease in latency offered by this topology progressively increases. The improvement in latency increases from 1.08% for N€=€32 to 6.79% for N€=€256, where APE€=€0.64€mm2. In addition, the area of the PE has no significant effect in this improvement, as depicted in Fig.€5.4, since this topology only alters the number of hops for packet switching. Note, however, that the absolute network latency increases for this topology, since the length of the busses increases with the area of the PEs. The 3-D IC – 2-D NoC exhibits the opposite behavior, since this topology reduces the length of the physical links. Thus, the improvement in latency increases in those networks with a large PE area. The latency decreases by 48.97% for APE€=€0.64€mm2 and 60.39% for APE€=€2.25€mm2 for a network size of N€=€256. The reduction in network latency for this topology decreases with increasing network size. As the network size increases, the greatest portion of the latency as described by (5.9) is due to the larger number of hops rather than the buss delay. Consequently, the benefits offered by the reduction in length of the busses decrease with network size for the 3-D IC – 2-D NoC topology. For example, the improvement in latency decreases from 54.12% for N€=€32 to 50.83% for N€=€128, where APE€=€0.64€mm2. The hybrid topology 3-D IC – 3-D NoC demonstrates the greater improvement in latency as compared to a 2-D network, since the third dimension decreases both the length of the busses and the number of hops [43]. Depending upon the network size, the area of the PE, and the interconnect impedance characteristics of the bus-

5â•… Physical Analysis of NoC Topologies for 3-D Integrated Systems

105

ses, n3 and np ensure that the 3-D IC – 3-D NoC topology can support either the 2-D IC – 3-D NoC (i.e., n3€=€nmax) or the 3-D IC – 2-D NoC (i.e., np€=€nmax). The results shown in Fig.€5.4 include the increase in delay caused by the rise in temperature within a 3-D stack. Based on the methodology discussed in the previous section, the resulting temperature rise does not significantly affect the improvement in latency provided by the 3-D topologies. The resulting higher temperatures within the 3-D system cause a small 3% increase in the interconnect latency for all of the investigated networks. The thermal effects are similar to those discussed in [44, 45]. Consequently, the overall effect of the 3-D topologies is that the network latency is significantly decreased, although an inevitable increase in temperature will consume a small portion of this improvement. The effect of the third dimension on the power consumed by the network and the PEs is discussed in the following section.

5.5.2  Power Dissipation of 3-D NoCs The decrease in the power of a conventional 2-D on-chip network achieved by the 3-D topologies is presented in this section. Two different power consumption metrics are used to characterize the benefits of these topologies. First, the 3-D topology that minimizes the power consumed by the network is described by (5.29), ignoring the power of the PEs. For the second metric, the overall power dissipation of the system, including both the power of the network and the PEs, is described by (5.30). Those topologies that minimize each of these two metrics are determined. Furthermore, the distribution of the network nodes in terms of the physical dimensions (i.e., n1, n2, n3, and np) can be quite different for the same 3-D topology. The power consumed by these 3-D topologies is illustrated in Fig.€5.5, where the power dissipated by the PEs is ignored. Similar to the discussion related to latency, the 2-D IC – 3-D NoC and 3-D IC – 2-D NoC topologies lower the power dissipated 0.7 0.6

1

0.4 0.3

0.6 0.4

0.1

0.2 4

5 6 7 8 Number of nodes log2N

9

0

b

2D ICs - 2D NoCs 2D ICs - 3D NoCs 3D ICs - 2D NoCs 3D ICs - 3D NoCs

0.8

0.2

0

a

1.2

Pbit [mW]

Pbit [mW]

0.5

1.4

2D ICs - 2D NoCs 2D ICs - 3D NoCs 3D ICs - 2D NoCs 3D ICs - 3D NoCs

4

5 6 7 8 Number of nodes log2N

9

Fig. 5.5↜渀 Power consumed by the network with delay constraints (↜渀fc€=€1€GHz) for several network sizes where (a) APE€=€0.64€mm2 and (b) APE€=€2.25€mm2 for fPE€=€1€GHz

106

V. F. Pavlidis and E. G. Friedman

by the network through a reduction in the number of hops and the capacitance of the wires, respectively. Note that the y-axis in Fig.€5.5 corresponds to the power required to transfer a single bit over an average distance within the network where this distance is determined by the number of hops for packet switching, as described by (5.2). Comparing Fig.€5.5a, b the power consumed by a planar on-chip network increases with the area of the PEs interconnected by this network. For example, the power almost doubles for the same network size as the area of the PE increases from 0.64€mm2 to 2.25€mm2. Similar to the network latency, the power consumption decreases in the 2-D IC – 3-D NoC topology by reducing the number of hops for packet switching. Again, the increase in the number of ports adds to the power consumed by the crossbar switch; however, the effect of this increase in power is not as significant as the corresponding increase in the latency of the network. A three-dimensional network can therefore reduce power even in small networks. The power savings achieved with this topology is greater in larger networks. This situation occurs because the reduction in the average number of hops for a three-dimensional network increases in larger network sizes. With the 3-D IC – 2-D NoC topology, the number of hops in the network is the same as for a two-dimensional network. The horizontal buss length, however, is shortened by implementing the PEs in more than one physical plane. The greater the number of physical planes that can be integrated in a 3-D system, the larger the power savings; meaning that the optimum value for np with this topology is always nmax regardless of the network size and operating frequency (if temperature is not the target objective). The savings is practically limited by the number of physical planes that can be integrated in a 3-D technology. For this type of NoC, the topology resulting in the maximum speed is identical to the topology minimizing the power consumption, as the key element of either objective originates solely from the shorter buss length. Finally, the 3-D IC – 3-D NoC can achieve the minimum power consumption for a 3-D on-chip network by properly adjusting n3 and np depending upon the interconnect impedance characteristics, the available number of physical planes, and the clock frequency of the network. Interestingly, when the power metric described in (5.30) is utilized, the topologies that minimize the total power are different, as illustrated in Fig.€ 5.6. The distribution of the network nodes within those topologies also changes. The total power of a network-based 3-D system is plotted in Fig.€5.6, where the clock frequency of the network is fc€=€1€GHz and the area of the PE is APE€=€0.64€mm2 and APE€=€2.25€mm2. The clock frequency of the PE is equal to fc in this case. A common characteristic of Fig.€5.6a, b is that for larger network sizes, the topology that produces the lowest power dissipation changes from the 3-D IC – 2-D NoC to the 2-D IC – 3-D NoC topology (for this specific example, the 3-D IC – 3-D NoC coincides with either of these topologies). For small networks and PE area (see Fig.€5.6a), the reduction in power originates from the shorter buss length of the network and the shorter interconnects within the PEs since the PEs are implemented in multiple planes. As the network size increases, however, the number of hops increases considerably, making the power dissipated by the network the dominant

5â•… Physical Analysis of NoC Topologies for 3-D Integrated Systems 5 4.5

8

3.5

7

3

6

2.5 2

4 3

1

2

0.5

1 4

5 6 7 8 Number of nodes log2N

9

0

b

2D ICs - 2D NoCs 2D ICs - 3D NoCs 3D ICs - 2D NoCs 3D ICs - 3D NoCs

5

1.5

0

a

9

Ptotal [W]

Ptotal [W]

4

10

2D ICs - 2D NoCs 2D ICs - 3D NoCs 3D ICs - 2D NoCs 3D ICs - 3D NoCs

107

4

5 6 7 8 Number of nodes log2N

9

Fig. 5.6↜渀 Total power consumed by a PE and the adjacent network busses according to (5.30) with delay constraints (╛fc€ =€ 1€ GHz) for several network sizes where (a) APE€ =€ 0.64€ mm2 and (b) APE€=€2.25€mm2 for fPE€=€1€GHz

power component consumed by the system. Consequently, the 3-D IC – 2-D NoC does not offer the maximum power savings; the maximum savings is now achieved with the 2-D IC – 3-D NoC topology. If the PE is larger, the network size at which the optimum topology changes increases. This behavior occurs since larger PEs include a greater number of gates leading to additional longer interconnections within the PEs, as described by the interconnect distribution model presented in Section€5.4. The greater number and length of the wires within a PE are the primary power component of the entire system. The 3-D IC – 2-D NoC topology offers a greater improvement for even larger network sizes before the power caused by the increasing number of hops starts to dominate. This behavior occurs since the 3-D IC – 2-D NoC topology reduces the length of the interconnects within the PEs in addition to the length of the network busses. Another interesting result is that the clock frequency of the PE fPE affects the overall power dissipation, a factor typically ignored when evaluating the performance of a network-based integrated system. In Fig.€5.7, fPE increases from 1 to 3€GHz. This increase has a profound effect on the overall power of the system. To satisfy this aggressive timing specification while limiting the interconnect power consumption, the 3-D IC – 2-D NoC topology exhibits the best results for most of the network sizes depicted in Fig.€5.7a. Note that this behavior is more pronounced for larger PE areas, as depicted in Fig.€5.7b, where the 3-D IC – 2-D NoC topology performs better than the 2-D IC – 3-D NoC topology for any network size. Furthermore, the 3-D IC – 3-D NoC topology can lead to the lowest power consumption with appropriate adjustment of the parameters n3 and np. To demonstrate the distribution of the networks nodes within the three physical dimensions in addition to the effect on the topology, the node distribution is listed in Table€5.2 for specific network and PE characteristics. From Table€5.2, both the operating frequency and the area of the PEs affect the distribution of nodes within the NoC. Large PE areas (↜APE€=€2.25€mm2) and high operating frequencies (╛↜fPE€=€3€GHz) require 3-D NoC topologies where some of the physical planes are

108

V. F. Pavlidis and E. G. Friedman

6

14

2D ICs - 2D NoCs 2D ICs - 3D NoCs 3D ICs - 2D NoCs 3D ICs - 3D NoCs

5

10 Ptotal [W]

4 Ptotal [W]

2D ICs - 2D NoCs 2D ICs - 3D NoCs 3D ICs - 2D NoCs 3D ICs - 3D NoCs

12

3 2

8 6 4

1

2

0

4

a

5 6 7 8 Number of nodes log2N

0

9

4

b

5 6 7 8 Number of nodes log2N

9

Fig. 5.7↜渀 Total power consumed by a PE and the adjacent network busses according to (5.30) with delay constraints (↜fc€ =€ 1€ GHz) for several network sizes where (a) APE€ =€ 0.64€ mm2 and (b) APE€=€2.25€mm2 for fPE€=€3€GHz

Table 5.2↜渀 Node distribution for minimum power consumption of different network sizes N fPE€=€1€GHz fPE€=€3€GHz APE€=€2.25€mm2

APE€=€0.64€mm2

╇ 16 ╇ 32 ╇ 64 128 256 512

APE€=€0.64€mm2

APE€=€2.25€mm2

n1

n2

n3

np

n1

n2

n3

np

n1

n2

n3

np

n1

n2

n3

np

4 8 8 4 8 8

4 4 8 4 4 8

1 1 1 8 8 8

8 8 8 1 1 1

╇ 4 ╇ 8 ╇ 8 16 16 ╇ 8

╇ 4 ╇ 4 ╇ 8 ╇ 8 16 ╇ 8

1 1 1 1 1 8

8 8 8 8 8 1

╇ 4 ╇ 8 ╇ 8 16 16 ╇ 8

╇ 4 ╇ 4 ╇ 8 ╇ 8 16 ╇ 8

1 1 1 1 1 8

8 8 8 8 8 1

╇ 4 ╇ 8 ╇ 8 16 16 32

╇ 4 ╇ 4 ╇ 8 ╇ 8 16 16

1 1 1 1 1 1

8 8 8 8 8 8

used for the PEs (i.e., np€>€1). For small PEs (↜APE€=€0.64€mm2) and low operating frequencies (↜fPE€=€1€GHz), a simple 3-D network (i.e., n3€>€1 and np€=€1) is typically the best choice. Note that the selection of the optimum topology for either a latency or power objective depends strongly on the interconnect and device characteristics of the specific technology node. Consequently, even for system level exploratory design, the analysis methodology presented in Section€5.4 provides a first estimate of the behavior of a network-based 3-D system. The related temperature rise for these 3-D topologies, which is another design objective for this type of integrated system, is discussed in the following section.

5.5.3  Temperature in 3-D NoCs Elevated temperatures are expected to become an important challenge in vertical integration, specifically where several high performance circuits form a multi-plane

5â•… Physical Analysis of NoC Topologies for 3-D Integrated Systems

109

integrated system. The increased power densities per unit volume can potentially increase the operating temperature of the system to prohibitive levels, greatly affecting the performance characteristics and severely degrading the reliability of this system. Consequently, the temperature rise resulting from these 3-D topologies is of primary interest. Based on the methodology described in Section€5.4, the temperature of the substrate and each metal layer within a physical plane is determined assuming a one-dimensional flow of heat towards the heat sink. The heat sink is assumed to be attached to the lowest plane within the 3-D stack. The change in temperature rise due to the different 3-D topologies, area and operating frequency of the PEs, and number of physical planes are discussed in this section. Considering the 3-D NoC topologies discussed in this chapter, the 2-D IC – 3-D NoC topology will result in higher temperatures as compared to the 3-D IC – 2-D NoC topology since the former topology simply stacks the PEs, while the latter topology utilizes more than one plane to implement a PE. The 2-D IC – 3-D NoC topology leads to higher temperatures for two reasons. Several PEs, determined by n3, are placed adjacent to the vertical direction. Consequently, the power density generated by both the devices and metal layers increases. In addition, each of these PEs is implemented in one physical plane (i.e., np€=€1) and, hence, no reduction in power density is possible. Alternatively, the 3-D IC – 2-D NoC topology utilizes more than one plane for each PE, reducing the interconnect load capacitance and, consequently, the temperature within the 3-D system. The temperature rise resulting from the different 3-D topologies is illustrated in Fig.€ 5.8 for different number of planes. These temperatures correspond to the temperature rise at the topmost metal layer of the uppermost physical plane over the nominal temperature (here assumed to be 27°C). From Fig.€5.8, as the number of planes increases, the temperature naturally increases for the 2-D IC – 3-D NoC topology (for this topology np€=€1). Alternatively, when some or all of the physical planes are used to implement the PEs, as occurs for the 3-D IC – 3-D NoC and 3-D 8

np = 1 np = 2 np = 4 np = 8

10 Temp. rise [°C]

6 Temp. rise [°C]

12

np = 1 np = 2 np = 4 np = 8

7

5 4 3

8 6 4

2 2

1

a

0

1

2

4 # of planes

8

b

0

1

2

4 # of planes

8

Fig. 5.8↜渀 Temperature rise within the 3-D topologies for different combinations of n3 and np. For all of the topologies, n3€×€np€=€n, where n is the number of planes within the 3-D stack. A maximum number of planes nmax€=€8 is assumed, according to Table€5.1. The clock frequency of the PE is fPE€=€1€GHz and the area is (a) APE€=€0.64€mm2 and (b) APE€=€2.25€mm2

110

V. F. Pavlidis and E. G. Friedman

8

np = 1 np = 2 np = 4 np = 8

10 Temp. rise [°C]

6 Temp. rise [°C]

12

np = 1 np = 2 np = 4 np = 8

7

5 4 3

8 6 4

2 2

1 0

a

1

2

4 # of planes

8

0

b

1

2

4 # of planes

8

Fig. 5.9↜渀 Temperature rise within the 3-D topologies for different combinations of n3 and np. For all of the topologies, n3€×€np€=€n; where n is the number of planes in the 3-D stack. A maximum number of planes nmax€=€8 is assumed according to Table€5.1. The area of the PE is APE€=€0.64€mm2 and the clock frequency is (a) fPE€=€1€GHz and (b) fPE€=€3€GHz

IC – 2-D NoC topologies, the temperature rise is considerably smaller. Note, for example, that a 3-D system consisting of eight planes exhibits comparable temperatures to another system comprised of only four planes, as long as the former system uses two physical planes for the PE. This behavior is more pronounced for PEs with larger area, as depicted in Fig.€5.8b. For this case, using more than one plane for the PE significantly reduces the power density per plane. Most importantly, however, the number of metal layers required for a two-plane PE can be smaller [36]. This construction decreases the length and resistance of the vertical thermal path to remove the heat within the 3-D stack. An increase in temperature also occurs when the operating frequency of the PEs increases, as illustrated in Fig.€5.9. This behavior can be explained by noting that an increase in frequency produces a linear increase in the (dynamic) power consumed by the 3-D system. Note that the temperature rise for higher frequencies within the PEs is comparable to the increase observed for PEs with larger areas. In Fig.€5.8, the area of the PE is almost quadrupled, while in Fig.€5.9 the operating frequency is tripled, resulting in approximately the same rise in temperature. This behavior can be explained as follows. A larger PE includes additional gates that require additional wiring resources. Alternatively, tighter timing constraints can be satisfied, in this example, by increasing the wire pitch. If in either case, an additional tier is required, the thermal resistance of the heat flow path increases. Additionally, for both cases, the power consumption increases, resulting in higher temperatures. The increase in temperature shown in Figs.€5.8 and 5.9 is for the highest metal layer of the uppermost physical plane within a 3-D system. Although this increase may not be catastrophic, the timing specifications for the PE or the network can possibly not be satisfied if temperature is ignored. To better explain this situation, the different metal pitches for the PEs considering thermal effects are listed in Table€5.3 for the 2-D IC 3-D NoC topology where n3€=€8. In columns 2 to 7, thermal effects are not considered in the analysis flow diagram depicted in Fig.€5.3, while

5â•… Physical Analysis of NoC Topologies for 3-D Integrated Systems

111

Table 5.3↜渀 Pitch of the interconnect layers for each plane for the 2-D IC – 3-D NoC topology where n3€=€8, APE€=€1€mm2, and fPE€=€3€GHz. Two cases are considered, where the system operates at nominal T0 and at temperature T0€+€ΔΤ. At the uppermost plane ΔΤ€=€20.1°C T0€+€ΔΤ T0 Plane # of Metal pitch (nm) # of Metal pitch (nm) tiers tiers € Tier 1 Tier 2 Tier 3 Tier 4 Tier 5 € Tier 1 Tier 2 Tier 3 Tier 4 Tier 5 1 2 3 4 5 6 7 8

5 5 5 5 5 5 5 5

90 90 90 90 90 90 90 90

270 270 270 270 270 270 270 270

900 900 900 900 900 900 900 900

1,440 1,440 1,440 1,440 1,440 1,440 1,440 1,440

2,000 2,000 2,000 2,000 2,000 2,000 2,000 2,000

5 5 5 5 5 5 5 5

90 90 90 90 90 90 90 90

270 270 270 270 270 270 270 270

900 900 900 900 900 900 900 900

1,280 1,280 1,280 1,280 1,280 1,280 1,280 1,280

2,250 2,500 2,500 2,750 2,750 3,000 3,000 3,000

in columns 8–13, thermal effects are considered. Note that a different temperature is determined for each tier in a plane according to the flow diagram shown in Fig.€5.3. For the uppermost plane, the maximum rise in temperature is ΔΤ€=€20.1°C. As reported in Table€5.3, neglecting the rise in temperature, particularly in the upper planes, results in a smaller interconnect pitch which is insufficient to satisfy the timing requirements. Another tier (not shown in Table€5.3) should be used for the network and, therefore, separate timing specifications would apply for this tier. The pitch of this global interconnect tier is not determined by the analysis procedure described in Section€5.4, but a small pitch is selected to constrain the area allocated for the physical links within the network. The power consumption and related temperature rise also depend upon the switching activity of both the network as_noc and the PEs as. The relative magnitude of these two parameters can greatly affect the behavior of a 3-D topology. In these examples, as_noc€=€0.25 and as€=€0.15 have been assumed. These parameters do not affect those traits of the 3-D topologies that improve the performance of a conventional 2-D network but can considerably affect the extent to which each of the 3-D topologies can improve a specific design objective.

5.6â•…Summary 3-D NoC are a natural evolution of 2-D NoC, exhibiting superior performance. Several 3-D NoC topologies are discussed. Models for the zero-load latency and power consumed by a network are presented for these 3-D topologies. Expressions for the power dissipation of the entire system including the PEs are also provided. A methodology that predicts the distribution of the interconnects within a system based on an on-chip network is extended to accommodate the 3-D nature of the investigated topologies. Thermal effects of the interconnect distribution are also considered in this analysis methodology.

112

V. F. Pavlidis and E. G. Friedman

In 3-D NoCs, the minimum latency and power consumption can be achieved by reducing both the number of hops per packet and the length of the communication channels. The topology that best achieves this reduction, however, changes according to the design objective. The network size, speed, and gate count of the PEs, as well as the particular 3-D technology are some important aspects that need to be considered when a 3-D topology is chosen. Selecting a topology that minimizes the power dissipated by an on-chip network does not necessarily guarantee that the power dissipated by the overall system will be minimized. Consequently, the analysis methodology described in this chapter can be a useful tool for exploring early in the design cycle the topological and architectural choices of a 3-D NoC-based system.

References ╇ 1. G. De Micheli and L. Benini, Networks on Chips: Technology and Tools, Morgan Kaufmann, San Francisco, CA, 2006. ╇ 2. A. Jantsch and H. Tenhunen, Networks on Chip, Kluwer Academic, San Francisco, CA, 2003. ╇ 3. M. Millberg et al., “The Nostrum Backbone—A Communication Protocol Stack for Networks on Chip,” Proceedings of the IEEE International Conference on VLSI Design, pp.€693–696, January 2004. ╇ 4. J. M. Duato, S. Yalamanchili, and L. Ni, Interconnection Networks: An Engineering Approach, Morgan Kaufmann, San Francisco, CA, 2003. ╇ 5. D. Park et al., “MIRA: A Multi-Layered On-Chip Interconnect Router Architecture,” Proceedings of the IEEE International Symposium on Computer Architecture, pp.€251–261, June 2008. ╇ 6. C. Addo-Quaye, “Thermal-Aware Mapping and Placement for 3-D NoC Designs,” Proceedings of the IEEE International System-on-Chip Conference, pp.€25–28, September 2005. ╇ 7. F. Li et al., “Design and Management of 3D Chip Multiprocessors Using Network-in-Memory,” Proceedings of the IEEE International Symposium on Computer Architecture, pp.€130– 142, June 2006. ╇ 8. V. F. Pavlidis and E. G. Friedman, “Three-Dimensional (3-D) Topologies for Networks-onChip,” Proceedings of the IEEE International System-on-Chip Conference, pp.€ 285–288, September 2006. ╇ 9. C. Seiculescu, S. Murali, L. Benini, and G. De Micheli, “SunFloor 3D: A tool for Networks on Chip Topology Synthesis for 3D Systems on Chips,” ACM/IEEE Design, Automation and Test in Europe Conference and Exhibition, pp.€9–14, April 2009. 10. W. J. Dally and B. Towles, Principles and Practices of Interconnection Networks, Morgan Kaufmann, San Francisco, CA, 2004. 11. L.-S. Peh and W. J. Dally, “A Delay Model for Router Microarchitectures,” IEEE Micro, Vol.€21, No.€1, pp.€26–34, January/February 2001. 12. T. Sakurai, “Closed-Form Expressions for Interconnection Delay, Coupling, and Crosstalk in VLSI’s,” IEEE Transactions on Electron Devices, Vol.€40, No.€1, pp.€118–124, January 1993. 13. K. A. Bowman et al., “A Physical Alpha-Power Law MOSFET Model,” IEEE Journal of Solid States Circuits, Vol.€34, No.€10, pp.€1410–1414, October 1999. 14. S. L. Garverick and C. G. Sodini, “A Simple Model for Scaled MOS Transistors that Includes Field-Dependent Mobility,” IEEE Journal of Solid States Circuits, Vol.€SC-22, No.€2, pp.€111–114, February 1987. 15. The International Technology Roadmap for Semiconductors Reports, 2009 [Online]. Available: http://www.itrs.net/Links/2008ITRS/Home2008.htm

5â•… Physical Analysis of NoC Topologies for 3-D Integrated Systems

113

16. Predictive Technology Model [Online]. Available: http://www.eas.asu.edu/~ptm 17. W. Zhao and Y. Cao, “New Generation of Predictive Technology Model for Sub-45€nm Design Exploration,” Proceedings of the IEEE International Symposium on Quality Electronic Design, pp.€585–590, March 2006. 18. T. Sakurai and A. R. Newton, “Alpha-Power Law MOSFET Model and Its Applications to CMOS Inverter Delay and Other Formulas,” IEEE Journal of Solid State Circuits, Vol.€25, No.€2, pp.€584–594, April 1990. 19. G. Chen and E. G. Friedman, “Low-Power Repeaters Driving RC and RLC Interconnects with Delay and Bandwidth Constraints,” IEEE Transactions on Very Large Integration (VLSI) Systems, Vol.€12, No.€2, pp.€161–172, February 2006. 20. X. Xi et al., BSIM4.5.0 MOSFET Model User’s Manual, University of California, Berkeley, CA, 2004. 21. H. B. Bakoglu, Circuits, Interconnections, and Packaging for VLSI, Addison-Wesley, Reading, MA,1990. 22. Y. I. Ismail, E. G. Friedman, and J. L. Neves, “Equivalent Elmore Delay for RLC Trees,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, Vol.€19, No.€1, pp.€83–97, January 2000. 23. Y. I. Ismail, E. G. Friedman, and J. L. Neves, “Figures of Merit to Characterize the Importance of On-Chip Inductance,” IEEE Transactions on Very Large Integration (VLSI) Systems, Vol.€7, No.€4, pp.€442–449, December 1999. 24. V. F. Pavlidis, I. Savidis, and E. G. Friedman, “Clock Distribution Networks for 3-D Integrated Circuits,” Proceedings of the IEEE International Conference on Custom Integrated Circuits, pp.€651–654, September 2008. 25. Massachusetts Institute of Technology Lincoln Laboratory, FDSOI Design Guide, Cambridge, 2006. 26. H. Hua et al., “Performance Trends in Three-Dimensional Integrated Circuits,” Proceedings of the International IEEE Interconnect Technology Conference, pp.€45–47, June 2006. 27. K. Banerjee and A. Mehrotra, “A Power-Optimal Repeater Insertion Methodology for Global Interconnects in Nanometer Design,” IEEE Transactions on Electron Devices, Vol.€ 49, No.€11, pp.€2001–2007, November 2002. 28. H. J. M. Veendrick, “Short-Circuit Dissipation of Static CMOS Circuitry and Its Impact on the Design of Buffer Circuits,” IEEE Journal of Solid State Circuits, Vol.€SC-19, No.€4, pp.€468–473, August 1984. 29. K. Nose and T. Sakurai, “Analysis and Future Trend of Short-Circuit Power,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, Vol.€ 19, No.€ 9, pp.€1023–1030, September 2000. 30. G. Chen and E. G. Friedman, “Effective Capacitance of Inductive Interconnects for ShortCircuit Power Analysis,” IEEE Transactions on Circuits and Systems I: Brief Papers, Vol.€55, No.€1, pp.€26–30, January 2008. 31. P. R. O’Brien and T. L. Savarino, “Modeling the Driving-Point Characteristic of Resistive Interconnect for Accurate Delay Estimation,” Proceedings of the International IEEE/ACM Conference on Computer-Aided Design, pp.€512–515, April 1989. 32. H. Wang, L.-S. Peh, and S. Malik, “Power-Driven Design of Router Microarchitectures in On-Chip Networks,” Proceedings of the IEEE International Symposium on Microarchitecture, pp.€105–116, December 2003. 33. V. F. Pavlidis and E. G. Friedman, Three-Dimensional Integrated Circuit Design, Morgan Kaufmann, San Francisco, CA, 2009. 34. V. F. Pavlidis and E. G. Friedman, “Interconnect-Based Design Methodologies for ThreeDimensional Integrated Circuits,” Proceedings of the IEEE, Special Issue on 3-D Integration Technology, Vol.€97, No.€1, pp.€123–140, January 2009. 35. J. W. Joyner and J. D. Meindl, “Opportunities for Reduced Power Distribution Using ThreeDimensional Integration,” Proceedings of the IEEE International Interconnect Technology Conference, pp.€148–150, June 2002.

114

V. F. Pavlidis and E. G. Friedman

36. J. W. Joyner et al., “Impact of Three-Dimensional Architectures on Interconnects in Gigascale Integration,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol.€9, No.€6, pp.€922–927, December 2000. 37. R. Venkatesan, J. A. Davis, K. A. Bowman, and J. D. Meindl, “Optimal n-tier Multilevel Interconnect Architectures for Gigascale Integration (GSI),” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol.€9, No.€6, pp.€899–912, December 2001. 38. T.-Y. Chiang, S. J. Souri, C. O. Chui, and K. C. Saraswat, “Thermal Analysis of Heterogeneous 3D ICs with Various Integration Scenarios,” Proceedings of the IEEE International Electron Device Meeting, pp.€681–684, December 2001. 39. T.-Y. Chiang, K. Banerjee, and K. C. Saraswat, “Analytical Thermal Model for Multilevel VLSI Interconnects Incorporating Via Effect,” IEEE Electron Device Letters, Vol.€23, No.€1, pp.€31–33, January 2002. 40. C. Marcon et al., “Exploring NoC Mapping Strategies: An Energy and Timing Aware Technique,” Proceedings of the ACM/IEEE Design, Automation and Test in Europe Conference and Exhibition, Vol.€1, pp.€502–507, March 2005. 41. P. P. Pande et al., “Performance Evaluation and Design Trade-Offs for Network-on-Chip Interconnect Architectures,” IEEE Transactions on Computers, Vol.€54, No.€8, pp.€1025–1039, August 2005. 42. X. Zhao, D. L. Lewis, H.-S. H. Lee, and S. K. Lim, “Pre-Bond Testable Low-Power Clock Tree Design for 3D Stacked ICs,” Proceedings of the IEEE/ACM International Conference on Computer Aided Design, pp.€184–190, November 2009. 43. V. F. Pavlidis and E. G. Friedman, “3-D Topologies for Networks-on-Chip,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol.€15, No.€10, pp.€1081–1090, October 2007. 44. A. H. Ajami, K. Banerjee, and M. Pedram, “Modeling and Analysis of Nonuniform Substrate Temperature Effects on Global ULSI Interconnects,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, Vol.€24, No.€6, pp.€849–861, June 2005. 45. J. C. Ku and Y. Ismail, “Thermal-Aware Methodology for Repeater Insertion in Low-Power VLSI Circuit,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol.€15, No.€8, pp.€963–970, August 2007.

Chapter 6

Three-Dimensional Networks-on-Chip: Performance Evaluation Brett Stanley Feero and Partha Pratim Pande

6.1â•…Introduction The current trend in System-on-Chip (SoC) design in the ultra deep sub-micron (UDSM) regime and beyond is to integrate a huge number of functional and storage blocks in a single die [1]. The possibility of this enormous degree of integration gives rise to new challenges in designing the interconnection infrastructure for these big SoCs. Extrapolating from the existing CMOS scaling trends, traditional on-chip interconnect systems have been projected to be limited in their ability to meet the performance needs of SoCs at the UDSM technology nodes and beyond [2]. This limit stems primarily from global interconnect delay significantly exceeding that of gate delays. While copper and low-k dielectrics have been introduced to decrease the global interconnect delay, they only extend the lifetime of conventional interconnect systems a few technology generations. According to the International Technology Roadmap for Semiconductors (ITRS) [2], for the longer term, material innovation with traditional scaling will no longer satisfy the performance requirements. New interconnect paradigms are in need. Continued progress of interconnect performance will require employing approaches that introduce materials and structures beyond the conventional metal/dielectric system, and one of the promising approaches is 3D integration. Shown in Fig.€6.1, three-dimensional (3D) ICs, which contain multiple layers of active devices, have the potential for enhancing system performance [3–6]. According to [3], three-dimensional ICs allow for performance enhancements even in the absence of scaling. A clear way to reduce the burden of high frequency signal propagation across monolithic ICs is to reduce the line length needed, and this can be done by employing stacking of active devices using 3D interconnects. Here, the multiple layers of active

B. S. Feero () ARM Inc., 3711 S. Mopac Expy. Austin, TX 78731, USA e-mail: [email protected] P. P. Pande () School of Electrical Engineering and Computer Science, Washington State University, PO BOX-642752, Pullman, WA 99164-2752, USA e-mail: [email protected] A. Sheibanyrad et al. (eds.), 3D Integration for NoC-based SoC Architectures, Integrated Circuits and Systems, DOI 10.1007/978-1-4419-7618-5_6, ©Â€Springer Science+Business Media, LLC 2011

115

116

B. S. Feero and P. P. Pande

Fig. 6.1↜渀 3DIC from a SOI process Second Device Layer Vertical Via

p+

n+ First Device Layer

Silicon

devices are separated by a few tens of micrometers. Consequently, 3D interconnects allow communication among these active devices with smaller distances required for signal propagation. Three-dimensional ICs will have a significant impact on the design of multi-core SoCs. Recently, Network-on-Chip (NoC) has emerged as an effective methodology for designing big multi-core SoCs [7, 8]. However, the conventional two dimensional (2D) IC has limited floor-planning choices and, consequently, limits the performance enhancements arising out of NoC architectures. The performance improvement arising from the architectural advantages of NoCs will be significantly enhanced if 3D ICs are adopted as the basic fabrication methodology. The amalgamation of two emerging paradigms, namely NoCs in a 3D IC environment, allows for the creation of new structures that enable significant performance enhancements over more traditional solutions. With freedom in the third dimension, on-chip network architectures that were impossible or prohibitive due to wiring constraints in planar ICs are now possible [9, 10]. However, 3D ICs are not without limitations. Thermal effects are already impacting interconnect and device reliability in 2D circuits [11]. Due to the reduction of chip size in a 3D implementation, 3D integrated circuits exhibit a profound increase in power density. Consequently, increases in heat dissipation will give rise to circuit degradation and chip cracking, among other side-effects [12]. As a result, there is a real need to keep the temperature low for reliable circuit operation. Furthermore, in ICs implementing NoCs, the interconnect structure dissipates a large percentage of energy. In certain applications [13], this percentage has been shown to approach 50%. As a result, the interconnection network has a significant contribution to the thermal performance of 3D NoCs. This chapter introduces multiple NoC architectures that are enabled through 3D integration. The following sections characterize the performance of these 3D NoC architectures in terms of five metrics: throughput, latency, energy dissipation, silicon area, and thermal profile. Through the application of realistic traffic patterns in cycle-accurate simulation, this chapter demonstrates that three-dimensional integration can facilitate topology choices that dramatically outperform two-dimensional topologies in terms of throughput, latency, energy, and area, and it demonstrates that these improvements begin to mitigate some of the thermal concerns presented by 3D ICs.

6â•… Three-Dimensional Networks-on-Chip: Performance Evaluation

117

6.2â•…3D NoC Architectures Enabling design in the vertical dimension permits a large degree of freedom in choosing an on-chip network topology. Due to wire-length constraints and layout complications, the more conventional two-dimensional integrated circuits have placed limitations on the types of network structures that are possible. With the advent of 3D ICs, a wide range of on-chip network structures that were not explored earlier are being considered [9, 10]. This chapter investigates five different topologies in 3D space and compares them with three well-known NoC architectures from 2D implementations. This analysis considers a SoC with a 400€mm2 floor plan and 64 functional IP blocks. This system size was selected to reflect state of the art of emerging SoCs. At ISSCC 2007, design of an 80-core processor arranged in an 8×10 regular grid built on fundamental NoC concepts was demonstrated [14]. Moreover, Tilera Corp. has recently announced design of a multi-core platform with 100 cores [15]. Therefore, the system size assumed in this work is representative of the latest trends. IP blocks for a 3D SoC are mapped onto four 10â•›×â•›10€mm layers, in order to occupy the same total area as a single-layer, 20â•›×â•›20€mm layout.

6.2.1  Mesh-Based Networks One of the well-known 2D NoC architectures is the 2D Mesh as shown in Fig.€6.2a. This architecture consists of an mâ•›×â•›n mesh of switches interconnecting IP blocks placed along with them. It is known for its regular structure and short inter-switch wires. From this structure, a variety of three-dimensional topologies can be derived. The straightforward extension of this popular planar structure is the 3D Mesh. Figure€6.2b shows an example of 3D Mesh NoC. It employs 7-port switches: one port to the IP block, one each to switches above and below, and one in each cardinal direction (North, South, East, and West), as shown in Fig.€6.3a. A second derivation, 3D Stacked Mesh (Fig.€6.2c), takes advantage of the short inter-layer distances that are characteristics of a 3D IC, which can be around 20€μm [3]. The 3D Stacked Mesh architecture is a hybrid between a packet-switched network and a bus. It integrates multiple layers of 2D Mesh networks by connecting them with a bus spanning the entire vertical distance of the chip. As the distance between the individual 2D layers in 3D IC is extremely small, the overall length of the bus is also small, making it a suitable choice for communicating in the z-dimension [9]. Furthermore, each bus has only a small number of nodes (i.e. equal to the number of layers of silicon), keeping the overall capacitance on the bus small and greatly simplifying bus arbitration. For consistency with [9], this analysis considers the use of a dynamic, time-division multiple-access (dTDMA) bus, although any other type of bus may be used as well. A switch in a 3D Stacked Mesh network has, at most, 6 ports: one to the IP, one to the bus, and four for the cardinal directions (Fig.€6.3b). Additionally,

118

B. S. Feero and P. P. Pande

l2DIC

2.5 mm

a

c

20 µm

l2DIC 2

= l3DIC

b

d y

x

z x

y Coordinate Axes

Interconnect

IP Block

Bus

Switch

Bus Node

e

Fig. 6.2↜渀 Mesh-based NoC architectures: a 2D Mesh, b 3D Mesh, c Stacked Mesh, and d Ciliated 3D Mesh

IP

IP

N IP W

Switch

N

IP

W

E

Switch

E

W

E

S

S

a

Switch

N

b

S

bus

c

Fig. 6.3↜渀 Switches for mesh-based NoCs: a 3D Mesh, b Stacked Mesh, and c Ciliated 3D Mesh

6â•… Three-Dimensional Networks-on-Chip: Performance Evaluation

119

it is possible to utilize ultra wide buses similar to the approach introduced in [16] to implement cost-effective, high-bandwidth communication between layers. A third method of constructing a 3D NoC is by adding layers of functional IP blocks and restricting the switches to one layer or a small number of layers, such as in the 3D Ciliated Mesh structure. This structure is essentially a 3D Mesh network with multiple IP blocks per switch. The 3D Ciliated Mesh is a 4â•›×â•›4â•›×â•›2 3D mesh-based network with 2 IPs per switch, where the two functional IP blocks occupy, more or less, the same footprint but reside at different layers. This is shown in Fig.€6.2d. In a Ciliated 3D Mesh network, each switch contains seven ports (one for each cardinal direction, one either up or down, and one to each of two IP blocks) as shown in Fig.€6.3c. This architecture will clearly exhibit lower overall bandwidth than a complete 3D Mesh due to multiple IP blocks per switch and reduced connectivity; however, Sect.€6.4 will show that this type of network offers an advantage in terms of energy dissipation, especially in the presence of specific traffic patterns.

a

b

l2DIC 20 µm

l2DIC 2 l2DIC 2

c

= l3DIC

d 40 µm

Interconnect

IP Block

Switch

l3DIC

l3DIC/4

e

Fig. 6.4↜渀 Tree architectures: a Butterfly Fat Tree, b SPIN, c 2D BFT Floorplan, d 3D BFT Floorplan for the first two layers, and e the first three layers of a 3D BFT Floorplan as seen in elevation view

120

B. S. Feero and P. P. Pande

To Parent Nodes

To Parent Nodes

Switch

Switch

To Child Nodes

a

To Child Nodes

b

Fig. 6.5↜渀 Switches for tree networks: a BFT and b SPIN

6.2.2  Tree-Based Networks Two types of tree-based interconnection networks that have been considered for network-on-chip applications are Butterfly Fat Tree (BFT) [17, 18] and the generic Fat Tree, or SPIN [19]. Unlike the work with mesh-based NoCs, this chapter does not propose any new topologies for tree-based systems. Instead, it investigates the achievable performance benefits by instantiating already-existing tree-based NoC topologies in a 3D environment. The BFT topology considered is shown in Fig.€6.4a. For a 64-IP SoC, a BFT network will contain 28 switches. Each switch (Fig.€6.5a) in a Butterfly Fat Tree network consists of 6 ports, one to each of four child nodes and two to parent nodes, with the exception of the switches at the topmost layer. When mapped to a 2D structure the longest inter-switch wire length for a BFT-based NoC is l2DIC/2, where l2DIC is the die length on one side [18, 20]. If the NoC is spread over a 20â•›×â•›20€mm die, then the longest inter-switch wire is 10€mm [20], as shown in Fig.€6.4c. Yet, when the same BFT network is mapped onto a four-layer 3D SoC, wire routing becomes simpler, and the longest inter-switch wire length is reduced by at least a factor of two, as can be seen in Fig.€6.4d. This will lead to reduced energy dissipation as well as less area overhead. The fat tree topology of Fig.€6.4b will have the same advantages when mapped on to a 3D IC as the BFT.

6.3â•…Performance Evaluation 6.3.1  Performance Metrics In order to properly analyze the various 3D network-on-chip topologies, a standard set of metrics must be used [21]. Wormhole routing [22] is assumed as the data

6â•… Three-Dimensional Networks-on-Chip: Performance Evaluation

121

transport mechanism where the packet is divided into fixed length flow control units or flits. The header flit holds the routing and control information. It establishes a path, and subsequent payload or body flits follow that path. This comparative analysis focuses on the four established benchmarks [21] of throughput, latency, energy, and area overhead. Throughput is a metric that quantifies the rate in which message traffic can be sent across a communication fabric. It is defined as the average number of flits arriving per IP block per clock cycle, so the maximum throughput of a system is directly related to the peak data rate that a system can sustain. For purposes of a message-passing system, throughput T is given by the equation 

T =

(Total Messages Completed) × (Message Length) . (Number of IP Blocks) × (Time)

(6.1)

Total Messages Completed are the number of messages which successfully traverse the network from source to destination. Message Length refers to the number of flits a message consists of, and Number of IP Blocks signifies the number of intellectual property units that send data over the network. Time is length of time in clock cycles between the generation of the first packet and the reception of the last. It can be seen that throughput is measured in flits/IP block/cycle, where a throughput of 1 signifies that every IP block is accepting a flit in each clock cycle. Accordingly, throughput is a measure of the maximum amount of sustainable traffic. Throughput will be dependent on a number of parameters including the number of links in the architecture, the average hop count, the number of ports per switch, and injection load. Injection load is measured by the number of flits injected in to the network per IP block per cycle. Consequently, it has the same unit as the throughput, and an injection load of 1 signifies that every IP block is injecting a flit in each clock cycle. Next, latency refers to the length of time elapsed between the injection of a message header at the source node and the reception of the tail flit at the destination. Latency is defined as the time in clock cycles elapsed from the transfer of the header flit by the source IP to the acceptance of the tail flit by the destination IP block. Latency is characterized by three delays: sender overhead, transport latency, and receiver overhead. 

Li = Lsender + Ltransport + Lreceiver

(6.2)

Flits must traverse a network while traveling from source to destination. With different routing algorithms and switch architectures, each packet will experience a unique latency. As a result, network topologies will be compared by average latency. Let P be the number of packets received in a given time period, and let Li be the latency of the ith packet. Average latency is therefore given by the equation: 

Lavg =

P 

i=1

Li

P

.

(6.3)

122

B. S. Feero and P. P. Pande

Additionally, the transport of messages across a network leads to a quantifiable amount of energy dissipation. Activity in the logic gates of the network switches as well as the charging and discharging of interconnection wires lead to the consumption of energy. The analysis in this chapter examines two types of energy: cycle energy and packet energy. Cycle energy is defined as the amount (in Joules) of energy dissipated by the entire network in one clock cycle. On the other hand, packet energy is defined as the amount of energy incurred by a single packet as it traverses the network from source to destination over many clock cycles. It will be shown that each of these types of energy reveals unique information about the behavior of the varying network architectures. Lastly, the amount of silicon area used by an interconnection network is a necessary consideration. As the network switches form an integral part of the infrastructure, it is important to determine the amount of relative silicon area they consume. Additionally, area overhead arising from layer-to-layer vias, inter-switch wires, and buffers incurred by relatively longer wires need to be considered. The evaluation of area in this chapter includes each form of area overhead.

6.3.2  Performance Analysis of 3D Mesh-Based NoCs Here, the performance of the 3D mesh-based NoC architectures is analyzed in terms of the parameters mentioned above: throughput, latency, energy dissipation, and area overhead. Throughput is given in the number of accepted flits per IP per cycle. This metric, therefore, is closely related to the maximum amount of sustainable traffic in a certain network type. Any throughput improvements in 3D networks are principally related to two factors: the number of physical links and the average number of hops. In general, for a mesh-based NoC, the number of links is given as follows: 

links = N1 N2 (N3 − 1) + N1 N3 (N2 − 1) + N2 N3 (N1 − 1) ,

(6.4)

where Ni represents the number of switches in the ith dimension. For instance, in an 8â•›×â•›8 2D Mesh NoC, this yields 112 links. In a 4â•›×â•›4â•›×â•›4 3D Mesh NoC, the number of links is 144. With a greater number of links, a 3D Mesh network, for example, is able to contain a greater number of flits and therefore transmit a greater number of messages. However, only considering the number of links will not characterize the overall throughput of a network. The average hop count also has a definitive effect on throughput. A lower average hop count will also allow more flits to be transmitted through the network. With a lower hop count, a wormhole-routed packet will utilize fewer links, thus leaving more room to increase the maximum sustainable traffic. It is important to note that hop count is also very application-dependent. For instance, if a particular application produces more localized traffic, where the majority of traffic is between source and destination nodes that are spatially close, average hop

6â•… Three-Dimensional Networks-on-Chip: Performance Evaluation

123

count will be reduced. It is easier to first approach average hop count by considering uniform spatial traffic distribution. The case of localized traffic is discussed in detail in Sect.€6.3.7. Following [10], assuming a uniform spatial traffic distribution, the average number of hops in a mesh-based NoC is given by 

hopsMesh =

n1 n2 n3 (n1 + n2 + n3 ) − n3 (n1 + n2 ) − n1 n2 , 3 (n1 n2 n3 − 1)

(6.5)

where ni is the number of nodes in the ith dimension. This equation applies both to the 4â•›×â•›4â•›×â•›4 3D Mesh and 4â•›×â•›4â•›×â•›2 3D Ciliated Mesh networks. The number of hops for the 3D Stacked Mesh is equal to 

hopsStacked =

n1 + n2 n3 − 1 . + 3 n3

(6.6)

For the 4â•›×â•›4â•›×â•›4 3D Mesh and 8â•›×â•›8 2D Mesh, average hop counts are 3.81 and 5.33, respectively. There are 40% more hops in the 2D Mesh compared to that in 3D Mesh. Consequently, flits in the 3D Mesh traverse fewer stages between source and destination than in the 2D counterpart. As a result of this, a corresponding increase in throughput is expected. Transport latency, like throughput, is also affected by average hop count. The number of links and the injection load also affects it heavily. In 3D architectures, a decrease in latency is expected due to a lower hop count and an increased number of links. In the System-on-Chip realm, energy dissipation characteristics of the interconnect structures are crucial, as the interconnect fabric can consume a significant portion of the overall energy budget [13]. The energy dissipation in a NoC depends on the energy dissipated by the switch blocks and the inter-switch wire segments. Both of these factors depend on the network architecture. Additionally, the injection load has a significant contribution, as it is the cause for any activity in the switches and inter-switch wires. Intuitively, it is clear that with more packets traversing the network, power will increase. This is why packet energy is an important attribute for characterizing NoC structures. The energy dissipated per flit per hop is given by 

Ehop = Eswitch + Ewire ,

(6.7)

where Eswitch and Ewire are the energy dissipated by each switch and inter-switch wire segments respectively. The energy of a packet of length n flits that completes h hops is given by 

Epacket = n

h 

Ehop, j .

(6.8)

j=1

From this, a formula for packet energy can be realized. If P packets are transmitted then the average energy dissipated per packet is given as

124



B. S. Feero and P. P. Pande

Epacket =

P 

i=1

Epacket,i P

=

P 

i=1



ni

hi 

j=1

P

Ehop, j

 .

(6.9)

Now, it is clear that a strong correlation exists between packet energy and the number of hops from source to destination. Consequently, a network topology that exhibits smaller hop counts will also exhibit correspondingly lower packet energy. As all 3D mesh-based NoC architectures exhibit a lower hop count they should also dissipate less energy per packet. Lastly, the area overhead for mesh-based NoCs must be established. Area overhead for a NoC includes switch overhead and wiring overhead. Switch area is affected by the overall number of switches and the area per switch, which is highly correlated to the number of ports. Since all 3D mesh-based NoCs have more ports, the area per switch will increase. However, wire overhead is reduced when moving to a 3DIC. This is not due to reductions in the length of most inter-switch wires in the case of mesh-based NoCs. Horizontal wire length is given by lIC/nside, where nside represents the number of IPs in one dimension of the IC and lIC is the die length on one side as shown earlier in Fig.€6.2a, b. For the 8â•›×â•›8 2D Mesh, this evaluates to 20€mm/8 or 2.5€mm, and for all 3D mesh-based architectures, the expression evaluates to 10€mm/4, also 2.5€mm. With this in mind, reductions in wire overhead come from the interlayer wires. The 3D structures have a reduced number of horizontal links due to the presence of interlayer wires. These interlayer wires are very small and hence, they are the source of wire overhead savings in mesh-based 3D NoCs. These effects are quantified in Sect.€6.4.4.

6.3.3  Performance Analysis of 3D Tree-Based NoCs Unlike the previous discussion pertaining to mesh-based NoCs, the tree-based networks considered for 3D implementations have identical topologies to their 2D counterparts. The only variable is the inter-switch wire length. As a result, there are significant improvements both in terms of energy and area overhead. In 2D space, the longest inter-switch wire length in a BFT or SPIN network is equal to l2DIC/2 [18, 20], where l2DIC is the die length on one side. This inter-switch wire length corresponds to the top-most level of the tree. In a 3D IC, however, this changes significantly. For instance, as shown in Fig.€6.4d, e, the longest wire length for 3D, tree-based NoC is equal to the length of horizontal travel in addition to the length of the vertical via. Considering a 20â•›×â•›20€mm 2D die, the longest inter-switch wire length is equal to 10€mm, whereas with a 10â•›×â•›10€mm stack of four layers, the maximum wire length is equal to the sum of l3DIC/4, or 2.5€mm, and the span of two layers, 40€μm. This is almost a factor-of-4 reduction compared to 2D implementations. Similarly, mid-level wire lengths are reduced by a factor of 2. As a result, this reduction in wire length, shown in Table€6.1, causes a significant reduction in energy.

6╅ Three-Dimensional Networks-on-Chip: Performance Evaluation Table 6.1↜渀 Inter-switch wire lengths in 3D tree-based NoCs

â•– 1st Level 2nd Level 3rd Level

125

2D NoC

4-layer 3D NoC

≤â•›l/8€= 2.5€mm l/4€=€5€mm l/2€=€10€mm

≤â•›l/4€= 2.5€mm l/4€= 2.5€mm l/4€= 2.5€mm

In addition to benefits in terms of energy, 3D ICs effect area improvements for tree-based NoCs. Again, as with energy, area gains pertain only to the inter-switch wire segments; there is neither a change in the number of switches nor in the design of the switch. As with the 3D mesh-based NoCs, wire overhead in a 3D tree-based NoC consists of the horizontal wiring in addition to the area incurred by the vertical wires and vias. Also, the longer inter-switch wires, which are characteristics of 2D treebased NoCs, require repeaters, and this is taken into account. For a Butterfly Fat Tree, the number of wires in an arbitrary tree level l as defined in [17] is 

wireslayerl = wlink

 N · l−1 , 2 

(6.10)

where N is the number of IP blocks and wlink is the link width in bits. For a generic Fat Tree, the number of wires in a tree level l is given by 

wireslayerl = wlink · N .

(6.11)

For instance, in a 64-IP BFT network with 32-bit wide bi-directional interswitch links, there are 2,048 wires in the first level, 1,024 wires in the second level, and 512 wires in the third. Similarly, a 64-IP Fat Tree will have 2,048 wires in every level.

6.3.4  Simulation Methodology To model performance of different NoC structures, a cycle-accurate network simulator is employed that can also simulate dTDMA buses. The simulator is flit-driven, uses wormhole routing, and assumes a self-similar injection process [21–24]. This type of traffic has been observed MPEG-2 video applications [25], as well as various other networking applications [24]. It has been shown to closely model real traffic [25]. In terms of spatial distribution the simulator is capable of producing both uniform and localized traffic patterns for injected packets. In order to acquire energy and area characteristics, the network switches, dTDMA arbiter, and FIFO buffers were modeled in VHDL. The network switches were designed in such a way that their delay can be constrained within the limit of one clock cycle. The clock cycle is assumed to be equal to 15FO4 (fan-out-of 4) delay units. With a 90€nm standard cell library from CMP [26], this corresponds to a clock frequency of 1.67€GHz. As the switches were designed with differing numbers of ports, their delays vary with one another. However, it was important to ensure that all the delay numbers were

126

B. S. Feero and P. P. Pande

Table 6.2↜渀 Wire delays Wire type Interlayer Vertical bus Horizontal Horizontal╛+╛Interlayer Horizontal Horizontal a b

Wire length 20€µm 60€µm 2.5€mm 2.54€mm 5€mm 10€mm

Delay (ps) 16 110/450a 219 231 436b 550b

Architectures used All 3D mesh-based 3D Stacked Mesh Mesh-based, 2D tree-based All 3D tree-based Mid-level in all 2D tree-based Top-level in all 2D tree-based

Bus arbitration included Repeaters necessary

kept within the 15FO4 timing constraint. Consistent with [20], the longest delays were in the 2D/3D Fat Tree switches as they had the highest number of ports. Yet, even it can be run with a clock frequency of 11FO4, well within the 15FO4 limit. To provide a consistent comparison, all the switches were run with a 15FO4 clock. Similarly, all inter-switch wire delays must hold within the same constraints. As shown in Table€6.2, wire RC delays remain within the clock period of 600€ps [26]. For Stacked Mesh, even considering the bus arbitration, the delay is constrained within one clock cycle. For the vertical interconnects, the via resistance and capacitance are included in the analysis. As such, all network architectures are able to run at the same clock frequency of 1.67€GHz. Additional architectural parameters for each topology are shown in Table€6.3. Each switch was designed with 4 virtual channels per port and 2-flit-deep virtual channel buffers as discussed in [21]. Synopsys Design Vision was used to synthesize the hardware description, and Synopsys PrimePower was used to gather energy dissipation statistics. To calculate Eswitch and Ewire from (6.7), the methodology discussed in [21] is followed. The energy dissipated by each switch, Eswitch, is determined by running its gate-level netlist through Synopsys PrimePower using large sets of input data patterns. In order to determine the interconnect energy, Einterconnect, the interconnects’ capacitance is estimated, taking into account each inter-switch wire’s specific layout, by the following expression [21]: 

Cinterconnect = Cwire · Wa+1;a + n · m · (CG + CJ ) ,

Table 6.3↜渀 Architectural parameters Topology Port count 2D Mesh 3D Mesh 3D Stacked Mesh Ciliated 3D Mesh 2D BFT 3D BFT 2D Fat tree 3D Fat tree

5 7 6 (+â•›bus arbitration) 7 6 6 8 8

Switch area (mm2) 0.0924 0.1385 0.1225 0.1346 0.1155 0.1155 0.1616 0.1616

Switch static energy (pJ) ╇ 65.3 ╇ 91.4 ╇ 81.3 ╇ 91.2 ╇ 78.3 ╇ 78.3 104.5 104.5

(6.12)

Longest wire delay (ps) 219 219 219 219 550 231 550 231

6â•… Three-Dimensional Networks-on-Chip: Performance Evaluation

127

where Cwire represents the capacitance per unit length of the wire, waâ•›+â•›1;â•›a is the wire length between two consecutive switches, n is the number of repeaters, m represents the size of those repeaters with respect to minimum-size devices, and lastly, CG and CJ represent the gate and junction capacitance, respectively, of a minimum size inverter. While determining Cwire, the worst-case scenario is considered, where adjacent wires switch in opposite directions [27]. The simulation was initially run for 10,000 cycles to allow the 64-IP network to stabilize, and it was subsequently run for 100,000 more cycles, reporting statistics for energy, throughput, and latency.

6.3.5  Experimental Results for Mesh-Based Networks This chapter first considers the performance of 3D mesh-based NoC architectures. Figure€6.6a shows the variation of throughput as a function of the injection load. A network cannot accept more traffic than is supplied, and limitations in routing and collisions cause saturation before throughput reaches unity. From Fig.€ 6.6a, it is clear that both the 3D Mesh and Stacked Mesh topologies exhibit throughput improvements over their two-dimensional counterparts. It is also clear that the ciliated 3D Mesh network shows only a small throughput improvement. However, this is not where a ciliated structure exhibits the best performance. It will be shown later that this network topology has significant benefits both in terms of energy dissipation and silicon area. These results coincide with the analysis of a 3D mesh-based NoC provided earlier. Equation (6.4) shows that a 3D mesh will have 29% more interconnection links than a 2D version; hop count calculations have shown that a flit in a 2D mesh network will traverse 40% more hops than a flit navigating a 3D mesh (see Table€6.4); and 3D mesh switches have higher connectivity with the increased number of ports. These all account for throughput improvements. In general, the lower hop count allows a packet to occupy fewer resources, freeing up links for additional packets. Consequently, there is a corresponding increase in throughput. Next, the 3D Stacked Mesh architecture is considered. An increase in throughput is evident, as shown in Fig.€6.6a. However, with a 32-bit bus (corresponding to the flit width) connecting the layers of the NoC, throughput improvements are not as substantial as with the 3D Mesh. Contention issues in the bus limit the attainable performance gains. Yet, since communication between layers is bus-based, the bus width is easily increased to 128 bits without modifying the switch architectures in order to limit contention. Any further increases do not have any significant impact on throughput, except to increase the capacitance on the bus. With this improvement, 3D Stacked Mesh saturates at a slightly higher injection load than a 3D Mesh network. The 3D Stacked Mesh topology also offers a lower hop count in comparison to a strict 3D Mesh. From (Eq. 6.6), the average hop count is equal to 3.42. With the lower hop count in addition to the wide, 128-bit bus for vertical transmission, this architecture offers the highest throughput among all the 3D mesh-based networks.

5000 4500 4000 3500 3000 2500 2000 1500 1000 500 0

Throughput

Cycle Energy (pJ)

0

0.9

Injection Load

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

2D MESH 3D MESH Ciliated 3D Mesh Stacked Mesh

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Injection Load

1

1

b

d

10000 9000 8000 7000 6000 5000 4000 3000 2000 1000 0

1000 900 800 700 600 500 400 300 200 100 0 0

2D MESH

9139.84

6111.85

Ciliated 3D Stacked Mesh Mesh

4867.94

Topology

3D MESH

5264.09

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Injection Load

Stacked Mesh (32-bit)

Stacked Mesh (128-bit)

Ciliated 3D Mesh

3D MESH

2D MESH

1

Fig. 6.6↜渀 Experimental results for mesh-based NoCs: a Throughput vs. injection load, b Latency vs. injection load, c Cycle energy vs. injection load, and d Packet energy

c

0

2D MESH 3D MESH Ciliated 3D Mesh Stacked Mesh (128-bit) Stacked Mesh (32-bit)

Latency (clock cycles) Packet Energy (pJ)

a

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

128 B. S. Feero and P. P. Pande

6╅ Three-Dimensional Networks-on-Chip: Performance Evaluation Table 6.4↜渀 Average hop count in mesh-based NoCs

2D Mesh 3D Mesh Stacked Mesh Ciliated 3D Mesh

129 5.33 3.81 3.42 3.10

Throughput characteristics of the ciliated 3D Mesh topology differ significantly from the other 3D networks. This network has a saturating throughput that is slightly higher than a 2D Mesh network and considerably less than both 3D Mesh and Stacked Mesh networks. This is true despite having the lowest hop count at an average of 3.10 hops. However, with only 64 inter-switch links, compared to 144 in the 3D Mesh and 112 in the 2D Mesh, throughput improvements due to hop count are negated by the reduced number of links. The fact that there are multiple functional IP blocks for every switch is also responsible for considerable lower throughput due to contention issues in the switches. Figure€6.6b depicts the latencies for the architectures under consideration. Here, it is seen that 3D mesh-based NoCs have superior latency characteristics over the 2D versions. This is a product of the reduced hop count characteristic of 3D meshbased topologies. Energy dissipation characteristics for three-dimensional mesh-based NoCs reveal a substantial improvement over planar NoCs. The energy dissipation profiles of the mesh-based NoC architectures under consideration are shown in Fig.€6.6c. Energy dissipation is largely dependent on two factors: architecture and injection load. These two parameters are considered as independent factors in this analysis. As shown in (6.7), the energy dissipation in a NoC depends on the energy dissipated by the switch blocks and the inter-switch wire segments. Both these factors depend on the architectures. The design of the switch varies with the architecture and interswitch wire length is also architecture dependent [21]. Besides the network architecture, injection load has a clear effect on the total energy dissipation of a NoC, in accordance with Fig.€6.6c. Intuitively, it is clear that with more packets traversing the network, power will increase. This is why packet energy, in Fig.€6.6d, is an important attribute for characterizing NoC structures. Notice that, at saturation, a 2D Mesh network dissipates less power than both 3D Stacked Mesh and 3D Mesh networks. This is the result of the lower 2D Mesh throughput, and the 3D networks consume more energy because they transmit more flits at saturation. Packet energy is a more accurate representation of the cost of data transmission. With packet energy in mind, it can be seen that every 3D topology provides a very substantial improvement over a 2D Mesh. Also, the energy dissipation of the ciliated mesh topology is less, still, than that of 3D Mesh network. These results follow closely the hop count calculations summarized in Table€6.4, with the exception of the packet energy for a 3D Stacked Mesh network. Energy is heavily dependent on interconnect energy, and this is where the 3D Stacked Mesh suffers. Since vertical communication takes place through wide busses, the capacitive loading on those busses results in a significant amount of energy. As a result, though 3D Stacked

130

B. S. Feero and P. P. Pande

Fig. 6.7↜渀 Area overhead for mesh-based NoCs

8

Switch Wiring

[%] of SoC Area

7 6 5 4 3 2 1 0 2D Mesh

3D Mesh

Stacked Mesh

Ciliated 3D Mesh

Topology

Mesh has a lower hop count compared to 3D Mesh, it dissipates more packet energy on average. Regardless, the profound energy savings possible in these 3D architectures provides serious motivation for a SoC designer to consider a three-dimensional integrated circuit. The final performance metric considered in this study is the overall area overhead incurred with the instantiation of the various networks in the 3D environment. Figure€6.7 shows the area penalty from each NoC design, both in terms of switch area and interconnects area. It shows that while the 3D Mesh and 3D Stacked Mesh NoCs reduce the amount of wiring area, switch overhead is increased. For both 3D Mesh and 3D Stacked Mesh NoCs, the number of longer inter-switch links in x-y plane is reduced. There are 96 x-y links for both topologies, for 3D Stacked Mesh, 16 buses are present, and for the 3D Mesh, 48 vertical links are present. In comparison, the conventional 2D mesh-based NoC has 112 links in the horizontal plane. As the 3D NoCs have fewer long horizontal links they incur less wiring area overhead. Although there are a large number of vertical links, the amount of area incurred by them is very small due to the 2â•›×â•›2€μm interlayer vias. However, an increased number of ports per switch results in larger switch overhead for both of these NoC architectures, ultimately causing the 3D Mesh and 3D Stacked Mesh topologies to incur more silicon area in spite of wiring improvements. On the other hand, 3D Ciliated Mesh shows a significant improvement in terms of area. The 4â•›×â•›4â•›×â•›2 3D Ciliated Mesh structure involves half the number of switches as the other meshbased architectures in addition to only 64 links. As a result, the area overhead is accordingly smaller.

6.3.6  Experimental Results for Tree-Based Networks Here, the performance of the three-dimensional tree-based NoCs is evaluated. It has already been established that 2D and 3D versions of the tree topologies should have identical throughput and latency characteristics, and Fig.€6.8a, b support this.

a

0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Injection Load

3D Fat Tree 3D BFT 2D Fat Tree 2D BFT

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Injection Load

3D Fat Tree 3D BFT 2D Fat Tree 2D BFT

1

1

d

0

2000

4000

6000

8000

10000

12000

14000

16000

b

1000 900 800 700 600 500 400 300 200 100 0

6598.97

3D BFT 2D Fat Tree Topology

6604.84

12962.67

2D BFT

12978.42

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Injection Load

3D Fat Tree

0

3D Fat Tree 3D BFT 2D Fat Tree 2D BFT

1

Fig. 6.8↜渀 Experimental results for tree-based NoCs: a Throughput vs. injection load, b Latency vs. injection load, c Cycle energy vs. injection load, and d Packet energy

c

0

2000

4000

6000

8000

10000

12000

Throughput

Cycle Energy (pJ)

Latency (clock cycles) Packet Energy (pJ)

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

6â•… Three-Dimensional Networks-on-Chip: Performance Evaluation 131

132

B. S. Feero and P. P. Pande

Consistent with the analysis of mesh-based NoCs, Fig.€6.8a shows the variation of throughput as a function of injection load, and Fig.€6.8b shows the effect of injection load on latency. The assumption here was that the switches and the inter-switch wire segments are driven by the same clock as explained earlier. Consequently under this assumption, in terms of throughput and latency there is no advantage to choosing a 3D IC over a traditional planar IC for a tree-based NoC. However, this is eclipsed by the superior performance achieved in terms of energy and area overhead. The energy profiles for 3D tree-based NoCs (Fig.€6.8c) reveal significant improvements over 2D implementations. Both BFT and Fat Tree (SPIN) networks show a very large reduction in energy when 3D ICs are used. Once again, energy dissipation is largely dependent both on architecture and injection load. Each NoC shows that energy dissipation increases with injection load until the network becomes saturated, similar to the throughput curve shown in Fig.€ 6.8a. The energy profiles show that the Fat Tree networks cause higher energy dissipation than the Butterfly Fat Tree instantiations, but this is universally true only at high injection load. Again, this is the motivation to consider packet energy of the networks as a relevant metric for comparison, shown in Fig.€6.8d. Energy savings in excess of 45% are achievable by adopting 3D ICs as a manufacturing methodology, and both BFT and Fat Tree networks show similar improvements. In case of tree-based NoCs, where the basic network topology remains unchanged in 3D implementations, all improvements in energy dissipation are caused by the shorter wires. As showed earlier in Table€6.1, a three-dimensional structure greatly reduces the inter-switch wire length. The overall energy dissipation in a NoC is heavily dependent on the interconnect energy, and this reduction in inter-switch wire length effects very large savings. Besides advantages in terms of energy, three-dimensional ICs enable tree-based NoCs to reduce silicon area overhead by a sizable margin. Figure€6.9 shows the overall area overhead of tree-based NoCs. Although no improvements are made in terms of switch area, the reductions in inter-switch wire lengths and amount of repeaters are responsible for substantial reductions in wiring overhead. This is 25

Switch Wiring

[%] of SoC Area

20 15 10 5

Fig. 6.9↜渀 Area overhead for tree-based NoCs

0

2D BFT

2D Fat Tree 3D BFT Topology

3D Fat Tree

6â•… Three-Dimensional Networks-on-Chip: Performance Evaluation

133

especially true of the Fat Tree network, which has more interconnects in the higher levels of the tree; wiring overhead is reduced more than 60% by instantiating the network into a 3D IC.

6.3.7  Effects of Traffic Localization Until this point, a uniform spatial distribution of traffic has been assumed. In a SoC environment, different functions would map to different parts of the chip and the traffic patterns are expected to be localized to different degrees [28]. We therefore consider the effect of traffic localization on the performance of the 3D NoCs, and in particular it considers the illustrative case of spatial localization where local messages travel from a source to the set of the nearest destinations. In the case of BFT and Fat Tree, localized traffic is constrained to within a cluster consisting of a single sub-tree while, in the case of 3D Mesh, it is constrained to within the destinations placed at the shortest Manhattan distance [21]. On the other hand, the 3D Stacked Mesh architecture is created simply to take advantage of the inexpensive vertical communication. The research pursued by Li et€al. in [9] suggested that in a 3D multi-processor SoC, much of the communication should take place vertically, taking advantage of the short inter-layer wire segments. This is a result of a large proportion of network traffic occurring between processor and the closest cache memories, which are often placed along the z-dimension. Consequently, in these situations, the traffic will be highly localized, and this study therefore considers localized traffic to be constrained to within a pillar for 3D Stacked Mesh. Figure€6.10 summarizes these effects, revealing the benefits of traffic localization. More packets can be injected into the network, improving the throughput characteristics of each topology as shown in Fig.€6.10a, c, which also shows the throughput profile of the 2D topologies for reference. Analytically, increasing localization reduces the average number of hops that a flit must travel from source to destination. Figure€ 6.10a reveals that the 3D Stacked Mesh network provides best performance in terms of throughput in the presence of localized traffic. However, this is achieved by using a wide bus for vertical communication. Let us consider what occurs when the bus size is equal to the flit width of 32 bits. With low localization, the achieved throughput is higher than that in a 2D Mesh network. However, when the fraction of localized traffic in the vertical pillars is increased, huge performance degradation is seen. This is due to the contention in the bus. When the bus width is increased to 128 bits, throughput increases significantly with increase in localized traffic. This happens due to less contention in a wider communication channel. Figure€6.10b, d depict the effects of localization on packet energy, and, unsurprisingly, there is a highly linear relationship between these two parameters. Packet energy is highly correlated with the number of hops from source to destination, and the resultant reduction of packet energy with localization supports this correlation. For the mesh-based networks, 3D Ciliated Mesh exhibits the lowest packet energy due to its low hop count and very short vertical wires. In fact, at highest localization,

a

Maximum Throughput

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0

0

2D BFT

3D Fat Tree 3D BFT 2D Fat Tree

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 % Localized Traffic

% Localized Traffic

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Stacked Mesh (32-bit)

Stacked Mesh (128-bit)

Ciliated 3D Mesh

2D MESH 3D MESH Localized

1

1

d

0

2000

4000

6000

8000

10000

12000

14000

16000

b

0

2000

4000

6000

8000

10000

12000

0

0

2D Fat Tree 2D BFT

3D Fat Tree 3D BFT

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 % Localized Traffic

% Localized Traffic

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Stacked Mesh (32-bit)

Stacked Mesh (128-bit)

Ciliated 3D Mesh

3D MESH Localized

2D MESH

1

1

Fig. 6.10↜渀 Localization effects on mesh-based NoCs in terms of: a Throughput and b Packet energy; and on tree-based NoCs in terms of c Throughput and d Packet energy

c

Maximum Throughput

Packet Energy (pJ) Packet Energy (pJ)

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

134 B. S. Feero and P. P. Pande

6â•… Three-Dimensional Networks-on-Chip: Performance Evaluation

135

the packet energy for a 3D Ciliated Mesh topology is less than 50% of that of the next-best-performing topology: 3D Mesh. For the tree-based NoCs, both 3D networks have much-improved packet energy with traffic localization. As can be seen from Fig.€6.10, there are tradeoffs between packet energy and throughput. For instance, the best-performing topology in terms of energy, ciliated Mesh, operates at the lowest throughput even when traffic is highly localized. On the other hand, although a 3D Stacked Mesh network with wider bus width achieves superior throughput without necessitating a highly local traffic distribution, it incurs more energy dissipation than other structures under local traffic due to the capacitive loading on the interlayer busses. However, the other topologies lie in some middle ground between these two extremes, and in general, it is clear that 3D ICs continue to effect improvements on NoCs under localized traffic.

6.3.8  Effects of Wire Delay on Latency and Bandwidth In NoC architectures, the inter-switch wire segments, along with the switch blocks, constitute a pipelined communication medium. The overall latency will be governed by the slowest pipelined stage. Table€ 6.2 showed earlier that the maximum wire delays for the network architectures are different. Though the vertical wire delays are very small, still the overall latency will be dependent on the delay of the switch blocks. Though the delays of the switch blocks were constrained within the 15FO4 limit, they were still the limiting stages in the pipeline, specifically when compared to the fast vertical links. Yet, considering a hypothetical case, which ignores the implications of switch design, where the clock period of the network is equal to the inter-switch wire delay, then the clock frequency can be increased, and, resultantly, the latency can be reduced significantly. With this in mind, latency in nanoseconds (instead of latency in clock cycles) and bandwidth (instead of throughput) are calculated. All other network parameters are kept consistent with the previous analysis. A plot of latency for all network topologies is shown in Fig.€6.11, and Table€6.5 depicts the network bandwidth in units of Terabits per second. To calculate bandwidth, the following expression is followed: 

BW = TPmax ·

1 · wflit · N , f

(6.13)

where TPmax represents the throughput at saturation, f represents the clock frequency, wflit is the flit width, and N is the number of IP blocks. Table€ 6.5 shows the performance difference achieved by running the NoC with a clock as fast as the inter-switch wire, disregarding the switch design constraints. It is evident that the tree-based architectures show the greatest performance improvement in this scenario going from 2D to 3D implementations, as the horizontal wire lengths are also reduced.

B. S. Feero and P. P. Pande

136 Fig. 6.11↜渀 Latency in ns at hypothetical clock frequencies

400

2D MESH 3D MESH Ciliated 3D MESH Stacked MESH 3D Fat Tree 3D BFT 2D Fat Tree 2D BFT

350 Latency (ns)

300 250 200 150 100 50 0

Table 6.5↜渀 Bandwidth of network architectures at simulated and hypothetical frequencies (Terabits/s)

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Injection Load

â•–

f€=€1.67€GHz

2D Mesh 3D Mesh Ciliated 3D Mesh 3D Stacked Mesh 2D BFT 2D Fat Tree 3D BFT 3D Fat Tree

1.357 2.412 1.457 2.488 0.9543 2.515 0.9543 2.515

f╛€= 1/(max wire delay) 3.711 6.596 3.983 6.804 1.039 2.738 2.474 6.520

1

% increase 173.5 173.5 173.5 173.5 ╇╇ 8.9 ╇╇ 8.9 159.2 159.2

6.3.9  Network Aspect Ratio The ability to stack layers of silicon is not without nuances. Upcoming 3D processes have a finite number of layers due to manufacturing difficulties and yield issues [3]. Furthermore, it is speculated [3] that the number of layers in a chip stack are not likely to scale with transistor geometries. This has a nontrivial effect on the performance of 3D NoCs. Consequently, future NoCs may have a greater number of intellectual property blocks in the horizontal dimensions than vertically. The effect of this changing aspect ratio must be characterized. For a more in-depth illustration of these effects, the overall performance of a mesh-based NoC in a 2-layer IC will be evaluated in comparison to the previouslyanalyzed 3D 4â•›×â•›4â•›×â•›4 Mesh and 2D 8â•›×â•›8 Mesh. Here, a 64-IP 8â•›×â•›4â•›×â•›2 Mesh is considered to match the 64-IP network size, in order to make the comparison of latency and energy as fair as possible, along with a 60-IP 6â•›×â•›5â•›×â•›2 Mesh to show a network which is similar in size and that results in a more square overall footprint than the 8â•›×â•›4â•›×â•›2 Mesh. Figure€6.12 summarizes the analysis of these 2-layer ICs. Throughput characteristics are seen in Fig.€ 6.12a. It shows clearly that the 6â•›×â•›5â•›×â•›2 Mesh achieves a significantly higher throughput than the 2D 8â•›×â•›8 Mesh and the 8â•›×â•›4â•›×â•›2 Mesh, which suffers from a high average hop count (4.44 vs 4.11

a

Throughput

1

0.1

0.2

0.3

0.5

0.6

0.7

Injection Load

0.4

0.8

0.9

8x4x2 MESH

6x5x2 MESH

500

0

2D MESH 4x4x4 MESH

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Injection Load

1000

1500

2000

2500

3000

3500

4000

0

8x4x2 MESH

6x5x2 MESH

4x4x4 MESH

2D MESH

1

1

b

d

10000 9000 8000 7000 6000 5000 4000 3000 2000 1000 0

1000 900 800 700 600 500 400 300 200 100 0 0

2D MESH

9139.84

6196.86 5264.09

8x4x2 MESH 6x5x2 MESH 4x4x4 MESH

Topology

7115.65

Injection Load

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

8x4x2 MESH

6x5x2 MESH

4x4x4 MESH

2D MESH

1

Fig. 6.12↜渀 Comparing two 2-layer NoCs: a Throughput vs. injection load, b Latency vs. injection load, c Cycle energy vs. injection load, and d Packet energy

c

Cycle Energy (pJ)

Latency (clock cycles) Packet Energy (pJ)

0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

6â•… Three-Dimensional Networks-on-Chip: Performance Evaluation 137

138

B. S. Feero and P. P. Pande

for the 6â•›×â•›5â•›×â•›2 Mesh), while achieving a lower maximum throughput than the 4-layer mesh. Likewise, the 2-layer mesh NoCs outperform the 2D Mesh in terms of latency, shown in Fig.€6.12b, without exceeding the performance of the 4-layer 3D instantiation. This trend continues when considering cycle energy (Fig.€6.12c) and packet energy (Fig.€6.12d). These results are as expected. With the first layer added, significant improvements are apparent in terms of each performance metric over the 2D case. Though the multi-layer NoC exhibits superior performance characteristics compared to a 2D implementation, it will have to circumvent significant manufacturing challenges. Yet, even if implementations are limited to two-layer 3D realizations, they will still significantly outperform the planar NoCs.

6.3.10  Multi-Layer IPs Throughout this chapter, each IP block has been assumed to be instantiated in one layer of silicon. However, as discussed in [10], it is possible for the IP blocks to be designed using multiple layers, although major issues like clock synchronization must be addressed. So, each network architecture is analyzed with multi-layer IPs. The pipelined communication shown in Fig.€ 6.13 is assumed; i.e. the NoCs are constrained by the switch delay and it cannot be driven as fast as the inter-switch wire. Considering this, multi-layer IPs have no effect on either throughput or latency (assuming the same clock frequency for all networks), but there are nontrivial effects on the energy dissipation profile. This effect on packet energy is depicted in Fig.€6.14. The energy savings come from reduced horizontal wire lengths. For

Source IP

Dest. IP

Fig. 6.13↜渀 The pipelined nature of NoCs

Switches

14000

2D MESH 2D BFT Ciliated 3D Mesh 3D Fat Tree

Packet Energy (pJ)

12000 10000

2D Fat Tree 3D MESH Stacked Mesh (128-bit) 3D BFT

8000 6000 4000 2000

Fig. 6.14↜渀 The effect of multilayer IPs

0

1

2 3 Number of Layers for IP

4

6â•… Three-Dimensional Networks-on-Chip: Performance Evaluation

139

instance, if a 2.5â•›×â•›2.5€mm IP block is instantiated in 2 layers, the IP’s circuitry is spread over 2 layers, and the footprint diagonal reduces by a factor of 1.414. Similarly, if instantiated in 3 layers, the footprint diagonal reduces by a factor of 1.732, and with 4 layers, the factor is 2. Although the vertical wire lengths are increased 2, 3, and 4 times, respectively, in order to span the entire multi-layer IP, the negative effects on energy incurred by this are eclipsed by the significant reductions in horizontal wire lengths. However, multi-layer IPs increase the number of layers in a 3D IC, placing an increased burden on manufacturability.

6.4╅Heat Dissipation Profile of 3D NoCs Heat dissipation is an extremely important concern in 3D ICs. Already, thermal effects have been known to have significant implications on device reliability and interconnect in traditional 2D circuits [29]. With the reduced footprint inherent to 3D ICs, this problem is exacerbated as the energy dissipated throughout the entire chip is now constrained to a smaller area, therefore increasing the energy density of these circuits. As a result, it is imperative that thermal issues are addressed in any system involving 3D integration. Accordingly, an analysis of 3D NoC is incomplete without an examination of temperature. It is especially important since the interconnect structure of a NoC can consume close to 50% of the overall power budget [13]. As temperature is closely related to the energy dissipation of the IC, this analysis will draw heavily upon the discussion of energy from Sects.€6.3.1 and 6.3.2. This section considers the 2D and 3D NoC architectures introduced in Sect.€6.3 and evaluates them in the presence of real traffic patterns. Furthermore, Sect.€6.3 has shown that the energy dissipated by the interconnection infrastructure, i.e. the communication energy, can be reduced compared to a 2D implementation by virtue of the inherent nature of the network architecture. Consequently, it will have a positive effect on heat dissipation.

6.4.1  Temperature Analysis Temperature in a 3D IC is related to a variety of factors including power dissipation and power density. In an integrated circuit, according to [30], the steady state temperature distribution is given by the following Poisson equation: 

∇ 2 T (r) =

−g(r) . kl (r)

(6.14)

Here, r is the three-dimensional coordinate (↜x,y,z). T(r) is the temperature inside the chip at point r, g(r) is the volume power density at that point, and kl(r) is the thermal conductivity. An important fact to note is that kl(r), the thermal conductivity, is

140

B. S. Feero and P. P. Pande

constrained by the manufacturing process and a SoC designer has little or no control over it. Therefore, the volume power density, g(r), is the parameter over which a designer has the most control. The challenge facing all designers of 3D ICs is to exercise control over this parameter. In a 3D integrated circuit, the volume power density of the chip is increased. The lateral dimensions are significantly smaller, and as a result, the total power of the circuit is dissipated in a much smaller area. For instance, in a four-layer 3D IC, the floor area is reduced by a factor of 4, for an eight-layer 3D IC, that area is reduced by a factor of 8, etc. Clearly, the energy of the entire chip is now constrained to a much smaller footprint, and the volume power density increases with respect to this.

6.4.2  The Relationship Between Temperature and Energy According to (6.14), it is clear that lower energy corresponds to lower heat. With an increase in volume power density, there is a corresponding increase in temperature. However, with 3D integration, the density of the chip is increased. As a result, there is a factor leading to higher heat in 3D NoCs. On the other hand, in 3D NoCs the communication energy can be reduced compared to a 2D implementation due to the various factors explained earlier. Consequently, it will lead to lesser heat dissipation in 3D NoCs. To quantify the overall effects of these two opposing factors, the heat dissipation profile for the aforementioned 3D NoC architectures is evaluated in presence of realistic traffic patterns.

6.4.3  Simulation Methodology The temperature profiles of the 3D NoCs were obtained through simulations following the methodology shown in Fig.€ 6.15. First, the network architecture is chosen. Subsequently, the network switches are instantiated in VHDL and synthesized using Synopsys DesignVision and a 90-nm standard cell library from CMP [26]. Here, Synopsys PrimePower is run to generate the energy profiles of the network switches. Next, the overall floorplan of the NoC is created. In order to generate the energy profile of the entire NoC it is necessary to incorporate the energy dissipated by each inter-switch stage. This is calculated taking into account the specific layout of each topology following the method elaborated in [20]. Following this, the NoC simulator is run in order to generate the overall energy profile of the NoC. Finally, with a complete floorplan and power profile, the Hotspot tool, developed by a research team at the University of Virginia [31], is used to generate the temperature profile. Hotspot takes the floorplan and applies the power profile to it, and with this information, it calculates the volume power density. From this, the temperature profile is generated.

6╅ Three-Dimensional Networks-on-Chip: Performance Evaluation Fig. 6.15↜渀 Design flow

141

Network topology Instantiate in VHDL Synthesize using DesignVision & analyze power of switches using PrimePower Energy profile of switches Run NoC Simulator Energy Profile of entire NoC

Generate Floorplan Complete floorplan

Temperature analysis using HotSpot Temperature profile of NoC

6.4.4  Experimental Results In accordance with the prescribed methods, 64-IP instantiations of each 3D NoC architecture were analyzed for thermal performance, with temperature taken as a function of injection load. As explained in Sect.€6.4.1, temperature is closely related to power density, so, likewise, these temperature profiles are very similar in form to the energy profiles, shown in Fig.€6.16a. The analysis begins with the 2D topologies. A plot of the temperature characteristics of the two architectures is shown in Fig.€6.16b with the temperature normalized to the maximum temperature of the 2D Mesh, considered as the baseline case. Figure€6.16a shows temperature saturating at different values and at different injection loads for each topology, like the communication energy dissipation profiles. With a 3D network implementation, this chapter has shown significant improvements in terms of energy dissipation, particularly packet energy, which is revisited in Fig.€6.16e. These significant improvements in energy have substantial effects on the temperature characteristics of these 3D networks. Let us first consider the hypothetical case where 3D implementations of these topologies dissipate the same communication energy per packet as the 2D versions. This case is shown by the dotted lines in Fig.€6.16c. It is very clear that in the absence of any packet energy gains, the result is a much hotter network. As discussed in Sect€6.2, when moving to a 3D NoC, the overall chip area remains constant while the footprint is reduced. In these 10â•›×â•›10€ mm 4-layer, 3D implementations, the entire energy dissipation of the chip is constrained to an area one quarter the size of the 20â•›×â•›20€mm 2D implementations. As a result, the power density should be significantly increased. However, the actual temperature profiles of the 3D networks, depicted by the solid lines in Fig.€6.16d, show a marked difference. This highlights a very important

142

B. S. Feero and P. P. Pande

a

3

2D Mesh 3D Mesh 3D Stacked Mesh 2D BFT 3D BFT

Normalized Temperature

Cycle Energy (pJ)

5000 4500 4000 3500 3000 2500 2000 1500 1000 500 0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Injection Load

1

Normalized Temperature

2 1 0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Injection Load 12978.42

Packet Energy (pJ)

12000 9139.84

8000 5264.09

6000

6111.85

6604.84

4000 2000 0 2D Mesh

2D BFT

3D Mesh 3D Stacked Mesh

Topology

3D BFT

1

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Injection Load

1

4 3 2 1 0

f

3D Mesh 3D Stacked 3D BFT

5

d Normalized Temperature per Packet

Normalized Temperature

3

14000

e

0

6

4

10000

1

b

3D Mesh 3D Stacked 3D BFT

5

0

2

0 0

6

c

2D Mesh 2D BFT

5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Injection Load

1

2.78 2.23 1.82 1.45 1.00

2D Mesh

2D BFT

3D Mesh 3D Stacked Mesh

3D BFT

Topology

Fig. 6.16↜渀 Experimental results for 3D NoCs: a Cycle energy, b Maximum temperature in 2D architectures, c Hypothetical temperature for 3D architectures, d Maximum temperature for 3D architectures, e Packet energy, and f the normalized contribution to temperature per packet

characteristic of NoC in a 3D environment: the savings in communication energy incurred by choosing a 3D NoC implementation partially mitigate what would otherwise be a drastic increase in temperature. To help describe this effect, Fig.€6.16f presents the normalized temperature contribution per packet, using the 2D mesh architecture again as the baseline case. The dotted bars represent the hypothetical case discussed earlier. The contribution to temperature per packet metric follows a similar idea to that of packet energy. Each packet sent through the network is responsible for a certain amount of energy dissipation. This, in turn, causes a rise in temperature. Therefore, as packet energy quantifies the energy efficiency of a NoC, the temperature contribution per packet thus quantifies the temperature efficiency of a NoC. All topologies show real improvements over the hypothetical case, and, in fact, the 3D version of the BFT

6â•… Three-Dimensional Networks-on-Chip: Performance Evaluation

143

network have lower temperature than its 2D counterpart. This can be attributed, in part, to the very high (49%) decrease in packet energy that is characteristic of a 3D BFT implementation over a 2D BFT instantiation.

6.5â•…Conclusion This chapter has demonstrated that besides reducing the footprint in a fabricated design, three-dimensional network structures have better performance compared to traditional, 2D NoC architectures. It has demonstrated that both mesh- and treebased NoCs are capable of achieving better performance when instantiated in a 3D IC environment compared to more traditional 2D implementations. The meshbased architectures show significant performance gains in terms of throughput, latency and energy dissipation with a small area overhead. On the other hand, the 3D tree-based NoCs achieve significant gain in energy dissipation and area overhead without any change in throughput and latency. However, if the NoC switches are designed to be as fast as the interconnect, even the 3D tree-based NoCs will exhibit performance benefits in terms of latency and bandwidth. Furthermore, 3D NoCs are efficient in addressing the temperature issues characteristic of 3D integrated circuits. The Network-on-Chip (NoC) paradigm continues to attract significant research attention in both academia and industry. With the advent of 3D ICs, the achievable performance benefits from NoC methodology will be more pronounced as this chapter has shown. Consequently this will facilitate adoption of the NoC model as a mainstream design solution for larger multi-core system chips.

References 1. P. Magarshack and P. G. Paulin, “System-on-Chip Beyond the Nanometer Wall,” Proceedings of 40th Design Automation Conf. (DAC 03), ACM Press, 2003, pp.€419–424. 2. International Technology Roadmap for Semiconductors 2005: Interconnect, [online] http:// www.itrs.net/ 3. A. W. Topol et al., “Three-Dimensional Integrated Circuits,” IBM Journal of Research & Development, vol.€50, no.€4/5, July/Sept. 2006. pp. 491–506. 4. W. R. Davis et al., “Demystifying 3D ICs: The Pros and Cons of Going Vertical,” IEEE Design and Test of Computers, vol.€22, no.€6, Nov. 2005. 5. Y. Deng et al., “2.5D System Integration: A Design Driven System Implementation Schema,” Proceedings of the Asia South Pacific Design Automation Conference, 2004. 6. M. Ieong et al., “Three Dimensional CMOS Devices and Integrated Circuits,” Proceedings of IEEE Custom Integrated Circuits Conference, 2003. 7. L. Benini and G. De Micheli, “Networks on Chips: A New SoC Paradigm,” IEEE Computer, Jan. 2002, pp.€70–78. 8. W. J. Dally and B. Towles, “Route Packets, Not Wires: On-Chip Interconnection Networks,” Proceedings of the 2001 DAC, June 18–22, 2001, pp.€683–689.

144

B. S. Feero and P. P. Pande

╇ 9. F. Li et al., “Design and Management of 3D Chip Multiprocessors Using Network-in-Memory,” Proceedings of the 33rd International Symposium on Computer Architecture (ISCA’06), pp.€130–141. 10. V. F. Pavlidis and E. G. Friedman, “3-D Topologies for Networks-on-Chip,” IEEE Transactions on Very Large Scale Integration (VLSI), October 2007, pp.€1081–1090. 11. J. Srinivasan et al., “Exploiting Structural Duplication for Lifetime Reliability Enhancement,” Proceedings of the 32nd International Symposium on Computer Architecture (ISCA’05), pp.€520–531. 12. J. Tsai, C. C. Chen, G. Chen, B. Goplen, H. Qian, Y. Zhan, S. Kang, M. D. F. Wong, and S. S. Sapatnekar, “Temperature-Aware Placement for SoCs,” Proceedings of the IEEE, vol.€94, no.€8, Aug. 2006, pp.€1502–1518. 13. T. Theocharides, G. Link, N. Vijaykrishnan, and M. Irwin, “Implementing LDPC Decoding on Network-on-Chip,” Proceedings of the International Conference on VLSI Design, 2005 (VLSID 2005), pp.€134–137. 14. S. Vangal et al., “An 80-Tile 1.28TFLOPS Network-on-Chip in 65€nm CMOS,” Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), 2007, pp.€98–99. 15. Tilera Co. http://www.tilera.com 16. P. Jacob et al., “Predicting the Performance of a 3D Processor-Memory Stack,” IEEE Design and Test of Computers, Nov. 2005, pp.€540–547. 17. R. I. Greenberg and L. Guan, “An Improved Analytical Model for Wormhole Routed Networks with Application to Butterfly Fat Trees,” Proceedings of the International Conference on Parallel Processing (ICPP 1997), pp.€44–48. 18. C. Grecu et al., “A Scalable Communication-Centric SoC Interconnect Architecture,” Proceedings of the 5th International Symposium on Quality Electronic Design, 2004, pp.€343– 348. 19. P. Guerrier and A. Greiner, “A Generic Architecture for On-Chip Packet-Switched Interconnections,” Proceedings of Design and Test in Europe (DATE), Mar. 2000, pp.€250–256. 20. C. Grecu, P. P. Pande, A. Ivanov, and R. Saleh, “Timing Analysis of Network on Chip Architectures for MP-SoC Platforms,” Microelectronics Journal, Elsevier, vol.€36, no.€9, Mar. 2005, pp.€833–845. 21. P. P. Pande, C. Grecu, M. Jones, A. Ivanov, and R. Saleh, “Performance Evaluation and Design Trade-offs for Network on Chip Interconnect Architectures,” IEEE Transactions on Computers, vol.€54, no.€8, Aug. 2005, pp.€1025–1040. 22. J. Duato, S. Yalamanchili, and L. Ni, Interconnection Networks—An Engineering Approach, Morgan Kaufmann, San Francisco, CA,2002. 23. K. Park and W. Willinger, Self-Similar Network Traffic and Performance Evaluation, John Wiley & Sons, New York,2000. 24. D. R. Avresky, V. Shubranov, R. Horst, and P. Mehra, “Performance Evaluation of the ServerNetR SAN under Self-Similar Traffic,” Proceedings of 13th International and 10th Symposium on Parallel and Distributed Processing, April 12–16th, 1999, pp.€143–147. 25. G. V. Varatkar and R. Marculescu, “On-Chip Traffic Modeling and Synthesis for MPEG-2 Video Applications,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol.€8, no.€3, June 2000, pp.€335–339. 26. Circuits Multi-Projects. http://cmp.imag.fr/ 27. K. C. Saraswat et al., “Technology and Reliability Constrained Future Copper Interconnects—Part II: Performance Implications,” IEEE Transaction Electron Devices, vol.€ 49, no.€4, Apr. 2002, pp.€598–604. 28. P. P. Pande, C. Grecu, M. Jones, A. Ivanov, and R. Saleh, “Effect of Traffic Localization on Energy Dissipation in NoC-based Interconnect,” Proceedings of IEEE International Symposium on Circuits and Systems, 23rd–26th May 2005, pp.€1774–1777. 29. J. A. Davis, R. Venkatesan, A. Kaloyeros, M. Beylansky, S. J. Souri, K. Banerjee, K. C. Saraswat, A. Rahman, R. Reif, and J. D. Meindl, “Interconnect Limits on Gigascale Integration (GSI) in the 21st Century,” Proceedings of the IEEE, vol.€89, no€3, Mar. 2001, pp.€305–324.

6â•… Three-Dimensional Networks-on-Chip: Performance Evaluation

145

30. D. Meeks, “Fundamentals of Heat Transfer in a Multilayer System,” Microwave Journal, vol.€1, no.€1, Jan. 1992, pp.€165–172. 31. W. Huang, K. Sankaranarayanan, R. J. Ribando, M. R. Stan, and K. Skadron, “An Improved Block-Based Thermal Model in HotSpot 4.0 with Granularity Considerations,” Proceedings of the Workshop on Duplicating, Deconstructing, and Debunking, in conjunction with the 34th International Symposium on Computer Architecture (ISCA), June 2007.

â•…

Part III

System and Architecture Design

â•…

Chapter 7

Asynchronous 3D-NoCs Making Use of Serialized Vertical Links Abbas Sheibanyrad and Frédéric Pétrot

7.1â•…Introduction 3D-Integration, a breakthrough in increasing transistor density by vertically stacking multiple dies with a high-speed die-to-die interconnection [1], is becoming a viable solution for the consumer electronic market segment. 3D-Integration results in a considerable reduction in the length and the number of long global wires which are the dominant factors on delays and power consumption, and allows stacking dies of different technologies (e.g. DRAM, CMOS, MEMS, RF) in a single package. However, even though this new technology introduces a whole new set of application possibilities, it also aggravates some current problems in VLSI design and, introduces several new ones. Delivering the clock signal to each die and dealing with clock synchronization are critical design problems for 3D integrated circuits [2]. The so-called GALS (Globally Asynchronous Locally Synchronous) paradigms seem indispensable to be exploited [3]. In that situation, and in order to provide the necessary computational power and communication throughput, NoCs offer a structured solution to construct GALS architectures. Since a NoC spans the entire chip, the network could be the globally asynchronous part of the system, while the subsystem modules are the locally synchronous parts. While the utilization of the 3D-Integration technology is at the moment very ad-hoc, the innovative exploitation of this major novel key technology with the Networks-on-Chip paradigm for the design and fabrication of advanced integrated circuits will allow a more generic use. The introduction of the NoC concept in early 2000 [4] was a big paradigm shift and opened a new, active, and practical area of research and development in the academia and the industry. Supposing a NoC with a complete 3D-Mesh (3D-Cube) topology, the number of vertical channels is equal to 2(↜Nâ•›−â•›3√N 2), where N is the number of network nodes. As generally each channel of a NoC consists of tens and even in some architectures A. Sheibanyrad () TIMA Laboratory, 46, Avenue Felix Viallet, 38000 Grenoble, France e-mail: [email protected] A. Sheibanyrad et al. (eds.), 3D Integration for NoC-based SoC Architectures, Integrated Circuits and Systems, DOI 10.1007/978-1-4419-7618-5_7, ©Â€Springer Science+Business Media, LLC 2011

149

150

A. Sheibanyrad and F. Pétrot

hundreds of physical wire links, such a network with a large number of nodes requires a large number of physical vertical interconnections. In a 3D-Integrated circuit the die-to-die interconnect pitch (mainly due to the bonding pads which need to be large enough to compensate for misalignment of dies) imposes a larger area overhead than corresponding horizontal wires. Moreover, fabricating such a circuit involves several extra and costly manufacturing steps, and each extra manufacturing step adds a risk for defects, resulting in potential yield reduction. The approach can be cost-effective only for very high yields. As a consequence looking for cost-efficiency introduces a trade-off between the great benefit of exploiting high-speed, short-length, vertical interconnects, and a serious limitation on the number of them. In order to reduce the number of vertical links to be exploited by the network, we suggest making use of serialized vertical links in asynchronous 3D-NoCs. Section€ 7.2 of this chapter elaborates on 3D-Integration technologies and Through-Silicon-Vias (TSVs) which, at the time being, are the most promising vertical interconnects. Then in Sect.€7.3 we explain how a three-dimensional design space leads to the design of 3D-NoCs, and describe how the incorporation of the third dimension provides a major improvement in the network performance. Section€7.4 details the advantageous aspects of exploiting asynchronous circuits and Sect.€7.5 explains how the use of asynchronous serialized vertical links minimizes the number of die-to-die interconnects and maximizes the exploitation of the potentially high bandwidth of these vertical connections. Finally we conclude the paper in Sect.€7.6.

7.2â•…3D-Integration Technology The shrinking of processing technology in the deep submicron domain aggravates the imbalance between gate delays and wire delays: while gate delays decrease, global wire delays, because of the growth of wire resistance, increase [5]. However, since the length of local wires usually shrinks with traditional scaling, the impact of their delay on performance is minor. On the contrary, as the die size does not necessarily scale down, global wire lengths do not reduce. Global wires connect different functional units of a system and spread over the entire chip. The largest part of delays now is related to global wires. Whereas the operating frequency and transistor density need to continue to grow, global wires are likely to have propagation delays largely exceeding the required clock period. Furthermore the total amount of energy dissipated by long global wires is not negligible. While Networks-on-Chip [6] systematically tackle these challenges by differentiating between local and global interconnections, 3D-Integration results in a considerable reduction in the length and the number of long global wires, by folding the die into multiple layers and using short vertical links instead of long horizontal interconnects.

7â•… Asynchronous 3D-NoCs Making Use of Serialized Vertical Links

151

There are several methods for die stacking and various vertical interconnection technologies. A comparison in terms of vertical density and practical limits can be found in (Chap.€ 1 of the present book). Through-Silicon-Via (TSV) has the potential to offer the greatest vertical interconnect density and therefore is the most promising among the vertical interconnect technologies. Furthermore, it features an extremely small inter-wafer distance of about 50€µm. Such a short distance guarantees low interconnect resistance about 50 times smaller than for a typical Metal 8 horizontal wire in 0.13€µm technology [7]. The authors of [7] have also indicated that for a whole via, capacitance is about 10 times smaller than for a typical Metal 2/3 horizontal wires of 1.5€mm in 0.13€µm. They have shown that while a 1.5€mm horizontal link delay is around 200€ps, for a whole vertical interconnect delay is 16–18.5€ ps, turning out to be substantially faster and more energy efficient than moderate size planar links.

7.2.1  TSV Technology Challenges Figure€ 7.1 shows a side view of the inter-die connection using Through-SiliconVias. The TSV technologies are assembled at the wafer-level, rather than the dielevel. Since assembly cannot be performed with known-good dies, the fabrication yield with this approach drops quickly as more dies are added. Furthermore, additional processing steps are required and so some new defects can be generated, including misalignment, void formation during bonding phase, dislocation and defects of copper grains, oxide film formation over Cu interface, partial or full Pad detaching due to thermal stress, etc [8].

Metal Layers Silicon Substrate

Pitch

Fig. 7.1↜渀 Side view of the inter-die vertical connections using Through-Silicon-Via (TSV) technology

152

A. Sheibanyrad and F. Pétrot

Although 3D-Integration using TSV technology is not limited in the number of layers to be assembled, the yield can be a limiting factor. The approach can be cost-effective only for a very high yield. According to results of the leading 3D technology owners (shown in Fig.€9.4), the yield is an exponential function of TSV defect rate (DBI) and the number of TSVs, and thus, exponentially decreases when the number of TSVs reaches a technology dependent limit, from 1,000–10,000 in the sample technologies of the figure. Looking for cost-efficiency, TSV defect rate and consequently the chip fabrication yield is a major limiting factor and introduces a significant trade-off between the great benefit of exploiting high-speed, short-length, vertical TSV interconnects, and a serious limitation on the maximum number of them should (could) be exploited.

7.3â•…NoC and 3D-Design Contribution In order to take full advantage of the 3D-Integration, the decision on the use of TSVs must come upfront in the architecture planning process rather than as a packaging decision after circuit design is completed. This requires taking the 3D design space into account right from the start of the system design. In fact, a 3D integrated complex system contains different partitions of the whole pilled-up system connected by a Three-Dimensional NoC architecture [9, 10], allowing massive integration of processing elements, SRAMs, DRAMs, I/Os, etc. The incorporation of the third dimension (offered by the 3D-Integration paradigm) in the design of the integrated networks allows the exploitation of three dimensional topologies [11]. By dividing a system into more dies and stacking them up, we can provide a major improvement in the network performance. Table€7.1 presents a formal comparison between 2D-Mesh and 3D-Cube topologies. While a 3D-Cube is more complex as its switch degree is 7 (in comparison with 5 of a 2DMesh), it offers a lower average communication latency as the network diameter is smaller. Furthermore, according to the higher number of channels, and especially the higher number of bisection channels, we can anticipate a higher network saturation threshold for a 3D-Cube topology. The number of channels determines the maximum possibility of the simultaneous communications and the number of bisection channels determines the possibility of concurrent global communications (i.e. it Table 7.1↜渀 Formal comparison between 2D-Mesh and 3D-Cube Number Switch Network Number of Number of nodes degree diameter channels of vertical channels 2D-Mesh Nâ•›=â•›n2 5 2√N 6Nâ•›−â•›4√N 0 3D-Cube Nâ•›=â•›m3 7 33√N 8Nâ•›−â•›63√N2 2Nâ•›−â•›23√N2

Number of bisection channels

Load of the busiest channelsa

2√N 23√N2

Câ•›×â•›¼√N Câ•›×â•›¼3√N

C average load injected to the network by each node Assuming uniform destination distribution and dimension-ordered routing

a

7╅ Asynchronous 3D-NoCs Making Use of Serialized Vertical Links Fig. 7.2↜渀 Saturation threshold of a 64-Core system arranged in two different network topologies

300

8×8 4×4×4

250 Latency (Cycles)

153

200 150 100 50 0 15

20

25 30 35 Offered Load (%)

40

45

is a measure of bottleneck channels which could be used concurrently by members of a subnet communicating with members of other subnets). In order to experimentally testify the network latency and saturation gain when using networks with 3D-Cube topologies rather than 2D-Mesh, we have developed Cycle-Accurate SystemC simulation models for a general NoC architecture (i.e. an input buffered, wormhole packet-switching network using dimension-ordered routing algorithm). Assuming uniform random traffic pattern, Fig.€7.2 shows the average communication latencies of a system with 64 cores generating traffic. As can be seen, while an 8â•›×â•›8 2D-Mesh topology saturates when the offered load is about 25%, a 4â•›×â•›4â•›×â•›4 3D-Cube saturates for about 40%. The question which may arise here is: from the network performance point of view what would be the number of dies should be stacked [12]? Table€7.2 shows some figures of a system with 900 cores generating traffic (which can likely be expected in the near future [13]), when arranged in (1) one, (2) four, and (3) nine layers. This formal comparison testifies that the value of the third dimension (i.e. the number of dies a system is divided into) has a direct influence on the network performance, namely on communication latencies and the saturation threshold. However for a given operating point, there would be an optimum number of dies to be exploited. Figure€7.3 depicts the average communication latency in a 900-core Table 7.2↜渀 Formal comparison between three different network topologies of a 900-Core system Number of Load of Number Switch Network Number of Number the busiest of nodes degree diameter channels of vertical bisection channelsa channels channels 900 5 60 5,280 0 60 Câ•›×â•›¼â•›×â•›30 30â•›×â•›30 7 34 6,510 1,350 120 Câ•›×â•›¼â•›×â•›15 4â•›×â•›15â•›×â•›15 900 7 29 6,640 1,600 180 Câ•›×â•›¼â•›×â•›10 9â•›×â•›10â•›×â•›10 900 C average load injected to the network by each node a Assuming uniform destination distribution and dimension-ordered routing

154

A. Sheibanyrad and F. Pétrot

Fig. 7.3↜渀 Average packet latency of a 900-Core system arranged in three different network topologies

300

Latency (Cycles)

250 200 150 100 30×30 4×15×15 9×10×10

50 0

0

5

10 15 Offered Load (%)

20

25

system arranged in the three different topologies. We can see that, for example, while at the average offered load of 12% a 30â•›×â•›30 network is saturated, a 9â•›×â•›10â•›×â•›10 network works properly. We can also see that this offered load could be an operating point for a 4â•›×â•›15â•›×â•›15 network, whose fabrication would result in a better yield.

7.4â•…Asynchronous Circuit Exploitation Delivering clock to each die and dealing with clock synchronization is a critical problem on design of the 3D integrated circuits. Remembering that even in twodimensional chips making a highly symmetric fractal structured clock distribution network (e.g. an H-tree), which is essentially needed to route the clock signal to all parts of a chip with equal propagation delays, is very hard to achieve [14], we can imagine that constructing a three-dimensional clock distribution network (tree) is almost infeasible, as a huge number of vertical links (TSVs) will be needed. Furthermore, the run-time temperature variations and stresses and the thermal concerns (due to the increased power densities), which are significant issues of 3D integration, will introduce additional uncontrollable time-varying clock skew. In consequence, the GALS approaches seem to be the best solution to be used in 3D integrated systems. A GALS system is divided into several physically independent clusters and each cluster is clocked by a different clock signal. The advantage of this method is the reduction of the problem of clock distribution to a number of smaller sub-problems: since the different clocks do not need to be related in frequency and phase, the clock distribution problem on each planar cluster (which is much smaller than the whole system) becomes feasible. Moreover, the use of GALS paradigm enables the implementation of various forms of DPM (Dynamic Power Management) and DVFS (Dynamic Voltage and Frequency Scaling) methods which, because of heat extraction and energy dissipation, seem essential to be exploited in future

7â•… Asynchronous 3D-NoCs Making Use of Serialized Vertical Links

155

SoCs. DPM and DVFS contain a set of techniques that achieve energy-efficient computation by selectively turning off or reducing the performance of system components when they are idle or partially unexploited. In these methods the need for physical distinction between system power and frequency segments has emerged. Potentially NoCs are compatible with the idea of GALS that needs to clusterize the chip into several physically independent subsystems, but the question that remains is how the network itself must be clocked, and how we can deal with the problem of synchronization and metastability on clock boundaries. Since one obvious way to eliminate the problem of clock skew is the utilization of asynchronous logic, a network with a fully asynchronous circuit design is a natural approach to construct GALS architectures [15]. A large number of locally planar synchronous islands can communicate together via a global three-dimensional asynchronous network (see Fig.€7.4). An asynchronous NoC (which, itself, does not involve synchronization issues) limits the synchronization failure (metastability [16]) only at the network interfaces in the first and last steps of a packet path, i.e. in the source cluster where the synchronous data enters the asynchronous network and in the destination cluster where the asynchronous data goes into synchronous subsystems. Due to the fact that the robust synchronization is often accompanied by an unavoidable penalty on the latency, the absence of synchronizing elements at each hop of the path leads to an extremely lower communication latency for an asynchronous NoC, compared with a GALS-compatible multi-synchronous one [17]. The instantiation of two special types of FIFO (Sync-to-Async and Async-to-Sync) at network boundaries, between the synchronous Network Interface Controller (NIC) and the asynchronous network, provides the requested synchronous compliant interfaces [18]. Each link of an asynchronous network works with its own speed as fast as possible, as opposed to the synchronous approach in which the slowest element determines the highest operating frequency. The highest possible clock frequency in a synchronous system is limited by the worst-case combination of some parameters, such as power supply variation, temperature variation, transistor speed variation (e.g. due to the fabrication variability), data-dependent operations, and data prop-

CK0

CK3

CK1

CK4

CK2

CK5

IP

Async/Sync Interfaces (FIFOs)

Loca lnterconnect NIC

Asynchronous Network

Fig. 7.4↜渀 Multi-clocked 2D-clusters in a GALS system using an asynchronous 3D-Network

156

A. Sheibanyrad and F. Pétrot

agation delays. Typically, the worst-case combination is encountered very infrequently and the system performance usually is less than what it could be. Asynchronous circuits automatically adjust their speed of operation according to the current conditions and they are not restricted by a fixed clock frequency. Their operating speed is determined by actual local latencies rather than global worst-case. In a 3DNoC this property offers the opportunity of totally exploiting the potentially high bandwidth of vertical links. The average load of a link in a NoC depends on several parameters, including traffic distribution pattern of the system application, the network routing algorithm, and the flit injection rate. The last columns of Table€7.1 and 7.2 show the average load of the busiest links in a general NoC with 2D-Mesh and 3D-Cube topologies when the destination distribution is uniform (i.e. each traffic generator sends randomly packets to all other network nodes with a fixed load equivalent in all clusters). The nominal estimated average load of a link is much less than its maximum capacity, especially in a GALS-compatible asynchronous NoC in which the flit injection rate is much lower than the network throughput [17]. Figure€7.5 demonstrates the average packet latency of a system with 256 cores. The curve in which the points are represented by + corresponds to when the system is arranged in a mesh of 16â•›×â•›16 and the curve in which the point are × when arranged in a cube with 4 layers (4â•›×â•›8â•›×â•›8). In these two cases the network is synchronous. In contrast, the curves with the points in * and □ display the average packet latency of the same system (corresponding to the + and × curves, respectively) when the network is asynchronous. We should mention here that from the system point of view the physical characteristics of an asynchronous network have no sense and so for this level of simulation we have used cycle accurate SystemC models of the network for both synchronous and asynchronous cases. In the case of asynchronous we have used two different clocks symbolically with different frequency ratio, one for the network and the other for the system (i.e. traffic generators). In fact in a system-level simulation and to obtain system-level latencies the only parameter of an asynchronous network has to be taken into account is its speed ratio to the clock frequency of clusters. 120

“2D-Sync” “3D-Sync” “2D-Async” “3D-Async”

Fig. 7.5↜渀 Average packet latency of a 256-Core system using synchronous (the speed ratio of 1) and asynchronous (the speed ratio of 2) networks arranged in two and three dimensional topologies

Latency (cycles)

105 90 75 60 45 30 15 0

0

10

20

30

40 50 Load (%)

60

70

80

7â•… Asynchronous 3D-NoCs Making Use of Serialized Vertical Links

157

The curves of Fig.€7.5 are the results when the speed ratio between system clock frequency and the network throughput is two. It means for example if the average link throughput of the network is 1,000€Mflits/s, the cores inject the flits with a rate of 500€ Mflits/s. According to [19] the maximum clock frequency of usual SoCs using STMicroelectronics 90€nm GPLVT technology is about 400€MHz, while as stated in [20] the throughput of an asynchronous implementation of the same NoC using the same technology is about 1,000€Mflits/s. Hence, in our simulations the speed ratio of two could be a worst-case assumption. In a packet switching network when a packet is traversing the network all resources in the path between the header (first flit) and the trailer (last flit) are allocated to the packet and no other packet can use those resources. The other packets must wait until the path is released, that is after the packet trailer is passed. In a synchronous NoC the flits injected to the network (as well as the trailer) move through the hops cycle by cycle, with a throughput of one flit per cycle. In contrast in the asynchronous approach flits propagate as fast as possible. When the speed ratio between the asynchronous network and flit injectors (subsystems) is larger than one, the trailer releases the path faster than in a synchronous one which works with the speed of subsystems. We believe that fast path liberation is why asynchronous NoCs have a better saturation threshold. All told, we can easily state that in an asynchronous 3D-NoC making use of high-bandwidth TSVs, the vertical links exploit only a small fraction of their capacities. And, this fraction is lower than that of the horizontal links as their bandwidth is much higher, encouraging to search for solutions that make a more efficient usage of TSVs.

7.5╅Serialized Vertical Links Serializing the data communication [21] of vertical links is an innovative solution for better utilization of these high-speed connections in an asynchronous 3D-NoC, particularly because 3D integrated circuits are strictly TSV limited to ensure an acceptable yield and area overhead. The serialization allows minimizing the number of die-to-die interconnects and simultaneously maximizing the exploitation of the high bandwidth of these vertical connections, hence addressing the cost-efficiency trade-off of 3D-Integration using TSV technologies. Additionally, reducing the number of TSVs to be exploited allows for hardware redundancy that is often used to improve the yield. As the principal cause of TSV defects is the misalignments, a simple and effective way to add redundancy and improve yield is to use larger pads. Serialization and consequently using fewer TSVs leads to increase the vertical interconnection pitches and so in the same area we can use square pads larger than standard pads. Another efficient example of hardware redundancy is the use of redundant TSVs which can be used to replace defected TSVs [22]. Figure€7.6 depicts a conceptual architecture of an asynchronous 3D-NoC and the inter-router vertical connections using asynchronous serializer/deserializer (instan-

158

A. Sheibanyrad and F. Pétrot

Router Through-Silicon-Via Serializer Deserializer

Fig. 7.6↜渀 Inter-router vertical connections using asynchronous circuits

tiated just before and after of vertical connections). The communication throughput of serialized links depends on the speed of data serializing and deserializing, as well as the serialization level (i.e. the number of parts a flit must be divided into). If the serialization degrades the vertical throughput to a value much lower than that of horizontal links, these serial vertical links may become bottlenecks for all paths crossing them. As a consequence, the circuit design optimization of the serializer and deserializer plays a key role. It is also very important to properly determine the optimum serialization level. Figure€7.7 shows the packet latency of a 256-core system with a 3D dimensional topology of 4â•›×â•›8â•›×â•›8 using networks with serialized vertical links. These results are obtained from cycle accurate SystemC simulations. The name of curves is a notation determining the type of the network (i.e. synchronous or asynchronous) and the level of serialization. For example “3D-Async-V3to1” means an asynchronous network using serialized vertical links that divide a parallel data word into three serial parts. The serialization level has a direct impact on the network performance. The interesting point of this figure is that the performance of a normal synchronous 3D-NoC is between the performance of an asynchronous network with vertical serialization level of 3 and 4, which means, having a reduction in the number of vertical links between 66% and 50% when using asynchronous dual-rail data encoding

7â•… Asynchronous 3D-NoCs Making Use of Serialized Vertical Links 120

“3D-Sync” “3D-Async” “3D-Async-V2to1” “3D-Async-V3to1” “3D-Async-V4to1”

105 Latency (cycles)

Fig. 7.7↜渀 Average packet latency of a 256-Core system using synchronous (the speed ratio of 1) and asynchronous (the speed ratio of 2) 3D networks using serialized vertical links with different serialization level

159

90 75 60 45 30 15 0

0

10

20

30 40 50 60 Offered Load (%)

70

80

90

method. Dual-rail data encoding is a delay-insensitive code which uses two wires to represent a single bit of data with which the validity information is carried along and thereby the receiver is enable to unambiguously detect the word completion regardless of delays. Seeing that in these system-level simulations we have not taken into account that the bandwidth of vertical links (TSVs) is higher than horizontals, we can easily expect that the performance of such asynchronous 3D networks is better than those presented in the figure!

7.5.1  Implementation Figure€ 7.8 demonstrates a schema for vertical communication serialization. The router output data going down (or up), encoded on n bits at the transfer rate of f, is serialized in p ( 1 . For example, in Fig.€8.3c, e7 corresponds to a multicast flow from source v4 to destinations v2, v5 and v6. Based on the optimization goals and cost functions specified by the user, the output of our 3D-NoC architecture synthesis problem is an optimized custom network topology with pre-determined routes for the specified traffic flows on the network such that the data rate requirements are satisfied. For example, Fig.€8.3d, e show two different topologies for the CDG shown in Fig.€8.3c. Figure€8.3d shows a network topology where all flows share a common network. In this topology, the pre-determined route for the multicast flow e7 travels from v4 to v2 to first reach v2, and then it bifurcates at v2 to reach v5 and v6. Figure€8.3e shows an alternative topology comprising of two separate networks. In this topology, the multicast flow e7 bifurcates in the source node to reach v6, then it is transferred over the network link between v4 to v2 to reach v2, and then bifurcates to reach v5. Observe that in both cases, the amount of network resources consumed by routing of multicast traffic is less than what would be required if the traffic is sent to each destination as a separate unicast flow.

174

S. Yan and B. Lin

8.4.2  Problem Formulation In general, the solution space of possible application-specific network architectures is quite large. Depending on the communication demand requirements of the specific application under consideration, the best network architecture may indeed be comprised of multiple networks, among each, many flows sharing the same network resources. To address the 3D-NoC synthesis problem, we formulate the problem as a combination of a rip-up and reroute procedure for routing flows and a router merging procedure for optimizing network topologies. The key part of the algorithm is a ripup and reroute procedure that routes multicast flows by way of finding the optimum multicast tree on a condensed multicast routing graph using the directed minimum spanning tree formulation and the efficient algorithms [36, 37]. Then a router merging procedure follows after to further optimize the implementation and reduce cost. The router merging algorithm iteratively considers all possible mergings of two routers connected with each other and merges them if the cost of the resulting topology after merging is reduced. In order to obtain the best topology solutions with minimum power consumption, accurate power models for 3D interconnects and routers are derived in Sect.€8.5. They are provided to the synthesis design flow as a library and utilized by the synthesis algorithm as evaluation criteria. The RIPUP-REROUTE algorithm for routing flows and the ROUTER-MERGING algorithm to optimize topologies are based on using these power costs of network links and router ports as edge weights. The application-specific 3D-NoC synthesis problem can be formulated as follows: Input: • The communication demand graph H(↜V, E, π, λ) of the application. • The 3D-NoC network component library Φ(↜I, J), where I provides the power and area models of routers with different sizes, and J provides power models of physical links with different lengths. • The target clock frequency, which determines the delay constraint for links between routers. • The floorplanning of the cores. Output: • A 3D-NoC architecture T(↜R, L, C), where R denotes the set of routers in the synthesized architecture, L represents the set of links between routers, and a function C:V€→€R that represents the connectivity of a core to a router. • A set of ordered paths P, where each pij€∈€P€=€(↜ri, rj,…, rk), ri,…, rk€∈€R, represents a route for a traffic flow e(↜vi, vk)€∈€E. Objective: • The minimization of power consumption for the synthesized 3D-NoC architecture.

8â•… Design of Application-Specific 3D Networks-on-Chip Architectures

175

8.5â•…3D Design Models Power dissipation is a critical issue in 3D circuits due to the increased power density of stacked ICs and the low conductivity of the dielectric layers between the device layers. Therefore, designing custom 3D-NoC topologies that offer low power characteristics is of significant interests. The different power consumption components that are comprised in 3D-NoC topologies are routers, horizontal interconnects that connect modules in the same 2D layer, and the through-silicon vias (TSVs) that connect modules or horizontal interconnects on different layers. We will discuss the details of modelling these components in the following sections.

8.5.1  3D Interconnect Modelling In 3D-NoCs, interconnect design imposes new constraints and opportunities compared to that of 2D NoC designs. There is an inherent asymmetry in the delay and power costs in a 3D architecture between the vertical and the horizontal interconnects due to differences in wire lengths. The vertical TSVs are usually few tens of μm in length whereas the horizontal interconnects can be thousands of μm in length. Consequently, extending a traditional 2D NoC fabric to the third dimension by simply adding routers at each layer and connecting them using vertical vias is not a good option, as router latencies may dominate the fast vertical interconnect. Hence, we explore an alternate option: a 3D interconnect structure that connects modules on different layers as shown in Fig.€8.4a and we derive an accurate model for it. As discussed in Sect.€8.3, the target clock frequency is provided to our 3D-NoC synthesis design flow as a design parameter. However, depending on the network topology, long interconnects may be required to implement network links between routers, which may have wire delays that are larger than the target clock frequency. To achieve the target frequency, repeaters may need to be inserted. In the 2D design problem, interconnects can be modelled as distributed RC wires. One way to optimize the interconnect delay is to evenly divide the interconnect into k segments with repeaters inserted between them that are s times as large as a minimum-sized repeater. When minimizing power consumption is the objective, the optimum size sopt and number kopt of repeaters that minimize power consumption while satisfying the

a

b

Fig. 8.4↜渀 3D interconnect model. a 3D interconnect. b Distributed RC model with repeaters

176

S. Yan and B. Lin

Table 8.1↜渀 Interconnect parameters Interconnect Parameter structure Electrical Horizontal bus Vertical bus

ρ€=€2.53€μΩ€×€cm rh€=€46€Ω/mm ρ€=€5.65€μΩ€×€cm cv€=€600€fF/mm

Physical kILD€=€2.7 ch€=€192.5€fF/mm rv€=€51.2€Ω/mm

w€=€500€nm t€=€1,100€nm w€=€1,050€nm

s€=€500€nm h€=€800€nm Lvia€=€50€μm

delay constraint can be determined for the interconnect [38]. For the 3D interconnect structure, we extended this distributed RC model. As shown in Fig.€8.4b, a 3D interconnect is divided into k segments by repeaters. Among the k segments, k€−€1 segments are part of the horizontal interconnect with the same structure. The other one is a different structure with two horizontal parts connected by a vertical via. The delay and power consumption per bit of this interconnect can be modelled using the Elmore model, as in [16, 38, 39]. In order to take the vertical via into account for the delay and power calculation of the entire interconnect, we first consider the interconnect as k segments with the same structure. We use the methodology described in [38] to find sopt and kopt for an interconnect with specific length to minimize power while satisfying the delay constraint1. After that, the delay and power of each segment are known. Given the fixed length and the physical parameters of the via, the detailed structure of the segment including the via which gives the same delay as the delay of the original segment without via can be determined by properly choosing the length of the horizontal wire parts in this segment. Finally, the total length of the 3D interconnect can be adjusted to the original length by evenly adjusting the length of each segment. Besides deciding the segment structure with vertical via, the via position on the interconnect also needs to be determined. That is, which wire segment is selected to include the via. As an example, in order to determine the influence of via positioning on the delay and power of the entire 3D interconnect, we performed experiments to evaluate the delay and power of an 8€mm 3D interconnect with a via length of 150€μm under different via positions. In the experiments, the physical and electrical parameters in 70€nm technology are used and are listed in Table€8.1. The horizontal wires are implemented on the global metal layers and their parameters are extracted from IRTS [40]. The parameters of vertical vias are obtained from [16]. For the vertical via, the length of 50€μm is assumed for a via that connects adjacent layers. For a 3D interconnect of 8€mm in length, if the target frequency is 1€GHz, then the power optimum solution using the methodology described in [38] is to divide the interconnect into three segments. Thus, there are three possible via positions with three interconnect structures correspondingly, which are shown in Fig.€8.5. The optimization result of each structure together with the result of the interconnect without vertical via (labelled as 2D-wire) are shown in Table€8.2. The differences of delay and power results of all structures relative to the 2D-wire results 1╇ Since inserting TSV adds delay, we tighten the delay constraints by some extent to get valid solutions.

8â•… Design of Application-Specific 3D Networks-on-Chip Architectures

a

b

c

d

177

Fig. 8.5↜渀 Different structures for an 8€mm 3D interconnect. a 2D-wire. b Model A. c Model B. d Model C Table 8.2↜渀 Power and delay comparison of 3D interconnect models Model Power Delay (↜mW╛╛) % diff to 2D-wire (↜ns) 0.3909 0.00 0.1951 2D-wire A 0.4020 2.85 0.1956 B 0.4020 2.85 0.1956 C 0.4020 2.85 0.1956

% diff to 2D-wire 0.00 0.25 0.25 0.25

are also listed. The results show that the influence of the vertical via on the total delay and power consumption of the entire interconnect is very small. The 150€μm via results in 0.25% increase in delay and 2.85% increase in power over the 8€mm interconnect. The results also show that the position of via on the interconnect has little effect on the delay and power. All the structures result in the same total delay and power. Thus, for our 3D-NoC synthesis algorithm, we can safely choose to position the via in the first segment of the interconnect for all 3D interconnects in the synthesized NoC topology for the purpose of computing the interconnect power costs. In our 3D-NoC synthesis design flow, we use the above 3D interconnect model to evaluate optimum power consumption of interconnects with different wire lengths under the given design frequency and delay constraint. These results are provided to the design flow in the form of a library. We emphasize that the focus of this chapter is on new 3D-NoC synthesis algorithms. We readily admit that 3D interconnect optimization is a complex problem and a subject of separate research. New or alternative 3D interconnect models can be easily used with our synthesis algorithms and design flow.

8.5.2  Modelling Routers To evaluate the power of the routers in the synthesized NoC architecture, we extended the router power model in two dimensions to three dimensions. The routers

178

S. Yan and B. Lin

Table 8.3↜渀 Power consumption of routers using Orion [42] Ports (inâ•›×â•›out) 2â•›×â•›2 3â•›×â•›2 3â•›×â•›3 Leakage power (W) Switching bit energy (pJ/bit)

0.0069 0.3225

0.0099 0.0676

0.0133 0.5663

4â•›×â•›3 0.0172 0.1080

4â•›×â•›4 0.0216 0.8651

5â•›×â•›4 0.0260 0.9180

5â•›×â•›5 0.0319 1.2189

are still located on a 2D layer. The ports of routers on the same layer are connected by horizontal interconnects whereas the ports of routers on different layers are connected by 3D interconnects. We use a state-of-the-art NoC power-performance simulator called Orion [41, 42] that can provide detailed power characteristics for different power components of a router for different input/output port configurations. It accurately considers leakage power as well as dynamic switching power. The power per bit values are also used as the basis for the entire router power estimation under different configurations. The leakage power and switching bit energy of some example router configurations with different number of ports in 70€nm technology are shown in Table€8.3.

8.6â•…Design Algorithms In this section, we present algorithms for the 3D topology synthesis process. The entire process is decomposed into the inter-related steps of constructing an initial network topology, rip-up and rerouting flows to design the network topology, inserting the corresponding network links and router ports to implement the routing, and merging routers to optimize network topology based on design objectives. In particular, we propose an algorithm called Ripup-Reroute-and-Router-Merging (RRRM). The details of the algorithm are discussed in this section.

8.6.1  Initial Network Construction The details of RRRM are described in Algorithm 1. RRRM takes a communication demand graph (CDG) and an evaluation function as inputs and generates an optimized network architecture as output. It starts with initializing a network topology by a simple router allocation and flow routing scheme. Then it uses a procedure of rip-up and rerouting flows to refine and optimize the network topology. After that, a router merging step is done to further optimize the topology to obtain the best result. In the initialization, every flow is routed using its own network. To construct the initial network topology, router allocation is considered at each core. A router is allocated to a core if there are more than two flows coming into that core or more than two flows going out from that core. After router allocation, a Routing Cost Graph (RCG) is generated (Algorithm 1 line 2). RCG is a very important graph used in the whole rip-up and reroute procedure of RRRM algorithm.

8â•… Design of Application-Specific 3D Networks-on-Chip Architectures

179

Definition 2╇ The RCG(↜R, E) is a weighted directed complete graph (a full-mesh) with each vertex ri€∈€R represents a router, and each directed edge eij€=€(↜ri, rj)€∈€E from ri to rj corresponds to a connection from ri to rj. A weight w(↜eij) is attached to each edge which represents the incremental cost of routing a flow f through eij. Please note that RCG does not represent the actual physical connectivity between routers and its edge weights change during the whole RIPUP-REROUTE procedure for different flows. Also, the actual physical connectivity between the routers is established during RIPUP-REROUTE procedure, which is explained in the following sections. Before RIPUP-REROUTE, initial network topology is constructed using InitialNetworkConstruction() procedure. Each flow ek€=€(↜sk, dk) in the CDG is routed using a direct connection from router rsk to router rdk , where ri is the router that core i connects to, and the path is saved in path(↜ek). Multicast flows are routed as a sequence of unicast flows from the source to each of their destinations. If either core i is not connected to any router, a direct connection is added between core i and the other router if any. The links and router ports are configured and saved. If a connection between routers can not meet the delay constraints, its corresponding edge weight in RCG is set to infinity. This can be used to guide the rerouting of the flows to use other valid links instead of this one in the RIPUP-REROUTE procedure. As an example, after initial network construction, the connectivity of routers for the example shown in Fig.€8.3a is shown in Fig.€8.6a. In this initial solution, each core is connected to a dedicated router.

180

S. Yan and B. Lin 0.80 0.80

R0

200

R2

R1

R5

100

200

100

200 200 R3

R4

R6

R1

100

0.40

R2

0.75

0.75

0.40 0.25

R3

R4 0.80 0.80

0.90

R5

0.70 0.47

0.35 0.35

400

200

0.75

R0

0.70 0.70

0.72 0.35 0.80 0.80

R6

0.90

a

b 0.40

R2

0.40

R2

R5

R5

0.70 0.75

0.70 0.45

0.75

0.35

0.70

R0

200

100 100 400

R3

R6

d R2

400 200

0.35

R4

R6

0.72

c

R1

0.47 0.35

R4

0.35

0.70

R4

100

R5

400

R0

R2

100

400 200

R6

e

R1

R3 400

300

R5

400

R4

R6 100

f

Fig. 8.6↜渀 Illustration of the RIPUP-REROUTE procedure. a Initial connectivity. b RCG. c MRG. d MRTree. e Connectivity before reroute e7. f Connectivity after reroute e7

8.6.2  Flow Ripup and Rerouting Once the initial network is constructed and the initial flow routing is done, the key procedure of the algorithm—RIPUP-REROUTE procedure is invoked to route flows and find optimized network topology. The details are described in Algorithm 2. In the RIPUP-REROUTE, each multicast routing step is formulated as a minimum directed spanning tree problem. Two important graphs, Multicast Routing Graph (MRG) and Multicast Routing Tree (MRTree), are used to help facilitate the rip-up and rerouting procedure. They are defined as follows.

8â•… Design of Application-Specific 3D Networks-on-Chip Architectures

181

Definition 3╇ Let f be a multicast flow with source s€∈€V and one or more destinations D€⊆€V. i.e., D = {d1 , d2 , . . . , d|D| }, each di€∈€V. A Multicast Routing Graph (MRG) is a complete graph (N , A) defined for f as follows: • N = s ∪ D . • There is a directed arc between every pair of nodes (↜i, j) in N. Each arc ai, j€∈€A corresponds to a shortest path pi, j between the same nodes in the corresponding RCG, pi, j€=€e1€→€e2€→€…€→€ek. • The weight for arc ai, j, w(↜ai, j), corresponds to the path weight of the corresponding shortest path pi, j in RCG. i.e., w(ai, j ) =



w(ei )

ei ∈ p

Definition 4╇ A Multicast Routing Tree (MRTree) is the Minimum Directed Spanning Tree for multicast routing graph (N , A) with s€∈€N as the root. When a flow is rip-up and rerouted, its current path is deleted and the links and router ports resources it occupies are released (line 3). Then based on the current network connectivity and resources occupation, the RCG related to this flow is built and the weights of all edges in RCG are updated (line 4). In particular, for every pair of routers in RCG, the cost of this flow using those routers and the link connecting them is evaluated. This cost depends on the sizes of the routers, the traffic already routed on the routers and the connectivity of the routers to other routers. It also depends on whether an existing physical link will be used or a new physical link needs to be installed. If there are already router ports and links that can support the traffic, the marginal cost of reusing those resources is calculated. Otherwise, the cost of opening new router ports and installing new physical links to support the traffic is calculated. The cost is assigned as edge weight to the edge connecting the pair of routers in RCG. If the physical links used to connect the routers can not satisfy the delay constraints, a weight of infinity is assigned to the corresponding edges in RCG. Once the RCG is constructed, the multicast routing graph (MRG) for the flow is generated from RCG (line 5). MRG is build by including every source and destination router of the flow as its nodes. For each pair of the nodes in MRG, the least cost directed path with least power consumption on RCG is found for the corresponding routers using Floyd-Warshall’s all pair shortest path algorithm and the cost is assigned as edge weight to the edge connecting the two nodes in MRG. Then the Chu-Liu/Edmonds algorithm [36, 37] is used to find the rooted directed minimum spanning tree of MRG with the source router as root. A rooted directed spanning tree of a graph is defined as a graph which connects, without any cycle, all n nodes in the graph with n€−€1 arcs such that the sum of the weight of all the arcs is minimized. Each node, except the root, has one and only one incoming arc. This directed minimum spanning tree is obtained as the multicast routing tree (MRTree) so that the routes of the multicast flow follows the structure of this tree. The details of ChuLiu/Edmonds Algorithm is summarized in Algorithm 3. The multicast routing for

182

S. Yan and B. Lin

flow f in RCG can be obtained by projecting MRTree back to RCG by expanding the correspond arcs to paths. A special case is when f is a unicast flow with source s and destination d. In this case, MRG will just consist of two nodes, namely s and d, and one directed arc from s to d. Therefore, the routing between s and d in RCG is simply a shortest path between s and d. After the path is determined, the routers and links on the chosen path are updated. As an example, Fig.€8.6b shows the RCG for rerouting the multicast flow e7. For clarity, only part of the edges are shown for RCG. The MRG and MRTree for e7 are shown in Fig.€8.6c, d respectively. By projecting MRTree back to RCG, the routing path for e7 is determined, namely e7 is bifurcates in the source router R4 to reach R6 and v6, then it transferred over the network link between R4 to R2 to reach v2, and then bifurcates to reach R5 and v5. The real physical connectivity between routers before and after rip-up and rerouting e7 are also shown in Fig.€8.6e, f. From them, we observe that the link between R4 and R5 and their corresponding ports are saved thus the power consumptions are reduced after rerouting e7 by utilizing the existing network resources for routing other flows. This RIPUP-REROUTE process is repeated for all the flows. The results of this procedure depend on the order that the flows are considered, so the entire procedure can be repeated for several times to reduce the dependency of the results on flow ordering2. Once the path of each flow is decided, the size of each router, the links that connect the routers are determined. Those routers and links constitute the network topology. The total implementation cost of all the routers and links in this topology is evaluated and the network topology is obtained.

2╇ In the experiments, we’ve tried several flow ordering strategies such as largest flow first, smallest flow first, random ordering et al., and we found the ordering of smallest flow first gave the best results. Thus we used this ordering in our experiments. Also, we observed that repeating the whole RIPUP-REROUTE procedure twice is enough to generate good results.

8â•… Design of Application-Specific 3D Networks-on-Chip Architectures

183

8.6.3  Router Merging After the physical network topology has been generated using RIPUP-REROUTE, a router merging step is used to further optimize the topology to reduce the power consumption cost. The router merging step was first proposed by Srinivasan in [43]. Their router merging was based on the distance between routers. However, in this work, we propose a new router merging algorithm for reducing the power consumption of the network and improving the performance. As has been observed, routers that connect with each other can be merged to eliminate router ports and links and thus possibly the corresponding costs. Routers that connect to the same common routers can also be merged to reduce ports and costs. We propose a greedy router merging algorithm, which is shown in Algorithm 4. The algorithm works iteratively by considering all possible mergings of two routers connected with each other. In each iteration, each router’s adjacent routers list is constructed and sorted by the distance between them in increasing order. They are possible candidate mergings. Then the routers are considered to merge in the decreasing order of the number of neighbors they have. For each candidate merging, if the topology from the merging result is valid, the total power consumption of the resulting topology after merging is evaluated using the power models. Routers are merged if they have not merged in this iteration and the cost is improving. After all routers are considered in the current iteration, they are updated by replacing the routers merged with the new one generated. Those routers are reconsidered in the next iteration. The algorithm keeps merging routers until no improvement can be made further. After router merging, the optimized topology is generated and the routing paths of all flows are updated. Since the router merging will always reduce the number of routers in the topology, it will not increase the hop counts for all the flows thus will not worsen the performance of the application.

184

S. Yan and B. Lin

The topology generated after router merging represents the best solution with the minimum power consumption. It is returned as the final solution for our NoC synthesis algorithm.

As an example, the connectivity graphs before and after ROUTER-MERGING procedure for the example of Fig.€8.3a are shown in Fig.€8.7a, b. It is shown that after router merging, the network resources are reduced from four routers to three routers and the total power consumption is reduced as well.

R2

v0

100

400 200 R1 400

a

300

v3

R4

400

100

v0

R2

400 200

100 400

v5

R1

R6

v3 400

300

R4 500

v5

v6

b

Fig. 8.7↜渀 Illustration of the RIPUP-MERGING procedure. a Before router merging. b After router merging

8â•… Design of Application-Specific 3D Networks-on-Chip Architectures

185

8.6.4  Complexity of the Algorithm For an application with |V | IP cores and |E| flows, the initial network construction step needs O(|E|) time. In the rip-up and reroute procedure, each flow is ripped-up and rerouted once. The edge weight calculation for router cost graph takes O(|V |2 ) . For a multicast flow with m destinations, the construction of multicast routing graph takes O((m + 1)2 |V |2 ) by finding shortest path between each pair of nodes. Then it takes O(|V |2 ) to find the rooted directed minimum spanning tree as the multicast tree by using the Chu-Liu/Edmonds algorithm. So the overall complexity of our algorithm is O(|E||V |2 ) .

8.7â•…Deadlock Considerations Deadlock-free routing is an important consideration for the correct operation of custom NoC architectures. In our previous work [9, 10], we’ve proposed two mechanisms to ensure the deadlock-free operation in our NoC synthesis results. In this section, we adopt the same mechanisms into our new Noc synthesis algorithm to ensure deadlock-free in the deterministic routing problem we consider. The first method is Statically Scheduled Routing. For our NoC solutions, the required data rates are specified and the routes are fixed. In this setting, data transfers can be statically scheduled along the pre-determined paths with resource reservations to ensure deadlock-free routing [44, 45]. The second method is Virtual Channels insertion. As shown in [46], a necessary and sufficient condition for deadlock-free routing is the absence of cycles in a channel dependency graph. In particular, we use an extended channel dependency graph construction to find resource dependencies between multicast trees3 and break the cycles by splitting a channel into two virtual channels (or by adding another virtual channel if the physical channel has already been split). The added virtual channels are implemented in the corresponding routers. We applied this method into our NoC synthesis procedure and we have found that virtual channels are rarely needed to resolve deadlocks in practice for custom networks. In all the benchmarks that we tested in Sect.€8.8, no deadlocks were found in the synthesized solutions. Therefore, we did not need to add any virtual channel.

8.8â•…Results 8.8.1  Experimental Setup We have implemented our proposed algorithm RRRM in C++. In our experiment, we aim to evaluate the performance of our proposed algorithm RRRM on bench3╇

This extended channel dependency graph construction treats unicast flows as a special case.

186

S. Yan and B. Lin

marks with the objective of minimizing the total power consumption of the synthesized NoC architectures under the specific performance constraint for the traffic flows. The performance constraint is specified in the form of average hop counts of all the traffic flows in the benchmarks. The total power consumption includes both the leakage power and the dynamic switching power of all network components. As discussed in Sect.€8.5, we use Orion [41, 42] to estimate the power consumptions of router configurations generated. We applied the design parameters of 1€GHz clock frequency, four-flit buffers, and 128-bit flits. For the link power parameters, we use the 3D interconnect models to evaluate the optimum powers for links with different lengths under the given delay constraint of 1€ns. Both routers and links are evaluated using the 70€nm technology and they are provided in a library. All existing published benchmarks are targeted to 2D architectures. However, their sizes are not big enough to take advantage of 3D network topologies. In the absence of published 3D benchmarks with a large number of available cores and traffic flows, we generated a set of synthetic benchmarks by extending the NoCcentric bandwidth-version of Rent’s rule proposed by Greenfield et€al. [47]. They showed that the traffic distribution models of NoC applications should follow a similar Rent’s rule distribution as in conventional VLSI netlists. The bandwidthversion of Rent’s rule was derived showing that the relationship between the external bandwidth B across a boundary and the number of blocks G within a boundary obeys B€=€kGβ, where k is the average bandwidth for each block and β is the Rent’s exponent. We used this NoC-centric Rent’s rule [47, 48] to generate large 3D NoC benchmarks for 3D circuits with varying number of cores in each layer and flows of varying data rate distributions. The average bandwidth k for each block and Rent’s exponent β are specified by the user. In our experiments, we generated large NoC benchmarks by varying k ranging from 100 to 500€kb/s and varying β from 0.65 to 0.75. We formed multicast traffic with varying group sizes for about 10% of the flows. Thus our multicast benchmarks cover a large range of applications with mixed unicast/multicast flows and varying hop count and data rate distributions. The benchmarks are generated for 3D circuits with three layers and four layers respectively with face-to-back bounding between each layer. The total number of cores for these benchmarks are ranging from 48 to 120. The total number of flows are ranging from 101 to 280. Our work is among the first in the area of application-specific NoC synthesis of 3D network topologies. In the absence of previously published work on this area, direct comparison with others’ work is unavailable. To evaluate the effectiveness of our proposed algorithm, we have generated a full 3D mesh implementation for each benchmark for comparisons. In a full 3D mesh implementation, each module is connected to a router with seven input/output ports, with one local port, four ports connecting to the four directions in the same layer, and two ports connecting to the upper and lower adjacent layers. Packets are routed using XYZ routing over the mesh from source to destination. We also generated a variant of the basic mesh topology called optimized mesh (opt-mesh) by eliminating router ports and links that are not used by the traffic flows. All experimental results were obtained on a 1.5€GHz Intel P4 processor machine with 512€MB memory running Linux.

101 133 149 169 177 203 228 248 280

0.497 0.654 0.788 0.850 0.867 1.089 1.083 1.335 1.426

0.25 0.26 0.28 0.26 0.24 0.27 0.24 0.27 0.25

0.50 0.49 0.46 0.51 0.53 0.48 0.44 0.45 0.44

╇ 48 ╇ 60 ╇ 64 ╇ 75 ╇ 80 ╇ 90 100 108 120

B1 B2 B3 B4 B5 B6 B7 B8 B9

3 3 4 3 4 3 4 3 4

Ratio to opt-mesh

Table 8.4↜渀 3D NoC synthesis results Bench. |L| |Cores| |Flows| RRRM Power Ratio to (W) mesh 2.48 2.73 2.05 2.38 2.25 2.65 2.68 2.22 2.59

Avg. hops 0.98 0.99 0.60 0.88 0.82 0.85 0.88 0.63 0.81

Ratio to mesh/opt ╇╇╇╛╛4 ╇╇╛╛15 ╇╇╛╛37 ╇╇╛╛79 ╇╛╛137 ╇╛╛235 ╇╛╛443 ╇╛╛446 1,144

Time (s) 1.951 2.530 2.769 3.269 3.550 4.032 4.599 4.975 5.670

Mesh Power (W) 2.54 2.77 3.40 2.69 2.75 3.11 3.06 3.52 3.21

Avg. hops

0.990 1.326 1.712 1.678 1.637 2.254 2.461 2.979 3.233

Opt-mesh Power (W)

2.54 2.77 3.40 2.69 2.75 3.11 3.06 3.52 3.21

Avg. hops

8â•… Design of Application-Specific 3D Networks-on-Chip Architectures 187

188

S. Yan and B. Lin

8.8.2  Comparison of Results The synthesis results of our algorithm on all benchmarks at 70€nm with comparison to results using mesh and opt-mesh topologies are shown in Table€ 8.4. For each algorithm, the power results and the average hop counts are reported. In the experiments, we used the average hop count results of 3D mesh topologies as the performance constraints feeding to RRRM for each benchmark. The average hop count results for the different benchmarks of 3D mesh topologies reported in Table€8.4 are small, all under 3.5. The average hop count results of RRRM on all benchmarks relative to opt-mesh/ mesh implementation are graphically compared in Fig.€ 8.8b. The results show that the results of RRRM satisfies constraints for all the benchmarks. On average, RRRM can achieve 17% average hop count reduction over the mesh topology.

Power ratio over mesh

1 RRRM opt-mesh mesh

0.8 0.6 0.4 0.2

a

0 B1

B2

B3

B4

B5

B6

B7

B8

B9

Avg. hops ratio over opt/mesh

1

Fig. 8.8↜渀 Comparisons of all algorithms on benchmarks. a Power. b Hop count

0.8 0.6

0.2 0

b

RRRM opt-mesh/mesh

0.4

B1

B2

B3

B4

B5

B6

B7

B8

B9

8â•… Design of Application-Specific 3D Networks-on-Chip Architectures

189

The power consumption results of RRRM and opt-mesh relative to mesh implementations are graphically compared in Fig.€8.8a. The results show that RRRM can efficiently synthesize NoC architectures that minimize power consumption under the performance constraint. It can achieve substantial reduction in power consumption over the standard mesh and opt-mesh topologies in all cases. In particular, it can achieve on average a 74% reduction in power consumption over standard mesh topologies and a 52% reduction over the optimized mesh topologies. The execution times of RRRM are also reported in Table€8.4. The results show that RRRM works very fast. For the largest benchmarks with 120 cores and 280 flows, it can finish within 20€min.

8.9â•…Conclusions In this chapter, we proposed a very efficient algorithm called Ripup-Reroute-andRouter-Merging (RRRM) that synthesizes custom 3D-NoC architectures. The algorithm is based on a ripup-reroute formulation for routing flows to find network topology followed by a router merging procedure to optimize network topology. Our algorithm takes into consideration both unicast and multicast traffic and our objective is to construct an optimized 3D interconnection architecture such that the communication requirements are satisfied and the power consumption is minimized. We also derived accurate power and delay models for 3D wiring. For the network topology derived, the routes for the corresponding flows and the bandwidth requirements for the corresponding network links are determined and the implementation cost is evaluated based on design objective and constraints. Experimental results on a variety of benchmarks using the accurate power consumption cost model show that our algorithms can produce effective solutions comparing to 3D mesh implementations.

References 1. W. J. Dally, B. Towles, “Route packet, not wires: On-chip interconnection networks,” DAC, 2001. 2. L. Benini, G. De Micheli, “Networks on chips: A new SoC paradigm,” IEEE Computer, vol.€35, no.€1, pp.€70–78, Jan. 2002. 3. M. B. Taylor et al., “The RAW microprocessor: A computational fabric for software circuits and general-purpose programs,” IEEE Micro, vol.€22, no.€6, pp.€25–35, Mar./Apr. 2002. 4. K. Sankaralingam et al., “Exploiting ILP, TLP, and DLP with the polymorphous TRIPS architecture,” ISCA, 2003. 5. J. Hu, R. Marculescu, “Energy-aware mapping for tile-based NoC architectures under performance constraints,” ASP-DAC, 2003. 6. S. Murali, G. De Micheli, “Bandwidth constrained mapping of cores onto NoC architectures,” DATE, 2004. 7. K. Srinivasan, K. S. Chatha, G. Konjevod, “Linear-programming-based techniques for synthesis of network-on-chip architectures,” IEEE Transactions on VLSI Systems, vol.€14, no.€4, pp.€407–420, Apr. 2006.

190

S. Yan and B. Lin

╇ 8. S. Murali et al., “Designing application-specific networks on chips with floorplan information,” ICCAD, 2006. ╇ 9. S. Yan, B. Lin, “Application-specific network-on-chip architecture synthesis based on set partitions and Steiner trees,” ASPDAC, 2008. 10. S. Yan, B. Lin, “Custom networks-on-chip architectures with multicast routing,” IEEE Transactions on VLSI Systems, accepted for publication, 2008. 11. K. Lee et al., “Three-dimensional shared memory fabricated using wafer stacking technology,” IEDM Technical Digest, Dec. 2000. 12. L. Xue et al., “Three dimensional integration: Technology, use, and issues for mixed-signal applications,” IEEE Transactions on Electron Devices, vol.€50, pp.€601–609, May 2003. 13. W. R. Davis et al., “Demystifying 3D ICs: The pros and cons of going vertical,” IEEE Design & Test of Computers, vol.€22, no.€6, pp.€498–510, 2005. 14. M. Kawano et al., “A 3D packaging technology for 4Gbit stacked DRAM with 3Gbps data transfer,” IEEE International Electron Devices, pp.€1–4, 2006. 15. J. Kim et al., “A novel dimensionally-decomposed router for on-chip communication in 3D architectures,” ISCA, 2007. 16. V. F. Pavlidis, E. G. Friedman, “3-D topologies for networks-on-chip,” IEEE Transactions on VLSI Systems, vol.€15, no.€10, pp.€1081–1090, Oct. 2007. 17. H. Matsutani, M. Koibuchi, H. Amano, “Tightly-coupled multi-layer topologies for 3-D NoCs,” ICPP, 2007. 18. T. Kgil et al., “PICOSERVER: Using 3D stacking technology to enable a compact energy efficient chip multiprocessor,” ASPLOS-XII, 2006. 19. F. Li et al., “Design and management of 3D chip multiprocessors using network-in-memory,” ISCA, 2006. 20. P. Morrow et al., “Design and fabrication of 3D microprocessor,” Material Research Society Symposium, 2007. 21. W. A. Dees, Jr., P. G. Karger “Automated rip-up and reroute techniques,” DAC, 1982. 22. H. Shin, A. Sangiovanni-Vincentelli, “A detailed router based on incremental routing modifications: Mighty,” IEEE Transactions on CAD of Integrated Circuits and Systems, vol.€CAD6, no.€6, pp.€942–955, Nov. 1987. 23. H. Shirota, S. Shibatani, M. Terai, “A new rip-up and reroute algorithm for very large scale gate arrays,” ICICC, May 1996. 24. J. Cong, J. Wei, Y. Zhang, “Thermal-driven floorplanning algorithm for 3D ICs,” ICCAD, 2004. 25. J. Cong, Y. Zhang, “Thermal-driven multilevel routing for 3-D ICs,” ASPDAC, 2005. 26. B. Goplen, S. Sapatnekar, “Efficient thermal placement of standard cells in 3D ICs using a force directed approach,” ICCAD, 2003. 27. M. Pathak, S. K. Lim, “Thermal-aware Steiner routing for 3D stacked ICs,” ICCAD, 2007. 28. C. Addo-Quaye, “Thermal-aware mapping and placement for 3-D NoC designs,” IEEE International SOC Conference, 2005. 29. X. Lin, P. K. McKinley, L. M. Ni, “Deadlock-free multicast wormhole routing in 2-D mesh multicomputers,” IEEE Transactions on Parallel and Distributed Systems, vol.€ 5, no.€ 8, pp.€793–804, Aug. 1994. 30. M. P. Malumbres, J. Duato, J. Torrellas, “An efficient implementation of tree-based multicast routing for distributed shared-memory,” IEEE Symposium on Parallel and Distributed Processing, 1996. 31. K. Goossens, J. Dielissen, A. Radulescu, “The Ethereal network on chip: Concepts, architectures, and implementations,” IEEE Design & Test of Computers, vol.€22, no.€5, pp.€414–421, 2005. 32. M. Millberg et al., “Guaranteed bandwidth using looped containers in temporally disjoint networks within the Nostrum network on chip,” DATE, 2004. 33. Z. Lu, B. Yin, A. Jantsch, “Connection-oriented multicasting in wormhole-switched networks on chip,” Emerging VLSI Technologies and Architectures (ISVLSI), 2006.

8â•… Design of Application-Specific 3D Networks-on-Chip Architectures

191

34. F. A. Samman, T. Hollstein, M. Glesner, “Multicast parallel pipeline router architecture for network-on-chip,” DATE, 2008. 35. E. A. Carara, F. G. Moraes, “Deadlock-free multicast routing algorithm for wormholeswitched mesh networks-on-chip,” ISVLSI, 2008. 36. Y. J. Chu, T. H. Liu, “On the shortest arborescence of a directed graph,” Science Sinica, vol.€14, pp.€1396–1400, 1965. 37. J. Edmonds, “Optimum branchings,” Research of the National Bureau of Standards, vol.€71B, pp.€233–240, 1967. 38. G. Chen, E. G. Friedman, “Low-power repeaters driving RC and RLC interconnects with delay and bandwidth constraints,” IEEE Transactions on VLSI Systems, vol.€14, no.€2, pp.161– 172, Feb. 2006. 39. L. Zhang et al., “Repeated on-chip interconnect analysis and evaluation of delay, power, and bandwidth metrics under different design goals,” ISQED, 2007. 40. The international Technology roadmap for semiconductors, 2007. 41. H. Wang et al., “Orion: A power-performance simulator for interconnection networks,” MICRO 35, Nov. 2002. 42. X. Chen, L.-S. Peh, “Leakage power modeling and optimization in interconnection networks,” ISPLED, 2003. 43. K. Srinivasan, K. S. Chatha, G. Konjevod, “Application specific network-on-chip design with guaranteed quality approximation algorithms,” ASPDAC 2007, Jan 2007. 44. E. Rijpkema et al., “Trade-offs in the design of a router with both guaranteed and best-effort services for networks on chip,” DATE, 2003. 45. N. Enright-Jerger, M. Lipasti, L.-S. Peh, “Circuit-switched coherence,” IEEE Computer Architecture Letters, vol.€6, no.€1, pp.€5–8, Mar. 2007. 46. W. J. Dally, C. L. Seitz, “Deadlock-free message routing in multiprocessor interconnection networks,” IEEE Transactions on Computers, vol.€C-36, no.€5, pp.€547–550, May 1987. 47. D. Greenfield et al., “Implications of rent’s rule for NoC design and its fault-tolerance,” NOCS, May 2007. 48. D. Stroobandt, P. Verplaetse, J. van Campenhout, “Generating synthetic benchmark circuits for evaluating CAD tools,” IEEE Transactions on CAD of Integrated Circuits and Systems, vol.€19, no.€9, pp.€1011–1022, Sep. 2000.

â•…

Chapter 9

3D Network on Chip Topology Synthesis: Designing Custom Topologies for Chip Stacks Ciprian Seiculescu, Srinivasan Murali, Luca Benini and Giovanni De Micheli

9.1â•…Introduction Today, many integrated circuits contain several processor cores, memories, hardware cores and analog components integrated on the same chip. Such Systems on Chips are widely used in high volume and high-end applications, ranging from multimedia, wired and wireless communication systems to aerospace and defense applications. As the number of cores integrated on a SoC increases with technology scaling, the two-dimensional chip fabrication technology is facing lot of challenges in utilizing the exponentially growing number of transistors. As the number of transistors and the die size of the chip increase, the length of the interconnects also increases. With smaller feature sizes, the performance of the transistors have increased dramatically. However, the performance improvement of interconnects has not kept pace with that of the transistors [1]. With reducing geometries, the wire pitch and cross section also reduce, thereby increasing the RC delay of the wires. This coupled with increasing interconnect length leads to long timing delays on global wires. For example, in advanced technologies, long global wires could require up to 10 clock cycles for traversal [2]. Another major impact of increased lengths and RC values is that the power consumption of global interconnects become significant, thereby posing a big challenge for system designers.

9.1.1  3D-Stacking Recently, 3D-stacking of silicon layers has emerged as a promising solution that addresses some of the major challenges in today’s 2D designs [1, 3–8]. In the 3D C. Seiculescu () Doctoral-assistent in Integrated Systems Laboratory, Swiss Federal Institute of Technology Lausanne (EPFL), EPFL IC ISIM LSI1 INF 339 (Bâtiment INF), Station 14, 1015 Lausanne, Switzerland Tel.: +41 21 693 0916 e-mail: [email protected] A. Sheibanyrad et al. (eds.), 3D Integration for NoC-based SoC Architectures, Integrated Circuits and Systems, DOI 10.1007/978-1-4419-7618-5_9, ©Â€Springer Science+Business Media, LLC 2011

193

194

C. Seiculescu et al.

stacked technology, the design is partitioned into multiple blocks, with each block implemented on a separate silicon layer. The silicon layers are stacked on top of each other. Each silicon layer has multiple metal layers for routing of horizontal wires. Unlike the 3D packaging solutions that have been around for a long time (such as the traditional system-in-package), the different silicon layers are connected by means of on-chip interconnects. The 3D-stacking technology has several major advantages: (1) the foot-print on each layer is smaller, thus leading to more compact chips (2) smaller footprints lead to shorter wires within each layer. Inter layer connections are obtained using efficient vertical connections, thereby leading to lower delay and power consumption on the interconnect architecture (3) allows integration of diverse technologies, as each could be designed as a separate layer. A detailed study of the properties and advantages of 3D interconnects is presented in [1] and [9]. There are several methods for performing 3D integration of silicon layers, such as the Die-to-Die, Die-to-Wafer and Wafer-to-Wafer bonding processes. In the Dieto-Die bonding process, individual dies are glued together to form the 3D-IC. In the Die-to-Wafer process, individual dies are stacked on top of dies which are still not cut from the wafer. The advantages of these processes are that the wafers on which the different layers of the 3D stack are produced can be of different size. Another advantage is that the individual dies can be tested before the stacking process and only “known-good-dies” can be used, thereby increasing the yield of the 3D-IC. In the Wafer-to-Wafer bonding, full wafers are bonded together. The vertical interconnection is usually achieved using Through Silicon Vias (TSVs). For connection from one layer to another, a TSV is created in the upper layer and the vertical interconnect passes through the via form the top layer to the bottom layer. Connections across non-adjacent layers could also be achieved by using TSVs at each intermediate layer. The integration of the different layers could be done with either face to face or face to back topologies [10]. A die’s face is considered to the metal layers and the back is the silicon substrate. The copper half of the TSV is deposited on each die and the two dies are bonded using thermal compression. Typically, the dies are thinned to reduce the distance between the stacked layers. Several researches have addressed 3D technology and manufacturing issues [1, 4, 11]. Several industrial labs, CEA-LETI [12], IBM [13], IMEC [14] and Tezzaron [15], to name a few, are also actively developing methods for 3D integration. In Fig.€ 9.1, we show a set of vertical wires using TSVs implemented in SOI and bulk silicon technologies [11]. We also show the schematic representation of a bundle of TSV vias in Fig.€9.2. In [11], a 4â•›×â•›4€µm via cross section, 8€µm via pitch, 1€µm oxide thickness and 50€µm layer thickness are used. The use of 3D technology introduces new opportunities and challenges. The technology should achieve a very high yield and commercial CAD tools should evolve to support 3D designs. Another major concern in 3D chips is about managing heat dissipation. In 2D chips, the heat sink is usually placed at the top of the chip. In 3D designs, the intermediate layers may not have a direct access to the heat sink to effectively dissipate the heat generated. Several researchers have been working on all these issues and several methods have been proposed to address them. For example, the problem of partitioning and floorplanning of designs for 3D integration has been

9â•… 3D Network on Chip Topology Synthesis

195 Bulk Si

SiO2

Via

Tier n

Tier n+1

Bonding Pad

z

a

x

y

b

Fig. 9.1↜渀 An example set of nine vertical links

Pads

L

HSiO2

SiO2

HBulk

Si-Bulk

W Pitch

tox

Fig. 9.2↜渀 3D bundle cross-section

addressed in [4–8]. Today, several 3D technologies have matured to provide high yield [16]. Many methods have been presented for achieving thermally efficient 3D systems. The methods span from architectural to technology level choices. At the architectural level, works have addressed efficient floorplanning to avoid thermal hot-spots in 3D designs [17]. At the circuit level, use of thermal vias for specifically conducting heat across the different silicon layers has been used [4]. In [13], use of liquid cooling across the different layers is presented.

9.1.2  Networks on Chips for 3D ICs One of the major challenges that designers face today in 3D integration is how to achieve the interconnection across the components within a layer and across the layers

196

C. Seiculescu et al.

in a scalable and efficient manner. The use of Networks on Chips (NoCs) has emerged as the solution to the 3D integration problem. The NoC paradigm has recently evolved to address the communication issues on a chip [2, 18, 19]. NoCs consist of switches and links and use circuit or packet switching technology to transfer data inside a chip. NoCs have several advantages, including achieving faster design closure, higher performance and modularity. An example NoC architecture is shown in Fig.€9.3. A NoC consists of set of switches (or routers), links and interfaces that packetize data from the cores. A detailed introduction to NoC principles can be found in [19]. NoCs differ from macro-networks (such as the wide area networks) because of local proximity and predictable behavior. The on-chip networks should have low communication latency, power consumption and could be designed for particular application traffic characteristics. Unlike a macro-network, the latency inside a chip should be in the order of few clock cycles. Use of complex protocols will lead to large latencies, NoCs thereby require streamlined protocols. Power consumption is a major issue for SoCs. The on-chip network routers and links should be highly power efficient and occupy low area. The use of NoCs in SoCs has been a gradual process, with the interconnects evolving from single bus structures to multiple buses with bridges, crossbars and a packet-switching network. Compared to traditional bus based systems, a network is clearly more scalable. Additional bandwidth can be obtained by adding more switches and links. Networks are inherently parallel in nature with distributed arbitration for resources. Thus, multiple transactions between cores take place in parallel in different parts of a NoC. Whereas, a bus-based system use centralized arbitra-

IP core master

NI

NI

IP core master

switch

switch

NoC topology

switch NI

switch IP core slave

NI

IP core slave

Fig. 9.3↜渀 Example NoC design

NI

NI

IP core slave

IP core master

9â•… 3D Network on Chip Topology Synthesis

197

tion, thereby leading to large congestion. Also, the structure and wiring complexity can be well controlled in NoCs, leading to timing predictability and fast design closure. The switches segment long global wires and the load on the links are smaller, due to their point-to-point nature. NoCs are a natural choice for 3D chips. A major feature of NoCs is that a large degree of freedom is available that can be exploited to meet the requirements. For example, the number of wires in a link (i.e. the link data width) can be tuned according to the application and architecture requirements. The data can be efficiently multiplexed on a small set of wires if needed. This is unlike bus based systems, that require several set of wires for address, data and control. Thus, communication across layers can be established with fewer vertical interconnects and TSVs. NoCs are also scalable, making the integration of different layers easy. Several different data streams from different sources and destinations can be transferred in parallel in a NoC, thereby increasing performance. The combined use of 3D integration technologies and NoCs introduces new opportunities and challenges for designers. Several researchers are working on building NoC architectures for 3D SoCs. Router architectures tuned specifically for 3D technologies have been presented in [20] and [21]. Using NoCs for 3D multi-processors has been presented in [22]. Cost models for 3D NoCs, computed analytically has been presented in [23]. Designing regular topologies for 3D has been addressed in [24].

9.1.3  Designing NoCs for 3D ICs Designing NoCs for 3D chips that are application-specific, with minimum powerdelay is a major challenge. Successful deployment of NoCs require dedicated solutions that are tailored to specific application needs. Thus, the major challenge will be to design hardware-optimized, customizable platforms for each application domain. The designed NoC should satisfy the bandwidth and latency constraints of the different flows of the application. Several works have addressed the design of bus based and interconnect architectures for 2D ICs [25, 26]. Several methods have addressed the mapping of cores on to NoC architectures [27–31]. Custom topology synthesis for 2D designs has been addressed in [32–39]. Compared to the synthesis of NoCs for 2D designs, the design for 3D systems present several unique challenges. The design process needs to support the constraints on the number of TSVs that can be established across any two layers. In some 3D technologies, only connections across adjacent layers can be supported, which needs to be considered. Finally, the layer assignment and placement of switches in each layer need to be performed. The yield of a 3D IC can be affected by the number of TSVs used, depending on the technology. In Fig.€9.4, we show how the yield for different processes vary with the number of TSVs used across two layers. The graphs show a trend that after a threshold, the yield decreases with increasing number of TSVs. Thus, the topology

198

C. Seiculescu et al.

120% (HRI-JP) DBI=9.75E-6

(HRI-JP)

100%

Yield [%]

80% (IMEC) DBI=40.0E-6

60%

(IMEC) (IBM) DBI=13.9E-6

40%

HRI-JP IMEC (IEDM2006) IBM/SOI (IEDM2005)

20% 0%

1

10

100 1000 10000 Number of TSV [piece/chip]

(IBM) 100000

1E+06

Fig. 9.4↜渀 Yield vs. TSV count

synthesis process should be able to design valid topologies meeting a specific yield and hence a TSV constraint. Moreover, with increasing TSV count, more area needs to be reserved for the TSV macros in each layer. Thus, to reduce area, a bound on allowed TSV is important. In [3], the pitch of a TSV is reported to be between 3 and 5€m. Reserving area for too many TSVs can cause a considerable reduction in the active silicon area remaining for transistors. The TSV constraint can significantly impact the outcome of the topology synthesis process. Intuitively, we can see that when more TSVs are permitted, more vertical links (or larger data widths) can be deployed. The resulting topologies could have lower latencies, while using more area for the TSVs. On the other hand, a tight TSV constraint would force fewer inter-layer links, thereby increasing congestion on such links and affecting performance. In Figs.€9.5 and 9.6, we show the best topologies synthesized by our flow for the same benchmark for two different TSV constraints. In the first case 13 interlayer links are used and in the second only eight inter-layer links are used. Building power-efficient NoCs for 3D systems that satisfy the performance requirements of applications, while satisfying the technology constraints, is an important problem. To address this issue, new architectures and design methods are needed. In this chapter, we present synthesis methods for designing NoCs for 3D ICs. The objective of the synthesis process is to obtain the most power efficient topology for the NoC for the 3D design. The process has to meet the 3D design constraints and the application performance requirements. We also take the floorplan of each layer of the 3D design, without the interconnect, as an optional input to the design process. In our design process, we also compute the position of the switches in the floorplan and place them, while minimally perturbing the position of the other cores. We apply our methods to several SoC benchmarks which show large power and latency improvements when compared to the use of standard topologies.

9â•… 3D Network on Chip Topology Synthesis

199

Smm 31

Switch 15

Int 32

Switch 16

Switch 14

ShM 30

PrM 22 PrM 21

Switch 9

PrM 25 PrM 26

Switch 11 PrM 24

Switch 13

PrM 29 PrM 16 PrM 15

PrM 28 PrM 17

Switch 8

Switch 10

Switch 5

Switch 1

Core 4

PrM 19

Switch 12

PrM 18

Switch 6

Core 5 Core 14

PrM 27

Switch 7

PrM 23

PrM 20

Core 2

Core 9 Core 8

Core 3 Switch 3

Core 1

Core 0

Switch 2 Core 11

Core 10

Switch 4 Core 6

Core 7

Switch 0 Core 13

Core 12

Fig. 9.5↜渀 Topology using 13 inter-layer links

While many works have addressed the architectural issues in the design of NoCs for 3D ICs, relatively fewer works have focused on the design aspects. In [17], the authors have presented methods for mapping cores on to 3D NoCs with thermal constraints. We have presented our methods to design NoCs for 3D ICs in [40, 41]. We also make a comparative study of NoC designs for the corresponding 2D implementation of the benchmarks. The objective is to evaluate the actual power and latency advantages when moving to a 3D implementation. For this study, we apply a flow developed by us earlier for NoC design for 2D systems [39]. Our results show that a 3D design can significantly reduce the interconnect power consumption (38% on average) when compared to the 2D designs. However, the latency savings is lower (13% on average), as the number of additional links that require pipelining in 2D were few. We use TSVs to establish the vertical interconnections. In Fig.€9.7, an example of how a vertical link is established across two layers is presented. In our architecture, the TSV needs to be drilled only on the top layer, and the interconnect uses horizontal metal at the bottom layer. In our synthesis flow, we allocate area for a TSV macro at the top layer for the link during the floorplanning phase. The TSV macro is actually placed directly at the output port of the corresponding switch. For links that go across multiple silicon layers, we also place TSV macros in each intermediate layer. In Fig.€9.8, we show an example link that spans multiple layers.

200

C. Seiculescu et al. Smm 31

Switch 15

Int 32

Switch 16

Switch 14

ShM 30

PrM 21 PrM 22

Switch 11 PrM 25

PrM 23 Switch 9

Switch 13

PrM 24

PrM 26 PrM 27 Switch 7 PrM 28

PrM 29 Switch 8

PrM 16 PrM 15

Switch 10 PrM 17

Switch 12

PrM 18 PrM 20 PrM 19

Core 9

Core 5

Core 3 Core 8

Core 4 Switch 1

Core 14

Core 0

Core 1

Switch 5

Switch 6

Switch 2

Switch 4

Core 11

Core 10

Core 6

Core 7

Core 2 Switch 3

Switch 0

Core 13

Core 12

Fig. 9.6↜渀 Topology using eight inter-layer links

9.2â•…3D Architecture and Design Flow If there is a core that is connected to a switch that is in another layer below the core layer, then the network interface that translates the core communication protocol to the network protocol will be the one that will contain the necessary TSV macros. If there are intermediate layers among the core’s network interface and the switch it is connected to then TSV macros will be added in the intermediate layers just as in the

9â•… 3D Network on Chip Topology Synthesis

201

TSV macro Core

Switch Layer 1 vertical link

horizontal links Switch

Core

Layer 0

Fig. 9.7↜渀 Example vertical link

switch Core

TSV macro

Layer 2 vertical link

horizontal link

TSV macro

Layer 1 switch Core

Layer 0

Fig. 9.8↜渀 Example vertical link

case of the switch to switch link from Fig.€9.8. Active silicon area is lost every time a TSV macro is placed as the area reserved by the macro will be used to construct the TSV.

9.3â•…Design Flow Assumptions In the design approach we use several realistic assumptions. • The computation architecture is designed separately from the communication architecture. Several works (such as [42]) have shown the need to separate compu-

202

C. Seiculescu et al.

tation from communication design to tame the complexity. We assume hardware/ software partitioning of the design has been performed and tasks are statically assigned to the cores. For the communication architecture design, we assume that the hardware/software partition of application tasks onto the processor/hardware cores has been performed. • The assignment of the cores to the different layers of the 3D are performed using existing methods/tools. There have been several works that address this issue and our work is complementary to them. • The floorplan of the cores in each layer (without the NoC) has been performed by existing methods. We use the floorplan estimates as inputs to our design flow to better estimate wire delay and power consumption. • Though the synthesis methods presented in this chapter are general and applicable to wide range of NoCs, we use a particular architecture ([43]) to validate the approach.

9.4╅Design Approach Our design flow for topology synthesis is presented in Fig.€9.9. The topology synthesis procedure produces a set of valid design points that meet the application performance and 3D technology constraints, with different power, delay and area values. From the Pareto curves, the designer can then choose the best design point.

Communication characteristics

Application bandwith requirements Latency constraints Message type of traffic flows Communication specification file

NoC area, power and timing models

Vertical link power, latency models

Fig. 9.9↜渀 Design flow

3D Specs

Technology constraints

Core assignment to layer in 3D Optionally, floorplan of cores in each layer

Max. # TSVs across adjacent layers Constraint on links only across adjacent layers

Core specification file

3D NoC Topology Synthesis

Application-specific 3D NoCs

9â•… 3D Network on Chip Topology Synthesis

203

The placement and floorplan of the switches and TSVs on each layer is also produced as an output. The different steps in the topology synthesis process are outlined as follows. In the outer most loop, the architectural parameters, such as the NoC frequency of operation, data width are varied and for each architectural point the topology synthesis process is repeated. Then, the number of switches in the design are varied. When fewer switches are used to connect the cores, the size of each switch is large and the inter-switch links are longer. However, the packets traverse a shorter path. On the other hand, when more switches are used, the size of each switch is smaller, but packets travel more hops. Depending on the application characteristics, the optimal power point in terms on the number of switches used varies. Hence, our synthesis tool sweeps a large design space, building topologies with different switch counts. For a chosen architectural point and switch count, we establish connectivity across the cores and switches. Then, we also determine the layer assignment for each of the switches. If there is a tight constraint on the TSVs or when the design objective is to minimize the number of vertical connections, a core in a layer can be constrained to be connected to a switch in the same layer. In this way, core to switch links will not require vertical connections. On the other hand, this will require inter-layer traffic flows to traverse at least two switches, thereby increasing latency and power consumption. This choice is application-specific and should be chosen carefully. To address this issue, we present a two-phase method for establishing core to switch connectivity. In the first phase, a core can be connected to a switch in any layer, while in the second phase, cores can be connected to only those switches in the same layer. The second phase is invoked when TSV constraints are not met in Phase 1 or when the objective is to minimize the number of vertical interconnections used. These phases are explained in detail in the next sections. Several inputs are obtained for the topology synthesis process. The name, size and position of the different cores, assignment of cores to the 3D layers, the bandwidth and latency constraints of the different flows are obtained. A constraint on the number of TSVs that can be established is also taken as an input. In some 3D technologies, vertical interconnects can established only across adjacent layers. This is also taken as an input. We model the maximum TSV constraint by constraining the number of NoC links that can be established across adjacent layers. We denote this by max_ill. For a chosen link width, the value of max_ill can be computed easily from the TSV constraint. For the synthesis procedure, the power, area and timing models of the NoC switches and links are also taken as inputs. For the experimental validation, we use the library of network components from [43] and the models are obtained from layout level implementations of the library components. The design process is general and models for other NoC architectures can also be easily integrated with the design flow. The delay and power consumption values of the vertical interconnects are also taken as inputs. We use the models from [11] for the vertical interconnects.

204

C. Seiculescu et al.

9.5â•…Algorithm We will now go on to describe the algorithm for synthesizing applications specific Noc topologies for 3D ICs. We will start by formally defining the inputs to the algorithm. The first input is the core specification which describes the number of cores, their position and the layer assignment. The core specification is defined as follows: Definition 1╇ For a design with n cores. The x and y co-ordinate positions of a core i are represented by xci and yci respectively, ∀i ∈ 1 · · · n. The assignment of core i to the 3D layer represented by layeri. The second input is the communication specification which describes the communication characteristics of the application and it is represented by a graph [28, 30, 31]. The graph is defined as follows: Definition 2╇ The communication graph is a directed graph, G(↜V, E) with each vertex vi∈V representing a core and the directed edge (↜vi, vj) representing the communication between the cores vi and vj . The bandwidth of traffic flow from cores vi to vj is represented by bwi, j and the latency constraint for the flow is represented by lati, jâ•›. Setting of several NoC architectural parameters can be explored, like the frequency at which the topology has to run and the data width of the links. The ranges in which the design parameters are varied are taken as inputs. The algorithm sweeps the parameters in steps, and designs the best topology points for each setting. For each architectural point, the algorithm performs the steps shown in Algorithm 1. The algorithm will create a list of all the switch counts for which topologies will be generated (steps 2–5). By default, the switches are varied from one to the maximum number of cores in the design or in each layer. However, the designer can also manually set the range of switch counts to be explored. The objective function of topology synthesis is initially set to minimize power consumption. However, for each topology point, if the 3D technology constraints are not met, the objective function is slowly driven to minimize the number of vertical interconnections. For this purpose, we use the scaling parameter θ. To obtain designs with lower inter-layer links, θ is varied from θmin to θmax in steps of θscale, until the maximum number of inter-layer links constraints is met. After several experimental runs, we determined that varying θ from 1 to 15 in steps of 3 gives good results. In step 7, the algorithm tests if inter-layer links can cross multiple layers, and if not, then phase one is skipped and phase 2 is used directly. In step 8, the parameter θ used for setting the importance of the 3D constraints is set to the minimum value to try to optimize for power. The function to build topologies is called in step 10 on the initial list of switch counts to be explored. A detailed description of the BuildTopologyGeneral is given in the Sect.€9.5.1. If the Unmet set is not empty, then some topology points may not have met the technology constraints. Thus, θ is increased and the function is called again. Phase 2 of the algorithm detailed in Sect.€9.5.2 is more restricted, as cores can only be connected to switches in the same layer of the 3D stack. Topologies built

9â•… 3D Network on Chip Topology Synthesis

205

using this restriction are usually consume more power, as more switches are required. Also the average hop count increases, as inter-layer flows traversing different layers have to go through at least two hops. The advantage of the method from phase 2 is that it can build topologies with a very tight constraint on the number of inter-layer links. In step 15, the algorithm tests if there are entries in the Unmet set for which topologies were not built in phase 1. This could be either because the constraints on the maximum number of inter-layer links were too tight or because the technology did not allow for inter-layer links to cross more than one layer and phase one was skipped completely. If Unmet is not empty then in step 16, the algorithm calls BuildTopologyLayerByLayer function which tries to build topologies using the restrictive approach.

9.5.1  Phase 1 Since different switch counts are explored and the number of switches rarely equals the number of cores, the first problem that arises is to decide how to connect the cores to switches. The assignment of cores to switches can have a big impact on the power consumption, but also on the number of inter-layers links required as switches from different layers can be assigned to the same switch. As multiple cores have to be assigned to the same switch, we partition the cores in as many blocks as there are switches. For this, we define the Partitioning Graph as follows: Definition 3╇ The partitioning graph is a directed graph, PG(↜U, H, α), that has same set of vertices and edges as the communication graph. The weight of the edge (↜ui, uj), defined by hi, j, is set to a combination of the bandwidth and the latency constraints of the traffic flow from core ui to uj: hi,j = α × bwi,j /max_bw + (1 − α) × min_lat/lati,j ,

206

C. Seiculescu et al.

where max_bw is the maximum bandwidth value over all flows, min_lat is the tightest latency constraint over all flows and α is a weight parameter.

The weights on the edges in the partitioning graph are calculated as a linear combination of the bandwidth required by the communication flow and the latency constraint. The parameter α can be used to make trade-offs between power and latency. The intuition is that when α is large, cores that have high bandwidth communication flows will be assigned to the same switch. This will minimize the switching activity in the NoC and therefore reduce the power consumption. On the other hand, when α is small, cores that have tight latency constraints will be assigned to the same switch minimizing the hop count. The parameter α is given as input or can be varied in a range as well to explore the trade-offs between power consumption and latency. However, the partitioning graph has no information on the layer assignment of the cores and cannot be used if the number of inter-layer links has to be reduced. For this purpose, we define the it Scaled partitioning Graph: Definition 4╇ A scaled partitioning graph with a scaling parameter θ, SPG(↜W, L, θ), is a directed graph that has the same set of vertices as PG. A directed edge li, j exists between vertices i and j, if â•› ∃(ui , uj ) ∈ P or layeri€=€layerjâ•›. In the scaled partitioning graph, the edges that connect vertices that correspond to cores that are in different layers are scaled down. This way cores that are on dif-

9â•… 3D Network on Chip Topology Synthesis

207

ferent layers will be assigned to different switches. This can lead to a reduction in the inter-layer links, because the links that connect switches can be reused by many flows while links that connect cores to switches can only be used by the communication flows of that core. As the parameter θ scales, to drive the partitioner to cluster cores that are on the same layer, edges between the vertices that correspond to cores in the same layer are added. It is important that these edges have lower weight than the real edges. If too much weight is given to the new edges, then the clustering is no more communication based and it will lead to an increase in the power consumption. Equation 9.1 shows how the weights are calculated in the SPG. The weight from the new edges is calculated based on the maximum weight of the edge in the PG and it is denoted by max_wt.



li,j =

 if (ui , uj ) ∈ PG & layeri = layerj hi,j    h  i,j    θ × |layer − layer | if (ui , uj ) ∈ PG & layeri = layerj i

θ × max_wt        10 × θmax 0

j

if (ui , uj ) ∈ / PG & layeri = layerj

(9.1)

otherwise

From the definition, we can see that the newly added edges have at most one-tenth the maximum edge weight of any edge in PG, which was obtained experimentally after trying several values. The steps of the BuildTopologyGeneral function are presented in Algorithm 2. In the first step, the partitioning graph is build. If θ is larger than the initial value (step 2), it means that feasible topologies could not be built for all switch counts using the core to switch assignment based on power and latency only. Therefore in step 3, the scaled partitioning graph is built from the partitioning graph using the current value of θ and replaces the partition graph in step 4. The design points from the Unmet set are explored in step 7. For each switch count that is explored, the cores are partitioned in as many blocks as the value of the switch count for the current point (step 8). Once the cores are connected to switches, the switch layer assignment can be computed. Switches are assigned to layers in the 3D stack based on the layer assignment of the core it connects to. A switch is placed at the average distance in the third dimension among all the cores it connects (steps 11–13). For the current core to switch assignment, the inter-switch flows have to be routed (steps 14, 15). The function CheckConstraints(↜cost) enforces the constraints imposed by the upper bound on inter-layer links. A more detailed description on how the constraints are enforced and how the routes are found is provided in Sect.€9.5.3. If paths for all the inter switch flows were found with the given constraints, then the topology for the design point is saved and the entry corresponding to the current switch count is removed from the Unmet set (steps 18 and 19).

208

C. Seiculescu et al.

9.5.2  Phase 2 As previously stated, phase 2 is more conservative in the sense that cores can only be connected to switches in the same layer. To ensure that the blocks that result from the partitioning do not contain cores that are assigned to different layers on the 3D stack, the partitioning is done layer by layer. To do a layer by layer partitioning, we define the Local Partitioning Graph as follows: Definition 5╇ A local partitioning graph, LPG(↜Z, M, ly), is a directed graph, with the set of vertices represented by Z and edges by M. Each vertex represents a core in the layer ly. An edge connecting two vertices is similar to the edge connecting the corresponding cores in the communication graph. The weight of the edge (↜mi, mj), defined by hij, is set to a combination of the bandwidth and the latency constraints of the traffic flow from core mi to mjâ•›: hi,j = α × bwi,j /max_bw + (1 − α) × min_lat/lati,j , where max_bw is the maximum bandwidth value over all flows, min_lat is the tightest latency constraint over all flows and α is a weight parameter. For cores that do not communicate with any other core in the same layer, edges with low weight (close to 0) are added between the corresponding vertices to all other vertices in the layer. This will allow the partitioning process to still consider such isolated vertices. A local partitioning graph is built for each layer and the partitioning and therefore the core to switch assignment is done layer by layer. Another restriction is imposed on the switches, as they can be connected to other switches in the same layer, but only to switches that are in adjacent layers when it comes to connections in the third dimension. Since there can be different number of cores on the different layers, it is important to be able to distribute the switches to the layers of the 3D stack unevenly. Therefore, the algorithm in phase 2 starts by determining the minimum number of switches in each layer necessary to connect the cores. The operating frequency determines the maximum size of a switch, as the critical path in a switch depends on the number of input/output ports. The maximum switch size is determined in step 1 based on switch frequency models given as inputs. In steps 2–5, the minimum number of switches in each layer is determined from the number of cores in the layer and the maximum supported size for the switches at the desired operating frequency. In step 5, the local partitioning graphs are built, one per layer. Then for each design point remaining in the Unmet set, the algorithm distributes the switches on the different layers (step 8). Then we calculate the actual number of switches to be used in each layer, starting from the minimum number of switches in each layer previously calculated (steps 12–16). We also makes sure that the number of switches in each layer does not grow beyond the number of cores. For the calculated switch count on each layer, the local partitioning graphs are partitioned in as many blocks as the switch count (step 17). Once the cores are assigned to switches, the CheckConstraints(↜cost) is called to enforce the routing constraints and paths are found for the inter switch flows (steps 19, 20). If paths are found for all flows, the topology for the design point is saved.

9â•… 3D Network on Chip Topology Synthesis

209

9.5.3  Find Paths When routing the inter switch flows, new physical links have to be opened between switches as in the beginning the switches are not connected among themselves. To establish the paths and to generate the physical connectivity for the inter switch flows, a similar procedure is used as in the 2D case [39]. The procedure finds minimum cost paths and the cost is based on the power increase generated by routing the new flow on that path. By using marginal power as the cost metric, the algorithm minimizes the overall power consumption. The full description of finding paths is beyond the scope of this work and we refer the reader’s attention to [39]. The work also shows how to find deadlock free routes in the design, which can also be used in 3D. However, in 3D we must take care of the maximum number of inter-layer links constraint (↜max_ill) together with the constraint on the maximum switch size imposed by the operating frequency. Therefore, in this section we will focus on how these constraints can be enforced by modifying the cost on which the paths are calculated. The routine to check and enforce the constraints is presented in Algorithm 4. Before describing the algorithm we have to make the following definitions:

210

C. Seiculescu et al.

Definition 6╇ Let nsw be the total number of switches used across all the layers and let layeri be the layer in which switch i is present. Let ill(↜i, j) be the number of vertical links established between layers i and j. Let the switch_size_inpi and switch_size_outi be the number of input and output ports of switch i. Let costi, j be the cost of establishing a physical link between switches i and j.

In the algorithm, we use two types of threshold. One type refers to hard thresholds that are given by the constraints and the other type is the soft thresholds which are set to be just a bit less than the hard constraints. While violating a hard threshold means that it is impossible to build a topology, soft thresholds can be violated. These allow the algorithm to reserve the possibility to open new links for special flows that otherwise cannot be routed due to other constraints (e.g. to enforce deadlock freedom). The algorithm tests if a link can be opened between every pair (↜i, j) of switches (steps 3, 4). First, the constraint on the number of vertical links are checked. In the case of phase 2, when inter-layer links cannot cross multiple layers and the distance in the third dimension between switch i and switch j is larger than 1, then the cost for that pair is set to INF. Also if the number of inter-layer links between the layer containing switch i and switch j reached the max_ill value, then the cost is also set to INF (steps 7, 8). By setting the cost to INF, we make sure that when finding paths, we will not open a new link between switch i and j. If only the hard threshold is used, then the algorithm would be able open links until reaching the limit and abruptly hit

9â•… 3D Network on Chip Topology Synthesis

211

an infeasible point. In a similar manner to the hard constraints, the soft constraints are enforced by setting the cost to SOFT_INF when the number of inter-layer links is already close to the hard constraint (steps 9, 10). The SOFT_INF value is chosen to be several orders of magnitude larger than the normal cost based on power. The constraints to limit the size of the switches are very similar to the constraints on the maximum number of inter-layer links and are enforced in steps 11–15. When paths are computed, if it is not feasible to meet the max_switch_size constraints, we introduce new switches in the topology that are used to connect the other switches together. These indirect switches help in reducing the number of ports needed in the direct switches. Due to space limitations, in this chapter, we do not explain the details of how the indirect switches are established. If we look at Algorithm 1, we can see that many design points are explored, especially when the constraint on the maximum number of inter-layer links is tight. Several methods can be employed to stop the exploration of design points when it becomes clear that a feasible topology cannot be built. To prune the search space, we propose three strategies. First, as the number of input/output ports of a switch increases, the maximum frequency of operation that can be supported by it reduces, as the combinational path inside the crossbar and arbiter increases with size. For a required operating frequency of the NoC, we first determine the maximum size of the switch (denoted by max_sw_size) that can support that frequency and determine the minimum number of switches needed. Therefore, design points where the switch count is less than that can be skipped. Second, for phase 2 we initialize the number of switches layer by layer as above. Thus, the starting design point can have different number of switches in each layer. The third strategy is applied after partitioning. The number of inter-layer links used to connect the cores to the switches is evaluated, before finding the paths. If the topology requires more inter-layer links than the threshold, we directly ignore the design point.

9.5.4  Switch Position Computation In modern technology nodes, a considerable amount of power is used to drive the wires. To be able to evaluate more accurately the power consumption of a topology point, we have to estimate the power used to drive the links. In order to evaluate the lengths of the links, we have to find positions for the switches and to place them in the floorplan. While the positions of the cores is given as input, the switches are added by the algorithm and their positions has to be calculated. Since a switch is connected to several cores, one way to calculate the switch position is to minimize the distance between the switch and the cores it is connected to. This can easily be done by averaging the coordinates of the cores the switches is connected to. This is a simple strategy and can provide good results and can be further improved by weighing the distance between the switch and cores with the bandwidth that the core generates, so that links that carry more bandwidth would be shorter. However, this strategy does not take into consideration that the switch can be connected to

212

C. Seiculescu et al.

other switches as well and minimizing the distance between switches is desirable. To achieve this, a strategy that uses a linear program formulation that minimizes the distance between the cores and switches at the same time is presented in [40]. If an inter-layer link crosses more than one layer then macros have to be placed on the floorplan to reserve space to create the TSVs. However, finding the position for TSV macros is much easier because the TSV macro is connected between only two components (core to switch or switch to switch). Therefore the TSV macro can be placed anywhere in the rectangle defined by the two component as it would not increase the wire length (Manhattan distance is considered). Placing the switches and TSV macros at the computed position may result in overlap with the existing cores. For most real designs, moving the cores from their relative positions is not desirable as there are many constraint to be satisfied.

CONT2

IMPRO2

100

0

20

20

FLASH

10

20

20

CONT1

40 20

ARM

IMPRO1

SD RAM1

L2CC

100 100

20

100

20

100 P1

200

MEM5

P2

L2C

20 100 DSP L2CC

DSP

DMA

400

20

40

0

MEM3

SD RAM2

DSP L2C

DEBUG

40

0 20

10

100

40

40

40

MEM4

200 20

20

20

0

20

0 20

Fig. 9.10↜渀 D26_media communication graph

100

MEM2

MEM1

PE3

PE2

P3

MEMPE1

9â•… 3D Network on Chip Topology Synthesis

213

A standard floorplanner can be used, but it can produce poor results if it is not allowed to swap the cores. A custom routine designed to insert the NoC component in the existing floorplan can give better results removing the overlap. The routine considers one switch or TSV macro at a time. It tries to find a free space near its ideal location to place it. If no space is available, we displace the already vld 0 70

run le dec 0

inv scan 0 362

acdc pred 0

49

353

27 srtipe mem 0

samp up 0 300

313

126

pad 0 94

313

vop rec 0

540

vop mem 0

ARM 0 16

357 362

362

idct 0

iquam 0

500 run le dec 1

iquam 1

357

12 inv scan 1 362

acdc pred 1

27

49

srtipe mem 1

353

126

94

mem out

samp up 1 300

313 vop mem1

ARM 1 0

mem in

362

16

54

362

idct 1

pad 1

313

vop rec 1

540

70

6

vld 1

500

vld 2

70

run le dec 2 362

362 inv scan 2

362

iquam 2

acdc pred 2

49

357

353

27

samp up 2

srtipe mem 2

300

313 vop mem 2

94

pad 2

500

Fig. 9.11↜渀 D38_tvopd communication graph

idct 2

313

vop rec 2

16

ARM 2

M10

M7

ARM1

M11

M6

ARM0

ARM2

M8

M9

ARM3

Fig. 9.12↜渀 D36_4 communication graph

Layer 0

ARM4

ARM5

ARM12

M18

M23

ARM17

Layer 1

ARM13

M19

M22

ARM16

ARM14

M20

M21

ARM15

400 MB/s

ARM24

M30

M35

ARM29

300 MB/s

Layer 2

ARM25

M31

M34

ARM28

100 MB/s

ARM26

M32

M33

ARM27

214 C. Seiculescu et al.

9â•… 3D Network on Chip Topology Synthesis

Switch power Core-to-switch link power Switch-to-switch link power Total power

70 60 Power consumption (mW)

215

50 40 30 20 10 0

4

6

8

10

12

14 16 Switch count

18

20

22

24

26

14 16 Switch count

18

20

22

24

26

Fig. 9.13↜渀 Power consumption in 2D

Switch power Core-to-switch link power Switch-to-switch link power Total power

70

Power consumption (mW)

60 50 40 30 20 10 0

4

6

8

10

Fig. 9.14↜渀 Power consumption in 3D

12

216

C. Seiculescu et al.

25

2D 3D Layer-by-layer

Number of wires

20

15

10

5

0

0–1

1–2

2–3

3–4

4–5 5–6 Wire length

6–7

7–8

8–9

9–10

Fig. 9.15↜渀 Wire length distributions

Relative power consumption

2.5

3D Application specific 3D Layer-by-layer

2

1.5

1

0.5

0

D_36_4

D_36_6

D_36_8

D_35_bot D_65_pipe D_38_tvopd

Fig. 9.16↜渀 Comparison with layer-by-layer

placed blocks from their positions in the x or y direction by the size of the component, creating space. Moving a block to create space for the new component can cause overlap with other already placed blocks. We iteratively move the necessary blocks in the same direction as the first block, until we remove all overlaps. As more components are placed, they can re-use the gap created by the earlier components (See Figs. 9.10 to 9.18).

9â•… 3D Network on Chip Topology Synthesis DSP L2C

DSP

DSP L2CC

L2C

217

IMPRO2

L2CC

IMPRO1

CONT1

CONT2

DMA

L2CC

Layer 0 MEM1 MEM2 Switch 2

Switch 1

MEM4

Layer 1

Switch 0

MEM3

Layer 2

SD RAM2

PE3

FLASH

MEM5

SD RAM1

DEBUG

PE2

P1

P3

PE1

P2

Fig. 9.17↜渀 Most power-efficient topology

DMA

DSPL2CC IMPROC2

IMPROC1

MEM2

DSPL2C

P2 DSP

PE2

SW1 SW0

CONT1 ARM

MEM1

L2CC

SW2

L2C

Layer 0 NI

Switch

Layer 1

MEM5 PE1

P3

MEM3 DEBUG SDRAM1

CONT2

Core

P1

FLASH MEM4

SDRAM2

PE3

Layer 3

Flip-Flop

Fig. 9.18↜渀 Resulting 3D floorplan with switches

9.6â•…Experiments and Case Studies To show how the algorithm performs, we will show different results on a realistic multimedia and wireless communication benchmark. We will also make a comparison on topologies built with the two phases of the described algorithm to show the advantages and disadvantages of each phase. We will also make a comparison between NoCs designed for different applications for 2D-ICs and 3D-ICs. Experiments showing the advantage of custom topologies over regular ones are also presented. In order to better estimate power, the NoC component library from [43] is used. The power and latency values of the switches and links of the library are

218

C. Seiculescu et al.

determined from post-layout simulations, based on 65€nm low power libraries. The vertical links are shown to have an order of magnitude lower resistance and capacitance than a horizontal link of the same dimension [11]. This translates to a traversal delay of less than 10% of clock cycle for 1€GHz operation and negligible power consumption on the vertical links.

9.6.1  Multimedia SoC Case Study For the case study, a realistic multimedia and wireless communication SoC was chosen as benchmark. The communication graph for the benchmark (denoted as D_26_media) is shown in Fig.€5.4. From the figure, it can be observed that there are 26 cores in the SoC. A part of the SoC which is constructed around an ARM processor is used for multimedia applications and it is aided by hardware video accelerators and controllers to access external memory. The other part of the SoC built around a DSP is used for wireless communication. The communication between the two parts is facilitated by several large on-chip memories and a DMA core. A multitude of peripherals are also present for off-chip communication. To compare between NoC designed for 2D-ICs and 3D-ICs, we performed 2D floorplans of the cores as well as 3D floorplan of the cores distributed to three layers using existing tools [44]. The assignment of cores to the layers of the 3D stack was performed manually, but there are solutions presented for 3D floorplanning that can give also the assignment of cores to layers. The tool from [39] was used to generate the application specific topologies for the 2D case. To data width of the links was set to 32 bits to correspond to the data width of the cores and the frequency was set to 400€MHz (this being the lowest frequency at which topologies can be designed to support the required bandwidth for the chosen data width). The max_ill constraint was set at 25. The impact of the constraint on power is analyzed later on. The power consumption on the different components of the NoCs, as well as the total power consumption is presented in Fig.€9.9 for the 2D-IC and in Fig.€9.10 for 3D. The plots represent the power consumption for topologies generated for different switch counts. Both plots start with topologies containing three switches. Because there are 26 cores in the design, topologies with less than three switches could not be built, as they would require the use of switches too large to support the operating frequency. These design points are pruned from exploration to reduce the run time, as explained in Sect.€9.5. The power consumption on individual components: switches, switch to switch links and core to switch links are presented in the figures. Several trends can be observed. The switch power grows as the number of switches grow. The core to switch link power goes down with more switches as the switches are placed closer to cores and the wire length decreases. The switch to switch link power consumption grows with the number of switches as the number of such links increases. However, the switch to switch link power does not increase as fast as the switch power with the switch count. The trends are similar for both 2D and 3D cases, but the absolute values for the link power in the 3D case is less than the ones

9â•… 3D Network on Chip Topology Synthesis

219

for 2D, as long and power hungry links from the 2D layout are replaced by short and efficient vertical links. For this particular benchmark, a power saving of 24% is achieved in 3D over 2D due to shorter wires. To give a better understanding, we show the wire-length distribution of the links in 2D and 3D cases in Fig.€9.11. From the figure, as expected, the 2D design has many long wires. In Fig.€9.12 the topology designed using phase 1 for the best power point is shown and in Fig.€9.13 the floorplan of the cores and network components in 3D for the corresponding topology is presented. For a more complete comparison between topologies for 2D-ICs and 3D-ICs, we designed topologies using different SoC benchmarks. We consider three distributed benchmarks with 36 cores (18 processors and 18 memories): D_36_4 (communication graph in Fig.€9.14), D_36_6 and D_36_8, where each processor has 4, 6 and 8 traffic flows going to the memories. The total bandwidth is the same in the three benchmarks. We consider a benchmark, D_35_bot that models bottleneck communication, with 16 processors, 16 private memories (one processor is connected to one private memory) and 3 shared memories to which all the processors communicate. We also consider two benchmarks where all the cores communicate in a pipeline fashion: 65 core (↜D_65_pipe) and 38 core designs (↜D_tvopd) (communication graph in Fig.€5.4). In the last two benchmarks, each core communicates only to one or few other cores. We selected the best power points for both the 2D case and the 3D case and we report the power and zero load latency in Table€9.1. As most of the power difference comes from the reduced wire length in 3D, the power savings differs from benchmark to benchmark. For the benchmarks with spread traffic and many communication flows the power savings are considerable, as they benefit from the reduction of the long wires of the 2D design. In the bottleneck benchmark, there are many long wires that go to the shared memory in the 2D case. Even though the traffic to shared memories is small, we can still see a reasonable power saving when moving to 3D. For the pipeline benchmarks, most of the links are between neighboring cores, so links are short even in 2D. So, going for a 3D design dose not lead to large savings. The average power reduction is 38% and the average zero load latency reduction is 13% for the different benchmarks when comparing 3D to a 2D implementation.

Table 9.1↜渀 2D vs 3D NoC comparison Benchmark Power (mW) Link power Switch power 2D 3D 2D 3D

Total power 2D 3D

2D

3D

D_36_4 D_36_6 D_36_8 D_35_bot D_65_pipe D_38_tvopd

215 230 320 116 169 ╇ 89.5

3.28 3.57 4.37 6.04 2.53 4

3.14 3.5 3.65 4.2 2.57 3.6

150 154.5 215 ╇ 68 106 ╇ 52.5

╇ 41.5 ╇ 43.5 ╇ 55.5 ╇ 36.2 104 ╇ 22.67

╇ 65 ╇ 76.5 105 ╇ 48 ╇ 63 ╇ 37

╇ 70.5 ╇ 82 104.5 ╇ 43.3 ╇ 58 ╇ 38.11

Latency (cyc)

112 125.5 160 ╇ 79.5 162 ╇ 60.78

220

C. Seiculescu et al.

In Fig.€9.13, we show the power consumption of the topologies synthesized using Phase 2 of the algorithm, with respect to topologies synthesized using Phase 1 for the different benchmarks. Since in Phase 2 cores in a layer are connected to switches in the same layer, the inter-layer traffic needs to traverse more switches to reach the destination. This leads to an increase in power consumption and latency. As seen from Fig.€9.13, Phase 1 can generate topologies that lead to a 40% reduction in NoC power consumption, when compared to the Phase 2. However Phase 2 can generate topologies with a much tighter inter-layer link constraint.

9.6.2  I mpact of Inter-layer Link Constraint and Comparisons with Mesh

Minimum power consumption (mW)

Limiting the number of inter-layer links has a great impact on power consumption and average latency. Reducing the number of TSVs is desirable for improving the yield of a 3D design. However, a very tight constraint on the number of inter-layer links can lead to a significant increase in power consumption. To see the impact of the constraint, we varied the value of max_ill constraint and performed topology synthesis for each value, for the D_36_4 benchmark. The power and latency values for the different max_ill design points are shown in Figs.€ 9.19 and 9.20. When there is a tight constraint on the inter-layer links, the cores are connected to switches in the same layer, so that only switch-to-switch links need to go across layers. This results in the use of more switches in each layer, increasing switch power consumption and average latency. Please note that our synthesis algorithm also allows the designers to perform such power, latency trade-offs for yield, early in the design cycle. Custom topologies that match the application characteristics can result in large power-performance improvement when compared to the standard topologies, such as mesh and torus [39]. A detailed comparison between a custom topol-

Fig. 9.19↜渀 Impact of max_ill on power

190 180 170 160 150 140 130 120 110

0

5 10 15 20 25 Maximum number of inter-layer links (max_ill)

9╅ 3D Network on Chip Topology Synthesis Fig. 9.20↜渀 Impact of max_ill on latency

221

4 Minimumlatency (cycles)

3.5 3 2.5 2 1.5 1 0.5 0

0

5 10 15 20 25 Maximum number of inter-layer links (max_ill)

ogy and several standard topologies for different benchmarks for the 2D case has been presented in [39]. For completeness, we compared the application specific topologies generated by our algorithm with an optimized 3D mesh topology (↜3D Opt-mesh), where core placement is optimized such that cores that communicate are connected to nearby switches. The power consumption value for the topologies for different benchmarks is presented in Fig.€9.21. The custom topologies result in large power reduction (average of 51%) when compared to the 3D mesh topology. 450 3D Application specific 3D Opt-mesh

400

Power consumption (mW)

350 300 250 200 150 100 50 0

D_36_4

D_36_6

Fig. 9.21↜渀 Comparisons with mesh

D_36_8

D_35_bot

D_65_pipe D_38_tvopd

222

C. Seiculescu et al.

9.7╅Conclusions Networks On Chips (╛NoCs) are a necessity for achieving 3D integration. One of the major design issues when using NoCs for 3D is the synthesis of the NoC topology and architecture. In this chapter, we presented synthesis methods for designing power-efficient NoC topologies. The presented methods not only address classic 2D issues, such as meeting application performance requirements, minimizing power consumption, but also the 3D technology constraints. We showed two flavors of the general algorithm, one for achieving low power solution and the other to achieve tight control on the number of vertical connections established. We also presented comparisons with 2D designs to validate the benefits of 3D integration for interconnect delay and power consumption. Acknowledgment╇ We would like to acknowledge the financial contribution of CTI under project 10046.2 PFNM-NM and the ARTIST-DESIGN Network of Excellence.

References ╇ 1.╇╛K. Banerjee et al., “3-D ICs: A Novel Chip Design for Deep-Submicrometer Interconnect Performance and Systems-on-Chip Integration”, Proc. of the IEEE, vol. 89, no. 5, p. 602, 2001. ╇ 2.╇╛L. Benini and G. De Micheli, “Networks on Chips: A New SoC Paradigm”, IEEE Computers, vol. 35, no. 1, pp. 70–78, Jan. 2002. ╇ 3.╇╛E. Beyne, “The Rise of the 3rd Dimension for System Integration”, International Interconnect Technology Conference, pp. 1–5, 2006. ╇ 4.╇╛B. Goplen and S. Sapatnekar, “Thermal Via Placement in 3D ICs”, Proc. Intl. Symposium on Physical Design, p. 167, 2005. ╇ 5.╇╛J. Cong et al., “A Thermal-Driven Floorplanning Algorithm for 3D ICs”, ICCAD, Nov. 2004. ╇ 6.╇╛W.-L. Hung et al., “Interconnect and Thermal-Aware Floorplanning for 3D Microprocessors”, Proc. ISQED, March 2006. ╇ 7.╇╛S. K. Lim, “Physical Design for 3D System on Package”, IEEE Design & Test of Computers, vol. 22, no. 6, pp. 532–539, 2005. ╇ 8. P. Zhou et al., “3D-STAF: Scalable Temperature and Leakage Aware Floorplanning for ThreeDimensional Integrated Circuits”, ICCAD, Nov. 2007. ╇ 9. R. Weerasekara et al., “Extending Systems-on-Chip to the Third Dimension: Performance, Cost and Technological Tradeoffs”, ICCAD, 2007. 10. G. H. Loh, Y. Xie, and B. Black. “Processor Design in 3D Die-Stacking Technologies”, IEEE Micro Magazine, vol. 27, no. 3, pp. 31–48, May--June 2007. 11. I. Loi, F. Angiolini, and L. Benini, “Supporting Vertical Links for 3D Networks on Chip: Toward an Automated Design and Analysis Flow”, Proc. Nanonets, 2007. 12. C. Guedj et al., “Evidence for 3D/2D Transition in Advanced Interconnects”, Proc. IRPS, 2006. 13. http://www.zurich.ibm.com/st/cooling/interfaces.html 14. IMEC, http://www2.imec.be/imec_com/3d-integration.php 15. http://www.tezzaron.com 16. N. Miyakawa, “A 3D Prototyping Chip Based on a Wafer-level Stacking Technology”, ASPDAC, 2009. 17. C. Addo-Quaye, “Thermal-Aware Mapping and Placement for 3-D NoC Designs”, Proc. SOCC, 2005.

9â•… 3D Network on Chip Topology Synthesis

223

18. P. Guerrier and A. Greiner, “A Generic Architecture for On-Chip Packet Switched Interconnections”, Proc. DATE, 2000. 19. G. De Micheli and L. Benini, “Networks on Chips: Technology and Tools”, Morgan Kaufmann, San Francisco, CA, First Edition, July 2006. 20. J. Kim et al., “A Novel Dimensionally-Decomposed Router for On-Chip Communication in 3d Architectures”, ISCA, 2007. 21. D. Park et al., “MIRA: A Multi-Layered On-Chip Interconnect Router Architecture”, ISCA, 2008. 22. F. Li et al., “Design and Management of 3D Chip Multiprocessors Using Network-in-Memory”, ISCA, 2006. 23. V. F. Pavlidis and E. G. Friedman, “3-D Topologies for Networks-on-Chip”, IEEE TVLSI, 2007. 24. B. Feero and P. P. Pande, “Performance Evaluation for Three-Dimensional Networks-onChip”, Proc. ISVLSI, 2007. 25. J. Hu et al., “System-Level Point-to-Point Communication Synthesis Using Floorplanning Information”, Proc. ASPDAC, 2002. 26. S. Pasricha et al., “Floorplan-Aware Automated Synthesis of Bus-Based Communication Architectures”, Proc. DAC, 2005. 27. S. Murali and G. De Micheli, “An Application-Specific Design Methodology for STbus Crossbar Generation”, Proc. DATE, 2005. 28. S. Murali and G. De Micheli, “SUNMAP: A Tool for Automatic Topology Selection and Generation for NoCs”, Proc. DAC, 2004. 29. S. Murali and G. De Micheli, “Bandwidth Constrained Mapping of Cores on to NoC Architectures”, Proc. DATE, 2004. 30. J. Hu and R. Marculescu, “Exploiting the Routing Flexibility for Energy/Performance Aware Mapping of Regular NoC Architectures”, Proc. DATE, 2003. 31. S. Murali et al., “Mapping and Physical Planning of Networks on Chip Architectures with Quality-of-Service Guarantees”, Proc. ASPDAC, 2005. 32. A. Pinto et al., “Efficient Synthesis of Networks on Chip”, ICCD 2003, Oct. 2003. 33. W. H. Ho and T. M. Pinkston, “A Methodology for Designing Efficient On-Chip Interconnects on Well-Behaved Communication Patterns”, HPCA, 2003. 34. T. Ahonen et al., “Topology Optimization for Application Specific Networks on Chip”, Proc. SLIP, 2004. 35. K. Srinivasan et al., “An Automated Technique for Topology and Route Generation of Application Specific On-Chip Interconnection Networks”, ICCAD, 2005. 36. J. Xu et al., “A Design Methodology for Application-Specific Networks-on-Chip”, ACM TECS, 2006. 37. A. Hansson et al., “A Unified Approach to Mapping and Routing on a Combined Guaranteed Service and Best-Effort Network-on-Chip Architectures”, Technical Report No: 2005/00340, Philips Research, Apr. 2005. 38. X. Zhu and S. Malik, “A Hierarchical Modeling Framework for On-Chip Communication Architectures”, ICCD, 2002. 39. S. Murali et al., “Designing Application-Specific Networks on Chips with Floorplan Information”, ICCAD, 2006. 40. S. Murali et al., “Synthesis of Networks on Chips for 3D Systems on Chips”, ASPDAC, 2009. 41. C. Seiculescu, S. Murali, L. Benini, and G. De Micheli, “SunFloor 3D: A Tool for Networks on Chip Topology Synthesis for 3D Systems on Chip”, Proc. DATE, 2009. 42. K. Keutzer et al., “System-Level Design: Orthogonalization of Concerns and Platform-Based Design”, IEEE TCAD, 2000. 43. S. Stergiou et al., “×pipesLite: a Synthesis Oriented Design Library for Networks on Chips”, Proc. DATE, 2005. 44. S. N. Adya and I. L. Markov, “Fixed-outline Floorplanning: Enabling Hierarchical Design”, IEEE TVLSI, 2003.

â•…

Chapter 10

3-D NoC on Inductive Wireless Interconnect Hiroki Matsutani, Michihiro Koibuchi, Tadahiro Kuroda and Hideharu Amano

10.1â•…Introduction: Wired vs. Wireless Three-dimensional chip implementation has been used in real commercial products to increase the real estate without stretching the implementation size and wiring delay. Most commercial 3-D chips are implemented using wafer-bonding technologies: face-to-face micro-bump [8, 2] or face-to-back through-silicon-via [3, 4]. On the other hand, wireless approaches have received a lot of attention because of their flexibilities. Wireless interconnects can be achieved by using capacitive coupling [5] and inductive coupling [12, 13]. The former approach is used only in surface-surface interconnections, while a number of dies can be stacked with the latter approach. The latter approach, inductive coupling, has the following advantages: (1) Dies can be stacked after fabrication and testing, thus only known-good-dies can be used. (2) A high speed and low power data transfer can be achieved [11]. (3) It is highly scalable; that is, from two to at least sixteen dies can be easily stacked. (4) Addition, removal, and swapping of dies are also all possible after fabrication. However, there are also several limitations: (1) In spite of remarkable improvements, the area of the inductors is still larger than those for through-silicon-vias. (2) The location of the inductors and the time to data transfer must be arranged so as to avoid introducing interference in the electric field. These inductive coupling advantages and limitations create challenges for using 3-D NoCs on it. Scalable and changeable network protocols are required to avoid any interference. We introduce 3-D NoCs on inductive-coupled wireless interconnects.

H. Matsutani () The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, 113-8656 Tokyo, Japan e-mail: [email protected] A. Sheibanyrad et al. (eds.), 3D Integration for NoC-based SoC Architectures, Integrated Circuits and Systems, DOI 10.1007/978-1-4419-7618-5_10, ©Â€Springer Science+Business Media, LLC 2011

225

226

H. Matsutani et al.

RX Die TX

Square Coil using metal layer

Fig. 10.1↜渀 Inductive coupling

10.2╅Wireless Interconnect with Inductive Coupling An inductor is implemented in wireless inductive coupling approach as a square coil of metal in a common CMOS layout. The data modulated by a driver are transferred between two inductors placed at exactly the same position of two stacked dies, and it is received at the other die using the receiver, as shown in Fig.€10.1. This method allows the stacking of at least 16 of dies if the power consumption of each die is low enough to work without power dissipation facilities. Although more than two inductors can be stacked, multiple transmitters at the same location cannot simultaneously send the data in order to avoid the interference. The techniques have recently been improved for inductive coupling, and the contact-less interface without an electrostatic-discharge (ESD) protection device achieves a high speed of more than 1€GHz with a low energy dissipation (0.14€pJ per bit) and a low bit-error rate (BER