GPU-Based Interactive Visualization Techniques

Mathematics and Visualization Series Editors Gerald Farin Hans-Christian Hege David Hoffman Christopher R. Johnson Konra...

Author: Daniel Weiskopf

83 downloads 1134 Views 8MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Mathematics and Visualization Series Editors Gerald Farin Hans-Christian Hege David Hoffman Christopher R. Johnson Konrad Polthier Martin Rumpf

Daniel Weiskopf

GPU-Based Interactive Visualization Techniques With 112 Figures, 42 in Color and 11 Tables

ABC

Daniel Weiskopf School of Computing Science Simon Fraser University Burnaby, BC V5A 1S6, Canada E-Mail: [email protected]

Library of Congress Control Number: 2006931796 Mathematics Subject Classification: 68U05, 68W10, 76M27 ISBN-10 3-540-33262-6 Springer Berlin Heidelberg New York ISBN-13 978-3-540-332626 Springer Berlin Heidelberg New York This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable for prosecution under the German Copyright Law. Springer is a part of Springer Science+Business Media springer.com c Springer-Verlag Berlin Heidelberg 2006 The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. A EX macro package Typesetting by the author and SPi using a Springer LT Cover picture by Daniel Weiskopf Cover design: design & production GmbH, Heidelberg

Printed on acid-free paper

SPIN: 11730132

46/SPi/3100

543210

F¨ ur Elisabeth und Gerhard

Preface

Graphics processing units (GPUs) have been revolutionizing the way computer graphics and visualization are practiced. Driven by the computer-games industry and its demand for efficient hardware support for 3D graphics, GPUs have dramatically increased in performance and functionality within only a few years. Although graphics hardware is primarily designed for the fast rendering of 3D scenes, it can also be used for other types of computations. In fact, GPUs have evolved to programmable processors that can facilitate applications beyond traditional real-time 3D rendering. This book addresses scientific visualization as one application area that significantly benefits from the use of GPUs. In general, scientific visualization has become an important tool for visual analysis in many scientific, engineering, and medical disciplines. For example, scientists and engineers regularly use visualization to interpret simulations of air or water flow in computational fluid dynamics. Another example is the medical imaging of 3D CT (computer tomography) or MRI (magnetic resonance imaging) scans. The interactive exploration of data sets is becoming increasingly more important with the growing amount and complexity of those data sets: the user – as an expert in his or her field – uses visualization as a tool to investigate the data and extract insight from it. This book focuses on efficient visualization techniques, which are the prerequisite for interactive exploration. High performance is primarily achieved by algorithms specifically designed for GPUs and their special features. Other aspects discussed in this work include parallelization on cluster computers with several GPUs, adaptive rendering methods, multi-resolution models, and non-photorealistic rendering techniques for visualization. This book also addresses the effectiveness of visualization methods, which can be improved by taking into account perceptual aspects and user interaction. Covering both the theoretical foundations and practical implementations of algorithms, this text provides a basis to understand and reproduce modern GPU-based visualization approaches.

VIII

Preface

This work constitutes my Habilitationsschrift, written at the University of Stuttgart. Being a research thesis, this Habilitationsschrift aims at describing visualization methods on a scientific level. Therefore, the intended audience includes researchers and students of computer science who are interested in interactive visualization methods. This book may serve as a starting point to delve into the current research in GPU-based visualization. It may also serve as reading material for a course that covers scientific visualization on an advanced undergraduate or a graduate level. This work also includes a discussion of practical issues such as how algorithms are mapped to GPU programs or performance characteristics of implementations. Therefore, practitioners and software developers might also find this book interesting. The reader is expected to have a basic background in scientific visualization, but not in GPU-based visualization methods. In addition, some familiarity with GPU programming is recommended. Although this book does not cover these background topics in detail, it nevertheless contains some introductory material on the basics of visualization and GPU programming. In particular, a wealth of references is provided to guide the reader to background reading.

How to Read This Book This book is structured in a way that it can be read from cover to cover. However, you may also pick out some passages that are most interesting to you, and you may read through the book in non-sequential order. This section gives some hints on what topics are covered in which chapter and which parts of the book are built on other parts. It is recommended reading the introductory Chap. 1 because it describes basic concepts used to organize the book. In particular, the abstract visualization pipeline is discussed along with a classification scheme for visualization methods (Sect. 1.1). Throughout this book, visualization techniques are related to the three main stages of the visualization pipeline, namely filtering, mapping, and rendering. This chapter also covers fundamentals of GPUs (in Sect. 1.2) that can be skipped if you are already familiar with GPU programming. In addition, Sect. 1.3 presents methods and goals of this work. The main part of the book is organized in three large chapters on 3D scalar field visualization (Chap. 2), vector field visualization (Chap. 3), and perception-oriented and non-photorealistic rendering (Chap. 4). To a large extent, each of these chapters represents a portion of the book that can be read independently of the other main chapters. However, a few interdependencies are present as outlined below. Chapter 2 addresses methods for direct volume visualization of 3D scalar fields. It is recommended reading the basics of volume rendering as laid out in Sects. 2.1–2.3. Sections 2.1 and 2.2 describe the underlying optical model and the volume rendering pipeline. Section 2.3 discusses basic volume rendering

Preface

IX

methods, focusing on real-time GPU rendering. In particular, texture-based volume rendering (also called texture slicing) is described because it is the method of choice in this work. A multi-bricking approach for 3D texturebased rendering is introduced in Sect. 2.4. This approach maintains an approximately constant rendering performance for real-time applications. Sections 2.5 and 2.6 discuss advanced topics that could be read independently. Section 2.5 focuses on a number of new techniques for volume clipping that allow for complex clipping geometries. Clipping plays an important role in improving the perception of a 3D data set because it enables the user to explore otherwise hidden internal parts of the data set. Here, object-space and image-space clipping methods are compared, pre-integrated volume clipping is described, and issues of consistent volume shading are discussed. The visualization of very large, time-dependent volume data is addressed in Sect. 2.6. One element of this large-data approach is parallelization on a cluster computer with commodity-of-the-shelf GPUs. Another element is wavelet compression in combination with adaptive rendering. Chapter 2 concludes with a brief summary of the described volume rendering techniques. Chapter 3 discusses techniques for vector field visualization, with the focus on texture-based methods. Section 3.1 is recommended as basis for this chapter because it presents the fundamental concept of particle tracing. Section 3.2 provides an overview and a classification of vector field visualization methods. This section contains an extensive list of references, serving as a good starting point to delve into state-of-the-art vector field visualization. Section 3.3 continues with a more detailed discussion of texture-based vector field visualization. It describes semi-Lagrangian texture advection and shows how advection can be used for dense, noise-based vector field visualization and sparse, dye-based visualization alike. In part, this section relies on volume rendering techniques that are described in Chap. 2 (especially Sects. 2.1– 2.3). Section 3.3 is recommended as basis for the following sections of this chapter. These subsequent sections cover advanced topics and can be read independently of each other. A novel level-set advection scheme is introduced in Sect. 3.4 to overcome numerical diffusion that is inherent to semi-Lagrangian dye advection. While the methods discussed so far work in Cartesian 2D and 3D space, Sect. 3.5 addresses vector field visualization on curved surfaces, for example, on the boundary surface of an automobile model enclosed by wind flow. A hybrid object-space and image-space method is introduced to achieve an efficient, yet accurate dense flow representation. Section 3.6 describes a generic framework for the visualization of time-dependent vector fields. This framework comprises all relevant previous visualization methods and allows us to compare them on a mathematical basis. In addition, the flexibility of the framework leads to the development of novel visualization approaches. The final section of Chap. 3 summarizes the presented flow visualization methods. Perception-oriented rendering and non-photorealistic rendering are discussed in Chap. 4. This chapter focuses on the third stage of the visualization pipeline – the rendering stage. It is recommended reading the brief

X

Preface

discussion of previous work in Sect. 4.1. Sections 4.2–4.5 can be read independently of each other because they cover different aspects of perceptionoriented and non-photorealistic rendering. Section 4.2 addresses the influence of color on the visual perception of moving patterns. Based on an extensive review of psychophysics and psychology literature, a set of design guidelines is derived for effective animated visualization. These guidelines are especially useful for texture-based flow visualization; therefore, some background reading in Chap. 3 is recommended. Section 4.3 improves depth-perception by utilizing perception-oriented color cues. In particular, depth-dependent intensity and saturation modifications are employed. Section 4.4 introduces nonphotorealistic rendering methods that improve the perception of spatial structures in complex tone-shaded illustrations. A view-dependent transparency model is proposed in Sect. 4.4.2, whereas alternative cutaway methods are described in Sect. 4.4.3. These tone-shaded illustrations are tightly connected to non-photorealistic volume rendering and volume clipping. Therefore, some background reading in Chap. 2 is recommended. Non-photorealistic halftoning approaches are explicated in Sect. 4.5; here, the focal point is frame-to-frame coherent halftoning and a generic GPU-based concept for G-buffer operations. Frame-to-frame coherent halftoning relies on texture advection as a basic technique. Therefore, background material from Sect. 3.3 is useful for understanding Sect. 4.5. Chapter 4 ends with a summary of presented rendering methods. Following the main part outlined above, Chap. 5 concludes this book. This chapter classifies the visualization methods discussed in this book and puts them in context. The appendix contains lists of figures, tables, and color plates, as well as a bibliography, an index, and color plates.

Acknowledgments I conducted the research for this book while I was with the Institute for Visualization and Interactive Systems (VIS) at the University of Stuttgart. I would like to thank all VIS members for the great time I had in Stuttgart and for many fruitful discussions. In particular, I thank Thomas Ertl for lots of support and advise; Joachim Diepstraten, Mike Eißele, and Sabine IserhardtBauer for having been great roommates; and Matthias Hopf, Guido Reina, Ulrike Ritzmann, and Simon “Grbic” Stegmaier for their help. I especially thank the following VIS members for successful collaborations on research that has become part of this work: Joachim Diepstraten [88, 89], Mike Eißele [108, 109], Klaus Engel [469, 470], Matthias Hopf [473, 478, 479], Stefan R¨ ottger [356], Martin Rotard [352], Tobias Schafhitzel [481], Magnus Strengert and Marcelo Magall´ on [402], as well as Manfred Weiler [484]. In addition, I had the pleasure to work with many VIS members on topics that, although not directly part of this book, have influenced my research in general: Ralf Botchen [27, 468], Joachim Diepstraten [87, 90], Mike

Preface

XI

Eißele [107, 216], Thomas Klein [216], Guido Reina [351], Dirc Rose [351], Martin Rotard [353], Tobias Schafhitzel [374, 480], Frederik Schramm [483], and Simon Stegmaier [107, 351]. Furthermore, I would like to thank the following collaborators outside VIS: Gordon Erlebacher [115, 471, 472, 473, 483], Stefan Guthe [356, 402], Bob Laramee and Helwig Hauser [240, 242], as well as Frits Post [240]. Finally, I thank Manfred Weiler for proof-reading Chaps. 1 and 2, and Nikolaus Weiskopf for help and discussions on various topics. Financial support from the following funding organizations has made my research possible: the “Landesstiftung Baden-W¨ urttemberg” for support within the program “Elitef¨ orderprogramm f¨ ur Postdoktoranden”; the “Deutsche Forschungsgemeinschaft” (DFG) for funding the projects D4 and D8 within SFB 382 and the project C5 within SFB 627; and the University of Stuttgart for financing the project “i-modanim” within the program “self-study online”. I also would like to thank the BMW group and Roger Crawfis for providing data sets, Gerik Scheuermann for making available the image for Fig. 3.1 (d), and Martin Schulz for providing the image for Fig. 3.3. The following publishers have kindly provided the permission to use material for this book from papers (co-)authored by myself: A K Peters, Ltd. [475]; Elsevier/Academic Press [471]; the Eurographics Association [88, 89, 356, 402, 464, 481]; IEEE [108, 465, 469, 470, 472, 484]; UNION Agency/WSCG [476]; and the publisher of the SimVis 2004 proceedings (SCS Publishing House) [109]. I thank Thomas Ertl, Eduard Gr¨ oller, Kurt Rothermel, and Hans-Peter Seidel for reviewing my Habilitationsschrift. I also thank the Faculty 5 (Computer Science, Electrical Engineering, and Information Technology) at the University of Stuttgart for a very timely organization and pleasant handling of the Habilitation procedure. I am indebted to all people at Springer who supported this book project. In particular, I thank Martin Peters (Executive Editor, Mathematics, Computational Science, and Engineering) and the editors of the book series Mathematics and Visualization, Gerald Farin, Hans-Christian Hege, David Hoffman, Christopher R. Johnson, Konrad Polthier, and Martin Rumpf. I also thank Ute McCrory for organizing the publishing process. Very special thanks to Bettina A. Salzer for thorough proof-reading. This work would not have been possible without her help, patience, and love. Finally, I thank my parents Elisabeth and Gerhard for all their support.

Vancouver, April 2006

Daniel Weiskopf

Contents

1

2

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Visualization Pipeline and Classification of Visualization Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 GPU Rendering Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.1 Programming Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.2 APIs and Effect Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Methods and Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Visualization of 3D Scalar Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Optical Model for Volume Rendering . . . . . . . . . . . . . . . . . . . . . . . 2.2 Volume Rendering Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Volume Rendering Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 3D Texture Slicing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.2 2D Texture Slicing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.3 Ray Casting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.4 Shear-Warp Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.5 Splatting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.6 Cell Projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.7 Pre-Integrated Volume Rendering . . . . . . . . . . . . . . . . . . . 2.3.8 Variants and Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Maintaining Constant Frame Rates in 3D Texture-Based Volume Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Volume Clipping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.1 Depth-Based Clipping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.2 Clipping Based on Volumetric Textures . . . . . . . . . . . . . . 2.5.3 Clipping and Pre-Integration . . . . . . . . . . . . . . . . . . . . . . . . 2.5.4 Clipping and Volume Illumination . . . . . . . . . . . . . . . . . . . 2.5.5 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 Hierarchical Volume Visualization on GPU Clusters . . . . . . . . . . 2.6.1 Previous and Related Approaches to Large-Data Volume Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 2 4 5 8 8 11 13 15 17 18 20 21 25 26 27 30 32 33 38 39 47 52 55 63 65 66

XIV

Contents

2.6.2 Distributed Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.3 Accelerated Compositing Scheme . . . . . . . . . . . . . . . . . . . . 2.6.4 Hierarchical Compression and Adaptive Rendering . . . . . 2.6.5 Implementation and Results . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

67 68 69 73 77

Vector Field Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 3.1 Basics of Particle Tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 3.1.1 Time-Dependent Vector Fields . . . . . . . . . . . . . . . . . . . . . . 82 3.1.2 Lagrangian Particle Tracing . . . . . . . . . . . . . . . . . . . . . . . . 84 3.1.3 Eulerian Particle Tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 3.2 Classification and Overview of Flow Visualization Methods . . . 85 3.2.1 Point-Based Direct Flow Visualization . . . . . . . . . . . . . . . 86 3.2.2 Sparse Representations for Particle-Tracing Techniques . 87 3.2.3 Dense Representations for Particle-Tracing Methods . . . 92 3.2.4 Feature-Based Visualization Approaches . . . . . . . . . . . . . . 95 3.3 Semi-Lagrangian Noise and Dye Advection on Cartesian Domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 3.3.1 Semi-Lagrangian Transport Mechanism . . . . . . . . . . . . . . 98 3.3.2 GPU Implementation of Semi-Lagrangian Advection . . . 99 3.3.3 Lagrangian-Eulerian Advection . . . . . . . . . . . . . . . . . . . . . . 105 3.3.4 Particle Injection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 3.3.5 Visual Mapping and Rendering . . . . . . . . . . . . . . . . . . . . . . 109 3.4 Dye Advection Based on Level-Sets . . . . . . . . . . . . . . . . . . . . . . . . 116 3.4.1 Distance Field as Level-Set . . . . . . . . . . . . . . . . . . . . . . . . . 118 3.4.2 Reinitialization of the Level-Set . . . . . . . . . . . . . . . . . . . . . 119 3.4.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 3.4.4 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 3.4.5 Implementation and Results . . . . . . . . . . . . . . . . . . . . . . . . 126 3.5 Flow Visualization on Curved Surfaces . . . . . . . . . . . . . . . . . . . . . 128 3.5.1 Lagrangian Particle Tracing on Surfaces . . . . . . . . . . . . . . 129 3.5.2 Visual Mapping and Noise Injection . . . . . . . . . . . . . . . . . 134 3.5.3 Effective Object and Flow Rendering . . . . . . . . . . . . . . . . 136 3.5.4 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 3.6 Spacetime Framework for Time-Dependent Vector Fields . . . . . 140 3.6.1 Continuous Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 3.6.2 Texture-Based Discretization . . . . . . . . . . . . . . . . . . . . . . . 143 3.6.3 Visualization Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 3.6.4 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 3.6.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 3.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

Contents

XV

4

Perception-Oriented and Non-Photorealistic Rendering . . . . 161 4.1 Previous Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 4.2 Color and the Perception of Motion . . . . . . . . . . . . . . . . . . . . . . . . 163 4.2.1 Psychophysical and Physiological Research . . . . . . . . . . . 164 4.2.2 Design Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 4.2.3 Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 4.2.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 4.3 Color-Based Depth-Cueing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 4.3.1 Linear Transformations and Weighted Sum of Colors . . . 177 4.3.2 Appropriate Parameters for the Generic Depth-Cueing Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 4.3.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 4.3.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 4.4 Non-Photorealistic Rendering of Depth Structures via Continuous Tone-Shading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 4.4.1 Traditional Visualization of Semi-Transparency in Illustrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 4.4.2 Semi-Transparency in Computer-Generated Illustrations 194 4.4.3 Overview of Cutaway Illustrations . . . . . . . . . . . . . . . . . . . 201 4.4.4 Cutout Drawings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 4.4.5 Breakaway Illustrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 4.4.6 Implementation and Performance of Surface Rendering . 213 4.4.7 Extension to Volume Illustrations . . . . . . . . . . . . . . . . . . . 215 4.5 Non-Photorealistic Halftoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 4.5.1 The G-Buffer Concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 4.5.2 Frame-to-Frame Coherent Halftoning . . . . . . . . . . . . . . . . 220 4.5.3 The G2 -Buffer Framework . . . . . . . . . . . . . . . . . . . . . . . . . . 229 4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

5

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 Color Plates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

1 Introduction

Visualization has become an essential part in today’s engineering, research, and business workflows. The typical amount of data that originates from numerical simulations or sensor measurements is very large and, thus, visualization is indispensable for understanding this data. The ever increasing performance of supercomputers and resolution of scanning devices is a challenge that requires us to keep on developing improved visualization methods. This book addresses this challenge in manifold ways. Both the efficiency and effectiveness of visualization methods are improved. High efficiency is achieved by using appropriate algorithms and data structures. This work specifically considers the usage of the fast graphics processing units (GPUs) of modern graphics hardware to obtain high performance – the performance gain can be as much as two to three orders of magnitude compared to a CPU (central processing unit) implementation. The effectiveness of visualization methods is improved by taking into account perceptual aspects and an appropriate representation of the underlying data model. Efficiency and effectiveness are not completely separated but interlinked: a highly efficient visualization method is the prerequisite for an interactive application, which, in return, leads to significantly improved effectiveness. For example, an interactive exploration may promote spatial perception by motion parallax or result in a better understanding of the large parameter space that controls data generation and visualization. Particularly interesting is real-time visualization for time-dependent data. Increased efficiency can lead to a qualitatively improved effectiveness because it might make the difference between a visualization that allows for effective user interaction and a noninteractive inefficient implementation. The use of GPUs for high-performance visualization is motivated by several aspects. First, GPUs typically have more transistors than CPUs. For example, an NVIDIA GeForce 6800 Ultra GPU comprises 222 million transistors, whereas an Intel Pentium 4 EE (Extreme Edition) CPU contains 29 million transistors for the processor core and 149 million transistors for the L2 (level 2) cache [419]. Second, the speed of GPUs and their number of transis-

2

1 Introduction

tors increase faster than described by Moore’s law for CPUs. Third, graphics hardware has wide data paths to its memory, and efficient bilinear and trilinear reconstruction of texture data is available. Fourth, the programming model for GPUs is rather restricted and results in highly parallel execution and deep pipelining. Finally, a transfer of visualization results to the graphics board for final display is superfluous for GPU-based implementations.

1.1 Visualization Pipeline and Classification of Visualization Methods An abstract visualization pipeline is used throughout this book to structure the large variety of visualization methods. This pipeline approximately follows the description by Haber and McNabb [149] – up to some slight modifications. Figure 1.1 sketches the elements of the visualization pipeline. Input to the visualization pipeline is acquired from a data source, such as a numerical simulation, a measurement of physical data, or a database. This raw data is transformed by the filtering stage of the visualization pipeline into abstract visualization data. Typical filtering operations are denoising by convolution with a filter kernel or data enhancement by segmentation. The next stage is visualization mapping, which constructs a renderable representation from visualization data. The renderable representation has extension in space and time, and it contains attribute fields that may comprise geometry, color, transparency, reflectance, and surface texture. A typical example for a

databases

simulation

sensors

raw data

visualization pipeline

filtering visualization data

mapping renderable representation

rendering displayable image

interaction

Fig. 1.1. Abstract visualization pipeline

data acquisition

1.1 Visualization Pipeline and Classification of Visualization Methods

3

mapping operation is the application of a color table that assigns colors to input data values. Finally, the rendering stage generates a displayable image from the renderable representation. Typical rendering operations include view transformations, scene illumination, and shading. For some visualization algorithms, it may be difficult to find a clear and unique assignment of algorithmic elements to the filtering, mapping, or rendering stages. GPU-based methods, in particular, often require a reordering of the conceptual visualization pipeline to achieve an efficient implementation. Examples for such a reordering are given for corresponding visualization methods when their details are discussed later in this work. Nevertheless, the concept of the visualization pipeline has proven to be useful in classifying and comparing different visualization methods. For example, this scheme has been successfully used as a basis for teaching visualization courses (see the article by Rotard, Weiskopf, and Ertl [352]). Therefore, all visualization techniques of this book are analyzed by taking into account the concept of the visualization pipeline. As pointed out in Fig. 1.1, a user may potentially interact with all three stages of the visualization pipeline. Interaction plays a crucial role throughout this work because the large parameter space that controls data acquisition and visualization can only be explored interactively. As a prerequisite, visualization techniques need to be efficient to allow for real-time application. In addition to the structure of the visualization pipeline, a more detailed taxonomy is useful for classifying visualization methods. Following Bergeron and Grinstein [17] and Brodlie [31], the dimensionality of independent variables (on the domain) and the dimensionality of dependent variables (i.e., data type) are important categories. Therefore, visualization methods are ordered according to their data type, starting with scalar data in Chap. 2. Vector fields, which are more complex than scalar fields, are discussed in Chap. 3. In addition, the dimensionality of the domain plays an important role in substructuring this chapter on vector field visualization. Chapters 2 and 3 cover all steps of the visualization pipeline for the specific cases of scalar and vector fields, whereas Chap. 4 focuses on the rendering stage. This chapter discusses perception-oriented and non-photorealistic rendering methods that could be applied to scalar field and vector field visualization, but also to a much wider range of visualization applications. A third category that can be used to classify visualization methods is the type of data structure that holds the data set. The data structure is important from an algorithmic point of view and therefore influences the design of visualization algorithms. Throughout this book, visualization techniques for different data structures are discussed. Usually, data is represented by a set of data points located in the domain space. The data structure forms a grid if a connectivity between data points is established. In contrast, scattered data does not have such a connectivity. We can distinguish two classes of grids: structured and unstructured grids. Structured grids have an implicitly given connectivity, whereas unstructured grids need their connectivity explic-

4

1 Introduction

itly stored. Examples of structured grids are Cartesian, uniform, rectilinear, or curvilinear grids. Cartesian, uniform, and rectilinear grids have all their cells aligned with the coordinate axes of the domain. A Cartesian grid has the same overall grid spacing regardless of direction. A uniform grid has an equidistant spacing along each of the directions, although the spacing is independently chosen for the different directions. A rectilinear grid has varying cell sizes. A curvilinear grid exhibits the most flexibility by allowing for deformed cells, provided the topological structure is still regular. A typical example of an unstructured grid is a grid consisting of simplical cells, i.e., a tetrahedral grid in the 3D case. More details on these basic grid types and other, more advanced grid structures are given in the book by Schroeder et al. [382].

1.2 GPU Rendering Pipeline This book discusses efficient visualization algorithms for GPUs. The basic properties of GPUs, their programming model, and the underlying rendering pipeline are outlined in this section. The stages of the GPU rendering pipeline are only briefly reviewed; detailed background information on the traditional fixed-function pipeline can be found in a textbook by Foley et al. [122] or the OpenGL programming guide [496]. Additional information on using programmable graphics hardware is given in Cg manuals [118, 312] and documentations of the OpenGL shading language [207] and Direct3D [288], which is part of DirectX. Figure 1.2 sketches the GPU rendering pipeline. A 3D graphics application communicates with graphics hardware via a 3D API (application programming interface). Typically, either OpenGL or Direct3D are used as 3D API. 3D scenes are usually described by a boundary representation (BRep) formed by a collection of primitives. In most cases, primitives are triangles defined by vertices. Vertex information consists of geometric position and further attributes such as normal vector, texture coordinates, color, or opacity. Primitives are sent from CPU to GPU as a stream of vertices, which are then transformed by the vertex processor. In the traditional fixed-function graphics pipeline, vertex processing accomplishes transform and lighting operations. A vertex of the input geometry is transformed from its original object coordinates into subsequent world, eye, and normalized clip coordinates by multiplication with respective matrices for model, view, and normalization transformations. Matrix formulations of these affine or projective transformations are adequate because all computations are based on homogeneous coordinates. Lighting is usually based on the Blinn-Phong model and performed with respect to eye coordinates. GPUs allow the programmer to replace these fixed-function vertex operations by a flexible vertex program (OpenGL lingo) or vertex shader (Direct3D lingo). Primitives are converted into fragments after vertex processing. Today’s GPUs exclusively process triangles at these stages of the pipeline; i.e., prim-

1.2 GPU Rendering Pipeline

5

3D Application

3D API CPU − GPU Boundary

GPU Front End

Primitive Assembly

Vertex Processor

Rasterization & Interpolation

Raster Operations

Frame Buffer

Fragment Processor

Fig. 1.2. GPU graphics pipeline (inspired by [312])

itive and triangle are synonymous in this context. During primitive assembly and rasterization the following operations are executed. Individual vertices of the vertex pipeline are organized into primitives, and primitives are clipped against the viewing frustum. At the transition from the vertex pipeline to the fragment pipeline, rasterization employs scanline conversion to generate the fragments covered by an input triangle. At the same time, vertex attributes are interpolated for fragments that lie inside the input triangle. Position coordinates are transformed from normalized clip space into normalized device coordinates by a homogeneous division by the w clip coordinate. Also, the viewport transformation to window coordinates is applied. Primitive assembly, rasterization, and interpolation are fixed and cannot be programmed by the user. The subsequent fragment processor modifies fragment attributes and applies textures. Texture lookup can make use of efficient built-in bilinear or trilinear texture interpolation. Analogously to the vertex processor, fragment processing can either be used as a traditional fixed-function element or freely specified in the form of a fragment program (OpenGL lingo) or pixel shader (Direct3D lingo). The final raster operations combine the current fragment with pixel information given at the same position in the frame buffer: first, the alpha, stencil, and depth tests (the latter is also called z test) are executed; then, if a fragment passes all tests, it may be directly written into the frame buffer or composited with frame buffer information by blending. 1.2.1 Programming Model GPU-based visualization methods utilize the programmability of the vertex processor and fragment processor stages. Typical operations for the vertex processor comprise the calculation of quantities that can later be interpolated

6

1 Introduction

linearly via scanline conversion. For example, texture coordinates can be generated to attach a 3D vector field stored in a 3D texture to a surface that cuts through the vector field. Much more important, however, is the fragment processor because it allows us to compute information on a per-fragment level. Moreover, an efficient access to texture data is only possible at this stage. Therefore, the visualization methods of this book make extensive use of pixel shaders. Fragment processing can be considered as a streaming model of computation [38]. The streaming model extends the SIMD (single instruction, multiple data) concept that was used as abstraction of the OpenGL architecture [324], where each rendering pass implements a SIMD instruction that performs a basic arithmetic operation and updates the frame buffer atomically. The stream programming model represents computational locality that is not present in the SIMD model. The key elements are streams and kernels. A stream consists of a set of data elements for which similar computations are executed. A streaming processor applies a computing kernel to all elements of an input stream and writes the results into an output stream. Dally et al. [70] and Buck et al. [38] discuss how the stream model supports programs with high arithmetic and computational intensity (i.e., a large ratio of arithmetic operations to memory bandwidth). Figure 1.3 depicts the basic execution model of a GPU that applies to fragment and vertex processing alike. A graphical input element (e.g., a vertex or a fragment) is transmitted through read-only input registers into the shader program. The result of shader execution is written to output registers. The shader program has read and write access to temporary registers, and read-only access to textures and constants. Therefore, a shader can be regarded as a stream kernel that is applied to stream data held in input registers or textures. Please note that vertex programs used to have no access to texture; however, with Shader Model 3 compliant GPUs (e.g., NVIDIA GeForce 6 and GeForce 7 series), texture access is also available in vertex programs. Although the streaming model slightly restricts the broader functionality of GPUs, it is well-suited for most visualization methods of this book. Efficient visualization techniques thrive on a good mapping to the streaming model. For example, such a mapping has to achieve a uniform kernel structure with a high degree of parallelism, no branching, and little or no conditional expressions. A regular data access to textures is highly advantageous to utilize optimized data paths to internal memory. If possible, efficient built-in bilinear or trilinear interpolation should be exploited. The stream output often needs to be re-used as input to a subsequent stream computation. In this case, the output register is directly transferred into a texture by using the efficient render-to-texture functionality of GPUs. A number of specific GPU properties should be taken into account. First, the accuracy of GPUs is usually limited and might vary along the rendering pipeline. For example, color channels in the frame buffer or in textures have a typical resolution of eight bits, whereas fragment processing may take place

1.2 GPU Rendering Pipeline

7

Input Registers

Textures

Shader Program

Constants

Temp Registers

Output Registers

Fig. 1.3. Programming model for current GPUs (after [38]). A shader program processes a single input element and writes the result to an output register

at higher precision. Even floating-point accuracy within textures and fragment processing, which is provided by modern GPUs, is not comparable to double-precision numbers, which are available on CPUs. Second, the number of instructions might be limited. Therefore, the algorithms have to be designed in a way that allows a concise implementation – sometimes at the cost of accuracy – or a less efficient multi-pass rendering approach has to be pursued. Similarly, the number of indirection steps in fragment processing (i.e., dependent texture lookups) may be restricted. Third, an efficient abortion of a fragment program (texkill command) is not possible on many GPUs. Therefore, the early z test is often (mis)used to emulate such a program abortion. Fourth, GPUs can simultaneously apply numerical operations to four different color channels (red, green, blue, alpha) of a register, which can be used for an efficient implementation. Fifth, a read-back of information from GPU to main memory should be avoided because the bandwidth from GPU to CPU can be significantly smaller than the bandwidth from CPU to GPU. Finally, a simultaneous read and write access to textures is not specified; e.g., an in-place replacement of texel information is not feasible. This problem can be overcome by a ping-pong rendering scheme: two copies of a texture are held, one for the read access to “old” information (regular texture read) and the other one for writing “new” information (with a render-to-texture operation); after each computational step, both textures are swapped and their roles are exchanged, i.e., a ring buffer with two elements is employed. Although GPUs have a potential to increase visualization performance by more than one order of magnitude, an inappropriate implementation or algorithm could decrease the speed of a GPU version significantly, compared to a reference CPU version. Therefore, the above issues and restrictions have to be carefully taken into account. Specific problems and their solutions are discussed throughout this work in the context of respective visualization algorithms.

8

1 Introduction

1.2.2 APIs and Effect Files Another issue is the choice of API for GPU programming. High-level scene graph APIs, such as OpenInventor, OpenGL Performer, OpenSceneGraph, or OpenSG, are not considered in this book because they anyway employ lowlevel APIs for GPU configuration and actual rendering. Therefore, only the widespread low-level APIs OpenGL and Direct3D are used in this work. GPU programs can be developed on assembler level or with high-level shading languages. Typical assembly-like programming environments are ARB vertex programs and ARB fragment programs for OpenGL and vertex shader and pixel shader programs for Direct3D. The advantage of assembler-level shaders is the direct control of shader instructions, which allows for handcoded optimization. The disadvantages are a time-consuming and error-prone shader development and program codes that are difficult to read and maintain. These problems are overcome by high-level shading languages, such as the OpenGL shading language [207], Direct3D’s HLSL [288], or NVIDIA’s Cg [118, 312]. These high-level shading languages for GPUs are all very similarly structured, oriented along the C programming style, and strongly influenced by the Renderman shading language [152]. Old GPU programming concepts such as NVIDIA’s register combiners or texture shaders [210] do not play a role in this book because they are vendor-specific and inflexible. However, references to older publications based on such GPU programming models are sometimes included in the text. In addition to actual vertex and pixel shaders, states of the graphics pipeline strongly influence rendering results, i.e., GPU programming is more than shader development. Traditionally, different parts of the program source code (for example, in the C++ code) are responsible for setting graphics states. Effect files (FX files) were introduced in Direct3D 8 and further enhanced in Direct3D 9 [288] to simplify GPU programming. They encapsulate the whole setup and programming of the graphics pipeline within a so-called effect. An effect is described by a character string or an external text file and thus this concept is mainly referred to as effect file. The source code responsible for the GPU programming is decoupled from the main program code, which deals with CPU programming. In this way, a better maintainability and readability of the source code for both parts is achieved.

1.3 Methods and Goals The goal of this book is the development of methods that improve both the efficiency and effectiveness of visualization. Efficiency mainly concerns technical aspects of algorithms, data structures, and implementations, whereas effectiveness is strongly affected by perceptual issues related to the human observer and domain knowledge for a specific application.

1.3 Methods and Goals

9

A wide spectrum of subgoals requires an equally broad and interdisciplinary set of methods. Typical methods from practical computer science are concerned with the use of efficient algorithms and memory-friendly data structures with fast access. Due to the restricted programming model of GPUs, specific algorithms that are efficiently supported by GPUs are developed. Algorithms are typically validated by running tests on their implementation, along with performance measurements. Actual implementation on GPUs is indispensable to verify these algorithms because GPU programming is subject to various subtle pitfalls. Moreover, visualization methods are discussed in the context of the generic visualization pipeline to show differences and analogies. Parallelization on distributed-memory architectures, such as cluster computers, is another computer-science approach used in this book. Mathematical methods are equally important to achieve efficient visualization techniques. For example, appropriate numerical methods have to be employed for differential equations and level-sets, wavelets are used for multiresolution modeling and compression, and reconstruction and filter theory plays a significant role in analyzing convolution-based vector field visualization. The last major aspect takes into account human perception and cognition because visualization techniques have to target comprehension by a human observer. This book thrives on knowledge from fields like psychophysics, psychology, and neurophysiology to achieve perception-oriented rendering. Literature in these fields is analyzed to find formalized methods that can be transferred into a computer program. Likewise, well-established experience from artists, designers, and illustrators is condensed into perception-guided and non-photorealistic rendering algorithms. The generic goals of efficiency and effectiveness are pursued for specific but widely useful applications: the visualization of 3D scalar fields and vector fields in 2D and 3D. Interactivity plays a crucial role in all applications because it greatly enhances their effectiveness. For example, interactivity allows a user to explore the large parameter space of visualization and data generation, it promotes spatial perception by motion parallax, and naturally represents temporal changes of time-dependent data. Real-time capable visualization is a requirement achieved by efficient GPU techniques that accelerate the complete visualization pipeline.

2 Visualization of 3D Scalar Fields

Volume rendering is a widely used technique in many fields of application, ranging from direct volume visualization of scalar fields for engineering and sciences to medical imaging and finally to the realistic rendering of clouds or other gaseous phenomena in visual simulations and virtual environments. In recent years, texture-based volume rendering on consumer graphics hardware has become a popular approach for direct volume visualization. The performance and functionality of GPUs have been increasing rapidly, while their prices have been kept attractive even for low-cost PCs. Volume rendering targets the visualization of 3D scalar fields. A timedependent scalar field, in general, is a map s : M × I −→ R , where M is an m-dimensional manifold with boundary and I ⊂ R is an open interval of real numbers. An element t ∈ I serves as a description for time, x ∈ M is a position in space, and the result of s yields the associated scalar value. For the more specific case of volume rendering, data is given on a flat 3D manifold. A corresponding 3D time-dependent scalar field is described by s : Ω × I −→ R ,

(x, t) −→ s(x, t) .

This scalar field is defined on the 3D Euclidean space Ω ⊂ R3 and depends on time t ∈ I. For a stationary 3D scalar field, the time-dependency is obsolete: s : Ω −→ R ,

x −→ s(x) .

This chapter focuses on the visualization of scalar fields without explicit timedependency: either the input data set is already stationary or just a single time step of a time-dependent data set is considered at a time. The basic idea of direct volume visualization is to simultaneously show all important aspects of a data set in a single image. Semi-transparent rendering has to be applied to represent a full 3D data set on a 2D image plane.

12


Section 2.1 describes the prevalent optical model used for semi-transparent volume rendering. Essentially, viewing rays starting at the camera are traversed through the data set. Color contributions are accumulated while stepping along a viewing ray. This image synthesis process can be structured via the volume rendering pipeline, which consists of the stages data traversal, interpolation, gradient computation, classification, shading, and compositing (see Sect. 2.2). The volume rendering pipeline can benefit from direct GPU support. Section 2.3 describes basic volume rendering methods, focusing on real-time GPU rendering. To a large extent, this chapter addresses the visualization of data sets given on uniform grids. This type of grid has a natural representation via 3D textures or stacks of 2D textures. Both the scalar data set and its gradients can be held in these textures. An advantage of a GPU is that interpolation on textures is efficient due to built-in hardware support. Section 2.3 also discusses alternative methods that support unstructured grids, especially tetrahedral grids. Furthermore, Sect. 2.3 shows how data traversal can be implemented using graphics hardware and how classification, shading, and compositing can be handled by fragment processing. The stages of the volume rendering pipeline can be related to the steps of the more abstract visualization pipeline. The classification stage can be identified as the mapping step, whereas shading and compositing can be considered as rendering step. Direct volume visualization exemplifies the fact that the visualization pipeline should be regarded as a conceptional pipeline. The actual implementation shows that a direct combination of the three steps may be more efficient than a clear separation with a subsequent execution of the steps of the visualization pipeline. For volume rendering, the order of operations is significantly modified – the roles of the outer processing of the visualization pipeline (the filtering, mapping, and rendering steps) and the inner loop (stepping along a viewing ray) are exchanged. This chapter particularly focuses on two aspects of direct volume visualization. The first one is volume clipping, which is discussed in detail in Sect. 2.5. Clipping provides a means of selecting regions of the data set on a purely geometric basis and plays a decisive role in understanding 3D volumetric data sets because it allows the user to cut away selected parts of the volume based on the position of voxels in the data set. Clipping reduces the domain of the data set to be visualized; therefore, clipping can be regarded as an example of a selection process (with respect to the domain), which is part of the filtering step of the visualization pipeline. From an alternative point of view, one may associate clipping with the mapping step (in the form of a space-dependent mapping to opacities) or the rendering step (as a clipping of the renderable geometry). Clipping shows that the assignment to the different steps of the visualization pipeline is not always unique. In this chapter, however, clipping is considered as part of the filtering step. The second important aspect covered in this chapter is the efficient rendering of large volume data sets. Section 2.6 addresses large-data visualization by applying compression techniques, adap-

2.1 Optical Model for Volume Rendering

13

tive rendering, and parallel processing on GPU clusters. A complementary multi-bricking approach leads to constant frame rates for volume rendering with 3D textures, independent of the viewing positions (see Sect. 2.4). Both volume clipping and large-data visualization rely on efficient texture-based volume rendering as a basic technique (see Sect. 2.3).

2.1 Optical Model for Volume Rendering Direct volume visualization can be derived from optical models for volume rendering and volume shading. Here, a brief overview of a widely used approach is given. A detailed presentation of the underlying physical and mathematical models can be found in the review articles by Hege et al. [160] or Max [275]. This chapter follows the terminology used by Max. By ignoring scattering, the generic equation of transfer for light can be reduced to the simpler emission-absorption model or density-emitter model [362]. The respective volume rendering equation can be formulated as the differential equation dI(t) = g(t) − τ (t)I(t) . (2.1) dt The amount of radiative energy is described by radiance I(t), and its derivative with respect to the length parameter t is taken along the direction of light flow. The source term g(t) describes the emission of light from the participating medium (and later, through reflection as well). The extinction coefficient τ (t) defines the rate that light is attenuated by the medium. The optical properties are determined by a transfer function that assigns a value for the extinction coefficient and the source term to each sample point of the data set. Radiance and optical properties are written as scalars, which is appropriate for luminance-based gray-scale images. A wavelength-dependent behavior or multiple wavelength bands (for example, red, green, and blue) can be taken into account by computing these quantities for each wavelength separately. The volume rendering equation (2.1) can be integrated to compute the radiance that leaves the volume and finally arrives at the eye point. Integrating from the initial point t = t0 to the end point t = D along the light direction results in D D D − τ (t ) dt − τ (t ) dt + g(t) e t dt . I(D) = I0 e t0 t0

The background light that enters the volume at the position t = t0 is denoted by I0 . The radiance that leaves the volume at t = D is described by I(D). With the definition of transparency, −

T (t) = e

D t

τ (t ) dt

,

14


one obtains the volume rendering integral D I(D) = I0 T (t0 ) +

g(t)T (t) dt .

(2.2)

t0

So far, the optical model is restricted to emission and absorption. The visual realism of volume rendering can be substantially increased by considering scattering of light. A simple and widely used volume illumination model implements single scattering by assuming that light from an external light source unimpededly reaches the scattering location. Light attenuation is neglected on the way from the light source to the scattering point, which is similar to neglecting shadowing in surface graphics. The result of single scattering can be computed similarly to local illumination models known from surface rendering, such as the Phong or the Blinn-Phong models. The gradient of the scalar field is used as the normal vector in local illumination because the gradient is identical to the normal vector of an isosurface. In this way, volume illumination essentially imitates the effect of lighting isosurfaces. The source term of the volume rendering integral (2.2) can be extended to include local illumination: g(x, ω) = E(x) + S(x, ω) . (2.3) Here, the source term is given with respect to position x, which directly corresponds to the length parameter t along the light ray. The non-directional emissivity E(x) is identical to the source term in the pure emission-absorption model. The additional scattering term S(x, ω) depends on the position x, the direction of the reflected light, ω, and the properties of the light source. The integral in (2.2) is typically approximated by a Riemann sum over n equidistant segments of length ∆x = (D − t0 )/n. The approximation yields I(D) ≈ I0

n i=1

where

ti +

n i=1

gi

n

ti ,

(2.4)

j=i+1

ti = e−τ (i∆x+t0 )∆x

(2.5)

is the transparency of the ith segment and gi = g(i∆x + t0 )∆x

(2.6)

is the source term for the ith segment. Note that both the discretized transparency ti and source term gi depend on the segment length ∆x. Opacities αi = 1 − ti are often used instead of transparencies ti . When the RGB color model is applied, the source term gi is described by its RGB values. The emissive RGB part is denoted Ci . Therefore, source and opacity terms can be combined in the form of an RGBA contribution. These terms have to be adapted if the sampling distance ∆x is changed.

2.2 Volume Rendering Pipeline

15

Although the emission-absorption model is predominant in real-time volume graphics, it should be pointed out that there exist more advanced rendering algorithms that strive for a higher degree of realism by including further elements of global illumination. Background information on the physical models for advanced global illumination can be found in the physics literature, for example, in a book by Chrandrasekhar [53]. Light transport can be regarded as a special case of a generic transport mechanism described by the Boltzmann equation. Mathematical details of the Boltzmann equation are discussed, for example, by Duderstadt and Martin [100] or Case and Zweifel [47]. Transport theory can be directly applied to volume visualization, as shown by Krueger [230, 231], and to image synthesis in general, as shown by Arvo and Kirk [4]. There are many previous papers on advanced global illumination specifically designed for participating volumetric media. Due to their complexity, most of these global illumination algorithms are implemented on CPUs and do not facilitate interactive rendering. Some examples are volume photon mapping [186, 95], the zonal method for radiosity [360], the use of spherical harmonics [199], or a method for scattering in layered surfaces [151]. Global illumination and physics-based light transport for computer graphics in general are presented in both breadth and detail in textbooks by Pharr and Humphreys [333] and Dutré et al. [103]. The overview by Cerezo et al. [51] focuses on state-of-the-art rendering techniques for participating media.

2.2 Volume Rendering Pipeline The evaluation of the emission-absorption model for a given data set can be split into several subsequent stages of the volume rendering pipeline. This section reviews the volume rendering pipeline only briefly; a more detailed description is given by Pfister [330]. The following stages are commonly found in volume rendering techniques: (a) data traversal, (b) interpolation, (c) gradient computation, (d) classification, (e) shading, and (f) compositing. During data traversal, resampling positions are chosen throughout the volume. Some kind of interpolation scheme is applied at these positions to reconstruct the data set at locations that differ from grid points. Typical filters are nearest-neighbor interpolation or variations of linear filtering. Trilinear interpolation is most common for uniform grids and is also used in most of the methods presented in this book. The gradient of a discretized volumetric data set is typically approximated by using discrete gradient filters. Many gradient filters are extensions of 2D edge filters from image processing, e.g., the Sobel operator. Central differences are a popular way of computing the partial derivatives for the gradient because they require only a small number of numerical operations. Classification maps properties of the data set to optical properties for the volume rendering integral (2.2), i.e., it represents the relationship between

16


data and the source term g and the extinction coefficient τ . The classification typically assigns the discretized optical properties Ci and αi , which are combined as RGBA values. Pre-classification first maps grid points of the data set to optical properties and afterwards applies the interpolation scheme to the RGBA values. Post-classification, in contrast, first evaluates the interpolation scheme for the input data set and then assigns the corresponding optical properties. The mapping to optical properties is represented by a transfer function. The transfer function typically depends on the scalar value s, which is given at position x by the term s(x). This leads to a slightly modified description of the source term via g˜: g(x) = g˜(s(x)) . Similarly, the extinction coefficient can be written as τ˜ according to τ (x) = τ˜(s(x)) . An analogous description can be applied for the RGBA transfer function used in the discrete approach. Additional parameters are often included in transfer functions, e.g., gradient magnitude or even higher-order derivatives [212, 217] in multi-dimensional transfer functions. Volume shading can be incorporated into transfer functions by adding an illumination term, as shown in (2.3). The design of useful transfer functions, in general, is a crucial aspect of volume visualization that attracts a lot of attention [331]. The discretized volume rendering integral (2.4) can be iteratively computed by compositing. Two types of compositing schemes are most common: front-to-back and back-to-front compositing. Front-to-back compositing is used for stepping along viewing rays from the eye point into the volume. The front-to-back iteration equations are Cdst ← Cdst + (1 − αdst )Csrc , αdst ← αdst + (1 − αdst )αsrc . Color is accumulated in Cdst and opacity is accumulated in αdst . The subscript dst is an abbreviation for “destination”, denoting a variable that can be modified by an update operation. The transfer function assigns the color contribution Csrc and opacity αsrc as a representation of the optical properties at the current location on the ray. The subscript src is an abbreviation for “source”. The same terminology of “source” and “destination” is used by OpenGL in the context of frame-buffer blending. In fact, the front-to-back iteration equation can be implemented by blending, as discussed in Sect. 2.3. The values Cdst and αdst are initialized with zero before ray traversal. The iteration ends once the ray has left the volume. Then, background light can be added, corresponding to the term I0 in (2.2) and (2.4). Finally, Cdst contains the color that is transported to the eye.

2.3 Volume Rendering Approaches

17

Back-to-front compositing is an alternative scheme. Here, a viewing ray is traversed from the backside of the volume to the eye. The corresponding iteration equation is Cdst ← (1 − αsrc )Cdst + Csrc . Again, color is accumulated in Cdst , and Csrc and opacity αsrc are assigned by the transfer function. Background light is used to initialize Cdst . Note that opacity does not need to be accumulated to compute the final color. In addition to these compositing schemes, a number of alternative approaches might be used. For example, maximum intensity projection (MIP) is useful in some medical imaging applications. Sometimes, the simplified emission-only or absorption-only models are applied, which only consider the emission term or the absorption term, respectively. The MIP, emission-only, and absorption-only schemes share the advantage of being order-independent: their compositing equations are commutative and therefore do not require any particular traversal order. Thus, they do not need any kind of spatial sorting. In contrast, the emission-absorption model is order-dependent and requires spatial sorting, e.g., according to a front-to-back or back-to-front scheme. In what follows, this book focuses on the compositing schemes for the emissionabsorption model. Although the volume rendering pipeline provides the usual way of describing the structure of a volume rendering algorithm, the stages of this pipeline can be identified with steps of the generic visualization pipeline: classification corresponds to the mapping step, whereas shading and compositing implement the rendering step. The other stages (data traversal, interpolation, and gradient computation) deal with internal data handling and serve as input for the mapping and rendering steps. Since these data handling stages do not provide any non-trivial filtering element, they are not considered as part of the filter step of the visualization pipeline, but are rather included in the respective mapping and rendering steps.

2.3 Volume Rendering Approaches Several ways of implementing the generic volume rendering pipeline are discussed in this section. Volume rendering methods primarily differ in the way the volume data set is traversed. In particular, methods can be classified either as image-order approaches or as object-order approaches. Object-order methods traverse the 3D volume in its object space and project the volumetric information onto the image plane. Here, this book focuses on 2D and 3D texture slicing, which are popular object-order approaches directly supported by graphics hardware (see Sects. 2.3.1 and 2.3.2). In contrast, image-order methods use the 2D image plane as starting point for data traversal: the image is scanned and the result of volume rendering is

18


computed for each pixel on the image plane. Ray casting (Sect. 2.3.3) is the most prominent example of image-order volume rendering. There is a large body of previous work on CPU-based ray casting, but this section primarily focuses on the recent developments in GPU-based ray casting. The subsequent sections cover alternative object-order methods: shearwarp factorization in Sect. 2.3.4, splatting in Sect. 2.3.5, and cell projection in Sect. 2.3.6. Furthermore, pre-integration is discussed in Sect. 2.3.7 as a means of improving most of the previously mentioned rendering methods. Finally, Sect. 2.3.8 briefly presents some variations and extensions of volume rendering methods. In general, this book focuses on visualization methods for Cartesian or uniform grids. Therefore, volume rendering techniques for these types of grids play a dominant role in this chapter. Nevertheless, alternative methods for other grid structures, especially tetrahedral grids, are also discussed. The reader is referred to the following detailed descriptions for information on supported grid types. 2.3.1 3D Texture Slicing Texture slicing is a prominent example of an object-order approach for GPU volume rendering. The specific example of 3D texture slicing – also called 3D texture-based volume rendering – makes use of image-aligned slices [1, 41, 68]. These view-aligned slices can be considered as 2D proxy geometry for the actual 3D volume that has to be rendered. The data set itself is stored in a 3D texture that represents volume data given on a Cartesian or uniform grid. An advantage of 3D texture slicing is its good support by graphics hardware, which leads to efficient volume rendering. The stages of the volume rendering pipeline are now described in the context of 3D texture slicing. For data traversal, view-aligned planar polygons are rendered. Rasterization generates fragments on the image-aligned slices, producing sampling positions throughout the volume. Figure 2.1 illustrates the layout of the viewaligned slices. The view-dependent slice polygons are generated by the CPU and clipped against the bounding box of the volume. Sampling locations in the volume are addressed by 3D texture coordinates that are attached to the vertices of the slice polygons. Trilinear interpolation is a built-in feature of 3D texture mapping on GPUs, i.e., the interpolation scheme is directly and efficiently supported. Gradients can be pre-computed for each grid point of the data set and stored along with the scalar data. For example, the RGB channels can be used for the gradient and the alpha channel for the scalar value [485]. Data sets with pre-computed gradients need four times the memory of the original scalar data, which can be a problem because texture memory is a limited resource on most, if not all, GPUs. Alternatively, gradients can be computed on-thefly during rasterization. Here, only scalar data has to be stored in texture


19

eye

image plane virtual rays

Fig. 2.1. 3D texture slicing. View-aligned slices are rendered, leading to sampling positions marked by dots. The dashed rays indicate onto which pixels the volume samples are projected

memory, but the evaluation of the gradient needs more numerical operations and texture accesses than the approach of pre-computed gradients. Post-classification is implemented by a transfer function in the form of a color table represented by a texture. During rasterization, the interpolated scalar value is interpreted as the texture coordinate for a dependent texture lookup in the transfer function. The dimensionality of the transfer function determines the dimensionality of the dependent texture, i.e., multi-dimensional transfer functions with up to three different parameters are directly supported. Pre-classification can be implemented by paletted textures, where the index that maps to the color palette is stored in the data texture and the color palette itself represents a 1D transfer function. The shading stage is wellsupported by programmable GPUs because typical illumination models such as the Blinn-Phong model can be evaluated on a per-fragment level if the gradient information is available. Even on older graphics hardware, however, volume shading is possible. For example, Westermann and Ertl [485] introduced ambient and diffuse illumination for texture-based isosurfaces. This approach could be extended to semi-transparent volume rendering [284, 285]. Compositing is typically implemented by alpha blending with the frame buffer. Both front-to-back and back-to-front strategies with their respective compositing equations are supported by alpha blending. Accumulated opacities have to be stored for front-to-back compositing, i.e., an RGBA frame buffer is required. Current graphics hardware is often restricted to 8-bit resolution for the RGBA channels of the frame buffer. Quantization artifacts can therefore occur when opacity is close to zero; see Bitter et al. [22] for a thorough discussion of quantization effects. Texture formats with 16-bit or 32-bit resolution can be used to overcome this accuracy problem [110]; here, compositing is performed by ping-pong rendering with these high-resolution textures.

20


2.3.2 2D Texture Slicing 2D texture slicing is an object-order approach tightly related to 3D texture slicing. The main difference is that 2D slicing employs object-aligned slices instead of view-aligned slices. The scalar data set is stored in stacks of 2D textures that fill the entire bounding box of the volume. Three stacks of textures are required, one for each of the main coordinate axes. Since 2D slicing is closely related to 3D slicing, only the differences between both approaches are mentioned here. Data traversal is based on rendering slice polygons. Here, however, the polygons are parallel to one of the faces of the cube-shaped volume data set. Figure 2.2 illustrates the layout of those axis-aligned slices. One of the three stacks of 2D textures is chosen as input for the rendering process. The stack direction is determined by minimizing the angle between viewing direction and processing axis. Similarly to data traversal, the interpolation stage is affected by changing from 3D to 2D slicing. Only bilinear interpolation is applied as the object-aligned slice polygons have a one-to-one correspondence to respective 2D textures of the data set. Due to more coherent memory access and fewer numerical operations, bilinear interpolation is faster than trilinear interpolation, i.e., 2D slicing is usually faster than 3D slicing. However, trilinear interpolation is possible on additional slices [340] by applying multi-texturing. Similarly, additional copies for three texture stacks can be avoided at the cost of decreased rendering performance [340]. All other stages (gradient computation, classification, shading, compositing) are analogous to 3D slicing. In conclusion, 3D texture slicing offers some advantages compared to the alternative of using stacks of 2D textures. First, only one third of the texture memory is required for the 3D texture method; the 3D approach just holds a single instance of the volume, whereas the 2D approach has to store the complete volume for each of the three main axes. Second, view-aligned slicing through a 3D texture avoids the artifacts that occur when texture stacks are switched. Third, the intrinsic trilinear interpolation in 3D textures directly

eye

image plane virtual rays

Fig. 2.2. 2D texture slicing. Axis-aligned slices are rendered, leading to sampling positions marked by dots. The dashed rays indicate onto which pixels the volume samples are projected


21

allows for an arbitrary number of slices with an appropriate resampling on these slices, i.e., the quality of the rendering can be easily adjusted by adapting the slice distance. The main advantage of 2D slicing is its support by almost any graphics hardware and its higher rendering performance. Most of the rendering techniques in later parts of this book are compatible with 2D and 3D slicing alike. All other cases are specifically noted. 2.3.3 Ray Casting Ray casting is an image-order approach for volume rendering. The starting point is the image plane, which is scanned in a pixel-by-pixel fashion. Typically, a viewing ray is traversed from the eye point through each pixel into the volume. For improved image quality, supersampling on the image plane leads to more than a single ray per pixel. While a ray is being traversed, the volume rendering integral is evaluated for that ray. The natural compositing order is front-to-back, i.e., from the eye into the volume. Figure 2.3 illustrates the principle of ray casting. Ray casting can be regarded as the volume-rendering analogue of traditional ray tracing for surface-based geometry according to Whitted [489]. One important difference is that ray casting does not spawn secondary rays like ray tracing does. Ray casting is often used for data given on uniform or Cartesian grids. Here, it is common practice to employ an equidistant sampling along the viewing rays, as shown in Fig. 2.3. Equidistant sampling leads to the Riemann sum approximation (2.4) of the volume rendering integral. An interpolation filter is applied during sampling because the sampling positions usually differ from grid locations. Trilinear interpolation is the most popular reconstruction filter for uniform grids. The ray casting algorithm can be described by the pseudo code from Fig. 2.4. Data traversal is split in two parts. In the first part – ray setup – the initial sample position along the ray is determined by computing the

eye

image plane rays

Fig. 2.3. Ray casting idea. Rays are traversed from the eye into the volume. The volume is sampled at positions along the rays (indicated by dots) to compute the volume rendering integral

22

2 Visualization of 3D Scalar Fields Ray setup (entry position and ray direction) Traversal loop (while ray in volume) Interpolate data value at current position Perform classification and compositing Propagate position along ray End loop Fig. 2.4. Ray casting: pseudo code

first intersection between the viewing ray and the data volume. Ray setup also calculates the other parameters of the viewing ray, e.g., in the form of a root point at the eye and the viewing direction. The second part of data traversal is the actual traversal loop. Here, positions are sampled along the ray by iteratively advancing them along the ray direction. In addition, it has to be checked whether the sample position is still inside the volume. Ray traversal has to be stopped once the ray exits the volume, i.e., a mechanism for ray termination has to be included. In addition to data traversal, Fig. 2.4 explicitly includes three other stages of the generic volume rendering pipeline from Sect. 2.2: interpolation, classification, and compositing. The stages gradient computation and shading are optional and can be added as additional operations within the traversal loop. Ray casting can use the same approaches for classification, gradient computation, and shading as used for texture slicing (see Sect. 2.3.1). The implementation of compositing is described below. Ray casting is a very well-known approach for CPU-based volume rendering. It has been used early on, dating back to the 1980s, as it can be seen in the discussion by Levoy [245]. For an overview of the large body of earlier research on CPU-based ray casting, the reader is referred to the survey chapter by Kaufman and Mueller [201]. Whereas CPU ray casting is a well-established approach, its GPU counterpart is rather new: GPU ray casting for the visualization of uniform grids was introduced by R¨ ottger, Guthe, Weiskopf, Ertl, and Straßer [356], followed by a variant by Kr¨ uger and Westermann [232]. In general, GPU ray casting requires advanced fragment shader functionality that has only recently become available. The basic idea is to map the processing steps from Fig. 2.4 to a fragment program. GPUs that support loops and branching in fragment programs are most suitable for GPU ray casting. For example, Shader Model 3 compliant GPUs such as the NVIDIA GeForce 6 and GeForce 7 series or the ATI X1900 series provide this kind of shader functionality. The implementation by Stegmaier et al. [398] follows such a direct mapping of Fig. 2.4 to a fragment program. Their implementation just needs a single render pass: fragments are generated by rendering the front faces of the volume; the code from Fig. 2.4 is implemented as an assembler-level fragment program. An important element of the fragment program is the branching


23

instruction that realizes the ray-termination mechanism. A 3D texture serves as data structure that holds the data set, and graphics hardware supports trilinear interpolation within that 3D texture. Front-to-back compositing is implemented by updating the accumulated color and opacity values held in a floating-point fragment-program variable (register). Therefore, no blending with frame buffer contents is required, and compositing is always performed with high floating-point quality. The source code of the implementation can be downloaded from Stegmaier et al.’s web page (see [398]). The earlier approaches [232, 356] are designed for Shader Model 2 GPUs, without relying on the availability of branching and loops in fragment programs. This lack of functionality is overcome by multi-pass rendering that essentially splits the ray setup and the iterations of the traversal loop in several render passes. Ray termination makes use of the early z test (see [232] for details) or occlusion query in combination with the early z test (see [356] for details). Compositing is implemented by blending with accumulated values stored in the frame buffer. In general, ray casting has the advantage of directly supporting a variety of acceleration techniques. The overview by Kaufman and Mueller [201] provides references to CPU-based acceleration methods. Many of these approaches can be mapped to GPU ray casting. A prominent example is early ray termination: ray traversal can be stopped once a high level of opacity is accumulated in front-to-back volume rendering because contributions to the volume rendering integral from further away regions are negligible. The GPU implementation of early ray termination essentially requires an additional test in the ray-termination mechanism to check the current value of the accumulated opacity. For example, this extension can be added to the branching operation in Shader Model 3 GPUs [398]. Alternatively, the early z test can be used to efficiently break out of fragment processing in multi-pass ray casting [232, 356]. Another example is empty-space skipping (also called empty-space leaping), which efficiently passes over completely transparent regions. Two different GPU approaches for empty-space skipping are proposed by Kr¨ uger and Westermann [232] and Stegmaier et al. [398]. Adaptive sampling can be regarded as a generalization of empty-space skipping, modifying the sampling rate along a viewing ray according to the properties of the data set or the transfer function. For example, a larger step size can be used in almost uniform data regions, which subsumes empty-space skipping as a special case. Röttger, Guthe, Weiskopf, Ertl, and Straßer [356] discuss a GPU approach for adaptive sampling. GPU ray casting for uniform grids and 3D texture slicing rely on the same scalar field representation: a 3D texture. In addition, both techniques apply a fragment-oriented point sampling of the data set to evaluate the volume rendering integral. Therefore, almost all of the rendering techniques in later parts of this book that are designed for 3D slicing can also be used with GPU ray casting. Ray casting and 3D slicing can be regarded as basic volume rendering techniques in this book.

24


eye

image plane rays

Fig. 2.5. Ray casting for tetrahedral grids. Rays are traversed from the eye into the volume. The volume is sampled at the intersections of rays and cell boundaries (indicated by dots)

So far, ray casting has only be considered for uniform grids. However, unstructured grids can be useful in many simulation applications because their resolution can be adapted to the required accuracy of the numerical computation. For example, computational fluid dynamics (CFD) is often based on unstructured grids. In general, an unstructured grid may consist of different types of cells such as tetrahedra, hexahedra, pyramids, or prisms. By decomposing complex cells into tetrahedra, an unstructured grid can always be transformed to a tetrahedral grid. Therefore, a tetrahedral grid is the most important example of an unstructured grid. The above ray casting algorithm needs only slight modifications to facilitate tetrahedral grids. Garrity [133] describes a corresponding CPU approach, illustrated in Fig. 2.5. A viewing ray is traversed in a front-to-back fashion from cell to cell, following the faces between adjacent cells. Again, ray setup determines the first intersection with the data set in order to initialize the viewing ray. In other words, the main difference between ray casting for uniform grids and tetrahedral grids is the choice of sampling positions along rays. Weiler et al. [459] were the first to propose tetrahedral grid volume rendering on GPUs, adopting Garrity’s algorithm. Designed for Shader Model 2 GPUs, Weiler et al.’s implementation performs the ray traversal in multiple render passes. A fragment program is used to compute intersections between rays and tetrahedral cells in order to find sampling positions. In each render pass, the color and opacity contribution of the current cell is computed analogously to the GPU-based view-independent cell projection by Weiler et al. [460]. The basic idea is to pre-compute the volume rendering integral for the ray segment within a cell and to use this information during volume rendering. Section 2.3.7 discusses this pre-integration approach for tetrahedral cells [357] in more detail. One issue of the method by Weiler et al. [459] is its restriction to convex meshes. Their method does not support a re-entry of a viewing ray into the volume. This problem can be overcome by convexification: a non-convex


25

mesh is transformed into a convex mesh by filling the empty space between the boundary of the mesh and its convex hull with imaginary cells. For example, the convexification approach by Williams [490] can be employed. An alternative approach [18, 461] applies a technique similar to depth-peeling [116] to facilitate non-convex meshes. Depth-peeling decomposes the tetrahedral mesh into several depth layers each of which can be processed by the original algorithm for convex meshes. Weiler et al. [461] also include the compression of tetrahedral strips to accommodate large data sets in GPU memory. More background information can be found in the survey article by Silva et al. [394], which discusses GPU methods for tetrahedral meshes in general. 2.3.4 Shear-Warp Factorization Shear-warp volume rendering [97, 235] is another object-space approach, tightly related to 2D texture slicing. The order of volume traversal is identical: an axis-aligned stack of slices is processed in a slice-by-slice order. The main difference is that slices are not directly projected onto the final image plane, but onto the base plane, which is an intermediate image plane aligned with two of the main axes of the data set. The simplest case involves an orthographic camera. Here, the volume grid is sheared before the projection so that the projection can be performed along a direction perpendicular to the base plane. In this way, the projection of a slice can be reduced to a resampling of a 2D image. All slices are combined via compositing (in front-to-back or backto-front order), similarly to 2D texture slicing. In the final step, the image of the base plane is warped to the final image plane. The warp operation is performed only once per frame. The name of this volume rendering approach is derived from a factorization of the view transformation in a shearing part (applied to the 3D volume) and a subsequent warp part (applied to the base plane). In summary, shear-warp rendering and 2D texture slicing differ in the order of executing compositing and warping: 2D texture slicing warps each slice and performs compositing on the final image, whereas the shear-warp approach first applies the compositing scheme and only warps the intermediate image. An advantage of shear-warp volume rendering is its high efficiency provided by an optimized CPU implementation. As described by Lacroute and Levoy [235], run-length encoding of the data set and early ray termination can be incorporated. It should be noted that the implementation by Lacroute and Levoy also contains some aspects of an image-space method because early ray termination is evaluated based on image-space information. Unfortunately, the optimizations for shear-warp rendering lead to non-uniform patterns in accessing memory, which is unfavorable for a GPU implementation. Another issue is that perspective projection makes the rendering algorithm more complex and introduces additional computations, as discussed by Schulze et al. [386]. Therefore, shear-warp factorization has not yet played a role in GPU volume rendering.

26


2.3.5 Splatting Splatting is an alternative object-order approach: it traverses the data set in 3D object space and projects volumetric elements, e.g., cells or voxels, onto the image plane. A key element is to project 3D reconstruction kernels onto the 2D image plane. Westover [488] uses the term splat to describe this flattening of the kernel on the image plane. Footprint is an alternative term for the projected image of a kernel. For splatting to work, the data set has to be expressed in terms of 3D kernels. The scalar field needs to be written as si h(x − xi ) , s(x) = i

where h is the 3D kernel function, xi are center points for the kernel, and si are corresponding weights. The index i serves as label for copies of the 3D kernel. The basic objective of volume rendering is the efficient evaluation of the volume rendering integral. Let us start with a slightly simplified integration of s along ray positions x(t) parameterized by t: si h(x(t) − xi ) dt = si h(x(t) − xi ) dt . s(x(t)) dt = i

i

This equation allows us to separate the data representation si from the kernel representation h. The integration along a ray reduces the dimensionality of the 3D kernel function to its 2D projection. The footprint of the 3D kernel can be expressed as footprint(u, v) = h(x(t)) dt , where the 2D coordinates (u, v) are computed by projection along the rays: (u, v) = project(x) . Note that, for a single ray, all positions x(t) are projected onto the same image location (u, v). By using the footprint function, the overall integration can be rewritten as s(x(t)) dt = si footprint(project(x(t) − xi )) . (2.7) i

Therefore, the integral is the sum of translated copies (translation due to xi ) of the footprint function, weighted by the data values si . The idea of splatting is to pre-compute the footprint function and rely on the pre-computed images during rendering. Graphics hardware can be used for volume rendering by accessing the footprint stored in a 2D texture. Typically, spherically symmetric kernels are used because their footprint function is


27

view-independent, which greatly reduces the amount of pre-computation. The actual volume rendering integral is slightly more complex than the prototypical integral in (2.7). However, the basic idea of pre-computing the footprint of the integration works in the same way as before. More background information on footprint computation can be found in the survey chapter [64]. In general, splatting is quite flexible in the order of traversing the volume, i.e., in choosing the positions xi . Often, splatting is applied to traverse a uniform grid in a voxel-by-voxel order. It can also be used for scattered data and is especially efficient for sparse data. The only requirement is a correct spatial sorting in order to apply back-to-front or front-to-back compositing. An issue with splatting occurs when 3D kernels overlap in 3D space. Here, a separation of the footprint computation and the summation similar to (2.7) is no longer valid, leading to color bleeding and popping artifacts. Image-aligned sheet-based splatting by Mueller and Crawfis [294] offers a solution for this problem. Their approach uses image-aligned sheets (i.e., slicing slabs) to clip the 3D kernels outside a thin slab. In this way, the overlap issue is overcome, however, at the cost of more compositing steps and several footprint sections per cell. Other improvements of the original splatting technique include blur reduction described by Mueller et al. [295], an anti-aliasing method proposed by Swan II et al. [407], and perspectively accurate splatting discussed by Zwicker et al. [511]. EWA (elliptical weighted average) reconstruction can be used for anti-aliasing in splatting, as shown by Zwicker et al. [510]. Chen et al. [56] describe a GPU implementation of EWA splatting to improve the rendering speed. Other approaches to accelerate splatting include opacity-based culling according to Mueller et al. [296], post-convolution described by Neophytou and Mueller [304], and hierarchical splatting investigated by Hopf et al. [171]. Vega-Higuera et al. [443] describe interactive GPU-based splatting in the context of a medical application. 2.3.6 Cell Projection Cell projection is an object-order approach that traverses the cells of a grid and projects them onto the image plane. In general, cell projection is suitable for unstructured grids that may consist of different types of cells. A most prominent approach to cell projection is the projected-tetrahedra algorithm by Shirley and Tuchman [393]. This algorithm works for an important case of unstructured grids: grids consisting of tetrahedra, i.e., simplicial cells. A key element of the algorithm is the observation that the projection of a tetrahedron onto the image plane can be decomposed into a collection of triangles. Figure 2.6 illustrates the three classes of non-degenerate cases that can occur. A tetrahedron is either decomposed into three triangles after projection (class 1a and class 1b) or into four triangles (class 2). Triangles can be efficiently rendered by using graphics hardware – even without the need for GPU shaders. Compositing is supported in the form of frame-buffer blending.

28


cell configuration

triangle decomposition

class 1a

class 1b

class 2

Fig. 2.6. Classification of non-degenerate cases for the projected-tetrahedra algorithm: cell configuration (top) and decomposition into projected triangles (bottom)

The projected-tetrahedra algorithm can also be applied to general unstructured grids. In this case, a pre-processing step needs to split complex cells into a collection of tetrahedra. For example, hexahedral cells can be decomposed into five or six tetrahedra. Similarly, pyramid and prism cells are split into two or three tetrahedra. Working on a purely tetrahedral grid, the projected-tetrahedra algorithm consists of the following steps: 1. The cells are sorted according to their spatial structure in back-to-front or front-to-back order. 2. Each tetrahedron is classified according to its projected profile (see Fig. 2.6). The projection is decomposed into a number of triangles. 3. Color and opacity is assigned to the vertices of the triangles, based on an evaluation of the volume rendering integral. 4. The triangles are rendered and blended. One of the issues of cell projection is an efficient spatial sorting for step 1 of the algorithm. Spatial sorting is necessary for non-commutative blending, for example, for the emission-absorption model of volume rendering. Depth sorting is view-dependent and needs to be recomputed for each frame when the camera parameters are changing. Efficient sorting algorithms exist for unstructured grids [58, 280, 490]. In particular, there are specific algorithms for non-convex grids (i.e., grids with non-convex hull) [60, 399] and for cyclic grids (i.e., grids with cycles in the depth structure of their cells) [226]. All these algorithms are designed for execution on the CPU. In fact, a traditional implementation of the projected-tetrahedra algorithm uses a hybrid CPU/graphics hardware approach: steps 1–3 of the above algorithm are performed on the CPU, whereas the actual rendering and blending (step 4) is performed by the graphics hardware. A more recent approach by Callahan et al. [45] uses a GPU for parts of the sorting process: the CPU performs only an approximate spatial sorting of


29

the tetrahedral grid, followed by an accurate final sorting by GPU fragment operations. The initial sorting on the CPU is fast because it does not need to be completely accurate. Other improvements of the original projected-tetrahedra algorithm address the evaluation of the volume rendering integral in step 3, as well as the interpolation of the opacities and colors within the projected triangles. The original algorithm by Shirley and Tuchman uses a linear interpolation of color and opacity, which may introduce errors. Stein et al. [399] overcome this problem for the case of linear transfer functions. To compute the interpolated opacity, they use a 2D texture parameterized by the thickness of the projected tetrahedron and the averaged absorption coefficient. Color is still interpolated linearly. Pre-integration by R¨ ottger et al. [357] is capable of accurately evaluating the volume rendering integral within projected triangles. The idea is to compute the integral in a pre-processing step and to use this information during actual rendering. Section 2.3.7 describes pre-integrated volume rendering in general. Another subtle problem occurs for perspective projection: linear interpolation on the image plane (for the parameters of pre-integration) is inconsistent with a linear interpolation of scalar values and the thickness parameter in 3D object space. However, linear (i.e., barycentric) interpolation is the typical interpolation scheme in a tetrahedron. Kraus et al. [229] show that this problem can be overcome by perspectively correct interpolation. One of the main issues of the projected-tetrahedra algorithm and its variants is that the decomposition into triangles is view-dependent. Therefore, the geometric description of the projected triangles needs to be transferred to graphics hardware for each frame, leading to a possible bottleneck. Ideally, we would like to use a rendering primitive that represents a volumetric tetrahedron, not a planar triangle. Then, we could use a static geometric description of the tetrahedral grid (provided the data set is static) – without the need for repeated transfer of data to graphics hardware. Unfortunately, GPUs have no native support for tetrahedral primitives. This problem can be overcome by realizing a view-independent scan conversion of tetrahedra by means of GPU programs. For example, Weiler and co-workers [458, 460] describe a GPU approach that allows us to render a tetrahedron without an explicit decomposition into triangles. The basic idea is to adopt GPU ray casting (see Sect. 2.3.3) for a single tetrahedron, whose implementation primarily relies on fragment programs. Wylie et al. [497] discuss an alternative approach for a GPU-based projected-tetrahedra algorithm based on vertex processing. Although view-independent scan conversion works well for displaying tetrahedral cells, it still requires spatial sorting (step 1 of the projected-tetrahedra algorithm) when a non-commutative compositing scheme is applied. For example, for the emission-absorption model a spatial sorting with a corresponding transfer of grid data to the GPU is required. Therefore, view-independent cell rendering is best suited for order-independent compositing, such as emissiononly or absorption-only models.

30


More information and further references on cell projection in general can be found in the survey articles by Kaufman and Mueller [201] or Silva et al. [394]. 2.3.7 Pre-Integrated Volume Rendering This section briefly describes pre-integrated volume rendering, following the detailed discussion by Kraus and Ertl [228]. The discrete version (2.4) of the volume rendering integral (2.2) converges to the true result when the sampling distance ∆x → 0. The sampling theorem requires the sampling rate to be higher than the Nyquist frequency. It is important to note that the volume rendering integral computes the contributions of the scalar data set after the transfer function has been applied. Therefore, the Nyquist frequency depends on the properties of both the actual data set and the transfer function. Non-linear features of the transfer function may increase the required sampling rate significantly because this sampling rate depends on the characteristic of the scalar data set and the transfer function [225, 385]. Therefore, it is not sufficient to sample the volume rendering integral with the Nyquist frequency of the scalar data set. Pre-integrated volume rendering improves the approximation of the volume rendering integral to overcome the aforementioned issues. Pre-integrated classification assumes a piecewise linear, continuous reconstruction of the scalar field along a viewing ray. With this assumption, the contribution of a ray segment (which shows a linear behavior of its scalar value) can be precomputed in a preprocessing step according to an accurate numerical solution of the volume rendering integral (2.2). For example, the discrete Riemann sum of this rendering integral (2.4) can be evaluated with a very small sampling distance ∆x to obtain an accurate solution. The pre-computed contributions of a ray segment depend on three parameters: the scalar value at the beginning of the segment, the scalar value at the end point of the segment, and the length of the segment. Therefore, the pre-integrated values can be stored in a 3D array with respect to these three parameters. During runtime, the contribution to the current segment is obtained from this 3D lookup-table. The opacities and colors of several subsequent segments along a viewing ray are blended to a final color according to one of the compositing schemes from Sect. 2.1 – either a front-to-back or a back-to-front scheme is possible. In the literature, the idea of pre-integration was first used for cell projection. Under the assumption that the color is constant in a cell, pre-integration is related to the shading of polyhedral cells according to Max et al. [280]. For linear transfer functions, Stein et al. [399] use a 2D lookup-table parameterized by the thickness of the projected tetrahedron and the averaged absorption coefficient. Full pre-integration is introduced by R¨ ottger et al. [357] in the context of the projected-tetrahedra algorithm. Their observation is that barycentric interpolation is the natural interpolation scheme for tetrahedra. Since barycentric interpolation is linear, the linear behavior that is assumed


31

L

sf

sb

slab

Fig. 2.7. Slab-based representation for pre-integrated texture-based volume rendering. The scalar data values at the entry and the exit point of the viewing ray are denoted by s f and s b , respectively. The thickness of the slab is denoted by L

along a viewing ray during pre-processing is compatible with the interpolant for the 3D data set, i.e., no error is introduced by the assumption of a piecewise linear interpolation along light rays. Pre-integration can also be used for a view-independent projection of tetrahedra [458, 460]. A strength of the concept of pre-integration is its applicability to other volume rendering methods. For example, slice-based rendering for uniform grids can be enhanced by pre-integration [111]. Both 2D and 3D slicing can be combined with pre-integration: the original point-sampling on slices is replaced by a slab-based representation that assumes a linear interpolation along a viewing ray crossing a slab. Figure 2.7 illustrates this slab-oriented point of view. A slab is constructed from the two adjacent slices, i.e., two slices have to be considered at a time (instead of only one slice in slice-based rendering). The scalar values at the entry face, sf , and the exit face of a slab, sb , serve as parameters for the lookup in the pre-integration table. When a fixed slab thickness L is assumed, the pre-integration table depends only on the two scalar values. The lookup in this 2D pre-integration table can be implemented by means of a dependent texture lookup within a fragment program [111]. It should be noted that the assumption of a piecewise linear behavior along a viewing ray is not compatible with the trilinear interpolation in uniform grids. However, the error introduced by this approximation is typically negligible – at least it tends to be much smaller than the approximation error caused by point-sampling with a Riemann sum computation along viewing rays. Pre-integration can also be incorporated in other volume rendering techniques: shear-warp volume rendering according to Schulze et al. [385], CPUbased ray casting according to Knittel [222], and special-purpose hardware for ray casting according to Meißner et al. [286]. R¨ ottger, Guthe, Weiskopf, Ertl, and Straßer [356] use pre-integration in the context of GPU-based ray casting in uniform grids. Analogously, pre-integration can be used for GPUbased ray casting in tetrahedral grids, as shown by Weiler et al. [459]. For the projected-tetrahedra algorithm, Kraus et al. [229] propose a perspectively correct interpolation of texture coordinates for the lookup in the pre-integration

32


table. They also present an approach for a more accurate representation of the pre-integration table. 2.3.8 Variants and Extensions Frequency domain volume rendering – also called Fourier volume rendering – is an approach that differs significantly from the previous ways of performing volume rendering. Frequency domain volume rendering, introduced by Levoy [246] and Malzbender [268], makes use of the projection slice theorem and computations in the frequency domain. The key observation is that projections in the spatial domain correspond to a slicing in the frequency domain. The extraction of one slice in the frequency domain and the subsequent inverse Fourier transformation to the image plane lead to a computational complexity O(m2 log m), where m is a measure for the size of a slice in one dimension. Therefore, frequency domain volume rendering has a lower computational complexity than other volume rendering methods: the rendering algorithms discussed earlier tend to be O(n3 ), where n is the size of the data set in one dimension. A problem of frequency domain volume rendering is its limitation to x-ray images (i.e., an absorption-only model). Totsuka and Levoy [420] address the quality issues of frequency domain volume rendering by including depth cues and shading calculations. Illumination models are also discussed by Entezari et al. [113]. While the above papers are exclusively based on CPU implementations, Viola et al. [450] describe how frequency domain volume rendering can be mapped to an efficient GPU implementation. Another line of research targets volume illumination models beyond the emission-absorption model (see Sect. 2.1). A survey of advanced rendering techniques for participating media is presented by Cerezo et al. [51]. Unfortunately, a general model of global illumination for participating media leads to complex numerical computations that typically prohibit interactive rendering. In fact, most of these rendering methods are implemented on CPUs and limited to non-interactive rendering. However, restricted models of global illumination can be employed for real-time volumetric rendering on GPUs. A first, yet important, step toward increased optical realism is the support for shadowing. For example, Kniss et al. [218] include shadow computations in 3D texture slicing by using a common slicing direction for simultaneously accumulating the weights for shadowing and the color and opacity contributions for the final image. In this way, only little computational and memory costs are added to the costs of emission-absorption rendering. Zhang and Crawfis [504, 505] independently developed a similar shadowing method for splatting. One step further toward optical realism is to add translucent light transport within a volume, taking into account multiple scattering. Typically, the scattering model needs to be restricted to allow for real-time rendering. Kniss et al. [220, 221] follow this approach in their GPU implementation of multiple scattering based on texture slicing. In important application of global

2.4 Maintaining Constant Frame Rates in Texture-Based Volume Rendering

33

illumination is GPU-based cloud rendering, as discussed by Riley et al. [345] and Schpok et al. [381]. In a related paper, Riley et al. [346] describe the GPU rendering of atmospheric phenomena. Magnor et al. [267] propose a multi-resolution model to approximate multiple scattering in astronomical reflection nebulae and demonstrate how their model leads to an efficient GPU implementation. Finally, Hegeman et al. [162] introduce a lighting model for inhomogeneous media that can be mapped to programmable graphics hardware.

2.4 Maintaining Constant Frame Rates in 3D Texture-Based Volume Rendering Texture-based volume rendering is one of the most popular approaches for interactive volume visualization due to its direct and efficient support by graphics hardware. Therefore, the remainder to this book focuses on texture slicing as basic technique. This section, in particular, addresses the issue of efficient rendering by means of 3D texture-based volume visualization. 3D texture slicing typically relies on a trilinear interpolation of the input data set. Since this interpolation scheme needs access to a neighborhood of eight voxels, 3D texture slicing is associated with high bandwidth requirements for data transfer from texture memory. Usually, the texture cache is optimized for fetch operations in 2D textures, which are important for most applications. On the other hand, different hardware architectures use different memory layouts for 3D textures. In one approach, the volume is essentially built from a collection of 2D textures in a slice-by-slice fashion. Here, a good caching strategy is only available within a 2D slice, but not along the stacking axis. Therefore, the cache behavior greatly depends on the order of accessing the 3D texture and the performance of volume rendering can become viewdependent. This characteristic is inappropriate for real-time applications that have to maintain constant frame rates. In a second approach, a memory layout is chosen that achieves a comparable cache coherence along all directions. Here, the volume is not constructed slice-by-slice, but in a more complex, staggered manner. Both memory layouts can be found in today’s GPUs since there are good reasons for both approaches. For example, the latter architecture shows a uniform performance behavior for volume rendering and therefore is well-suited for real-time applications like virtual environments. On the other hand, the slice-by-slice approach allows the application to update a volume slice very efficiently (because of its direct mapping to the memory); with the superbuffer extension [327] or the frame-buffer object extension [390], even a direct rendering into a volume slice is possible. With the increasing popularity of general-purpose computations on GPUs (GPGPU [142]), this ability will become even more important in the future.


IMAGE: WEISKOPF, WEILER, ERTL [484]

c 2004 IEEE

34

Fig. 2.8. Partitioning a volume data set into 43 bricks

This section describes a modification of 3D texture-based volume rendering that maintains approximately constant frame rates on any GPU architecture. This approach was proposed by Weiskopf, Weiler, and Ertl [484], and the following discussion is based on their work.1 The idea of their technique is to split the volume into smaller sub-volumes, i.e., bricks [145]. These bricks can be oriented in different directions; thus, the varying performance for different viewing directions is balanced out within a single image. Multi-Oriented Bricking The volume can be partitioned into smaller bricks to overcome the potential view dependency in texture-based volume rendering. Figure 2.8 shows an example in which the volume data set (that is visualized as a sphere) is split into sub-volumes, 43 in number. The borders of the bricks are marked by white lines. These bricks can be oriented in different directions. Figure 2.9 illustrates (in the simplified analogue of a 2D drawing) that the number of bricks which are oriented along the viewing direction can be kept constant. Therefore, the number of elements that can be rendered at high speed is independent of the viewing direction. Note that the “fast” bricks may change. For example, in the left image of Fig. 2.9, the bricks in light gray lead to higher rendering speed, whereas, in the right image, the bricks in dark gray are faster to render. This idea works as long as the whole volume is in the visible view frustum and roughly the same number of fragments are associated with each brick. These two assumptions are only partly fulfilled in real applications. Here, the uniformity of rendering speed is the better the finer the granularity of the 1

c 2004 IEEE. Reprinted, with permission, from Weiskopf, Weiler, and Ertl [484]


Eye

35

Eye IMAGE: WEISKOPF, WEILER, ERTL [484]

c 2004 IEEE

Fig. 2.9. For any viewing direction, two bricks are oriented along the viewing direction and the other two bricks are roughly perpendicular. The lines inside the bricks indicate the orientation of internal slices for the corresponding 3D texture

bricking is chosen, i.e., the smaller the sub-volumes become. On the other hand, finer granularity leads to a larger number of slicing polygons to be computed and rendered because each slice through the complete volume has to be partitioned into sub-slices for the different bricks. Moreover, a brick should have a one-texel-wide overlap with neighboring bricks [145]. Therefore, additional texture memory is needed, of which the consumption increases when the number of bricks is increased. Note, however, that the sizes of the additional bricks can be chosen independently of the other bricks’ sizes and thus only little memory is wasted by the restriction to power-of-two textures. In summary, the granularity should be optimized by taking into account the above aspects. The optimal number of bricks is highly dependent on the application and the characteristics of user navigation. From experience, bricking into 23 or 43 sub-blocks leads to reasonable results, as long as the viewing parameters are not too extreme. Corresponding performance measurements can be found in the following section. Bricking is frequently used in volume rendering already. For example, large data sets that do not fit into texture memory at once can only be handled by bricking [219, 237, 451]. The same approach is also taken in Sect. 2.6 for large-data volume visualization on a GPU cluster computer. Moreover, bricking overcomes the waste of texture memory that occurs when data sets of arbitrary size have to be extended by empty space to meet the restriction to power-of-two textures. Therefore, bricking is already available in many volume rendering applications (e.g., in OpenGL Volumizer [19] or the volume node of OpenSG [316]) and multi-oriented bricking causes only minimal additional implementation efforts. Thus, multi-oriented bricking is a practical and easy-to-handle solution that, in particular, improves the usage of volume rendering in real-time sensitive applications such as virtual reality and visual simulations. Typical implementations of virtual environments freeze the overall frame rate to the lowest achievable frame rate, which would be un-

36


acceptable if the ratio between maximum and minimum rendering speeds is large. Performance Comparison This section shows a performance analysis for the bricking approach and compares the bricking method with the original non-bricking rendering. Two typical representatives for the two types of memory layout are used: an ATI Radeon 9800 Pro as an example of a GPU with slice-oriented layout and an NVIDIA GeForce FX 5950 Ultra as an example of a graphics board with an optimized memory structure for volume rendering. The size of the test data set is 2563 and shows a spherically symmetric behavior with a linear dependency on the distance from the center point. Figure 2.8 depicts a volume visualization of this data set. Pre-classification is used with a two-component 3D texture that stores pre-computed luminance and alpha values. The slicing distance is kept constant, i.e., the number of slices may change with the viewing direction. Just the fixed-function vertex and fragment pipeline is used. In this way, the test scenario focuses on the 3D texture access speed. The viewport has a size of 8002 pixels in all performance tests. Figure 2.10 compares the rendering speed for the two architectures, based on a single brick (i.e., standard view-aligned volume slicing). The rendering speeds are shown in fps (frames per second) along the vertical axis. The horizontal axis describes the viewing direction in degrees. The angles denote a rotation of the volume about the y axis, which forms a vertical line on the viewing plane. The z axis is assumed to be parallel to the stacking axis for building the 3D texture from slices. For an angle of zero, the viewer looks along the z axis of the texture. Figure 2.10 demonstrates that the rendering performance of the slice-oriented architecture heavily depends on the viewing angle. In this case, the minimum and maximum frame rates form a ratio of 1:4.8. In contrast, the other hardware architecture maintains a much more uniform frame rate; there is only a 12 percent difference between minimum and maximum frame rates. Figure 2.11 shows that multi-oriented bricking achieves a much more even rendering performance than the standard approach for the Radeon 9800 Pro GPU. Here, bricking is employed with a total number of 43 sub-volumes. The directions of the bricks are alternated between the x, y, and z axes, based on the index of the bricks. The ratio between minimum and maximum frame rates is reduced to 1:1.2. Figure 2.12 shows the same comparison between 43 bricking and standard volume rendering for the FX 5950 Ultra GPU. The performance numbers indicate that the bricking approach is a little bit slower, which is mainly caused by the additional work for the increased number of slice polygons. These measurements illustrate how much performance penalty is associated with bricking: the overall rendering speed is reduced by less than five percent.


37

Single-brick 3D texture-based rendering 50

FX 5950 Ultra Radeon 9800 Pro

45 40 35 fps

30 25 20 15 10 5 0 0

50

100

150

200

250

300

350

rotation angle (in degrees)

Fig. 2.10. Single-brick volume rendering on FX 5950 Ultra and Radeon 9800 Pro Single-brick vs multi-brick rendering (Radeon 9800 Pro) 50

Single-brick rendering Rendering with 4*4*4 bricks

45 40 35 fps

30 25 20 15 10 5 0 0

50

100

150

200

250

300

350


Fig. 2.11. Comparing volume rendering with single brick and 43 bricks on Radeon 9800 Pro Single-brick vs multi-brick rendering (FX 5950 Ultra) 50

Single-brick rendering Rendering with 4*4*4 bricks

45 40 35 fps

30 25 20 15 10 5 0 0

50

100

150

200

250

300

350


Fig. 2.12. Comparing volume rendering with single brick and 43 bricks on FX 5950 Ultra c 2004 IEEE ALL IMAGES: WEISKOPF, WEILER, ERTL [484]

38


2.5 Volume Clipping

IMAGE: WEISKOPF, ENGEL, ERTL [469]

c 2002 IEEE

Volume clipping plays a decisive role in understanding 3D volumetric data sets because it allows the user to cut away selected parts of the volume based on the position of voxels in the data set. Clipping is an important example of a filtering method in the volume rendering pipeline because it may be the only way to uncover important, otherwise hidden details of a data set. This geometric approach can be regarded as complementary to the specification of transfer functions that are based on the data values and their derivatives only. Therefore, a combination of data-driven transfer functions and geometryguided clipping can be used to achieve effective volume visualizations. Although clipping is frequently used in volume rendering applications, only a straightforward approach is usually adopted – with one or multiple clip planes serving as clip geometry. For example, Van Gelder and Kim [436] use clip planes to specify the boundaries of the data set in 3D texture-based volume rendering. Although some clip geometries can be approximated by multiple clip planes, many useful and important geometries are not supported: cutting a cube-shaped opening into a volume (Fig. 2.13) is a prominent example. To achieve more flexibility in volume clipping, a number of more advanced clipping techniques are described in this section. They can handle complex clip geometries and are designed to be used in the context of GPU-based volume rendering. Although the presented clipping techniques are discussed in combination with texture slicing, most of them could be adapted for GPU ray casting as well. Two basic approaches to volume clipping, as proposed by

Fig. 2.13. Volume clipping in an MR data set

2.5 Volume Clipping

39

Weiskopf, Engel, and Ertl [469, 470], are discussed in the beginning. In the first approach (Sect. 2.5.1), a clip object is represented by a tessellated boundary surface. The basic idea is to store the depth structure of the clip geometry in 2D textures whose texels have a one-to-one correspondence to pixels on the viewing plane. The depth structure of the clip object can be used by fragment operations to clip away unwanted parts of the volume. This depthbased approach is particularly well-suited for producing high-quality images because clipping is performed with per-pixel accuracy. In the second approach (Sect. 2.5.2), a clip object is voxelized and represented by an additional volume data set. Clip regions are specified by marking corresponding voxels in this volume. The description of both approaches is based on Weiskopf, Engel, and Ertl [470].2 These approaches were originally designed for slice-based volume rendering. The extension to pre-integrated volume rendering needs a couple of modifications that were introduced by R¨ ottger, Guthe, Weiskopf, Ertl, and Straßer [356]. Besides the core clipping techniques for volume visualization, issues related to volume illumination are specifically considered. Lighting introduces further information on the spatial structure and orientation of features in the volume data set and can thus facilitate a better understanding of the original data. The combination of volume shading and volume clipping, however, introduces issues of how illumination needs to be computed in the vicinity of the clipping object. On the one hand, the orientation of the clipping surface should be represented. One the other hand, properties of the scalar field should still influence the appearance of the clipping surface. Weiskopf, Engel, and Ertl [470] proposed an optical model that takes into account a consistent shading of the clipping surface. This model is described in Sect. 2.5.4. In addition to applications in scientific visualization, volume clipping can play an important role in volume illustrations. As detailed later in Sect. 4.4.7, volume illustration is an example of non-photorealistic rendering (NPR) and it has the goal of emphasizing important elements of an image, while deemphasizing or removing unimportant elements. Clipping is an extreme, yet very useful basis technique for such a volume illustration because it allows us to completely remove unimportant regions. Section 4.4.7 also discusses the generalization to the rendering of deformed volumes and it describes how deformation can be used for illustrative rendering. Similarly, volume clipping is related to volume sculpting (see, e.g., Wang and Kaufman [453] or Perry and Frisken [329]), although the typical focal point of sculpting is modeling, not rendering. 2.5.1 Depth-Based Clipping This section focuses on rendering algorithms for volume clipping that exploit the depth structure of the clip geometry. In all depth-based approaches, it is 2

c 2003 IEEE. Reprinted, with permission, from Weiskopf, Engel, and Ertl [470]

40

2 Visualization of 3D Scalar Fields ray eye

image plane

object

depth structure inside

z value IMAGE: WEISKOPF, ENGEL, ERTL [470]

c 2003 IEEE

Fig. 2.14. Depth structure of a clip geometry for a single pixel in the frame buffer

assumed that clip objects are defined by boundary surface representations. Graphics hardware expects surfaces to be tessellated – usually in the form of triangular meshes. Figure 2.14 illustrates the depth structure of a typical clip object. In what follows, the underlying 3D scenario is reduced to just a 1D geometry along a single light ray that originates from the eye point and reaches into the scene. From this point of view, only the corresponding single pixel on the image plane is taken into account. Therefore, the following descriptions can be mapped to operations on those fragments that correspond to the respective position of the pixel in the frame buffer. Let us start with a generic algorithm for clip objects of arbitrary topology and geometry. First, the depth structure of the clip geometry is built for the current pixel. This structure stores the depth values for each intersection between eye ray and object. In this way, the 1D depth values are partitioned into intervals whose bounds are given by the boundaries of the objects. In addition, the intervals are classified either as inside or as outside of the clip object. The classification is determined by the clip geometry and can be based on the orientation of the tessellated surface or on the direction of normal vectors associated with the clip geometry. For a closed, non-selfintersecting clip surface, the classification alters repeatedly from outside to inside and vice versa. The user can choose to clip away either the inside or outside of the clip object. In this way, the intervals are uniquely specified as visible or invisible. Second, the rendering of each fragment of a volume slice has to check to which class of interval the fragment belongs. Based on the visibility, the fragment is blended into the frame buffer or clipped away. As mentioned before, intervals are either marked as visible or invisible. The term volume probing is used for a scenario where the volume is clipped away outside the clip geometry and the volume inside the clip object remains unchanged. Conversely, a volume cutting approach inverts the role of the visibility property. Here, only the volume outside the clip object remains visible.

2.5 Volume Clipping

41

The following issues have to be considered for an actual implementation of the above algorithm. What kind of data structure is appropriate to store the depth structure? How is the depth structure generated from clip geometry? How is the visibility property of a fragment evaluated? To be more specific, the question is how the above algorithm can be efficiently implemented on graphics hardware. Since the depth structure is stored on a per-pixel basis, a pixel-oriented mechanism has to be considered, such as a texture on the graphics board. Depth-Based Clipping against a Single Boundary A straightforward approach exploits the depth buffer as storage medium for interval boundaries. A major disadvantage of this approach is the fact that only one depth value per pixel can be stored. Therefore, just a single boundary can be implemented, which greatly reduces the fields of application. The implementation of this approach begins with rendering a simple clip geometry to the depth buffer, i.e., the depth structure is generated from the clip geometry by standard scan conversion of the graphics hardware. Then, writing to the depth buffer is disabled and depth testing is enabled. Finally, the slices of the volume data set are rendered and blended into the frame buffer. Here, the depth test implements the evaluation of the visibility property of a fragment. The user can choose to clip away the volume in front or behind the geometry, depending on the logical operator for the depth test. Two-Pass Rendering for Convex Volume Cutting Volume cutting with a convex and closed clip object can be implemented by two-pass rendering based on depth tests. A convex and closed object implies that the number of boundaries along the depth structure is not larger than two because, regardless of the viewing parameter, an eye ray cannot intersect the object more than twice. Concave clip geometries can benefit from this approach as well: a concave clip geometry can often be approximated by two boundaries without introducing inappropriate artifacts. The algorithm is illustrated in Fig. 2.15. First, the depth values zback for the second boundary are determined by rendering the back faces of the clip geometry into the depth buffer. The depth buffer contains the initial value for the far plane of the viewing frustum in regions where the clip object is not apparent. Then, volume rendering is applied with the depth test set to “greater” and without modifying the depth buffer. In this way, the volume behind the clip object is drawn (cf. the gray portion in Fig. 2.15). The second rendering pass is responsible for drawing the volume in front of the clip object and in regions not covered by the clip object (i.e., the hatched portion in Fig. 2.15). First, the depth buffer is cleared and then filled with the depth values zfront for the first boundary by rendering the front faces of the clip geometry. Afterwards, volume rendering is accomplished with the depth test set to “less” and without modifying the depth buffer.

42

2 Visualization of 3D Scalar Fields rendered in second pass

eye

convex clip object

rendered in first pass

rectangular volume IMAGE: WEISKOPF, ENGEL, ERTL [470]

c 2003 IEEE

Fig. 2.15. Illustration of depth-based two-pass rendering for a convex clip geometry

Single-Pass Rendering for Convex Volume Probing Based on Depth Clipping A volume probing approach for a convex clip object is now considered. The corresponding maximum number of two boundaries along the depth structure can be implemented by single-pass rendering. The basic algorithm for depth-based volume probing is as follows. First, the depth values zfront for the first boundary are determined by rendering the front faces of the clip geometry into the depth buffer. Second, the contents of the depth buffer are stored in a texture and the fragment shader is configured to shift the depth values of all fragments in the following rendering passes by −zfront . Third, the depth buffer is cleared and the backside of the clip geometry is rendered into the depth buffer (with depth shift enabled) to build the second boundary. Finally, slices through the volume data set are rendered and blended into the frame buffer, without modifying the depth buffer. However, depth shift and depth testing have to be enabled. The following discussion describes in more detail how the depth of a fragment determines its visibility. The visibility depends on both depth clipping and depth testing. The decision whether a fragment passes depth clipping and depth testing can be expressed by a Boolean function df (zf ), where zf is the depth of the fragment. Depth values are assumed to be described with respect to normalized window coordinates, i.e., valid depth values lie in [0, 1]. In general, the value of df (zf ) can formally be written as a logical expression, df (zf ) = dclip (zf ) ∧ (zf op zb ) ,

(2.8)

with dclip (zf ) = (zf ≥ 0) ∧ (zf ≤ 1) . In this notation, ∧ symbolizes a logical “and”. The function dclip (zf ) describes depth clipping against the bounds 0 and 1 of the view frustum. The binary

2.5 Volume Clipping

43

ray eye convex clip object

removed by clipping test

removed by depth test

visible

z

front

z

back IMAGE: WEISKOPF, ENGEL, ERTL [470]

c 2003 IEEE

Fig. 2.16. Illustration of depth-based volume probing for a convex clip geometry

logical operator op is used for depth testing against zb , the current entry in the depth buffer. Figure 2.16 illustrates how the clipping algorithm operates on the depth structure. A shift by −zfront is applied to all depth values. By rendering the back faces of the clip object with the same shift, entries in the depth buffer are initialized to zback − zfront , where zback is the unmodified depth of the back face’s fragment. During the final rendering of the volume slices, the logical operator for the depth test is set to “≤”. Therefore, (2.8) can be written as df (zf ) = ((zf − zfront ≥ 0) ∧ (zf − zfront ≤ 1)) ∧(zf − zfront ≤ zback − zfront ) = dprobe (zf ) ∧ (zf ≤ 1 + zfront ) , with dprobe (zf ) = (zf ≥ zfront ) ∧ (zf ≤ zback ) .

(2.9)

The term dprobe (zf ) exactly represents the required logical operation for displaying the volume only in the interval [zfront , zback ]. The last term, (zf ≤ 1 + zfront ), implies just a clipping against a modified far clipping plane. A shift of depth values by −zfront can be efficiently accomplished on a per-fragment basis by modern graphics hardware. The implementation on NVIDIA GeForce 3/4 is described in detail by Weiskopf, Engel, and Ertl [469]. A high-resolution texture is used to store the respective depth values zfront and a fragment program is employed to replace the z value of a fragment by z − zfront . The details of the vertex and fragment programs are also presented in [469]. Figure 2.17 illustrates clipping in a medical MR data set. Image (a) shows the data set without clipping. Depth-based volume probing is applied in image (b). This picture demonstrates that depth-based volume clipping allows for geometries with a large number of triangles – such as the Stanford bunny.



c 2002 IEEE

44

(a)

(b)

Fig. 2.17. Depth-based volume probing for an MR data set: (a) shows the data set without clipping; (b) illustrates depth-based volume probing with the Stanford bunny as clip object

Single-Pass Rendering for Convex Volume Cutting Based on Depth Clipping The previous clipping mechanism can also be used to invert the role of the visibility property. Volume cutting replaces the above volume probing approach, i.e., only the volume outside the clip object remains visible. Once again, a maximum number of two boundaries along the depth structure is assumed. By inverting (2.9) for volume probing, one obtains dcutting (zf ) = (zf ≤ zfront ) ∨ (zf ≥ zback ) ,

(2.10)

for volume cutting, with “∨” representing the logical “or” operator. The cutting function dcutting (zf ) is almost identical to the negation of the probing function, ¬ dprobe (zf ) – except for the fact that zfront and zback satisfy the Boolean functions for both volume probing and volume cutting. Unfortunately, logical operations for depth clipping and depth testing have to be of the form of (2.8), which is not compatible with (2.10). The solution to this problem is based on the following considerations. First, a simplified scenario is discussed. The expression (x ≥ −1) ∧ (x ≤ 1) can be rewritten as (f (x) ≥ −1) ∧ (f (x) ≤ 1) by introducing an identity function f (x) = x. The valid values x are determined by the intersection of f (x) with the horizontal lines at ±1. Now a similar situation is considered in which the function f (x) is replaced by another function g(x) = 1/x. In this case, valid values of the logical expression (g(x) ≥ −1) ∧ (g(x) ≤ 1) also satisfy (x ≤ −1) ∨ (x ≥ 1). Therefore, a mapping between the form of (2.10) and the required form of (2.8) can be achieved. As a matter of fact, there are other possible choices of mapping functions to achieve a similar inversion of z clipping. The advantage of a hyperbolic mapping is its availability in the fragment operations unit even on older graphics boards.

2.5 Volume Clipping

45

1

z scale 0 z mid

z back

z

z front valid z values IMAGE: WEISKOPF, ENGEL, ERTL [469]

c 2002 IEEE

Fig. 2.18. Schematic illustration of the depth structure for volume cutting via 1/z mapping

The adaptation of this idea to graphics hardware needs to take into account that window coordinates lie in [0, 1] and not in [−1, 1]. Moreover, the vertical asymptote of the hyperbola must not meet the origin, z = 0, but has to be located at the midpoint of the clip geometry. The midpoint is obtained by zmid = (zfront +zback )/2. In addition, the actual distance between the front and the back boundaries of the clip geometry is taken into account by the scaling factor zscale = (zback − zfront )/2. Figure 2.18 illustrates how the quantities of the clip object can be used for volume cutting via 1/z mapping. The correct function has to be 1 1 zscale (2.11) + , g(z) = 2 z − zmid 2 in order to produce the graph of Fig. 2.18. The algorithm for depth-based volume cutting can now be formulated as follows. First, the depth values of the front faces of the clip geometry are rendered into the depth buffer. Then, these values zfront are transferred to main memory. Analogously, the depth values of the back faces, zback , are determined and copied to main memory. Based on zfront and zback , the CPU computes zmid and zscale and stores them as first and second components of a high-resolution texture. A fragment program is enabled to implement the mapping according to (2.11) by employing the above mentioned texture to modify the depth value of each fragment. Finally, slices through the volume data set are rendered and blended into the frame buffer. In contrast to volume probing described in the previous section, the depth buffer does not contain any information on the clip geometry and depth testing is not required. Volume cutting is only based on the data from the high-resolution texture and on view frustum clipping.

46


Once again, the paper by Weiskopf, Engel, and Ertl [469] contains details of the implementation on GeForce 3/4, including both the fragment program and the vertex program that issues texture coordinates on a per-vertex basis. Convex Volume Clipping Based on Shadow Test The two previous clipping techniques are based on high-resolution textures, transformation of depth values, and view frustum clipping. An alternative, similarly structured approach exploits shadow mapping and the shadow test to achieve comparable results. A related approach is taken by Everitt [116] for spatial sorting in surface-based rendering. Compared to the algorithm from the previous two sections, shadow-based clipping introduces the following modifications. First, the contents of the z buffer is transferred to a depth texture that can store values for a subsequent shadow test with high accuracy. Second, the fragment operations now perform a comparison between the depth of a fragment and the depth structure stored in the depth texture by means of the shadow test. In typical applications of shadow mapping, the positions of the virtual light source and the camera are different. Here, however, the virtual light for shadow testing is identical with the camera. The test result is stored in the alpha component of the fragment. Finally, the fragment may be removed by the alpha test, depending on the result of the preceding depth comparison. With a Shader Model 2 compliant GPU, a related alternative implementation is feasible. Here, the depth structure is stored in floating-point textures. The fragment program has to include a conditional removal of the fragment by a texkill instruction, depending on the comparison between the fragment’s depth value and the stored depth structure. Multi-Pass Rendering for Concave Clip Objects So far, only convex clip objects can be correctly handled. In order to allow for concave objects, the management of the depth structure has to be extended to consider a larger number of possible intersections between the clipping geometry and an eye ray. The following multi-pass rendering technique is proposed for concave clipping. First, the depth structure of the clip object is analyzed and neighboring pairs of z values in this depth structure are combined. These pairs are organized in a way which ensures that each pair represents one visible segment of the viewing ray, i.e., the volume remains visible within the interval bounded by such a pair of z values. Note that, in the case of volume cutting, this organization generates single, unpaired depth entries at the front and back parts of the depth structure. The depth-peeling algorithm [116] can be used to determine the depth structure from a triangle mesh. Each pair of z values is stored in separate textures, i.e., n pairs of textures are required if n describes the maximum number of depth pairs in the depth structure. Note that this process is performed only once per frame.

2.5 Volume Clipping

47

Second, the actual multi-pass rendering of the volume is accomplished. For each pair of z values from the depth structure, the corresponding region of the volume is rendered. Since each depth pair represents a local volume probing against two boundaries, rendering is based on the previously introduced techniques for volume probing. To put it another way, the complete volume is reconstructed in a layer-by-layer fashion by processing the depth structure pair-by-pair. The depth structure is processed in the same direction as the direction of slice compositing. For example, a back-to-front compositing implies that the depth pairs are also processed back-to-front. The case of volume cutting – with single depth values at the front and back of the depth structure – can be handled by clipping only at a single boundary. 2.5.2 Clipping Based on Volumetric Textures In contrast to the depth-based clipping algorithms from the previous section, volumetric clipping approaches model the visibility information by means of a second volume texture of which the voxels provide the visibility information for voxels of the “real” volume. Here, the clip geometry is voxelized and stored as a binary volume. This clipping texture is either a 3D texture or a stack of 2D textures. It is initialized with one for all voxels inside the clip geometry and with zero for all voxels outside. Thus, arbitrary voxelized clip objects are possible. Clipping techniques based on a volumetric description of clipping objects are related to so-called tagged or attributed volumes. Volume tagging is particularly useful in medical applications that provide segmentation data. For example, Hastreiter et al. [153] modify texture-based volume rendering with pre-classification by applying different transfer functions to different regions of the segmented data set. In their approach, volume tags are spatially fixed and cannot be modified interactively. Tiede et al. [418] use a similar approach for the visualization of attributed volumes by ray casting. Pflesser et al. [332] extend this ray casting approach to incorporate arbitrarily shaped cut surfaces. Konrad-Verse et al. [223] use a deformable cutting plane to specify and modify virtual resections in a medical application. Two-level volume rendering by Hauser et al. [154] is related to the visualization of attributed volumes because it combines different volume rendering techniques for different subsets of a 3D data set. In a publication on texture-based volume visualization with OpenGL Volumizer [19], a clipping mechanism based on a volumetric description of clipping objects is described. This implementation requires two-pass rendering and models visibility by means of a stencil test on each slice. In the volumetric clipping approach by Weiskopf, Engel, and Ertl [469, 470], the clipping texture and the volume data set are combined by multiplying their texture entries. In this way, voxels to be clipped are multiplied by zero. During actual rendering, a texture slice of the data set and a slice of the clip texture are simultaneously mapped onto the same slice polygon and combined

48


using a per-component multiplication. Fragments lying outside the clip object have an alpha value of zero and can be discarded by a subsequent alpha test. The clip object can be scaled and moved by adapting the texture coordinates of the clipping texture. Additionally, when using 3D textures, a rotation of the clip object is possible. To be more precise, any affine transformation of the 3D texture-based clip geometry can be represented by a corresponding transformation of texture coordinates. Switching between volume probing and volume cutting is easily achieved by applying a per-fragment invert mapping to the clipping texture. In this way, the original texture value is replaced by (1 − value). Since all these operations only require the change of texture coordinates or a re-configuration of per-fragment operations, all above transformations of the clip geometry can be performed very efficiently. Only a complete change of the clip geometry’s shape requires a revoxelization and a reload of the clipping texture. The above technique is based on a binary representation and therefore requires a nearest-neighbor sampling of the clipping texture. If a bilinear (for a stack of 2D textures) or a trilinear (for a 3D texture) interpolation was applied, intermediate values between zero and one would result from a texture fetch in the clipping texture. Therefore, a clearly defined surface of the clip geometry would be replaced by a gradual and rather diffuse transition between visible and clipped parts of the volume, which is inappropriate for most applications. However, a missing interpolation within the clipping texture introduces quite visible, jaggy artifacts similar to those in aliased line drawings. To overcome this problem, the following modification of the original approach can be used. The voxelized clipping texture is no longer used as a binary data set but interpreted as a distance volume: each voxel stores the (signed) Euclidean distance to the closest point on the clip object. When a 3D texture format with fixed-point numbers (from the interval [0, 1]) is used, the distance is mapped to this range. In this way, the isosurface for the isovalue 0.5 represents the surface of the clip object. During rendering, a trilinear interpolation in the 3D clipping texture is applied (a stack of 2D textures is not considered here because of the missing interpolation between slices). Based on a comparison between the isovalue 0.5 and the value from the clipping texture, either the RGBA value from the data set (for a visible fragment) or zero (for an invisible fragment) is used as final fragment color. In this way, a clearly defined surface without jaggy artifacts is introduced at the isovalue 0.5. In Fig. 2.19, a cube-shaped voxelized clip object is used for volume cutting within the data set of an engine block. Trilinear interpolation is applied in image (a), nearest-neighbor sampling in (b). Due to trilinear interpolation, the corners and edges of the cubic clip object are smoothed out unless the faces of the cube are identical with voxel boundaries in the clip volume. Therefore, the original clip geometry cannot be reproduced in (a). An example of a significant difference between both methods is marked by red arrows. For this specific example of an axis-aligned cube, nearest-neighbor sampling gives acceptable

2.5 Volume Clipping

(a)

49

(b) IMAGES: WEISKOPF, ENGEL, ERTL [469]

c 2002 IEEE

Fig. 2.19. Comparison for a voxelized cube-shaped clip object: (a) trilinear interpolation; (b) nearest-neighbor sampling (see also Color Plate 1)

results. However, a generic clip object that contains both smooth regions and sharp features can be more accurately handled by depth-based clipping. The main problem of volumetric clipping is that the resolution of the voxel representation may not be sufficient to capture all details of the clipping surface. This issue could be addressed by locally increasing the voxel resolution, for example, by means of an adaptively sampled distance field [127]. However, an efficient implementation of adaptively sampled distance fields on GPUs has not yet been done and should be addressed in future work. Another solution is the use of a procedural description of the distance volume that represents the clipping surface. For example, simple shapes such as spheres or cylinders could be compactly described by a procedural shader. Advantages of a procedural representation are a highly accurate clipping model and savings in required texture memory. The main disadvantage is the restriction to a small class of supported shapes. Therefore, the remainder of this section focuses on a voxel representation of the clipping geometry. Fortunately, due to the availability of multiple texturing units, usually no or only little performance penalty is introduced for the actual rasterization of a voxelized clip object. However, performance might be affected by additional memory access for the clipping texture. An important advantage of this method is the fact that any kind of clip geometry can be specified – without any limitation with respect to topology or geometry. The main disadvantages of the approach are additional texture memory requirements for the clipping volume, the need for a voxelization of the clip geometry, and possible artifacts due to a limited resolution of the clipping texture. Moreover, the clipping surface can only be represented as an isosur-


IMAGES: WEISKOPF, ENGEL, ERTL [469]

c 2002 IEEE

50

(a)

(b)

Fig. 2.20. Volumetric clipping in a medical MR data set, with a voxelized cylindrical clip object in (a) and a complex clip object in (b)

face in a piecewise trilinearly interpolated scalar field, which makes it difficult to reproduce clip objects with sharp edges and creases. Figure 2.20 shows volumetric clipping in a medical MR data set. Image (a) uses a voxelized cylindrical clip object. The clipped part of the volume is moved away from the remaining parts of the volume. Figure 2.20 (b) demonstrates that volumetric clipping is capable of processing complex clip geometries and topologies. Figure 2.21 compares the different volume clipping methods of this section and the previous section. The underlying test data set is of size 1283 and shows a piecewise linear change of scalar values along the vertical axis, which are mapped to yellow and blue colors. The boundary of the cubic volume has a different scalar value that is mapped to a brownish–orange color. Lighting is disabled and texture-based rendering with pre-classification based on paletted textures is used. The first three images in the top row of Fig. 2.21 are produced by volume probing, the images in the second row are produced by volume cutting. Images (a), (b), (e), and (f) show clipping based on a volumetric texture. The clipping texture has the same resolution as the data set. In (a) and (e), nearest-neighbor sampling is applied; in (b) and (f), a trilinear interpolation within the distance field is used. Artifacts are clearly visible for nearest-neighbor sampling but not for trilinear interpolation. Images (c) and (g) demonstrate clipping as presented in Sect. 2.5.1 (single-pass depth-based clipping). Depth-based clipping achieves high-quality visualization because clipping is performed with per-pixel accuracy. For image (h), only a single clip surface is implemented according to the depth-based approach that makes use of the z buffer only. This technique erroneously removes parts of the volume data set around the closest vertical edge. In an interactive application the introduced errors are quite disturbing; the spatial structure is hard to grasp

Fig. 2.21. Comparison of volume clipping techniques (see also Color Plate 2)

2.5 Volume Clipping 51

c 2002 IEEE IMAGES: WEISKOPF, ENGEL, ERTL [469]

52


because clipping properties become view-dependent in this simple approach. Finally, image (d) shows the clip geometry in the form of an opaque sphere. 2.5.3 Clipping and Pre-Integration The aforementioned clipping approaches are based on a slice-by-slice representation of the volume data set. Therefore, the visibility computation can be reduced to checking whether a fragment on a slice is visible or not. Preintegrated volume rendering is a modification of this slice-by-slice rendering (see Sect. 2.3.7) and makes use of slabs between two neighboring slices, i.e., the basic building blocks have a finite thickness. The extension of volume clipping in the context of pre-integrated volume rendering was introduced by R¨ ottger, Guthe, Weiskopf, Ertl, and Straßer [356] and the following discussion is based on their paper.3 To combine preintegration with volumetric clipping, two steps are necessary: (a) a modification of the scalar data values within a clipped slab, and (b) an adaptation of the length of a ray segment. In what follows, a volumetric description of the clip geometry is used via a signed distance field (as in Sect. 2.5.2). Once again, the clipping surface is assumed to be located at isovalue 0.5. Let df and db be the distance values of the clip volume at the entry and exit points of the ray segment (i.e., of the slab). If both distance values are below or above 0.5, the complete slab is either invisible or visible and no special treatment is necessary. Considering the case df < 0.5 and db > 0.5 (as in Fig. 2.22), only the dark gray part of the volume has to be rendered. In this case, the front scalar data value sf has to be changed to the data value sf at entry point into the clipped region. Then, a lookup into the pre-integration table is performed with the parameters (sf , sb ) rather than with (sf , sb ). In general, the parameter sf is obtained by

[0.5 − df ] µ= , sf = (1 − µ)sf + µ sb . db − df The brackets denote clamping to the range [0, 1]. Similarly, the parameter sb is replaced by sb according to

[0.5 − db ] ν =1− , sb = (1 − ν)sf + ν sb . df − db If both clip values are below 0.5, the slab is completely invisible and the scalar values do not matter. The second issue is the reduced length L of the clipped ray segment. In general, the numerical integration depends on the parameters sf , sb , and L. The volume rendering integral (2.2) can be used to obtain the contribution 3

c The Eurographics Association 2003. Reprinted, with permission, from R¨ ottger, Guthe, Weiskopf, Ertl, and Straßer [356]

2.5 Volume Clipping

53

L

s ′f

sf

sb

df

db

slab L′

Fig. 2.22. Using slabs instead of slices for pre-integrated volume clipping. The scalar data values at the entry point and the exit point of the viewing ray are denoted by s f and s b , respectively. The corresponding distance values from the clip texture are d f and d b . The thickness of the slab is denoted by L; L is the length of the visible part of the slab. The dark gray region remains after volumetric clipping

for a ray segment. The scalar field sL (t) is assumed to be linear between the entry point and exit point. The respective mathematical expressions are: sL (t) = sb + L C(sf , sb , L) =

t (sf − sb ) L L −

g˜(sL (t)) e

τ˜(sL (q)) dq

dt

t

0 −

T (sf , sb , L) = e

L

τ˜(sL (t)) dt

0

α = 1−T . C denotes the emission (as RGB color) and T the transparency of the segment. The reduced ray segment length L can be taken into account in the following way. The pre-integration table is still based on a constant ray segment length L. The visible fraction of the slab is denoted by b = L /L. Then, the transparency T of the clipped ray segment is the pre-integrated transparency T (associated with the original segment length L) raised to the bth power because L L τ˜(sL (t )) dt = b τ˜(sL (t)) dt , 0 −b

T = e

0

L 0

⎛ τ˜(sL (t)) dt

−

⎜ = ⎝e

L 0

⎞b τ˜(sL (t)) dt

⎟ b ⎠ =T .


¨ IMAGES: ROTTGER, GUTHE, WEISKOPF, ERTL, STRASSER [356] c The Eurographics Association 2003

54

(a)

(b)


Fig. 2.23. Comparison of volumetric clipping approaches: (a) slice-based, (b) slab-based. See also Color Plate 3

(a)

(b)

Fig. 2.24. Rendering of a bonsai without (a) and with (b) pre-integrated volumetric clipping (see also Color Plate 4)

A first-order approximation is sufficient if slabs are reasonably thin. Furthermore, if self-attenuation is neglected, the emission of the clipped ray segment is given by C = bC. Instead of calculating the factors for the adjustment of scalar values, emission, and opacity in the fragment program, the factors for all combinations of the clip values df and db can be pre-computed and stored in a 2D dependent texture. The content of the RGBA channels of the dependent texture corre-

2.5 Volume Clipping

55

sponds to the respective adjustment factors. Within a fragment program, the scalar values can be adjusted by a single linear interpolation. A comparison of the rendering quality between the slab-based approach and the slice-based method from previous sections is depicted in Fig. 2.23. A sphere has been cut out of the Bucky Ball data set so that the holes of the carbon rings are visible as green spots. Both images have been rendered with only 32 slices or slabs. While the slices are clearly visible on the left, the slab-oriented method reproduces the clipped volume accurately, i.e., it is accurate along the viewing direction. As another example, Fig. 2.24 shows a bonsai [358] of size 2563 without and with pre-integrated volumetric clipping. 2.5.4 Clipping and Volume Illumination Volume illumination extends volume rendering by adding illumination terms. It is based on gradients of the scalar field and works very well if clipping is not applied. The combination of volume shading and volume clipping, however, introduces issues of how illumination has to be computed in the vicinity of the clip object. Here, illumination should not only be based on properties of the scalar field, but should rather represent the orientation of the clipping surface itself. On the other hand, the scalar field should still influence the appearance of the clipping surface, for example, with respect to color and transparency. Figure 2.25 (b) demonstrates how lighting the volume and the clipping surface improves the perception of the spatial structure of the clip object. For comparison, Fig. 2.25 (a) shows the same data set without specific lighting on the clipping surface; it is difficult to recognize the spherical cutting surface because of the inappropriate lighting conditions. This section discusses ways to combine illuminated clipping surfaces with shaded volumes in an appropriate way. As shown by Weiskopf, Engel, and Ertl [470], the depth-based and volumetric clipping techniques from Sects. 2.5.1 and 2.5.2 can be extended to take into account a consistent shading of the clipping surface. The following discussion is based on the paper by Weiskopf, Engel, and Ertl [470].4 An Optical Model for Clipping in Illuminated Volumes For a consistent combination of volume clipping and volume shading, the following requirements should be met. First, the illumination of the clipping surface should allow the viewer to recognize the orientation and shape of this surface. Second, the optical properties at the clipping surface should be compatible with the optical properties of the scalar data field at the same position to avoid discontinuities in the visual appearance. Third, volume shading in regions distant from clipping surfaces should not be affected by the clipping algorithm. Fourth, the generated images should be independent of the chosen sampling rate (e.g., slice distance) – up to improvements of accuracy by 4

c 2003 IEEE. Reprinted, with permission, from Weiskopf, Engel, and Ertl [470]



c 2003 IEEE

56

(a)

(b)

Fig. 2.25. Combining volume clipping and volume shading. Depth-based volume cutting with a spherical clip object is applied to a technical engine data set. (a) No specific lighting is applied at the cutting surface; (b) reveals lighting (e.g., a highlight) on the cutting surface by combining surface-based and volumetric shading. See also Color Plate 5

evaluating the volume rendering integral on a finer level. In particular, from a mathematical point of view, the image of a clipped volume has to converge to a reasonable limit value for an infinitely small sampling distance. Let us consider a first approach to combining clipping and volume shading. The idea is to apply unmodified clipping techniques (e.g., from Sects. 2.5.1 and 2.5.2) to the shaded volume. Illumination is based on the gradient of the scalar field and on the optical properties assigned by the transfer function. Afterwards, the shaded clipping surface is layered on top of the volume, applying illumination based on the normal vector of the surface. The optical properties and thus the material properties of the clipping surface are defined by the same transfer function and scalar values as for volume shading. Furthermore, the same local illumination model is applied as for volume shading. The surface of the clip object can be pictured as a “skin” surrounding the actual volume, as illustrated in Fig. 2.26 (a). This approach meets the first three requirements for a successful combination of clipping and volume shading. First, the lighting computation at the “skin” is based on the normal vector on the clipping surface. Therefore, the orientation of the cut can be recognized. Second, the appearance of the “skin” and the neighboring volume is compatible because identical material proper-

2.5 Volume Clipping

clip object

57

clip object

infinitesimally thin boundary

boundary of finite thickness

volume data set

volume data set

(a)


c 2003 IEEE

Fig. 2.26. Combining volume shading for a volumetric data set with surface-based shading on the boundary of the clip geometry: (a) shows an infinitesimally thin “skin” around the clip object; (b) illustrates the extension of the surrounding layer to finite thickness

ties are applied. Third, volume shading away from the clipping surface is not affected at all. Unfortunately, this first approach is not invariant if the sampling rate ∆x changes in the evaluation of the discretized version (see p. 14), I(D) ≈ I0

n

ti +

i=1

n i=1

n

gi

ti ,

(2.4)

j=i+1

of the continuous volume rendering integral (see p. 14), D I(D) = I0 T (s0 ) +

g(s)T (s) ds .

(2.2)

s0

In discretized form, the source term (see p. 14) gi = g(i∆x + s0 )∆x ,

(2.6)

converges to zero for ∆x → 0, i.e., the color contribution of an infinitely thin 2D surface is zero. Similarly, the transparency (see p. 14) ti = e−τ (i∆x+s0 )∆x

(2.5)

converges to one for ∆x → 0 unless τ is infinite. (The special case of completely opaque, surface-like regions in the volume – where transparency is zero and τ and g are delta distributions – are automatically handled by the “skin” and the following approach alike.) The underlying problem is that a continuous volumetric description of a 3D volume is combined with an infinitely thin 2D hypersurface. In the original, continuous representation by the volume

58


rendering integral (2.2), the terms g and τ have no contribution if their support is of zero length. This problem can be overcome by modifying the first approach in a way that the “skin” is broadened to a finite thickness. The volume is covered by a thick layer of which the illumination is governed by the normal vectors on the clipping surface. The influence of the orientation of the surface reaches into the volume for some distance, which could be pictured as “impregnating” the shaded volume with lighting information from the surrounding clipping surface. Figures 2.26 (a) and (b) compare both the original “skin” approach and the modified “impregnation” approach. The “impregnation” reaches only into the remaining volume; it may not change the geometric clipping (i.e, the visibility of voxels) but only the illumination in the layer region. The transition between clipping surface and interior volume is introduced into the volume shading model by extending the source term to g(x, ω) = E(x) + w(x)Ssrf (x, ω) + (1 − w(x)) Svol (x, ω) , with the weight function w(x), the surface-based illumination term Ssrf (x, ω), and the volumetric scattering term Svol (x, ω). From construction, the thickness of the layer is limited and therefore the support of the weight function w(x) has to be compact (i.e., closed and bounded). Moreover, the function values w(x) have to be from [0, 1] to ensure an affine and convex combination of the two illumination terms. When the model is restricted to a uniform layer thickness, one important parameter – the “impregnation” thickness – is introduced. It is assumed that the thickness parameter is specified by the user. Another choice concerns the functional behavior of the weight itself. A linear behavior with respect to the distance from the clipping surface (i.e., a linear ramp), a step function, or a more sophisticated function could be used. A continuous or even smooth weight function has the advantage that illumination shows a continuous or smooth transition between the “impregnation” layer and the remaining parts of the volume. One further issue is the evaluation of the surface-related illumination part, Ssrf (x, ω), at positions that are distant from the clipping surface. In particular, what is the normal vector in those regions? The definition of a surface-based normal vector in the “impregnation” layer makes use of the concept of parallel transport. For each point in the layer, the closest point on the clipping surface is determined. The normal vector is evaluated at that point on the surface and then transported to the respective location within the layer. The material parameters can be directly computed from the optical properties of the scalar data set at any point in the layer. The “impregnation” approach meets all four requirements for an appropriate combination of clipping and volume shading. By introducing a finite boundary region, the contribution of the illuminated clipping surface to the volume rendering integral (2.2) is finite. The discretized version of the integral (2.4) converges to this contribution for successive refinements of the sampling rate. Furthermore, volume shading in regions distant from clipping

2.5 Volume Clipping

59

surfaces is not affected by clipping because of the compact support of the weight function. The first two requirements are met due to the same arguments as for the “skin” approach. Combining Clipping and Volume Shading in an Object-Space Approach The above mathematical discussion of the “impregnation” approach leads to the following algorithm for combining clipping and volume shading. All considerations are based on object space. It is assumed that the clipping object is defined as the isosurface for the isovalue zero on a signed distance volume (analogously to volumetric clipping in Sect. 2.5.2). Furthermore, the distance field should be based on the Euclidean norm in 3D Euclidean space and the clipping surface should be non-selfintersecting and smooth. With these assumptions, a direct mapping of the above mathematical formulation can be accomplished. The generic weight function w(x) can be expressed in terms of distance volume vdistance (x) and an alternative weight function w(d) ˜ according to w(x) = w ˜ (vdistance (x)) .

(2.12)

The weight function w(d) ˜ has support [0, dmax ], where dmax describes the thickness of the “impregnation” layer. Function values w(d) ˜ are from [0, 1]. By the above construction, w(x) models a uniform thickness of the layer. Surface-based illumination also makes use of the distance volume. Not only is the signed distance stored but the gradient of the distance field as well. The gradient is identical to the normal vector of a respective isosurface, i.e., the gradient represents the normal vector on the clipping surface. Furthermore, even in a small neighborhood (ε neighborhood) around the smooth clipping surface, the gradient reflects the normal vector that could be obtained by parallel transport from the isosurface. This specific property of a distance volume is caused by the fact that the gradient represents the direction to or from the closest point on the isosurface. This behavior is illustrated in Fig. 2.27. The maximum size of the neighborhood depends on the curvature of the clipping surface; the neighborhood shrinks with increasing curvature. If the “impregnation” layer is contained in these local neighborhoods, the surface-shading term Ssrf (x, ω) can be computed inside the layer by using the gradient of the distance field and the optical properties of the scalar field in that region. Therefore, a quite generic formulation is available to combine clipping and volume shading. This object-space approach can be incorporated in a wide class of volume rendering algorithms, such as ray casting or splatting. This section, however, is focused on texture-based rendering. Thus, it is demonstrated how the previously described depth-based and volumetric clipping techniques can be extended to allow for consistent volume and surface shading.

60

2 Visualization of 3D Scalar Fields respective gradient point inside layer

clipping surface

point on isosurface with respective gradient IMAGE: WEISKOPF, ENGEL, ERTL [470]

c 2003 IEEE

Fig. 2.27. Gradients in distance field. The normal vector on an isosurface – on the clipping surface – is identical with the gradient of the distance field. The gradient can also be used to compute the parallel-transported normal vector inside the layer

Volumetric Clipping and Volume Shading Volumetric clipping from Sect. 2.5.2 already models the clipping object by means of an isosurface through a distance field. Accordingly, the above formulation can be directly mapped to volumetric clipping. The following minor changes need to be included in the original clipping algorithm. First, the texture representations of the volume data set and the distance field are extended to include both scalar values and gradients. For example, an RGBA texture format can be employed to store the scalar value in alpha and the gradient in RGB. The gradient fields serve as a basis to evaluate the local illumination model. Second, a fragment program is used to evaluate local illumination for the original data set and the distance field. The material properties for both volumes are based on the entries in the data set and the transfer function. Since lighting is not precomputed, the user may change lighting parameters, such as material properties or color and position of the light source, without redefining the RGBA textures for the volume data set and the distance field. Third, the weight function w(d) ˜ is evaluated per fragment to implement (2.12). The distance d is obtained from the distance field. The thickness of the “impregnation” layer can be adapted by scaling d. A linear ramp behavior for w(d) ˜ can be realized by a linear interpolation with subsequent clamping at 0 and 1; a step function is implemented by a comparison operation. More sophisticated weighting functions need respective numerical operations or a dependent texture lookup in a lookup-table. The final function value w(d) ˜ serves as weight for a linear interpolation between the lighting computation for the original data set and the distance field. All other parts of the original clipping algorithm are not affected. In particular, actual clipping is still based on a comparison between the entry in the distance field and the isovalue of the clip geometry. Moreover, the number of

2.5 Volume Clipping

61

texture fetches is not increased – two textures are still sufficient for the data set and the distance field. Figure 2.29 demonstrates the consistent combination of clipping and volume shading for an orbital data set. Figure 2.29 (a) shows the original data set, and Fig. 2.29 (d) is based on volumetric clipping. (Depth-based clipping, as shown in Figs. 2.29 (b) and (c), is explained in the following subsection.) Depth-Based Clipping and Volume Shading The ideas from the above object-space approach need some modifications before they can be applied to depth-based clipping. The main problem is that a measure for the distance from the clipping surface is not available. An approximation of the basic optical clipping model that does not require any distance information is proposed. An “impregnation” layer of finite thickness has been introduced to take into account that transparency ti and source term gi in (2.5) and (2.6) depend on slice distance. In particular, an infinitely thin layer will have no effect on the final image when the sampling rate is very high. This observation leads to the following modified approach. The clipping boundary is not modeled as a thick layer but as a 2D hypersurface. Shading inside the volume is based on transparency values and source terms that are adapted to the slicing distance ∆x. Surface-based shading on the clip geometry, however, yields modified contributions to transparency tsrf = e−τ (ssrf )∆xsrf , and source term gsrf = g(ssrf )∆srf , where ssrf describes the location of the clipping surface and ∆srf represents the thickness of the original “impregnation” layer, which is not dependent on the sampling rate. Once again, the optical properties are defined by the volume data set and the transfer function. To put it another way, geometrically the clip boundary is treated as 2D surface, whereas the contribution to the rendering integral comes from a thick virtual layer. This approach results in the following approximations of the original distance-based model. First, the terms g(ssrf ) and τ (ssrf ) are evaluated only at the clipping surface and thus surface-based transparency and source term are constant inside the “impregnation” layer. Second, the weight function w(d) ˜ is implicitly set to a step function. Third, the volume rendering integral has double contributions from the “impregnation” layer – both from the above terms tsrf and gsrf , and erroneously from the slice-based sampling of the volume data set. Fourth, the layer is rendered as a surface of infinitesimal thickness. The effects of these approximations depend on the layer thickness, rate of change of scalar values around the clipping surface, and opacity. In typical applications, the approximations are not noticeable to the user because

62


the layer thickness tends to be small and, more importantly, the opacity is quite high in regions that have strong illumination. A clipping surface of high opacity essentially hides parts of the volume that are further away. On the other hand, the following benefits are introduced by surface-based shading. First, surface shading offers higher visual quality than volume shading because of a better representation of normal vectors. Second, the normal vector is always well-defined because no parallel transport is employed. The normal vector field on the clipping surface does not even need to be continuous and thus clipping geometries with sharp edges and creases are possible. For the actual implementation, the following modifications are introduced into the original depth-based clipping techniques from Sect. 2.5.1. Essentially, pure slice-based volume rendering is extended to hybrid volume and surface rendering. The volume and the surface parts are interleaved according to the depth structure of the clip geometry, and so is the order of surface and volume rendering. Accordingly, multi-pass rendering approaches have to be used; single-pass rendering for convex volume cutting is not suitable because the surface part is contained within the volume part and cannot be separated. All other techniques from Sect. 2.5.1, however, are prepared to alternatingly render regions of the clipped volume and the surrounding clipping surfaces. Volume rendering and volume clipping are adopted from Sect. 2.5.1, with volume shading being added. Surface rendering is based on the tessellated representation of the clip object. Material properties are obtained from the 3D scalar data set and the transfer function, i.e., texture coordinates are assigned to the vertices of the surface in order to access the 3D texture of the data set. The normal vector is computed from the surface model. Fragment operations compute the modified source and transparency terms to obtain the contribution of the clipping surface to the image. Finally, this contribution is combined with the volume parts by blending into the frame buffer. Figures 2.28 and 2.29 demonstrate the consistent combination of clipping and volume shading. Depth-based clipping via shadow testing is applied to a medical CT data set in Fig. 2.28. A specular highlight on the clipping surface allows the viewer to recognize surface orientation. Here, the transfer function is chosen to render the head almost opaque, i.e., the image resembles surface-based graphics. In contrast, the visualization of the orbital data set in Fig. 2.29 reveals both transparent and opaque structures. Figure 2.29 (a) shows the original data set. Figures 2.29 (b) and (c) are based on depthbased clipping by means of shadow testing, whereas Fig. 2.29 (d) is based on volumetric clipping. This comparison demonstrates that the quality of the specular highlight is very high for surface-based shading in images (b) and (c). In contrast, the highlight in image (d) is based on the gradients of the clip volume and shows artifacts caused by an inaccurate reconstruction of the gradients and by quantization. In Fig. 2.29 (b), an extended version of surface shading is applied to specifically mark the clipping surface for a fast recognition of its shape, position, and orientation. An additional ambient lighting term makes the clipping object visible as a dark-gray surface.

63


c 2003 IEEE

2.5 Volume Clipping

Fig. 2.28. Depth-based clipping in an illuminated CT data set (see also Color Plate 6)

2.5.5 Implementation An example implementation of slice-based volume clipping (Sects. 2.5.1 and 2.5.2) is based on C++, OpenGL, and GLUT. The implementation makes use of NVIDIA’s OpenGL extensions [210] for GeForce 3/4. The performance measurements in Table 2.1 were conducted on a Windows XP PC with Athlon XP 1800+ CPU and NVIDIA GeForce 4 Ti 4600 GPU with 128 MB texture memory. The size of the data set was 2563 and the viewport sizes were 5122 or 10242 . A slicing approach for a 3D texture was chosen, pre-classification by paletted textures was applied, and volume shading was disabled. The performance numbers show that clipping based on a voxelized geometry causes only modest additional rendering costs. The performance of depthbased clipping depends on the chosen implementation. On GeForce 3/4, the shadow-testing approach is faster than the approach based on view frustum clipping. The main reason is the larger number of texture stages required by depth clipping. Because of the rather restricted functionality of the GeForce 3/4, a shift of depth values needs three texture stages. On the other hand, a lookup in a depth texture combined with a shadow test can be handled by a


(a)

(b)

(c)

(d)


c 2003 IEEE

64

Fig. 2.29. Clipping in an illuminated orbital data set: (a) without clipping; (b) and (c) depth-based clipping; (d) volumetric clipping. The clipping surface is additionally marked in dark gray in (b). See also Color Plate 7

2.6 Hierarchical Volume Visualization on GPU Clusters

65

Table 2.1. Performance measurements for volume clipping techniques in frames per second (NVIDIA GeForce 4 Ti 4600, 128 MB) [470] Viewport size

5122

10242

No clipping Voxelized clip object Depth-based probing (depth clipping) Depth-based cutting (depth clipping) Depth-based probing (shadow test) Depth-based cutting (shadow test)

28.0 15.3 8.5 7.1 18.5 12.2

8.8 5.2 2.5 2.2 5.6 3.4

single texture stage. Another reason is that data can be directly copied from the z buffer into a depth texture, whereas the high-resolution texture for the depth-clipping approach can only be filled by data from main memory. Moreover, the early z test can improve the performance of shadow-based probing. Depth-based cutting by means of shadow testing is slower than shadow-based probing because a second depth texture has to be used. For volume shading, the above relative performance behavior with respect to different clipping approaches remains roughly the same; however, the overall performance is reduced due to additional texture and fragment operations for post-classification and illumination. An example implementation of pre-integrated clipping (Sect. 2.5.3) is based on C++ and DirectX 9.0. A Shader Model 2 compliant GPU is required. The prototype was implemented and tested on an ATI Radeon 9700 GPU. The main functionality is included in a fragment program that provides data access and interpolation, pre-integrated classification, and modifications for clipping. The complete source code (assembler-like pixel shader code) can be found in [356].

2.6 Hierarchical Volume Visualization on GPU Clusters While the previous sections focus on improvements for single-GPU volume rendering, this section addresses the issue of large-data visualization on GPU clusters. Large-data visualization is becoming more and more important because, for example, the increasing resolution of medical CT scanners leads to increasing sizes of scalar data sets, which can be in the range of gigabytes. Even more challenging is the visualization of time-dependent CFD simulation data that can comprise several gigabytes for a single time step and several hundred or thousand time steps. Parallel visualization can be used to address the issues of large data processing in two ways: both the available memory and the visualization performance are scaled by the number of nodes in a cluster computer.

66


The idea is to combine the “traditional” benefits of parallel computing with the high performance offered by GPU-based techniques. The following approach is based on hierarchical wavelet compression, which is adapted to the distributed-memory architecture of a cluster computer to increase the effective size of volumes that can be handled. This visualization method was proposed by Strengert, Magall´ on, Weiskopf, Guthe, and Ertl [402] and the following discussion is based on their paper.5 Here, the wavelet hierarchy is also used for an adaptive, texture-based volume rendering approach for a PC cluster. An advanced compositing scheme takes into account the footprints of volume bricks to minimize costs for reading from frame buffer, network communication, and blending. Finally, performance numbers for different combinations of parameters are included to clarify performance and scaling characteristics. 2.6.1 Previous and Related Approaches to Large-Data Volume Visualization Previous and related techniques to large-data volume rendering are briefly reviewed in this section. The first class of techniques considers a single GPU only. Bricking (see also Sect. 2.4) is a simple method to handle large data sets that do not fit into texture memory at once [219, 237, 451]. Bricking, however, overcomes the limitations of texture memory only at cost of lower speed due to the permanent transfer of data over the AGP or PCI Express bus. Adaptive techniques aim at actually reducing the texture memory needed. They subdivide the volume to represent volumes of interest at high resolution and other parts of the volume at lower resolution. For example, an octree structure can be traversed in a top-to-down manner to find the first nodes that deliver a sufficiently high quality [237]. Special care has to be taken at the boundary of sub-blocks to avoid artifacts at the transition zone between different resolutions [462]. In a related approach, a variable-resolution texture model with an octree structure is utilized to render large volume data [24]. Packing techniques use sub-blocks of different sizes and pack non-empty blocks to save memory for empty sub-regions. In the context of GPU-based volume rendering, Kraus and Ertl [227] implemented this idea by means of fragment programs with texture indirections (dependent textures). A related method by Li and Kaufman [248] uses a vertex program to realize volume packing. Vector quantization is another well-known compression technique that can be applied in GPU-based volume visualization. Here, the volume is partitioned into sub-blocks of same size. Each sub-block holds one index pointing into a code book that represents the actual data. Therefore, the data set can be decompressed during volume rendering by applying a dependent texture lookup in the code book [379]. Another volume compression technique is used by Rosa et al. [350], who developed a system for the visualization of time-varying volume data from 5

c The Eurographics Association 2004. Reprinted, with permission, from Strengert, Magall´ on, Weiskopf, Guthe, and Ertl [402]


67

thermal flow simulations. The system is based on the work of Lum et al. [263], who apply quantization and lossy compression to texture data by means of a discrete cosine transformation and then store the result as indexed texture. A texture represented in this way can be decoded in graphics hardware by changing the texture palette. Binotto et al. [21] have recently presented a system that uses a hierarchical representation, but is oriented toward the compression of time-dependent, highly sparse and temporally coherent data sets. Their algorithm uses fragment programs to perform the decompression of the data sets. Finally, wavelet compression can be used to compress large data sets and build a hierarchy of different resolutions. Guthe and co-workers [147, 148] show how a wavelet hierarchy can be constructed in a preprocessing step and then used during adaptive volume rendering on a single GPU. Their work is the basis for the volume compression scheme of this chapter – a more detailed description of the wavelet algorithm can be found in Sect. 2.6.4. Another line of research focuses on parallelization to display a large amount of data. In general, intermediate results of distributed rendering have to be composited into a single final image. Parallel rendering approaches can be classified as sort-first, sort-middle, or sort-last. In the first category, graphical primitives (e.g., triangles) are sorted and then distributed to the different render nodes. A sort-middle method distributes data after geometric transformations and lighting, but before rasterization. Sort-last methods apply sorting after rasterization. WireGL [173] and Chromium [174] are generic systems that enable distributed rendering for existing OpenGL applications. To speed up the per-pixel compositing in sort-last approaches, special-purpose hardware can be used, such as Sepia [292], Lightning-2 [400], or Metabuffer [23]. Specifically targeting volume rendering, Garcia et al. [130] use a combination of object-space partitioning and image-space partitioning while Magall´ on et al. [266] employ object-space partitioning with a sort-last approach. Another implementation of volume rendering builds upon the Sepia-2 architecture [257]. In the related field of indirect volume visualization, Zhang et al. [507] describe parallel isosurface extraction and rendering with the Metabuffer system. 2.6.2 Distributed Visualization In this section, a sort-last strategy [293] is used to distribute the visualization process in a cluster environment. With increasing size of the input data set, this sorting scheme is favorable, since the input data becomes larger than the compositing data and a static partitioning in object space avoids communication regarding the scalar field during runtime. The basic structure of the implementation is based on the aforementioned parallel volume rendering system by Magall´ on et al. [266], which originally did not support adaptive and hierarchical representations. During a preprocessing step object-based partitioning is performed to split the input data set into multiple, identically sized sub-volumes, depending on

68


the number of nodes in the cluster configuration. To overcome possible memory limitations in connection with large data sets, this step is executed using the same set of nodes as the subsequent render process. The render loop is entered when all sub-volumes are created and transferred to their corresponding nodes. Rendering can be split into two consecutive tasks. The first task is to render each brick separately on its corresponding node. An intermediate image is generated by texture-based direct volume visualization with screenaligned slices (see Sect. 2.3.1). By adapting the model-view matrix for each node, it is ensured that each sub-volume is rendered at its correct position in image space. Since the partitioning is performed in object space, the rendering process of different nodes can produce intermediate images that overlap in image space. The second task blends these images and takes into account that multiple nodes can contribute to a single pixel in the final image. The distributed images are depth sorted and processed through a compositing step based on alpha blending. To this end, each node reads back its frame buffer, including the alpha channel, and sends it to other nodes. To take advantage of all nodes for the computationally expensive alpha blending, direct send is used as compositing scheme [305]. Each intermediate result is horizontally cut into a number of stripes that match the total number of nodes. All these regions are sorted and transferred between the nodes in a way that each node receives all stripes of a specific area in the image space. Then, each node computes an identically sized part of the final image. The alpha blending of the intermediate images is completely performed on the CPU. Although the GPU is highly specialized for this task, the additional costs for loading all stripes into texture memory and reading back the information after blending would lead to a lower overall performance. Instead, an optimized MMX [326] code is used to determine the result of the blend function for all four color channels of a pixel simultaneously (details are given in [402]). This approach can also handle time-dependent scalar fields without major changes. During the bricking process a static partitioning scheme is used for all time steps, i.e., each sub-volume contains the complete temporal sequence for the corresponding part of the input volume. To synchronize all nodes the information regarding the current time step is broadcast to the render nodes. 2.6.3 Accelerated Compositing Scheme The overall performance of the distributed visualization system may be limited by three factors (besides actual volume rendering): the process of reading back the results from the frame buffer, the data transfer between nodes, and the compositing step. In the following, these issues are addressed by minimizing the amount of image data to be processed. The key observation is that the image footprint of a sub-volume usually covers only a fraction of the intermediate image. For the scaling behavior, it is important to note that the relative size of the footprint shrinks with an increasing number of nodes. For


69

simplicity, an upper bound for the footprint is determined by computing the axis-aligned bounding box of the projected sub-volume in image space. Since the time needed to read back a rectangular region from the frame buffer depends almost linearly on the amount of data, reducing the size of the region leads to a performance increase of this part of the rendering process. Similarly, the communication speed also benefits from the reduction of image data. The compositing step is accelerated by avoiding unnecessary alpha blending operations for image regions outside the footprints. Similarly to SLIC (Scheduled Linear Image Compositing) [401], a line-based compositing scheme is employed. For each line, the span containing already blended data is tracked. Since the images are blended in the depth-sorted order of their corresponding volume blocks and all blocks together represent the convex shape of the non-partitioned volume, the tracked region always forms one segment instead of multiple separated spans. If a projected volume face is parallel to the image plane, depth sorting results in an ambiguous ordering that may break this property. In this case, the topology is considered to ensure the connectivity of the marked span. With this information the new image data of the next compositing step can be separated into a maximum number of three segments. Two segments contain pixels that map into the region outside the marked span. These pixels need no further processing and can be copied into the resulting image. The remaining segment maps into an area where already other color information resides and alpha blending has to be performed. An example of this procedure is given in Fig. 2.30. After one iteration, the size of the span containing data needs to be updated and the next image stripe can be processed. In doing so, only a minimal amount of blending operations for a given volume partitioning is carried out. 2.6.4 Hierarchical Compression and Adaptive Rendering Even with distributed rendering techniques the size of a data set can exceed the combined system memory of a cluster configuration and even a brick of the data set may be larger than one single node can handle. Another challenge is to further improve the rendering speed. The memory issue is addressed by using a hierarchical compression technique, and the performance issue by adaptive rendering. Single-GPU Wavelet Compression A single-GPU visualization approach is adopted that utilizes compression for large data sets [148]. The idea is to transform the input data set into a compressed hierarchical representation in a preprocessing step. An octree structure is created with the help of wavelet transformations. The input data set is split into cubes of size 153 , which serve as starting point for recursive preprocessing. Eight cubes that share one corner are transformed at a time using linearly interpolating spline wavelets. The resulting low-pass filtered portion

70


´ WEISKOPF, GUTHE, ERTL [402] IMAGE: STRENGERT, MAGALLON,

c The Eurographics Association 2004

Fig. 2.30. Depth-sorted blending of footprints of four volume blocks. The regions with and without the need for blending are marked for each compositing step

is a combined representation of the eight input cubes with half the resolution of the original data. The size of this portion is 153 voxels again. The wavelet coefficients representing high frequencies replace the original data of the eight input blocks. After all cubes of the original data set are transformed, the next iteration starts with the newly created low-pass filtered cubes as input. The recursion stops as soon as the whole volume is represented through one single cube. This cube forms the root node of the hierarchical data structure and is the representation with the lowest quality. Except for the root node, all other nodes hold only high-pass filtered data, which is compressed through an arithmetic encoder [147]. Although it is possible to increase the compression ratio by thresholding, a lossless compression is used for best visualization results in this chapter. During rendering an adaptive decompression scheme is used that depends on the viewing position and the data set itself. Starting at the root node of the hierarchical data structure, a priority queue determines which parts of the volume are decompressed next. Depending on the ratio between the resolution of a volume block and the actual display resolution, regions closer to the viewer are more likely decompressed than others. Additionally, an error criterion describing the difference between two representations of varying quality is used to identify regions that can be rendered in low quality without



71


Fig. 2.31. Texture interpolation at a sub-volume border for a 1D case. Texels on one side of the border (white cells) are filled with previously transferred information of the neighboring node. Errors arise if the quality level of the neighboring node is unknown and hence a wrong level is chosen. For the incorrect case, border information of level 0 is used for interpolation although the rendering of the neighboring node is performed on level 1

noticeable artifacts. After the quality classification is completed, the required blocks are decompressed by the CPU and then transferred to the graphics board’s texture memory for rendering. Depending on the reconstructed quality level of a block, the number of slices used for rendering is determined. With increasing reconstruction quality the number of slices increases as well, delivering higher quality for areas closer to the viewer. Additionally, a cache strategy is used to avoid the expensive decompression step for recently processed blocks. Unnecessary texture transfers are avoided by tracking already loaded textures. Extension to Parallel Rendering In a distributed visualization system, this approach leads to a problem concerning correct texture interpolation between sub-volumes rendered on different nodes. A typical solution is to create sub-volumes with an overlap of one voxel. With multi-resolution rendering techniques it is necessary to know not only the border voxels of the original data set but also the data value at the border of all other used quality levels [462]. This information can be determined in the preprocessing step. After creating the sub-volumes and constructing the hierarchical data structure, each node transfers the border information of all quality levels to its appropriate neighbors. But even with this information available on each node, a correct texture interpolation cannot be generated easily. As illustrated in Fig. 2.31, the remaining problem is to determine the quality level used for rendering a neighboring node in order to choose the correct border information. Since communication between nodes suffers from network latency, requesting information from a neighboring node is not suitable. Another approach is to compute the quality classification on each node for an expanded area. Unfortunately, this is also impractical because the quality classification is dependent on the volume data.

72




Fig. 2.32. In the left part of the image, the volume is rendered using different quality levels for each of the two sub-volume blocks. Assuming identical classification for interpolation leads to visible artifacts as seen in the left magnified area. For comparison, the right image is rendered with identical classification for the subvolume blocks

Instead, an approximate solution is proposed which assumes that there are no changes in quality classification at the border of the sub-volumes. With this approach errors only occur if different qualities are used on each side of a subvolume border (see the example visualization in Fig. 2.32). Due to the similar position of adjacent parts of the sub-volumes it is likely that both regions are classified with the same quality. Experimental data that shows the error rates for the approximate solution is given in Table 2.2 for both the nonweighted number of transitions (i.e., just the number of transitions) and the area-weighted ratio (i.e., transitions multiplied by the attached area). The measurement was performed while rendering the Visible Human male data set on 16 rendering nodes (Fig. 2.34). In this configuration, a total number of 185, 212 cube transitions is present in the whole data set. Only 723 cube borders show transitions that lead to an interpolation error, i.e., less than one percent of the total amount of transitions is affected. For a correct solution of the interpolation problem, another approach is proposed that separates the rendering process and the computation of quality classification. In each frame, an adaptive classification is determined, but the associated rendering is delayed by one frame. Therefore, the information about the currently used quality levels can be transferred to neighboring nodes at the same time when intermediate images are distributed for compositing. Since at this time communication between all nodes must be performed anyway, the additional data can be appended to the image data. Having the transferred data available, the rendering process can produce a properly interpolated visualization during the next frame. The downside is that the latency between user interaction and the system’s reaction is increased by one frame. To avoid this, a hybrid technique that exploits both described approaches is possible. While the viewing parameters are changed, the approximate solution is used to generate an image without increased latency times during user interac-


73

Table 2.2. Portion of changes in quality classification at block faces [402] non-weighted area-weighted total volume same quality different quality

89.8% 10.2%

81.7% 18.3%

sub-volume borders only same quality 91.2% different quality 8.8%

83.0% 17.0%

borders compared to total volume same quality 99.6% 99.1% different quality 0.4% 0.9%

tion. As soon as the camera parameters are kept constant, a correct image is rendered based on the quality classification that is transferred from the previous rendering step. Thus, a fast user interaction is combined with a correct sub-volume interpolation for the static case. 2.6.5 Implementation and Results The implementation is based on C++ and OpenGL. Volume rendering adopts post-classification, which is realized either through NVIDIA’s register combiners or alternatively through an ARB fragment program. MPI is used for all communication between nodes. Two different cluster environments were used for the following evaluation. The first one is a 16-node PC cluster. Each of the nodes runs a dual-CPU configuration with two AMD 1.6 GHz Athlon CPUs, 2 GB of system memory, and NVIDIA GeForce 4 Ti 4600 (128 MB) graphics boards. The interconnecting network is a Myrinet 1.28 GBit/s switched LAN providing low latency times. Linux is used as operating system, the SCore MPI implementation drives the communication [323]. The second environment consists of standard PCs with a Gigabit Ethernet interconnection. Each node has an Intel Pentium 4 2.8 GHz CPU and 4 GB system memory. The installed graphics boards are a mixture of NVIDIA GeForce 4 Ti 4200 and GeForce 4 Ti 4600, both with 128 MB memory. Running Linux, the MPI implementation LAM/MPI is used for node management and communication [238]. Three different large-scale data sets are used to evaluate the performance of the implemented visualization system. If not stated otherwise, all measurements were performed on the cluster interconnected through Myrinet. The first data set is an artificial scalar field of size 10243 , showing a radial distance volume that is distorted by Perlin noise [328] (Fig. 2.33). The second data set is derived from the anatomical RGB cryosections of the Visible Human male data set [415]. The slices are reduced to 8 bit per voxel and cropped to exclude


Fig. 2.33. Radial distance volume combined with Perlin noise. A high-frequency transfer function is applied. See also Color Plate 8

´ IMAGES: STRENGERT, MAGALLON, WEISKOPF, GUTHE, ERTL [402] c The Eurographics Association 2004

74

Fig. 2.34. Parallel volume rendering of anatomical cryosections through the Visible Human male data set. The whole body is rendered with a total of 16 nodes. This image shows the result of four render nodes that display the upper part of the body. See also Color Plate 9


´ WEISKOPF, GUTHE, ERTL [402] IMAGES: STRENGERT, MAGALLON,

75


Fig. 2.35. Visualization of the time-dependent CFD simulation. From top to bottom, timesteps 0, 45, and 89 are shown. See also Color Plate 10

external data like gray-scale cards and fidual markers. The obtained data set has a resolution of 2048 × 1024 × 1878 voxels (Fig. 2.34). The third data set is a time-dependent CFD simulation of a flow field with increasing turbulence. The sequence contains a total of 89 time bins, each sized 2563 (Fig. 2.35). The Visible Human male data set can be visualized on a 10242 viewport using 16 nodes with 5 frames per second. The quality classification is set to use the original resolution for most regions. Due to the uniform characteristics of the surroundings, these areas are displayed in a lower resolution without any noticeable artifacts. With a viewport of half size in each dimension, the obtained frame rate increases to 8 frames per second. Configurations with between two and 16 render nodes were measured to investigate the scaling behavior of the visualization system. The gigacube that contains the perturbed radial distance field (Fig. 2.33) is used for all these tests. Results are shown in Fig. 2.36. For a 16-node configuration, the data set can be rendered in 174 ms, which corresponds to a refresh rate of 5.7 Hz.

76

2 Visualization of 3D Scalar Fields 350

time in ms

300

250

200

150 2

4

6

8

10

12

14

16

Number of nodes

Fig. 2.36. Scalability of the visualization system with the number of rendering nodes 450

full resolution 1/2 resolution 1/4 resolution

400

time in ms

350 300 250 200 150 100 50 0

10

20

30

40 50 timestep

60

70

80

´ WEISKOPF, GUTHE, ERTL [402] BOTH IMAGES: STRENGERT, MAGALLON,

Fig. 2.37. Fig. 2.35

90


Rendering performance for the time-dependent data set shown in

Figure 2.37 shows the results for the time-dependent data set. The test was performed using three different quality levels. In case of the original (highest) quality, the required time clearly increases toward the end of the sequence. The reason for this behavior is found in the characteristics of the data set, which becomes more and more turbulent over time, leading to a higher amount of blocks that have to be decompressed. Furthermore, the cache becomes invalid with a progress in time and all blocks have to be decompressed starting at the root node. Therefore, rendering is rather slow for time-dependent data sets compared to the static ones. Using the second cluster environment with

2.7 Summary

77

its 8 nodes, only 2 frames per second are achieved for rendering the distance volume. Due to the similar configuration of each node this gap is solely caused by the Gigabit Ethernet in comparison to Myrinet. The limiting factor for this type of application is the latency rather than the bandwidth limitation of the used network. The Myrinet configuration clearly outperforms a conventional Gigabit Ethernet regarding latency times although both network technologies provide comparable bandwidth.

2.7 Summary This chapter has focused on efficient GPU-based volume rendering. It has been demonstrated that all parts of the volume rendering pipeline can benefit from the usage of GPUs. Traditionally, graphics hardware is used for efficient interpolation (in 2D and 3D textures), classification (by dependent textures for transfer functions), shading (by fragment operations), and compositing (by alpha blending). Real-time volume rendering is very important because it is a prerequisite for interactive volume visualization. Interaction is crucial for a user-centric exploration process in which the large parameter space for rendering and, possibly, the generation of the underlying data set (e.g., simulation or measurement parameters) has to be investigated. In particular, this chapter has provided details on two specific aspects of direct volume visualization: first, volume clipping and, second, large-data visualization by applying compression techniques, adaptive rendering, and parallel rendering on GPU clusters. In this work, volume clipping is considered as an example of the filtering step of the visualization pipeline because it provides a reduction of the input data by restricting the domain of the scalar field. Clipping facilitates the selection of regions of the data set on a purely geometric basis and, therefore, is an important interaction element for a user-controlled examination. A free choice of the shape of the clip geometry plays a substantial role – in combination with the support for real-time rendering. Two alternative GPU approaches for efficient volume clipping have been presented in Sect. 2.5: an image-space approach in which a depth structure is represented in 2D textures (Sect. 2.5.1) and an object-space approach that holds a voxelized clip geometry in a 3D texture (Sect. 2.5.2). When volume clipping and illumination have to be combined seamlessly, a modification of the assignment of color and opacity values has to be introduced. Section 2.5.4 has shown how the corresponding extended optical model for volume rendering can be incorporated into both the image-space method and the object-space method. Moreover, it has been demonstrated that the advanced clipping techniques of this chapter can be used in combination with texture slicing and pre-integrated volume rendering alike (Sect. 2.5.3). Experiments with the two basic clipping approaches of this chapter indicate that each method has specific advantages and disadvantages. The

78


best-suited technique has to be chosen by the type of application and the requirements stated by the user. The main advantage of the volumetric clipping approach is that it provides an explicit control over each voxel of the clip volume. Therefore, arbitrarily complex objects can be used as clip geometry. The main disadvantage of the volumetric clipping approach is a possibly inaccurate reproduction of clip geometry. Nearest-neighbor sampling of the clipping texture results in visible, jaggy artifacts and a poor image quality. Trilinear interpolation, on the other hand, does not allow for arbitrary sharp geometric features of the clip object. For volume shading, the image quality can suffer further because of the sometimes inaccurate reconstruction of normal vectors from an interpolated gradient field. Moreover, the voxelization approach prohibits changing the shape of a clip object in real time. The additional texture memory requirements for the clipping volume are another problem. An advantage of depth-based volume clipping is the high quality of the resulting images since clipping is performed with per-pixel accuracy. High image quality is even more noticeable for clipping in shaded volumes because the illumination of the clipping surface is derived from surface-based information and not from a volumetric gradient field. Aliasing problems related to texture mapping cannot occur because there is a one-to-one mapping between pixels on the image plane and texels representing the clip geometry. Arbitrary changes of the clip geometry cause no additional costs because no voxelization is required. Depth-based volume clipping methods benefit from the fact that the clip geometry has to be processed only once per frame. This is an important difference to the method by Westermann and Ertl [485], which needs to update the values in the stencil buffer for each slice by rendering the whole clip geometry. Their approach does not scale well for complex clip geometries or large volume data sets with many slices. The main problem of depth-based volume clipping is the costly handling of concave and complex clip geometries by means of multi-pass rendering. Therefore, depth-based clipping provides best performance for geometries with little depth complexity. Large-data visualization has been addressed by using a distributed rendering system for texture-based volume rendering (Sect. 2.6). By adapting a hierarchical wavelet compression technique to a cluster environment, the effective size of volume data that can be handled is further improved. The adaptive decompression and rendering scheme results in a reduction of rendering costs, depending on the viewing position and the characteristics of the data set. The problem of texture interpolation at brick borders in connection with multi-resolution rendering has been addressed and different solutions have been provided. Parts of the rendering process that are crucial to the system’s performance benefit from the applied reduction of the processed region in image space, especially with increasing numbers of rendering nodes. The achieved performance is often restricted by the capabilities of the interconnection between rendering nodes and the computation of blending operations during the compositing step. With a viewport of size 10242 , this upper bound is approximately 11 frames per second for the tested cluster

2.7 Summary

79

configuration with Myrinet network. To increase this upper limit, an exact calculation of the footprints instead of using a bounding box could be helpful. In the same way, the remaining unnecessary blending operations could be avoided and communication costs could be further reduced. In the case of time-dependent data sets, the performance is additionally bound by the decompression step because caching of decompressed blocks cannot be used in this context. Finally, a multi-bricking approach has been proposed to achieve constant frame rates for volume rendering with 3D textures, independent of the viewing positions (Sect. 2.4). Multi-bricking is complementary to the aforementioned distributed rendering of compressed data sets and can be combined with existing rendering methods.

3 Vector Field Visualization

Vector field visualization is another important and classical topic in scientific visualization that has been the subject of active research for many years. Typically, data originates from numerical simulations – such as those of computational fluid dynamics – or from measurements, and needs to be analyzed by means of visualization to gain an understanding of the flow. With the rapid increase of computational power for simulations, the demand for more advanced visualization methods has grown. This chapter focuses on a recent class of vector field visualization techniques: interactive texture-based methods. Texture-based techniques are well equipped to provide a dense representation of the vector field – they essentially cover the complete domain by visual patterns that show the underlying vector data. The visual patterns are constructed from particle tracing, which computes the motion of massless marker particles along the vector field. A dense representation has the advantage that the seed-positioning problem is automatically solved because particle traces are computed everywhere and, therefore, important features cannot be missed. All particle-tracing approaches for vector field visualization focus on the mapping step of the generic visualization pipeline. Here, the mapping step comprises the numerical aspect of particle tracing, as well as the actual mapping of particle traces to renderable representations. Texture-based vector field visualization greatly benefits from an implementation on GPUs because a large number of texels has to be processed by uniformly structured numerical operations. Only through the advent of programmable graphics hardware, interactive dense vector field visualization has become possible. Consequently, one important topic of this chapter is to address the efficiency of texture-based vector field visualization. The mapping to intuitively understandable visual representations, however, plays an equally important role because the user – as a human being – plays a crucial role in the complete visualization process. Section 3.3, for example, explicitly discusses both the numerical efficiency and the effective visual mapping for semi-Lagrangian texture advection.

82


Section 3.4 specifically addresses dye advection and solves the numerical diffusion problems that are inherent to traditional semi-Lagrangian advection schemes by using a level-set description. While the other parts of this chapter are restricted to Cartesian 2D or 3D domains, Sect. 3.5 extends texture-based vector field visualization to curved surfaces. Finally, Sect. 3.6 presents a novel description of texture-based vector field visualization that allows us to represent the various existing techniques within a unified framework. On the one hand, this unification facilitates a better understanding of the relationship between the different techniques, including their specific advantages and disadvantages. On the other hand, the flexibility of the framework allows us to come up with novel visualization approaches by just modifying parameters of the framework. As outlined before, the vector field visualization approaches of this chapter focus on the mapping step of the visualization pipeline. The filtering step is not discussed here, but could be easily added just before the mapping step. For example, denoising a measured vector field data set could be a useful filter module. Finally, the rendering step is rather simple for 2D vector fields because there is a natural mapping of a 2D texture-based vector field representation to the 2D screen. Rendering is more sophisticated for 3D vector fields; here, volume rendering techniques – as introduced in Chap. 2 and discussed in Sect. 3.3.5 in detail – are applied.

3.1 Basics of Particle Tracing The notion of a vector field can be defined mathematically by making use of concepts from differential geometry and the theory of differential equations. More detailed background information on these topics can be found in textbooks on differential topology and geometry [167, 168, 239, 289]. Although this mathematical approach might seem quite abstract for many applications, it has the advantage of being a flexible and generic description that is applicable to a wide range of problems. The following discussion is partly based on the work by Weiskopf and Erlebacher [471].1 3.1.1 Time-Dependent Vector Fields The definition of a vector field is given first. Let M be a smooth m-manifold with boundary, N be an nD submanifold with boundary (N ⊂ M ), and I ⊂ R be an open interval of real numbers. A map u : N × I −→ T M is a time-dependent vector field provided that 1

c 2005, with permission from Reprinted from Weiskopf and Erlebacher [471], Elsevier

3.1 Basics of Particle Tracing

83

u(x, t) ∈ Tx M . An element t ∈ I serves as a description for time; x ∈ N is a position in space. T M is a tangent bundle – the collection of all tangent vectors, along with the information of the point of tangency. Finally, Tx M is the tangent space associated with x. The vector field maps a position in space and time, (x, t), to a tangent vector located at the same reference point x. For a tangential time-dependent vector field, the mapping remains in the tangent bundle T N and therefore does not contain a normal component, i.e., u : N × I −→ T N . For a non-tangential vector field, a related tangential vector field can be computed by projection from Tx M to Tx N , i.e., by removing the normal parts from the vectors. Integral curves are directly related to vector fields. Let u : N × I → T N be a continuous (tangential) vector field. Let x0 be a point in N and J ⊂ I be an open interval that contains t0 . The C 1 map ξx0 ,t0 : J −→ N with

dξx0 ,t0 (t) = u (ξx0 ,t0 (t), t) dt is an integral curve for the vector field with initial condition x = x0 at t = t0 . The subscripts in the notation of ξx0 ,t0 denote this initial condition. These integral curves are usually referred to as pathlines, especially in the context of flow visualization. If u satisfies the Lipschitz condition, the above differential equation for ξx0 ,t0 has a unique solution. In all practical applications of flow visualization, the data is given on a manifold of two or three dimensions. To investigate a vector field on an arbitrary curved surface, the above formalism is necessary and, for example, the issue of describing the surface by charts has to be addressed. An alternative solution for vector fields on surfaces is described in Sect. 3.5. Very often, however, N is just a Euclidean space. This allows us to use a simpler form of the tangential vector field given by ξx0 ,t0 (t0 ) = x0

and

u : Ω × I −→ Rn ,

(x, t) −→ u(x, t) .

The vector field is defined on the nD Euclidean space Ω ⊂ Rn and depends on time t ∈ I. Here, the reference point x is no longer explicitly attached to the tangent vector. In general, boldface letters are used to denote vectors in Euclidean space. In this modified notation, the integral curve is determined by the ordinary differential equation dxpath (t; x0 , t0 ) = u(xpath (t; x0 , t0 ), t) . dt

(3.1)

84


It is assumed that the initial condition xpath (t0 ; x0 , t0 ) = x0 is given at time t0 , i.e., all integral curves are labeled by their initial conditions (x0 , t0 ) and parameterized by t. By construction, the tangent to the pathline at position x and time t is precisely the velocity u(x, t). In more general terms, the notation x(t; x0 , t0 ) is used to describe any curve parameterized by t that contains the point x0 for t = t0 . 3.1.2 Lagrangian Particle Tracing Integral curves play an important role in visualizing the associated vector field, and in understanding the underlying physics of the flow. There exist two important additional types of characteristic curves: streamlines and streaklines. Pathlines, streamlines, and streaklines are identical in steady flows. When the vector field depends explicitly on time, these curves are distinct from one another. In the special case of a steady flow, a streamline is defined as a solution to dxstream (t; x0 , t0 ) = u(xstream (t; x0 , t0 )) , dt

(3.2)

which is identical to a pathline. In the general case of an unsteady flow, a streamline is defined as the particle trace through the instantaneous vector field at a fixed time τ . In other words, the vector field is made artificially steady by virtually “freezing” it for one time. Accordingly, a streamline is generally defined as the solution to dxstream (t; x0 , t0 ) = u(xstream (t; x0 , t0 ), τ ) . dt Here, t and t0 are just parameters along the curve and do not have the meaning of physical time, in contrast to the physical time τ . A third type of curve is produced by dye released into the flow. If dye is released continuously into a flow from a fixed point x0 , it traces out a streakline. For example, smoke emanating from a chimney follows a streakline. It is instructive to derive integrated equations for pathlines, streamlines, and streaklines. The solution to the ordinary differential equation (3.1) is obtained by formal integration, which gives the pathline t u (xpath (s; x0 , t0 ), s) ds . (3.3) xpath (t; x0 , t0 ) = x0 + t0

Similarly, streamlines at time τ can be computed by t xstream (t; x0 , t0 ) = x0 + u (xstream (s; x0 , t0 ), τ ) ds .

(3.4)

t0

To obtain the snapshot of a streakline at time t, particles are released from x0 at times s ∈ [tmin , t] and their positions are evaluated at time t:

3.2 Classification and Overview of Flow Visualization Methods

85

xstreak (s; x0 , t) = xpath (t; x0 , s) . The streakline is parameterized by s, and tmin is the first time that particles are released. 3.1.3 Eulerian Particle Tracing The above approach to particle tracing adopts a Lagrangian point of view. Here, each single particle can be identified individually – labeled in some sense – and the properties of each particle are computed over time t. In the case of particle tracing, the properties – such as colors – can be subsumed by a function φ which does not change during time for a specific particle, i.e., the property function is time-independent: φLagrangian (x0 , t) = φLagrangian (x0 ). From an Eulerian point of view, the property φ is a function of position x and time t: the observer is located at a fixed position x and measures changes of φ caused by different particles that cross this position. As the property does not change for a specific particle, φ(x, t) = φ(x0 , t0 ) in the Eulerian approach if the spacetime positions (x, t) and (x0 , t0 ) belong to the pathline of the same particle, cf. (3.3). For a continuously differentiable property function, the advection equation can thus be obtained: ∂φ(x, t) + u(x, t) · ∇φ(x, t) = 0 . ∂t

(3.5)

In contrast to the Lagrangian approach, line-like structures are not directly available in an Eulerian description. Nevertheless, line-like patterns can be introduced by successive blending of Eulerian property fields or by convolution, respectively. Details are given in Sect. 3.3.4.

3.2 Classification and Overview of Flow Visualization Methods There exist many different vector field visualization techniques, which can be distinguished according to their properties with respect to a number of categories. The following classification should be rather considered as a collection of important issues than a complete taxonomy. In this overview section, the primary classification scheme for vector field visualization distinguishes techniques according to the relationship between a vector field and its associated visual representation. Point-based direct visualization approaches take into account the vector field at a point and possibly its neighborhood to obtain a visual representation. The vector field is directly

86


mapped to graphical primitives in the sense that no sophisticated intermediate processing of data is performed. Another class is based on characteristic curves obtained by particle tracing. The third class thoroughly preprocesses data to identify important features, which then serve as a basis for the actual visualization. Another type of property is the density of representation: the domain can be sparsely or densely covered by visualization objects. Density is particularly useful for subclassing particle-tracing approaches. Related to density is the distinction between local and global methods. A global technique essentially shows the complete flow, whereas important features of the flow can be missed by a local technique. A visualization method is also influenced by the structure of the data. The dimensionality of the manifold on which the vector field is defined plays an important role. For example, strategies that work well in 2D might be much less useful in 3D because of perceptual issues: the recognition of orientation and spatial position of graphical primitives is more difficult, and important primitives could be hidden by others. Dimensionality also affects performance; a 3D technique has to process substantially more data. If visualization is restricted to slices or more general hypersurfaces of a 3D flow, the projection of vectors onto the tangent spaces of the hypersurfaces has to be considered. Moreover, a distinction has to be made between time-dependent and time-independent data. A steady flow usually is much less demanding since frame-to-frame coherence is easy to achieve and streamlines, streaklines, and pathlines are identical. Finally, the type of grid has to be taken into account. Data can be provided, for example, on uniform, rectilinear, curvilinear, or unstructured grids. The grid type mainly affects the visualization algorithms with respect to data storage and access mechanisms or interpolation schemes. Ideally, the final visual representation does not depend on the underlying grid. In what follows, the techniques are roughly ordered according to their classification into: point-based direct methods, sparse representations for particle-tracing techniques, dense representations based on particle tracing, and feature-based approaches. The other types of properties are discussed along with the description of the individual methods. This classification approach and the corresponding review of flow visualization methods can also be found in a book chapter by Weiskopf and Erlebacher [471]. 3.2.1 Point-Based Direct Flow Visualization The traditional technique of arrow plots is a well-known example of direct flow visualization based on glyphs. Small arrows are drawn at discrete grid points, showing the direction of the flow and serving as local probes for the velocity field; see Fig. 3.1 (a). In the closely related hedgehog approach, the flow is visualized by directed line segments of which the lengths represent velocity magnitude. To avoid a possible distracting pattern for a uniform sampling by arrows or hedgehogs, their positions can be slightly perturbed by random


87

variations [96]. Arrow plots can be directly applied to time-dependent vector fields by letting the arrows adapt to the velocity field for the current time. For 3D representations, the following issues have to be considered. The position and orientation of an arrow is difficult to understand due to the projection onto the 2D image plane, and an arrow might occlude other arrows in the background. The problem of clutter can be addressed by highlighting arrows with orientations in a range specified by the user [26], or by selectively seeding the arrows. Illumination and shadows serve to improve spatial perception; for example, shadowing can be applied to hedgehog visualizations on 2D slices of a 3D flow [215]. More complex glyphs [80] can be used to provide additional information on the flow, as illustrated in Fig. 3.2. In addition to the actual velocity, information on the Jacobian of the velocity field is revealed. The Jacobian is presented in an intuitive way by decomposing the Jacobian matrix into meaningful components and by mapping them to icons based on easily understandable metaphors. Typical data encoded into glyphs comprises velocity, acceleration, curvature, local rotation, shear, or convergence. Glyphs can also be used to represent information on the uncertainty of the vector field data [493]. Glyphbased uncertainty visualization is also covered in [252, 320], which additionally discuss uncertainty representations for other visualization styles. Another strategy is to map flow properties to a single value and apply techniques known from the visualization of scalar data. Typically, the magnitude of the velocity or one of the velocity components are used. For 2D flow visualization, a mapping to color or to isolines (contour lines) is often applied. Volume visualization techniques have to be employed in the case of 3D data. Direct volume rendering, which avoids occlusion problems by selective use of semi-transparency, can be applied to single-component data derived from vector fields [106, 434]; recent developments are specifically designed for time-dependent data [59, 135, 314]. 3.2.2 Sparse Representations for Particle-Tracing Techniques Another class of visualization methods is based on the characteristic lines obtained by particle tracing. Among these are the aforementioned pathlines, streamlines, and streaklines. In addition, time lines, constructed from particles released at the same earlier time from different points along a curve, can be used. All these lines are quite intuitive because they represent some kind of transport along the flow. This section describes sparse representations, in which the spatial domain is not densely covered. A traditional particle-based approach computes characteristic curves and draws them as thin lines. Since many researchers handle time-independent vector fields, the notion of streamlines is used frequently. The visualization concepts can often be generalized to pathlines, streaklines, or time lines, even if not explicitly mentioned. Streamlines just serve as a role model for the


IMAGES A–C: WEISKOPF, ERLEBACHER [471]

c 2005 Elsevier

88

(a)

(b)

(c)

(d)

Fig. 3.1. Comparison of visualization techniques applied to the same 2D flow: (a) arrow plot, (b) streamlets, (c) LIC, (d) topology-based (image (d) courtesy of Gerik Scheuermann)

other characteristic lines. Particles traced for a very short time generate short streamlines or streamlets. Streamlines and streamlets can be used in 2D space, on 2D hypersurfaces of an underlying 3D flow, and for 3D flows. Hypersurfaces typically are sectional slices through the volume, or they are curved surfaces such as boundaries or other characteristic surfaces. It is important to note that the use of particle traces for vector fields projected onto slices may be misleading, even within a steady flow: a streamline on a slice may depict a closed loop, even though no particle would ever traverse the loop. The problem is caused by the fact that

89

IMAGE: WEISKOPF, ERLEBACHER [471]

c 2005 Elsevier


Fig. 3.2. Glyph-based 3D flow visualization, combined with illuminated streamlines (see also Color Plate 11)

flow components orthogonal to the slice are neglected during flow integration. For 3D flows, perceptual problems might arise due to distortions resulting from the projection onto the image plane. Moreover, issues of occlusion and clutter have to be considered. An appropriate solution is to find selective seed positions for particle traces that still show the important features of the flow, but do not overcrowd the volume; for example, a thread of streamlets along characteristic structures of 3D flow can be used [254]. The method of illuminated streamlines [508], based on illumination in diverse co-dimensions [6], improves the perception of those lines. In addition, depth information is increased and the problem of occlusion is addressed by making the streamlines partially transparent. An example is shown in Fig. 3.2. In 2D, particle traces are usually represented by thin lines, although the width of a line is sometimes modified to represent further information. Figure 3.1 (b) shows an example of a collection of streamlets. In 3D applications, however, the additional spatial dimension allows more information to be encoded into the graphical representation by using geometric objects of finite extent perpendicular to the particle trace. Examples of such an extension of streamlines in 3D are streamribbons and streamtubes. A streamribbon is the area swept out by a deformable line segment along a streamline. The strip-like shape of a streamribbon displays the rotational behavior of a 3D flow. Figure 3.3 shows a visualization of a 3D fluid simulation combining streamribbons, streamlines, arrows, and color coding [384]. An iconic streamtube [431] is a thick tube-shaped streamline of which the radial extent shows

90


Fig. 3.3. Combination of streamlines, streamribbons, arrows, and color coding for a 3D flow (courtesy of the BMW Group and Martin Schulz). See also Color Plate 12

the expansion of the flow. As a further extension of streamtubes, dash tubes [129] provide animated, opacity-mapped tubes. Stream polygons [383] trace an arbitrary polygonal cross section along a streamline and thus are closely related to streamtubes and streamribbons. The properties of the polygons – such as size, shape, or orientation – reflect properties of the vector field, including strain, displacement, and rotation. Streamballs [30] use their radii to visualize divergence and acceleration in a flow. Instead of spheres, other geometric objects such as tetrahedra [409] may be used. Another extension of streamlines is provided by stream surfaces, which are everywhere tangent to the vector field. A stream surface can be modeled by an implicit surface [439] or approximated by explicitly connecting a set of streamlines along time lines. Stream surfaces present challenges related to occlusion, visual complexity, and interpretation, which can be addressed by choosing an appropriate placement and orientation based on principal stream surfaces [44] or by user interaction [172]. Ray casting can be used to render several stream surfaces at different depths [128]. Stream arrows [255] cut out arrow-shaped portions from a stream surface and thus provide additional information on the flow,


91

such as flow direction and convergence or divergence. Stream surfaces can also be computed and visualized via surface particles [438], which are subject to less occlusion than a full-bodied surface. The generalization of the concept of particles released from single discrete points (for streamlines or streaklines) or from several points on a 1D line (for stream surfaces) leads to flow volumes [277]. A flow volume is a region of a 3D flow domain traced out by a 2D patch over time. The resulting volume can be visualized by volume rendering techniques. Since any flow volume can be (at least approximately) represented by a collection of tetrahedral cells, volume rendering techniques for unstructured grids can be applied, such as GPU-accelerated cell projection [357, 393] (also see Sect. 2.3.6). Flow volumes can be extended to unsteady flows [14], yielding the analogue of streaklines. Finally, time surfaces extend time lines to surfaces that are built from particles released from a 2D patch. The evolution of time surfaces can be handled by a level-set approach [486]. A fundamental issue of all particle-based techniques is an appropriate choice of initial conditions – seed point positioning – in order to catch all relevant features of the flow. Two main strategies can be identified: interactive or automatic placement of seed points. The interactive approach leaves the problem to the user and, in this sense, simplifies the problem from an algorithmic point of view. Nevertheless, the visualization system should be designed to support the user in identifying appropriate seed points. For example, the virtual wind tunnel [37] is an early virtual reality implementation of a flow visualization system where particles can be interactively released by the user. A useful approach for the automatic placement of seed points is to construct a uniform distribution of streamlines, which can be achieved for 2D vector fields [193, 430] or for surfaces within curvilinear grids of a 3D flow [269]. The idea behind a uniform distribution of streamlines is that such a distribution very likely will not miss important features of the flow. Therefore, this approach can be regarded as a step toward a completely dense representation, which is discussed in the following section. Equally spaced streamlines can be extended to multi-resolution hierarchies that support an interactive change of streamline density while zooming in and out of the vector field [196]. Moreover, with this technique, the density of streamlines can be determined by properties of the flow, such as velocity magnitude or vorticity. Evenly spaced streamlines for an unsteady flow can be realized by correlating instantaneous streamline visualizations for subsequent time steps [195]. Seeding strategies may also be based on vector field topology; for example, flow structures in the vicinity of critical points can be visualized by appropriately setting the initial conditions for particle tracing [445]. Since all particle-tracing techniques are based on solving the differential equation for particle transport, issues of numerical accuracy and speed must be addressed. Different numerical techniques can be applied for the initial value problem of ordinary differential equations. In many applications, ex-

92


plicit integration schemes are used, such as non-adaptive or adaptive RungeKutta methods. The required accuracy for particle tracing depends on the visualization technique; for example, first-order Euler integration might be acceptable for streamlets but not for longer streamlines. A comparison of different integration schemes [410] helps to judge the trade-off between computation time and accuracy. Besides the actual integration scheme, the grid on which the vector field is given is very important for choosing a particletracing technique. Point location and interpolation heavily depend on the grid and therefore affect the speed and accuracy of particle tracing. Both aspects are detailed in [307], along with a comparison between C-space (computational space) and P-space (physical space) approaches. The numerics of particle tracing is discussed, for example, for the tetrahedral decomposition of curvilinear grids [206], especially for the decomposition of distorted cells [363], for unstructured grids [431], for analytical solutions in piecewise linearly interpolated tetrahedral grids [308], for stiff differential equations originating from shear flows [410], and for sparse grids [412]. 3.2.3 Dense Representations for Particle-Tracing Methods Another class of visualization approaches is based on a representation of the flow by a dense coverage through structures determined by particle tracing. Typically, dense representations are built upon texture-based techniques, which provide images of high spatial resolution. A detailed description of texture-based and dense flow visualization can be found in a survey paper by Sanna et al. [373], in a state-of-the-art report by Laramee, Hauser, Doleisch, Vrolijk, Post, and Weiskopf [240], and in an overview paper by Weiskopf and Scheuermann [482]. The distinction between dense and sparse techniques should not be taken too rigidly because both classes of techniques are closely related by the fact that they form visual structures based on particle tracing. Therefore, dense representations also lead to the same intuitive understanding of the flow. Often, a transition between both classes is possible [444]; for example, texturebased techniques with only few distinct visual elements might resemble a collection of few streamlines and, on the other hand, evenly spaced streamline seeding can be used with a high density of lines. An early texture-synthesis technique for vector field visualization is spot noise [437], which produces a texture by generating a set of spots on the spatial domain. Each spot represents a particle moving over a short period of time and results in a streak in flow direction at the position of the spot. Enhanced spot noise [81] improves the visualization of highly curved vector fields by adapting the shape of the spots to the local velocity field. Spot noise can also be applied on boundaries and surfaces [74, 413]. A divide-and-conquer strategy makes possible an implementation of spot noise for interactive environments [73]. As an example application, spot noise was applied to the visualization of turbulent flow [75].


93

Line integral convolution (LIC) [43] is a widely used technique for the dense representation of streamlines in steady vector fields. It serves as a role model for the development of the visualization methods in this chapter. A LIC example is shown in Fig. 3.1 (c). LIC takes a vector field and a white noise texture as input. The noise texture is locally smoothed along streamlines by convolution with a filter kernel. This filtering leads to high correlation between the gray-scale values of neighboring pixels along streamlines, and to little or no correlation perpendicular to streamlines. The contrast and quality of LIC images can be improved by postprocessing techniques, such as histogram equalization, high-pass filtering, or a second pass of LIC [313]. The basic LIC technique does not show orientation and magnitude of the velocity field, an issue that is addressed by variants of LIC. Periodic motion filters can be used to animate the flow visualization, and a kernel phase shift can be applied to reveal the direction and magnitude of the vector field [43, 123]. Oriented Line Integral Convolution (OLIC) [457] exploits the existence of distinguishable, separated blobs in a sparse texture and smears these blobs into the direction of the local velocity field by convolution with an asymmetric filter kernel to show the orientation of the flow. In another approach, orientation is visualized by combining animation and adding dye advection [391]. Multi-frequency noise for LIC [214] visualizes the magnitude of the velocity by adapting the spatial frequency of noise. Other visualization techniques achieve LIC-like images by applying methods that are only indirectly related to line integral convolution. For example, fur-like textures [208] can be utilized by specifying the orientation, length, density, and color of fur filaments according to the vector field. Another method employs texture generation according to a reaction diffusion model [370], which can be also implemented on GPUs [371]. The integrate and draw [347] approach deposits random gray-scale values along streamlines. Pseudo LIC (PLIC) [444] is a compromise between LIC and sparse particle-based representations and therefore allows for a gradual change between dense and sparse visualizations. PLIC uses LIC to generate a template texture in a preprocessing step. For the actual visualization, the template is mapped onto thin or thick streamlines, thus filling the domain with LIC-like structures. The idea of thick streamlines can be extended to an error-controlled hierarchical method for a GPU-accelerated level-of-detail approach [25]. LIC can be extended to non-uniform grids and curved surfaces, for example, to curvilinear grids [123], and to 2D unstructured or triangular meshes [12, 271, 411]. Multi-granularity noise as input for LIC [270] compensates for the non-isometric mapping from texture space to the cells of a curvilinear grid that differ in size. The projection of the normal component of the vector field needs to be considered for LIC-type visualizations on hypersurfaces [375]. Unsteady Flow LIC (UFLIC) [392] or its accelerated version [250] incorporate time into the convolution to visualize unsteady flow. The issue of temporal coherence is addressed by successively updating the convolution results over time. In [123], a visualization of time-dependent flows on curvilinear surfaces

94


is presented. Dynamic LIC (DLIC) [405] is another extension of LIC, which allows for time-dependent vector fields, such as electric fields. A LIC-like image of an unsteady flow can also be generated by an adaptive visualization method using streaklines, where the seeding of streaklines is controlled by the vorticity [372]. Since LIC has to perform a line integral convolution for each element of a high-resolution texture, computational costs are an issue. One solution to this problem utilizes the coherence along streamlines to speed up the visualization process [161, 396]. Parallel implementations are another way of dealing with high computational costs [42, 509]. Finally, implementations based on graphics hardware can enhance the performance of LIC [163]. From a conceptional point of view, an extension of LIC to 3D is straightforward. The convolution along streamlines is performed within a volume; the resulting gray-scale volume can be represented by volume visualization techniques, such as texture-based volume rendering. However, computational costs are even higher than in 2D and therefore interactive implementations of the filtering process are hard to achieve. Even more importantly, possibly severe occlusion issues have to be considered: in a dense representation, there is a good chance of hiding important features behind other particle lines. A combination of interactive clipping and user intervention is one possible solution [341]. Alternatively, 3D LIC volumes can be represented by selectively emphasizing important regions of interest in the flow, enhancing depth perception, and improving orientation perception [177, 178, 179]. The main focus of this chapter is a closely related class of dense representations that is based on texture advection. The basic idea is to represent a dense collection of particles in a texture that is transported according to the motion of particles [276, 279]. For example, the Lagrangian coordinates for texture transport can be computed by a numerical scheme for convection equations [15]. The motion map [194] is an application of the texture-advection concept for animating 2D steady flows. The motion map contains a dense representation of the flow as well as the information required for animation. Lagrangian-Eulerian advection (LEA) [191] is a scheme for visualizing unsteady flows by integrating particle positions (i.e., the Lagrangian part) and advecting the color of particles based on a texture representation (i.e., the Eulerian aspect). LEA can be extended to visualizing vertical motion in a 3D flow by means of time surfaces [143]. Image-based flow visualization (IBFV) [440] is a recently developed variant of 2D texture advection. Not only is the (noise) texture transported along the flow, but additionally a second texture is blended into the advected texture at each time step. IBFV is a flexible tool that can imitate a wide variety of visualization styles. Another approach to the transport of a dense set of particles is based on nonlinear diffusion [91]. An initial noise image is smoothed along integral lines of a steady flow by diffusion, whereas the image is sharpened in the orthogonal direction. Nonlinear diffusion can be extended to the multi-scale visualization of transport in time-dependent flows [40].


95

Finally, some 3D flow visualization techniques adopt the idea of splatting, originally developed for volume rendering (see Sect. 2.3.5). Even though some vector splatting techniques do not rely on particle tracing, they are included in this section because their visual appearance resembles dense curve-like structures. Anisotropic “scratches” can be modeled onto texture splats that are oriented along the flow to show the direction of the vector field [62]. Line bundles [278] use the splatting analogy to draw each data point with a precomputed set of rendered line segments. These semi-transparent line bundles are composited together in a back-to-front order to achieve an anisotropic volume rendering result. Crawfis et al. [63] detail the splatting approach in the context of volume rendering techniques for flow visualization. For timedependent flows, the animation of a large number of texture-mapped particles along pathlines can be used [146]. For all splatting approaches, the density of representation depends on the number of splats. 3.2.4 Feature-Based Visualization Approaches The visualization concepts discussed so far operate directly on the vector field. Therefore, it is the task of the user to identify the important features of the flow from such a visualization. Feature-based visualization approaches seek to compute a more abstract representation that already contains the important properties in a condensed form and suppresses superfluous information. In other words, an appropriate filtering process is chosen to reduce the amount of visual data presented to the user. Examples of more abstract data are flow topology based on critical points, other flow features such as vortices and shock waves, or aggregated flow data via clustering. After features are computed, the actual visual representation has to be considered. Different features have different attributes; to emphasize special attributes for each type of feature, suitable representations must be used. Glyphs or icons can be employed for vortices or for critical points and other topological features. Examples are ellipses or ellipsoids that encode the rotation speed and other attributes of vortices. A comprehensive presentation of feature extraction and visualization techniques can be found in a survey article by Post et al. [334]. Topology-based 2D vector field visualization [165] aims to display only the essential information of the field. The qualitative structure of a vector field can be globally represented by portraying its topology. The field’s critical points and separatrices determine the nature of the flow. From a diagram of the topology, the complete flow can be inferred. Figure 3.1 (d) shows an example of a topology-based visualization. From a numerical point of view, the interpolation scheme is crucial for identifying critical points. Extended versions of topology-based representations make use of higher-order singularities [376] and C 1 -continuous interpolation schemes [377]. Another extension is based on the detection of closed streamlines, a global property that is not detected by the aforementioned algorithms [492]. Topology-based visualization can also

96


be extended to time-dependent vector fields by topology tracking [425]. The original topology-based techniques work well for data sets with a small number of critical points. Turbulent flows computed on a high-resolution grid, however, may show a large number of critical points, leading to an overloaded visual representation. This issue is addressed by topology simplification techniques [76, 77, 78, 423, 424, 495], which remove some of the critical points and leave only the important features. For the visualization of 3D topology, suitable visual representations need to be used. For example, streamlines that are traced from appropriate positions close to critical points connect to other critical points or the boundary to display the topology, whereas glyphs can be used to visualize various classes of critical points [136]. Topology can also serve as a means for determining the similarity between two different vector fields [11, 243]. Vector field clustering is another way of reducing the amount of visualization data. Vectors of the original high-resolution field are combined into fewer vectors that approximately represent the vector field at a coarser resolution, leading to a visualization of aggregated data. An important issue is to establish appropriate error measures to control the way vectors are combined into clusters [253, 413]. The extension of vector field clustering to several levels of clustering leads to hierarchical representations [132, 131, 159]. In related approaches, segmentation [91], multi-scale visualization [40], or topology-preserving smoothing based on level-set techniques [486] reduce the complexity of the displayed vector fields. An important class of feature detection algorithms is based on the identification of vortices and their respective vortex cores. Vortices are useful for identifying significant features in a fluid flow. One way of classifying vortex detection techniques is the distinction between point-based, local techniques, which directly operate on the vector data set, and geometry-based, global techniques, which examine the properties of characteristic lines around vortices [366]. Local techniques build a measure for vortices from physical quantities of a fluid flow. An overview on these techniques is given in [7, 354]; more recent contributions are presented in [13, 189, 205, 355]. A mathematical framework [325] makes it possible to unify several vortex detection methods. Point-based methods are rather simple to compute, but are more likely to miss some vortices. For example, weak vortices, which have a slow rotational component compared to the velocity of the core, are hard to detect by these techniques. Geometry-based, global methods [188, 364, 365] are usually associated with higher computational costs, but allow a more robust detection or verification of vortices. Shock waves are another important feature of a fluid flow because they can increase drag and even cause structural failure. Shock waves are characterized by discontinuities in physical quantities, such as pressure, density, or velocity. Therefore, shock detection algorithms are related to edge detection methods known from image processing. A comparison of different techniques for shock extraction and visualization is available [265]. Flow separation and

3.3 Semi-Lagrangian Noise and Dye Advection on Cartesian Domains

97

attachment, which occur when a flow abruptly moves away from, or returns to, a solid body, are another interesting feature of a fluid flow. Attachment and separation lines on surfaces in a 3D flow can be automatically extracted by means of phase plane analysis [204].

3.3 Semi-Lagrangian Noise and Dye Advection on Cartesian Domains The remaining parts of this chapter focus on texture-based techniques for flow visualization. The discussion begins with the display of vector fields by means of semi-Lagrangian texture advection. Semi-Lagrangian texture advection can be separated into two different parts: first, a transport mechanism and, second, a method to introduce visual patterns. The semi-Lagrangian approach is based on an Eulerian description of “particles”, which are represented indirectly via a property field φ that is a function of position x and time t (see Sect. 3.1.3). The observer is located at a fixed position x and measures changes of φ caused by the fact that different particles cross that position. The first aspect of texture advection deals with the transport mechanism, which implements the propagation of particles according to (3.5). Slightly different numerical schemes to solve this equation are discussed in Sects. 3.3.1 and 3.3.3. In contrast to the Lagrangian, particle-based approach, line-like structures are not directly available in this Eulerian description. Therefore, an additional visual processing is needed to introduce visual patterns that resemble characteristic curves of the vector field. These visual patterns are usually introduced by means of convolution that modifies the spatial frequency of the image along characteristic curves – a higher coherence is present along these lines than perpendicular to these lines. A clear distinction between transport mechanism and visual mapping is often missing in the previous literature. This book, in contrast, makes a distinction between the two fundamental aspects of texture-based flow visualization. One benefit of this point of view is that different methods for transport and visual mapping can be assembled in new, and possibly improved, combinations. Moreover, this point of view allows for a better analysis of the contributions and properties of existing visualization techniques. Dye advection and dense, noise-based representation are the two most dominant metaphors for texture-based flow visualization. They differ in their mapping to visual properties, but can be realized by the same underlying transport mechanism(s). For example, either a semi-Lagrangian approach (as described in this section) or a completely Lagrangian approach (see Sect. 3.6) can be used for both dye and noise transport. The advection of dye and noise textures allows for different but equally useful results. Both approaches are often combined within the same visualization system. Dye is released at userspecified positions and therefore supports an interactive and user-centric visual exploration process. At the same time, a dense representation can provide

98


additional information on the overall flow behavior and serve as a background context in which the user can focus on details of the dye-based visualization. 3.3.1 Semi-Lagrangian Transport Mechanism The in-depth discussion of semi-Lagrangian texture advection begins with the underlying transport mechanism. Here, particles or marker “objects” (such as dye) are modeled as massless material that is perfectly advected along the input vector field. From an Eulerian point of view (see Sect. 3.1.3), particle properties are stored in a field φ(x, t) of which the evolution is governed by ∂φ(x, t) + v(x, t) · ∇φ(x, t) = 0 , ∂t

(3.5)

where v can be identified with u in the notation of this equation on p. 85. This partial differential equation could be discretized and solved by different Eulerian techniques [317]. Instead, a semi-Lagrangian approach is often used because it is well-suited for such a convection process and leads to a stable evolution even for large step sizes [397]. The advection is performed along characteristic lines: typically, pathlines xpath (t; x0 , t0 ) or streamlines xstream (t; x0 , t0 ) are considered. Therefore, (3.1) or (3.2) can be used to compute the transport in a semi-Lagrangian manner (see, e.g., [397, 190]). One possible method is based on backward integration: φ(x(t0 ), t0 ) = φ(x(t0 − ∆t; x0 , t0 ), t0 − ∆t) .

(3.6)

Starting from the current time step t0 , an integration backward in time provides the position along the characteristic curve (e.g., pathline or streamline) at the previous time step, x(t0 − ∆t; x0 , t0 ). The property field is evaluated at this previous position to access the value that is transported to the current position. This method can be considered as a semi-Lagrangian approach because the property is represented on a texture in an Eulerian way, whereas the temporal evolution relies on a Lagrangian integration of particle paths. Similarly to (3.6), an integration forward in time leads to a forward mapping in which property values are actively scattered from their current location to a new position. In an analytic description, backward and forward mapping are equivalent. Slight differences may appear when a numerical approximation for pathline or streamline integration is employed. To tackle the advection problem on a computer, the particle-tracing equations (3.1) or (3.2) have to be solved numerically and the property function φ has to be discretized. A simple method to solve an ordinary differential equation is the first-order explicit Euler integration, which yields x(t0 − ∆t; x0 , t0 ) = x(t0 ; x0 , t0 ) − ∆t v(x(t0 ; x0 , t0 ), t0 ) for a pathline. An analogous expression describes the Euler integration for a streamline. Furthermore, the property φ is discretized on a Cartesian or


99

uniform grid, both spatially and temporally. Bilinear (in 2D) or trilinear (in 3D) interpolation is applied to reconstruct the property field at locations that are different from grid points. Combining a discrete grid for φ with Euler integration for particle positions, the following numerical scheme is obtained to propagate a solution from time (t0 − ∆t) to t0 : φt (x) = φt−∆t (x − ∆t vt (x)) .

(3.7)

In this slightly changed notation, the superscripts in φt and vt denote the time dependency of the property field and vector field, respectively. 3.3.2 GPU Implementation of Semi-Lagrangian Advection A GPU implementation has to address the problem of how data structures and operations working on that data can be mapped to graphics hardware. The uniform grid that stores the property field can be directly represented by 2D or 3D textures, depending on the dimensionality of the domain. In purely GPU-based implementations, the velocity field is also held in a 2D or 3D texture. For hybrid CPU and GPU-based implementations, the vector field is stored in main memory and used for advection computations by the CPU. In general, a single advection step computes the particle paths for the corresponding time step and transports the property field accordingly. Often, an explicit first-order Euler integration is used for particle tracing, which is either computed by the CPU (for a hybrid approach) or by the GPU. The transport of the property texture is based on an appropriate specification of texture coordinates, which can be done either on a per-vertex basis (for hybrid CPU / GPU approaches) or on a per-fragment basis (i.e., purely GPU-based). 2D Semi-Lagrangian Advection Three different approaches can be used for semi-Lagrangian advection with planar 2D vector fields. The first approach applies per-fragment operations: a fragment program determines the previous position (x − ∆t vt (x)), based on the vector field vt that stores the vx and vy components in a two-component texture format. The coordinates of the previous position are interpreted as texture coordinates for a dependent texture lookup in φt−∆t . This GPU-based method allows for an efficient update of the property texture in a single rendering pass; this technique was first proposed by Weiskopf, Hopf, and Ertl [479] and later described in more detail by Weiskopf and Hopf [478]. Figure 3.4 shows the DirectX pixel shader code for this advection method. A single quadrilateral spanning the spatial domain is rendered to generate the fragments for the pixel shader. The texture coordinates t0 and t1 are chosen to cover the domain by the textures. The dependent texture lookup (shader instruction texbem) uses the previously fetched vector field entry v, scales this vector by a constant value (a parameter that is specified outside the pixel

100

3 Vector Field Visualization ps.1.1 // Pixel shader V1.1 tex t0 // Load vector field data v texbem t1,t0 // Dependent lookup in tex φ with // shifted texture coordinates mov r0, t1; // Result of advection Fig. 3.4. DirectX pixel shader code for 2D backward advection

shader and that takes into account the step size −∆t), and adds the texture coordinates t0 in order to obtain the coordinates for accessing the property texture φt−∆t . An animated visualization is built from iterative executions of the above advection step. In all computations, the property field is accessed for only two time steps: the current time step (with a write access) and the previous time step (with a read access). Therefore, it is sufficient to provide two 2D textures to hold these two time steps. The texture φt−∆t is used as an input texture, whereas the texture φt is modified by directly rendering into it via the renderto-texture functionality. After each single advection computation, the roles of the two textures are exchanged, according to a ping-pong rendering scheme. The second approach also employs a backward advection. However, this approach computes the advection on a per-vertex basis, as described by Van Wijk [441]. The domain of the property field is covered by a mesh, with velocities assigned at its vertices. A CPU program calculates the previous positions (x − ∆t v(x)) based on the vector field held in main memory. These previous positions are used as per-vertex texture coordinates for a lookup in φt−∆t . The domain is rasterized by rendering the mesh. The third approach uses a per-vertex advection forward in time [440]. A forward Euler integration (x+∆t v(x)) provides the positions for the following time step. These new coordinates are used as the vertex coordinates for a distorted triangle mesh, i.e., vertex coordinates are modified instead of texture coordinates. All three approaches employ a texture lookup in the property field that is based on bilinear interpolation. An interpolation scheme is required because pathline or streamline integration usually results in texture or vertex coordinates that do not have a one-to-one correspondence between pixels on the image plane and grid points of the property field φt−∆t . 3D Semi-Lagrangian Advection The extension to 3D vector fields requires the following modifications, as proposed in a paper by Weiskopf and Ertl [476]. First, the property field and the vector field are represented by 3D textures. The transport of the property field along one time step can be realized by a dependent texture lookup that is analogous to the per-fragment approach for 2D fields; the only difference


101

load flow field into GPU memory for i = 1 to max slice render quad for slice i, with dependent texture lookup update slice i in new property field end for Fig. 3.5. Pseudo code for 3D advection [476]

is that 2D texture coordinates are replaced by 3D texture coordinates for a lookup in a 3D texture. The second modification takes into account that GPUs can only render 2D images, not directly a completely 3D output. Therefore, the property field for a subsequent time step is built in a slice-by-slice manner. Figure 3.5 shows the pseudo code for a single time step of the advection process. Each slice of the property field is updated by rendering a quadrilateral that represents this 2D subset of the full 3D domain. The quadrilateral is rasterized by the GPU to generate the fragments. The dependent-texture lookup can be realized by a fragment program that computes the modified texture coordinates according to the Euler integration of the flow field. Figure 3.6 shows the corresponding OpenGL ARB fragment program. Note that

!!ARBfp1.0 PARAM stepSize = program.local[0]; PARAM biasExtend = 2.0, -1.0, 0.0, 0.0; ATTRIB iTexCoord = fragment.texcoord[0]; OUTPUT oColor = result.color; # Temporary variables (registers) TEMP velocity; TEMP oldPos; # Fetch flow field TXP velocity, iTexCoord, texture[0], 3D; # Mapping to positive and negative values MAD velocity, velocity, biasExtend.x, biasExtend.y; # Compute previous position (Euler integration) MAD oldPos, velocity, stepSize, iTexCoord; # Dependent texture lookup: # Advected property value from previous time step TEX oColor, oldPos, texture[1], 3D; END Fig. 3.6. ARB fragment program for 3D texture advection [476]

102


the components of the vector field texture are stored as unsigned fixed-point numbers and thus a bias/extend transformation is applied to achieve positive and negative values. A related, but slower approach that requires multiple render passes per slice on an Octane2 with pixel textures was described by Weiskopf, Hopf, and Ertl [479]. The above advection method is completely formulated in 3D and therefore takes into account any flow field. In contrast, 3D image-based flow visualization (3D IBFV) [414] uses a restricted model with 2D textures and with gathering of values along the z axis, and thus is restricted to a limited class of flows: the z component of the vector field has to be very small so that the absolute value of ∆tvz is not larger than the slice distance along the z axis. Moreover, the 3D texture-based approach makes use of built-in trilinear interpolation and thus allows for different resolutions of particle and vector fields, whereas 3D IBFV supports only a bilinear resampling on 2D slices. Finally, 3D IBFV requires multiple render passes to update a single slice of the property field. 3D Advection Based on 2D Texture Memory A potential problem of the above 3D texture advection is connected with the slice-by-slice update of the 3D texture for the property field. An update of a 3D texture often is possible only by transferring data to and from main memory. For example, Direct3D does not provide a mechanism to directly modify 3D textures from other data on the GPU. Although OpenGL allows us to update a slice of a 3D texture by glCopyTexSubImage3D, the speed of such an update can vary between GPU architectures because of different internal memory layouts of 3D textures (see the related discussion on read access for 3D textures in Sect. 2.4). Even though the OpenGL superbuffer extension [327] or the frame-buffer object extension [390] allow us to render directly into a 3D texture, the fundamental issues of the speed of updating 3D textures and the extent of the vendor’s support for those extensions may remain. Therefore, Weiskopf, Schafhitzel, and Ertl [481] proposed an alternative approach that is based on 2D texture memory instead of 3D texture memory. The following discussion is based on their paper.2 2D textures have several advantages: they are available on any GPU, they provide good caching mechanisms and efficient bilinear resampling, and they support an extremely fast write access by the render-to-texture functionality. For a 2D texture-based implementation of 3D advection, logical memory and physical memory have to be distinguished. Logical memory is identical as for 3D texture advection – it is organized in the form of a uniform 3D grid. Physical memory is a 2D uniform grid represented by a 2D texture. The coordinates for addressing the logical memory are denoted by x = (x, y, z) and the coordinates for physical memory by u = (u, v). A slice of constant value 2

c The Eurographics Association 2005. Reprinted, with permission, from Weiskopf, Schafhitzel, and Ertl [481]

3.3 Semi-Lagrangian Noise and Dye Advection on Cartesian Domains lookup table Φ0

0

1

2

. . . . . . . . .

i tex

Φ

logical memory

103 z (u,v) i tex

i tex

. . .

physical memory IMAGE: WEISKOPF, SCHAFHITZEL, ERTL [481]


Fig. 3.7. Mapping between logical 3D memory and physical 2D memory by means of a lookup table

z in logical memory corresponds to a tile in physical memory, as illustrated in Fig. 3.7. Different tiles are positioned in physical memory with a row-first order. Since the maximum size of a 2D texture may be limited, several “large” 2D textures may be used to provide the necessary memory. These 2D textures are labeled by the integer-valued index itex . Since all numerical operations of 3D advection are conceptually computed in logical 3D space, an efficient mapping from logical to physical memory is needed. This mapping is described by the function Φ : (x, y, z) −→ ((u, v); itex ) . The 2D coordinates can be separated into the coordinates for the origin of a tile, u0 , and the local coordinates within the tile, ulocal : Φ : (x, y, z) −→ (u0 + ulocal ; itex ) ,

(3.8)

u0 = Φ0,u (z), itex = Φ0,itex (z), ulocal = (sx x, sy y) .

(3.9)

with The function Φ0 maps the logical z value to the origin of a tile in physical memory and is independent of the x and y coordinates. The map Φ0 can be represented by a lookup-table (see Fig. 3.7), which can be efficiently implemented by a dependent texture lookup. Conversely, the local tile coordinates are essentially identical to (x, y) – up to scalings (sx , sy ) that take into account the different relative sizes of texels in logical and physical memory. If more than one “large” 2D texture is used, multiple texture lookups in these physical textures may be necessary. However, multiple 2D textures are only required in tiles that are close to the boundary between two physical textures because the maximum difference vector for the backward lookup in (3.7) is bounded by ∆tvmax , where vmax is the maximum velocity of the data set. Different fragment programs are used for boundary tiles and internal tiles to reduce the number of texture samplers for the advection in internal regions.

104


Table 3.1. Performance for steady vs. unsteady 3D advection based on 2D textures (in frames per second) [481] Radeon X800 Domain size Steady Unsteady

128

3

114.2 51.2

GeForce 6800

3

256

1283

2563

13.0 9.9

96.4 39.0

16.7 6.7

Trilinear interpolation in logical space is implemented by two bilinear interpolations and a subsequent linear interpolation in physical space. Bilinear interpolation within a tile is directly supported by built-in 2D texture interpolation. A one-texel-wide border is added around each tile to avoid any erroneous influence by a neighboring tile during bilinear interpolation. The subsequent linear interpolation takes the bilinearly interpolated values from the two closest tiles along the z axis as input. This linear interpolation is implemented within a fragment program. While trilinear interpolation by the above mapping scheme is necessary for the read access in (3.6) or (3.7), write access is more regularly structured. First, write access does not need any interpolation because it is restricted to grid points (i.e., single texels). Second, the backward lookup for (3.6) or (3.7) allows us to fill logical memory in a slice-by-slice manner and, thus, physical memory in a tile-by-tile fashion. A single tile can be filled by rendering a quadrilateral into the physical 2D texture if the viewport is restricted to the corresponding subregion of physical memory. Table 3.1 shows performance measurements for the implementation of 3D advection with 2D textures on ATI Radeon X800 Platinum Edition (256 MB) and NVIDIA GeForce 6800 Ultra (256 MB) GPUs. This implementation is based on C++ and DirectX 9.0. All shader programs are formulated with high-level shading language (HLSL). The size of the property field is given in Table 3.1. A vector field of size 1283 (tornado from Fig. 3.11) is used. The measurements indicate that advection speed is roughly proportional to the number of texels. The table also compares the visualization performance for unsteady and steady flow. For unsteady flow, a new 3D texture for the vector field is transferred for each frame. Although the rendering speed is reduced for unsteady flow, the overall performance still facilitates interactive visualization for property fields up to 2563 . Table 3.2 compares the 3D advection method based on 2D textures (implemented in DirectX) with the advection technique realized by 3D textures (implemented in OpenGL with glCopyTexSubImage3D). These performance measurements were conducted on an ATI Radeon X800 GPU. Table 3.2 shows that the 2D texture-based method outperforms the 3D texture-based method, i.e., the benefit of fast read and write access to 2D textures outweighs the ad-


105

Table 3.2. Comparison between the 3D advection method based on 2D textures and the advection method based on 3D textures on a Radeon X800 GPU (in frames per second) [481] 2D texture Domain size Advection only

128

3

114.2

3D texture

3

256

1283

2563

13.0

44.4

9.7

ditional operations for the mapping of texture coordinates from logical 3D space to physical 2D space. 3.3.3 Lagrangian-Eulerian Advection Lagrangian-Eulerian advection (LEA) [191] addresses issues introduced by bilinear interpolation within the backward lookup in semi-Lagrangian integration (3.6). Especially for a noise signal stored in the property field, this bilinear resampling leads to a rapid decrease of contrast and high spatial frequencies: the property field is increasingly blurred. The following description of LEA specifically aims at improving the advection of white noise fields. The basic idea is to use nearest-neighbor interpolation for resampling, along with a mechanism that corrects errors introduced by nearest-neighbor interpolation. Once again, all information concerning particles or property values is stored in 2D arrays or textures, i.e., the discussion is restricted to planar 2D flow. A coordinate system is assumed in which the grid points of textures are located at integer-valued locations (i, j). LEA uses an additional 2D field c to store the coordinates of particles within a texture cell. Note that the two-component field c(i, j) describes only the fractional part of the coordinates – actual coordinates with respect to the grid are x = (i, j) + c(i, j). Stated differently, each texture cell contains a single virtual particle that can be moved within the cell and even be transported into different cells by advection. Based on the array of fractional coordinates, a backward integration step yields the coordinates x at the previous time step: (3.10) x = (i, j) + c(i, j) − ∆t v(i, j) . To maintain a high contrast in the advected noise and to avoid artificial diffusion, φ is computed by a nearest-neighbor sampling of the property field from the previous time step, based on the coordinates x for the previous time step. A correction step is introduced to counteract a duplication effect that occurs during the computation of (3.10). Effectively, if particles in neighboring cells of φ retrieve their property value from within the same cell of φ, this value will be duplicated in the corresponding cells of φ. This effect is undesirable since lower noise frequency reduces the spatial resolution of features that

106


Table 3.3. Performance measurements for Lagrangian-Eulerian advection in frames per second (ATI Radeon 8500 GPU vs. Athlon 1.2 GHz CPU) [473] Noise Advection Particle Size Flow Size GPU (steady) GPU (unsteady) CPU-based

Noise, Dye & Mask

2

1024 2562

2

1024 10242

10242 2562

10242 10242

24.8 24.3 0.8

23.8 19.4 0.8

20.5 20.3 NA

19.9 16.7 NA

can be represented. This duplication effect is caused by the discrete nature of texture sampling and is further reinforced in regions where the flow has a strong positive divergence. To break the undesirable formation of uniform blocks and to maintain a high frequency random noise, a user-specified percentage of noise is injected into φ. Random cells are chosen in φ and their value is inverted. From experience, a fixed percentage of two to three percent of randomly inverted cells provides adequate results over a wide range of flows. Finally, the array c of fractional coordinates is re-initialized to prepare a new collection of particles to be integrated backward in time for the next iteration. Nearest-neighbor sampling implies that a property value can only change if it originates from a different cell. If fractional coordinates were neglected in the next iteration, subcell displacements would be ignored and the flow would be frozen where the velocity magnitude or the integration step size is too small. Nearest-neighbor sampling without fractional coordinate tracking causes the flow to be partitioned into distinct regions within which the integer displacement vector is constant. Therefore, the array c is set to the fractional part of the newly computed coordinates. Since the original CPU-based implementation of LEA allows for the interactive visualization of only small datasets (in the range of 1282 ), a much faster GPU-based approach was proposed by Weiskopf, Erlebacher, Hopf, and Ertl [473]. This method makes use of efficient single-pass noise advection on an ATI Radeon 8500 GPU, which has DirectX pixel shader 1.4 support. A similar implementation is feasible with any of the more recent GPUs, like ATI Radeon 9700, NVIDIA GeForce FX, or newer. Table 3.3 compares performance measurements for the GPU-based implementation (ATI Radeon 8500, Athlon 1.2 GHz CPU, Windows 2000) with those for a CPU-based implementation on the same machine. Included are the numbers for mere noise advection (single-pass advection) and for advection of noise and dye images (two advection passes) with subsequent velocity masking (see Sect. 3.3.5 for a discussion of velocity masking). A final rendering to a 10242 viewport is included in all tests. The performance measurements indicate that the download of the 2D vector field data from main memory


107

to texture memory via AGP bus (line “GPU (unsteady)”) takes only a small portion of the overall rendering time even for large data sets. For many applications, the flow field is medium-sized and leads to typical overall frame rates of 15–25 fps. The cost of the CPU implementation is dominated by the advection of the noise texture and is quite independent of the size of the flow field data; a transfer of flow data to texture memory is not necessary. 3.3.4 Particle Injection So far, only basic advection mechanisms have been discussed. However, a useful visualization needs – besides the computation of particle traces – a mapping of the particle traces to a graphical representation. In the model of this section, the mapping is essentially restricted to an appropriate injection of property values. The basic idea of the IBFV approach is adopted and further extended. IBFV introduces new property values at each time step, described by an injection texture I. The structure of the injection mechanism of 2D IBFV [440] is illustrated in Fig. 3.8. The pure advection mechanism is typically based on semi-Lagrangian advection (Sects. 3.3.1 and 3.3.2) or LEA (Sect. 3.3.3). Original 2D IBFV is based on an affine combination of values from the previously advected texture φ(t − ∆t) and the injection texture I(t) to yield the new property texture φ(t): φ(t) = (1 − α)φ(t − ∆t) + αI(t) .

(3.11)

The variable α denotes a scalar blending parameter that is constant for the complete visualization domain. A repeated application of this alpha blending results in an exponential decay of the injected noise over time. For a timeindependent injection texture and a steady vector field, this alpha blending resembles the result of LIC with an exponential filter kernel. In the context of 3D texture advection, 3D IBFV [414] introduces a slightly extended injection scheme that allows for space-variant noise injection weights H(t): φ(t) = (1 − H(t))φ(t − ∆t) + H(t)I(t) . (3.12) The structure and contents of the injection texture determines the appearance of the final visualization. A dense, LIC-like result is achieved by specifying a

advection for the next time step

advected texture

injection texture blending

IMAGE: WEISKOPF, SCHAFHITZEL, ERTL [481]


Fig. 3.8. Basic structure of the injection mechanism of IBFV: new property values are introduced from an injection texture by blending

108


noise texture as injection texture. The spatial frequency of the noise corresponds to the spatial frequency of the visualization output. The injection texture typically contains filtered noise to avoid aliasing artifacts (see the discussion by Van Wijk [440]). Moreover, the injection may be time-variant to introduce time-dependent visualizations even for steady flows. As already proposed for IBFV [440], noise is used that is periodically switched on and off with slow-in and slow-out behavior. As long as the periodicity of this time dependency is the same for the complete domain, a single, time-independent texture that holds both the noise values and the random phases is sufficient. For a specific time step, the actual noise injection value is obtained from a lookup table that describes the temporal behavior of the slow-in and slow-out. The relative phase from the noise injection texture is added to the current global time to yield the local time of that noise. This time – modulo the temporal periodicity – serves as the basis for the aforementioned table lookup. The advantage of this approach is that a single noise-injection texture is sufficient to represent a time-dependent noise injection. Besides this noise-based dense representation, dye advection is a powerful and intuitive visualization metaphor. Virtual dye is released at user-specified positions and therefore supports an interactive and user-centric visual exploration process. Texture-based advection is well-suited for dye visualization because of its built-in support for transporting extended regions of dye. The visualization performance is independent of the number of cells covered by dye. Therefore, a user interface is possible that allows the user to control dye injection by virtually painting into the flow, as shown by Weiskopf, Erlebacher, Hopf, and Ertl [473]. This interaction approach is more difficult to implement for particle-based Lagrangian tracking. To maintain a dense and uniform coverage of the domain in regions of divergent or convergent flow, data structures have to be implemented that allow for dynamic insertion and deletion of particles in the Lagrangian approach [392]. Alternatively, a large number of particles can be traced to achieve a minimum density of particles in divergent flow areas, which decreases the efficiency of the implementation [79]. The combination of a dense representation by noise injection and a userguided exploration by injecting dye at isolated locations is very powerful because it combines both an overall view and a detailed visualization of specific features. To achieve this goal, the paper by Weiskopf and Ertl [476] generalizes the aforementioned injection schemes to allow for a unified description of both noise and dye advection. The extensions are: first, the restriction to an affine combination of the advected value and of the newly injected value is suspended and replaced by a generic combination of both; second, several materials can be advected and blended independently. The extended blending equation is given by φ(t) = W (t) ◦ φ(t − ∆t) + V (t) ◦ I(t) ,


109

where the two, possibly space-variant, weights W (t) and V (t) need not add up to one. The symbol “◦” denotes a component-wise multiplication of two vector quantities, i.e., W (t), V (t), φ(t), and I(t) must have the same number of components. In this approach, the different components of each texel in the property field describe the density of different materials that are transported along the flow (rather than color or gray-scale values). The advantages of this extended blending scheme are: first, different materials are blended independently from each other and may therefore have different lengths for exponential decay; second, material can be added on top of existing material (e.g., additional dye), which is impossible with an affine combination as in (3.11) and (3.12). A unified description of both dye and noise advection is supported by the extended blending. Typically, dye is faded out only very slowly or not at all. Since new dye has to be injected at seed points, both W (t) and V (t) need to be one or close to one, and therefore the sum is larger than one. A possible saturation of material can be controlled by clamping. Conversely to dye visualization, the streaklines generated by noise advection tend to be much shorter. Furthermore, the overall brightness of noise-based representations should be independent of blending weights. Therefore, noise advection typically makes use of an affine combination of advected and newly injected material. Dye injection is represented by an injection texture as well. The shape of the dye emitter is stored in a voxelized form. A random phase is not needed here; dye injection is either switched on or off, depending on user interaction. 3.3.5 Visual Mapping and Rendering In the aforementioned blending schemes, the property fields do not need to directly contain color values, but could represent densities of different materials, which are coded into the RGBA channels of the property texture (i.e., a maximum number of four materials is possible with a single texture). A separate transfer function is applied to each material to obtain the corresponding color values. A scenario is assumed with a fixed assignment of materials, so that a collection of different 1D textures can be used to represent the transfer functions. The different transfer functions are evaluated on a per-fragment level and the results are combined to a single color by summation. From a visualization point of view, it is very effective to interactively choose whether and how different materials are displayed. For example, the noise part could be rendered very faintly to give an overall context and, at the same time, the dye part could be emphasized by bright colors to focus on this detail. In another scenario, dye could be completely removed and noise could be rendered more prominently. It is important that these different visualization approaches can be interactively changed in order to adapt the appearance of the visualization to the users’ needs. This goal is achieved by modifying the respective transfer functions.

110


To further reduce the visual complexity and to highlight important features of the flow, additional scalar quantities can be mapped to the color and opacity values in the final rendering. Scalar quantities can either be derived from the vector field itself (e.g., velocity magnitude) or from other parameters of the data set (e.g., pressure or temperature in a fluid flow). For example, the velocity magnitude can be used to emphasize interesting regions of high flow magnitude and fade out parts with low speeds (i.e., velocity masking). Rendering is trivial for 2D flow visualization: the planar 2D property field is transformed into an RGB image by applying the aforementioned transfer functions, and the resulting image is directly displayed to the user. Figure 3.9 shows an example visualization of a 2D unsteady flow field by means of 2D texture advection. The transport of the noise texture is implemented by LEA (Sect. 3.3.3); the data set is produced by the interaction of a planar shock with a longitudinal vortex (in axisymmetric geometry) [114]. The images show a single time step from a time series of an unsteady flow simulation. The complete data set has 200 time steps and a spatial resolution of 256 × 151. In Fig. 3.9 (a), only noise advection is employed. The noise blending factor is α = 0.05. Uninteresting regions of the flow are faded out by reducing their brightness via velocity masking. Therefore, one clearly sees the primary shock, secondary shock, and the slip lines [176]. Figures 3.9 (b) and (c) additionally include the advection of colored dye which was released into the flow by the user. In Fig. 3.9 (c), velocity masking is applied to enhance important parts of the flow. In Fig. 3.9 (b), no velocity masking is applied. Direct volume visualization is used to render the results of 3D texture advection because it allows the user to view a volume data set at different depth positions simultaneously by means of semi-transparency. For texture advection based on 3D textures, volume rendering makes use of 3D texture slicing with image-aligned slices, as described in Sect. 2.3.1. Alternatively, GPU ray casting could be employed. A 3D texture holds the different material densities in different color channels, and the aforementioned transfer functions are applied on-the-fly in the form of dependent-texture lookups while a fragment of a slice or along a ray is processed. For 3D advection based on 2D textures, volume rendering with 2D slicing is employed (see Sect. 2.3.2). Volume rendering accesses tile after tile in frontto-back order by rendering object-aligned quadrilaterals. Texture coordinates are issued to address a tile in the physical memory of the “large” 2D texture. Otherwise, rendering is the same as for the 3D texture-based approach. In contrast to traditional 2D texture slicing, which holds three copies of a volume data set (one for each of the main axes), volume rendering of advection data maintains only a single copy of the property field. The stacking direction is changed on-the-fly during advection if the viewing angle becomes too large with respect to the stacking axis to avoid holes in the final display. Figure 3.10 illustrates a reordering of the stacking direction from y to x axis. A tile in the new stacking order is rendered in a stripe-by-stripe fashion, according to


111

(a)

IMAGES: WEISKOPF, ERLEBACHER, HOPF, ERTL [473] c 2002 Greiner, Niemann, Ertl, Girod, Seidel

(b)

(c) Fig. 3.9. Visualization of the interaction of a planar shock with a longitudinal vortex by 2D texture advection. The transport mechanism makes use of LEA. (a) shows a dense, noise-based visualization with a velocity mapping that modifies the grayscale image according to the magnitude of the velocity to enhance the important structures in the flow. (b) and (c) show the combination of noise (gray-scale) and dye (colored) advection – in (b) without velocity mapping and in (c) with velocity mapping. See also Color Plate 13

112

3 Vector Field Visualization y ordering

x ordering

physical memory

y logical memory

x IMAGE: WEISKOPF, SCHAFHITZEL, ERTL [481]


Fig. 3.10. Reordering of stack direction. A tile in the new stacking order is rendered in a stripe-by-stripe fashion, according to portions of tiles from the old stacking order

portions of tiles from the old stacking order. The reordering process takes into account that the number of tiles, their sizes, and their positions may change. Figure 3.11 shows results of advection with 3D textures. The underlying data set represents the behavior of a tornado (the data set courtesy of Roger Crawfis). The size of the flow field is 1283 , the particle textures are 2563 , and the noise injection texture is 1283 . Figures 3.11 (a)–(f) compare different visualization styles for the same viewpoint. Images (a)–(d) are rendered with velocity masking, i.e., only regions of high velocity magnitude are displayed. Image (a) employs a dense noise injection of two materials (bluish and green) and (b) a sparse noise injection. Due to the asymmetric, exponential filter kernel (based on iterative alpha blending), image (b) resembles oriented LIC [457]. In (c), additional red dye is injected at a user-specified position. The seed point is visualized by the intersection of three orangecolored, axis-oriented, thin tubes. In this way, the user can easily identify the spatial position of dye injection. In (d), different lengths for blue and green noise material demonstrate that different blending weights can be used for each material. Image (e) shows a dense visualization without masking. In this dense representation, interior parts of the flow are completely occluded by advected material in the front. Therefore, clipping approaches are required to view these inner regions, e.g., with a slanted clipping plane as in Fig. 3.11 (f). The 3D texture advection in Fig. 3.11 demonstrates that perception challenges have to be addressed in dense 3D representations. Perceptual issues are much more important for 3D flow visualization than for the 2D case because the fundamental problems of occlusion, clutter, and depth perception need to be addressed [178]. One basic approach is to reduce the visual complexity by removing unimportant elements of the 3D representation and, in this way, emphasizing the remaining, more important elements. As one important example of this approach, transfer functions can be used to selectively specify opacity. “Flow features” in their broadest sense are


(b)

(c)

(d)

(e)

(f)

IMAGES: WEISKOPF, ERTL [476]

c 2004 UNION Agency/WSCG

(a)

113

Fig. 3.11. Visualization of a tornado data set by 3D texture advection with velocity masking: (a) dense noise injection, (b) sparse noise injection, (c) additional red dye, (d) different lengths for blue (dark) and green (bright) material. (e) shows a dense visualization without masking; (f ) the same with a slanted clipping plane. See also Color Plate 14


IMAGE: WEISKOPF, SCHAFHITZEL, ERTL [481]


114

Fig. 3.12. Benard flow with λ2 vortex visualization. See also Color Plate 15

mapped to a single importance value that is used as measure for opacity. A “flow feature” can be as simple as the velocity magnitude in Figs. 3.11 (a)–(d). Other opacity measures could be based on traditional features such as vorticity or shock fronts (see Sect. 3.2.4). Figure 3.12 illustrates vortex detection by means of the λ2 criterion [187]. Vortex regions are rendered as opaque red or yellow material (see also Color Plate 15), whereas the surrounding flow regions are still visible as semi-transparent material with blue or green color. Features are, in general, derived from the vector field and possibly from additional attributes (such as temperature or pressure). The actual definition of a feature heavily depends on the application context and domain knowledge. Therefore, the expertise of a user could also be used to specify a feature, e.g., by means of user input via a feature-definition language [93]. The advection and rendering methods of this section are prepared to include this feature-based approach because they can immediately incorporate any of these features by adapting the transfer function. Volume clipping pushes the idea of selective rendering to its extremes – unimportant elements are completely removed (or rendered transparent). Therefore, clipping techniques from Sect. 2.5 can be adopted to filter the flow representation. For example, the clip geometry could be adapted to curved boundaries of the problem domain in flow simulations or the clip object could be aligned to the surface of a car body in automotive CFD applications to explore interesting features of a flow close to such boundaries. Therefore, volume clipping can play a crucial role in investigating dense 3D representations; see, e.g., the discussion by Rezk-Salama et al. [341]. When clipping is applied to an otherwise completely opaque volume visualization, the rendering is reduced to a flow visualization on curved surfaces defined by the clip geometry. Vector field visualization on curved surfaces is called a 2.5D approach because


115

it is a compromise between visualization on a 2D plane and in full 3D space. An efficient implementation of the 2.5D approach is discussed in Sect. 3.5. Finally, the perception of shape, orientation, and depth could be promoted by perception-guided rendering techniques. Some of these techniques are discussed in Chap. 4. A simple, yet very effective approach is to include volumetric illumination in the display of the property field. As discussed in Sect. 2.1, local illumination needs gradients, which serve as normal vectors. One issue is that the property field continuously changes due to advection. Therefore, gradients have to recomputed after each advection step. Weiskopf, Schafhitzel, and Ertl [481] use central differences to compute gradients after each advection and blending iteration and before volume rendering. Central differences provide an acceptable quality at moderate computational costs. In particular, they access the 3D uniform grid in a regular pattern and need only directly neighboring grid points along the three main axes. Therefore, gradient computation can be implemented as a fragment program that processes the texture representation of the property field. Once the gradient is available, any gradient-based volume shading method can be applied. Figure 3.13 compares a few shading techniques. The data set is the same tornado as in Fig. 3.11. Figure 3.13 (a) illustrates the emissionabsorption model, rendered with high opacities. Figure 3.13 (b) shows an example of volume illumination by the Phong model with diffuse and specular components, which are added on top of the emissive part present in Fig. 3.13 (a). Figure 3.13 (c) demonstrates cool/warm shading, which encodes the orientation of streamlines by warm or cool colors, respectively [137, 105]. An advantage of cool/warm shading is that orientation is represented by (almost) isoluminant colors. Therefore, brightness can still be used to visualize another attribute or property. In general, both cool/warm shading and Phong illumination provide a better impression of shape than the emissionabsorption model. More information on the effective use of color and illumination in animated visualizations is discussed in Sect. 4.2, which focuses on perception-oriented rendering. An example of a more advanced rendering approach is halo rendering, which is effective in visualizing relative depth between curves [178]: curves behind a closer streamline are partly hidden by dark halos. In Fig. 3.13 (d), halos are rendered as thick silhouette lines. Silhouettes are identified by examining the dot product of gradient and viewing directions because a silhouette is associated with a gradient perpendicular to the viewing vector. In addition, limb darkening through high opacities in the transfer function [164] also contributes to the halo effect. More details on volume illumination for GPU-based 3D texture advection can be found in a paper by Weiskopf, Schafhitzel, and Ertl [481]. The work by Interrante and Grosch [178] discusses perception-oriented rendering for 3D texture-based flow visualization in general.


(a)

(b)

(c)

(d)

c The Eurographics Association 2005 IMAGES: WEISKOPF, SCHAFHITZEL, ERTL [481]

116

Fig. 3.13. Volume rendering of 3D advection applied to a tornado data set: (a) emission-absorption model only, (b) with Phong illumination added, (c) cool/warm shading, and (d) halos. See also Color Plate 16

3.4 Dye Advection Based on Level-Sets Semi-Lagrangian dye advection is subject to numerical diffusion that may lead to an inappropriate blurring of the visualization, especially when short integration step sizes are employed. Figure 3.14 shows an example of numerical diffusion within a 2D circular flow. The underlying problem is the continuous resampling with a tensor product of linear filters (i.e., bilinear interpolation in 2D and trilinear interpolation in 3D). Figure 3.15 illustrates how linear interpolation leads to artificial diffusion in a simple 1D example. The top image shows the property values p for the time step t. The dye is represented

117

IMAGE: WEISKOPF [464]


3.4 Dye Advection Based on Level-Sets

Fig. 3.14. Numerical diffusion in a circular flow (see also Color Plate 17)

by large values p, the background by vanishing p. The bottom image shows the property values for the following time step (t + ∆t). A uniform vector field is applied to transport the dye to the right. In this example, the step size ∆t and the velocity v are chosen in a way that the dye is shifted by one half of the grid spacing ∆x, i.e., ∆tv = ∆x/2. Ideally, the profile for the values p should not be changed by such a translation. However, the access to positions in between grid points leads to a dilution of dye by linear interpolation. As a result, the width of the interface between dye and background is increased from one grid spacing to two grid spacings. Subsequent advection steps will continuously increase the size of this interface layer. This blurring effect is not related to physical diffusion that may occur in experimental flow visualization. In fact, the smearing is only caused by the advection algorithm and depends on many different parameters, such as step size, velocity, grid resolution, and the orientation with respect to the main axes of the texture. Furthermore, the blurring prevents users from clearly identifying streaklines or other isolated visual patterns. In summary, this artificial diffusion should be avoided for an effective vector field visualization. The use of a level-set approach was proposed by Weiskopf [464] to overcome the above mentioned problems and to provide a dye advection method that is able to accurately track the boundary between dye and background. The following discussion is based on the paper by Weiskopf [464].3 In general, level-set techniques were adapted to other computer graphics applications before, e.g., the interface between fluid and air can be modeled by level-sets in a water simulation [112, 124]. In addition, tracking of interfaces 3

c The Eurographics Association 2004. Reprinted, with permission, from Weiskopf [464]

118


p(t )

x p(t+∆t)

∆t v

∆x

x



Fig. 3.15. Numerical diffusion caused by linear interpolation. The top image shows the property values p for the time step t; the bottom image shows the property values for the following time step (t + ∆t)

plays an important role in computational fluid dynamics and computational physics. Here, only a brief list of related publications in this field can be given. One class of numerical methods makes use of a Lagrangian tracking of particles, as, e.g., described in [433]. Volume-of-fluid (VOF) techniques [169] model the fraction of a material in each computational cell to reconstruct the material interface; examples are the SLIC method (Simple Line Interface Calculation) [309], the PLIC technique (Piecewise Linear Interface Construction) [502], or a cubic-polynomial interpolation to represent the VOF fraction [499]. Another approach is the ghost fluid method [117], in which a level-set function is used to track the motion of a material interface. 3.4.1 Distance Field as Level-Set In the level-set approach of this section, the interface between dye and background is represented as an implicit surface within a signed distance function. A signed distance function φ(x) describes the distance of the point x to the surface. The interface is the implicit surface for φ(x) = 0. It is assumed that φ(x) is negative inside the dye region and positive on the outside. Dye is modeled as a massless marker material that is perfectly advected along the input vector field. For such a convection scenario, the evolution of the level-set is governed by ∂φ(x, t) + v(x, t) · ∇φ(x, t) = 0 , (3.5) ∂t


119

which was already used for the generic description of advection in an Eulerian context. The vector v can be identified with u in the original notation of above equation on p. 85. The same semi-Lagrangian approach is used as in Sect. 3.3, leading to the evolution equation φ(x(t0 ), t0 ) = φ(x(t0 − ∆t), t0 − ∆t) .

(3.13)

Besides the transport of dye, the injection of additional dye has to be modeled in this level-set approach. Newly injected dye is represented by a signed distance field φinject . After each integration step (3.13), the new dye and the “old”, transported dye φtransp are combined according to φinject (x) if φinject (x) < φtransp (x) φ(x) = . φtransp (x) otherwise In this way, the signed distance is modified by the injection field at positions that are closer to the new dye region, compared to the original distance from the interface. The distance field itself cannot be directly used for the final visualization. In fact, dye is drawn where the signed distance is negative and the background remains visible where the distance is positive. Figures 3.16 (c) and (d) demonstrate level-set dye advection. Dye is injected in the lower left part of the domain and transported by a uniform vector field that points toward the upper right of the domain. Image (c) visualizes the distance field: bright regions correspond to negative distances, whereas dark regions correspond to positive distances. This illustration uses a nonlinear gray-scale mapping to cover all relevant distance values. Figure 3.16 (d) shows the final visualization of the dye, which is displayed in yellow color. The traditional advection method is included in Figs. 3.16 (a) and (b) for comparison. Image (a) shows a gray-scale table applied to the property values, image (b) displays the same values by a color table that resembles the color coding from Fig. 3.16 (d). As discussed previously, traditional dye advection leads to a blurring and widening of the streakline. In contrast, the level-set approach in Fig. 3.16 (d) exhibits a sharp interface boundary. Unfortunately, the streakline shrinks and eventually disappears completely. The reason for this behavior can be seen in the underlying distance field (Fig. 3.16 (c)). The semi-Lagrangian level-set evolution builds upon a bilinear resampling in the distance field and therefore is affected by interpolation artifacts that are similar to those in traditional dye advection. Here, positive distance values surround the rather small dye region and dilute the negative distances inside the dye. Therefore, this simple distance-based scheme is not appropriate for dye advection. 3.4.2 Reinitialization of the Level-Set The underlying problem is that, as the level-set evolves, φ drifts away from its initialized value as signed distance. This issue is apparent in other level-

(a)

(c)

(e)

(b)

(d)

(f)



IMAGES: WEISKOPF [464]

120

Fig. 3.16. Comparison: (a, b) traditional dye advection; (c, d) the level-set approach without reinitialization; (e, f ) the level-set approach with reinitialization. See also Color Plate 18

set applications as well [317]. Therefore, in general, the level-set needs to be reinitialized periodically to a signed distance. Fortunately, a complete reconstruction of the signed distance field is not required for dye advection. In fact, only a local level-set around the interface has to be modeled as signed distance. Unlike many other applications of level-set techniques, a higher-order representation of the level-set function is not needed – it is sufficient to reinitialize only those grid points that are adjacent to the interface. In what follows, only a local “distance field” is used, where distance values are clamped to the range [−1, 1]. The spacing of the uniform grid is assumed to have unit length. Figure 3.17 illustrates the initialization method in 1D. First, the edges between grid points are classified as either intersection or non-intersection edges. Due to the linear interpolation between grid points, an intersection with the interface is uniquely identified by a sign change of φ between two adjacent grid points. Grid points that are not adjacent to the interface are reset to +1 or −1, depending on the sign of φ. Intersection edges, however, trigger a reinitialization to a signed distance. Here, the values at the two adjacent grid points are modified under the constraint that the position of the interface is not changed. Using indices i = 1, 2 for these two grid points, the reinitialized values are φri =

φi , ||φ1 || + ||φ2 ||

where the φi denote the values before reinitialization. For very small dye regions, the interface can cross two adjacent edges. Therefore, the reinitialization


121

φ 1

∆φ < 1

0 x

φ

Reinitialization

1

∆φ = 1

0 x



Fig. 3.17. Reinitialization of ∆φ back to unit distance along the intersection edge

method is extended to take into account the case that a grid point is attached to more than a single intersection edge. Here, the final distance is set to the average of the values from the reinitialization along the two edges. This averaging method directly leads to the extension of the reinitialization technique to more than one dimension. In nD, a grid point is attached to 2n edges. Once again, the edges are classified as intersection or non-intersection edges. For each intersection edge, the adjacent grid points receive a preliminary modified value φri that is calculated along this edge. The final distance value is computed by taking the average of all these contributions from adjacent edges. If a grid point is not located next to the interface, its value is reset to +1 or −1. In any dimension n, reinitialization only needs access to grid elements in the direct neighborhood, which leads to a fast computation of the updated level-set function. Figures 3.16 (e) and (f) show the result of this reinitialization process for the same test scenario as in Figs. 3.16 (a)–(d). Here, the reinitialization is applied after every tenth step of the level-set evolution. The reinitialization guarantees that the level-set is not diluted by surrounding positive distance values. Therefore, a constant width of the streakline is maintained. At the same time, a clearly defined interface between dye and background is achieved. Figure 3.18 demonstrates the effect of reinitialization in more detail. The left image represents the local distance field before reinitialization. The right picture shows the local distance field directly after the reinitialization, which




122

(a)

(b)



Fig. 3.18. Local distance field reinitialization in 2D: (a) shows the local distance field before reinitialization; (b) directly after reinitialization

Fig. 3.19. Level-set dye advection for a 3D vector field with a swirling feature (see also Color Plate 19)

clearly retains the position of the interface. The blurring has been removed and a thin boundary layer between dye and background has been restored. Figure 3.19 shows the visualization of a 3D vector field with a swirling convergent feature. The two images depict the same time step from different camera positions. Level-set dye advection was used with a reinitialization after each fifth integration step. The resolution of the domain is 1283 .


123

3.4.3 Discussion All texture-based advection techniques have the advantage that the regions covered by dye can change their topology while the dye evolves. A typical example is a streakline that is transported across the seed point of another streakline. Figure 3.20 illustrates this behavior as generated by the level-set approach with reinitialization. In Fig. 3.20 (a), the two streaklines are still separated. Figure 3.20 (b) shows the visualization just after the two streaklines have attached and Fig. 3.20 (c) depicts a later time step. Such a merger of different dye structures would require complex algorithms and data structures if a mesh-based representation with purely Lagrangian transport was used. Another advantage of texture-based advection is its built-in support for transporting extended regions of dye. As mentioned previously in Sect. 3.3, the visualization performance is independent of the number of cells covered by dye, and the user can control dye injection by virtually painting into the flow. The issue of numerical diffusion is also addressed by Jobard et al. [192] in the context of traditional texture-based dye advection. They use a method that re-adjusts the contrast of the dye density values after each advection step. To achieve an appropriate sharpening of dye streaks, the parameters of the contrast-enhancing mapping have to be chosen correctly. If the mapping function is too steep, contrast is increased too much and the visualization may appear “frozen”, similarly to the effect of nearest-neighbor sampling in LEA [192]. On the other hand, dye will smear out if the mapping function is too flat. In contrast, level-set reinitialization is independent of the rate of artificial diffusion and does not require a careful choice of parameters. Reinitialization results in a boundary layer that has a width of one or two texels everywhere, regardless of the size of the smeared-out zone before reinitialization (see Fig. 3.18). A disadvantage of level-set advection with reinitialization is the failure of transporting small structures that are in the range of only one cell. Figure 3.21 (a) shows a streakline that incorrectly ends after being “compressed” in a convergent region. For comparison, Fig. 3.21 (b) represents the correct streakline as generated at high resolution. In general, the level-set approach with reinitialization has a slight tendency toward shrinking the advected dye structures. 3.4.4 Extensions The visual representation of the distance fields from level-set dye advection can be improved by a number of extensions. The first extension addresses the final display of distance-based dye. Since dye regions are extracted with a sharp threshold at isovalue 0, the underlying grid structure will become apparent in the form of jaggies, as shown in Fig. 3.22 (a). (Note that the final display has a higher resolution than the computational grid; a slight blurring




124

(a)

(b)

(c)



Fig. 3.20. Change of topology of the dye-covered area. The images show snapshots from an animated visualization, chronologically ordered from left to right. See also Color Plate 20

(a)

(b)

Fig. 3.21. (a) Truncation of streaklines in low-resolution level-set advection. (b) represents the correct streakline generated at high resolution. See also Color Plate 25


(b)

(c)

(d)



(a)

125

Fig. 3.22. Extensions for level-set dye advection: (a) an interface with jaggy artifacts caused by a binary decision with respect to the isovalue φ = 0; (b) jaggy artifacts smeared out by image-space smoothing; (c) an additional background LIC image; (d) dye faded out. See also Color Plate 21

126


of the dye boundary is caused by bilinear interpolation for texture fetches.) In Fig. 3.22 (b), jaggy artifacts are reduced by image-space smoothing. For example, a Gaussian filter or a box filter can be employed for smoothing. This filtering by convolution can be efficiently handled by GPU operations working in image-space [290]. Like in previous approaches with traditional dye advection, the dye visualization can be combined with a dense, noise-based vector field visualization. As an example, dye is modulated by a background LIC image in Fig. 3.22 (c). As another extension, the “age” of dye can be transported along the flow to gradually fade out dye structures. Age is stored in another property texture and advected in a semi-Lagrangian manner. The measure for age has values between 1 (“just born”) and 0 (“infinitely old”). Newly injected dye carries an age measure of 1. In each advection step, the age of the existing dye elements is decreased according to a user-specified factor. The original color from the dye extraction process is modulated by this age measure to provide the final image, as shown in Fig. 3.22 (d). 3.4.5 Implementation and Results Both the 2D and 3D level-set dye advection techniques were implemented in a C++ program. The grids for the distance field are represented by arrays onto which the above level-set methods can be directly mapped. The images for Figs. 3.16, 3.18, and 3.19 were generated by this CPU-based implementation. To improve rendering performance and achieve a real-time visualization, 2D dye advection can be realized on a GPU. A prototypical GPU version based on DirectX 9.0 was developed and tested on a Windows XP PC with ATI Radeon 9800 XT (256 MB) GPU and Pentium 4 (3.2 GHz) CPU. The images for Figs. 3.14 and 3.20–3.22 were generated by this GPU-based 2D advection tool. In the following, details of this implementation are presented. The local distance field is represented by a 2D texture with a single 16bit fixed-point channel. The vector field is also stored in a 2D texture. The current implementation represents the vector field in two 16-bit fixed-point channels. A 32-bit floating-point texture could be used if a higher accuracy is required. The semi-Lagrangian evolution of the distance field is realized by a pixel shader program that computes the backward mapping according to (3.13). The corresponding fragments are generated by rendering a single viewport-filling quadrilateral. First, the pixel shader accesses the vector field to compute a short particle path according to (3.1). An explicit second-order midpoint scheme is used because the accuracy of first-order Euler integration is inappropriate for tracing long streaklines. Afterwards, the computed particle position is used to access the distance field from the previous time step. The required bilinear interpolation is directly supported by 16-bit fixed-point textures. Actually, two copies of the distance texture are held on the GPU – one for the previous time step and the other one for the field that has to


127

Table 3.4. Rendering performance for level-set 2D dye advection in frames per second (ATI Radeon 9800 XT, 256 MB) [464] Viewport No reinitialization Permanent reinitialization Reinitialization after 10 steps

800 × 600

1200 × 900

326.8 152.2 267.5

151.1 68.8 121.5

be updated in the current time step. After each iteration, both textures are swapped in a ping-pong scheme. Periodically, the level-set texture has to be reinitialized by another pixel shader program. This pixel shader needs the values from the center point and from the adjacent 2n grid points along the main axes in nD, i.e., five texels have to be loaded in the 2D case. The grid points are accessed by nearest-neighbor sampling in the level-set texture. The corresponding texture coordinates are generated by a vertex shader program. In this way, dependent texture lookups (texture indirections) are avoided. The reinitialization method computes preliminary reinitialization values for each edge. Since this calculation has the same structure for all edges, the four-vector capabilities of a GPU can be exploited to process all four edges simultaneously and thus greatly reduce the number of instructions. In total, the implementation needs 23 numerical instruction slots and five slots for texture loading. In the 3D case, the number of adjacent edges increases from four to six. Therefore, seven texture accesses would be needed. As a rough estimate, the number of numerical instructions would be increased by a factor of two. Both numbers are still well within the limits of GPUs like ATI Radeon 9800 or NVIDIA GeForce FX. Table 3.4 documents the rendering performance for 2D GPU-based dye advection on the aforementioned hardware configuration. The viewport sizes were 800×600 and 1200×900 pixels, respectively. The first line shows the performance of the semi-Lagrangian advection scheme. These numbers apply to the traditional property-based dye advection and the distance-based representation alike. The numbers in the second and third rows reflect the rendering speed for a combination of semi-Lagrangian advection and reinitialization. If reinitialization is applied after each single iteration step, the performance is roughly fifty percent of the semi-Lagrangian advection scheme. The third line represents a more realistic example in which the level-set is periodically reinitialized after every tenth iteration. Here, the performance is only 20 percent below pure semi-Lagrangian advection.

128


3.5 Flow Visualization on Curved Surfaces Dense vector field visualization gives good results for planar 2D domains, but has intrinsic problems of clutter and occlusion in 3D (see Sect. 3.3.5). Flow visualization on curved 2D hypersurfaces through the complete 3D data set is one method to avoid most of the occlusion problems while it still provides a more flexible representation than on planar slices. Interesting fluid behavior often occurs in the direct neighborhood of a curved boundary and can be displayed by this 2.5D approach. A typical example is the visualization of the air flow around an aircraft wing or an automobile (see Fig. 3.23). Texture-based visualization techniques for 2D planar flow can be extended to 2.5D by considering a C-space (computational space) approach. Here, a parameterization of the surface has to be known, and all computations take place in the Cartesian 2D coordinate system. However, many surfaces are not equipped with a parameterization and thus do not allow for a direct application of C-space methods. In this section, an approach guided by image-space computations is followed. This technique was proposed by Weiskopf and Ertl [477] and is inspired and influenced by the work by Laramee et al. [241] and Van Wijk [441], who apply texture advection and image-based flow visualization on the image plane. The following discussion is based on the paper by Weiskopf and Ertl [477]. The image-space approach has a number of important advantages: neither a parameterization nor connectivity information of a surface mesh are

IMAGE: WEISKOPF, ERTL [477]

Fig. 3.23. Flow visualization on a curved surface for an automotive CFD simulation (data set courtesy of the BMW Group)

3.5 Flow Visualization on Curved Surfaces

129

required; it is well supported by GPUs; it does not need sophisticated data structures and is easy to implement; the main part of the algorithm – advection and blending – is output-sensitive, i.e., the performance is determined by the viewport size. Unfortunately, the restriction to pure image-space texture advection also causes some disadvantages: frame-to-frame coherence cannot be guaranteed when the camera position is changed; silhouette lines are inflow regions on the image plane, and it is difficult to maintain constant contrast in these inflow areas; silhouette lines have to be identified by an image-space edge detector; only an exponential filter kernel is supported for LIC. The idea behind the technique of this section is to combine the image-space methods with some aspects of object-space methods. The following aspects are described in detail. First, a hybrid Lagrangian particle tracing in physical and device spaces that retains all aforementioned benefits of the image-space approach and avoids its disadvantages. Second, a solid texturing of the input 3D noise which fixes the noise structure in object space and thus guarantees temporal coherence under camera motion while, at the same time, a constant spatial noise frequency is maintained on the image plane. Third, an approach to effectively combine the visualization of surface shape and flow by using appropriate color schemes. Fourth, a mapping of the algorithm to GPUs, which leads to interactive rendering. 3.5.1 Lagrangian Particle Tracing on Surfaces In contrast to the methods from Sect. 3.3, a Lagrangian approach to particle tracing is adopted. The path of a single massless particle is determined by the ordinary differential equation dxpath (t; x0 , t0 ) = u(xpath (t; x0 , t0 ), t) , dt

(3.1)

where xpath describes the position of the particle at time t and u(x, t) denotes the time-dependent vector field. In the case of a flat 2D or 3D domain, the vectors are 2D or 3D, respectively. In this section, domains are assumed to be 2D curved manifolds embedded in flat 3D space. Curved nD manifolds are often described by an atlas, which is the collection of charts; a chart consists of a coordinate system that is a subset of Cartesian space Rn , and of a mapping from the manifold to the coordinate system. Finding an appropriate set of charts – a set of parameterizations – for an arbitrary 2D manifold is a formidable and expensive task that is part of ongoing research (see, for example, the papers [140, 247]). To avoid the construction of a parameterization, another representation for the embedded manifold is chosen: just the set of points x in R3 that belong to the surface. The normal component (along the normal direction of the hypersurface) of the 3D vector u has to vanish to build a tangential vector field. A tangential vector field can either come from a direct construction, such as a fluid flow

130


computation on a surface, or from the projection of a non-tangential 3D vector field. For a tangential vector field, (3.1) leads to curves that stay on the surface. So far, the vector and point quantities were given with respect to physical space (P-space). Along the rendering pipeline of computer graphics, however, a large number of additional coordinate systems is used. In GPU-based interactive rendering, a vertex of the input geometry is usually transformed from its original object coordinates into subsequent world, eye, and clip coordinates by respective affine or projective transformations (see Chap. 1). The transformation from clip space into normalized device coordinates makes use of a homogeneous division by the w clip coordinate. The view frustum is assumed to be [0, 1]3 in normalized device space (D-space). In this notation, P-space coordinates are identical to their object coordinates. For a stationary surface geometry, P-space and world coordinates are related by a constant affine transformation that is described by the model matrix. On the one hand, a P-space approach is well-suited for representing everything that should be fixed with respect to object coordinates: for example, the noise input for LIC-like convolution (more details on specifying such a noise texture are given in Sect. 3.5.2) should be modeled in P-space; in this way, a frame-to-frame coherent display of the input noise is automatically achieved even when the virtual camera is moving. On the other hand, D-space is the natural representation for everything that is directly related to, or governed by aspects of, the image plane. For example, a dense vector field visualization is ideally computed on a per-pixel basis with respect to the image plane in order to achieve an output-sensitive algorithm and a uniform density on the image plane. The basic idea of the algorithm is to combine the advantages of P-space and D-space representations by computing particle paths in both spaces simultaneously and tightly connecting these two points of view. Figure 3.24 illustrates the coupled P/D-space approach. The starting point is (3.1), which is solved by explicit numerical integration. A first-order explicit Euler scheme is employed in the current implementation, but other, higher-order schemes (e.g., Runge-Kutta) could be used as well. The numerical solver works with P-space coordinates xP ≡ x and the original tangential vectors uP ≡ u. After each integration step, the corresponding position in D-space is computed by applying the model, view, projection, and homogeneous division operations, TMVPH : (3.14) xD = TMVPH (xP ) . An important point is that the vector field is no longer given on a P-space but a D-space domain, i.e., different representations for the vector components and the associated point on the surface are used. This alternative description with −1 (xD ), t). The differential respect to D-space is given by vP (xD , t) = uP (TMVPH equation then becomes dxP (t) = vP (xD (t), t) , dt

(3.15)


131

zD

far clipping plane

zD

zP

fa pla r clip ne pin

g

view frustum

xP near clipping plane

D-space oriented

xD

x

D

ne pla ar c ne lipp

vie fru w stu m

ing

P-space oriented IMAGE: WEISKOPF, ERTL [477]

Fig. 3.24. Coupled D-space (left) and P-space (right) representations of the same scene. The y axes are left out to simplify the illustration

in combination with (3.14). The crucial step in making the integration process efficient is to reduce the 3D representation of the quantities that depend on xD to a 2D representation with respect to the xD and yD components of xD . Since the flow fields are assumed to live on opaque surfaces, only the closest surface layer is considered and the zD component of xD (i.e., the depth component) can be neglected. The goal of all LIC-oriented techniques is to generate a texture φ by starting particle traces at each texel and computing the convolution integral along these traces. This texture can be parameterized by 2D device coordinates because only the visible surface parts are of interest. Since each particle trace can be uniquely identified by its corresponding seed point at the starting time t0 , the P-space positions along a path can be labeled by the x and y device 0 0 ): xP (t − t0 ; x0D , yD ). Finally, the vector coordinates of the seed point (x0D , yD 0 0 field with respect to 2D device coordinates, vP (xD , yD , t), can be computed by projecting the surface geometry onto the image plane. In summary, all quantities that are needed to solve (3.15) can be stored in 2D textures with 0 ). respect to the texel positions (x0D , yD Figure 3.25 shows the complete pseudo code for texture-based Lagrangian particle tracing on surfaces. The algorithm is split in two major parts. In the first part, the 2D textures for xP and vP are initialized by rendering the mesh representation of the hypersurface. The closest depth layer is extracted by the z test. The P-space positions are set according to the surface’s object coordinates (i.e., the standard vertex coordinates); interpolation during scanline conversion provides the positions within the triangles. Similarly, the vector field texture is filled by vP , which either comes from slicing through a 3D vec-

132

3 Vector Field Visualization // Initialize the 2D texture for vP (xD ): while rendering the surface with attached vector field: make vectors tangential transform vectors to device coordinates 0 ) → write result into tex2D vP (x0D , yD // Set the 2D texture for the initial positions xP : 0 ) render surface and write xP into 2D texture tex2D xP (x0D , yD // Initialize the convolution texture: 0 )←0 tex2D φ(x0D , yD // Iterate explicit solver until maximum length is reached: for i=1 to imax 0 ): for each visible texel (x0D , yD // Transform from P-space to D-space 0 )) (xD , yD ) ← TMVPH (tex2D xP (x0D , yD // Single Euler integration step: 0 0 ) ← tex2D xP (x0D , yD ) + ∆t tex2D vP (xD , yD ) tex2D xP (x0D , yD 0 0 ), based on accumulate convolution in tex2D φ(xD , yD 0 )) P-space noise tex3D noise(tex2D xP (x0D , yD endfor endfor

Fig. 3.25. Pseudo code for the iterative computation of surface particle paths [477]

tor field texture or from vector data attached to the vertices of the surface. If the vector field is not tangential by construction, it is made tangential by removing the normal component, which is computed according to the normal vectors of the surface mesh. In the second part, (3.15) is solved by iterating Euler integration steps. This part works on the 2D sub-domain of D-space; it successively updates the coordinate texture xP along the particle traces, while simultaneously accumulating the contribution of the convolution integral in texture φ. A contribution from backward particle traces can be computed by additionally executing the complete algorithm with negated step size. The algorithm in Fig. 3.25 supports only steady vector fields. In the case of a time-dependent flow, the projection of the vector data has to be performed within the integration loop. Note that the details of the noise accumulation step are covered in the following section. By restricting the device coordinates to the image plane during the texture lookup in the 2D vector field, the behavior of the image-space techniques [241, 441] is imitated, i.e., the velocity values are evaluated only at the surface and not at arbitrary 3D points. On the other hand, the positions in P-space are not projected onto the hypersurface. This is one crucial difference to the image-space methods. This approach avoids the inflow boundaries that are generated by silhouette lines on the image plane and that cause some of the artifacts in [241, 441]. Moreover, the P-space representation provides a


133


Fig. 3.26. Demonstrating the treatment of silhouette lines and object boundaries for surface flow on a teapot

better numerical representation of positions in the vicinity of silhouette lines by taking into account three coordinates instead of only two. Finally, the full 3D representation of P-space points allows us to distinguish two objects that touch each other in image space, but are at different depths. Therefore, it is automatically guaranteed that flow structures do not extend across object boundaries. Figure 3.26 shows that the flow structure is correctly generated at silhouette lines and boundaries. Even discontinuities of the tangential vector field are well represented. Figure 3.27 illustrates a uniform vertical 3D flow projected onto two tori. The LIC approach (left image) normalizes the projected vector field to unit length and therefore produces a discontinuity of the flow on the top and bottom rings of the torus. Even these problematic parts can be treated by the P/D-space approach. The right image in Fig. 3.27 shows the same vector field without normalization to unit length. The streaks are shorter at the top of the torus due to the small magnitude of the tangential vector field. Electronic videos on the web page [467] illustrate the frame-to-frame coherence by means of animations.

134



Fig. 3.27. Vertical flow projected onto two tori: LIC visualization (top left) vs. representation of the magnitude of the vectors (bottom right)

3.5.2 Visual Mapping and Noise Injection So far, the focus has been on the computation of particle paths. But how can these paths serve as the basis for generating effective images? For dense flow visualization, the idea of LIC on 2D planar surfaces [43] is adopted to produce patterns of different colors or gray-scale values. Generalizing LIC to surfaces in 3D and taking into account time-dependent noise input, one obtains ∞ 0 ) φ(x0D , yD

0 k(t − t0 )I(xP (t − t0 ; x0D , yD ), t) dt ,

=

(3.16)

−∞ 0 0 ) describes the particle path starting at point (x0D , yD ) where xP (t − t0 ; x0D , yD at time t0 , k is the filter kernel, and I(xP , t) is the time-dependent 3D noise input. After discretizing the integral, one obtains 0 0 φ(x0D , yD )= k i I i (xiP (x0D , yD )) ∆t , (3.17) i

where the superscripts i indicate time dependency. Since procedural noise is not (yet) supported by GPUs, all textures are modeled as sampled textures. In a naive implementation, the spacetime noise I i with its four dimensions would require a large amount of texture memory. By adopting the time representation from IBFV [440], the temporal dimension is replaced by a random


135

temporal phase per texel. Furthermore, the memory for the remaining three spatial dimensions can be reduced by periodically repeating the 3D texture along the main axes (i.e., texture wrapping). From experience, noise texture sizes of 643 or 1283 are appropriate for typical applications. Repeating noise structures may become apparent only for planar slices – as long as the surface is at least slightly curved, texture wrapping leads to good results. However, one problem is introduced by using a solid noise texture: a constant spatial frequency in physical space leads to different frequencies after perspective projection to image space. The associated aliasing issues are overcome by appropriately scaling the noise texture in physical space and thus compensating the effects of perspective projection. Mao et al. [271] use a uniform scaling that is based on the distance of the surface from the camera, which is appropriate for generating a non-animated LIC image of an object with limited depth range. The following two features are added to allow for temporally coherent, animated visualizations of large objects. First, the scaling factor is computed for each particle seed point independently to ensure a constant image-space frequency for objects with large depth range. Second, the noise is only scaled with discretized scaling factors and snapped to fixed positions to achieve frame-to-frame coherence under camera motions. A model is assumed in which noise with all spatial frequencies I˜ is constructed by a sum of band-pass filtered noise: ˜ I(x) =

∞

Iband (2i x) ,

(3.18)

i=0

where Iband serves as role model for the noise and the multiplication by 2i results in an increase of spatial frequencies by a factor of 2i . When a band-pass ˜ low and high frequencies are removed and therefore the filter is applied to I, infinite sum is reduced to a finite sum of Iband (2i x) terms. As an approximation, a narrow band filter is assumed and only the linear superposition of two neighboring frequency bands is considered: Ifiltered (x) =

iscale +1

αi Iband (2i x) .

(3.19)

i=iscale

The two discrete scaling factors are determined by iscale , which is computed from the distance d between particle seed point and camera (by using log2 d). The weights αi provide a linear interpolation based on the fractional part of log2 d. This model can directly be realized by sampled textures in combination with a fragment program: a single copy of Iband is stored as 3D texture, the measure iscale is computed for each seed point, two texture lookups with scalings 2iscale and 2(iscale +1) are performed, and finally these values are linearly interpolated. Since the noise frequency spectrum is variable in P-space, the step size of Lagrangian integration should be as well. The step size is changed in a

136


similar way as the scaling of noise. In this case, however, the snapping to discrete scaling factors is not needed, but the distance d can be directly used as measure. 3.5.3 Effective Object and Flow Rendering For the final rendering, both the LIC-like texture that represents the flow, and the shape and orientation of the hypersurface should be taken into account. That is, two types of information need to be visualized at the same time: object shape and flow structure. Figure 3.28 (a) demonstrates that the geometry of the surface is extremely hard to recognize if only the flow texture is projected onto the image plane. For the display on a 2D image plane, visual cues are essential to allow the user to recognize shape and structure. Colors play a crucial role because shape recognition heavily depends on the color variations introduced by illumination. Figure 3.28 (b) shows a straightforward way of combining colors from the illumination with the structure from the flow texture: gray-scale values are computed according to the diffuse illumination of the gray object surface and then modulated (i.e., multiplied) with the gray-scale values from the LIC-like texture. This image gives a good impression of both flow and object shape. However, both aspects are coded only by luminance variations and, therefore, other color dimensions are not used. Based on the tristimulus theory, colors are given with respect to a three-dimensional space. According to perceptionoriented color systems, luminance, hue, and saturation can be distinguished as three dimensions. From this point of view, the approach of Fig. 3.28 (b) completely neglects hue and saturation. Therefore, an alternative method is proposed: the LIC-like texture is first mapped from gray-scale variations to either hue or saturation variations; afterwards, the transformed flow texture is modulated with the illuminated gray surface. Figure 3.28 (c) demonstrates a mapping to hue variations between blue and yellow. From experience, hue gradients are preferable because they result in a better visual contrast than saturation gradients. Building on the idea of using different color dimensions for encoding flow and shape structure, an alternative technique can be proposed. Here, the roles of hue and luminance are exchanged, i.e., the LIC-like texture is represented by luminance variations and the surface by hue variations. Cool/warm shading [137] is a well-established hue-based illumination model that leads to intuitive shape perception. In Fig. 3.28 (d), cool/warm shading with yellowish and bluish colors is modulated with the gray-scale LIC-like texture. The different approaches from Fig. 3.28 (b)–(d) have specific advantages and disadvantages. A pure gray-scale representation (b) makes possible to encode additional properties in the hue and saturation channels and thus is useful for multivariate or multi-field visualization. Furthermore, gray-scale images are, of course, a good basis for black-and-white printing. Since luminance variations are a strong visual cue for shape perception, the technique


(b)

(c)

(d)


(a)

137

Fig. 3.28. Different shading approaches for object and flow rendering: (a) grayscale flow visualization without additional surface shading; (b) gray-scale flow modulated with illuminated gray object; (c) yellow/blue flow visualization modulated with illuminated gray object; (d) gray-scale flow modulated with cool/warm shaded surface (data set courtesy of the BMW Group). See also Color Plate 22

from Fig. 3.28 (c) provides a good impression of the geometry. On the other hand, the flow texture is hard to see in dimly illuminated parts of the surface. Finally, cool/warm surface shading is excellent in showing the flow structure in all visible surface regions, but gives a weaker impression of the surface geometry. Section 4.2 discusses strongly related issues of motion, shape, and texture perception in more detail, focusing on animated visualizations.

138


3.5.4 Implementation The implementation is based on C++ and DirectX 9.0. GPU states and shader programs are configured within effect files. All shader programs are formulated with high-level shading language (HLSL). Since the descriptions of Lagrangian integration in Sect. 3.5.1 and noise representation in Sect. 3.5.2 are already based on textures, most parts of this visualization approach can be readily mapped to the functionality of a DirectX 9.0 compliant GPU. A comparable implementation should be feasible with OpenGL and its vertex and fragment program support. An advantage of the proposed approach is that most operations take place on a texel-by-texel level. This essentially reduces the role of the surrounding C++ program to allocating memory for the required textures and executing the pixel shader programs by drawing a single domain-filling quadrilateral. The early z test skips the pixel shader programs for the pixels that are not covered by the projection of the surface onto the image plane. Textures are updated by using the render-to-texture functionality of DirectX. As an example, the HLSL code for a single Lagrangian particle integration step is given in Fig. 3.29, which demonstrates that (3.14) and (3.15) can be readily transferred into a corresponding pixel shader program. Similarly, the other elements of the algorithm can be mapped to shader programs in a direct way. Some of the textures hold data that is replaced by new values during each iteration step. For example, the P-space coordinates are updated, based on the current position. Ping-pong rendering is applied for this texture. To limit the required texture memory and make full use of the internal memory bandwidth of the GPU, the depth of the color channels in the textures should be reduced to the smallest possible level. From experience, the following choices are appropriate: 32-bit floating-point resolution for P-space positions, 16-bit fixed-point numbers for the accumulated noise values, 16-bit fixed-point or floating-point numbers for the flow data, and 8-bit fixed-point numbers for the input noise and random phases. Performance measurements are shown in Table 3.5. All tests were performed on an ATI Radeon 9800 Pro GPU with 256 MB. Filter length describes the number of integration steps to compute the LIC. The teapot scene from Table 3.5. Performance measurements for surface flow visualization in frames per second (ATI Radeon 9800 Pro, 256 MB) [477] Domain size Filter length Teapot Automobile

6002 25

70

9502 25 70

26.2 10.1 13.1 5.1 12.6 5.7 8.0 3.4

3.5 Flow Visualization on Curved Surfaces // Parameters: float4x4 matMVP; float4 stepSize;

139

// Model-view-projection matrix // Integration step size

// Tex coords for viewport-filling quad struct VS Output { float2 TexCoord: TEXCOORD0; }; // Homogeneous P-space position struct PS Output { float4 RGBA : COLOR0; }; // High-level pixel shader program PS Output IntegratePS (VS Output In) { PS Output Output; // Lookup current P-space position in texture PosTex float4 posP = tex2D(PosTex, In.TexCoord); // One integration step, split into three parts: // (1) Transform from P-space into D-space // Applies model-view-projection matrix: float4 posD = mul(posP, matMVP); posD = posD / posD.w; // Homogeneous division posD.x = .5 * posD.x + .5; // Maps x from [-1,1] to [0,1] posD.y = -.5 * posD.y + .5; // Maps y from [1,-1] to [0,1] // (2) Get P-space velocity from the 2D texture float3 velocity = tex2D(FlowField2dTex, (float2) posD); // (3) Compute new P-space position with Euler integration float3 posNew = posP + velocity * stepSize; // Output new position in homogeneous P-space coordinates Output.RGBA = float4 (posNew, 1.0); return Output; } Fig. 3.29. HLSL pixel shader code for one integration step in hybrid P/D-space particle tracing [477]

Fig. 3.26 is an example for a surface with a low polygon count, while the automobile scene from Figs. 3.23 and 3.28 with its 1, 143, 796 triangles and 597, 069 vertices represents a realistic industry data set. Please note that the performance numbers for the two scenes should not directly be compared with each other because the two scenes cover different ratios of the domain; the car almost completely fills the viewport, while the teapot covers a smaller percentage. However, the numbers for each scene show that the complexity of the surface geometry plays only a small role and that the performance depends on domain size and filter length in a nearly linear fashion. Therefore, the implementation allows the user to balance speed against visualization quality by gradually changing the filter length and/or the viewport size.

140


This implementation works on Shader Model 2 GPUs that do not need to support long fragment programs or fragment-shader loops. However, on GPUs that allow for longer fragment programs, the iteration along particle traces (see Fig. 3.25, the loop over integration and accumulation steps) can be incorporated in a single fragment program. Here, the P-space position and accumulated noise value are represented by fragment-program variables instead of textures. In this way, the multiple rendering passes with the pingpong scheme can be reduced to a single rendering pass, which decreases the amount of data that has to be written to render targets and read from textures.

3.6 Spacetime Framework for Time-Dependent Vector Fields This section specifically addresses the visualization of time-dependent vector fields by means of dense representations. A crucial observation is that two types of coherence are essential for understanding animated visualizations. First, spatial coherence within a single picture reveals image patterns indicating some vector field structure. Second, frame-to-frame coherence allows the user to identify a consistent motion of these patterns. To achieve separate control over both properties, a generic and flexible framework was proposed by Weiskopf, Erlebacher, and Ertl [472], along with further background information in a book chapter by Erlebacher, Jobard, and Weiskopf [115]. The following description is based on the paper by Weiskopf, Erlebacher, and Ertl [472].4 This framework provides spacetime-coherent dense representations of timedependent vector fields by a two-step process. First, continuous trajectories are constructed in spacetime to guarantee temporal coherence. Second, convolution along another set of paths through the above spacetime results in spatially correlated patterns. The framework can be mapped to a discrete texture-based representation in combination with a streaming architecture and therefore makes feasible an efficient implementation on graphics hardware. The actual visualization result depends on the choice of parameters, leading to a wide range of possible applications. For example, the framework is capable of mimicking known visualization techniques, such as LIC for steady vector fields, Unsteady Flow LIC (UFLIC) [392], LEA [192], IBFV [440], and Dynamic LIC (DLIC) [405]. Although the same qualitative visualization results are achieved, the underlying computational models may differ significantly, causing differences with respect to image quality, performance, and flexibility. Since the framework makes use of two different sets of paths for temporal and spatial coherence, even more advanced visualization techniques 4

c 2003 IEEE. Reprinted, with permission, from Weiskopf, Erlebacher, and Ertl [472]

3.6 Spacetime Framework for Time-Dependent Vector Fields

141

have become possible. For example, Unsteady Flow Advection–Convolution (UFAC) is introduced as a novel approach for unsteady fluid flows that performs time evolution governed by pathlines, but builds spatial correlation according to instantaneous streamlines of which the spatial extent is related to the degree of unsteadiness. 3.6.1 Continuous Framework The discussion begins with a generic framework that describes spatial and temporal correlation for vector field visualization on a continuous level. In what follows, a 2D spatial domain is assumed, but the framework itself can be directly extended to flat 3D domains or curved hypersurfaces through 3D space. In a flat, Euclidean space, a time-dependent vector field is a map u(x, t) that assigns a vector to each point x in space at time t. Pathlines xpath are the integral curves of a vector field, governed by the ordinary differential equation dxpath (t; x0 , t0 ) = u(xpath (t; x0 , t0 ), t) , dt

(3.1)

with the initial condition xpath (t0 ; x0 , t0 ) = x0 . Particle motion can be investigated in spacetime, i.e., in a manifold with one temporal and two spatial dimensions. It is assumed that changes of reference frames are governed by the Galilei group of Newtonian physics. The world line (i.e., the spacetime curve) traced out by a particle can be written as Y(t; x0 , t0 ) = (xpath (t; x0 , t0 ), t). In general, curves in spacetime are denoted by scripted variables and have three components: two spatial and one temporal. Y(t; x0 , t0 ) is parameterized by its first argument and passes through the point (x0 , t0 ) when t = t0 . The term trajectory is used for the spacetime description of a pathline. Figure 3.30 shows a spacetime diagram with a few trajectories. A dense representation of a vector field employs a large number of particles so that the intersection between each spatial slice and the trajectories yields a dense coverage by points. Accordingly, spacetime itself is densely filled by trajectories. Properties can be attached to particles to distinguish them from one another. In what follows, a property is assumed to be a gray-scale value from the range [0, 1]. Properties are allowed to change continuously along the trajectory; very often, however, they remain constant. From a collection of trajectories, along with the corresponding particle properties, a function φ(x, t) can be defined by setting its value to the property of the particle crossing through the spacetime point (x, t). The function value will be zero if no trajectory touches the point. The function φ(x, t) filling spacetime is an important element of the framework. The continuous behavior of trajectories and their attached properties ensures that spatial slices through the property field φ(x, t) at nearby times are strongly related – and therefore temporal coherence is achieved. An animated sequence built from spatial slices with uniformly increasing time yields the motion of particles governed by the vector field.

142

3 Vector Field Visualization t

trajectories

convolution

x

Fig. 3.30. Spacetime diagram with a single spatial and the temporal axis. A sample collection of trajectories is drawn as black lines, a typical convolution path is shown as light-gray box

Since, in general, different particles are not correlated, spatial slices of the property field do not exhibit any coherent spatial structures. To achieve spatial correlation, a filtered spatial slice Dt (x) is defined through the convolution ∞ Dt (x) = k(s)φ(Z(t + s; x, t)) ds (3.20) −∞

along a path Z(s; x, t) through spacetime. The subscript on Dt is a reminder that the filtered image depends on time. For generality, the convolution is performed along the entire time axis; the domain of integration can be restricted by choosing a filter kernel k(s) with compact support. The kernel need not be the same for all points on the filtered slice and may depend on additional parameters, such as data derived from a vector field. For simplicity of notation, these parameters are not explicitly marked in the mathematical expressions. An example of a convolution path is illustrated in the spacetime diagram of Fig. 3.30. Z(s; x, t) can be any path through spacetime and need not be the trajectory of a particle. However, the spatial components of the path are given by the pathlines of another vector field w(x, t). The temporal component of Z(s; x, t) may depend on s and t. In the visualization approaches of this book, either Z(s; x, t) = ( · , t) or Z(s; x, t) = ( · , s) are used. An animated visualization produces filtered images Dt for uniformly increasing times t. Two issues have to be considered for a useful animation sequence. First, each single filtered image should introduce some intuitively understandable visual patterns. Therefore, the property field φ and the family of convolution paths Z should not be chosen arbitrarily, but should be constructed to lead to these patterns. Second, the spatial structures should be coherent from frame to frame. It is important to note that the coherence of φ along the time axis does not automatically imply a temporal coherence of the filtered images Dt . Thus, further restrictions are imposed on the choice of φ and Z.


143

In summary, the structure of Dt is defined by the triplet [φ, Z, k], where φ is built from trajectories Y; some useful combinations for this triplet are discussed in Sect. 3.6.3. The main advantage of this generic framework is its direct, explicit, and separate control over the temporal evolution along pathlines of u(x, t) and over the spatial structures that result from convolution along paths based on w(x, t). 3.6.2 Texture-Based Discretization The continuous framework must be discretized to compute images. A texturebased approach is used for storing and processing the dense representation, the vector field itself, and additional auxiliary data. Since the algorithms only utilize textures and no other complex data structures, an efficient, purely GPU-based implementation is possible. The rough outline of the texturebased discretization is as follows. First, spatial slices of the property field φ are constructed from trajectories. For each texel, trajectories backward in time are iteratively computed in a Lagrangian manner; property contributions from particles starting at previous times and propagating to the current texel position of φ are collected along the trajectory. By combining spatial slices, the complete property field defined on a spacetime domain is built. Second, a convolution is computed along Z within the resulting 3D property field. A Lagrangian integration of the corresponding paths is iteratively calculated to obtain contributions for the respective texel. A more detailed description of the discretization follows, starting with the first major component – the construction of the property field. Still in a continuous notation, a finite lifetime τ is assigned to each particle, and therefore each trajectory Z has a maximum temporal length. A particle is born at (x0 , t0 ) and dies at time t = t0 + τ . Newly introduced particles are described by a field P (x, t) defined on the spacetime domain. The field yields a continuous description – particles are no longer considered as point objects, but rather are continuously spread over space, i.e., the particle field describes a particle density. The invariant property of a particle born at (x0 , t0 ) is described by P (x0 , t0 ). The actual property of a particle may change during its lifetime, governed by a weight function w(tage ), where tage ∈ [0, τ ] is the age of the particle. For example, w = 1 describes a constant property along the entire trajectory, whereas some other choice for the weight may be used for a gradual phase-in and phase-out of a particle’s brightness. The weight might also depend on outside parameters, such as information derived from the data set or the particle injection field P . These parameters are not explicitly noted. The contribution to the property field φ at time t by a particle born at position x0 and at an earlier time t0 (within the lifetime) is φ(Y(t; x0 , t0 )) = w(t − t0 )P (x0 , t0 ) . Due to the symmetry of Newtonian mechanics with respect to time reversal, the trajectory can equally well be computed backward in time, starting from

144


t. By adding contributions from all possible particle paths originating from different discrete times tl between (t − τ ) and t, one obtains the property field φ(x, t) = w(t − tl )P (Y(tl ; x, t)) . tl

The fields φ and P are discretized by a uniform sampling in space and time. Stated differently, φ and P can be represented by stacks of equidistant 2D textures. Textures can be regarded as uniform grids, and texels as cells of these grids. Accordingly, the discrete property field is φi (xjk ) =

i−∆i τ

w(ti − tl )Pl (xpath (tl ; xjk , ti )) .

l=i

Here, φi and Pi denote spatial slices of φ or P at time ti , respectively; ∆iτ is the maximum number of discrete time intervals per lifespan; xjk denotes the spatial position of a grid point. Being a texture, Pi can represent any distribution of released particles – ranging from white noise to frequency-filtered noise or some input with large-scale structures that is used for dye injection. Figure 3.31 illustrates how particles released at previous times contribute to the property at time ti . The property slice φi is only sampled at discrete grid points xjk , i.e., point sampling is assumed. Without changing the basic procedure, more complex filtering, for example by means of subsampling, could be applied. Due to convergence or divergence of the vector field, trajectories starting at a given spatial distance from each other might sample the injection textures Pi at varying distances (i.e., at different spatial frequencies). However, the limited lifetime of particles ensures that these differences do not grow without bound (provided that the vector field is smooth) and therefore subsampling and filtering is usually not required. Although the access to φi is restricted to grid points xjk , the lookup in Pl is not, and some kind of interpolation scheme has to be employed. In the current implementation, a tensor product of linear interpolations is used, i.e., a bilinear interpolation on 2D spatial slices. Alternatively, simple nearest-neighbor sampling or more sophisticated higher-order interpolation schemes are possible. In all discussions of this section, the existence of some interpolation scheme is assumed to access textures at arbitrary positions. Finally, the discretized pathline xpath (tl ; xjk , ti ) is computed from the vector field, which is stored either as a stack of 2D textures for different times or as a 3D texture for a spacetime domain. Furthermore, it is assumed that a function (3.21) ∆x(x, ti ) = x − xpath (ti − ∆t; x, ti ) yields the motion of a particle for a small time step ∆t. In the current implementation, first-order explicit Euler integration approximates ∆x(x, ti ) by ∆tu(x, ti ). Higher-order schemes could be applied if higher accuracy were needed.

3.6 Spacetime Framework for Time-Dependent Vector Fields t

145

texels ti

t i−1 t i−2

lifetime

trajectory

t i−3 particles released x IMAGE: WEISKOPF, ERLEBACHER, ERTL [472]

c 2003 IEEE

Fig. 3.31. Computing two example texels of φi at time ti (dashed horizontal slice) by adding all possible contributions from particles starting (marked by black dots) at previous times

Combining the aforementioned building blocks, the texture-based computation of a single spatial slice φi can be accomplished by iterating ∆iτ times the recurrence relations ˜ l+1 (xjk ) − ∆x(X ˜ l+1 (xjk ), tl+1 ) ˜ l (xjk ) = X X ˜ l (xjk )) , φ˜l (xjk ) = φ˜l+1 (xjk ) + w(ti − tl )Pl (X

(3.22) (3.23)

with l = i − 1, . . . , i − ∆iτ . The initial values are ˜ i (xjk ) = xjk X

and φ˜i (xjk ) = w(0)Pi (xjk ) .

(3.24)

The final result is φi (xjk ) = φ˜l−∆iτ (xjk ) . The 2D texture φ˜l holds the intermediate results for the accumulation of ˜ l is a two-component 2D texture that stores the iteratively property values; X updated spatial positions along pathlines. Figure 3.31 illustrates this process in a spacetime diagram. The iteration starts at the current time ti and scans starting points of trajectories at previous time steps, up to the maximum ˜ l and lifetime (which is three time steps in this example). The lookups in X ˜ φl are restricted to fixed grid points xjk and do not need an interpolation scheme. Since the above iteration equations and ∆x(x, t) only need access to various textures and a fixed set of numerical operations, a direct mapping to fragment processing on graphics hardware is feasible. Specific GPU-related implementation issues are discussed in Sect. 3.6.4. So far, a uniform lifetime τ was assumed. However, the lifetime of particles can be individually adapted by changing the support of the weight function w(tage ) – as long as some maximum age τmax is not exceeded. Since a large and possibly varying number of particles contributes to each texel of φi , the

146


sum of the weights w should be normalized to one for each cell of φi . Therefore, the weights are accumulated along with the accumulated properties φ˜l , and the final property value φi is the accumulated property divided by the accumulated weight. One issue concerns inflow regions, i.e., parts of the domain in which the backward integration of trajectories leaves the boundaries of the textures Pi . The solution is to enlarge the input textures so that the pathlines do not exceed these boundaries. The texture size depends both on the maximum magnitude of the vector field and on the lifetime of the particles. Typically, the particle injection textures P are enlarged only virtually by applying texture wrapping, provided that these textures are seamless, e.g., white noise. Similarly, the vector field textures can be virtually enlarged by clamping texture coordinates to the texture boundaries. Another issue is a possible change of spatial frequency of the input P along trajectories through regions of divergence or convergence. However, due to the limited lengths of trajectories, these effects do not continue over large distances and are usually not noticeable even in small regions. Finally, the accumulation of random particles released from various previous times in a noise-based approach (i.e., uncorrelated injection with respect to both space and time) essentially represents a summation of independent random variables, leading to a normal distribution dictated by the central limit theorem. The standard deviation depends on the number of elements added, i.e., on ∆iτ . For a long lifetime of particles, φi becomes a gray image with most intensities in a small interval around 0.5. Good contrast can be reestablished by histogram equalization [313]. It is sufficient to compute the histogram once because the distribution of property values changes only slightly over time. The second major component of the framework concerns the convolution along spacetime paths Z. The basic idea is to discretize the convolution integral (3.20) by a Riemann sum. Fortunately, the texture-based implementation of this part is strongly related to the above construction of φ. In what follows, analogous elements in the two implementations are briefly described and differences are discussed. Since the spatial components of both Y and Z are based on pathlines of some vector field, the integration process acting on co˜ i is the same. The filter kernel k(s) replaces the weight ordinate textures X w(tage ), including the accumulation of kernel contributions and the final normalization to unity. Moreover, the filter kernel can depend on the position in spacetime, additional outside parameters, or values derived from the given data set. The input to the convolution process is the stack of properties φi , which acts as a substitute for the description of particle injections Pi . The maximum lifetime of particles is analogous to the maximum support of the filter kernel. The following differences exist between constructing φ and computing the convolution. First, the vector fields for computing pathlines can be different. Second, the temporal component of Z(s; x, t) is not necessarily s. Therefore, a mechanism has to calculate the appropriate time ti for each iteration step


147

and address the corresponding slice φi . In all visualization approaches investigated so far, the temporal component of Z(s; x, t) is either s or t, i.e., the iteration either processes the stack of textures φi concurrently to the integration time or uses a single property texture at a fixed time. Third, a provision must be made for integrating the convolution forward and backward in time, which is implemented by performing both parts independently and combining their results. Fourth, the treatment of boundaries is different. The convolution integral is limited to the boundaries of the domain; once the integrated coordinates leave the domain, the kernel is set to zero. Due to the normalization of accumulated kernel weights, the property values exhibit no intensity loss by a decreased filter support near boundaries. Fifth, both histogram equalization, which is also used for reestablishing high contrast in the construction of φ, and edge enhancement by high-pass filtering (such as a Laplacian filter) are used to improve image quality [313]. The main features of the texture-based implementation of the continuous framework are the following. First, an explicit and separate control over the temporal evolution and the structure of spatial patterns is possible. Second, any input texture Pi can be used; one is not restricted to white noise or to dye textures that exhibit only large-scale spatial structures. Third, a complete Lagrangian integration scheme is employed for particle transport in each time step of an animation, which avoids the accumulation of sampling artifacts or artificial smoothing, prevalent in Eulerian or semi-Eulerian approaches (see the discussion in Sect. 3.4). Fourth, there is no restriction on the possible filter kernel k(s) or property weight w(tage ) – except for a compact support of these functions. They can even depend on parameters derived from data sets or current spacetime locations. This flexibility allows for custom kernel design for further quality enhancement. A similar argument holds for the temporal behavior of particle transport along Y; for example, popping artifacts can be avoided by a slow phase-in and phase-out of particles through a timedependent property weight. 3.6.3 Visualization Approaches Within the structure of the above framework and its texture-based discretization, several visualization approaches are possible. Here, some well-known visualization techniques for time-dependent data sets are revisited. The basic idea is that by choosing the parameters appropriately, visualization results that are qualitatively similar to existing techniques can be reproduced, even if the underlying computational models differs significantly. The discussion focuses on differences with respect to image quality, performance, and flexibility. Furthermore, a new approach is described: the Unsteady Flow Advection– Convolution, derived from the framework of this section.

148


Lagrangian-Eulerian Advection LEA [192] is realized within the framework by setting u(x, t) to the given vector field v(x, t), and w(x, t) = 0. LEA does not change the properties of particles along their paths (i.e., w(tage ) = 1). Particle transport is computed according to the description in Sect. 3.6.2. To emulate the exponential filter of the original version of LEA, successive alpha blending of the results of particle transport can be applied. For any other kernel, the implementation of Sect. 3.6.2 is employed – except for the superfluous update of positions ˜ l (xjk ). The advantage of the exponential filter is that only one blending X operation is required for each final image Di , whereas other kernels require an iterative computation of the entire convolution for each time step. However, some of these successive filter calculations could be accelerated by exploiting coherence analogously to FLIC methods [396]. Particle transport in the original LEA utilizes a hybrid LagrangianEulerian texture update. Unfortunately, this computation scheme does not always reproduce trajectories faithfully and white noise input becomes gradually duplicated over several texels. Therefore, larger homogeneous patterns, even perpendicular to pathlines, emerge after some time of particle motion. The degradation of noise frequency is overcome by an ad-hoc injection of noise during each time step. Furthermore, many particles are completely discarded from one time step to the following because a texel is only filled with information from a single texel of the previous time step (by nearest-neighbor sampling). As a consequence, the original LEA exhibits only suboptimal temporal correlation, which is apparent in the form of noisy and rather short spatial patterns after convolution. For the implementation of LEA within the framework, these issues do not arise and therefore the final visualization shows an improved temporal and spatial coherence. As another advantage, the framework supports any type of input texture Pi , and is not restricted to white noise as in the original LEA. Thus, high spatial frequencies can be removed in a prefiltered input to avoid aliasing artifacts after convolution. The disadvantage of the alternative implementation is its higher computational cost for processing ∆iτ input images per time step. However, for short lifetimes of particles (e.g., some 15 iterations), the alternative LEA implementation executes at interactive frame rates on current GPUs and yet shows a higher temporal correlation than the original implementation. Moreover, the framework can be easily adjusted to targeted frame rates by adapting ∆iτ ; for example, only a few iteration steps could be processed for particle transport during user interaction, while a high-quality update of a snapshot could be computed with an increased number of iterations. Image-Based Flow Visualization In the context of the framework, IBFV [440] is dual to LEA: now w(x, t) is identified with the given vector field v(x, t), and u(x, t) = 0. From a physical


149

point of view, IBFV shows streaklines of particles injected at various points according to the structure of the input Pi . The implementation of IBFV within the framework computes the convolution according to the description in Sect. 3.6.2. The construction of the property field φ is trivial: φi is set to the particle injection at the same time, Pi . The only essential difference between the original IBFV implementation and the alternative framework-based implementation is the convolution process. The original approach permanently and implicitly updates φ˜l , i.e., particles are resampled on a texture after each integration step. This resampling is based on bilinear interpolation and thus results in artificial smoothing (see Sect. 3.4). In contrast, the alternative IBFV implementation never resamples particles for intermediate steps, but rather employs a Lagrangian integration of the complete pathline and samples only the original particles released on Pi . Therefore, no numerical diffusion is introduced. The alternative implementation has the downside of higher computational costs because complete pathlines have to be calculated for each time step. A combination of both implementations is most useful: interactive visualization could be based on the faster original implementation, whereas high-quality, high-contrast snapshots with very long streaklines are produced by the alternative approach. At no additional cost, filters different from an exponential kernel (which is intrinsically built into the original IBFV) can be applied for these snapshots to further improve image quality or achieve different visualization styles. Unsteady Flow LIC UFLIC [392] visualizes unsteady flow fields by advecting a dense set of particles forward in time – particles are released in a noise-based fashion and then distributed over space and time. The original UFLIC implementation attaches a time stamp to each particle to feed the points of a trajectory into spacetime buckets. Figure 3.32 illustrates for a single trajectory how points are distributed into buckets of temporal size ∆T . Using forward integration and scattering, several buckets next to each other in a single spacetime slab (a subset of spacetime covering the entire spatial domain and having a temporal thickness of ∆T ) can be filled by spacetime curve drawing. At each time step Tj (i.e., an integer multiple of ∆T ), a large number of new particles is released and traced up to the maximum age τ to obtain a dense representation. The final image for one spacetime slab is constructed from the contributions of all particles crossing this slab. Actual visualization results are displayed at time intervals ∆T . Unlike the original UFLIC implementation, the framework of this section is completely texture-based and cannot make use of scattering or curve-based modeling. However, the same qualitative visualization approach is feasible by subsampling spacetime slabs at intervals ∆t that are shorter than ∆T , see Fig. 3.32. In the formalism of the framework, u(x, t) is set to the given vector

150

3 Vector Field Visualization t

∆t

spacetime slab

∆T x spacetime bucket IMAGE: WEISKOPF, ERLEBACHER, ERTL [472]

c 2003 IEEE

Fig. 3.32. Spacetime sampling of trajectories for UFLIC. Using forward integration and scattering, buckets in a spacetime slab are filled along the spacetime curve

field v(x, t), and w(x, t) = 0. Convolution is performed along straight lines parallel to the time axis, and the support of the filter is exactly the thickness of a spacetime slab, ∆T . Particle injection textures Pj are only needed at the coarser steps Tj . The original UFLIC approach reuses filtered images as input for releasing new particles to achieve highly correlated spatial structures. This feed-forward technique can be directly mapped to the framework by using Di as a particle injection texture Pl for a later time – at the expense of losing a clear separation between the mechanisms of spatial and temporal correlations. Although the framework is capable of reproducing the same visualization patterns as UFLIC, a purely texture-based approach does not seem to be appropriate because of inherently higher computational costs, since, in particular, no quality improvements can be achieved compared to the original implementation. Dynamic LIC DLIC [405] targets the animation of instantaneous streamlines (or fieldlines) of a time-dependent vector field, e.g., an electric field. More specifically, two vector fields are given: w(x, t) governs the streamline generation at each time step, while u(x, t) describes the evolution of streamlines. Therefore, w(x, t) and u(x, t) cannot be chosen independently. Using these variable names for the vector fields, DLIC maps one-to-one to the framework. Unlike the aforementioned techniques, DLIC sets the temporal component of the convolution lines to the time corresponding to the filtered image, i.e., Z(s; x, t ) = ( · , t ). In this way, convolution is computed on a single spatial slice through φ. In the original CPU-based implementation of DLIC, a large number of individual particles, representing discs of finite radius, is tracked in a Lagrangian approach. The particles are allowed to overlap. An almost uniform particle density is achieved by removing or adding particles on a per-cell basis after each integration step. A noise texture is produced by sampling the particles


151

on a uniform grid. Finally, streamlines are generated by FLIC [396]. The main advantage of the implementation within the framework is GPU support and thus an interactive visualization speed. (Sundquist [405] does not report performance measurements, but it can be assumed that the speed is well below interactive frame rates for typical data set sizes.) Another advantage is related to the LIC process. Sundquist takes special care in choosing seed points for FLIC to avoid temporal sampling artifacts. Unlike FLIC, the convolution in the framework employs an independent sampling at every texel and integration step, and therefore is not subject to these artifacts. Both implementations may exhibit problems in regions of strong convergence or divergence. The original DLIC algorithm loses contrast for contracting fields, whereas the spatial frequency of noise might be changed in the current implementation (see Sect. 3.6.2). Unsteady Flow Advection–Convolution Unsteady Flow Advection–Convolution (UFAC) is a novel approach for displaying unsteady fluid flows. The development of this method has been guided by the following observations. First, streaklines of an unsteady flow (or similarly, pathlines) may cross each other and, consequently, are an inappropriate basis for a dense representation. Second, streamlines never intersect (except at critical points). Third, a coherent temporal evolution of image patterns can be recognized by the user as a consistent motion, and can be used as an additional means of conveying paths. Fortunately, the spatial part of these paths may intersect at different times without destroying the impression of a consistent motion. UFAC aims to produce an animated sequence of streamline images, where the evolution of streamlines is governed by the motion of particles along pathlines. In this way, each image would contain only non-intersecting line structures, while the temporal evolution would reveal the trajectories of particles. Analogously to DLIC, this approach can be formulated in the context of the framework by setting u(x, t) to the given time-dependent flow field v(x, t) and by performing the convolution along instantaneous streamlines (at time t ) of the same field, w(x, t) = v(x, t ). The convolution is computed on a single spatial slice through φ, i.e., Z(s; x, t ) = ( · , t ). Unlike DLIC, the evolution of streamlines along pathlines might not lead to streamlines of the subsequent time step. In other words, the temporal evolution according to u(x, t) is, in general, not compatible with the structure of streamlines of w(x, t). For the special case of a steady flow, however, pathlines and streamlines are identical, and the time evolution is compatible with the construction of streamlines. Therefore, the length of spatial patterns should be linked to the degree of unsteadiness. In regions where the flow changes rapidly, the correlated segments along streamlines should be very short or could even degenerate to point particles specified on φi , while time-independent parts should be covered by long patterns. Note that the motion of unfiltered elements of φi is, by construction,

152


temporally correlated, regardless of the unsteadiness of the vector field. The framework allows for variable filter types and lengths, controlled by quantities derived from the input flow, and thus is prepared for spacetime-coherent flow visualization. This visualization approach exhibits the following advantageous features. First, it shows both the motion of particles along pathlines and instantaneous streamlines in the same animation. Second, the degree of unsteadiness is revealed even in a single, frozen image in the form of the spatial frequency of noise along streamlines. Third, in the limit of a time-independent flow, the direction of motion is visualized similarly to animated LIC [123]. UFAC cannot solve the fundamental dilemma of inconsistency between spatial and temporal patterns, but it explicitly addresses the problem and directly controls the length of the spatial structures. It maximizes the length of spatial patterns and the density of their representation, while retaining temporal coherence. Other unsteady flow methods only implicitly address the issue, or neglect it completely. IBFV and LEA construct streaklines or time-reversed streaklines. When these lines intersect, the visualization may become convoluted or smeared out – it loses understandable spatial structure. An even more important problem of these techniques is the hysteresis in their visualization. An image strongly depends on the particle’s past. In contrast, UFAC does not depend on visualization history, as long as particle injection by Pi is based on noise. Hysteresis is even more severe for UFLIC due to its feed-forward approach. Even for a steady flow, the structure of the display depends on how long the visualization has been running: the longer the time, the longer the correlation length along streamlines. Large-scale structures without high spatial frequencies in one direction can emerge and could therefore limit the spatial resolution perpendicular to the velocity field after a time-dependent flow has turned. 3.6.4 Implementation The implementation of the texture-based framework relies on C++ and DirectX 9.0. GPU states and fragment programs are configured within effect files. Most parts of the framework can be readily mapped to the functionality of a DirectX 9.0 compliant GPU. A comparable implementation should be feasible with OpenGL and its fragment program support. Any texture-related data is stored in 2D textures with fixed-point color channels; time dependency is represented by stacks of 2D textures. First, the implementation of the first part of the framework – the construction of the property field – is described. The vector field u is represented by fixedpoint numbers in the RG channels of a 2D texture by mapping the original floating-point components to the range [0, 1] (this mapping could be avoided ˜ l ), the by using floating-point textures). The two coordinates of a particle (X accumulated property (φ˜l ), and the accumulated weight are stored together in the RGBA channels of another 2D texture. The coordinates are chosen


153

so that the range [0.25, 0.75]2 corresponds to the domain of the vector field; in this way, an area around the domain can be covered by valid fixed-point coordinates. First, this texture is filled with the initial values by rendering a quadrilateral that covers the computational domain. The corresponding fragment program implements (3.24). The particle injection Pl is represented by a 2D texture with a single color channel for the brightness of the particle; the weight w is an outside program parameter. Then, the recurrence relations (3.22) and (3.23) are evaluated by another fragment program that is executed by rendering a domain-filling quadrilateral ∆iτ times. This multi-pass approach makes use of the render-to-texture functionality of DirectX. The required values from φ˜l , Pl , and u are obtained by texture lookups, and the computations in (3.21)–(3.23) are directly supported by the numerical operations in the fragment program. In the last step, the accumulated values φ˜l−∆iτ are transferred to a single-channel representation of the final result by a trivial fragment program. The second part of the framework – the convolution – can be mapped to graphics hardware in a very similar way. After each of these two parts, histogram equalization is applied to the accumulated values to restore good contrast. A user-specified parameter makes possible a gradual change between full histogram equalization and no equalization. The histogram is built only once, by a CPU-based implementation. Equalization is applied to each fragment by a dependent texture lookup in the integrated histogram (i.e., in a color table). In addition, a Laplacian filter enhances the line-like patterns in the result of the final convolution; a 3 × 3 filter is employed in the current implementation. The strength of edge enhancement is also controlled by a user-set parameter. In addition, optional dye advection is implemented as a separate process as described in Sect. 3.3; its results can be overlaid onto the noise-based visualization for the final display. One important caveat is the resolution of texture channels. The accumulated property values are encoded as 16-bit numbers to allow for an adequate accuracy for the weighted sum of a large number of samples along a path. For example, a filter length of 300 samples is quite common for the convolution step. Accuracy is even more important when histogram equalization with its non-uniform mapping is applied. 16-bit channels are also used to represent coordinate arrays. From experience, these fixed-point numbers provide enough accuracy for the integration of particle paths and for addressing texture coordinates. Even higher accuracy could be achieved by floating-point numbers. Conversely, particle injection textures such as input noise can be 8 bit or less. Finally, the vector fields are represented by fixed-point 16-bit numbers in the current implementation. Again, floating-point textures could be used for higher accuracy and range.


(a)

(b)

(c)

(d)

IMAGES: WEISKOPF, ERLEBACHER, ERTL [472]

c 2003 IEEE

154

Fig. 3.33. Visualization of a shock data set: (a) UFAC with velocity masking; (b) UFAC without masking; (c) LIC; (d) LEA (see also Color Plate 23)

3.6.5 Results In this section, results obtained by the framework are presented and discussed. Videos contained on the project web page [463] demonstrate the effects of temporal correlation through animations. Figures 3.33 (a)–(d) show the same snapshot from the animated visualization of a time-dependent shock data set from CFD. The complete data set has 200 time steps and a spatial resolution of 256 × 151. Figures 3.33 (a) and 3.33 (b) present UFAC. An additional velocity masking (see Sect. 3.3.5) is applied to image (a) to emphasize regions of high velocity magnitude. In regions of large unsteadiness, such as the shock regions in the center of the image, the correlation length along streamlines is reduced. In this way, unsteadiness is visualized even in a single image. Steady regions of the flow are represented by clearly defined, long streamline structures. Figure 3.33 (c) is based on

155


c 2003 IEEE


Fig. 3.34. Electric field of an oscillating electric dipole

LIC, i.e., UFAC without controlling the length of streamlines. The important regions of unsteadiness are hard to recognize. Even worse, the animation exhibits substantial flickering in these regions because the temporal evolution is not compatible with the spatial structure of the streamlines. Figure 3.33 (d) demonstrates the LEA implementation within the framework. LEA shows velocity magnitude by the length of (inverse) streaklines, but tends to produce noisy images. The computational domain for all images is 512 × 302. For images (a)–(c), the maximum filter length is 120 and the maximum lifetime of particles is 40. The blending weight for LEA in Fig. 3.33 (d) is 0.05. Figure 3.34 shows two frames of the visualization of a time-dependent electric field generated by an oscillating electric dipole. The evolution of field lines is governed by a velocity field specified in [405]. The images were produced by the framework-based implementation of DLIC, on a resolution of 10242 with τ = 20 and filter length 1200. This example shows that a high-quality and fast DLIC visualization is feasible on a purely texture-based representation. Figure 3.35 compares IBFV-guided techniques. Both images represent the very same underlying continuous model – with two different kinds of discretization. The same exponential filter kernel and the same accuracy of 16 bits for the property fields is used for both images. The computational domain has size 5122 . Figure 3.35 (a) exhibits long, clearly defined streaklines generated by Lagrangian integration of trajectories in the framework-based approach. In contrast, Fig. 3.35 (b) shows strong blurring effects caused by the numerical diffusion in the original IBFV implementation (cf. the discussion in Sects. 3.4 and 3.6.3). This example is extreme because it uses only a few seed points and a filter with a long decay time. However, the blurring effect remains, to a smaller degree, even for shorter kernels and dense noise input. The performance of the implementation depends mainly on the maximum lifetime of particles and the support of the convolution kernel. Table 3.6 shows



c 2003 IEEE

156

(a)

(b)

Fig. 3.35. Comparison of IBFV-guided techniques: (a) long, clearly defined streaklines for the framework-based method; (b) numerical diffusion for the original IBFV implementation. See also Color Plate 24

the computation times of a single particle transport or convolution iteration (obtained by performing many steps and dividing by the number of steps). Times were measured on a PC with ATI Radeon 9700 GPU and Athlon XP 2200+ CPU (1.8 GHz), running under Windows XP and DirectX 9.0. Convolution is slightly slower than constructing the property field because the fragment program is slightly more complex. The timings show an almost perfect linear dependency with respect to the number of texels in the domain. Furthermore, the complete visualization process depends linearly on the number of iteration steps. Consequently, the overall performance can be adjusted to a targeted frame rate by adapting spatial resolution, filter length, and/or maximum lifetime, i.e., quality and speed can be smoothly balanced. In addition to the iterative computations, a small time offset is introduced by the general management of the visualization system (event handling etc.), edge enhancement, histogram equalization, velocity masking, color table mappings, final display to the screen, and transfer of time-dependent vector data from Table 3.6. Performance measurements for one iteration step of the texture-based framework in frames per second (ATI Radeon 9700, 128 MB) [472]

Domain size Constructing φ Convolution

2562

5122 10242

5,100 1,380 3,800 1,055

350 268

3.7 Summary

157

main memory to GPU. This transfer has only a minor effect on performance because just a single 2D texture for each new time step is passed to the GPU (since data for previous time steps has already been transferred). To give an impression of typical speeds for a complete visualization cycle that combines all these aspects: 12.7 fps (frames per second) are achieved for τ = 20, filter length of 60, and a 5122 domain; 3 fps for τ = 50, filter length of 300, and a 5122 domain. The above described implementation works on Shader Model 2 GPUs that do not need to support long fragment programs or fragment-shader loops. However, on GPUs that allow for longer fragment programs, the iteration along trajectories can be incorporated in a single fragment program. In this way, the multiple rendering passes with the ping-pong scheme can be reduced to a single rendering pass, which decreases the amount of data that has to be written to render targets and read from textures.

3.7 Summary In this chapter, several texture-based techniques for visualizing vector fields have been presented. A common goal has been to achieve interactive exploration by providing efficient, real-time visualization methods. This goal has been accomplished by introducing algorithms that are well-supported by GPUs. Another important aspect has been to explicitly distinguish the particle transport mechanism from the mechanisms for building visual patterns. Although, at some stage, all techniques rely on an Eulerian description of particle properties on a uniform grid, large variations of numerical particle transport mechanisms are present in the different techniques. For example, semi-Lagrangian advection employs a Lagrangian integration at each advection step, along with a permanent resampling on an Eulerian texture, which introduces numerical diffusion artifacts. Traditional semi-Lagrangian advection applies bilinear or trilinear interpolation (in 2D or 3D, respectively), whereas LEA is based on nearest-neighbor interpolation in conjunction with fractional sub-texel motion. In contrast, a complete Lagrangian particle integration has been used for the implementation of surface flow visualization in Sect. 3.5 and the generic spacetime framework in Sect. 3.6. Finally, the level-set approach for dye advection (Sect. 3.4) relies on a semi-Lagrangian level-set evolution, combined with a continuing reinitialization of the level-set. These different numerical techniques show different characteristics with respect to performance, memory footprint, and quality. Semi-Lagrangian advection is very fast because it computes particle transport in an incremental fashion – only a single integration step has to be computed for each displayed frame of an animation. Moreover, semi-Lagrangian advection is memory-friendly since only the current time step of the vector field and the property field has to be accessed. The main disadvantage is the numerical diffusion that leads to artificial blurring and, thus, reduced visualization quality.

158


In contrast, a complete Lagrangian integration scheme avoids this blurring as resampling is performed only at the final rendering stage. Unfortunately, the Lagrangian approach requires the re-computation of complete particle traces at each frame of an interactive visualization. Therefore, a much larger number of numerical operations is involved and, similarly, the memory footprint is much larger since all time steps along the integration path have to be stored (both the vector field data and the particle injection textures). For this reason, a purely Lagrangian approach is not suitable for dye advection, which has to follow dye patterns for a very long time. Therefore, the level-set description is a good alternative for dye advection because it combines the benefits of an incremental computation with a diffusion-free description. Unfortunately, the level-set method is not suitable for noise advection, for which no distinct interface is available. The actual mapping to visual primitives uses the concept of line integral convolution to introduce patterns in the texture-based property fields. Iterative alpha blending is completely compatible with the iterative semiLagrangian approach because it requires only the property field of the current time step. The according high performance of iterative alpha blending is achieved at the expense of a very restricted filter kernel: only an exponential filter kernel is possible here. In contrast, Lagrangian particle tracing accesses the fields at all points along the integration path anyway and, therefore, allows for a straightforward implementation of an arbitrary filter kernel function. The different methods for visual mapping are dependent on the underlying numerical scheme for particle tracing and they show the same behavior with respect to differences in performance on the one hand and quality or flexibility on the other hand. While the mapping of 2D property fields to the 2D image plane is trivial, the projection of 3D property fields onto a 2D plane is quite challenging. In this chapter, the visual representation of 3D vector fields has been accomplished by means of volume rendering. Direct volume visualization has the advantage of showing information at different depths simultaneously by semitransparency. However, the application of volume rendering to dense vector field representations leads to issues of shape and depth perception, occlusion, and visual clutter. This chapter has discussed ways to reduce the visual information in order to overcome these perceptual problems. One approach is based on transfer functions to selectively emphasize or hide parts of the data set by specifying the opacity based on flow features. Another, closely related method applies volume clipping techniques (from Sect. 2.5) to remove unimportant regions. Alternatively, advanced shading and illumination techniques can be employed to improve depth and shape perception. More details on sophisticated perception-oriented rendering methods are discussed in the following chapter. Although the interactive approaches from this chapter contribute to an improved visualization of vector fields, the visualization of 3D vector fields is not yet solved in all generality. In fact, there is not a single solution that

3.7 Summary

159

would provide perfect results for data sets from completely different sources and applications. To give a better understanding of the specific advantages and disadvantages of different flow visualization methods, Laramee, Weiskopf, Schneider, and Hauser [242] investigated and compared a large variety of techniques for the specific example of CFD simulations of swirl and tumble flow in automotive engine design. One result of this investigation is the general guideline that a good visualization system should provide a variety of different visualization methods so that a user can choose the most appropriate technique, based on the type of input data and exploration goals. Such a visualization system ideally allows the user to combine the results of different methods within the same image.

4 Perception-Oriented and Non-Photorealistic Rendering

The main goal of all fields of visualization and computer graphics is to produce images for a human observer. Therefore, visual perception has been playing an important role in the visualization and computer graphics communities and will continue to do so. Utilizing the knowledge of the properties of the Human Visual System (HVS), rendering techniques have been enhanced by making them more effective and/or more efficient. An effective visual representation tries to achieve a specific visual sensation, which is particularly important for any visualization technique. This chapter focuses on rendering methods guided by properties of the HVS, i.e., it mainly concerns the rendering stage of the generic visualization pipeline. In particular, Sect. 4.2 addresses issues of motion perception in animated visualizations. Here, relevant results from psychophysical and physiological research are reviewed to clarify the role of color in motion perception. Based on these psychophysical experiments, guidelines are proposed for the design and rendering of effective animated visualizations. Section 4.3 specifically investigates color-based depth cues to improve spatial perception by a generic and flexible depth-cueing scheme that subsumes well-known and new color-based depth-cueing methods alike. While the above approaches are based on early stages of the HVS, nonphotorealistic rendering (NPR) can be considered as a way of making use of higher-level processing stages in the human brain. Since high-level cognitive processes are less well understood and formalized than early stages of perception, NPR techniques essentially exploit the knowledge of illustrators and artists in producing expressive drawings. The underlying assumption is that traditional illustrations are based on an implicit, yet effective, understanding of human perception. One of the major advantages of non-photorealistic illustrations – in contrast to photorealistic renderings or actual photographs – is that they can provide a selective view on important details while extraneous details are omitted [94]. For example, NPR styles can be used to improve the recognition of the shape and structure of objects, their orientation, or spatial relationships [20, 29, 57].

162


Two specific aspects of NPR are discussed in detail. The first aspect deals with the visualization of objects with complex depth structure. The goal is to show more than just the foremost surface of an object and reveal the spatial relationships inside the object. Based on traditional illustration methods, sets of rules are extracted for computer-based non-photorealistic illustrations. Here, the focus lies on methods that build upon tone-shading for continuous surface rendering. Both semi-transparency (Sects. 4.4.1 and 4.4.2) and cutaway illustrations (Sects. 4.4.3–4.4.5) are investigated, along with a performance analysis (Sect. 4.4.6). Section 4.4.7 demonstrates that semi-transparent NPR and cutaway illustrations can be adopted and extended for volume illustrations. This section is strongly related to volume rendering (Chap. 2), especially to volume clipping (Sect. 2.5). The second part of the NPR discussion deals with halftoning (Sect. 4.5). Halftoning is typically applied to achieve a clearer and more pleasant display style. For example, many illustrations in textbooks on archaeology or medical science are drawn in a stippling style, even when photographs of good quality are available. Image-space techniques for halftoning are discussed in detail. In particular, the issue of frame-to-frame coherence is addressed (Sect. 4.5.2) and a generic GPU-based G-buffer framework is introduced for image-space operations (Sect. 4.5.3). As outlined above, this chapter focuses on rendering – and thus on the rendering step of the visualization pipeline. The rendering methods of this chapter are generic and can be applied to many different visualization applications by just replacing the rendering stage with an extended rendering method. This is particularly true for the continuous-tone techniques from Sects. 4.2 and 4.3, which can enrich any existing continuously shaded display.

4.1 Previous Work In recent years, a lot of research has been conducted on perceptual issues in computer graphics. Various aspects of human visual perception are presented in a number of SIGGRAPH courses [52, 102, 121, 158, 180, 408], IEEE Visualization tutorials [120, 244], a book by Ware [455], and an overview article by Tumblin and Ferwerda [429]. One line of research uses the knowledge of the HVS to improve the quality of displayed images, based on a photorealistic computation of illumination [119, 282, 322]. Quantitative quality measures are typically based on a Contrast Sensitivity Function (CSF) that depends on spatial frequencies of the image and additional viewing parameters. Examples for elaborated spatial and chromatic quality metrics are the Visible Difference Predictor (VDP) [71], Sarnoff’s Just-Noticeable Difference (JND) model [262], or Rushmeier et al.’s models [359]. The spatiotemporal CSF [202] describes the detectability of moving patterns and is an important element in motion-aware metrics [301, 302]. Saliency models [185] simulate where people focus their attention in images to

4.2 Color and the Perception of Motion

163

improve the efficiency of rendering [32, 101]; some models take into account motion as part of the saliency computation [33, 500]. Motion perception has also been investigated for effective visualizations. Since motion provides additional perceptual dimensions, such as phase, amplitude, or frequency of sinusoidal motion [249], these perceptual dimensions can be used in the visualization of multivariate data to display additional data dimensions. Motion is preattentive and allows moving patterns to pop out [303]. Therefore, motion can be used in fast, preattentive visualization of complex data [156, 157] or for filtering and brushing techniques in information visualization [9, 10]. Also, motion can promote the perception of shape; this structure-from-motion effect is exploited by kinetic visualization [264], using moving particles. Similarly to perception-based rendering, non-photorealistic rendering has attracted much attention in recent years. An overview of NPR techniques can be found in textbooks by Gooch and Gooch [138], Strothotte and Schlechtweg [404], and in a SIGGRAPH course on NPR [144]. Saito and Takahashi [368] introduce the G-buffer as a generic framework for image-space NPR techniques. Mitchell et al. [291], for example, present a corresponding imagespace algorithm to detect object outlines. Different classes of NPR styles are widely used – pen-and-ink illustrations [85, 125, 273, 335, 338, 369, 387, 491] and halftone rendering [126] are important types of NPR. Another class of NPR methods makes explicit use of color tone variations. For example, Curtis et al. [69] imitate the appearance of water-colored images, Gooch and co-workers [137, 139] facilitate airbrush techniques for tone-based cool/warm shading. Meier [283] employs several blended layers of differently shaped and oriented strokes to achieve diffuse object boundaries in a painterly rendering approach. Interrante and co-workers [181, 182, 183] render several semi-transparent depth layers with a stroke-based approach. They visualize the 3D shape of smoothly curving transparent surfaces by utilizing the principal curvature directions of surfaces. In a subsequent work [177], they extend this approach to 3D line integral convolution in order to illustrate surface shape in volume data. Finally, Seligman and Feiner [389] describe a rule-based illustration system, Markosian et al. [273] introduce a real-time NPR system, and Hamel et al. [150] present a semi-automatic system to handle transparency in line drawings.

4.2 Color and the Perception of Motion “The perception of dynamic patterns is less well understood than the perception of static patterns. But we are very sensitive to patterns in motion and, if we can learn to use motion effectively, it may be a very good way of displaying

164


certain aspects of data.” Colin Ware [455], p. 230 The goal of this section is to provide assistance in improving the use of color in conjunction with motion. Here, we have to distinguish between the detection of patterns in motion (seeing their “existence”) and the actual perception of motion (recognizing speed and direction of motion). The detection of moving patterns is quite well understood. Measured quantitative data for the corresponding CSFs is frequently used in the perception-based rendering techniques mentioned in the previous section. However, the sensation of motion is less well understood for color stimuli and, in particular, purely chromatic, isoluminant stimuli are extensively discussed in psychophysics and neuroscience literature. Since motion perception can be impaired for color stimuli, it is a common assessment in the visualization and computer graphics communities that color is irrelevant to perceiving how objects are moving [456]. Although this statement is useful as a general guideline, there are some noticeable chromatic effects that lead to a differentiated view on the perception of moving colored objects. This section directly follows the paper by Weiskopf [465]1 and covers the following aspects in detail. First, relevant results from psychophysical and physiological research are presented (Sect. 4.2.1). Here, ample evidence is provided for an important role of color in motion detection. Second, the interpretation of results from psychophysical experiments leads to a proposal of some guidelines that assist the design of animated graphics (Sect. 4.2.2). Third, it is inferred that a calibration is needed to represent data by the perceived speeds of colored patterns; and a corresponding calibration procedure is described for a desktop work environment in Sect. 4.2.3. Finally, visualization applications are discussed in Sect. 4.2.4 to demonstrate how the guidelines and the calibration approach can be used. 4.2.1 Psychophysical and Physiological Research Some aspects of the HVS are briefly discussed for color vision in general and motion detection in particular. Detailed background information on neuroscience and human vision can be found in [175, 200]. Two categories of photoreceptors in the retina serve as detectors for light that is collected by the eye: rods and cones. Here, only cones are considered because they dominate the initial response at normal light levels. Cones can be classified as shortwavelength (S) cones, medium-wavelength (M) cones, and long-wavelength (L) cones. The responses of the S, M, and L cones are coded in terms of opponent color channels. The achromatic channel represents the intensity of light and reflects the sum of M and L cones. The red-green (RG) channel measures the difference between the L and M signals, the yellow-blue (YB) 1

c 2004 IEEE. Reprinted, with permission, from Weiskopf [465]


165

channel measures the difference between the sum of L and M signals and the S response. The geniculostriate system (lateral geniculate nuclei, LGN) consists of two major pathways with distinct physiological properties: the color-opponent and the broad-band (achromatic) pathways, which remain separated through several cortical stages. Conventional views of visual perception propose that different visual capacities are connected to the different pathways. The “colorblind” (magnocellular) broad-band pathway provides motion information and the “motion-blind” (parvocellular) pathway conveys (opponent) color information [251, 503], i.e., the visual system’s primary analysis of color and motion takes place in parallel at different areas in the brain. Therefore, motion analysis would be compromised if color had to be analyzed before motion. Numerous psychophysical experiments have been conducted in the vision community to gain a detailed understanding of the HVS’s motion-detection capabilities. In the following, results are presented that are relevant for the design of practical visualization techniques. The main parameters for psychophysical investigations are the type of stimulus that is presented to the human observer and the kind of task that has to be solved. Typical stimuli are sinusoidal gratings or random dot patterns that either move spatially or flicker temporally. Two main tasks can be distinguished. The first task is the detection of motion to reveal thresholds for motion discrimination. Actually, two different sub-tasks with the same moving stimulus are performed to measure forced-choice contrast thresholds for stimulus detection (discrimination from blank field) and direction-of-motion detection. The motion discrimination threshold is then given as the ratio of the motion threshold to the detection threshold, i.e., the M/D ratio. The unit-free M/D ratio has the advantage that reduced cone contrasts under different stimuli conditions are canceled out [319], i.e., the influence of the spatiotemporal CSF is absorbed. The second task is the detection of velocity magnitude. The perceived speed is usually described with respect to the speed of an achromatic stimulus [50]. In accordance with the aforementioned notion of separated motion and color pathways in the HVS, some psychophysical investigations show that motion perception of purely chromatic, isoluminant stimuli is degraded under some conditions. Ramachandran and Gregory [336] report a failure to perceive motion in RG random-dot kinematograms at isoluminance. Subsequent investigations support this point of view by demonstrating that the perceived speed of moving isoluminant stimuli is slower in comparison to that of luminance stimuli [50, 155, 251, 258, 299, 306, 426]. On the other hand, the loss of motion perception in color vision is not complete, especially for suprathreshold stimuli. Both the motion direction of drifting sinewave gratings and the frequency of flicker can be discriminated at contrasts close to, although not at, detection threshold [48, 84, 134, 299, 319]. Cropper [65] reports that, for suprathreshold chromatic contrast, the discrimination of both motion direction and velocity magnitude improves to performance levels comparable to that of luminance gratings. In addition, there is

166


an indication that a low-level motion-detection mechanism is prevalent at high contrast [66] and high speeds of isoluminant stimuli [388]. Experiments with combined motions of luminance and chromatic patterns also suggest that color plays a substantial role in motion perception [321]. Another indication for a chromatic contribution to motion detection is given by the motion aftereffect from isoluminant stimuli [49, 83, 298]. Furthermore, the motion-detection mechanisms of the HVS can identify even very brief signals, which shows that chromatic motion provides input for early stages of motion analysis [67]. Finally, neurophysiological experiments with primates indicate that there are mechanisms in the motion-detection pathway that have some (reduced) color sensitivity [256, 367]. In addition to the detection of motion direction, the perceived velocity magnitude of patterns plays an important role. As mentioned before, there is extensive evidence for color-induced apparent motion being slower than luminance-induced motion [50, 155, 299, 306]. The perceived speed of chromatic stimuli is strongly contrast-dependent [65, 299], compared to luminance stimuli (even luminance gratings, however, “slow down” at reduced contrasts [417]). Some possibilities that would allow a chromatic stimulus to produce a response in the luminance mechanism of the HVS are discussed extensively in vision literature and are briefly reviewed here. This luminance effect would explain the aforementioned results within the concept of different pathways for motion and color. The possibility of luminance “artifacts” that directly originate in the stimulus (for example, from an inaccurate overall isoluminant point, from variations in individual cells’ isoluminant points, or from chromatic aberration) is extremely unlikely [155, 300]. However, some psychophysical experiments indicate that chromatic stimuli can be contaminated by dynamic luminance artifacts (for example, nonlinearities in the responses to colors, or differences in the temporal phase of neural response) [300, 501]. Finally, Lu and Sperling [259, 261] propose a three-systems theory of human visual motion perception, where the third-order system has simultaneous access to inputs of form, color, depth, and texture. Third-order motion detection would be based on the analysis of moving features in a saliency map. Although attention may play a greater role for chromatic than for achromatic motion processing, there are other experiments with motion aftereffects which indicate that chromatic motion does not appear to rely more on attentional processing than achromatic motion does [339]. In general, it is not quite clear to what extent the different pathways in human perception can interact with each other; in addition to the issues discussed in this chapter, there are numerous other phenomena that indicate manifold interactions between different elements of stimulus processing (see, e.g., Kandel et al. [200]).


167

4.2.2 Design Guidelines Here, the above psychophysical findings are adapted for the needs of visualization applications. Although there is not yet a well-established computational model that would describe all previously mentioned results, some useful guidelines are provided for the application of color in animated computer graphics. The goal of psychophysical research is to gain a better understanding of the structure and properties of the HVS and therefore much effort is made to clearly separate the influences on different pathways by choosing specifically designed stimuli (e.g., sinusoidal gratings). In contrast, visualization applications contain a variety of different visual objects and typically do not activate an isolated response within the HVS. To assist the design of effective and efficient rendering techniques, the results of the aforementioned literature are combined to propose some guidelines. The main goal is to stimulate an improved awareness for both the opportunities and limitations associated with moving color patterns. In accordance with a prevalent assessment in the visualization community, the first guideline is: [G1]

Use luminance contrasts for best motion perception.

Whenever possible, luminance signals should be applied to provide a reliable motion perception. It is a well-established fact in vision research that the HVS is efficient in analyzing moving luminance patterns. Therefore, luminance stimuli are the “golden standard” in all previously mentioned psychophysical studies. Luminance contrast is not only useful for motion perception, but for many other perceptual tasks; for example, high levels of spatial detail are best displayed by a luminance-based color map [349]. Unfortunately, pure luminance patterns cannot always be used. For example, the visualization of multivariate data often relies on a mapping to several color channels. We have to consider the limited budget of color dimensions to find an optimal choice of colors. If the data dimension associated with motion is less important than another data dimension, luminance contrast will be mapped to the latter dimension, while the moving pattern will be realized by color contrast. In the previously mentioned psychophysical literature, motion discrimination is degraded only for very special conditions. M/D ratios are almost always close to one, except for nearly perfect isoluminant stimuli; e.g., Palmer et al. [319] report that already a one-percent luminance mismatch is sufficient to restore M/D ratios to one. Moreover, there is extensive evidence that suprathreshold chromatic contrasts promote the perception of motion for a variety of tasks and scenarios [65, 66, 67, 84, 388]. These investigations show that direction discrimination is possible for isoluminant patterns. Even though different tests lead to different M/D ratios, suprathreshold contrast generally supports motion detection. These psychophysical results are compatible with neurophysiological experiments in which color contrast is shown to support the perception of apparent motion by rhesus monkeys [92]. Some psychophysical experiments [299] even indicate that the M/D ratio is close to one in any

168


case. As a common finding, the following guideline for non-luminance patterns is stated: [G2]

High chromatic contrasts support the discrimination of the direction of motion for (nearly) isoluminant patterns.

In this context, it is important to note that perfect isoluminance is extremely difficult to achieve for practical visualization applications, which further supports the use of colored objects. First, typical computer-generated images simultaneously cover a range of different spatial frequencies: edges contribute high spatial frequencies, whereas larger, interior parts result in low frequencies. Since isoluminance is affected by spatial frequency [2], a single, overall isoluminance point is hard to find. Second, isoluminance also depends on temporal frequency [287, 403]; and there is some indication for dynamic luminance artifacts from moving chromatic stimuli [300, 501]. Motion, in general, strongly attracts visual attention [494] and is an important feature to steer the user’s attention. Furthermore, the results from [98] indicate that motion can serve as a feature mask and lead to a selective filtering of objects, which is the basis for visual grouping. There is some indication that chromatic stimuli contribute to a salience map and that the analysis of their motion is directly computed from this map [259, 261]. Therefore, the following guideline is proposed: [G3]

Highlighting and visual grouping can be based on the motion of color patterns.

Motion discrimination has to be distinguished from the perception of smooth motion [299] and the speed of motion. There is psychophysical evidence that the perceived speed of a stimulus with fixed luminance contrast is strongly affected by changing the chromatic contrast. In experiments with moving sinusoidal gratings, the luminance mechanism is “diluted” by chromatic contrast [49, 50]. From a naive point of view, this dilution is surprising because, in stationary displays, redundant color scales, in which two or more channels are varied together, reinforce a signal. In contrast to the results for moving sinusoidal gratings, experiments with another, largely different set of stimuli and user tasks indicate that performance of observers can sometimes be improved by combining luminance and chromatic motion (both compared to stimuli that only match in luminance or color) [321]. In conclusion, the existence of possible interference problems leads to the following guideline: [G4]

Take into account that chromatic and achromatic channels can influence each other, especially for the perception of apparent speed.

Accordingly, this guideline proposes a restriction to variations in only a single color channel, which is compatible with other rules for constructing effective color tables [343, 344]. Often, only a single color-model component is varied


169

for univariate data. For example, luminance can be held constant to minimize interpretive errors caused by perceptual effects such as simultaneous contrast [454]. Considering [G4] and the previously mentioned investigations [50, 65, 299, 306], there is evidence for an influence of luminance and/or chromaticity contrasts on the perception of motion. Since this effect is particularly important for the perception of apparent speed, this last rule is proposed: [G5]

Calibration is needed to represent ordinal data by the perceived speed of colored patterns.

Note that [G4] and [G5] apply even when significant luminance contrasts are present in a color display. 4.2.3 Calibration Two calibration procedures are required to implement guidelines [G4] and [G5]: the determination of isoluminance and a measurement of apparent speed. The first procedure is the calibration of isoluminance. Different interpretations of isoluminance have to be distinguished. First, luminance could be equated according to the luminous efficiency function Vλ of the CIE standard observer [198]. Second, isoluminance can be determined from psychophysical experiments with the individual user. The second approach is chosen here because isoluminance settings can vary markedly among individuals [86]. The calibration is performed with the same viewing conditions under which the actual application will be run. In this way, additional parameters that influence the isoluminant point are taken into account, e.g., different color reproduction by different monitors and graphics cards, the adaptation to the environment, or the effect of the distance between user and screen. The minimum-motion method [2, 260] or the minimization of apparent flicker for a flickering stimulus [28] are often used in psychophysical experiments. To avoid the sometimes annoying flickering, an alternative approach by Kindlmann et al. [213], which relies on the brain’s ability to distinguish human faces, is applied. Their method is easy to use, comfortable, fast, and allows the user to match the luminance of color stimuli and determine the monitor’s gamma. An implementation can be downloaded from their web page [211]. This calibration method is acceptable for practical applications because we do not have to achieve perfect isoluminance with respect to motion (as it is required for psychophysical experiments). In fact, many different parameters (such as spatial and temporal frequencies of the stimuli, or parafoveal vs. peripheral areas of the retina) have subtle effects on the isoluminance point and, therefore, it is dubious whether a perfect calibration can be achieved for animated computer-generated images. The goal of the second, subsequent calibration procedure is to determine the apparent speed of stimuli that differ in luminance and/or color. The principal setup used by Cavanagh et al. [50] is adopted. Figure 4.1 shows a typical



c 2004 IEEE

170

Fig. 4.1. Calibration of perceived speeds. Two texture stimuli are rendered: a benchmark luminance contrast in the lower half of the screen and, in the upper half of the screen, a texture with the stimulus of which the apparent speed has to be calibrated. The speed of both textures has to be matched by the user. See also Color Plate 26

screen configuration of the calibration system. Two horizontally drifting textures are used as stimuli. An achromatic luminance texture is presented in the lower half of the screen, serving as a benchmark for perceived speed. The color stimulus is rendered in the upper half of the screen. The two textures move in opposite directions to avoid any apparent cue for matching the speeds. Both textures are separated by a small, horizontal, and achromatic strip that has the mean luminance of the two stimuli. During the calibration task, the human observer has to match the perceived speeds of the two textures. Velocity, spatial structure, and contrast of the benchmark achromatic texture is determined by user-specified parameters and fixed during the calibration process to rule out side effects. (Even achromatic gratings exhibit non-constant perceived speed; high spatial frequency and low contrast gratings appear to move more slowly [46, 417].) The speed of the color texture can be adjusted by the observer. To prevent a potential buildup of motion aftereffect, the user can reverse the directions of drift at any time by pressing a keyboard button. The calibration is carried out for a few user-specified “key” stimuli that should represent the range of possible contrasts in the actual application. If needed, perceived speeds for intermediate contrasts are computed by interpolating between the “key” points. The corresponding interpolation of colors takes into account the monitor’s gamma. In contrast to [50], sinusoidal gratings are not necessarily used as drifting textures, but the textures are mod-


171

eled according to the targeted application. This approach has two advantages. First, the stimuli represent the needs of the actual application. Second, this method avoids monotonous gratings that can be disturbing for the user, especially if the matching process takes some time. In Fig. 4.1, the moving stimuli are generated by a GPU-based implementation of line integral convolution (LIC) [43], according to the Lagrangian integration scheme for the spacetime framework for texture-based flow visualization in Sect. 3.6. The speed calibration process was tested by an informal user study with computer science students (six males and one female) from the University of Stuttgart. All users had normal or corrected-to-normal vision. After an introduction to the calibration system and a trial phase to become familiar with the system, the users were asked to match the stimulus shown in Fig. 4.1. All tests were performed under rather typical viewing conditions (desktop environment with dim surrounding light, 21 inch iiyama CRT monitor, approximately 18 inch distance between eye and screen, ATI Radeon 9800 XT graphics board, gamma γ = 2.3, Windows XP operating system without a built-in gamma correction). The calibration times for one stimulus varied between half a minute and three minutes. The relative (physical) speed of the YB stimulus and the achromatic stimulus of the same perceived speed was in the range between 1 and 1.5 (for the different observers). Some participants of the study frequently reversed the directions of drift, whereas others did not use the reversal function at all. None of the users experienced a disturbing motion aftereffect or complained about other unpleasant visual effects of the moving textures. 4.2.4 Applications In this section, the guidelines from Sect. 4.2.2 are applied to typical examples from information visualization and flow visualization. The actual implementation of the guidelines usually amounts to a specific choice of color table for the final display. Within a GPU approach, such a perception-guided color table can typically by implemented in the form of a dependent texture lookup, e.g., in a slightly modified fragment program. Information Visualization An important field in information visualization is concerned with the visual representation of multivariate data or complex interrelations between elements, e.g., in graphs or large tree hierarchies. Motion provides further useful perceptual dimensions in addition to color or shape [249]. Therefore, motion plays an important role in conveying additional data dimensions in the representation of multivariate data. Taking into account guidelines [G2] and [G3], motion can be combined with color or other perceptual channels to extend the visualization space. Moreover, the preattentive [421] character of motion allows simple motion to effectively pop out objects in a crowded



c 2004 IEEE

172

Fig. 4.2. Grouping through motion in graph visualization. A subset of four nodes moves coherently. This motion is illustrated by a duplication and slight shift of the moving edges and points. A zoomed-in view of this region of interest is shown in the bottom-left part of the image. See also Color Plate 27

display which are dissimilar in all except their motion parameters [98, 303]. Therefore, motion can be used for filtering and brushing techniques [9, 10]. Brushing [16] highlights a subset of data interactively and thus allows the user to link related elements. Since this approach needs its own coding dimension, guidelines [G2] and [G3] relax the restriction to specific choices of colors and extend the range of possible combinations with other display dimensions. Figure 4.2 illustrates the grouping behavior within the visualization of a graph. Four of the nodes move coherently (cf. the accompanying video on the project web page [466]) and show a strong grouping property – they pop out as a connected element within the visualization. Background and foreground colors are chosen isoluminant. Here, the luminance calibration was performed by the author under the same viewing conditions as described in Sect. 4.2.3. The same viewing conditions are also assumed for the following example application. This extreme example of Fig. 4.2 demonstrates that even isoluminant colors are capable of supporting effective motion perception. Another effect that is not connected to motion perception should be noted here: the acuity limit for high spatial frequencies, as displayed in this example, is reduced at isoluminance (even in a steady image) [297].

173


c 2004 IEEE


Fig. 4.3. LIC visualization of a 2D vector field. An additional data dimension is represented by color coding. The lower part contains a nearly isoluminant YB contrast, which appears to move more slowly than the luminance patterns in the upper part (shown in the accompanying video [466]). See also Color Plate 28

Texture-Based Flow Visualization As discussed in Chap. 3, a popular way of visualizing vector fields is based on dense texture representations. LIC is a classical texture-based flow visualization approach. In a standstill visualization, LIC shows only the orientation of the vector field, but not its direction. This problem can be overcome by displaying the direction of motion in an animation. The animation can be computed by employing a periodic filter kernel, where the phase of the filter is changed according to time [43]. Additional information – such as pressure or temperature of an underlying flow field – is often encoded by means of a univariate color table. Figure 4.3 shows an example of a combined visualization of a vector field and one additional data dimension. The implementation is the same as for the spacetime framework in Sect. 3.6. Perception issues may arise when color and animation are applied together. Considering guideline [G5], a calibration of perceived speed is generally needed to faithfully represent the speed of moving patterns. The accompanying video [466] shows animated versions of Fig. 4.3. The first sequence illustrates the animation with the same physical speed of LIC structures in all image regions. Due to the influence of different colors, some regions appear to move more slowly than others. The second sequence demonstrates that a correction of physical speed according to the calibration from Sect. 4.2.3

4 Perception-Oriented and Non-Photorealistic Rendering Fig. 4.4. Surface flow visualization: (a) shape and flow structures are represented by luminance contrasts; (b) cool/warm shading for the surface and luminance contrasts for LIC patterns; (c) shape is represented by luminance, flow structures by isoluminant YB contrast (data set courtesy of the BMW Group). See also Color Plate 29

(a)

c 2004 IEEE

(b)


174

(c)


175

leads to a constant apparent speed. Here, the animation speed for the LIC computation is space-variant and takes into account the calibration data. Figure 4.4 shows the visualization of the vector field from a CFD simulation of wind flow around an automobile. The real-time implementation of flow visualization on surfaces is described in Sect. 3.5. Additional perception issues become relevant because the shape of the surface has to be conveyed. One problem is that both shape and motion are best visualized by luminance contrasts. Different principal ways of simultaneously representing shape and vector field are possible (see also Sect. 3.5.3). First, both shape and vector field could be mapped to the luminance channel, for example, by multiplying both contributions. This approach is illustrated in Fig. 4.4 (a). A drawback is that the partly different needs of shape and vector field visualization interfere; for example, the left side of the car is only dimly illuminated and therefore the contrast is insufficient for the LIC structures. This observation leads to another approach in which the vector field is represented by luminance contrasts and shape by color contrasts (Fig. 4.4 (b)). Cool/warm shading [137] is adopted for the surface, with isoluminant cool/warm colors. According to guideline [G4], the LIC texture is represented by luminance contrasts without any chromatic contrast. This approach gives a good visualization of motion [G1], but a less distinct visualization of shape. Finally, the roles of luminance and chromaticity can be exchanged, as demonstrated in Fig. 4.4 (c). Here, shape impression is predominant, at the cost of degraded motion perception. An advantage is that the LIC patterns do not stimulate an apparent, usually undesirable depth impression that is present in typical luminance-based LIC animations. Once again, perceived speed can be controlled by calibration [G5]. Other Potential Applications The guidelines could also influence other visualization and rendering techniques. One example could be kinetic shape visualization. Structure-frommotion perception [452] can be exploited to improve the recognition of shape. For example, structure-from-motion can be induced by a movement of points [422], which is used in kinetic visualization [264]. Additional information can be encoded by colors. Lum et al. [264] apply cool/warm variations for shading and reserve luminance variations for outlines and highlights. The guidelines from Sect. 4.2.2 can be used to choose appropriate colors that support a good perception of motion. For example, high chromaticity contrasts should be applied and luminance and chromaticity channels could be separated. As another example, efficient rendering of dynamic scenes could benefit from a sophisticated treatment of color and motion. In particular, there is a potential for improving saliency-oriented rendering frameworks [101, 500]. While motion computation is often based on the luminance channel only (e.g., for the image-space velocity map in [500]), the motion map could take into account both luminance and chromatic channels in a more sophisticated approach. If chromatic and luminance inputs are treated equally, no further

176


changes are required for the actual rendering framework and an extension to moving color attractors could readily be included. A more advanced approach, however, would require an extension of the visual attention model to incorporate color.

4.3 Color-Based Depth-Cueing While the previous section has focused on how color affects motion perception, this section addresses issues of depth perception in combination with color. Since the retina of the human eye is only two-dimensional, depth cues are essential for the visual perception of spatial structures. Primary and secondary cues can be distinguished. Primary cues include binocular disparity (stereoscopic viewing), convergence, and accommodation. On a computer system, the primary cues can only be implemented in the form of specialized hardware (such as shutter glasses or color-based ChromaDepth glasses [5]) and are not widely facilitated. Secondary cues include perspective, relative size, occlusion, shadows, texture, color gradients, motion parallax, and active movement. A description of these depth cues can be found, for example, in a survey article by Keley [203]. These secondary cues do not rely on specific hardware and can thus be readily used to improve depth perception in computer-generated images. This section presents a color-based depth-cueing scheme that is especially appropriate for non-photorealistic or perception-oriented rendering techniques. Intensity depth-cueing is a well-known example of a color-oriented technique for enhancing depth perception. This approach creates an effect similar to viewing a scene through haze by dimming the colors of far-away objects [122]. Intensity depth-cueing is influenced by aerial perspective, which has been used by painters for many centuries. Early examples can be found in the works by the Flemish painter Jan van Eyck (approximately 1390–1441), e.g., [435]. In many drawings, an additional slight color modulation is noticeable – remote objects experience a subtle blue shift [105]. These depthcueing techniques can be easily implemented by exploiting the standard fogging functionality on graphics chips. With a more advanced rendering method, atmospheric scattering can be represented in a procedural texturing approach to aerial perspective [104]. A change in saturation is another effect prominent in aerial perspective – the colors of far-away objects are less saturated. Nevertheless, this effect has been widely neglected in computer-generated imagery, although investigations of the HVS by Troscianko et al. [427] indicate that color gradients at isoluminance affect perceived depth. Their study reveals that both saturation and luminance gradients, which occur in natural scenes, facilitate depth perception. On the contrary, for example, a red–green hue gradient, which does not occur in nature, does not affect depth perception. Troscianko et al.’s work is the psychological motivation for saturation depth-cueing.

4.3 Color-Based Depth-Cueing

177

This section describes a generic depth-cueing scheme that comprises all of the above color-based cues. This scheme has been introduced by Weiskopf and Ertl [474, 475] and the following discussion is based on their work.2 Despite of its generality, this approach is computationally simple and can be directly mapped to GPU programs. It is transparent to the implementation of 3D viewers and can readily be combined with other non-photorealistic rendering or visualization techniques, assisting the idea that graphical excellence is nearly always multivariate [428]. In particular, the use of saturation gradients is a very effective monocular depth cue. Unlike intensity-based cueing, saturation depth-cueing leaves intensity invariant and therefore preserves features in a scene. This is close to Tufte’s strategy of the smallest effective difference [428], making visual distinctions as subtle as possible, but still clear and effective. Because of the flexibility of the proposed scheme, even more sophisticated applications become possible, of which some are presented in this section. In particular, it is shown how depth perception is improved by combining different depth cues. Furthermore, semantic depth-cueing is introduced as an abstract formulation of depth-cueing. 4.3.1 Linear Transformations and Weighted Sum of Colors This section presents the basic idea of the generic depth-cueing scheme of which the development is guided by the well-known intensity depthcueing [122]. This original model produces a final color from a weighted sum of an original input color and a user-specified background color. In this section, the model is extended by two new, crucial features. First, the weights for the combination of the two colors can be chosen independently for each of the three color components. Second, the weighted sum does not need to be calculated with respect to a fixed color coordinate system (which would usually be RGB). In fact, an arbitrary linear transformation of the input colors is applied before the summation is performed. This depth-cueing scheme is based on fundamental concepts of color vision and colorimetry. Here, only the relevant aspects are briefly reviewed. More detailed background information on color science can be found, for example, in a book by Wyszecki and Stiles [498] and the references in Sect. 4.2.1. A fundamental result of experimental color matching and a key element of colorimetry is trichromatic generalization, which states that, over a wide range of observations, many color stimuli can be matched by an additive mixture of three fixed primary stimuli. All other stimuli have to be mixed with one or two primary stimuli before a complete match with the remaining stimuli can be achieved. In the stronger form of trichromatic generalization, linearity laws of additivity and proportionality are valid, giving rise to a three-dimensional vector space – the tristimulus space of colors. From a physiological point of view, this property of color perception is based on the fact that three different 2

c 2002 A K Peters, Ltd. In part, material is reused from Weiskopf and Ertl [475],

178


types of cones are responsible for color detection in the human eye’s retina. Neglecting further levels of processing in the HVS (as they are important in Sect. 4.2), signals from these receptors essentially add up to the finally perceived color. A couple of different color spaces are in wide use today. One of these was introduced by the CIE (Commission International de l’Eclairage) for measurements of their CIE 1931 Standard Colorimetric Observer. It consists of the primaries red, green, and blue as monochromatic stimuli of wavelengths λred = 700 nm, λgreen = 546.1 nm, and λblue = 435.8 nm. Since the phosphors of contemporary computer monitors are not identical to these CIE RGB primaries, related RGB systems may be based on measurements of these phosphors. Another important color system is the XYZ system, which was introduced as a standard colorimetry system by the CIE. All these color systems are directly based on the vector space concept of tristimulus space. They just represent different basis vectors – the primary colors – of the same three-dimensional vector space. Therefore, a change of basis can be realized by a linear transformation. Suppose that an arbitrary color C is given with respect to a basis of primary colors, {Pi | i = 1, 2, 3}, and can accordingly be represented as a three-component tristimulus vector (C1 , C2 , C3 ): 3 C= Ci Pi . i=1

˜i | i = It is assumed that an old basis {Pi | i = 1, 2, 3} and a new basis {P 1, 2, 3} are related by 3 ˜i , Pj = Mij P i=1

where Mij are the elements of a 3 × 3 matrix. Then, the color (C˜1 , C˜2 , C˜3 ) ˜ i can be expressed in terms of the represented with respect to the new basis P transformation matrix M and the original tristimulus vector: C˜i =

3

Mij Cj ,

j=1

i.e., the linear transformation between different representations is accomplished by the matrix M . In this notation, the depth-cueing scheme of this section can be formulated as follows. As input data, an original color Corig and an admixing color Cmix shall be given with respect to an arbitrary, but fixed basis {Pi }. As most modeling and design software is based on RGB, the basis {Pi } can usually be ˜ i } or the transformation identified with RGB. Furthermore, a second basis {P matrix M have to be supplied. Appropriate choices for the second basis are discussed in the following section.

4.3 Color-Based Depth-Cueing Input RGB color C orig

Admixing RGB color C mix

Transformation to ~ basis Pi

Transformation to ~ basis Pi

179

Linear combination

Transformation to RGB basis

Final color IMAGE: WEISKOPF, ERTL [475]

c 2002 A K Peters, Ltd.

Fig. 4.5. Structure of the color-based depth-cueing scheme. The part with dashed lines (on the right) is evaluated in a preprocessing step, the part with solid lines (on the left and bottom) is processed for each single pixel

Figure 4.5 shows the structure of the depth-cueing scheme. One part of the process has to be performed only once in a preprocessing step (dashed lines), whereas the other part (solid lines) has to be evaluated for each single pixel in the image plane. In the first step of the process, the original color Corig and the admixing color Cmix are transformed into the second basis, Cõrig,i =

3

Mij Corig,j ,

C˜mix,i =

j=1

3

Mij Cmix,j .

(4.1)

j=1

The second step accomplishes a weighted sum of these two transformed colors, C˜dst,i = fi (d) Cõrig,i + gi (d) C˜mix,i ,

(4.2)

where d is some scalar distance measure depending on the distance between camera and object. Typical choices for this distance measure are either the z value (depth value) of the object or the Euclidean distance between camera and object (i.e., similar to radial fog). However, even some abstract “distance” measures may be useful for specific applications, as described in Sect. 4.3.4. Both weight functions fi (d) and gi (d) may differ for each color component i, allowing for a distinct mixture of the two colors. This is one of the important differences, compared to standard intensity depth-cueing, which has equal weights for each color component. In many applications, the weight functions are not independent of each other, but give rise to an affine combination of the two input colors, i.e., fi (d) + gi (d) = 1 ,

180


for all valid values of d. In the third and final step, the blended color Cdst is transformed into the original basis {Pi } by means of the inverse matrix M −1 , Cdst,i =

3

−1 ˜ Cdst,j . Mij

(4.3)

j=1

The actual color of that part of the object is then given by (Cdst,i ). The mixing color is constant during the whole rendering process. Therefore, the corresponding transformation (4.1) needs to be performed only once in a preprocessing step. Contrarily, the input color Corig might change from pixel to pixel and thus the corresponding transformation in (4.1) has to be computed for each pixel. As a consequence, (4.2) and (4.3) have to be evaluated for each pixel as well. 4.3.2 Appropriate Parameters for the Generic Depth-Cueing Scheme How can the above scheme facilitate color-based depth-cueing? What are ap˜ i }, the weight functions fi (d) propriate choices for the intermediate basis {P and gi (d), and the admixing color Cmix ? This section describes some useful sets of parameters for the basis vectors and weight functions and establishes the relationship between these parameters and corresponding physiologically based depth cues. Intensity Depth-Cueing Intensity depth-cueing can be realized by choosing appropriate parameters. Here, colors in the foreground are unaltered; objects further away are reduced in intensity, ultimately fading away into the black background. Therefore, the admixing color has to be completely black, i.e., (Cmix,i ) = (0, 0, 0). The same positive weight function is employed for an affine combination of each color component, i.e., fi (d) = f (d) and gi (d) = 1 − f (d). Typical choices for f (d) represent a linear or exponential behavior within an interval d ∈ [dmin , dmax ], cf., for example, the OpenGL reference [496]. The bounds dmin and dmax can be specified by the user or by the 3D viewer. For ranges outside [dmin , dmax ], f (d) is constant. For intensity depth-cueing, fi (d) is identical for all color components and therefore the weighted sum (4.2) and the basis transformations (4.1) and (4.3) are commutative. Consequently, the basis transformations become superfluous and can be canceled. The weighted sum can be performed with respect to any basis, even in the original color system, for example RGB. This application is usually implemented by standard fogging, but can be represented in the proposed scheme as well.


(a)

(b)

(c)

(d)

181

Fig. 4.6. Comparison of different color-based depth-cueing approaches: (a) the test scene without depth-cueing; (b) intensity depth-cueing; (c) saturation depth-cueing; (d) combination of saturation and softened intensity depth-cueing. See also Color Plate 30

Another, almost identical application just replaces black by white as admixing color. In this way, both intensity and saturation are changed. This approach is especially useful for illustrations on a white background. In another variation of this depth-cueing model, some additional color tone is admixed. Ebert and Rheingans [105], for example, add a bluish tone to enhance the effect of atmospheric depth-cueing. All these approaches have in common an identical weight function, independent of the color component. Figure 4.7 provides a simple visualization of intensity-based depth-cueing. Image (a) represents the original yellow object. Image (b) illustrates the effect of intensity depth-cueing by gradually changing yellow to black from bottom to top. Figure 4.6 shows depth-cueing for a 3D example. Figure 4.6 (a) depicts the original test scene that consists of differently colored spheres. Figure 4.6 (b) illustrates depth-cueing by fading to black. Saturation Depth-Cueing The following depth-cueing approach is based on a saturation gradient and cannot be reproduced by standard fogging. Here, more distant objects are

182


Fig. 4.7. Comparison of color-mapping techniques for depth-cueing: (a) a simple box-shaped “test” object of which the distance to the camera increases from bottom to top; (b) scene with intensity depth-cueing; (c) saturation variations; (d) combination of saturation and softened intensity changes; (e) saturation changes with a superimposed brightness “sawtooth” structure. See also Color Plate 31

rendered in more desaturated colors; however, the intensities of their colors should remain constant. Figure 4.8 illustrates how such a desaturation can be described geometrically in RGB tristimulus space. For clarity of the diagram, the blue axis is removed; it would be perpendicular to the red and green axes. The new color Csat has to lie on a plane that contains the original color Csrc and that is perpendicular to the gray axis. This plane is a hyperplane of (approximately) isoluminant colors. In this way, the intensities of Csrc and Csat are identical. Furthermore, Csrc needs to be shifted toward the gray axis, while preventing any hue change. Both criteria can be met by the following construction. First, a linear interpolation of the original color Csrc and a color on the gray axis, Cgray , is computed. The intermediate result, Cinterpolation , is desaturated, but shows a shift in intensity. This change in intensity is compensated by a subsequent projection of Cinterpolation onto a plane perpendicular to the gray axis. The same desaturation effect can be generated within the generic depth˜ i }, one basis vector, cueing scheme. In an appropriate intermediate basis {P ˜ for example P1 , is collinear with the gray axis. The other two basis vectors ˜ 2 and P ˜ 3 themselves do not need to ˜ 1; P should be chosen perpendicular to P be perpendicular. This construction ensures that intensity is only described ˜ 1 . Hue and saturation are given by a combination of by the coefficient of P ˜ ˜ 3 . The admixing color Cmix is set to an arbitrary the coefficients of P2 and P color on the gray axis, for example, to white or black. The weight function is of the form, f1 (d) = 1 , f2 (d) = f3 (d) = f (d) , where f (d) describes depth dependency, for example, by a linear or exponential behavior (as in the previous intensity depth-cueing approach). An affine combination of colors is employed, i.e., gi (d) = 1 − fi (d). By setting f1 (d)

4.3 Color-Based Depth-Cueing green

183

C gray

C sat

gr

ay

ax

is

C interpolation

C src red

Fig. 4.8. Change of saturation. The new color Csat can be produced by a weighted sum of the original color, Csrc , and a color on the gray axis, Cgray , followed by a subsequent projection onto a plane perpendicular to the gray axis

identically to one, it is ensured that the final color Cdst always has the same intensity as the input color Corig . By choosing gray as Cmix , it is made sure that the final color lies between the original color and the gray axis and that any change in hue is prevented. ˜ i }, To obtain explicit values for the transformation matrix to and from {P ˜ one has to determine an appropriate intermediate basis {Pi } with the aforementioned properties. Since the gray axis stretches from black to white in RGB ˜ 1 is the sum of the RGB space (i.e., from (0, 0, 0) to (1, 1, 1)), the gray vector P ˜ 1 = α 3 Pi . The basis vectors, up to an arbitrary scaling factor α, i.e., P i=1 ˜ 1 and to each other. other two basis vectors may be chosen perpendicular to P One possible choice for the transformation matrix is √ √ ⎞ ⎛ 1 −1 + √3 −1 − √3 1 Mij = ⎝1 −1 − 3 −1 + 3⎠ . 3 1 2 2 The transformation back to RGB color space is based on the inverse √ matrix M −1 . The maximum and minimum values of Mij are 1/3 and (−1 − 3)/3, respectively. In this way, the parameters lie in the fixed-point range [−1, 1] and can be processed by most GPUs. For the implementation on GPUs that do not support floating-point fragment operation (such as NVIDIA GeForce 3/4), the matrix elements have to be mapped from [−1, 1] to [0, 1] before they can be transferred as parameters to the fragment program. Other choices of matrices are also possible. In particular, the definition of the luminous vector Y from the CIE XYZ color space could serve as a basis for the transformation matrix. Figure 4.7 (c) provides a simple visualization of saturation-based depthcueing. The weight function is almost identical to the weight function for intensity depth-cueing in Fig. 4.7 (b), the only difference is that the first component is f1 (d) = 1. Analogously, Fig. 4.6 (c) shows the 3D test scene with saturation-based depth-cueing. On the one hand, the saturation depth

184


cue is not as strong as the intensity depth cue, Fig. 4.6 (b). On the other hand, a saturation gradient does not influence intensity contrast and therefore is able to visualize even background objects. These far-away objects are not visible with intensity depth-cueing. Hue Gradient So far, depth cues based on brightness and saturation have been presented. The third psychological term is the hue of a color. Therefore, it seems to be obvious that a hue gradient could be used as a depth cue as well. Experimental findings [427], however, indicate that a red–green hue gradient does not affect depth perception. Therefore, hue gradient is not considered as a direct depth cue in this chapter. Hue changes, however, could make sense in applications that visualize features in a more abstract way. For example, hue changes could be determined by other means than the geometric distance, such as features or semantics of the scene, in order to visualize these features. Combination of Depth Cues In addition to pure intensity or saturation gradients, flexible combinations of these cues can be readily combined within the proposed framework. By choosing different functions for f1 (d) and for f2 (d) = f3 (d), a great variety of combinations of intensity and saturation depth-cueing can be achieved. For example, ranges and scales for the weighting functions can be different, so that saturation cues would be effective in the proximity of the camera and intensity fading would be prominent in distant regions. Figure 4.7 (d) visualizes the combination of saturation-based and intensitybased depth-cueing. Here, the weight function is given by f1 (d) = 1 − (1 − f (d))/2 and f2 (d) = f3 (d) = f (d), where f (d) is the weight function used for intensity depth-cueing in Fig. 4.7 (b). Figure 4.6 (d) shows the same depth cue for the 3D test scene. The weight function has parameters analogous to those in Fig. 4.6 (b). In Fig. 4.6 (d), the dimming effect is only half of the desaturation effect, combining the advantages of both approaches: the depth cue is stronger than in mere saturation-cueing and, in contrast to intensitycueing, background objects are still visible. Fig. 4.7 (e) shows an example of a quite complex combination of intensity and saturation cues. A saturation gradient as of Fig. 4.7 (c) is superimposed by a “sawtooth” function for intensity depth-cueing, i.e., the weight function f1 (d) has recurrent maximum and minimum values that are linearly interpolated. The resulting image allows the user to identify areas of equal distance from the camera. In this way, depth-cueing can even facilitate quantitative distance measures. Sect. 4.3.4 illustrates how this choice of parameters supports depth perception in terrain visualization.


185

4.3.3 Implementation Although the presented depth-cueing scheme is quite flexible, the required arithmetic operations are – computationally – not very demanding and low in number. Only a few multiplications and summations are necessary and, for example, no division is needed. Therefore, the depth-cueing scheme can be implemented on moderately powerful graphics hardware by using vertex and fragment programs. It can be implemented transparently to the user or programmer of a graphics viewer; a 3D viewer just needs to be extended by an additional initialization step that loads a vertex program and a fragment program, as described below. The test implementation is based on the OpenSceneGraph API [315] and requires only minimal extensions to the standard OpenGL renderer. Moreover, the implementation comes with (almost) no additional rendering costs and therefore allows for interactive applications. The only assigned task for the vertex program – besides standard transform and lighting calculations – is to determine the depth value d of a vertex and set the texture coordinate of a one-dimensional texture to this depth value. For the calculation of the Euclidean (radial) distance between camera and vertex, for example, only four additional vertex program commands are necessary. Even simpler is the computation of the standard depth value, i.e., the distance perpendicular to the image plane. The next change affects the fragment shader. Here, one additional texture lookup for the three components, fi (d), of the weight function has to be included if an affine combination of colors is implemented. If fi (d) and gi (d) are independent of each other, one further lookup for gi (d) is required. Texture coordinates are used to transfer depth information from the vertex program to the fragment program because GPUs may not provide direct access to the depth value within a fragment program. The second task of the fragment program is the transformation and blending of a fragment color according to (4.1), (4.2), and (4.3), which can be directly mapped to numerical per-fragment operations. An implementation on an NVIDIA GeForce 3 GPU is described by Weiskopf and Ertl [475]. This implementation is based on vertex programs, multi-textures, and register combiners. Here, special attention is paid to the restricted programmability of the fragment unit and the limited accuracy and numerical range in register combiners. For example, the negative values in the matrices M and M −1 (see Sect. 4.3.2) are remapped to the interval [0, 1] at intermediate stages of the pipeline. All example images in the following section were generated by this GPU implementation for NVIDIA GeForce 3. The frame rates for the depth-cueing application are identical to those of the original OpenSceneGraph viewer for all example scenes. This shows that the additional operations in the transform and lighting part and the fragment unit do not impair rendering performance in typical applications. Only for scenes with extreme rasterization demands – such as texture-based volume rendering – the additional operations in the

186


fragment unit could decrease the overall frame rate, depending on the actual properties of a GPU. In an alternative approach, the first transformation step for Corig according to (4.1) is detached from the fragment program and transferred to a vertex program. Rasterization is always based on a triangulated surface; and colors are interpolated linearly within a triangle during scanline conversion. As linear interpolation and a linear transformation via a matrix are commutative, colors may very well be transformed before scanline conversion. In this way, the fragment stage could be unburdened and many unnecessary calculations could be avoided – at the cost of only three transformations per triangle within the transform and lighting part of the rendering pipeline. 4.3.4 Applications This section demonstrates the aptitude of the color-based depth-cueing scheme by means of various applications in non-photorealistic rendering and visualization. These examples are, of course, not exhaustive, but show what type of applications can benefit from color-based depth cues. Figure 4.9 demonstrates depth-cueing for the non-photorealistic rendering of technical illustrations. Figure 4.9 (a) shows the original engine block. In Fig. 4.9 (b), cool/warm tone shading [137] is applied to enhance the perception of surface orientation. A cool (blue) to warm (tan) transition of color tones indicates a change of the surface normal from left to right. Figure 4.9 (c) illustrates cool/warm shading and saturation depth-cueing, and Fig. 4.9 (d) illustrates cool/warm shading and intensity depth-cueing. The main differences between Figs. 4.9 (b)–(d) appear in furthermost parts of the engine, especially in the upper portion of the images. Saturation depth-cueing causes rather subtle color changes, whereas intensity depth-cueing harshly affects the image by completely fading away parts of the engine. This example demonstrates that saturation depth-cueing is close to Tufte’s strategy of the smallest effective difference, making visual distinctions as subtle as possible, but still clear and effective. In particular, non-photorealistic illustrations benefit from saturation depth-cueing because all structures – even in the background – are retained. Figure 4.10 shows a typical application in information visualization. Here, a large amount of hierarchically structured data is visualized by a cone tree [348]. The elements of the tree are represented by boxes and cones, connected by thin lines. These graphical objects are located and viewed in 3D space. Therefore, depth perception is quite important for the understanding of the tree structure. The original rendering in Fig. 4.10 (a) includes only perspective and occlusion as depth cues. With the saturation depth cue being enabled in Fig. 4.10 (b), the depth structure becomes much clearer. The top and bottom images in the center of Fig. 4.10 show magnified details of the respective renderings. The in-between image provides a comparison between


(a)

(b)

(c)

(d)

187

Fig. 4.9. Non-photorealistic rendering of an engine block with different depth-cues: (a) original scene; (b) cool/warm shading without depth-cueing; (c) cool/warm shading with saturation depth-cueing; (d) cool/warm shading with intensity depthcueing. See also Color Plate 32

4 Perception-Oriented and Non-Photorealistic Rendering 188

(a)

(b)

Fig. 4.10. Depth-cueing for a cone tree representation in information visualization. The left image shows the original scene; in the right image, saturation depth-cueing is applied to the same scene. The top and bottom images in the middle show magnified details; the central image visualizes the difference between the two detail images. See also Color Plate 33


(a)

(b)

(c)

(d)

189

Fig. 4.11. Terrain rendering with color-based depth-cueing: (a) original scene; (b) saturation depth-cueing; (c) intensity depth-cueing; (d) combination of saturation depth-cueing and “sawtooth” intensity depth-cueing. See also Color Plate 34

these two detail images, showing the differences of saturation levels of the two images as gray-scale values. Figure 4.11 demonstrates depth-cueing for a terrain rendering example. Figure 4.11 (a) shows the original scene, Fig. 4.11 (b) shows the scene with saturation depth-cueing, and Fig. 4.11 (c) shows the scene with intensity depthcueing. In all images of Fig. 4.11, depth-cueing is only applied to the terrain itself, the background sky and ocean keep their original colors. In Fig. 4.11 (d), saturation depth-cueing is combined with a “sawtooth” function for intensity depth-cueing, analogously to the parameters for Fig. 4.7 (e). The resulting image allows us to identify lines of equal distance from the camera. In this way, depth-cueing even facilitates quantitative distance measures. This technique is related to classical contour lines (isolines) in topographical relief maps and to enridged contour maps [442] used for height field visualization. It is also influenced by works by the Flemish painter Jan Brueghel the Elder (1568–1625), who enhances depth perception by alternating darker and lighter areas, as in [36]. The “sawtooth” approach is especially useful for extended objects like terrains. However, it is less suited for scenes consisting of many small and separate objects because the context of lines of equal distance is destroyed by blank regions between objects.

190

4 Perception-Oriented and Non-Photorealistic Rendering Fig. 4.12. Semantic depth-cueing: (a) traditional 3D view on a chess board; (b) some chessmen emphasized by highly saturated colors; (c) combination of semantic and geometric depth-cueing. See also Color Plate 35

(a)

(b)

(c)

In Fig. 4.12, saturation depth-cueing is applied to an abstract “distance” measure. Figure 4.12 (a) shows a traditional 3D view on a chess board. In Figs. 4.12 (b) and (c), some chessmen are accentuated by using highly saturated colors, whereas the less important objects appear less prominent due to desaturated colors. Figure 4.12 (b) shows an extreme case: the highlighted objects are fully saturated and the “background” objects are only in grayscale. Here, the weight functions f2,3 (d) take only the two extreme values 1

4.4 Non-Photorealistic Rendering of Depth Structures

191

and 0 for the “foreground” and the “background” objects, respectively. In the depth-cueing framework, two different depth values are assigned to the two groups of objects, depending on their semantic value. In this approach, the “depth” value is determined by the user or some other outside source, but not by purely geometric quantities. This type of visualization could be called semantic depth-cueing, similarly to the techniques by Kosara et al. [224] for semantic depth-of-field. Figure 4.12 (c) illustrates that semantic depth-cueing can be combined with geometric depth-cueing. The “foreground” objects are still completely saturated. The saturation of the “background” objects, however, is controlled by their Euclidean distance to the camera. On the one hand, this combined approach conveys more information than mere semantic depth-cueing. On the other hand, the mapping from saturation value to semantic value is no longer unique. Therefore, the benefit of the variations of semantic depthcueing largely depends on the available colors, semantics, and depth structure of the scene. As another application, depth-cueing can be applied to non-photorealistic volume rendering. Ebert and Rheingans [105] use a standard fogging approach for depth-cueing in volume illustrations, which could be replaced by the more flexible depth-cueing scheme of this section. A further field of application is image-based rendering (IBR). It was shown that real-time processing and rendering of image-based representations can be achieved by using GPUs [39, 82, 378]. Sophisticated depth cues could improve the perception of spatial structures in this context. As a closely related application, range images could also be visualized. In a completely different setting, saturation control is required in virtual reality facilities. In an immersive virtual environment like a CAVE, maximum luminance values are low because projection devices are not able to produce very bright images and stereo glasses further reduce intensity. The user is often limited to viewing conditions in the mesoscopic range, where color perception by the HVS is significantly reduced [380]. Therefore, colors are perceived in low saturation in such VR environments. The depth-cueing scheme of this chapter allows us to increase the saturation of colors by supplying negative values and values larger than 1 in the weight functions, i.e., by shifting colors further away from the gray axis. In this way, the saturation of the displayed colors can be adjusted to the viewing conditions in real time.

4.4 Non-Photorealistic Rendering of Depth Structures via Continuous Tone-Shading This section addresses the non-photorealistic rendering of objects with complex depth structure. The goal is to visualize more than just the foremost surface of an object and reveal the spatial relationships inside the object. In general, one of the major advantages of non-photorealistic illustrations, in contrast to photorealistic renderings or actual photographs, is that they

192


can provide a selective view on important details while extraneous details are omitted [94]. Previous research on computer-generated illustrations focused on imitating different rendering styles traditionally used by illustrators. However, these computer-based techniques only communicate some aspects like shape, structure, or depth information, but often neglect complex spatial relationships and the issue of occlusion. In this section, algorithms for the visualization of spatial structures are derived from well-established knowledge for drawing manual illustrations [170, 274, 416]. Traditional methods are evaluated to extract a small and effective set of rules for computer-based non-photorealistic illustrations; the focus lies on methods that build upon tone-shading for continuous surface rendering (halftoning is discussed later in Sect. 4.5). The first rendering approach of this section adopts semi-transparency to show spatial relationships between objects, as proposed by Diepstraten, Weiskopf, and Ertl [88]. Traditional drawing techniques are briefly reviewed in Sect. 4.4.1 to derive rules for a rendering algorithm in Sect. 4.4.2. Here, the concept of view-dependent transparency, which is widely applied in traditional technical illustrations, is introduced into the field of computer graphics. Two different GPU-based depth-sorting algorithms are described to meet the requirements of this transparency model. The second approach allows the viewer to have a look into an otherwise solid opaque object by means of cutaway illustrations. In these drawings, parts lying inside or going through an opaque object are of more interest than the surrounding object itself. Instead of letting the inner object shine through the girdling surface, parts of the exterior object are removed, which produces a visual appearance as if someone had cutout a piece of the object or sliced it into parts. Sect. 4.4.3 gives an overview of cutaway illustrations and describes rules for corresponding rendering algorithms. Two slightly different types of cutaway illustrations are discussed in detail, along with algorithms for real-time GPU rendering: cutout drawings in Sect. 4.4.4 and breakaway illustrations in Sect. 4.4.5. Both rendering methods were introduced by Diepstraten, Weiskopf, and Ertl [89]. Note that cutaway drawings are related to clipping techniques in volume rendering (Sect. 2.5) as both approaches remove parts of an object. Performance characteristics of all above rendering techniques are discussed in Sect. 4.4.6. Performance measurements show that GPU implementations lead to interactive computer-based illustration tools that provide a variety of methods for visualizing complex depth structure. Finally, Sect. 4.4.7 demonstrates that semi-transparent NPR and cutaway illustrations can be adopted and extended for volume illustrations.


193

4.4.1 Traditional Visualization of Semi-Transparency in Illustrations Previously, transparency has not been addressed extensively in the field of non-photorealistic rendering. The paper by Hamel et al. [150] is one of the few other works that specifically deal with transparency in NPR. Hamel et al. concentrate on transparency in line drawings, whereas this section focuses on illustrations that consist of smoothly shaded surfaces. The following presentation adopts the description by Diepstraten, Weiskopf, and Ertl [88].3 In this section, guidelines are first extracted from books that describe artistic techniques for technical and scientific illustrations [170, 274, 416]. These rules serve as basis for handling transparency to communicate the location of occluding and occluded objects. A simple method would be to draw only the outlines of transparent objects. These outlines are rendered in a line style different from the other outlines to make them distinct from the opaque objects. This line style is often described as phantom lines [416], as shown in Fig. 4.13. Although this technique can be applied to a wide variety of drawing styles, ranging from color illustrations to simple line or sketch drawings, there are certain drawbacks. First, details of the transparent objects are lost as only their outlines are drawn. Second, material and surface information of transparent objects is ignored. Finally, there are only two transparency states: fully opaque or fully non-opaque; semi-transparency cannot be visualized. A different approach can be found in “The Guild Handbook of Scientific Illustration” by Hodges [170]. For color illustrations, Hodges recommends light-

IMAGE: DIEPSTRATEN, WEISKOPF, ERTL [88]


Fig. 4.13. Phantom lines showing semi-transparent objects (see also Color Plate 36) 3

c The Eurographics Association 2002. Reprinted, with permission, from Diep straten, Weiskopf, and Ertl [88]

194


ening the color of object regions occluded by transparent objects and proposes the following basic rules: • •

Strengthen the shade where an opaque object enters a non-opaque object. Set the intensity of the opaque object to zero at the edge of the surrounding object and slowly increase its intensity with increasing distance from the edge.

To put it another way, transparency falls off close to the edges of transparent objects and increases with the distance to edges. Besides these fundamental rules for visualizing transparent objects, there are others which are not directly described by Hodges and which focus on the correlation between objects. These precepts may be easily handled by an illustrator, but they cannot be directly transformed into rules that are appropriate for a computer-based implementation. By analyzing color-shaded manual drawings, however, the following simplified rules can be identified: • • •

Back faces or front faces of the same non-opaque object never shine through. Opaque objects occluded by two transparent objects do not shine through to the closer transparent object. Two transparent objects are only blended with each other if they do not distract the viewer or if they are very close to each other and belong to the same semantic group.

This set of rules is based on the fact that in illustrations transparency is used to look into objects and to show objects that lie inside or go through nonopaque ones. Often these interior objects are opaque. As semantic grouping cannot be achieved without additional user interaction in pre-processing or post-processing steps, the last rule is changed to: •

Two transparent objects never shine through each other.

4.4.2 Semi-Transparency in Computer-Generated Illustrations From the above traditional illustration methods, the following small and effective set of rules is extracted for rendering semi-transparent illustrations: [R1] [R2] [R3]

Faces of transparent objects never shine through. Opaque objects occluded by two transparent objects do not shine through. Transparency falls off close to the edges of transparent objects and increases with the distance to edges.

Based on these rules, an algorithm for semi-transparent rendering can be formulated. First, the objects that are blended have to be determined, following the first two rules [R1] and [R2]. This task essentially corresponds to viewdependent spatial sorting. An efficient GPU-based solution to this problem is


195

described later in this section. Second, transparency values have to be computed for a correct blending between transparent and opaque objects according to the third rule [R3]. In what follows, opacity α replaces transparency T according to α = 1 − T . View-Dependent Transparency The concept of view-dependent transparency is introduced to implement the third rule: the alpha value for blending between transparent and opaque objects depends on the distance to the outline of the transparent object; the outlines of an object projected onto a 2D screen consist of silhouettes lines, which are view-dependent. Silhouette edges can either be determined in 2D image space or in 3D object space. Hamel et al. [150] recommend using an image-space method by establishing an object ID buffer and an edge extraction filter. However, this method is time-consuming as you have to render the objects with an ID tag, find the edges, vectorize these edges to 2D lines, and – with the help of these lines – calculate the distance of each pixel to the outline. In this section, a 3D object-space approach is pursued instead. All silhouettes are determined before rasterization to avoid most of the previously mentioned steps. In 3D space, a silhouette is an edge connecting two faces, where one face of the edge is back-facing and the other one is front-facing. This classification can be formulated as (n1 · (p − o))(n2 · (p − o)) < 0 , where p is an arbitrary, yet fixed point on the edge, ni are the outward-facing surface normal vectors of the two faces sharing the edge, and o is the position of the camera. In the current implementation, all edges are checked for the above criterion and the detected silhouette edges are stored in a list, based on Appel’s original algorithm [3]. The performance of silhouette detection could be improved by more sophisticated techniques, such as fast back face culling [506], the Gauss map method [139], or by exploiting frame-to-frame coherence. Then, the distance of each vertex to the closest silhouette edge is computed. First, the distances d of a vertex to all silhouette edges are determined according to (v − p) · e , e v − p + d= e where v is the position of the vertex, p is an arbitrary, yet fixed point on the silhouette edge, and e is the silhouette edge. The minimum distance of a vertex to all silhouettes, dmin , is used to calculate the alpha value for the respective vertex of the transparent object: α=1−

dmin dobject,max

k ,

(4.4)

196


(a)

(b) IMAGES: DIEPSTRATEN, WEISKOPF, ERTL [88]


Fig. 4.14. Comparing transparency models: (a) traditional view-independent transparency; (b) view-dependent transparency. See also Color Plate 37

where dobject,max denotes the maximum distance between the center of the object and all surface points; k is a user-specified fall-off term. The above alpha value is used as blending weight for the transparent surface and another, fixed value is used as weight for the opaque object. Therefore, the alpha value of the opaque object determines this object’s weight in the final image. Note that the background color itself may be blended with transparent surfaces as well. Different alpha values for opaque objects and for the background can be specified by the user. A value α = 0.5 for the background is a good choice because it makes sure that the background color does not shine completely through non-opaque objects. Figure 4.14 compares view-dependent transparency to traditional view-independent transparency. Depth Sorting with Implicit Interior Boundaries The next step needs to determine which objects have to be blended and which objects are hidden and do not contribute to the final image. In full generality, rendering semi-transparent scenes would require view-dependent depth sorting. The two main approaches to depth sorting are either based on image space or on object space. Here, an image-space algorithm is chosen because it uses graphics hardware to efficiently perform depth sorting. Image-space algorithms also benefit from the fact that this specific application does not require complete depth sorting – only the closest transparent objects and the opaque objects directly behind the corresponding transparent surfaces have to be determined. In the following discussion, volumetric objects are assumed to be defined by boundary surface representations. In a first, quite common scenario, an opaque object may be contained inside the volume of a transparent object, as illustrated in Fig. 4.15. Such an opaque object should be visualized by transparent rendering. Here, the interior boundary of the transparent volume is given implicitly by the surface of the surrounded opaque object. First, only convex transparent objects are

4.4 Non-Photorealistic Rendering of Depth Structures camera

197

camera

opaque

blocking back face

semi−transparent

(a)



Fig. 4.15. Scenario with opaque objects embedded in the volume of a transparent object. The left image shows a convex transparent object, the right image shows a concave transparent object

considered, as shown in Fig. 4.15 (a), while opaque objects may be of arbitrary shape. Following the aforementioned rules, the front-facing surface of the transparent object closest to the camera has to be blended with opaque objects contained within the volume of this transparent object. All objects – both opaque and transparent – which are located further away should be hidden by this nearest transparent object. Note that these invisible objects are not included in Fig. 4.15 (a). Finally, opaque objects in the foreground are drawn on top of all other objects that are further behind. In a slightly more complicated scenario, concave transparent objects are considered, as sketched in Fig. 4.15 (b). Surrounded opaque objects should be visible only up to the first back-facing part of the transparent object in order to blend out unnecessary and distracting visual information. To put it another way, only those opaque objects are visible which are closer than the nearest back face of the surrounding transparent object. In Fig. 4.15 (b), the circular opaque object is visible, whereas the squared object is hidden by a back-facing surface of the surrounding object. Figure 4.16 shows the rendering pipeline for this scenario. The depth buffer is used to select objects for blending and to hide unwanted objects. In the first part, back faces of all transparent objects are rendered into the depth buffer. In this way, the depth buffer contains the distance of the closest back-facing transparent surfaces. In the second part, the front faces and the silhouettes of all opaque objects are rendered to both frame and depth buffers. The depth test rejects all opaque objects lying behind any “transparent” back-facing surface. Finally, the front faces and silhouettes of the transparent objects are rendered and blended into the depth and frame buffers, respectively. Here, the depth test rejects all parts of the transparent objects hidden by an opaque

198

4 Perception-Oriented and Non-Photorealistic Rendering clear depth/frame buffer

render back faces of transparent objects to depth buffer render front faces/silhouettes of opaque objects to depth/frame buffer blend front faces/silhouettes of transparent objects to depth/frame buffer IMAGE: DIEPSTRATEN, WEISKOPF, ERTL [88]

Fig. 4.16. objects


Rendering pipeline for opaque objects embedded inside transparent

foreground object. Blending is applied only in those regions of the frame buffer where a transparent surface is visible. Depth Sorting with Explicit Interior Boundaries In a more complex scenario, objects may no longer be contained inside the volume of another object. In fact, surrounding transparent objects are modeled with respect to both the outside and the inside boundary. Figure 4.19 illustrates this scenario for the example of an opaque cuboid inside a transparent cylindrical, mug-shaped object; Fig. 4.17 (a) shows a color-shaded rendering of a similar scene. The above algorithm for implicit interior boundaries fails for this scenario, as all surrounded opaque objects are hidden by a back-facing transparent surface. To overcome this problem, another classification of the visibility of opaque objects is necessary: only those opaque objects located between the closest and second-closest front-facing transparent surfaces are visible; objects (transparent or opaque) further behind are hidden. The corresponding rendering algorithm is as follows. In the first step, all front-facing transparent surfaces are rendered to the depth buffer; afterwards, the depth buffer contains the depth values of the closest transparent surfaces. As second step, the depth buffer is stored in a high-resolution texture and then the depth buffer is cleared. In the third step, the front-facing transparent surfaces are rendered to the depth buffer except for the foremost ones, virtually peeling off the closest surfaces. After this step, the second-closest transparent front faces are stored in the depth buffer. The principal idea behind this step is closely related to depth-peeling [116] or the multi-pass approach to volume clipping with concave clip objects (Sect. 2.5.1). In the fourth step, opaque


(a)

(c)

199

(b)

(d) IMAGES: DIEPSTRATEN, WEISKOPF, ERTL [88]


Fig. 4.17. Semi-transparent illustration: (a) semi-transparent mug with two boundaries surrounding an opaque box, based on explicit interior boundaries; (b, c) multiple semi-transparent mugs; (d) crank within a semi-transparent cylinder. See also Color Plate 38

objects are rendered. The depth test rejects all surfaces but those lying in front of the second-closest transparent front faces. Finally, just the foremost transparent surfaces are blended into the frame buffer. This algorithm can be directly mapped to Shader Model 2 compliant GPUs, which support comparison operators in fragment programs and floating-point textures to hold the depth values. Details of an implementation for NVIDIA GeForce 3 are described by Diepstraten, Weiskopf, and Ertl [88]. Figures 4.17 and 4.18 show examples of semi-transparent illustrations. Figure 4.17 (a) displays a transparent mug with two boundaries surrounding an

200


(a)

(b)

(c)



Fig. 4.18. Semi-transparent illustration. Pictures (a) and (b) show different parts of a Lancia engine block as a typical example of a technical illustration. Image (c) depicts a semi-transparent rendering of the scene from Fig. 4.13 (with phantom lines). Image (d) shows a simplified wheel axis. See also Color Plate 39

opaque box; Figs. 4.17 (b) and (c) show a similar scene, including multiple transparent mugs. Figure 4.17 (d) depicts a crank within a transparent cylinder. Figures 4.18 (a) and (b) show different parts of a Lancia engine block as a typical example of a technical illustration. Finally, Figs. 4.18 (c) and (d) display another engine and a simplified wheel axis, respectively.


201

exterior boundary interior boundary

exterior boundary

interior boundary cutting plane

opaque



Fig. 4.19. Scenario with an opaque cuboid inside a transparent mug-like object of which the interior boundary is explicitly modeled. The left image shows a side view with a horizontal cutting plane, the right image shows a top view onto the cutting plane

4.4.3 Overview of Cutaway Illustrations Cutaway techniques provide an alternative means of visualizing the depth structure of complex 3D objects. This section briefly reviews how cutaway illustrations are traditionally created by illustrators to extract requirements for an automatic generation process. The following description adopts the presentation by Diepstraten, Weiskopf, and Ertl [89].4 More detailed background information on traditional hand-made illustrations in general and cutaway drawings in particular can be found in classical art books [274, 416]. The purpose of a cutaway drawing is to allow the viewer to have a look into an otherwise solid opaque object. Instead of letting the inner object shine through the surrounding surface, parts of the outside object are simply removed. From an algorithmic point of view, the most interesting question is where to cut the outside object. The answer to this fundamental question depends on many different factors, for example, size and shape of the inside and outside objects, semantics of objects, personal taste, etc. Many of these factors cannot be formalized within a simple algorithm and need some user interaction. Nevertheless, some interesting common properties can be found in many examples of traditional cutaway drawings, which leads to an automatic process to generate quite reasonable cutaway illustrations. Two different subclasses of the general notion of a cutaway drawing can be distinguished: cutout illustrations and breakaway illustrations. Fig4

c The Eurographics Association 2003. Reprinted, with permission, from Diep straten, Weiskopf, and Ertl [89]

202


(a)



Fig. 4.20. Comparison of computer-generated cutout and breakaway illustrations. The left image demonstrates the cutout technique with a jittering boundary. In the right picture, the breakaway method is applied to the same scene. See also Color Plate 40

ures 4.20 (a) and 4.20 (b) compare the two subclasses, based on an identical input scene. A cutout removes a considerable part of the outside object to show the interior parts. Artists and illustrators tend to restrict themselves to simple and regularly shaped cutout geometries. Only a small number of planar slices is often cut into the outside object; just two planes are sufficient in many cases. The location and orientation of the cutting planes are determined by the spatial distribution of the interior objects and, more importantly, by the geometry of the outside body. Just enough is taken away from the outside object to allow the observer to view internal details. Many cutout illustrations have been analyzed to come to the conclusion that two planes intersecting at an angle between 90 to 140 degrees are sufficient for a wide class of applications. Another common property describes the location of the slicing planes. The cut through the object of interest often takes place at or around the object’s main axis, which is the axis with the greatest spread of the object geometry. A cutout in an illustration has not always to be smooth. For example, sawtooth-like or jittering cutting is often applied to better distinguish between outer and inner objects and produce a higher level of abstraction. Figure 4.20 (a) shows such a jittering cutout for a simple example scene. The cutout approach is particularly useful when many objects or large objects are inside, covering a large portion of the interior of the girdling object. In contrast, if only a few small interior objects lie densely to each other, another approach is more appropriate: a breakaway drawing. Here, an illustrator breaks a virtual hole into the boundary to show the interior objects. This boundary hole should be just wide enough to see these objects. Figure 4.20 (b) shows a simple example of a breakaway illustration. It has already been stated that hand-made illustrations are influenced by various aspects of which many are hard to be formalized for computer-based processing. Nevertheless, a small number of rules that lead to quite convincing,


203

fully automatic cutaway drawings have been extracted. In addition, the user is still able to change parameters after the initial automatic construction. Cutout Illustration The rules for the cutout approach are discussed first. A basic question is: which objects are potentially subject to cutting? Interior objects are usually not sliced by the cutout geometry; cutting is only applied to outside objects. Therefore, the first requirement is: [R1]

Inside and outside objects have to be distinguished from each other.

Please note that this requirement not only covers scenes with a single outside object, but also scenarios with several disjoint outside objects and even nested layers of outside objects. Another issue is the shape of the cutout geometry. Only a restricted class of shapes is used: [R2]

The cutout geometry is represented by the intersection of (a few) half spaces (A half space can be represented by the plane that separates the space from its complement).

By construction, this cutout geometry is convex. Good results are often achieved by: [R2’] A cutout geometry is represented by the intersection of two half spaces. The next rule determines the position of the cutout geometry: [R3]

The cutout is located at or around the main axis of the outside object.

For a cutout with two planes, the intersection line between the two planes lies on the main axis of the outside object. The angle of rotation around this axis is a parameter that can be adjusted by the user. A cutout does not always have to be as smooth as the geometry defined by a collection of half spaces. For example, a sawtooth-like or jittering cutting is often applied to such a simple cutout geometry: [R4]

An optional jittering mechanism is useful to allow for rough cutouts.

Finally, a cutout produces new parts of the objects’ surfaces at the sliced walls. This leads to: [R5]

A mechanism to make the walls visible is needed.

This requirement is important in the context of boundary representations (BReps) of scene objects, which do not explicitly represent the solid interior of walls. Therefore, special care has to be taken to compute a correct illumination of cutout walls.

204


Breakaway Illustration The alternative cutaway approach in the form of a breakaway illustration is based on a slightly different set of rules. The first requirement [R1] for a distinction between inside and outside objects is the same as in cutout drawings. However, the shape and position of the breakaway geometry is not based on rules [R2] and [R3], but on: [R6]

A breakaway illustration should be realized by a single hole in the outside object.

If several small openings were cut into the outside surface, a disturbing and complex visual appearance would be generated. Nevertheless, [R7]

All interior objects should be visible from any given viewing angle.

The above rule for making the walls visible [R5] can be applied to breakaway illustrations as well. Jittering breakaway illustrations are seldom and therefore [R4] is not a hard requirement for these illustrations. The following two sections present two different rendering algorithms that meet the above characteristics for cutout and breakaway illustrations, respectively. 4.4.4 Cutout Drawings Rendering algorithms for cutout drawings are presented in this section. It is shown how a computer-based process can fulfill rules [R1]–[R5]. Classification It is assumed that the classification of objects as interior or exterior [R1] is provided by an outside mechanism. The problem is that a generic classification criterion solely based on the spatial structure of a scene is not available. For special cases, objects can be recognized as being inside or outside by observing their geometry. For example, the important class of nested surfaces can be handled by an algorithm by Nooruddin and Turk [310]. However, this method does not support exterior objects that already have openings before the cutting process is performed. A different approach is taken in this section. The objects are stored in a scene graph structure; the classification is based on an additional Boolean attribute that is attached to each geometry node of the scene graph. For practical purposes, geometric modeling of scenes is performed in an outside, commercial modeling and animation tool (such as 3D Studio Max or Maya) and the scenes are afterwards imported via a 3D data file. Since these modeling tools do not directly support the additional classification attribute, other information stored in the 3D file format has to be exploited. For example, the opacity value can be (mis)used, or the classification is coded in the form of


205

a string pattern in the name of 3D objects. This approach allows the user to explicitly specify interior and exterior objects and to introduce some external knowledge into the system. Main Axis Another issue is the computation of the main axis of an outside object that is typically represented by a triangulated surface. Information on the connectivity between triangles is not required, i.e., a “triangle soup” can be used. The algorithm makes use of first-order and second-order statistics that summarize the vertex coordinates: the mean value and the covariance matrix [99]. The algorithm is identical to Gottschalk et al.’s method for creating object-oriented bounding boxes (OBBs) [141]. If the vertices of the i’th triangle are the points pi , qi , and ri , then the mean value µ and the covariance matrix C can be expressed as 1 i (p + qi + ri ) , 3n i=1 n

µ=

1 i i i i i i (p j p k + q j q k + r j r k ) , 3n i=1 n

Cjk =

where n is the number of triangles, p = pi − µ, q = qi − µ, r = ri − µ, and Cjk are the elements of the 3 × 3 covariance matrix. Actually, not the vertices of the original mesh are used, but the vertices of the convex hull. Moreover, a uniform sampling of the convex hull is applied to avoid potential artifacts caused by an uneven distribution of triangle sizes. These improvements are also described by Gottschalk et al. [141]. The eigenvectors of a symmetric matrix with different eigenvalues are mutually orthogonal. Therefore, the eigenvectors of the symmetric covariance matrix can be used as an orthogonal basis. Of special interest is the eigenvector corresponding to the largest eigenvalue because it serves as the main axis of the object. i

i

i

CSG Cutout Finally, the cut geometry has to be defined and then applied at the previously determined location. An object-space approach could be realized by techniques known from constructive solid geometry (CSG). The intersection of half spaces [R2] can be implemented by a CSG intersection operation working on half spaces. An intersection operation is equivalent to a logical “and” applied to the corresponding elements of the spaces. A half space can be represented by the plane that separates the space from its complement. Therefore, this plane serves as a slicing plane in the cutout approach. The half space is defined in a way that outside objects are removed at all locations of this half space; outside objects are left untouched in the complementary space. The actual cutting process is modeled by a CSG difference operation applied to

206


the cutout geometry and the geometry of the exterior objects. Intrinsic to all CSG operations is the creation of new boundary surfaces at cuts. Therefore, cutout walls are automatically modeled and can be rendered afterwards, as required by [R5]. In the general approach of rule [R2], the number, locations, and orientations of cutting planes have to be defined by the user. The more restricted rule [R2’] fixes the intersection line between the two cut planes along the main axis of the outside object. The only parameters are the relative angle between the planes and the angle of rotation of the cutout geometry with respect to the main axis. The relative angle can be set to a default value between 90 and 140 degrees (e.g., to 110 degrees); the default orientation with respect to the main axis can be set to any fixed angle. With these initial values, quite good results are achieved without user interaction. For an optional sawtooth-like or jittered cutout [R4], the cutout geometry has to be perturbed, for example, by a displacement mapping technique [61]. Appropriate displacement maps are presented in the description of texture-based cutouts. Although all requirements [R1]–[R5] can be directly mapped to a CSGbased implementation, this approach is devaluated by the fact that CSG operations can be very time-consuming. Therefore, parameter changes are unlikely to work in real time and an interactive application is not possible. This is a major drawback because cutout drawings – even if they work almost automatically – need some user interaction to adjust parameters for improved final results. If highly detailed and jittered cutout geometries or complex exterior objects are used, CSG operations become particularly time-consuming. Another issue is rendering time itself. A high number of new primitives may be introduced by the re-tesselation steps required for accurate intersections between objects. A large amount of new primitives could prevent interactive rendering and might require further object optimization steps. All these aspects limit the applicability of the CSG approach for an interactive application. Therefore, alternative methods are pursued in the following discussion. They are based on image-space calculations and make use of graphics hardware to accelerate rendering. Planar Cutout The first image-space approach of this section is based on the concept of clipping planes and allows for piecewise planar, convex cutouts according to [R2]. Each planar element of the cutout geometry is identified by a clipping plane. The exterior object is rendered n times, where n is the number of different planes. In each rendering pass, the respective clipping plane is activated. Afterwards, the interior objects are rendered in a single pass, with clipping planes being deactivated. The advantages of the clipping-plane based method are a rather simple implementation and the support by virtually any graphics hardware. A drawback is the increase in rendering costs for multiple rendering passes – especially for


207

more complex cutouts with several cutting planes. Another issue is the restriction to smooth cutouts. Jittering boundaries according to [R4] are not possible. Since no explicit modeling of the cutout surface is implemented, the wall cannot be made visible [R5]. Cutout via Stencil Test The following image-space technique uses the stencil buffer and stencil test to represent the cutout geometry. The advantage of this method is the fact that jittering boundaries are supported. However, this method is restricted to convex exterior objects. Rappoport and Spitz [337] demonstrate that a related stencil-based approach can be used to realize interactive Boolean operators for CSG. The algorithm is similar to the implementation of shadow volumes by means of the stencil buffer [209]. The following extensions are needed for stencil-based cutouts. First, the algorithm has to affect the visibility of objects and thus their z values. Therefore, a mechanism to adjust depth values has to be incorporated. Second, the algorithm has to allow for front and back faces of the exterior convex object because the line-of-sight (from the camera) may have intersections with one front and one back face of the exterior object. The core algorithm is applied twice: once for front faces and once for back faces. The details of the rendering algorithm are as follows. In the first step, the front faces of the exterior object that should be cut are rendered to the depth buffer; the color buffer is masked out. Afterwards, the front faces of the cutout geometry are rendered with the depth test being activated, but without changing the depth buffer entries. A stencil operator increases the stencil value by one each time a fragment passes the depth test. Similarly, the back faces of the cutout geometry are rendered without changing the depth buffer. This time the stencil value is decreased when a fragment passes the depth test. This ensures that pixels of the front face lying inside the cutout geometry have a stencil value of one. All other pixels have a stencil value of zero. It has been assumed that the camera is not located inside the cutout geometry. If the camera is within the cutout geometry, the stencil buffer needs to be initialized to one. In the next step, the depth buffer is cleared and the back faces of the exterior object are rendered to the depth buffer in the regions where the stencil value is greater than zero. Then, the front faces of the cutout geometry and afterwards the back faces of the cutout geometry are rendered with depth test being enabled. The stencil value is increased for front faces passing the depth test and decreased for back faces passing the depth test. It is now possible to decide for each pixel if only the back face of the exterior object (stencil value one), both the back and front faces (stencil value zero) or neither (stencil value two) are visible. For the final step, the depth buffer is cleared and then the back faces of the exterior object are rendered into depth and color buffers in regions where the stencil value is one. Afterwards, the front faces of the

208


exterior object are rendered to depth and color buffers in those parts of the screen where the stencil value is zero. Finally, the interior objects are rendered without stencil test. The advantage of this approach is that complex cutout geometries are supported. In addition, the number of rendering passes does not increase with the complexity of the cutout geometry. Stencil buffer and stencil test are already included in standard OpenGL 1.0. Therefore, the algorithm is widely supported by graphics hardware. Unfortunately, the structure of the exterior object is subject to an important restriction: the object has to be represented by a single convex surface without boundaries. The above algorithm makes explicit use of the fact that (at the most) one front and one back face is cut by a ray originating from the camera. However, many 3D scenes contain nested surfaces or explicitly represent all boundaries – both inside and outside – i.e., they can have more than just one front face and one back face intersecting the same line of sight. In this case, the above algorithm fails because it is no longer guaranteed that all back faces of the exterior object lie behind the cutout geometry. Moreover, cutout walls are not modeled [R5]. Texture-Based Cutout The restriction to convex exterior objects is overcome by a novel rendering algorithm that uses textures to represent the cutout geometry. The basic idea is first illustrated for the example of a single cutting plane, as depicted in Fig. 4.21. The operations required for a cut into a single triangle are considered. The decision whether a fragment of the triangle lies inside the clipped half space or in the complement space is based on the signed Euclidean distance of the fragment from the plane. Fragments with a negative distance d are assumed to be clipped and fragments with a positive value d are assumed to pass. Signed distances are computed for each vertex and then interpolated across the triangle to obtain values for each fragment. The distance is transferred into the rasterization stage via an interpolated texture coordinate. The per-vertex distances can either be computed on the CPU or by a vertex program. The necessary parameters for the plane equation of the cutout can be provided by passing vertex program parameters. In addition, the original signed distance dplane can be perturbed to allow for jittering cutouts. The idea is based on displacement mapping techniques [61]. Figure 4.22 illustrates the displacement of the signed distance to a cutting plane. The final distance is computed by fragment operations according to d = dplane + dperturb . The perturbation dperturb is stored in a 2D texture and is superimposed onto the original distance. From experience, fractals, synthetic procedural textures used for clouds and virtual terrain height fields, and real terrain data produce good visual effects. In particular, a sawtooth-like boundary can be realized by a simple and memory-friendly perturbation texture: just a tiny 2×2 texture as


209

d

object

v1

v2

d1

d2

e

plan

d2 v2

d1 v1

Fig. 4.21. Planar cutout. Signed distance values are attached to the vertices and linearly interpolated in between d

object v2 d2

e

plan

v1 d1

d2 v2

d1 v1

Fig. 4.22. Jittering cutout. A perturbation function displaces the original distances to the cutting plane

Texel 0

Texel 1

Texel 2

Texel 3

Fig. 4.23. A 2 × 2 perturbation texture used for sawtooth-shaped boundaries. A height-field visualization of the interpolated texture is shown in the right image ALL IMAGES: DIEPSTRATEN, WEISKOPF, ERTL [89]


210


(a)



Fig. 4.24. Cutout illustrations with different rendering styles, showing a part of an engine block: (a) toon shading and silhouette rendering; (b) cool/warm tone shading and black silhouette lines. See also Color Plate 41

illustrated in Fig. 4.23 is needed. Thanks to texture repeat and bilinear texture interpolation a repeated fall-off is generated, which leads to the desired visual effect. Figure 4.20 (a) shows an example of a cutout illustration based on this 2 × 2 perturbation texture. In the last step, a fragment clipping operation has to be executed according to the corresponding distance value. If the value is below zero, the fragment has to be clipped, otherwise it is kept. This can either be done through a texkill command or by setting the alpha value and using the alpha test. Only a single perturbed cutout plane is supported so far. The above texture-based algorithm can be easily extended to several cutout planes. A separate texture coordinate is used for each plane to store the respective signed distance and the perturbation is applied to each distance value. An additional fragment operation determines the minimal distance value. The following approximation can be used to avoid several texture coordinates and the additional comparison operation in the fragment program. The minimal distance value can also be computed per vertex, and just the according signed distance can be used as texture coordinate. This approximation yields correct results for most cases. If, however, the three different vertices of a triangle do not have the same closest plane, inaccuracies are introduced by interpolating signed distances that are attached to different planes. The smaller the triangle, the smaller is the possible error. The presented texture-based cutout algorithm meets requirements [R1]– [R4]. The geometry of the interior and exterior objects is not subject to any restrictions. Since the algorithm can be mapped to graphics hardware, interactive frame rates are possible even for complex illustrations. The only drawback is the missing modeling of cutout walls [R5].


211

Figure 4.24 shows examples of cutout illustrations of an engine block with two different rendering styles. Additionally, a 512×512 shadow map is used to simulate shadow casting inside the interior parts. In Fig. 4.24 (a), silhouette rendering is employed, with a toon-shading technique described by Lake et al. [236]. Specular lighting is added to the original diffuse toon-like lighting. The diffuse and specular terms are used to access a 2D texture that contains the final color. Silhouettes are generated by using a GPU-based method according to Gooch et al. [139]. Silhouettes at cutting boundaries cannot be created by the original approach because these boundaries are not explicitly modeled as a triangular mesh. Therefore, the criterion for a silhouette – an edge connecting a front with a back face – is not valid at a boundary. This problem can be overcome by using an idea also described by Gooch et al. [139]. In Fig. 4.24 (b), the same engine scene is rendered with cool/warm tone shading [137], which is implemented in a vertex program. 4.4.5 Breakaway Illustrations In this section, a rendering algorithm that meets the requirements for breakaway illustrations is presented. It is shown how a computer-based process can fulfill the breakaway-specific rules [R6] and [R7]. The classification of objects as interior or exterior [R1] is provided by the same mechanism as in the previous section. The basic idea is to clip away those parts of the surrounding object that would otherwise occlude the interior objects as seen from the camera’s position. Therefore, this approach is intrinsically view-dependent and allows for [R7]. The other requirement is that only a single hole is cut into the exterior object [R6]. The convex hull of the interior objects is used as a basis for breakaway illustrations. Just enough is removed from the outside object to make this convex hull visible. The convex hull has two advantages. First, it contains all interior objects. If the convex hull is visible, all interior objects are visible. Second, the projection of the convex hull onto the image plane always yields a convex geometry and cannot contain any holes. The following algorithm is proposed for breakaway illustrations. In a preprocessing step, the convex hull of the interior objects is computed, for example, by the Quick Hull algorithm [8]. The convex hull is extended into all directions by some additional spatial offset. In this way, all interior objects are enclosed with a non-zero minimum distance to the hull. The extended convex hull serves as a virtual clipping object during the actual rendering process. Only those parts of surrounding objects that are not in front of the convex hull are rendered. This is achieved by using the foremost part of the convex hull as a clipping object. Finally, the interior objects are displayed. The crucial point of the algorithm is the clipping at the foremost surface of the convex hull. Here, the same clipping mechanism is used as for view-dependent transparency in Sect. 4.4.2. Figure 4.25 shows the details of

212

4 Perception-Oriented and Non-Photorealistic Rendering clear depth/frame buffer

render convex hull to depth buffer read depth buffer define clip texture

generate clip texture containing depth values of the extended convex hull of the clipping object

clear depth buffer

enable clipping

render exterior objects to frame/depth buffer

while rendering, test against clipping area

render interior objects to frame/depth buffer

disable clipping



Fig. 4.25. Rendering pipeline for the breakaway technique: computation of the clipping area (first four boxes) and actual scene rendering (last four boxes)

the corresponding rendering pipeline. This pipeline is tailored for NVIDIA GeForce 3 but could be directly applied on more advanced GPUs as well. In the first four boxes, the clipping area in screen space is determined. First, the depth buffer is initialized with a depth of zero (which corresponds to the near clipping plane of the view frustum). Then, the extended convex hull is rendered twice into the depth buffer. The first rendering pass uses “greater” as logical operation for the depth test, the second rendering pass uses the standard “less” depth test. In this way, the depth buffer contains the depth values of the foremost parts of the convex hull. Furthermore, the depth buffer is still initialized with zero in the areas that are not covered by the convex hull. In the third box, the depth values are transferred into a highresolution 2D texture (a HILO texture on GeForce 3) that will serve as the clip texture. Then, the depth buffer is cleared. In the fifth box, a fragment program is enabled to clip away all fragments that have equal or smaller depth values than those given by the above clip texture. Essentially, this fragment program replaces the z value of a fragment by (z − zclip ), where zclip represents the depth value stored in the clip texture.


(a)

213



Fig. 4.26. Breakaway illustrations with different rendering styles, showing a curved conduit: (a) toon shading and silhouette rendering; (b) cool/warm tone shading with black silhouette lines. See also Color Plate 42

This fragment program causes all fragments with z < zclip to be clipped away. Then, the exterior object is rendered into the frame and depth buffers; all parts in front of the convex hull are clipped away. Finally, the interior objects are displayed and the fragment program is reset to the standard configuration. This algorithm can take into account more than one “cluster” of interior objects by computing several corresponding convex hulls and rendering them into the depth buffer in the second step of the rendering pipeline. Surrounding objects with boundary surfaces of finite thickness are also possible. Here, separate clipping objects have to be defined for the front face and the back face of the boundary. By choosing a smaller clipping geometry for the back face, the effect of cutting through an object of finite wall thickness can be imitated. In this way, requirement [R5] for visible walls can be met in part. Figure 4.26 shows examples of breakaway illustrations of a curved conduit with two different rendering styles. In Fig. 4.26 (a), silhouette rendering and toon shading [236] are employed. Specular lighting is added to the original diffuse toon-like lighting. Silhouettes are generated by a GPU-based method according to Gooch et al. [139]. In Fig. 4.26 (b), the same scene is rendered with cool/warm tone shading [137]. 4.4.6 Implementation and Performance of Surface Rendering The implementation of semi-transparent illustrations and cutaway drawings is based on OpenGL, C++, and GLUT. The development and testing of the implementation was performed on NVIDIA GeForce 3/4; therefore, NVIDIA-

214


Table 4.1. Performance measurements in frames per second (FPS) for semitransparent NPR on a 5122 viewport (NVIDIA GeForce 3 Ti 200) [88] Scene

# Polygons Total Non-opaque

Fig. Fig. Fig. Fig.

4.17 (d) 25,192 4.17 (d) 25,192 4.18 (a) 40,000 4.13 70,955

192 192 835 3,112

Lines?

FPS I

II

No 21.4 9.0 Yes 13.7 5.7 No 11.4 5.6 No — 1.2

specific OpenGL extensions were used to access the texture shader and register combiner units. The algorithms from Sects. 4.4.2, 4.4.4, and 4.4.5 are mapped to per-vertex and per-fragment operations on GeForce 3/4. Table 4.1 shows performance measurements for semi-transparent illustrations (Sect. 4.4.2). Both depth-sorting approaches are considered: the method for implicitly modeled interior boundaries is denoted “I”, the method for explicitly modeled interior boundaries is denoted “II”. The performance tests were carried out on a Windows PC with AMD Athlon 900 MHz CPU and GeForce 3 Ti 200. Viewport size was 5122 . The first column refers to the figure in which the respective scene is depicted. The second and third columns contain the number of all polygons and only the transparent polygons, respectively. The fourth column indicates whether silhouette lines are rendered. The fifth and sixth columns reflect the frame rates for methods “I” and “II”, respectively. Note that method “I” does not render the scene in row four correctly and thus the corresponding frame rate is omitted. These performance figures indicate that most of the rendering time is spent in the computation of silhouettes, as frame rate drops rapidly with scene complexity. Therefore, speedup techniques previously described in Sect. 4.4.2 should lead to improved frame rates. The reduced performance of method “II” compared to method “I” is mainly caused by the larger number of rendering passes and the read-back from the z buffer. Table 4.2 shows performance measurements for cutout and breakaway illustration from Sects. 4.4.4 and 4.4.5. The tests were carried out on a Windows PC with AMD Athlon 1533 MHz CPU and NVIDIA GeForce 4 Ti 4600. The test scene is depicted in Fig. 4.24 and contains 145, 113 triangles. Both cutaway techniques were tested with two different rendering styles (cool/warm shading and toon shading) and two different viewport sizes (5122 and 10242 ). For comparison, rendering times are included for each style without applying cutaway methods. As it can be seen in Table 4.2, the cutout technique hardly affects the final rendering time. For the larger viewport, the breakaway technique loses nearly half of its original performance. This might be caused by


215

Table 4.2. Performance measurements for cutout and breakaway illustrations in frames per second (NVIDIA GeForce 4 Ti 4600) [89]. The test model is illustrated in Fig. 4.24 and contains 145, 153 triangles Render style

Cutaway technique

Viewport size 5122

10242

Cool/warm Cool/warm Cool/warm

None Cutout Breakaway

6.11 6.07 5.67

6.10 5.69 3.27

Toon shading Toon shading Toon shading

None Cutout Breakaway

6.12 5.75 5.52

6.10 5.32 3.30

the double read-back from the depth buffer for the convex hulls of the interior objects and by the usage of large clipping textures. 4.4.7 Extension to Volume Illustrations The NPR techniques discussed so far expect a boundary representation (BRep) of scene objects. However, the idea of cutout illustrations and nonphotorealistic transparency can also be applied in the context of volume illustrations. One aspect of illustrative volume rendering is concerned with the actual rendering style. Similarly to surface NPR, volume illustrations typically imitate artistic drawing styles. For example, Ebert and Rheingans [105] describe techniques for continuous tone-shading. Another important aspect of volume illustrations is the question which parts of the volume should be drawn in which style. This aspect is related to the challenge of finding an appropriate level of visual abstraction: important elements should be emphasized in the image, unimportant elements should be de-emphasized or removed to reduce the visual complexity. One approach to achieve this goal is to adopt the concept of viewdependent transparency from Sects. 4.4.1 and 4.4.2. Context-preserving volume rendering by Bruckner et al. [34] is an example of view-dependent transparency in GPU-based volume rendering. The basic idea is to extend the description of the transfer function in order to include view-dependent effects. For example, the volume illumination term is used to modify opacity: parts of the volume that receive little illumination are emphasized, other parts are decreased in opacity. In this way, contours are drawn more prominently. In addition, the effect of a view-aligned clipping plane is mimicked by controlling the opacity according to the distance from the camera. Finally, the amount of accumulated opacity and the gradient magnitude affect opacity in a way that the inner parts of a volume become visible, while preserving the outer

216


context. The model by Bruckner et al. [34] can be easily used in combination with most variants of GPU volume rendering (see Sect. 2.3) because it only requires a slightly modified definition of the transfer function. Importance-driven volume rendering by Viola et al. [448, 449] is another example of view-dependent transparency. Their approach generalizes breakaway illustrations (see Sect. 4.4.5): an importance value is associated with volumetric regions so that the most important regions are always visible. During volume rendering, importance values are accumulated along viewing rays to control the density of the visual display – a region is rendered more sparsely if it would otherwise occlude more important parts. At the moment, importance-driven volume rendering is restricted to a CPU implementation and a fast GPU version needs yet to be developed. The overview article by Viola and Gr¨ oller [447] provides more references on the state of the art in “smart” visibility for visualization. Another extension adopts cutout illustrations (see Sect. 4.4.4), where cutouts are used to deform volumetric objects. Similarly to explosion illustrations, some elements of a volume can be deformed or moved to other locations so that interior regions are no longer occluded. In general, volume deformations can be performed at the modeling stage; the explicitly deformed model can afterwards be processed by traditional rendering techniques. For interactive deformation, however, a direct rendering of the deformation is more useful because it avoids the additional step of explicitly building a modified volume representation. For example, Kurzion and Yagel [233] introduce a socalled ray deflector that bends the viewing rays in order to render deformed volumes. Kurzion and Yagel [234] subsequently extend their method to 3D texture-based volume rendering. Westermann and Rezk-Salama [487] also use 3D texturing hardware for interactive volume deformations. The GPU technique by Rezk-Salama et al. [342] allows for an efficient adaptive subdivision of patches of a deformed volume. Singh et al. [395] achieve interactive volume deformation by manipulating a volume proxy geometry based on skeletons. More background literature on volume deformation and animation can be found in the state-of-the-art report by Chen et al. [54]. The aforementioned papers discuss volume deformations in general. In the specific context of cutout and explosion illustrations, Islam et al. [184] use volume splitting, which is based on the spatial transfer functions by Chen et al. [55], to selectively transform subregions of a volume. A related method by McGuffin et al. [281] uses deformations to browse volumetric data. It should be noted that these two cutout and deformation approaches are not yet designed for efficient GPU rendering. In contrast, VolumeShop by Bruckner and Gr¨ oller [35] is an example of an integrated, interactive system for GPU volume illustrations that, amongst other features, allows for cutout drawings. The system by Svakhine et al. [406] is another example of a system for interactive GPU-based volume illustrations. Their implementation uses illustration motifs for a high-level specification of visualization goals.

4.5 Non-Photorealistic Halftoning

217

In general, cutout and breakaway illustrations are strongly related to volume clipping. Therefore, the clipping techniques from Sect. 2.5 can be adopted for interactive cutaways. The main challenge is an appropriate definition of the clipping geometry, which could be based on the approaches of this section. For volume illustrations, the visibility is sometimes controlled by a fuzzy, gradually and smoothly varying parameter in order to give the illustrator more flexibility in the visual design. This flexibility can be regarded as an extension of the binary visibility description in volume clipping. For example, importancedriven volume rendering [448, 449] allows for a fuzzy visibility. Nevertheless, clearly defined, binary cutaways are useful for many illustration applications.

4.5 Non-Photorealistic Halftoning Instead of continuous tone shading, different patterns can also be used to convey shading and texture for effective visualization. Halftoning methods [446], for example, simulate a wide color spectrum with only a few discrete color values. In terms of non-photorealistic rendering, halftoning is applied to introduce patterns in the drawing in order to achieve a clearer and more pleasant display style. For example, many illustrations in textbooks on archaeology or medical science are drawn in a stippling style, even when photographs of good quality are available. This section focuses on screening as the basis for halftoning, where different tones are represented through predefined patterns [318]. Two possible approaches have evolved for the automatic generation of halftoning images: halftoning in object space as illustrated by Freudenberg et al. [126] or Praun et al. [335], and halftoning in image space as, e.g., described by Secord [387]. Both approaches have specific advantages and disadvantages. In object-space methods, frame-to-frame coherence can quite easily be achieved by attaching halftoning structures to scene objects. In this way, halftoning patterns always move with their corresponding object and, therefore, temporal coherence is preserved. Halftoning elements are typically stored in the form of a texture; texture coordinates are specified in a way to “glue” these textures onto the objects, thereby automatically linking the halftoning fragments with the object surface [126, 335]. In pen-and-ink illustrations, halftoning fragments are often drawn beyond object borders to achieve a more natural drawing style. Object-space approaches cannot easily meet this demand, as the halftoning process is restricted to the object surface. Moreover, a prerequisite for object-space methods is a 3D scene with adequately assigned texture coordinates, i.e., input data cannot be processed without the information of the complete 3D scene. Another important drawback of object-space techniques shows up for screening illustrations. A screening process of the complete image plane is not possible, as screening is applied separately for each object surface. Moreover, the screening patterns should be independent of the object scale and orientation, which is hard to achieve in object-space approaches.

218


Conversely, image-space techniques overcome many restrictions of objectspace methods. Halftoning elements can easily cross object borders, as the halftoning operation is performed on the image plane. Image-space algorithms often rely only on 2D information and, therefore, can also be applied to video sequences. In contrast to object-space methods, image-space approaches can easily meet the requirement that screening patterns are rendered independently of an object’s scale or orientation. On the other hand, the main problem for image-space methods is how to maintain temporal coherence since, in general, the halftoning process is not directly related to the objects. Section 4.5.2 addresses the issue of frame-to-frame coherent halftoning. The idea is to preserve the advantages of image-space techniques, while overcoming the problem of temporal incoherence. The proposed rendering technique transports halftoning structures in image space according to the motion of the underlying objects. It is capable of generating time-coherent animations of all halftoning techniques that are based on screening matrices. Therefore, many display styles such as ordered dithering [404], clustered dot screening [432], or void and cluster screening [432] can be implemented. Additionally, stippling [387] or hatching [126] illustrations can be simulated with appropriate screening matrices. Moreover, it is demonstrated that the proposed rendering algorithm can be mapped to graphics hardware to generate non-photorealistic animations in real time. Finally, Sect. 4.5.3 generalizes the idea of image-space GPU methods by introducing a G-buffer [368] framework tailored to graphics hardware. This framework can be integrated into the existing effect files concept and, thus, the proposed system can be easily adopted. 4.5.1 The G-Buffer Concept Image-space rendering techniques have been and will be an active area of research, since all renderings finally end up in image space. In particular, the field of non-photorealistic rendering shows the usefulness of image-space approaches because important NPR rendering styles operate directly in image space, such as halftoning, screening, stippling, or hatching methods. The G-buffer (geometric buffer) by Saito and Takahashi [368] is a general framework for such image-space rendering algorithms. This buffer forms an enriched image space, also referred to as 2.5D image space, where each pixel of the image space holds additional attributes. The G-buffer approach separates rendering in two parts. In a first step, G-buffer attributes are generated on a 2D domain. Many attributes come from the standard graphics pipeline itself; for example, G-buffer attributes can store color, alpha, depth, texture coordinates, or normal vector per pixel. Other typical G-buffer attributes are object ID, world-space position, screen-space position, or parameterization of higher-order surfaces. Additionally, further attributes might be implemented such as the screen-space velocity for motion blur effects or light-space depth values for shadow calculations. Most attributes are computed in object space


Fragment Intensity

Threshold Screening Matrix

219

Result of the Screening Process IMAGE: EISSELE, WEISKOPF, ERTL [108]

c 2004 IEEE

Fig. 4.27. Illustration of threshold screening: the gray values (fragment intensities) of the original picture are compared with entries in the screening matrix

and stored for each pixel of the image plane. The subsequent rendering passes, referred to as G-buffer operations, receive this information and have therefore access to more than only simple 2D color information. Arbitrary calculations upon this data structure can be applied to evaluate the final result per pixel. This approach makes it possible to implement some classes of algorithms that cannot be realized by object-space techniques. In this book, NPR is the most prominent application for the G-buffer. Many algorithms for artistic drawing styles work in image space, e.g., hatching or stippling illustrations generated in image space do not suffer from perspective distortion, introduced by most object-space techniques. More importantly, the rendering algorithms are not restricted by object borders and so, for example, hatching lines can cross object boundaries to achieve a more natural drawing style. Some NPR algorithms are exclusive image-space operations that can directly be implemented via the G-buffer framework. Of particular interest are screening techniques, which combine a grayscale or color image with an underlying screening mask to obtain halftoning. Figure 4.27 illustrates the idea of a typical screening technique – threshold screening. Here, the gray value of a fragment from the original picture is compared to entries in a screening matrix. The final pixel is black when the original gray value is below the corresponding entry, otherwise white. The screening matrices are located on a uniform grid and fill the screening mask, which covers the complete image space. Other screening approaches may use more complicated ways to determine the color of the final pixel. Another class of G-buffer algorithms comprises filter operations. Depending on the filter kernel, different computations can be performed, e.g., edge detection by the Sobel operator or blurring by Gaussian filtering. Also morphological operations like dilate or erode could be realized. More filter operations are known in the computer vision [72] and image-processing [361] literature, and most of them could be implemented within the G-buffer concept. Figure 4.28 demonstrates that NPR and image processing techniques can be handled by the G-buffer within the same image. Several G-buffer operations are performed to calculate the final result: edges are detected and

220


IMAGE: EISSELE, WEISKOPF, ERTL [109]

Fig. 4.28. G-buffer for NPR and image processing: edges are detected and dilated, and an NPR screening technique is applied afterwards

dilated (by image processing methods) and an NPR screening technique is applied afterwards. Deferred shading is a third field of application for the G-buffer. The basic idea is to separate shading and lighting from other parts of rendering. This has several advantages. If the lighting conditions of a scene are changed but the geometry remains static, only the lighting has to be recomputed. Moreover, if a scene suffers from massive overdraw, a conventional object-space algorithm will compute lighting for each fragment regardless if it is overdrawn by other, closer fragments. In contrast, deferred shading evaluates the illumination only for visible pixels. 4.5.2 Frame-to-Frame Coherent Halftoning This section addresses issues of temporal coherence in animated halftoning based on G-buffer screening. Screening techniques combine an input image

221

Scene Movement


c 2004 IEEE


Fig. 4.29. Advection of a screening matrix

with an underlying screening mask to obtain the final image. As the screening matrices are located on a fixed uniform grid, the position of elements of the screening mask is independent of the motion of the underlying scene. Therefore, previously known screening techniques suffer from the “shower door effect”, where objects of the scene appear to swim through the texture. This section describes a novel method to achieve frame-to-frame coherence, as proposed by Eißele, Weiskopf, and Ertl [108].5 This approach advects the screening mask in image space according to the velocity of each pixel in the image plane. In this way, a coupling of the screening mask and the moving objects of the scene is established. Figure 4.29 shows two snapshots taken from a rotating cube to illustrate how a single screening matrix is moving with the underlying object. The image-space velocities of fragments – in the context of computer vision also referred to as optical flow [144, 166] – can either be calculated directly from the 3D scene, for example, by using rigid body kinematics, or they are computed from video sequences [166]. In both variants, the G-buffer framework is extended by an additional attribute that provides the image-space velocity of objects. Overview of the Algorithm The following issues are introduced by an advection of the screening mask. First, screening matrices may move to positions that are no longer aligned with the underlying grid, which is shown in the right image in Fig. 4.29. A second problem is caused by velocity fields that contain regions of divergence, convergence, or discontinuity. For example, divergence and convergence appear in the case of a rotating object or camera zooming, discontinuities show up at object borders. In these areas, in the following referred to as distortion areas, the velocity field causes distortions in the screening mask if only a simple advection is performed. In addition, the spatial frequencies of the mask might change. Stated differently, image-space advection of screening masks is subject to the following contradicting requirements: screening matrices should have equal 5

c 2004 IEEE. Reprinted, with permission, from Eißele, Weiskopf, and Ertl [108]

222


Fig. 4.30. Combining different masks (adopted from [108])

size and be aligned to an underlying grid; and, screening matrices should accurately follow the motion of objects. Multiple screening masks are introduced to approximately meet the above requirements: (1) a static, non-advected mask C that accounts for screening matrices aligned to the underlying grid; (2) a screening mask S that is advected to follow the motion of objects; and (3) a so-called blend-in screening mask B used to couple the above two masks and make possible a smooth insertion of new screening matrices. Figure 4.30 illustrates the different masks and how they are combined for the generation of the final image. Mask S is advected, based on the velocities of the scene objects. Whenever artifacts caused by distortion areas become too large, new elements of the mask C replace these regions of S. The blend-in screening mask allows these new elements to be blended in smoothly and thus eliminates popping artifacts. In the actual implementation, masks C and B are not combined to form another mask; in fact, both masks are applied to render intermediate halftoning images that are afterwards combined to the final image. The complete process can be split into three computational stages: update of the blend-in screening mask B, update of screening mask S, and calculation


223

Static Screening Mask C

Blend-In Screening Mask B

Advect Mask B

Detect Distortion Insert new Matrices from Mask C

New Blend-In Screening Mask B

G-Buffer Velocity Field Copy threshold

Screening Mask S

Advect

from Mask S or

Mask S

from Mask B

Screening Blending

New Screening Mask S

Final Image

Next Frame


c 2004 IEEE

Fig. 4.31. Computational scheme for one time step of frame-to-frame coherent halftoning

of the final image. Figure 4.31 illustrates the flow chart for the combination of all three tasks. These three steps are detailed in the following discussion. Calculation of the Final Image To generate the final image, the results of the screening processes with mask S and blend-in mask B have to be combined. Therefore, each fragment of mask S has an appended status information that consists of the total advection distance in x- and y-direction. Each time a fragment is advected, these distance counters are updated accordingly. The scale for the counters are chosen in a way that an absolute value of 1.0 corresponds to an advected distance equal to the extent of the screening matrix in this direction. Additionally, the blend-in screening mask B holds a timer that is incremented after each frame and thus reflects the age of the fragment in mask B. A timer value of 1.0 corresponds to the maximum age of a fragment. The maximum of these values is calculated to determine the weight for blending the results of the screening mask S and the blend-in mask B. This ensures that a fragment is completely blended when it has either been advected by the size of one screening matrix in x- or y-direction or the maximum time for blending has elapsed. The timer restricts the duration of the blending process to a user-specified threshold, while the direction counters accelerate the blending, depending on the advection velocity. Update of the Blend-In Screening Mask The first step for this part of the algorithm advects the blend-in screening mask B according the velocity values in the G-buffer attribute and increments

224



c 2004 IEEE

Fig. 4.32. Alignment of the insert mask with the layout of the static screening mask to ensure the insertion of complete matrices

the timer for each fragment. The backward Lagrangian-Eulerian advection (LEA) scheme (see Sect. 3.3.3) is applied to transport elements of B by displacing them at each time step. LEA makes use of nearest-neighbor sampling in the backward texture lookup to maintain the contrast of the original input texture and to preserve the structure of the screening mask. As described in Sect. 3.3.3, fractional coordinates are used to model subtexel motion. The primary purpose of the blend-in mask is to detect distortion regions in the velocity field and schedule these for a replacement in the blend-in mask B with new matrices from the static screening mask C. The difference between the velocities at the current fragment position and at the previous position is considered to compute a measure for the distortion. Unfortunately, the velocity field can contain a wide range of velocity values and therefore simple thresholds for finite differences are not appropriate. Thus, the implementation computes the ratio of both velocities. If the ratio is below a given divergence threshold, the fragment is assumed to be located in a divergence region and therefore marked for an overwrite with the static screening mask C. A ratio above the convergence threshold also initiates an overwrite of the fragment. Additionally, the distance counters are examined to check whether the fragment has been advected for more than the extent of the screening matrix, which also initiates an overwrite by the value from the static screening mask C. To ensure that only complete matrices are inserted, distortion regions have to be exactly aligned with the layout of the static screening mask C as shown in Fig. 4.32. All fragments of a newly inserted screening matrix receive an initial value of 0.0 for the timer and the distance counters. Along subsequent frames, the contribution of the blend-in screening mask B to the final image increases monotonically, depending on the distance and the timer value, until one of the values reaches 1.0 and the information of the blend-in screening mask B of the current fragment is transferred to the screening mask S.


225

Calculation of the Screening Mask For each frame, the screening mask S is advected according to the velocity value for each pixel. Due to distortion areas in the velocity field, the advected screening mask S contains areas in which screening matrices are skewed. As mentioned above, new matrices are inserted into the blend-in screening mask B to smoothly replace distorted matrices in the screening mask S over a couple of frames. During the calculation of the screening mask S, it is only important to detect when the blend-in screening mask B has been completely blended. Then, the screening matrices are copied from the mask B to the screening mask S. Implementation The implementation of frame-to-frame coherent halftoning is based on C++ and DirectX 9.0. The G-buffer framework utilizes the render-to-texture functionality of DirectX to store the G-buffer attributes of each pixel in textures. The velocity attribute stores the velocity in screen-space coordinates multiplied by the frame-to-frame time, which yields the distance the fragment moves in one time step. For rotating objects, the velocity magnitude is very small near the rotation center, whereas the velocity could be very high further away from the rotation center. Thus, the velocity field could contain a huge range of values, and a 32-bit floating-point texture is used to store distances for the G-buffer. The image-space rendering algorithm uses pixel shader 2.0 functionality through the DirectX effect files feature. For the calculation of the final image, the screening technique is applied separately to the screening mask S and the blend-in mask B. Threshold screening patterns are encoded in a 2D texture, and a smooth threshold function [126] is applied to evaluate screening. The results are blended according to a factor f that is calculated from the maximum of the timer and the two distance counter values. If f is greater than 1.0, the blend-in mask has already been completely blended and copied to the screening mask S. In this case, the blend-in screening mask has no contribution to the final image and only the result of the screening with the mask S is used for the final image. During the update of the blend-in screening mask B, velocity values are used to apply GPU-based Lagrangian-Eulerian advection (see Sect. 3.3.3) to the mask, utilizing dependent-texture lookups and maintaining fractional coordinates. The extra information for the timer and the distance counters for each fragment are encoded in free components of the blend-in mask and calculated during the blend-in screening mask update. Additionally, distortion areas are detected by computing the ratio of the G-buffer’s distance value at the position in the previous frame to the distance at the current position. If the ratio is below a user-specified divergence threshold, the current fragment will be regarded as distorted; a difference above the user-specified convergence threshold also marks the fragment as distorted. The total advection distance

226


of a fragment is used as a third criterion to determine a distortion. It is compared to the corresponding extent of the screening matrix in each direction. If the advection distance exceeds the matrix dimension in either direction, the fragment is considered to be distorted and is scheduled for override. In a subsequent rendering pass, distortion areas are enlarged to fit the size of a screening matrix, as illustrated in Fig. 4.32. Here, a render target with the size of the image plane divided by the size of the screening matrix is used, so that each fragment corresponds to one screening matrix. To determine whether a fragment of the downsized render target has to be marked as distorted, all texels of the corresponding screening matrix in the image plane should be tested if any of them is marked. To access all texels of an 8×8 screening matrix, 64 texture lookups are needed, which is more than some GPUs can provide. However, to determine a distortion of a fragment in the downsized distortion mask, it is enough to know if at least one fragment of the corresponding screening matrix is marked as distorted. Bilinear texture filtering accesses a 2 × 2 block of texels within a single texture lookup operation and combines the four values to a single interpolated value. It is sufficient to check this result against 0 to determine if at least one texel has a value greater than 0 because only binary flags of 0 or 1 are stored in the textures. Therefore, just 16 texture lookups are necessary to calculate the enlarged distortion area for a 8 × 8 screening matrix. For very big screening matrices, a hierarchical multi-pass approach can be applied, where each pass reduces the number of texels that map to a screening matrix until one texel maps to one matrix. In an additional pass, this render target is resized to the output resolution by using nearest-neighbor sampling. Now each distortion area is aligned with the underlying screening matrix and, depending on the insertion mask, either the advected value of the blend-in screening mask from the previous frame is used or a new value is inserted from the static screening mask C. The update of the screening mask S employs advection in the same way as for the blend-in mask. Afterwards, the distance counters and the timer, which are stored in the blend-in mask B, are checked whether any of them exceeds their maximum of 1.0. This situation initiates a copy of the screening mask value from the blend-in mask B to the screening mask S. This transfer of a screening value is also triggered when the fragment of the blend-in mask B is scheduled for override with the static mask C, since otherwise the content of the screening mask B would be lost. Results Figure 4.33 shows a sequence of a rotating car in a computer-generated animation. Here, the velocity at each vertex of the mesh is evaluated by using rigid body kinematics in object space with a subsequent projection to image space. Linear interpolation of vertex attributes is used to calculate the velocity of fragments within each polygon of the mesh. Figure 4.33 demonstrates a scale and orientation independent representation of the screening patterns

227

IMAGE: EISSELE, WEISKOPF, ERTL [108] c 2004 IEEE


Fig. 4.33. Sequence of a computer-generated animation with frame-to-frame coherent halftoning

over the entire image plane. The correspondence of the screening patterns is shown in two enlarged parts at the bottom of Fig. 4.33: the motion of screening patterns is coherent with the motion of the underlying scene. In areas of extreme distortion at the left side of the car, some screening matrices are still distorted. In these cases, it is hard to decide whether an advection of the matrix or a replacement should be performed. Figure 4.34 depicts a sequence of an animated rotating pot which shows that the patterns of the screening algorithm follow the texture of the pot. This scene contains less severe distor-

4 Perception-Oriented and Non-Photorealistic Rendering 228


Fig. 4.34. Screening illustration of a rotating pot

c 2004 IEEE


229

Table 4.3. Performance of frame-to-frame coherent halftoning in frames per second (ATI Radeon 9700 Pro) [108] Viewport size With G-buffer Without G-buffer

256 × 256

512 × 512

1024 × 1024

238 295

65 68

16.4 18.6

tions areas, which leads to better results in the illustration of the screening patterns. Table 4.3 shows performance measurements for different viewport sizes. The frame rates with and without generating G-buffer attributes are documented, i.e., the last row shows the performance numbers for the screening mechanism itself. The tests were conducted on a Windows PC with AMD Athlon and ATI Radeon 9700 Pro. The car model of Fig. 4.33 was used as test scene. 4.5.3 The G2 -Buffer Framework As described in Sects. 4.5.1 and 4.5.2, the G-buffer is a flexible concept to implement image-based rendering algorithms, especially for non-photorealistic halftoning. The goal of this section is to present a generic framework that maps the G-buffer concept and associated image-space operations to GPUs. This GPU-G-buffer (G2 -buffer) framework has been introduced by Eißele, Weiskopf, and Ertl [109], and the following discussion is based on their paper. The G2 -buffer framework consists of two major components: first, a texturebased representation of G-buffer attributes and, second, an implementation of image-space operations by fragment programs. The G-buffer setup and the actual rendering algorithm are described in a plain text file, of which the syntax is an extended version of the effect files in DirectX [288] or CgFX [311]. The proposed approach provides a fast and generic method to implement Gbuffer algorithms on GPUs. Extended Effect Files for G-Buffers Effect files, as provided by DirectX or CgFX, greatly simplify the programming of modern GPUs by encapsulating the complete setup and programming of the graphics pipeline within a unified description – the effect. In this way, states of the graphics pipeline can be configured, vertex shader and pixel shader programs can be specified. However, the concept of effect files is mainly designed to describe object-space rendering techniques. In contrast, the G-buffer is a pure image-space approach and therefore additional functionality is needed to make effect files useful in this context. The following two essential aspects of the G-buffer cannot be configured in the original effect

230

4 Perception-Oriented and Non-Photorealistic Rendering G-Buffer Storage Per-fragment data

attribute 0

G-Buffer Operation Buf IN Tex IN Operation Pass 0

. attribute 1 . . attribute n

Buf IN Tex IN Operation Pass 1 . . . Buf IN Tex IN Operation Pass n

Buf IN Tex IN Final Pass

Pass Result Tex 0

Pass Result Tex 1 . . . Pass Result Tex n

Final Output


Fig. 4.35. Possible data flow configurations that can be configured within an extended effect file

files: first, the G-buffer attribute storage and, second, the G-buffer operations with an associated data flow. In the proposed framework, multiple 2D render-target textures are used to model the G2 -buffer storage, one for each attribute. Effect files do not provide a mechanism to define render targets; therefore, an attribute is defined as a 2D texture within the description of the effect. The G2 -buffer framework recognizes these definitions and updates the attribute texture for each frame. Typically, the algorithms to generate attribute values are object-space methods that are also implemented via effect files. A G-buffer operation is specified by a technique consisting of one or more rendering passes. The computations are performed per-fragment, i.e., the Gbuffer operation is described via a set of pixel shader programs. One pass often needs information generated by a previous pass. Therefore, the possibility to model data flow – to specify which pass result is used as input for another pass – has be introduced in the effect description. Figure 4.35 shows all possible data flow configurations that are configurable within an extended effect file. Textures can be used as data input to pixel shader programs, as it is already possible in standard effect files. To store the result of a rendering pass in a texture that can afterwards be used as input for subsequent pixel shader programs, render-target textures, referred to as pass-result textures, are used. Since effect files do not support render-target settings, pass-result textures are defined as ordinary textures within an effect. The G2 -buffer framework detects them and allocates corresponding render-target textures in which the result of a rendering pass is stored. During rendering, the framework automatically activates the respective render target for each pass of the G-buffer operation.


231

Through this mechanism, it is possible to have fragment programs of arbitrary length because subsequent passes can continue the calculation of prior passes, similarly to the idea by Mark and Proudfoot [272]. More importantly, subsequent passes can access results of previous calculations of neighboring fragments in image space, e.g., to implement filter operations. The input of a single pass labeled with Buf In refers to an indirect input of data already contained in the frame buffer via alpha test, alpha blending, z test, or stencil test. Implementation The implementation of the G2 -buffer framework is based on DirectX 9.0 and its effect file feature. DirectX provides the functionality to read and parse an effect file. The extensions are encoded by the naming of the textures and attributes within the effect file definition. The G2 -buffer framework examines the description and arranges the setup of the attributes and pass-result textures. The actual storage of G-buffer attributes and intermediate processing results is realized via render-target textures. Therefore, any transfer of per-pixel information between GPU and CPU is avoided. For pass-result and attribute textures, the number of components, the precision per element, and the relative scale factor for the size of the complete texture are specified in the effect. These parameters are set up by modifiers within the name of the declared texture. Since the allocated texture size, dimension, and precision can be configured to exactly match the requirements, any memory overhead is avoided. For example, it is possible to set up the normal attribute with three components of 16-bit fixed-point precision while the color attribute is generated with four components of 8-bit precision. The choice of precision and number of components per element is only restricted by the available formats for render-target textures. The first rendering step is to update G-buffer attributes required during the subsequent execution of G-buffer operations. Afterwards, the framework processes the G-buffer operations by rendering the corresponding passes step by step. Before a pass is rendered, the framework sets the appropriate render target, which is either the frame buffer or a pass-result texture. Then, a quadrilateral that matches the size of the image plane is drawn, where the texture coordinates of the vertices are set up to represent the fragment positions in image space. During fragment processing these texture coordinates are used to access the G-buffer attributes or results of prior passes. Since the assigned texture coordinates must exactly address the corresponding texel, special care has to be taken to ensure a one-to-one pixel-to-texel mapping. The size of a texel must be known by the effect to access adjacent texels. Since the size of pass-result or attribute textures depends on the frame buffer dimensions, the texel size changes when the frame buffer is resized. To overcome this problem, the texel size is defined as a vector within the effect description and updated after each viewport resize.

232


Results The performance of the G2 -buffer implementation depends on various factors. The costs of the first part, where the attributes are computed and stored in the G-buffer, are affected by the number and complexity of attributes used. Many operations on G-buffers use standard graphics pipeline properties, such as color or normal vector, for whose computation GPUs are optimized. For other attributes, however, the computational costs might differ significantly, depending on the operations needed to evaluate the attributes. As a G-buffer operation is an image-space technique, its performance is linearly dependent on the number of fragments on the image plane. Other parameters that affect performance are complexity of the fragment processing, number of passes in the G-buffer operation, and precision and dimension of intermediate results. The main part of the implementation is rasterizationbound and vertex processing imposes no restriction on performance because each pass of a G-buffer operation just renders a single quadrilateral. The following performance numbers were measured on a Windows PC with Intel P4 2.8 GHz CPU and ATI Radeon 9700 Pro GPU. Figure 4.36 shows the result of a G-buffer operation that performs Robert’s cross edge detection and a subsequent dilation. This example is rendered at 64 frames per second on a 1024 × 1024 viewport. Figure 4.37 illustrates the combination of Robert’s cross edge detection and halftoning. This image is rendered at 185 frames per second on a 1024 × 1024 viewport. Figure 4.28 shows another example, using an NPR screening technique. Several G-buffer operations are performed to calculate the final result: the contrast of the scene is enhanced, edges are detected and dilated, and screening is applied. This example is rendered at 78 frames per second on a 1024 × 768 viewport.

4.6 Summary In this chapter, several perception-guided and non-photorealistic rendering approaches have been proposed to improve the final rendering stage of the visualization pipeline. While the “classical” perception-oriented methods are based on early stages of the HVS, NPR can be regarded as a way to make use of more abstract and higher-level processing in the human brain. The main goal of Sect. 4.2 has been to stimulate a better awareness for the opportunities and problems involved with the low-level perception of moving color stimuli. Although luminance contrast plays a predominant role in motion perception, the effects of chromatic contrast should not be underestimated. First, even the motion of purely chromatic stimuli can be discriminated, which allows us to “stretch” the available design space for animated visualizations. Second, luminance stimuli can be affected by chromatic contrast. For example, adding chromatic contrast to a luminance stimulus may degrade the perceived velocity, i.e., the design space is restricted in this case.

Fig. 4.36. Dilated feature lines, implemented as a G-buffer operation

IMAGES: EISSELE, WEISKOPF, ERTL [109]

Fig. 4.37. Combination of Robert’s cross edge detection method and halftoning by means of G-buffer operations

4.6 Summary 233

234


Based on psychophysical evidence, guidelines have been proposed for effective color design in animated visualization, along with a practical calibration process. A generic and widely applicable color-based depth-cueing scheme has been introduced in Sect. 4.3. It rests upon the fundamental principle of the trichromatic generalization of human color perception. Arbitrary hyperplanes in the 3D tristimulus space of colors can be specified, and different ways of combining colors can be applied to color differences in these planes and perpendicular to them. Parameters have been derived for this generic scheme to allow for wellknown intensity depth-cueing and rather neglected saturation depth-cueing. In addition, semantic depth-cueing has been introduced as an abstract depthcueing approach. Although the proposed depth-cueing scheme is very flexible and powerful, it requires only a few and simple computational operations – just two linear transformations of 3D vectors and a weighted sum of two vectors – and can be efficiently implemented on graphics hardware. Section 4.4 has shown how NPR illustrations can be modified to improve the perception of spatial structures. A transparency model that depends on viewing direction has been introduced. Furthermore, image-space GPU methods for efficient depth sorting have been described. The transparency model can be readily combined with other techniques for NPR illustrations, such as cool/warm tone shading or silhouette rendering, in order to enhance visual perception and understanding. Alternatively, a small and effective set of rules has been presented for the computer-based rendering of cutaway illustrations. One class of rules leads to cutout drawings, which are most appropriate for scenes with large interior objects. The other class of rules is used for breakaway illustrations, which are suitable for scenes with smaller, densely packed interior objects. Both cutout and breakaway drawings are supported by GPUs and, thus, achieve interactive frame rates. Even though the rules facilitate a completely automatic generation of reasonable cutaway illustrations, interactivity is still very useful for a fine adjustment of illustration parameters. An advantage of this rule-based approach is the small number of parameters that effectively control the visual appearance of a drawing. Cutaway techniques can also be combined with existing NPR styles. While the above NPR techniques are mainly used for continuous tone shading, Sect. 4.5 has focused on NPR halftoning, which is typically employed in scientific illustrations. In particular, an approach has been introduced to achieve frame-to-frame coherent screening in image space. Screening matrices are advected to link these matrices with the object surfaces and to maintain temporal coherence. Since the input to this technique only requires an intensity image and the velocity of the optical flow in image space, the approach is also applicable to video sequences. Finally, the G2 -buffer framework has been described. This framework offers a flexible, generic, and abstract way of implementing G-buffer operations on graphics hardware. The G2 -buffer has been integrated into the existing effect files framework; since only minor ex-

4.6 Summary

235

tensions to the original syntax are introduced, the proposed system can be easily adopted and existing code can be reused.

5 Conclusion

This book has focused on efficient and effective ways to visualize data from scientific, engineering, and medical disciplines. High performance has been primarily achieved by using GPU-based algorithms. It has been shown that all stages of the visualization pipeline greatly benefit from GPU implementations – not only the graphics-oriented rendering stage but also the numerics-oriented filtering and mapping stages. Since data transfer from GPU to main memory is typically slower than from main memory to GPU, an implementation that runs completely on GPU without any read-back is one important goal that has been achieved for most visualization techniques of this book. In particular, GPU-based methods for direct volume rendering and texture-based vector field visualization have been investigated in detail. Parallelization has been successfully used as an approach that is complementary to GPU acceleration. It has been demonstrated that parallelization of GPU methods leads to a significant speedup on cluster architectures with distributed memory. Large-data visualization additionally benefits from adaptive, multi-resolution approaches, exemplified by wavelet-based and view-dependent parallel volume rendering. Visualization quality has been addressed in different ways. One quality aspect is related to numerical accuracy, which has been mainly investigated for texture-based vector field visualization. On the one hand, a level-set method for dye advection has been introduced to resolve the issue of numerical diffusion that is inherent to Eulerian or semi-Lagrangian advection. On the other hand, completely Lagrangian particle tracing has been employed to advect texture information without numerical diffusion. Moreover, higher-order explicit integration has been applied to improve the accuracy of particle tracing. Another research activity has addressed the quality of temporal behavior in animated visualization: mathematical and algorithmic models have been devised to achieve frame-to-frame coherence for non-photorealistic halftoning and the visualization of time-dependent vector fields. Animated visualization has also been examined under perceptual aspects. Based on results from psychophysical and psychological research, guidelines

238

5 Conclusion

for an effective use of color in animation have been proposed. Analogously, spatial perception has been improved by a generic color-based depth-cueing scheme. The non-photorealistic rendering methods of this book also address spatial perception – on a high level of cognitive processing: knowledge about traditional, manual illustrations has been analyzed to derive rules and algorithms for computer-generated images. The perception-oriented and nonphotorealistic rendering approaches specifically target an extended conceptual model of visualization that not only comprises the technical visualization pipeline but also takes into account that a human observer is an essential part of the visualization process. After all, the final goal of visualization is that the user gains insight. Interactive visualization targets this extended visualization cycle by tightly integrating the user in the exploration process. Therefore, interaction has been pursued as a superordinate goal throughout this work. On the one hand, the prerequisite in the form of real-time visualization has been achieved by efficient algorithms. On the other hand, interaction metaphors, such as volume clipping, painting dye into vector fields, or modeling cutouts in illustrations, have been investigated. A last aspect deals with the tendency toward generalization and abstraction, which is inherent to all fields of science. This direction of research is becoming increasingly relevant in computer-based visualization as the field is maturing. This book has contributed a generic and theoretical framework that describes the dense visualization of time-dependent vector fields by means of a two-step particle-tracing and convolution scheme in spacetime. At this point, the contributions of this work should be related to the 15 most important issues and research directions in visualization, as recently formulated by Johnson [197]. This list represents topics that are well-accepted by the visualization community because it includes feedback from a large number of researchers in the field. I believe that this book has primarily contributed to five of these topics. Johnson identifies “efficiently utilizing novel hardware architectures” as an important aspect of visualization. This work directly addresses this issue by extending the use of graphics hardware beyond areas for which GPUs were originally designed, in particular, by using GPUs for all stages of the visualization pipeline. Another research direction comprises “perceptual issues”, which have been specifically investigated in the chapter on perception-oriented rendering. The third issue of “time-dependent visualization” has been addressed for scalar field and vector field visualization. Parallel wavelet-based volume rendering has contributed to “scalable, distributed, and grid-based visualization”. Finally, the proposed framework for texture-based vector field visualization has provided one element of a “theory of visualization”. Despite these contributions to the field of visualization, many unsolved problems and open questions remain. I believe that the following research directions will be important in the years to come. Although perceptual issues of computer graphics have been investigated to some degree, the complexity

5 Conclusion

239

of human perception and cognition will drive further research in this field. Perception-guided visualization and graphics is still far from being optimal, which is mainly caused by the lack of a complete understanding of the processing in the human brain. The ultimate goal would be a model-based visualization approach, optimizing all parameters that describe the complete visualization process (data model, actual visualization pipeline, and human observer). In this context, feature extraction will play a prominent role because it allows the user to focus on relevant elements of the data set while unimportant parts are suppressed. Texture-based 3D vector field visualization, in particular, could be significantly improved by advanced feature definitions. Feature extraction will typically be domain-specific and take into account a model that describes data generation or measurement. Improved humancomputer interaction (HCI) methods play an equally important role in the visualization–user process and provide much room for future improvement. Finally, efficient algorithms for novel hardware architectures will remain an ongoing research effort. Since the amount of input data will keep on growing due to the increasing performance of simulation computers, sensors, and scanners, increasingly efficient visualization methods will be required and parts of the efficiency gain will be achieved by pushing the limits of future hardware.

List of Figures

1.1 1.2 1.3

Abstract visualization pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . GPU graphics pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Programming model for current GPUs . . . . . . . . . . . . . . . . . . . . . .

2 5 7

2.1 2.2 2.3 2.4 2.5 2.6

3D texture slicing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2D texture slicing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ray casting idea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ray casting: pseudo code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ray casting for tetrahedral grids . . . . . . . . . . . . . . . . . . . . . . . . . . . Classification of non-degenerate cases for the projectedtetrahedra algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Slab-based representation for pre-integrated texture-based volume rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Partitioning a volume data set into 43 bricks . . . . . . . . . . . . . . . . . For any viewing direction, two bricks are oriented along the viewing direction and the other two bricks are roughly perpendicular . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Single-brick volume rendering on FX 5950 Ultra and Radeon 9800 Pro . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Comparing volume rendering with single brick and 43 bricks on Radeon 9800 Pro . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Comparing volume rendering with single brick and 43 bricks on FX 5950 Ultra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Volume clipping in an MR data set . . . . . . . . . . . . . . . . . . . . . . . . . Depth structure of a clip geometry for a single pixel in the frame buffer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Illustration of depth-based two-pass rendering for a convex clip geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Illustration of depth-based volume probing for a convex clip geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Depth-based volume probing for an MR data set. . . . . . . . . . . . . .

19 20 21 22 24

2.7 2.8 2.9

2.10 2.11 2.12 2.13 2.14 2.15 2.16 2.17

28 31 34

35 37 37 37 38 40 42 43 44

242

List of Figures

2.18 Schematic illustration of the depth structure for volume cutting via 1/z mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.19 Comparison of trilinear interpolation and nearest-neighbor sampling for a voxelized cube-shaped clip object . . . . . . . . . . . . . . 2.20 Volumetric clipping in a medical MR data set . . . . . . . . . . . . . . . . 2.21 Comparison of volume clipping techniques . . . . . . . . . . . . . . . . . . . 2.22 Using slabs instead of slices for pre-integrated volume clipping . 2.23 Comparison of volumetric clipping approaches: slice-based vs. slab-based . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.24 Rendering of a bonsai with and without pre-integrated volumetric clipping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.25 Combining volume clipping and volume shading . . . . . . . . . . . . . . 2.26 Combining volume shading for a volumetric data set with surface-based shading on the boundary of the clip geometry . . . 2.27 Gradients in distance field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.28 Depth-based clipping in an illuminated CT data set . . . . . . . . . . 2.29 Clipping in an illuminated orbital data set . . . . . . . . . . . . . . . . . . . 2.30 Depth-sorted blending of footprints of four volume blocks . . . . . 2.31 Texture interpolation at a sub-volume border for a 1D case . . . . 2.32 Comparing the choice of quality levels for two sub-volume blocks 2.33 Radial distance volume combined with Perlin noise . . . . . . . . . . . 2.34 Parallel volume rendering of anatomical cryosections through the Visible Human male data set . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.35 Visualization of different timesteps of a time-dependent CFD simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.36 Scalability of the visualization system with the number of rendering nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.37 Rendering performance for a time-dependent data set . . . . . . . . . 3.1

45 49 50 51 53 54 54 56 57 60 63 64 70 71 72 74 74 75 76 76

Comparison of visualization techniques applied to the same 2D flow: arrow plot, streamlets, LIC, topology-based . . . . . . . . . . 88 3.2 Glyph-based 3D flow visualization, combined with illuminated streamlines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 3.3 Combination of streamlines, streamribbons, arrows, and color coding for a 3D flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 3.4 DirectX pixel shader code for 2D backward advection . . . . . . . . . 100 3.5 Pseudo code for 3D advection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 3.6 ARB fragment program for 3D texture advection . . . . . . . . . . . . . 101 3.7 Mapping between logical 3D memory and physical 2D memory by means of a lookup table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 3.8 Basic structure of the injection mechanism of IBFV . . . . . . . . . . . 107 3.9 Visualization of the interaction of a planar shock with a longitudinal vortex by 2D LEA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 3.10 Reordering of stack direction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 3.11 Visualization of a tornado data set by 3D texture advection . . . . 113

List of Figures

3.12 3.13 3.14 3.15 3.16

3.17 3.18 3.19 3.20 3.21 3.22 3.23 3.24 3.25 3.26 3.27 3.28 3.29 3.30 3.31 3.32 3.33 3.34 3.35 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8

243

Benard flow with λ2 vortex visualization . . . . . . . . . . . . . . . . . . . . 114 Volume rendering of 3D advection applied to a tornado data set 116 Numerical diffusion in a circular flow . . . . . . . . . . . . . . . . . . . . . . . 117 Numerical diffusion caused by linear interpolation . . . . . . . . . . . . 118 Comparison: traditional dye advection; the level-set approach without reinitialization; the level-set approach with reinitialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 Reinitialization of ∆φ back to unit distance along the intersection edge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Local distance field reinitialization in 2D . . . . . . . . . . . . . . . . . . . . 122 Level-set dye advection for a 3D vector field with a swirling feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 Change of topology of the dye-covered area . . . . . . . . . . . . . . . . . . 124 Truncation of streaklines in low-resolution level-set advection . . 124 Extensions for level-set dye advection . . . . . . . . . . . . . . . . . . . . . . . 125 Flow visualization on a curved surface for an automotive CFD simulation (data set courtesy of the BMW Group) . . . . . . . . . . . . 128 Coupled D-space and P-space representations . . . . . . . . . . . . . . . . 131 Pseudo code for the iterative computation of surface particle paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 Demonstrating the treatment of silhouette lines and object boundaries for surface flow on a teapot . . . . . . . . . . . . . . . . . . . . . . 133 Vertical flow projected onto two tori: LIC visualization vs. representation of the magnitude of the vectors . . . . . . . . . . . . . . . 134 Different shading approaches for object and flow rendering . . . . . 137 HLSL pixel shader code for one integration step in hybrid P/D-space particle tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Spacetime diagram with a single spatial and the temporal axis . 142 Computing two example texels of φi at time ti by adding all possible contributions from particles starting at previous times . 145 Spacetime sampling of trajectories for UFLIC . . . . . . . . . . . . . . . . 150 Visualization of a shock data set, comparing UFAC with velocity masking, UFAC without masking, LIC, and LEA . . . . . 154 Electric field of an oscillating electric dipole . . . . . . . . . . . . . . . . . . 155 Comparison of IBFV-guided techniques . . . . . . . . . . . . . . . . . . . . . 156 Calibration of perceived speeds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 Grouping through motion in graph visualization . . . . . . . . . . . . . . 172 LIC visualization of a 2D vector field with an additional data dimension represented by color coding . . . . . . . . . . . . . . . . . . . . . . 173 Surface flow visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 Structure of the color-based depth-cueing scheme . . . . . . . . . . . . . 179 Comparison of different color-based depth-cueing approaches . . . 181 Comparison of color-mapping techniques for depth-cueing . . . . . 182 Illustration of the change of saturation . . . . . . . . . . . . . . . . . . . . . . 183

244

List of Figures

4.9 4.10 4.11 4.12 4.13 4.14 4.15 4.16 4.17 4.18 4.19 4.20 4.21 4.22 4.23 4.24 4.25 4.26 4.27 4.28 4.29 4.30 4.31 4.32 4.33 4.34 4.35 4.36 4.37

Non-photorealistic rendering of an engine block with different depth-cues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 Depth-cueing for a cone tree representation in information visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 Terrain rendering with color-based depth-cueing . . . . . . . . . . . . . . 189 Semantic depth-cueing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 Phantom lines showing semi-transparent objects . . . . . . . . . . . . . . 193 Difference between traditional transparency blending and view-dependent transparency blending . . . . . . . . . . . . . . . . . . . . . . 196 Scenario with opaque objects embedded in the volume of a transparent object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 Rendering pipeline for opaque objects embedded inside transparent objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 Semi-transparent illustrations with mugs and a crank . . . . . . . . . 199 Semi-transparent illustrations with technical objects . . . . . . . . . . 200 Scenario with an opaque cuboid inside a transparent mug-like object of which the interior boundary is explicitly modeled . . . . 201 Comparison of computer-generated cutout and breakaway illustrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 Planar cutout. Signed distance values are attached to the vertices and linearly interpolated in between . . . . . . . . . . . . . . . . . 209 Jittering cutout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 A perturbation texture used for sawtooth-shaped boundaries . . . 209 Cutout illustrations with different rendering styles . . . . . . . . . . . . 210 Rendering pipeline for the breakaway technique . . . . . . . . . . . . . . 212 Breakaway illustrations with different rendering styles . . . . . . . . . 213 Illustration of threshold screening . . . . . . . . . . . . . . . . . . . . . . . . . . 219 G-buffer for NPR and image processing . . . . . . . . . . . . . . . . . . . . . 220 Advection of a screening matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 Combining different masks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 Computational scheme for one time step of frame-to-frame coherent halftoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 Alignment of the insert mask with the layout of the static screening mask to ensure the insertion of complete matrices . . . . 224 Sequence of a computer-generated animation with frame-to-frame coherent halftoning . . . . . . . . . . . . . . . . . . . . . . . . . 227 Screening illustration of a rotating pot . . . . . . . . . . . . . . . . . . . . . . 228 Possible data flow configurations that can be configured within an extended effect file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 Dilated feature lines, implemented as a G-buffer operation . . . . . 233 Combination of Robert’s cross edge detection method and halftoning by means of G-buffer operations . . . . . . . . . . . . . . . . . . 233

List of Tables

2.1 2.2 3.1 3.2

3.3

3.4 3.5 3.6

4.1 4.2 4.3

Performance measurements for volume clipping techniques in frames per second (NVIDIA GeForce 4 Ti 4600, 128 MB) . . . . . . 65 Portion of changes in quality classification at block faces . . . . . . 73 Performance for steady vs. unsteady 3D advection based on 2D textures (in frames per second) . . . . . . . . . . . . . . . . . . . . . . . . . 104 Comparison between the 3D advection method based on 2D textures and the advection method based on 3D textures on a Radeon X800 GPU (in frames per second) . . . . . . . . . . . . . . . . . . . 105 Performance measurements for Lagrangian-Eulerian advection in frames per second (ATI Radeon 8500 GPU vs. Athlon 1.2 GHz CPU) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 Rendering performance for level-set 2D dye advection in frames per second (ATI Radeon 9800 XT, 256 MB) . . . . . . . . . . . 127 Performance measurements for surface flow visualization in frames per second (ATI Radeon 9800 Pro, 256 MB) . . . . . . . . . . . 138 Performance measurements for one iteration step of the texture-based framework in frames per second (ATI Radeon 9700, 128 MB) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 Performance measurements for semi-transparent NPR (NVIDIA GeForce 3 Ti 200) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 Performance measurements for cutout and breakaway illustrations in frames per second (NVIDIA GeForce 4 Ti 4600) 215 Performance of frame-to-frame coherent halftoning in frames per second (ATI Radeon 9700 Pro) . . . . . . . . . . . . . . . . . . . . . . . . . 229

List of Color Plates

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

19 20

Comparison of trilinear interpolation and nearest-neighbor sampling for a voxelized cube-shaped clip object . . . . . . . . . . . . . . 277 Comparison of volume clipping techniques . . . . . . . . . . . . . . . . . . . 278 Comparison of volumetric clipping approaches: slice-based vs. slab-based . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 Rendering of a bonsai with and without pre-integrated volumetric clipping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 Combining volume clipping and volume shading . . . . . . . . . . . . . . 280 Depth-based clipping in an illuminated CT data set . . . . . . . . . . 280 Clipping in an illuminated orbital data set . . . . . . . . . . . . . . . . . . . 281 Radial distance volume combined with Perlin noise . . . . . . . . . . . 282 Parallel volume rendering of anatomical cryosections through the Visible Human male data set . . . . . . . . . . . . . . . . . . . . . . . . . . . 282 Visualization of different timesteps of a time-dependent CFD simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 Glyph-based 3D flow visualization, combined with illuminated streamlines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284 Combination of streamlines, streamribbons, arrows, and color coding for a 3D flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284 Visualization of the interaction of a planar shock with a longitudinal vortex by 2D LEA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 Visualization of a tornado data set by 3D texture advection . . . . 286 Benard flow with λ2 vortex visualization . . . . . . . . . . . . . . . . . . . . 287 Volume rendering of 3D advection applied to a tornado data set 287 Numerical diffusion in a circular flow . . . . . . . . . . . . . . . . . . . . . . . 288 Comparison: traditional dye advection; the level-set approach without reinitialization; the level-set approach with reinitialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 Level-set dye advection for a 3D vector field with a swirling feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 Change of topology of the dye-covered area . . . . . . . . . . . . . . . . . . 289

248

List of Color Plates

21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42

Extensions for level-set dye advection . . . . . . . . . . . . . . . . . . . . . . . 290 Different shading approaches for object and flow rendering . . . . . 291 Visualization of a shock data set, comparing UFAC with velocity masking, UFAC without masking, LIC, and LEA . . . . . 292 Comparison of IBFV-guided techniques . . . . . . . . . . . . . . . . . . . . . 293 Truncation of streaklines in low-resolution level-set advection . . 293 Calibration of perceived speeds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294 Grouping through motion in graph visualization . . . . . . . . . . . . . . 295 LIC visualization of a 2D vector field with an additional data dimension represented by color coding . . . . . . . . . . . . . . . . . . . . . . 295 Surface flow visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296 Comparison of different color-based depth-cueing approaches . . . 297 Comparison of color-mapping techniques for depth-cueing . . . . . 297 Non-photorealistic rendering of an engine block with different depth-cues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298 Depth-cueing for a cone tree representation in information visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 Terrain rendering with color-based depth-cueing . . . . . . . . . . . . . . 300 Semantic depth-cueing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301 Phantom lines showing semi-transparent objects . . . . . . . . . . . . . . 302 Difference between traditional transparency blending and view-dependent transparency blending . . . . . . . . . . . . . . . . . . . . . . 302 Semi-transparent illustrations with mugs and a crank . . . . . . . . . 303 Semi-transparent illustrations with technical objects . . . . . . . . . . 304 Comparison of computer-generated cutout and breakaway illustrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 Cutout illustrations with different rendering styles . . . . . . . . . . . . 305 Breakaway illustrations with different rendering styles . . . . . . . . . 306

References

1. K. Akeley. Reality Engine graphics. In SIGGRAPH 1993 Conference Proceedings, pages 109–116, 1993. 2. S. M. Anstis and P. Cavanagh. A minimum motion technique for judging equiluminance. In J. D. Mollon and E. T. Sharpe, editors, Colour Vision, pages 155–166, New York, 1983. Academic Press. 3. A. Appel. The notion of quantitative invisibility and the machine rendering of solids. In Proceedings of ACM National Conference, pages 387–393, 1967. 4. J. Arvo and D. Kirk. Particle transport and image synthesis. Computer Graphics (Proceedings of ACM SIGGRAPH 90), 24(4):63–66, 1990. 5. M. J. Bailey and D. Clark. Using ChromaDepth to obtain inexpensive singleimage stereovision for scientific visualization. Journal of Graphics Tools, 3(3):1–9, 1998. 6. D. C. Banks. Illumination in diverse codimensions. In SIGGRAPH 1994 Conference Proceedings, pages 327–334, 1994. 7. D. C. Banks and B. A. Singer. Vortex tubes in turbulent flows: Identification, representation, reconstruction. In IEEE Visualization ’94, pages 132–139, 1994. 8. C. B. Barber, D. P. Dobkin, and H. Huhdanpaa. The Quickhull algorithm for convex hulls. ACM Transactions on Mathematical Software, 22(4):469–483, 1996. 9. L. Bartram. Perceptual and interpretative properties of motion for information visualization. Technical Report CMPT-TR-1997-15, School of Computing Science, Simon Fraser University, 1997. 10. L. Bartram and C. Ware. Filtering and brushing with motion. Information Visualization, 1(1):66–79, 2002. 11. R. Batra and L. Hesselink. Feature comparisons of 3-D vector fields using earth mover’s distance. In IEEE Visualization ’99, pages 105–114, 1999. 12. H. Battke, D. Stalling, and H.-C. Hege. Fast line integral convolution for arbitrary surfaces in 3D. In H.-C. Hege and K. Polthier, editors, Visualization and Mathematics, pages 181–195. Springer, 1997. 13. D. Bauer and R. Peikert. Vortex tracking in scale-space. In EG / IEEE TCVG Symposium on Visualization ’02, pages 233–240, 2002. 14. B. Becker, D. A. Lane, and N. Max. Unsteady flow volumes. In IEEE Visualization ’95, pages 329–335, 1995.

250

References

15. J. Becker and M. Rumpf. Visualization of time-dependent velocity fields by texture transport. In EG Workshop on Visualization in Scientific Computing, pages 91–101, 1998. 16. R. A. Becker and W. S. Cleveland. Brushing scatterplots. Technometrics, 29(2):127–142, 1987. 17. R. D. Bergeron and G. G. Grinstein. A reference model for the visualization of multi-dimensional data. In Eurographics ’89, pages 393–399, 1989. 18. F. Bernardon, C. Pagot, J. Comba, and C. Silva. GPU-based tiled ray casting using depth peeling. Technical Report UUSCI-2004-006, SCI Institute, University of Utah, 2004. 19. P. Bhaniramka and Y. Demange. OpenGL Volumizer: A toolkit for high quality volume rendering of large data sets. In 2002 Symposium on Volume Visualization and Graphics, pages 45–53, 2002. 20. I. Biederman and G. Ju. Surface versus edge-based determinants of visual recognition. Cognitive Psychology, 20:38–64, 1988. 21. A. P. D. Binotto, J. L. D. Comba, and C. M. D. Freitas. Real-time volume rendering of time-varying data using a fragment-shader compression approach. In IEEE Symposium on Parallel and Large-Data Visualization and Graphics, pages 69–75, 2003. 22. I. Bitter, N. Neophytou, K. Mueller, and A. E. Kaufman. Squeeze: Numericalprecision-optimized volume rendering. In 2004 Eurographics / SIGGRAPH Workshop on Graphics Hardware, pages 25–34, 2004. 23. W. Blanke, C. Bajaj, D. Fussell, and X. Zhang. The Metabuffer: A scalable multiresolution multidisplay 3-D graphics system using commodity rendering engines. TR2000-16, University of Texas at Austin, 2000. 24. I. Boada, I. Navazo, and R. Scopigno. Multiresolution volume visualization with a texture-based octree. The Visual Computer, 17(3):185–197, 2001. 25. U. Bordoloi and H.-W. Shen. Hardware accelerated interactive vector field visualization: A level of detail approach. Computer Graphics Forum (Eurographics 2002), 21(3):605–614, 2002. 26. E. Boring and A. Pang. Directional flow visualization of vector fields. In IEEE Visualization ’96, pages 389–392, 1996. 27. R. P. Botchen, D. Weiskopf, and T. Ertl. Texture-based visualization of uncertainty in flow fields. In IEEE Visualization ’05, pages 647–654, 2005. 28. R. M. Boynton. Human Color Vision. Holt, Rinehart & Winston, New York, 1979. 29. W. Braje, B. Tjan, and G. Legge. Human efficiency for recognizing and detecting low-pass filtered objects. Vision Research, 35(21):2955–2996, 1995. 30. M. Brill, H. Hagen, H.-C. Rodrian, W. Djatschin, and S. V. Klimenko. Streamball techniques for flow vizualization. In IEEE Visualization ’94, pages 225– 231, 1994. 31. K. W. Brodlie. Chapter 3: Visualization techniques. In K. W. Brodlie, L. A. Carpenter, R. Earnshaw, J. R. Gallop, R. J. Hubbold, A. M. Mumford, C. D. Osland, and P. Quarendon, editors, Scientific Visualization – Techniques and Applications, pages 37–86. Springer, 1992. 32. R. Brown, B. Pham, and A. Maeder. A fuzzy model for scene decomposition based on preattentive visual features. In Proceedings of SPIE, 3644, pages 461–472, 1999.

References

251

33. R. Brown, B. Pham, and A. Maeder. Visual importance-biased image synthesis animation. In Proceedings of the 1st International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia, pages 63–70, 2003. 34. S. Bruckner, S. Grimm, A. Kanitsar, and E. Gr¨ oller. Illustrative contextpreserving volume rendering. In Eurovis 2005 (EG / IEEE VGTC Symposium on Visualization), pages 69–76, 2005. 35. S. Bruckner and E. Gr¨ oller. VolumeShop: An interactive system for direct volume illustration. In IEEE Visualization ’05, pages 671–678, 2005. 36. J. Brueghel the Elder. Christ preaching at the seaport. Oil on wood, Alte Pinakothek, Munich, 1598. 37. S. Bryson and C. Levit. The virtual wind tunnel. IEEE Computer Graphics and Applications, 12(4):25–34, 1992. 38. I. Buck, T. Foley, D. Horn, J. Sugerman, K. Fatahalian, M. Houston, and P. Hanrahan. Brook for GPUs: Stream computing on graphics hardware. ACM Transactions on Graphics, 23(3):777–786, 2004. 39. C. Buehler, M. Bosse, L. McMillan, S. J. Gortler, and M. F. Cohen. Unstructured lumigraph rendering. In SIGGRAPH 2001 Conference Proceedings, pages 425–432, 2001. 40. D. B¨ urkle, T. Preußer, and M. Rumpf. Transport and anisotropic diffusion in time-dependent flow visualization. In IEEE Visualization ’01, pages 61–67, 2001. 41. B. Cabral, N. Cam, and J. Foran. Accelerated volume rendering and tomographic reconstruction using texture mapping hardware. In 1994 Symposium on Volume Visualization, pages 91–98, 1994. 42. B. Cabral and C. Leedom. Highly parallel vector visualization using line integral convolution. In SIAM Conference on Parallel Processing for Scientific Computing, pages 802–807, 1995. 43. B. Cabral and L. C. Leedom. Imaging vector fields using line integral convolution. In SIGGRAPH 1993 Conference Proceedings, pages 263–270, 1993. 44. W. Cai and P.-A. Heng. Principal stream surfaces. In IEEE Visualization ’97, pages 75–80, 1997. 45. S. P. Callahan, M. Ikits, J. L. D. Comba, and C. T. Silva. Hardware-assisted visibility sorting for unstructured volume rendering. IEEE Transactions on Visualization and Computer Graphics, 11(3):285–295, 2005. 46. F. W. Campbell and L. Maffei. The influence of spatial frequency and contrast on the perception of moving patterns. Vision Research, 21:713–721, 1981. 47. K. M. Case and P. F. Zweifel. Linear Transport Theory. Addison–Wesley, Reading, MA, 1967. 48. P. Cavanagh and S. Anstis. The contribution of color to motion in normal and color-deficient observers. Vision Research, 31:2109–2148, 1991. 49. P. Cavanagh and O. E. Favreau. Color and luminance share a common motion pathway. Vision Research, 25:1595–1601, 1985. 50. P. Cavanagh, C. W. Tyler, and O. E. Favreau. Perceived velocity of moving chromatic gratings. Journal of the Optical Society of America A, 1(8):893–899, 1984. 51. E. Cerezo, F. Pérez, X. Pueyo, F. J. Seron, and F. X. Sillion. A survey on participating media rendering techniques. The Visual Computer, 21(5):303– 328, 2005.

252

References

52. A. Chalmers, S. Daly, A. McNamara, K. Myszkowski, H. Rushmeier, and T. Troscianko. Seeing is believing: Reality perception in modeling, rendering, and animation. SIGGRAPH 2001 Course #21 Notes, 2001. 53. S. Chandrasekhar. Radiative Transfer. Clarendon Press, Oxford, 1950. 54. M. Chen, S. Islam, M. W. Jones, P.-Y. Shen, D. Silver, S. J. Walton, and P. J. Willis. Deforming and animating discretely sampled object representations. In Eurographics 2005, State of the Art Reports, pages 113–140, 2005. 55. M. Chen, D. Silver, A. S. Winter, V. Singh, and N. Cornea. Spatial transfer functions – a unified approach to specifying deformation in volume modeling and animation. In Workshop of Volume Graphics 2003, pages 35–44, 2003. 56. W. Chen, L. Ren, M. Zwicker, and H. Pfister. Hardware-accelerated adaptive EWA volume splatting. In IEEE Visualization ’04, pages 67–74, 2004. 57. C. Christou, J. J. Koenderink, and A. J. van Doorn. Surface gradients, contours and the perception of surface attitude in images of complex scenes. Perception, 25:701–713, 1996. 58. P. Cignoni, C. Montani, D. Sarti, and R. Scopigno. Optimization of projective volume rendering. In EG Workshop on Visualization in Scientific Computing, pages 58–71, 1995. 59. J. Clyne and J. M. Dennis. Interactive direct volume rendering of time-varying data. In EG / IEEE TCVG Symposium on Visualization ’99, pages 109–120, 1999. 60. J. Comba, J. T. Klosowski, N. Max, J. S. B. Mitchell, C. T. Silva, and P. L. Williams. Fast polyhedral cell sorting for interactive rendering of unstructured grids. Computer Graphics Forum (Eurographics 1999), 18(3):369–376, 1999. 61. R. L. Cook. Shade trees. Computer Graphics (Proceedings of ACM SIGGRAPH 84), 18(3):223–231, 1984. 62. R. Crawfis and N. Max. Texture splats for 3D scalar and vector field visualization. In IEEE Visualization ’93, pages 261–266, 1993. 63. R. Crawfis, H.-W. Shen, and N. Max. Flow visualization techniques for CFD using volume rendering. In 9th International Symposium on Flow Visualization, pages 64/1–64/10, 2000. 64. R. Crawfis, D. Xue, and C. Zhang. Volume rendering using splatting. In C. D. Hansen and C. R. Johnson, editors, The Visualization Handbook, pages 175–188. Elsevier, Amsterdam, 2005. 65. S. J. Cropper. Velocity discrimination in chromatic gratings and beats. Vision Research, 34:41–48, 1994. 66. S. J. Cropper and A. M. Derrington. Motion of chromatic stimuli: First-order or second-order? Vision Research, 34:49–58, 1994. 67. S. J. Cropper and A. M. Derrington. Rapid colour-specific detection of motion in human vision. Nature, 379:72–74, 1996. 68. T. J. Cullip and U. Neumann. Accelerating volume reconstruction with 3D texture mapping hardware. Technical Report TR–93–027, Department of Computer Science at the University of North Carolina, Chapel Hill, 1993. 69. C. J. Curtis, S. E. Anderson, J. E. Seims, K. W. Fleischer, and D. H. Salesin. Computer-generated watercolor. In SIGGRAPH 1997 Conference Proceedings, pages 421–430, 1997. 70. W. J. Dally, P. Hanrahan, M. Erez, T. J. Knight, F. Labonte, A. Ahn, N. Jayasena, U. J. Kapasi, D. Das, J. Gummaraju, and I. Buck. Merrimac: Supercomputing with streams. In Proc. Supercomputing 2003, 2003.

References

253

71. S. Daly. The visible differences predictor: An algorithm for the assessment of image fidelity. In A. B. Watson, editor, Digital Images and Human Vision, pages 179–206, Cambridge, 1993. MIT Press. 72. E. R. Davies. Machine Vision: Theory, Algorithms, Practicalities. Academic Press, 2nd edition, 1996. 73. W. C. de Leeuw. Divide and conquer spot noise. In Supercomputing ’97 Conference, 1997. 74. W. C. de Leeuw, H.-G. Pagendarm, F. H. Post, and B. Walter. Visual simulation of experimental oil-flow visualization by spot noise images from numerical flow simulation. In EG Workshop on Visualization in Scientific Computing, pages 135–148, 1995. 75. W. C. de Leeuw, F. H. Post, and R. W. Vaatstra. Visualization of turbulent flow by spot noise. In EG Workshop on Virtual Environments and Scientific Visualization ’96, pages 286–295, 1996. 76. W. C. de Leeuw and R. van Liere. Collapsing flow topology using area metrics. In IEEE Visualization ’99, pages 349–354, 1999. 77. W. C. de Leeuw and R. van Liere. Visualization of global flow structures using multiple levels of topology. In EG / IEEE TCVG Symposium on Visualization ’99, pages 45–52, 1999. 78. W. C. de Leeuw and R. van Liere. Multi-level topology for flow visualization. Computers and Graphics, 24(3):325–331, 2000. 79. W. C. de Leeuw and R. van Liere. Spotting structure in complex time dependent flow. In H. Hagen, G. M. Nielson, and F. Post, editors, Proceedings Dagstuhl ’97 Scientific Visualization, pages 47–53. IEEE Computer Society, 2000. 80. W. C. de Leeuw and J. J. van Wijk. A probe for local flow field visualization. In IEEE Visualization ’93, pages 39–45, 1993. 81. W. C. de Leeuw and J. J. van Wijk. Enhanced spot noise for vector field visualization. In IEEE Visualization ’95, pages 233–239, 1995. 82. P. E. Debevec, Y. Yu, and G. D. Borshukov. Efficient view-dependent imagebased rendering with projective texture-mapping. In Eurographics Rendering Workshop ’98, pages 105–116, 1998. 83. A. M. Derrington and D. R. Badcock. The low level motion system has both chromatic and luminance inputs. Vision Research, 25:1879–1884, 1985. 84. A. M. Derrington and G. Henning. Detecting and discriminating the direction of motion of luminance and colour gratings. Vision Research, 33:799–811, 1993. 85. O. Deussen, S. Hiller, C. van Overveld, and T. Strothotte. Floating points: A method for computing stipple drawings. Computer Graphics Forum (Eurographics 2000), 19(3):40–51, 2000. 86. H. L. DeVries. The heredity of the relative numbers of red and green receptors in the human eye. Genetica, 24:199–212, 1948. 87. J. Diepstraten, D. Weiskopf, and T. Ertl. Automatic generation and nonphotorealistic rendering of 2+1D Minkowski diagrams. In WSCG 2002 Conference Proceedings, pages 139–146, 2002. 88. J. Diepstraten, D. Weiskopf, and T. Ertl. Transparency in interactive technical illustrations. Computer Graphics Forum (Eurographics 2002), 21(3):317–325, 2002. 89. J. Diepstraten, D. Weiskopf, and T. Ertl. Interactive cutaway illustrations. Computer Graphics Forum (Eurographics 2003), 22(3):523–532, 2003.

254

References

90. J. Diepstraten, D. Weiskopf, M. Kraus, and T. Ertl. Vragments – relocatability as an extension to programmable rasterization hardware. In WSCG 2004 Short Communication Papers Proceedings, pages 181–188, 2004. 91. U. Diewald, T. Preußer, and M. Rumpf. Anisotropic diffusion in vector field visualization on Euclidean domains and surfaces. IEEE Transactions on Visualization and Computer Graphics, 6(2):139–149, 2000. 92. K. R. Dobkins and T. D. Albright. What happens if it changes color when it moves?: The nature of chromatic input to macaque visual area MT. Journal of Neuroscience, 14:4854–4870, 1994. 93. H. Doleisch, M. Gasser, and H. Hauser. Interactive feature specification for focus+context visualization of complex simulation data. In EG / IEEE TCVG Symposium on Visualization ’03, pages 239–248, 2003. 94. D. Dooley and M. Cohen. Automatic illustration of 3D geometric models: Lines. Computer Graphics, 24(2):77–82, 1990. 95. J. Dorsey, A. Edelman, H. W. Jensen, J. Legakis, and H. Pedersen. Modeling and rendering of weathered stone. In SIGGRAPH 1999 Conference Proceedings, pages 225–234, 1999. 96. D. Dovey. Vector plots for irregular grids. In IEEE Visualization ’95, pages 248–253, 1995. 97. R. A. Drebin, L. Carpenter, and P. Hanrahan. Volume rendering. Computer Graphics (Proceedings of ACM SIGGRAPH 88), 22(4):65–74, 1988. 98. J. Driver, P. McLeod, and Z. Dienes. Motion coherence and conjunction search: Implications for guided search theory. Perception and Psychophysics, 51(1):79– 85, 1992. 99. R. O. Duda and P. E. Hart. Pattern Classification and Scene Analysis. Wiley, New York, 1973. 100. J. J. Duderstadt and W. R. Martin. Transport Theory. Wiley, New York, 1979. 101. R. Dumont, F. Pellacini, and J. A. Ferwerda. Perceptually-driven decision theory for interactive realistic rendering. ACM Transactions on Graphics, 22(2):152–181, 2003. 102. F. Durand, M. Agrawala, B. Gooch, V. Interrante, V. Ostromoukhov, and D. Zorin. Perceptual and artistic principles for effective computer depiction. SIGGRAPH 2002 Course #13 Notes, 2002. 103. P. Dutré, P. Bekaert, and K. Bala. Advanced Global Illumination. A K Peters, Natick, MA, 2003. 104. D. Ebert, K. Musgrave, D. Peachey, K. Perlin, and S. Worley. Texturing and Modeling. Academic Press, San Diego, 1998. 105. D. Ebert and P. Rheingans. Volume illustration: Non-photorealistic rendering of volume models. In IEEE Visualization ’00, pages 195–202, 2000. 106. D. S. Ebert, R. Yagel, J. Scott, and Y. Kurzion. Volume rendering methods for computational fluid dynamics visualization. In IEEE Visualization ’94, pages 232–239, 1994. 107. M. Eißele, S. Stegmaier, D. Weiskopf, and T. Ertl. Orientation as an additional user interface in mixed-reality environments. In Workshop “Virtuelle und Erweiterte Realit¨ at” der GI-Fachgruppe AR / VR, 2004. 108. M. Eißele, D. Weiskopf, and T. Ertl. Frame-to-frame coherent halftoning in image space. In Proceedings of Theory and Practice of Computer Graphics (TPCG), pages 188–195, 2004. 109. M. Eißele, D. Weiskopf, and T. Ertl. The G2 -buffer framework. In Proceedings of SimVis 2004, pages 287–298, 2004.

References

255

110. K. Engel. High-quality volume rendering. In SIGGRAPH 2004 Course #28 Notes: Real-Time Volume Graphics, pages 114–144, 2004. 111. K. Engel, M. Kraus, and T. Ertl. High-quality pre-integrated volume rendering using hardware-accelerated pixel shading. In 2001 Eurographics / SIGGRAPH Workshop on Graphics Hardware, pages 9–16, 2001. 112. D. P. Enright, S. R. Marschner, and R. P. Fedkiw. Animation and rendering of complex water surfaces. ACM Transactions on Graphics, 21(3):736–744, 2002. 113. A. Entezari, R. Scoggins, T. M¨ oller, and R. Machiraju. Shading for Fourier volume rendering. In 2002 Symposium on Volume Visualization, pages 131– 138, 2002. 114. G. Erlebacher, M. Y. Hussaini, and C.-W. Shu. Interaction of a shock with a longitudinal vortex. Journal of Fluid Mechanics, 337:129–153, 1997. 115. G. Erlebacher, B. Jobard, and D. Weiskopf. Flow textures. In C. D. Hansen and C. R. Johnson, editors, The Visualization Handbook, pages 279–293. Elsevier, Amsterdam, 2005. 116. C. Everitt. Interactive order-independent transparency. White paper, NVIDIA, 2001. 117. R. Fedkiw, T. Aslam, B. Merriman, and S. Osher. A non-oscillatory Eulerian approach to interfaces in multimaterial flows (the ghost fluid method). Journal of Computational Physics, 152(2):457–492, 1999. 118. R. Fernando and M. J. Kilgard. The Cg Tutorial: The Definitive Guide to Programmable Real-Time Graphics. Addison-Wesley, Boston, MA, 2003. 119. J. A. Ferwerda, S. N. Pattanaik, P. Shirley, and D. P. Greenberg. A model of visual adaptation for realistic image synthesis. In SIGGRAPH 1996 Conference Proceedings, pages 249–258, 1996. 120. J. A. Ferwerda, H. Rushmeier, and B. Watson. Psychometrics 101: How to design, conduct, and analyze perceptual studies of computer graphics visualization techniques. IEEE Visualization 2002 Tutorial #7, 2002. 121. J. A. Ferwerda, H. Rushmeier, and B. Watson. Frontiers in perceptually-based image synthesis: Modeling, rendering, display, validation. SIGGRAPH 2003 Course #3 Notes, 2003. 122. J. D. Foley, A. van Dam, S. K. Feiner, and J. F. Hughes. Computer Graphics: Principles and Practice. Addison-Wesley, Reading, MA, 1990. 123. L. K. Forssell and S. D. Cohen. Using line integral convolution for flow visualization: Curvilinear grids, variable-speed animation, and unsteady flows. IEEE Transactions on Visualization and Computer Graphics, 1(2):133–141, 1995. 124. N. Foster and R. Fedkiw. Practical animation of liquids. In SIGGRAPH 2001 Conference Proceedings, pages 23–30, 2001. 125. B. Freudenberg, M. Masuch, and T. Strothotte. Walk-through illustrations: Frame-coherent pen-and-ink style in a game engine. Computer Graphics Forum (Eurographics 2001), 20(3):184–191, 2001. 126. B. Freudenberg, M. Masuch, and T. Strothotte. Real-time halftoning: A primitive for non-photorealistic shading. In Eurographics Rendering Workshop ’02, pages 227–231, 2002. 127. S. F. Frisken, R. N. Perry, A. P. Rockwood, and T. R. Jones. Adaptively sampled distance fields: A general representation of shape for computer graphics. In SIGGRAPH 2000 Conference Proceedings, pages 249–254, 2000. 128. T. Fr¨ uhauf. Raycasting vector fields. In IEEE Visualization ’96, pages 115–120, 1996.

256

References

129. A. Fuhrmann and E. Gr¨ oller. Real-time techniques for 3D flow visualization. In IEEE Visualization ’98, pages 305–312, 1998. 130. A. Garcia and H.-W. Shen. An interleaved parallel volume renderer with PCclusters. In Workshop on Parallel Graphics and Visualization 2002, pages 51–59, 2002. 131. H. Garcke, T. Preußer, M. Rumpf, A. Telea, U. Weikard, and J. van Wijk. A continuous clustering method for vector fields. In IEEE Visualization ’00, pages 351–358, 2000. 132. H. Garcke, T. Preußer, M. Rumpf, A. Telea, U. Weikard, and J. J. van Wijk. A phase field model for continuous clustering on vector fields. IEEE Transactions on Visualization and Computer Graphics, 7(3):230–241, 2001. 133. M. P. Garrity. Raytracing irregular volume data. In Workshop on Volume Visualization ’90, pages 35–40, 1990. 134. K. R. Gegenfurtner and M. J. Hawken. Temporal and chromatic properties of motion mechanisms. Vision Research, 35:1547–1563, 1995. 135. T. Glau. Exploring instationary fluid flows by interactive volume movies. In EG / IEEE TCVG Symposium on Visualization ’99, pages 277–283, 1999. 136. A. Globus, C. Levit, and T. Lasinski. A tool for visualizing the topology of three-dimensional vector fields. In IEEE Visualization ’91, pages 33–40, 1991. 137. A. Gooch, B. Gooch, P. Shirley, and E. Cohen. A non-photorealistic lighting model for automatic technical illustration. In SIGGRAPH 1998 Conference Proceedings, pages 101–108, 1998. 138. B. Gooch and A. A. Gooch. Non-Photorealistic Rendering. A K Peters, Natick, MA, 2001. 139. B. Gooch, P.-P. Sloan, A. Gooch, P. Shirley, and R. Riesenfeld. Interactive technical illustration. In 1999 ACM Symposium on Interactive 3D Graphics, pages 31–38, 1999. 140. G. Gorla, V. Interrante, and G. Sapiro. Texture synthesis for 3D shape representation. IEEE Transactions on Visualization and Computer Graphics, 9(4):512–524, 2003. 141. S. Gottschalk, M. C. Lin, and D. Manocha. OBB-tree: A hierarchical structure for rapid interference detection. In SIGGRAPH 1996 Conference Proceedings, pages 171–180, 1996. 142. GPGPU (General-Purpose Computation on GPUs). General-purpose computation using graphics hardware. http://www.gpgpu.org, 2004. 143. J. Grant, G. Erlebacher, and J. O’Brien. Case study: Visualizing ocean flow vertical motions using Lagrangian-Eulerian time surfaces. In IEEE Visualization ’02, pages 529–532, 2002. 144. S. Green, D. Salesin, S. Schofield, A. Hertzmann, P. Litwinowicz, A. A. Gooch, C. Curtis, and B. Gooch. Non-photorealistic rendering. SIGGRAPH 1999 Course #17 Notes, 1999. 145. R. Grzeszczuk, C. Henn, and R. Yagel. Advanced geometric techniques for ray casting volumes. SIGGRAPH 1998 Course #4 Notes, 1998. 146. S. Guthe, S. Gumhold, and W. Straßer. Interactive visualization of volumetric vector fields using texture based particles. In WSCG 2002 Conference Proceedings, pages 33–41, 2002. 147. S. Guthe and W. Straßer. Real-time decompression and visualization of animated volume data. In IEEE Visualization ’01, pages 349–356, 2001. 148. S. Guthe, M. Wand, J. Gonser, and W. Straßer. Interactive rendering of large volume data sets. In IEEE Visualization ’02, pages 53–60, 2002.

References

257

149. R. B. Haber and D. A. McNabb. Visualization idioms: A conceptual model for visualization systems. In G. M. Nielson, B. D. Shriver, and L. J. Rosenblum, editors, Visualization in Scientific Computing, pages 74–93. IEEE Computer Society Press, Los Alamitos, CA, 1990. 150. J. Hamel, S. Schlechtweg, and T. Strothotte. An approach to visualizing transparency in computer-generated line drawings. In IEEE Proceedings of Information Visualization, pages 151–156, 1998. 151. P. Hanrahan and W. Krueger. Reflection from layered surfaces due to subsurface scattering. In SIGGRAPH 1993 Conference Proceedings, pages 165–174, 1993. 152. P. Hanrahan and J. Lawson. A language for shading and lighting calculations. Computer Graphics (Proceedings of ACM SIGGRAPH 90), 24(4):289– 298, 1990. 153. P. Hastreiter, H. C ¸ akmak, and T. Ertl. Intuitive and interactive manipulation of 3D datasets by integrating texture mapping based volume rendering into the OpenInventor class hierarchy. In T. Lehman, I. Scholl, and K. Spitzer, editors, Bildverarbeitung f¨ ur die Medizin - Algorithmen, Systeme, Anwendungen, pages 149–154, Universit¨ at Aachen, 1996. Verl. d. Augustinus Buchhandlung. 154. H. Hauser, L. Mroz, G. I. Bischi, and M. E. Gr¨ oller. Two-level volume rendering. IEEE Transactions on Visualization and Computer Graphics, 7(3):242– 252, 2001. 155. M. J. Hawken, K. R. Gegenfurtner, and C. Tang. Contrast dependence of colour and luminance motion mechanisms in human vision. Nature, 367:268– 270, 1994. 156. C. G. Healey, K. S. Booth, and J. T. Enns. Visualizing real-time multivariate data using preattentive processing. ACM Transactions on Modeling and Computer Simulation, 5(3):190–221, 1995. 157. C. G. Healey, K. S. Booth, and J. T. Enns. High-speed visual estimation using preattentive processing. ACM Transactions on Computer-Human Interaction, 3(2):107–135, 1996. 158. C. G. Healey, V. Interrante, and P. Rheingans. Fundamental issues of visual perception for effective image generation. SIGGRAPH 1999 Course #6 Notes, 1999. 159. B. Heckel, G. Weber, B. Hamann, and K. I. Joy. Construction of vector field hierarchies. In IEEE Visualization ’99, pages 19–25, 1999. 160. H.-C. Hege, T. H¨ ollerer, and D. Stalling. Volume rendering – mathematical models and algorithmic aspects. Report TR 93-7, ZIB (Konrad-Zuse-Zentrum), Berlin, 1993. 161. H.-C. Hege and D. Stalling. Fast LIC with piecewise polynomial filter kernels. In H.-C. Hege and K. Polthier, editors, Mathematical Visualization, pages 295– 314. Springer, 1998. 162. K. Hegeman, M. Ashikhmin, and S. Premoze. A lighting model for general participating media. In Proceedings of the 2005 Symposium on Interactive 3D Graphics and Games, pages 117–124, 2005. 163. W. Heidrich, R. Westermann, H.-P. Seidel, and T. Ertl. Applications of pixel textures in visualization and realistic image synthesis. In 1999 ACM Symposium on Interactive 3D Graphics, pages 127–134, 1999. 164. A. Helgeland and O. Andreassen. Visualization of vector fields using seed LIC and volume rendering. IEEE Transactions on Visualization and Computer Graphics, 10(6):673–682, 2004.

258

References

165. J. Helman and L. Hesselink. Representation and display of vector field topology in fluid flow data sets. Computer, 22(8):27–36, 1989. 166. A. Hertzmann and K. Perlin. Painterly rendering for video and interaction. In Proceedings of the Symposium on Non-Photorealistic Animation and Rendering, pages 7–12, 2000. 167. M. W. Hirsch. Differential Topology. Springer, Berlin, 6th edition, 1997. 168. M. W. Hirsch and S. Smale. Differential Equations, Dynamical Systems, and Linear Algebra. Academic Press, New York, 1974. 169. C. W. Hirt and B. D. Nichols. Volume of fluid (VOF) method for the dynamics of free boundaries. Journal of Computational Physics, 39(1):201–225, 1981. 170. E. R. S. Hodges. The Guild Handbook of Scientific Illustration. Van Nostrand Reinhold, New York, 1989. 171. M. Hopf, M. Luttenberger, and T. Ertl. Hierarchical splatting of scattered 4D data. IEEE Computer Graphics and Applications, 24(4):64–72, 2004. 172. J. P. M. Hultquist. Interactive numerical flow visualization using stream surfaces. Computing Systems in Engineering, 1(2-4):349–353, 1990. 173. G. Humphreys, M. Eldridge, I. Buck, G. Stoll, M. Everett, and P. Hanrahan. WireGL: A scalable graphics system for clusters. In SIGGRAPH 2001 Conference Proceedings, pages 129–140, 2001. 174. G. Humphreys, M. Houston, R. Ng, R. Frank, S. Ahern, P. Kirchner, and J. Klosowski. Chromium: A stream-processing framework for interactive rendering on clusters. ACM Transactions on Graphics, 21(3):693–702, 2002. 175. L. M. Hurvich. Color Vision. Sinauer Associates, Sunderland, 1981. 176. M. Y. Hussaini, G. Erlebacher, and B. Jobard. Real-time visualization of unsteady vector fields. In AIAA paper 2002-0749 (40th AIAA Aerospace Sciences Meeting), 2002. 177. V. Interrante. Illustrating surface shape in volume data via principal directiondriven 3D line integral convolution. In SIGGRAPH 1997 Conference Proceedings, pages 109–116, 1997. 178. V. Interrante and C. Grosch. Strategies for effectively visualizing 3D flow with volume LIC. In IEEE Visualization ’97, pages 421–424, 1997. 179. V. Interrante and C. Grosch. Visualizing 3D flow. IEEE Computer Graphics and Applications, 18(4):49–53, 1998. 180. V. Interrante, P. Rheingans, J. Ferwerda, R. Gossweiler, and C. Healey. Applications of visual perception in computer graphics. SIGGRAPH 1998 Course #32 Notes, 1998. 181. V. L. Interrante, H. Fuchs, and S. Pizer. Enhancing transparent skin surfaces with ridge and valley lines. In IEEE Visualization ’95, pages 52–59, 1995. 182. V. L. Interrante, H. Fuchs, and S. Pizer. Illustrating transparent surfaces with curvature-directed strokes. In IEEE Visualization ’96, pages 211–218, 1996. 183. V. L. Interrante, H. Fuchs, and S. Pizer. Conveying the 3D shape of smoothly curving transparent surfaces via texture. IEEE Transactions on Visualization and Computer Graphics, 3(2):98–117, 1997. 184. S. Islam, S. Dipankar, D. Silver, and M. Chen. Spatial and temporal splitting of scalar fields in volume graphics. In 2004 Symposium on Volume Visualization, pages 87–94, 2004. 185. L. Itti and C. Koch. Computational modelling of visual attention. Nature Reviews Neuroscience, 2(3):194–203, 2001.

References

259

186. H. W. Jensen and P. H. Christensen. Efficient simulation of light transport in scenes with participating media using photon maps. In SIGGRAPH 1998 Conference Proceedings, pages 311–320, 1998. 187. J. Jeong and F. Hussain. On the identification of a vortex. J. Fluid Mech., 285:69–94, 1995. 188. M. Jiang, R. Machiraju, and D. Thompson. Geometric verification of swirling features in flow fields. In IEEE Visualization ’02, pages 307–314, 2002. 189. M. Jiang, R. Machiraju, and D. Thompson. A novel approach to vortex core region detection. In EG / IEEE TCVG Symposium on Visualization ’02, pages 217–225, 2002. 190. B. Jobard, G. Erlebacher, and M. Y. Hussaini. Hardware-accelerated texture advection for unsteady flow visualization. In IEEE Visualization ’00, pages 155–162, 2000. 191. B. Jobard, G. Erlebacher, and M. Y. Hussaini. Lagrangian-Eulerian advection for unsteady flow visualization. In IEEE Visualization ’01, pages 53–60, 2001. 192. B. Jobard, G. Erlebacher, and M. Y. Hussaini. Lagrangian-Eulerian advection of noise and dye textures for unsteady flow visualization. IEEE Transactions on Visualization and Computer Graphics, 8(3):211–222, 2002. 193. B. Jobard and W. Lefer. Creating evenly-spaced streamlines of arbitrary density. In EG Workshop on Visualization in Scientific Computing, pages 43–55, 1997. 194. B. Jobard and W. Lefer. The motion map: Efficient computation of steady flow animations. In IEEE Visualization ’97, pages 323–328, 1997. 195. B. Jobard and W. Lefer. Unsteady flow visualization by animating evenlyspaced streamlines. Computer Graphics Forum (Eurographics 2000), 19(3):31– 39, 2000. 196. B. Jobard and W. Lefer. Multiresolution flow visualization. In WSCG 2001 Conference Proceedings, pages P34–P37, 2001. 197. C. Johnson. Top scientific visualization research problems. IEEE Computer Graphics and Applications, 24(4):13–17, 2004. 198. D. B. Judd. Report of U.S. Secretariate Committee on Colorimetry and Artificial Daylight. In CIE Proceedings. Bureau Central CIE, Paris, 1951. 199. J. T. Kajiya and B. P. von Herzen. Ray tracing volume densities. Computer Graphics (Proceedings of ACM SIGGRAPH 84), 18:165–174, 1984. 200. E. R. Kandel, J. H. Schwartz, and T. M. Jessell, editors. Essentials of Neural Science and Behavior. Appleton & Lange, Norwalk, 1995. 201. A. Kaufman and K. Mueller. Overview of volume rendering. In C. D. Hansen and C. R. Johnson, editors, The Visualization Handbook, pages 127–174. Elsevier, Amsterdam, 2005. 202. D. H. Kelly. Motion and vision. II. Stabilized spatio-temporal threshold surface. Journal of the Optical Society of America, 69(10):1340–1349, 1979. 203. C. A. Kelsey. Detection of visual information. In W. R. Hendee and P. Wells, editors, Perception of Visual Information, pages 30–51, Wien, 1993. Springer. 204. D. N. Kenwright. Automatic detection of open and closed separation and attachment lines. In IEEE Visualization ’98, pages 151–158, 1998. 205. D. N. Kenwright and R. Haimes. Automatic vortex core detection. IEEE Computer Graphics and Applications, 18(4):70–74, 1998. 206. D. N. Kenwright and D. A. Lane. Interactive time-dependent particle tracing using tetrahedral decomposition. IEEE Transactions on Visualization and Computer Graphics, 2(2):120–129, 1996.

260

References

207. J. Kessenich, D. Baldwin, and R. Rost. The OpenGL shading language (language version 1.10). http://www.opengl.org, 2004. 208. L. Khouas, C. Odet, and D. Friboulet. 2D vector field visualization using furlike texture. In EG / IEEE TCVG Symposium on Visualization ’99, pages 35–44, 1999. 209. M. Kilgard. Creating reflections and shadows using stencil buffers (NVIDIA technical demonstration). Game Developer Conference, 1999. 210. M. J. Kilgard, editor. NVIDIA OpenGL Extension Specifications. NVIDIA Corporation, 2004. 211. G. Kindlmann. Face-based luminance matching: Software. http://www. cs.utah.edu/~gk/lumFace, 2002. 212. G. Kindlmann and J. Durkin. Semi-automatic generation of transfer functions for direct volume rendering. In 1998 Symposium on Volume Visualization, pages 79–86, 1998. 213. G. Kindlmann, E. Reinhard, and S. Creem. Face-based luminance matching for perceptual colormap generation. In IEEE Visualization ’02, pages 299–306, 2002. 214. M.-H. Kiu and D. C. Banks. Multi-frequency noise for LIC. In IEEE Visualization ’96, pages 121–126, 1996. 215. R. V. Klassen and S. J. Harrington. Shadowed hedgehogs: A technique for visualizing 2D slices of 3D vector fields. In IEEE Visualization ’91, pages 148–153, 1991. 216. T. Klein, M. Eißele, D. Weiskopf, and T. Ertl. Simulation and high-quality rendering of incompressible fluids in real time. In Vision, Modeling, and Visualization VMV ’03 Conference, pages 365–373, 2003. 217. J. Kniss, G. Kindlmann, and C. Hansen. Interactive volume rendering using multi-dimensional transfer functions and direct manipulation widgets. In IEEE Visualization ’01, pages 255–262, 2001. 218. J. Kniss, G. Kindlmann, and C. Hansen. Multidimensional transfer functions for interactive volume rendering. IEEE Transactions on Visualization and Computer Graphics, 8(3):270–285, 2002. 219. J. Kniss, P. McCormick, A. McPherson, J. Ahrens, J. Painter, A. Keahey, and C. Hansen. Interactive texture-based volume rendering for large data sets. IEEE Computer Graphics and Applications, 21(4):52–61, 2001. 220. J. Kniss, S. Premoze, C. Hansen, and D. Ebert. Interactive translucent volume rendering and procedural modeling. In IEEE Visualization ’02, pages 109–116, 2002. 221. J. Kniss, S. Premoze, C. Hansen, P. Shirley, and A. McPherson. A model for volume lighting and modeling. IEEE Transactions on Visualization and Computer Graphics, 9(2):150–162, 2003. 222. G. Knittel. Using pre-integrated transfer functions in an interactive software system for volume rendering. In Eurographics 2002 Short Presentations, pages 199–123, 2002. 223. O. Konrad-Verse, B. Preim, and A. Littmann. Virtual resection with a deformable cutting plane. In Proceedings of SimVis 2004, pages 203–214, 2004. 224. R. Kosara, S. Miksch, and H. Hauser. Semantic depth of field. In IEEE Symposium on Information Visualization 2001 (InfoVis 2001), pages 97–104, 2001.

References

261

225. M. Kraus. Direct Volume Visualization of Geometrically Unpleasant Meshes. Dissertation, University of Stuttgart, Faculty of Computer Science, Electrical Engineering and Information Technology, 2003. 226. M. Kraus and T. Ertl. Cell-projection of cyclic meshes. In IEEE Visualization ’01, pages 215–222, 2001. 227. M. Kraus and T. Ertl. Adaptive texture maps. In 2002 Eurographics / SIGGRAPH Workshop on Graphics Hardware, pages 7–15, 2002. 228. M. Kraus and T. Ertl. Pre-integrated volume rendering. In C. D. Hansen and C. R. Johnson, editors, The Visualization Handbook, pages 211–228. Elsevier, Amsterdam, 2005. 229. M. Kraus, W. Qiao, and D. S. Ebert. Projecting tetrahedra without rendering artifacts. In IEEE Visualization ’04, pages 27–34, 2004. 230. W. Krueger. The application of transport theory to visualization of 3D scalar data fields. In IEEE Visualization ’90, pages 273–280, 1990. 231. W. Krueger. Volume rendering and data feature enhancement. Computer Graphics, 24(5):21–26, 1990. 232. J. Kr¨ uger and R. Westermann. Acceleration techniques for GPU-based volume rendering. In IEEE Visualization ’03, pages 287–292, 2003. 233. Y. Kurzion and R. Yagel. Space deformation using ray deflectors. In Rendering Techniques ’95 (Proceedings of the Eurographics Workshop on Rendering), pages 21–30, 1995. 234. Y. Kurzion and R. Yagel. Interactive space deformation with hardware-assisted rendering. IEEE Computer Graphics and Applications, 17(5):66–77, 1997. 235. P. Lacroute and M. Levoy. Fast volume rendering using a shear-warp factorization of the viewing transformation. In SIGGRAPH 1994 Conference Proceedings, pages 451–458, 1994. 236. A. Lake, C. Marshall, M. Harris, and M. Blackstein. Stylized rendering techniques for scalable real-time 3D animation. In Proceedings of the Symposium on Non-Photorealistic Animation and Rendering, pages 13–20, 2000. 237. E. C. LaMar, B. Hamann, and K. I. Joy. Multiresolution techniques for interactive texture-based volume visualization. In IEEE Visualization ’99, pages 355–362, 1999. 238. LAM/MPI Parallel Computing. http://www.lam-mpi.org. 239. S. Lang. Differential and Riemannian Manifolds. Springer, New York, 3rd edition, 1995. 240. R. S. Laramee, H. Hauser, H. Doleisch, B. Vrolijk, F. H. Post, and D. Weiskopf. The state of the art in flow visualization: Dense and texture-based techniques. Computer Graphics Forum, 23(2):143–161, 2004. 241. R. S. Laramee, B. Jobard, and H. Hauser. Image space based visualization of unsteady flow on surfaces. In IEEE Visualization ’03, pages 131–138, 2003. 242. R. S. Laramee, D. Weiskopf, J. Schneider, and H. Hauser. Investigating swirl and tumble flow with a comparison of visualization techniques. In IEEE Visualization ’04, pages 51–58, 2004. 243. Y. Lavin, R. Batra, and L. Hesselink. Feature comparisons of vector fields using earth mover’s distance. In IEEE Visualization ’98, pages 103–109, 1998. 244. H. Levkowitz, V. Interrante, and H. P. Meinzer. Perception for visualization: From design to evaluation. IEEE Visualization 1998 Tutorial #4, 1998. 245. M. Levoy. Display of surfaces from volume data. IEEE Computer Graphics and Applications, 9(3):29–37, 1988.

262

References

246. M. Levoy. Volume rendering using the Fourier projection-slice theorem. In Proceedings of Graphics Interface, pages 61–69, 1992. 247. B. Levy, S. Petitjean, N. Ray, and J. Maillot. Least squares conformal maps for automatic texture atlas generation. ACM Transactions on Graphics, 21(3):362– 371, 2002. 248. W. Li and A. Kaufman. Texture partitioning and packing for accelerating texture-based volume rendering. In Proceedings of Graphics Interface, pages 81–88, 2003. 249. S. Limoges, C. Ware, and W. Knight. Displaying correlations using position, motion, point size or point color. In Proceedings of Graphics Interface, pages 262–265, 1989. 250. Z. P. Liu and R. J. Moorhead. AUFLIC: An accelerated algorithm for unsteady flow line integral convolution. In EG / IEEE TCVG Symposium on Visualization ’02, pages 43–52, 2002. 251. M. Livingstone and D. Hubel. Psychophysical evidence for separate channels for the perception of form, color, movement, and depth. Journal of Neuroscience, 7:3416–346, 1987. 252. S. K. Lodha, A. Pang, R. E. Sheehan, and C. M. Wittenbrink. UFLOW: Visualizing uncertainty in fluid flow. In IEEE Visualization ’96, pages 249– 254, 1996. 253. S. K. Lodha, J. C. Renteria, and K. M. Roskin. Topology preserving compression of 2D vector fields. In IEEE Visualization ’00, pages 343–350, 2000. 254. H. L¨ offelmann and E. Gr¨ oller. Enhancing the visualization of characteristic structures in dynamical systems. In EG Workshop on Visualization in Scientific Computing, pages 59–68, 1998. 255. H. L¨ offelmann, L. Mroz, E. Gr¨ oller, and W. Purgathofer. Stream arrows: Enhancing the use of stream surfaces for the visualization of dynamical systems. The Visual Computer, 13(8):359–369, 1997. 256. N. K. Logothetis, P. H. Schiller, E. Charles, and A. C. Hurlbert. Perceptual deficits and the activity of the color-opponent and broad-band pathways at isoluminance. Science, 247(4939):214–217, 1990. 257. S. Lombeyda, L. Moll, M. Shand, D. Breen, and A. Heirich. Scalable interactive volume rendering using off-the-shelf components. In IEEE Symposium on Parallel and Large-Data Visualization and Graphics, pages 115–121, 2001. 258. Z.-L. Lu, L. Lesmes, and G. Sperling. Perceptual motion standstill from rapidly moving chromatic displays. In Proceedings of National Academy of Science, 96, pages 15374–15379, 1999. 259. Z.-L. Lu and G. Sperling. Attention-generated apparent motion. Nature, 379:237–239, 1995. 260. Z.-L. Lu and G. Sperling. Sensitive calibration and measurement procedures based on the amplification principle in motion perception. Vision Research, 41:2355–2374, 2001. 261. Z.-L. Lu and G. Sperling. Three-systems theory of human visual motion perception: Review and update. Journal of the Optical Society of America A, 18(9):2331–2370, 2001. 262. J. Lubin. A visual system discrimination model for imaging system design and evaluation. In E. Peli, editor, Visual Models for Target Detection and Recognition, pages 245–283, New York, 1995. World Scientific.

References

263

263. E. B. Lum, K.-L. Ma, and J. Clyne. A hardware-assisted scalable solution for interactive volume rendering of time-varying data. IEEE Transactions on Visualization and Computer Graphics, 8(3):286–301, 2002. 264. E. B. Lum, A. Stompel, and K.-L. Ma. Using motion to illustrate static 3D shape—kinetic visualization. IEEE Transactions on Visualization and Computer Graphics, 9(2):115–126, 2003. 265. K.-L. Ma, J. V. Rosendale, and W. Vermeer. 3D shock wave visualization on unstructured grids. In 1996 Volume Visualization Symposium, pages 87–94, 1996. 266. M. Magall´ on, M. Hopf, and T. Ertl. Parallel volume rendering using PC graphics hardware. In Proceedings of Pacific Graphics, pages 384–389, 2001. 267. M. Magnor, K. Hildebrand, A. Lintu, and A. Hanson. Reflection nebula visualization. In IEEE Visualization ’05, pages 255–262, 2005. 268. T. Malzbender. Fourier volume rendering. ACM Transactions on Graphics, 12(3):233–250, 1993. 269. X. Mao, Y. Hatanaka, H. Higashida, and A. Imamiya. Image-guided streamline placement on curvilinear grid surfaces. In IEEE Visualization ’98, pages 135– 142, 1998. 270. X. Mao, L. Hong, A. Kaufman, N. Fujita, and M. Kikukawa. Multi-granularity noise for curvilinear grid LIC. In Proceedings of Graphics Interface, pages 193–200, 1998. 271. X. Mao, M. Kikukawa, N. Fujita, and A. Imamiya. Line integral convolution for 3D surfaces. In EG Workshop on Visualization in Scientific Computing, pages 57–69, 1997. 272. W. R. Mark and K. Proudfoot. The F-buffer: A rasterization-order FIFO buffer for multi-pass rendering. In 2001 Eurographics / SIGGRAPH Workshop on Graphics Hardware, pages 57–64, 2001. 273. L. Markosian, M. A. Kowalski, S. J. Trychin, L. D. Bourdev, D. Goldstein, and J. F. Hughes. Real-time nonphotorealistic rendering. In SIGGRAPH 1997 Conference Proceedings, pages 415–420, 1997. 274. J. Martin. Technical Illustration: Material, Methods, and Techniques. Macdonald and Co Publishers, 1989. 275. N. Max. Optical models for direct volume rendering. IEEE Transactions on Visualization and Computer Graphics, 1(2):99–108, 1995. 276. N. Max and B. Becker. Flow visualization using moving textures. In Proceedings of the ICASE/LaRC Symposium on Visualizing Time-Varying Data, pages 77–87, 1995. 277. N. Max, B. Becker, and R. Crawfis. Flow volumes for interactive vector field visualization. In IEEE Visualization ’93, pages 19–24, 1993. 278. N. Max, R. Crawfis, and C. Grant. Visualizing 3D velocity fields near contour surfaces. In IEEE Visualization ’94, pages 248–255, 1994. 279. N. Max, R. Crawfis, and D. Williams. Visualizing wind velocities by advecting cloud textures. In IEEE Visualization ’92, pages 179–184, 1992. 280. N. Max, P. Hanrahan, and R. Crawfis. Area and volume coherence for efficient visualization of 3D scalar functions. Computer Graphics (San Diego Workshop on Volume Visualization), 24(5):27–33, 1990. 281. M. J. McGuffin, L. Tancau, and R. Balakrishnan. Using deformations for browsing volumetric data. In IEEE Visualization ’03, pages 401–408, 2003.

264

References

282. A. McNamara, A. G. Chalmers, T. Troscianko, and I. Gilchrist. Comparing real and synthetic scenes using human judgements of lightness. In Eurographics Workshop on Rendering, pages 207–219, 2000. 283. B. J. Meier. Painterly rendering for animation. In SIGGRAPH 1996 Conference Proceedings, pages 477–484, 1996. 284. M. Meißner, M. Guthe, and W. Straßer. Interactive lighting models and preintegration for volume rendering on PC graphics accelerators. In Proceedings of Graphics Interface, pages 209–218, 2002. 285. M. Meißner, U. Hoffmann, and W. Straßer. Enabling classification and shading for 3D-texture based volume rendering using OpenGL and extensions. In IEEE Visualization ’99, pages 207–214, 1999. 286. M. Meißner, U. Kanus, G. Wetekam, J. Hirche, A. Ehlert, W. Straßer, M. Doggett, P. Forthmann, and R. Proksa. VIZARD II: A reconfigurable interactive volume rendering system. In 2002 Eurographics / SIGGRAPH Workshop on Graphics Hardware, pages 137–146, 2002. 287. A. B. Metha and K. T. Mullen. Failure of direction discrimination at detection threshold for both fast and slow chromatic motion. Journal of the Optical Society of America A, 15(12):2945–2950, 1998. 288. Microsoft. DirectX. http://www.microsoft.com/directx, 2004. 289. J. W. Milnor. Topology from the Differentiable Viewpoint. University Press of Virginia, Charlottesville, 1965. 290. J. L. Mitchell. Image processing with 1.4 pixel shaders in Direct3D. In W. F. Engel, editor, Direct3D ShaderX: Vertex and Pixel Shader Tips and Tricks, pages 258–269, Plano, TX, 2002. Wordware. 291. J. L. Mitchell, C. Brennan, and D. Card. Real-time image-space outlining for non-photorealistic rendering. SIGGRAPH 2002 Sketch, 2002. 292. L. Moll, A. Heirich, and M. Shand. Sepia: Scalable 3D compositing using PCI Pamette. In IEEE Symposium on FPGAs for Custom Computing Machines, pages 146–155, 1999. 293. S. Molnar, M. Cox, D. Ellsworth, and H. Fuchs. A sorting classification of parallel rendering. IEEE Computer Graphics and Applications, 14(4):23–32, 1994. 294. K. Mueller and R. Crawfis. Eliminating poping artifacts in sheet buffer-based splatting. In IEEE Visualization ’98, pages 239–245, 1998. 295. K. Mueller, T. M¨ oller, and R. Crawfis. Splatting without the blur. In IEEE Visualization ’99, pages 363–370, 1999. 296. K. Mueller, N. Shareef, J. Huang, and R. Crawfis. High-quality splatting on rectilinear grids with efficient culling of occluded voxels. IEEE Transactions on Visualization and Computer Graphics, 5(2):116–134, 1999. 297. K. T. Mullen. The contrast sensitivity of human colour vision to red–green and blue–yellow chromatic gratings. The Journal of Physiology, 359(1):381– 400, 1985. 298. K. T. Mullen and C. L. Baker, Jr. A motion aftereffect from an isoluminant stimulus. Vision Research, 25:685–688, 1985. 299. K. T. Mullen and J. C. Boulton. Absence of smooth motion perception in color vision. Vision Research, 32:483–488, 1992. 300. K. T. Mullen, T. Yoshizawa, and C. L. Baker, Jr. Luminance mechanisms mediate the motion of red-green isoluminant gratings: The role of “temporal chromatic aberration”. Vision Research, 43:1235–1247, 2003.

References

265

301. K. Myszkowski, P. Rokita, and T. Tawara. Perception-based fast rendering and antialiasing of walkthrough sequences. IEEE Transactions on Visualization and Computer Graphics, 6(4):360–379, 2000. 302. K. Myszkowski, T. Tawara, H. Akamine, and H.-P. Seidel. Perception-guided global illumination solution for animation rendering. In SIGGRAPH 2001 Conference Proceedings, pages 221–230, 2001. 303. K. Nakayama and G. H. Silverman. Serial and parallel processing of visual feature conjunctions. Nature, 320:264–265, 1986. 304. N. Neophytou and K. Mueller. Post-convolved splatting. In EG / IEEE TCVG Symposium on Visualization ’03, pages 223–230, 2003. 305. U. Neumann. Parallel volume-rendering algorithm performance on meshconnected multicomputers. In IEEE/SIGGRAPH Parallel Rendering Symposium, pages 97–104, 1993. 306. D. Nguyen-Tri and J. Faubert. Perceived speed of drifting chromatic gratings is mechanism dependent. Vision Research, 42:2073–2079, 2002. 307. G. M. Nielson, H. Hagen, and H. M¨ uller. Scientific Visualization: Overviews, Methodologies, and Techniques. IEEE Computer Society Press, Los Alamitos, CA, 1997. 308. G. M. Nielson and I.-H. Jung. Tools for computing tangent curves for linearly varying vector fields over tetrahedral domains. IEEE Transactions on Visualization and Computer Graphics, 5(4):360–372, 1999. 309. W. F. Noh and P. R. Woodward. SLIC (Simple line interface calculation). In Lecture Notes in Physics 59: Proceedings of the Fifth International Conference on Numerical Methods in Fluid Dynamics, pages 330–340. Springer, 1976. 310. F. S. Nooruddin and G. Turk. Interior/exterior classification of polygonal models. In IEEE Visualization ’00, pages 415–422, 2000. 311. NVIDIA. Cg homepage. http://developer.nvidia.com/cg, 2004. 312. NVIDIA. Cg toolkit user’s manual: A developer’s guide to programmable graphics (version 1.2). http://developer.nvidia.com/cg, 2004. 313. A. Okada and D. Lane. Enhanced line integral convolution with flow feature detection. In Proceedings of IS&T/SPIE Electronic Imaging ’97, pages 206– 217, 1997. 314. K. Ono, H. Matsumoto, and R. Himeno. Visualization of thermal flows in an automotive cabin with volume rendering method. In EG / IEEE TCVG Symposium on Visualization ’01, pages 301–308, 2001. 315. Open Scene Graph. Web page. http://www.openscenegraph.org, 2004. 316. OpenSG Web Page. http://www.opensg.org, 2003. 317. S. Osher and R. Fedkiw. Level Set Methods and Dynamic Implicit Surfaces. Springer, New York, 2003. 318. V. Ostromoukhov and R. D. Hersch. Artistic screening. In SIGGRAPH 1995 Conference Proceedings, pages 219–228, 1995. 319. J. Palmer, L. A. Mobley, and D. Y. Teller. Motion at isoluminance: Discrimination/detection ratios and the summation of luminance and chromatic signals. Journal of the Optical Society of America A, 10:1353–1362, 1993. 320. A. T. Pang, C. M. Wittenbrink, and S. K. Lodha. Approaches to uncertainty visualization. The Visual Computer, 13(8):370–390, 1997. 321. T. V. Papathomas, A. Gorea, and B. Julesz. Two carriers for motion perception: Color and luminance. Vision Research, 31:1883–1892, 1991.

266

References

322. S. N. Pattanaik, J. A. Ferwerda, D. A. Greenberg, and M. D. Fairchild. A multiscale model of adaptation and spatial vision for realistic imaging. In SIGGRAPH 1998 Conference Proceedings, pages 287–298, 1998. 323. PC Cluster Consortium. http://www.pccluster.org, 2004. 324. M. S. Peercy, M. Olano, J. Airey, and U. Ungar. Interactive multi-pass programmable shading. In SIGGRAPH 2000 Conference Proceedings, pages 425– 432, 2000. 325. R. Peikert and M. Roth. The “parallel vectors” operator – a vector field visualization primitive. In IEEE Visualization ’99, pages 263–270, 1999. 326. A. Peleg and U. Weiser. MMX technology extension to the Intel architecture. IEEE Micro, 16(4):42–50, 1996. 327. J. Percy. OpenGL extensions. ATI Presentation at SIGGRAPH 2003, access via http://www.ati.com/developer, 2003. 328. K. Perlin. An image synthesizer. Computer Graphics (Proceedings of ACM SIGGRAPH 85), 19(3):287–296, 1985. 329. R. N. Perry and S. F. Frisken. Kizamu: a system for sculpting digital characters. In SIGGRAPH 2001 Conference Proceedings, pages 47–56, 2001. 330. H. Pfister. Hardware-accelerated volume rendering. In C. D. Hansen and C. R. Johnson, editors, The Visualization Handbook, pages 229–258. Elsevier, Amsterdam, 2005. 331. H. Pfister, B. Lorensen, C. Baja, G. Kindlmann, W. Schroeder, L. S. Avila, K. Martin, R. Machiraju, and J. Lee. Visualization viewpoints: The transfer function bake-off. IEEE Computer Graphics and Applications, 21(3):16–23, 2001. 332. B. Pflesser, U. Tiede, and K. H. H¨ ohne. Specification, modelling and visualization of arbitrarily shaped cut surfaces in the volume model. In Medical Image Computing and Computer-Assisted Intervention, Proc. MICCAI ’98, volume 1496, pages 853–860. Springer, Berlin, 1998. 333. M. Pharr and G. Humphreys. Physically Based Rendering. Elsevier, Morgan Kaufmann, Amsterdam, 2004. 334. F. H. Post, B. Vrolijk, H. Hauser, R. S. Laramee, and H. Doleisch. The state of the art in flow visualization: Feature extraction and tracking. Computer Graphics Forum, 22(4):775–792, 2003. 335. E. Praun, H. Hoppe, M. Webb, and A. Finkelstein. Real-time hatching. In SIGGRAPH 2001 Conference Proceedings, pages 579–584, 2001. 336. V. S. Ramachandran and R. L. Gregory. Does colour provide an input to human motion perception? Nature, 275:55–57, 1978. 337. A. Rappoport and S. Spitz. Interactive Boolean operations for conceptual design for 3-D solids. In SIGGRAPH 1997 Conference Proceedings, pages 269– 278, 1997. 338. R. Raskar. Hardware support for non-photorealistic rendering. In ACM SIGGRAPH/Eurographics Workshop on Graphics Hardware, pages 41–46, 2001. 339. A. Rezec, B. Krekelberg, and K. Dobkins. Effects of contrast and attention on chromatic vs. achromatic motion processing [Abstract]. Journal of Vision, 4(8):553a, 2004. 340. C. Rezk-Salama, K. Engel, M. Bauer, G. Greiner, and T. Ertl. Interactive volume rendering on standard PC graphics hardware using multi-textures and multi-stage rasterization. In 2000 Eurographics / SIGGRAPH Workshop on Graphics Hardware, pages 109–118, 2000.

References

267

341. C. Rezk-Salama, P. Hastreiter, C. Teitzel, and T. Ertl. Interactive exploration of volume line integral convolution based on 3D-texture mapping. In IEEE Visualization ’99, pages 233–240, 1999. 342. C. Rezk-Salama, M. Scheuering, G. Soza, and G. Greiner. Fast volumetric deformation on general purpose hardware. In 2001 Eurographics / SIGGRAPH Workshop on Graphics Hardware, pages 17–24, 2001. 343. P. Rheingans. Dynamic color mapping of bivariate qualitative data. In IEEE Visualization ’97, pages 159–166, 1997. 344. P. Rheingans and C. Landreth. Perceptual principles for effective visualizations. In G. Grinstein and H. Levkowitz, editors, Perceptual Issues in Visualization, pages 59–74. Springer Verlag, 1995. 345. K. Riley, D. Ebert, C. Hansen, and J. Levit. Visually accurate multi-field weather visualization. In IEEE Visualization ’03, pages 279–286, 2003. 346. K. Riley, D. S. Ebert, M. Kraus, J. Tessendorf, and C. Hansen. Efficient rendering of atmospheric phenomena. In Eurographics Symposium on Rendering, pages 375–386, 2004. 347. C. P. Risquet. Visualizing 2D flows: Integrate and draw. In EG Workshop on Visualization in Scientific Computing (Participant Edition), pages 57–67, 1998. 348. G. G. Robertson, J. D. Mackinlay, and S. K. Card. Cone trees: Animated 3D visualizations of hierarchical information. In Proceedings of ACM CHI’91 Conference on Human Factors in Computing Systems, pages 189–194, 1991. 349. B. Rogowitz and L. Treinish. How NOT to lie with visualization. Computers in Physics, 10(3):268–274, 1996. 350. G. G. Rosa, E. B. Lum, K.-L. Ma, and K. Ono. An interactive volume visualization system for transient flow analysis. In Workshop of Volume Graphics 2003, pages 137–144, 2003. 351. D. Rose, S. Stegmaier, G. Reina, D. Weiskopf, and T. Ertl. Non-invasive adaptation of black-box user interfaces. In Proceedings of the Australasian User Interface Conference (AUIC) 2003, pages 19–24, 2003. 352. M. Rotard, D. Weiskopf, and T. Ertl. Curriculum for a course on scientific visualization. In Eurographics / ACM SIGGRAPH Workshop on Computer Graphics Education, 2004. 353. M. Rotard, D. Weiskopf, and T. Ertl. An introductory course on graphicalinteractive systems: Combining human-computer interaction and computer graphics. In Eurographics / ACM SIGGRAPH Workshop on Computer Graphics Education, 2004. 354. M. Roth and R. Peikert. Flow visualization for turbomachinery design. In IEEE Visualization ’96, pages 381–384, 1996. 355. M. Roth and R. Peikert. A higher-order method for finding vortex core lines. In IEEE Visualization ’98, pages 143–150, 1998. 356. S. R¨ ottger, S. Guthe, D. Weiskopf, T. Ertl, and W. Straßer. Smart hardwareaccelerated volume rendering. In EG / IEEE TCVG Symposium on Visualization ’03, pages 231–238, 2003. 357. S. R¨ ottger, M. Kraus, and T. Ertl. Hardware-accelerated volume and isosurface rendering based on cell-projection. In IEEE Visualization ’00, pages 109–116, 2000. 358. S. R¨ ottger and B. Thomandl. Three little bonsais that did survive CT scanning and contrast agent but not the graphix lab conditions. Data set available through http://www.volvis.org, 2004.

268

References

359. H. Rushmeier, G. J. Ward, C. Piatko, P. Sanders, and B. Rust. Comparing real and synthetic images: Some ideas about metrics. In EG Rendering Workshop 1995, pages 82–91, 1995. 360. H. E. Rushmeier and K. E. Torrance. The zonal method for calculating light intensities in the presence of a participating medium. Computer Graphics (Proceedings of ACM SIGGRAPH 87), 21:293–302, 1987. 361. J. C. Russ. The Image Processing Handbook. CRC Press, 4th edition, 2002. 362. P. Sabella. A rendering algorithm for visualizing 3D scalar fields. Computer Graphics (Proceedings of ACM SIGGRAPH 88), 22(4):51–58, 1988. 363. I. A. Sadarjoen, A. J. de Boer, F. H. Post, and A. E. Mynett. Patricle tracing in σ-transformed grids using tetrahedral 6-decomposition. In EG Workshop on Visualization in Scientific Computing, pages 71–80, 1998. 364. I. A. Sadarjoen and F. H. Post. Geometric methods for vortex extraction. In EG / IEEE TCVG Symposium on Visualization ’99, pages 53–62, 1999. 365. I. A. Sadarjoen and F. H. Post. Detection, quantification, and tracking of vortices using streamline geometry. Computers and Graphics, 24(3):333–341, 2000. 366. I. A. Sadarjoen, F. H. Post, B. Ma, D. C. Banks, and H.-G. Pagendarm. Selective visualization of vortices in hydrodynamic flows. In IEEE Visualization ’98, pages 419–422, 1998. 367. H. Saito, K. Tanaka, M. Yasuda, and A. Mikami. Directionally selective response of cells in the middle temporal area (MT) of the macaque monkey to the movement of equiluminous opponent color stimuli. Brain Research, 15:1–14, 1989. 368. T. Saito and T. Takahashi. Comprehensible rendering of 3-D shapes. Computer Graphics (Proceedings of ACM SIGGRAPH 90), 24(4):197–206, 1990. 369. M. P. Salisbury, S. E. Anderson, R. Barzel, and D. H. Salesin. Interactive pen-and-ink illustration. In SIGGRAPH 1994 Conference Proceedings, pages 101–108, 1994. 370. A. R. Sanderson, C. R. Johnson, and R. M. Kirby. Display of vector fields using a reaction-diffusion model. In IEEE Visualization ’04, pages 115–122, 2004. 371. A. R. Sanderson, R. M. Kirby, C. R. Johnson, and L. Yang. Advanced reactiondiffusion models for texture synthesis. Journal of Graphics Tools, 2006. Accepted for publication. 372. A. Sanna, B. Montrucchio, and R. Arina. Visualizing unsteady flows by adaptive streaklines. In WSCG 2000 Conference Proceedings, pages 84–91, 2000. 373. A. Sanna, B. Montrucchio, and P. Montuschi. A survey on visualization of vector fields by texture-based methods. Recent Res. Devel. Pattern Rec., 1:13– 27, 2000. 374. T. Schafhitzel, D. Weiskopf, and T. Ertl. Interactive exploration of unsteady 3D flow with linked 2D/3D texture advection. In Conference on Coordinated and Multiple Views in Exploratory Visualization (CMV 2005), pages 96–105, 2005. 375. G. Scheuermann, H. Burbach, and H. Hagen. Visualizing planar vector fields with normal component using line integral convolution. In IEEE Visualization ’99, pages 255–261, 1999. 376. G. Scheuermann, H. Hagen, H. Kr¨ uger, M. Menzel, and A. Rockwood. Visualization of higher order singularities in vector fields. In IEEE Visualization ’97, pages 67–74, 1997.

References

269

377. G. Scheuermann, X. Tricoche, and H. Hagen. C1 -interpolation for vector field topology visualization. In IEEE Visualization ’99, pages 271–278, 1999. 378. H. Schirmacher, L. Ming, and H.-P. Seidel. On-the-fly processing of generalized lumigraphs. Computer Graphics Forum (Eurographics 2001), 20(3):C165– C173, 2001. 379. J. Schneider and R. Westermann. Compression domain volume rendering. In IEEE Visualization ’03, pages 39–48, 2003. 380. F. Sch¨ offel, W. Kresse, S. M¨ uller, and M. Unbescheiden. Do IPT systems fulfill application requirements? – A study on luminance on large-scale immersive projection devices. In Proceedings of the 3rd International Immersive Projection Technology Workshop (IPT ’99), pages 281–292. Springer, 1999. 381. J. Schpok, J. Simons, D. S. Ebert, and C. Hansen. A real-time cloud modeling, rendering, and animation system. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pages 160–166, 2003. 382. W. J. Schroeder, K. M. Martin, and W. E. Lorensen. The Visualization Toolkit: An Object-Oriented Approach to 3D Graphics. Kitware, New York, 3rd edition, 2002. 383. W. J. Schroeder, C. R. Volpe, and W. E. Lorensen. The stream polygon: A technique for 3D vector field visualization. In IEEE Visualization ’91, pages 126–132, 1991. 384. M. Schulz, F. Reck, W. Bartelheimer, and T. Ertl. Interactive visualization of fluid dynamics simulations in locally refined Cartesian grids. In IEEE Visualization ’99, pages 413–416, 1999. 385. J. Schulze, M. Kraus, U. Lang, and T. Ertl. Integrating pre-integration into the shear-warp algorithm. In Workshop of Volume Graphics 2003, pages 109–118, 2003. 386. J. P. Schulze, R. Niemeier, and U. Lang. The perspective shear-warp algorithm in a virtual environment. In IEEE Visualization ’01, pages 207–214, 2001. 387. A. Secord. Weighted Voronoi stippling. In Proceedings of the Symposium on Non-Photorealistic Animation and Rendering, pages 27–43, 2002. 388. A. E. Seiffert and P. Cavanagh. Position-based motion perception for colour and texture stimuli: Effects of contrast and speed. Vision Research, 39:4172– 4185, 1999. 389. D. D. Seligmann and S. Feiner. Automated generation of intent-based 3D illustrations. Computer Graphics (Proceedings of ACM SIGGRAPH 91), 25(4):123–132, 1991. 390. SGI. OpenGL extension registry. Web site: http://oss.sgi.com/projects/ ogl-sample/registry, 2005. 391. H.-W. Shen, C. R. Johnson, and K.-L. Ma. Visualizing vector fields using line integral convolution and dye advection. In 1996 Volume Visualization Symposium, pages 63–70, 1996. 392. H.-W. Shen and D. L. Kao. A new line integral convolution algorithm for visualizing time-varying flow fields. IEEE Transactions on Visualization and Computer Graphics, 4(2):98–108, 1998. 393. P. Shirley and A. Tuchman. A polygonal approximation to direct scalar volume rendering. In Workshop on Volume Visualization ’90, pages 63–70, 1990. 394. C. Silva, J. Comba, S. Callahan, and F. Bernardon. A survey of GPU-based volume rendering of unstructured grids. In RITA 2005, 2005.

270

References

395. V. Singh, D. Silver, and N. Cornea. Real-time volume manipulation. In Workshop of Volume Graphics 2003, pages 45–52, 2003. 396. D. Stalling and H.-C. Hege. Fast and resolution independent line integral convolution. In SIGGRAPH 1995 Conference Proceedings, pages 249–256, 1995. 397. J. Stam. Stable fluids. In SIGGRAPH 1999 Conference Proceedings, pages 121–128, 1999. 398. S. Stegmaier, M. Strengert, T. Klein, and T. Ertl. A simple and flexible volume rendering framework for graphics-hardware-based raycasting. In Workshop of Volume Graphics 2005, pages 187–195, 2005. 399. C. M. Stein, B. G. Becker, and N. L. Max. Sorting and hardware assisted rendering for volume visualization. In 1994 Symposium on Volume Visualization, pages 83–89, 1994. 400. G. Stoll, M. Eldridge, D. Patterson, A. Webb, S. Berman, R. Levy, C. Caywood, M. Taveira, S. Hunt, and P. Hanrahan. Lightning-2: A high-performance display subsystem for PC clusters. In SIGGRAPH 2001 Conference Proceedings, pages 141–148, 2001. 401. A. Stompel, K.-L. Ma, E. B. Lum, J. P. Ahrens, and J. Patchett. SLIC: Scheduled linear image compositing for parallel volume rendering. In IEEE Symposium on Parallel and Large-Data Visualization and Graphics, pages 33– 40, 2003. 402. M. Strengert, M. Magall´ on, D. Weiskopf, S. Guthe, and T. Ertl. Hierarchical visualization and compression of large volume datasets using GPU clusters. In Eurographics Symposium on Parallel Graphics and Visualization, pages 41–48, 2004. 403. C. F. Stromeyer, R. E. Kronauer, A. Ryu, A. Chaparro, and R. T. Eskew. Contributions of human long-wave and middle-wave cones to motion detection. The Journal of Physiology, 485(1):221–243, 1995. 404. T. Strothotte and S. Schlechtweg. Non-Photorealistic Computer Graphics – Modeling, Rendering, and Animation. Morgan Kaufmann, San Francisco, 2002. 405. A. Sundquist. Dynamic line integral convolution for visualizing streamline evolution. IEEE Transactions on Visualization and Computer Graphics, 9(2):273– 282, 2003. 406. N. Svakhine, D. S. Ebert, and D. Stredney. Illustration motifs for effective medical volume illustration. IEEE Computer Graphics and Applications, 25(3):31– 39, 2005. 407. J. E. Swan II, K. Mueller, T. M¨ oller, N. Shareef, R. Crawfis, and R. Yagel. An anti-aliasing technique for splatting. In IEEE Visualization ’97, pages 197–204, 1997. 408. R. M. Taylor II, C. Ware, and V. Interrante. Perceptually-based visualization design. SIGGRAPH 2003 Course #45 Notes, 2003. 409. C. Teitzel and T. Ertl. New approaches for particle tracing on sparse grids. In EG / IEEE TCVG Symposium on Visualization ’99, pages 73–84, 1999. 410. C. Teitzel, R. Grosso, and T. Ertl. Efficient and reliable integration methods for particle tracing in unsteady flows on discrete meshes. In EG Workshop on Visualization in Scientific Computing, pages 31–41, 1997. 411. C. Teitzel, R. Grosso, and T. Ertl. Line integral convolution on triangulated surfaces. In WSCG 1997 Conference Proceedings, pages 572–581, 1997. 412. C. Teitzel, R. Grosso, and T. Ertl. Particle tracing on sparse grids. In EG Workshop on Visualization in Scientific Computing, pages 81–90, 1998.

References

271

413. A. Telea and J. J. van Wijk. Simplified representation of vector fields. In IEEE Visualization ’99, pages 35–42, 1999. 414. A. Telea and J. J. van Wijk. 3D IBFV: Hardware-accelerated 3D flow visualization. In IEEE Visualization ’03, pages 233–240, 2003. 415. The National Library of Medicine’s Visible Human Project. http://www.nlm. nih.gov/research/visible. 416. T. A. Thomas. Technical Illustration. McGraw-Hill, New York, second edition, 1968. 417. P. Thompson. Perceived rate of movement depends on contrast. Vision Research, 22:377–380, 1982. 418. U. Tiede, T. Schiemann, and K. H. H¨ ohne. High quality rendering of attributed volume data. In IEEE Visualization ’98, pages 255–262, 1998. 419. Tom’s Hardware. Performance leap: NVIDIA GeForce 6800 Ultra. http://graphics.tomshardware.com/graphic/20040414, 2004. 420. T. Totsuka and M. Levoy. Frequency domain volume rendering. In SIGGRAPH 1993 Conference Proceedings, pages 271–278, 1993. 421. A. Treisman. Preattentive processing in vision. Computer Vision, Graphics, and Image Processing, 31:156–177, 1985. 422. S. Treue, M. Husain, and R. A. Andersen. Human perception of structure from motion. Vision Research, 31:59–75, 1991. 423. X. Tricoche, G. Scheuermann, and H. Hagen. A topology simplification method for 2D vector fields. In IEEE Visualization ’00, pages 359–366, 2000. 424. X. Tricoche, G. Scheuermann, and H. Hagen. Continuous topology simplification of planar vector fields. In IEEE Visualization ’01, pages 159–166, 2001. 425. X. Tricoche, T. Wischgoll, G. Scheuermann, and H. Hagen. Topology tracking for the visualization of time-dependent two-dimensional flows. Computers and Graphics, 26(2):249–257, 2002. 426. T. Troscianko and M. Fahle. Why do isoluminant stimuli appear slower? Journal of the Optical Society of America A, 5(6):871–880, 1988. 427. T. Troscianko, R. Montagnon, J. L. Clerc, E. Malbert, and P.-L. Chanteau. The role of colour as a monocular depth cue. Vision Research, 31(11):1923–1930, 1991. 428. E. R. Tufte. Visual Explanations: Images and Quantities, Evidence and Narrative. Graphics Press, Cheshire, CN, 1997. 429. J. Tumblin and J. A. Ferwerda (eds.). Issue on Applied Perception. IEEE Computer Graphics and Applications, 21(5):20–85, 2001. 430. G. Turk and D. Banks. Image-guided streamline placement. In SIGGRAPH 1996 Conference Proceedings, pages 453–460, 1996. 431. S.-K. Ueng, C. Sikorski, and K.-L. Ma. Efficient streamline, streamribbon, and streamtube constructions on unstructured grids. IEEE Transactions on Visualization and Computer Graphics, 2(2):100–110, 1996. 432. R. Ulichney. A review of halftoning techniques. In Color Imaging: DeviceIndependent Color, Color Hardcopy, and Graphic Arts V, volume 3963 of Proceedings of SPIE, pages 378–391, 2000. 433. S. O. Unverdi and G. Tryggvason. A front-tracking method for viscous incompressible multi-fluid flows. Journal of Computational Physics, 100(1):25–37, 1992. 434. S. P. Uselton. Volume rendering for computational fluid dynamics: Initial results. Technical Report RNR-91-026, NASA Ames Research Center, Sept. 1991.

272

References

435. J. van Eyck. The virgin of chancellor Rolin. Oil on wood, Musée du Louvre, Paris, 1435. 436. A. Van Gelder and K. Kim. Direct volume rendering with shading via threedimensional textures. In 1996 Symposium on Volume Visualization, pages 23–30, 1996. 437. J. J. van Wijk. Spot noise – texture synthesis for data visualization. Computer Graphics (Proceedings of ACM SIGGRAPH 91), 25:309–318, 1991. 438. J. J. van Wijk. Flow visualization with surface particles. IEEE Computer Graphics and Applications, 13(4):18–24, 1993. 439. J. J. van Wijk. Implicit stream surfaces. In IEEE Visualization ’93, pages 245–252, 1993. 440. J. J. van Wijk. Image based flow visualization. ACM Transactions on Graphics, 21(3):745–754, 2002. 441. J. J. van Wijk. Image based flow visualization for curved surfaces. In IEEE Visualization ’03, pages 123–130, 2003. 442. J. J. van Wijk and A. Telea. Enridged contour maps. In IEEE Visualization ’01, pages 69–74, 2001. 443. F. Vega-Higuera, P. Hastreiter, R. Fahlbusch, and G. Greiner. High performance volume splatting for visualization of neurovascular data. In IEEE Visualization ’05, pages 271–278, 2005. 444. V. Verma, D. Kao, and A. Pang. PLIC: Bridging the gap between streamlines and LIC. In IEEE Visualization ’99, pages 341–348, 1999. 445. V. Verma, D. Kao, and A. Pang. A flow-guided streamline seeding strategy. In IEEE Visualization ’00, pages 163–170, 2000. 446. O. Veryovka and J. Buchanan. Halftoning with image-based dither screens. In Proceedings of Graphics Interface, pages 167–174, 1999. 447. I. Viola and M. E. Gr¨ oller. Smart visibility in visualization. In L. Neumann, M. Sbert, B. Gooch, and W. Purgathofer, editors, Proceedings of EG Workshop on Computational Aesthetics in Graphics, Visualization and Imaging, pages 209–216, 2005. 448. I. Viola, A. Kanitsar, and E. Gr¨ oller. Importance-driven volume rendering. In IEEE Visualization ’04, pages 139–145, 2004. 449. I. Viola, A. Kanitsar, and E. Gr¨ oller. Importance-driven feature enhancement in volume visualization. IEEE Transactions on Visualization and Computer Graphics, 11(4):408–418, 2005. 450. I. Viola, A. Kanitsar, and M. E. Gr¨ oller. GPU-based frequency domain volume rendering. In Spring Conference on Computer Graphics (SCCG ’04), pages 55– 64, 2004. 451. W. R. Volz. Gigabyte volume viewing using split software/hardware interpolation. In 2000 Symposium on Volume Visualization, pages 15–22, 2000. 452. H. Wallach and D. N. O’Connell. The kinetic depth effect. Journal of Experimental Psychology, 45(4):205–217, 1953. 453. S. W. Wang and A. E. Kaufman. Volume sculpting. In 1995 ACM Symposium on Interactive 3D Graphics, pages 151–156, 1995. 454. C. Ware. Color sequences for univariate maps. IEEE Computer Graphics and Applications, 8(5):41–49, 1988. 455. C. Ware. Information Visualization: Perception for Design. Morgan Kaufmann, San Francisco, 2000. 456. C. Ware. Lecture 2: Early vision and color. In SIGGRAPH 2003 Course #45 Notes: Perceptually Based Visualization Design, 2003.

References

273

457. R. Wegenkittl, E. Gr¨ oller, and W. Purgathofer. Animating flow fields: Rendering of oriented line integral convolution. In Computer Animation ’97, pages 15–21, 1997. 458. M. Weiler, M. Kraus, and T. Ertl. Hardware-based view-independent cell projection. In 2002 Symposium on Volume Visualization, pages 13–22, 2002. 459. M. Weiler, M. Kraus, M. Merz, and T. Ertl. Hardware-based ray casting for tetrahedral meshes. In IEEE Visualization ’03, pages 333–340, 2003. 460. M. Weiler, M. Kraus, M. Merz, and T. Ertl. Hardware-based view-independent cell projection. IEEE Transactions on Visualization and Computer Graphics, 9(2):163–175, 2003. 461. M. Weiler, P. N. Mall´ on, M. Kraus, and T. Ertl. Texture-encoded tetrahedral strips. In 2004 Symposium on Volume Visualization, pages 71–78, 2004. 462. M. Weiler, R. Westermann, C. Hansen, K. Zimmerman, and T. Ertl. Levelof-detail volume rendering via 3D textures. In 2000 Symposium on Volume Visualization and Graphics, pages 7–13, 2000. 463. D. Weiskopf. Spacetime-coherent visualization of time-dependent vector fields, 2003. http://www.vis.uni-stuttgart.de/ufac. 464. D. Weiskopf. Dye advection without the blur: A level-set approach for texturebased visualization of unsteady flow. Computer Graphics Forum (Eurographics 2004), 23(3):479–488, 2004. 465. D. Weiskopf. On the role of color in the perception of motion in animated visualizations. In IEEE Visualization ’04, pages 305–312, 2004. 466. D. Weiskopf. Perception-based visualization, 2004. http://www.vis.unistuttgart.de/perception. 467. D. Weiskopf. Texture-based flow visualization, 2004. http://www.vis.unistuttgart.de/texflowvis. 468. D. Weiskopf, R. Botchen, and T. Ertl. Interactive visualization of divergence in unsteady flow by level-set dye advection. In Proceedings of SimVis 2005, pages 221–232, 2005. 469. D. Weiskopf, K. Engel, and T. Ertl. Volume clipping via per-fragment operations in texture-based volume visualization. In IEEE Visualization ’02, pages 93–100, 2002. 470. D. Weiskopf, K. Engel, and T. Ertl. Interactive clipping techniques for texturebased volume visualization and volume shading. IEEE Transactions on Visualization and Computer Graphics, 9(3):298–312, 2003. 471. D. Weiskopf and G. Erlebacher. Overview of flow visualization. In C. D. Hansen and C. R. Johnson, editors, The Visualization Handbook, pages 261– 278. Elsevier, Amsterdam, 2005. 472. D. Weiskopf, G. Erlebacher, and T. Ertl. A texture-based framework for spacetime-coherent visualization of time-dependent vector fields. In IEEE Visualization ’03, pages 107–114, 2003. 473. D. Weiskopf, G. Erlebacher, M. Hopf, and T. Ertl. Hardware-accelerated Lagrangian-Eulerian texture advection for 2D flow visualization. In Vision, Modeling, and Visualization VMV ’02 Conference, pages 77–84, 2002. 474. D. Weiskopf and T. Ertl. A depth-cueing scheme based on linear transformations in tristimulus space. Technical Report TR–2002/08, Department of Computer Science at the University of Stuttgart, 2002. 475. D. Weiskopf and T. Ertl. Real-time depth-cueing beyond fogging. Journal of Graphics Tools, 7(4):83–90, 2002.

274

References

476. D. Weiskopf and T. Ertl. GPU-based 3D texture advection for the visualization of unsteady flow fields. In WSCG 2004 Short Communication Papers Proceedings, pages 259–266, 2004. 477. D. Weiskopf and T. Ertl. A hybrid physical/device-space approach for spatiotemporally coherent interactive texture advection on curved surfaces. In Proceedings of Graphics Interface, pages 263–270, 2004. 478. D. Weiskopf and M. Hopf. Real-time simulation and rendering of particle flows. In W. Engel, editor, Direct3D Shaderx: Vertex and Pixel Shader Tips and Tricks, pages 414–425. Wordware, Plano, TX, 2002. 479. D. Weiskopf, M. Hopf, and T. Ertl. Hardware-accelerated visualization of timevarying 2D and 3D vector fields by texture advection via programmable perpixel operations. In Vision, Modeling, and Visualization VMV ’01 Conference, pages 439–446, 2001. 480. D. Weiskopf, T. Schafhitzel, and T. Ertl. GPU-based nonlinear ray tracing. Computer Graphics Forum (Eurographics 2004), 23(3):625–633, 2004. 481. D. Weiskopf, T. Schafhitzel, and T. Ertl. Real-time advection and volumetric illumination for the visualization of 3D unsteady flow. In Eurovis 2005 (EG / IEEE VGTC Symposium on Visualization), pages 13–20, 2005. 482. D. Weiskopf and G. Scheuermann. Textur- und topologiebasierte Vektorfeldvisualisierung. it – Information Technology, 46(3):111–116, 2004. 483. D. Weiskopf, F. Schramm, G. Erlebacher, and T. Ertl. Particle and texture based spatiotemporal visualization of time-dependent vector fields. In IEEE Visualization ’05, pages 639–646, 2005. 484. D. Weiskopf, M. Weiler, and T. Ertl. Maintaining constant frame rates in 3D texture-based volume rendering. In Proceedings of IEEE Computer Graphics International (CGI), pages 604–607, 2004. 485. R. Westermann and T. Ertl. Efficiently using graphics hardware in volume rendering applications. In SIGGRAPH 1998 Conference Proceedings, pages 169–179, 1998. 486. R. Westermann, C. Johnson, and T. Ertl. Topology-preserving smoothing of vector fields. IEEE Transactions on Visualization and Computer Graphics, 7(3):222–229, 2001. 487. R. Westermann and C. Rezk-Salama. Real-time volume deformations. Computer Graphics Forum (Eurographics 2001), 20(3):443–451, 2001. 488. L. Westover. Footprint evaluation for volume rendering. Computer Graphics (Proceedings of ACM SIGGRAPH 90), 24(4):367–376, 1990. 489. T. Whitted. An improved illumination model for shaded display. Communications of the ACM, 23(6):343–349, 1980. 490. P. L. Williams. Visibility ordering meshed polyhedra. ACM Transactions on Graphics, 11(2):103–126, 1992. 491. G. Winkenbach and D. H. Salesin. Computer-generated pen-and-ink illustration. In SIGGRAPH 1994 Conference Proceedings, pages 91–100, 1994. 492. T. Wischgoll and G. Scheuermann. Detection and visualization of closed streamlines in planar flows. IEEE Transactions on Visualization and Computer Graphics, 7(2):165–172, 2001. 493. C. M. Wittenbrink, A. T. Pang, and S. K. Lodha. Glyphs for visualizing uncertainty in vector fields. IEEE Transactions on Visualization and Computer Graphics, 2(3):266–279, 1996. 494. W. Wolfe. Visual search. In H. E. Pashler, editor, Attention, pages 13–73, Hove, 1998. Psychology Press.

References

275

495. P. C. Wong, H. Foote, R. Leung, E. Jurrus, D. Adams, and J. Thomas. Vector fields simplification – A case study of visualizing climate modeling and simulation data sets. In IEEE Visualization ’00, pages 485–488, 2000. 496. M. Woo, J. Neider, T. Davis, and OpenGL Architecture Review Board. OpenGL Programming Guide. Addison-Wesley, 1997. 497. B. Wylie, K. Moreland, L. A. Fisk, and P. Crossno. Tetrahedral projection using vertex shaders. In 2002 Symposium on Volume Visualization, pages 7– 12, 2002. 498. G. Wyszecki and W. S. Stiles. Color Science. John Wiley & Sons, New York, second edition, 1982. 499. T. Yabe, T. Ishikawa, P. Y. Wang, T. Aoki, Y. Kadota, and F. Ikeda. A universal solver for hyperbolic equations by cubic-polynomial interpolation II. Two- and three-dimensional solvers. Computer Physics Communications, 66(2– 3):233–242, 1991. 500. H. Yee, S. Pattanaik, and D. P. Greenberg. Spatiotemporal sensitivity and visual attention for efficient rendering of dynamic environments. ACM Transactions on Graphics, 20(1):39–65, 2001. 501. T. Yoshizawa, K. T. Mullen, and C. L. Baker, Jr. Absence of a chromatic linear motion mechanism in human vision. Vision Research, 40:1993–2010, 2000. 502. D. L. Youngs. Time-dependent multi-material flow with large fluid distortion. In K. W. Morton and M. J. Baines, editors, Numerical Methods for Fluid Dynamics, pages 273–285. Academic Press, 1982. 503. S. M. Zeki. Uniformity and diversity of structure and function in rhesus monkey prestriate visual cortex. The Journal of Physiology, 277:273–290, 1978. 504. C. Zhang and R. Crawfis. Volumetric shadows using splatting. In IEEE Visualization ’02, pages 85–92, 2002. 505. C. Zhang and R. Crawfis. Shadows and soft shadows with participating media using splatting. IEEE Transactions on Visualization and Computer Graphics, 9(2):139–149, 2003. 506. H. Zhang and K. Hoff. Fast backface culling using normal masks. In 1997 ACM Symposium on Interactive 3D Graphics, pages 103–106, 1997. 507. X. Zhang, C. Bajaj, and W. Blanke. Scalable isosurface visualization of massive datasets on COTS clusters. In IEEE Symposium on Parallel and Large-Data Visualization and Graphics, pages 51–58, 2001. 508. M. Z¨ ockler, D. Stalling, and H.-C. Hege. Interactive visualization of 3D-vector fields using illuminated stream lines. In IEEE Visualization ’96, pages 107–113, 1996. 509. M. Z¨ ockler, D. Stalling, and H.-C. Hege. Parallel line integral convolution. In EG Workshop on Parallel Graphics and Visualisation, pages 111–127, 1996. 510. M. Zwicker, H. Pfister, J. van Baar, and M. Gross. EWA volume splatting. In IEEE Visualization ’01, pages 29–36, 2001. 511. M. Zwicker, J. R¨ as¨ anen, M. Botsch, C. Dachsbacher, and M. Pauly. Perspective accurate splatting. In Proceedings of Graphics Interface, pages 247–254, 2004.

Color Plates

Chapter 2: Visualization of 3D Scalar Fields

(a)


c 2002 IEEE

Plate 1. Comparison for a voxelized cube-shaped clip object: (a) trilinear interpolation; (b) nearest-neighbor sampling (see also gray-scale Figure 2.19 on page 49)

Color Plates IMAGES: WEISKOPF, ENGEL, ERTL [469]

278

c 2002 IEEE

Plate 2. Comparison of volume clipping techniques (see also gray-scale Figure 2.21 on page 51)

279



(a)

(b)


Plate 3. Comparison of volumetric clipping approaches: (a) slice-based, (b) slabbased. See also gray-scale Figure 2.23 on page 54

(a)

(b)

Plate 4. Rendering of a bonsai without (a) and with (b) pre-integrated volumetric clipping (see also gray-scale Figure 2.24 on page 54)

Color Plates


c 2003 IEEE

280

(a)

(b)


c 2003 IEEE

Plate 5. Combining volume clipping and volume shading. Depth-based volume cutting with a spherical clip object is applied to a technical engine data set. (a) No specific lighting is applied at the cutting surface; (b) reveals lighting (e.g., a highlight) on the cutting surface by combining surface-based and volumetric shading. See also gray-scale Figure 2.25 on page 56

Plate 6. Depth-based clipping in an illuminated CT data set (see also gray-scale Figure 2.28 on page 63)


(b)

(c)

(d)


c 2003 IEEE

(a)

281

Plate 7. Clipping in an illuminated orbital data set: (a) without clipping; (b) and (c) depth-based clipping; (d) volumetric clipping. The clipping surface is additionally marked in dark gray in (b). See also gray-scale Figure 2.29 on page 64

Color Plates

Plate 8. Radial distance volume combined with Perlin noise. A high-frequency transfer function is applied. See also gray-scale Figure 2.33 on page 74

´ IMAGES: STRENGERT, MAGALLON, WEISKOPF, GUTHE, ERTL [402] c The Eurographics Association 2004

282

Plate 9. Parallel volume rendering of anatomical cryosections through the Visible Human male data set. The whole body is rendered with a total of 16 nodes. This image shows the result of four render nodes that display the upper part of the body. See also gray-scale Figure 2.34 on page 74



283


Plate 10. Visualization of the time-dependent CFD simulation. From top to bottom, timesteps 0, 45, and 89 are shown. See also gray-scale Figure 2.35 on page 75

284

Color Plates

IMAGE: WEISKOPF, ERLEBACHER [471]

c 2005 Elsevier

Chapter 3: Vector Field Visualization

Plate 11. Glyph-based 3D flow visualization, combined with illuminated streamlines (see also gray-scale Figure 3.2 on page 89)

Plate 12. Combination of streamlines, streamribbons, arrows, and color coding for a 3D flow (courtesy of the BMW Group and Martin Schulz). See also gray-scale Figure 3.3 on page 90


285

(a)

IMAGES: WEISKOPF, ERLEBACHER, HOPF, ERTL [473] c 2002 Greiner, Niemann, Ertl, Girod, Seidel

(b)

(c) Plate 13. Visualization of the interaction of a planar shock with a longitudinal vortex by 2D texture advection. The transport mechanism makes use of LEA. (a) shows a dense, noise-based visualization with a velocity mapping that modifies the grayscale image according to the magnitude of the velocity to enhance the important structures in the flow. (b) and (c) show the combination of noise (gray-scale) and dye (colored) advection – in (b) without velocity mapping and in (c) with velocity mapping. See also gray-scale Figure 3.9 on page 111

Color Plates

(a)

(b)

(c)

(d)

(e)

(f)


c 2004 UNION Agency/WSCG

286

Plate 14. Visualization of a tornado data set by 3D texture advection with velocity masking: (a) dense noise injection, (b) sparse noise injection, (c) additional red dye, (d) different lengths for blue (dark) and green (bright) material. (e) shows a dense visualization without masking; (f ) the same with a slanted clipping plane. See also gray-scale Figure 3.11 on page 113

287

IMAGE: WEISKOPF, SCHAFHITZEL, ERTL [481] c The Eurographics Association 2005


Plate 15. Benard flow with λ2 vortex visualization. See also gray-scale Figure 3.12 on page 114

(b)

(c)

(d)

IMAGES: WEISKOPF, SCHAFHITZEL, ERTL [481] c The Eurographics Association 2005

(a)

Plate 16. Volume rendering of 3D advection applied to a tornado data set: (a) emission-absorption model only, (b) with Phong illumination added, (c) cool/warm shading, and (d) halos. See also gray-scale Figure 3.13 on page 116

Color Plates



288

(c)

(e)

(b)

(d)

(f)


(a)


Plate 17. Numerical diffusion in a circular flow (see also gray-scale Figure 3.14 on page 117)

Plate 18. Comparison: (a, b) traditional dye advection; (c, d) the level-set approach without reinitialization; (e, f ) the level-set approach with reinitialization. See also gray-scale Figure 3.16 on page 120

289






Plate 19. Level-set dye advection for a 3D vector field with a swirling feature (see also gray-scale Figure 3.19 on page 122)

(a)

(b)

(c)

Plate 20. Change of topology of the dye-covered area. The images show snapshots from an animated visualization, chronologically ordered from left to right. See also gray-scale Figure 3.20 on page 124

Color Plates

(a)

(b)

(c)

(d)



290

Plate 21. Extensions for level-set dye advection: (a) an interface with jaggy artifacts caused by a binary decision with respect to the isovalue φ = 0; (b) jaggy artifacts smeared out by image-space smoothing; (c) an additional background LIC image; (d) dye faded out. See also gray-scale Figure 3.22 on page 125


(b)

(c)

(d)


(a)

291

Plate 22. Different shading approaches for object and flow rendering: (a) gray-scale flow visualization without additional surface shading; (b) gray-scale flow modulated with illuminated gray object; (c) yellow/blue flow visualization modulated with illuminated gray object; (d) gray-scale flow modulated with cool/warm shaded surface (data set courtesy of the BMW Group). See also gray-scale Figure 3.28 on page 137

Color Plates

(a)

(b)

(c)

(d)


c 2003 IEEE

292

Plate 23. Visualization of a shock data set: (a) UFAC with velocity masking; (b) UFAC without masking; (c) LIC; (d) LEA (see also gray-scale Figure 3.33 on page 154)

293


c 2003 IEEE


(a)

(b)



Plate 24. Comparison of IBFV-guided techniques: (a) long, clearly defined streaklines for the framework-based method; (b) numerical diffusion for the original IBFV implementation. See also gray-scale Figure 3.35 on page 156

(a)

(b)

Plate 25. (a) Truncation of streaklines in low-resolution level-set advection. (b) represents the correct streakline generated at high resolution. See also gray-scale Figure 3.21 on page 124

294

Color Plates


c 2004 IEEE

Chapter 4: Perception-Oriented and Non-Photorealistic Rendering

Plate 26. Calibration of perceived speeds. Two texture stimuli are rendered: a benchmark luminance contrast in the lower half of the screen and, in the upper half of the screen, a texture with the stimulus of which the apparent speed has to be calibrated. The speed of both textures has to be matched by the user. See also gray-scale Figure 4.1 on page 170

295


c 2004 IEEE



c 2004 IEEE

Plate 27. Grouping through motion in graph visualization. A subset of four nodes moves coherently. This motion is illustrated by a duplication and slight shift of the moving edges and points. A zoomed-in view of this region of interest is shown in the bottom-left part of the image. See also gray-scale Figure 4.2 on page 172

Plate 28. LIC visualization of a 2D vector field. An additional data dimension is represented by color coding. The lower part contains a nearly isoluminant YB contrast, which appears to move more slowly than the luminance patterns in the upper part (shown in the accompanying video [466]). See also gray-scale Figure 4.3 on page 173

Color Plates Plate 29. Surface flow visualization: (a) shape and flow structures are represented by luminance contrasts; (b) cool/warm shading for the surface and luminance contrasts for LIC patterns; (c) shape is represented by luminance, flow structures by isoluminant YB contrast (data set courtesy of the BMW Group). See also gray-scale Figure 4.4 on page 174

(a)

c 2004 IEEE

(b)


296

(c)


(a)

(b)

(c)

(d)

297

Plate 30. Comparison of different color-based depth-cueing approaches: (a) the test scene without depth-cueing; (b) intensity depth-cueing; (c) saturation depthcueing; (d) combination of saturation and softened intensity depth-cueing. See also gray-scale Figure 4.6 on page 181

Plate 31. Comparison of color-mapping techniques for depth-cueing: (a) a simple box-shaped “test” object of which the distance to the camera increases from bottom to top; (b) scene with intensity depth-cueing; (c) saturation variations; (d) combination of saturation and softened intensity changes; (e) saturation changes with a superimposed brightness “sawtooth” structure. See also gray-scale Figure 4.7 on page 182

298

Color Plates

(a)

(b)

(c)

(d)

Plate 32. Non-photorealistic rendering of an engine block with different depth-cues: (a) original scene; (b) cool/warm shading without depth-cueing; (c) cool/warm shading with saturation depth-cueing; (d) cool/warm shading with intensity depthcueing. See also gray-scale Figure 4.9 on page 187


299

Plate 33. Depth-cueing for a cone tree representation in information visualization. The left image shows the original scene; in the right image, saturation depth-cueing is applied to the same scene. The top and bottom images in the middle show magnified details; the central image visualizes the difference between the two detail images. See also gray-scale Figure 4.10 on page 188

300

Color Plates

(a)

(b)

(c)

(d)

Plate 34. Terrain rendering with color-based depth-cueing: (a) original scene; (b) saturation depth-cueing; (c) intensity depth-cueing; (d) combination of saturation depth-cueing and “sawtooth” intensity depth-cueing. See also gray-scale Figure 4.11 on page 189


301

Plate 35. Semantic depth-cueing: (a) traditional 3D view on a chess board; (b) some chessmen emphasized by highly saturated colors; (c) combination of semantic and geometric depth-cueing. See also gray-scale Figure 4.12 on page 190

(a)

(b)

(c)

302

Color Plates

Plate 36. Phantom lines showing semi-transparent objects (see also gray-scale Figure 4.13 on page 193)

(a)

(b)

Plate 37. Comparing transparency models: (a) traditional view-independent transparency; (b) view-dependent transparency. See also gray-scale Figure 4.14 on page 196 c The Eurographics Association 2002 ALL IMAGES: DIEPSTRATEN, WEISKOPF, ERTL [88]


(a)

(c)

303

(b)



Plate 38. Semi-transparent illustration: (a) semi-transparent mug with two boundaries surrounding an opaque box, based on explicit interior boundaries; (b, c) multiple semi-transparent mugs; (d) crank within a semi-transparent cylinder. See also gray-scale Figure 4.17 on page 199

304

Color Plates

(a)

(b)

(c)



Plate 39. Semi-transparent illustration. Pictures (a) and (b) show different parts of a Lancia engine block as a typical example of a technical illustration. Image (c) depicts a semi-transparent rendering of the scene from Color Plate 36 (with phantom lines). Image (d) shows a simplified wheel axis. See also gray-scale Figure 4.18 on page 200


(a)

305

(b)

Plate 40. Comparison of computer-generated cutout and breakaway illustrations. The left image demonstrates the cutout technique with a jittering boundary. In the right picture, the breakaway method is applied to the same scene. See also gray-scale Figure 4.20 on page 202

(a)

(b)

Plate 41. Cutout illustrations with different rendering styles, showing a part of an engine block: (a) toon shading and silhouette rendering; (b) cool/warm tone shading and black silhouette lines. See also gray-scale Figure 4.24 on page 210 ALL IMAGES: DIEPSTRATEN, WEISKOPF, ERTL [89]


306

Color Plates

(a)



Plate 42. Breakaway illustrations with different rendering styles, showing a curved conduit: (a) toon shading and silhouette rendering; (b) cool/warm tone shading with black silhouette lines. See also gray-scale Figure 4.26 on page 213

Index

2D texture slicing, see texture slicing, 2D 3D texture slicing, see texture slicing, 3D adaptive rendering, 69–78, 237 advection, 81, 221 dye, 82, 93, 97–127, 144, 147, 153, 157, 158, 237 equation, 85 noise, 97–115 semi-Lagrangian texture, 81, 97–107, 116, 119, 127, 157, 237 texture, 94, 97–109, 127, 128 aerial perspective, 176 alpha blending, see blending alpha test, 5, 46, 48, 210, 231 ARB fragment program, see fragment program, ARB ARB vertex program, see vertex program, ARB arrow plot, 86, 87, 89 attributed volume, see tagged volume back-to-front compositing, see compositing, back-to-front barycentric interpolation, 29, 30 binary volume, 47 blending, 5, 19, 23, 62, 66, 68, 69, 77, 78, 85, 107–109, 112, 129, 148, 158, 185, 195, 197, 198, 223, 231 Blinn-Phong, see illumination, Blinn-Phong Boltzmann equation, 15

boundary representation, 4, 40, 203, 215 breakaway, see illustration, breakaway Brep, see boundary representation bricking, 34, 66, 68, 69, 78 multi-oriented, 34–36, 79 calibration, 164, 169–175, 234 cell projection, 27–30, 91 central differences, 15, 115 CFD, 24, 65, 75, 114, 154, 159, 175 Cg, 4, 8 classification, 12, 15, 77 vector field visualization, see vector field visualization, classification visualization methods, 3–4 clipping, 12, 38–65, 77, 94, 112, 114, 158, 238 concave, 46–47 convex, 46 convex cutting, 41, 44–46 convex probing, 42–43 depth-based, 39–47, 50, 61–63, 77, 78 illumination, 55–65, 77 optical model, 55–59 plane, 38, 112, 206 pre-integrated, 52–55, 65 single boundary, 41 volumetric texture, see clipping, voxelized voxelized, 47–52, 60–61, 63, 77, 78 cluster computer, 9, 13, 65–78 clustered dot screening, 218 clutter, 87, 89, 112, 128, 158

308

Index

compositing, 12, 16, 19, 67–69, 72, 77, 78 back-to-front, 17, 19, 30, 47 front-to-back, 16, 19, 30 compression, 12, 69–73, 77 computational fluid dynamics, see CFD constructive solid geometry, see CSG Contrast Sensitivity Function, see CSF convex hull, 205, 211–213, 215 convolution, 9, 85, 97, 126, 131, 132, 140, 142, 143, 146–151, 153, 156, 238 cool/warm shading, 136, 137, 163, 175, 186, 211, 213, 214, 234 covariance matrix, 205 CSF, 162, 164, 165 CSG, 205–206 cutaway, see illustration, cutaway cutout, see illustration, cutout D-space, 130–133 data traversal, 12, 15, 18, 20 data type, 3 deferred shading, 220 dense representation, see vector field visualization, dense density-emitter model, see emissionabsorption model dependent texture, 7, 19, 54, 77, 99–101, 110, 127, 153, 171, 225 depth buffer, 41, 42, 45, 197, 198, 207, 212, 213, 215 depth sorting, 196–200, 234 depth structure, 39–41, 46 depth test, see z test depth-based clipping, see clipping, depth-based depth-cueing, 161, 176–191, 234, 238 intensity, 176, 177, 179–181, 183, 184, 186–191, 234 saturation, 176, 177, 181–184, 186–191, 234 semantic, 191, 234 depth-peeling, 25, 46, 198 design guidelines, 167–169 differential geometry, 82 dimensionality of the domain, 3, 86 direct volume visualization, see volume rendering

Direct3D, 4, 8, 102 DirectX, 4, 65, 99, 104, 106, 126, 138, 152, 153, 156, 225, 229, 231 displacement mapping, 208 displayable image, 3 distance signed, 208 distance field signed, 48, 52, 59, 60, 118–119 distributed visualization, see parallelization dithering, 218 DLIC, 94, 140, 150–151, 155 dye advection, see advection, dye Dynamic LIC, see DLIC early ray termination, 23, 25 early z test, see z test, early effect, see effect file effect file, 8, 138, 152, 218, 225, 229 extended, 229–231 effectiveness, 1 efficiency, 1 elliptical weighted average, see EWA emission-absorption model, 13–15, 17, 32, 115 equation of transfer, 13 Euclidean space, 11, 59, 83, 141 Euler integration, 92, 98–101, 126, 130, 132, 144 EWA, 27 extinction coefficient, 13 eye coordinates, 4 filter kernel, 93, 134, 142, 146–148, 158, 173 exponential, 107, 112, 129, 149, 155 filtering stage, 2, 12, 38, 77, 82, 237 fixed-function pipeline, see GPU, fixed-function pipeline flow visualization, see vector field visualization footprint, 26 Fourier volume rendering, see frequency domain volume rendering fractional coordinates, 105, 106 fragment, 4, 40 fragment program, 5, 22–24, 29, 31, 43, 45, 46, 54, 60, 65, 66, 99, 101, 103,

Index 115, 135, 138, 140, 152, 156, 171, 183, 185, 186, 199, 210, 212, 229, 231 ARB, 8, 73, 101 frame buffer, 5 frame-buffer object, 33, 102 frame-to-frame coherence, 86, 129, 130, 133, 135, 140, 162, 195, 217, 218, 220–229, 234, 237 frequency domain volume rendering, 32 front-to-back compositing, see compositing, front-to-back FX file, see effect file G-buffer, 162, 163, 218–232, 234 Gauss map, 195 glyph, 86, 87, 96 GPU API, 8 fixed-function pipeline, 4 programming model, 5–7 rendering pipeline, 4–5 gradient, 14, 15, 18, 56 computation, 12, 15, 115 grid Cartesian, 4, 18, 21, 99 curvilinear, 4, 86 rectilinear, 86 structured, 3 tetrahedral, 12, 18 type, 86 uniform, 4, 12, 15, 18, 21, 27, 31, 86, 99, 120, 144, 151, 157, 219, 221 unstructured, 3, 12, 27, 86, 91–93 halftoning, 217–235, 237 hatching, 218, 219 hedgehog, 86, 87 High-Level Shading Language, see HLSL histogram equalization, 93, 146, 147, 153, 156 HLSL, 8, 104, 138, 139 homogeneous coordinates, 4 homogeneous division, 5, 130 Human Visual System, see HVS HVS, 161, 162, 164, 167, 176, 178, 191, 232

309

hypersurface, 57, 61, 86, 88, 93, 128, 129, 131, 132, 136, 141 IBFV, 94, 107, 108, 134, 140, 148, 152, 155 2D, 107 3D, 102, 107 illuminated streamline, see streamline, illuminated illumination Blinn-Phong, 4, 14, 19 local, 56, 60 Phong, 14, 115, 116 surface, 58, 61 volumetric, 39, 55–65, 115 illustration, 234 breakaway, 201, 204, 211–215, 234 cutaway, 192, 201–215, 234 cutout, 201–211, 214–215, 234, 238 semi-transparent, 193–200, 214, 234 traditional, 161, 162, 192–194, 201 image-aligned slices, see texture slicing, 3D image-based flow visualization, see IBFV impregnation approach, 58–61 inflow region, 129, 132, 146 information visualization, 171–172 injection texture, 107–109 integral curve, 83, 84, 141 interaction, 3, 9, 108, 148, 238 interactive exploration, 1, 77, 81, 91, 97, 106, 108, 109, 157, 238 interpolation, 12, 15, 20, 86, 116, 144 isoluminance, 164–169, 176, 182 isosurface, 14, 50, 59, 60, 67 JND, 162 Just-Noticeable Difference, see JND Lagrangian-Eulerian advection, see LEA large-data visualization, 65–78 lateral geniculate nuclei, see LGN LEA, 94, 105–107, 148, 152, 155, 157, 224, 225 level-set, 9, 82, 91, 96, 116–127, 157, 158, 237 evolution equation, 119

310

Index

reinitialization, 119–127 LGN, 165 LIC, 93, 94, 107, 112, 126, 129, 131, 133–136, 138, 140, 151, 152, 154, 163, 171, 173, 175 line integral convolution, see LIC M/D ratio, 165, 167 manifold, 82 curved, 129 mapping stage, 2, 12, 17, 81, 82, 237 maximum intensity projection, 17 MIP, see maximum intensity projection MMX, 68 model transformation, 4, 130 model-based visualization, 239 motion parallax, 1, 9, 176 multi-bricking, see bricking, multioriented multi-dimensional transfer function, see transfer function, multi-dimensional multi-oriented bricking, see bricking, multi-oriented Myrinet, 73, 77, 79 noise advection, see advection, noise noise injection, 107, 108, 112, 134–136 noise texture, 93, 97, 108, 130, 135, 144 non-photorealistic rendering, 3, 9, 39, 161–163, 191–235, 238 normalization transformation, 4 normalized clip coordinates, 4 normalized device coordinates, 5, 130 NPR, see non-photorealistic rendering numerical diffusion, 82, 116–118, 123, 149, 155, 157, 237 Nyquist frequency, 30 object coordinates, 4 object-aligned slices, see texture slicing, 2D occlusion, 87, 89, 94, 112, 128, 158, 176, 186, 192 occlusion query, 23 OLIC, 93 opacity, 14 OpenGL, 4, 8, 16, 63, 67, 73, 102, 104, 138, 152, 180, 208, 213

shading language, 4, 8 OpenGL Performer, 8 OpenGL Volumizer, 35, 47 OpenInventor, 8 OpenSceneGraph, 8 OpenSG, 8, 35 optical model, see volume rendering, optical model optical property, 13, 15 Oriented LIC, see OLIC P-space, 130–133 parallelization, 9, 13, 65–69, 71–77, 237 participating medium, 13, 15, 32 particle tracing, 81–95, 129–133, 158 Eulerian, 85, 97, 98, 119, 157 Lagrangian, 84–85, 158 partitioning image-space, 67 object-space, 67, 68 pathline, 83–87, 98, 141–146, 148, 149, 151, 152 pathway, 165–167 perception, 9, 161 color, 163–191 depth, 94, 112, 115, 158, 176–191 motion, 163–176 orientation, 94, 115 shape, 136–137 spatial, 1, 9, 87, 161, 234, 238 perception-oriented rendering, 3, 9, 161–191, 232–234, 238 phantom lines, 193 Phong, see illumination, Phong photon mapping, 15 physiology, 161, 164–166, 177, 180 ping-pong rendering, 7, 19, 100, 127, 138 pixel shader, 5, 8, 65, 99, 100, 106, 126, 127, 138, 139, 225, 229, 230 post-classification, 16, 19, 73 pre-classification, 16, 19, 63 pre-integrated classification, see volume rendering, pre-integrated pre-integration, see volume rendering, pre-integrated preattentive, 163, 171 primitive, 4 assembly, 5

Index projected tetrahedra, 27–30 property function, 85, 97–102, 105–107, 109, 110, 116, 119, 126, 141–144, 147, 149, 152, 155–158 psychophysics, 9, 161, 164–171, 234 radiance, 13 radiative transfer, 15 radiosity, 15 raster operations, 5 rasterization, 5 raw data, 2 ray casting, 21–25 ray termination, 22 register combiner, 8, 214 render-to-texture, 7, 100, 138, 153, 225, 230 renderable representation, 2 rendering stage, 3, 12, 17, 82, 162, 237 Renderman shading language, 8 Riemann sum, 14, 30, 146 Runge-Kutta method, 92, 130 saliency, 162, 163, 166, 175 scalar data, 3, 9, 11 scalar field, see scalar data 3D, 11 scanline conversion, 5, 131 scattered data, 27 scattering, 14, 15 screening, 217–229, 234 threshold, 219 seed point positioning, 81, 91, 92, 94 segmentation, 47 semi-Lagrangian particle tracing, see advection, semi-Lagrangian texture semi-Lagrangian texture advection, see advection, semi-Lagrangian texture Shader Model 2, 23, 46, 65, 140, 157, 199 Shader Model 3, 6, 22 shading, 12 shadow mapping, 46, 63, 211 shear-warp factorization, 25 shower door effect, 221 silhouette edge, 195–196, 211, 213, 234 SIMD, 6

311

skin approach, 56–59 slab, 31, 52, 55 SLIC, 69, 118 slicing, see texture slicing Sobel operator, 15, 219 sort-first, 67 sort-last, 67 sort-middle, 67 source term, 13 spacetime framework, 140–157, 171, 173, 238 sparse representation, see vector field visualization, sparse splatting, 26–27, 95 stencil buffer, 78, 207 stencil test, 5, 47, 207, 231 stippling, 162, 217–219 streakline, 84–87, 91, 94, 109, 117, 119, 121, 123, 126, 149, 151, 152, 155 stream, 6 kernel, 6 stream surface, 90, 91 streamball, 90 streaming model, 6 streamline, 84, 86, 87, 90, 91, 93, 94, 98, 141, 150–152, 154, 155 illuminated, 89 streamribbon, 89, 90 streamtube, 89, 90 superbuffer, 33, 102 tagged volume, 47 tangent bundle, 83 tangent space, 83 taxonomy of vector field visualization, 85 taxonomy of visualization, 3 texkill, 7, 46, 210 texture advection, see advection, texture texture cache, 33 texture memory layout, 33 texture shader, 8, 214 texture slicing, 31, 39, 55, 62, 63, 77, 94 2D, 20–21, 110 3D, 18–19, 110 texture-based volume rendering, see volume rendering, texture-based

312

Index

time-dependent data, 9, 11, 65, 75, 76, 79, 82–84, 86, 87, 93, 95, 96, 129, 140–157, 237 toon shading, 211, 213, 214 trajectory, 140–146, 148, 149, 151, 155 transfer function, 13, 16, 38, 56, 61, 109, 110, 112, 114, 158 multi-dimensional, 16, 19 transparency, 14, 193 artistic, 193–194 view-dependent, 192, 195–196, 211 transport mechanism, 97–99, 157 transport theory, 15 triangle, 4, 40 UFAC, 141, 151–152, 154, 155 UFLIC, 93, 140, 149–150, 152 Unsteady Flow Advection–Convolution, see UFAC Unsteady Flow LIC, see UFLIC user study, 171 VDP, 162 vector data, 3, 9 non-tangential, 83, 130, 132 tangential, 83, 129, 130, 132 vector field, see vector data vector field visualization, 3, 9, 81–159 classification, 85–86 curved surface, 128–140 dense, 81, 86, 92–95, 97, 107, 108, 112, 114, 126, 128, 134, 140, 141, 143, 158 feature-based, 86, 95–97 point-based direct, 85–87 sparse, 86–93, 112 texture-based, 81–159, 171, 173–175, 237 topology-based, 95–96 velocity masking, 106, 110, 112, 154, 156 vertex, 4 vertex processor, 4 vertex program, 4, 46, 66, 138, 185, 186, 208, 211 ARB, 8 vertex shader, 4, 8, 127, 229

view transformation, 4, 130 viewing frustum, 5 viewport transformation, 5 visibility, 41, 47 Visible Difference Predictor, see VDP visual mapping, 81, 97, 109–115, 134–136, 158 visual patterns, 81, 97, 117, 142, 157 visualization data, 2 visualization pipeline, 2–4, 12, 17, 77, 81, 82, 162, 237 VOF, 118 volume clipping, see clipping volume cutting, 40–41, 44–46, 48, 50 volume deformation, 39, 216 volume probing, 40, 42–43, 48, 50 volume rendering, 11–79, 82, 87, 91, 94, 95, 110, 158, 237 illumination, see illumination, volumetric image-order, 21–25 object-order, 18–21, 25–30 optical model, 12–15, 39, 55–59 pipeline, 12, 15–17 pre-integrated, 30–32, 39, 52–55, 77 texture-based, 18–21 two-level, 47 volume rendering equation, 13 volume rendering integral, 14, 15, 30, 52, 57 volume sculpting, 39 volume tagging, see tagged volume volume visualization direct, see volume rendering volume-of-fluid, see VOF vortex detection, 96 voxelization, 39, 47, 48, 63 wavelet, 9, 66, 67, 69–71, 78, 237 window coordinates, 5, 42, 45 world coordinates, 4 z buffer, see depth buffer z test, 5, 41, 43, 131, 231 early, 7, 23, 65 zonal method, 15

GPU-Based Interactive Visualization Techniques

Information Visualization, Second Edition: Perception for Design (Interactive Technologies)

Trends in Interactive Visualization. State-of-the-Art Survey

Interactive Visualization of Large Networks: Technical challenges and practical applications

Visualization Handbook

Information Visualization

Visualization Handbook

Software Visualization

Visualization Handbook

Computer Visualization: Graphics Techniques for Engineering and Scientific Analysis

The Visualization Handbook

The Visualization Handbook

Numerics - interactive

Handbook of Data Visualization

Introduction to Information Visualization

Introduction to Information Visualization

Visualization of Categorical Data

Applied Security Visualization

Knowledge and Information Visualization

Handbook of Data Visualization

Visualization and Processing of Tensor Fields (Mathematics and Visualization)

Information Visualization, Second Edition: Perception for Design (The Morgan Kaufmann Series in Interactive Technologies)

Information Visualization, Second Edition: Perception for Design (The Morgan Kaufmann Series in Interactive Technologies)

Designing Interactive Systems

Perl 5 Interactive Course

Interactive Storytelling - ICIDS 2011

Topology-based Methods in Visualization

Understanding Intrusion Detection through Visualization

SQL Interactive Workbook

SPSS Interactive Graphics 10.0

Immunobiology Interactive, 5th Edition

GPU-Based Interactive Visualization Techniques

Information Visualization, Second Edition: Perception for Design (Interactive Technologies)

Trends in Interactive Visualization. State-of-the-Art Survey

Interactive Visualization of Large Networks: Technical challenges and practical applications

Visualization Handbook

Information Visualization

Visualization Handbook

Software Visualization

Visualization Handbook

Computer Visualization: Graphics Techniques for Engineering and Scientific Analysis

The Visualization Handbook

The Visualization Handbook

Numerics - interactive

Handbook of Data Visualization

Introduction to Information Visualization

Introduction to Information Visualization

Visualization of Categorical Data

Applied Security Visualization

Knowledge and Information Visualization

Handbook of Data Visualization

Visualization and Processing of Tensor Fields (Mathematics and Visualization)

Information Visualization, Second Edition: Perception for Design (The Morgan Kaufmann Series in Interactive Technologies)

Information Visualization, Second Edition: Perception for Design (The Morgan Kaufmann Series in Interactive Technologies)

Designing Interactive Systems

Perl 5 Interactive Course

Interactive Storytelling - ICIDS 2011

Topology-based Methods in Visualization

Understanding Intrusion Detection through Visualization

SQL Interactive Workbook

SPSS Interactive Graphics 10.0

Immunobiology Interactive, 5th Edition

Recommend Documents